Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability

ArXiv: 2602.02477

🎯 Pitch

Training language models to explicitly decompose hard math problems into subproblems, rather than just grinding through chain-of-thought, unlocks an entirely new level of test-time compute scaling—it's not just better, it keeps improving with more inference budget while CoT flatlines. This divide-and-conquer RL training boosts Pass@1 by 8.6 points and even lifts CoT performance when mixed in, suggesting we've been training these models in a way that's fundamentally misaligned with their most powerful reasoning strategies.

---

1. Executive Summary

This paper proposes an end-to-end reinforcement learning framework, DAC-RL, that trains LLMs to perform divide-and-conquer (DAC) reasoning — decomposing a problem into subproblems, solving them sequentially, and synthesizing the final answer — rather than relying solely on standard sequential chain-of-thought (CoT). Training Qwen2.5-7B-Instruct and Qwen3-4B-Instruct-2507 on DAPO-Math-17k with GRPO, DAC-RL yields absolute gains of 8.6% in Pass@1 and 6.3% in Pass@32 averaged across AIME 2024/2025, Beyond-AIME, and HMMT-25 over CoT-based RL, while also improving CoT reasoning itself by over 10% when mixed into training. The framework demonstrates that DAC-style reasoning elevates test-time scalability — allocating the inference budget across more diverse subproblem groups consistently outperforms independent CoT sampling — establishing that DAC training unlocks a higher reasoning ceiling than CoT only on problems where the model can benefit from structured decomposition, and that dedicating post-training specifically to DAC is necessary to bridge the misalignment between standard CoT-centric post-training and DAC-style inference.

2. Context and Motivation

The Core Problem: CoT Reasoning Hits a Ceiling, and We Don't Know How to Break Through It

The fundamental problem this paper addresses is that chain-of-thought (CoT) reasoning, despite being the dominant paradigm for LLM problem-solving, has a finite performance ceiling that cannot be transcended by simply scaling up training or inference within the CoT framework. The authors observe (Section 1, Figure 4 middle/right panels) that when they apply CoT-style RL to Qwen3-4B-Instruct-2507 — a model already extensively post-trained for reasoning — further CoT RL training yields no additional gains. The performance curve flattens. This saturation is not just an empirical inconvenience; it represents a structural limitation: on problems that the model consistently answers incorrectly with CoT, no amount of additional CoT-style training or sampling can help, because the model never produces correct trajectories to reinforce.

This gap is significant for several reasons the paper highlights, both explicitly and implicitly:

• Frontier mathematical reasoning remains unsolved. While models like OpenAI O1 and DeepSeek-R1 handle competition-level problems like AIME reasonably well, they struggle on harder benchmarks — International Mathematical Olympiad problems, advanced theorem proving, and the most challenging subsets of Beyond-AIME and HMMT-25. The paper's experiments (Table 1) show that even Qwen3-4B-Instruct-2507, a strong post-trained model, achieves only 42.7% Pass@1 averaged across these benchmarks with CoT inference. A 57% failure rate on competition math is not a small gap — it represents problems that are fundamentally out of reach for CoT alone.

• Test-time compute scaling is bottlenecked by reasoning structure. The paper draws an important connection (Section 1, Figure 6) between the structure of reasoning and test-time scalability. When you allocate 1024 samples to CoT inference, you generate 1024 independent sequential reasoning chains. But if each chain follows essentially the same high-level approach (because CoT has no mechanism for structured exploration of alternative solution strategies), those 1024 samples are highly correlated — you're exploring the same neighborhood of solution space 1024 times. The paper shows that DAC-style reasoning, by decomposing problems into subproblems whose solutions can be varied independently, creates a combinatorially richer exploration space at test time. This is a structural argument, not just an empirical one: a reasoning paradigm that supports independent variation of solution components will scale better with compute than one that doesn't.

• Self-improvement pipelines have a hidden ceiling. The paper operates in a reinforcement learning framework (GRPO) where the model learns from its own generated solutions — a self-improvement loop. If the model cannot generate correct solutions for certain problems under any sampling condition, no RL algorithm can improve it on those problems, because there are no positive training signals to learn from. By switching to DAC reasoning, the model gains a new mechanism for generating correct solutions (via decomposition and structured conquering) that CoT alone couldn't produce, thereby expanding the set of problems that can enter the self-improvement loop. This is a theoretical motivation: DAC increases the effective exploration space of RL training.

Why CoT Hits a Ceiling: The Sequential Rigidity

To understand why CoT saturates, we need to understand what CoT fundamentally is as a reasoning strategy. In CoT, the model generates a single linear sequence of tokens that progresses from problem statement to final answer. There is no explicit decomposition, no independent sub-goals, no branching. If the model makes an error at step 7 of a 20-step reasoning chain, the remaining 13 steps are conditioned on that error — they cannot "recover" except through the model's ad-hoc self-correction mechanisms (which are unreliable, as documented by Huang et al., 2023 and others). Moreover, when the model samples multiple CoT solutions for the same problem, the diversity is limited: all solutions follow the same high-level approach (start from the given equations, manipulate algebraically, solve for variables) and differ only in local lexical choices, arithmetic details, or self-correction patterns.

This matters because reasoning diversity is the engine of test-time scaling. Best-of-N sampling improves accuracy only to the extent that the N samples represent genuinely different attempts at the problem. If all N attempts fail in the same way (e.g., all make the same sign error when manipulating a particular equation), no amount of sampling helps. The paper's observation that CoT RL fails to improve Qwen3-4B-Instruct-2507 (Figure 4, middle panel — the CoT curve is essentially flat) suggests that the model has exhausted the useful diversity in its CoT generation: every new CoT trajectory is essentially a minor variation of trajectories the model already knows how to produce, and none of these variations can crack the hardest problems.

Prior Work on Structured Reasoning: Promising but Misaligned

The divide-and-conquer strategy is not new — it's a classical algorithm design paradigm (Cormen et al., 2022). Nor is its application to LLM reasoning new. The paper acknowledges a substantial body of prior work that applies DAC-style approaches:

Prompting-based decomposition. Least-to-Most Prompting (Zhou et al., 2022) uses few-shot examples to guide LLMs in generating simpler subproblems from the original problem and solving them sequentially. Decomposed Prompting (Khot et al., 2022) modularizes complex tasks into sub-tasks handled by different prompts. Tree-of-Thought (Yao et al., 2023) and Graph-of-Thought (Besta et al., 2024) use explicit search over reasoning trees or graphs, where nodes represent intermediate reasoning states and edges represent transformations. DeAR (Xue et al., 2024) implements a Decompose-Analyze-Rethink cycle that explicitly breaks problems into components.

DAC in frontier models. Recent systems like Seed-Prover (Chen et al., 2025) and DeepSeek-Prover-V2 (Ren et al., 2025) incorporate DAC-style decomposition as a key component for tackling formal theorem proving and frontier mathematical reasoning, demonstrating that decomposition is not just a theoretical curiosity but a practically useful strategy at the cutting edge.

The common thread — and the common weakness. All of these approaches share a critical property: they implement DAC reasoning only at inference time, through prompts, while the model itself is trained exclusively with standard CoT-style reasoning. This creates what the paper identifies as a fundamental misalignment (Section 1, Figure 1): the model has been optimized during post-training to produce direct, step-by-step CoT solutions, but at test time it's suddenly asked to operate in a completely different mode — first decomposing the problem, then solving subproblems, then combining solutions. There is no training signal that reinforces good decomposition or punishes bad decomposition. The model is relying entirely on its generalization capabilities to handle a reasoning structure it was never explicitly taught.

The Evidence for Misalignment: Why Naive DAC Inference Fails

The paper provides direct evidence for this misalignment in Figure 2 (Section 1). The authors evaluate a range of instruction-following and reasoning models on four competition-level benchmarks (AIME 2024, AIME 2025, Beyond-AIME, HMMT-25) using both CoT-style inference and DAC-style inference (without any DAC-specific training). The results are striking:

• For Qwen2.5-7B-Instruct, naive DAC inference achieves an average Pass@32 of only 9.2% compared to CoT's 24.1% — DAC is nearly 3× worse.
• Even for the stronger Qwen3-4B-Instruct-2507, naive DAC achieves 71.9% versus CoT's 72.1% — essentially no improvement, despite the theoretically richer reasoning structure.
• The gap is most severe for smaller or less post-trained models: the Qwen2.5-7B-Instruct model goes from 0.4% Pass@1 with DAC to 5.6% with CoT (Table 1), a 14× difference.

This is the central empirical puzzle that motivates the paper: DAC is a theoretically superior reasoning strategy (more structured, better exploration, proven in classical algorithms), yet LLMs perform worse when asked to use it at inference time. The explanation, the paper argues, is the training-inference mismatch: the model has never been rewarded for good decomposition, never been shown examples of DAC-style solutions during training, and has no learned heuristics for how to divide problems effectively. It's analogous to asking a student who has only ever practiced solving math problems directly to suddenly explain their solution by first breaking it into subproblems — the metaskill of decomposition is itself a capability that requires training.

Where Existing Training Approaches Fall Short

The paper also positions itself relative to training-based approaches, though this is more implicit in the related work (Section 5) than explicitly contrasted:

Standard CoT RL training is the dominant but insufficient baseline. Most modern reasoning LLMs (OpenAI O1, DeepSeek-R1, Qwen3) are post-trained with reinforcement learning on CoT-style reasoning trajectories. The reward signal is simple: did the final answer match the ground truth? This works well for problems where CoT can eventually produce correct answers through self-exploration, but it creates no pressure toward structured decomposition. The model learns to reason better within the CoT paradigm, but never learns that there might be a better paradigm.

Ladder (Simonds and Yoshiyama, 2025) attempts DAC training but is limited. The paper acknowledges Ladder as the closest prior work — it decomposes integral problems into simpler sub-tasks and incorporates the solutions into training. However, Ladder focuses specifically on integral problems (a narrow domain) and uses a different training methodology (not end-to-end RL). The paper positions DAC-RL as a more general and principled framework that (1) works on arbitrary mathematical reasoning problems, (2) optimizes both decomposition and solving jointly through RL rather than treating decomposition as a pre-processing step, and (3) provides theoretical justification (Lemma 2.1) for why the final-answer reward signal is sufficient to train both stages.

No prior work addresses the misalignment head-on. The key gap the paper fills is not "DAC hasn't been tried" — as the authors document, DAC has been tried extensively at inference time. The gap is that no prior work has trained LLMs specifically for DAC reasoning in an end-to-end RL framework, with both decomposition and solving optimized jointly against the same final-answer reward signal. Prior work either (a) used DAC only at inference with CoT-trained models, or (b) trained models for DAC but only on narrow domains, or (c) used DAC as a data generation strategy rather than as the primary reasoning objective.

The Theoretical Motivation: Why Subproblem Decomposition Should Help

The paper provides a theoretical grounding for why DAC should be superior — but importantly, this grounding is conditional on the model being capable of effective decomposition, which is precisely what the training framework aims to provide.

Causal structure of the conquering step. Section 2.3 and Lemma 2.1 formalize the relationship between subproblem correctness and final answer correctness. The conquering phase first solves subproblems sequentially, then answers the original problem conditioned on those subproblem solutions. This induces a causal structure: $s_1, s_2, \dots, s_m \rightarrow C$, where $s_i$ is the correctness of subproblem $i$ and $C$ is the correctness of the final answer. Under the mild assumption that $P(C=1 \mid \mathbf{s})$ is monotonically non-decreasing in each $s_i$ (solving more subproblems correctly can't hurt), the final-answer reward $R(y_c) = \mathbf{1}\{\text{Extract}(y_c) = a\}$ is a consistent surrogate for subproblem correctness. In plain language: optimizing the policy to produce correct final answers also optimizes it to produce correct subproblem solutions, even though we never directly reward subproblem correctness.

This is important for two reasons. First, it means DAC-RL doesn't require any additional supervision beyond what CoT-RL already uses — the same ground-truth final answers suffice. Second, it means the feedback loop is aligned: the model learns to decompose more effectively because better decompositions lead to more correct final answers and thus higher rewards. There's no "credit assignment" problem where the model can't tell whether a given decomposition was good or bad.

Exploration via subproblem diversity. A second theoretical motivation — not formalized in a lemma but central to the paper's argument about test-time scalability — is that DAC creates an exponentially larger exploration space. Under CoT, sampling 1024 solutions gives you 1024 points in the space of complete reasoning trajectories. Under DAC with $n$ subproblem groups and $m$ conquering solutions per group, you effectively explore $n$ different decompositions, each of which is tested with $m$ different solving strategies. If decompositions and solutions are independent sources of variation, the effective exploration space is combinatorially larger, increasing the chance that at least one (decomposition, solution) pair leads to the correct answer.

How This Paper Positions Itself

The paper's positioning can be understood as making three nested claims:

Claim 1 (Diagnosis): There is a fundamental misalignment between CoT-centric post-training and DAC-style inference. This is an empirical claim, supported by Figure 2 and the baseline results in Table 1 (Init-CoT vs. Init-DAC). It's not that DAC is inherently worse — it's that current models haven't been trained for it.

Claim 2 (Solution): End-to-end RL training can resolve this misalignment and unlock DAC reasoning capabilities. This is the core methodological contribution: DAC-RL, which jointly optimizes problem division and subproblem conquering through GRPO with a combined reward scheme (Equations 1-3). The evidence is in Table 1 (RL-DAC vs. RL-CoT) and Figure 4.

Claim 3 (Implication): DAC reasoning has a higher performance ceiling and better test-time scalability than CoT. This is the forward-looking claim: even after both CoT and DAC are trained to convergence, DAC continues to improve and scales better with compute. The evidence is in Figure 4 (DAC curves continue rising while CoT flats), Figure 6 (DAC scales better with test-time compute), and the Deep DAC experiments (Table 1, bottom section).

Importantly, the paper does not claim that DAC replaces CoT for all problems. Section 4.1 (Mix-RL) explicitly studies a hybrid approach where simple problems use CoT and challenging problems use DAC, finding that this too outperforms pure CoT. The positioning is pragmatic: DAC is a more powerful tool for the hardest problems, and training models to use it appropriately raises the overall reasoning ceiling.

The Deeper Context: Why This Matters for the Field

Beyond the specific technical contributions, this paper engages with a broader question in the LLM reasoning literature: is CoT the final form of LLM reasoning, or is it just the first paradigm we figured out how to train? The saturation of CoT RL training (Figure 4) suggests that for frontier models, we're approaching the limits of what sequential step-by-step reasoning can achieve. If we want continued progress on the hardest reasoning benchmarks — the ones that matter for scientific discovery, formal verification, and advanced mathematics — we need new reasoning paradigms that structure the solution space differently.

DAC is not the only candidate (tree-of-thought, graph-of-thought, program-of-thought, and multi-agent debate are others), but it is one of the most principled, drawing on decades of computer science wisdom about how to solve complex problems. The paper's contribution is showing that this classical strategy can be learned by LLMs through reinforcement learning, not just prompted at inference time, and that learning it unlocks capabilities that CoT training alone cannot reach.

This connects to a larger narrative about the relationship between architecture, training, and reasoning. The paper's findings suggest that the "reasoning architecture" (how a model structures its problem-solving process) is partially orthogonal to model scale — a 4B model with DAC training can outperform its own CoT-trained self by substantial margins, and mixing DAC training into CoT training improves CoT performance by over 10% (Section 4.1). This implies that some of the reasoning capability we've been attributing to model scale might actually be attributable to reasoning structure, and that better reasoning paradigms could make smaller models significantly more capable — an important consideration for practical deployment.

3. Technical Approach

3.1 Reader Orientation

This paper presents DAC-RL, an end-to-end reinforcement learning framework that trains language models to solve difficult math problems by first decomposing them into subproblems (the "divide" step) and then solving those subproblems sequentially to arrive at a final answer (the "conquer" step), rather than relying solely on step-by-step chain-of-thought reasoning. The core problem the system solves is the performance ceiling of CoT reasoning: when a model consistently fails to produce correct CoT solutions for certain problems, standard CoT-based RL cannot improve it because there are no positive training signals. DAC-RL breaks through this ceiling by teaching the model a more structured reasoning paradigm that creates new pathways to correct answers — pathways that CoT alone cannot discover — and jointly optimizes both the decomposition strategy and the solving strategy through reinforcement learning with only final-answer correctness as the reward signal.

3.2 Big-Picture Architecture

The DAC-RL system has five major components that operate in a training loop (Algorithm 1):

1. Base Policy Model ($\pi_\theta$) — a pretrained LLM (Qwen2.5-7B-Instruct or Qwen3-4B-Instruct-2507) that serves as the starting point for RL training. It is fine-tuned via GRPO to perform both division and conquering.

2. Division Stage — given a problem $x$, the policy generates $G_d = 4$ groups of subproblems $\{\mathcal{P}_g\}_{g=1}^{4}$, each containing $n_g \geq 3$ subproblems. The division prompt (Figure 12) instructs the model to break the problem down without solving it.

3. Conquering Stage — for each subproblem group $\mathcal{P}_g$, the policy generates $G_c = 8$ complete solution candidates $\{y_{g,v}\}_{v=1}^{8}$ using a conquering prompt (Figure 13) that concatenates the original problem with the subproblems and asks the model to solve subproblems sequentially, then produce the final answer.

4. Reward Assignment — the correctness of each conquering response is evaluated against the ground-truth answer $a$ to produce a binary reward $\mathbf{R}(y_c) \in \{0, 1\}$. These conquering rewards are then aggregated per subproblem group to compute a division reward $\mathbf{R}(y_d)$ (Equation 2), which also incorporates format and quantity validity checks.

5. Policy Optimization (GRPO) — both division and conquering experiences are stored in a buffer $\mathcal{B}$ and used to update $\pi_\theta$ via Group Relative Policy Optimization (Equation 6), extended with Clip-Higher and token-level loss techniques from DAPO.

Information flows cyclically: a batch of problems enters → the policy divides each problem into $G_d$ subproblem groups → each group is used to generate $G_c$ conquering solutions → conquering rewards are computed from final-answer correctness → division rewards are derived from conquering outcomes and format checks → both sets of experiences update the policy → the updated policy generates better divisions and solutions in the next iteration.

3.3 Roadmap for the Deep Dive

• First, the formal optimization objective (Equation 1), which defines what DAC-RL optimizes and how division and conquering rewards are combined — this sets up the overall learning problem.
• Second, the subproblem division mechanism — how the model is prompted to decompose problems, the reward scheme that shapes this behavior (Equation 2), and the critical design choices that prevent collapse into CoT-like behavior.
• Third, the conquering mechanism and its reward structure (Equation 3), including Lemma 2.1 which provides the theoretical justification for why final-answer correctness alone can train subproblem solving — a key insight that makes the framework practical.
• Fourth, the training algorithm (Algorithm 1) and GRPO optimization details, connecting the reward assignments to the actual policy gradient updates, including hyperparameters and implementation choices like Clip-Higher and token-level loss.
• Fifth, the inference procedure, which differs from training — explaining how DAC-style reasoning is deployed at test time and how the budget allocation between division diversity and conquering depth affects scalability (Figure 6).

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a reinforcement learning systems paper whose core idea is that divide-and-conquer reasoning is a learnable skill that can be acquired through end-to-end RL with only final-answer supervision, and that acquiring this skill raises the model's reasoning ceiling beyond what CoT training alone can achieve.

---

The Joint Optimization Objective

The paper frames DAC-RL training as maximizing the expected sum of two reward terms: one for the quality of problem decomposition (the division response $y_d$) and one for the quality of solution execution (the conquering response $y_c$). The objective is stated in Equation 1:

$\mathcal{J}(\theta) = \mathbb{E}_{y_d, y_c \sim \pi_\theta} [\mathbf{R}(y_d) + \mathbf{R}(y_c)]$

where $\theta$ denotes the parameters of the policy model $\pi_\theta$, $y_d$ is the division response (text containing subproblems), $y_c$ is the conquering response (text solving subproblems and the original problem), $\mathbf{R}(y_d)$ is the division reward defined in Equation 2 (Section 2.2), and $\mathbf{R}(y_c)$ is the conquering reward defined in Equation 3 (Section 2.3).

What it computes: The objective is the expected total reward per problem, summing the division-stage reward and the conquering-stage reward, where the expectation is taken over the policy's own generated responses. During training (Algorithm 1, lines 5-13), the expectation is approximated by sampling: for each problem, the policy generates $G_d = 4$ division responses and $G_c = 8$ conquering responses per division, and the sample average of $\mathbf{R}(y_d) + \mathbf{R}(y_c)$ across these rollouts serves as the empirical objective for that iteration's gradient update.

Why this form: This additive decomposition separates two distinct capabilities that the policy must learn — what to decompose (division) and how to execute the decomposition (conquering) — while allowing them to be optimized jointly from the same underlying model. An alternative would be to only reward the final conquering answer, but this would provide no direct training signal for the division step. Without $\mathbf{R}(y_d)$, the policy could learn to ignore the division prompt entirely and simply solve the original problem directly (which is exactly what happens in the failure mode described in Appendix C). The division reward creates explicit pressure to produce useful subproblems, while the conquering reward ensures those subproblems are actually solved correctly. The additive combination means both capabilities are equally important — a perfect decomposition with terrible execution scores no better than a terrible decomposition with perfect execution.

A critical implementation detail: these two rewards are not computed independently. The division reward $\mathbf{R}(y_d)$ is derived from the conquering outcomes (Equation 2 uses $\text{CA}(\mathcal{P}_g)$, the conquering accuracy for subproblem group $\mathcal{P}_g$), creating a dependency where the division stage is evaluated by how well its outputs enable the conquering stage. This coupling is what makes the training end-to-end: improving division improves conquering success rates, which increases division rewards, which reinforces better division, creating a virtuous cycle.

---

Subproblem Division: Prompting, Constraints, and Reward Design

The division stage is where the policy learns to decompose a complex problem into simpler subproblems that, if solved, would lead to the solution of the original problem. This is not a free-form generation — it is carefully constrained through prompt design, formatting requirements, quantity requirements, and a reward scheme that balances multiple objectives.

The division prompt (Figure 12). The policy receives the original problem text along with explicit instructions: "Your task is NOT to solve the problem. Instead, break it down into more than 3 subproblems that, if solved, would naturally lead to the solution of the original problem." The prompt provides a template with XML-style tags (<SUBPROBLEM i> ... </SUBPROBLEM i>) and emphasizes format compliance: "Do not provide any explanations or answers. Only list the subproblems using the tags."

This prompt design addresses a specific failure mode documented in Appendix C. Early experiments used a simpler division reward — the average conquering accuracy — as the reward for division responses. The authors found that "when conquer-accuracy-based rewards are used to reward subproblems division, the model tends to solve the problem prematurely during the division stage (as illustrated in Case 8), rather than decomposing it into subproblems." In Case 8 (Figure 8), the model ignores the division instruction and instead generates what are essentially intermediate solution steps: "53.463 + 12.9873 = 66.4503" labeled as a "subproblem." This is not division — it is just doing the original problem in small chunks while calling the chunks "subproblems."

The prompt's explicit prohibition against solving ("Your task is NOT to solve the problem") and the strict format requirements are designed to prevent this collapse. But the authors learned through their initial failure that prompts alone are insufficient — the reward structure must also be aligned. The division reward in Equation 2 is specifically designed to not directly reward high conquering accuracy, which would incentivize the model to embed solution steps in the division to "help" the conquering stage. Instead, it rewards a lower bound on subproblem helpfulness, as we will see.

The quantity constraint: preventing subproblem collapse. The paper observes another failure mode: "without this constraint, it collapses to producing no useful subproblems in division response $y_d$ as the training goes, and the conquering stage degrades to directly solving the original problem, as in standard CoT reasoning" (Section 2.2). The model's natural tendency is to take the path of least resistance — if it can get high conquering rewards by just solving the original problem directly (ignoring the subproblems), the division stage becomes vestigial. The minimum subproblem count $N_s = 3$ prevents this by docking the division reward to zero if fewer than $N_s$ subproblems are generated, forcing the model to maintain the division structure even when it could solve the problem without it.

The division reward scheme (Equation 2). The reward for a division response $y_d$ producing subproblem group $\mathcal{P}_g$ is:

$\mathbf{R}(y_d) = \begin{cases} 0, & \underbrace{|\mathcal{P}_g| < N_s}_{\text{quantity}} \vee \underbrace{\neg \text{Format}(y_d)}_{\text{format}} \\ 0, & \underbrace{\text{CA}(\mathcal{P}_g) = 0 \wedge \text{CA}(\{\mathcal{P}_i\}_{i=1}^{G_d}) > 0}_{\text{helpfulness}} \\ 1, & \text{otherwise,} \end{cases}$

where $|\mathcal{P}_g|$ is the number of subproblems in group $\mathcal{P}_g$, $N_s = 3$ is the minimum required subproblems, $\text{Format}(y_d)$ is a boolean indicating whether the division response is parseable via regular expressions (the XML tags are extractable), $\text{CA}(\mathcal{P}_g)$ is the conquering accuracy for that subproblem group (the fraction of the $G_c = 8$ conquering solutions that produce the correct final answer), and $\text{CA}(\{\mathcal{P}_i\}_{i=1}^{G_d})$ is the conquering accuracy across all $G_d = 4$ groups.

What it computes: The reward is 1 for "good enough" division and 0 otherwise, evaluated through three conditions in order. First, format and quantity must both pass — if the response is unparseable or has too few subproblems, reward is 0. Second, the helpfulness condition: if this subproblem group produces zero correct conquering solutions ($\text{CA}(\mathcal{P}_g) = 0$) but some other group from the same problem produces at least one correct solution ($\text{CA}(\{\mathcal{P}_i\}_{i=1}^{G_d}) > 0$), then this group gets a reward of 0 — it failed to help when help was possible. Third, otherwise (format and quantity pass, and either this group produced a correct solution or no group did), the reward is 1.

Why this form — the lower-bound logic. This is the most subtle design choice in the paper and directly addresses the failure mode from Appendix C. The reward does not increase with higher conquering accuracy — a group with 8/8 correct solutions and a group with 1/8 correct solutions both get a reward of 1 (assuming format and quantity pass). This is deliberate. If the reward were proportional to conquering accuracy, the model would be incentivized to make the conquering stage easier by embedding solution information in the division — exactly the behavior observed in Case 8 (Figure 8). By only requiring that the division be sufficient to enable at least one correct solution (and penalizing only when another group succeeded while this one failed), the reward optimizes a lower bound on helpfulness rather than maximal helpfulness.

The authors explain: "This relaxation reduces greedy behavior in early training and prevents the policy from prematurely optimizing for the original problem, thereby preventing this failure mode." The key insight is that over-optimizing the division to maximize conquering accuracy is actually harmful because it leads to the division stage doing the work of the conquering stage, collapsing the two-stage structure into a single stage. By rewarding only whether the division is "adequate" (enables at least one correct solution when solutions are possible), the training preserves the functional separation between decomposition and execution.

The third condition — giving a reward of 1 when no group produces a correct solution — is also important. On genuinely hard problems where the model cannot find any correct solution regardless of decomposition, penalizing all divisions would provide no useful gradient. Instead, the model is "let off the hook" on these problems for the division stage (reward 1), allowing it to focus its learning on problems where division quality actually matters.

Format validity. The paper requires that subproblems be parsable by regular expressions — meaning the XML tag structure must be extractable. If the <SUBPROBLEM i> tags cannot be parsed, the subproblems cannot be inserted into the conquering prompt (Figure 13), making the division useless regardless of its content. This is a hard constraint: $\neg\text{Format}(y_d)$ forces $\mathbf{R}(y_d) = 0$ regardless of everything else.

---

Subproblem and Original Conquering: Execution and Reward Assignment

Once subproblems are generated, the conquering stage takes over. For each subproblem group $\mathcal{P}_g$, a conquering prompt is constructed by filling the template in Figure 13: the original problem is inserted at {REPLACE}, and the subproblems are inserted in their XML-tagged format. The prompt instructs the model to "Solve the subproblems sequentially (in the given order)," "Show detailed reasoning for each subproblem," "Do not use \boxed when solving subproblems," and finally "After solving all subproblems, combine their results to solve the original problem" and present the final answer in \boxed{}.

The conquering reward (Equation 3). The reward for a conquering response $y_c$ is simply:

$\mathbf{R}(y_c) = \mathbf{1}\{\text{Extract}(y_c) = a\}$

where $\text{Extract}(y_c)$ is the final answer extracted from the generated response (presumably by parsing the \boxed{} content), $a$ is the ground-truth answer to the original problem, and $\mathbf{1}\{\cdot\}$ is the indicator function (1 if the condition is true, 0 otherwise).

What it computes: A binary reward — 1 if the final extracted answer matches the ground truth, 0 otherwise. There is no partial credit, no reward for solving subproblems correctly, no reward for good formatting. The only thing that matters is whether the final answer to the original problem is correct.

Why this form — the theoretical justification (Lemma 2.1). At first glance, using only final-answer correctness to train subproblem solving seems problematic: how does the model learn to solve subproblems if subproblem solutions are never directly rewarded? The answer is Lemma 2.1, which states:

$\text{Cov}_\theta(\mathbf{1}\{s_i = 1\}, \mathbf{1}\{C = 1\}) \geq 0$

where $s_i \in \{0, 1\}$ indicates whether subproblem $i$ is solved correctly, $C \in \{0, 1\}$ indicates whether the original problem is solved correctly, and $\text{Cov}_\theta$ is the covariance under the policy's distribution.

In plain language: There is a non-negative correlation between solving subproblems correctly and solving the original problem correctly. Under the monotonicity assumption (solving more subproblems correctly never decreases the probability of a correct final answer), trajectories with more correct subproblems are more likely to also have $C = 1$. Therefore, a policy update that increases $P_\theta(C = 1)$ (which is what the gradient from Equation 3 does) will, on average, also increase the probability of correct subproblem solutions.

The proof (Appendix B) walks through this: expand $P_\theta(C = 1) = \sum_{\mathbf{s}} P(C = 1 \mid \mathbf{s}) P_\theta(\mathbf{s})$, note that $P(C = 1 \mid \mathbf{s})$ is an increasing function of $\mathbf{s}$ (by monotonicity), so increasing $P_\theta(C = 1)$ requires shifting probability mass toward configurations $\mathbf{s}$ with more 1s. Using Bayes' rule, $P_\theta(S_i = 1 \mid C = 1) \geq P_\theta(S_i = 1)$ — correct subproblems are overrepresented among correct final answers — and therefore the covariance is non-negative.

The practical implication: The conquering reward serves as a consistent surrogate for subproblem correctness. Even though the model never sees whether individual subproblems were solved correctly, the policy gradient automatically favors trajectories that solve subproblems correctly because those trajectories are more likely to produce correct final answers and thus receive positive rewards. This is what makes the framework "end-to-end": the same scalar reward trains both subproblem solving and final problem solving, without requiring any subproblem-level supervision (which would be expensive or impossible to obtain, since subproblems are generated by the model itself and have no pre-existing ground-truth answers).

The causal direction requirement. The lemma assumes the causal direction $\mathbf{s} \rightarrow C$ — subproblem correctness determines final answer correctness, not vice versa. This is enforced by the conquering prompt structure (Figure 13): the model must solve subproblems sequentially first, then address the original problem conditioned on those subproblem solutions. The sequential ordering in the prompt creates this causal structure operationally, even though the underlying transformer processes all tokens in parallel.

The $G_c$ parallel samples. For each subproblem group, the policy generates $G_c = 8$ independent conquering solutions (Algorithm 1, line 7). This parallelism serves two purposes. First, it provides more reliable estimates of conquering accuracy for the division reward calculation — with 8 samples, $\text{CA}(\mathcal{P}_g)$ is a better estimate of whether the subproblem group is genuinely helpful than a single sample would be. Second, it increases the diversity of conquering trajectories, which feeds into the GRPO advantage computation (Equation 5): the advantage of each token depends on the group-normalized rewards, so having more samples in the group provides a more stable baseline.

---

Policy Optimization: GRPO with Extensions

The paper uses Group Relative Policy Optimization (GRPO) as the underlying RL algorithm (described in Appendix A), extended with techniques from DAPO (Yu et al., 2025a). GRPO is a variant of PPO (Schulman et al., 2017) that eliminates the need for a separate critic (value function) model by computing advantages relative to a group of rollouts from the same prompt.

Group-relative advantage computation (Equation 5). For a group of $G$ responses $\{y_i\}_{i=1}^G$ to the same prompt (in DAC-RL, these are either $G_d$ division responses for the same problem with their computed division rewards, or $G_c$ conquering responses for the same subproblem group with their binary conquering rewards), the advantage for each token in response $y_i$ is:

$A_{i,t} = \frac{r_i - \text{mean}(\{r_i\}_{i=1}^G)}{\text{std}(\{r_i\}_{i=1}^G)}$

where $r_i$ is the scalar reward for response $y_i$ (either $\mathbf{R}(y_d)$ or $\mathbf{R}(y_c)$), $\text{mean}(\{r_i\})$ is the average reward across the group, and $\text{std}(\{r_i\})$ is the standard deviation.

What it computes: A z-score normalized advantage signal. Responses that score above the group mean get positive advantages (their tokens are "good" and should be reinforced); responses below the mean get negative advantages (their tokens are "bad" and should be suppressed). The advantage is the same for every token in a given response — GRPO does not assign per-token rewards, only per-response rewards.

Why this form: The group-relative normalization eliminates the need for a learned value baseline. In standard PPO, the advantage is $r - V(s)$ where $V(s)$ is a learned value function that estimates expected future reward. GRPO approximates $V(s)$ with the group mean, using the fact that all responses in the group were generated for the same prompt, so their expected rewards should be similar. The standard deviation normalization keeps the advantage scale roughly constant across training, preventing gradient magnitudes from exploding as the policy improves.

Policy gradient objective (Equation 6). The full GRPO objective is:

$\mathcal{J}(\theta) = \mathbb{E}_{x \sim \mathcal{D}, \mathbf{Y} \sim \pi_{\theta_{\text{old}}}(\cdot|x)} \left[ \frac{1}{G} \sum_{i=1}^G \frac{1}{|y_i|} \sum_{t=1}^{|y_i|} \left( \min \left( k_{i,t}(\theta) A_{i,t}, \text{clip}(k_{i,t}(\theta), 1 - \epsilon, 1 + \epsilon) A_{i,t} \right) - \beta D_{\text{KL}}(\pi_{\theta} \parallel \pi_{\text{ref}}) \right) \right]$

where $\mathcal{D}$ is the training dataset (DAPO-Math-17k), $\pi_{\theta_{\text{old}}}$ is the policy from the previous iteration (used to generate the rollouts $\mathbf{Y}$), $G$ is the group size (varies — $G_d = 4$ for division, $G_c = 8$ for conquering), $|y_i|$ is the number of tokens in response $y_i$, $k_{i,t}(\theta) = \frac{\pi_{\theta}(y_{i,t}|x,y_{i,<t})}{\pi_{\theta_{\text{old}}}(y_{i,t}|x,y_{i,<t})}$ is the probability ratio (how much more or less likely the current policy is to generate token $y_{i,t}$ compared to the old policy), $A_{i,t}$ is the advantage from Equation 5, $\epsilon$ is the clipping threshold, $\beta$ is the KL penalty coefficient, and $D_{\text{KL}}$ is the Kullback-Leibler divergence from the reference model $\pi_{\text{ref}}$.

What it computes: A per-token loss that is averaged across all tokens in all responses in the group. For each token, the "raw" objective is $k_{i,t}(\theta) A_{i,t}$ — the probability ratio times the advantage. If the advantage is positive, the policy should increase $k_{i,t}$ (make this token more likely); if negative, decrease it. The $\min$ with the clipped version implements a trust region: $k_{i,t}$ is clipped to $[1 - \epsilon, 1 + \epsilon]$, preventing any single update from changing token probabilities too drastically (this is the PPO-style clipping that stabilizes training). The KL term $\beta D_{\text{KL}}(\pi_{\theta} \parallel \pi_{\text{ref}})$ penalizes the policy for deviating too far from the reference model (typically the initial pretrained model), preventing catastrophic forgetting and reward hacking.

Why this form — the DAPO extensions. The paper incorporates two specific techniques from DAPO that modify the standard GRPO objective:

1. Clip-Higher: The clipping upper bound $\varepsilon_h = 0.28$ is set higher than the lower bound. This allows the policy to increase probabilities of high-advantage tokens more aggressively than it decreases probabilities of low-advantage tokens. The intuition is that in reasoning tasks, positive signals are rarer and more informative than negative signals — most tokens in an incorrect solution are perfectly reasonable, and only a few are "wrong." Clip-Higher prevents the policy from being overly conservative about reinforcing good behavior.

2. Token-level loss: The loss is computed and averaged at the token level ($\frac{1}{|y_i|} \sum_{t=1}^{|y_i|}$), rather than at the response level. This ensures that longer responses don't dominate the gradient simply because they contain more tokens. Without token-level averaging, the policy would have a bias toward generating shorter responses (fewer tokens = smaller per-response loss contribution), which could lead to degenerate short solutions that happen to be correct.

Training hyperparameters. The paper specifies (Section 3.1): training batch size = 256 (problems per iteration), maximum rollout length = 8,192 tokens, sampling temperature = 1.0, Clip-Higher upper bound $\varepsilon_h = 0.28$, mini-batch size = 64 (for gradient accumulation), and 400 training steps (approximately 6 epochs over DAPO-Math-17k) for the main experiments. For the Deep DAC setting, an additional ~200 steps are performed on a difficulty-filtered subset of 3.7k problems where the initial policy solves with less than 50% accuracy across 16 responses, with the token budget increased to 16,384 for training and 24,576 for inference.

The experience buffer and update cycle (Algorithm 1). The training loop follows a specific sequence designed to generate diverse, on-policy experiences before each update:

1. Sample a mini-batch of problems from the training set.
2. Divide: For each problem, generate $G_d = 4$ subproblem groups (line 5). Each group $\mathcal{P}_g$ may contain a variable number of subproblems $n_g$ (the model decides how many, as long as $n_g \geq N_s = 3$).
3. Conquer: For each subproblem group, generate $G_c = 8$ conquering solutions (line 7). Compute their rewards against the ground-truth answer (line 8) and store $([x; \mathcal{P}_g], y_{g,v}, \mathbf{R}(y_{g,v}))$ tuples in the buffer (line 9).
4. Compute division rewards: After all conquering solutions for a problem are generated, evaluate format and quantity validity (line 11), compute conquering accuracy per group, and derive division rewards via Equation 2 (line 12). Store division tuples $(x, \mathcal{P}_g, \mathbf{R}(\mathcal{P}_g))$ in the buffer (line 13).
5. Update policy: After processing the entire mini-batch, update $\pi_\theta$ using all stored experiences according to the GRPO objective (line 15). Clear the buffer (line 16).

This "generate-then-update" cycle is standard for on-policy RL: all experiences in the buffer were generated by the same policy $\pi_\theta$, so the gradient computed from them is an unbiased estimate of the policy gradient for that policy.

---

Inference: DAC-Style Reasoning at Test Time

At inference time, DAC reasoning follows a simplified version of the training pipeline, with two key differences:

1. No RL updates: The policy is frozen after training. All generation is for evaluation only.
2. Flexible budget allocation: The number of subproblem groups and conquering samples can be varied independently, trading off between division diversity and conquering depth.

Standard DAC inference. For the main results in Table 1, DAC inference uses the same pipeline as training: the policy divides the problem into one subproblem group (the prompt does not request multiple groups at test time), then generates a single conquering solution. For Pass@1 evaluation, this is done 32 times independently, and the average accuracy is reported. For Pass@32 evaluation, the unbiased estimator from Chen et al. (2021) is used: generate $n \geq k$ solutions, then compute Pass@k as $1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}$ where $c$ is the number of correct solutions, which provides an unbiased estimate of the probability that at least one of $k$ samples is correct.

Test-time scalability analysis (Figure 6). The paper investigates how DAC scales with compute by varying the allocation of a fixed budget of $k = 1024$ total generations between:

• $n$ = number of independent subproblem groups (division diversity)
• $m$ = number of conquering solutions per group (conquering depth) such that $n \times m = k$. The configurations tested range from $(n=1, m=1024)$ (one decomposition, 1024 solving attempts — essentially CoT-like since there's no diversity in decomposition) to $(n=1024, m=1)$ (1024 different decompositions, one solving attempt each).

The key finding (Figure 6, described in Section 4.2) is that allocating more budget to division diversity (larger $n$) consistently outperforms allocating more budget to conquering depth (larger $m$) at the same total compute. This empirically validates the theoretical motivation from Section 1: subproblem diversity expands the model's exploration space, increasing the chance that at least one decomposition-solution pair succeeds.

The conquering prompt at inference (Figure 13). The conquering prompt is the same as in training, with the original problem and subproblems inserted via text replacement. The prompt explicitly instructs the model to solve subproblems sequentially ("in the given order"), which creates the causal structure assumed by Lemma 2.1. The instruction "Do not use \boxed when solving subproblems" ensures that the final answer extraction only captures the answer to the original problem, not intermediate subproblem answers. The final answer format \boxed{\text{final answer here}} enables reliable parsing of the model's answer for comparison with ground truth.

Format constraint experiments (Section 4.5, Table 3). The paper also experiments with a stricter conquering format during training: requiring that responses explicitly label and answer each subproblem ("subproblem i" for $i = 1, \dots, n_g$). Responses that fail to follow this format receive a negative reward. The results in Table 3 show that while the format-following rate increases from 42.6% to 92.1%, the actual benchmark performance drops significantly — from 51.3% average Pass@1 to 45.2%. This is an example of the alignment tax (Lin et al., 2023; Wang et al., 2025): forcing strict instruction-following degrades downstream reasoning quality. The default DAC-RL configuration does not enforce this strict format constraint, allowing the model to solve subproblems in a more natural, less structured way as long as the final answer is correct.

---

Cold-Start Initialization (Section 4.4)

For experiments with cold-start distillation (Table 2), the training pipeline is modified to include a supervised fine-tuning (SFT) phase before RL:

1. Distillation data generation: 3,000 problems are randomly selected from the DeepMath-103K dataset with difficulty labels greater than 6. A stronger instruction model (Qwen3-235B-A22B-2507-Instruct) generates both CoT and DAC-style solutions for each problem, yielding 6,000 problem-response pairs total.

2. Mixed SFT training: The CoT and DAC distillation data are combined into a single training set. The Qwen3-4B-Instruct-2507 model is fine-tuned on this mixture for 5 epochs with a maximum token length of 32,768.

3. Subsequent RL: The SFT-checkpointed model then undergoes the standard DAC-RL training procedure (or CoT-RL for the CD-RL-CoT baseline).

This cold-start approach addresses a concern that smaller models may have "limited instruction-following capacity, which may hinder exploration during solution generation and policy optimization" (Section 4.4). By first teaching the model what good DAC solutions look like through supervised distillation from a larger model, the subsequent RL phase starts from a better initialization with higher-quality exploration.

The results (Table 2, CD-RL-DAC) show that cold-start distillation improves DAC-RL performance: CD-RL-DAC achieves 53.5% average Pass@1 compared to 46.1% for the standard RL-DAC (without cold-start). Interestingly, the distilled model already performs better with DAC than with CoT before any RL (CD-DAC: 46.4% vs. CD-CoT: 43.4%), even though the initial model performs better with CoT (Init-CoT: 42.7% vs. Init-DAC: 40.2%). This suggests that the distillation process transfers DAC-specific knowledge that the smaller model cannot acquire through RL alone — possibly because the larger teacher model can demonstrate decomposition strategies that the smaller model would not discover through self-exploration.

4. Key Insights and Innovations

Innovation 1: Training–Inference Misalignment as the Root Cause of DAC Underperformance

Prior work on divide-and-conquer reasoning for LLMs (Tree-of-Thought, Least-to-Most Prompting, DeAR, Seed-Prover, DeepSeek-Prover-V2) operated under an implicit assumption: that an LLM trained to reason well with chain-of-thought should, when given appropriate prompts at inference time, be able to generalize to DAC-style reasoning. The reasoning was "if the model can solve problems step-by-step, it should also be able to decompose problems and solve subproblems — it just needs the right instructions." This paper’s central diagnostic move is to reject that assumption and instead identify a structural misalignment between CoT-centric post-training and DAC-style inference as the fundamental bottleneck.

The evidence for this diagnosis is stark. Figure 2 and Table 1 show that Qwen2.5-7B-Instruct, when asked to perform DAC reasoning at inference time without any DAC-specific training, achieves an average Pass@32 of only 9.2% across four competition-level benchmarks — compared to 24.1% for CoT. That’s nearly a 3× regression from using what is, in principle, a richer reasoning structure. Even the stronger Qwen3-4B-Instruct-2507 sees essentially no benefit (71.9% vs. 72.1%), meaning the model’s extensive post-training for CoT reasoning provides zero transfer to DAC — the reasoning capability that should generalize does not.

This is not merely an empirical observation; it is a reframing of the problem. The field has been treating DAC as a prompting strategy — something you do at inference time to get better outputs from an already-capable model. This paper argues that DAC is actually a skill — something the model must be trained to do, because the meta-cognitive operations involved (identifying independent subproblems, structuring their solutions to feed into a final synthesis, maintaining the causal relationship between subproblem correctness and final answer correctness) are not acquired through CoT training. A model trained only on "start from the given information, reason forward, produce the answer" has never been rewarded for saying "this problem actually consists of three independent subproblems, let me solve each one and combine them."

What makes this diagnosis fundamental rather than incremental is that it explains the pattern of conflicting results in prior work. Systems like Seed-Prover and DeepSeek-Prover-V2 showed DAC-style reasoning working well — but they built specialized pipelines with explicit search, verifiers, and prompt engineering that effectively compensated for the model's lack of DAC training. Meanwhile, simpler prompting-based approaches like Least-to-Most showed modest gains on some tasks but not others — likely depending on whether the model could "fake" DAC reasoning by essentially doing CoT with subproblem labels. The paper's diagnosis unifies these observations: DAC fails when the gap between training and inference is large, succeeds (partially) when external scaffolding bridges that gap, but can only reach its full potential when the gap is closed through training.

---

Innovation 2: End-to-End RL as a Mechanism for Acquiring Decomposition as a Learned Skill

The paper’s methodology contribution — training DAC reasoning through reinforcement learning — is significant not because RL is novel (it’s standard in LLM post-training), but because it demonstrates that problem decomposition can be acquired as an emergent skill through self-exploration with only final-answer feedback. This is a conceptually important finding: it means the model can learn what makes a good decomposition without ever being shown examples of good decompositions, without ever receiving direct feedback on its subproblem quality, and without any subproblem-level ground-truth answers.

Prior work on structured reasoning either (a) used hand-crafted decomposition templates or few-shot examples to tell the model how to decompose (Least-to-Most, Decomposed Prompting), or (b) used external verifiers or search procedures to evaluate decomposition quality (Tree-of-Thought, DeepSeek-Prover-V2). DAC-RL does neither. The division reward (Equation 2) provides only a binary signal — was the decomposition "adequate enough" to enable at least one correct final answer? — and the conquering reward (Equation 3) provides only final-answer correctness. Yet from these sparse signals, the policy learns to produce varied, non-trivial subproblems that become progressively more useful over training (Figure 4, where DAC accuracy starts below CoT and rises past it).

The theoretical underpinning (Lemma 2.1) is essential here: the final-answer reward is a consistent surrogate for subproblem correctness because of the causal structure enforced by the conquering prompt (subproblem solutions → final answer, not vice versa). This means the RL credit assignment problem — "did the solution fail because the decomposition was bad or because the execution was bad?" — is partially resolved by the structure of the task itself, not by a clever algorithmic trick. The model learns that decompositions leading to correct final answers are good, and decompositions that consistently fail (when other decompositions for the same problem succeed) are bad. No explicit credit assignment mechanism is needed; the training loop’s own statistics surface this signal.

This is a fundamental advance because it shows that LLMs can acquire meta-cognitive skills through RL in ways that go beyond what prompting or supervised fine-tuning can achieve. A model shown DAC examples through SFT learns to imitate decomposition patterns. A model trained through RL learns to invent decompositions that work for novel problems, because it has been optimized for the outcome (correct final answer) rather than for mimicking a process. The cold-start experiments (Table 2) provide suggestive evidence here: distillation from a strong teacher helps, but the largest gains come from subsequent RL (CD-RL-DAC: 53.5% vs. CD-DAC: 46.4%), consistent with the idea that RL enables the model to go beyond imitation into genuine exploration of decomposition strategies.

---

Innovation 3: The Performance Ceiling of CoT as a Trainable Limit, Not an Architectural Limit

One of the most theoretically significant findings in this paper is that CoT reasoning has a performance ceiling that can be characterized and exceeded by switching reasoning paradigms — and that this ceiling is a property of the reasoning structure, not the model architecture or scale. Section 4.1 reports that for Qwen3-4B-Instruct-2507, further CoT-style RL training "yields no additional gains, as the policy has already been extensively post-trained with CoT reasoning." The CoT RL curve in Figure 4 (middle panel) is essentially flat. Yet DAC-RL training on the same model, using the same base architecture, same dataset, same RL algorithm, and same reward signal (only final-answer correctness), continues to improve, reaching 46.1% Pass@1 compared to CoT’s saturated 37.5%.

This is not just "DAC is better than CoT" — it’s evidence for a stronger claim: CoT saturates because of what it is as a reasoning strategy, not because of limits on the model’s capacity to improve. When a model fails on a problem with CoT, it fails because all CoT trajectories for that problem are incorrect. RL cannot improve CoT performance on these problems because there are no positive training signals — every trajectory gets zero reward. DAC breaks this deadlock by providing an alternative pathway: even if the model cannot produce a correct CoT solution for a problem, it can produce a decomposition that makes the problem tractable, and then solve it through the conquering mechanism. Once correct solutions enter the training data via DAC, the self-improvement loop can operate.

This finding has broad implications for the field’s understanding of reasoning capabilities in LLMs. It suggests that the limits of a model’s reasoning are partially determined by the reasoning paradigm it employs, not just by its parameter count or training data. This is a more optimistic view than the implicit assumption in much of the scaling literature — that bigger models trained on more data will naturally develop whatever reasoning strategies are needed. The paper’s results argue for a more active role for reasoning paradigm design: continuing to scale CoT training on a model that has already saturated may be wasted compute, when switching to DAC training could unlock additional gains at the same or lower cost.

The Mix-RL experiments (Section 4.1, Figure 5) provide convergent evidence. Incorporating DAC training on challenging problems into an otherwise CoT-focused training run improves CoT performance by over 10% across all benchmarks — even though the model is doing less CoT training than a pure CoT-RL baseline. This is a striking transfer effect: learning to decompose problems well makes the model better at solving them directly, even when not using the decomposition at test time. The most natural interpretation is that DAC training teaches the model something about problem structure that generalizes back to CoT reasoning — perhaps helping the model recognize when a direct approach is going down a futile path, or teaching it to structure its CoT steps more effectively.

---

Innovation 4: Subproblem Diversity Over Conquering Depth as the Engine of Test-Time Scalability

The paper’s analysis of test-time compute allocation (Section 4.2, Figure 6) reveals a finding that is counterintuitive given the standard intuition about verification: when allocating a fixed inference budget across DAC-style reasoning, spending more on generating diverse subproblem decompositions (larger $n$) consistently outperforms spending more on verifying each decomposition through multiple conquering attempts (larger $m$). At a total budget of $k = 1024$ generations, the best performance comes from configurations with many decompositions and few conquering samples per decomposition — the exact opposite of what one might expect if the primary bottleneck were the reliability of the conquering stage.

This matters because it challenges the prevailing emphasis in the test-time compute literature on verification. Recent work on best-of-N sampling, process reward models (Lightman et al., 2023), and search-based reasoning (Tree-of-Thought) has focused primarily on how to select among candidate solutions — the verifier side of the proposer-verifier decomposition. The implicit assumption is that generating candidates is easy (just sample from the model) and the hard part is identifying the right one. DAC-RL’s results suggest that for structured reasoning paradigms, the opposite can be true: the bottleneck is in generating a good decomposition in the first place, not in selecting among conquering attempts.

Why does decomposition diversity matter so much? The paper’s explanation (Section 1, Section 4.2) is that different subproblem decompositions represent fundamentally different approaches to the problem — they partition the solution space differently. One decomposition might break an algebra problem into "isolate variable $a$," "isolate variable $b$," "solve the resulting system"; another might break the same problem into "rewrite equations in symmetric form," "introduce auxiliary variables," "solve the linear system in auxiliary variables," "back-substitute." These are not minor variations — they are different high-level strategies. If one strategy is fundamentally flawed (e.g., the first approach hits a dead end), generating 1000 conquering attempts with that strategy still fails. But a single conquering attempt with the second strategy might succeed. The diversity is in the decomposition space, not the execution space.

This has practical implications for how to spend inference compute on DAC-trained models: generate more decompositions, not more solutions per decomposition. It also has theoretical implications for understanding test-time compute scaling laws: the relationship between compute and accuracy likely depends on the dimensionality of the exploration space, not just the number of samples. CoT explores a low-dimensional space (all solutions follow the same high-level approach with local variations), while DAC explores a higher-dimensional space (different decompositions × different execution paths), making each additional sample more informative.

---

Innovation 5: The Alignment Tax of Format Enforcement in Structured Reasoning

Section 4.5 (Table 3) reports a negative result: enforcing strict subproblem-answering format during training — requiring the model to explicitly label and solve each subproblem in order — substantially degrades benchmark performance (51.3% → 45.2% average Pass@1), even though it dramatically improves format compliance (42.6% → 92.1%). This is positioned as an instance of the "alignment tax" (Lin et al., 2023; Wang et al., 2025), but in the specific context of DAC reasoning, it carries a deeper lesson.

The lesson is that the model’s learned DAC reasoning is not a rigid pipeline; it is a flexible strategy where the model uses subproblems as scaffolding rather than as a script. When forced to explicitly answer each subproblem, the model loses the ability to adapt its reasoning flow — to skip subproblems that turn out to be unnecessary, to merge subproblems whose solutions interact, or to recognize when a subproblem decomposition was imperfect and adjust accordingly. The 92.1% format following rate tells us the model can learn to follow the format; the 45.2% accuracy tells us that following the format makes the model worse at actually solving problems.

This finding is conceptually significant because it reveals a tension that is likely to recur as the field develops more structured reasoning paradigms: there is a tradeoff between the interpretability of structured reasoning (where each sub-step is explicit and verifiable) and the effectiveness of flexible reasoning (where sub-steps can be implicit, reordered, or adapted on the fly). The paper’s default DAC-RL configuration resolves this tension in favor of flexibility — the conquering prompt instructs the model to solve subproblems sequentially, but the reward only depends on final answer correctness, allowing the model to discover its own optimal balance between structure and adaptability.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary training dataset is DAPO-Math-17k (Yu et al., 2025a), consisting of 17,000 mathematical reasoning problems. The Deep DAC experiments use a difficulty-filtered subset of 3,700 problems from this dataset where the initial Qwen3-4B-Instruct-2507 policy achieves less than 50% accuracy across 16 responses. For cold-start distillation (Section 4.4), 3,000 problems are randomly selected from the DeepMath-103K dataset with difficulty labels greater than 6. Evaluation is performed on four competition-level mathematical reasoning benchmarks: AIME 2024, AIME 2025 (MAA), Beyond-AIME (ByteDance-Seed, 2025), and HMMT-25 (Balunović et al., 2025). The paper evaluates only on benchmarks with integer-valued answers to ensure precise assessment, following the recommendation of rStar2 (Shang et al., 2025) that rule-based verifiers struggle with open-ended mathematical formats.

• Base models. Two model families are used: Qwen2.5-7B-Instruct (a 7B parameter model) and Qwen3-4B-Instruct-2507 (a 4B parameter model). Both are instruction-tuned models with substantial post-training for reasoning, making them strong baselines that already exhibit non-trivial performance on competition math (Qwen3-4B-Instruct-2507 achieves 42.7% Pass@1 average across the four benchmarks with CoT inference). The choice of these models is motivated by the paper's central question: can DAC reasoning break through the performance ceiling that CoT hits on already-well-post-trained models? Using extensively trained models provides a harder test for DAC-RL than starting from base models would.

• Metrics. The primary metrics are Pass@1 and Pass@32 accuracy on the four competition benchmarks. Pass@1 is reported as the average over 32 independent runs to reduce variance from single evaluations. Pass@32 is computed using the unbiased estimation method from Chen et al. (2021): given $n$ total samples with $c$ correct solutions, $\text{Pass@k} = 1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}$, which provides an unbiased estimate of the probability that at least one of $k$ samples is correct without requiring $k$ independent evaluations. For the test-time scalability analysis (Figure 6), Pass@k is evaluated at $k = 1024$ with varying allocations between subproblem groups ($n$) and conquering samples per group ($m$) such that $n \times m = k$.

• Baselines. The paper compares against several baselines. Init-CoT: the initial model (before any RL training) using standard chain-of-thought inference. Init-DAC: the same initial model using DAC-style inference (with the division and conquering prompts from Figures 12 and 13) but without any DAC-specific training — this is the critical baseline that demonstrates the training-inference misalignment. RL-CoT: the model after GRPO-based RL training using standard CoT-style reasoning (same RL algorithm, same dataset, same reward signal on final-answer correctness, but with a standard solving prompt rather than the DAC pipeline). This is the primary competitive baseline that isolates the effect of DAC reasoning structure from the effect of RL training itself. RL-D-CoT (for the Deep experiments): CoT RL training under the same extended training and inference token budgets as the Deep DAC setting, with 32 rollouts per training problem — this ensures the Deep DAC comparison is fair regarding allocated compute. CD-CoT and CD-DAC: cold-start models distilled from Qwen3-235B-A22B-2507-Instruct on 3,000 problems, evaluated before any RL. CD-RL-CoT and CD-RL-DAC: cold-start models after subsequent RL training. Mix-RL (Section 4.1): a hybrid training approach where problems with CoT accuracy below 25% are trained with DAC-style rollouts while simpler problems use CoT-style rollouts.

• Generation budget / compute accounting. During RL training, the generation budget per problem per iteration is $G_d \times G_c = 4 \times 8 = 32$ complete solutions: 4 subproblem groups are generated, and for each group, 8 conquering solutions are sampled. This is the training-time compute. For the Deep DAC baseline, the CoT comparison is given an identical budget of 32 rollouts per training problem to ensure fair comparison ("this configuration with CoT-style RL training under an identical budget of 32 rollouts per training problem," Section 3.2). At inference time, test-time compute is measured in number of complete generations. For the test-time scalability analysis (Figure 6), a fixed budget of $k = 1024$ total generations is allocated across $n$ subproblem groups and $m$ conquering solutions per group. For the main Pass@1 and Pass@32 results, the generation budget is implicit in the sampling — Pass@1 uses 32 independent DAC runs (each involving one division and one conquering generation, though the exact token cost differs from CoT), and Pass@32 uses the unbiased estimator with a larger number of samples. Maximum token lengths are set to 8,192 for standard training, 16,384 for Deep DAC training, and 24,576 for Deep DAC inference.

• Cross-validation / statistical protocol. The paper does not explicitly describe a cross-validation protocol for strategy selection, unlike the compute-optimal test-time scaling framework in the reference example. The main results in Table 1 are direct evaluations on the four competition benchmarks, which are held-out test sets not used during training. For the Pass@1 metric, averaging over 32 runs provides variance reduction but the paper does not report confidence intervals or standard errors. For Pass@32, the unbiased estimator (Chen et al., 2021) corrects for the finite-sample bias inherent in computing Pass@k from $n$ samples, but the paper does not specify the total number of samples $n$ used for this estimation. The Mix-RL threshold ($t_{acc} = 25\%$) and the Deep DAC difficulty threshold (less than 50% accuracy across 16 responses) appear to be chosen based on empirical observation rather than through systematic hyperparameter sweeps.

Main Quantitative Results

DAC Reasoning vs. CoT Reasoning After RL Training (Table 1, Figure 4)

The central empirical finding is that DAC-RL training consistently and substantially outperforms CoT-RL training on competition-level mathematics benchmarks, with the gains being largest on the model where CoT has already saturated.

Qwen2.5-7B-Instruct results (Table 1, top section). Starting from an initial model that achieves 5.6% Pass@1 and 24.1% Pass@32 with CoT, and only 0.4% Pass@1 and 9.2% Pass@32 with naive DAC inference (confirming the severe misalignment), RL training dramatically changes the picture. After RL-CoT training, the model reaches 8.2% Pass@1 and 27.0% Pass@32 — modest gains of 2.6 and 2.9 percentage points respectively. After RL-DAC training, the model achieves 10.4% Pass@1 and 30.4% Pass@32 — gains of 2.2 and 3.4 percentage points over RL-CoT. The per-benchmark breakdown shows DAC outperforming CoT on every single benchmark for Pass@1 (AIME 2024: 15.5 vs. 13.5; AIME 2025: 15.5 vs. 11.4; Beyond-AIME: 7.0 vs. 5.1; HMMT-25: 4.8 vs. 2.7) and on all but one for Pass@32 (AIME 2025 is 34.2 for DAC vs. 30.8 for CoT, but the Init-CoT baseline already shows a higher number there, suggesting some variance). The absolute numbers are modest (this is a 7B model on hard competition math), but the pattern is clear: DAC-RL provides consistent improvements over CoT-RL.

Qwen3-4B-Instruct-2507 results (Table 1, middle section) — the headline result. This is where the paper makes its strongest claim. The initial model is already strong: 42.7% Pass@1 and 72.1% Pass@32 with CoT. After RL-CoT training, performance actually degrades to 37.5% Pass@1 and 69.0% Pass@32 — a regression of -5.2 and -3.1 percentage points. This is the CoT saturation phenomenon: further training on the same reasoning paradigm provides no benefit, and the stochasticity of RL training causes some forgetting or distribution shift. In contrast, RL-DAC training improves the model to 46.1% Pass@1 and 75.3% Pass@32 — gains of +8.6 and +6.3 percentage points over the RL-CoT baseline. The per-benchmark gains are substantial: AIME 2024 Pass@1 improves by 18.0 points (45.9 → 63.9); HMMT-25 Pass@1 improves by 10.1 points (21.8 → 31.9); Beyond-AIME Pass@32 improves by 9.8 points (58.1 → 67.9). These are large effect sizes on established competition benchmarks.

A critical detail: the improvement is measured relative to RL-CoT, not relative to Init-CoT. Compared to the initial model, RL-DAC improves by 3.4 points on Pass@1 (42.7 → 46.1) and 3.2 points on Pass@32 (72.1 → 75.3). This is a more modest improvement than the 8.6/6.3 numbers suggest, because RL-CoT actively degrades performance. The paper's framing emphasizes that DAC-RL prevents the degradation that CoT-RL suffers and additionally provides gains beyond the initial model — it's the differential effect of DAC training relative to further CoT training that constitutes the main result. This is a valid framing but worth noting: DAC-RL doesn't produce an 8.6 point improvement over the best available model; it produces a 3.4 point improvement, and the 8.6 number reflects the gap between DAC's success and CoT's failure under identical RL training conditions.

Deep DAC results (Table 1, bottom section). Training Qwen3-4B-Instruct-2507 exclusively on the 3.7k hardest problems (those with less than 50% accuracy from the initial policy), with extended training and inference token budgets, further amplifies the DAC advantage. RL-D-DAC achieves 51.3% Pass@1 and 81.6% Pass@32, compared to RL-D-CoT's 49.9% and 76.9% — gains of 1.4 and 4.7 percentage points. The Pass@32 improvement is particularly notable on HMMT-25, where DAC achieves 76.4% vs. 65.5% for CoT, a 10.9 point gap. Even more telling is the right panel of Figure 4, which shows that "simply increasing the rollout budget for CoT reasoning during RL training does not yield performance improvements, while DAC-style training delivers a 4.7% improvement in Pass@32 over the CoT baseline." This is a direct test of the claim that CoT has a structural ceiling: giving CoT more compute during training (32 rollouts per problem instead of fewer) does not help, while DAC with the same compute budget continues to improve.

Training dynamics (Figure 4). The intermediate checkpoint evaluations reveal the trajectory of improvement. For Qwen2.5-7B-Instruct (left panel), both CoT and DAC start at low accuracy (roughly 5-8% Pass@1) and improve over 400 steps, with DAC pulling ahead around step 200 and maintaining a growing lead. For Qwen3-4B-Instruct-2507 (middle panel), the pattern is dramatically different: CoT accuracy is essentially flat throughout training (hovering around 38-40%), while DAC accuracy starts below CoT (around 32%) but climbs steadily, crossing over around step 200 and continuing to rise. The right panel (Deep DAC) shows DAC continuing to improve over 200 additional steps of training on the difficult subset, while CoT with matched rollouts remains flat. The figure caption notes that "the training set is updated to the difficult subset after the 400th iteration" for Qwen3-4B-Instruct-2507, explaining the shift in the middle panel's rightmost portion.

Initial vs. Trained DAC Performance (Table 1, Init-DAC vs. RL-DAC)

The misalignment diagnosis depends on showing that naive DAC inference performs poorly but post-training fixes it. The numbers are dramatic:

• Qwen2.5-7B-Instruct: Init-DAC achieves 0.4% Pass@1 and 9.2% Pass@32 (averaged across benchmarks). After RL-DAC training, this jumps to 10.4% and 30.4% — a 26× improvement in Pass@1 and a 3.3× improvement in Pass@32. For comparison, Init-CoT achieves 5.6% and 24.1%, meaning the trained DAC surpasses the initial CoT by substantial margins (10.4% vs. 5.6%).
• Qwen3-4B-Instruct-2507: Init-DAC achieves 40.2% Pass@1 and 71.9% Pass@32 — already close to Init-CoT (42.7% and 72.1%). After RL-DAC training, this rises to 46.1% and 75.3%. The improvement is smaller in absolute terms (5.9 and 3.4 points) but represents the model learning to use DAC effectively rather than just overcoming a catastrophic failure mode.

The Qwen3-4B-Instruct-2507 results reveal something interesting: the stronger model already has some ability to perform DAC reasoning from its general post-training (40.2% is not terrible), but DAC-specific RL training still provides meaningful gains. This suggests the misalignment is partial for stronger models — they can partially fake DAC reasoning using CoT-learned skills — and that dedicated DAC training closes the remaining gap.

Mix-RL: DAC Training Improves CoT Reasoning (Section 4.1, Figure 5)

The Mix-RL experiments test a hybrid training approach where challenging problems (CoT accuracy below 25%) receive DAC-style training while simpler problems use CoT-style training. The results are reported in Figure 5 and discussed in Section 4.1.

Headline finding: "Incorporating DAC-style training on challenging problems significantly enhances the model's CoT performance — by over 10% across all benchmarks." This is evaluated under CoT-style inference — the model is tested using standard CoT, not DAC. The left panel of Figure 5 shows CoT-style Pass@1 accuracy across four benchmarks for CoT-RL and Mix-RL training over 400 steps. Mix-RL's CoT performance (roughly 42% average) substantially exceeds CoT-RL's (roughly 37-38%). The middle panel shows AIME-specific scores with a similar pattern.

This is perhaps the most surprising result in the paper: training the model to decompose problems using DAC makes it better at solving problems directly with CoT, even on problems that received only CoT training. The authors explain this as DAC training teaching the model something general about problem structure that transfers: "DAC-style training is even beneficial for the policy's CoT reasoning ability" (Section 1).

The right panel of Figure 5 shows DAC-style inference performance for both Mix-RL and full DAC-RL. Mix-RL achieves DAC performance (roughly 43-44% average) that is significantly above its CoT performance but below full DAC-RL (roughly 46%). This demonstrates that DAC reasoning capability can be partially acquired even when DAC training is only applied to a subset of problems, and that exposing the model to more DAC training (full DAC-RL) further improves that capability.

Test-Time Scalability (Section 4.2, Figure 6)

The test-time scalability analysis examines how DAC performance scales when the inference budget is allocated between subproblem group diversity and conquering depth.

Headline finding: "Allocating more groups (i.e., larger $n$ and smaller $m$) consistently improves performance on competition-level benchmarks compared to the CoT baseline." Figure 6 shows Pass@1024 for configurations ranging from $(n=1, m=1024)$ (far left: one decomposition, 1024 conquering samples — minimal decomposition diversity, maximum verification) to $(n=1024, m=1)$ (far right: 1024 decompositions, one conquering sample each — maximum decomposition diversity, minimal verification). The curve generally increases from left to right, with the best performance achieved at high-$n$ configurations. The CoT baseline (1024 independent CoT generations with majority voting) is shown as a horizontal reference line.

The exact numbers are not provided in the main text or visible in Figure 6 at sufficient resolution, but the qualitative pattern is unambiguous: decomposition diversity dominates conquering depth for test-time scaling. The paper interprets this as evidence that "increased subproblem diversity at test time expands the model's exploration space, improving its chances to discover correct trajectories" (Section 4.2).

This finding has a direct practical implication: when deploying DAC-trained models, budget should be spent on generating many different decompositions rather than on verifying each decomposition many times. It also validates the theoretical motivation that DAC's primary advantage is structural exploration rather than improved verification.

Cold-Start Distillation (Section 4.4, Table 2)

The cold-start experiments investigate whether initializing the policy with DAC demonstrations from a stronger model improves subsequent RL training.

After SFT only (CD-CoT vs. CD-DAC): The distilled DAC model achieves 46.4% Pass@1 and 76.2% Pass@32, compared to the distilled CoT model's 43.4% and 74.8%. This is notable because the initial model (before any distillation) performs better with CoT (42.7% vs. 40.2% for DAC), yet after equal-budget distillation, the DAC model pulls ahead. This suggests that the stronger teacher model provides DAC knowledge that the smaller model cannot discover through RL alone from its initial state.

After subsequent RL (CD-RL-CoT vs. CD-RL-DAC): The DAC model reaches 53.5% Pass@1 and 82.1% Pass@32, versus the CoT model's 52.2% and 79.7% — improvements of 1.3 and 2.4 percentage points. While the absolute gains from adding RL to the distilled models are substantial for both (CD-RL-DAC improves 7.1 Pass@1 points over CD-DAC; CD-RL-CoT improves 8.8 points), the DAC model maintains an edge. The per-benchmark results show DAC outperforming CoT on 5 of 8 metric-benchmark combinations (AIME 2024 Pass@1: 71.3 vs. 66.5; AIME 2025 Pass@32: 91.7 vs. 88.2; Beyond-AIME Pass@32: 69.0 vs. 66.1; HMMT-25 Pass@32: 75.9 vs. 71.8) and tying or losing narrowly on the remaining 3.

The paper interprets these results as showing that "DAC constitutes a more effective advanced reasoning paradigm" and that "DAC enables richer and more diverse exploration during policy optimization, leading to more efficient performance improvement than standard CoT reasoning." However, the cold-start gains are modest — the 1.3 point Pass@1 advantage could be within the range of variance (no confidence intervals are reported).

Concise Reasoning and Training Efficiency (Section 4.3, Figure 7)

The paper reports a counterintuitive finding about reasoning length: "DAC training produces more concise reasoning than CoT-style RL" (Section 4.3). Figure 7 shows three metrics measured during training:

• Left panel (response length): The batch-averaged response length is lower for DAC than for CoT, despite the intuitive expectation that solving subproblems would add length. The paper attributes this to DAC reducing "narrative redundancy" — CoT responses include extensive self-verification and re-derivation of intermediate results, while DAC responses map directly to predefined subproblem decompositions, avoiding repetitive restatements. Appendix D provides a case study (Figures 9, 10, 11 and Listings 1 and 2) comparing a 3,328-token DAC solution to a 5,072-token CoT solution for the same algebraic problem.

• Middle panel (clip ratio): The fraction of rollouts clipped by the maximum length constraint is lower for DAC than for CoT. This is a direct consequence of shorter responses and has an important training implication: truncated positive rollouts become false negatives (correct solutions that are cut off before the answer and thus scored as incorrect), which degrades the training signal. DAC's lower clip ratio means fewer false negatives and higher-quality gradient updates.

• Right panel (policy entropy): DAC maintains higher policy entropy than CoT throughout training. Higher entropy indicates greater diversity in the policy's output distribution — the model is exploring a wider range of possible responses rather than converging to a narrow set of high-probability outputs. The paper interprets this as DAC enabling "broader exploration" that "does not compromise exploration" and "removes redundant steps while preserving — and even enhancing — diversity in solution space exploration, thereby enabling more efficient training without premature convergence."

The combination of shorter responses, lower clip ratios, and higher entropy suggests that DAC training is not just more accurate but also more efficient: each training step processes more information (less wasted computation on redundant text), suffers from fewer artifacts (fewer truncated correct solutions), and maintains healthier exploration dynamics (less entropy collapse).

Format Constraint Ablation (Section 4.5, Table 3)

This ablation tests the effect of imposing strict subproblem-solving format during conquering — requiring responses to explicitly label and answer each subproblem in order.

Headline finding: The format-following constraint dramatically improves format compliance (42.6% → 92.1%) but substantially degrades benchmark performance (51.3% → 45.2% Pass@1 average). Per-benchmark, the constrained model underperforms on every single metric: AIME 2024 Pass@1 drops from 66.3 to 56.2; AIME 2025 from 61.5 to 55.5; Beyond-AIME from 38.8 to 34.3; HMMT-25 from 38.7 to 34.6. The Pass@32 metrics show similar degradation.

The paper attributes this to the alignment tax: "training LLMs for alignment, especially under strict formatting constraints, can introduce an alignment tax, which enhances instruction-following behavior while degrading downstream performance." In the context of DAC reasoning, this means the model learns to perform the format of decomposition without actually using that decomposition effectively — it becomes more compliant but less capable.

This is a practically important negative result that informs the default DAC-RL configuration, which does not enforce this constraint. It also raises an important question about the relationship between the division stage (where format constraints are enforced via Equation 2) and the conquering stage (where they are not): does the asymmetric enforcement of format constraints across the two stages introduce a subtle misalignment, or is it necessary to prevent the conquering stage from degenerating into format-following rather than problem-solving?

Ablation Studies and Robustness Checks

Conquering accuracy as division reward (Appendix C): Early experiments used average conquering accuracy directly as the division reward, rather than the binary scheme in Equation 2. This caused the model to solve the problem prematurely during division, embedding solution steps in the "subproblems" rather than actually decomposing. Figure 8 provides an example where the division output contains intermediate calculations labeled as subproblems. This failure mode motivated the relaxed reward scheme that only enforces a lower bound on subproblem helpfulness. The finding is important because it demonstrates that reward design for meta-cognitive skills (like decomposition) requires different principles than reward design for execution skills — directly optimizing for downstream task accuracy can cause the meta-cognitive step to collapse into the execution step.

Number of subproblems constraint (Section 2.2): The requirement that $N_s \geq 3$ was motivated by an observed failure mode where "without this constraint, it collapses to producing no useful subproblems in division response $y_d$ as the training goes, and the conquering stage degrades to directly solving the original problem, as in standard CoT reasoning." This is not presented as a formal ablation with varying $N_s$ values in a table, but as a design choice justified by observed training dynamics. The paper does not report what happened with $N_s = 1$, $N_s = 2$, or $N_s = 5$, leaving open the question of whether 3 is optimal or simply sufficient.

Division group size $G_d$ and conquering group size $G_c$ (Section 3.1): These are set to 4 and 8 respectively, but no ablation study varies these numbers to determine whether different configurations would yield different results. The choice of $G_d = 4$ determines the diversity of decompositions explored per problem during training, while $G_c = 8$ determines the number of conquering attempts per decomposition. Given the test-time scalability finding that decomposition diversity dominates conquering depth (Figure 6), it would be valuable to know whether larger $G_d$ with smaller $G_c$ during training would improve learning — but this is not tested.

Cold-start vs. no cold-start (Table 2 vs. Table 1): The cold-start experiments effectively serve as an ablation of initialization quality. Comparing the standard RL-DAC on Qwen3-4B-Instruct-2507 (46.1% Pass@1) with the cold-start CD-RL-DAC (53.5%) shows that better initialization provides substantial gains (7.4 points). However, comparing CD-DAC (46.4%) with CD-RL-DAC (53.5%) shows that RL provides an additional 7.1 points even from a strong initialization — RL and cold-start are complementary rather than redundant. Comparing CD-DAC (46.4%) with standard RL-DAC from non-distilled initialization (46.1%) shows that distillation alone achieves comparable performance to full RL training from scratch, which raises the question of whether RL is actually necessary if a strong enough teacher is available — though the paper's core claim is about RL-based training, not distillation.

Training data difficulty filtering (Deep DAC): The Deep DAC experiments filter training data to the 3.7k hardest problems (those where the initial policy achieves less than 50% accuracy across 16 responses). This is compared against standard training on all 17k problems. The Deep DAC setting produces stronger results (51.3% vs. 46.1% Pass@1), but this is confounded with increased training steps (additional ~200 steps), extended token budgets (16,384 for training, 24,576 for inference vs. 8,192 standard), and the difficulty filtering itself. The paper does not ablate these factors separately, so the relative contribution of each is unknown.

Model scale (7B vs. 4B): Both model sizes benefit from DAC-RL training, but the gains differ in character. For the weaker 7B model, DAC-RL provides absolute gains over both the initial model and CoT-RL (10.4% vs. 8.2% Pass@1). For the stronger 4B model, DAC-RL prevents the degradation seen in CoT-RL and adds modest absolute gains over the initial model (46.1% vs. 42.7%). This suggests that DAC-RL is beneficial across model scales, but the nature of the benefit shifts from "teaching a fundamentally new capability" (for weaker models where DAC is initially unusable) to "refining and extending an existing capability" (for stronger models that already have some DAC ability). However, with only two model sizes tested, no scaling trend can be reliably established.

RL algorithm choice (GRPO with DAPO extensions): The paper uses GRPO with Clip-Higher and token-level loss, but provides no ablation comparing against vanilla GRPO, PPO, or other RL algorithms. The hyperparameters (Clip-Higher $\varepsilon_h = 0.28$, batch size 256, etc.) are stated without justification or sensitivity analysis. This is a standard limitation of RL papers — the computational cost of full RL training runs makes thorough algorithm ablations expensive — but it means the results are conditional on these specific algorithmic choices.

Critical Assessment

Claim: DAC-RL training raises the performance ceiling beyond what CoT-RL can achieve

Does the evidence support this? Yes, with important qualifications about what "ceiling" means and how it was tested. The strongest evidence is Figure 4 (middle panel) and Table 1 (Qwen3-4B-Instruct-2507): CoT-RL training degrades performance (37.5% vs. initial 42.7%), while DAC-RL training improves it (46.1%). The CoT curve is flat, the DAC curve rises. This is direct evidence of a structural difference in trainability — CoT has saturated, DAC has not.

However, "ceiling" could mean two different things: (1) CoT RL with this specific algorithm (GRPO), dataset (DAPO-Math-17k), and training duration (400 steps) cannot improve this model, or (2) CoT reasoning as a paradigm cannot improve this model's performance on these benchmarks regardless of training methodology. The paper's evidence supports interpretation (1) but not necessarily (2). It's possible that CoT RL with a different algorithm, different reward shaping, different dataset, or longer training could break through the apparent ceiling. The paper does not explore CoT RL with the same extended budget as Deep DAC (32 rollouts, longer token limits, difficulty-filtered data) and report those results separately — the Deep DAC baseline (RL-D-CoT) uses 32 rollouts but the standard RL-CoT does not, making the comparison partially confounded.

Additionally, the "ceiling" is demonstrated relative to a specific set of benchmarks (AIME 2024/2025, Beyond-AIME, HMMT-25). These are all competition-level math problems with integer answers. Whether CoT has saturated on other types of reasoning (code generation, theorem proving, multi-step planning) is untested.

Claim: DAC-RL achieves 8.6% absolute improvement in Pass@1 and 6.3% in Pass@32 over CoT-RL

Does the evidence support this? Yes, but these numbers are the gap between DAC-RL and RL-CoT, not between DAC-RL and the best available CoT model. The numbers come directly from Table 1 (Qwen3-4B-Instruct-2507, averaging across four benchmarks): RL-DAC (46.1%, 75.3%) minus RL-CoT (37.5%, 69.0%) = 8.6, 6.3. The calculation is straightforwardly correct.

The important context is that RL-CoT is worse than Init-CoT (37.5% vs. 42.7% Pass@1), meaning CoT-RL actively damaged the model. The improvement of DAC-RL over the initial model is 3.4 points (46.1% - 42.7%), which is a more conservative but still meaningful gain. The 8.6% number reflects not just DAC's strength but CoT-RL's weakness under these specific training conditions. Framing the result as "DAC-RL improves over the initial model by 3.4 points, while identical CoT-RL training degrades it by 5.2 points" would be more precise, though the paper's abstract emphasizes the 8.6% figure, which is the headline comparison between the two RL-trained variants.

The average masks substantial per-benchmark variation. The 8.6% average is composed of: AIME 2024 (+18.0), AIME 2025 (+2.1), Beyond-AIME (+4.2), HMMT-25 (+10.1). The AIME 2024 gain is enormous while AIME 2025 is much smaller — without per-benchmark reporting, the average would be misleading. The paper does report the per-benchmark numbers in Table 1, so this is transparent, but the abstract's aggregate number should be understood as summarizing a heterogeneous pattern.

Claim: DAC-trained models exhibit stronger test-time scalability

Does the evidence support this? Yes, but the demonstration is limited to one budget point (k=1024) and one model. Figure 6 shows the scaling pattern for a fixed budget across different allocations, but this is a single cross-section of the test-time compute curve. The paper does not show Pass@k curves for varying k (e.g., k = 1, 2, 4, 8, ..., 1024, 2048) for DAC vs. CoT, which would directly demonstrate that DAC's accuracy rises faster with additional compute. Instead, Figure 6 shows that at k=1024, how you allocate the budget within DAC matters, and that DAC outperforms CoT at this budget point. This demonstrates that DAC uses a fixed compute budget more efficiently than CoT, but does not directly demonstrate that DAC's performance scales better (i.e., has a steeper slope in a Pass@k vs. k plot) — the paper infers this from the exploration space argument, but the empirical demonstration is incomplete.

A full test-time scaling analysis would require plots of accuracy vs. total compute budget for both DAC and CoT, showing that DAC not only achieves higher accuracy at matched budgets but also benefits more from each additional unit of compute. The paper's Figure 4 (training dynamics) shows DAC improving more over training, which is a related but distinct concept — training compute scaling vs. inference compute scaling. These are conflated in the paper's discussion but are not the same phenomenon.

Claim: DAC training improves CoT reasoning by over 10%

Does the evidence support this? Yes, based on Figure 5 (left and middle panels), but with caveats about what is being compared. The 10% figure refers to the relative improvement of Mix-RL over CoT-RL when both are evaluated under CoT inference. The left panel of Figure 5 shows Mix-RL achieving roughly 42% average Pass@1 across benchmarks vs. CoT-RL's roughly 37-38%, which is approximately a 10% relative improvement or a ~4 percentage point absolute improvement. The text says "over 10% across all benchmarks" — this is a relative percentage, not absolute percentage points.

This is a surprising and important transfer effect: training on DAC reasoning improves CoT reasoning even on problems that never received DAC training (since Mix-RL only applies DAC to problems with <25% CoT accuracy). The mechanism is unclear — the paper speculates about learning general problem structure, but there's no mechanistic analysis. An ablation where DAC training is applied to random subsets (rather than difficulty-filtered subsets) would help distinguish whether the transfer is due to training on different problems with DAC vs. training on hard problems with DAC.

Missing Experiments and Weaknesses

No confidence intervals or statistical significance testing. All results in Table 1, Table 2, and Table 3 are reported as point estimates without any measure of variance besides the 32-run averaging for Pass@1. Given that the test sets are relatively small (AIME has 15-30 problems per year, Beyond-AIME and HMMT-25 are similarly sized), differences of 1-3 percentage points could easily fall within the range of sampling variability. The paper would be strengthened by reporting bootstrap confidence intervals or at minimum the per-benchmark problem counts so readers can assess statistical reliability.

Single training dataset. All RL experiments use DAPO-Math-17k as the training set. It's unclear whether the results are specific to this dataset's composition or would generalize to other math reasoning datasets. The paper does not test on, for example, the MATH training set or other math corpora. The training data distribution inevitably shapes what kinds of decompositions the model learns — DAPO-Math-17k may contain problems that are particularly amenable to DAC-style reasoning.

Limited model diversity. Only two model families are tested (Qwen2.5-7B-Instruct and Qwen3-4B-Instruct-2507), both from the same model developer and sharing architectural similarities. The paper does not test on Llama, Mistral, DeepSeek, or other model families. This matters because the ability to learn DAC reasoning through RL may depend on the base model's instruction-following capabilities, which vary substantially across model families.

No analysis of subproblem quality. The paper never evaluates the quality of the generated subproblems directly — are they logically sound? Do they cover all aspects of the original problem? Are there redundant or missing subproblems? The only Quality metric is downstream conquering accuracy, which confounds decomposition quality with execution quality. A human evaluation of subproblem quality (even on a small sample) would substantially strengthen the claim that the model is learning genuine decomposition rather than just finding decompositions that happen to work for this specific RL setup.

The difficulty estimation for Mix-RL and Deep DAC is oracle-based. Mix-RL assigns DAC training to problems based on actual CoT accuracy (<25%), and Deep DAC filters based on the initial policy's accuracy (<50% across 16 responses). Both require knowing ground-truth answers to compute accuracy. In a deployment setting where the ground-truth answer is unknown, the system cannot make these determinations. The paper does not propose or evaluate a practical difficulty estimation method for routing problems between CoT and DAC.

The alignment between training and inference DAC procedures is imperfect. During training, the model generates 4 subproblem groups with 8 conquering samples each. During inference for Pass@1, it generates one division and one conquering solution. The mismatch in the number of groups could mean the model learns strategies that depend on having multiple decompositions to "average over," and these strategies don't transfer to the single-decomposition inference setting. Conversely, the test-time scalability analysis (Figure 6) shows benefits from using many decompositions at inference, suggesting that the standard Pass@1 inference (single decomposition) is suboptimal — but the paper reports Pass@1 as the primary metric anyway.

No comparison to non-RL DAC methods. The paper compares DAC-RL to CoT-RL, but does not compare to DAC methods that don't use RL — for example, DAC inference with a model fine-tuned via SFT on DAC demonstrations, or DAC inference with a stronger model used only for decomposition. This makes it difficult to assess whether RL specifically is necessary, or whether any form of DAC training would work. The cold-start experiments partly address this (showing that distillation helps), but a direct comparison between CD-DAC (SFT only, no RL) and standard RL-DAC (RL only, no SFT) would isolate the contribution of RL — currently these use different training data (DeepMath-103K vs. DAPO-Math-17k) and are not directly comparable.

Test sets are small and potentially contaminated. The four evaluation benchmarks (AIME 2024, AIME 2025, Beyond-AIME, HMMT-25) are publicly available competition problems. They may have appeared in the pretraining data of Qwen3-4B-Instruct-2507, particularly since the model was released in 2025 and these benchmarks were available before then. The paper does not discuss contamination analysis or decontamination procedures. If the model has seen these problems during pretraining, the measured performance may reflect memorization rather than genuine reasoning capability, and the differences between CoT and DAC may reflect which reasoning style better recovers memorized solutions.

The purported conciseness advantage may be dataset-specific. The case study in Appendix D compares one problem's DAC solution (3,328 tokens) to its CoT solution (5,072 tokens). This is a single example, and the paper does not report aggregate statistics on response length distributions across all problems. It's possible that DAC is shorter on average, or that the example was chosen to illustrate the effect, or that DAC responses are shorter for some problem types but not others. Without aggregate statistics (mean, median, percentiles of response length for DAC vs. CoT across all evaluation problems), the conciseness claim is anecdotal.

Alignment tax experiment (Table 3) conflates format enforcement with reward penalty. When format following is enforced, incorrect-format responses receive negative rewards regardless of answer correctness. This is a different reward structure than the default, where format-incorrect conquering responses are simply scored 0 if the answer is wrong (since the answer is unextractable). The degradation could be due to the reward penalty rather than the format constraint itself — the model might be learning to play it safe with shorter, format-compliant but less ambitious solutions, rather than suffering from format constraints per se.

6. Limitations and Trade-offs

6.1 DAC-RL Provides No Benefit on Problems Fundamentally Outside the Model's Capability Range

The constraint. The entire DAC-RL framework rests on a hidden premise: that the model is capable of producing at least one correct solution trajectory (whether CoT or DAC) for a problem, so that the RL training loop has positive reward signals to reinforce. On problems where the base policy's pass rate is zero — where no amount of sampling, decomposition, or search can produce a correct answer — DAC-RL cannot improve performance any more than CoT-RL can. The paper demonstrates this implicitly rather than stating it as an explicit limitation, but the evidence is clear: training on the difficulty-filtered subset of DAPO-Math-17k (problems where the initial policy achieves less than 50% accuracy) still leaves a substantial gap between the Deep DAC model's 51.3% Pass@1 and anything approaching saturation on these competition benchmarks.

The consequence. DAC-RL does not solve the cold start problem for genuinely novel reasoning: if the model has no conception of how to approach a problem class, decomposition doesn't create that conception — it only reorganizes existing capabilities. A practitioner facing problems where their base model achieves near-zero accuracy (say, difficulty bin 5 in the compute-optimal scaling framework from the reference paper) should not expect DAC-RL to help. The gains demonstrated in this paper are substantial but bounded: DAC raises the ceiling, it does not remove it.

Evidence in the paper. The paper never evaluates DAC-RL on a problem set where the base model has zero correct solutions. The Deep DAC setting uses problems with <50% accuracy (Section 3.1), meaning the model already has a non-trivial chance of getting them right — this is a filter for "hard but solvable," not "impossible." The per-benchmark results in Table 1 show that even Deep DAC achieves only 38.7% Pass@1 on HMMT-25 (which is comparable to the hardest bins in difficulty-quintile analyses), and the paper provides no analysis of whether these failures are concentrated on problems where the base model never succeeds. The missing experiment is a Pass@k analysis on the evaluation benchmarks with very large k (e.g., k = 1024) for the initial model, showing the fraction of problems that are fundamentally unsolvable — those problems would be DAC-RL's ceiling.

Mitigation status. The paper does not address this limitation. It acknowledges that DAC is motivated by the need to tackle "the most challenging problems at the edge of its capability" (Section 3.2), which correctly frames DAC as pushing the frontier rather than expanding the territory. But it does not characterize where that frontier lies or how to detect when a problem is beyond it. The cold-start distillation experiments (Section 4.4) partially address the capability-bound problem by injecting knowledge from a stronger model — if the small model has no idea how to approach a problem, the teacher might — but this simply shifts the capability bound from the small model to the large model; it does not eliminate it.

---

6.2 The Difficulty-Based Routing Mechanisms Require Oracle Knowledge of Problem Difficulty

The constraint. Two of the paper's most practically significant findings — the Mix-RL training strategy (Section 4.1) and the Deep DAC training strategy (Section 3.2) — depend on knowing in advance which problems are "hard" for the current model. Mix-RL routes problems with CoT accuracy below t_acc = 25% to DAC-style training; Deep DAC filters the training set to problems where the initial policy achieves less than 50% accuracy across 16 responses. Both thresholds are computed using ground-truth answers, which are unavailable at deployment time for genuinely novel problems. This is the same oracle-difficulty problem that the compute-optimal test-time scaling paper (the reference example) grapples with, and DAC-RL's solution is even less developed.

The consequence. In a real deployment where the system encounters new problems with unknown answers, the Mix-RL strategy cannot be applied: you cannot compute CoT accuracy without knowing the correct answer, and you cannot know the correct answer without solving the problem (which is the whole point). Similarly, the Deep DAC data filtering cannot be replicated on a new dataset without first evaluating the model's accuracy on that dataset, which requires ground-truth labels. The paper's headline results — particularly the 10% CoT improvement from Mix-RL and the Deep DAC gains — are therefore conditional on oracle knowledge that would not exist in a practical self-improvement pipeline operating on unlabeled problems.

Evidence in the paper. Section 4.1 explicitly states the Mix-RL criterion: "all batch problems are first answered in CoT-style rollouts, and those with accuracy below t_acc = 25% are replaced by DAC-style solutions." Section 3.1 describes the Deep DAC filtering: "a difficulty-filtered subset containing 3.7k problems that the initial policy solves with less than 50% accuracy across 16 responses." In both cases, the filtering depends on ground-truth answers. The paper does not propose or evaluate a predicted difficulty mechanism (e.g., using the policy's own confidence, using a learned difficulty estimator, or using the variance of CoT answers as a proxy for uncertainty). This is a gap that the authors do not acknowledge as a limitation, but it is the single largest barrier to deploying Mix-RL or Deep DAC in a self-improvement loop without human-labeled data.

Mitigation status. Not addressed. The paper does not discuss the oracle-difficulty problem, does not propose practical difficulty estimators, and does not include the cost of difficulty estimation in any compute budget. This is in contrast to the compute-optimal test-time scaling paper, which explicitly acknowledges the difficulty estimation cost, proposes a PRM-based estimator as a practical approximation, and flags it as future work (Section 3.2). DAC-RL's analogous limitation is unexamined.

---

6.3 Single Training Dataset and Narrow Evaluation Domain Leave Cross-Task Generalization Untested

The constraint. All RL training is performed on a single dataset (DAPO-Math-17k, a collection of mathematical reasoning problems), and all evaluation is performed on four competition-level mathematics benchmarks (AIME 2024/2025, Beyond-AIME, HMMT-25). The paper's claims about DAC reasoning as a general paradigm — as opposed to a strategy that works specifically for competition math — are untested. The paper states explicitly that evaluation is restricted to "benchmarks with integer answers to ensure precise assessment" (Section 3.1), which rules out a large fraction of mathematical reasoning tasks with non-integer or symbolic answers, not to mention non-mathematical reasoning domains entirely.

The consequence. A practitioner considering DAC-RL for code generation, scientific reasoning, multi-step planning, formal theorem proving, or any domain outside integer-answer competition math has no evidence from this paper about whether DAC training would help, hurt, or have no effect. The domain restriction also interacts with the DAC mechanism itself: competition math problems have a particular structure (well-defined subproblems, unique correct answers, verifiable solutions) that makes them especially amenable to divide-and-conquer. It is unclear whether DAC training would transfer to domains where "subproblems" are ambiguous, overlapping, or undefined — e.g., writing an essay, designing a software architecture, or formulating a scientific hypothesis.

Evidence in the paper. The paper acknowledges the evaluation restriction in Section 3.1 ("following rStar2 (Shang et al., 2025), which suggests that rule-based verifiers often struggle with certain open-ended mathematical formats, we evaluate model performance only on benchmarks with integer answers to ensure precise assessment") but frames it as a methodological choice for reliable evaluation rather than as a limitation on the generality of the findings. There is no discussion of whether DAC reasoning might behave differently on other problem types, no experiment on even a single non-math benchmark, and no analysis of whether the decomposition strategies learned on DAPO-Math-17k would transfer to structurally different problems.

Mitigation status. Not addressed. The paper does not suggest that the findings are math-specific, nor does it discuss future work on other domains. The introduction frames DAC reasoning in general terms ("an intuitive and classical approach to solving complex problems") and cites applications to theorem proving (Seed-Prover, DeepSeek-Prover-V2), implying generality that the experiments do not support.

---

6.4 The Cold-Start Teacher Model Is ~59× Larger, Making Cost Comparisons Ambiguous

The constraint. The cold-start distillation experiments (Section 4.4, Table 2) use Qwen3-235B-A22B-2507-Instruct as the teacher model to generate 6,000 CoT and DAC demonstrations for fine-tuning Qwen3-4B-Instruct-2507. The teacher is approximately 59× larger than the student (235B vs. 4B parameters). The paper reports that CD-DAC (distilled DAC, before any RL) achieves 46.4% Pass@1, which is comparable to standard RL-DAC (46.1%) trained from scratch without distillation. This raises a critical question: is the gain from distillation actually a gain from DAC reasoning, or is it primarily a gain from access to a much larger model's knowledge, with DAC being incidental?

The consequence. If a practitioner has access to a 235B-parameter teacher model, they could simply use that teacher model for inference rather than distilling it into a 4B model and then applying RL. The paper's comparison of CD-RL-DAC (53.5%) to RL-DAC (46.1%) conflates the effect of distillation (knowledge transfer from a much larger model) with the effect of DAC reasoning structure, making it impossible to isolate how much of the improvement comes from DAC specifically versus from model scale. The fair comparison would be: can the 4B model, trained with DAC-RL alone (no distillation), match the performance of the 235B model using CoT? Alternatively: if the 235B model's knowledge were distilled into the 4B model using CoT demonstrations (CD-CoT), how does that compare to distillation using DAC demonstrations (CD-DAC)?

Evidence in the paper. Table 2 provides the relevant numbers: Init-CoT (42.7%), CD-CoT (43.4%), CD-DAC (46.4%). The gains from distillation alone are +0.7 points for CoT and +6.2 points for DAC, suggesting DAC distillation transfers more effectively than CoT distillation. This is evidence that the DAC reasoning structure matters beyond just model scale. However, the subsequent RL gains (CD-RL-CoT: 52.2%, CD-RL-DAC: 53.5%) show both distilled models improving substantially, with DAC maintaining only a 1.3-point edge. The marginal benefit of DAC over CoT after both have been cold-started and RL-trained is modest relative to the massive scale difference between teacher and student.

Mitigation status. The paper presents the cold-start results as evidence that "DAC constitutes a more effective advanced reasoning paradigm" (Section 4.4), but does not directly address the teacher-student scale confound. The comparison of CD-DAC to CD-CoT within the same distillation budget (3k samples each, same teacher) partially controls for scale, but the missing experiment is a comparison to the teacher model's own performance on these benchmarks, which would establish an upper bound and clarify how much of the teacher's capability is actually transferred.

---

6.5 The Training Budget Is 32× Larger for DAC Than CoT per Problem, Yet This Is Not Counted in Efficiency Claims

The constraint. Per training iteration, DAC-RL generates G_d × G_c = 4 × 8 = 32 complete solution candidates per problem (4 subproblem groups × 8 conquering solutions each). CoT-RL generates some number of CoT rollouts per problem — the paper does not explicitly state the CoT rollout count for the standard RL-CoT baseline, but the standard GRPO setup typically uses a group size comparable to the number of responses needed for stable advantage computation, which would be much smaller than 32. For the Deep DAC comparison, the paper explicitly states that "this configuration with CoT-style RL training under an identical budget of 32 rollouts per training problem" is used (Section 3.2). But for the standard RL-CoT baseline in Table 1 (the one that achieves 37.5% Pass@1 for Qwen3-4B-Instruct-2507), the rollout budget is not stated, making it unclear whether DAC-RL's advantage comes from better reasoning structure or simply from more training-time exploration.

The consequence. If standard RL-CoT uses, say, 4-8 rollouts per problem (typical for GRPO with reasonable batch sizes), then DAC-RL is getting 4-8× more training compute per problem. The paper's framing as a comparison of reasoning paradigms rather than compute budgets would be misleading if the DAC model simply sees more training data (in the form of more on-policy rollouts). The Deep DAC comparison explicitly controls for this (32 rollouts for both), and DAC still wins — but the standard RL-CoT baseline's compute budget is unspecified, and the headline 8.6% Pass@1 improvement is computed against that unspecified baseline.

Evidence in the paper. Section 3.1 states: "In each iteration, every input problem is divided into G_d = 4 subproblem groups containing varying numbers of subproblems, and each group, together with the original problem, is used to generate G_c = 8 conquering solutions." This is a clear statement of DAC's per-problem generation budget (32). Section 3.2, describing the Deep DAC comparison, states: "We further compare this configuration with CoT-style RL training under an identical budget of 32 rollouts per training problem, with results shown in the right panel of Figure 4 and the bottom section of Table 1." But for the standard RL-CoT baseline in the middle section of Table 1 (the one with 37.5% Pass@1), the rollout budget is never stated. The GRPO description in Appendix A mentions a group of G responses but does not specify G for the CoT baseline.

Mitigation status. Partially addressed. The Deep DAC experiments explicitly control for training budget, and DAC still outperforms CoT (51.3% vs. 49.9% Pass@1). However, the standard RL-CoT baseline's budget remains unspecified, and the paper's abstract and introduction emphasize the 8.6%/6.3% numbers (which compare DAC to the standard RL-CoT, not the budget-matched Deep CoT baseline) without noting the potential compute confound. A practitioner reading the headline numbers should know that the comparison may not be compute-matched.

---

6.6 Evaluation Is Conducted on Small Test Sets Without Reported Confidence Intervals

The constraint. The four evaluation benchmarks — AIME 2024, AIME 2025, Beyond-AIME, and HMMT-25 — each contain a relatively small number of problems. AIME has 15 problems per year (30 total across the two years), Beyond-AIME and HMMT-25 are similarly sized competition sets. The total evaluation set across all four benchmarks is likely on the order of 60-100 problems. With such small test sets, the standard error on a Pass@1 estimate can be substantial: for 60 problems with 50% accuracy, the margin of error is approximately ±13 percentage points for a 95% confidence interval. The paper reports Pass@1 as the average of 32 runs (reducing variance from sampling stochasticity) but does not report confidence intervals, standard errors, or per-benchmark problem counts for any of the metrics.

The consequence. Many of the paper's detailed comparisons — particularly the per-benchmark breakdowns in Table 1 and Table 2 — involve differences of 1-3 percentage points. For example, RL-DAC vs. RL-CoT on Qwen2.5-7B-Instruct on Beyond-AIME Pass@32 is 27.4 vs. 25.5 (a 1.9 point difference). Without knowing the number of test problems and the associated sampling variance, the reader cannot assess whether such differences are statistically reliable or could easily arise from which specific problems happened to appear in the test set. The problem is compounded by the paper's use of the unbiased Pass@k estimator (Chen et al., 2021), which has its own variance that depends on the total number of samples n used for estimation — a parameter that is not reported.

Evidence in the paper. The paper reports point estimates throughout (Tables 1, 2, 3; Figures 4, 5, 6, 7) without any measure of uncertainty. Pass@1 is "averaged over 32 runs to ensure a more stable and precise evaluation" (Section 3.1), but this averaging addresses variance from the model's stochastic sampling, not variance from the finite test set. AIME is described as having "15-30 problems per year" elsewhere in the literature; Beyond-AIME and HMMT-25 are cited as competition benchmarks with similarly small problem sets. The paper provides URLs for AIME and Beyond-AIME in the references but does not state the exact problem counts used for evaluation.

Mitigation status. Not addressed. The paper makes no mention of confidence intervals, statistical testing, or the limitations of small test sets. This is a common practice in the LLM reasoning literature (particularly on competition benchmarks where the test sets are inherently small), but it means that fine-grained comparisons between methods (particularly the per-benchmark breakdowns) should be interpreted as suggestive rather than definitive. A practitioner deciding between DAC-RL and an alternative method based on a 2-point difference on a 30-problem test set is making a decision under substantial uncertainty that the paper does not quantify.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a diagnostic reframing rather than a paradigm shift: it identifies the performance ceiling of CoT reasoning not as a limitation of model scale or training data, but as a structural property of the reasoning paradigm itself. The finding that CoT RL training on Qwen3-4B-Instruct-2507 actively degrades performance (37.5% vs. initial 42.7% Pass@1, Table 1) while DAC RL training on the same model with the same dataset and reward signal produces meaningful gains (46.1%) constitutes direct evidence that the bottleneck is not the model's capacity to reason, but the reasoning structure it was taught to use. This is a conceptually important shift: it moves the field's understanding of reasoning limitations from "bigger models needed" toward "better reasoning paradigms needed."

The paper resolves a tension that has been implicit in the structured reasoning literature. Prior work showed DAC-style reasoning succeeding (Seed-Prover, DeepSeek-Prover-V2) and failing (naive prompting with off-the-shelf models) without a clear explanation for the discrepancy. The paper's diagnosis — that DAC is a learned skill requiring dedicated training, not a prompting strategy that any reasoning-trained model can execute — provides a unified account: prior successes involved systems with substantial external scaffolding or specialized training that compensated for the misalignment, while prior failures involved models with no DAC-specific training at all. This reframes DAC from an inference-time add-on to a first-class training objective, which changes how researchers should think about building structured reasoning systems.

The paper also establishes CoT saturation as a measurable, actionable phenomenon. Before this work, the fact that frontier models might plateau on reasoning benchmarks was a suspicion supported by anecdotes. The paper provides a clean empirical demonstration: the CoT RL curve in Figure 4 (middle panel) is flat across 400 training steps, while DAC RL on the same model continues to rise. This creates a concrete diagnostic that other researchers can apply — if further CoT training on your model produces no gains, consider that CoT itself may be the bottleneck. The Mix-RL finding (DAC training on hard problems improving CoT performance by over 10%, Section 4.1, Figure 5) further demonstrates that the gains from better reasoning structure can transfer back to the original paradigm, suggesting that DAC training teaches something general about problem-solving that CoT training alone cannot.

Methodologically, the paper demonstrates that end-to-end RL with only final-answer supervision is sufficient to train a meta-cognitive skill (problem decomposition) — something not obvious a priori. Lemma 2.1 provides the theoretical justification (final-answer correctness is a consistent surrogate for subproblem correctness), and the training dynamics (Figure 4, DAC rising from below CoT to above it) provide the empirical validation. This opens the door to RL-based training of other meta-cognitive reasoning strategies (analogical decomposition, problem re-framing, constraint identification) without requiring expensive step-level or subproblem-level annotations.

The finding that decomposition diversity dominates execution depth for test-time scaling (Section 4.2, Figure 6) challenges the prevailing emphasis in the test-time compute literature on verification and selection. The standard framing — generate many candidates, then use a verifier to pick the best one — implicitly assumes that candidate diversity is limited and the bottleneck is identifying the right answer. DAC-RL's results suggest the opposite: when the reasoning paradigm supports structured exploration, the bottleneck shifts to generating a good decomposition in the first place, and additional verification within a single decomposition provides diminishing returns. This reframes test-time compute allocation away from "more verification" toward "more structural exploration."

Research directions this work makes more attractive: training reasoning strategies (not just reasoning ability), studying the transfer between reasoning paradigms, and developing difficulty-aware routing between reasoning strategies. Research directions this work makes less attractive (or at least complicates): the assumption that scaling models and data within the CoT paradigm will naturally unlock higher reasoning capabilities, and the treatment of structured reasoning as purely an inference-time prompt engineering problem.

Follow-Up Research This Work Enables

Characterize the exact mechanism by which DAC training improves CoT reasoning. The Mix-RL result (Figure 5) — DAC training on hard problems producing a ~10% relative improvement in CoT performance on all problems — is the most surprising finding in the paper and the least explained. Does DAC training teach the model to recognize dead-ends earlier in CoT trajectories? To structure its CoT steps more hierarchically? To allocate more tokens to the critical sub-steps of a CoT solution? A strong follow-up would instrument the Mix-RL model's CoT generations and compare them to the CoT-RL model's generations on the same problems, measuring specific properties: average reasoning depth before first self-correction, fraction of tokens spent on verification vs. forward reasoning, and semantic clustering of CoT trajectories (do Mix-RL models explore a wider diversity of high-level approaches even when using CoT?). This would transform "DAC transfers to CoT" from a black-box observation into a mechanistic understanding.

Test whether the subproblem diversity advantage extends to combined proposer-verifier architectures. The test-time scalability analysis (Figure 6) shows that decompositions matter more than conquering samples at a fixed budget. But this is evaluated with a single verifier — the policy's own final-answer checking. A natural extension pairs DAC's decomposition diversity with a trained process reward model (PRM) or outcome reward model (ORM) that scores conquering solutions, similar to the PRM-guided search in Lightman et al. (2023) or the compute-optimal framework from the reference example. The concrete experiment: for a fixed total budget of k = 1024, compare five allocation strategies — (a) DAC with high n, low m, no verifier (as in the paper); (b) DAC with low n, high m, best-of-N weighted selection using a trained verifier; (c) CoT with best-of-N at k = 1024 using the same verifier; (d) DAC with intermediate n and m, with the verifier selecting the best conquering solution per decomposition and then majority voting across decompositions; (e) an adaptive strategy that uses the verifier's scores on early conquering attempts to decide whether to invest more budget in that decomposition or try a new one. The question is whether the diversity advantage persists when a strong verifier is available, or whether verification-based selection can compensate for fewer decompositions.

Investigate the failure boundary: when does DAC-RL provide zero benefit? The paper implicitly demonstrates that DAC-RL helps on problems where the base model has non-trivial pass rates (the Deep DAC training set filters to problems with <50% but >0% initial accuracy), but the cold-start problem — problems with zero initial pass rate — is unexplored. A direct experiment: partition the evaluation benchmarks into problems where the initial Qwen3-4B-Instruct-2507 model produces at least one correct answer in 2048 samples (solvable) versus problems where it produces zero correct answers (unsolvable), then measure DAC-RL's improvement on each subset separately. The hypothesis from the paper's implicit framework is that DAC-RL should provide gains only on the "solvable" subset and near-zero gains on the "unsolvable" subset. If DAC-RL does improve on unsolvable problems (by enabling correct solutions through decomposition that CoT could never produce), that would dramatically strengthen the case for DAC as a capability-unlocking mechanism rather than just a capability-reorganizing one. Conversely, if DAC-RL provides zero gain on unsolvable problems, the practical implication is clear: invest in DAC training only for problems within the model's rough capability horizon, and look elsewhere (larger models, more pretraining) for problems beyond it.

Can the division reward scheme be simplified without sacrificing performance? The division reward (Equation 2) is the most complex component of DAC-RL, combining format checks, quantity checks, and a lower-bound helpfulness condition developed through trial and error (Appendix C). A systematic ablation would test simpler alternatives: (a) binary division reward of 1 if any conquering solution in the group is correct, 0 otherwise (eliminating the "no group succeeded" exception and the format/quantity checks); (b) average conquering accuracy as the division reward, accepting the premature-solving failure mode but testing whether it can be mitigated by stronger KL regularization to the initial model; (c) removing the division reward entirely and relying purely on the conquering reward (testing whether the training dynamics naturally preserve division because the conquering prompt requires it); (d) keeping the current scheme but varying N_s (the minimum subproblem count) to find whether 3 is optimal or an arbitrary choice. The goal is to identify which components of the reward design are essential and which are historical artifacts.

Replicate on code generation benchmarks to test domain generality. All experiments use competition math with integer answers. Code generation shares important structural properties (decomposable into sub-functions, verifiable via unit tests, unique correct outputs for many problems) and lacks others (no clear "subproblem" decomposition for algorithmic problems, non-integer outputs, different error modes). A replication on HumanEval, MBPP, or LiveCodeBench would test whether DAC training transfers. The DAC prompt structure would need adaptation — instead of "divide into subproblems," the division stage would produce function stubs and specifications, and the conquering stage would implement each function and compose them. The key metric is whether DAC-trained code models show the same test-time scalability pattern (decomposition diversity > implementation sampling) and whether DAC training on hard coding problems transfers to improved CoT coding on easier ones (the Mix-RL result).

Develop a predicted-difficulty mechanism to make Mix-RL and Deep DAC practical. The Mix-RL strategy (Section 4.1) requires knowing which problems have CoT accuracy below 25%, which depends on ground-truth answers. A practical system needs to estimate this without ground truth. Concrete approaches to evaluate: (a) use the variance of CoT final answers as a difficulty proxy — problems where the model produces many different answers may be harder (the model is uncertain); (b) use the average token-level probability of the generated CoT solutions as a confidence estimate; (c) train a lightweight difficulty classifier on the DAPO-Math-17k training set (where ground-truth answers are available) using the initial policy's CoT accuracy as labels, then evaluate whether this classifier's predictions on held-out competition benchmarks correlate with the actual benefit of DAC training. The question is whether any of these practical proxies recovers the 10% CoT improvement that Mix-RL achieves with oracle difficulty.

Practical Applications and Downstream Use Cases

Self-improvement pipelines for math reasoning with unlabeled problems. An organization with access to a base LLM and a large corpus of unlabeled math problems (e.g., from textbooks, competition archives, or synthetic generation) wants to improve the model's reasoning through RL self-play. Using standard CoT RL, the model plateaus once it stops producing correct solutions for the remaining problems — the RL loop runs out of positive training signals. DAC-RL provides a mechanism to re-enter the self-improvement loop: even if the model cannot solve a problem via CoT, training it to decompose that problem (via the DAC pipeline) may enable correct solutions that feed back into training. The Mix-RL strategy (Section 4.1) is particularly relevant here: apply DAC training only to problems where CoT accuracy has dropped below a threshold, while maintaining CoT training on easier problems. The paper's numbers suggest this could improve both DAC and CoT performance — DAC accuracy rises by 8.6 points over CoT-RL (Table 1), and CoT itself improves by ~10% (Figure 5) from the transfer effect. The primary implementation barrier is the difficulty estimation problem (knowing which problems are "hard" without ground-truth answers), which can be partially addressed by using the model's own CoT answer distribution statistics as difficulty proxies.

Test-time compute allocation for competition math solvers. A team building a system to compete on math benchmarks (AIME, HMMT, IMO) with a fixed inference budget wants to maximize Pass@k for a given k. The standard approach — k independent CoT generations with majority voting or verifier-based selection — implicitly invests all compute in execution diversity within a single reasoning paradigm. The paper's test-time scalability analysis (Figure 6) provides a concrete alternative: for a DAC-trained model, allocate the k budget primarily across diverse subproblem decompositions (n large, m small) rather than across many conquering attempts per decomposition. At k = 1024, the high-n DAC configuration consistently outperforms both CoT baselines and low-n DAC configurations. The practical recipe would be: (1) train the model with DAC-RL, (2) at inference, generate many different decompositions for each problem (e.g., n = 128), (3) for each decomposition, generate a small number of conquering solutions (e.g., m = 8), (4) select the final answer via majority voting or verifier scoring across all decompositions. The paper's results suggest this should substantially outperform 1024 independent CoT samples at the same total budget, though the exact allocation would need tuning per benchmark.

Cost-efficient fine-tuning of small models for specialized reasoning domains. A practitioner with a small model (4B-7B parameters) wants to deploy it for a specific reasoning task where labeling is expensive but the task has verifiable answers (integer-output math problems, code with unit tests, formal proofs with verifiers). Standard CoT fine-tuning on a few thousand examples may plateau below the practitioner's accuracy target. DAC-RL provides a path to extract more capability from the same small model without requiring a larger teacher or more labeled data: the training uses only final-answer verification (which is cheap when answers are verifiable) and the RL loop generates its own training signals through self-exploration. The paper's numbers on Qwen3-4B-Instruct-2507 show DAC-RL reaching 46.1% Pass@1 (Table 1) — 3.4 points above the initial model — using only 17k training problems. If the practitioner additionally has access to a larger model for cold-start distillation (Section 4.4), the combined pipeline (distill from large teacher, then RL with DAC-RL) reaches 53.5% Pass@1. The cost profile is attractive: the teacher model is queried once per distillation example (3k queries in the paper's setup), and subsequent RL training uses only the small model's self-generated solutions, which are cheap compared to repeated large-model inference.

Training data generation for harder benchmarks via DAC decomposition. A team building a new dataset of challenging reasoning problems (e.g., for training the next generation of math models) needs to generate high-quality solution trajectories. Standard CoT generation on hard problems produces many incorrect solutions (which are useless for training) and few correct ones (which may be overly long and redundant, as shown in Appendix D, where the CoT solution is 5,072 tokens vs. DAC's 3,328). Using a DAC-trained model as the generator offers two advantages: (1) higher pass rate on hard problems (DAC-RL achieves 46.1% vs. CoT's 37.5% on Qwen3-4B-Instruct-2507, meaning ~23% more problems get correct solutions for the same generation budget), and (2) more compact solutions (Figure 7, left panel, showing DAC responses are shorter), which reduces the token cost of storing, processing, and training on the generated data. The decomposition structure also provides natural metadata: the subproblems become intermediate reasoning targets that downstream models can learn from, and the diversity of decompositions for the same problem creates a richer training signal than multiple near-identical CoT solutions.