LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

ArXiv: 2605.08083

🎯 Pitch

Instead of hand-crafting test-time scaling heuristics, this paper shows that an LLM agent can automatically discover superior branching, pruning, and stopping strategies—improving the accuracy–cost Pareto frontier while the entire automated search costs only $39.90 and 160 minutes. The discovered controller generalizes across held-out benchmarks and model scales, even cutting token consumption by ~70% compared to Self-Consistency@64 at matched accuracy.

1. Executive Summary

This paper proposes AutoTTS, an environment-driven framework that reframes test-time scaling strategy design as an automated discovery problem — shifting the human role from hand-crafting branching, pruning, and stopping heuristics to constructing offline replay environments where an explorer LLM (Claude Code) synthesizes controllers automatically. The discovery instantiates width–depth test-time scaling as controller synthesis over pre-collected reasoning trajectories and probe signals, evaluating candidate controllers cheaply against fixed replay data without repeated LLM calls, and introduces beta parameterization to collapse hyperparameter search to a single scalar trade-off knob and execution trace feedback to enable the agent to diagnose failure modes beyond scalar accuracy–cost outcomes. On mathematical reasoning benchmarks (AIME24 as search set; AIME25 and HMMT25 as held-out sets) across four Qwen3 model scales (0.6B–8B), the discovered Confidence Momentum Controller (CMC) improves the accuracy–cost Pareto frontier over strong hand-crafted baselines — for example, at β = 0.5, reducing token consumption by approximately 69.5% compared to Self-Consistency@64 while maintaining on-par accuracy on held-out benchmarks — with the entire five-round discovery process costing only $39.9 and 160 minutes. The discovered controller generalizes to held-out benchmarks and across model families (including a Llama-based distilled model on math and GPQA-Diamond on science QA), establishing that effective TTS strategies can be automatically discovered given the right environment, provided the control space is structured to be tractable and the discovery loop supplies rich, diagnostic feedback.

2. Context and Motivation

The Core Problem: Test-Time Scaling Strategies Are Hand-Crafted and Underexplored

The fundamental problem this paper addresses is that effective strategies for allocating inference-time computation are discovered through manual trial-and-error rather than systematic search. Test-time scaling (TTS) — the practice of spending additional computation during inference to improve LLM outputs — has become a widely adopted paradigm. The key insight from prior work (Snell et al., 2024) is that performance depends not merely on how much computation is used, but critically on how it is allocated across different reasoning paths. Yet despite rapid proliferation of TTS methods, the design process remains stubbornly manual: researchers hypothesize heuristics for when to branch into parallel reasoning chains, when to deepen existing chains, when to probe intermediate answers, when to prune unpromising paths, and when to stop — then implement these heuristics and tune their thresholds by intuition against benchmark results.

This manual approach creates a profound bottleneck. The space of possible TTS strategies is combinatorially large — any controller operating in a width–depth allocation space must make sequential decisions about branching, continuation, probing, pruning, and termination, with the consequences of early decisions propagating through the remainder of the inference budget. Human designers can explore only a tiny fraction of this space, guided by intuition that may be systematically biased toward familiar patterns (e.g., "parallel sampling with majority voting works, so let's add early stopping") rather than discovering genuinely novel allocation mechanisms. This means the field is likely leaving substantial performance on the table — not because existing methods are weak, but because the design process itself is constrained by what humans can conceive and tune by hand.

The motivating example in Figure 2 makes this concrete. When viewed through the lens of a width–depth control space (where width = number of parallel reasoning branches explored, and depth = how far each branch is developed), existing methods correspond to specific hand-drawn trajectories:

• Self-Consistency@64 (Wang et al., 2022) occupies a fixed full-budget corner — always sample 64 complete reasoning chains, then majority vote. There is no adaptivity: every question gets identical treatment regardless of difficulty.
• Adaptive Consistency (ASC) (Aggarwal et al., 2023) and Early-Stopping Consistency (ESC) (Li et al., 2024) adapt only along the width axis — they sample branches sequentially or in chunks and stop early when answer consensus emerges, but never selectively deepen individual branches.
• Answer Consistency (Liu and Wang, 2025) adapts only along the depth axis — it follows a single reasoning chain and stops when intermediate answer convergence is detected, never exploring parallel alternatives.
• Self-Truncation Best-of-N (ST-BON) (Wang et al., 2025) follows a fixed pattern: expand wide to generate many candidates, prune to a single branch, then deepen that branch — a hand-specified trajectory through the 2D space.
• Parallel-Probe (Zheng et al., 2026) starts wide with a fixed cohort of parallel chains and progressively prunes while deepening — the most adaptive hand-crafted method, but still following a predetermined structural template where the cohort size, pruning triggers, and stopping rules are set by human designers.

The critical observation is that all of these are special cases — manually specified policies within a shared underlying control space. The paper argues that this perspective is not intended to reduce all TTS algorithms to a two-dimensional abstraction (many methods involve richer structures like tree search or verifier-guided refinement). Rather, it reveals that human-designed TTS strategies are fundamentally sampling from a constrained subset of possible allocation policies. The unexplored regions of this control space may contain strategies that outperform anything designed by hand, but accessing them requires a systematic search mechanism rather than manual intuition.

Why This Problem Matters: Scaling Inference Compute Is Becoming a Primary Lever for LLM Performance

The importance of solving this problem stems from a structural shift in how LLM performance is achieved. Historically, the dominant paradigm was scaling pretraining: train larger models on more data, and improvements follow from the resulting increase in model capability. This approach has been well-characterized by scaling laws (Hoffmann et al., 2022) and drove the progression from GPT-2 to GPT-4 and beyond.

However, the economics of this approach are shifting. Pretraining runs for frontier models now cost hundreds of millions of dollars and require increasingly scarce high-quality data. Simultaneously, inference-time computation has emerged as an alternative lever: rather than baking all capability into model weights during pretraining, systems can spend additional computation after the prompt is received to improve outputs through search, verification, and refinement. The landmark finding from Snell et al. (2024) — that a smaller model with compute-optimal test-time strategies can outperform a ~14× larger model on problems within its capability range — demonstrated that test-time and pretraining compute are partially substitutable under certain conditions.

This reorients the optimization problem: instead of "how do we train the largest possible model?", the question becomes "given a total compute budget spanning training and inference, how should we allocate between them?" Answering this requires understanding not just how much test-time compute to use, but how to allocate it — which strategy to deploy, with which hyperparameters, for which types of problems. The paper's core claim is that we are currently leaving enormous efficiency on the table because the strategy design process is manual and therefore cannot explore the full space of possible allocation policies.

Practical implications of this gap include:

• Cost efficiency at scale: For organizations running millions of inference queries, a strategy that achieves equivalent accuracy with 70% fewer tokens (as the discovered controller does at β = 0.5 compared to SC@64) translates directly to massive cost savings. Without systematic discovery, such strategies may exist but remain unfound.
• Deployment flexibility: Smaller models augmented with discovered TTS strategies could match larger models on certain problem distributions, enabling on-device or edge deployment where large models are infeasible. But this requires discovering strategies that work across model scales and problem types, not just tuning heuristics to a specific (model, benchmark) pair.
• Rapid adaptation to new models: As new model families are released (Qwen3, Llama-4, etc.), optimal TTS strategies likely shift because model capabilities, error patterns, and calibration properties change. Manual re-tuning of heuristics for each new model is labor-intensive and slow. An automated discovery pipeline that can re-discover strategies for new models in hours ($39.9, 160 minutes) would dramatically accelerate deployment cycles.
• Self-improvement pipelines: If LLMs can automatically discover better ways to use inference compute, this enables a meta-level self-improvement loop — the model discovers strategies for better reasoning, those strategies generate higher-quality training data, the model is fine-tuned on that data, and the cycle repeats with an improved base model and re-discovered strategies. Manual strategy design cannot keep pace with such a loop.

Where Prior Approaches Fall Short: Manual Design, Isolated Mechanisms, and Brittle Heuristics

The paper identifies several specific limitations in how TTS strategies are currently developed:

Manual design cannot explore the full allocation space. Even sophisticated hand-crafted methods like Parallel-Probe, which combines parallel chains, intermediate probing, off-track pruning, and stable-majority termination, represent a single point in the space of possible controllers. The human designer chose a particular structural template (fixed initial cohort → probe → classify branches → prune deviants → stop on consensus), but there is no principled reason to believe this template is optimal. The space of possible controllers includes alternative mechanisms — momentum-based stopping rather than instantaneous confidence gates, coupled width–depth control where widening and deepening decisions share a feedback signal, priority-based depth allocation rather than uniform or alignment-biased deepening — that would be difficult to conceive through intuition alone because they require reasoning about complex interactions between multiple adaptive mechanisms operating simultaneously.

The discovered Confidence Momentum Controller (CMC) described in Appendix D illustrates this concretely. It incorporates four non-obvious mechanisms that emerged from the discovery process rather than human design: trend-based stopping via EMA momentum (prevents premature termination on transient confidence spikes), coupled width–depth control through a shared evidence signal (confidence gains suppress new branch spawning while stagnation triggers widening), alignment-aware depth allocation (concentrates computation on branches matching the emerging consensus while still advancing all active branches), and conservative branch abandonment (only drops branches after persistent deviation). None of these individually is beyond human conception, but their coordinated interaction — where the EMA trend simultaneously gates stopping, triggers widening, and influences depth allocation priority — represents a level of joint design that would be extremely difficult to arrive at through manual tuning.

Prior discovery methods cannot be directly applied. There is a rich literature on automated algorithm discovery, ranging from classical AutoML (Zoph and Le, 2016; Elsken et al., 2019) to LLM-driven program search methods like FunSearch (Romera-Paredes et al., 2024), Evolution of Heuristics (Liu et al., 2024), AlphaEvolve (Novikov et al., 2025), and ADAS (Hu et al., 2024). These methods use LLMs to iteratively propose and refine algorithms in code, demonstrating that program-space search can discover novel solutions in domains like combinatorial optimization, mathematical discovery, and agent design. Meta-Harness (Lee et al., 2026) further advances this paradigm by exposing full execution histories to the proposer, enabling targeted diagnosis of failure modes.

However, applying these discovery frameworks to TTS strategy design faces a fundamental bottleneck: evaluation cost. In prior algorithm discovery settings, evaluating a candidate algorithm is cheap — running a sorting heuristic on test cases, computing a mathematical score function, or evaluating a harness configuration takes milliseconds to seconds. In contrast, evaluating a TTS controller online would require invoking the base LLM repeatedly to generate reasoning trajectories on demand for every candidate strategy across many benchmark questions. A single evaluation run with 64 branches × 500 tokens per probe interval × 128 questions would consume millions of tokens, and a discovery loop with tens of candidates would be prohibitively expensive (potentially thousands of dollars per round). This evaluation cost bottleneck is why TTS strategy design has remained hand-crafted despite advances in algorithmic discovery — the naive application of existing discovery methods would be economically infeasible.

Existing TTS methods are studied in isolation, not as instances of a shared control problem. The literature has produced a diverse array of methods — adaptive consistency, early-stopping consistency, answer convergence, self-truncation, deep pruning, Parallel-Probe, and many others — each evaluated independently against baselines on specific benchmarks. But there is no unified framework for comparing them as policies within a shared computation-allocation space, understanding their complementary strengths, or systematically exploring the regions of the space they leave uncovered. This fragmentation means the field lacks a cumulative understanding: each new paper proposes a new heuristic and evaluates it against prior heuristics, but the design principles that generalize across methods remain implicit. The width–depth perspective in Figure 2 is the paper's attempt to provide this unifying lens, revealing that apparently different methods are tracing different trajectories through the same underlying space — and that most of the space remains unexplored.

Hand-crafted strategies are brittle and may not transfer. A strategy tuned on one benchmark with one model often requires re-tuning for a new benchmark or model scale, because the optimal thresholds for pruning, stopping, and branching depend on the model's calibration properties and the problem difficulty distribution. The paper's experimental results show that the discovered controller does transfer across model scales (0.6B to 8B) and to held-out benchmarks (AIME25, HMMT25) without modification, suggesting that systematic discovery in a structured control space yields strategies that capture more fundamental properties of effective allocation rather than dataset-specific tuning. Hand-crafted strategies, by contrast, embed the designer's implicit assumptions about problem difficulty, model reliability, and answer convergence patterns — assumptions that may not hold when the distribution shifts.

How AutoTTS Positions Itself: From Strategy Design to Environment Design

The paper's fundamental reframing is to shift the human role from designing TTS strategies to designing discovery environments. This is not merely an automation argument ("let the LLM do the work"). It is a claim about where human insight provides the most leverage.

The key insight is that human researchers are good at understanding the structure of a problem — defining the relevant state space, identifying the available actions, specifying the objectives and constraints — but poor at exploring large combinatorial spaces of sequential decisions to find optimal policies within that structure. Conversely, search procedures (whether classical optimization or LLM-driven program synthesis) are good at exploring structured spaces but require that structure to be well-defined. AutoTTS positions human effort where it provides the most value: constructing the replay environment that makes the control space tractable, defining the feedback signals that enable diagnosis, and specifying the objectives (accuracy–cost tradeoff) and constraints (beta parameterization to prevent overfitting). The actual search over controller designs is delegated to the agent.

This positioning draws on a broader trend in AI research where the boundary between human design and automated search is shifting. In neural architecture search, humans moved from designing individual architectures to designing search spaces and training protocols. In program synthesis, humans moved from writing programs to writing specifications. AutoTTS applies the same logic to test-time scaling: humans define the environment, agents discover the strategies.

The paper is careful to position this as a proof-of-concept instantiation rather than a final solution. The width–depth control space, the offline replay construction from pre-collected trajectories, and the specific instantiation with Claude Code as the explorer are one concrete realization of the environment-driven discovery paradigm. The broader claim is that this paradigm — constructing environments where TTS strategies can be systematically discovered rather than hand-crafted — is the right direction for the field, and that richer environments with more complex action spaces (tree search, verifier-guided refinement, revision-based mechanisms) are natural extensions.

Critically, the paper acknowledges that the discovered controller is not necessarily globally optimal — it is the best strategy found within five rounds of agent-driven search in a particular instantiation of the environment. The contribution is not the specific controller (CMC) but rather the demonstration that the discovery paradigm works: it produces strategies that outperform strong hand-crafted baselines, generalize to held-out benchmarks and model scales, and cost only $39.9 to discover. This establishes a new baseline for how TTS research can be conducted — not by proposing the next heuristic, but by building better environments in which better heuristics can be systematically found.

Relationship to the Example Paper (Snell et al., 2024)

For readers familiar with the Snell et al. (2024) paper on compute-optimal test-time scaling (analyzed in the prior sections of this document), there is an important conceptual relationship and a key difference.

Conceptual relationship: Both papers study how to optimally allocate test-time compute. Snell et al. showed that the optimal allocation strategy depends on problem difficulty — easy problems benefit from sequential revisions, hard problems benefit from parallel search, and a difficulty-conditioned policy recovers 4× efficiency gains. That paper's contribution was demonstrating that allocation matters and that it should be adaptive.

Key difference: Snell et al. studied a small set of pre-specified strategies (best-of-N, beam search, sequential revisions, lookahead search) and selected among them per difficulty bin. The space of possible strategies was fixed by the researchers. AutoTTS asks: what if the optimal strategy is none of the pre-specified ones? What if the best controller combines mechanisms (momentum-based stopping, coupled width–depth feedback, priority-based depth allocation) that no human has conceived? The discovery paradigm opens the space of possible strategies far beyond what humans can enumerate, while the environment construction (replay, beta parameterization, execution traces) makes searching this space feasible.

In this sense, AutoTTS builds on the insight from Snell et al. (that allocation strategy matters) but addresses the next problem: how do we find good allocation strategies without manually designing and evaluating each candidate? The two papers are complementary — Snell et al. established the importance of the problem, and AutoTTS provides a methodology for solving it at scale.

3. Technical Approach

3.1 Reader Orientation (Approachable Technical Breakdown)

The paper builds an automated discovery pipeline that lets a coding agent (Claude Code) iteratively design, evaluate, and refine controllers — algorithmic programs that decide how to allocate a fixed inference budget across multiple parallel reasoning branches for each math problem — without ever invoking the base LLM during the discovery process itself. The core problem it solves is that human-designed test-time scaling strategies explore only a tiny fraction of the possible allocation policies, so the solution's "shape" is an offline replay environment where candidate controllers are evaluated cheaply against pre-collected reasoning traces, combined with feedback mechanisms (scalar accuracy–cost curves plus detailed execution traces) and a constrained search space (beta parameterization) that together make it tractable for an explorer agent to discover strategies that outperform anything designed by hand.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components connected in a closed discovery loop:

1. Offline Replay Environment — pre-collected reasoning trajectories and probe signals for each question, stored before discovery begins. All LLM calls happen once during data collection; controller evaluation reads deterministically from this stored data with zero generation cost.

2. Controller (Policy π) — a code-defined program that implements the solve(question) interface. It observes the current 2D probing state (active branches, their depths, revealed probe answers, remaining budget) and selects atomic actions: BRANCH (start a new reasoning path), CONTINUE(i) (advance branch i by one generation interval), PROBE(i) (reveal branch i's intermediate answer without advancing it), PRUNE(i) (abandon branch i), or ANSWER (terminate and aggregate final answer).

3. Explorer Agent (Claude Code) — an LLM that reads accumulated history (all prior controller implementations, their accuracy–cost outcomes, and full execution traces), analyzes failure modes, and proposes an improved controller by directly editing the OptimalController class in method.py.

4. History / Memory — stores for each round: (a) the exact controller source code, (b) scalar accuracy–cost scaling curves across all β values on the search set, and (c) per-question execution traces showing every decision (which branches were probed, pruned, abandoned, and why the controller stopped).

5. Evaluator (eval.py) — sweeps each proposed controller across a grid of β values, runs it on every question in the search set (against the offline replay data), records accuracy and token cost, and appends results to history.

Information flows in a loop: History → Explorer → Proposed Controller → Evaluator (against replay data) → Updated History → Explorer reads history and proposes again. After a fixed number of rounds (5), the controller achieving highest accuracy on the search set is selected and evaluated on held-out benchmarks.

3.3 Roadmap for the Deep Dive

I'll explain the system in this order, which builds from the foundational formalization outward to the discovery mechanisms:

• First, the formal MDP for test-time control (Section 2) — the state space, action space, transition dynamics, cost model, and objective function. This is the language in which all controllers and the environment are defined, so it must come first.

• Second, the offline replay environment (Section 3.1) — how pre-collected trajectories and probe signals instantiate the MDP into a cheap, deterministic evaluation substrate. This is what makes discovery affordable and is the single most important design choice.

• Third, the discovery loop mechanics (Section 3.2) — the round-by-round process: what the explorer sees in the history, how it proposes new controllers, and how controllers are selected.

• Fourth, beta parameterization (Section 3.3) — the mechanism that collapses a high-dimensional hyperparameter search space into a single scalar knob, preventing overfitting to the search set.

• Fifth, execution trace feedback (Section 3.2, detailed) — how per-step decision traces enable the explorer to diagnose why a controller fails, beyond what scalar accuracy–cost numbers reveal.

• Sixth, the discovered controller (CMC) — the concrete output of one discovery run, analyzed to show how the four non-obvious mechanisms (trend-based stopping, coupled width–depth control, alignment-aware depth allocation, conservative abandonment) emerge from agent-driven search.

3.4 Detailed, Sentence-Based Technical Breakdown

This is an automated discovery paper whose core idea is that test-time scaling strategies should not be hand-crafted but instead discovered by an agent operating in a carefully constructed offline replay environment that makes controller evaluation cheap, deterministic, and diagnostic. The contribution is not the specific discovered controller (though it outperforms baselines), but rather the framework and environment design that makes discovery feasible at all.

The Formal MDP for Test-Time Control

The paper formalizes test-time scaling as a Markov Decision Process (MDP) over pre-collected reasoning trajectories, where a controller sequentially selects actions that allocate a generation budget across a branching 2D space of width (parallel reasoning chains) and depth (how far each chain is developed). This formalization provides the shared language in which all controllers — both hand-crafted baselines and discovered policies — are expressed, and defines the interface between controllers and the replay environment.

State space. At decision step $t$, the state $s_t$ captures everything the controller knows about the current allocation:

$s_t = (q, m_t, I_t, \ell_t, Z_t, \Omega_t)$

where:

• $q \in \mathcal{Q}$ is the question text (immutable context),
• $m_t \in \mathbb{Z}_{\geq 0}$ is the number of branches instantiated so far (some may be finished or pruned),
• $I_t \subseteq [m_t]$ is the set of currently active (not yet pruned or completed) branch indices, with $[m_t] = \{1, \ldots, m_t\}$ and $[0] = \emptyset$,
• $\ell_t = (\ell_{t,i})_{i \in [m_t]}$ records the current depth (number of generation intervals completed) of every instantiated branch, where $\ell_{t,i} \geq 1$ for all $i \in [m_t]$ (every branch has at least one interval after instantiation),
• $Z_t = (Z_{t,i})_{i \in [m_t]}$ records the generated prefix sequences, where $Z_{t,i} = (z_{i,1}, \ldots, z_{i,\ell_{t,i}})$ contains the prefixes produced on branch $i$ up to its current depth — but critically, these are stored in the offline data and not directly observed by the controller (the controller only sees probe answers, not raw token sequences),
• $\Omega_t \subseteq \{(i, k, \omega_{i,k}) : i \in [m_t], 1 \leq k \leq \ell_{t,i}\}$ is the set of revealed probe feedback, where $\omega_{i,k}$ is the intermediate answer extracted from prefix $z_{i,k}$ — and these are only known to the controller if it explicitly took a PROBE action at that $(i, k)$ position.

What this state represents operationally: at any moment during inference, the controller knows which branches exist, which are still active, how deep each has been developed, and which probe answers have been explicitly requested (the $\Omega_t$ set). It does not know probe answers for $(i, k)$ pairs that exist in the offline data but haven't been revealed — those are hidden until a PROBE(i) action is taken. For pruned branches $i \in [m_t] \setminus I_t$, $\ell_{t,i}$ and $Z_{t,i}$ record the depth and prefixes at the moment of pruning, and the branch is frozen — no further actions can target it, but its historical data remains.

Why this state representation: the key design choice is that $\Omega_t$ is partial and controller-controlled. The controller must actively decide when to probe, creating an information-gathering cost (the probe action itself has cost, as shown below) and a strategic tension: probe too early and you pay cost for potentially noisy intermediate answers; probe too late and you waste generation on branches that could have been pruned earlier. This partial observability is what makes the control problem non-trivial — it requires balancing exploration (probing to gather information) against exploitation (continuing branches that appear promising).

The initial state is $s_0 = (q, 0, \emptyset, \emptyset, \emptyset, \emptyset)$ — no branches exist yet, no history.

Action space. Given state $s_t$, the admissible actions $\mathcal{A}(s_t)$ are:

$\mathcal{A}(s_t) = \{\texttt{BRANCH}\} \cup \{\texttt{CONTINUE}(i) : i \in I_t\} \cup \{\texttt{PROBE}(i) : i \in I_t, \nexists \omega \text{ s.t. } (i, \ell_{t,i}, \omega) \in \Omega_t\} \cup \{\texttt{PRUNE}(i) : i \in I_t\} \cup \{\texttt{ANSWER}\}$

What each action does operationally:

• BRANCH: Creates a new branch $m_t + 1$ and immediately advances it through one fixed-length generation interval to produce prefix $z_{m_t+1, 1}$. The branch is added to the active set. This is the only way to increase width — the controller cannot create a branch without also investing one interval of generation into it, so branching always has a minimum cost.

• CONTINUE(i): Advances active branch $i$ by exactly one generation interval. The new prefix $z_{i, \ell_{t,i}+1}$ is appended to the branch's history. Depth increases by 1. This action does not reveal the intermediate answer at the new depth — that requires a separate PROBE action. This separation is important: the controller can invest computation to deepen a branch (consuming generation budget) without necessarily paying the additional cost to read the intermediate answer, if it plans to read answers in batches later.

• PROBE(i): Reveals the intermediate answer $\omega_{i, \ell_{t,i}}$ at the current depth of branch $i$. This does not advance the branch — it simply adds the existing $\omega_{i, \ell_{t,i}}$ to $\Omega_{t+1}$, making it visible to the controller. A probe is only admissible if the current depth's answer hasn't already been revealed for that branch. This means the controller can probe the same depth only once, preventing redundant information gathering.

• PRUNE(i): Removes branch $i$ from the active set $I_t$ but preserves its history (prefixes and probe answers recorded up to that point). A pruned branch is frozen — no more CONTINUE or PROBE actions can target it. The branch's completed answer (if it was finished) or latest intermediate answer remains available for final aggregation.

• ANSWER: Terminates the episode immediately. The terminal state $s_T$ is passed to an aggregation rule $\text{Agg}$ that produces the final answer $\hat{y}$.

Why this action space design: the separation of CONTINUE (spend generation budget) from PROBE (spend probe cost to read answer) enables fine-grained control over information acquisition. A controller can, for example, deepen a branch for several intervals (investing in the branch's reasoning) and then probe once to read the accumulated answer, rather than probing at every step. This models real inference where reading intermediate outputs (e.g., extracting answer strings from partially generated text) has a cost (tokenization, parsing, verifier computation) that is distinct from the generation cost. The PRUNE action introduces an irreversibility — once pruned, a branch cannot be resumed — creating a commitment problem: prune too early and you lose potentially useful computation; prune too late and you waste budget.

Transition dynamics (informally). The transition function $P(s_{t+1} | s_t, a_t)$ is deterministic given the offline data, which is the key insight enabling cheap evaluation. When $a_t = \texttt{BRANCH}$, the next state has $m_{t+1} = m_t + 1$, the new branch is added to $I_{t+1}$ with $\ell_{t+1, m_t+1} = 1$ and its first prefix $z_{m_t+1, 1}$ loaded from storage. When $a_t = \texttt{CONTINUE}(i)$, only $\ell_{t+1,i} = \ell_{t,i} + 1$ changes, with $z_{i, \ell_{t,i}+1}$ appended. When $a_t = \texttt{PROBE}(i)$, only $\Omega_{t+1}$ gains the entry $(i, \ell_{t,i}, \omega_{i,\ell_{t,i}})$. When $a_t = \texttt{PRUNE}(i)$, only $I_{t+1} = I_t \setminus \{i\}$ changes. When $a_T = \texttt{ANSWER}$, the episode terminates.

Cost model. The computation cost of a state is:

$\text{Cost}(s_t) = \sum_{i=1}^{m_t} \ell_{t,i} + \kappa_{\text{probe}} |\Omega_t|$

where $\kappa_{\text{probe}} \geq 0$ is the relative cost of reading one probe signal compared to one generation interval. In the paper's experiments, probing is treated as essentially free relative to generation ($\kappa_{\text{probe}} = 0$ in practice), reflecting that extracting an answer string from a generated prefix is negligible compared to the cost of generating the prefix itself. The first term $\sum_{i=1}^{m_t} \ell_{t,i}$ counts the total number of generation intervals spent across all branches ever instantiated (including pruned and completed ones) — this is the dominant cost.

What this cost model captures: it measures the total "reasoning effort" invested up to time $t$. A branch that was started, deepened for 10 intervals, and then pruned cost 10 intervals of generation. A branch that ran to completion at depth 20 cost 20 intervals. The sum over all branches gives the total token budget consumed. The $\kappa_{\text{probe}}|\Omega_t|$ term accounts for any additional cost of reading intermediate answers (e.g., running a verifier model on partial outputs), though the paper sets this to zero in practice.

Objective. The discovery process aims to find a code-defined policy $\pi$ (a Python class implementing the solve method) and a hyperparameter $\beta$ (a scalar controlling budget aggressiveness) that maximize:

$\max_{(\pi, \beta)} \mathbb{E}_{(q,y) \sim \mathcal{D}, \tau \sim P_{\pi,\beta}(\cdot|q)} \left[ \mathbf{1}\{\hat{y}_{\pi,\beta}(\tau) = y\} - \gamma \, C(\tau) \right]$

where:

• $\mathcal{D}$ is the distribution over question-answer pairs,
• $\tau = (s_0, a_0, s_1, a_1, \ldots, s_T)$ is the full execution trajectory — the sequence of states visited and actions taken — induced by running $(\pi, \beta)$ from the initial state,
• $P_{\pi,\beta}(\cdot|q)$ is the distribution over trajectories, including any randomness in the generation process (though in the replay setting this randomness is fixed — the set of pre-collected branches and their probe answers are deterministic given the sampled trajectories),
• $\hat{y}_{\pi,\beta}(\tau)$ is the final answer produced by the controller's aggregation rule at termination,
• $C(\tau) = \text{Cost}(s_T)$ is the total computation cost at the terminal state,
• $\gamma$ is a trade-off parameter weighting accuracy against cost (not explicitly set — instead, the discovery loop searches over controllers that achieve different points on the accuracy–cost Pareto frontier by varying $\beta$).

What this objective computes: it formalizes that the ideal controller maximizes the probability of correct answers while minimizing computation cost, with relative importance controlled by $\gamma$. In practice, rather than optimizing this scalarized objective, the discovery loop searches for a controller family (parameterized by $\beta$) that traces out a Pareto frontier — for each $\beta$ value, the controller achieves some (accuracy, cost) pair, and better controllers push this frontier upward and leftward.

Why this MDP formulation: it abstracts away from the specifics of any particular TTS method (majority voting, beam search, early stopping, etc.) and instead captures the common control problem underlying all of them: at each step, given partial information about parallel reasoning paths, decide where to invest the remaining computation budget. This abstraction is what enables the discovery framework — any controller that conforms to this interface (observes state, selects actions, produces final answer) can be evaluated in the same replay environment, regardless of whether it was designed by hand or discovered by an agent.

The Offline Replay Environment

The central challenge in making controller discovery affordable is that evaluating a candidate controller online — invoking the base LLM to generate reasoning trajectories on demand for every action — would be prohibitively expensive. If a single evaluation run involves, say, 64 branches × 20 intervals each × 500 tokens per interval, that's 640,000 tokens per question, and with 128 questions in the search set, a single controller evaluation could cost millions of tokens. A discovery loop with tens of candidates would be economically infeasible.

The offline replay environment solves this by moving all LLM calls to a one-time pre-computation phase before any discovery begins.

Offline data collection (one-time cost). For each question $q \in \mathcal{Q}$ (the search set $\mathcal{Q}_{\text{search}}$ is AIME24 across all four Qwen3 models), the authors pre-sample 128 independent reasoning trajectories from the base LLM (the Qwen3 model being evaluated) at temperature 0.7. Each trajectory is a complete chain-of-thought solution generated from scratch, not a continuation of any other trajectory. Following the data collection protocol of Parallel-Probe (Zheng et al., 2026), each trajectory is segmented into fixed-length intervals of $\Delta \text{tokens}$ (the paper specifies a probing interval of 500 tokens in Section 4, though the exact value $\Delta$ is configurable).

For each trajectory $i$ and each interval $k$ (where $k$ ranges from 1 up to the trajectory's total length in intervals):

• The prefix $z_{i,k}$ is the concatenation of the first $k$ generation intervals — that is, the partial chain-of-thought up to token $k \cdot \Delta$.
• The intermediate answer $\omega_{i,k}$ is extracted from this prefix. The paper doesn't detail the extraction method, but it involves parsing the model's output to identify the answer it would produce if generation stopped at this point (e.g., by finding the last stated answer before the interval boundary, or by appending a prompt like "Therefore, the answer is:" and extracting the prediction).

All of this data — all prefixes $z_{i,k}$ and all intermediate answers $\omega_{i,k}$ for all $i = 1 \ldots 128$, all $k = 1 \ldots \text{length}_i$ — is stored offline in a data structure keyed by question and (branch_index, depth). This storage is what the replay environment reads from.

What this buys: for 128 questions × 128 trajectories × average length of, say, 10 intervals × 500 tokens, the one-time collection generates roughly 82 million tokens. At standard API pricing (~$1–2 per million tokens for small models), this is maybe$100–200 in one-time cost. But once collected, every subsequent controller evaluation is free — it just reads from this stored data. This transforms controller evaluation from an O(number_of_candidates × tokens_per_eval) cost to O(1) amortized cost.

Evaluation via offline replay. To evaluate a specific controller $(\pi, \beta)$ on a question $q$:

1. The evaluator initializes the state $s_0 = (q, 0, \emptyset, \emptyset, \emptyset, \emptyset)$.
2. At each step $t$, the controller's solve method observes the current state $s_t$ and selects an action $a_t \in \mathcal{A}(s_t)$.
3. If $a_t = \texttt{BRANCH}$: the evaluator samples a fresh branch from the pre-collected pool for question $q$. Specifically, there are 128 pre-collected trajectories, and each BRANCH action draws one of these (without replacement) as a new branch. The branch's first interval prefix $z_{m_t+1, 1}$ and the intermediate answer $\omega_{m_t+1, 1}$ are loaded from storage. The state updates: $m_{t+1} = m_t + 1$, new branch added to active set with depth 1. However, $\omega_{m_t+1, 1}$ is not automatically added to $\Omega_{t+1}$ — the controller only sees it if it explicitly probes.
4. If $a_t = \texttt{CONTINUE}(i)$: the evaluator increments branch $i$'s depth by 1. The prefix $z_{i, \ell_{t,i}+1}$ is loaded from storage (it was pre-computed). The intermediate answer at this new depth is available in storage but remains hidden ($\Omega_t$ unchanged).
5. If $a_t = \texttt{PROBE}(i)$: the evaluator retrieves the pre-stored intermediate answer $\omega_{i, \ell_{t,i}}$ at branch $i$'s current depth and adds it to $\Omega_{t+1}$. No generation occurs — the cost is $\kappa_{\text{probe}}$ (typically 0).
6. If $a_t = \texttt{PRUNE}(i)$: branch $i$ is removed from the active set. Its existing data is preserved but frozen.
7. If $a_T = \texttt{ANSWER}$: the episode terminates. The controller's aggregation rule $\text{Agg}_{\pi,\beta}$ (typically majority voting over completed answers or over latest answers from all branches) produces final answer $\hat{y}$. The answer is compared to ground truth to compute accuracy, and $\text{Cost}(s_T)$ gives the total token cost.

Key property: determinism. Because all prefixes and probe answers are pre-stored, the replay evaluation is completely deterministic for a given controller and a given sampling of which 64 (or fewer) trajectories are used as the branch pool. This removes variance from the evaluation, making comparisons between controllers clean and reproducible. The paper reduces remaining variance by evaluating each controller 64 times independently, each time randomly sampling a subset of 64 trajectories from the 128 pre-collected ones (or fewer, depending on how many branches the controller actually spawns), and averaging results.

Why this replay design is the linchpin: without it, discovering TTS controllers through iterative agent search would be economically infeasible — each evaluation would require fresh LLM calls, and a five-round loop with dozens of β values per round would cost thousands of dollars. The replay environment collapses the evaluation cost to essentially zero (just reading from disk/memory and running Python control flow), making the entire discovery loop cost only $39.9 and 160 minutes. This is the single most important design decision in the paper.

Cost accounting caveat. The one-time cost of collecting the offline data (generating 128 trajectories per question from the base LLM) is not included in the $39.9 figure. That$39.9 covers only the discovery loop itself (API calls to Claude Code for proposing controllers, plus compute for running the replay evaluations). For a fair total-cost comparison, one would add the cost of building the replay environment — but the authors argue this cost is amortized across many discovery runs, since the same replay data can be used to discover controllers for many different configurations and the discovery itself can be repeated. Additionally, in many deployment scenarios, companies already pre-generate large numbers of trajectories for evaluation purposes, so the replay data may already exist as part of standard workflows.

The Discovery Loop Mechanics

The discovery loop is an iterative process where an explorer LLM (Claude Code, accessed via API) proposes, evaluates, and refines controller implementations across five rounds. The key design elements are: what the explorer sees in its context, how it proposes improvements, and how the final controller is selected.

Search set construction. The search environment $\mathcal{E}_{\text{search}}$ is the union of AIME24 environments across all four Qwen3 models (0.6B, 1.7B, 4B, 8B). This means a single controller is evaluated on AIME24 four times — once with trajectories generated by Qwen3-0.6B, once with 1.7B, once with 4B, once with 8B — and the aggregate accuracy across all four model scales is used for selection. The rationale is to force the discovered controller to work across model capacities, preventing overfitting to a single model's error patterns or calibration properties. The held-out evaluation sets (AIME25, HMMT25) are never seen during discovery or selection.

Round structure. Each discovery round proceeds as follows:

1. Agent reads history. The explorer (Claude Code) is provided with the full accumulated history $\mathcal{H}$, which contains for every previous round:

   • The exact source code of the OptimalController class that was proposed and evaluated in that round (stored in method.py snapshots).
   • The scalar accuracy–cost results: for each β value swept, the average accuracy and total tokens across all questions in the search set, organized by model scale.
   • The per-question execution traces: JSONL files containing every decision the controller made on every question — which branches were spawned, probed, continued, pruned, abandoned; when the controller terminated; what the EMA history, confidence scores, and gate evaluations were at each step.

2. Agent analyzes failure modes. The prompt instructs the explorer to inspect the history and identify: what the prior controllers did well, where they failed, and what mechanisms would address those failures. For example, if a prior controller (IBC, from round 1) showed that instantaneous Beta-majority confidence gates caused premature stopping on transient confidence spikes, the agent might propose adding momentum (EMA smoothing) to the confidence signal. If a prior controller (SCR, from round 2) had asymmetric depth allocation but still used an instantaneous gate, the agent might combine the asymmetric allocation with momentum.

3. Agent proposes improved controller. The explorer directly edits the OptimalController class in the current round's copy of method.py (which is reset from a template at the start of each round, so no state carries over except what's in the agent's prompt). The controller must:

   • Subclass LLMDesignedMethod and implement solve(self, question) -> Optional[str].
   • Accept configuration through config={"beta": beta}.
   • Implement the MethodTraceRecorder interface for emitting execution traces.
   • Respect all design constraints from the prompt: adaptive width–depth allocation, budget-family coverage via β, monotonicity (larger β → larger budget), single-knob schedule (all hyperparameters are functions of β), novelty relative to seeds and prior proposals.

4. Evaluator sweeps β. The evaluator (eval.py) sweeps across a grid of β values (the paper doesn't specify the exact grid, but it covers a range from conservative/low-budget to aggressive/high-budget). For each β, the controller is instantiated with config={"beta": beta} and run on every question in $\mathcal{E}_{\text{search}}$. Each question is evaluated 64 times with different random subsets of the pre-collected trajectories, and results (accuracy, token cost) are averaged.

5. Results appended to history. The scalar results (CSV files per model per dataset) and execution traces (JSONL files per model per dataset per β) are saved to a new round directory and appended to $\mathcal{H}$.

6. Loop repeats. The explorer reads the updated history and proposes again. The total is 5 rounds.

Why 5 rounds: the paper notes in Section 3.3 that with 5 rounds, agents tended to overfit when allowed unrestricted hyperparameters (leading to the beta parameterization solution). Five rounds is a practical budget — enough for the agent to see the failures of initial proposals, iterate a few times, and converge to a reasonable controller, without being so many rounds that the search becomes computationally expensive or the agent starts overfitting by exploiting quirks of the search set.

Controller selection. After 5 rounds, the controller that achieved the highest accuracy on $\mathcal{E}_{\text{search}}$ (averaged across all β values? or at its best β? The paper says "the final controller is selected as the one achieving the highest accuracy on $\mathcal{E}_{\text{search}}$" — this is somewhat ambiguous, but in practice, the controller from the final round (round 5) is typically the best because each round builds on prior ones). This selected controller — with its code fixed — is then evaluated on the held-out sets (AIME25, HMMT25) at multiple β values (typically 0.5 and 1.0) to produce the generalization results in Table 1 and Figure 3.

The explorer's prompt (Appendix C). The prompt provided to Claude Code is extensive (~4 pages) and specifies:

• The environment API: question.probe_new(), question.probe_more(branch_index), question.get_new_branch_final_answer(), question.get_seq_and_total_tokens().
• Design constraints: must subclass LLMDesignedMethod, must accept beta through config, must implement trace recording, must be adaptive in width and depth (not just a fixed schedule), must support budget-family coverage (varying β traces a meaningful frontier), must have monotonic β behavior (larger β → more budget).
• Novelty requirements: must be meaningfully different from the seed algorithms (ASC, ESC, Parallel-Probe) and from prior proposals.
• Robustness requirements: single-knob schedule (all hyperparameters are functions of β), fewer hyperparameters is strictly better, avoid brittle thresholds.
• The exact interface for trace recording: _reset_trace(), _trace_step(event, goal, step_input, step_output, state, decision), get_last_trace(), solve_with_trace(question).

Why this round-based agent-driven approach: it leverages the explorer LLM's ability to read code, analyze execution traces, and synthesize new algorithms by combining mechanisms from prior attempts. A purely random search or grid search over controller hyperparameters would be infeasible because the space of possible controllers is a space of programs — it's combinatorial and discrete, not a continuous parameter space. The LLM acts as a learned prior over plausible controller designs, proposing candidate implementations that are syntactically valid and semantically coherent, which dramatically reduces the search space compared to random program generation. The history mechanism (showing prior proposals and their failure modes) enables the agent to learn from experience — it doesn't start from scratch each round, but builds on what was tried before.

Beta Parameterization for Tractable Search

The paper identifies a critical failure mode in preliminary experiments: without constraints on the search space, the explorer agent proposes controllers with up to 10 independent hyperparameters — pruning thresholds, confidence thresholds, stability windows, patience counters, branch caps, warm-up periods, burst sizes, etc. With only 5 discovery rounds, navigating this high-dimensional space causes the agent to collapse onto extreme solutions that happen to minimize token cost on the search set but fail to represent robust allocation strategies. For example, a controller with an overly aggressive pruning threshold (prune_thresh = 0.99) might prune almost all branches after 1–2 steps, achieving near-zero token cost on AIME24 while maintaining some accuracy because the search set's easy questions happen to have high-confidence early answers — but this strategy completely fails on any distribution where early answers are less reliable.

The solution is beta parameterization: a hard constraint that every controller must expose exactly one hyperparameter $\beta$ and derive all internal thresholds, counters, and budgets as deterministic functions of $\beta$. This collapses the search space from a high-dimensional space of independent knobs to a one-dimensional sweep.

Formal constraint. The controller must implement a method (conventionally _schedule(self, beta) -> dict) that maps $\beta \in [0, 1]$ to all internal hyperparameters. Given a single β value, the controller must be fully determined — there are no additional tunable knobs exposed to the evaluator. The config dict carries only beta (and at most a few fixed structural constants like max_branch = 64 that are shared across all β values and are not tuned between runs).

Monotonicity requirement. The mapping from $\beta$ to each hyperparameter must be monotonic in the direction that larger β corresponds to larger budget consumption. Specifically, the prompt states:

"As $\beta$ increases, the controller should become systematically more willing to widen exploration, deepen uncertain branches, defer termination, and in general spend more of the available budget."

This means:

• Parameters that increase budget use (warm-up rounds, patience counters, EMA window length, per-branch depth cap, burst sizes, number of initial branches) must be non-decreasing in β.
• Parameters that gate termination (confidence thresholds, stability requirements) must be non-decreasing in β — that is, harder to satisfy at high β, so the controller keeps running longer.
• Parameters that trigger widening (trend thresholds for confidence growth) must be non-increasing in β — at high β, even modest confidence gains don't suppress widening, leading to more exploration.

The monotonicity constraint serves two purposes: (1) it makes the β → behavior mapping interpretable ("higher β = more budget"), which helps the explorer reason about the controller family's coverage; and (2) it prevents pathological schedules where, say, β = 0.3 is aggressive and β = 0.7 is conservative (which would make the β sweep meaningless for tracing a frontier).

Conservative anchor at β = 1. The prompt includes a "conservative-anchor requirement": when $\beta = 1$, the controller should be conservative in its termination decisions — reluctant to stop based on shallow consensus, avoiding aggressive early pruning, and willing to approach full use of the 64-branch budget if the state remains ambiguous. This anchors the β = 1 endpoint near the "accuracy-first" regime of the Pareto frontier. Lower β values should achieve lower token consumption mainly through more selective width/depth allocation (e.g., fewer initial branches, stricter pruning), not through brittle premature stopping rules.

Coverage requirement. By sweeping β from 0 to 1, it should be possible to trace a meaningful accuracy–cost frontier from low-budget regimes (β near 0, spending perhaps 10–20% of the max budget) up to near-full use of the shared max_branch = 64 ceiling (β = 1, spending 80–100% of the budget). This ensures the discovered controller family can be deployed at different budget levels by simply varying β, without needing to re-discover for each budget point.

Why this works: beta parameterization is effectively a regularization on the search space. Instead of the agent searching over 10 independent hyperparameters and potentially overfitting to the search set by finding a sharp optimum, the agent must commit to a family of controllers related by a smooth, monotonic schedule. This forces the agent to think in terms of "what is the right mechanism?" rather than "what is the right threshold value?" — because the threshold values for any given β are determined by the schedule, and the schedule must work across the full β range, not just at the β that happens to optimize search-set accuracy.

What the agent designs, vs. what is fixed. The agent designs:

• The structure of the controller logic: what mechanisms it uses (EMA momentum, confidence-trend widening, priority-based depth allocation, etc.).
• The functional form of each hyperparameter's dependence on β — e.g., conf_thresh = 0.85 + 0.12 * β, ema_alpha = 0.70 - 0.40 * β, n_init = max(2, round(2 + 6 * β)).

The agent does not design:

• The fixed structural constants: max_branch = 64, max_outer = 500 (hard cap on loop iterations).
• The evaluator's β sweep grid — that's fixed by the framework.
• The data collection protocol (128 trajectories, temperature 0.7, 500-token intervals) — that's fixed before discovery starts.

The specific schedule functions in the discovered CMC (Appendix D) use simple analytic forms — mostly linear ramps with clipping:

n_init           = max(2, round(2 + 6 * β))        # 2 → 8 branches
max_branch_use   = min(64, round(4 + 60 * β))       # 4 → 64 branches
warm_up          = max(2, round(2 + 8 * β))         # 2 → 10 rounds
abandon_patience = max(3, round(3 + 9 * β))         # 3 → 12 rounds
T_ema            = max(2, round(2 + 6 * β))         # 2 → 8 window length
ema_alpha        = 0.70 - 0.40 * β                  # 0.70 → 0.30 (NON-INCREASING)
conf_thresh      = 0.85 + 0.12 * β                  # 0.85 → 0.97
delta_slack      = 0.04 - 0.03 * β                  # 0.04 → 0.01 (NON-INCREASING)
burst_aligned    = max(1, round(1 + 2 * β))         # 1 → 3
widen_burst      = max(1, round(1 + 3 * β))         # 1 → 4
trend_thresh     = 0.04 - 0.03 * β                  # 0.04 → 0.01 (NON-INCREASING)
min_complete     = max(2, round(2 + 3 * β))         # 2 → 5

Each of these is explicitly monotonic in the required direction. The agent selected these forms (linear ramps with simple coefficients like 0.40, 0.12, 0.03) rather than more complex sigmoids or piecewise functions, likely because simpler schedules are more robust and less likely to overfit to the search set. The prompt explicitly encourages this:

"prefer a small number of simple analytic forms (linear, power, sigmoid, clipped ramps) with a handful of fixed structural constants"

The ablation study validates this design (Table 3). Removing beta parameterization (the "w/o Beta Parameterization" row) leads to controllers with excessive free hyperparameters that overfit dramatically: accuracy drops from 53.1 to 49.0 on held-out benchmarks, while token consumption plummets from 575.5K to 93.3K — the controller discovered overly aggressive pruning thresholds that "worked" on AIME24 but fail to generalize. This is the smoking gun for the overfitting hypothesis: without the beta constraint, the agent finds sharp, search-set-specific optima that don't represent robust allocation strategies. Beta parameterization forces the agent to design mechanisms that work across a range of budgets, which inherently selects for more generalizable strategies.

Execution Trace Feedback for Diagnostic Discovery

Scalar outcomes — "the controller achieved 65.2% accuracy with 483.4K tokens at β = 1.0 on AIME24" — tell the explorer whether a controller is good or bad, but reveal almost nothing about why. Without understanding why a controller fails, the explorer can only propose random variations, which is unlikely to improve performance within a limited number of rounds (5, in this paper). The solution is to augment the history with per-question execution traces that log every decision the controller made, enabling the explorer to diagnose specific failure modes and propose targeted fixes.

What execution traces contain. For each question and each β value, the evaluator records a JSONL file where each line corresponds to a decision step in the controller's execution. The trace interface (enforced by MethodTraceRecorder) requires the controller to emit events at key decision points:

• start: initialization, records all hyperparameter values derived from β.
• init_branches: the initial batch of branches spawned (via probe_new), how many completed immediately.
• forward: the main per-round event, recording the current outer step, pool statistics (winner, confidence), EMA state (ema_conf, ema_delta), number of active/completed branches, how many were probed this round, which branches were abandoned, and what the controller decided to do next.
• terminate_check: the evaluation of the stopping gate — what the relevant thresholds are (conf_thresh, delta_slack, min_complete), what the current EMA values are, whether the gate is eligible and whether it fires.
• update_states: when widening occurs — what triggered it (ema_delta vs. trend_thresh), how many new branches were spawned, total spawned count.
• prune (implicit in forward): which branches were abandoned due to persistent deviance.
• finish: the terminal event, recording the final answer, stop reason (which gate fired, or loop exhausted), and final EMA/pool statistics.

What these traces enable the agent to do:

1. Identify premature stopping. If the trace shows that a controller stopped at round 5 with pool_conf = 0.92 and ema_conf = 0.88 because gate_fires = True, but the final answer was wrong, the agent can diagnose: the instantaneous confidence was high, but the underlying distribution hadn't stabilized yet. This motivated the shift from instantaneous confidence gates (all prior proposals: IBC, SCR, DGCC) to momentum-based EMA gates (CMC), where stopping requires both high EMA level and non-declining momentum — a one-round spike in confidence won't fire the gate.

2. Identify over-aggressive pruning. If the trace shows branches being abandoned after disagree_rounds = 3 and those branches would have converged to the correct answer if given more depth, the agent can diagnose: the pruning patience is too low. This motivated the abandon_patience schedule (3 to 12 rounds depending on β) and the safe-minimum of 2 active branches (never prune below 2 alive).

3. Identify inefficient depth allocation. If the trace shows that the controller uniformly deepened all active branches (1 step each per round), but many of those branches were deviant and eventually abandoned, the agent can diagnose: computation is being wasted on branches that should be pruned earlier or deepened less aggressively. This motivated the priority-queue-based depth allocation in CMC, where aligned branches get burst_aligned extra probe steps per round while deviant branches still get 1 step (so they're not completely starved, but aligned branches receive proportionally more investment).

4. Identify missed widening opportunities. If the trace shows the controller never spawned new branches after the initial batch, but the EMA confidence stagnated (delta ≈ 0) and the final answer was wrong (because none of the initial branches had the correct answer), the agent can diagnose: the controller should have widened to explore alternative reasoning paths when depth alone wasn't producing confidence gains. This motivated the confidence-trend widening mechanism: if ema_delta <= trend_thresh (confidence isn't growing), trigger widening.

The contrast with scalar-only feedback. Without execution traces, the explorer sees only: "Controller A: 65% accuracy, 480K tokens. Controller B: 64% accuracy, 700K tokens." From this, the agent knows B is worse, but it has no idea why — did B over-invest in depth on wrong branches? Did B fail to prune? Did B stop too early? Did B widen too aggressively? Any fix would be a shot in the dark. With traces, the agent sees: "Controller B ran for 25 rounds, maintaining 32 active branches (many deviant) until the budget exhausted, never triggering the stop gate because conf_thresh = 0.97 was too strict." The fix is targeted: lower the confidence threshold, or add more aggressive pruning, or both.

The ablation study validates this design (Table 3). Removing execution traces (the "w/o Execution Traces" row) degrades performance: held-out accuracy drops from 53.1 to 51.6, and token consumption increases from 575.5K to 824.3K — the discovered controller is both less accurate and less efficient. This suggests that without traces, the agent cannot learn effective allocation strategies; it essentially guesses at mechanisms and tunes them based on scalar outcomes, which produces controllers that are either too conservative (spending too many tokens on unproductive branches) or too aggressive (stopping prematurely), or both in different parts of the state space.

Why this is consistent with prior work. The paper cites Meta-Harness (Lee et al., 2026) as demonstrating that "fine-grained execution feedback improves agentic discovery for harness engineering." AutoTTS applies the same principle to a different domain (test-time scaling rather than evaluation harness design), showing that the insight transfers: in any domain where a controller interacts with an environment through a sequence of decisions, scalar outcomes lose information about which decisions were good or bad, and execution traces recover that information.

Trace payload size discipline. The prompt explicitly constrains trace payloads to be "small and JSON-serializable (primitive types, short lists); avoid dumping full branch histories at every step." This prevents trace files from becoming unwieldy (hundreds of megabytes for 128 questions × 64 branches × 50 rounds) and ensures the explorer can process them within its context window. The traces record summaries (counts, EMA values, confidence scores, decisions) rather than the full text of every branch's prefixes and probe answers.

The Discovered Controller: Confidence Momentum Controller (CMC)

The output of one discovery run (5 rounds with Claude Code on the AIME24 search set across all four Qwen3 models) is the Confidence Momentum Controller (CMC) , documented in Appendix D. This controller embodies four non-obvious mechanisms that interact in a coordinated way — mechanisms that would be extremely difficult to arrive at through manual design because they involve coupled feedback loops between width and depth decisions mediated by a momentum signal.

Mechanism 1: Trend-based stopping via EMA momentum. Unlike all seed baselines and prior discovery-round proposals, which gate termination on instantaneous pool confidence (the Beta-majority confidence computed from the currently completed answers), CMC maintains an exponential moving average (EMA) of pool confidence over the last T_ema rounds:

$\text{ema\_conf}_t = (1 - \alpha) \cdot \text{ema\_conf}_{t-1} + \alpha \cdot \text{pool\_conf}_t$

where $\alpha = \text{ema\_alpha}$ is the blending factor (lower α = more smoothing/inertia). The EMA delta tracks recent trend:

$\text{ema\_delta}_t = \text{ema\_conf}_t - \text{ema\_conf}_{t - T_{\text{ema}}}$

What the stopping gate evaluates: termination requires ALL of:

1. warm_enough — the controller has run at least warm_up rounds (prevents stopping before any meaningful evidence is gathered).
2. n_complete >= min_complete — at least min_complete branches have reached their final answer (prevents stopping based on very few data points).
3. ema_conf >= conf_thresh — the smoothed confidence is above the threshold (level requirement).
4. ema_delta >= -delta_slack — the trend is NOT significantly negative; the EMA is at least flat or rising (momentum requirement).

Why this form: condition (3) alone — checking instantaneous confidence — can fire on a single lucky round where several branches coincidentally agree on the same (potentially wrong) answer before the distribution stabilizes. This is the failure mode observed in all prior proposals (IBC, SCR, DGCC). Condition (4) adds a trend requirement: even if the current EMA is high, if it's actively declining (because recent rounds are producing more disagreement), the gate does not fire — the controller keeps running to see if the trend stabilizes or reverses. The delta_slack parameter provides a small tolerance: the EMA can decline very slightly without blocking termination, preventing infinite loops from numerical noise.

Mechanism 2: Coupled width–depth control through EMA delta. The confidence trend (ema_delta) serves double duty: it gates stopping (as above) AND controls widening. The widening decision is:

$\text{want\_widen} = \text{can\_widen} \land (\text{ema\_delta} \leq \text{trend\_thresh}) \land (\text{outer\_step} \geq \max(1, \text{warm\_up} // 2)) \land (\text{ema\_conf} < \text{conf\_thresh})$

If want_widen is true, the controller spawns widen_burst new branches (up to max_branch_use total). The logic: if confidence is growing (ema_delta > trend_thresh), depth alone is producing progress — no need to widen. If confidence is stagnant or declining (ema_delta ≤ trend_thresh), the current set of branches isn't converging on a confident answer, so the controller widens to explore alternative reasoning paths. The additional constraint ema_conf < conf_thresh prevents unnecessary widening when the controller is already near the stop threshold (widening at that point would just add noise).

What this creates: a closed feedback loop where the same signal (EMA delta) controls two decisions in opposite directions:

• Positive delta → suppress widening, continue deepening (exploit current branches).
• Negative/zero delta → trigger widening (explore new branches), and if the decline persists, continue running (stop gate won't fire due to condition 4).

This coupling is absent in all hand-crafted baselines — ASC and ESC make stopping decisions based on consensus but never use the consensus trend to control branch spawning; Parallel-Probe has fixed cohort size and never widens; ST-BON has a fixed expand-then-prune pattern. The coupling emerges naturally from the agent-driven discovery because the agent can observe in traces that when confidence is growing, widening is wasteful (it adds branches that will duplicate the already-converging answer), and when confidence is stagnant, deepening alone is insufficient (the existing branches don't contain the correct answer).

Mechanism 3: Alignment-aware depth allocation with priority scheduling. Each round, the controller allocates probe steps (generation intervals) across active unfinished branches using a priority queue sorted by probe_count descending — branches that have received the most investment get served first. Each branch then gets a multiplier based on its classification:

• Aligned (latest answer matches the current pool winner): receives burst_aligned probe steps (1 to 3, depending on β) — these branches are on-track and worth investing in.
• Neutral (no pool winner yet, or warm-up not reached): receives 1 probe step.
• Deviant (latest answer differs from pool winner): receives 1 probe step but accumulates disagree_rounds.

What this achieves: it concentrates computation on branches that agree with the emerging consensus (because they're most likely to be correct), while still giving at least minimal attention to all active branches. This is more sophisticated than:

• Uniform allocation (all branches get 1 step each per round, as in prior proposals) — wastes computation on deviant branches.
• Purely aligned-only allocation (only probe aligned branches) — risks starving a branch that initially deviates but would converge to the correct answer if given more depth (a wrong intermediate answer might later be corrected).
• Lazy sleeping (DGCC's approach of "locking" aligned branches after they reach a confidence threshold) — misses the opportunity to further strengthen aligned branches and increase confidence.

The priority-queue aspect (serving most-invested branches first) ensures that branches closest to completion (which have the most informative probe answers) are advanced first, maximizing the information gained per unit of computation.

Mechanism 4: Conservative branch abandonment. A branch is abandoned (pruned) only after:

• It has been classified as "deviant" for abandon_patience consecutive rounds (3 to 12 rounds, depending on β).
• After abandonment, at least 2 branches remain alive (the controller never prunes below 2 active branches, unless branches naturally finish).

What this achieves: the consecutive-rounds requirement prevents abandoning a branch that temporarily disagrees with the pool winner due to an intermediate answer that will be corrected with more depth. The safe-minimum of 2 active branches prevents the pathological case where all but one branch are pruned, and that remaining branch is also deviant (just not yet for enough rounds) — the controller would be forced to follow a single potentially wrong branch. By keeping at least 2 active, there's always a comparison point.

Why these mechanisms are "non-obvious" in combination: the key insight is that these four mechanisms are not independent knobs — they interact through the shared EMA signal. Specifically:

1. The EMA delta controls both when to stop (gate condition 4) and when to widen (widening trigger).
2. Widening adds new branches, which (once they complete) add answers to the completed pool, which affects pool confidence, which feeds into the EMA, which affects the delta, which affects widening... creating a feedback loop.
3. Alignment-aware depth allocation concentrates computation on branches that agree with the pool winner, which may increase confidence faster, which may trigger the stop gate earlier — but if those aligned branches were wrong (the pool winner is incorrect), the EMA will eventually decline as more branches complete and disagree, preventing the stop gate from firing and triggering widening instead.
4. Conservative abandonment removes persistently deviant branches, which cleans up the completed pool (removing "noise" answers), which may increase confidence — but if too many branches are abandoned, the pool becomes artificially pure and the stop gate fires prematurely. The safe-minimum of 2 prevents this.

The coordinated complexity of these interactions — a momentum signal that simultaneously gates stopping, triggers widening, and influences which branches receive depth investment — would be extremely difficult to design by hand because human designers tend to decompose problems into independent modules (separate stopping logic, separate widening logic, separate depth allocation logic). The agent-driven discovery, operating over code and receiving fine-grained execution traces, can discover the benefits of coupling these decisions through a shared signal, because it can observe in traces when decoupled decisions lead to incoherent behavior (e.g., stopping due to high instantaneous confidence while simultaneously wanting to widen due to stagnant trend — a contradiction that CMC's coupling resolves by using the same EMA for both decisions).

The evolution trajectory (Figure 4). The discovery process didn't arrive at CMC in one step. Figure 4 shows the accuracy–cost trajectory over rounds:

• Round 1 (t1): The initial controller was overly aggressive in reducing token usage, leading to relatively low accuracy (the agent prioritized the cost reduction objective too heavily).
• Round 2 (t2): After observing this accuracy degradation, the explorer increased the computation budget (higher conf_thresh, more branches), substantially recovering accuracy. This moved the point rightward and upward on the cost–accuracy plane.
• Round 3 (t3): Fine-grained efficiency adjustment — pushed cost down slightly with small accuracy loss (efficiency improvement).
• Round 4 (t4): Accuracy push — allocated more budget to recover the small accuracy loss from t3 and push beyond t2.
• Round 5 (t5): Further accuracy push — achieved the best accuracy–cost point on the search set.

The trajectory alternates between efficiency-oriented adjustments and accuracy-oriented recovery, gradually moving toward a better Pareto frontier. This is visible in Figure 4's zoomed inset (t2–t5), labeled "t2→t3: efficiency", "t3→t4: push acc", "t4→t5: further push acc." The final controller generalizes well to held-out benchmarks (right panel), suggesting the improvements are not overfitting to AIME24.

Design choices in CMC vs. rejected alternatives. The agent tried and rejected several mechanisms visible in prior proposals:

• Instantaneous gates (IBC, SCR, DGCC): Rejected because traces showed they fire prematurely on confidence spikes.
• Uniform depth allocation (IBC): Rejected because traces showed computation wasted on deviant branches that were eventually abandoned.
• Asymmetric burst without priority scheduling (SCR): Improved upon by adding the priority queue (most-invested first) and maintaining minimal attention to deviant branches rather than purely aligned-burst.
• Dual-gate soft corroboration (DGCC): Replaced by single EMA gate, which is conceptually simpler (one signal, not two) and prevents both transient spikes (through smoothing) and declining trends (through delta check).
• Vote-gap proportional widening (DGCC): Replaced by EMA-trend widening, which couples the widening decision to the same signal that controls stopping, creating a coherent controller rather than two independently-triggered mechanisms.

The beta schedule functions (listed earlier) were selected through sandbox experimentation — the agent tested alternative forms (sigmoids, piecewise linear, different coefficient ranges) and selected the linear forms with the specific coefficients (0.40, 0.12, 0.03, etc.) that achieved the most robust frontier. The paper notes: "prototype alternative schedule forms, plot the induced accuracy-cost frontier across the evaluated β grid against the seed baselines, and keep the simplest schedule that achieves a robust frontier." The committed schedule is the simplest form that works — linear ramps with a handful of fixed coefficients — which aligns with the robustness requirement in the prompt ("a slightly weaker controller with fewer hyperparameters is preferred over a slightly stronger one that needs careful multi-knob tuning").

Summary of Design Choices and Their Justifications

• Offline replay environment over online evaluation: moves all LLM calls to a one-time pre-computation phase, making controller evaluation essentially free during discovery. This is the linchpin that makes the entire framework economically feasible.

• Width–depth MDP formulation as the shared control abstraction: unifies all TTS strategies — hand-crafted and discovered — into a common language of states, actions, and costs, enabling systematic comparison and automated search.

• Agent-driven discovery (Claude Code) over random search or grid search: the space of possible controllers is a space of programs, which is combinatorial and discrete. The explorer LLM acts as a learned prior over plausible designs, proposing syntactically valid and semantically coherent controllers, dramatically reducing the effective search space.

• Five discovery rounds rather than more: enough for the agent to iterate on failures, not so many that overfitting to the search set becomes severe or costs become excessive.

• Beta parameterization as search space regularization: collapses 10+ independent hyperparameters into a single β knob with monotonic schedules, preventing the agent from discovering sharp, search-set-specific optima that fail to generalize. Validated by the ablation showing dramatic overfitting (accuracy drop + token collapse) when removed.

• Execution trace feedback over scalar-only outcomes: enables the agent to diagnose why controllers fail — premature stopping, over-aggressive pruning, inefficient depth allocation, missed widening opportunities — and propose targeted fixes. Validated by the ablation showing degraded performance when traces are removed.

• Multi-model search set (AIME24 across all four Qwen3 scales): forces the discovered controller to work across model capacities, preventing overfitting to a single model's error patterns.

• Single-knob schedule with simple analytic forms (linear ramps): keeps the β → hyperparameter mapping interpretable, smooth, and monotonic, avoiding the complexity and brittleness of piecewise functions with many hand-placed breakpoints.

• Trace interface alignment with seeds (MethodTraceRecorder, solve_with_trace): ensures the discovered controller emits diagnostic traces in the same format as the hand-crafted baselines, enabling side-by-side comparison and enabling the explorer to learn from seed traces as well as prior proposals.

4. Key Insights and Innovations

Innovation 1: Reframing TTS Strategy Design from Manual Engineering to Environment Construction

The paper's most fundamental conceptual move is not automating the design of test-time scaling strategies — it is changing what humans design at all. Before AutoTTS, the research workflow was: hypothesize a heuristic (e.g., "stop sampling when the most common answer exceeds 95% confidence"), implement it, tune its thresholds on a validation set, and evaluate against baselines. Each new strategy was a one-off artifact, and the design process was a direct mapping from researcher intuition to algorithm. AutoTTS reframes this entirely: the human role shifts from designing strategies to designing environments in which strategies can be systematically discovered. The human now defines the state space, action space, cost model, and feedback signals — the "rules of the game" — and delegates the search over controller designs to an agent operating within that structured space.

This is not merely an automation argument ("let an LLM do the trial-and-error"). It is a claim about where human cognitive effort provides the most leverage. Humans excel at understanding problem structure: identifying that width–depth allocation is the right abstraction, recognizing that partial observability of intermediate answers creates an information-gathering tension, and designing constraints (beta parameterization, coverage requirements) that channel search toward robust solutions. Humans are poor at exploring large combinatorial spaces of sequential decision rules, where the consequences of early choices propagate through the inference budget in non-obvious ways and where coordinated mechanisms (like coupling the stopping signal to the widening signal) are difficult to conceive through intuition alone. The division of labor — human structures the space, agent searches within it — is the core intellectual contribution.

This reframing connects to a broader intellectual shift visible across AI research. Neural architecture search (Zoph and Le, 2016) moved human effort from designing individual architectures to designing search spaces and training protocols. Program synthesis moved human effort from writing programs to writing specifications. FunSearch (Romera-Paredes et al., 2024) and AlphaEvolve (Novikov et al., 2025) demonstrated that LLM-driven program search can discover novel algorithms in mathematics and combinatorial optimization. AutoTTS applies the same logic to test-time scaling, but with a crucial additional challenge that prior discovery work did not face: the evaluation cost bottleneck. In FunSearch, evaluating a candidate scoring function on a mathematical problem costs milliseconds. In AutoTTS, evaluating a candidate controller online would require invoking the base LLM to generate reasoning trajectories on demand, costing millions of tokens per evaluation. The paper's insight that this bottleneck can be circumvented by constructing an offline replay environment — moving all LLM calls to a one-time pre-computation phase and making controller evaluation a cheap, deterministic replay — is what makes the reframing practically viable rather than theoretically interesting but economically infeasible.

Evidence for the significance of this reframing is indirect but compelling: the discovered Confidence Momentum Controller (CMC) incorporates four non-obvious mechanisms that interact through a shared feedback signal (EMA momentum simultaneously gates stopping and triggers widening), a level of coordinated complexity that would be extremely difficult to arrive at through manual design. The fact that a five-round discovery loop costing $39.9 produced a controller that improves the accuracy–cost Pareto frontier over strong hand-crafted baselines designed by human experts over months of research is the empirical validation that environment-driven discovery is not just an alternative workflow — it can produce better results than manual design, because it explores regions of the controller space that human intuition does not reach.

Innovation 2: The Width–Depth MDP as a Unifying Abstraction That Collapses the TTS Literature

The paper introduces a formal Markov Decision Process over a 2D width–depth allocation space that serves as a common language for expressing and comparing all test-time scaling strategies. This abstraction (Section 2) reveals that apparently disparate methods — Self-Consistency@64, ASC, ESC, Answer Consistency, ST-BON, Parallel-Probe — are not fundamentally different types of algorithms but rather different manually-specified trajectories through the same underlying control space, as visualized in Figure 2. Self-Consistency occupies a fixed corner (max width, max depth, no adaptivity); ASC and ESC adapt width only (stopping early when consensus emerges but never selectively deepening); Answer Consistency adapts depth only (following a single chain); Parallel-Probe adapts both width and depth but follows a predetermined structural template. Each is a special case of a controller that could, in principle, make any sequence of BRANCH, CONTINUE, PROBE, PRUNE, and ANSWER decisions.

This is more than a taxonomic contribution. By formalizing TTS as a controller synthesis problem in a partially observable MDP, the paper makes visible the vast regions of the allocation space that hand-crafted methods leave unexplored and frames the discovery problem in a way that is amenable to systematic search. Prior work treated TTS strategy design as an ad-hoc, method-by-method activity. The MDP abstraction reveals it as an instance of a well-studied class of problems — sequential decision-making under uncertainty with a budget constraint — and connects it to the broader literature on MDPs, reinforcement learning, and program synthesis. This reframing is what enables the discovery loop: once controllers are cast as policies in this MDP, any code-defined policy can be evaluated in the same replay environment, regardless of whether it was hand-crafted or discovered, and the discovery agent can search over this space of policies using standard iterative refinement.

The significance of this abstraction is that it enables cumulative progress. Prior work on TTS was fragmented: each new method was evaluated against its own chosen baselines on its own chosen benchmarks, and the design principles that generalized across methods remained implicit. The width–depth MDP provides a substrate in which all methods can be compared on equal footing — they are evaluated in the same replay environment, on the same questions, with the same cost model. Future work can build on this substrate: researchers can construct richer environments (e.g., adding tree-search actions, verifier-guided refinement, or revision-based mechanisms to the action space) and deploy the same discovery framework to find controllers in those expanded spaces. The MDP is therefore not just a paper-specific tool but a shared infrastructure for TTS research comparable to what the Chinchilla scaling laws (Hoffmann et al., 2022) provided for pretraining — a formal framework that organizes existing knowledge and guides future exploration.

Innovation 3: Beta Parameterization as a Search-Space Regularization Technique That Prevents Overfitting in Algorithm Discovery

The paper identifies a subtle but critical failure mode in program-space search: when an explorer agent can tune many independent hyperparameters, it discovers sharp, search-set-specific optima that fail to generalize to held-out distributions. This is not unique to TTS — it is a general phenomenon in automated algorithm design — but the paper provides a concrete solution (beta parameterization) and an ablation study (Table 3) that quantifies the severity of the problem and the effectiveness of the fix.

The diagnostic insight is that program-space search with unrestricted hyperparameters is analogous to high-capacity model training without regularization: the agent can overfit to the search set by finding precise combinations of thresholds (pruning patience, confidence gates, window sizes) that happen to minimize cost on the specific questions and model trajectories in $\mathcal{E}_{\text{search}}$, without learning mechanisms that generalize. The solution — collapsing all hyperparameters into deterministic, monotonic functions of a single scalar $\beta$ — is a form of structural regularization on the search space. It forces the agent to design mechanisms rather than thresholds, because the threshold values for any given $\beta$ are determined by the schedule, and the schedule must produce sensible behavior across the full $\beta$ range, not just at a single operating point.

What makes this distinctive from standard hyperparameter management advice ("use fewer knobs") is the monotonicity and coverage requirements. The beta schedule must be monotonic (larger $\beta$ → larger budget) and must trace a meaningful accuracy–cost frontier from low-budget to near-full-budget regimes. This transforms the search from "find a single good controller" to "find a coherent controller family that behaves sensibly across budgets." The family-based objective inherently penalizes overfitting because a sharp optimum at one $\beta$ value that produces degenerate behavior at other $\beta$ values will perform poorly when evaluated across the full sweep.

The ablation study (Table 3) provides stark evidence: removing beta parameterization causes held-out accuracy to drop from 53.1 to 49.0 (a 4.1 percentage point loss) while token consumption plummets from 575.5K to 93.3K (an 83.8% reduction). This dramatic token collapse — rather than just accuracy degradation — is the signature of overfitting: the agent discovered aggressive pruning and stopping thresholds that "worked" on the search set but are far too aggressive for general use. The fact that the regularized search (with beta parameterization) produces controllers that both are more accurate and use more tokens (in a sensible, adaptive way) demonstrates that the regularization doesn't just prevent a particular failure mode — it fundamentally redirects the search toward robust, generalizable mechanisms.

This insight has implications beyond TTS. Any domain where LLM-driven program search can tune many hyperparameters (autoML pipelines, agent design, evaluation harnesses) faces the same overfitting risk. Beta parameterization demonstrates a general design pattern: constrain the search space to families parameterized by a single trade-off knob, and require the family to cover a meaningful range of operating points. This shifts the search objective from point-wise optimization to frontier optimization, which is inherently more robust to distribution shift.

Innovation 4: Execution Traces as a Diagnostic Medium That Enables Targeted Iteration in Program-Space Search

The paper identifies that scalar feedback — accuracy and token cost — is information-poor for program-space search, because it tells the explorer agent that a controller failed but not why. The solution is to augment the discovery loop's history with per-question execution traces that log every decision the controller made: which branches were probed, continued, pruned, or abandoned; what the EMA, confidence scores, and gate evaluations were at each step; what triggered widening or stopping. This turns the discovery process from blind trial-and-error (propose a controller, see scalar outcome, guess at what to change) into diagnosis-driven iteration (propose a controller, inspect its decision traces to understand which decisions were good or bad, propose targeted fixes).

This is not the first work to use execution traces in automated discovery — the paper cites Meta-Harness (Lee et al., 2026), which used full execution histories for harness engineering — but AutoTTS demonstrates this principle in a domain where the control decisions are sequential and interdependent, making diagnosis particularly challenging. In harness engineering, the decisions are relatively independent (e.g., which validation checks to include). In width–depth TTS control, decisions compound: early pruning decisions affect which branches are available for depth allocation later; early widening decisions determine how many branches compete for the remaining budget; the interaction between stopping, widening, and depth allocation is mediated by a shared confidence signal. Diagnosing failures in such a system without execution traces would require essentially simulating the controller's decision process from scratch to infer what went wrong — which is what the traces provide directly.

The ablation study (Table 3) quantifies the value of traces: removing them degrades held-out accuracy from 53.1 to 51.6 while increasing token consumption from 575.5K to 824.3K. The controller discovered without traces is both less accurate and less efficient — a pattern suggesting that the agent, lacking diagnostic information, proposed controllers that were poorly calibrated (sometimes too aggressive, sometimes too conservative, but never quite right) because it couldn't identify which specific mechanisms needed adjustment. With traces, the agent could identify particular failure modes — premature stopping on transient confidence spikes (leading to the EMA momentum gate), over-aggressive pruning of branches that would have converged (leading to the abandon_patience schedule and the 2-branch safe minimum), wasted computation on deviant branches (leading to alignment-aware depth allocation) — and propose targeted fixes that compound into substantial improvements.

This finding has a broader methodological implication for the field of LLM-driven algorithm discovery. As the complexity of the artifacts being discovered increases — from simple scoring functions to sequential decision-making controllers to multi-agent systems — the bandwidth of the feedback channel becomes a bottleneck. Scalar rewards (accuracy, cost, F1) provide perhaps a few bits of information per evaluation (better/worse, and roughly how much). Execution traces provide orders of magnitude more information (which decisions mattered, in which contexts, with what consequences). The paper demonstrates that investing in rich feedback — even if it costs more to record and process — pays off in sample efficiency: with traces, 5 rounds of discovery suffice to find a strong controller; without traces, 5 rounds produce a substantially worse one, and many more rounds would be needed to stumble upon good mechanisms through scalar-guided random variation.

Assessment of These Innovations: Incremental vs. Fundamental

Environment-driven discovery (Innovation 1) is a fundamental reframing of the TTS research paradigm. It changes what the field produces — from individually designed strategies to reusable discovery environments — and opens a new research direction (environment design) that was previously invisible because the dominant paradigm assumed strategies must be hand-crafted.

The width–depth MDP (Innovation 2) is an important synthesis rather than a fundamental theoretical advance. MDP formalisms are well-established in RL, and the specific formalization (states, actions, costs) follows standard conventions. The contribution is applying this lens to TTS specifically and demonstrating that it unifies the fragmented literature, not inventing the formal machinery.

Beta parameterization (Innovation 3) is a practical insight with general applicability. The core idea — regularizing program-space search by constraining to a single-knob family — is simple, but the paper provides the first clear empirical demonstration (via ablation) that unrestricted hyperparameter search leads to catastrophic overfitting in LLM-driven algorithm discovery. This makes it more than a "use fewer hyperparameters" recommendation; it quantifies the risk and provides a specific, validated design pattern.

Execution trace feedback (Innovation 4) is an engineering contribution that validates a broader principle (rich feedback improves sample efficiency in program search) in a particularly challenging domain. The principle is not new — Meta-Harness demonstrated it first — but AutoTTS shows it transfers to sequential decision-making domains where the feedback structure is more complex, and the ablation provides clean evidence that the principle holds even when the underlying search agent (Claude Code) and domain (TTS control) differ substantially from the prior demonstration.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All discovery and evaluation use mathematical reasoning benchmarks. The search set is AIME24 (30 competition-level math problems from the 2024 American Invitational Mathematics Examination). The held-out evaluation sets are AIME25 (the 2025 AIME, also 30 problems) and HMMT25 (the 2025 Harvard-MIT Mathematics Tournament, with problems covering algebra, geometry, combinatorics, and number theory — exact problem count not specified). These are standard benchmarks in the TTS literature that require multi-step symbolic reasoning. The paper also evaluates on GPQA-Diamond (a graduate-level science QA benchmark) for its non-math generalization test (Section 5.3), though this is not part of the main experimental protocol.

• Base model(s). The primary experiments use four models from the Qwen3 family (Yang et al., 2025): Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, and Qwen3-8B. This spans nearly two orders of magnitude in parameter count (0.6B to 8B) and tests whether discovered controllers transfer across model scales without modification. The authors argue these models are "representative of the capabilities of many contemporary LLMs" and sit in a useful regime where performance on MATH-style problems is non-trivial but far from saturated, leaving room for test-time compute strategies to make a meaningful difference. For the generalization experiment (Section 5.3), the paper additionally evaluates on DeepSeek-R1-Distill-Llama-8B, a Llama-based model distilled from DeepSeek-R1 (Guo et al., 2025), to test transfer to a different model family with a different pretraining and distillation lineage.

• Metrics. The paper reports two metrics for every controller evaluation:

  • Accuracy (%): The fraction of questions for which the controller's final aggregated answer matches the ground truth. Answers are compared using exact match after standard parsing (the exact grading protocol is not detailed, but it follows the MATH benchmark convention of extracting the final answer string and checking equivalence).
  • Total Tokens: The total number of tokens consumed across all branches used by the controller, summed over all questions in the benchmark. This measures the computation cost of running the strategy. Since each generation interval has a fixed token length (Δ = 500 tokens), total tokens = (total number of generation intervals across all branches ever instantiated) × 500, plus any probe-reading cost (κ_probe ≈ 0 in practice). The paper does not report tokens per question or variance estimates (standard deviations or confidence intervals) for either metric.

• Baselines. The paper compares the discovered controller against four hand-crafted test-time scaling methods, all of which are implemented in the same codebase and evaluated in the same replay environment for fair comparison:

  • Self-Consistency (SC@64) (Wang et al., 2022): Sample 64 complete reasoning trajectories from the base LLM and perform majority voting over their final answers. This is the simplest and most widely used TTS baseline — no adaptivity, every question gets identical treatment.
  • Adaptive Consistency (ASC) (Aggarwal et al., 2023): Sample trajectories one-by-one, stopping early when the Beta-majority confidence (a statistical measure of whether the most common answer has enough support relative to the runner-up) exceeds a pre-defined threshold. The paper uses the original threshold of 0.95. This adapts only along the width axis — it never selectively deepens individual branches, and it uses full-read branches (complete trajectories) rather than incremental probing.
  • Early-Stopping Consistency (ESC) (Li et al., 2024): A chunk-based approach that generates trajectories in parallel batches (chunk size 8) and terminates early when answer stability is detected within a sliding window. Like ASC, this adapts width only via early stopping on consensus, using full-read branches.
  • Parallel-Probe (Zheng et al., 2026): A more sophisticated method that uses incremental probing: start with a fixed cohort of parallel chains, probe intermediate answers at each depth interval, classify branches as "on-track" or "off-track" based on whether their intermediate answer matches the current majority, prune off-track branches, and terminate when a stable majority emerges. This is the strongest hand-crafted baseline and the one that most closely resembles the adaptive width–depth control that the discovered controller aims to improve upon.

  All baselines are evaluated at multiple budget levels by varying the maximum number of sampled trajectories (for SC, ASC, ESC) or the initial cohort size (for Parallel-Probe), producing the scaling curves shown in Figure 3.

• Generation budget / compute accounting. The universal unit of computation is a generation interval — a fixed-length chunk of Δ = 500 tokens generated by the base LLM. Every action in the MDP carries a generation cost:

  • BRANCH: costs 1 interval (the new branch is advanced through its first interval immediately).
  • CONTINUE(i): costs 1 interval (advances branch i by one chunk).
  • PROBE(i): costs κ_probe ≈ 0 (reading a pre-computed intermediate answer is treated as free relative to generation).
  • PRUNE(i): costs 0 (no generation, just changes the active set).
  • ANSWER: costs 0 (termination is free).

  Total token cost for a controller run is (total number of generation intervals ever consumed) × 500. This cost model allows direct comparison between controllers with different branching and deepening patterns — a controller that spawns 10 branches and deepens each by 20 intervals costs 200 intervals (100,000 tokens), regardless of when it probed or pruned. The budget ceiling for all controllers is max_branch = 64 total branches (not intervals — the controller can spawn at most 64 distinct branches, each of which can be deepened arbitrarily until it reaches the end of its pre-collected trajectory). The discovered controller's beta parameter controls how aggressively it approaches this ceiling, with β = 1 corresponding to near-full budget use and lower β values achieving lower cost through more selective allocation (fewer initial branches, stricter stopping, less widening).

• Cross-validation / statistical protocol. The paper does not use standard k-fold cross-validation or report confidence intervals. Instead, it employs two forms of variance reduction:

  • Replay determinism: Because all branch prefixes and probe answers are pre-collected offline, controller evaluation is fully deterministic for a fixed set of sampled trajectories. This removes stochasticity from the base LLM's generation process (the trajectories are fixed once collected) and from the controller's decision process (the controller logic is deterministic given the observed state).
  • Multi-sampling for robustness: To reduce variance from which trajectories happen to be in the branch pool, each controller is evaluated 64 times independently per question. In each of the 64 runs, a random subset of trajectories is sampled from the 128 pre-collected ones (the exact subset size depends on how many branches the controller spawns — typically at most 64 out of 128). Accuracy and token cost are then averaged over these 64 runs. This provides some robustness to trajectory sampling variability, though the paper does not report standard deviations, so the magnitude of this variability is unknown.

  For controller selection (Section 4), the paper uses a straightforward performance-based criterion: after R = 5 discovery rounds, the controller achieving the highest accuracy on E_search (AIME24 across all four Qwen3 models) is selected as the final controller. There is no hold-out selection set or cross-validation within the search set — the selected controller is directly evaluated on the held-out benchmarks (AIME25, HMMT25). This means there is a risk of mild overfitting to AIME24, though the paper argues that the beta parameterization constraint mitigates this (Section 3.3, validated by the ablation in Table 3).

Main Quantitative Results

Search Set Performance and Comparison to Baselines

Table 1 (column "AIME24 (search)") reports accuracy and total tokens on the search set across all four Qwen3 models for each baseline and the discovered controller at two β values (0.5 and 1.0). The discovered controller was selected based on performance on this set, so these numbers reflect the optimization target rather than generalization. Key observations:

• On Qwen3-4B: The discovered controller at β = 1.0 achieves 83.5% accuracy with 424.9K tokens, a clear improvement over SC@64 (80.0% accuracy, 886.8K tokens) — higher accuracy with less than half the token cost. At β = 0.5, it achieves 82.0% accuracy with 236.7K tokens — slightly higher accuracy than SC@64 with roughly one-quarter the cost. Both β values also outperform Parallel-Probe (79.7% accuracy, 688.9K tokens).

• On Qwen3-8B: The discovered controller at β = 1.0 achieves 85.8% accuracy with 467.4K tokens, compared to SC@64 at 80.4% accuracy and 910.8K tokens — a 5.4 percentage point accuracy improvement with approximately half the tokens. At β = 0.5, it achieves 84.3% accuracy with 255.3K tokens, outperforming SC@64 by 3.9 percentage points with 72% fewer tokens.

• On Qwen3-1.7B: The pattern is more nuanced. SC@64 achieves 72.5% accuracy with 1025.8K tokens. The discovered controller at β = 1.0 achieves a lower accuracy (70.4%) but with substantially fewer tokens (499.1K) — trading 2.1 percentage points of accuracy for a 51% token reduction. At β = 0.5, it achieves 68.5% accuracy with 276.3K tokens, compared to ASC at 72.3% accuracy with 482.6K tokens — ASC wins on accuracy but uses more tokens. This suggests the discovered controller may be slightly suboptimal on the 1.7B model scale, though the accuracy–cost tradeoff (Figure 3) is still competitive.

• On Qwen3-0.6B: At this smallest scale, the discovered controller at β = 1.0 achieves 20.9% accuracy with 542.2K tokens, slightly below SC@64 (21.4% accuracy) but with roughly half the tokens. At β = 0.5, accuracy drops to 19.2% with 283.6K tokens. The absolute performance is low across all methods on this model scale, and the discovered controller does not substantially improve peak accuracy.

Overall, the discovered controller at β = 1.0 pushes peak accuracy beyond all hand-crafted baselines in 5 out of 8 cases (across four models × two β values, counting β = 1.0 on models where it exceeds all baselines: 4B, 8B; plus β = 0.5 where it exceeds or matches baselines while using far fewer tokens: 4B, 8B, 1.7B vs. ASC in efficiency). At β = 0.5, the controller generally achieves similar or slightly lower accuracy than the best baseline but with dramatic token savings — for example, 69.5% fewer tokens than SC@64 on average across all four models while maintaining on-par accuracy (45.3 vs. 45.2, as stated in Section 5.1), though this "on-par accuracy" claim is not explicitly shown in Table 1 at the per-model level and appears to be computed as an aggregate across models.

Held-Out Generalization: AIME25 and HMMT25

Table 1 (columns "AIME25 (held-out)" and "HMMT25 (held-out)") reports performance on benchmarks never seen during discovery or controller selection. This is the critical test of whether the discovered controller captures generalizable allocation principles rather than overfitting to AIME24.

On AIME25:

• Qwen3-0.6B: Discovered (β = 1.0) achieves 31.1% accuracy with 474.7K tokens, outperforming SC@64 (28.9%, 890.5K) and Parallel-Probe (29.7%, 697.8K) in both accuracy and efficiency.
• Qwen3-1.7B: Discovered (β = 1.0) achieves 49.0% accuracy with 612.6K tokens, substantially better than all baselines: SC@64 (44.4%, 1054.1K), ASC (44.4%, 600.9K), ESC (44.4%, 913.8K), Parallel-Probe (44.7%, 775.8K). This is a 4.3–4.6 percentage point accuracy gain over the best baseline with lower or comparable token cost.
• Qwen3-4B: Discovered (β = 1.0) achieves 74.4% accuracy with 610.4K tokens, slightly below SC@64 (76.6%, 1088.1K) and ASC (76.4%, 277.3K) in accuracy but with a much stronger accuracy–efficiency tradeoff (roughly half the tokens of SC@64 for 2.2 points less accuracy). β = 0.5 achieves 73.8% with 332.3K tokens.
• Qwen3-8B: Discovered (β = 1.0) achieves 75.8% accuracy with 672.4K tokens, compared to SC@64 (76.7%, 1124.4K) — slightly lower accuracy but 40% fewer tokens. Parallel-Probe achieves 76.9% accuracy with 846.7K tokens, slightly edging out the discovered controller in peak accuracy.

On HMMT25:

• Qwen3-0.6B: Discovered (β = 1.0) achieves 18.0% accuracy with 487.1K tokens, comparable to SC@64 (18.1%, 937.8K) with roughly half the tokens.
• Qwen3-1.7B: Discovered (β = 1.0) achieves 32.1% accuracy with 679.6K tokens, a substantial improvement over all baselines: SC@64 (24.2%, 1132.9K), ASC (24.2%, 586.3K), ESC (24.2%, 1014.2K), Parallel-Probe (22.6%, 860.2K). This is a 7.9 percentage point accuracy gain over the best baseline (SC@64/ASC) with 40% fewer tokens than SC@64. The β = 0.5 variant achieves 30.5% accuracy with 359.1K tokens — still 6.3 points above SC@64 with 68% fewer tokens. This is the strongest single result in the paper.
• Qwen3-4B: Discovered (β = 1.0) achieves 46.5% accuracy with 686.8K tokens, outperforming SC@64 (43.6%, 1168.3K) and slightly edging Parallel-Probe (44.7%, 872.3K) in accuracy while using fewer tokens.
• Qwen3-8B: Discovered (β = 1.0) achieves 49.5% accuracy with 749.1K tokens, slightly above SC@64 (48.9%, 1267.0K) in accuracy with substantially fewer tokens, and above Parallel-Probe (47.1%, 897.2K) in both dimensions.

Aggregate held-out average (Table 1, final column): Across both held-out benchmarks averaged over all four models, the discovered controller at β = 1.0 achieves:

• Qwen3-0.6B: 24.6% accuracy, 480.9K tokens vs. SC@64 at 23.2%, 914.2K — better accuracy, 47% fewer tokens.
• Qwen3-1.7B: 40.6% accuracy, 646.1K tokens vs. SC@64 at 34.3%, 1093.5K — substantially better accuracy, 41% fewer tokens.
• Qwen3-4B: 60.5% accuracy, 648.6K tokens vs. SC@64 at 60.1%, 1128.2K — comparable accuracy, 43% fewer tokens.
• Qwen3-8B: 62.7% accuracy, 710.8K tokens vs. SC@64 at 62.8%, 1195.7K — essentially tied in accuracy, 41% fewer tokens.

The discovered controller outperforms all hand-crafted baselines in three out of four models on average held-out accuracy and remains competitive (within 0.1 percentage points of SC@64) on Qwen3-8B, while consistently using 40–50% fewer tokens than SC@64.

Accuracy–Cost Scaling Curves (Figure 3)

Figure 3 plots accuracy against total tokens (log scale) for the discovered controller and hand-crafted baselines on held-out benchmarks. Each curve is obtained by varying the budget parameter: for hand-crafted baselines, by sweeping the maximum number of sampled trajectories; for the discovered controller, by sweeping β across its range.

Key patterns visible in Figure 3:

• Figure 3a (Qwen3-0.6B on AIME25): The discovered controller's curve lies above all baselines in the low-to-mid budget regime (~100K–300K tokens) but converges with SC@64 and ASC at higher budgets. The Pareto frontier is largely overlapping at the high-budget end.

• Figure 3b (Qwen3-4B on HMMT25): The discovered controller's curve is clearly above all hand-crafted baselines across the full budget range from ~200K to ~900K tokens. At a given accuracy level (e.g., 44%), the discovered controller achieves it with roughly half the tokens of Parallel-Probe. At a given token budget (e.g., 500K), it achieves 3–5 percentage points higher accuracy than any baseline.

• Figure 3c (Qwen3-1.7B on AIME25): The discovered controller shows the strongest relative advantage on this setting. Its curve dominates across the full range, with particularly large gaps at mid-to-high budgets: at ~600K tokens, it achieves ~49% accuracy vs. ~44–45% for all baselines — a gap of 4–5 percentage points at the same cost.

• Figure 3d (Qwen3-8B on HMMT25): The discovered controller's curve is competitive and generally at or above the baselines, though the gap is narrower than for smaller models. At ~400K tokens, it achieves ~46% vs. ~42–44% for baselines; at ~700K tokens, ~49.5% vs. ~48–49% for SC@64/ASC.

The paper notes (Section 5.2) that the discovered controller "does not simply reduce inference cost at a fixed accuracy level" — it can also push the attainable peak performance beyond what hand-crafted baselines achieve, as indicated by the rightmost points of its curves in Figures 3b, 3c, and 3d being above all baseline curves. This is important because it suggests the discovery process found mechanisms that improve both efficiency and peak capability, not just a more efficient way to reach the same accuracy ceiling.

Comparison at Specific Budget Regimes

The β = 0.5 and β = 1.0 points in Table 1 represent two operating regimes of the discovered controller:

• β = 0.5 (efficiency regime): Across the held-out average, the discovered controller uses 254.0K to 379.0K tokens (depending on model) while achieving accuracy competitive with or slightly below the best baseline. Compared to SC@64, this represents token reductions of approximately 72% (0.6B: 254.0K vs. 914.2K), 69% (1.7B: 343.5K vs. 1093.5K), 69% (4B: 348.7K vs. 1128.2K), and 68% (8B: 379.0K vs. 1195.7K). The paper's claim of "69.5% reduction while maintaining on-par accuracy" (Section 5.1) is computed as an average across models, though per-model accuracy does drop slightly (e.g., from 60.1 to 59.8 on 4B, from 62.8 to 61.1 on 8B).

• β = 1.0 (accuracy regime): The discovered controller uses 480.9K to 710.8K tokens while achieving accuracy that matches or exceeds the best baseline in most cases. Compared to SC@64, token reductions range from 41% (8B: 710.8K vs. 1195.7K) to 47% (0.6B: 480.9K vs. 914.2K), while accuracy improves (0.6B, 1.7B), stays comparable (4B), or is essentially tied (8B).

The key takeaway from these two regimes is that the discovered controller, parameterized by a single β, provides a unified family that spans the Pareto frontier — from extreme efficiency (β = 0.5) to peak accuracy (β = 1.0) — without requiring separate strategy designs for different budget points.

Generalization Beyond Qwen Models and Math Tasks (Table 2)

Table 2 tests whether the discovered controller (discovered on Qwen3 models with AIME24) transfers to (a) a different model family and (b) a non-math benchmark.

DeepSeek-R1-Distill-Llama-8B on HMMT25:

• The discovered controller at β = 1.0 achieves 27.2% accuracy with 533.9K tokens, the highest accuracy among all methods while using substantially fewer tokens than SC@64 (26.7%, 985.7K). This is a 0.5 percentage point accuracy gain with 46% fewer tokens.
• At β = 0.5, it achieves 26.3% accuracy with 279.0K tokens — a small accuracy drop (0.4 points below SC@64) but with 72% fewer tokens.
• The controller outperforms ASC (26.5%, 582.7K) in both accuracy and cost at β = 1.0, and in cost only at β = 0.5.

This result is significant because the DeepSeek-R1-Distill-Llama model has a completely different pretraining lineage (Llama-based, distilled from a reasoning model) compared to the Qwen3 models the controller was discovered on. The fact that the controller transfers without modification suggests that the allocation principles it discovered (momentum-based stopping, coupled width–depth control, etc.) are not specific to Qwen3's error patterns or calibration properties.

Qwen3-1.7B on GPQA-Diamond (science QA):

• The discovered controller at both β = 1.0 and β = 0.5 achieves 41.6% accuracy, matching SC@64's 41.3% while using substantially fewer tokens: 270.1K (β = 1.0) and 151.0K (β = 0.5) vs. 510.0K for SC@64.
• Compared to ASC (41.0%, 186.3K), the discovered controller at β = 0.5 achieves higher accuracy (41.6 vs. 41.0) with fewer tokens (151.0K vs. 186.3K) — a strict improvement in both dimensions.
• ESC achieves 41.3% with 391.6K tokens, worse in cost-efficiency than both β variants.

GPQA-Diamond tests graduate-level science knowledge rather than mathematical reasoning, so this result tests whether the discovered allocation strategy (designed for math) transfers to a domain with different answer patterns, difficulty characteristics, and model behaviors. The positive transfer suggests the discovered mechanisms capture general properties of effective computation allocation, not math-specific heuristics.

Ablation Studies and Robustness Checks

Beta parameterization (Table 3, "w/o Beta Parameterization"): Removing the constraint that all hyperparameters must be deterministic functions of a single β — allowing the agent to propose controllers with multiple independent tunable knobs — leads to catastrophic overfitting. On held-out benchmarks, accuracy drops from 53.1 to 49.0 (average across four models and AIME25/HMMT25), while token consumption plummets from 575.5K to 93.3K — an 83.8% reduction. This pattern (accuracy down, tokens way down) is the signature of the agent discovering overly aggressive pruning and stopping thresholds that happen to work on AIME24 but fail to allocate sufficient computation for held-out problems. The search cost also increases ($46.4 vs.$39.9), likely because the agent spends more rounds exploring the larger hyperparameter space without converging to a robust solution. This ablation provides the strongest empirical evidence for the paper's central claim that beta parameterization is essential for preventing overfitting in TTS strategy discovery.

Execution traces (Table 3, "w/o Execution Traces"): Removing the per-step execution trace feedback from the discovery loop degrades both accuracy and efficiency. Held-out accuracy drops from 53.1 to 51.6, while token consumption increases from 575.5K to 824.3K — the discovered controller is both less accurate (by 1.5 percentage points) and less efficient (using 43% more tokens). The search cost is lower ($30.9 vs.$39.9), likely because the agent has less context to process per round, but the resulting controller is substantially worse. This supports the paper's argument that scalar accuracy–cost feedback alone is insufficient to guide effective search — the agent needs fine-grained execution traces to diagnose why controllers fail and propose targeted improvements. Without traces, the agent essentially guesses at mechanism design, producing controllers that are poorly calibrated (sometimes too aggressive, sometimes too conservative) and fail to achieve the Pareto improvements that trace-guided discovery enables.

Controller generalization across model scales (Table 1, all rows): The same discovered controller (CMC) is evaluated on all four Qwen3 models (0.6B to 8B) without any model-specific tuning. The fact that it outperforms or matches hand-crafted baselines across this 13× parameter range — and shows the largest relative gains on Qwen3-1.7B (particularly on HMMT25, where it achieves a 7.9 point accuracy gain over SC@64) — is evidence that the discovered allocation principles are not tightly coupled to a specific model capacity. However, the gains are not uniform: the controller provides the most dramatic improvements on 1.7B and 4B, while on 8B it is roughly tied with SC@64 in accuracy and on 0.6B the absolute gains are small (though relative efficiency is high). This pattern suggests that discovered strategies are most valuable for medium-capability models where the base pass@1 is non-trivial but far from saturation — consistent with the findings from Snell et al. (2024) that test-time compute helps most when problems are within the model's rough capability range but not trivially easy.

Controller generalization across benchmarks (Table 1, AIME25 and HMMT25 columns): The controller was discovered and selected on AIME24 only. Performance on AIME25 and HMMT25 tests generalization to held-out problem distributions. The controller shows strong transfer, with particularly large improvements on HMMT25 for Qwen3-1.7B and on AIME25 for Qwen3-1.7B. However, there is some inconsistency: on Qwen3-8B, the discovered controller at β = 1.0 achieves slightly lower accuracy than SC@64 on AIME25 (75.8 vs. 76.7) and slightly higher on HMMT25 (49.5 vs. 48.9), suggesting that the controller is not uniformly better across all settings and that the margin of improvement is small at the highest model scale.

Controller transfer to different model family (Table 2, DeepSeek-R1-Distill-Llama-8B): The discovered controller transfers to a Llama-based model with minimal accuracy degradation and substantial token savings, supporting the claim that the discovered mechanisms are model-agnostic. However, this is tested on only one non-Qwen model and one benchmark; broader testing would be needed to establish robust cross-family generalization.

Controller transfer to non-math domain (Table 2, GPQA-Diamond): The controller maintains accuracy while reducing tokens on a science QA benchmark, suggesting the allocation principles generalize beyond math. However, GPQA-Diamond is still a multiple-choice/short-answer benchmark with clear correctness signals; whether the controller would transfer to open-ended generation tasks or tasks requiring factual recall rather than reasoning is unknown.

Critical Assessment

Central Claim 1: "The discovered strategies improve the accuracy–cost tradeoff over strong manually designed baselines."

This claim is supported with qualifications that vary by model scale and benchmark. On Qwen3-1.7B and Qwen3-4B, the discovered controller at β = 1.0 achieves both higher accuracy and lower token cost than all hand-crafted baselines on held-out benchmarks — a strict Pareto improvement (Table 1). On Qwen3-8B, it roughly matches SC@64 in accuracy while using ~40% fewer tokens, which is a Pareto improvement along the cost axis but not along the accuracy axis. On Qwen3-0.6B, absolute accuracy is low across all methods (18–31% on held-out sets), and the discovered controller's advantage is primarily in cost reduction rather than accuracy improvement. The paper's abstract claim of "improved accuracy–cost tradeoffs" is broadly accurate, but the improvement is not uniform — it is most pronounced on medium-scale models (1.7B–4B) and on harder benchmarks (HMMT25), while on the largest model (8B) the gains are primarily in efficiency rather than peak accuracy.

A caveat: the "strong manually designed baselines" include ASC and ESC, which were designed for full-read (non-incremental) probing and may not be fully optimized for the incremental-probing paradigm used by the discovered controller. The discovered controller operates in a richer action space (probe, continue, prune interactively) than ASC/ESC (which only do full reads and early stopping), so part of its advantage may come from the action-space expansion rather than superior strategy design per se. Parallel-Probe uses the same incremental-probing paradigm and is the most appropriate baseline; the discovered controller outperforms it in most settings (e.g., +4.6 points on AIME25 for 1.7B, +2.0 points on HMMT25 for 4B), though the advantage is not universal (Parallel-Probe slightly edges out the discovered controller on Qwen3-8B AIME25: 76.9 vs. 75.8).

Central Claim 2: "The discovered strategies generalize to held-out benchmarks and model scales."

This claim is strongly supported within the tested range. The controller was discovered on AIME24 (search set) and generalizes to AIME25 and HMMT25 (held-out sets) across all four Qwen3 models, with performance generally maintaining or exceeding baselines. The generalization to DeepSeek-R1-Distill-Llama-8B (different model family) and GPQA-Diamond (different domain) in Table 2 provides additional cross-family and cross-domain evidence.

However, the claim's scope should be qualified:

• Held-out benchmarks are both math competitions (AIME and HMMT). They test the same type of reasoning (multi-step mathematical problem-solving) as the search set, just with different specific problems. Generalization to genuinely different task types (code generation, logical reasoning, open-ended QA) is tested only on GPQA-Diamond, and only on one model (1.7B). This is a narrow test of domain generalization.
• Model scales span 0.6B to 8B within a single model family (Qwen3). While this is a 13× range, all models share the same pretraining data, tokenizer, and architectural decisions (transformer variants within the Qwen family). Generalization to models with different architectures (e.g., non-autoregressive models, mixture-of-experts, different training objectives) is not tested, except for the single DeepSeek-R1-Distill-Llama checkpoint.
• The ablation showing overfitting without beta parameterization (Table 3) provides strong evidence that the generalization is not accidental — the beta constraint actively prevents search-set overfitting. However, the paper does not report how the discovered controller would perform if re-discovered on each held-out benchmark individually (i.e., what is the ceiling vs. the transfer gap). It's possible that benchmark-specific discovery would yield even stronger controllers, and the reported held-out performance reflects a tradeoff between generalization and benchmark-specific optimization.

Central Claim 3: "The entire discovery costs only $39.9 and 160 minutes."

This claim is factually reported but the cost accounting has important exclusions:

• **The $39.9 covers only the five-round discovery loop** — API calls to Claude Code for proposing controllers, plus compute for running replay evaluations. It does **not** include the one-time cost of collecting offline replay data: generating 128 trajectories per question from the base LLM for each (model, benchmark) pair. For AIME24 alone across 4 models × 30 questions × 128 trajectories × ~5K tokens per trajectory (average reasoning length), this is roughly 77 million tokens of generation. At typical API pricing for Qwen3 models (which are open-weight and can be run locally, but the paper does not disclose whether cloud API or local inference was used), this might cost$20–100 depending on hardware. If data collection were done from scratch for every new model or benchmark, the amortized cost would be higher.

• The 160 minutes presumably measures wall-clock time for the discovery loop plus evaluation, but the paper does not specify what hardware was used or whether this includes the time for the explorer agent (Claude Code) to process history and generate proposals, which involves network latency to an external API.

• Reusability is the paper's argument for why this cost is acceptable: the same replay data can be used for many discovery runs (e.g., re-running discovery with different random seeds, exploring different environment designs, or re-discovering controllers after model updates). The paper does not explicitly compute the amortized cost per discovery run if data collection is included, but the one-time collection cost divided by, say, 10 discovery runs would add perhaps $2–10 per run, keeping the total well under$50.

Central Claim 4 (implicit): "The discovered controller's mechanisms represent a level of coordinated complexity that would be difficult to arrive at through manual intuition."

This is a qualitative claim supported by the novelty of the CMC mechanisms (trend-based stopping via EMA momentum, coupled width–depth control, alignment-aware depth allocation, conservative abandonment) relative to hand-crafted baselines. However, the paper does not provide direct evidence that these mechanisms could not have been designed by humans — it only demonstrates that they were not (prior hand-crafted methods use instantaneous confidence gates, fixed cohort sizes, or uniform depth allocation). A human designer, given the same environment and feedback, might have arrived at similar mechanisms after sufficient iteration. The claim's strength rests on the reader's assessment of whether the specific combination of EMA momentum + coupled width–depth feedback + priority-based depth allocation represents a non-obvious design that manual intuition would be unlikely to produce. The trajectory in Figure 4, showing alternating accuracy pushes and efficiency adjustments over rounds, does suggest that the discovery process involved back-and-forth refinement that would be tedious and difficult to do manually, but it does not prove impossibility.

Potential weaknesses and missing experiments:

1. No variance reporting. The paper reports point estimates (averages over 64 evaluation runs) but never reports standard deviations, standard errors, or confidence intervals. For a test set of only 30 questions (AIME) or an unreported number for HMMT, accuracy differences of 1–3 percentage points could be within sampling noise. The paper's strongest results (e.g., 7.9 point gain on HMMT25 for Qwen3-1.7B at β = 1.0) are large enough to likely be statistically significant, but smaller differences (e.g., 62.7 vs. 62.8 on Qwen3-8B held-out average) are almost certainly within noise. Without variance estimates, the reader cannot assess reliability.

2. Small test sets. AIME24 and AIME25 each have only 30 questions. HMMT25's problem count is not specified but is likely similar (25–30 problems, based on typical HMMT formats). This means held-out evaluation is based on roughly 55–60 total questions. Accuracy differences of a few percentage points correspond to only 1–2 additional correct answers, which could be affected by random trajectory sampling despite the 64-run averaging.

3. No comparison to the best possible controller (oracle). The paper does not report an upper bound on what an optimal controller could achieve in the replay environment. For example, what accuracy would be achieved by an oracle that always selects the correct answer from the 128 available trajectories? What is the best possible accuracy given the trajectory data? Without such bounds, the reader cannot assess whether the discovered controller is close to optimal or whether there is still substantial room for improvement.

4. Single explorer agent (Claude Code). The discovery uses Claude Code as the explorer. The paper does not test whether other coding agents (GPT-4, Gemini, open-source alternatives) would produce comparable or better controllers. The ablation studies remove components of the framework (beta parameterization, traces) but not the agent. It's possible that Claude Code has specific strengths (or weaknesses) in program synthesis that affect the results.

5. No comparison to directly optimizing the beta schedule parameters. The beta parameterization collapses hyperparameter search to a single β knob, but the β → hyperparameter mapping (the _schedule function) is itself designed by the agent. The paper does not compare against a baseline where the schedule coefficients are directly optimized (e.g., via Bayesian optimization over the 11 coefficients in the linear schedule) rather than designed by LLM-driven code editing. This would test whether the LLM's mechanism design (the structure of the controller logic) is what provides value, or whether simple coefficient tuning on a fixed controller structure would suffice.

6. No ablation of the "multi-model search set" design choice. The search environment E_search includes AIME24 trajectories from all four Qwen3 models simultaneously. The paper does not test whether discovering on a single model (e.g., only Qwen3-4B) and evaluating on others would yield comparable cross-model generalization, or whether the multi-model search set is essential for the strong transfer results.

7. The discovered controller's code is complex (Appendix D, ~500 lines). While the paper argues this complexity reflects coordinated mechanisms that manual design would miss, it also makes the controller harder to interpret, debug, and trust. A simpler discovered controller with slightly worse performance might be preferable in practice, but the discovery prompt's incentives (emphasizing novelty and performance) may push toward complexity. The paper does not discuss this complexity–performance tradeoff.

8. No test of whether the controller works in online (non-replay) settings. All evaluation uses the offline replay environment. While the replay data is drawn from actual LLM generations, there may be subtle distribution shifts: in online inference, the controller's decisions affect which branches are explored, which could change the distribution of generated prefixes (e.g., if the base LLM uses different sampling for branches that are continuations vs. fresh starts). The replay environment treats each branch's trajectory as fixed regardless of the controller's decisions, which eliminates any such interaction effects. Whether a controller discovered in replay would perform identically in online inference is not verified.

9. The cost metric (total tokens) ignores latency. The discovered controller uses sequential adaptive decisions (probe, evaluate, decide to continue/prune/widen) that may have higher wall-clock latency than embarrassingly parallel methods like SC@64, even if total tokens are lower. For latency-sensitive applications, the token reduction may not translate to end-to-end speed improvements. The paper does not report wall-clock inference time.

10. Hand-crafted baselines may not be optimally tuned. ASC uses the default threshold of 0.95; ESC uses chunk size 8; Parallel-Probe uses its published hyperparameters. These methods might perform better with tuning on AIME24 (the same search set used for discovery), but the paper deliberately avoids tuning them to maintain a fair comparison (the baselines are used as-is from their original papers). However, this means the baselines may be suboptimal for the specific (model, benchmark) pairs tested, and the discovered controller's advantage partly reflects the benefit of search-set-specific optimization (even with beta regularization) versus off-the-shelf baseline configurations. A fairer comparison might include tuned versions of the baselines where their hyperparameters are optimized on AIME24 to the same extent the discovered controller is.

Summary of evidential strength:

The experiments demonstrate that an agent-driven discovery framework, constrained by beta parameterization and guided by execution trace feedback, can produce a controller that matches or exceeds the accuracy of strong hand-crafted baselines while using 40–70% fewer tokens on held-out math benchmarks across multiple model scales. The generalization evidence (to different model families and a non-math benchmark) is positive but limited in breadth. The ablation studies provide credible evidence that both beta parameterization and execution trace feedback are essential components — removing either substantially degrades the discovered controller. However, the absolute magnitude of improvement over the strongest baseline (Parallel-Probe) is modest in many settings (1–5 percentage points, sometimes within plausible noise), and the small test sets, lack of variance reporting, and absence of online evaluation prevent a definitive claim that the discovered controller is practically superior in deployment rather than just in the specific offline replay setting tested. The paper's strongest contribution is not the specific controller's performance numbers but rather the demonstration that the discovery paradigm works at all — that an agent can find non-obvious allocation mechanisms that improve over human design — and that the framework components (replay, beta, traces) are individually validated as necessary for this success.

6. Limitations and Trade-offs

The Offline Replay Environment Assumes Trajectories Are Independent of Controller Decisions

The assumption or constraint. The offline replay environment is constructed by pre-collecting 128 reasoning trajectories per question from the base LLM in a single-shot manner — each trajectory is generated independently from scratch with no knowledge of which branches a controller will later spawn, continue, or prune. The controller then replays its decisions against this fixed pool, observing intermediate answers at each depth interval as if the generation had occurred in response to its actions. This assumes that the prefixes $z_{i,k}$ and intermediate answers $\omega_{i,k}$ are identical regardless of the controller's decision sequence — that is, generation is path-independent. The paper acknowledges this implicitly in Section 3.1 when constructing the replay environment, but does not discuss what breaks if this assumption fails.

The consequence. In online (live) inference, a controller's decisions could affect the distribution of generated text in ways the replay data does not capture. For instance, if a controller prunes branches early and then widens to explore new reasoning paths, the newly spawned branches in live inference might benefit from the pruned branches' partial computation (e.g., through shared key-value caches or batch effects that influence sampling), or the model might generate different continuations if it "knows" certain paths have been abandoned. More subtly, if the base LLM is non-deterministic at temperature > 0, the trajectories in the replay data represent one specific set of sampled outputs — a controller optimized against this fixed set might overfit to the idiosyncrasies of those particular generations. A controller that performs well on the replay data might make decisions that, in live inference with fresh samples, lead to different intermediate answers at the same depths, invalidating the strategy's calibrated thresholds.

What evidence exists in the paper. The paper does not evaluate any discovered controller in an online setting where the base LLM is invoked in response to the controller's actions. All reported results (Table 1, Figure 3, Table 2) use the offline replay environment exclusively. The 64-run averaging (Section 4) reduces variance from which subset of the 128 pre-collected trajectories is sampled, but does not test whether the controller's behavior differs when trajectories are generated on-the-fly. This is a fundamental limitation because the entire discovery paradigm rests on the assumption that offline replay fidelity is high enough that controllers discovered in replay will transfer to online deployment.

Mitigation status. The paper does not address this limitation beyond the design of the replay environment itself. It suggests in Appendix A that "richer environments that support more complex control structures" would be an interesting extension, but does not propose validating replay-to-online transfer. A natural mitigation — evaluating the final discovered controller in a live setting on a small subset of questions and comparing to replay performance — is not performed. Without such validation, the claimed improvements over baselines are technically replay-environment improvements and may not fully materialize in deployment.

Difficulty Estimation Is Absent, and Hard Problems Receive No Benefit

The assumption or constraint. The controller discovered by AutoTTS allocates computation based solely on the 2D probing state — active branches, their depths, revealed probe answers, and the accumulated EMA confidence signal. It has no mechanism for estimating question difficulty before or during inference and therefore deploys the same adaptive strategy regardless of whether the problem is trivially easy for the base model or far beyond its capabilities. This contrasts with the compute-optimal framework of Snell et al. (2024), which explicitly conditions strategy selection on estimated prompt difficulty and shows that difficulty-agnostic allocation leaves large efficiency gains on the table.

The consequence. On hard problems where the base model's pass@1 is near zero — problems in difficulty bin 5 in the Snell et al. taxonomy, or the hardest AIME/HMMT questions — the discovered controller will still spawn its initial n_init branches, run through warm-up rounds, maintain EMA tracking, and potentially widen when confidence stagnates, consuming significant token budget on branches that have negligible probability of containing the correct answer. The controller's stopping gate (EMA momentum) may not fire if confidence never stabilizes, and the coupled widening trigger (ema_delta <= trend_thresh) may repeatedly spawn new branches that are equally unlikely to be correct, burning budget until max_branch_use is exhausted. The paper's own data shows that on the smallest model (Qwen3-0.6B), absolute accuracy on held-out benchmarks is only 18–31% across all methods (Table 1) — suggesting a substantial fraction of problems are essentially unsolvable by this model — yet the discovered controller still consumes hundreds of thousands of tokens on these problems.

More subtly, the absence of difficulty awareness means the controller cannot distinguish between "the problem is hard and none of my branches are correct, so I should stop early" and "the problem is hard but my branches are on the right track and I should invest more." The EMA confidence signal reflects answer consensus among branches, not external difficulty. On a genuinely hard problem where all branches converge on the same wrong answer with high confidence, the EMA gate may fire prematurely, terminating with a confident but incorrect answer. The controller has no signal to detect this failure mode.

What evidence exists in the paper. The per-model accuracy numbers in Table 1 demonstrate that absolute accuracy varies dramatically across model scales — from ~20% for Qwen3-0.6B to ~80% for Qwen3-8B on the search set — confirming that problem difficulty relative to model capability spans a wide range. The paper does not report accuracy broken out by problem difficulty or show how token consumption varies with problem difficulty. There is no analysis of whether the controller wastes tokens on problems where the base model's pass@1 is near zero, or whether a difficulty-conditioned variant could improve the Pareto frontier further. Figure 4 (evolution trajectory) shows the controller progressively adjusting budget allocation over rounds, but these adjustments are made at the strategy design level (changing the beta schedule), not at the per-problem level during inference. The discovered controller treats every problem identically given the same probing state.

Mitigation status. The paper does not address difficulty estimation or difficulty-conditioned allocation. This is a deliberate scoping choice — the width–depth MDP formalization (Section 2) does not include difficulty as a state variable, and the discovery environment does not provide difficulty signals to the controller. However, the paper also does not acknowledge this as a limitation in Appendix A, which focuses only on extending the action space to richer control structures. For practitioners, this means the discovered controller may be inefficient on problem distributions that are skewed toward problems beyond the base model's capability, and combining AutoTTS with a difficulty estimator (as in Snell et al., 2024) remains an open challenge.

Discovered Controllers Are Black-Box Programs With No Correctness Guarantees

The assumption or constraint. AutoTTS produces a code-defined controller — a Python class implementing the solve method with hundreds of lines of logic, schedule functions, and interacting mechanisms (the CMC in Appendix D is approximately 500 lines). This controller was synthesized by an LLM (Claude Code) through iterative improvement over five rounds, and the paper provides no formal verification, testing on adversarial cases, or analysis of failure modes beyond aggregate accuracy–cost curves. The controller's behavior emerges from the interaction of EMA momentum tracking, confidence thresholds, branch classification, priority-queue scheduling, and trend-based widening — each of which depends on beta-parameterized hyperparameters with specific coefficient values (0.40, 0.12, 0.03, etc.) that were discovered through sandbox experimentation.

The consequence. In deployment, this controller could exhibit unexpected or pathological behavior on inputs that differ from the mathematical reasoning benchmarks it was discovered and evaluated on. For example:

• The EMA stopping gate requires ema_conf >= conf_thresh AND ema_delta >= -delta_slack. If conf_thresh is set to 0.97 at β = 1.0, and the problem's answer distribution naturally produces confidence hovering around 0.95–0.96, the controller might never stop, exhausting the max_outer = 500 loop iterations and falling through to a final majority vote that may be based on incomplete information.
• The conservative branch abandonment rule (never prune below 2 active branches) could, on a problem where all but one branch are genuinely unproductive, keep a deviant branch alive for abandon_patience rounds (up to 12 at β = 1.0), wasting computation.
• The beta schedule coefficients (e.g., ema_alpha = 0.70 - 0.40 * β) were tuned against the AIME24 search set. On a different problem distribution where answer convergence patterns are different — e.g., science QA where confidence stabilizes more quickly, or code generation where intermediate answers are less reliable — these coefficients may produce miscalibrated behavior (stopping too early or too late).
• The controller was discovered against Qwen3 models with trajectories generated at temperature 0.7. If deployed with a different base model, a different temperature, or a different prompt format, the calibration of intermediate probe answers could shift, and the controller's thresholds (which assume a certain relationship between probe depth and answer reliability) may no longer be appropriate.

More broadly, the controller is a non-interpretable artifact: even with the execution traces, understanding why it made a particular decision on a particular problem requires tracing through multiple interacting mechanisms (EMA smoothing, branch classification, priority scheduling, trend evaluation) that have nonlinear interactions. Debugging a failure — e.g., "the controller stopped too early on problem 17" — is substantially harder than for hand-crafted heuristics where the decision logic is explicit and modular.

What evidence exists in the paper. The generalization experiments (Table 2) provide partial evidence that the controller is not catastrophically brittle: it transfers to DeepSeek-R1-Distill-Llama-8B (different model family, different training lineage) and to GPQA-Diamond (different task domain) with maintained or improved accuracy–cost tradeoffs. However, these are only two additional settings, both using similar QA-format benchmarks. The paper does not test the controller on:

• Adversarially constructed problems designed to trigger premature stopping or over-widening.
• Problems with very different answer distribution characteristics (e.g., binary yes/no questions, open-ended generation).
• Extreme β values (β < 0.2 or β > 0.95) that might expose edge cases in the schedule functions.
• Different sampling temperatures or prompting strategies that could shift intermediate answer reliability.

The ablation study (Table 3) shows what happens when the discovery framework is weakened, but does not test the robustness of the specific discovered controller to input distribution shift.

Mitigation status. The paper's use of beta parameterization and the multi-model search set (Section 4) is an implicit regularization strategy to prevent the controller from overfitting to narrow conditions. The discovery prompt (Appendix C) includes robustness requirements ("prefer stable operating regions and smooth budget-performance curves over sharp search-set optima") that partially address this concern. However, the paper provides no post-hoc analysis of controller robustness — no sensitivity analysis varying the schedule coefficients, no stress-testing on out-of-distribution inputs, and no discussion of what failure modes to expect. Appendix A acknowledges only the need for "richer environments" and testing with "open-source coding agents," not the need for controller verification or robustness validation.

For practitioners, this means deploying the discovered controller requires trust that the discovery process (offline replay on math benchmarks with Qwen3 models) produces a controller that behaves reasonably on their specific deployment distribution. The $39.9 discovery cost is low enough that practitioners could re-run discovery on their own data and models, but the resulting controller would still be a black-box program with no correctness guarantees beyond the empirical evaluation they perform.

The One-Time Data Collection Cost Is Excluded From the Headline $39.9 Figure

The assumption or constraint. The paper reports that "the entire discovery costs only $39.9 and 160 minutes" (Abstract, Section 5.5). This figure covers the five-round discovery loop: API calls to Claude Code for proposing controllers, plus compute for running replay evaluations against pre-collected trajectories. It explicitly **excludes the cost of constructing the offline replay environment** — generating 128 reasoning trajectories per question from the base LLM for each (model, benchmark) pair used in the search environment `$\mathcal{E}_{\text{search}}$`.

The consequence. For a practitioner who wants to discover a controller for their own model and benchmark, the true end-to-end cost is data collection + discovery, not just discovery. The data collection cost scales with:

• Number of questions in the search set (30 for AIME24; potentially hundreds for larger benchmarks).
• Number of model scales included (4 in this paper: 0.6B, 1.7B, 4B, 8B).
• Number of trajectories per question (128).
• Average trajectory length in tokens.

For AIME24 across four Qwen3 models, assuming an average trajectory length of ~5,000 tokens (typical for math reasoning), the total generation is: 4 models × 30 questions × 128 trajectories × 5,000 tokens ≈ 77 million tokens.

At typical cloud API pricing for models of this scale ($0.10–0.50 per million tokens for open-weight models run on cloud GPUs, or potentially free if run locally), the data collection cost could range from ~$8–40 for inference alone, plus any costs for extracting intermediate answers from partial generations (which may require additional parsing or verifier calls). This is comparable to or larger than the $39.9 discovery cost, roughly doubling the total.

More importantly, data collection must be repeated for every new model a practitioner wants to discover controllers for, and for every new benchmark used as the search set. The paper's multi-model search set (AIME24 across all four Qwen3 scales) required data collection for four separate model–benchmark pairs. If a practitioner wants to discover controllers for a 70B model on a 500-question benchmark, the data collection cost could easily reach hundreds of dollars. The replay environment's reusability — the paper's argument for why the amortized cost is acceptable — only applies if the same replay data is used for many discovery runs. The first discovery run on a new model still pays the full data collection cost.

What evidence exists in the paper. Section 5.5 states the $39.9 and 160 minutes explicitly but does not break down the data collection cost or report total tokens generated for the replay environment. Section 3.1 describes the offline data collection protocol but does not quantify its cost. The paper's framing — "the one-time discovery cost of$39.87 and 160 minutes demonstrates that environment-driven discovery is practical today" (Section 7) — could mislead a reader into thinking this is the total cost of deploying AutoTTS from scratch, when in fact data collection adds non-trivial overhead that is not included in the headline figure.

Mitigation status. The paper does not report data collection costs, does not include them in the headline figure, and does not discuss how they scale with model size, benchmark size, or trajectory count. The paper frames data collection as a one-time amortized expense, but this amortization only applies to repeated discovery runs on the same (model, benchmark) pair — for the initial setup on a new model, the cost must be paid in full. Practitioners should budget for data collection as a separate line item, and the paper would be strengthened by providing per-million-token generation costs for the Qwen3 models used, enabling readers to estimate the data collection cost for their own setups.

The Controller Discovery Depends on a Single Frontier Coding Agent

The assumption or constraint. The entire discovery loop relies on Claude Code (accessed via API) as the explorer agent that reads history, analyzes execution traces, proposes controller improvements, and edits the OptimalController class. The paper provides no experiments testing whether other coding agents — GPT-4, Gemini, open-source models like DeepSeek-Coder or CodeLlama — would produce comparable or better controllers. The discovery prompt (Appendix C) is extensive and highly structured, requiring the agent to understand the MDP formalization, the environment API, the design constraints (monotonicity, coverage, conservative anchor), and the history format. Different agents may have different strengths in program synthesis, debugging from traces, and creative mechanism design.

The consequence. The specific controller discovered (CMC) is the product of a particular agent + prompt + search set + random seed combination. If a practitioner does not have access to Claude Code (due to cost, API restrictions, or organizational policy), they cannot simply re-run the discovery with a different agent and expect comparable results. The ablation studies (Table 3) remove framework components (beta parameterization, traces) but never change the agent. This means the paper provides no evidence about the agent-dependence of the discovery results.

Potential failure modes with a different agent include:

• Weaker code synthesis: An open-source agent might struggle with the 500-line controller implementation, producing buggy code that fails to execute or contains subtle logic errors. The paper's evaluation framework catches runtime errors (the controller must execute without exceptions), but logical errors that produce valid but suboptimal behavior would not be detected.
• Different exploration biases: Claude Code might have particular tendencies — e.g., preferring EMA-based mechanisms, or avoiding certain types of coupling — that influenced the discovered controller's design. A different agent with different biases might discover a completely different, potentially better, controller family that Claude Code never considered.
• Prompt sensitivity: The discovery prompt is complex and includes specific requirements (monotonicity, coverage, conservative anchor, single-knob schedule). Different agents may interpret these requirements differently, leading to controllers that satisfy the letter but not the spirit of the constraints (e.g., a controller that is technically monotonic but has near-discontinuous jumps at certain β values).

What evidence exists in the paper. None. All discovery experiments use Claude Code exclusively. The paper does not report:

• Attempts to use other agents.
• Ablation of the prompt (e.g., removing certain requirements to see if the agent still discovers strong controllers).
• Multiple discovery runs with different random seeds to assess variance in the discovered controller's performance (the paper reports results from a single discovery run).
• Whether the discovered controller is robust to re-initialization of the discovery process (if you run the same agent with the same prompt 5 times, do you get 5 different controllers with similar performance, or does the outcome depend heavily on the first round's proposal?).

Mitigation status. Appendix A acknowledges this limitation explicitly: "the discovery process currently relies on a frontier coding agent; exploring whether open-source coding agents can achieve comparable discovery performance is an interesting direction for future work." This is a frank admission, but the limitation remains unresolved in the current paper. For practitioners who want to use AutoTTS but do not have access to Claude Code, the paper offers no guidance on agent selection, prompt adaptation, or expected performance degradation with alternative agents.

The Held-Out Benchmarks Are All Mathematical Reasoning Tasks With Small Test Sets

The assumption or constraint. The paper's generalization evaluation (Table 1, Figure 3) tests the discovered controller on AIME25 and HMMT25, both of which are mathematical competition benchmarks of the same type as the search set (AIME24). The generalization to non-math tasks is tested only on GPQA-Diamond (a graduate-level science QA benchmark) using a single model (Qwen3-1.7B) at a single β value. All test sets are small: AIME has 30 questions, HMMT25 likely has a similar number (25–30, based on typical HMMT formats), and GPQA-Diamond's size is not specified but is typically 198–228 questions depending on the split. The paper reports only point estimates (averages over 64 evaluation runs) without standard deviations or confidence intervals.

The consequence. The paper's central generalization claim — that "the discovered strategies generalize to held-out benchmarks and model scales" (Section 5.1) — is supported only for a narrow distributional shift: from one math competition to another math competition with similar problem formats, difficulty ranges, and reasoning patterns. This does not demonstrate generalization to:

• Different task types: Code generation (HumanEval, MBPP), logical reasoning (FOLIO, ARC), commonsense QA (StrategyQA), or open-ended generation tasks where correctness evaluation is less clear-cut.
• Different difficulty profiles: MATH has a characteristic difficulty distribution (competition-level problems that require multi-step reasoning but have clear ground-truth answers). A benchmark with a higher proportion of "easy" problems (where consensus emerges quickly) or "impossible" problems (where the base model never gets the correct answer) might expose different failure modes.
• Different answer formats: MATH problems produce short final answers (numbers, expressions, multiple-choice options). Tasks requiring long-form answers, ranked lists, or structured outputs may not work well with the controller's answer aggregation mechanism (majority voting over completed answers).

The small test set sizes compound this issue. With 30 questions in AIME25, each correctly answered question contributes ~3.3 percentage points to accuracy. The difference between the discovered controller at β = 1.0 (75.8%) and the best baseline (76.7% for SC@64) on Qwen3-8B AIME25 is 0.9 percentage points — less than a single question's worth of accuracy. Without variance estimates, it's unclear whether this difference is statistically meaningful or just sampling noise from the specific 30 questions and the random trajectory subsetting. A difference of 1–2 questions correct out of 30 could easily be within noise, especially when the 64-run averaging only addresses trajectory sampling variance, not benchmark sampling variance.

What evidence exists in the paper. Table 1 reports performance on AIME25 and HMMT25 across all four models, and Table 2 reports performance on GPQA-Diamond for one model. Figure 3 shows scaling curves for selected (model, benchmark) pairs. The GPQA-Diamond result (Table 2) provides the only evidence of non-math generalization, and it is positive: the discovered controller matches SC@64 accuracy while using 47–70% fewer tokens. However, this is a single data point, and the paper does not test on additional non-math benchmarks or with additional models for GPQA. The supplementary material (Appendix C) provides no additional generalization experiments.

Mitigation status. The paper does not discuss the narrow scope of held-out evaluation as a limitation. Appendix A focuses on extending the action space rather than on more comprehensive generalization testing. The GPQA-Diamond result is presented as evidence of generalization to non-math tasks (Section 5.3), but the paper does not acknowledge that it is the only such test. For practitioners considering deployment on non-math tasks, the evidence is suggestive at best: the controller might transfer, based on one positive result, but there is no systematic characterization of which task properties (answer format, difficulty distribution, reasoning requirements) affect transfer and which do not. The paper would be stronger with evaluations on a diverse suite of benchmarks beyond math — code generation, multi-hop QA, logical entailment — to establish the boundaries of where the discovered controller provides benefits.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a paradigm-level reframing of how test-time scaling research is conducted. Before AutoTTS, the dominant workflow was: researcher conceives a heuristic → implements it → tunes thresholds on validation data → evaluates against baselines → publishes. Each paper contributed one point in the space of possible TTS strategies, and the field progressed through the accumulation of individually-designed methods. AutoTTS changes the unit of progress from strategies to environments — the researcher's output is no longer a specific branching/pruning/stopping algorithm, but a structured discovery environment in which many algorithms can be systematically found.

The magnitude of this shift is comparable to what happened in neural architecture search (NAS) when the field moved from hand-designing individual architectures (AlexNet, VGG, ResNet) to designing search spaces and optimization protocols within which architectures are automatically discovered. In both cases, human effort shifts from exploring the solution space directly to shaping the search space so that automated exploration is productive. The key difference — and what makes AutoTTS more than a straightforward application of program synthesis to TTS — is the discovery that the search space must be carefully regularized (beta parameterization) and richly instrumented (execution trace feedback) to avoid catastrophic overfitting to the search set. These insights are not obvious ex ante; they emerged from observing failure modes in preliminary experiments and constitute methodological contributions that transfer to other domains where LLM-driven algorithm discovery faces high-dimensional, brittle search spaces.

The paper resolves a tension that has been brewing implicitly in the TTS literature. On one side, the proliferation of hand-crafted methods (ASC, ESC, ST-BON, Parallel-Probe, Answer Consistency, and many others) demonstrated that allocation matters — different strategies produce different accuracy–cost tradeoffs, and no single strategy dominates across all settings. On the other side, the design process for these strategies was fundamentally unprincipled: each was a product of researcher intuition, and the field had no framework for understanding why one strategy worked where another failed, or for systematically exploring the space of possible strategies beyond what humans could conceive. AutoTTS resolves this tension by showing that the space of effective strategies is large enough to justify automated search, but structured enough (through the width–depth MDP) that search can succeed within a practical budget ($39.9, 160 minutes). The implication is that hand-crafted TTS strategy design — while it has produced valuable methods — is approaching the limits of what manual intuition can achieve, and the field should redirect effort toward building better discovery environments.

This reframing redirects research attention in several concrete ways:

• Strategy design becomes less attractive as a primary contribution. A paper proposing "Adaptive Momentum Pruning with Confidence-Guided Widening" — a new hand-crafted heuristic — would now face the question: can AutoTTS discover this or a better strategy automatically? If the answer is yes, the marginal contribution of manual design is diminished. This does not invalidate hand-crafted methods (Parallel-Probe, which inspired the replay environment design, remains a strong baseline), but it raises the bar: new hand-crafted strategies must demonstrate that they outperform not just prior hand-crafted baselines, but also what automated discovery produces in the same environment.

• Environment design becomes the primary research activity. The paper demonstrates that effective environments require specific properties: an offline replay substrate to make evaluation cheap, a structured MDP to define the control space, a parameterization scheme (beta) to prevent overfitting, and rich feedback (execution traces) to enable diagnostic iteration. Future research can extend this blueprint to richer environments — environments with tree-search actions, verifier-guided refinement, revision-based mechanisms, or multi-modal reasoning — and the quality of the environment (measured by the performance of controllers discovered within it) becomes the new evaluation metric.

• Verifier quality becomes an infrastructure concern for discovery, not just inference. In prior work (Snell et al., 2024), verifier quality was important because it directly affected the accuracy–cost tradeoff of test-time strategies (better verifiers → better beam search → better performance). AutoTTS reveals a second-order effect: verifier quality shapes the discovery landscape. If intermediate probe answers (the ω_{i,k} in the MDP) are noisy or unreliable, the controller receives misleading signals, and the discovery agent may learn to distrust probes entirely or overfit to probe patterns that don't generalize. The paper uses simple answer extraction from partial generations as the probe signal — improving probe quality (e.g., via trained process reward models, as in Lightman et al., 2023) would likely enable the discovery of even stronger controllers by providing more reliable intermediate feedback.

• The pretraining–inference compute tradeoff gains a new dimension. Snell et al. (2024) established that test-time compute can substitute for pretraining compute under certain conditions (easy-to-medium problems, low inference-to-pretraining token ratios). AutoTTS adds a layer to this picture: the discoverability of allocation strategies is itself a function of how the environment is constructed. A well-designed environment might discover strategies that extract more performance per unit of test-time compute, shifting the tradeoff further in favor of inference-time scaling. Conversely, a poorly-designed environment might fail to discover strategies that a human could design, making test-time compute appear less effective than it actually is. The FLOPs-matched comparisons in Snell et al. implicitly assume optimal (or at least strong) allocation strategies; AutoTTS provides a methodology for approaching that optimum.

Follow-Up Research This Work Enables

Online validation of replay-discovered controllers to characterize the simulation-to-reality gap. The paper's central enabling technique — offline replay evaluation — assumes that pre-collected trajectories faithfully represent what the base LLM would generate in response to live controller actions. This assumption is untested. A direct follow-up would evaluate the CMC and a few ablation variants in a live inference setting on AIME25, comparing per-question accuracy and token cost to the replay-predicted values. The key measurement is the replay-to-online accuracy gap and the replay-to-online cost correlation: do controllers that are Pareto-optimal in replay remain Pareto-optimal in live inference, or does the ranking of controllers change? If the gap is small (<2 percentage points accuracy difference, high rank correlation), the replay environment is validated as a reliable discovery substrate. If the gap is large or rank order reverses, it reveals that the path-independence assumption is violated, and future environment designs must incorporate online evaluation (e.g., by periodically re-sampling trajectories from the live model during discovery, or by training a surrogate model of generation dynamics).

Extending the action space to tree search and verifier-guided refinement. The current width–depth MDP provides BRANCH, CONTINUE, PROBE, PRUNE, and ANSWER actions. This covers parallel sampling and adaptive pruning, but excludes richer control structures that have proven effective in prior work: tree search (where branches can spawn sub-branches, creating hierarchical exploration), verifier-guided refinement (where a trained process reward model scores partial trajectories and guides which branches to deepen, as in Snell et al., 2024), and revision-based mechanisms (where the model conditions on previous incorrect answers to produce improved ones). An extended environment would add actions like REFINE(i) (re-generate branch i with previous context as a revision), VERIFY(i) (score branch i's current prefix with a learned verifier and return a scalar), and SPLIT(i) (create a sub-branch from branch i's current prefix, exploring alternative continuations). The discovery loop would remain identical (agent proposes controllers, evaluator replays against pre-collected data, history accumulates), but the offline data collection would need to be expanded: pre-generating revision trajectories, verifier scores at each depth, and tree-structured branching data. A strong follow-up would construct this richer environment for AIME24, run the same five-round discovery protocol, and measure whether the discovered tree-search+verifier controller improves the Pareto frontier beyond what the flat width–depth CMC achieves. The hypothesis is that richer action spaces enable stronger controllers, but the discovery challenge also increases (larger search space, more opportunities for overfitting), testing the robustness of the beta parameterization + execution trace recipe.

Difficulty-conditioned controller families that adapt allocation per-problem. The current CMC deploys the same adaptive strategy regardless of problem difficulty — it has no mechanism to detect that a problem is trivially easy (where aggressive early stopping is safe) or impossibly hard (where further computation is wasted). The Snell et al. (2024) framework demonstrated that difficulty-conditioned allocation yields up to 4× efficiency gains over uniform allocation. A natural synthesis would extend the AutoTTS discovery environment to include difficulty signals as part of the controller state. Concretely, after a small number of initial probes (say, 4–8 branches at shallow depth), compute the average PRM score or the entropy of intermediate answers as a difficulty proxy, and expose this as a state variable difficulty_estimate ∈ [0,1]. The beta schedule would then become hyperparameters = f(beta, difficulty_estimate) — a 2D schedule where the controller can be more aggressive (earlier stopping, fewer branches) on easy problems and more conservative on hard problems. The discovery agent would design both the difficulty estimation mechanism (how to compute the estimate from early probe signals) and the 2D schedule. A strong evaluation would compare the Pareto frontier of the difficulty-conditioned controller against both the uniform CMC and the compute-optimal selection from Snell et al. (which selects among pre-specified strategies per difficulty bin), testing whether automated discovery in a difficulty-aware environment can outperform manual strategy selection.

Stress-testing controller robustness through adversarial benchmark construction. The paper evaluates the CMC on held-out benchmarks (AIME25, HMMT25) from the same distribution family as the search set. This tests benign generalization — transfer within the same task type. An important negative result would test adversarial generalization: can we construct benchmark variants that specifically break the discovered controller while leaving hand-crafted baselines mostly unaffected? Concrete adversarial constructions include: (a) Consensus traps: problems where the base model's most common wrong answer has high confidence across multiple branches, testing whether the EMA momentum gate (which requires high confidence + non-declining trend) can be fooled into stopping on wrong answers; (b) Late divergence: problems where branches agree on intermediate answers for the first 10–15 intervals but diverge near the end, testing whether the alignment-aware depth allocation (which concentrates computation on branches matching the pool winner) prematurely commits to a wrong consensus; (c) Needle-in-haystack: problems where the correct answer appears only in a small fraction of branches (<5%) and requires substantial depth to surface, testing whether the conservative abandonment rule (keep at least 2 alive) preserves the rare correct branch or abandons it in favor of the more common wrong ones; (d) Rapid convergence: trivially easy problems where consensus emerges after 1–2 intervals, testing whether the warm-up requirement and min_complete gate cause unnecessary computation. A finding that the CMC performs poorly on specific adversarial categories would identify structural weaknesses in its design, inform improvements to the discovery prompt (e.g., "the controller should be robust to consensus traps by incorporating answer verification"), and establish safety boundaries for deployment.

Open-source agent replication and agent-dependence analysis. The paper's entire discovery loop relies on Claude Code as the explorer. A critical replication study would replace Claude Code with an open-source coding agent — e.g., DeepSeek-Coder, CodeLlama-70B, or a fine-tuned Llama-3 variant with code editing capabilities — and re-run the same five-round discovery protocol with the identical prompt, search set, and evaluation framework. The primary measurement is the performance gap between the best controller discovered by the open-source agent and the CMC discovered by Claude Code. If the gap is small (<2–3 percentage points accuracy on held-out benchmarks at matched token budgets), it demonstrates that the discovery framework's value is in its structure (replay, beta, traces) rather than in any particular frontier model's capabilities, making AutoTTS accessible to practitioners without API access to proprietary agents. If the gap is large, it characterizes the current dependency and motivates research on improving open-source agents for program synthesis tasks, or on simplifying the discovery prompt to work within the capabilities of weaker agents. A parallel experiment would run the discovery with Claude Code five times with different random seeds (the agent's sampling temperature, the order of trajectory subsetting) to measure the variance in discovered controller performance — is the CMC a typical outcome or a lucky draw? This would establish whether single-run discovery (as reported) is reliable or whether practitioners should run multiple independent discovery runs and ensemble or select among the resulting controllers.

Transfer learning across model families and the amortization of environment construction. The paper shows that a controller discovered on Qwen3 models transfers to DeepSeek-R1-Distill-Llama-8B (Table 2) with minimal accuracy loss and substantial token savings. This raises a broader question: how much does the discovery environment need to be model-specific? A systematic study would construct replay environments for multiple model families (Qwen3, Llama-3, Mistral, Gemma) on the same benchmark (AIME24), discover controllers independently for each family, and measure: (a) within-family generalization: how well does a controller discovered on Qwen3-1.7B transfer to Qwen3-8B? (b) cross-family generalization: how well does a controller discovered on Qwen3-4B transfer to Llama-3-8B? (c) cross-family discovery transfer: if you run discovery on Qwen3-4B to get an initial controller, then fine-tune it with 1–2 additional discovery rounds on Llama-3-8B, does this achieve performance comparable to full discovery on Llama-3-8B from scratch? The findings would characterize the amortization properties of environment construction: if controllers transfer well across model families, a few carefully-constructed environments (one per task type) could serve an entire ecosystem of models, making the data collection cost a true one-time expense. If transfer is poor, practitioners must budget for model-specific discovery, increasing the total cost.

Practical Applications and Downstream Use Cases

Cost-efficient batch evaluation at scale. Organizations that run large-scale LLM evaluation pipelines — grading student answers, scoring candidate solutions in programming competitions, evaluating model outputs during RLHF data collection — can deploy the discovered controller (CMC) at β = 0.5 to achieve comparable accuracy to Self-Consistency@64 while using ~70% fewer tokens. For a pipeline processing 100,000 questions per month with Qwen3-4B, SC@64 at ~1,100K tokens per question (Table 1, held-out average) would consume ~110 billion tokens monthly. The discovered controller at β = 0.5 uses ~350K tokens per question (roughly one-third), reducing the monthly token consumption to ~35 billion — a savings of 75 billion tokens per month, which at typical cloud inference pricing ($0.10–0.50 per million tokens) translates to$7,500–37,500 per month in direct API cost reduction. The one-time discovery cost of $39.9 (plus data collection amortized over many months) is negligible against these operational savings. This use case requires no changes to the existing pipeline beyond swapping the inference strategy from SC@64 to the CMC, and the controller's deterministic replay behavior (once the controller code is fixed, its decisions are reproducible) simplifies debugging and auditing compared to stochastic methods.

On-device deployment with small models for educational applications. The Qwen3-0.6B and Qwen3-1.7B models can run on consumer GPUs or even CPUs with quantization, making them candidates for on-device educational math assistants. The discovered controller at β = 1.0 achieves 31.1% accuracy on AIME25 with Qwen3-0.6B and 49.0% accuracy on AIME25 with Qwen3-1.7B (Table 1), compared to 28.9% and 44.4% for SC@64. While these absolute accuracies are modest (AIME problems are competition-level and difficult), for easier problem sets (e.g., high school algebra, SAT math), the base model's pass@1 would be substantially higher, and the controller's adaptive allocation would likely push accuracy significantly above what majority voting achieves. The practical benefit is that a 1.7B model running on a laptop with the CMC controller could match or exceed the accuracy of a 1.7B model with SC@64 — which requires 64× the computation — making real-time interactive math tutoring feasible without cloud API calls, data privacy concerns, or latency from network round-trips. The controller's behavior at different β values provides a natural "quality slider": β = 0.5 for quick, low-cost answers (suitable for practice problems where occasional errors are acceptable); β = 1.0 for high-stakes answers (exam preparation, graded assignments). This is a concrete deployment scenario where the paper's generalization evidence (transfer to held-out benchmarks without re-tuning, transfer across model scales) directly reduces the engineering burden of per-model, per-task strategy design.

Automated strategy re-discovery for model updates in continuous deployment. In production LLM systems, base models are frequently updated — new fine-tuning runs, RLHF iterations, or model version bumps change the model's output distribution, error patterns, and calibration. Hand-crafted TTS strategies tuned for the previous model version may become suboptimal (or even counterproductive) for the new version, requiring manual re-tuning that is labor-intensive and slow. AutoTTS provides a push-button re-discovery pipeline: when a new model checkpoint is released, collect 128 trajectories per question on the search set (a one-time cost of ~20 million tokens for a 30-question set like AIME24), run the five-round discovery loop ($39.9, 160 minutes, fully automated), and deploy the re-discovered controller. The entire process from model release to updated strategy deployment could complete in under 3 hours with minimal human intervention (only the initial data collection trigger). This turns strategy maintenance from a manual research task into an automated CI/CD step, analogous to how hyperparameter optimization is integrated into ML training pipelines. The$39.9 per update cost is trivially small compared to the engineering time saved, and the 160-minute turnaround is fast enough to keep pace with weekly or even daily model updates.

Test-time compute allocation as a service for multi-tenant inference platforms. Cloud inference platforms (e.g., Together AI, Fireworks, Anyscale) serve diverse customers with different accuracy requirements, latency constraints, and cost sensitivities. A platform could offer AutoTTS-discovered controllers as a configurable inference backend: customers specify their desired β value (or equivalently, their accuracy–cost tradeoff preference) via an API parameter, and the platform routes requests through the appropriate controller variant. The platform would construct the replay environment once per supported model, run discovery to produce a β-parameterized controller family, and expose the β knob to customers. A customer running batch evaluation with relaxed accuracy requirements might set β = 0.3 (extreme efficiency), while a customer building a high-stakes financial analysis tool might set β = 0.9 (near-peak accuracy). The platform benefits from the 40–70% token reduction (translating to lower GPU costs) while offering customers fine-grained control over the accuracy–cost tradeoff without requiring them to understand or tune the underlying TTS mechanisms. The multi-model search set approach (discovering on multiple model scales simultaneously, as the paper does with all four Qwen3 variants) would produce a controller family that works across the platform's model catalog, reducing the per-model discovery burden.