ArXiv: 2601.08605

🎯 Pitch

ExpSeek shows that a small 4B model can significantly boost a larger web agent's performance—but only when guidance is triggered on demand. The magic is in using the agent’s own step-level entropy to sense confusion, dynamically fetching tailored tips exactly when and where uncertainty spikes, rather than drowning the model in upfront advice.

1. Executive Summary

This paper proposes ExpSeek, a self-triggered experience-seeking framework that shifts experience intervention in web agents from passive global injection to proactive, step-level guidance. Evaluating on GAIA, WebWalkerQA, xbench-DeepSearch, and Seal-Hard using Qwen3-8B and Qwen3-32B agents, ExpSeek uses the model's own step entropy as an intrinsic signal to determine intervention timing (via logistic regression with bootstrap-derived threshold intervals) and an experience model to dynamically generate contextualized guidance from a repository of experience triplets (behavior–mistake–guidance tuples grouped by topic, extracted from paired successful and failed trajectories). ExpSeek achieves absolute improvements of 9.3% and 7.5% over vanilla ReAct on the 8B and 32B models respectively, substantially outperforming passive experience injection baselines, while establishing that even a 4B experience model can significantly boost larger agent performance — but only when guidance is triggered adaptively at steps where the model exhibits high uncertainty rather than injected uniformly as global context.

2. Context and Motivation

The Core Problem: Web Agents Need Help, but Not All the Time

The fundamental question this paper tackles is deceptively simple: when a web agent is searching for information to answer a complex question, when should it receive guidance from past experience, and what should that guidance say? Existing approaches dump all available experience into the agent's prompt before it starts working — essentially handing it a thick manual and saying "good luck." ExpSeek argues this is both inefficient and often counterproductive. The agent needs guidance at the right moments — when it's confused — and that guidance needs to be tailored to the current situation, not a generic list of tips.

This gap matters because web agents are increasingly deployed for real-world information-seeking tasks where reliability is critical. The paper opens Section 1 with a vivid description of the challenge:

"the open web is noisy and partially observable with sparse useful evidence, posing the challenges to the agent's reliability. Agents powered by LLMs, particularly small-scale, cost‑effective models, often exhibit inefficient exploration in multi‑turn interactions with the environment or respond prematurely, resulting in unreliable answers"

In plain terms: web agents have to navigate a messy, ever-changing landscape of search results and web pages where relevant evidence is buried in noise. Small models — which are attractive for cost and deployment reasons — are especially prone to two failure modes. They either wander inefficiently, chasing irrelevant leads, or they give up too early and produce incorrect answers based on incomplete information. This is not merely a benchmark problem; it has real-world stakes for any application where users rely on agent-generated answers constructed from web evidence, from research assistants to customer-facing search tools.

Two Failure Modes of Prior Experience-Based Approaches

The paper identifies a specific pathology in how existing methods inject experience into agent reasoning. To understand why, we need to first understand what experience looks like in these systems.

What is experience? Experience, in the web agent context, refers to distilled insights from past interaction trajectories — records of what the agent did, what it observed, and whether it ultimately succeeded or failed. These are not raw logs but structured reflections: "In this situation, I made this mistake because I relied on search snippets rather than visiting the actual page; next time, I should verify claims by accessing the primary source." Experience serves as long-term memory, distinct from short-term contextual memory (the current conversation history).

Prior work falls into two camps (reviewed in Section 2.1):

Offline refinement methods (ExpeL, RaDA, Synapse, Agent KB) process training trajectories after the fact, extract reusable patterns, and retrieve relevant cases at inference time. These are essentially case-based reasoning systems: store examples of successful reasoning, find similar situations, and inject the stored insights.

Online self-evolution methods (Agent Workflow Memory, Contextual Experience Replay, ReasoningBank) accumulate experience through iterative interaction and feedback, building up a repository that improves over time through continuous updates, sometimes modifying the agent's behavior through gradient-free training or distributional shifts.

Both families share a critical design choice that the paper argues is the root problem: experience is passively injected as global context before task execution begins. Figure 1 (Panel A) illustrates this: the experience repository is consulted once at the start, and the retrieved experience is concatenated to the initial prompt. The agent then runs its entire trajectory with this static context.

This leads to two specific failure modes:

Failure mode 1: Experience is disconnected from actual reasoning. The paper offers a reflective analysis in Appendix B.2:

"Most experience repository construction methods meticulously process training trajectories, perform multiple rounds of careful denoising, and cautiously distill core experience items... However, the construction process itself is challenging, and when such heavily processed experiences are given to reasoning agents, the agents may not even understand some abstract expressions."

This is a subtle point. The process of extracting "clean" experience — removing noise, generalizing patterns, distilling insights — can strip away the very contextual richness that makes experience useful. The result is abstract guidance that the agent struggles to interpret or apply to its concrete situation. The paper contrasts this with human experience recall, where "the amount of information retrieved instantly is enormous, including scenes, behavioral details, and even emotions, far exceeding simplified experience items."

Failure mode 2: Static experience cannot adapt to changing contexts. Even when the injected experience is relevant and understandable, it sits frozen in the initial prompt while the agent's situation evolves across multiple interaction turns. The paper states in Section 1:

"During the agent's interaction with the environment, the contextual observations continuously change. If the agent relies solely on the initial context without proactively acquiring and integrating fresh experience, its decision-making may become suboptimal or even misaligned with the current situation."

Concretely: an agent might receive useful experience at step 0 about how to verify sources, but by step 7 — after visiting several pages and following tangents — the original guidance may no longer be salient, or new forms of confusion may arise that the initial context didn't address. The agent has no mechanism to ask for help when it needs it; it must work with whatever was provided at the start.

The paper provides direct empirical evidence for these limitations. In Table 2, the two passive experience baselines — Training-Free GRPO and ReasoningBank+ — show improvements under 3% or even performance degradation across benchmarks:

"Both baseline types struggle with challenging open web reasoning tasks, showing improvements under 3% or even performance degradation. This indicates that carefully designed global experience injection struggles to adapt to noisy web environments while adding reasoning burden to smaller agents."

This last phrase is key: passive experience injection can actually hurt by adding "reasoning burden" — the agent must process a longer context containing potentially abstract, decontextualized advice that may distract from the current task.

The Contradiction Between Experience Value and Experience Delivery

The paper is motivated by a genuine tension in the literature. On one hand, there is broad agreement that experience should help: agents, like humans, learn from accumulated interaction history (Silver and Sutton, 2025, cited in the opening of Section 2.1). Prior work has demonstrated that structured experience — when properly aligned to the task — can improve agent performance across a range of domains.

On the other hand, the empirical results with web agents tell a different story. The carefully designed global injection methods produce negligible or negative gains. This is not a failure of experience per se, but a failure of when and how experience is delivered. The paper's core intuition, expressed in Section 1, is to flip the paradigm:

"Compared with passively injecting experience, why not empower the agent to proactively seek experience during its interaction with the environment for more precise guidance?"

This reframing raises two immediate sub-problems that the paper must solve: (1) How does the agent know when it needs help? — it cannot simply ask for experience at every step, both for efficiency and because over-intervention can be as harmful as under-intervention; and (2) What form should that help take? — retrieved raw experiences may be too generic, while hand-crafted rules may miss nuance.

Why Entropy? The Missing Piece

The paper's choice of entropy as the self-trigger signal is not arbitrary — it builds on an established but previously separate line of research on uncertainty quantification in LLMs (Section 2.2).

Entropy has been extensively used for static evaluation: detecting hallucinations (Farquhar et al., 2024), calibrating confidence (Chen and Mueller, 2024), and selecting answers (Ren et al., 2023). More recently, it has been extended to multi-step reasoning where responses are decomposed into atomic steps, particularly in reinforcement learning settings. Entropy reflects sampling diversity and facilitates exploration during training (Wang et al., 2025c; Zheng et al., 2025), and acts as a fine-grained signal for step-level credit assignment in long-horizon reasoning (Wang et al., 2025b). Critically, Dong et al. (2025) showed entropy can incentivize exploration across multi-turn interactions — the exact setting of web agents.

What was missing was the connection: no prior work had used the agent's own step-level entropy as a real-time trigger for deciding when to seek external guidance during deployment. The paper identifies this gap explicitly:

"This demonstrates the potential of entropy as a self-trigger signal for web agents." (Section 2.2, final sentence)

The motivation is cleanly stated. Entropy is intrinsic — it comes from the agent model itself, requiring no external reward model, no separate critic network, no expensive API calls. It is computed from the token-level probability distributions that the model already produces during autoregressive generation. If entropy can reliably distinguish steps where the agent is on-track from steps where it's floundering, it becomes a "free" guidance trigger. But the paper must establish whether it can do so, which it tackles in Section 4.2.1 and Figure 3.

How This Paper Positions Itself

The paper positions ExpSeek as a paradigm shift from passive to proactive experience intervention, formalized in the framework of Section 3. The key conceptual move is in the definition of the utilization function G\mathcal{G}:

Traditional methods implement experience intervention as:

e=G(E,q)e = \mathcal{G}(\mathcal{E}, q)

where E\mathcal{E} is the experience base and qq is the query. Experience is retrieved once based on the question text alone, producing static guidance ee that persists unchanged throughout the trajectory.

ExpSeek redefines intervention as:

et=G(E,ht)e_t = \mathcal{G}(\mathcal{E}, h_t)

where hth_t is the full interaction history up to step tt. The mapping function G\mathcal{G} is now invoked at any step to provide fresh, contextualized guidance based on the current state.

This is not a minor modification — it changes the role of experience from a background resource to an active collaborator. The experience model becomes a kind of supervisor that monitors the agent's progress and provides feedback when the agent's own confidence (as measured by entropy) drops below a threshold. The paper explicitly connects this to the metaphor of human learning: a student who actively asks questions when stuck learns more effectively than one who reads a static study guide before starting.

The paper does not claim to have solved all problems — Section 8 (Limitations) is candid about dependence on training set quality for threshold estimation, the unexplored extension to non-web domains, and the potential for integration with agentic RL training. But the core contribution is clearly scoped: a method for when to intervene and what intervention to provide, validated across four challenging benchmarks with consistent gains over both no-experience and passive-experience baselines.

3. Technical Approach

3.1 Reader Orientation

ExpSeek is a two-component system consisting of (1) a self-triggering mechanism that uses the web agent's own token-level prediction entropy to decide when to ask for help at each interaction step, and (2) an experience model that dynamically generates step-specific, contextualized guidance by retrieving and adapting structured experience triplets from a pre-built repository. The system solves the problem of when and how to inject past experience into a running web agent — instead of dumping all experience into the initial prompt (the prevailing paradigm), ExpSeek monitors the agent's internal uncertainty signal in real time and provides tailored coaching only when the agent appears confused, balancing guidance frequency against agent autonomy.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components connected in a pipeline:

  1. Agent Model (ℳ_a) — the web agent itself (e.g., Qwen3-8B, Qwen3-32B), which follows the ReAct framework to interleave reasoning thoughts and tool-calling actions across multiple steps. It produces responses token-by-token with associated probability distributions.

  2. Entropy Monitor — a lightweight computation layer that, for each agent response $R_t$, computes the average token-level entropy $\bar{H}_t$ (Equation 2) from the logits the agent already produces. This is the intrinsic signal — no separate model, no extra inference.

  3. Trigger Decision Module — takes $\bar{H}_t$ and maps it to an intervention probability $p_{\text{intervene}}$ (Equation 6) using pre-computed threshold intervals (Table 1), which were derived offline via bootstrap logistic regression on training trajectories (Algorithm 1). The decision is probabilistic in the threshold region to balance intervention frequency with agent autonomy.

  4. Experience Base (ℰ) — a structured repository of experience triplets, each containing a Behavior description, a Mistake analysis, and Guidance cues, organized into topic groups (e.g., 17 topics for process steps and 11 for answer steps with the 8B agent). Built offline from paired successful and failed training trajectories using a tool model.

  5. Experience Model (ℳ_e) — a separate LLM (e.g., Qwen3-235B-A22B-Instruct-2507, or smaller variants down to 4B) that, when a trigger fires, selects the three most relevant topics from the experience base based on the current context $h_t$, then generates fresh, step-specific guidance $e_t$ by adapting the stored triplets to the ongoing situation.

Information flows as follows: the agent produces step $t$ → entropy is computed → if $p_{\text{intervene}}$ triggers and the previous step was not intervened → the experience model reads the current context $h_t$, selects topics, retrieves related triplets, and generates guidance $e_t$ → guidance is appended to the observation $O_t$ (for process steps) or treated as a new observation $O_T$ enabling an extra step (for answer steps) → the agent continues with augmented context → intervention is suppressed at step $t+1$ to allow the agent to incorporate guidance.

3.3 Roadmap for the Deep Dive

  • First, the experience base construction pipeline — how triplets are created from paired trajectories, what the three fields contain, how topics are induced, and why this structured form matters. This is the knowledge foundation that all guidance draws from.

  • Second, the entropy analysis that justifies using entropy as a trigger signal — the statistical evidence that correct and incorrect steps have separable entropy distributions, with process and answer steps showing different separability patterns (Figure 3 and the KS test results).

  • Third, the threshold estimation procedure — the logistic regression formulation, the bootstrap resampling algorithm, how the $[\theta_{\text{lower}}, \theta_{\text{upper}}]$ intervals are computed, and why a probabilistic (rather than hard) trigger is used.

  • Fourth, the guided intervention at inference — how the experience model operates at test time: topic selection, guidance generation, and the distinct injection mechanisms for process versus answer steps.

  • Fifth, the design choices and their justifications — why bootstrap logistic regression over a learned classifier, why generative guidance over retrieval, why topic-based organization over flat retrieval, and the "one-step silence" rule.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and methods paper whose core idea is that web agents should proactively seek experience guidance at moments of high uncertainty (self-triggered by entropy), rather than passively receiving global experience before execution, and that the guidance content should be dynamically generated for the specific step context rather than retrieved as static templates.

Experience Base Construction

The experience base $\mathcal{E}$ is the static knowledge repository from which the experience model draws. It is built offline from training trajectories and organized for efficient topical retrieval at inference time. The construction has three stages: trajectory pairing, triplet generation, and topic induction.

Guiding Experience Schema. Each entry in the experience base is a triplet containing three fields:

  • Behavior: An objective description of what the agent did at a specific step — the state it observed and the action it took. This is not an evaluation; it is a neutral recording of the agent's behavior.

  • Mistake: An identification of what went wrong at that step, derived by contrasting the failed trajectory with a successful one for the same query. The mistake analysis pinpoints why the behavior led to failure, not just that it did.

  • Guidance: Directional advice based on the error analysis, formulated as suggestions for what the agent should have done differently. Critically, the guidance does not provide direct answers or specific clues — it offers process-level redirection (e.g., "verify claims by visiting authoritative sources rather than relying on snippets").

The paper provides concrete examples in Table 6 (Appendix B.4). For process steps, the Behavior might describe what source was consulted, the Mistake identifies that the agent relied on search snippets rather than authoritative pages, and the Guidance suggests prioritizing official sources. For answer steps, the Mistake might flag failure to faithfully reproduce complete proper nouns, and the Guidance emphasizes detail verification and trajectory extension when needed.

The design motivation for this triplet structure (explained in Appendix B.4) is that it "reproduces step-level error patterns from training trajectories and provides targeted guidance, enabling the experience model to more accurately guide both process and final steps during test inference." The three fields serve different functions: Behavior provides context for retrieval relevance, Mistake enables the experience model to diagnose similar situations, and Guidance offers actionable direction without giving away the answer.

Construction Process (Figure 2, Panel A). The construction proceeds through three sequential stages:

Stage 1: Trajectory sampling and pairing. For each query $q$ in the training corpus $\mathcal{D}_{\text{train}}$, the agent model $\mathcal{M}_a$ samples $k = 5$ complete trajectories at temperature 1.0 and top-p 0.95 (the same generation parameters used at test time). Each trajectory $\tau$ is a sequence of steps $S_1, S_2, \ldots, S_T$ where each step $S_t$ is either a process step $S_t^p = (R_t, O_t)$ (containing the agent's response and the environment observation) or a final answer step $S_T^a = R_T$ (containing only the final answer). The trajectories are scored against the ground-truth answer $y$ to label them as successful ($\tau^+$, the agent reached the correct answer) or failed ($\tau^-$, the agent did not).

For each failed trajectory, it is paired with a successful trajectory for the same query to enable contrastive analysis. This pairing is the critical design choice: the mistake analysis is not based on abstract heuristics but on a direct comparison with what a correct solution looks like for the same problem.

Stage 2: Step-level mistake analysis and triplet generation. A tool model (Qwen3-235B-A22B-Instruct-2507 in the main experiments) processes each pair $(\tau^+, \tau^-)$. For each step $S_t$ in the failed trajectory $\tau^-$, the tool model:

  • Assigns a binary correctness label $y_t \in \{0, 1\}$ indicating whether the step was correct or erroneous, using the successful trajectory as a reference for what correct behavior looks like at that stage of the task.
  • For incorrect steps (where $y_t = 0$), generates a triplet containing the Behavior, Mistake, and Guidance fields.

The prompt for this stage is provided in Table 12 (Appendix D). The tool model is instructed to analyze systematically, identifying not just surface-level errors but underlying reasoning failures.

The paper reports the resulting repository sizes: for the 8B agent model, $|\mathcal{E}_p| = 196$ triplets across 17 topics for process steps, and $|\mathcal{E}_a| = 190$ triplets across 11 topics for answer steps. For the 32B agent model, $|\mathcal{E}_p| = 276$ triplets across 18 topics and $|\mathcal{E}_a| = 143$ across 23 topics. These are modest numbers — the entire repository for the 8B agent fits in under 400 structured entries — which is feasible for practical deployment and topic-based retrieval.

Stage 3: Topic induction. The tool model is prompted (Table 13, Appendix D) to organize the triplets into topic groups using an iterative batch processing approach:

"when processing each new batch, the model takes all previously generated triplets with their assigned topics as input, then either assigns existing topics, modifies them, or creates new topics for the current batch."

This means topics are not pre-defined by human annotators — they emerge bottom-up from the data. The process is incremental: as new triplets arrive, the model can either slot them into existing topics (e.g., "source verification"), refine an existing topic's scope, or create a new topic if the pattern doesn't fit any existing category. The final topics represent common failure modes distilled from the training data.

The topic structure serves two purposes: (1) it enables efficient retrieval at inference time — the experience model selects among ~15-20 topics rather than over ~200 individual triplets; and (2) it groups related experiences so the generative guidance can draw from multiple exemplars of the same failure pattern, producing richer, more generalized advice.

Why not human annotation? The paper implicitly addresses this through the design: the construction pipeline is fully automated, requiring only training queries with ground-truth answers. This makes it scalable and model-specific — the triplets are generated by analyzing the actual behavior of the deployed agent model, so the mistakes captured are real failure modes of that specific model, not hypothetical errors a human might anticipate.

Why contrastive pairs? The pairing of successful and failed trajectories for the same query is the mechanism for ensuring that mistake identification is grounded rather than speculative. The tool model doesn't have to guess what correct behavior looks like — it can read the successful trajectory and identify exactly where the failed one diverged. This contrastive signal is what makes the "Mistake" field specific and actionable rather than generic.

Entropy Analysis for Step Correctness

Before building the trigger mechanism, the paper establishes the empirical foundation: can entropy distinguish correct from incorrect steps in web agent reasoning? This is the question answered by Figure 3 and the associated statistical tests in Section 4.2.1.

Computing step entropy. The entropy computation is defined in Equation 2 of Section 3. Recall that the agent produces its response $R_t$ token-by-token. For the $i$-th token $x_i$ with preceding context $h_i$ (all prior tokens and observations), the model computes a probability distribution $P(v \mid h_i)$ over the vocabulary $\mathcal{V}$. The token-level entropy quantifies how "spread out" this distribution is — whether the model is strongly committed to a few tokens (low entropy) or uncertain across many tokens (high entropy):

H(xi)=−∑v∈VP(v∣hi)log⁡P(v∣hi)H(x_i) = -\sum_{v \in \mathcal{V}} P(v \mid h_i) \log P(v \mid h_i)

where $H(x_i)$ measures the uncertainty in the model's prediction for token $x_i$. The step entropy $\bar{H}_t$ is simply the arithmetic mean of token-level entropies over the entire response:

Hˉt=1∣Rt∣∑x∈RtH(x)\bar{H}_t = \frac{1}{|R_t|} \sum_{x \in R_t} H(x)

where $|R_t|$ is the number of tokens in the response $R_t$.

What it computes: The average predictive uncertainty of the agent model across all tokens in its step-$t$ response. When the model is confident — producing tokens with high probability — individual $H(x)$ values are low, and $\bar{H}_t$ is low. When the model is hesitant — spreading probability mass across many alternatives — entropies are high, and $\bar{H}_t$ is high.

Why this form: Averaging across tokens captures the overall uncertainty of the step rather than being sensitive to single-token fluctuations (e.g., a low-entropy token like "the" during a high-entropy reasoning passage). The arithmetic mean treats all tokens equally, which is appropriate since the agent's confidence should be reflected across the entire response, not just at key words. Alternative aggregations like the maximum or minimum token entropy would be dominated by outliers.

Data construction for analysis. The paper constructs two labeled step collections from the training trajectories (Equation 3):

S+={St∈τ+}∪{St∈τ−∣yt=1}\mathcal{S}^+ = \{S_t \in \tau^+\} \cup \{S_t \in \tau^- \mid y_t = 1\} S−={St∈τ−∣yt=0}\mathcal{S}^- = \{S_t \in \tau^- \mid y_t = 0\}

where $\mathcal{S}^+$ is the set of correct steps — all steps from successful trajectories plus any steps from failed trajectories that were labeled correct by the tool model (steps where the agent was on the right track before a later mistake). $\mathcal{S}^-$ is the set of incorrect steps — only the steps from failed trajectories that the tool model labeled as erroneous.

These collections are further partitioned by step type into $\mathcal{S}^+_p, \mathcal{S}^-_p$ (process steps) and $\mathcal{S}^+_a, \mathcal{S}^-_a$ (answer steps). The paper then computes $\bar{H}_t$ for every step in each collection and visualizes the distributions (Figure 3).

Key finding 1: Entropy distinguishes correct from incorrect steps. Figure 3 shows the entropy distributions for Qwen3-8B on the WebWalkerQA training set. For both process and answer steps, the distribution of $\mathcal{S}^+$ (blue) is shifted left (lower entropy) compared to $\mathcal{S}^-$ (orange). The Kolmogorov-Smirnov test confirms statistical significance:

"process steps: KS = 0.1998, p < 0.001; answer steps: KS = 0.3809, p < 0.001"

The KS statistic measures the maximum vertical distance between two empirical CDFs — larger values indicate greater separation. Both are significant at $p < 0.001$, confirming that the distributions are genuinely different, not an artifact of sampling.

Key finding 2: Answer steps are much more separable than process steps. This is the critical nuance. The paper reports AUC (Area Under the ROC Curve) values: 0.6223 for process steps versus 0.7187 for answer steps. AUC of 0.5 means random discrimination; 0.62 is "weak" and 0.72 is "acceptable" by conventional standards (Bradley, 1997). The paper explicitly interprets this:

"process steps show considerable overlap as agents naturally explore multiple paths, producing high entropy even when correct"

This makes intuitive sense. During process steps, the agent might be exploring legitimate alternatives — considering different search queries, evaluating multiple sources — and this exploration naturally produces higher entropy even when the step is ultimately correct. The agent might think "I could search for X, or Y, or Z" and this deliberation shows up as elevated entropy without indicating an error.

In contrast, answer steps are more decisive — the agent must commit to a specific answer, and if it's confident in the correct answer, entropy drops sharply. If it's uncertain or guessing wrong, entropy stays high. This makes entropy a much cleaner signal for answer steps.

Design implications. The differential separability means that a single entropy threshold would not work well. A threshold calibrated to catch errors in answer steps (where separation is good) would over-trigger on process steps (where many correct steps have elevated entropy). Conversely, a threshold calibrated for process steps would miss many answer-step errors. The paper's solution — separate threshold intervals for process and answer steps, derived independently from their respective distributions — is a direct response to this finding.

Figure 8 (Appendix) shows the 32B model's entropy distributions. The same pattern holds, but with an important difference: the process-step distributions for $\mathcal{S}^+$ and $\mathcal{S}^-$ show even more overlap for the 32B model (visually, the blue and orange curves are nearly superimposed in the middle range). The paper notes that this leads to a "predominance of the yellow region" (the probabilistic trigger zone), meaning the algorithm adaptively randomizes trigger decisions when separation is poor.

Threshold Estimation via Bootstrap Logistic Regression

Given the entropy distributions for correct and incorrect steps, the paper needs a principled way to convert a raw entropy value $\bar{H}_t$ into a trigger decision. The approach has two levels: (1) fit a logistic regression to model the probability that a step is incorrect given its entropy, and (2) use bootstrap resampling to estimate the uncertainty in the decision boundary, producing a threshold interval rather than a single hard threshold.

Level 1: Logistic regression. The paper frames the problem as binary classification: predict $y_t \in \{0, 1\}$ (0 = correct, 1 = incorrect) from the single feature $\bar{H}_t$. The logistic regression model learns:

P(yt=0∣Hˉt)=11+e−(w⋅Hˉt+b)P(y_t = 0 \mid \bar{H}_t) = \frac{1}{1 + e^{-(w \cdot \bar{H}_t + b)}}

where $w$ is a scalar weight, $b$ is a scalar bias term, and the expression computes the probability that the step is incorrect (note: the equation in the paper has $y_t = 0$ for incorrect, which is consistent with the convention that $\mathcal{S}^-$ steps are labeled 0 — wait, let me re-read. The paper says:

"P(y_t = 0 \mid \bar{H}_t) = \frac{1}{1 + e^{-(w \cdot \bar{H}_t + b)}}, modeling the probability of incorrectness"

So $y_t = 0$ means incorrect, $y_t = 1$ means correct in the training data construction. The logistic regression outputs the probability of being incorrect.

What it computes: Given a step entropy value, the model outputs a scalar between 0 and 1 representing the estimated probability that this step is erroneous. The sigmoid function $\sigma(z) = 1/(1+e^{-z})$ maps the linear combination $w\bar{H}_t + b$ to $(0,1)$. The decision boundary (where $P = 0.5$) occurs when $w\bar{H}_t + b = 0$, yielding the threshold:

θ=−b/w\theta = -b/w

Why this form: Logistic regression is chosen for two reasons. First, it's the simplest well-calibrated probabilistic classifier — crucial because the paper uses the output probability for probabilistic triggering (Equation 6), not just the binary decision. Second, with a single input feature, the model has exactly two parameters $(w, b)$, making bootstrap estimation computationally trivial (seconds on CPU) while still capturing the monotonic relationship "higher entropy → higher error probability" that the distributions in Figure 3 suggest. A more complex model (neural network, gradient boosting) would risk overfitting to the modest step counts (~thousands of steps) and would not provide the clean, interpretable threshold geometry that the bootstrap procedure exploits.

Level 2: Bootstrap confidence intervals. A single logistic regression fit on limited training data yields an unstable threshold — small changes in which examples are included could shift $\theta$ meaningfully. To quantify this uncertainty, the paper uses bootstrap resampling (Algorithm 1, Appendix A.1):

  1. For $N = 1000$ iterations, sample with replacement from $\mathcal{S}^+$ and $\mathcal{S}^-$ (preserving their original sizes) to create bootstrap replicates.
  2. Fit a logistic regression on each replicate to obtain $\theta^{(i)} = -b_i/w_i$.
  3. Form the distribution $\Theta = \{\theta^{(1)}, \ldots, \theta^{(N)}\}$.
  4. Compute the 2.5th and 97.5th percentiles to get a 95% confidence interval:

θlower=Q0.025(Θ)\theta_{\text{lower}} = Q_{0.025}(\Theta) θupper=Q0.975(Θ)\theta_{\text{upper}} = Q_{0.975}(\Theta)

where $Q_p$ is the $p$-th quantile function.

This procedure is performed independently for process steps (yielding $[\theta_{\text{lower}}^p, \theta_{\text{upper}}^p]$) and answer steps (yielding $[\theta_{\text{lower}}^a, \theta_{\text{upper}}^a]$).

The resulting intervals are reported in Table 1. For Qwen3-8B (the values are visible in the paper's Table 1), the process-step interval $\theta_{\text{lower}}^p$ is lower than the answer-step interval $\theta_{\text{lower}}^a$, reflecting the fact that process steps naturally have higher entropy even when correct (as seen in Figure 3).

Why bootstrap instead of a parametric confidence interval? The standard approach — computing the standard error of $\theta$ from the Hessian of the log-likelihood — assumes the sampling distribution of $\theta$ is approximately normal. With a single predictor and potentially non-linear relationships between entropy and correctness, this assumption may not hold. Bootstrap makes no distributional assumptions; it directly estimates the empirical sampling distribution of $\theta$ by resampling the data. The paper's choice of 1000 iterations is standard for percentile bootstrap confidence intervals.

Why a confidence interval rather than a single threshold? This is the key design insight. A hard threshold would make two types of errors: (1) classifying low-entropy incorrect steps as correct (false negatives — missing interventions when needed), and (2) classifying high-entropy correct steps as incorrect (false positives — unnecessary interventions). The $[\theta_{\text{lower}}, \theta_{\text{upper}}]$ interval represents the region of uncertainty — entropy values where the model cannot confidently distinguish correct from incorrect. The paper's probabilistic intervention scheme (Equation 6) exploits this: steps below $\theta_{\text{lower}}$ are clearly correct (no intervention), steps above $\theta_{\text{upper}}$ are clearly incorrect (always intervene), and steps in between are handled probabilistically, with the intervention probability increasing linearly with entropy. This balances the tradeoff between missing necessary interventions and over-intervening.

Self-Triggered Intervention at Inference (Equation 6)

At inference time, for each step $S_t$, the agent model produces a response $R_t$ with associated token probabilities. The entropy monitor computes $\bar{H}_t$ (Equation 2), and the trigger decision module maps it to an intervention probability:

pintervene={0Hˉt<θlowerHˉt−θlowerθupper−θlowerθlower≤Hˉt≤θupper1Hˉt>θupperp_{\text{intervene}} = \begin{cases} 0 & \bar{H}_t < \theta_{\text{lower}} \\[4pt] \frac{\bar{H}_t - \theta_{\text{lower}}}{\theta_{\text{upper}} - \theta_{\text{lower}}} & \theta_{\text{lower}} \leq \bar{H}_t \leq \theta_{\text{upper}} \\[4pt] 1 & \bar{H}_t > \theta_{\text{upper}} \end{cases}

where for process steps, the interval $[\theta_{\text{lower}}^p, \theta_{\text{upper}}^p]$ is used; for answer steps, $[\theta_{\text{lower}}^a, \theta_{\text{upper}}^a]$ is used.

What it computes: A scalar in $[0, 1]$ representing the probability that the experience model will be invoked for this step. The three cases correspond to three regimes: (1) low entropy → high confidence → no need for guidance; (2) intermediate entropy → uncertain → probabilistic guidance, with probability increasing linearly as entropy rises; (3) high entropy → low confidence → almost certainly needs guidance.

Why this form: The piecewise linear function has three desirable properties. First, it is monotonic in entropy — higher uncertainty always means higher intervention probability, aligning with the logistic regression finding that entropy and error probability are positively correlated. Second, it is smooth at the boundaries — the function is continuous at $\theta_{\text{lower}}$ (0 to 0) and $\theta_{\text{upper}}$ (1 to 1), avoiding discontinuous jumps in intervention behavior. Third, the linear interpolation in the middle region provides graduated response — a step with entropy just above $\theta_{\text{lower}}$ has a small chance of intervention (e.g., 10-20%), while a step near $\theta_{\text{upper}}$ is very likely to trigger (e.g., 80-90%). This avoids the brittleness of a hard threshold where entropy 0.501 triggers intervention and 0.499 does not.

A hard-threshold alternative — intervene if and only if $\bar{H}_t > \theta$ — would be simpler but would lose the uncertainty quantification that the bootstrap interval provides. The probabilistic scheme recognizes that in the $[\theta_{\text{lower}}, \theta_{\text{upper}}]$ region, the model genuinely cannot tell whether the step is correct or incorrect, so it's appropriate to sometimes intervene and sometimes not, in proportion to the estimated risk.

The "one-step silence" rule. To prevent the agent from being bombarded with guidance, the paper implements a cooldown mechanism: after an intervention at step $t$, no intervention is allowed at step $t+1$. This gives the agent one full step to process and act on the guidance before potentially receiving more. The paper justifies this as preventing "over-intervention" — the agent needs time to incorporate the feedback rather than being interrupted at every turn.

Empirical trigger rates. Table 3 provides concrete numbers. On GAIA, the entropy-based trigger results in 1.0× the step count and 1.0× the time of the vanilla agent (i.e., minimal overhead), with a 69.3% rejection rate (the proportion of trigger checks that result in non-intervention). On the more challenging xbench, the trigger rate increases by 25.6% — meaning the system adaptively intervenes more frequently on harder problems — while maintaining similar accuracy. This adaptive behavior is a key advantage: the trigger rate is not a fixed hyperparameter but emerges from the agent's own uncertainty signal.

Guided Intervention: How the Experience Model Operates

When the trigger fires at step $S_t$, the experience model $\mathcal{M}_e$ is invoked. Its operation has two stages: topic selection and guidance generation.

Stage 1: Topic selection. The experience model reads the full interaction history up to step $t$, denoted $h_t = (q, R_1, O_1, \ldots, R_t, O_t)$ for process steps or $h_T = (q, \ldots, R_T)$ for answer steps. It selects the three most relevant topics from the appropriate repository ($\mathcal{E}_p$ for process steps, $\mathcal{E}_a$ for answer steps). The selection prompt is in Table 14 (Appendix D).

The paper does not provide detailed retrieval mechanics, but the description in Section 4.3 implies a semantic selection process: the experience model evaluates which topics are most relevant to the current context, not a simple keyword match. This is consistent with using an LLM as the selector — it can understand the nature of the current difficulty (e.g., "the agent seems to be trusting search snippets without verification") and map it to the appropriate topic (e.g., "source verification").

Limiting to three topics balances breadth and specificity: one topic might miss relevant patterns, while many topics would dilute the generated guidance with tangential advice.

Stage 2: Guidance generation. Once three topics are selected, the experience model is provided with all experience triplets under those topics from the repository. It generates fresh guidance $e_t$ that is contextualized to the current step (prompt in Table 15, Appendix D). This is not retrieval — the output is not a copy-paste of stored text but a dynamically generated piece of advice that synthesizes the relevant experiences and applies them to the ongoing situation.

The paper emphasizes this distinction in Section 6.2 when comparing against a retrieval-only baseline (Table 3): "retrieval increases time without improving accuracy, confirming the necessity of generative guidance." Simply selecting the most similar stored experience and inserting it verbatim is insufficient — the generative step adapts the general pattern to the specific context.

Injection mechanism by step type. The guidance is injected differently depending on whether the current step is a process step or an answer step:

  • Process steps ($S_t^p$): The guidance $e_t$ is appended to the environment observation $O_t$. In the ReAct framework, the agent sees $O_t$ as the result of its action (e.g., the content of a visited webpage or search results). By appending guidance to this observation, the agent processes it alongside the environmental feedback when generating its next response $R_{t+1}$. This integrates naturally: the agent was about to reason about what it observed, and now it also has coaching about what to pay attention to.

  • Answer steps ($S_T^a$): This requires special handling because answer steps are terminal — the agent produces a final answer and the episode would normally end. The paper introduces an extension mechanism: the guidance $e_t$ is treated as a synthetic observation $O_T$, creating an artificial next step $S_{T+1}$ where the agent can either refine its answer or invoke additional tools for further reasoning. The paper describes this in Section 4.3:

"we extend it to $\{R_T, O_T\}$ with $e_t$ as $O_T$, enabling the agent to continue at step $T+1$ (either refining the answer or invoking tools for further reasoning)."

This is a clever design: instead of forcing the agent to accept the guidance and produce a new answer immediately, it creates space for the agent to verify, search for additional evidence, or reconsider its conclusion. The guidance acts as a prompt to re-examine, not as a command to change the answer.

What happens when the experience base is removed? Figure 7 includes an ablation where the repository is entirely removed — the experience model must generate guidance based solely on its own "world experience" (pretraining knowledge) without reference to stored triplets. The paper reports: "while removing the repository entirely degrades performance, the remaining substantial accuracy indicates that $\mathcal{M}_e$'s inherent world experience alone is beneficial." This is an interesting finding: even without task-specific experience, a capable model can provide useful guidance just by analyzing the current context. However, the repository provides task-specific precision that boosts performance further.

Design Choices and Their Justifications

Why entropy over learned reward models? The paper explicitly compares against a reward-model-based trigger in Section 6.2, using Claude-Sonnet-4-20250514 to judge intervention necessity at each step. The RM-based approach achieves similar accuracy but with 2.2-2.9× time overhead, because each trigger check requires an expensive API call to a separate model. Entropy is computed from the agent's own logits — it adds negligible computational cost. The paper frames this as a deliberate efficiency choice: the self-trigger "balances efficiency and performance by adapting intervention intensity to problem difficulty."

Why bootstrap logistic regression over a learned trigger classifier? A neural classifier could potentially achieve better discrimination between correct and incorrect steps, especially for process steps where logistic regression (a linear model in log-odds of entropy) may be insufficient. However, the paper's probabilistic triggering scheme requires well-calibrated probabilities, which logistic regression naturally provides (it optimizes log-likelihood directly). A neural classifier trained with cross-entropy can be miscalibrated (overconfident on some inputs, underconfident on others), which would distort the $p_{\text{intervene}}$ mapping. The bootstrap provides principled uncertainty quantification without distributional assumptions. The computational cost (seconds on CPU) makes this a practical choice.

Why topic-based organization over flat retrieval? A flat retrieval approach — embedding the current context, embedding all stored experiences, and retrieving the top-k by cosine similarity — would be the standard choice. The paper's topic-based approach adds a level of indirection: first select topics, then generate from all experiences under those topics. The advantage (though not explicitly justified in the paper) is that topic selection operates at a higher level of abstraction. A single triplet might be highly similar to the current context in surface-level features (similar query words, similar tool calls) but address the wrong type of mistake. Topic selection groups experiences by failure mode, not by surface similarity. The experience model can then synthesize guidance from multiple examples of the same failure pattern, producing more robust advice.

Why generative guidance over retrieval? This is the most consequential design choice and is explicitly evaluated in Section 6.2 (Table 3). Retrieval-based guidance — selecting the most similar experience via text embeddings — "increases time without improving accuracy." The paper's explanation (Appendix B.2) provides the intuition: stored experiences are abstracted patterns, and "when such heavily processed experiences are given to reasoning agents, the agents may not even understand some abstract expressions." Generative guidance adapts the abstract pattern to the concrete situation — it translates "verify sources" into "the current webpage you just visited claims X, but you should cross-check this against the official documentation at Y." This translation step is what makes the guidance actionable.

Why the "one-step silence" rule? Without this rule, the agent could receive guidance at step $t$, act on it at step $t+1$, and then receive more guidance at $t+1$ about its response to the first guidance — creating a cascade where the experience model and the agent play ping-pong. The one-step silence creates space for the agent to incorporate the guidance and produce a response before being evaluated again. It also prevents the over-intervention pattern that Section 6.2 identifies in the rule-based trigger (which intervenes at step 1 and onwards): the rule-based trigger produces 1.7× step overhead and 2.6× time overhead on GAIA.

Why separate process and answer step thresholds? This is a direct consequence of the entropy analysis in Figure 3. Process steps have inherently higher and more variable entropy (AUC = 0.6223) compared to answer steps (AUC = 0.7187). Using a single threshold would either over-trigger on process steps (if calibrated for answer-step separability) or under-trigger on answer steps (if calibrated for process-step separability). Separate thresholds acknowledge that the relationship between entropy and correctness is qualitatively different for deliberation (process) versus commitment (answer).

Why $N = 1000$ bootstrap iterations? The paper does not explicitly justify this number. Standard practice for percentile bootstrap confidence intervals is 1000-2000 iterations; 1000 provides stable quantile estimates (the 2.5th and 97.5th percentiles) while keeping computation trivial. Since each iteration fits a logistic regression with one feature on at most a few thousand data points, 1000 iterations completes in seconds on CPU.

4. Key Insights and Innovations

Innovation 1: Entropy as an Intrinsic, Cost-Free Self-Trigger Signal for Interactive Agents

Prior to ExpSeek, the dominant paradigm for triggering interventions in multi-turn agent systems — when triggering was even considered as a separate design problem — relied on external evaluation. Methods either injected experience at every step (the rule-based baseline in Table 3), consulted a separate reward model or critic to assess step quality (the RM-based baseline), or simply injected all experience at the start and made no further triggering decisions at all (the global injection baselines). Each approach carried a fundamental cost: uniform triggering produces inefficiency and over-intervention, while external evaluation requires expensive API calls or a separately trained verifier that may itself introduce errors.

ExpSeek's core conceptual move is recognizing that the agent model's own token-level predictive entropy can serve as a proxy for step-level correctness — and that this signal is already being computed during generation at zero additional inference cost. This is not an obvious claim. Entropy has been used for static evaluation tasks — confidence calibration (Chen and Mueller, 2024), hallucination detection (Farquhar et al., 2024), answer selection (Ren et al., 2023) — and recently extended to multi-step reasoning in reinforcement learning settings as an exploration bonus (Wang et al., 2025c; Zheng et al., 2025) or a credit assignment signal (Wang et al., 2025b). But no prior work had taken the step of treating real-time entropy as a binary trigger for when to seek external guidance during deployment, transforming it from a training-time or post-hoc analysis tool into an online control mechanism.

The intellectual contribution is not the computation of entropy — that is elementary — but the empirical validation that web agent step entropy is statistically separable between correct and incorrect steps, coupled with the recognition that this separability is qualitatively different for process versus answer steps (Figure 3). The paper reports AUC values of 0.6223 for process steps versus 0.7187 for answer steps on Qwen3-8B. This means that for deliberation steps — where the agent is exploring alternatives, evaluating sources, or formulating search strategies — entropy is a noisy signal (many correct steps exhibit high entropy due to genuine exploration). For commitment steps — where the agent must produce a final answer — entropy becomes substantially more reliable as an error indicator. This finding is not a metric gain but a diagnostic insight: it establishes boundary conditions for entropy-based triggering and motivates the design of separate threshold intervals for different step types.

The significance extends beyond web agents. Any LLM-based interactive system produces token-level probability distributions as a byproduct of generation. ExpSeek demonstrates that this "waste product" of inference can be repurposed as a control signal for orchestrating external resources (experience models, retrieval systems, human feedback) — provided the relationship between uncertainty and correctness is empirically characterized for the specific task. This opens a design space where agents monitor their own confidence and proactively request assistance rather than passively accepting whatever context was provided at initialization, shifting the role of external knowledge from a static background resource to an on-demand collaborative partner.

Innovation 2: The Paradigm Shift from Passive Experience Injection to Proactive, Contextualized Guidance Seeking

The paper's second fundamental contribution is a reframing of how experience intervenes in agent reasoning, formalized as a change in the utilization function from $\mathcal{G}(\mathcal{E}, q)$ to $\mathcal{G}(\mathcal{E}, h_t)$. This notation may appear minor, but it encodes a structural shift with deep implications.

What the field did before. Prior experience-based methods (ExpeL, RaDA, Agent Workflow Memory, ReasoningBank, and the two baselines in this paper) all follow the same architectural pattern: construct or update an experience repository offline, then at inference time, retrieve relevant experience and inject it into the agent's initial context. The experience is static — it sits frozen in the system prompt or initial message, unchanged as the agent's situation evolves across potentially dozens of interaction turns. This architecture implicitly treats experience as a background knowledge resource, analogous to a textbook that the agent consults once before starting a task.

The paper provides direct evidence that this architecture fails for web agents. Table 2 shows both passive baselines (Training-Free GRPO and ReasoningBank+) struggling to improve over vanilla ReAct, with improvements under 3% or even performance degradation on challenging benchmarks. The paper's diagnosis (Appendix B.2) identifies two root causes: (1) abstracted, decontextualized experience items are difficult for the agent to interpret and apply to concrete situations, and (2) even when the experience is relevant at the start, its salience decays as the agent's context window fills with new observations, making it difficult for the model to "precisely locate a few short effective experience items across ultra-long contexts."

What ExpSeek changes. By redefining experience utilization as $\mathcal{G}(\mathcal{E}, h_t)$ — a function of the current context that can be invoked at any step — ExpSeek transforms experience from a static prefix to a dynamic, interactive resource. The experience model $\mathcal{M}_e$ does not retrieve stored text and paste it verbatim; it generates fresh guidance that synthesizes relevant past experiences and adapts them to the concrete situation of the current step. The paper explicitly contrasts this with human experience recall:

"when humans recall experience, the amount of information retrieved instantly is enormous, including scenes, behavioral details, and even emotions, far exceeding simplified experience items."

The generative approach approximates this richness by producing guidance that is embedded in the agent's current observational context — it says "given where you are right now and what you just observed, here's what similar situations have taught us" rather than "here are some general tips about web searching."

Why this is more than an engineering improvement. The shift from $\mathcal{G}(\mathcal{E}, q)$ to $\mathcal{G}(\mathcal{E}, h_t)$ changes the role of the experience model from a librarian (retrieving relevant documents at the start) to a supervisor (monitoring the agent's ongoing behavior and providing just-in-time coaching). This has architectural implications beyond the specific entropy-trigger mechanism: any signal that indicates the agent needs help — entropy, reward model scores, human feedback, anomaly detection — could drive the same $\mathcal{G}(\mathcal{E}, h_t)$ invocation. The contribution is the interaction paradigm, not just the specific trigger.

The empirical validation of this paradigm shift is substantial: ExpSeek achieves 9.3% and 7.5% absolute improvements over vanilla ReAct on 8B and 32B models (Table 2), while the passive baselines show near-zero or negative gains. The 6.7% and 6.0% margins over the best passive baselines directly quantify the value of when and how experience is delivered, independent of the content of the experience base. This is a fundamental architectural finding, not an incremental gain: how you inject knowledge matters as much as what knowledge you inject.

Innovation 3: Weak-to-Strong Guidance — A Small Experience Model Can Materially Improve a Larger Agent

The paper's scaling experiments in Section 6.3 (Figure 5) reveal a finding that, while not the central thesis, carries significant practical and theoretical implications: a 4B-parameter experience model can improve a 32B-parameter agent by 5.2 and 9.7 percentage points on GAIA and xbench respectively. This is not the main contribution — the paper does not frame itself around model compression or distillation — but it is an emergent property of the architecture that deserves recognition as a distinct insight.

Why this is surprising. The standard assumption in the experience-based agent literature is that the model providing guidance should be at least as capable as the agent receiving it — or that experience comes from a superior source (e.g., a larger teacher model distilling into a smaller student). The baselines in this paper use a 235B model for experience generation. ExpSeek's finding that even a 4B model — 8× smaller than the 32B agent — provides substantial gains suggests that the quality of the guidance depends more on the structured experience repository and the contextualization mechanism than on the raw capability of the guiding model.

This is consistent with the paper's architectural insight: the experience model's job is not to solve the task but to recognize failure patterns and map them to stored guidance. Pattern recognition requires less capability than solution generation. The experience base, constructed by a larger tool model (235B) from contrastive trajectory pairs, encodes the reasoning about mistakes — the experience model at inference time primarily performs retrieval (topic selection) and adaptation (contextualizing stored templates), tasks that are well within the reach of smaller models given a high-quality repository.

The practical implication is significant: deploying ExpSeek does not require maintaining a large, expensive model for guidance. A small, fast model can provide near-real-time coaching, making the approach feasible for latency-sensitive applications. The theoretical implication is that knowledge can be transferred from a powerful model to a weaker model through a structured intermediate representation (the experience base), and that weaker model can then effectively deploy that knowledge to guide a stronger model — a form of "knowledge compression for coaching" that is distinct from standard distillation.

Innovation 4: Experience as a Step-Level, Topic-Organized, Contrastively-Derived Triplet Schema

The paper's experience representation — the triplet schema of (Behavior, Mistake, Guidance) organized into topics and derived from contrastive trajectory pairs — is a design contribution that solves a specific problem with prior experience formalisms. Most prior work represents experience as either raw trajectory exemplars (Synapse), abstracted patterns without step-level granularity (ExpeL), or continuously updated knowledge without structured error analysis (ReasoningBank). These representations suffer from what the paper identifies as a disconnect between experience construction and actual reasoning: "when such heavily processed experiences are given to reasoning agents, the agents may not even understand some abstract expressions" (Appendix B.2).

The triplet schema addresses this through three deliberate design properties:

Contrastive derivation ensures groundedness. Each triplet is not generated from the failed trajectory alone — it is produced by a tool model that has access to both the failed trajectory and a successful trajectory for the same query. The mistake identification is therefore anchored in a specific, concrete divergence: "at step 3, the successful agent visited the official documentation, while you relied on the search snippet." This contrasts with prior methods that induce patterns from successes and failures independently, which risks producing abstract advice ("verify sources") that the agent cannot operationalize.

Step-level granularity enables targeted intervention. Because triplets are generated for individual incorrect steps rather than whole trajectories, the experience model can provide guidance that addresses the specific type of error occurring at the current moment. A trajectory-level experience ("this task requires careful source verification") is too broad to help at step 7 when the agent is struggling with answer formatting. The step-level schema enables the experience model to say "your current step shows signs of premature answer formulation — here's what went wrong in similar situations."

Topic organization enables synthesis across exemplars. The iterative topic induction process — where topics emerge bottom-up from the data rather than being pre-defined — groups related failure patterns. At inference time, the experience model selects three topics and generates guidance from all triplets under those topics, not just the single most similar one. This synthesis across multiple exemplars of the same failure mode produces richer guidance than retrieval of a single matched case, which the ablation in Table 3 confirms: retrieval-based guidance (selecting the most similar experience via embedding similarity) fails to outperform generative guidance that synthesizes across topical groups.

This schema is not a theoretical advance but a practical design contribution that solves a recognized engineering problem — the disconnect between abstracted experience and concrete reasoning. It provides a template that could be adapted to other interactive agent domains (code generation, dialogue, robotics) where step-level error analysis from contrastive trajectories is feasible.

format to avoid conflicts with the models' original output structure (Appendix C). The paper chooses these as representative small-to-medium-scale agents where experience guidance is most needed — larger models already achieve strong performance, while very small models may lack the fundamental capability to benefit from guidance. Both are hybrid reasoning models but are used in their base (non-reasoning) configuration. The experience model $\mathcal{M}_e$ and tool model (for trajectory analysis) default to Qwen3-235B-A22B-Instruct-2507 in the main experiments, with ablations down to 4B and 30B variants (Section 6.3, Figure 5).

  • Metrics. The primary metric is accuracy (%) , defined as the fraction of test questions for which the agent's final answer matches the ground truth, evaluated using an LLM-as-a-Judge approach. The paper reports mean accuracy over five independent runs to account for sampling variability (Table 2). The judge prompt is provided in Table 19 (Appendix D). For sampling diversity evaluation, the paper also reports Pass@3 — the fraction of questions for which at least one correct answer appears among three sampled trajectories (Table 5, Appendix B.1). For efficiency analysis (Section 6.4 and Table 3), the paper tracks average reasoning steps per question and average wall-clock time per question, though the latter is not formally defined (no hardware specifications or timing methodology).

  • Baselines. The paper compares against two carefully selected experience-based methods:

    • Training-Free GRPO (Cai et al., 2025a): Uses semantic advantages from offline trajectories to iteratively acquire and update high-quality experiences, globally leveraging the experience repository at test time. This represents the offline refinement paradigm (experience is extracted from pre-collected trajectories and injected before task execution).
    • ReasoningBank+ (Ouyang et al., 2025): A self-evolving experience acquisition scheme that accumulates experiences from online tasks and retrieves them from a continuously updated repository in the system prompt during subsequent reasoning. The "+" variant represents an enhanced implementation using the 235B model (rather than a weaker agent) to generate experiences, making it a stronger baseline than the original. This represents the online self-evolution paradigm.

    Both baselines inject experience as global context before task execution (Figure 1, Panel A), making them direct comparisons against the proactive seeking paradigm. The vanilla ReAct agent with no experience intervention serves as the absolute baseline.

  • Generation budget / compute accounting. The paper does not conduct a compute-matched comparison between methods — there is no concept of "generation budget" as a controlled variable as in the inference-time scaling literature. Instead, efficiency is measured post hoc through step counts and wall-clock time (Table 3, Figure 6). The maximum number of ReAct steps is capped at 30, with episodes exceeding this limit treated as failures. The agent sampling parameters are temperature 1.0 and top-p 0.95 throughout. During experience base construction, $k = 5$ trajectories are sampled per training query. The bootstrap resampling uses $N = 1000$ iterations. There is no explicit accounting for the computational cost of the 235B tool model during experience base construction, nor for the cost of the experience model during inference — the efficiency comparisons in Table 3 and Figure 6 focus on agent-side steps and time, not total system FLOPs.

  • Cross-validation / statistical protocol. The paper reports mean accuracy over five independent runs with different random seeds (Section 5.1), which accounts for the stochasticity in agent sampling (temperature 1.0, top-p 0.95) and the probabilistic trigger mechanism. However, there is no explicit cross-validation for threshold estimation — the threshold intervals in Table 1 are computed once from the WebWalkerQA training set and applied to all four benchmarks without further tuning. This means the test-time trigger mechanism is sensitive to the representativeness of the WebWalkerQA training entropy distributions. The paper does not report confidence intervals or standard deviations on the accuracy numbers, making it difficult to assess the statistical significance of the reported gains (e.g., whether the 9.3% improvement on 8B is reliably distinguishable from random variation). The threshold estimation itself uses 95% bootstrap confidence intervals (Equation 5), but this quantifies uncertainty in $\theta$, not in downstream accuracy.

Main Quantitative Results

Aggregate Performance Across Benchmarks (Table 2)

Table 2 reports the headline results across all four benchmarks, two agent models, and three method variants (full ExpSeek, process-only guidance, answer-only guidance). The key findings are:

ExpSeek substantially outperforms all baselines. On Qwen3-8B, ExpSeek achieves average absolute improvements of 9.3% over vanilla ReAct across the four benchmarks (the per-benchmark improvements are: GAIA +2.8%, WebWalkerQA +14.6%, xbench +12.4%, Seal-Hard +8.8%). On Qwen3-32B, the average improvement is 7.5% over vanilla ReAct. The absolute accuracy numbers (reading from Table 2) place Qwen3-8B ExpSeek at roughly 31-44% across benchmarks (up from ~22-34% for vanilla), and Qwen3-32B ExpSeek at roughly 38-53% (up from ~31-45% for vanilla). These are substantial gains for a method that adds no new model parameters to the agent itself and uses only its own uncertainty signal as a trigger.

The margin over passive baselines is 6.7% (8B) and 6.0% (32B). Training-Free GRPO and ReasoningBank+ show minimal improvement over vanilla — typically under 3% on most benchmarks, and in several cases negative gains (performance degradation). For example, on xbench with Qwen3-8B, Training-Free GRPO achieves roughly 28% versus 32.8% for ExpSeek (roughly 4.8 percentage points lower). On WebWalkerQA with Qwen3-32B, ReasoningBank+ achieves roughly 38% versus 44.6% for ExpSeek. The paper attributes this to the "reasoning burden" that passive global experience places on smaller agents — they must process a longer context with abstract advice that may not apply to their current situation.

Cross-task generalization is strong. Despite the experience base being constructed entirely from the WebWalkerQA training set (170 examples), ExpSeek maintains robust performance on the three out-of-distribution benchmarks (GAIA, xbench, Seal-Hard). The improvement over vanilla on GAIA is smaller (+2.8% for 8B) than on WebWalkerQA (+14.6%), which is expected since the experience is in-distribution for WebWalkerQA. However, the xbench gain (+12.4%) is nearly as large, suggesting the experience capture failure modes (source verification, premature answer formulation, search strategy) that generalize across web agent tasks.

Answer-only guidance approaches full ExpSeek performance more closely than process-only. The ablation in Table 2 shows that guiding only process steps drops average performance by -2.44% (8B) and -4.12% (32B) relative to full ExpSeek, while guiding only answer steps drops by -4.91% (8B) and -4.51% (32B). However, the paper notes a nuance: "guiding only answer steps achieves performance closer to the full method." This aligns with the entropy analysis finding (Figure 3) that answer steps have stronger distributional separability (AUC 0.7187 vs. 0.6223) — the trigger is more reliable on answer steps, so answer-only guidance captures a larger fraction of the benefit. Process-only guidance still provides gains but is less targeted due to the noisier entropy signal.

Pass@3 Results (Table 5, Appendix B.1)

Table 5 reports Pass@3 performance — the fraction of questions where at least one of three sampled trajectories produces the correct answer. This measures sampling diversity: can the method produce correct solutions in its top-k outputs, even if it doesn't consistently select them?

ExpSeek amplifies sampling diversity more than it amplifies mean accuracy. For Qwen3-8B, the absolute Pass@3 improvement over vanilla is 12.9% (vs. 9.3% for mean accuracy). For Qwen3-32B, it is 8.8% (vs. 7.5% for mean accuracy). In contrast, both passive baselines (Training-Free GRPO and ReasoningBank+) show worse Pass@3 than vanilla on several benchmarks, indicating they not only fail to improve selection but actually reduce the diversity of correct solutions in the sampling distribution.

This has implications beyond the main results: the paper suggests ExpSeek could serve as "a rollout augmentation strategy in agentic RL training" (Appendix B.1), where higher Pass@k improves the quality of training data for self-improvement loops. The fact that ExpSeek increases Pass@3 more than Pass@1 (mean accuracy) suggests it encourages the agent to explore a wider range of potentially correct paths, not just converge more reliably on a single answer.

Efficiency Analysis (Table 3, Section 6.2)

Table 3 compares the entropy-based self-trigger against two alternative trigger mechanisms on GAIA and xbench, measuring both accuracy and efficiency:

  • Rule-based trigger (continuous intervention from step one with one-step post-trigger silence): similar accuracy, 1.7× step overhead and 2.6× time overhead on GAIA (1.5× and 2.1× on xbench). The agent receives guidance at every other step regardless of whether it needs it, inflating the trajectory length without corresponding accuracy gains.

  • RM-based trigger (Claude-Sonnet-4-20250514 judges intervention necessity at each step): similar accuracy, 1.3-1.5× step overhead but 2.2-2.9× time overhead. The reward model correctly identifies that many steps don't need intervention (reducing step overhead vs. rule-based), but the per-step API calls dominate wall-clock time.

  • Entropy-based self-trigger (ExpSeek) : 1.0× step and 1.0× time overhead on GAIA (i.e., nearly identical to vanilla), while maintaining the full accuracy gain. The "Rej" (rejection rate) column shows that 69.3% of trigger checks result in non-intervention on GAIA, rising to higher intervention rates on the more challenging xbench — exactly the adaptive behavior the paper claims.

Adaptivity to problem difficulty. On xbench, the entropy-based trigger intervenes 25.6% more frequently than on GAIA while maintaining similar accuracy. The paper interprets this as evidence that "the self-trigger balances efficiency and performance by adapting intervention intensity to problem difficulty" — harder problems naturally produce higher agent entropy, triggering more interventions automatically without any difficulty-estimation overhead.

Intervention Frequency Scaling (Figure 6, Section 6.4)

Figure 6 shows a sweep of 21 configurations where the paper varies the intervention interval (the number of silent steps after each trigger, expanded from 1 to {0, 1, 2}) and shifts the threshold by ±0.05 three times. The x-axis is the average number of interventions per question; the y-axis is accuracy.

Key finding: accuracy rises rapidly with the first ~2 interventions, then plateaus. At approximately 2 interventions per question, accuracy reaches 43.01%; beyond 6 interventions, accuracy is essentially flat. This means the marginal benefit of additional guidance drops to near-zero after a small number of interventions — the experience model's first few guidance instances capture most of the value, and subsequent interventions are redundant or unhelpful. The paper interprets this as evidence that "while increasing intervention intensity does not degrade performance, it also fails to yield higher gains" — over-intervention is not harmful (unlike the rule-based trigger in Table 3, which increases steps without increasing accuracy) but also not beneficial.

Stable performance under threshold perturbation. The ±0.05 threshold shifts (which are substantial relative to the threshold values in Table 1) produce consistent performance, indicating that the system is not brittle to the exact threshold calibration — the bootstrapped intervals provide a reasonable operating range, and small errors in threshold estimation do not cause catastrophic failures.

Experience Model Scaling (Figure 5, Section 6.3)

Figure 5 reports accuracy on GAIA and xbench when the experience model $\mathcal{M}_e$ is varied among 4B, 30B, and 235B parameter variants (all from the Qwen3 family). The key findings:

All three sizes produce substantial gains over vanilla. The smallest tested experience model (4B) improves the 32B agent by +5.2% on GAIA and +9.7% on xbench. The largest (235B) improves by roughly +12% on GAIA and +15% on xbench (estimated from Figure 5 lines). This establishes a scaling law for experience models: larger guidance models produce more effective guidance, but even very small models provide meaningful value.

The 4B → 32B result is particularly striking: a model 8× smaller than the agent it's guiding produces nearly half the gain of a model 7× larger (235B vs. 32B). This validates the paper's claim in Section 1 that "even a 4B small-scale experience model can significantly boost the performance of larger agent models" and supports the architectural insight that the experience model's job (pattern recognition from a high-quality repository) requires less capability than the agent's job (solving the task from scratch).

Experience Repository Transfer and Scaling (Table 4, Figure 7, Section 6.3)

Table 4 reports a cross-swapping experiment: the experience repository built for the 8B agent is used with the 32B agent, and vice versa. Both directions show performance drops compared to using the dedicated repository, confirming that experience has model dependency — the failure patterns of the 8B agent are not identical to those of the 32B agent. However, the performance is still above vanilla in both cases, indicating that "the abstract guidance knowledge it contains still holds transfer value" — the failure modes captured by the triplets capture general web-agent reasoning pitfalls, not just model-specific quirks.

Figure 7 shows accuracy as a function of repository size when experiences are downsampled per topic. Even a single experience per topic maintains high accuracy, only modestly below the full-repository performance. This suggests that the topic structure itself — the categorization of failure modes — provides substantial value independent of the number of exemplars. The experience model can generalize from a single example of a failure pattern to provide useful guidance in related situations. Removing the repository entirely (guidance from the experience model's pretraining knowledge alone) causes a larger drop but still leaves substantial accuracy above vanilla, indicating the 235B model's "inherent world experience" provides useful coaching even without task-specific examples.

Ablation Studies and Robustness Checks

Process-only vs. answer-only guidance (Table 2, Section 5.3): Guiding only process steps drops average performance by 2.44% (8B) and 4.12% (32B). Guiding only answer steps drops by 4.91% (8B) and 4.51% (32B). Neither matches full ExpSeek, confirming that both step types benefit from guidance and that their effects are complementary. The paper notes that answer-only guidance achieves performance closer to full ExpSeek, consistent with the stronger entropy-correctness correlation for answer steps (Figure 3). This ablation validates the design choice of separate trigger thresholds and guidance repositories for process and answer steps — if both step types behaved identically, a single threshold and a single repository would suffice, and this ablation shows they don't.

Trigger mechanism comparison (Table 3, Section 6.2): As discussed in the main results, RM-based and rule-based triggers achieve similar accuracy to entropy-based self-triggering but with substantially worse efficiency. RM-based: 2.2-2.9× time overhead. Rule-based: 1.7× step and 2.6× time overhead. This ablation directly validates the central claim that entropy is not only sufficient as a trigger signal (matching RM performance) but superior in efficiency (zero additional inference cost) and adaptive (trigger rate scales with problem difficulty). The RM baseline represents the upper bound on what a perfect external evaluator could achieve; ExpSeek matches its accuracy with dramatically lower cost.

Retrieval-based vs. generative guidance (Table 3, Section 6.2): Replacing the experience model's generative guidance with retrieval of the most similar stored experience (via text embeddings) "increases time without improving accuracy." The paper attributes this to the decontextualization problem: stored experiences are abstracted patterns that may not be directly actionable in the current context, and verbatim retrieval does not adapt them. Generative guidance — which synthesizes across multiple triplets and contextualizes to the current state — is necessary for the guidance to be effective. This ablation validates the paper's choice of $\mathcal{M}_e$ as a generative model rather than a retrieval engine.

Experience repository removal (Figure 7, Section 6.3): When the experience base is removed entirely, the experience model generates guidance based solely on its pretraining knowledge (prompt in Table 16). The resulting accuracy is above vanilla but below the full ExpSeek with repository. The paper interprets this as evidence that both the structured experience (from the repository) and the experience model's own capabilities contribute to the gains, and that each provides non-redundant value. This is a strong robustness check: even in a zero-shot setting with no task-specific experience, the experience model provides useful coaching, suggesting the approach could bootstrap itself with no initial repository.

Experience per topic downsampling (Figure 7, Section 6.3): Reducing the number of triplets per topic to 1 (from the original distribution, which varies by topic) maintains high accuracy. This suggests the topic structure is the primary value carrier — the categorization of failure modes into distinct, named topics enables the experience model to quickly identify the relevant pattern and generate guidance, and additional exemplars within a topic provide diminishing returns.

Intervention interval and threshold sensitivity (Figure 6, Section 6.4): Expanding the silence interval from 1 to 2 (i.e., two agent steps without guidance after each intervention) and shifting thresholds by ±0.05 produces a smooth tradeoff curve between intervention frequency and accuracy, with no catastrophic drops. Accuracy rises with ~2 interventions and plateaus beyond ~6. This robustness check validates that the system is not brittle to hyperparameter choices and that the benefits are not contingent on precise threshold calibration.

Experience repository cross-swapping (Table 4, Section 6.3): Swapping the 8B and 32B repositories produces intermediate performance: better than vanilla but worse than the dedicated repository. The paper interprets this as evidence of model dependency in experience (different agents exhibit different failure patterns) but also some transfer value (abstract guidance knowledge generalizes). This is a practical finding: deploying ExpSeek for a new agent model would ideally use a repository built from that specific model's trajectories, but sharing repositories across model scales provides partial benefits.

Negative result: ReSTEM^{\text{EM}}-style revision model (not applicable): Unlike the reference example paper (which studied revisions), ExpSeek does not involve iterative self-revision or reinforcement learning. There is no analogous "revision model degrades" negative result. However, the paper does report a negative result for passive experience baselines (Table 2), which underperform or even degrade relative to vanilla ReAct in multiple settings — this is the key empirical demonstration that global passive injection is not just suboptimal but can be counterproductive.

Negative result: Retrieval-based guidance (Table 3): The retrieval-only variant "increases time without improving accuracy" — a clean demonstration that simply finding the most similar stored experience and inserting it into context is insufficient, and that contextualized generation is necessary.

Critical Assessment

Does ExpSeek genuinely demonstrate that entropy is an effective self-trigger signal?

What the experiments show: Table 3 demonstrates that the entropy-based trigger achieves similar accuracy to an RM-based trigger (Claude-Sonnet-4-20250514, representing a strong external evaluator) while maintaining ~1× step and time overhead. The RM-based trigger effectively represents the upper bound of what external evaluation could achieve — if entropy were a poor signal, the accuracy gap between entropy-based and RM-based triggering would be large. It is not.

What the experiments do not show: The RM baseline uses a single proprietary model (Claude-Sonnet-4). There is no comparison against an ensemble of reward models, against a fine-tuned step-level verifier trained specifically on web agent trajectories (analogous to the PRM in the reference example), or against human evaluation of intervention necessity. The claim that entropy matches external evaluation therefore holds only in comparison to one specific external evaluator, and the RM itself may not be perfectly calibrated. Additionally, the threshold estimation depends on the WebWalkerQA training set — there is no experiment where thresholds are estimated from one benchmark and tested on the same distribution with varying difficulty, which would more cleanly isolate the entropy signal from the representativeness of the training data.

Does the "proactive seeking" paradigm genuinely outperform passive injection, or is the gain attributable to other factors (the experience model, the triplet schema, the generative guidance)?

What the experiments show: Table 2 directly compares ExpSeek against two passive global injection baselines (Training-Free GRPO and ReasoningBank+) that use the same 235B tool model for experience construction and the same underlying agent models. The 6.7% (8B) and 6.0% (32B) margins over the best passive baseline cannot be attributed to model scale or experience quality — they isolate the value of step-level, entropy-triggered, generative intervention over global, static, retrieved intervention.

What a skeptic could argue: The passive baselines do not use the same experience representation (triplets) or the same experience model (generative $\mathcal{M}_e$). An ideal controlled experiment would compare ExpSeek against a variant where the same triplets and same experience model are used, but injected globally at the start rather than triggered step-by-step. This ablation would isolate the "when" and "how" from the "what," and it is not present. The paper's claim that the paradigm shift matters is supported by the overall performance margin, but the separate contributions of (a) step-level triggering, (b) contextualized generative guidance, and (c) the triplet schema are not disentangled with full factorial ablations.

Does the cross-task generalization claim hold up?

What the experiments show: Three of four benchmarks (GAIA, xbench, Seal-Hard) are entirely out-of-distribution for experience construction, and ExpSeek shows substantial gains on all three (Table 2). The GAIA gain is smallest (+2.8% on 8B), which the paper does not discuss but which is consistent with GAIA being the most distinct domain (general AI assistant tasks vs. web search tasks).

What is missing: The paper does not report which experience topics are triggered on which benchmarks. It would strengthen the generalization claim to show that specific failure modes captured in the WebWalkerQA experience base (e.g., "premature answer formulation," "source verification") are triggered on GAIA and xbench, demonstrating that the same abstract failure patterns recur across domains. Without this analysis, the generalization claim rests on the aggregate accuracy numbers, which could be driven by the experience model's pretraining knowledge (which is domain-general) rather than the WebWalkerQA-specific triplets.

Does the "weak-to-strong" guidance claim (4B → 32B) hold up?

What the experiments show: Figure 5 clearly shows the 4B experience model improving the 32B agent by 5.2% (GAIA) and 9.7% (xbench). This is well above the vanilla baseline and demonstrates that a small model can provide useful guidance to a larger agent.

What is missing: There is no comparison against a baseline where the 4B model is used as the agent (with or without experience). If the 4B model itself performs at, say, 20% accuracy on GAIA while providing guidance that lifts the 32B agent to 45%, the finding is more striking than if the 4B model is already at 40% on its own. The paper does not report 4B agent performance. Additionally, the 4B experience model is from the same model family (Qwen3) and likely shares substantial training data and architectural similarity with the 32B agent — the "weak-to-strong" result may not transfer to cross-family settings (e.g., a Llama-4B guiding a Qwen-32B).

Are the efficiency metrics adequate?

What the experiments show: Table 3 reports step counts and wall-clock time for different trigger mechanisms, with the entropy-based trigger matching vanilla efficiency.

What is missing: The efficiency accounting is incomplete in several ways. (1) The cost of the experience model $\mathcal{M}_e$ at inference time is not included in the time or step counts — each intervention requires a full generation from $\mathcal{M}_e$ (topic selection + guidance generation), which for the 235B model adds substantial inference cost that is not reflected in the agent's step count. Table 3 only tracks agent-side overhead. (2) The cost of experience base construction (trajectory sampling, tool model inference for triplet generation, topic induction) is entirely unaccounted for — this is a substantial upfront computation using a 235B model. (3) The paper reports "average time" but does not specify hardware, batching, or whether the experience model runs on separate infrastructure. These omissions make the efficiency claims (while directionally plausible) difficult to evaluate rigorously.

What experiments would strengthen the paper?

Several controlled experiments would address the gaps identified above:

  1. Ablation isolating triggering from content: Compare ExpSeek against a variant where the same triplet repository and same generative experience model are used, but all guidance is injected as a single block at the start (mimicking passive global injection with ExpSeek's content quality). This would isolate the value of step-level triggering from the value of the triplet schema and generative model.

  2. Cross-family weak-to-strong: Test whether a Llama-4B (or equivalent non-Qwen model) can provide similar gains to a Qwen-32B agent, establishing whether the weak-to-strong property is model-family-specific.

  3. Full cost accounting: Report total inference FLOPs or wall-clock time including the experience model's forward passes, and amortize the upfront construction cost across test queries. This would provide a fair comparison against baselines and against simply using a larger agent model with more inference budget.

  4. Difficulty-stratified analysis: Report accuracy gains separately for easy, medium, and hard questions (as labeled by the WebWalkerQA difficulty levels used for stratified sampling). The reference example paper's central finding — that test-time compute effectiveness is strongly difficulty-dependent — raises the natural question of whether ExpSeek's entropy trigger is uniformly effective across difficulty levels or whether (as one might expect) easy questions trigger rarely and hard questions trigger frequently, with different marginal benefits.

  5. Confidence intervals on accuracy: Report standard deviations or confidence intervals across the five independent runs to enable statistical significance assessment of the claimed improvements, particularly the smaller gains (e.g., +2.8% on GAIA 8B) where noise may rival the signal.

  6. Topic triggering analysis: For each benchmark, report the distribution of triggered topics to demonstrate that specific failure patterns (not just the experience model's general world knowledge) are driving the cross-task generalization gains.

The paper's central claims — that entropy-driven proactive seeking substantially outperforms passive global injection, that a small experience model can boost a larger agent, and that the method generalizes across benchmarks — are supported with clear evidence and consistent margins. The primary limitations are in the completeness of the efficiency accounting (experience model cost not included) and the granularity of the ablation analysis (the separate contributions of triggering, content, and generation are not fully disentangled). These are typical of a systems paper proposing a new interaction paradigm, where the priority is demonstrating overall effectiveness rather than fully decomposing every component.

6. Limitations and Trade-offs

The Cost of Threshold Estimation and Experience Base Construction Is Not Factored Into Efficiency Claims

The assumption or constraint. The headline efficiency gains — that entropy-based self-triggering matches the step count and wall-clock time of vanilla ReAct (Table 3, Section 6.2) — implicitly assume that the threshold intervals $[\theta_{\text{lower}}, \theta_{\text{upper}}]$ and the structured experience base $\mathcal{E}$ already exist. The paper is transparent about the offline construction pipeline but does not factor its cost into any efficiency metric. Building the experience base requires: (1) sampling $k = 5$ trajectories per training query from the agent model, (2) invoking a 235B tool model to analyze each failed-successful trajectory pair and generate triplets with step-level correctness labels, and (3) running iterative batch topic induction with the same 235B model (Section 4.1, Appendix D). Threshold estimation requires collecting all step entropies from these trajectories and running 1000 bootstrap logistic regression fits (Algorithm 1). The tool model inference alone — processing hundreds of trajectory pairs with detailed prompts — represents a substantial computational investment that is never quantified.

The consequence. A practitioner evaluating whether to deploy ExpSeek cannot determine the total cost of ownership from the paper's numbers. The efficiency comparison in Table 3 measures incremental inference cost over vanilla ReAct but excludes the amortized construction cost per query. For a deployment with a small number of test queries, the construction overhead dominates the total compute budget and the reported ~1× inference efficiency is misleading. More subtly, the paper does not report how the experience base degrades as the training set size changes — if only 50 training examples are available (rather than 170), does the triplet quality drop sharply? Is the bootstrap threshold estimation stable with fewer step samples? These questions are unanswered, making it difficult to assess whether ExpSeek is practical in low-resource settings where collecting hundreds of training trajectories is infeasible.

What evidence exists in the paper. Table 2 reports the size of the constructed repositories (196/190 triplets for 8B, 276/143 for 32B) and the paper mentions using 170 WebWalkerQA training examples (Section 5.1). The offline pipeline components (triplet generation prompt in Table 12, topic induction prompt in Table 13) confirm the reliance on a large tool model. However, the paper provides no measurement of tool model inference cost, no FLOPs accounting, and no analysis of how repository quality scales with training set size. The experience model scaling experiment (Figure 5, Section 6.3) studies the inference-time guidance model size but does not study the construction-time tool model size or cost.

Mitigation status. The paper does not address this limitation. The construction cost is mentioned only in passing — Algorithm 1 notes threshold estimation completes "within seconds" on CPU, but this is a tiny fraction of the total pipeline cost (tool model inference dominates). There is no suggestion of future work on cheaper experience acquisition (e.g., using smaller models for triplet generation, few-shot construction, or cross-model repository transfer). The paper frames efficiency only in terms of inference-time overhead (Table 3, Figure 6), not total lifecycle cost.

No Accounting for the Experience Model's Inference Cost at Deployment Time

The assumption or constraint. The efficiency analysis in Table 3 and Figure 6 tracks only the agent's step count and wall-clock time, not the cost of running the experience model $\mathcal{M}_e$ when a trigger fires. Each intervention requires two forward passes from $\mathcal{M}_e$: one for topic selection (Table 14 prompt) and one for guidance generation (Table 15 prompt). In the main experiments, $\mathcal{M}_e$ is Qwen3-235B-A22B-Instruct-2507 — a 235B-parameter model, substantially larger than the 8B or 32B agent it is guiding. Even with the 69.3% rejection rate on GAIA (Table 3), each intervention that does fire adds inference cost proportional to generating structured outputs from a 235B model. The paper does report that the entropy-based trigger has ~1× agent-side step and time overhead — but this metric excludes the experience model's contribution entirely.

The consequence. A practitioner reading Table 3 might conclude that ExpSeek adds zero runtime cost compared to vanilla ReAct. In reality, the total system cost depends on the intervention rate and the experience model size. On xbench, where the trigger rate is 25.6% higher than GAIA, the experience model is invoked more frequently, and the total system latency may be dominated by guidance generation rather than agent reasoning. The bright spot — and what partially mitigates the concern — is the experience model scaling experiment (Figure 5): a 4B model (which is cheaper than the 32B agent) still provides meaningful gains (+5.2% on GAIA, +9.7% on xbench for the 32B agent). A deployment using a 4B or 30B experience model would have much lower per-intervention cost than the 235B default. But the paper never quantifies this cost even for the 4B variant, and it never compares total system FLOPs against simply allocating the same compute to the agent (e.g., more search steps, more sampling).

What evidence exists in the paper. Table 3 reports agent-side step count and "Time" (presumably wall-clock) for GAIA and xbench with various triggers. The "Time" numbers are 1.0× for ExpSeek versus 1.0× for vanilla, which can only be true if the experience model's inference time is either excluded from the measurement or negligible (unlikely for a 235B model). The paper does not specify the timing methodology, hardware, or whether guidance generation runs on separate infrastructure. Figure 5 shows performance vs. experience model size but reports no latency or cost metrics for the different sizes. Figure 6 shows interventions-per-question vs. accuracy but does not convert intervention count to total system cost.

Mitigation status. The paper partially mitigates this concern through the experience model scaling results — the existence of a viable 4B guidance model means the inference cost can be made modest, even if the paper never measures it. The one-step silence rule (Section 4.3) also caps the maximum intervention frequency to at most every other step. However, the absence of any latency or FLOPs measurement for the experience model is a gap. A practitioner deploying ExpSeek would need to run their own cost analysis. The paper does not identify this as a limitation or suggest future work on cost accounting.

Difficulty Estimation Is Implicit and Uniform — There Is No Mechanism for Dynamic Difficulty-Aware Allocation

The assumption or constraint. ExpSeek uses a single set of entropy threshold intervals $[\theta_{\text{lower}}, \theta_{\text{upper}}]$ for all queries, estimated once from the WebWalkerQA training set and applied uniformly across all four benchmarks. The threshold intervals capture the average relationship between entropy and step correctness across all training questions, regardless of query difficulty. The paper provides no mechanism for estimating query difficulty before or during execution, and no mechanism for adjusting the trigger sensitivity based on whether the current query is easy or hard.

The consequence. The reference example paper on compute-optimal test-time scaling demonstrated that the optimal allocation of inference compute is strongly difficulty-dependent — easy problems benefit from different strategies than hard problems, and uniform allocation leaves substantial efficiency on the table. The same principle likely applies to experience intervention. On an easy query, the agent may rarely produce high-entropy steps, so the trigger rarely fires, and the system behaves essentially like vanilla ReAct (which is appropriate — no guidance needed). On a hard query, the agent may consistently produce high-entropy steps, triggering frequent interventions. The fixed threshold intervals cannot distinguish between "this query is hard, so high entropy is expected and perhaps less indicative of correctable errors" and "this step is genuinely off-track." A query that pushes the agent to the limits of its capability might produce elevated entropy across all steps — not because the agent is making specific, correctable mistakes, but because the task is fundamentally difficult and the model is uncertain throughout. In this regime, more interventions may not help, similar to the "hardest problems" finding in the reference example (difficulty bin 5) where no test-time strategy improved performance. The fixed-threshold approach has no way to detect this regime and could waste interventions — and experience model compute — on steps that are uncertain for reasons guidance cannot fix.

What evidence exists in the paper. The paper provides indirect evidence for this concern. Table 2 shows that the gain on GAIA is +2.8% (8B) versus +14.6% on WebWalkerQA. The paper does not analyze whether this difference is due to (a) GAIA having different difficulty distribution, (b) the WebWalkerQA-trained thresholds being less appropriate for GAIA, or (c) the experience base being less relevant to GAIA tasks. Figure 6 shows that accuracy plateaus after ~2 interventions and does not improve beyond ~6 — this could indicate that on some queries, additional interventions are unhelpful because the agent's difficulty cannot be resolved by guidance. However, the paper never stratifies these results by query difficulty, never reports per-difficulty-bin accuracy, and never proposes a difficulty-adaptive allocation mechanism.

Mitigation status. The paper does not address difficulty adaptation. The threshold intervals are static and uniform. The probabilistic trigger (Equation 6) introduces some adaptivity at the step level — entropy values near $\theta_{\text{lower}}$ trigger rarely, those near $\theta_{\text{upper}}$ trigger often — but this adaptivity is within the fixed threshold framework and does not adjust to query-level difficulty. The paper's limitations section (Section 8) focuses on threshold estimation accuracy ("more accurate threshold calculation strategies need to be investigated") and domain extension, but does not mention difficulty-aware allocation as a direction.

Single Benchmark Family for Experience Construction, Single Model Family for Agents

The assumption or constraint. All experience bases are constructed from the WebWalkerQA training set (170 examples sampled across difficulty levels). All agents are from the Qwen3 model family (8B, 32B, with ablations using 4B and 30B variants). All benchmarks (GAIA, WebWalkerQA, xbench, Seal-Hard) are web-based question-answering tasks. The paper states it "believe[s] this model is representative of the capabilities of many contemporary LLMs" (implied framing in Section 5.1, though not explicitly stated), but this belief is untested across model families.

The consequence. Several aspects of ExpSeek's behavior could be model-family-specific or benchmark-family-specific. The entropy-correctness separability (Figure 3) — the foundational empirical finding that justifies entropy as a trigger — may differ substantially across model families. Qwen3 models are trained with specific objectives and data mixtures; a model with different calibration properties (e.g., Llama, Gemma, Claude) might exhibit different entropy distributions for correct vs. incorrect steps, potentially weakening or strengthening the trigger signal. The paper provides Figure 8 (Appendix) showing the 32B model's entropy distributions, confirming the pattern holds within Qwen3. But cross-family generalization remains unverified. Similarly, the experience triplets capture failure modes for web search and navigation tasks (source verification, query formulation, answer extraction). It is unclear whether these failure patterns — and the topic structure that organizes them — transfer to non-web agent domains (code generation, embodied navigation, dialogue). The paper acknowledges this in Section 8 ("It remains unexplored whether ExpSeek has the potential to extend to other non-web domains") but does not frame it as a limitation of the current evidence.

What evidence exists in the paper. Table 2 shows strong cross-benchmark generalization within the web agent domain (GAIA +2.8–12.4%, xbench +12.4%, Seal-Hard +8.8%). This demonstrates that WebWalkerQA-derived experience captures failure modes relevant to other web agent tasks. Table 4 shows cross-model repository transfer (8B ↔ 32B) within the Qwen3 family — performance degrades versus dedicated repositories but remains above vanilla. However, this is within-family transfer. No experiment tests cross-family generalization (e.g., Qwen3-8B experience used with a Llama-8B agent) or cross-domain generalization (e.g., WebWalkerQA experience used for code generation agents).

Mitigation status. The paper partially addresses this in Section 8 (Limitations), noting that "it remains unexplored whether ExpSeek has the potential to extend to other non-web domains and integrate more tools." The authors treat this as future work. The cross-benchmark generalization within the web domain is a mitigating factor — the method demonstrably generalizes across four distinct web agent benchmarks — but the single model family constraint is not acknowledged as a limitation of the current evidence.

The Bootstrap Threshold Estimation Procedure Assumes Stationary Entropy Distributions

The assumption or constraint. The logistic regression models and bootstrap-derived threshold intervals (Section 4.2.2, Algorithm 1) are estimated once from training trajectories and then frozen for all test-time inference. This assumes that the relationship between step entropy and step correctness — encoded in the learned parameters $(w, b)$ and the resulting thresholds $\theta$ — is stationary across test queries and deployment conditions. The training trajectories are generated by the agent model at temperature 1.0 and top-p 0.95 (Section 5.1); the thresholds are valid only for this sampling configuration.

The consequence. If the deployment sampling parameters change (e.g., lower temperature for more deterministic behavior, different top-p), the entropy distributions shift, and the frozen thresholds may no longer correctly identify steps needing intervention. At temperature 0, all token distributions become near-deterministic, entropy drops, and the trigger would rarely fire — even if the agent is producing incorrect answers confidently. Conversely, at higher temperature, entropy inflates globally, and the trigger may fire excessively. More subtly, if the agent model is fine-tuned, updated, or quantized, its calibration and entropy characteristics may change, silently degrading the trigger's reliability. The probabilistic trigger mechanism (Equation 6) provides some robustness to small distributional shifts via the $[\theta_{\text{lower}}, \theta_{\text{upper}}]$ interval, but large shifts (e.g., from 1.0 to 0.3 temperature) would likely require re-estimation.

What evidence exists in the paper. Figure 6 provides partial evidence of robustness: shifting the thresholds by ±0.05 (a modest perturbation) produces stable performance across 21 configurations. This suggests the system is not brittle to small threshold errors, which is encouraging. However, this experiment perturbs the thresholds themselves, not the underlying distribution that generated the entropy values. It does not test robustness to sampling parameter changes, model quantization, or distribution shift in the test queries. The paper does not report whether the entropy distributions of the four test benchmarks match the WebWalkerQA training distribution — the cross-benchmark accuracy gains (Table 2) are indirect evidence that the thresholds remain useful out-of-distribution, but no direct distribution comparison is provided.

Mitigation status. The paper does not identify this as a limitation. The assumption of stationarity is implicit in the offline threshold estimation design. The probabilistic trigger and the one-step silence rule provide some operational robustness (occasional mistriggers don't catastrophically degrade performance, as Figure 6 shows), but they do not address systematic distribution shift. A deployment that differs substantially from the training conditions (different temperature, different model version, different query domain) would require re-estimating thresholds, which adds deployment friction not acknowledged in the paper.

The Test Set Size (Per Benchmark) and Absence of Confidence Intervals Limit Statistical Certainty

The assumption or constraint. The paper evaluates on four benchmarks with a total test set that is the union of the GAIA, WebWalkerQA (remainder after 170 training examples), xbench-DeepSearch, and Seal-Hard test sets. The paper does not report the exact test set sizes, but WebWalkerQA's training set is 170 out of roughly 680 total (based on the 25% sampling from three difficulty levels), leaving ~510 test examples. GAIA's standard validation set is ~165 examples, and the other benchmarks likely have comparable or smaller sizes. The paper reports mean accuracy over five independent runs (Section 5.1) but does not report standard deviations, confidence intervals, or statistical significance tests for any of the claimed improvements.

The consequence. The absolute improvement numbers reported in Table 2 — particularly the smaller gains like +2.8% on GAIA (8B) — may be within the noise range of the five-run variability. Without standard deviations, a practitioner cannot assess whether the reported improvement is reliably above zero. The per-benchmark test sets (potentially ~100–500 questions each) are small enough that sampling variability across five runs could produce apparent gains that are not statistically significant. This is especially relevant for the cross-benchmark generalization claim: the +2.8% on GAIA might be a real but small effect, or it might be noise, and the paper provides no way to distinguish. The same concern applies to the comparison against baselines: a claimed 6.7% margin over the best passive baseline (8B average) may or may not be statistically significant depending on the variance, which is unreported.

What evidence exists in the paper. The five-run averaging is explicitly mentioned ("We report mean accuracy (%) over five independent runs," Section 5.1), which is good practice. However, the paper never reports the variance across runs. The only uncertainty quantification in the paper is the bootstrap confidence interval for $\theta$ (Equation 5), which is a methodological detail, not a downstream performance uncertainty estimate. Figure 6 shows the range of outcomes across 21 threshold-interval configurations — this provides some evidence of performance stability but is not a direct substitute for per-run variance on the main results.

Mitigation status. The paper does not address this. The limitations section (Section 8) discusses threshold estimation, domain extension, and RL integration as future directions, but does not acknowledge the statistical uncertainty in the main results. This is a standard weakness in many systems papers — reporting mean over multiple runs is already better than reporting a single run — but it limits the strength of the quantitative claims, particularly the smaller per-benchmark gains.

7. Implications and Future Directions

How This Work Changes the Landscape

ExpSeek introduces a new interaction paradigm for experience intervention: proactive, self-triggered seeking rather than passive, global injection. This is not an incremental improvement in experience quality — the paper's passive baselines (Training-Free GRPO and ReasoningBank+) already used carefully constructed experience repositories and 235B models for generation — but a structural shift in when and how experience reaches the agent. The empirical magnitude of the shift is substantial: the 6.7% and 6.0% margins over the best passive baselines (Table 2, 8B and 32B respectively) quantify the value of delivery mechanism independent of content quality, since the underlying models and tools are comparable. For a field that has largely treated experience as a retrieval-and-prefix problem, this reframes the design question from "what experience should we extract?" to "when does the agent need help, and how should that help be contextualized to the current moment?"

The paper's most significant reframing is the formalization in Section 3: shifting the experience utilization function from $\mathcal{G}(\mathcal{E}, q)$ (experience retrieved once from the query) to $\mathcal{G}(\mathcal{E}, h_t)$ (experience generated at any step from the full interaction history). This generalization converts experience from a static background resource — analogous to a textbook consulted at the start — into an interactive supervisor that monitors the agent's ongoing behavior and provides just-in-time coaching. The metaphor matters because it changes architectural thinking: future agent systems should not be designed as "agent + experience database" with a single injection point, but as "agent + experience model + trigger mechanism" that form a collaborative loop. The agent asks for help when it's uncertain; the experience model provides help that is specific to what the agent is currently struggling with. This is a fundamentally different division of labor than the prevailing paradigm.

The paper resolves a specific contradiction in the experience-based agent literature. Prior work showed that structured experience could improve agent performance in controlled settings (ExpeL, Synapse, Agent KB), but other evaluations — particularly on noisy, multi-turn web tasks — showed minimal or negative gains. ExpSeek provides a unified explanation through two mechanisms identified in Appendix B.2. First, abstracted experience items are difficult for agents to interpret when decontextualized from the concrete situation ("the agents may not even understand some abstract expressions"). Second, even relevant experience loses salience in long contexts ("it is difficult to require models to precisely locate a few short effective experience items across ultra-long contexts"). The generative, step-triggered guidance addresses both: it contextualizes abstract patterns to the current observation, and it inserts guidance at the moment of need rather than burying it in a growing context window. This explains why prior methods failed — they suffered from a delivery problem, not a content problem — and provides a diagnostic framework for evaluating future experience-based methods: are you delivering guidance when the agent needs it, in a form it can immediately use?

The paper establishes entropy as a viable, cost-free trigger signal for interactive agent systems. Prior to this work, entropy had been used for static evaluation (confidence calibration, hallucination detection, answer selection) and as an exploration bonus in reinforcement learning training. ExpSeek demonstrates that the same signal — which is a byproduct of token-level generation and requires zero additional inference — can serve as a real-time control mechanism for orchestrating external resources during deployment. The RM-based trigger comparison (Table 3) is particularly important: entropy-based triggering achieves similar accuracy to a strong external evaluator (Claude-Sonnet-4-20250514, representing what a perfect critic could achieve) while adding negligible time overhead (1.0× vs. 2.2-2.9×). This is not a marginal efficiency gain — it demonstrates that an intrinsic signal can match an expensive external oracle, fundamentally changing the cost calculus for intervention systems. The implication extends beyond web agents: any LLM-based interactive system produces token-level probabilities, and ExpSeek provides a template for repurposing them as confidence monitors that drive resource allocation (retrieval, tool invocation, human escalation, model switching).

The paper redirects research attention from experience extraction to experience delivery and trigger design. The past several years of work on agent experience (reviewed in Section 2.1) has focused overwhelmingly on the construction problem: how to extract reusable patterns from trajectories, how to denoise, how to structure, how to update online. ExpSeek's results suggest that for complex interactive tasks, the delivery problem may be the binding constraint — and that solving it can unlock gains from even modest-quality experience bases. This shifts the research frontier: rather than investing ever more effort in perfecting experience extraction pipelines (multi-round denoising, sophisticated distillation), the priority should be developing mechanisms for just-in-time, contextualized guidance and reliable trigger signals that know when to invoke them. The paper's finding that a 4B experience model can boost a 32B agent (Figure 5) reinforces this: the bottleneck is not the raw capability of the experience source but the architecture that connects experience to agent reasoning at the right moments.

The paper establishes a weak-to-strong guidance phenomenon that has implications for model deployment economics. The finding that a 4B experience model can improve a 32B agent by 5.2-9.7 percentage points (Figure 5) suggests that the guidance model does not need to be more capable than the agent — it needs access to structured experience and a mechanism for contextualizing it. This is practically important because it means ExpSeek can be deployed with a small, fast guidance model running alongside the agent, rather than requiring a large, expensive model for coaching. The experience base — constructed offline by a powerful tool model — serves as a form of knowledge compression that decouples guidance model size from guidance quality. This has not been demonstrated in prior agent experience work, where the assumption has typically been that experience should come from a model at least as capable as the agent, often the same model (in self-evolution approaches) or a teacher (in distillation approaches). ExpSeek shows this assumption can be relaxed when knowledge is transmitted through a structured intermediate representation rather than through direct model outputs.

Follow-Up Research This Work Enables

Direct factorial ablation: isolating the value of step-level triggering from the value of generative contextualization. The paper demonstrates that ExpSeek substantially outperforms passive global injection, but does not disentangle the two key architectural departures — when guidance is delivered (step-level vs. global) and how guidance is generated (contextualized to $h_t$ vs. static retrieval). A controlled experiment would compare four conditions on the same experience base and agent models: (1) global static injection of retrieved triplets (mimicking prior work's delivery with ExpSeek's content), (2) global static injection of generative guidance (all guidance generated at step 0 from the query alone), (3) step-level triggered retrieval of most-similar triplets (ExpSeek's trigger with the retrieval-only variant from Table 3), and (4) full ExpSeek. This 2×2 (delivery timing × guidance type) ablation would quantify the independent contributions of when guidance arrives and whether it's contextualized. The paper's Table 3 already suggests retrieval-only step-level guidance underperforms — but does not compare against global generative guidance, which would test whether contextualization alone can compensate for poor timing. A strong follow-up would run this on WebWalkerQA (in-distribution) and GAIA (out-of-distribution) to assess whether the relative importance of timing vs. contextualization shifts across domains.

Cross-model-family validation of entropy as a trigger signal. The paper's entropy-correctness separability analysis (Figure 3, KS tests) and the derived threshold intervals (Table 1) are validated only on Qwen3 models. A critical stress test would replicate the full pipeline — trajectory sampling, entropy distribution analysis, threshold estimation, and ExpSeek evaluation — on an architecturally distinct model family, such as Llama-3-8B or Gemma-2-9B. The specific hypothesis to test is whether the AUC values for process and answer steps (0.6223 and 0.7187 for Qwen3-8B) are model-family-specific or reflect a more universal property of web agent reasoning. If Llama models show substantially different entropy-correctness separability — e.g., lower AUC on process steps, making the trigger less reliable — this would establish boundary conditions for entropy-based triggering and motivate model-specific or adaptive threshold estimation. Conversely, if the pattern holds across families, entropy becomes a robust, architecture-agnostic trigger signal. The experiment would also test whether thresholds estimated for one model family transfer to another, which the paper's cross-repository experiment (Table 4, within Qwen3) does not address. A strong follow-up would report both within-family and cross-family accuracy, plus the correlation between per-model AUC and ExpSeek's downstream gain.

Difficulty-stratified analysis of trigger behavior and guidance effectiveness. The reference example paper on test-time compute scaling established that the effectiveness of inference strategies is strongly difficulty-dependent — the same method can help on medium problems while hurting on easy ones. ExpSeek's fixed-threshold design implicitly assumes a uniform relationship between entropy and guidance benefit across difficulty levels, but this is unlikely. On easy queries, the agent may rarely trigger (low entropy throughout), so ExpSeek behaves like vanilla ReAct — this is appropriate but unverified. On hard queries, the agent may produce consistently high entropy regardless of whether it's making correctable mistakes or simply operating beyond its fundamental capability, leading to frequent triggers that may not help (analogous to the "hardest problems" finding where no test-time strategy improved performance). A difficulty-stratified analysis would use WebWalkerQA's built-in difficulty labels (easy, medium, hard, from which the training set was sampled uniformly) and report: (1) per-difficulty trigger rates and rejection percentages, (2) per-difficulty accuracy gains over vanilla, and (3) whether the experience topics triggered differ by difficulty (do hard questions trigger "insufficient evidence" topics while easy questions trigger "premature answer" topics?). This would reveal whether ExpSeek's gains are concentrated in medium-difficulty questions (where the agent is capable of improvement but prone to specific errors) and whether it provides zero or negative value on the hardest tier — information that would guide deployment decisions and motivate difficulty-adaptive threshold adjustment.

Experience model scaling with efficiency measurement. The paper's Figure 5 shows accuracy scaling with experience model size (4B, 30B, 235B) but reports no latency or cost metrics. A practical follow-up would measure total system wall-clock time and inference FLOPs for each experience model size, then plot a Pareto frontier of accuracy vs. total cost. The key question is whether the diminishing returns in accuracy from larger experience models (the curve in Figure 5 appears to flatten between 30B and 235B) are worth the increased per-intervention cost. Concretely: does the 4B model, which costs ~8× less per forward pass than the 32B agent, provide 50-60% of the 235B model's gain at a fraction of the cost, making it the optimal operating point? The experiment would also measure how intervention frequency interacts with experience model size — if larger models produce more effective guidance that reduces the agent's step count (by resolving confusion faster), the total cost gap might narrow. A strong follow-up would compare ExpSeek's total cost against a budget-matched baseline where the agent simply gets more ReAct steps or more parallel samples (best-of-N with majority voting), establishing whether the experience model is a more compute-efficient way to spend inference budget than simply letting the agent explore longer.

Combining ExpSeek with agentic reinforcement learning for self-improvement. The paper notes in Section 8 that "since ExpSeek can also significantly improve pass@k performance, it has not yet been studied whether it can serve as an enhancement technique for Agentic Reinforcement Learning rollout." The Pass@3 results (Table 5, Appendix B.1) show that ExpSeek improves sampling diversity more than mean accuracy (+12.9% vs. +9.3% on 8B), suggesting it helps the agent explore a wider range of potentially correct paths. This is directly valuable for RL training pipelines where rollout quality determines the training signal. A concrete experiment would compare an RL training loop (e.g., GRPO or ReST-style self-improvement on web agent tasks) with two rollout strategies: (1) vanilla agent sampling and (2) ExpSeek-guided sampling. The hypothesis is that ExpSeek-guided rollouts produce higher-quality training trajectories (more correct answers in the sampling distribution, as Pass@3 suggests), which accelerates training convergence and raises the asymptotic performance ceiling. The follow-up would measure training curves (accuracy vs. RL steps) and final test accuracy, with a specific focus on whether ExpSeek's guidance during rollouts reduces the variance of the training signal (since guidance steers trajectories away from common failure modes). A strong follow-up would also test whether the experience base itself can be iteratively updated from the improving agent's trajectories, creating a co-evolution loop where the agent gets better and the experience repository gets richer simultaneously.

Cross-domain extension with domain-specific experience base construction. The paper evaluates exclusively on web search and navigation benchmarks, and acknowledges in Section 8 that extension to other domains is unexplored. A natural follow-up would apply the ExpSeek framework to a non-web interactive domain — code generation (e.g., SWE-Bench, where agents navigate repositories and edit files) or embodied task planning (e.g., ALFWorld, where agents navigate virtual environments). The key adaptation would be the experience base construction pipeline: the triplet schema (Behavior, Mistake, Guidance) and topic induction should transfer directly, but the failure modes would be domain-specific (e.g., incorrect API usage vs. premature answer formulation). The experiment would measure whether the entropy-correctness separability observed in web agents (AUC 0.62-0.72) replicates in other domains, and whether the ExpSeek architecture (entropy trigger + generative experience model) provides similar relative gains. A negative result — e.g., code generation agents show near-zero AUC because entropy is uniformly low during syntax-level token generation — would establish that entropy-based triggering is domain-dependent and that alternative trigger signals (compilation errors, test failures) are needed for structured-output domains. A positive result would establish ExpSeek as a general architecture for interactive agent guidance rather than a web-agent-specific technique.

Practical Applications and Downstream Use Cases

Cost-efficient deployment of small web agents for enterprise search and research assistance. The paper's headline result — 9.3% absolute improvement on Qwen3-8B (Table 2) — means that a relatively small, cost-effective model augmented with ExpSeek can approach or exceed the performance of a much larger vanilla model on complex web reasoning tasks. For organizations deploying internal research assistants, customer-facing search agents, or automated due-diligence tools, this translates to a concrete deployment recipe: deploy a small agent model (8B parameters, cheap to serve) with a small experience model (4B parameters, as validated in Figure 5) and a pre-built experience base constructed from domain-specific training queries. The 6.7% margin over passive experience injection (Table 2) is the key number for practitioners — it means switching from "dump all experience into the system prompt" (the current default in many agent frameworks) to ExpSeek's triggered guidance provides a substantial accuracy lift without changing the underlying models or tools. The cross-benchmark generalization (+12.4% on xbench, +8.8% on Seal-Hard from WebWalkerQA-trained experience) suggests the experience base does not need to be rebuilt for every new deployment domain, reducing the upfront construction cost.

Improving sampling diversity for LLM-as-a-judge and self-consistency pipelines. ExpSeek's Pass@3 gains (+12.9% on 8B, Table 5) exceed its Pass@1 gains (+9.3%), indicating it substantially improves the diversity of correct solutions in the sampling distribution. For applications that use self-consistency (majority voting over multiple sampled trajectories) or LLM-as-a-judge evaluation (where a judge model selects among candidate answers), this increased sampling diversity directly translates to improved final accuracy without changing the selection mechanism. A practitioner running a self-consistency pipeline with an 8B web agent would expect 3-shot majority voting accuracy to improve by more than the 9.3% single-sample gain, since more trajectories now contain the correct answer. The paper's actuator RL suggestion — using ExpSeek as a rollout augmentation strategy — applies equally to offline data generation pipelines: generating training data for distillation or fine-tuning with ExpSeek-guided rollouts should produce higher-quality datasets than vanilla sampling, since more trajectories reach correct answers and the diversity of solution paths is higher.

Real-time debugging and monitoring of agent behavior in production. The entropy trigger mechanism provides a naturally interpretable monitoring signal. When the entropy monitor fires, it indicates — based on the agent's own confidence — that the current step may be erroneous. In a production setting where agents handle user queries autonomously, these trigger events could be logged and used for: (1) real-time escalation to human reviewers when intervention frequency exceeds a threshold for a single query, (2) post-hoc auditing of agent behavior (focusing human review on steps where the agent knew it was uncertain), and (3) continuous improvement of the experience base by collecting examples of steps that triggered but still failed, using them to generate new triplets. The 69.3% rejection rate on GAIA (Table 3) means most steps pass without triggering, keeping the monitoring overhead low, while the adaptive increase in trigger rate on harder benchmarks (25.6% higher on xbench) means the system naturally escalates attention when tasks are challenging. This is a lightweight alternative to deploying a separate critic model for every agent step, which the RM-based trigger ablation showed to be accurate but 2.2-2.9× slower.

When to Prefer This Method

The paper does not articulate an explicit tradeoff matrix positioning ExpSeek against named alternatives under specific operational conditions. The comparison in Table 3 (entropy-based vs. RM-based vs. rule-based triggering) and the baseline comparisons in Table 2 (vs. passive global injection) are empirical evaluations rather than prescriptive decision rules. The paper does not, for example, state conditions under which a practitioner should prefer passive injection over ExpSeek, or under which they should use an external reward model despite its cost. The abstract and conclusion frame ExpSeek as a general improvement over the passive paradigm rather than as one option in a tradeoff space.

What the paper does provide: Section 6.4 and Figure 6 show that accuracy plateaus after approximately 2-6 interventions per question, with diminishing returns beyond that. This implies ExpSeek is most valuable when the natural intervention rate (determined by the entropy threshold) falls in this range — frequent enough to correct genuine errors, infrequent enough to avoid redundancy. The paper also demonstrates (Table 3) that the entropy-based trigger is preferable to RM-based triggers when inference latency matters (1.0× vs. 2.2-2.9× time overhead) and to rule-based triggers when step efficiency matters (1.0× vs. 1.5-1.7× step overhead). These are implicit tradeoffs grounded in the data, but the paper does not formalize them as a decision framework or define operational boundaries (e.g., minimum training set size for reliable thresholds, maximum acceptable accuracy gap for using a 4B vs. 235B experience model).

Given the paper's empirical framing — demonstrating the superiority of proactive seeking over passive injection across four benchmarks and two model scales — constructing a "prefer A when, prefer B when" matrix would impose a prescriptive structure that the paper's own analysis does not provide. The results consistently show ExpSeek outperforming all compared alternatives, without identifying regimes where alternatives are preferable (except in cost, where the 4B experience model is cheaper than the 235B default). Any tradeoff matrix would therefore be extrapolation rather than faithful representation of the paper's claims.