Learning to Retrieve from Agent Trajectories

🎯 Pitch

Search agents don’t use retrieval like humans do—they browse, reject, and reason over documents across multi-turn loops, yet today’s retrievers are still trained on human click data. This paper shows that mining signals like browsing actions and post-browse reasoning length from agent trajectories yields training supervision that boosts task success by 28.6% across six agent backbones, completely sidestepping the need for expensive human labels.

1. Executive Summary

This paper introduces learning to retrieve from agent trajectories as a new training paradigm that addresses a fundamental mismatch between human-centric retrieval training and agentic search, where retrievers are consumed by LLM-powered agents within multi-turn reasoning loops rather than by humans. Through systematic analysis of deep research agent trajectories on InfoSeekQA using Tongyi-DeepResearch-30B, the authors identify key behavioral signals—browsing actions as necessary positive indicators, unbrowsed documents as reliable negatives without position bias, and post-browse reasoning length as a proxy for relevance intensity (analogous to dwell time in human search)—and propose LRAT, a framework that mines query-document supervision from search-browse transitions, filters positives via LLM-based reasoning trace verification, and incorporates relevance intensity through weighted contrastive learning. Experiments across six agent backbones spanning 4B to 358B parameters on both in-domain (InfoSeek-Eval) and out-of-domain (BrowseComp-Plus) benchmarks demonstrate consistent improvements, with an average 28.6% gain in task success rate and 27.5% gain in evidence recall, establishing that agent trajectories can serve as a scalable, practical supervision source for retrieval training while also sustaining a self-improving data flywheel—though the approach depends on the agent producing sufficiently rich interaction traces.

2. Context and Motivation

The Core Problem: Retrievers Are Trained for Humans but Consumed by Agents

The fundamental problem this paper addresses is a mismatch between training and deployment in modern information retrieval systems. For decades, the IR community has built and optimized retrieval models—from early probabilistic models like BM25 to modern dense embedding retrievers—under a single, unquestioned assumption: the end user is a human being. This assumption is baked into every layer of the training pipeline. Learning-to-rank methods are trained from human click logs, human dwell-time signals, and human relevance judgments. The objective functions, evaluation metrics, and even the implicit biases we correct for (like position bias, where humans disproportionately click top-ranked results regardless of relevance) all assume a human consumer with human behavioral patterns.

The paper argues, compellingly, that this assumption has become not just outdated but actively harmful. The primary consumers of search results are increasingly not humans at all, but LLM-powered search agents—autonomous systems that issue queries, consume retrieved documents, reason about their contents, and iteratively refine their search strategies within multi-turn reasoning-and-action loops. These agents, exemplified by systems like Search-o1, Search-R1, and Tongyi-DeepResearch, embed retrieval as a core tool within a broader problem-solving architecture. The retriever is no longer a standalone endpoint that serves a final ranked list to a person; it is a component within an agent's cognitive process, feeding information into reasoning states that determine subsequent actions.

The mismatch manifests in concrete, measurable ways. Consider what happens when a human issues a search query versus when an agent does:

Humans typically issue queries to satisfy an immediate informational need. Their queries are often short, sometimes ambiguous, and driven by a desire to find a specific piece of information or answer. The relevance of a retrieved document is judged by whether it directly addresses the query intent. Behavioral signals—clicks, dwell time, scrolling—reflect this judgment.
Agents, by contrast, issue intermediate queries that are not endpoints in themselves but tools for advancing a reasoning process. An agent researching "Which station merged with Immaculate Heart Radio in 2017?" might issue a query like "Immaculate Heart merged 2017" not because it wants to read about radio station mergers, but because it needs to identify a specific entity to answer the original question. The relevance of a retrieved document is judged not by whether it matches the surface-level query intent, but by whether it provides useful evidence for the downstream reasoning task—a fundamentally different and more complex criterion.

Moreover, agents exhibit behavioral patterns that violate the assumptions embedded in human-centric retrieval models. As the paper's trajectory analysis reveals (Section 4), agents browse documents distributed relatively uniformly across ranking positions, not concentrated at the top like human clicks. They frequently inspect and reject documents after examining snippets, and they generate explicit reasoning traces that reveal why a document was useful or not. These behaviors are not noise to be corrected for—they are rich signals that a human-centric model cannot interpret because it was never designed to.

The consequence of this mismatch is that retrieval has become the primary performance bottleneck for search agents. Even state-of-the-art agents with sophisticated reasoning capabilities are hamstrung by retrievers that return documents optimized for human satisfaction rather than agent utility. The paper's Figure 1 quantifies this: across six different agent backbones spanning three orders of magnitude in parameter count (4B to 358B), simply replacing a base retriever with an LRAT-trained retriever yields an average 28.6% improvement in task success rate on InfoSeek-Eval. This is not a marginal gain—it demonstrates just how severely misaligned the base retrievers are with agent needs.

Why This Problem Matters: Practical and Theoretical Significance

The misalignment between human-centric retrievers and agent consumers is not merely an academic curiosity. It has immediate practical consequences for the rapidly growing ecosystem of agentic search applications, and it exposes a conceptual gap in how the IR community thinks about retrieval training.

Practical Impact: Retrieval as the Bottleneck in Agentic Search

The deployment of LLM-powered search agents is accelerating across industry and research. Deep research agents are being used for complex information-seeking tasks that require multi-step reasoning, evidence synthesis, and iterative refinement—tasks like competitive intelligence gathering, scientific literature review, legal research, and investigative journalism. In these applications, the agent's ability to succeed is directly constrained by what the retriever surfaces. If the retriever returns documents that are topically relevant but evidentially useless, the agent wastes turns browsing irrelevant content, consumes its step budget without making progress, and either fails or produces an incorrect answer.

The paper's trajectory analysis makes this concrete. Table 1 shows a striking pattern: in incorrect trajectories, agents issue far more [Search] actions (average 34.68 vs. 11.77 for correct trajectories) but browse far fewer documents relative to their search volume (B/S ratio of 0.20 vs. 0.31). This means failing agents are stuck in a search-only loop—they keep issuing queries because the retrieved snippets are never compelling enough to warrant browsing. The retriever is returning documents that look plausible from their snippets but fail to provide the specific evidence the agent needs. The agent recognizes this implicitly (by not browsing) but has no mechanism to communicate back to the retriever what would actually be useful.

This bottleneck will only intensify as agents become more capable. A more sophisticated reasoning engine can formulate better sub-queries and integrate evidence more effectively, but it cannot conjure evidence that the retriever never surfaces. As the paper demonstrates, even the largest agents tested (GLM-4.7 at 358B parameters, MiniMax-M2.1 at 229B) show substantial gains from LRAT-trained retrievers—proving that retrieval quality, not agent capability, is the limiting factor.

Theoretical Significance: A New Supervision Paradigm for IR

Beyond the practical bottleneck, the paper addresses a conceptual gap in how retrieval models are trained. The traditional supervision sources for learning to rank fall into two categories:

Explicit relevance judgments: Human annotators manually assess document relevance for a set of queries, providing clean but expensive and non-scalable labels. This has been the gold standard since the early days of IR (Cao et al., 2006; Liu et al., 2009) but is fundamentally limited by annotation cost and the static nature of the judgments—they capture a single assessor's judgment at a single point in time, not the dynamic, context-dependent utility that matters to an agent mid-task.
Implicit user feedback: Click logs, dwell time, scrolling patterns, and other behavioral signals provide abundant and naturally occurring supervision (Joachims, 2002; Joachims et al., 2005; Kelly and Belkin, 2004). This approach solved the scalability problem but introduced new ones: position bias (users click top results regardless of relevance), selection bias (users can only click what they see), and the fundamental ambiguity of negative signals (does "not clicked" mean "irrelevant" or simply "not examined"?).

Neither paradigm was designed for agents as the primary consumer. Human relevance judgments reflect human notions of topical relevance, not agent utility. Human click logs encode human behavioral biases that agents do not share. Training a retriever on human click data and deploying it to serve agents is like training a restaurant recommendation system on Yelp reviews and expecting it to work for a nutritionist planning therapeutic diets—the consumer's needs, behaviors, and evaluation criteria are fundamentally different.

The paper's key theoretical contribution is identifying agent trajectories as a third supervision paradigm that is both scalable (trajectories are a natural byproduct of every agent invocation) and aligned with the actual consumer (the training signal comes from the same type of entity that will use the retriever at deployment). This is not merely a new dataset or a new training trick—it is a reconceptualization of what supervision means for retrieval systems in the agent era.

Prior Approaches and Where They Fall Short

The paper situates its contribution against several lines of prior work, each of which addresses part of the problem but leaves the fundamental misalignment unresolved.

Static Retrievers in Agentic Search (Section 2, "Search Agent & Retriever Optimization for RAG")

The dominant approach in current search agent systems is to treat the retriever as an off-the-shelf, frozen component. Agents like Search-o1 (Li et al., 2025), Search-R1 (Jin et al., 2025), and Tongyi-DeepResearch (Team et al., 2025b) all rely on existing retrieval infrastructure—Google Search API, BM25, or pretrained dense embedding models—without any attempt to adapt the retriever to the agent's specific needs. The research focus has been almost exclusively on improving the agent: better reasoning architectures, more effective action selection, improved multi-turn planning. The retriever is treated as part of the environment, not as something to be optimized.

This approach has a clear limitation: no amount of agent improvement can compensate for a retriever that surfaces the wrong documents. An agent can be perfectly calibrated in its reasoning, brilliant in its sub-query formulation, and optimal in its browse decisions—but if the retriever never returns the evidence document in the first place, the agent has no path to success. The paper's empirical results make this concrete: simply swapping the retriever (keeping the agent identical) produces 17–38% relative improvements in success rate (Table 2). This is direct evidence that the retriever, not the agent, is the bottleneck.

Retriever Optimization for Single-Turn RAG (Section 2)

The Retrieval-Augmented Generation (RAG) literature has explored retriever optimization, but almost exclusively in the single-turn setting. Methods like REPLUG (Shi et al., 2024), preference alignment approaches (Dong et al., 2025; Qu et al., 2025), and utility-oriented training (Salemi and Zamani, 2024; Zamani and Bendersky, 2024) all assume a fixed user query and optimize the retriever to surface documents that help a language model produce a correct answer in one shot. The supervision signal typically comes from comparing the model's output against a ground-truth answer and back-propagating utility through the retrieval step.

This paradigm breaks down completely in the multi-turn agent setting for two reasons:

First, agent queries are intermediate, not final. An agent investigating "Which station merged with Immaculate Heart Radio in 2017?" might issue queries like "Immaculate Heart Radio corporate history," "Relevant Radio merger 2017 announcement," and "WMJR Nicholasville Kentucky call sign history." None of these sub-queries has a "correct answer" in the traditional sense—their purpose is to gather evidence toward the final answer, not to be self-contained Q&A pairs. Single-turn RAG optimization methods that require a ground-truth answer for each query cannot operate here, because the ground truth is only defined for the original user question, not for the agent's intermediate steps.

Second, document utility is context-dependent and cumulative. A document that seems irrelevant when retrieved for one sub-query might prove crucial when combined with evidence from a later step. An agent's judgment of document utility emerges from its full reasoning trajectory—the document is useful if it contributes to eventual task success, not if it matches the surface semantics of the query that retrieved it. Single-turn optimization cannot capture this trajectory-level utility signal.

The paper explicitly distinguishes itself from this prior work (end of Section 2): "In contrast to prior work, our work shifts the focus from merely refining the agent to optimizing the retriever, enabling it to respond effectively to the agents' information needs as they evolve across multiple turns."

Learning to Retrieve/Rank from Human Feedback (Section 2)

The traditional learning-to-rank literature is the most mature body of prior work, but it is fundamentally human-centric in its assumptions. Methods based on explicit relevance judgments (Cao et al., 2006, 2007; Liu et al., 2009) assume human assessors applying human relevance criteria. Methods based on implicit feedback—clicks (Joachims, 2002; Joachims et al., 2005), dwell time (Kelly and Belkin, 2004; Kim et al., 2014), scrolling (Agichtein et al., 2006a,b)—assume human behavioral patterns and explicitly model human-specific biases like position bias and trust bias.

The paper's trajectory analysis (Section 4.2) reveals just how different agent behavior is from human behavior, and why methods designed for humans fail when applied to agent data:

Position bias reversal: Figure 4(c) shows that agent browsing actions are distributed relatively uniformly across ranking positions, in stark contrast to the well-documented human tendency to click predominantly on top-ranked results (Craswell et al., 2008). This means that classical click models, which spend enormous effort modeling and debiasing position effects (e.g., the Examination Hypothesis, cascade models), are solving a problem that doesn't exist for agent data. Conversely, the heuristics developed for human click logs—like "skip-above" sampling, which treats unclicked documents above the last click as pseudo-negatives (Joachims et al., 2005)—would be actively harmful if applied to agent trajectories, because they would incorrectly penalize documents at higher ranks that the agent examined and genuinely rejected.
Negative signal reliability: The ambiguity of negative signals is one of the hardest problems in learning from human clicks. An unclicked document might be irrelevant, or it might simply not have been seen (due to position bias, screen size, or user impatience). The paper's analysis in Section 4.2.2 provides evidence that agent unbrowsed documents do not suffer from this ambiguity: because agents actively evaluate all candidates (as shown by the uniform position distribution), an unbrowsed document genuinely reflects explicit rejection after snippet inspection. This is a stronger signal than anything available in human click logs, and it enables simpler, more aggressive negative sampling strategies.
Dwell time vs. reasoning length: Human dwell time on a document is a well-known proxy for relevance (Kelly and Belkin, 2004; Kim et al., 2014)—longer reading time generally indicates higher interest or utility. The paper identifies a striking agent-analog of this phenomenon: the length of the agent's post-browse reasoning trace (Figure 4d). When an agent browses a genuinely useful document, it engages in substantially longer reasoning about how to integrate that evidence into its ongoing investigation. When it browses an unhelpful document, it quickly dismisses it with a short reasoning trace. This provides a continuous, fine-grained relevance signal that is strictly richer than the binary click/no-click signal from human logs.

These differences mean that existing learning-to-rank methods cannot simply be ported to agent trajectories. The assumptions they encode about user behavior are not just different—they are actively counterproductive. A click model trained to correct for position bias would systematically misinterpret the uniform browsing distribution as evidence of low-quality results. A dwell-time model trained on human reading patterns would misunderstand the semantic significance of reasoning trace length.

How This Paper Positions Itself

The paper positions its contribution along three axes, each addressing a gap in prior work:

1. A New Data Source for Retrieval Training

The central claim is that agent trajectories constitute a supervision source that is simultaneously more aligned and more scalable than existing alternatives. They are more aligned than human relevance judgments because they come from the same type of entity (an agent) that will consume the retrieval results at deployment time. They are more scalable than human judgments because they are generated as a natural byproduct of every agent invocation—no additional annotation cost is incurred. And they are richer than human click logs because they contain explicit reasoning traces that reveal not just whether a document was useful but why and how much.

The paper draws an explicit historical analogy (Section 1) to the transition from manual relevance judgments to implicit user feedback: just as click logs revolutionized retrieval training by providing abundant, naturally occurring supervision, agent trajectories may represent the next evolution—the "agent-era counterpart of user click logs." This analogy is rhetorically effective but also conceptually precise: in both cases, the key insight is that the interaction between user and system generates supervision signals that are implicitly aligned with the user's true utility function, if only we can learn to interpret them correctly.

2. A Framework for Mining Supervision from Trajectories

The paper does not merely argue that agent trajectories could be useful—it provides a concrete, multi-stage framework (LRAT) for extracting high-quality training supervision from raw trajectories. The framework is explicitly guided by empirical analysis of agent behavior (Section 4), not by a priori assumptions about what should work. Each component of LRAT—naive relevance mining from search-browse transitions, reasoning-aware positive filtering, and intensity-aware weighted training—is justified by a specific empirical finding from the trajectory analysis:

Browsing as a necessary condition (Section 4.2.1) motivates using browsed documents as positives. The analysis shows that successful trajectories have substantially higher browse-to-search ratios, and that zero evidence documents browsed implies zero chance of success.
Unbrowsed documents as reliable negatives (Section 4.2.2) motivates the simple but effective negative sampling strategy of treating all unbrowsed candidates in a retrieved set as negatives, without position bias correction.
Post-browse reasoning as a utility signal (Section 4.2.3) motivates both the filtering step (using an LLM judge to remove browsed-but-unhelpful documents) and the intensity weighting (using reasoning length as a proxy for document importance).

This evidence-driven design distinguishes LRAT from approaches that might naively treat all agent actions as equally informative. The paper acknowledges that browsing alone is a noisy signal (agents sometimes browse and then reject documents) and that the real value lies in the reasoning traces that explain why the agent acted as it did.

3. Empirical Validation Across Diverse Settings

The paper's experimental design (Section 6) is deliberately comprehensive, testing LRAT across:

Two retriever architectures: an encoder-based model (Multilingual-E5-Large-Instruct) and a decoder-based model (Qwen3-Embedding-0.6B), demonstrating that the approach is not tied to a specific architecture.
Six agent backbones spanning 4B to 358B parameters: three task-optimized search agents (AgentCPM-Explore, WebExplore, Tongyi-DeepResearch) and three generalist agentic foundation models (GPT-OSS, MiniMax-M2.1, GLM-4.7), demonstrating that the benefits persist regardless of agent design or scale.
Two benchmarks: in-domain (InfoSeek-Eval) and out-of-domain (BrowseComp-Plus), demonstrating that the training signal generalizes beyond the specific queries used for trajectory collection.

This breadth of evaluation is important because it addresses a natural skepticism: perhaps agent-specific retriever training only helps when the agent and retriever are closely coupled, or only on the exact same task distribution used for training. The results systematically refute this: LRAT-trained retrievers improve performance even for agents that were not involved in trajectory generation (e.g., GPT-OSS, MiniMax-M2.1, GLM-4.7), and even on out-of-domain benchmarks with different query distributions and evidence requirements.

4. The Data Flywheel Vision

Beyond the immediate empirical gains, the paper articulates a broader vision: agent trajectories can sustain a self-improving data flywheel for retrieval systems. This is the conceptual extension of the click-log analogy. Just as search engines continuously improve their ranking models by training on fresh user click data, agent-based systems could continuously improve their retrievers by training on fresh agent trajectories. The paper provides preliminary evidence for this vision through a simulated flywheel experiment (Section 6.5, Figure 9), showing that retrievers iteratively trained on agent-generated trajectories show steady improvement across multiple loop steps.

Crucially, the paper also shows that the flywheel does not require perfect trajectories. The analysis in Table 3 demonstrates that training on incorrect trajectories (where the agent ultimately fails to produce the right answer) still yields improvements over the base retriever, though smaller than training on correct trajectories. This is significant because in realistic deployments, trajectory correctness labels may not be available, and many trajectories will be imperfect. The fact that even failed trajectories contain useful supervision signals (the agent's intermediate browsing and rejection decisions still reflect genuine utility judgments) makes the flywheel practical rather than merely aspirational.

The Key Insight That Distinguishes This Work

What makes this paper more than an incremental contribution is the reconceptualization of what constitutes supervision for retrieval training. Prior work, whether based on human judgments or implicit feedback, treated supervision as something that must be explicitly provided by an external oracle—either a human assessor or a measurable user action. This paper argues that in the agent era, supervision is latent in the structure of agent behavior itself. The agent's decision to browse one document and not another, to reason extensively about some retrieved content and dismiss others quickly—these are not merely actions to be optimized over, but judgments that reveal what the agent truly finds useful.

This shift in perspective has deep implications. If agent trajectories are a valid and sufficient supervision source, then retrieval models can be continuously adapted to the agents they serve without any manual annotation, without any ground-truth relevance labels, and without any assumptions about human search behavior. The retriever and agent can co-evolve—the retriever learns to surface what the agent needs, and the agent (if retrained or fine-tuned) can learn to formulate queries that the retriever can satisfy. This vision of retriever-agent co-adaptation is not realized in the current paper (the agents are frozen during retriever training), but it is the natural endpoint of the research direction the paper opens.

3. Technical Approach

3.1 Reader Orientation

The system is a retriever training pipeline that takes raw execution traces from LLM-powered search agents and converts them into high-quality training data for dense embedding models—without requiring any human annotations or ground-truth relevance labels. It solves the problem that retrievers trained on human click data perform poorly when serving agents, because agents issue different types of queries and judge document usefulness differently from humans. The solution's shape is a multi-stage mining and training framework that starts with coarse behavioral signals (which documents did the agent browse?), progressively refines them using reasoning traces (did the agent actually find the document useful?), and finally trains the retriever with continuous importance weights derived from the depth of agent engagement (how much did the agent reason about this document?).

3.2 Big-Picture Architecture (Diagram in Words)

The LRAT system has five major components arranged as a pipeline from raw data to trained model:

Trajectory Generator — runs a deep research agent (Tongyi-DeepResearch-30B) on seed queries with a frozen base retriever, producing multi-turn execution traces containing search queries, retrieved document lists, browse decisions, and reasoning traces. This is the data source, and it produces raw trajectories as output.
Naive Relevance Miner — scans each trajectory for [Search] → [Browse] transitions, treating browsed documents as candidate positives and unbrowsed documents in the same retrieved set as negatives. No content analysis, just action-sequence signals. Output: coarse (query, positive_doc, negative_set) triples.
Reasoning-Aware Filter — takes the coarse positives from the miner and feeds each (query, browsed_document, post-browse_reasoning_trace) to an LLM judge, which classifies whether the reasoning trace indicates the document was genuinely useful or merely skimmed and rejected. Filters out false positives. Output: cleaned positives.
Relevance Intensity Estimator — computes a continuous weight for each positive document by measuring the token length of the post-browse reasoning trace and mapping it through an exponential saturation function (analogous to dwell-time models in human search). Longer reasoning → higher weight, with diminishing returns. Output: scalar weight per positive instance.
Weighted Contrastive Learner — fine-tunes a dense bi-encoder retriever using a modified InfoNCE loss where each positive pair's loss contribution is scaled by its relevance intensity weight. Negatives come from both the trajectory (unbrowsed documents) and the mini-batch (other queries' documents). Output: a trained retriever.

Information flows sequentially: raw agent executions → trajectory logs → naive mining → reasoned filtering → intensity weighting → model training. The only external component is the LLM judge used in the filtering step, which is a separate frozen model (Qwen3-30B-A3B-Thinking-2507).

3.3 Roadmap for the Deep Dive

First, the trajectory generation setup (Section 4.1, but its components are essential infrastructure for LRAT)—what environment produces the raw data, what retriever is used during collection, what agent generates the traces, and what the resulting dataset looks like. This establishes the "raw material" that LRAT processes.
Second, the naive relevance mining procedure (Section 5.1.1)—how search-browse transitions are converted into positive and negative document labels. This is the foundation: it defines what supervision is extractable from action sequences alone, before any content analysis.
Third, the reasoning-aware positive filtering step (Section 5.1.2)—how post-browse reasoning traces are used to remove false positives among browsed documents, and why an LLM judge is needed rather than simpler heuristics.
Fourth, the relevance intensity estimation scheme (Section 5.2.1)—how reasoning trace length is mapped to a continuous importance weight using an exponential saturation function, the mathematical form of this mapping, and its justification by analogy to human dwell-time models.
Fifth, the weighted contrastive learning objective (Section 5.2.2)—how the mined supervision and intensity weights are combined into a training loss, the architecture of the dense retriever being trained, and the negative sampling strategy.

This order mirrors how data is processed in LRAT: raw traces → coarse labels → refined labels → importance weights → model training. Each step depends on the output of the previous step, so understanding them in sequence builds the complete picture.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a data mining and training framework paper whose core idea is that agent execution trajectories contain latent relevance judgments—revealed through browsing decisions, rejection patterns, and reasoning traces—that can be systematically extracted and converted into supervised training signals for retrieval models, without any manual annotation.

3.4.1 Trajectory Generation Environment (Raw Data Production)

Before explaining how LRAT mines supervision from trajectories, we must understand what a trajectory looks like and how it is produced. The paper generates trajectories using a specific agent, retriever, corpus, and query set—these choices determine the characteristics of the supervision signal that LRAT can extract.

Seed queries. The trajectory generation uses 10,000 queries from InfoSeekQA (Xia et al., 2025), a deep research benchmark containing over 50,000 question-answer pairs designed to require hierarchical reasoning and iterative information acquisition. Queries are selected based on having verified ground-truth answers, ensuring that trajectory correctness can be evaluated (though LRAT itself does not require correctness labels during training—this is for analysis and filtering in experiments, not a training requirement). InfoSeekQA queries are substantially harder than traditional QA benchmarks, producing "significantly longer interaction trajectories" than simpler datasets, which is important because LRAT needs trajectories with enough search-browse interactions to generate training pairs.

Corpus. All retrieval is performed over Wiki-25-Dump, comprising over 11.2 million document chunks, each truncated to 512 tokens. This is a standard Wikipedia-derived corpus, and the truncation to 512 tokens means the retriever only ever sees the first ~512 tokens of each document during both the embedding step and the snippet generation step. This length constraint shapes what information the agent can access: if critical evidence appears in the latter part of a long article, it will be invisible to both the retriever (which embeds only the first 512 tokens) and the agent (which sees only a snippet from the first 64 tokens before deciding whether to browse).

Base retrievers during collection. Four retrieval models are deployed during trajectory collection to generate diverse interaction patterns:

BM25: a sparse lexical retriever that matches on term overlap, representing traditional keyword-based retrieval.
Qwen3-Embedding-0.6B: a small dense retriever with 600 million parameters, decoder-based architecture.
Qwen3-Embedding-4B: a medium dense retriever with 4 billion parameters.
Qwen3-Embedding-8B: a large dense retriever with 8 billion parameters.

Each [Search] action returns the top-10 candidate documents. The agent observes, for each candidate, a short snippet consisting of the first 64 tokens of the document. This snippet length is explicitly chosen to approximate "the average length of web search snippets measured under the Qwen3 tokenizer," making the environment a realistic simulation of web search where agents initially rely on coarse-grained evidence (titles and brief text fragments) before deciding to expend the effort of browsing full documents. The key constraint is that the agent must make browse decisions based on 64-token snippets, not full documents—this is the same information asymmetry that exists in real web search, and it means the agent's browsing decisions implicitly encode judgments about which snippets are informative enough to warrant deeper inspection.

Agent configuration. The agent is Tongyi-DeepResearch-30B-A3B, a 30-billion-parameter mixture-of-experts model specifically designed for long-horizon deep research tasks and supporting over 100 interaction steps. The maximum number of interaction rounds is set to T=100. If the agent fails to reach a conclusion within this limit, it must produce a final answer based on whatever information it has collected so far (forced termination). This ensures that every trajectory ends with an answer that can be verified against ground truth.

The agent follows a ReAct-style execution pattern (Yao et al., 2022), alternating between reasoning ([Think]), searching ([Search]), browsing ([Browse]), and answering ([Answer]). At each turn, the agent first produces an explicit reasoning trace (rt), then selects an action (at), executes it, and incorporates the observation (ot) into its state. A full trajectory T is a sequence of (reasoning, action, observation) tuples: T = {(rt, at, ot)}Tt=1.

Verification and filtering. After trajectory generation, answer correctness is verified by comparing the agent's final output against the InfoSeekQA ground truth using Qwen3-30B-A3B-Thinking-2507 as an automated judge. Trajectories that exceed the 100-step limit or produce incorrect final answers are filtered out, leaving only completed, correct trajectories for the primary training set. The paper uses 26,482 correct trajectories for training (Table 1, Total row for correct trajectories), plus 8,137 incorrect trajectories that are used in auxiliary experiments (Section 6.5, Table 3) to show that even imperfect trajectories provide useful supervision.

Resulting dataset characteristics. Table 1 provides detailed statistics of the generated trajectories. Across all four retrievers, correct trajectories average 11.77 [Search] actions and 3.61 [Browse] actions, yielding a browse-to-search ratio (B/S) of 0.31 and an average of 15.38 total steps. Incorrect trajectories are substantially longer (average 34.68 searches, 6.95 browses, B/S ratio 0.20, 41.63 total steps), reflecting the pattern that failing agents get stuck in search loops without finding browsable content. The full dataset (correct + incorrect) comprises 34,619 trajectories averaging 17.25 searches, 4.41 browses, and 21.66 steps.

These numbers matter for LRAT because they quantify how much supervision is available: each correct trajectory provides, on average, 3.61 positive training instances (one per browsed document), along with associated negative instances from unbrowsed candidates in each search turn. The total training data extracted from 26,482 correct trajectories yields 91,713 training pairs after filtering (as stated in Section 1 and confirmed by the ablation in Figure 7, where "+Filter" and "+Reweight" both use n=91,713).

3.4.2 Naive Relevance Mining from Search-Browse Transitions

The first stage of LRAT extracts coarse supervision from the raw action sequences in trajectories. This stage operates entirely on behavioral signals—what the agent did, not what it said or thought. It does not analyze reasoning traces or document content. It simply identifies patterns in the action sequence and converts them into training labels.

The core signal: Search → Browse transitions. Consider a turn t where the agent performs a [Search] action, issuing an intermediate query qt and receiving a ranked candidate set Dt = {dt,1, dt,2, ..., dt,K} with K=10 documents. If, in the immediately following turn (t+1), the agent performs a [Browse] action on one specific document from this set—call it dt+1—then the paper treats this document as a naive positive for query qt. The reasoning, grounded in the trajectory analysis from Section 4.2.1, is that browsing is a necessary precondition for successful task completion: the analysis showed that task success increases monotonically with the number of browsed evidence documents and drops to zero when no evidence documents are browsed. Therefore, a document that the agent chooses to browse is, at minimum, one that passed the agent's initial snippet-level relevance assessment and warranted the expenditure of a valuable interaction turn on full-text inspection.

Negative construction: unbrowsed candidates. For the same search turn, any document in Dt that is not browsed is treated as a negative. Formally, the negative set Nt for browsed document dt+1 is:

$N_t = D_t \setminus \{d_{t+1}\}$

where Dt is the full set of K=10 documents returned for query qt, and dt+1 is the single document that the agent chose to browse.

Why this negative sampling is reliable. This is a critical design choice that the paper justifies through empirical analysis (Section 4.2.2). In human click logs, treating unclicked documents as negatives is dangerous because of position bias: humans disproportionately click top-ranked results regardless of relevance, so an unclicked document at rank 3 might be perfectly relevant but simply unseen. The standard remedy is "skip-above" sampling—treat only documents ranked above the last click as pseudo-negatives, conservatively assuming documents below the click were never examined.

The paper shows that agent browsing behavior does not exhibit this position bias. Figure 4(c) demonstrates that browsing actions are distributed relatively uniformly across ranking positions, not sharply concentrated at the top. This means the agent actively evaluates all 10 candidates and makes a deliberate decision about which one to browse, based on snippet content rather than rank position. An unbrowsed document is therefore likely the result of explicit rejection after inspection, not limited exposure. The paper states: "unbrowsed documents are typically the result of explicit rejection after inspection, rather than limited exposure. Therefore, in contrast to human click logs, all unbrowsed documents within a retrieved candidate set can be treated as reliable negatives without requiring position bias correction."

What this yields. Each [Search] → [Browse] transition produces one training instance of the form:

Query: qt (the intermediate search query the agent issued)
Positive document: dt+1 (the document the agent chose to browse)
Negative set: Nt (the other 9 candidates in Dt that were not browsed)

A single trajectory with B browse actions yields B such training instances. Across 26,482 correct trajectories averaging 3.61 browses each (Table 1), this yields approximately 95,600 naive training instances—before any filtering. The ablation in Figure 7 reports 120,579 training instances at the "+Naive" stage (which also includes data from additional collection rounds), consistent with this estimate.

What this stage does NOT do. The naive miner does not:

Check whether the browsed document actually helped the agent (it might have been browsed and then rejected as unhelpful)
Distinguish between documents that received deep engagement versus superficial skimming
Use any information from the agent's reasoning traces—only the action sequence is used
Require any document content analysis or LLM judging

It is intentionally coarse, trading precision for recall: it captures nearly all documents that might be useful (since browsing is necessary for document utility) but also includes documents that the agent browsed and then dismissed. The next stage addresses this noise.

3.4.3 Reasoning-Aware Positive Filtering

The naive miner over-generates positives because browsing alone is an imperfect proxy for document utility. The trajectory analysis in Section 4.2.3 showed that agents sometimes browse documents and then quickly determine they are not useful, producing short dismissive reasoning traces. These browsed-but-unhelpful documents would become false positives if included in training—they would train the retriever to surface documents that look promising from snippets but fail to deliver useful content.

The solution: use post-browse reasoning as a verifier. Immediately after every [Browse] action, the agent produces a reasoning trace (rt+2 in the trajectory sequence) that analyzes the just-read document and integrates it into the ongoing investigation. The key insight from Section 4.2.3 is that these reasoning traces contain explicit statements about document utility. When a document is genuinely useful, the reasoning trace is long and substantive—it extracts specific facts, connects them to the research question, and plans next steps based on the new information. When a document is unhelpful, the reasoning trace is short and dismissive—it states that the document does not contain the needed information and moves on.

LRAT leverages this by applying an LLM-based verifier to classify each naive positive. For each (qt, dt+1) pair identified by the naive miner, the procedure is:

Retrieve the agent's post-browse reasoning trace rt+2 (the reasoning produced immediately after browsing dt+1).
Feed the query qt, the document dt+1, and the reasoning trace rt+2 to an LLM judge.
The judge classifies the document as Relevant or Irrelevant based on whether the reasoning trace indicates that the document content was used to support progress on the task.

The LLM judge used is Qwen3-30B-A3B-Thinking-2507, the same model used for answer verification. The paper does not provide the exact prompt used for this filtering step, but the criterion is described as: "determine whether the reasoning explicitly uses the document content to support progress on the task."

What this filtering achieves. The paper validates the filter's performance on BrowseComp-Plus, which provides ground-truth evidence document annotations (each query has a known set of documents that contain the necessary evidence). The results:

Recall of true evidence documents: 97.2%. The filter correctly identifies nearly all documents that actually contain evidence, ensuring that genuine positives are almost never discarded. This is crucial because losing true positives is far more damaging than retaining some false positives—the weighted training (next stage) can down-weight low-utility documents, but it cannot recover documents that were filtered out entirely.
Retention of browsed non-evidence documents: 74.8%. The filter retains about three-quarters of the documents that the agent browsed but that do not contain ground-truth evidence. This means it removes 25.2% of non-evidence browsed documents as clear noise, but keeps the majority. The paper interprets the retained non-evidence documents not as failures but as capturing "agent-specific utility that may go beyond rigid dataset evidence labels"—documents that, while not explicitly annotated as evidence, the agent genuinely found useful for its reasoning process.

Why LLM judging rather than simpler heuristics. The paper considered alternatives but settled on LLM-based filtering for two reasons. First, reasoning traces are free-form natural language, not structured data—a simple length threshold (e.g., "filter out documents with reasoning traces shorter than X tokens") would be brittle, because a short trace could still indicate utility if expressed concisely, and a long trace could indicate confusion rather than utility. The LLM judge can interpret the semantic content of the reasoning, distinguishing between "this document doesn't help" (dismissal) and "this document tells us that..." (genuine engagement). Second, the LLM judge provides a binary label that cleanly separates training instances into keep/discard, making the filtering step a simple gate before the continuous weighting applied in the next stage.

The filtering reduces training data size. The ablation in Figure 7 shows that "+Filter" uses 91,713 training instances, down from 120,579 at the "+Naive" stage—a reduction of roughly 24%, consistent with filtering out about a quarter of naive positives. This means the filtering step removes approximately 29,000 likely-false-positive instances while preserving 91,713 high-quality training pairs.

3.4.4 Reasoning-Length Induced Relevance Intensity Estimation

After filtering removes clearly unhelpful documents, LRAT addresses a subtler problem: not all useful documents are equally useful. A document that provides a crucial missing piece of evidence and triggers extensive agent reasoning is more valuable than one that provides minor corroborating detail. Treating all positives as equally relevant would ignore this heterogeneity and produce a retriever that cannot distinguish between "nice to have" and "essential" documents.

The trajectory analysis in Section 4.2.3 provides the signal needed to make this distinction: post-browse reasoning length. Figure 4(d) shows that evidence documents elicit markedly longer reasoning traces than non-evidence documents, and that correct trajectories have longer reasoning traces overall. Qualitatively, when an agent browses a highly useful document, it engages in extensive reasoning: extracting specific facts, connecting them to the research question, updating its understanding, and planning subsequent actions. When it browses a marginally useful document, the reasoning is shorter—it might note one relevant fact and move on. This relationship between reasoning length and document utility is the agent-era analog of the well-established relationship between human dwell time and document relevance in web search (Kelly and Belkin, 2004; Kim et al., 2014).

The paper formalizes this intuition mathematically by adapting the time-aware click model from Liu et al. (2016), which models human click satisfaction as a function of dwell time. The adaptation substitutes reasoning length for dwell time and uses the same mathematical form: an exponential saturation function that captures diminishing returns.

Step 1: Define a marginal gain function. The time-aware click model assumes that the marginal gain from each additional unit of dwell time follows an exponentially decaying function. LRAT adopts this directly for reasoning length:

$g(x) = \exp\left(-\frac{\ln 2}{\beta} x\right)$

where x represents reasoning length (in tokens), β is a half-life parameter controlling how quickly additional reasoning length becomes less informative, and ln 2 normalizes so that β is the length at which marginal gain drops to half its initial value.

What this function represents: g(x) is the incremental value contributed by the x-th token of reasoning. The first few tokens of reasoning are highly informative—they establish whether the agent found the document useful at all. Additional tokens beyond the first few provide progressively less new information about utility, because once the agent has demonstrated engagement, the marginal signal weakens. The exponential decay captures this pattern: early tokens contribute more to the utility estimate than later tokens.

Step 2: Integrate to get cumulative utility. The total utility u(l) contributed by a reasoning trace of length l is the integral of the marginal gain from 0 to l:

$u(l) = \int_0^l g(x) \, dx = \frac{\beta}{\ln 2}\left(1 - \exp\left(-\frac{\ln 2}{\beta} l\right)\right)$

where l is the token length of the reasoning trace immediately following the browse action, and β remains the half-life parameter.

What this computes: u(l) is a scalar between 0 and β/ln 2 that quantifies the total estimated utility of a document based on how long the agent reasoned about it. For very short reasoning (l ≈ 0), u(l) ≈ 0—a document that is immediately dismissed contributes near-zero utility. As l grows, u(l) increases but saturates—reasoning for 200 tokens versus 100 tokens provides additional utility, but not twice as much, because the exponential form captures diminishing returns. The saturation behavior is important: it prevents very long reasoning traces from dominating training by having exponentially larger weights.

Step 3: Normalize to produce training weights. The raw u(l) values vary in scale depending on β and the dataset's typical reasoning lengths. To produce stable training weights, LRAT normalizes across the dataset:

$w = \frac{1}{\mu_{\text{raw}}}\left(1 - \exp\left(-\frac{\ln 2 \cdot l}{\beta}\right)\right)$

where w is the final relevance intensity weight for a training instance, μraw is the global mean of the unnormalized scores (1 - exp(-ln 2 · l / β)) computed across all training instances, and β is set to "the median reasoning length across all trajectories."

Why normalization matters. The division by μraw ensures that E[w] ≈ 1 across the training set. This means that documents with average utility receive weight near 1 (leaving their contribution to the loss unchanged relative to unweighted training), documents with above-average utility receive weight > 1 (increasing their influence), and documents with below-average utility receive weight < 1 (decreasing their influence). Without normalization, the absolute scale of weights would interact unpredictably with the learning rate and batch normalization in the retriever model, potentially causing training instability or requiring per-dataset hyperparameter tuning.

The paper omits the constant factor β/ln 2 from the final weight computation because, as stated: "For simplicity, we omit the constant factor β/ln 2, as the final weights are normalized across the dataset and the relative ranking of documents is preserved." This is valid because multiplying all weights by a constant does not change the relative importance of different instances.

Why this functional form rather than linear or threshold-based weighting. A linear mapping (w ∝ l) would over-weight very long reasoning traces, making the retriever excessively sensitive to documents that trigger unusually extended reasoning—which could be an artifact of agent verbosity rather than document utility. A threshold-based approach (w = 1 if l > threshold, else w = 0.5) would lose the continuous gradation that distinguishes between degrees of usefulness. The exponential saturation function provides a principled middle ground: it increases with reasoning length, so more engagement means higher weight, but the rate of increase diminishes, preventing outlier reasoning lengths from dominating. This form is directly inherited from the time-aware click model (Liu et al., 2016), which was developed for human dwell time and has been empirically validated in that domain.

The dwell-time analogy. The paper explicitly draws this parallel: "This phenomenon is analogous to classical human search, where dwell time has long been recognized as an effective proxy for relevance intensity. In both cases, increased cognitive effort reflects deeper engagement with the retrieved content." The analogy is not merely rhetorical—it provides theoretical justification for using the same mathematical form. Both human dwell time and agent reasoning length are measures of cognitive engagement with retrieved content, and both exhibit the same diminishing-returns pattern: the first few seconds (or tokens) are most informative, and extremely long engagement (or reasoning) rarely means proportionally higher utility.

Parameter selection: β = median reasoning length. Setting β to the median reasoning length across all trajectories means that half of all training instances have reasoning length l ≤ β and half have l > β. For these instances:

When l = β (exactly median), 1 - exp(-ln 2 · β / β) = 1 - exp(-ln 2) = 1 - 1/2 = 0.5. After normalization, these documents receive weight 0.5 / μraw.
The half-life property means that reasoning at length β contributes half the marginal gain of reasoning at length 0. This is a reasonable default that doesn't require per-dataset tuning.

The paper does not report the actual numerical value of β (the median reasoning length in tokens), but Figure 6 shows the distribution of reasoning lengths, with a mode around 250-500 tokens and a long tail extending past 1,750 tokens.

3.4.5 Weighted Contrastive Learning

The final stage of LRAT takes the mined, filtered, and weighted training instances and uses them to fine-tune a dense retriever. The training procedure has three components: the model architecture, the loss function, and the negative sampling strategy.

Model architecture: standard bi-encoder dense retriever. LRAT works with any embedding model that follows the bi-encoder paradigm, where queries and documents are independently encoded into fixed-dimensional vectors and relevance is computed as their similarity. The paper experiments with two architectures:

Multilingual-E5-Large-Instruct (Wang et al., 2024): an encoder-based model (likely BERT-derived) with a dedicated embedding head. Encoder-based models process the full input sequence with self-attention and produce a pooled representation (typically [CLS] token or mean pooling).
Qwen3-Embedding-0.6B (Zhang et al., 2025): a decoder-based embedding model built on a 600M-parameter autoregressive language model backbone, adapted for embedding tasks. Decoder-based embedding models process input through a causal transformer and produce a representation, usually from the last token position or through a specialized pooling mechanism.

For both architectures, the training procedure is the same: each query q and document d is independently encoded into vectors eq, ed ∈ ℝh, where h is the embedding dimension (architecture-dependent, not specified in the paper). The relevance score is computed via a similarity function:

$s(q, d) = \text{sim}(e_q, e_d)$

The paper does not specify whether the similarity function is dot product or cosine similarity, but standard practice in dense retrieval training uses cosine similarity with a temperature scaling, or dot product with normalized embeddings (which is equivalent to cosine similarity).

Loss function: weighted InfoNCE. The standard contrastive loss for dense retrieval training is InfoNCE (Gutmann and Hyvärinen, 2010), which treats retrieval as a classification problem: given a query and one positive document among a set of negative documents, the model must identify which document is the positive. The standard (unweighted) InfoNCE loss for a batch of N query-positive pairs is:

$\mathcal{L}_{\text{standard}} = -\frac{1}{N}\sum_{i=1}^{N} \log \frac{\exp(s(q_i, d_i^+) / \tau)}{\exp(s(q_i, d_i^+) / \tau) + \sum_{d^- \in \mathcal{N}_i} \exp(s(q_i, d^-) / \tau)}$

where τ is a temperature parameter, d+i is the positive document for query qi, and Ni is the set of negative documents for query qi.

LRAT modifies this by introducing instance-wise weights derived from the relevance intensity estimator. The weighted objective is:

$\mathcal{L} = -\frac{1}{N}\sum_{i=1}^{N} w_i \cdot \log \frac{\exp(s(q_i, d_i^+) / \tau)}{\exp(s(q_i, d_i^+) / \tau) + \sum_{d^- \in \mathcal{N}_i} \exp(s(q_i, d^-) / \tau)}$

where wi is the relevance intensity weight computed in Section 3.4.4, and all other terms remain as in the standard InfoNCE loss.

What this weighting achieves operationally. During backpropagation, the gradient contribution of training instance i is scaled by wi. Instances with wi > 1 (documents that triggered above-average reasoning) contribute more strongly to parameter updates, making the model more sensitive to getting these documents right. Instances with wi < 1 (documents that triggered below-average reasoning) contribute less, preventing them from dominating the training signal. In the extreme, a document with weight near 0 (immediately dismissed) contributes essentially nothing to training—though such documents should have been filtered out in the reasoning-aware filtering stage, so in practice all training instances have non-trivial weights.

Why weight the loss rather than sample based on weight. An alternative would be to sample training instances with probability proportional to wi (importance sampling). LRAT uses direct loss weighting instead, which has two advantages: (1) it uses all available data, preventing information loss from down-sampling low-weight instances, and (2) it is simpler to implement—weights are just multipliers on the per-instance loss, requiring no changes to the data loading pipeline.

Negative sampling strategy. The negative set Ni for query qi is constructed from two complementary sources:

Trajectory negatives: The unbrowsed documents from the same retrieved candidate set Dt that produced the positive document d+i. These are typically 9 documents (10 candidates minus the 1 browsed positive), providing query-specific hard negatives—documents that were retrieved for the exact same query but explicitly rejected by the agent after snippet inspection.
In-batch negatives: Documents paired with other queries in the same mini-batch. If the batch contains N query-positive pairs (q1, d+1), (q2, d+2), ..., (qN, d+N), then for query qi, all documents d+j for j≠i serve as in-batch negatives. These provide diverse, query-irrelevant negatives that help the model learn to separate truly relevant documents from generally plausible ones.

This hybrid strategy is standard in dense retrieval training and provides complementary benefits: trajectory negatives teach the model to distinguish the agent's chosen document from others returned for the same query (fine-grained discrimination), while in-batch negatives teach the model to separate relevant documents from unrelated ones (coarse-grained discrimination). The paper notes that this combination "improves discriminative power, helping the retriever separate high-utility evidence from explicitly rejected candidates and unrelated documents, while avoiding representation collapse."

Training hyperparameters. The paper specifies (Section 6.1.4):

Training framework: FlagEmbedding (FlagOpen Team, 2023)
Epochs: 2
Batch size: 32
Learning rate: 1 × 10-6
Maximum input length: 512 tokens
InfoNCE group size: 10 (this likely means 10 candidate documents per query in the loss computation, including the positive)
Temperature τ: 0.02

The small learning rate (1e-6), relatively few epochs (2), and moderate batch size (32) suggest that the retriever is being fine-tuned from a strong pretrained initialization (the base Qwen3-Embedding or E5 model) rather than trained from scratch. This is consistent with the paper's framing: LRAT adds agent-specific alignment on top of existing general-purpose retrieval capabilities.

Temperature analysis. The temperature τ = 0.02 is notably low. The temperature controls the sharpness of the softmax distribution over candidates in the InfoNCE loss. Lower temperatures make the loss more sensitive to small differences in similarity scores, effectively increasing the penalty for ranking negatives above positives. A temperature of 0.02 is on the aggressive end—it means the model is strongly penalized if the positive document's similarity score is even slightly below a negative's score. This is appropriate for the dense retrieval setting where the model needs to produce well-separated representations for positive and negative documents, but it may also make training more sensitive to label noise. The reasoning-aware filtering step (Section 3.4.3) partially addresses this by reducing noise in the positives, allowing the aggressive temperature to be used without overfitting to false positives.

3.4.6 Summary of Design Choices and Their Justifications

The LRAT pipeline makes several non-obvious design choices, each justified by empirical findings from the trajectory analysis:

Position-bias-free negative sampling rather than skip-above or other click-model correction: justified by Figure 4(c) showing uniform browsing distribution across ranks, meaning unbrowsed documents genuinely reflect rejection, not lack of exposure.
LLM-based positive filtering rather than length-threshold or content-overlap heuristics: justified by the need to interpret free-form reasoning traces, where semantic content—not surface length—determines whether a document was found useful.
Exponential saturation weighting rather than linear or threshold-based: justified by analogy to dwell-time models in human search and the empirical observation that reasoning length exhibits diminishing returns (the marginal informativeness of additional tokens decreases).
Instance-wise loss weighting rather than importance sampling: justified by simplicity and the desire to use all available training data, since even low-weight instances provide some signal.
Hybrid trajectory + in-batch negatives rather than pure trajectory negatives: justified by the complementary benefits—trajectory negatives provide query-specific hard negatives (teach fine discrimination), while in-batch negatives provide diverse easy negatives (teach coarse discrimination and prevent collapse).

4. Key Insights and Innovations

Innovation 1: Agent Trajectories as a Self-Supervision Paradigm—Not Just a New Dataset, but a New Training Regime

The paper's most fundamental contribution is not the LRAT framework itself, but the reconceptualization of what constitutes valid supervision for retrieval training. Before this work, the IR field recognized exactly two categories of supervision: explicit relevance judgments (human annotations, expensive and static) and implicit user feedback (clicks, dwell time, abundant but biased by human behavioral patterns). Both assume the end user is human, and both encode human-specific assumptions about how queries are formulated, how results are examined, and what relevance means.

This paper introduces a third category that is qualitatively different from both: supervision derived from the internal decision-making of an autonomous agent as it pursues a complex, multi-step task. The key conceptual move is recognizing that an agent's actions—which documents it browses, which it rejects, how extensively it reasons about what it reads—are not merely behaviors to be optimized by better retrieval, but implicit relevance judgments that can be mined to train the retriever itself.

This is not an incremental extension of learning from implicit feedback. The nature of the signal is fundamentally different. Human click logs provide binary feedback (clicked/didn't click) that requires extensive debiasing because humans have systematic perceptual biases (position bias, trust bias, selection bias). Agent trajectories provide structured, multi-dimensional feedback:

Binary decisions (browse vs. don't browse) that, unlike human clicks, are not distorted by position bias (Figure 4c shows uniform browsing distribution across ranks).
Post-decision reasoning that explains why the agent chose to browse and what it gained, providing a semantic layer absent from human click data.
Continuous engagement intensity (reasoning length) that captures how much value the agent extracted, providing a graded relevance signal richer than the binary click/no-click from human logs.

The paper explicitly frames this as a historical analog to the transition from manual relevance judgments to implicit user feedback (Section 1): "positioning trajectories as the agent-era counterpart of user click logs." This analogy is conceptually precise. Just as click logs solved the scalability crisis of manual annotation by exploiting naturally occurring user behavior as supervision, agent trajectories solve a different scalability crisis—the impossibility of manually annotating relevance for the diverse, intermediate, context-dependent queries that agents generate during multi-step problem solving. An agent investigating "Which station merged with Immaculate Heart Radio in 2017?" might issue a dozen sub-queries. None of these sub-queries is a self-contained information need for which a human annotator could provide stable relevance judgments—they only make sense within the trajectory's unfolding context. The agent's own behavior within that context provides the only scalable path to supervision.

Why this matters beyond this paper. This reconceptualization opens a research direction that extends far beyond retriever training. If agent interactions can supervise retrieval, what else could they supervise? The query formulation component? The agent's own decision-making about when to search versus when to reason? The paper does not explore these extensions, but the conceptual foundation—"agent behavior as latent supervision"—is general. It recasts the relationship between agent and retriever from unidirectional (agent consumes, retriever serves) to potentially co-adaptive (agent generates training signal, retriever improves, agent benefits from better retrieval in future interactions). The data flywheel simulation in Section 6.5 provides preliminary evidence for this co-adaptive vision—supporting gains across iterative update cycles—though the current paper keeps the agent frozen during retriever training, leaving full co-adaptation as future work.

Distinction from prior work. Traditional learning-to-rank from implicit feedback (Joachims, 2002; Joachims et al., 2005; Agichtein et al., 2006a,b) treats user actions as weak labels that must be modeled and debiased to approximate true relevance. The paper's insight is that agent actions require the opposite treatment: they are already strong labels, because the agent actively evaluates and explicitly rejects candidates in ways that humans do not. The position-bias-correction machinery developed for human clicks is not just unnecessary for agent trajectories—it would actively degrade the signal by imposing assumptions about rank-dependent exposure that don't hold for agents. This is a fundamental shift, not a transfer of existing techniques to a new data source.

Evidence anchor. The necessity and sufficiency of agent trajectories as supervision is demonstrated throughout the experimental section. The "+Naive" baseline in Figure 7, which uses only coarse browse/no-browse signals with no content analysis whatsoever, already produces substantial gains over the base retriever (9.0% → ~11.5% success rate for GPT-OSS). This confirms that the raw behavioral signal alone is informative. Table 3 further shows that even incorrect trajectories—where the agent ultimately fails—provide useful supervision (10.7% for GPT-OSS with incorrect trajectories vs. 9.0% base), proving that the intermediate judgments embedded in trajectories carry value independent of whether the trajectory succeeds. This robustness to trajectory quality is what makes the paradigm scalable: in real deployments, most trajectories will be imperfect, and the finding that even failed interactions contain useful training signal is critical for the practical viability of the approach.

Innovation 2: The Diagnostic Decoupling of Retrieval Quality from Agent Capability

A persistent challenge in evaluating retrieval for agentic search is that retrieval quality and agent capability are confounded. If an agent using retriever A outperforms the same agent using retriever B, is it because A is genuinely better at retrieving useful documents, or because A's superficial result patterns happen to interact favorably with the agent's specific reasoning quirks? Conversely, if two agents using the same retriever perform differently, is the difference in retrieval consumption or in downstream reasoning? Prior work largely sidestepped this by evaluating agents end-to-end—did the agent succeed?—without isolating the retriever's contribution.

This paper provides a clean experimental design that decouples these factors through two structural choices that together constitute a methodological innovation.

First, the evidence recall metric. On BrowseComp-Plus, the paper reports not just task success (did the agent answer correctly?) but also evidence recall: the fraction of queries for which the annotated ground-truth evidence document is retrieved at any point during the agent's execution. This metric is independent of the agent's reasoning quality—a document either appears in the retrieved sets across the trajectory or it does not. Evidence recall isolates the retriever's contribution from the agent's ability to exploit what is retrieved. The consistent gains in evidence recall across all agent backbones (Table 2, BrowseComp-Plus Recall column) demonstrate that LRAT improves retrieval per se, not just retrieval-as-filtered-through-an-agent.

Second, the cross-agent evaluation design. The paper trains retrievers on trajectories from one agent (Tongyi-DeepResearch-30B) and evaluates them with six different agent backbones spanning three orders of magnitude in parameter count (4B to 358B) and two architectural categories (task-optimized search agents vs. generalist foundation models). If LRAT's gains were agent-specific—if the retriever learned only to serve the particular agent that generated the training trajectories—we would expect large gains on Tongyi-DeepResearch and small or zero gains on other agents. The results show the opposite: gains are substantial and consistent across all agents, including those with entirely different training procedures and architectures (e.g., GPT-OSS, GLM-4.7). The average gain of 28.6% on InfoSeek-Eval (calculated across the six agents with Qwen3-Embedding as base retriever) demonstrates that LRAT-trained retrievers capture agent-general properties of useful documents, not idiosyncrasies of the trajectory-generating agent.

Why this decoupling matters. It addresses a natural objection: perhaps LRAT-trained retrievers just overfit to Tongyi-DeepResearch's specific search patterns—its query formulation style, its snippet evaluation heuristics, or its browse decision thresholds. The cross-agent results refute this. The retriever is learning something more fundamental: what kinds of documents actually advance an agent's task progress, as revealed through the agent's behavioral engagement signals. This generalizes because the relationship between document utility and agent engagement (browsing, extended reasoning) is not specific to Tongyi-DeepResearch—it is a property of how any competent agent interacts with useful information.

This finding has a deeper implication: it suggests that the bottleneck in agentic search is not agent architecture or scale, but retrieval quality. Even the largest agents tested (GLM-4.7 at 358B, MiniMax-M2.1 at 229B) show substantial gains from LRAT (Table 2: GLM-4.7 improves from 67.7% to 82.0% on InfoSeek-Eval, a 21.1% relative gain). If agent capability were the primary constraint, we would expect diminishing returns from retrieval improvement as agent scale increases—the largest agents would already be making optimal use of whatever the retriever provides. Instead, retrieval improvement helps more at larger scales, suggesting that more capable agents are better able to exploit improved retrieval, but are also more severely bottlenecked when retrieval is poor. This is a diagnostic finding that should influence how resources are allocated in agent system development: investing in retriever quality may yield higher returns than further scaling the agent.

Evidence anchor. Table 2 is the core evidence for the decoupling claim. Consider the BrowseComp-Plus Recall column for Qwen3-Embedding base vs. +LRAT: across six agents, the recall improvement ranges from +16.8% (GLM-4.7) to +37.9% (AgentCPM), with an average gain of roughly +24%. This metric is computed independent of agent success rate—it measures only whether the evidence document appeared in any search result set. The fact that recall improves consistently across agents means the retriever is genuinely better at surfacing evidence, not just better at satisfying one agent's particular preferences. The success rate gains then follow naturally: better recall enables better task outcomes, but the recall improvement itself is agent-independent.

Innovation 3: Post-Browse Reasoning as a Continuous Utility Signal—The Agent Analog of Dwell Time, but Richer

The paper identifies a signal—post-browse reasoning trace length—that serves as the agent-era analog of human dwell time for estimating document relevance intensity, but with a crucial difference: it is semantically interpretable in ways that dwell time is not. This is both a practical technique (the intensity estimation in Section 5.2.1) and a conceptual insight about what agent behavior reveals.

The dwell-time problem in human search. In human-centric IR, dwell time—how long a user spends reading a clicked document—has been recognized for decades as a proxy for relevance (Kelly and Belkin, 2004; Kim et al., 2014). The intuition is straightforward: people spend more time reading documents they find useful. The limitation is equally straightforward: dwell time is purely behavioral and semantically opaque. A long dwell time could mean the document is highly relevant, or that the user is confused, or that they stepped away from their computer, or that the document is long but shallow. The signal is continuous and naturally scaled, but it is noisy in ways that cannot be resolved without additional information.

What post-browse reasoning adds. Agent reasoning traces are fundamentally different because they are generated, not measured. Unlike dwell time, which is an observed side effect of cognitive engagement, the reasoning trace is the engagement—it is the explicit, token-by-token record of how the agent processes and integrates the browsed document into its ongoing task. This means:

The signal is semantic, not just temporal. A long reasoning trace that extracts specific facts and connects them to the research question is reliably high-utility. A long reasoning trace that expresses confusion and rephrases the problem without making progress is distinguishable from the former. The LLM-based filtering step in LRAT exploits precisely this: it reads the reasoning trace to determine whether the document was useful, not just how long the agent reasoned about it. Human dwell time offers no such semantic layer.
The signal explains relevance, not just indicates it. The reasoning trace contains the agent's explicit assessment of what it learned from the document. This is relevance with a rationale attached—it tells us not just that the document was useful, but why it was useful: what specific information gap it filled, what specific evidence it provided. This makes the supervision signal more robust to noise because the rationale can be verified (by the LLM judge) against the document content.
The signal naturally handles heterogeneity. Different documents serve different roles in an agent's process. Some provide a critical missing fact that immediately advances the task; others provide context or corroboration. The reasoning trace reflects this: a critical document triggers extensive integration reasoning, while a corroborating document might trigger a brief check followed by continuation. The continuous intensity weighting (Section 5.2.1) captures this gradation in ways that binary click/no-click signals cannot.

The analogy to dwell time is precise but the difference is fundamental. The paper explicitly invokes the dwell-time analogy, and it is apt for why the signal works: both dwell time and reasoning length reflect engagement depth. But the analogy also highlights what is new. Dwell-time models required sophisticated statistical machinery (click models, survival analysis) to extract a weak relevance signal from noisy behavioral data. Post-browse reasoning traces provide a strong, semantically rich signal that can be interpreted directly by an LLM. The analogy establishes continuity with prior IR research—this is not an ad hoc heuristic but an agent-specific instantiation of a well-understood principle. But the implementation is categorically more powerful because the underlying signal is richer.

Why this innovation matters for future work. It suggests that agent reasoning is a general-purpose supervision channel for any component that feeds into the agent's decision process. The paper uses reasoning to supervise the retriever, but the same principle could supervise query rewriting (which queries lead to productive reasoning?), tool selection (which tools trigger substantive reasoning vs. surface-level interaction?), or even agent self-evaluation (does the reasoning trajectory look like successful problem-solving?). The key insight is that explicit agent cognition—the reasoning trace—is not merely an output to be evaluated, but a source of training signal. This inverts the usual relationship between reasoning and supervision: in most prior work, reasoning is something the agent must learn to do well; here, reasoning is something the retriever learns from.

Evidence anchor. Figure 4(d) establishes the core empirical relationship: evidence documents are followed by substantially longer reasoning traces than non-evidence documents, and correct trajectories have longer reasoning than incorrect trajectories. This validates reasoning length as a utility proxy. The ablation in Figure 7 quantifies the value added: "+Reweight" (which incorporates reasoning-length-based intensity weighting) improves over "+Filter" (which only uses binary filtering) across all three agents tested (GPT-OSS: 11.8% → 12.2%, MiniMax-M2.1: 44.8% → 48.3%, GLM-4.7: 55.3% → 54.6%). The gains from continuous weighting are smaller than the gains from the binary filter step, which is expected—the filter removes clear noise, while the reweighting fine-tunes relative importance among already-filtered positives. The critical point is that the reweighting adds value without requiring any additional data collection—it extracts more information from the same reasoning traces that were already collected for the filtering step.

Innovation 4: The Uniform Browsing Distribution as a Diagnostic Finding—Agents Are Not Position-Biased

One of the paper's most counterintuitive empirical findings is also one of its most consequential for methodology: agent browsing behavior does not exhibit the rank-position concentration that defines human search behavior. Figure 4(c) shows that agent browsing actions are distributed relatively uniformly across the top-10 ranking positions, in stark contrast to the well-documented human pattern where clicks are heavily concentrated at ranks 1–3 (Craswell et al., 2008). This is not a minor quantitative difference—it upends a foundational assumption of learning-to-rank methodology that has been treated as universally applicable.

Why this is surprising. Position bias in human search is considered so fundamental that entire subfields of IR research are devoted to modeling and correcting for it. The Examination Hypothesis—the assumption that users are more likely to examine higher-ranked results—underlies virtually all click models (cascade model, dependent click model, dynamic Bayesian network). "Skip-above" heuristics for constructing negative training examples from click logs (treating unclicked documents above the last click as pseudo-negatives) are standard practice precisely because of the ambiguity that position bias creates: an unclicked high-ranked document might be irrelevant or might simply be unseen.

The paper's finding that agents do not exhibit this bias means these carefully developed corrections are not just unnecessary—they would be actively harmful if applied to agent trajectory data. A skip-above heuristic applied to agent trajectories would systematically discard perfectly valid negative signals from higher-ranked unbrowsed documents, reducing the effective training data and potentially introducing bias where none existed.

Why agents behave differently. The paper does not deeply analyze the cognitive reasons, but the implications are clear from the agent's operational constraints. Human searchers are subject to genuine perceptual limitations: screen real estate, attention decay, impatience, and the physical effort of scrolling and scanning. Agents have none of these constraints—they process all 10 snippet results with equal attention because snippet evaluation is a fixed-cost computation (a forward pass through the language model) regardless of rank position. The agent "reads" the snippet at rank 10 as thoroughly as the snippet at rank 1, because reading is computation, not perception. This means the agent's browse decision genuinely reflects its assessment of snippet content, not a compromise between content quality and positional convenience.

Implications beyond this paper. This finding has practical consequences for anyone building retrieval systems for agents:

Negative sampling can be aggressive. Because unbrowsed documents are reliably rejected rather than unseen, all unbrowsed candidates in a retrieved set can be used as negatives without any position-based correction. This yields 9 negatives per positive (for K=10), substantially more than what position-bias-corrected human click data typically provides, and all of them are query-specific hard negatives—documents retrieved for the same query but explicitly rejected.
Ranking optimization should target snippet informativeness, not click probability. Human-centric ranking models optimize for click-through rate, implicitly assuming that rank position and click probability are related. For agents, the relationship is different: what matters is whether the snippet conveys enough information for the agent to make an accurate browse decision. A highly relevant document with a poor snippet may never be browsed; a moderately relevant document with an informative snippet may be browsed and found useful. This suggests that snippet generation—which is typically an afterthought in retrieval system design—may be as important as document ranking for agentic search.
Click-model machinery is deadweight for agent contexts. The sophisticated statistical apparatus developed for debiasing human click data (position-based models, trust bias models, examination models) adds complexity without adding value for agent trajectory data. This simplifies the training pipeline substantially: LRAT's naive relevance mining directly treats browsed = positive, unbrowsed = negative, with no debiasing step.

Evidence anchor. Figure 4(c) is the primary evidence, with the dashed reference line at y=0.1 showing the uniform distribution that would be expected if browsing were random with respect to rank. The agent browsing distribution closely tracks this uniform line, with slight elevation at middle ranks and slight depression at the extremes, but no monotonic decline from rank 1 downward. This is contrasted with the well-known human click distribution from Craswell et al. (2008), which shows a sharp peak at rank 1–2 and rapid drop-off thereafter. The ablation in Figure 7 provides indirect validation: the "+Naive" variant, which uses unbrowsed documents as negatives without any position-bias correction, already yields substantial gains over the base retriever, confirming that the "unbrowsed = negative" assumption is not introducing systematic error due to unseen-but-relevant documents at high ranks.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses two benchmarks. For in-domain evaluation, InfoSeek-Eval (Luo et al., 2025): 300 multi-hop information-seeking queries strictly disjoint from the 10K InfoSeekQA queries used for trajectory generation. For out-of-domain evaluation, BrowseComp-Plus (Chen et al., 2025): a reproducible benchmark with 830 complex, human-authored questions requiring multi-step reasoning and evidence aggregation, with retrieval performed over a corpus of 100,195 documents and trace-level evidence annotations available.
Base model(s). The retrieval backbones are two complementary architectures: Multilingual-E5-Large-Instruct (Wang et al., 2024), an encoder-based embedding model, and Qwen3-Embedding-0.6B (Zhang et al., 2025), a decoder-based embedding model. These represent standard off-the-shelf dense retrievers that agents would typically use. The agent backbones span six models across three orders of magnitude: three task-optimized search agents (AgentCPM-Explore-4B, WebExplore-8B, Tongyi-DeepResearch-30B) and three generalist agentic foundation models (GPT-OSS-120B, MiniMax-M2.1-229B, GLM-4.7-358B).
Metrics. Three metrics are reported. Success Rate (SR) is assessed by an automated LLM judge (Qwen3-30B-A3B-Thinking-2507) that verifies whether the agent's final answer matches the ground truth—this is the primary end-to-end metric. Average Step Count measures execution efficiency by counting the total interaction turns (search + browse actions) before the agent produces an answer; fewer steps indicate more direct information acquisition enabled by the retriever. Evidence Recall (BrowseComp-Plus only) measures the proportion of queries for which the annotated ground-truth evidence document is successfully retrieved at any point during the agent's execution, isolating retrieval quality from the agent's reasoning capability.
Baselines. The primary baseline is the base retriever without LRAT training—the off-the-shelf Qwen3-Embedding-0.6B or Multilingual-E5-Large-Instruct model as-is. This represents the standard deployment practice where agents use frozen, general-purpose retrievers. The ablation study (Section 6.3, Figure 7) additionally includes intermediate variants: +Naive (trajectory supervision without filtering or weighting), +Filter (adds reasoning-aware positive filtering), and +Reweight (the full LRAT with intensity weighting). No external baseline methods (e.g., prior retriever optimization techniques) are compared directly, since the paper argues those methods target single-turn RAG and are not designed for multi-turn agent trajectories.
Generation budget / compute accounting. The paper's primary axis of comparison is retriever quality under identical agent execution budgets. Retrieval configuration during evaluation is kept consistent with the training setup: each [Search] action returns top-K=10 candidate documents, agents are limited to 50 turns per query during evaluation (for computational constraints, vs. 100 during training). The "generation budget" concept from the reference example does not directly apply—here the cost is measured in agent interaction turns (searches and browses), and retriever quality is assessed by how efficiently those turns are used (fewer steps = better retrieval). The ablation in Figure 8(b) additionally sweeps top-K ∈ {1, 5, 10, 20} to test robustness to varying retrieval budgets.
Cross-validation / statistical protocol. No explicit cross-validation or statistical significance testing is reported. The paper uses fixed train/test splits: trajectory generation uses 10K queries from InfoSeekQA (training), evaluation uses InfoSeek-Eval (300 queries, disjoint from training) and BrowseComp-Plus (830 queries, entirely out-of-domain). Reproducibility is ensured by fixing the random seed to 2025 and setting agent generation parameters (temperature=0.85, top-p=0.95, presence penalty=1.1) consistently. Error bars or confidence intervals are not reported for any experiment, which is a methodological limitation—the test sets are moderately sized (300 and 830 queries), and 95% confidence intervals would be roughly ±5–6 percentage points for a 50% success rate at n=300, meaning some of the smaller gains reported (e.g., E5-Large on AgentCPM: 47.3% → 49.7%) may fall within statistical noise.

Main Quantitative Results

Overall Performance Across Agent and Retriever Backbones (Table 2)

The headline result is that LRAT consistently improves both task success and retrieval quality across all six agent backbones and both retriever architectures, with gains on in-domain InfoSeek-Eval averaging 28.6% relative improvement for Qwen3-Embedding-0.6B (calculated across the six agents: ((55.7-40.3)/40.3 + (68.7-52.0)/52.0 + (68.0-52.7)/52.7 + (47.0-40.0)/40.0 + (78.3-58.7)/58.7 + (82.0-67.7)/67.7) / 6 ≈ 0.286) and evidence recall gains on out-of-domain BrowseComp-Plus averaging 27.5% across the same setups.

Task-optimized search agents with Qwen3-Embedding:

AgentCPM-Explore (4B): Success rate improves from 40.3% to 55.7% (+38.2% relative), with average steps decreasing from 38.0 to 34.4. On BrowseComp-Plus, evidence recall improves from 23.2% to 32.0% (+37.9%) and success rate from 13.5% to 15.8% (+17.0%).
WebExplore (8B): Success rate improves from 52.0% to 68.7% (+32.1% relative), with average steps decreasing from 24.1 to 19.0—a 21% reduction in interaction cost. BrowseComp-Plus evidence recall improves from 47.7% to 55.9% (+17.2%), and success rate from 21.0% to 27.2% (+29.5%).
Tongyi-DeepResearch (30B): Success rate improves from 52.7% to 68.0% (+29.0% relative), with average steps decreasing from 26.7 to 20.7. BrowseComp-Plus evidence recall improves from 49.2% to 60.7% (+23.4%), and success rate from 17.8% to 23.7% (+33.1%). Note that this is the same agent architecture that generated the training trajectories, though the evaluation queries are disjoint from training.

Generalist agentic foundation models with Qwen3-Embedding:

GPT-OSS (120B): Success rate improves from 40.0% to 47.0% (+17.5% relative), with average steps decreasing from 34.9 to 30.5. BrowseComp-Plus evidence recall improves from 43.7% to 56.4% (+29.1%), and success rate from 9.0% to 12.1% (+34.4%).
MiniMax-M2.1 (229B): Success rate improves from 58.7% to 78.3% (+33.4% relative), with average steps decreasing from 21.4 to 14.7—a 31% reduction. BrowseComp-Plus evidence recall improves from 57.2% to 69.2% (+21.0%), and success rate from 38.2% to 48.3% (+26.4%).
GLM-4.7 (358B): Success rate improves from 67.7% to 82.0% (+21.1% relative), with average steps decreasing from 27.5 to 18.5—a 33% reduction. BrowseComp-Plus evidence recall improves from 66.6% to 77.8% (+16.8%), and success rate from 43.9% to 54.6% (+24.4%).

Results with Multilingual-E5-Large-Instruct:

The pattern holds for the encoder-based retriever, though gains are generally smaller than with Qwen3-Embedding. For example, Tongyi-DeepResearch with E5-Large improves from 56.7% to 68.0% (+19.9%), and GLM-4.7 improves from 73.7% to 81.7% (+10.9%). The smaller relative gains with E5-Large are partly because the E5-Large baseline already achieves higher absolute performance than Qwen3-Embedding (e.g., 73.7% vs. 67.7% for GLM-4.7), leaving less room for improvement. However, even the strongest baseline (GLM-4.7 + E5-Large at 73.7%) benefits from LRAT, reaching 81.7%, demonstrating that retrieval quality remains a bottleneck even for large agents with strong base retrievers.

Key patterns across all results:

Efficiency gains are universal. Average step count decreases for every single agent-retriever combination when LRAT is applied (Table 2, Avg. Steps columns), with reductions ranging from roughly 5% to over 30%. This means LRAT-trained retrievers not only help agents succeed more often, but help them succeed with fewer interactions—a direct efficiency improvement that compounds the accuracy gains.
Evidence recall improvements are larger than success rate improvements in several cases. On BrowseComp-Plus, the evidence recall gain often exceeds the success rate gain (e.g., GPT-OSS: +29.1% recall vs. +34.4% success rate for Qwen3-Emb). This suggests that better retrieval helps the agent find evidence documents, but the agent does not always capitalize on finding them—some retrieved evidence documents may not be properly exploited. This is evidence for the paper's decoupling argument: retrieval quality and agent reasoning quality are separable factors.
The largest agents still benefit substantially. GLM-4.7 at 358B parameters and MiniMax-M2.1 at 229B both show large absolute and relative gains. If agent capability were the primary constraint, we would expect diminishing returns from retrieval improvement at these scales. The continued large gains indicate that even very capable agents are severely bottlenecked by retrieval quality—they have the reasoning capacity to solve complex tasks, but only if the retriever surfaces the necessary evidence.

Ablation Study: Progressive Component Contribution (Figure 7)

The ablation isolates the marginal contribution of each LRAT component by incrementally adding them, evaluated on BrowseComp-Plus with Qwen3-Embedding-0.6B across three agent backbones (GPT-OSS, MiniMax-M2.1, GLM-4.7):

Base (no LRAT): GPT-OSS: 9.0%, MiniMax-M2.1: 38.2%, GLM-4.7: 43.9%.
+Naive (n=120,579 training pairs): Uses only search-browse transitions—browsed documents as positives, unbrowsed as negatives, no filtering or weighting. This already yields substantial gains: GPT-OSS ≈11.5% (+27.8% relative from 9.0%), MiniMax-M2.1 ≈39.0% (+2.1%), GLM-4.7 ≈53.2% (+21.2%). The large jump for GLM-4.7 and GPT-OSS confirms that the basic behavioral signal (what the agent browses) is highly informative, even without any content analysis.
+Filter (n=91,713 training pairs): Adds LLM-based reasoning-aware positive filtering, removing roughly 24% of naive positives (120,579 → 91,713). This further improves performance: GPT-OSS ≈11.8%, MiniMax-M2.1 ≈44.8% (+14.9% relative over +Naive), GLM-4.7 ≈55.3% (+3.9%). The filtering step's value is most pronounced for MiniMax-M2.1, suggesting this agent's trajectories contained more false-positive browsed documents that the filter successfully removed.
+Reweight (n=91,713 training pairs, same as +Filter): Adds reasoning-length–based intensity weighting to the already-filtered training data. Further gains: GPT-OSS ≈12.2% (+3.4% relative over +Filter), MiniMax-M2.1 ≈48.3% (+7.8%), GLM-4.7 ≈54.6% (−1.3%—a slight decrease, which the paper does not discuss). The weighting adds value for two of three agents, with MiniMax-M2.1 benefiting most, confirming that not all useful documents are equally useful and that reasoning length captures meaningful gradations in utility. The slight decrease for GLM-4.7 may indicate that this agent's reasoning length distribution is less well-modeled by the exponential saturation function, or simply statistical noise given the absence of error bars.

Key takeaway: The progressive improvement validates each component's design. The largest jump comes from the naive mining step, confirming that agent browsing behavior alone provides strong supervision. Filtering and weighting add further gains, with filtering being particularly important for agents whose browsing is noisier (more false positives), and weighting being more beneficial when the agent exhibits clear gradations in engagement intensity across useful documents.

Scalability Analysis: Training Data Size (Figure 8a)

The paper investigates whether LRAT-trained retrievers continue to benefit from additional agent interaction data, evaluating performance with 10K, 20K, and 30K collected trajectories:

Agent success rates generally improve as training data grows across all three agents tested (GPT-OSS, MiniMax-M2.1, GLM-4.7), with no evidence of performance saturation at 30K trajectories. The exact numerical values are not provided in the text (only visible in Figure 8a), but the trend is described as indicating that "LRAT can effectively exploit larger volumes of agent trajectories and does not suffer from early performance saturation."
The baseline (no LRAT) is shown as a flat reference line in Figure 8a, with all LRAT-trained variants substantially above it regardless of data quantity.

This is a practically important finding because it suggests that the data flywheel vision is sustainable: as more agent interactions are collected, the retriever continues to improve rather than hitting a ceiling. This is not obvious a priori—one might worry that 10K trajectories already capture most of the learnable signal about agent document preferences, and additional data would provide diminishing returns. The continued improvement at 30K trajectories suggests the signal is diverse enough that more data helps.

Robustness Analysis: Varying Retrieval Budget (Figure 8b)

The paper evaluates agent performance under different top-K retrieval settings (K ∈ {1, 5, 10, 20}) for both the base retriever and the LRAT-enhanced retriever:

Increasing K does not always improve performance. For GLM-4.7 with the base retriever, moving from K=10 to K=20 decreases success rate, likely because "increased noise and limited effective context capacity" cause the agent to be overwhelmed by irrelevant results. The optimal K is agent-dependent: MiniMax-M2.1 performs well at high K, while GLM-4.7 prefers moderate K.
LRAT consistently outperforms the base retriever across all K values. This holds even when K is small (K=1, information-scarce) or large (K=20, noise-heavy), demonstrating that LRAT's benefits are not dependent on a particular retrieval budget. This is evidence that LRAT is learning something fundamental about document utility, not just optimizing for a particular K=10 retrieval scenario.

The practical implication: practitioners deploying LRAT-trained retrievers do not need to carefully tune K to realize gains—the retriever provides benefits across a range of retrieval budgets, making it robust to deployment conditions that may differ from training.

Data Flywheel Simulation (Figure 9, Table 3)

The paper simulates a self-improving data flywheel where the retriever is iteratively updated using trajectories collected from agent interactions with the current retriever, creating a feedback loop:

Setup (Figure 9a): At iteration 0, the base retriever is used to collect agent trajectories on 10K InfoSeekQA queries. The retriever is then trained with LRAT on those trajectories (iteration 1). At iteration 1, the updated retriever is used to collect new trajectories, which are then used for the next training round (iteration 2), and so on for 5 total loop steps. Evaluation is conducted at each step using the Tongyi-DeepResearch agent.
Results (Figure 9b): Both agent success rate and retrieval recall show steady improvement across iterations, with no evidence of collapse or degradation. The success rate increases from roughly 18–19% (iteration 0) to roughly 23–24% (iteration 5) for the agent, and recall increases from roughly 50–52% to roughly 58–60%. The iteration-0 point (base retriever) establishes the baseline; each subsequent iteration builds on the previous.
Incorrect trajectories also provide useful supervision (Table 3): When training data is restricted to 10K correct vs. 10K incorrect trajectories, both yield improvements over the base retriever. For GPT-OSS: base = 9.0%, LRAT with incorrect trajectories = 10.7% (+18.9% relative), LRAT with correct trajectories = 11.8% (+31.1%). For MiniMax-M2.1: base = 38.2%, incorrect = 43.6% (+14.1%), correct = 45.3% (+18.6%). For GLM-4.7: base = 43.9%, incorrect = 50.6% (+15.3%), correct = 52.6% (+19.8%). Correct trajectories provide larger gains, as expected, but incorrect trajectories are far from useless—they provide roughly 60–80% of the improvement of correct trajectories (e.g., for GPT-OSS: (10.7-9.0)/(11.8-9.0) = 1.7/2.8 ≈ 61%).

This is a critical finding for practical deployment: in real-world settings, trajectory correctness labels may not be available, and most trajectories will be imperfect or incorrect. The fact that even failed interactions contain useful training signal means the flywheel can operate without expensive answer verification, using all available trajectory data.

Trajectory Statistics and Behavioral Patterns (Table 1, Figure 4)

While these are presented as analysis rather than evaluation, they provide essential context for interpreting the main results:

Table 1 shows that correct trajectories average 11.77 searches, 3.61 browses, B/S ratio 0.31, 15.38 total steps. Incorrect trajectories are dramatically longer: 34.68 searches, 6.95 browses, B/S ratio 0.20, 41.63 steps. The 2.7× higher total step count for incorrect trajectories quantifies the cost of poor retrieval: agents waste turns searching without finding browsable content.
Figure 4(a) shows action transition probabilities: correct trajectories have 59.7% probability of S→B (search followed by browse) vs. 40.3% S→S (search loop). Incorrect trajectories have only 43.0% S→B vs. 57.0% S→S, confirming that failed agents get stuck in search loops.
Figure 4(b) shows that task success increases monotonically with the number of browsed evidence documents and drops to zero when no evidence documents are browsed—directly validating the browsing-as-necessary-condition premise of LRAT's naive mining.

Ablation Studies and Robustness Checks

Naive relevance mining (position-bias-free negatives): The "+Naive" variant in Figure 7 demonstrates that treating all unbrowsed documents as negatives—without any position-bias correction—yields substantial performance gains over the base retriever (e.g., GPT-OSS: 9.0% → ~11.5%). This validates the paper's claim from Section 4.2.2 that agent browsing is not subject to position bias and that unbrowsed documents are reliable negatives without the "skip-above" heuristics required for human click data.
Reasoning-aware filtering: Adding LLM-based positive filtering ("+Filter" in Figure 7) improves over "+Naive" for all three agents, with the largest gain for MiniMax-M2.1 (~39.0% → ~44.8%). The filtering reduces training data from 120,579 to 91,713 pairs (24% reduction), yet improves performance—confirming that the removed instances were indeed noisy (browsed-but-unhelpful documents) and that their removal improves supervision quality despite reducing data quantity.
Relevance intensity weighting: Adding exponential-saturation weighting based on reasoning length ("+Reweight" in Figure 7) further improves performance for GPT-OSS (~11.8% → ~12.2%) and MiniMax-M2.1 (~44.8% → ~48.3%), but slightly decreases for GLM-4.7 (~55.3% → ~54.6%). The paper does not discuss this negative result, but it suggests that the exponential saturation function with β set to median reasoning length may not be universally optimal—GLM-4.7's reasoning patterns may produce a different length-utility relationship, or the small decrease may be statistical noise (no error bars are reported).
Training trajectory correctness: Table 3 shows that incorrect trajectories provide 60–80% of the improvement of correct trajectories (e.g., GPT-OSS: +18.9% vs. +31.1% relative). This validates that agent intermediate judgments (browsing and rejection decisions) carry useful signal even when the trajectory ultimately fails, supporting the data flywheel vision where all trajectories can be used.
Retriever architecture generalization: Table 2 shows that LRAT works with both encoder-based (E5-Large) and decoder-based (Qwen3-Embedding) architectures, though gains are larger for Qwen3-Embedding. This is partly because the Qwen3-Embedding baseline is weaker (leaving more room for improvement), but also because the trajectory-generating agent (Tongyi-DeepResearch) may have a query style more aligned with the Qwen3 embedding space. The paper does not ablate the embedding model's pretraining data or architecture to isolate this effect.
Agent backbone generalization: The gains persist across six agents spanning task-optimized search agents and generalist foundation models, from 4B to 358B parameters. The fact that agents not involved in trajectory generation (GPT-OSS, MiniMax-M2.1, GLM-4.7) benefit nearly as much as Tongyi-DeepResearch itself suggests that LRAT captures agent-general properties of useful documents, not idiosyncrasies of the trajectory-generating agent's query style.
Training data scalability: Figure 8a shows continued improvement from 10K to 30K trajectories, with no saturation. This is a non-obvious result—one might expect the signal from agent browsing/reasoning patterns to be quickly exhausted. The continued improvement suggests that the space of useful query-document relationships is large and that additional trajectories provide genuinely new information.
Retrieval budget robustness: Figure 8b shows LRAT outperforms the base retriever across top-K ∈ {1, 5, 10, 20}, despite performance being non-monotonic in K (e.g., GLM-4.7 degrades at K=20). This robustness is practically important because deployed systems may use different K than the training setting (K=10).
Data flywheel stability: Figure 9b shows iterative improvement over 5 loop steps without collapse, validating that LRAT can sustain a self-improving feedback loop. However, the experiment uses the same 10K queries at each step (sampled from InfoSeekQA), and the agent is frozen—a true flywheel would require continuously novel queries and potentially co-adaptation where the agent also improves.

Critical Assessment

Claim 1: "LRAT consistently improves evidence recall and end-to-end task success across diverse agent architectures and scales"

What the experiments demonstrate: Table 2 provides consistent evidence across six agents and two retriever architectures, with every single agent-retriever combination showing improvement on both success rate and (where measured) evidence recall. The gains are substantial in absolute terms—for example, MiniMax-M2.1 with Qwen3-Embedding improves from 58.7% to 78.3% success rate—and the pattern holds across agent scales from 4B to 358B parameters. The cross-agent generalization (training on Tongyi-DeepResearch trajectories, testing on five other agents) is a particularly strong test that the paper passes.

What is not tested: The claim of "diverse agent architectures" is empirically supported across six agents, but all agents are ReAct-style LLM-based search agents that follow a similar think-search-browse-answer pattern. The paper does not test on fundamentally different agent paradigms—e.g., agents that use different action spaces (no explicit browse action), agents that use retrieval differently (e.g., as a single-step tool rather than multi-turn interaction), or non-LLM-based agents. The generalization claim is within a specific agent architecture family, not across all possible agent designs.

The evidence recall metric is only available on BrowseComp-Plus, not InfoSeek-Eval (which lacks trace-level annotations). This means for half the evaluation, we cannot disentangle whether LRAT improves retrieval per se or merely improves agent-retriever interaction dynamics.

Claim 2: "LRAT improves execution efficiency (fewer steps)"

What the experiments demonstrate: Every agent-retriever pair in Table 2 shows reduced average step count with LRAT, with some reductions exceeding 30% (e.g., MiniMax-M2.1: 21.4 → 14.7 steps). This is consistent and substantial.

What is not tested: The step count reduction could be partially mechanical rather than indicative of better retrieval. If LRAT retrieves documents that are easier for the agent to process (e.g., more concise, better-structured), the agent might finish faster even if the documents are not more informative. The paper does not control for document length, readability, or structure when measuring step count, so the efficiency gain could conflate retrieval quality with document surface properties. Additionally, the 50-turn evaluation budget (vs. 100 during training) may truncate some trajectories, and the effect of truncation on step count is not analyzed.

Claim 3: "Agent trajectories can serve as a practical and scalable supervision source"

What the experiments demonstrate: The data flywheel simulation (Figure 9) shows iterative improvement without collapse. Table 3 shows that incorrect trajectories still provide useful training signal (60-80% of the gain from correct trajectories). Figure 8a shows continued improvement with more data (10K → 30K trajectories).

What is not tested or is inadequately tested:

Scalability of trajectory generation cost is never quantified. The paper generates 34,619 trajectories averaging 21.66 steps each (Table 1), with each step involving LLM inference (for reasoning and action selection) and retrieval (for search results). The computational cost of generating this training data—which involves running a 30B-parameter agent for up to 100 turns per query across 10K queries—is likely substantial. The paper frames trajectories as a "scalable" supervision source because they are a byproduct of agent execution, but in the experimental setup, they are explicitly generated for training. The data flywheel vision assumes trajectories will be available from production use, but the paper does not estimate how many trajectories are needed for meaningful improvement or what the cost-per-trajectory is relative to the improvement gained.
The flywheel simulation is limited. Only 5 iterations are tested, using the same 10K query pool at each step, with a frozen agent. A true flywheel would operate over continuously novel queries, potentially for hundreds or thousands of iterations. The paper does not test whether improvements plateau after more iterations, whether distribution shift (from the retriever changing what documents appear at top ranks) eventually destabilizes training, or whether the agent's behavior changes when retriever quality improves (e.g., the agent might develop different browsing patterns when retrieval is better, changing the nature of the supervision signal).
Difficulty estimation cost for trajectory quality is not addressed. The paper uses correct trajectories for primary training (26,482 of them, Table 1) and shows that incorrect trajectories provide smaller gains. In deployment, correctness labels require answer verification (an LLM judge or ground truth), which may not be available. The paper shows that incorrect trajectories still help (Table 3), but does not test a setting where the proportion of correct vs. incorrect trajectories matches what would occur in production (e.g., if only 30% of trajectories are correct, does the mixed training signal still provide net-positive improvement?).

Claim 4: "LRAT identifies key behavioral signals—browsing as necessary, unbrowsed as reliable negatives, post-browse reasoning as intensity indicator"

What the experiments demonstrate: The trajectory analysis in Section 4.2 provides correlational evidence for each signal: browsing is associated with success (Figure 4a,b), unbrowsed documents show uniform rank distribution (Figure 4c), post-browse reasoning length correlates with document utility (Figure 4d). The ablation in Figure 7 shows that each mining stage (naive, filter, reweight) adds value, which is consistent with each signal being informative.

What is not tested or is correlational rather than causal:

Browsing as "necessary" is correlational, not causal. Figure 4b shows that success increases with browsed evidence count and is zero when no evidence is browsed. But this does not establish that browsing causes success—it could be that successful trajectories happen to have more browsable documents, or that the agent's initial query quality determines both subsequent browsing and eventual success. A stronger test would intervene: force the agent to browse documents it would not naturally browse and measure whether success improves, or prevent browsing of certain documents and measure degradation.
Unbrowsed-as-negative is tested indirectly. The "+Naive" ablation uses this assumption and improves performance, which is consistent with the assumption being correct. But the paper does not compare against alternative negative sampling strategies (e.g., using only in-batch negatives, using a position-bias-corrected sampling scheme). The claim that "unbrowsed documents are reliable negatives" would be more strongly supported by showing that alternative negative strategies do worse—the current evidence only shows that the simple strategy works, not that it works because of the claimed lack of position bias.
Reasoning length as utility proxy is validated by improvement from "+Reweight" over "+Filter" for two of three agents. However, the paper does not test alternative intensity functions (linear, step-function, learned weights) to validate that the exponential saturation form is optimal. The slight decrease for GLM-4.7 under "+Reweight" (Figure 7) is unexplained and may indicate that the intensity function is not universally appropriate. Additionally, reasoning length could be confounded by the agent's verbosity—some agents simply produce longer reasoning traces regardless of document quality—and the paper does not control for this.

Claim 5: "LRAT requires no additional human annotation and can be applied to trajectories from arbitrary agents and retrievers"

What the experiments demonstrate: All training data is derived from agent trajectories without human labels. The cross-agent and cross-retriever results in Table 2 show generalization.

What is not tested: The LLM-based filtering step (Section 5.1.2) uses Qwen3-30B-A3B-Thinking-2507 as a judge to classify documents as Relevant/Irrelevant. This is not "human annotation" but it is a large-model inference cost that is not accounted for. For 120,579 naive positives, evaluating each with an LLM judge represents a non-trivial computational expense. The paper does not ablate whether a cheaper filtering method (e.g., reasoning length threshold alone, keyword matching in reasoning traces) could achieve comparable filtering quality. The claim of "no additional annotation" is true in the sense of no human labels, but the LLM judge is itself a form of automated annotation that carries computational cost.

Additionally, the "arbitrary agents and retrievers" claim is tested across six agents and two retriever architectures—a reasonable diversity, but "arbitrary" is a strong claim. The paper does not test on agents that do not produce explicit reasoning traces, agents that use different retrieval interfaces (e.g., API-based search without top-K candidate lists), or agents whose browse decisions are not recorded in the trajectory format LRAT assumes.

Missing Experiments That Would Strengthen the Paper

Comparison to single-turn retriever optimization baselines. The paper argues that prior RAG retriever optimization methods (REPLUG, preference alignment approaches) target single-turn retrieval and are not designed for multi-turn agents. While this claim is reasonable conceptually, an empirical comparison—even on a simplified setting—would quantify how much the multi-turn trajectory signal adds over single-turn optimization. For example: train a retriever on the same queries using only the first search turn and browsing decision (simulating single-turn RAG), and compare to the full multi-turn LRAT training.
Ablation of trajectory-generating agent diversity. All training trajectories come from Tongyi-DeepResearch-30B. An experiment training on trajectories from multiple different agents and testing on a held-out agent would test whether trajectory diversity improves generalization.
Scaling to larger training sets. Figure 8a shows improvement from 10K to 30K trajectories. Testing at 50K or 100K would reveal whether there is a saturation point, which is practically important for estimating the data requirements of a production data flywheel.
Direct measurement of position bias. Figure 4c shows browsing distribution is uniform-ish, but a formal statistical test (e.g., comparing the empirical distribution to a uniform null hypothesis) would strengthen the claim. Additionally, measuring whether the browsing distribution changes after LRAT training (when retrieval quality improves) would test whether the "no position bias" property is stable—an improved retriever might surface clearly better documents at top ranks, potentially inducing a rank effect.
Error analysis on failure cases. The paper reports only aggregate metrics. An analysis of cases where LRAT decreases performance (there are some, per the non-monotonic behavior in Figure 8b and the GLM-4.7 decrease under +Reweight in Figure 7) would provide insight into failure modes and boundary conditions.

Assessment Summary

The experimental section provides strong evidence for the paper's core practical claim: LRAT-trained retrievers consistently improve agent task success and retrieval quality across a diverse set of agent backbones and two retriever architectures. The gains are substantial (averaging 28.6% relative improvement in success rate for the primary retriever) and are supported by ablation studies that isolate the contribution of each LRAT component.

The evidence is weaker for the more ambitious conceptual claims: that agent trajectories constitute a "scalable supervision source" (scalability is demonstrated to 30K trajectories but no further, and the cost of trajectory generation is not quantified), that unbrowsed documents are "reliable negatives" without position bias (the claim is plausible and consistent with the data but not rigorously tested against alternative negative sampling strategies), and that reasoning length is a universal proxy for utility (the GLM-4.7 negative result under +Reweight is unexplained).

The most robust findings are: (1) agent browsing behavior provides a strong supervision signal even without content analysis (+Naive baseline), (2) filtering browsed-but-unhelpful documents using reasoning traces improves supervision quality despite reducing data quantity, (3) the benefits generalize across agent architectures not involved in training, indicating that the learned signal captures agent-general document utility rather than agent-specific preferences, and (4) even incorrect trajectories contain useful training signal, enabling a practical data flywheel without requiring answer verification.

The largest methodological weakness is the absence of confidence intervals or statistical testing on a moderately-sized test set (300 queries for InfoSeek-Eval, 830 for BrowseComp-Plus), making it difficult to distinguish genuine improvement from sampling noise for the smaller reported gains. The absence of comparison to any alternative retriever optimization method (even simple baselines like fine-tuning on query-document pairs from the first search turn only) leaves open the question of how much the multi-turn trajectory signal specifically contributes versus the general value of in-domain fine-tuning.

6. Limitations and Trade-offs

The Difficulty Estimation Problem Is Unresolved—Trajectory Quality Varies, and No Method Exists to Identify High-Quality Trajectories Without Ground Truth

The assumption or constraint. The entire LRAT framework assumes access to agent trajectories that contain informative supervision signals—browsing decisions that reflect genuine document utility, reasoning traces that honestly assess content, and sufficient interaction depth to generate training pairs. The paper uses 26,482 correct trajectories for primary training (Table 1), where correctness is verified against InfoSeekQA ground-truth answers using an LLM judge. In a realistic deployment, correctness labels are unavailable, and the proportion of correct trajectories in the wild could be much lower than the ~76% in the paper's curated dataset (26,482 correct out of 34,619 total). The paper acknowledges this implicitly in Section 6.5: "real user queries are open-ended, and agent trajectories are not always fully correct, raising the question of whether imperfect trajectories can still provide useful supervision."

The consequence. If a practitioner deploys LRAT without ground-truth answer verification, the training data will be a mixture of correct and incorrect trajectories whose composition is unknown. Table 3 provides partial reassurance: incorrect trajectories still improve over the base retriever, yielding 60–80% of the gain of correct trajectories. However, this experiment uses trajectories that are known to be incorrect (verified against ground truth) and deliberately paired in equal quantity (10K correct vs. 10K incorrect). In deployment, the practitioner cannot separate trajectories by correctness, cannot control the mix ratio, and cannot exclude trajectories that are not merely incorrect but pathological—where the agent's browsing decisions are effectively random, where reasoning traces are hallucinated, or where the agent gets stuck in degenerate loops that produce training pairs with no relationship to document utility. The paper does not test LRAT on trajectories from agents that are known to be unreliable (e.g., a weak agent with high failure rate), nor does it characterize the minimum trajectory quality threshold below which LRAT training becomes harmful rather than helpful. There is no "trajectory quality estimator" proposed—the paper offers no method for a practitioner to determine whether a batch of unlabeled trajectories is suitable for training.

What evidence exists in the paper. Table 3 is the only experiment probing this limitation, and it is designed to be favorable: it uses equal quantities of known-correct and known-incorrect trajectories, generated by a strong agent (Tongyi-DeepResearch-30B), on well-formed queries with verifiable answers. The incorrect trajectories in this experiment still come from a capable agent that made genuine attempts—they are not the kind of degenerate trajectories that might dominate production logs from a poorly-configured agent or from adversarial queries. The paper provides no experiment where the training data consists of mixed-quality trajectories in unknown proportions, which is the deployment-relevant setting.

Mitigation status. Not addressed. The paper identifies the question ("whether imperfect trajectories can still provide useful supervision") and provides preliminary positive evidence, but does not develop any mechanism for trajectory quality assessment, filtering, or robust training under unknown mixture proportions. Section 8 (Conclusion) frames this as a future direction only implicitly, by stating that agent trajectories "can support iterative retriever improvement" without specifying how to handle quality variance. A practitioner currently has no guidance on what proportion of incorrect trajectories is tolerable, what signals indicate trajectory unsuitability, or how to design a trajectory filtering step that does not itself require ground truth.

The Cost of Trajectory Generation Is Not Accounted for in the Headline Efficiency Gains

The assumption or constraint. The paper frames agent trajectories as "naturally abundant" and "generated as a byproduct of every agent invocation" (Section 1), implying that the training data is essentially free—it would be produced anyway during normal agent operation. The headline results in Table 2 report improvements in agent success rate and step efficiency when using LRAT-trained retrievers, but these numbers do not amortize the cost of generating the trajectories used for training. The paper's own training data required running Tongyi-DeepResearch-30B—a 30-billion-parameter mixture-of-experts model—for up to 100 turns per query across 10,000 queries with four different retrievers, producing 34,619 total trajectories averaging 21.66 steps each (Table 1). Each step involves an LLM forward pass for reasoning and action selection, plus retrieval and (when browsing) full-document processing.

The consequence. The true cost of deploying LRAT in a new domain or with a new agent includes the trajectory generation phase, which may dwarf the cost of retriever fine-tuning itself. If a practitioner needs to collect trajectories specifically for training (rather than having them available from existing production traffic), the total compute budget is: (cost of running the agent on N queries) + (cost of LLM-based filtering of ~120K naive positives) + (cost of retriever fine-tuning). For the paper's setup, this likely represents millions of LLM inference calls before the retriever is ever updated. The data flywheel simulation (Section 6.5, Figure 9) suggests that the retriever can improve iteratively, but each iteration requires collecting new trajectories with the updated retriever—so the trajectory generation cost recurs at every flywheel step. The paper's claim of "scalable supervision" (Section 1) is only true conditional on the trajectories already existing; the cost of producing them in the first place is an unacknowledged barrier to entry for practitioners who are not already operating a high-volume agent deployment.

What evidence exists in the paper. The paper never quantifies the computational cost of trajectory generation. Table 1 provides trajectory counts and average step counts, but no FLOP or token estimates. The LLM-based filtering step (Section 5.1.2) uses Qwen3-30B-A3B-Thinking-2507 to judge 120,579 naive positives—another large-model inference cost that is not separately reported. The retriever fine-tuning itself (2 epochs, batch size 32, on 91,713 pairs) is likely the cheapest component of the pipeline, but the paper provides no breakdown. The data flywheel experiment (Figure 9) shows iterative improvement but does not report the cumulative compute cost across iterations relative to the cumulative performance gain—we see that performance improves, but not at what total cost.

Mitigation status. Partially acknowledged, not resolved. The paper notes in Section 3.2 that "the difficulty estimation step alone consumes more compute than the largest test-time budgets studied" in the context of an analogous problem, but for its own trajectory generation cost, the acknowledgment is only implicit—the paper states that trajectories are a "byproduct" (Section 1) and that the data flywheel "mimics a realistic streaming environment" (Section 6.5) without estimating what "realistic" costs would be. The suggestion that trajectories are free because they are produced during normal operation only applies to practitioners who already have high-volume agent deployments generating diverse trajectories—for anyone building a new agent or entering a new domain, the cold-start cost is substantial and uncharacterized.

The Approach Has Only Been Demonstrated on a Single Task Family (Deep Research QA) with a Single Trajectory-Generating Agent

The assumption or constraint. All training trajectories are generated by Tongyi-DeepResearch-30B (Section 4.1.3) operating on InfoSeekQA queries (Section 4.1.1) over a Wikipedia-derived corpus (Wiki-25-Dump). The task family is deep research question-answering: complex, multi-hop information-seeking questions requiring evidence aggregation. The evaluation benchmarks (InfoSeek-Eval and BrowseComp-Plus) are both within this same task family, even though the paper labels BrowseComp-Plus as "out-of-domain"—it is out-of-domain with respect to the specific queries, but both benchmarks involve multi-hop QA over textual corpora with ground-truth answers and annotated evidence documents. The paper generalizes across agent backbones (six of them), but does not generalize across task types.

The consequence. A practitioner cannot assume LRAT will work for fundamentally different retrieval contexts. Consider an agent that uses retrieval for code generation (searching documentation and Stack Overflow to write functions), for legal research (searching case law to construct arguments), for customer support (searching knowledge bases to answer user questions), or for open-ended exploration (browsing the web without a specific answer target). In each of these settings, the relationship between browsing and document utility may differ substantially from the deep research QA setting. For code generation, the agent might browse a document, extract a single API signature, and move on—reasoning length may be short even for highly useful documents. For legal research, documents may be browsed extensively but ultimately found not to contain the specific precedent needed—reasoning length would be long for a document that turns out to be a dead end. For open-ended exploration, there may be no clean notion of "task success" to validate trajectory quality. The paper's behavioral signals—browsing as necessary for success, unbrowsed as reliable negatives, reasoning length as utility proxy—were discovered through analysis of deep research QA trajectories specifically (Section 4.2) and may not transfer.

What evidence exists in the paper. The paper provides no evidence across task types. All experiments (Table 2, Figures 7–9) are on InfoSeek-Eval and BrowseComp-Plus, both multi-hop QA benchmarks. The trajectory analysis in Section 4.2, which motivates the entire LRAT design, is conducted exclusively on BrowseComp-Plus trajectories generated by Tongyi-DeepResearch with Qwen3-Embedding-0.6B. The paper does not analyze trajectories from code-generation agents, dialogue agents, or any non-QA agent, and does not claim the behavioral signals will generalize—it simply does not address the question.

Mitigation status. Not addressed. The paper makes no claims about cross-task generalization and does not acknowledge the task-specificity of its findings as a limitation. The phrase "deep research" appears throughout to scope the claims (the title mentions "agent trajectories," the abstract mentions "deep research benchmarks," Section 3.1 defines "Deep Research Agents"), but the Introduction frames the contribution broadly as "learning to retrieve from agent trajectories" without restricting to deep research tasks. A practitioner working on non-QA agent retrieval would find no guidance on which aspects of LRAT are task-invariant and which would need re-validation.

The LLM-Based Filtering Step Introduces a Hidden Dependency on a Large, Frozen Judge Model That May Not Be Available or Appropriate in All Settings

The assumption or constraint. The reasoning-aware positive filtering step (Section 5.1.2) uses Qwen3-30B-A3B-Thinking-2507—a 30-billion-parameter model with specialized thinking capabilities—as a judge to classify each naive positive as Relevant or Irrelevant. This model is larger than the base agent used for trajectory generation in some configurations (e.g., AgentCPM at 4B, WebExplore at 8B) and is comparable in scale to the trajectory-generating agent itself (30B). The judge must process the query, the full browsed document, and the post-browse reasoning trace for each of 120,579 naive positives, producing binary relevance labels. The paper validates the judge's quality on BrowseComp-Plus evidence annotations, reporting 97.2% recall of true evidence documents and 74.8% retention of non-evidence browsed documents.

The consequence. This introduces two practical problems. First, the filtering step requires access to a capable LLM judge that may not be available to all practitioners—particularly those deploying smaller agents on resource-constrained hardware. If the judge is weaker than Qwen3-30B-A3B-Thinking-2507, its filtering decisions will be noisier, potentially removing true positives (reducing training data quality) or retaining false positives (reintroducing the noise the filtering step is designed to remove). The paper provides no ablation on judge model quality or scale—we do not know whether a smaller judge (e.g., 7B parameters) would achieve comparable filtering, or whether the filtering step's value is contingent on the judge being at least as capable as the trajectory-generating agent.

Second, the filtering step's computational cost scales linearly with the number of naive positives. For a deployment with millions of trajectories, filtering every naive positive with a 30B-parameter model may be prohibitively expensive. The paper does not explore whether filtering can be done on a subset of trajectories, whether a distilled classifier can replace the LLM judge, or whether simpler heuristics (e.g., reasoning length threshold) can approximate the filtering quality at lower cost. The ablation in Figure 7 shows that the "+Filter" step adds value over "+Naive" for all three tested agents, but does not compare against alternative filtering methods that might achieve similar gains with lower computational cost.

What evidence exists in the paper. The only evidence about the judge's behavior is the BrowseComp-Plus validation (Section 5.1.2): 97.2% recall of evidence documents, 74.8% retention of non-evidence browsed documents. This tells us the judge is good at the specific filtering task, but does not tell us whether a simpler method would be nearly as good. The paper does not ablate the judge model (e.g., comparing Qwen3-30B to a smaller model), does not compare to heuristic filters (e.g., reasoning length below a threshold, keyword-based rejection detection), and does not report the computational cost of the filtering step relative to trajectory generation or retriever training.

Mitigation status. Not addressed. The LLM judge is presented as a component of the LRAT framework without discussion of its cost, its availability requirements, or alternatives. The paper's claim that LRAT "requires no additional human annotation" (Section 1) is true, but it substitutes automated annotation with a large model whose inference cost is unaccounted for. A practitioner reading the paper would not know whether the filtering step is essential (could be removed with minimal loss?) or critical (the entire gain comes from filtering?), because the "+Naive" baseline in Figure 7 already shows substantial improvement without filtering, and the marginal gain from "+Filter" over "+Naive" varies substantially across agents (large for MiniMax-M2.1, modest for GPT-OSS, smaller for GLM-4.7).

The Weighted Contrastive Loss Makes an Untested Parametric Assumption About the Relationship Between Reasoning Length and Document Utility

The assumption or constraint. The relevance intensity weighting function (Section 5.2.1, Equation 3) maps reasoning trace length l to a scalar weight w using an exponential saturation function: w ∝ (1 - exp(-ln 2 · l / β)), where β is set to the median reasoning length across all trajectories. This functional form is inherited from the time-aware click model (Liu et al., 2016), which was developed for human dwell time in web search. The paper justifies the transfer by analogy: "increased cognitive effort reflects deeper engagement with the retrieved content" in both cases. The parameter β is set to the dataset median, and weights are normalized so that E[w] ≈ 1.

The consequence. The exponential saturation function makes a strong and untested assumption: that the marginal informativeness of additional reasoning tokens decays exponentially, with half of the total utility signal contained in the first β tokens. If this assumption is wrong for a particular agent or task, the weights will be miscalibrated—documents that genuinely contributed to task progress but elicited concise reasoning (because the agent is terse, or the document's contribution was obvious) will be underweighted, while documents that triggered verbose but unproductive reasoning (because the agent is verbose, or the document was confusing) will be overweighted. The intensity weighting step then amplifies this miscalibration during training, pushing the retriever toward documents that trigger long reasoning regardless of whether that reasoning reflects genuine utility.

The ablation in Figure 7 provides evidence that this concern is not merely theoretical. Under "+Reweight" (the full LRAT with intensity weighting), GLM-4.7 shows a slight decrease in success rate compared to "+Filter" (~55.3% → ~54.6%), while the other two agents improve. The paper does not discuss this negative result. It could be statistical noise (no error bars are provided), but it could also indicate that GLM-4.7's reasoning length distribution has a different relationship to document utility than the exponential saturation function assumes. GLM-4.7 is the largest agent tested (358B parameters), and larger models may exhibit different reasoning verbosity patterns—they might produce longer reasoning traces for the same document utility, or their reasoning length might be less tightly correlated with utility because they are more capable of extracting value from briefly-inspected documents.

What evidence exists in the paper. The only validation of the intensity weighting function is the "+Reweight" ablation in Figure 7. This shows improvement for two of three agents and a small decrease for the third—mixed evidence at best. The paper does not compare the exponential saturation function against alternative intensity formulations: linear weighting (w ∝ l), step-function weighting (w = 1 if l > threshold, else w = 0.5), learned weights from a small model, or simply omitting intensity weighting entirely. The distribution of reasoning lengths is shown in Figure 6, but the paper does not analyze whether this distribution supports the exponential saturation assumption (e.g., by showing that documents in the long tail of reasoning length are indeed higher-utility than documents at the median).

Mitigation status. Not addressed. The exponential saturation function is presented without alternative comparison and without discussion of its assumptions. The GLM-4.7 negative result under "+Reweight" is visible in Figure 7 but not mentioned in the text. The paper does not provide guidance on how a practitioner should set β for a new agent or task, or how to validate that the intensity weighting is helping rather than hurting. The median-based β is a reasonable default, but its adequacy is assumed, not tested.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not introduce a new retrieval architecture, a novel loss function, or a better embedding technique. Instead, it makes a methodological intervention that shifts what the IR field considers to be valid training data. The change it causes is not in the mechanism of retrieval training but in the source of supervision—and this has deeper implications than any single architectural improvement could.

The core shift: from human behavioral traces to agent cognitive traces as supervision. For decades, the IR field has operated within a binary supervision taxonomy: explicit human judgments (expensive, static, gold-standard) versus implicit human behavioral signals (abundant, noisy, requiring debiasing). Both assume a human consumer whose relevance criteria are stable, whose behavioral biases are well-characterized, and whose interaction patterns—clicks, dwell time, scrolling—are the sole windows into utility. This paper introduces a third category: supervision derived from the explicit, multi-step reasoning of an autonomous agent as it pursues a complex task. This is not a variant of implicit feedback—it is a qualitatively different signal with properties that invert the assumptions of human-centric training.

The inversion is most visible in the paper's treatment of negative signals. The ambiguity of "not clicked" in human search—was the document irrelevant, unseen, or skipped due to position bias?—has been a central methodological challenge since Joachims (2002). Entire subfields (click models, counterfactual learning, position-bias estimation) exist to address it. For agent trajectories, this problem disappears. The paper's Figure 4(c) demonstrates that agent browsing is distributed uniformly across ranks, meaning "not browsed" genuinely reflects rejection after inspection. The "+Naive" baseline in Figure 7, which treats all unbrowsed documents as negatives with no debiasing whatsoever, already yields substantial gains. This is not because the paper invented a clever debiasing technique—it's because the data source doesn't have the bias that debiasing techniques were designed to correct. The implication is profound: when training data comes from agents rather than humans, an entire class of statistical machinery becomes unnecessary. This simplifies the training pipeline, reduces the expertise required to build retrieval systems for agents, and eliminates a source of potential error (incorrectly applied debiasing can introduce bias where none existed).

Why this is more than a new dataset. A superficial reading might categorize this work as "training retrievers on a new dataset of agent interactions." That interpretation misses the conceptual move. The paper is not proposing that the specific 26,482 trajectories used in its experiments are a valuable resource (though they may be). It is proposing that the act of agent execution itself constitutes a supervision-generation process that can be harnessed systematically. The data flywheel simulation in Figure 9 is the critical evidence for this interpretation: the retriever improves, which changes the retrieval distribution, which changes agent behavior, which generates new trajectories, which provide new supervision, which further improves the retriever. This is not a static dataset transfer; it is a self-sustaining training regime where the agent and retriever co-evolve. The paper demonstrates only the first few iterations and keeps the agent frozen, but the vision is clear: retrieval systems can continuously adapt to the agents they serve without any manual annotation, simply by observing and learning from agent behavior.

This reframes the relationship between agents and retrieval from unidirectional dependency (agents need retrieval, so build better retrieval) to potential co-adaptation (retrieval shapes agent behavior, agent behavior trains retrieval). Current search agent research treats retrieval as infrastructure—a fixed, external resource to be queried. This paper suggests that retrieval should instead be treated as a trainable component within the agent's learning loop, analogous to how policy networks are trained in reinforcement learning from environment interaction. The trajectory is not just the agent's output; it is the training signal for the retriever that feeds the agent.

Reconciling prior contradictions. The paper resolves a latent tension in the search agent literature that has been visible but unarticulated. On one hand, the community has invested enormous effort in improving agent architectures—better reasoning, better planning, better tool use—under the implicit assumption that agent capability is the primary bottleneck. On the other hand, practitioners have consistently observed that agents fail in ways that seem retriever-limited: they issue good queries but get back irrelevant results; they browse documents that turn out to be unhelpful; they get stuck in search loops because nothing looks worth reading. The paper's experimental design, particularly the cross-agent evaluation where LRAT-trained retrievers benefit agents that were not involved in training, demonstrates that retrieval quality, not agent capability, is the limiting factor for current systems. The largest agent tested (GLM-4.7 at 358B parameters) shows a 21.1% relative improvement simply from swapping the retriever. If agent reasoning were the bottleneck, we would expect diminishing returns from retrieval improvement as agent scale increases. The opposite pattern—large gains persisting at the largest scales—indicates that even very capable agents are hamstrung by what the retriever surfaces.

This finding redirects research attention. Before this paper, a researcher studying why an agent fails on a complex QA task would naturally investigate the agent's reasoning: did it plan poorly? Did it misinterpret evidence? Did it fail to synthesize? After this paper, an equally natural first question is: did the retriever ever surface the necessary evidence? If not, the agent's reasoning quality is moot. The evidence recall metric on BrowseComp-Plus makes this diagnostic possible, and the paper's consistent finding that recall improvements (sometimes large, e.g., +37.9% for AgentCPM) precede task success improvements suggests that retrieval is the upstream constraint.

Directions that become more attractive. The paper makes several research directions newly compelling:

Trajectory-based supervision for other components. If agent reasoning traces can supervise retriever training, they can likely supervise query rewriting (which sub-queries lead to productive browsing?), tool selection (which tools trigger substantive reasoning vs. surface engagement?), and even agent self-evaluation. The principle that "agent cognition is a supervision channel" generalizes beyond retrieval.
Co-adaptation of agents and retrievers. The current paper freezes the agent during retriever training. Allowing the agent to also adapt—fine-tuning on its own successful trajectories, or using the improved retriever to generate better training data for itself—could create a virtuous cycle. The data flywheel simulation (Figure 9) is the first step toward this vision.
Verifier-free training pipelines. The paper's finding that incorrect trajectories still provide useful supervision (Table 3, 60–80% of the gain from correct trajectories) suggests that answer verification may not be necessary for the data flywheel to operate. This is practically important because verification requires either ground truth (unavailable in deployment) or a capable judge model (expensive). If all trajectories—regardless of outcome—contribute useful training signal, the pipeline becomes genuinely self-supervising.
Agent-aware evaluation of retrieval. The paper's evidence recall metric on BrowseComp-Plus demonstrates that retrieval quality can be measured independently of agent reasoning quality. This should become standard practice: when evaluating a retriever for agentic search, measure whether it surfaces the necessary evidence, not just whether the agent succeeds end-to-end. This decouples retrieval evaluation from the confounding variable of agent capability.

Directions that become less attractive. The paper also makes some previously popular directions seem less promising:

Applying human-centric click models to agent data is unnecessary and potentially harmful. The paper's demonstration that position bias is absent in agent browsing (Figure 4c) means that the sophisticated debiasing machinery developed for human clicks—cascade models, examination hypotheses, skip-above heuristics—adds complexity without adding value. Researchers should not port human-centric debiasing to agent trajectories; they should instead exploit the simpler signal structure that agent data provides.
Treating retrieval as a static, off-the-shelf component is increasingly indefensible. The dominant practice in search agent research—using Google Search API, BM25, or frozen dense retrievers with no agent-specific adaptation—should be viewed as a temporary convenience, not a principled design choice. This paper demonstrates that retrieval adaptation yields gains comparable to or exceeding architectural improvements (e.g., the difference between base Qwen3-Embedding and LRAT-trained Qwen3-Embedding for Tongyi-DeepResearch is 15.3 percentage points on InfoSeek-Eval, larger than the gap between different agent architectures using the same base retriever). Research that evaluates agent architectures while ignoring retriever quality is measuring a confounded signal.
Single-turn RAG optimization methods are insufficient for multi-turn agents. The paper argues, and the cross-agent results support, that supervision from multi-turn trajectories captures something that single-turn query-answer pairs cannot: the trajectory-level utility of a document, which depends on how it contributes to a multi-step reasoning process, not just whether it answers a single query. Researchers extending retriever optimization to agentic settings should look to trajectory-derived supervision rather than adapting single-turn methods.

Follow-Up Research This Work Enables

1. Trajectory quality estimation without ground truth. The most immediate barrier to deploying LRAT in production is the question: "Which of my agent trajectories are worth training on?" The paper shows that incorrect trajectories provide useful supervision (Table 3), but does not characterize how incorrect—are slightly-incorrect trajectories (agent almost succeeded) more useful than catastrophically-incorrect ones (agent hallucinated throughout)? A concrete follow-up would train a lightweight classifier to predict trajectory utility from trajectory-intrinsic features—browse-to-search ratio, average reasoning length, action transition entropy, final answer confidence score—and validate whether filtering trajectories by predicted utility (rather than by correctness) improves retriever training outcomes. The BrowseComp-Plus evidence annotations provide a partial ground truth for trajectory utility (did the agent browse evidence documents?), enabling supervised training of a utility estimator that could then be applied to unlabeled trajectories. A strong result would be: training on trajectories filtered by predicted utility matches or exceeds training on ground-truth-correct trajectories, eliminating the need for answer verification in the flywheel.

2. Agent architecture ablations on the retriever-utility relationship. The paper finds that GLM-4.7 shows a slight decrease under "+Reweight" while other agents improve (Figure 7). This is a fragment of evidence that the relationship between reasoning length and document utility may be agent-specific. A systematic follow-up would ablate the intensity weighting function across agents with known reasoning verbosity characteristics: compare a "verbose" agent (one that produces long, detailed reasoning traces) against a "concise" agent (one that produces terse, minimal traces) on the same task. If verbose agents show a weaker correlation between reasoning length and document utility (because they produce long reasoning for both useful and useless documents), then the exponential saturation weighting may need to be calibrated per-agent, or a different signal entirely (e.g., semantic content of reasoning rather than length) may be needed. A negative result—showing that intensity weighting helps for some agents but hurts for others, and that the helpful/harmful boundary corresponds to measurable agent reasoning characteristics—would establish boundary conditions for the technique and prevent practitioners from applying it blindly.

3. Cross-task generalization of the behavioral signals. The paper's trajectory analysis (Section 4.2) identifies three behavioral signals—browsing as necessary for success, unbrowsed as reliable negatives, reasoning length as utility proxy—based exclusively on deep research QA trajectories over Wikipedia. A critical stress-test would be: do these signals hold for fundamentally different retrieval contexts? Concrete experiments: (a) Code generation agents retrieving from documentation and Stack Overflow—here, a document might be highly useful (providing the one correct API call) but elicit very short reasoning ("I'll use this function"), breaking the reasoning-length-as-utility assumption. (b) Legal research agents retrieving from case law—documents might be browsed extensively but ultimately rejected as not precedential, creating long-reasoning negatives that would be overweighted by the current intensity function. (c) Open-ended exploration agents without a clear success criterion—can the data flywheel operate when there is no notion of "correct trajectory" at all? A finding that the behavioral signals transfer to code generation (perhaps with a modified intensity function) but not to legal research (where the relationship between browsing and utility is more ambiguous) would map the domain of applicability and guide practitioners on when LRAT is likely to work versus when new signal identification is needed.

4. Active retriever training from online agent interactions. The paper's data flywheel simulation (Figure 9) is offline and batch-based: collect trajectories, train retriever, deploy, repeat. A more ambitious extension would be online learning: the retriever is updated continuously as the agent interacts, with each new trajectory immediately contributing to training. This requires addressing several challenges that the paper doesn't face: catastrophic forgetting (the retriever must retain general retrieval capability while adapting to recent agent preferences), distribution shift (the agent's behavior changes as the retriever changes, making old trajectories off-policy), and computational efficiency (updating a dense retriever after every trajectory is expensive). A concrete experiment: implement online LRAT training where the retriever is updated every N trajectories (N swept from 1 to 1000) using a replay buffer of recent trajectories, and measure both task success and training stability over long horizons (thousands of trajectories). The key metric is whether online training matches or exceeds offline batch training at equivalent total trajectory counts—if online training underperforms due to catastrophic forgetting or off-policy issues, then the flywheel vision requires a batch retraining step that limits its responsiveness.

5. Combining LRAT with agent fine-tuning for co-adaptation. The current paper freezes the agent during retriever training. The natural extension is joint optimization: train the retriever on agent trajectories, then fine-tune the agent on the improved retrieval distribution, then collect new trajectories with the updated agent, and repeat. This creates a co-adaptation loop where both components improve. The critical question is whether this loop converges to a higher equilibrium than either component adapting alone, or whether it diverges (the agent over-adapts to the retriever's quirks, or vice versa). A concrete experiment: start with a base agent and base retriever, run 10 iterations of (train retriever on agent trajectories → fine-tune agent on successful trajectories with the new retriever → collect new trajectories), measuring both task success and the diversity of agent browsing behavior across iterations. A positive result would show monotonic improvement and maintained behavioral diversity (the agent does not collapse to a narrow strategy); a negative result would show improvement for a few iterations followed by collapse, indicating that co-adaptation requires explicit regularization to prevent over-specialization.

6. Lightweight filtering alternatives to the LLM judge. The paper's reasoning-aware filtering step uses Qwen3-30B-A3B-Thinking-2507—a 30B-parameter model—to judge 120,579 naive positives. This cost is unaccounted for and may be prohibitive in resource-constrained settings. A practical follow-up would develop and evaluate cheaper filtering methods: (a) Length-threshold filtering: discard naive positives with post-browse reasoning shorter than T tokens, sweeping T and measuring the recall of true evidence documents (using BrowseComp-Plus annotations). (b) Keyword-based filtering: use simple NLP heuristics (presence of rejection phrases like "does not contain," "not useful," "no information") to classify reasoning traces as acceptance or rejection. (c) Distilled classifier: train a small classifier (e.g., a 100M-parameter BERT-based model) on the LLM judge's outputs, then use the distilled classifier for filtering at a fraction of the inference cost. The key comparison is: for each method, what is the filtering quality (recall of true evidence, precision of retained positives) versus the computational cost (FLOPs or wall-clock time), and does the retriever trained on the filtered data achieve comparable downstream performance to retriever trained on LLM-judge-filtered data? A finding that keyword-based filtering achieves 90% of the LLM judge's quality at 0.1% of the cost would make LRAT substantially more accessible to practitioners without large-model access.

Practical Applications and Downstream Use Cases

1. Continuous retriever improvement for deployed search agents. The most direct application is for organizations operating search agents at scale—customer support systems, enterprise knowledge base assistants, or research tools. These systems generate trajectories as a natural byproduct of every user interaction. Instead of discarding these trajectories after the agent produces an answer, the organization can feed them into LRAT-style training to continuously improve their retriever. The paper provides the key enabling evidence: (a) the data flywheel works (Figure 9, steady improvement over 5 iterations), (b) incorrect trajectories still help (Table 3, 60–80% of the gain), and (c) the retriever improves even when the agent is different from the one that generated training data (Table 2, cross-agent generalization). The concrete benefit: if an organization's baseline agent achieves 40% success rate with a frozen retriever, deploying LRAT-style continuous training could push this to ~55% (extrapolating from the GPT-OSS results in Table 2) without any manual annotation, simply by learning from the agent's own interactions. The ongoing cost is the compute for periodic retriever fine-tuning, which is negligible compared to the agent inference cost that is already being incurred.

2. Cold-start retrieval for new domains where human relevance judgments don't exist. When deploying a search agent in a specialized domain—medical literature, legal documents, internal corporate knowledge bases—the standard approach requires either expensive domain experts to provide relevance judgments, or reliance on a general-purpose retriever that may not understand domain-specific terminology and relevance criteria. LRAT offers a third path: deploy the agent with a general-purpose retriever, collect trajectories even though early performance will be poor, and use those trajectories to adapt the retriever to the domain. The paper's finding that incorrect trajectories provide useful supervision is critical here—in a cold-start domain, most early trajectories will be incorrect, but the agent's browsing and rejection decisions still encode domain-specific judgments about what constitutes a useful document. After training on a few thousand trajectories (which the paper shows provides meaningful gains, Figure 8a, moving from 10K to 30K trajectories yields continued improvement), the retriever has adapted to the domain without any human annotation. The concrete benefit: a medical QA agent deployed on a hospital's internal knowledge base could improve its retrieval from "mostly returns general Wikipedia-level articles" to "returns the specific clinical guidelines relevant to the query" after observing its own failed attempts and learning from which documents it chose to browse.

3. Cost-efficient retriever adaptation across agent versions. When an organization upgrades its search agent—moving from AgentCPM to WebExplore, or from an open-source agent to a custom fine-tuned version—the standard practice is to keep the same retriever and hope for the best. The paper's cross-agent generalization results suggest a better approach: use trajectories from the old agent (which are already available from production logs) to train a retriever via LRAT, deploy it with the new agent, and then continue the flywheel with the new agent's trajectories. The key finding enabling this is that LRAT-trained retrievers work across agents: a retriever trained on Tongyi-DeepResearch trajectories improves performance for AgentCPM, WebExplore, GPT-OSS, MiniMax-M2.1, and GLM-4.7 alike (Table 2). The concrete workflow: when upgrading from agent version N to version N+1, train a retriever on version N's historical trajectories (which exist at zero additional collection cost), deploy with version N+1, and achieve immediate improvement over the base retriever while version N+1's trajectories accumulate for the next training round. This amortizes the trajectory generation cost across agent versions and ensures that retriever quality never resets to baseline during an agent upgrade.

4. Retriever personalization through user-specific agent trajectories. In multi-tenant agent deployments where different users or organizations have different information needs, trajectories from one user's interactions could be used to personalize the retriever for that specific user. A pharmaceutical researcher and a patent attorney might both use the same search agent to query the same biomedical corpus, but their notions of document utility differ substantially—the researcher wants mechanistic detail, the attorney wants claims language. If each user's agent trajectories are logged separately, LRAT can train user-specific or organization-specific retriever adaptations from the trajectories, learning to surface documents that match the specific consumption patterns revealed by browsing and reasoning behavior. The paper doesn't test this personalization scenario, but the mechanism is a direct extension: the supervision signal (browsing, rejection, reasoning length) is inherently per-trajectory, so grouping trajectories by user and training separate retriever heads (or using user-conditioned embeddings) would produce personalized retrieval without requiring users to explicitly specify their preferences. The concrete benefit: a legal research platform serving multiple law firms could give each firm a retriever that has learned—from that firm's associates' browsing patterns—to prioritize the types of cases and precedents that the firm actually relies on, without any firm needing to manually configure search preferences.

When to Prefer This Method

The paper does not explicitly position LRAT against named alternative retriever training methods with a clear tradeoff matrix (e.g., "use LRAT when X, use REPLUG-style training when Y"). The paper's framing is that existing retriever optimization methods target single-turn RAG and are not designed for multi-turn agent trajectories, and that the primary alternative—using a frozen, off-the-shelf retriever—is the baseline that LRAT consistently outperforms. The decision rule is therefore implicit in the paper's scope and results:

Prefer LRAT when the retriever will be consumed by a multi-turn search agent that generates trajectories containing browse decisions and post-browse reasoning. The paper demonstrates consistent gains across six agents on two QA benchmarks, with supervision derived entirely from agent behavior without human annotation. This covers a broad class of current search agent deployments, particularly deep research agents that follow a ReAct-style think-search-browse pattern.
LRAT's benefits are largest when (a) the agent's browsing decisions are reasonably informative (the agent is competent enough to distinguish useful from useless snippets, even if it ultimately fails the task), (b) post-browse reasoning traces are available and semantically meaningful, and (c) the retrieval task involves finding evidence documents that support multi-step reasoning rather than answering queries in a single turn.
LRAT is not demonstrated for (and practitioners should be cautious about) settings where (a) the agent does not produce explicit reasoning traces, (b) the agent's browsing behavior is unreliable (e.g., a very weak agent that browses essentially at random), (c) the task is single-turn retrieval with a clear query-answer correspondence (where standard RAG retriever optimization may be simpler and equally effective), or (d) the domain lacks a notion of browseable documents with distinguishable utility (e.g., retrieval of short factual answers where browsing vs. not-browsing carries little signal). The paper provides no evidence about these settings, and the behavioral signals identified in Section 4.2—browsing as necessary for success, unbrowsed as reliable negatives, reasoning length as utility proxy—were discovered in deep research QA trajectories specifically and may not transfer.