ArXiv: 2511.05489

🎯 Pitch

Rather than reasoning from a fixed set of frames, TimeSearch-R learns to dynamically search a long video mid-reasoning by interleaving text-based thinking with adaptive video retrieval, achieving state-of-the-art accuracy. Crucially, it trains this entire process end-to-end via reinforcement learning by using the model itself to self-verify whether the frames it chose are sufficient—an innovation that eliminates the need for frame-level supervision while ensuring the model’s reasoning is actually grounded in what it saw.

1. Executive Summary

This paper introduces TimeSearch-R, a framework that reformulates temporal search in long videos as an interleaved text-video thinking process — where the model alternates between textual reasoning and retrieving relevant video clips — and learns optimal search strategies end-to-end through reinforcement learning. To address the failure modes that arise when standard GRPO (Group Relative Policy Optimization) rewards only the final answer — leading to insufficient temporal exploration (the model arrives at correct answers without adequate visual grounding) and inconsistent logical reasoning (intermediate reasoning contradicts final answers) — the authors propose GRPO-CSV (GRPO with Completeness Self-Verification), which extracts the dynamically searched frames from the reasoning chain and requires the same policy model to re-answer the question using only those frames, thereby supervising intermediate search decisions without frame-level annotations. Evaluated on temporal search benchmarks (Haystack-LVBench, Haystack-Ego4D) and long-form video understanding benchmarks (VideoMME, MLVU, LongVideoBench) using Qwen2.5-VL-7B as the base model, TimeSearch-R improves temporal F1 on Haystack-LVBench by 5.6% over the previous SOTA, achieves 8.5% higher accuracy on Haystack-Ego4D, and establishes a new SOTA on LongVideoBench with a 4.1% improvement over the base model and 2.0% over the video reasoning model Video-R1, establishing that adaptive temporal search learned through RL with intermediate-process supervision substantially outperforms both hand-crafted search workflows and text-only video reasoning.

2. Context and Motivation

The Core Problem: Static Frame Sampling vs. Dynamic Reasoning

The fundamental tension this paper addresses is that long-form video understanding requires dynamic, adaptive access to video frames, but current models are forced to reason from a fixed, pre-selected set of frames. A typical long video might contain tens of thousands of frames spanning minutes to hours, yet only a tiny fraction of those frames contain the information needed to answer a specific question — identifying a brief object interaction, tracking an event sequence, or verifying whether an action was completed. The model must search through this massive temporal space to find the needles of relevant visual evidence.

The paper frames this as a temporal search problem: given a question about a long video, identify the minimal set of relevant frames that provide sufficient evidence to answer correctly. This is inspired by direct analogy to human visual cognition, where people "naturally alternate between broad scanning and targeted inspection, refining their focus iteratively based on intermediate findings" (Section 1). Humans don't stare at every frame equally; they saccade to regions of interest, form hypotheses, and gather evidence adaptively. Current large video-language models (LVLMs) do none of this — they receive a uniform or heuristically sampled subset of frames decided before reasoning begins, making their visual evidence static and decoupled from the dynamic process of building understanding.

This gap matters for several practical reasons:

  • Information density varies dramatically across time. A 10-minute video might have 5 seconds of critical action embedded in long stretches of static or irrelevant footage. Uniform frame sampling wastes the model's visual context budget on uninformative frames while potentially missing the crucial evidence entirely.
  • What is relevant depends on the question, which is unknown at sampling time. You cannot pre-select the "right" frames without knowing the question, since different questions target different temporal regions. A static sampler must either densely cover the entire video (prohibitively expensive for hour-scale videos) or risk information loss.
  • Reasoning is iterative and hypothesis-driven. Humans form partial hypotheses ("maybe the drawer was left open?") and seek targeted evidence to confirm or refute them. A model restricted to pre-sampled frames cannot engage in this confirmation-seeking behavior — it either has the needed evidence or it doesn't.

Where Existing Approaches Fall Short

The paper identifies three categories of prior work, each with distinct limitations that motivate the proposed approach:

1. Static frame sampling (the dominant paradigm). State-of-the-art LVLMs — including Qwen2.5-VL, GPT-4o, and Gemini-1.5-Pro — process videos by pre-selecting frames using uniform sampling, fixed FPS, or heuristic-based strategies before any reasoning occurs. The model then conducts all its thinking in a single pass over this fixed visual context. As Table 2 shows, even the strongest static-sampling models plateau in performance as video duration increases (Qwen2.5-VL drops from 76.3% on short videos to 54.6% on long videos on VideoMME). The paper identifies the core limitation: "video reasoning is a dynamic process where temporal search interleaves with video reasoning; however, the video frames accessible to the model remain fixed from the outset, ultimately hindering effective reasoning" (Section 1).

2. Hand-crafted interactive agents (VideoAgent, T*, VideoTree). These systems attempt to bridge the static-dynamic gap by enabling multi-turn temporal search, where an agent (typically an LLM) iteratively calls retrieval tools to fetch relevant frames. As illustrated in Figure 1(a):

  • VideoAgent (Wang et al., 2024) uses a large language model as a central agent that calls vision-language models and CLIP for frame captioning and retrieval, aggregating information in the textual modality to perform reasoning. The search strategy is prompt-driven: the LLM decides what to search for next based on a predefined instruction template.
  • T* (Ye et al., 2025) extends this by introducing object-oriented spatial-temporal search. It uses a VLM to extract target objects from the question, then employs object detection models (YOLO) to identify keyframes containing those objects, and finally uses the retrieved frame set for task completion.
  • VideoTree (Wang et al., 2025) introduces tree-structured search to explore multiple temporal paths, improving search efficiency through branching.

The paper identifies a critical, shared weakness across all these approaches: they rely on manually designed workflows rather than learned strategies. As stated in Section 1: "all of these approaches depend on manually designed workflows, which lead to suboptimal search strategies." The search behavior — when to search, what temporal window to query, what textual query to use, when to stop — is encoded through fixed prompts, heuristics, and tool-calling sequences designed by human engineers. These static strategies cannot adapt to the statistical patterns of what actually works across different video types, question types, and reasoning contexts. They represent human intuitions about good search, not empirically optimized search policies.

The performance gap illustrates the cost of this hand-crafted limitation. In Table 1, VideoAgent achieves a temporal F1 score of only 2.1 on Haystack-LVBench, and T* reaches 2.5–3.1. The paper's learned approach achieves 8.1 — more than three times better — while using a comparable frame budget (8–10 frames). In Table 2, the hand-crafted agents (VideoAgent, T*, VideoTree) consistently underperform TimeSearch-R on VideoMME despite using similar or larger frame budgets. The 2.5–10.6% accuracy gap demonstrates that learned search strategies provide substantial gains over human-designed workflows.

3. Text-only video reasoning models (Video-R1). Recent work has applied RL (specifically GRPO) to improve video reasoning capabilities. Video-R1 (Feng et al., 2025) uses GRPO with outcome-based rewards to train models to produce long chain-of-thought reasoning for video questions. However, the paper identifies a fundamental limitation: Video-R1 "limits the thinking process to pure text without visual interaction" (Section 4). The model reasons about the video but cannot search within the video — it must work with whatever frames were pre-sampled before reasoning begins. The thinking process operates in the textual modality only, with no mechanism to request additional visual evidence when the initial frames are insufficient. Table 2 quantifies this limitation: Video-R1-7B achieves 65.7% on VideoMME with 768 frames, while TimeSearch-R achieves 66.6% — and the gap widens on benchmarks specifically designed to test temporal search capabilities.

The RL-for-Multimodal-Reasoning Gap

The paper situates itself within a broader trend of applying reinforcement learning to enhance reasoning in multimodal models. Several recent works have demonstrated that outcome-based RL (particularly GRPO) can effectively elicit complex reasoning behaviors:

  • Search-R1 (Jin et al., 2025) applies GRPO to text-based search tasks, showing RL can train models to adaptively retrieve information from search engines.
  • MM-Eureka (Meng et al., 2025) and LMM-R1 (Peng et al., 2025) apply RL to static image understanding, improving multimodal reasoning capabilities.
  • DeepEyes (Zheng et al., 2025) uses RL for high-resolution image understanding through adaptive cropping — teaching models where to look spatially in images.

However, the paper identifies a crucial gap: "applying RL to interactive long video understanding remains largely unexplored and presents unique challenges" (Section 4). Temporal search in videos differs from spatial search in images along several axes:

  • Temporal extent: Videos span thousands of frames across minutes to hours, making the search space dramatically larger than a single image.
  • Sequential dependencies: Events unfold over time, requiring models to understand temporal ordering and causal relationships that static images don't capture.
  • Varying information density: Some temporal regions are dense with relevant information while others are sparse, requiring the model to learn when to search broadly vs. narrowly.
  • Query formulation: The model must learn to generate effective textual queries that align with the visual content of target frames, a skill absent from text-only search.

The paper's most specific critique of prior RL approaches is that directly applying standard GRPO to temporal search produces two systematic failure modes, illustrated in Figure 2:

Failure Mode 1: Insufficient temporal exploration. When GRPO rewards only the final answer correctness, the model can achieve high reward through "correct answers through partial evidence or language bias without proper visual grounding" (Section 1). In the example shown, the model searches once for "small tin cans and ASTRO CARDS" and produces the correct answer (2) based on frames at 697s, 700s, and 703s — but misses a critical frame that provides essential visual context for the count. The outcome reward provides no signal about whether the searched frames were sufficient; it only cares about final answer correctness. Since LVLMs can exploit linguistic shortcuts and partial visual cues to guess correctly without comprehensive exploration, the RL signal fails to incentivize thorough search behavior.

Failure Mode 2: Inconsistent logical reasoning. The model can produce plausible-sounding intermediate reasoning that is disconnected from its final answer — a phenomenon documented in text-only reasoning as the "unfaithful chain-of-thought" problem (Lanham et al., 2023). In the second example in Figure 2, the model reasons that apples, candles, and berries all have a count of 5, then states "each kind of decoration is the same," but outputs the final answer "Berries" — a direct contradiction between reasoning trace and conclusion. Because GRPO's outcome reward only evaluates the final answer, it provides no gradient signal to align the intermediate reasoning with the conclusion. The model can learn to produce reasoning text that looks plausible while functionally ignoring it when generating the answer.

These two failure modes are not merely aesthetic problems — they fundamentally undermine the purpose of temporal search. If the model can answer correctly without comprehensive frame exploration, there is no pressure to learn good search strategies. If the model's reasoning and answers are inconsistent, the search trace becomes an unreliable guide for understanding why the model arrived at a conclusion. The paper argues that addressing these failures requires supervision over intermediate search decisions — but doing so without expensive frame-level annotations is the central methodological challenge.

How the Paper Positions Itself

The paper's positioning synthesizes several complementary insights into a unified approach:

Reformulation as interleaved thinking. Rather than treating search as a pre-processing step or a separate agent module, the paper reformulates temporal search as integral to the reasoning process itself. The model alternates between textual analysis and temporal exploration within a single chain of thought, conditioning each search decision on its current reasoning state. This is framed as extending "Thinking with Images" — the paradigm where models actively request visual information during reasoning — to the long-video domain, now "Thinking with Videos." The key innovation is that the search strategy is not hand-crafted but learned end-to-end from data through RL, making it the first approach to discover optimal temporal search policies through direct optimization rather than human heuristics.

Self-verification as annotation-free intermediate supervision. To address the failure modes of outcome-only GRPO without requiring expensive frame-level annotations (which would be necessary to directly train a per-step search quality evaluator), the paper introduces Completeness Self-Verification (CSV). This is a cleverly designed auxiliary objective: after the model completes its search-and-reasoning trajectory, the same policy model is asked to re-answer the question using only the frames it chose to retrieve, with no further search allowed. If the searched frames genuinely contain sufficient evidence, the model should be able to answer correctly from them alone. If the frames are insufficient, the re-answer will likely be wrong or produce "I don't know," revealing the insufficiency. This provides a trainable signal on search quality without requiring any human annotation of what constitutes "good" search behavior — the model's own ability to answer from the retrieved frames serves as the supervisory signal.

This positions TimeSearch-R as fundamentally different from both the hand-crafted agent approaches (which cannot adapt their strategies to data) and the text-only video reasoning approaches (which cannot interact with video content during reasoning). The CSV mechanism is the linchpin that makes end-to-end RL for temporal search practical, addressing the exploration and consistency problems that would otherwise cause training to collapse (as the ablation in Figure 4 demonstrates — removing CSV causes the model to stop searching altogether around step 300).

Data filtering as enabling infrastructure. The paper also identifies that standard video QA datasets contain many questions answerable through pure linguistic shortcuts — e.g., commonsense knowledge or dataset biases — and some that remain unsolvable even with perfect temporal search. Both types of samples are toxic for RL: trivially answerable questions provide no gradient signal (all rollouts are equally correct, yielding zero advantage), while unsolvable questions provide no path to reward regardless of search behavior, leading to noisy, unproductive gradients. The two-stage filtering pipeline — removing questions solvable from 4 uniformly sampled frames and removing questions unsolvable even with extensive search — is a pragmatic contribution that directly enables the RL training to focus on samples where search quality actually differentiates good from bad trajectories.

3. Technical Approach

3.1 Reader orientation

TimeSearch-R is a video-language model that actively searches through long videos while reasoning — rather than passively consuming a fixed set of pre-sampled frames, it alternates between thinking in text and requesting specific video clips to inspect, much like a human analyst pausing to re-examine footage. The system learns this search behavior end-to-end through reinforcement learning, with a self-verification mechanism that ensures the model gathers sufficient visual evidence before concluding, addressing the core tension between static frame budgets and the dynamic, hypothesis-driven nature of video understanding.

3.2 Big-picture architecture (diagram in words)

The system consists of five major components:

  1. Base policy model (Qwen2.5-VL-7B) — the video-language model that generates interleaved reasoning text and search instructions, then produces final answers. It serves as both the reasoning engine and the temporal search controller.

  2. Video environment with search function — a non-learned retrieval mechanism that executes the model's search instructions (temporal window + textual query) against the full video, returning a diverse and relevant subset of frames using determinantal point process (DPP) sampling. This is the environment that the RL agent interacts with.

  3. GRPO trainer — the standard Group Relative Policy Optimization algorithm that provides outcome-based rewards (accuracy + format) and updates the policy model based on relative advantage within groups of rollouts.

  4. Completeness Self-Verification (CSV) module — an auxiliary mechanism that, after a full search trajectory, extracts the dynamically gathered frames and asks the same policy model to re-answer the question using only those frames. The correctness of this re-answer produces a completeness reward that supervises intermediate search quality without requiring frame-level annotations.

  5. Two-stage data filtering pipeline — a pre-processing step that removes samples solvable from minimal visual input (4 uniform frames) or unsolvable even with extensive search, ensuring the RL training focuses on questions where search quality genuinely differentiates good from bad trajectories.

Information flows as follows: a video and question enter → the policy model receives a short preview of uniformly sampled frames → it interleaves reasoning text with search instructions (specifying temporal windows and textual queries) → the video environment executes each search via DPP-based frame retrieval and returns the selected frames with timestamps → the model continues reasoning with the expanded visual context → this alternation repeats up to 8 times → the model produces a final answer → in the CSV phase, only the dynamically retrieved frames are fed back to the model for a re-answer → three rewards (accuracy, format, completeness) are computed → GRPO updates the policy parameters.

3.3 Roadmap for the deep dive

  • First, the task formulation (Section 2.1), which defines the interleaved text-video thinking process, the formal decomposition into temporal search and answer prediction, and the interface between the model and the video environment. This provides the language and notation for everything that follows.

  • Second, the search function, which is the non-learned retrieval mechanism that translates the model's search instructions into concrete frame selections. Understanding how frames are selected (DPP optimization) explains what feedback the model receives and why certain search behaviors succeed or fail.

  • Third, the GRPO-CSV algorithm (Section 2.2), which is the core technical contribution. I will explain the three reward components (accuracy, format, completeness), how CSV works mechanically during the rollout and re-answer phases, and why the completeness reward addresses the two failure modes identified in the introduction.

  • Fourth, the dataset construction pipeline (Section 2.3), which enables effective RL training. The two-stage filtering is essential context for understanding why naive application of GRPO fails and what properties of training data make temporal search learnable.

  • Fifth, the overall training procedure and hyperparameters, which ties together the components into a concrete recipe and provides the numerical details needed for reproducibility or critical assessment.

3.4 Detailed, sentence-based technical breakdown

This is primarily a systems and methods paper whose core idea is that temporal search in long videos can be learned end-to-end through RL, provided that (a) the search is interleaved into the reasoning chain itself, (b) intermediate search decisions receive supervision via self-verification, and (c) the training data is filtered to remove samples where search quality is irrelevant or unlearnable.

Task Formulation: Interleaved Text-Video Thinking

The paper reformulates temporal search as a multi-turn thinking process where text reasoning and video clip retrieval alternate within a single chain of thought. This is a departure from both static sampling (where all frames are fixed before reasoning) and agent-based approaches (where search is orchestrated by a separate controller module with hand-crafted logic).

The interleaved chain of thought. Given a video $V$ and a corresponding question $Q$, the system first constructs an initial preview $\tilde{V}$ by uniformly sampling a small number of frames from $V$. This preview serves as the initial visual context — the model needs some visual information to form initial hypotheses, but the preview alone is deliberately insufficient for answering most questions (this is enforced by the data filtering, which removes questions answerable from the preview).

At each thinking step $k$, the policy model $\pi_\theta$ (parameterized by $\theta$) generates a segment of textual reasoning $T_k$ followed by one of two possible actions:

  1. Search: If $T_k$ concludes that more visual evidence is needed, the model emits a search instruction specifying a temporal window $[t_s^k, t_e^k]$ (start and end times in seconds), a textual query $q^k$ describing what to look for, and the number of frames to retrieve (maximum 8). The video environment executes this search, returning a clip $V_k \subseteq V$ of selected frames, which is appended to the chain of thought as visual input for subsequent reasoning steps.

  2. Answer: If $T_k$ concludes that sufficient evidence has been gathered, the model emits a final answer $A$ enclosed in <answer> tags, terminating the process.

The accumulating chain of thought at step $k$ is formalized as:

Ck≜{(T1,V1),(T2,V2),…,(Tk,Vk)}C_k \triangleq \{ (T_1, V_1), (T_2, V_2), \ldots, (T_k, V_k) \}

where $C_k$ is the interleaved sequence of reasoning-text/temporal-clip pairs accumulated up to step $k$. The notation $\triangleq$ means "is defined as." Each element is a tuple: textual analysis $T_i$ followed by the video frames $V_i$ retrieved in response to the search instruction embedded in $T_i$.

What this represents: $C_k$ is the full multimodal context available to the model at the start of reasoning step $k+1$. It grows incrementally — step 1 produces $\{(T_1, V_1)\}$, step 2 produces $\{(T_1, V_1), (T_2, V_2)\}$, and so on. This is the model's "working memory" of what it has thought and what it has seen, analogous to a human analyst's notes and collected screenshots. Crucially, $V_i$ contains new frames not in the initial preview $\tilde{V}$, meaning the model's visual context expands dynamically based on its own decisions.

Why this form: the interleaved structure ensures that each search decision is conditioned on the entire history of prior reasoning and retrieved evidence. If the model's initial hypothesis was wrong, subsequent reasoning can detect the inconsistency and issue a corrective search. This is fundamentally different from agent approaches where the LLM controller plans a search about the video at a meta-level; here, the search is part of the cognitive trace, not a separate planning stage.

Decomposing the generation process. The paper decomposes the probability of generating a reasoning chain $C$ and final answer $A$ into two factors:

Pθ(A,C∣V~,Q)=Pθ(C∣V~,Q)⋅Pθ(A∣C,V~,Q)P_\theta(A, C \mid \tilde{V}, Q) = P_\theta(C \mid \tilde{V}, Q) \cdot P_\theta(A \mid C, \tilde{V}, Q)

where $P_\theta(C \mid \tilde{V}, Q)$ is the temporal search component (the probability of generating this particular sequence of search-and-reason steps given the preview and question), and $P_\theta(A \mid C, \tilde{V}, Q)$ is the answer prediction component (the probability of producing the correct final answer given the complete search trajectory).

What this decomposition computes: it separates the two sub-tasks that the policy must learn — what to search for (the $C$ generating process) and how to answer given what was found (the $A$ generating process). If search were perfect but answer prediction were broken, $P_\theta(A \mid C, \tilde{V}, Q)$ would be low. If answer prediction were perfect but search were lazy (gathering insufficient frames), $P_\theta(C \mid \tilde{V}, Q)$ would produce low-quality $C$ chains that happen to sometimes yield correct answers by chance.

Why this decomposition matters for RL: standard GRPO rewards both components jointly through the final answer accuracy — $P_\theta(A \mid C, \tilde{V}, Q)$ receives gradient signal directly, but $P_\theta(C \mid \tilde{V}, Q)$ receives signal only indirectly through whether the chosen $C$ leads to a correct $A$. If the model can achieve correct $A$ with lazy $C$ (through linguistic shortcuts or partial evidence), the gradient signal for $P_\theta(C \mid \tilde{V}, Q)$ is uninformative — it cannot distinguish between good search and lucky guessing. This decomposition precisely identifies where the insufficiency of outcome-only reward originates, motivating the CSV mechanism (which provides direct signal to the search component by evaluating whether $C$ alone is sufficient).

The Search Function: DPP-Based Frame Retrieval

When the policy model emits a search instruction, it specifies a temporal window $[t_s, t_e]$ and a textual query $q$. The video environment must translate this into a concrete set of $F$ frames (maximum 8) that are both relevant to the query and diverse among themselves. The paper uses a determinantal point process (DPP) for this selection, chosen for its ability to balance relevance against diversity — it penalizes selecting multiple frames that are visually similar, thereby maximizing the information gained per retrieved frame.

Step 1: Candidate subsampling. The search function first subsamples $N$ candidate frames uniformly from the specified temporal window:

F[ts,te]={vi}i=1NF_{[t_s, t_e]} = \{v_i\}_{i=1}^N

where each $v_i$ is a frame within $[t_s, t_e]$. The paper does not specify the subsampling rate, but the mechanics are straightforward: if the window is dense with frames (e.g., 30 fps video across a 60-second window = 1800 frames), $N$ is a downsampled set to keep the candidate pool manageable for DPP computation.

Step 2: Embedding extraction. Each candidate frame $v_i$ is encoded into a visual embedding $h_i \in \mathbb{R}^d$ using SigLIP (Zhai et al., 2023), a contrastively trained vision-language model. Simultaneously, the textual query $q$ is encoded into a query embedding $\mathbf{q} \in \mathbb{R}^d$ using the same SigLIP text encoder. The shared embedding space ensures that cosine similarity between $h_i$ and $\mathbf{q}$ is a meaningful measure of visual-textual relevance.

Step 3: Pairwise similarity matrix. A similarity matrix $S$ is constructed where each entry $S_{ij}$ is the cosine similarity between the visual embeddings of frames $i$ and $j$:

Sij=hi⊤hjS_{ij} = h_i^\top h_j

where $h_i^\top h_j$ is the dot product of $L_2$-normalized embeddings, equivalent to cosine similarity. $S_{ij} = 1$ means the two frames are visually nearly identical; $S_{ij} = 0$ means they are orthogonal in the embedding space.

Step 4: Relevance scores. An unnormalized relevance score $\tilde{r}_i$ is computed for each candidate frame as the cosine similarity between its visual embedding and the query embedding:

r~i=q⊤hi\tilde{r}_i = \mathbf{q}^\top h_i

These raw scores are then rescaled to $[0, 1]$ via min-max normalization:

ri=r~i−min⁡r~max⁡r~−min⁡r~+ϵr_i = \frac{\tilde{r}_i - \min \tilde{r}}{\max \tilde{r} - \min \tilde{r} + \epsilon}

where $\epsilon$ is a small constant (added to avoid division by zero when all $\tilde{r}_i$ are equal). $r_i = 1$ means frame $i$ is the most query-relevant among all candidates; $r_i = 0$ means it is the least relevant.

Step 5: DPP kernel construction. The DPP kernel $\tilde{L}$ is constructed by combining the relevance scores with the similarity matrix:

L~=diag(r) S diag(r)\tilde{L} = \text{diag}(r) \, S \, \text{diag}(r)

where $\text{diag}(r)$ is a diagonal matrix with the relevance scores on the diagonal, and $S$ is the frame-to-frame similarity matrix. Expanding this:

L~ij=ri rj hi⊤hj\tilde{L}_{ij} = r_i \, r_j \, h_i^\top h_j

What this kernel represents: $\tilde{L}_{ij}$ is large when both frames $i$ and $j$ are individually relevant (high $r_i$ and $r_j$) and they are visually similar (high $h_i^\top h_j$). The DPP selection objective will penalize subsets with large $\tilde{L}_{ij}$ values, because the determinant of a submatrix of $\tilde{L}$ decreases when off-diagonal entries are large — intuitively, highly similar frames "compete" for inclusion.

Why this form: pure relevance-based selection (picking the top-$F$ frames by $r_i$) would often select multiple nearly-identical frames if a particular visual pattern is strongly correlated with the query — for example, if the query is "person at a desk," the top-10 most relevant frames might all show nearly the same static shot of a person at a desk from the same angle. This wastes the frame budget on redundant evidence. The DPP formulation explicitly models diversity through the similarity matrix $S$: two frames can both be highly relevant, but if they are very similar to each other, only one will typically be selected, freeing budget for frames that are relevant but visually distinct. The $\text{diag}(r)$ scaling ensures that relevance drives selection priority, while $S$ ensures that selected frames are spread across distinct visual content.

Step 6: MAP inference. The optimal subset $V^* \subset F_{[t_s, t_e]}$ of size $|V^*| = F$ is selected by maximizing the determinant of the submatrix of $\tilde{L}$ corresponding to the chosen indices:

V∗=arg⁡max⁡S⊆F[ts,te], ∣S∣=Fdet⁡(L~S)V^* = \arg\max_{S \subseteq F_{[t_s, t_e]},\, |S| = F} \det(\tilde{L}_S)

where $\tilde{L}_S$ is the $F \times F$ submatrix of $\tilde{L}$ restricted to the rows and columns corresponding to the selected frames. This is solved via fast greedy MAP inference (Chen et al., 2018), which builds the subset iteratively — at each step, it adds the frame that maximizes the marginal gain in $\det(\tilde{L}_S)$, providing an efficient approximation to the NP-hard exact maximization.

What this optimization computes: it finds the set of $F$ frames that jointly maximize a "volume" in the feature space spanned by the selected embeddings, weighted by relevance. Geometrically, this selects frames whose embeddings span a large volume (diverse) while being close to the query direction (relevant). Frames that are redundant (parallel embeddings) reduce the volume and are implicitly penalized.

Edge cases. When the temporal window contains fewer than $F$ candidate frames (e.g., a very short clip), the search function falls back to uniform temporal sampling — there is no diversity-relevance tradeoff to optimize because the budget exceeds the available content.

Frame representation with timestamps. The selected frames are sparse and non-uniformly spaced (unlike uniform sampling, the DPP may pick frames from arbitrary timestamps within the window). To preserve temporal grounding, each frame is prefixed with an explicit absolute timestamp token — e.g., "12.3s" inserted immediately before the image token. This interleaving of timestamp text and image tokens provides the model with precise temporal coordinates for each retrieved frame, even when inter-frame intervals vary dramatically. For the initial preview frames $\tilde{V}$, the paper uses the native dynamic-FPS and absolute time encoding from Qwen2.5-VL, which binds image token sequences to temporal IDs aligned with real timestamps.

Why explicit timestamps: without absolute timestamps, a sparsely sampled frame set loses all temporal context — the model cannot know whether two frames are 2 seconds apart or 2 minutes apart, making temporal reasoning (sequencing, duration estimation, before/after judgments) impossible. The timestamp prefix is a lightweight solution that encodes precise temporal metadata directly into the token stream that the model processes.

GRPO-CSV: The Core RL Algorithm

The GRPO-CSV algorithm extends standard GRPO with an auxiliary self-verification phase that provides supervision over the intermediate temporal search decisions. Understanding the algorithm requires understanding three things: (1) how standard GRPO works for this task, (2) what CSV adds and how it computes the completeness reward, and (3) the full reward composition.

Standard GRPO for interleaved text-video thinking. GRPO (Group Relative Policy Optimization) is an outcome-based RL algorithm that updates the policy $\pi_\theta$ by comparing multiple rollouts (generations) for the same prompt. Given a video-question pair, the current policy generates $G$ complete trajectories (set to 8 in this paper), each consisting of an interleaved CoT $C$ and a final answer $A$. A reward function $R(A, A^*)$ scores each trajectory based only on the final answer relative to the ground truth $A^*$. The advantage for each trajectory is computed as its reward normalized relative to the group mean and standard deviation:

Ai=Ri−mean(R1:G)std(R1:G)A_i = \frac{R_i - \text{mean}(R_{1:G})}{\text{std}(R_{1:G})}

where $A_i$ is the advantage for trajectory $i$, $R_i$ is its reward, and the mean and standard deviation are computed over the $G$ trajectories in the group. Positive advantage means the trajectory performed better than the group average; negative means worse. The policy is then updated to increase the log-probability of tokens from high-advantage trajectories and decrease it for low-advantage trajectories, with a KL penalty term preventing the policy from diverging too far from a reference model.

What this computes: standard GRPO provides a learning signal only through the final outcome — trajectories that happen to produce correct answers are reinforced, trajectories that produce incorrect answers are suppressed. There is no direct signal about whether the search process was good, only whether the final answer was correct.

The problem: as explained in the failure mode analysis, LVLMs can arrive at correct answers through partial visual evidence or linguistic bias. A trajectory with a lazy search (retrieving only tangentially relevant frames) that happens to guess correctly receives the same positive advantage as a trajectory with thorough, systematic search. The gradient updates cannot distinguish between these cases — the search policy $P_\theta(C \mid \tilde{V}, Q)$ is updated based on the downstream answer correctness, not on the intrinsic quality of the search. Over training, the model may learn to minimize search effort (since search costs tokens and increases the probability of format errors) while maintaining acceptable answer accuracy, leading to the collapse observed in Figure 4(b), where the model gradually reduces search calls and eventually stops searching entirely.

The CSV mechanism. Completeness Self-Verification addresses this by introducing a second evaluation phase after the main trajectory is generated. The procedure works as follows:

  1. GRPO rollout phase: For a given video-question pair $(V, Q)$, the policy model generates a full trajectory producing an interleaved CoT $C$ and a final answer $A$. The video clips $V_1, V_2, \ldots, V_k$ retrieved during the $k$ search steps are extracted from $C$ and concatenated into a dynamic frame set $V_c$. This $V_c$ is the union of all frames the model chose to look at — it is the visual evidence the model deemed sufficient to answer the question.

  2. CSV rollout phase: The same policy model is now given the question $Q$ and only the dynamic frame set $V_c$, with no access to the original preview $\tilde{V}$ and with all search functionality disabled (the model cannot call the seek_video_frames tool). The model is prompted to answer as briefly as possible and to say "I don't know" if the visual evidence is insufficient. It produces a CSV answer $A_c$.

  3. Completeness condition: The CSV answer $A_c$ is expected to be consistent with the original answer $A$ — that is, if the original answer was correct and the searched frames truly contain sufficient evidence, the model should be able to reproduce the correct answer from those frames alone:

Pθ(Ac∣Vc,Q)≈Pθ(A∣C,V~,Q)P_\theta(A_c \mid V_c, Q) \approx P_\theta(A \mid C, \tilde{V}, Q)

where the left side is the probability under the policy of producing the correct answer given only the searched frames, and the right side is the probability under the policy of producing the correct answer given the full trajectory context.

Why this form works: the CSV phase tests whether the dynamic frame set $V_c$ is self-sufficient — whether an independent inference from those frames alone can recover the ground-truth answer. If the model searched well (retrieving frames that contain decisive visual evidence), $A_c$ should match $A^*$. If the model searched poorly (retrieving frames that lack key information but guessing correctly anyway), $A_c$ will likely be wrong since the model has no linguistic shortcuts or partial evidence to exploit — it must answer from exactly the frames it chose. The CSV phase is "completeness self-verification" because the model verifies its own search completeness without external annotation.

The completeness reward. The completeness reward $R_c$ is defined as:

Rc=1[Acc(A,A∗)>0.5]⋅Acc(Ac,A∗)R_c = \mathbf{1}[\text{Acc}(A, A^*) > 0.5] \cdot \text{Acc}(A_c, A^*)

where $\text{Acc}(A, A^*)$ is the correctness score (between 0 and 1, typically binary for multiple-choice and continuous from LLM-as-judge for open-ended) of the original answer $A$ against the ground truth $A^*$, $\text{Acc}(A_c, A^*)$ is the correctness score of the CSV answer $A_c$ against the same ground truth, and $\mathbf{1}[\cdot]$ is the indicator function that evaluates to 1 when its argument is True and 0 otherwise.

What this computes step by step:

  • First, check if the original answer $A$ is correct (score exceeds 0.5, meaning more likely correct than incorrect).
  • If $A$ is incorrect, $R_c = 0$ regardless of $A_c$. This conditional gating prevents the model from being rewarded for good search on trajectories where it ultimately answered wrong — the search might have been thorough but the reasoning was flawed, and the policy should not be reinforced for the search component of a failed trajectory.
  • If $A$ is correct, $R_c = \text{Acc}(A_c, A^*) — the reward equals how well the model can re-answer from the searched frames alone.
  • If the search was sufficient and logical reasoning was consistent, $A_c$ should be correct, yielding $R_c = 1$.
  • If the search was insufficient (the model guessed correctly without proper visual grounding), $A_c$ will likely be wrong, yielding $R_c = 0$.

Why this conditional form: the gating on original answer correctness $\mathbf{1}[\text{Acc}(A, A^*) > 0.5]$ is a design choice with specific implications. Without this gating, a trajectory with a wrong answer but thorough search could receive a positive completeness reward (if $A_c$ were also wrong but $\text{Acc}(A_c, A^*)$ happened to be high by chance), creating a perverse incentive: the model could learn to search thoroughly but reason poorly, and still be reinforced. The gating ensures that completeness reward is additive to accuracy reward, not compensatory — thorough search is only rewarded when it actually leads to correct reasoning. This aligns the completeness objective with the overall goal of correct answer prediction, preventing reward hacking where the model maximizes $R_c$ by searching exhaustively but then guessing randomly.

The format reward. A binary format reward $R_{\text{fmt}}$ enforces adherence to a predefined output schema:

Rfmt={1if all steps follow the required format0otherwiseR_{\text{fmt}} = \begin{cases} 1 & \text{if all steps follow the required format} \\ 0 & \text{otherwise} \end{cases}

The required format is: each thinking step must use thinking... response followed by either <tool_call>...</tool_call> (for search steps) or <answer>...</answer> (for the final answer step). The reward is assigned to the full trajectory (not per-step), meaning a single format violation anywhere in the chain zeros the format reward for the entire trajectory.

What this enforces: the format reward ensures that the model's output remains machine-parseable by the video environment and the answer extraction logic. Without this reward, the RL optimization might drift toward free-form text that is indistinguishable from tool calls or answers, breaking the environment interface. This is a standard technique in tool-use RL to maintain action-space constraints during policy optimization.

Why binary rather than continuous: a continuous format score (e.g., partial credit for mostly correct formatting) would be harder to define and might create perverse incentives (e.g., the model could learn to output syntactically valid but semantically empty tool calls). Binary scoring is simple, unambiguous, and directly penalizes any deviation from the required interaction protocol.

The accuracy reward. The accuracy reward $R_{\text{acc}}$ evaluates whether the final answer matches the ground truth:

Racc=Acc(A,A∗)R_{\text{acc}} = \text{Acc}(A, A^*)

The computation of $\text{Acc}(A, A^*)$ depends on question type:

  • Multiple-choice: extract the option letter from the model's output and perform exact string match with the ground-truth option. Score is 1 for match, 0 otherwise.
  • Open-ended: use an LLM-as-a-Judge approach (GPT-4o, following Zheng et al., 2023) to assess semantic agreement between the model's answer and the reference answer. The LLM judge produces a binary score (1 for semantically equivalent, 0 otherwise).

Why LLM-as-judge for open-ended: exact string matching fails for answers that are correct but phrased differently (e.g., "the drawer was not closed" vs. "No, the drawer remained open"). A semantic judge can recognize equivalence despite lexical variation, providing a more accurate reward signal. The binary scoring (rather than continuous) keeps the reward scale consistent with the format and completeness components, simplifying the advantage computation.

The overall reward. The total reward for a trajectory is the unweighted sum:

R=Rc+Rfmt+RaccR = R_c + R_{\text{fmt}} + R_{\text{acc}}

yielding a maximum possible reward of 3 (correct answer + valid format + complete search evidence) and a minimum of 0 (incorrect answer, or invalid format with incomplete search, etc.).

What the sum composition achieves: each component addresses a distinct failure mode. $R_{\text{acc}}$ provides the primary signal for answer quality. $R_{\text{fmt}}$ prevents structural degeneration. $R_c$ provides the missing signal on search quality — it penalizes trajectories where the answer is correct but the search was insufficient (addressing Failure Mode 1: insufficient temporal exploration) and trajectories where the search was good but the CSV re-answer fails due to inconsistent reasoning (addressing Failure Mode 2: inconsistent logical reasoning, since CSV forces the model to re-derive the answer from the same visual evidence, and inconsistency between $A$ and $A_c$ is penalized by low $\text{Acc}(A_c, A^*)$).

Why unweighted sum rather than learned weights: a weighted sum with learned coefficients would introduce additional hyperparameters and training instability — the model could learn to maximize the easiest reward component while ignoring harder ones. Equal weighting is simple, interpretable, and forces the policy to satisfy all three criteria simultaneously to achieve high total reward.

KL penalty and policy update. Following standard GRPO, the policy update includes a KL divergence penalty between the current policy $\pi_\theta$ and a reference policy $\pi_{\text{ref}}$ (typically the SFT checkpoint or the policy from the start of the RL stage):

LGRPO=−E(Q,V)[1G∑i=1GAi⋅log⁡πθ(Ai,Ci∣V~,Q)]+β⋅KL(πθ∥πref)\mathcal{L}_{\text{GRPO}} = -\mathbb{E}_{(Q,V)} \left[ \frac{1}{G} \sum_{i=1}^G A_i \cdot \log \pi_\theta(A_i, C_i \mid \tilde{V}, Q) \right] + \beta \cdot \text{KL}(\pi_\theta \parallel \pi_{\text{ref}})

where $\beta$ is the KL penalty coefficient (set to 0.005), $G = 8$ is the number of rollouts per prompt, and $A_i$ is the advantage computed from the total reward $R$ as described above.

What the KL penalty prevents: without it, the policy could drift far from the reference model during RL, potentially losing the base capabilities (general video understanding, language generation quality) that the SFT stage established. The KL penalty acts as a regularizer, ensuring that policy improvements are local and don't catastrophically forget pre-trained behaviors. The coefficient $\beta = 0.005$ represents a relatively light penalty — the paper prioritizes search capability acquisition over strict proximity to the SFT model.

Dataset Construction: Two-Stage Filtering Pipeline

The RL training's effectiveness critically depends on the quality of the training data. The paper identifies that standard video QA datasets contain two types of samples that are toxic for RL, and addresses them through a two-stage filtering pipeline shown in Figure 6.

The toxicity problem for RL. GRPO computes advantages by comparing rollouts within a group. For the advantage signal to be meaningful, there must be variation in reward across rollouts — some rollouts must be better than others, so the policy can learn which behaviors to reinforce. Two types of samples destroy this variation:

  • Trivially answerable questions: if the model can answer correctly from minimal visual input (4 uniformly sampled frames, or even zero frames via linguistic bias), then all rollouts will be correct regardless of search behavior. The advantage for every trajectory is approximately zero (since all rewards equal the group mean), providing no gradient signal. RL training on such samples wastes compute and can even be harmful — the optimizer may drift due to noise in the tiny residual advantages.

  • Impossibly hard questions: if the question cannot be answered correctly even with extensive search and many frames, then all rollouts will be incorrect. Again, zero advantage, no learning signal. Worse, the model might learn spurious search behaviors that sometimes produce correct answers by chance, creating unstable policy updates.

Stage 1: Visual dependency filtering. This stage removes questions that can be solved with minimal visual context:

  1. Sample 4 frames uniformly from the full video (a deliberately impoverished visual context).
  2. Feed these 4 frames + the question to Qwen2.5-VL for direct answer prediction (no search, no CoT — the model must answer immediately).
  3. Use an LLM-as-a-Judge to evaluate whether the answer is correct against the ground truth.
  4. If correct → filter out the sample (the question has low visual dependency and can be solved trivially).
  5. If incorrect → retain the sample (the question requires richer visual context that 4 frames cannot provide).

What this filtering achieves: it ensures that the remaining samples genuinely require temporal search to answer correctly. A model cannot rely on linguistic shortcuts or the initial preview frames — it must actively explore the video to find decisive evidence. This makes search behavior directly consequential for reward, creating the variation in rollouts that GRPO needs.

Stage 2: Search usefulness filtering. This stage removes questions that remain unsolvable even with extensive search, preventing the model from wasting training on impossible tasks:

  1. Provide the model with up to 64 frames and enable dynamic temporal search.
  2. Use different LVLMs for different purposes:
    • GPT-4o generates the text-video interleaved reasoning traces used for SFT training data.
    • An early version of TimeSearch (a preliminary RL-trained model) generates the question-answer pairs used for RL training data.
  3. For each sample, evaluate whether the LVLM can produce a correct answer with search.
  4. If incorrect → filter out (the question is too hard for the current model capacity).
  5. If correct → retain (the question is challenging but solvable with good search, making it suitable for RL).

What this filtering achieves: it bounds the difficulty of retained samples — they must be hard enough to require search (Stage 1) but easy enough to be solvable with good search (Stage 2). This "zone of proximal development" is where RL is most effective: the policy can learn because good search behaviors reliably lead to reward, but it must actually perform good search to get that reward.

Human selection for automatically generated QA pairs. The VideoMarathon dataset (which contributes 35.6% of the training data) contains automatically generated question-answer pairs that may have incorrect ground-truth labels or be unanswerable from the video content. The paper implements a manual verification protocol:

  1. Human annotators assess whether each question is reasonable based on the video content, filtering out unanswerable or ambiguous questions.
  2. Annotators provide their own answers and compare them against the synthetic ground-truth labels.
  3. Samples where human answers disagree with synthetic labels are removed.

Why this is necessary: RL with incorrect ground-truth labels would be catastrophic — the model would be reinforced for producing factually wrong answers, learning to replicate the labeling errors rather than genuinely understanding the video. Manual verification ensures label correctness for the portion of the data where automatic generation is least reliable.

Final dataset composition. After filtering, the training set consists of:

  • 49.5% from Haystack-Ego4D (egocentric daily activities with frequent viewpoint changes)
  • 35.6% from Panda-70M via VideoMarathon (diverse internet videos with heterogeneous motion patterns)
  • 9.5% from CinePile (short videos with narrative structure and rapid scene transitions)
  • 5.4% from other sources (to reduce distributional bias)

Question types are 60.3% open-ended and 39.7% multiple-choice. The mean video duration is 1,659 seconds (approximately 27.7 minutes), with a long tail extending beyond 10,000 seconds (approximately 2.8 hours). This duration distribution is deliberately challenging — at 2 FPS maximum, a 1,659-second video would contain over 3,300 frames, far exceeding the model's visual context budget, making adaptive search essential rather than optional.

Overall Training Procedure

The training proceeds in two stages: supervised fine-tuning (SFT) as a cold start, followed by RL with GRPO-CSV.

Stage 1: Supervised fine-tuning (SFT). The purpose of SFT is to teach the model the basic mechanics of interleaved text-video thinking — how to format search instructions, how to incorporate retrieved frames into reasoning, and how to terminate with an answer — before RL optimization begins. Without SFT, the model cannot engage with the video environment at all (as shown in Table 4a: zero-shot CoT with search achieves 0.0 temporal F1 and 0% completeness, meaning the model never successfully issues a search instruction that retrieves useful frames).

The SFT training data is constructed using the filtered dataset: for each retained sample, GPT-4o generates a text-video interleaved reasoning trace (the chain of thought $C$) and a final answer $A$. The model is fine-tuned to maximize the likelihood of these traces:

LSFT=−E(Q,V)[log⁡πθ(C,A∣V~,Q)]\mathcal{L}_{\text{SFT}} = -\mathbb{E}_{(Q,V)} \left[ \log \pi_\theta(C, A \mid \tilde{V}, Q) \right]

with a crucial modification: temporal search results (the retrieved video frames) are masked during training — gradient computation excludes tokens corresponding to video frames. This forces the model to learn to predict meaningful temporal windows $[t_s, t_e]$ and textual queries $q$ without conditioning on the actual search results, preventing the model from simply memorizing what GPT-4o saw and instead requiring it to learn the policy for deciding what to search.

Why mask video tokens: if video tokens were included in the loss, the model could learn a trivial shortcut — it sees the retrieved frames in the training data and learns to copy the final answer without learning why those particular frames were retrieved. Masking forces the model to predict the search parameters from the reasoning context alone, making the search decision the primary object of learning.

Stage 2: RL with GRPO-CSV. Building on the SFT checkpoint, the model is further optimized using the GRPO-CSV algorithm described in Section 3.4.3. Key hyperparameters from Table 5:

  • Number of generations per prompt: 8 (the group size for relative advantage computation)
  • KL penalty coefficient $\beta$: 0.005
  • Max search turns: 8 (the model can search at most 8 times per question)
  • Max completion length per turn: 256 tokens (limits verbose reasoning to prevent the model from wasting context)
  • Max frames per search operation: 8 (each seek_video_frames call retrieves at most 8 frames)
  • Max total video tokens: 10,240 (the total context budget for all video content, including preview and searched frames)
  • Max frames per video: 768 (hard cap on total frames across all search turns)
  • Min tokens per frame: 12, Max tokens per frame: 256 (dynamic token allocation based on content complexity)
  • Max FPS for video processing: 2 (videos are downsampled to at most 2 frames per second before search)
  • Optimizer: AdamW with learning rate $1 \times 10^{-6}$
  • Batch size per GPU: 1, with 2 gradient accumulation steps (effective batch size depends on the number of GPUs)
  • Infrastructure: 32 A100 GPUs, DeepSpeed ZeRO-3 with offloading, VLLM in colocate mode for efficient rollout generation, bfloat16 mixed precision, Flash Attention 2.0

The colocate VLLM setup means the inference engine for generating rollouts runs on the same GPUs as the training process, sharing GPU memory. This avoids the overhead of transferring model weights between separate inference and training servers but requires careful memory management (hence the ZeRO-3 offloading to CPU).

Interaction constraints. The environment enforces a maximum of 8 search turns per question. If the model issues a 9th search instruction, it is ignored or the trajectory is truncated. This prevents the model from learning an infinite-loop strategy where it searches indefinitely without ever committing to an answer. Combined with the 256-token limit per turn and the 8-frame limit per search, the total computational cost per trajectory is bounded, making RL training feasible within the 32-GPU budget.

The replay buffer. The paper notes that a replay buffer is enabled (true in Table 5). In RL for LLMs, a replay buffer stores previous rollouts and their rewards, allowing the policy to be updated on a mixture of on-policy and off-policy data. This can improve sample efficiency by reusing computations and stabilizing training by smoothing the data distribution. However, since GRPO is an on-policy algorithm (it computes advantages relative to the current policy's rollouts), the replay buffer here likely serves to store environment interactions for the CSV phase rather than for off-policy policy updates — the CSV re-answer uses frames from the original trajectory, so the trajectory must be preserved until CSV is complete.

4. Key Insights and Innovations

Innovation 1: Temporal Search as an Integral Part of Reasoning, Not a Pre-Processing Step

The most fundamental conceptual move in this paper is the reformulation of temporal search from a pre-reasoning filtering operation into an interleaved component of the reasoning chain itself. Prior to this work, all approaches to long-form video understanding treated frame selection as something that happens before the model starts thinking — either through static uniform sampling (Qwen2.5-VL, GPT-4o, Gemini-1.5-Pro), hand-crafted agent workflows where an LLM plans searches as a separate orchestrator (VideoAgent, T*, VideoTree), or even in text-only video reasoning (Video-R1) where the model thinks about pre-sampled frames without any ability to request new visual evidence. In every case, the reasoning process and the visual evidence were temporally decoupled: the frames were frozen before a single reasoning token was generated.

The paper's reframing — what it calls "Thinking with Videos" — makes search decisions conditioned on intermediate reasoning states. The model doesn't decide "what to look at" from a cold start; it formulates a partial hypothesis ("maybe the drawer was left open?"), then searches for evidence to confirm or refute that specific hypothesis, then revises its understanding based on what it finds, then potentially searches again. This is not a small architectural tweak — it represents a fundamental shift in what it means to "understand" a long video. Understanding is no longer a function applied to a fixed representation; it is a process of active evidence gathering where what you look at next depends on what you've already figured out.

What makes this intellectually distinctive — rather than an obvious extension of tool-use in LLMs — is that it closes the loop between reasoning and perception at the training level, not just the inference level. VideoAgent and T* already enabled multi-turn search at inference time, but their search strategies were hand-crafted through prompt engineering. The sequence of "what to search for when" was designed by humans, not learned from data. TimeSearch-R learns this mapping end-to-end through RL, meaning the model discovers from experience that certain reasoning states should trigger certain types of searches — a discovery that is inaccessible to prompt engineering because it depends on statistical patterns in what visual evidence actually proves decisive across thousands of videos.

The evidence for why this reframing matters is most visible in the ablation in Table 4a, where the model given search capability but no training (zero-shot CoT with search) achieves 0.0 temporal F1 — it literally cannot use the search tool effectively. SFT provides the basic mechanics (7.8 F1), but only RL (8.1 F1) optimizes the search policy. The collapse observed in Figure 4(b) when CSV is removed — where the model gradually stops searching entirely — further demonstrates that the interleaved search behavior is fragile and must be actively maintained through appropriate reward design, not merely enabled through architecture. This fragility is precisely what makes the learned policy valuable: it represents a non-trivial optimization that hand-crafted approaches cannot replicate.

Innovation 2: Outcome-Based Supervision of Intermediate Search Through Self-Verification

The paper's second distinctive contribution is the Completeness Self-Verification (CSV) mechanism, which achieves something that appears paradoxical: it provides per-step supervision over the search process without any per-step annotations. Standard approaches to training search or retrieval policies require ground-truth labels indicating which frames are relevant (as in T*'s evaluation framework, which uses temporal F1 against annotated keyframes). Collecting such annotations at scale for diverse videos is prohibitively expensive and limits the applicability of search-based methods.

CSV sidesteps this entirely through a cleverly designed auxiliary task: after the model completes its full search-and-reason trajectory, ask it to re-answer the question using only the frames it chose to retrieve, with search disabled. If the retrieved frames genuinely contain sufficient evidence, the model should be able to answer correctly. If not, the re-answer will fail. This transforms the problem from "do we have labels for which frames are good?" to "can the model answer from its own selected frames?" — a form of self-supervision on search quality that requires nothing beyond the video-question-answer triplets already available in video QA datasets.

What makes this intellectually significant — beyond the practical benefit of avoiding annotation costs — is that it resolves a fundamental credit assignment problem in temporal search RL. In standard outcome-based GRPO, the search policy $P_\theta(C \mid \tilde{V}, Q)$ receives gradient signal only indirectly through whether the final answer $A$ is correct. But LVLMs can guess correctly from partial evidence or linguistic bias, meaning correct answers do not imply good search. CSV breaks this confounding by forcing an independent evaluation: the re-answer task removes the model's ability to exploit shortcuts (it must answer from exactly the frames it searched) and the model's own knowledge of the question difficulty (it must commit to an answer or admit uncertainty). The completeness reward $R_c = \mathbf{1}[\text{Acc}(A, A^*) > 0.5] \cdot \text{Acc}(A_c, A^*)$ then gates reinforcement on both the original answer being correct and the searched frames being sufficient — a conjunction that is only satisfied when search was genuinely adequate.

The evidence for CSV's necessity is stark. Figure 4(b) shows that removing CSV causes the number of search turns to collapse to zero around step 300 — the model learns that searching costs tokens and format rewards without providing compensating benefit to the outcome reward. Table 4a quantifies the degradation: GRPO without CSV drops completeness from 60.5% to 57.2% and temporal F1 from 7.8 to 7.4. The conceptual message is clear: outcome rewards alone cannot sustain learned search behavior in long-form video reasoning. The search policy will collapse without intermediate supervision, and CSV provides that supervision at zero annotation cost. This finding has implications beyond video — it suggests that any domain where RL agents must learn to gather evidence for reasoning (document QA, web navigation, multi-step tool use) likely requires analogous intermediate verification mechanisms to prevent evidence-gathering from atrophying.

Innovation 3: The Two Failure Modes of Outcome-Only RL for Interactive Reasoning

The paper contributes an important diagnostic framework by identifying and naming the two specific failure modes — insufficient temporal exploration and inconsistent logical reasoning — that cause naive application of GRPO to fail for interactive video reasoning. This is not merely a taxonomy of bugs; it is a causal analysis of why outcome-based RL breaks when applied to tasks that require active evidence gathering.

The first failure mode, insufficient temporal exploration, arises because LVLMs possess linguistic biases and partial-visual-cue heuristics that allow them to answer many questions without thorough visual grounding (a well-documented phenomenon in the VQA literature, e.g., Niu et al., 2021). In an RL setting, this creates a perverse dynamic: the policy can maintain acceptable accuracy while gradually reducing search effort, since the marginal benefit of each additional search to outcome reward is small or zero for many questions. The RL optimizer sees no gradient signal distinguishing the "lazy but lucky" trajectory from the "thorough and correct" trajectory, and over training, the policy drifts toward the minimum-effort strategy that preserves accuracy. This is what Figure 4(b) captures — not a sudden catastrophic failure, but a gradual erosion of search behavior across ~300 training steps until the model stops searching entirely.

The second failure mode, inconsistent logical reasoning, is subtler but equally damaging. In text-based RL, a well-known problem is that models can produce plausible-sounding CoT that is functionally disconnected from their final answers — the reasoning text is optimized for coherence rather than causal fidelity to the conclusion (Lanham et al., 2023). In the video search setting, this means the model might produce detailed, seemingly analytical search instructions and reasoning traces while the final answer is effectively generated from a different "pathway" through the model that ignores the searched frames. Outcome rewards provide no penalty for this disconnect — the final answer might be correct even if the reasoning trace is a post-hoc rationalization rather than a genuine causal chain.

What makes this diagnostic contribution valuable is that it is predictive: it tells future researchers where to look for problems when outcome-based RL underperforms for interactive tasks. The two failure modes are not specific to video — they apply to any domain where an RL agent must balance evidence-gathering effort against task accuracy and where the agent has access to shortcuts that bypass proper evidence-gathering. The paper's identification of these modes also provides clear evaluation criteria (completeness and consistency metrics, defined in Appendix D) that can be measured independently of final accuracy, enabling more fine-grained analysis of why a particular RL training run succeeded or failed.

Evidence for the framework's validity comes from the ablation in Table 4a, where adding CSV to GRPO increases completeness from 57.2% to 61.2% (addressing insufficient exploration) and consistency from 69.3% to 75.3% (addressing inconsistent reasoning) — demonstrating that the identified failure modes are not merely hypothetical but are the actual mechanisms through which CSV improves performance.

Innovation 4: Learned Search Strategies Exhibit Emergent Cognitive Patterns

Beyond the quantitative improvements, the paper provides qualitative evidence that the RL-trained search policy discovers interpretable, task-adaptive search strategies that mirror human cognitive patterns — without any explicit programming of these strategies into the reward or training data. The case studies in Figures 5, 13, 14, and 15 reveal three distinct search behaviors that the model spontaneously organizes around:

  • Hypothesis-driven search (Figure 5): the model forms a specific claim about what is happening ("the dogs are posing for a photo") and searches for confirmatory evidence (the person taking the photo), using the search results to verify rather than to guess.

  • Confirmation/elimination (Figures 13 and 14): when faced with a multiple-choice question, the model searches for evidence pertaining to specific options — confirming the presence of one option or searching for a specific option to eliminate it when initial evidence is inconclusive.

  • Sequential search (Figure 15): for questions requiring temporal ordering, the model searches segment-by-segment through the video timeline, reconstructing the event sequence iteratively rather than attempting to locate all relevant frames in a single query.

These strategies are emergent — they were not specified in the training data (GPT-4o generated the SFT traces, but TimeSearch-R's RL policy diverged from these traces) or in the reward function (only accuracy, format, and completeness were rewarded, with no shaping toward particular search patterns). The model discovered that these strategies work better through the direct optimization pressure of the RL objective. This is a significant finding because it demonstrates that end-to-end RL can induce qualitatively sophisticated search policies that go beyond the heuristics a human engineer would encode. The strategies are adaptive to question type (hypothesis-driven for causal questions, elimination for multiple-choice, sequential for temporal-ordering questions), suggesting the model is learning a meta-policy for which search strategy to deploy when, not just a single fixed routine.

What makes this conceptually important — beyond being visually compelling — is that it validates the core premise of the paper against a plausible alternative hypothesis. One could argue that the gains from RL are merely optimizing superficial aspects of search (better query phrasing, slightly more precise temporal windows) rather than learning fundamentally different search behaviors. The emergence of distinct, task-appropriate strategies refutes this: the RL policy is not just a refined version of the SFT policy; it is a qualitatively different search agent that organizes its interactions with the video environment around cognitive patterns the SFT stage did not exhibit.

The failure cases (Figures 16 and 17) are equally informative: the model sometimes terminates search prematurely after examining only a subset of options (insufficient exploration, despite CSV) or hallucinates information not present in retrieved frames (visual hallucination). These residual failures reveal the boundaries of what CSV can correct — it provides pressure toward complete exploration but cannot guarantee it when the model halts early, and it cannot prevent the model from "seeing" things in frames that aren't there.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. The paper evaluates on two task categories. For temporal search, it uses Haystack-LVBench and Haystack-Ego4D (Ye et al., 2025) — benchmarks designed as needle-in-a-haystack tests where models must locate specific frames relevant to a question within long videos. For long-form video understanding, it uses VideoMME (Fu et al., 2024; 3,000 videos across short/medium/long durations, evaluated without subtitles), MLVU (Zhou et al., 2024; multi-task long video understanding), and LongVideoBench (Wu et al., 2024; long-context interleaved video-language understanding). The test set sizes and splits are not explicitly stated in the main paper for all benchmarks; Haystack-Ego4D uses a "test-tiny" subset per Ye et al. (2025).

  • Base model(s). All experiments use Qwen2.5-VL-7B-Instruct (Bai et al., 2025a), a 7-billion-parameter open-source video-language model. The paper states this model is "representative of the capabilities of many contemporary LLMs" (Section 1) and operates in a regime where temporal search can provide meaningful gains — the model has non-trivial baseline performance (e.g., 65.1% on VideoMME, 56.0% on LongVideoBench with 768 uniformly sampled frames) but substantial room for improvement, particularly on longer videos (54.6% on VideoMME long subset). The FLOPs-matched comparison in Table 2 includes GPT-4o, Gemini-1.5-Pro, and the reasoning model Video-R1 as additional reference points.

  • Metrics. For temporal search on Haystack-LVBench, the paper reports precision (P), recall (R), and F1 score for both temporal similarity (how well the model's searched timestamps align with ground-truth keyframe timestamps) and visual similarity (how well the selected frames match ground-truth relevant frames in embedding space), along with QA accuracy. For Haystack-Ego4D, only QA accuracy is reported. For video understanding benchmarks, standard accuracy (percentage of correctly answered questions) is reported, with VideoMME broken into short/medium/long duration subsets. The paper additionally introduces two ablation metrics for internal analysis: completeness (defined in Appendix D as the proportion of cases where the dynamic frame set alone suffices to produce the correct answer, measured by re-answering from only the searched frames) and consistency (alignment between intermediate reasoning text and final answer, judged by GPT-4o as a binary yes/no per Appendix D).

  • Baselines. The paper compares against three categories. (1) Static frame sampling: Qwen2.5-VL-7B with 768 uniformly sampled frames (the base model), GPT-4o with 384 frames, Gemini-1.5-Pro at 1 fps. (2) Adaptive temporal search agents: VideoAgent (Wang et al., 2024) using GPT-4 with approximately 10.1 keyframes on average, T* (Ye et al., 2025) using GPT-4o with 8 or 32 keyframes, VideoTree (Wang et al., 2025) using GPT-4 with approximately 128 keyframes, and a retrieval-based baseline from Ye et al. (2025) using GPT-4o. (3) Video reasoning models: Video-R1-7B (Feng et al., 2025) evaluated at both 32 and 768 frames, representing text-only chain-of-thought reasoning over static frames. The paper also includes an ablation baseline labeled "Qwen2.5VL-7B + Search" (Table 2) where search capability is added to the base model via zero-shot prompting without SFT or RL training, to isolate the effect of learned search strategies from mere search access.

  • Generation budget / compute accounting. Budget is measured in number of frames (Table 1 and 2 report "# Frame" for each method). For static sampling methods, this is the fixed number of uniformly sampled frames. For search-based methods, this is the dynamic number of keyframes retrieved across all search turns. TimeSearch-R uses an average of 8.8 frames on Haystack-LVBench and up to 768 total frames across all search turns during training (Table 5: max 8 search turns × max 8 frames per search = up to 64 dynamically retrieved frames, plus the initial preview frames within the 768-frame cap). The paper also reports latency (Table 4, Appendix E) to account for the computational cost of different retrieval backends (SigLIP-400M for TimeSearch-R vs. CLIP-1B for VideoAgent vs. YOLO-World-110M for T*). For the RL training itself, generation budget per prompt is 8 rollouts (GRPO group size), and training is conducted on 32 A100 GPUs.

  • Cross-validation / statistical protocol. The paper does not report cross-validation or statistical significance testing. For the difficulty-based analysis (comparable to the reference example's two-fold cross-validation), no analogous procedure is described — the ablation studies in Table 4a and Figure 4 appear to report single-run results without confidence intervals or multiple seeds. This is a notable methodological gap: the training dynamics collapse shown in Figure 4(b) (three separate curves for GRPO, GRPO-CSV, and GRPO-CSV without accuracy reward) would be more convincing with evidence that the collapse vs. stability pattern is reproducible rather than a single-training-run artifact. The comparison of search patterns in case studies (Section 3.4) is qualitative without quantitative frequency analysis of how often each pattern occurs.

Main Quantitative Results

Temporal Search on Haystack Benchmarks

Table 1 presents the paper's headline results on the temporal search task. TimeSearch-R achieves a temporal F1 score of 8.1 on Haystack-LVBench, more than three times the previous best result of 2.5 from T* (GPT-4o, 8 frames) and substantially exceeding the 2.1 from VideoAgent. The precision-recall breakdown shows this gain comes primarily from improved recall: TimeSearch-R achieves 22.3 recall compared to 7.1 for T* and 8.5 for VideoAgent, while precision (5.4) is also markedly higher than baselines (1.2–1.7). On visual similarity, TimeSearch-R achieves an F1 of 69.2, surpassing the previous SOTA VideoAgent (64.7) by 4.5 points and even outperforming retrieval-based methods with larger frame budgets (67.8 at 32 frames). On needle-in-a-haystack QA accuracy, TimeSearch-R achieves 52.1% on Haystack-LVBench and 53.5% on Haystack-Ego4D, consistently outperforming the API model GPT-4o (47.1% and 41.5% respectively with 8 frames, 50.5% and 45.5% with 32 frames) and T* (51.9% and 45.0% with 8 frames, 53.1% and 46.5% with 32 frames). A key comparison is against the retrieval-based baseline at 32 frames: despite using only 8.8 frames on average, TimeSearch-R achieves comparable or better QA accuracy (52.1% vs. 50.5% on LVBench, 53.5% vs. 45.5% on Ego4D), demonstrating the efficiency of learned frame selection over retrieval-based heuristics.

The paper does not report temporal F1 on Haystack-Ego4D, making the Haystack-LVBench F1 results the sole quantitative measure of temporal localization quality. This is a limitation — Ego4D's egocentric videos with frequent viewpoint changes present different search challenges than LVBench, and it is unclear whether the temporal precision gains transfer.

Long-Form Video Understanding

Table 2 reports results on three video understanding benchmarks. TimeSearch-R achieves an overall accuracy of 66.6% on VideoMME (without subtitles), improving over the base model Qwen2.5-VL-7B by 1.5% (65.1% → 66.6%). The improvement is duration-dependent: +0.5% on short videos, +1.1% on medium videos, and +1.4% on long videos — consistent with the paper's claim that temporal search becomes more valuable as video length increases. On MLVU, TimeSearch-R achieves 71.5% (m-avg), a 1.3% improvement over Qwen2.5-VL (70.2%). On LongVideoBench, the improvement is most substantial: 60.1% vs. 56.0%, a 4.1% absolute gain.

Compared against adaptive temporal search agents, TimeSearch-R outperforms VideoAgent (56.0% on VideoMME) by 10.6% and T* (64.1%) by 2.5%. Against the video reasoning model Video-R1, TimeSearch-R outperforms it across all benchmarks: 66.6% vs. 65.7% on VideoMME (at 768 frames), 71.5% vs. 68.4% on MLVU, and 60.1% vs. 58.1% on LongVideoBench. The gap is consistent but modest (~1–2 percentage points) against Video-R1 at the same 768-frame budget, suggesting that text-video interleaved thinking provides incremental but reliable gains over text-only reasoning on standard benchmarks.

A critical ablation appears in the "Qwen2.5VL-7B + Search" row of Table 2: simply adding search capability to the base model through prompting (no SFT, no RL) causes performance to degrade substantially — 51.8% on VideoMME vs. 65.1% for the base model without search, a 13.3% drop. This demonstrates that temporal search is not automatically beneficial; the model must be trained to use it effectively. The SFT stage recovers most of this gap (59.2% in Table 4a, close to the 65.1% baseline), and RL provides the remaining improvement to 66.6%.

Comparison Between SFT and RL Training Stages

Table 4a provides the most granular view of how each training stage contributes. Starting from the zero-shot CoT baseline with search (which achieves 0.0 temporal F1 and 44.2% completeness — the model cannot use search at all), SFT provides the largest single gain: temporal F1 jumps from 0.0 to 7.8, completeness from 44.2% to 60.5%, and VideoMME accuracy from 51.8% to 59.2%. This confirms that the format and mechanics of interleaved search must be explicitly taught; the base model cannot discover them from a prompt alone.

RL with standard GRPO (before collapse, presumably at an early checkpoint) shows a mixed pattern: temporal F1 drops from 7.8 to 7.4 (precision declines by 2.2 while recall improves by 7.2), completeness drops from 60.5% to 57.2%, and consistency is essentially flat (69.2% → 69.3%). However, VideoMME accuracy improves from 59.2% to 65.1% — a gain of 5.9 percentage points. This reveals a tension: standard GRPO improves final answer accuracy but degrades search quality (lower F1, lower completeness), consistent with the failure mode analysis where the model learns to answer correctly with less thorough search.

GRPO-CSV without accuracy reward (i.e., only completeness + format rewards) pushes completeness to its highest value (61.2%) and consistency to 75.3% (+6.1 over SFT alone, +6.0 over standard GRPO). Temporal F1 improves to 8.2. However, VideoMME accuracy dips to 64.8% — slightly below standard GRPO — suggesting that optimizing purely for search quality and consistency can come at the cost of final answer correctness, perhaps because the model becomes overly conservative in committing to answers.

GRPO-CSV with accuracy reward (the full method) achieves the best VideoMME accuracy (66.6%, +7.4 over SFT) while maintaining improved search quality (temporal F1 of 8.1, completeness of 60.2%) and consistency (71.8%, +2.6 over SFT). This configuration balances the three reward components and represents the Pareto-optimal point among the ablations for final task performance.

Efficiency Analysis

Table 4 (Appendix E) reports latency on Haystack-Ego4D. TimeSearch-R achieves an end-to-end latency of 13.4 seconds, compared to 34.9 seconds for VideoAgent (a 61.6% reduction), 32.2 seconds for the retrieval-based baseline, and 11.1 seconds for T*. T*'s slightly faster runtime (11.1 vs. 13.4 seconds) is attributed to its use of the lightweight YOLO-World-110M detector rather than SigLIP-400M for frame retrieval. The paper argues that TimeSearch-R's 13.4-second latency is "comparable" to T*'s while substantially outperforming it in temporal search metrics and QA accuracy, though the latency gap of 2.3 seconds (approximately 21% slower than T*) is not negligible for real-time applications. The latency measurement appears to include both search and answering time (the table header says "overall latency of temporal search and answering") but does not break down the proportion spent on frame retrieval vs. model inference. Evaluations were conducted on A100 GPUs, making the absolute latency numbers hardware-dependent.

Ablation Studies and Robustness Checks

GRPO without CSV causes training collapse (Figure 4b): When CSV is removed from the GRPO objective, the number of search turns issued by the model gradually declines across training and collapses to zero around step 300. This is the paper's primary evidence that outcome-only rewards cannot sustain learned search behavior. The collapse is not instantaneous but progressive — the model initially searches (2–4 turns at step 0), then progressively reduces search frequency over hundreds of steps, consistent with the gradual erosion of search effort as the policy discovers it can maintain accuracy without thorough exploration.

CSV improves search completeness and consistency but requires accuracy reward for best QA (Table 4a): The GRPO-CSV configuration without accuracy reward achieves the highest completeness (61.2%) and consistency (75.3%) but lower QA accuracy (64.8% vs. 65.1% for standard GRPO at the pre-collapse checkpoint). Adding accuracy reward to GRPO-CSV improves QA to 66.6% while maintaining completeness (60.2%) and consistency (71.8%) above the SFT baseline. This demonstrates that the completeness and accuracy signals are partially complementary — completeness alone can over-optimize for thorough search at the expense of efficient answering. A subtle finding is that the standard GRPO checkpoint (before collapse) achieves 65.1% accuracy — nearly matching GRPO-CSV without accuracy reward — despite having lower completeness (57.2%) and consistency (69.3%). This suggests that on some questions, less thorough search with higher-risk answering can outperform conservative, thorough search.

Data filtering is essential for RL training (Table 3): Training GRPO-CSV only on data that has undergone the two-stage filtering pipeline shows marked differences from unfiltered data. When the data is not filtered, RL training produces substantially worse performance than the original Qwen2.5-VL base model — the "Ego" only row (filtered but egocentric-only) achieves 62.8% overall on VideoMME vs. 65.1% for the base model, and the paper notes in text that training on unfiltered data leads to "a substantial performance drop compared to the original Qwen2.5-VL." The mechanism is explained as linguistic bias inducing zero advantage in GRPO: "when questions can be trivially answered through linguistic shortcuts, all rollouts achieve perfect accuracy and completeness, yielding no learning signal and severely hindering RL efficiency." This is a critical finding that validates the filtering pipeline's necessity — RL is not merely less efficient on unfiltered data, it is actively harmful, degrading performance below the pre-trained baseline.

Data diversity matters — exocentric data improves generalization (Table 3): Training on only egocentric data (Ego4D from Haystack-Ego4D, row "Ego") achieves 62.8% overall on VideoMME, below the base model's 65.1%. Training on egocentric + exocentric data (adding Panda-70M and CinePile, row "Ego + Exo") improves to 65.3%, essentially matching the base model. Adding the filtering pipeline (+Filter) enables further improvement to 66.6%. A notable finding is that RL training on the diverse, filtered dataset improves performance not just on general reasoning (+1.5% overall) but also on specific reasoning subcategories: temporal reasoning improves by 7.4% (51.4% → 58.8%) and action reasoning by 5.7% (56.8% → 62.5%). These subcategory improvements occur despite the training data containing only general long-video QA tasks without explicit temporal or action reasoning labels — the paper interprets this as evidence that "TimeSearch-R learns fundamental cognitive patterns through end-to-end policy optimization."

Zero-shot search degrades performance (Table 2, "Qwen2.5VL-7B + Search" vs. base model): Adding search capability to the base model through zero-shot prompting without SFT or RL causes VideoMME accuracy to drop from 65.1% to 51.8% (a 13.3% absolute decline). This confirms that the ability to search is not sufficient for improved performance; the model must learn how to formulate effective search queries, interpret retrieved frames, and integrate visual evidence into reasoning. The SFT stage recovers 59.2% accuracy, and RL further improves to 66.6%, demonstrating that both stages are necessary.

No ablation on CSV conditional gating design: The paper does not ablate the indicator function 1[Acc(A, A*) > 0.5] in the completeness reward (Equation 4). A natural ablation would be to remove this gating, rewarding completeness regardless of whether the original answer was correct. This would test whether the gating is necessary to prevent the hypothesized perverse incentive (model searches thoroughly but answers wrong, still receiving completeness reward), or whether the completeness signal alone is sufficient to drive good search behavior without conditioning on answer correctness.

No ablation on the number of search turns or frames per search: The paper fixes the maximum at 8 turns × 8 frames = 64 dynamically retrievable frames (Table 5) but does not vary these limits to test whether more turns with fewer frames per turn, or fewer turns with more frames per turn, would change search policy learning.

No ablation on DPP vs. simpler retrieval strategies: The search function uses DPP-based frame selection, but the paper does not compare against simpler alternatives (e.g., top-K by relevance score only, random sampling within the temporal window). It is possible that the learned search policy would be equally effective with a simpler retrieval backend, or that the DPP diversity mechanism is essential for maximizing information per frame.

No multi-seed training runs: All ablation results in Table 4a and Figure 4 appear to report single runs, including the training dynamics collapse in Figure 4(b). Without multiple seeds, it is unclear whether the collapse at step 300 is a reliable effect of removing CSV or an artifact of a particular random initialization and data ordering.

Critical Assessment

The experiments demonstrate that TimeSearch-R, combining end-to-end RL with CSV, improves over hand-crafted search agents and text-only video reasoning on both temporal search and video understanding benchmarks. However, several claims require qualification when mapped against the evidence:

On the claim that learned search outperforms hand-crafted search: The evidence is strong for Haystack-LVBench temporal F1 (8.1 vs. 2.5 for T*) and for VideoMME accuracy (66.6% vs. 64.1% for T*, 56.0% for VideoAgent). However, the comparison is confounded by base model differences — VideoAgent and T* use GPT-4/GPT-4o as their reasoning engines, while TimeSearch-R uses Qwen2.5-VL-7B. The paper does not include an experiment where the hand-crafted search pipeline is applied to Qwen2.5-VL-7B to isolate search strategy from base model capability. It is possible that some of the gain attributed to learned search strategy is actually due to different base model characteristics (e.g., Qwen2.5-VL may be stronger at certain visual reasoning tasks than GPT-4o, or vice versa). The latency results in Table 4 partially address this by using different retrieval backends for each method, but the reasoning model itself varies.

On the claim that CSV addresses insufficient temporal exploration and inconsistent reasoning: The ablation in Table 4a provides supporting evidence: adding CSV increases completeness from 57.2% to 61.2% (insufficient exploration) and consistency from 69.3% to 75.3% (inconsistent reasoning) compared to standard GRPO. However, these gains are modest — completeness improves by only 4 percentage points, and consistency by 6 percentage points. Moreover, the standard GRPO comparison point in Table 4a is labeled "Before Collapse," meaning it represents an early checkpoint before the training degenerated. The paper does not report what maximum completeness and consistency standard GRPO ever achieves (the "Before Collapse" numbers of 57.2% completeness and 69.3% consistency might not be the peak values, only the values at the point training was halted pre-collapse). A fairer comparison would report the best completeness/consistency achieved by standard GRPO at any point during training.

On the claim that TimeSearch-R establishes new SOTA on LongVideoBench: The 60.1% accuracy improves over Qwen2.5-VL (56.0%, +4.1%) and Video-R1-7B (58.1%, +2.0%). This is a legitimate SOTA claim backed by the numbers in Table 2. However, context is important: the Video-R1 comparison uses only 768 frames for Video-R1 (matching TimeSearch-R's frame budget), but Video-R1's own paper may report results at different frame budgets or with different base models. The 2.0% margin is meaningful but not overwhelming — it suggests text-video interleaved thinking provides a real but modest advantage over text-only reasoning on this benchmark. The paper does not report whether the improvement is concentrated in specific question types (temporal ordering, action recognition, etc.) or is uniform across the benchmark.

On the generalizability evidence: All experiments use Qwen2.5-VL-7B as the base model. The paper claims the model is "representative" but provides no evidence with other model families (LLaVA, InternVL, etc.) or scales (7B vs. 13B vs. 72B). The findings — particularly the CSV mechanism's effectiveness and the training collapse without it — may be specific to Qwen2.5-VL's architecture, pre-training data, or instruction tuning. Additionally, the data filtering pipeline is calibrated to Qwen2.5-VL's capabilities (Stage 1 uses Qwen2.5-VL to identify trivially answerable questions). The thresholds (4 frames for Stage 1, 64 frames for Stage 2) are not explored or justified; different models or benchmarks might require different thresholds.

On the training dynamics collapse claim: Figure 4(b) is compelling but is presented as three curves without error bars or multiple seeds. The collapse of the "GRPO" (no CSV) curve to zero search turns at step 300 is the paper's central evidence that CSV is necessary for stable training. Without replication across seeds, it is unclear whether this collapse is deterministic (happens every time) or stochastic (happened in this particular run). If the collapse is stochastic, it might be preventable with different hyperparameters (learning rate, KL coefficient, group size) rather than requiring CSV.

Missing experiment: scaling with video duration: The paper claims temporal search "becomes more valuable when the video length increases" based on the VideoMME duration breakdown (short: +0.5%, medium: +1.1%, long: +1.4%). However, VideoMME's "long" videos are only up to 60 minutes. The training dataset mean duration is 1,659 seconds (~27.7 minutes) with a tail extending past 10,000 seconds (~2.8 hours). The paper does not evaluate on hour-scale videos separately to test whether the gains continue to scale with extreme durations or plateau.

Missing evaluation: search precision on Haystack-Ego4D: Table 1 reports only QA accuracy on Haystack-Ego4D, not temporal F1 or visual F1. This is a significant omission because Ego4D's egocentric videos (with frequent viewpoint changes, rapid camera motion, and different visual statistics from LVBench) are precisely the domain where temporal search quality differences should manifest. Without temporal F1 on Ego4D, the claim that TimeSearch-R learns better search strategies is only partially validated.

On the efficiency claims: The 13.4-second latency on Haystack-Ego4D (Table 4) is compared against baselines using different hardware (VideoAgent uses GPT-4 API with unknown GPU configuration; T* uses LLaVA-OV-7B as the VLM). The latency numbers are therefore not strictly comparable — API latency includes network overhead, while TimeSearch-R's latency is measured locally on A100 GPUs. A fairer comparison would run all methods in the same hardware environment. Additionally, the paper does not report throughput (videos processed per GPU-hour), which matters for batch processing scenarios where latency is less important than total computational cost.

On the case study claims of emergent strategies: The qualitative case studies in Figures 5, 13, 14, 15 are illustrative but not systematically validated. The paper does not report how frequently each search pattern occurs, whether the patterns correlate with question types in a statistically significant way, or whether the patterns were present in the SFT model or only emerged during RL. A quantitative analysis of search pattern frequency by question category (temporal, spatial, action, object from Table 3) would substantially strengthen the claim that the model learns adaptive strategies.

6. Limitations and Trade-offs

Difficulty Estimation and Data Filtering Cost Is Not Accounted for in Headline Results

The assumption or constraint. The entire RL training pipeline depends on the two-stage data filtering procedure (Section 2.3, Appendix B.1) to remove samples that are trivially solvable from 4 uniformly sampled frames or unsolvable even with extensive search. This filtering requires running inference with Qwen2.5-VL (Stage 1), GPT-4o (Stage 2 for SFT data), and an early version of TimeSearch-R (Stage 2 for RL data) on every candidate training sample. For VideoMarathon's automatically generated QA pairs, additional manual human annotation is required to verify ground-truth labels and filter out unanswerable questions. The computational and human cost of this filtering pipeline is not included in any efficiency calculation — the paper's reported latency (13.4 seconds per query, Table 4) and training budget (32 A100 GPUs, Section F) cover only the RL training and inference, not the prerequisite data curation.

The consequence. The difficulty estimation problem here is analogous to the difficulty estimation cost flagged in the reference paper (Section 3.2 of that work): generating 2048 samples per question to estimate pass@1 difficulty. In TimeSearch-R, the filtering pipeline must process every candidate training sample through multiple LVLMs — for Stage 2 alone, GPT-4o API calls are needed for the SFT training data, and an early TimeSearch-R model must be trained and run for the RL training data. The cost of obtaining an "early version of TimeSearch" for this filtering is itself a chicken-and-egg problem: you need a functional temporal search model to filter data for training a temporal search model. The paper does not specify how this early model was obtained (presumably through a separate training run on unfiltered or partially filtered data, which Table 3 suggests degrades performance below the base model). In a deployment scenario where a practitioner wants to apply TimeSearch-R to a new video domain (e.g., surveillance, medical imaging, sports analytics), they would need to repeat the entire filtering pipeline — including manual annotation for any automatically generated QA pairs — before RL training can begin. This makes the approach substantially less "end-to-end" than the narrative suggests: the search policy is learned end-to-end, but the data that enables that learning requires a complex, multi-stage curation process with nontrivial human effort.

What evidence exists in the paper. Table 3 quantifies the necessity of filtering: training on unfiltered data produces performance below the base model ("a substantial performance drop compared to the original Qwen2.5-VL," Section 3.3). The filtering pipeline is described in Appendix B.1 with the specific LVLMs used (GPT-4o for SFT, early TimeSearch for RL), and the manual annotation protocol for VideoMarathon data is specified. However, the total compute cost of filtering (GPU-hours for inference, number of GPT-4o API calls, human annotator hours) is not reported anywhere. The paper acknowledges the filtering cost only implicitly by describing the pipeline stages, never flagging the cost as a limitation or including it in any budget calculation.

Mitigation status. Not addressed. The paper treats the filtering pipeline as an enabling infrastructure step and does not discuss amortizing its cost, reducing its computational requirements, or developing cheaper alternatives. No ablation explores whether cheaper filtering strategies (e.g., fewer frames in Stage 1, a smaller model for Stage 2, or skipping the manual annotation step) would be sufficient. A natural mitigation — training a lightweight difficulty classifier that predicts filtering decisions from question text and video metadata alone, analogous to the difficulty prediction model suggested in the reference paper — is not discussed.

Hard Problems: The Method Fails on Questions Requiring Capabilities Beyond the Base Model's Reach

The assumption or constraint. TimeSearch-R's temporal search operates entirely within the visual recognition and reasoning capabilities of the base model (Qwen2.5-VL-7B). The search function (DPP-based frame retrieval) can find diverse, query-relevant frames, but the model's ability to interpret those frames — to recognize objects, understand actions, or perform temporal reasoning — is bounded by what Qwen2.5-VL-7B can do. As the reference paper found for test-time compute in math reasoning (Section 7), "test-time compute can amplify existing capability but does not create it from nothing." The same principle applies here: temporal search can help the model find relevant evidence more efficiently, but it cannot enable the model to understand visual content that is beyond its pretraining capabilities.

The paper's data filtering pipeline explicitly acknowledges this boundary at the upper end: Stage 2 removes samples that "remain unsolvable even with multiple temporal searches and numerous video frames" (Section 2.3). This means the training data is curated to exclude precisely the hardest questions — the RL training never sees samples where the model fails despite good search, and the evaluation benchmarks may contain such questions that the model cannot address. The VideoMME difficulty breakdown is coarse (short/medium/long by duration, not by visual complexity), and the paper does not report per-question accuracy binned by estimated difficulty in the way the reference paper's Figure 3 (right) breaks down search performance by quintile.

The consequence. On questions where the base model cannot reliably interpret the relevant visual content even when the frames are perfectly selected — for instance, questions requiring fine-grained object counting in cluttered scenes, reading small text in video frames, or understanding subtle social interactions — TimeSearch-R provides no advantage over the base model or may even perform worse (due to the risk of searching inadequately and then guessing, as in the failure case of Figure 16 where the model terminates search after reviewing only 2 of 4 options). The failure case in Figure 17 shows visual hallucination: the model claims the protagonist "is seen riding his bike" when the search results contain no such evidence, suggesting the model's visual interpretation is unreliable even when search is functioning. The paper does not characterize what fraction of benchmark questions fall into this "fundamentally beyond capability" category, making it difficult for a practitioner to estimate what accuracy ceiling the approach can reach.

What evidence exists in the paper. Indirect evidence: the Stage 2 filtering explicitly removes unsolvable samples, confirming they exist in the training data. The failure cases in Figures 16 and 17 demonstrate specific regimes where the model fails. The VideoMME results (Table 2) show that TimeSearch-R achieves 56.0% on long videos — a 1.4% improvement over the base model, but still substantially below the 76.8% on short videos, suggesting that long videos contain more "hard" questions where temporal search provides limited benefit. However, the paper provides no difficulty-binned analysis comparable to the reference paper's Figure 3 (right) or Figure 7 (right) — there is no quantification of accuracy stratified by question difficulty, visual complexity, or temporal reasoning demand. Without such analysis, the claim that TimeSearch-R "learns fundamental cognitive patterns" (Section 3.3) cannot be separated from the possibility that gains are concentrated on questions that were already within the base model's capability range and merely required better frame selection.

Mitigation status. Not addressed. The paper acknowledges the existence of unsolvable samples (by filtering them from training data) but does not characterize the difficulty profile of evaluation benchmarks or test whether gains are uniform across difficulty levels. A natural mitigation — scaling to larger base models (Qwen2.5-VL-72B, for instance) or incorporating stronger visual encoders — is not explored. The paper frames the method as a general approach but only validates it at the 7B scale.

Single Model Family, Single Scale: No Evidence of Generalization Across Architectures or Model Sizes

The assumption or constraint. All experiments use Qwen2.5-VL-7B-Instruct as the base model. The paper states in Section 1 that they "believe this model is representative of the capabilities of many contemporary LLMs," but provides no experimental evidence with other model families (LLaVA, InternVL2, VideoLLaMA, LLaMA-VID, etc.) or different scales within the same family (e.g., Qwen2.5-VL-3B or -72B). The data filtering pipeline is calibrated specifically to Qwen2.5-VL-7B — Stage 1 uses this exact model to identify trivially answerable questions, and Stage 2 uses GPT-4o (for SFT data) and an early TimeSearch-R (which is itself based on Qwen2.5-VL-7B). The model's specific architecture (vision encoder, projection layer, language model backbone), pretraining data mixture, and instruction tuning recipe all affect how it interacts with the search environment and responds to RL training.

The consequence. Several findings may not transfer to other model families. The GRPO training collapse when CSV is removed (Figure 4b, collapsing at step ~300) could be specific to Qwen2.5-VL-7B's propensity to exploit linguistic shortcuts or its sensitivity to the KL penalty coefficient (β = 0.005). A model with stronger visual grounding (less reliant on language bias) might be less susceptible to the insufficient temporal exploration failure mode, potentially making CSV less necessary. Conversely, a model with weaker instruction-following might require more extensive SFT to learn the search format, or might never converge to a functional search policy at all. The frame retrieval pipeline uses SigLIP-400M embeddings for DPP-based selection — these embeddings are well-aligned with Qwen2.5-VL's training distribution (both use contrastively trained vision-language representations), but might be misaligned with models using different vision encoders (e.g., InternVideo, VideoMAE-based encoders).

The benchmark comparisons in Table 1 and Table 2 are confounded by base model differences: VideoAgent and T* use GPT-4/GPT-4o as reasoning engines, while TimeSearch-R uses Qwen2.5-VL-7B. When TimeSearch-R outperforms these baselines, it is unclear whether the gain comes from the learned search strategy or from Qwen2.5-VL being a stronger visual reasoner than GPT-4o for certain tasks (or vice versa — GPT-4o might be weaker on some visual tasks but the search strategy compensates). The paper does not include a control experiment where the hand-crafted search pipeline (e.g., T*'s object-oriented search) is applied to Qwen2.5-VL-7B to isolate the effect of the search strategy from the base model capability.

What evidence exists in the paper. All tables and figures use Qwen2.5-VL-7B. The comparison against Video-R1-7B (Table 2) is helpful — it shows TimeSearch-R outperforming another Qwen2.5-VL-7B-based approach — but Video-R1 uses text-only reasoning, making it a comparison of reasoning paradigms rather than base models. The paper provides no cross-model-family experiments and does not discuss model-specific assumptions in the limitations section.

Mitigation status. Not addressed. The paper's title and abstract present TimeSearch-R as a general framework, but the experimental validation is limited to one model at one scale. Applying the method to other model families would require re-running the data filtering pipeline with the target model (or accepting that the filtering thresholds optimized for Qwen2.5-VL-7B may not be appropriate), re-running SFT, and re-tuning RL hyperparameters — a significant barrier to adoption. The paper does not suggest this as future work or discuss the expected transferability.

CSV Does Not Prevent Search Collapse in All Regimes, and the Gating Design Is Unexplored

The assumption or constraint. The GRPO-CSV algorithm is designed to prevent the training collapse where the model gradually stops searching (Figure 4b). However, the paper's own results show that CSV does not fully solve the problem in all configurations. Table 4a shows that GRPO-CSV without accuracy reward — the configuration that maximizes completeness reward — achieves the highest completeness (61.2%) and consistency (75.3%) but lower QA accuracy (64.8%) than standard GRPO before collapse (65.1%). This suggests that optimizing purely for search quality can come at the cost of answer quality, creating a different failure mode: the model searches thoroughly but reasons less effectively, perhaps because it becomes overly conservative or because the CSV re-answer task interferes with the primary answer-generation capability.

The completeness reward's conditional gating design — R_c = 1[Acc(A, A*) > 0.5] · Acc(A_c, A*) — gates the completeness reward on the original answer being correct. This design choice has specific implications: if the original answer is wrong, the completeness reward is zero regardless of whether the search was thorough. This means that good search on a wrong-answer trajectory receives no reinforcement, which could slow down learning of search skills in the early stages of RL when the model's answer accuracy is low. Conversely, if the gating were removed, the model might learn to search thoroughly but guess randomly, receiving completeness reward for search quality without pressure to answer correctly. The paper does not ablate this gating design — there is no experiment comparing gated vs. ungated completeness reward, so the tradeoff between search learning speed and answer quality remains uncharacterized.

The consequence. A practitioner implementing GRPO-CSV faces an unguided hyperparameter choice: should the accuracy reward component be included, and if so, at what relative weight? The paper shows that including accuracy reward yields the best QA performance (66.6% vs. 64.8% without it), but does not explore whether a weighted combination (e.g., R = α·R_c + β·R_fmt + γ·R_acc with tuned coefficients) could achieve a better tradeoff. The gating design (1[Acc(A, A*) > 0.5]) represents a specific choice about how to balance search quality against answer quality during RL — the model only learns from search quality on trajectories that already produce correct answers, which could create a "rich get richer" dynamic where the policy only improves search on questions it can already answer, never learning to search well on questions just beyond its current capability. Additionally, the failure case in Figure 16 — where the model terminates search after examining only 2 of 4 options — shows that CSV does not prevent premature search termination even when the full method (with accuracy reward) is used. The completeness reward penalizes insufficient search after the fact (when CSV re-answer fails), but this penalty may come too late in the training process to correct the underlying "early stopping" behavior.

What evidence exists in the paper. Figure 4(b) shows that GRPO-CSV (both with and without accuracy reward) maintains non-zero search turns throughout training, while GRPO without CSV collapses to zero at step ~300. However, Figure 4(b) does not show the quality of those search turns — only their quantity. Table 4a shows that GRPO-CSV without accuracy reward achieves the highest search quality metrics (F1: 8.2, completeness: 61.2%, consistency: 75.3%) but lower QA accuracy (64.8%). The gating design is not ablated. The premature termination failure (Figure 16) is shown as a case study but not quantified across the test set.

Mitigation status. Partially addressed through the accuracy reward component, which the paper includes in the final configuration. However, the fundamental tradeoff — that optimizing completeness alone can reduce QA accuracy — is identified but not resolved. The paper does not propose mechanisms for balancing search depth against answering confidence dynamically (e.g., allowing the model to decide per-question how thorough to be), which would be a natural extension. Future work on dynamic difficulty estimation and adaptive search budgets is suggested only implicitly through the general framing, not as a specific extension to CSV.

Latency and Serial Dependency: The Interleaved Search Paradigm Is Inherently Slower Than Parallel Alternatives

The assumption or constraint. TimeSearch-R's interleaved text-video thinking is fundamentally serial: each reasoning step conditions on the results of the previous search, meaning the model must wait for frame retrieval to complete before generating the next reasoning token. The search function (DPP-based frame selection with SigLIP embeddings) adds 13.4 seconds of end-to-end latency on Haystack-Ego4D (Table 4) — this includes both search time and answer generation time, but the serial dependency means the model's reasoning is gated on the slowest search operation. If the model performs multiple searches (Figure 15 shows a case with 5 sequential searches), each search adds latency, and the total wall-clock time scales with the number of search turns. In contrast, static frame sampling methods (Qwen2.5-VL, GPT-4o, Video-R1) process all frames in a single forward pass — their latency is dominated by generation length, not by iterative environment interactions.

The paper does not report how latency scales with the number of search turns, nor does it break down the 13.4 seconds into search time vs. inference time. The SigLIP-400M embedding extraction (for DPP similarity computation) and the DPP greedy MAP inference (Equation 7, iteratively building the subset of F frames) add computational overhead on top of the base model's own inference cost. The paper compares latency against VideoAgent (34.9 seconds) and T* (11.1 seconds) in Table 4, but these comparisons use different hardware (VideoAgent uses GPT-4 API with unknown inference hardware; T* uses LLaVA-OV-7B as the VLM with YOLO-World-110M as the detector) — the latency numbers are not measured in a controlled environment.

The consequence. For latency-sensitive applications — interactive video assistants, real-time video monitoring, live-stream question answering — the 13.4-second latency (and potentially longer for multi-turn searches) may be prohibitive, regardless of accuracy gains. Even in batch processing scenarios, the serial dependency means TimeSearch-R cannot be parallelized across search turns: you cannot run search turn 2 until turn 1 completes, limiting throughput compared to batched static sampling where all videos can be processed independently and simultaneously. The paper frames the efficiency comparison against T* as favorable (13.4 vs. 11.1 seconds, "comparable runtime" per Appendix E), but the 21% latency increase over the fastest baseline is non-trivial for deployment. Moreover, T* achieves its 11.1-second latency using a lightweight detector (YOLO-World-110M) with a different VLM (LLaVA-OV-7B); a fair latency comparison would require running all methods on identical hardware with identical base models, which the paper does not do.

The efficiency analysis in Appendix E claims a "61.6% speed-up over the 34.9-second latency of VideoAgent," but this comparison is misleading because VideoAgent uses the GPT-4 API with network latency and unknown server-side batching, while TimeSearch-R uses local A100 inference. The speed-up may primarily reflect local vs. API inference rather than algorithmic efficiency.

What evidence exists in the paper. Table 4 reports end-to-end latency numbers with hardware specified for some baselines ("A100 GPUs" for TimeSearch-R, but unspecified for VideoAgent and Retrieval-based baselines). The paper does not report throughput (videos per GPU-hour) or latency breakdown by search turn. The serial interleaving nature is inherent in the framework design (Section 2.1) and is not discussed as a limitation.

Mitigation status. Not addressed. The paper does not discuss latency-accuracy tradeoffs, propose mechanisms for speculatively pre-fetching frames for likely future searches, or explore whether some search turns could be parallelized (e.g., searching for multiple hypotheses simultaneously). The maximum search turns (8) and frames per search (8) are fixed during training; whether a smaller budget (e.g., 4 turns × 4 frames) could achieve most of the accuracy gain with lower latency is not explored.

Frame Selection Quality Depends on the Alignment Between the Retrieval Embedding Space and the Base Model's Visual Understanding

The assumption or constraint. The search function (Appendix A) uses SigLIP-400M embeddings for computing query-frame relevance (via cosine similarity) and frame-frame similarity (for DPP diversity). These embeddings define what the search environment considers "relevant" and "diverse." However, SigLIP is a general-purpose vision-language model trained on image-text pairs; its notion of relevance may not align with what Qwen2.5-VL-7B actually needs to answer a specific question. A frame that SigLIP scores as highly relevant to the query might lack the specific visual detail (a particular object attribute, a subtle action, text in the frame) that the reasoning model requires. Conversely, a frame that is crucial for answering might receive a low relevance score because the model's textual query poorly describes the needed visual content. The paper does not investigate the alignment between SigLIP's relevance scores and the downstream utility of frames for question answering.

The consequence. This misalignment creates a bottleneck that RL cannot optimize through: the search function (DPP + SigLIP) is non-differentiable and frozen during training. The policy model learns to generate better temporal windows and textual queries, but it cannot improve the retrieval mechanism itself. If the retrieval function consistently fails to surface the right frames for certain types of queries — e.g., queries requiring fine-grained textual recognition (reading signs, labels, subtitles) that SigLIP's visual encoder handles poorly — the model will receive poor visual evidence regardless of how well it searches, creating a performance ceiling that no amount of RL can surmount. The failure case in Figure 17 (visual hallucination about "riding a bike") could stem from either the model misinterpreting correctly retrieved frames or the retrieval function failing to find frames showing bike-riding despite a well-formulated query. The paper cannot distinguish these failure modes because the retrieval-relevance bridge is unanalyzed.

The paper also does not ablate the DPP diversity mechanism against simpler alternatives (e.g., top-K by relevance only, random sampling within the temporal window, or uniform sampling at higher density). It is possible that the diversity-aware sampling is essential for maximizing information per frame (since highly relevant but redundant frames would waste the budget), but it is also possible that the model's search queries already provide sufficient specificity that simple top-K relevance ranking would work equally well. Without this ablation, the contribution of DPP to the overall system performance is unquantified.

What evidence exists in the paper. The DPP formulation is described in detail (Appendix A, Equations 6 and 7), and SigLIP-400M is named as the embedding model (Table 4, Appendix E). However, the paper provides no ablation comparing DPP against simpler retrieval methods, no analysis of retrieval recall (what fraction of ground-truth relevant frames are actually retrieved by the search function), and no investigation of whether certain query types systematically fail to retrieve useful frames. The temporal F1 and visual F1 metrics on Haystack-LVBench (Table 1) evaluate the final searched frame set against ground-truth keyframes, which implicitly measures retrieval quality, but these metrics conflate the policy model's search decisions (which windows and queries to use) with the search function's retrieval quality — a model could have poor F1 because it searched the wrong windows (policy error) or because it searched the right windows but SigLIP failed to identify the relevant frames (retrieval error).

Mitigation status. Not addressed. The paper treats the search function as a fixed environment and does not discuss the impact of embedding model choice, the potential for fine-tuning SigLIP on in-domain video data, or the possibility of making the retrieval function differentiable (e.g., through soft selection or learned scoring) to allow end-to-end optimization of the entire perception-search-reasoning pipeline. The choice of SigLIP-400M appears arbitrary — no justification is given for why this particular embedding model was selected over alternatives (CLIP, EVA-CLIP, DFN, etc.).

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a new paradigm for long-form video understanding where temporal search is not a pre-processing step or a separate agent module, but an integral, learned component of the reasoning process itself. Prior to this work, the field was split between two suboptimal approaches: static frame sampling (where models reason from a fixed, pre-selected set of frames regardless of what the reasoning process requires) and hand-crafted search agents (where human engineers design search heuristics through prompt engineering). TimeSearch-R collapses this distinction by making search decisions conditioned on intermediate reasoning states and optimizing the entire search policy end-to-end through reinforcement learning. The model does not merely search better — it learns when to search, where to search, and what to search for, discovering strategies (hypothesis-driven search, elimination, sequential exploration) that no human engineer explicitly programmed.

The magnitude of this shift is best understood as a conceptual reframing rather than a single algorithmic breakthrough. The paper does not introduce a fundamentally new RL algorithm — GRPO-CSV is a modification of existing GRPO with an auxiliary self-verification task. Nor does it introduce a new model architecture — Qwen2.5-VL-7B is an off-the-shelf foundation model. What it introduces is the idea that video understanding is active evidence gathering, not passive pattern recognition from fixed context. This reframing has implications that extend beyond the specific method: it suggests that future video models should be designed from the ground up to support interactive perception-reasoning loops, rather than being optimized for single-pass inference over pre-sampled frames.

The paper also reconciles a tension that has been emerging in the video reasoning literature. On one side, models like Video-R1 showed that RL with outcome rewards can improve video reasoning capabilities — but only in the text modality, without visual interaction. On the other side, interactive agents like VideoAgent and T* showed that multi-turn search can improve performance — but only through hand-crafted workflows that are brittle and suboptimal. TimeSearch-R demonstrates that these two lines of work are not alternatives but complementary halves of a unified solution: the interactive paradigm provides the architecture for dynamic visual access, while RL provides the optimization mechanism to make that access strategic rather than heuristic. The paper's evidence that RL without intermediate supervision causes search to collapse (Figure 4b) explains why prior work kept search and reasoning separate — without something like CSV, combining them is actively harmful. The CSV mechanism provides the missing piece that makes the unification viable.

Research directions that become more attractive:

  • Designing video-language models with native support for interactive perception. Current VLMs are optimized for single-pass inference over a fixed context window. Training them to interact with video environments — issuing queries, receiving frames, and integrating new evidence into ongoing reasoning — requires awkward prompting (the tool_call format, the 8-turn budget, the timestamp prefix encoding). A model architecture with first-class support for interleaved perception-reasoning — perhaps through dedicated memory banks for retrieved visual evidence, or through architectural mechanisms for updating beliefs in light of new perceptual input — could substantially improve the efficiency and naturalness of interactive video understanding.

  • Exploring self-verification as a general mechanism for intermediate supervision in interactive RL. The CSV mechanism provides annotation-free supervision over intermediate search decisions by evaluating sufficiency — can the model answer from what it gathered? This principle generalizes beyond video: in document QA, the model could verify whether its retrieved passages contain the answer. In web navigation, it could verify whether the pages it visited provide enough information to complete the task. In multi-step tool use, it could verify whether the tool outputs collected so far constrain the solution space. CSV suggests a general template for RL with interactive environments: after the agent completes its episode, re-run the task from scratch using only the agent's collected evidence, and reward the agent for evidence sufficiency. This template is annotation-free, domain-agnostic, and addresses the fundamental credit assignment problem that causes evidence-gathering to atrophy under outcome-only rewards.

Research directions that become less attractive:

  • Further investment in hand-crafted search heuristics for video agents. The paper's results quantify the gap between hand-crafted and learned search strategies: TimeSearch-R achieves temporal F1 of 8.1 vs. 2.5 for T* (the previous SOTA hand-crafted agent) on Haystack-LVBench, and 66.6% vs. 64.1% on VideoMME — with a frame budget of 8.8 vs. 32 for T*. The 3× improvement in temporal localization with 4× fewer frames suggests that human-designed search heuristics have approached their ceiling. Further effort spent on refining prompt templates, designing new tool-calling sequences, or introducing additional hand-crafted search modules is unlikely to close this gap.

  • Text-only video reasoning as a standalone approach for long-form video tasks. Video-R1 represents the state of the art in applying RL to video reasoning without visual interaction — it achieves 65.7% on VideoMME with 768 frames compared to TimeSearch-R's 66.6%. The consistent (~1–2 percentage point) gap across multiple benchmarks (VideoMME, MLVU, LongVideoBench) and the larger gap on explicitly temporal search tasks (52.1% vs. 51.9% on Haystack-LVBench at similar frame budgets) suggests that text-only reasoning over fixed frames has a fundamental ceiling: it cannot request additional evidence when the initial frames are insufficient. The paper's demonstration that simply adding search capability to the base model without training degrades performance (51.8% on VideoMME, a 13.3% drop from the 65.1% baseline) further confirms that making search work requires the full RL pipeline — it is not a simple upgrade to existing text-only reasoning approaches.

Follow-Up Research This Work Enables

1. Scaling TimeSearch-R to larger base models and characterizing the capability ceiling.

The paper validates TimeSearch-R only on Qwen2.5-VL-7B. A natural follow-up would apply the same pipeline (data filtering → SFT → GRPO-CSV) to Qwen2.5-VL-72B, testing whether the gains from learned search are additive with base model scale. If the larger model achieves proportionally similar gains (+1.5% on VideoMME, +4.1% on LongVideoBench), this would validate that temporal search provides independent value beyond base model capability. However, there are reasons to expect the gains might shrink: a stronger base model may have fewer "trivially answerable" questions in the training data (reducing the need for filtering), or may already implicitly attend to relevant temporal regions during single-pass inference (reducing the marginal benefit of explicit search). Conversely, a stronger model might formulate better search queries and interpret retrieved frames more accurately, amplifying the benefits of temporal search. Characterizing this scaling relationship would determine whether TimeSearch-R is most valuable for smaller models (as a way to compensate for limited capacity) or for larger models (as a way to unlock capabilities that even scale alone cannot provide). A strong follow-up would evaluate at 3–4 model scales (e.g., 3B, 7B, 13B, 72B) with the same training pipeline, reporting both absolute accuracy and the relative improvement over the static-sampling baseline at each scale.

2. Ablating the retrieval backend: Is DPP necessary, or would simpler mechanisms suffice?

The search function uses DPP-based frame selection with SigLIP-400M embeddings (Appendix A). Two ablated alternatives would substantially clarify where the system's performance comes from. First, replace DPP with top-K relevance-only selection (pick the K frames with the highest cosine similarity to the query, ignoring diversity). If this achieves comparable temporal F1 and QA accuracy, it would mean the model's learned search queries are already specific enough that diversity-aware selection provides marginal benefit. Second, replace SigLIP-400M with a weaker embedding model (e.g., CLIP-ViT-B/32) to test whether retrieval quality is a bottleneck or whether the learned search policy is robust to noisy embeddings. The paper's temporal F1 metric (Table 1) reflects the joint performance of the search policy and the retrieval function; these ablations would decompose the contribution of each. If DPP with SigLIP is essential, this would motivate future work on making the retrieval function trainable — for instance, fine-tuning SigLIP on the specific video domain, or incorporating the retrieval function into the RL loop through a differentiable approximation so the policy can learn to generate queries that the current retrieval function can effectively resolve.

3. Dynamic difficulty estimation and adaptive search budgets.

The paper fixes the maximum search budget at 8 turns × 8 frames = 64 frames, regardless of question difficulty. However, the case studies reveal that different question types require different amounts of search: hypothesis-driven search (Figure 5) needs only 1–2 turns to confirm a specific prediction, while sequential search (Figure 15) needs 5 turns to traverse the timeline. A natural extension would be to train the model to predict, from the initial preview frames and question text, how much search budget is needed — and to allocate budget dynamically, terminating search early when evidence is sufficient rather than always using the maximum turns. This connects directly to the paper's data filtering pipeline, which already estimates question difficulty (questions solvable from 4 frames are "easy"; questions unsolvable even with 64 frames are "hard"). A difficulty predictor could be trained as a lightweight classifier on the filtered dataset, using the Stage 1 and Stage 2 filtering results as supervision: questions filtered by Stage 1 are "easy" (no search needed), questions retained through both stages are "medium" (search needed and helpful), and questions filtered by Stage 2 are "hard" (search insufficient). At inference time, the difficulty predictor would select the search budget — easy questions get 0 turns, medium questions get the full 8 turns, hard questions get flagged for human review or routed to a larger model. This would address the latency concern in Section 6: the current 13.4-second latency is incurred uniformly, but easy questions could be answered in a fraction of that time. A strong follow-up would report accuracy vs. average latency curves by varying the difficulty threshold, showing the Pareto frontier of the speed-accuracy tradeoff.

4. On-policy data generation for self-improvement: Does RL-trained search generate better training data than GPT-4o?

The paper's SFT stage uses GPT-4o to generate text-video interleaved reasoning traces for the filtered training data. An intriguing follow-up would replace GPT-4o with the RL-trained TimeSearch-R model itself: after RL training converges, use the trained model to generate new reasoning traces on the training data, then fine-tune a fresh base model on these self-generated traces, then re-run RL, iterating. This is the "self-improvement loop" pattern that has been successful in text-based reasoning (STaR, ReSTEM^{EM}). The key question is whether the RL-trained model's search traces — which exhibit emergent strategies like hypothesis-driven search and elimination — provide better SFT supervision than GPT-4o's traces. If GPT-4o's traces contain suboptimal search patterns (overly broad queries, inefficient temporal windows) that the SFT model learns and must later unlearn during RL, then self-generated traces might provide a better initialization, accelerating RL convergence or achieving higher final accuracy. A negative result (self-generated SFT underperforms GPT-4o SFT) would be equally informative, suggesting that the diversity of GPT-4o's search strategies — even if individually suboptimal — provides a better exploration foundation for RL. The paper's note that the ReSTEM^{EM}-trained revision model in the reference work degraded (Appendix K) suggests that on-policy self-improvement for interactive tasks is fragile, making this a high-risk, high-reward experiment.

5. Causal analysis of CSV: Does gating on answer correctness help or hurt search learning?

The completeness reward definition (Equation 4) includes an indicator function 1[Acc(A, A*) > 0.5] that gates the completeness reward on the original answer being correct. This means that on trajectories where the model answers incorrectly, the model receives no signal about search quality, even if the search was excellent. The paper states this is to prevent the model from learning to search thoroughly but guess randomly, but provides no ablation testing this claim. A critical follow-up would compare three gating designs: (a) full gating (current design, only reward completeness when the answer is correct), (b) no gating (reward completeness on all trajectories regardless of answer correctness), and (c) soft gating (weight the completeness reward by the accuracy score, R_c = Acc(A, A*) · Acc(A_c, A*)). The prediction from the paper's stated reasoning is that (b) should produce better search quality (higher temporal F1, higher completeness) but lower QA accuracy (because the model learns that thorough search is rewarded even when it answers wrong). Design (c) is an unexplored middle ground. Additionally, the experiment should track the speed of search learning: gating reduces the number of trajectories that contribute to the search gradient (only correct-answer trajectories provide completeness signal), which could slow down search skill acquisition in early RL stages when answer accuracy is low. If design (b) learns search skills faster but with lower final QA accuracy, this would reveal a speed-accuracy tradeoff in the gating design and motivate adaptive gating that is aggressive early (no gating, to learn search quickly) and conservative late (gating, to align search with answer quality). A strong follow-up would plot temporal F1, completeness, consistency, and QA accuracy against training steps for all three gating designs, revealing the learning dynamics.

6. Stress-testing generalizability to other video domains and question types.

The training data is dominated by egocentric daily activities (49.5% from Haystack-Ego4D) and internet videos (35.6% from Panda-70M). The evaluation benchmarks span general video understanding (VideoMME, MLVU, LongVideoBench) and specific temporal search tasks (Haystack-LVBench, Haystack-Ego4D). A stress-test that would reveal the boundaries of learned search would evaluate TimeSearch-R on domain-shifted videos not represented in training: (a) instructional videos with step-by-step procedures (e.g., cooking, assembly, repair — where sequential search is essential and timestamps correspond to procedural steps), (b) surveillance videos with long static periods and brief action events (where the temporal search must discriminate rare events from background), (c) sports videos with rapid motion and fine-grained action recognition needs (where frame-level detail matters more than temporal window selection). For each domain, the critical metric is whether the RL-trained search policy transfers — does the model apply appropriate search strategies (sequential for procedural, event-triggered for surveillance, motion-cued for sports) without domain-specific training, or does it revert to generic search behavior? A negative result (search strategies are domain-specific and don't transfer) would motivate domain-adaptive training or meta-learning approaches. A positive result (strategies transfer) would strengthen the paper's claim that TimeSearch-R learns "fundamental cognitive patterns" rather than dataset-specific heuristics. The follow-up should also characterize failure modes per domain: in surveillance, does the model terminate search too early (missing rare events)? In sports, does the model struggle with query formulation for fast-moving objects?

Practical Applications and Downstream Use Cases

1. Efficient video corpus indexing and retrieval for enterprise video libraries.

Organizations with large video archives — media companies, educational institutions, legal firms managing deposition footage, insurance companies processing claim videos — need to answer specific questions about video content without manual review. The standard approach is uniform frame sampling followed by single-pass VLM inference, which applies the same computational budget to every query regardless of whether the answer is trivially visible or requires careful temporal search. TimeSearch-R's adaptive search offers a concrete efficiency gain: on Haystack-LVBench, it achieves 52.1% QA accuracy using 8.8 frames on average — matching the retrieval-based baseline at 32 frames (50.5%) while using 3.6× fewer frames. For a video library processing 10,000 hours of content, reducing the per-query frame budget from 32 to 9 translates to approximately 3.6× lower inference cost (fewer frames to encode, fewer visual tokens to process) while maintaining or improving accuracy. The latency numbers (13.4 seconds per query, Table 4) are suitable for batch processing where videos are analyzed offline and results are cached for future queries.

2. Interactive video assistants for accessibility and education.

For users who are blind or visually impaired, understanding video content requires descriptive narration that adapts to specific questions. A static-sampling model can describe only what it saw in pre-selected frames; if a user asks "Did the character pick up the keys before leaving the room?" and the key-pickup moment was not in the sampled frames, the model cannot answer. TimeSearch-R's interleaved search enables the model to investigate the video in response to the user's question — it can search for the key-pickup action, verify the sequence of events, and provide a temporally grounded answer. The 13.4-second latency (Table 4) is acceptable for asynchronous Q&A (user asks a question about a video, waits for the answer), though not for real-time narration where sub-second response is required. The sequential search pattern demonstrated in Figure 15 — where the model traverses a daily itinerary segment by segment — is precisely the kind of temporal reasoning needed for questions about event ordering ("What happened after the boy left school?"). The consumable output is not just the final answer but the search trace itself, which provides an auditable evidence trail showing which video segments were examined and why — critical for trust in accessibility applications where incorrect answers have direct impact on users.

3. Self-improving data annotation pipelines for video model training.

Training video-language models requires large-scale annotated datasets with temporal grounding — identifying when in a video a described event occurs. Manual annotation of temporal segments is expensive (requires annotators to watch videos and mark timestamps) and often imprecise. TimeSearch-R's RL training pipeline produces a model that can autonomously locate relevant temporal regions and answer questions about them. The search traces (temporal windows + queries + selected frames) generated by the trained model can serve as pseudo-labels for temporal grounding — each successful search trajectory provides a correspondence between a question and a set of timestamps where the answer is visually evident. The paper's temporal F1 of 8.1 on Haystack-LVBench (Table 1) represents 3× better localization than T*'s 2.5, suggesting the learned policy produces cleaner pseudo-labels than hand-crafted agents. These pseudo-labels could be used to train dedicated temporal grounding models (e.g., moment retrieval, video paragraph grounding) or to augment the training data for the next generation of video-language models. The key advantage over using GPT-4o or other static models for pseudo-labeling is that TimeSearch-R's labels are self-improving — as RL training progresses, the model's search accuracy improves, producing higher-quality pseudo-labels that can bootstrap further training.

4. Selective frame retrieval for bandwidth-constrained video streaming analysis.

In scenarios where video must be analyzed over limited-bandwidth connections — drone surveillance, remote infrastructure inspection, telemedicine consultations — transmitting full video streams for cloud-based VLM inference is infeasible. TimeSearch-R's architecture, where the policy model runs locally (potentially on-device) and issues targeted frame retrieval requests to a remote video server, enables query-driven bandwidth allocation: instead of streaming the entire video, the system transmits only the frames that the model specifically requests, at the timestamps it specifies. The 8-turn × 8-frame search budget means the model requests at most 64 frames for any question — a compression ratio of roughly 100:1 for a 30-second, 30fps video (900 frames). The DPP-based retrieval ensures those 64 frames are maximally informative (diverse and query-relevant), making efficient use of the limited bandwidth. Table 4's latency of 13.4 seconds includes frame retrieval time (SigLIP embedding extraction + DPP) and model inference; in a streaming scenario, the dominant cost is frame transmission latency, which depends on connection quality rather than computation. The use case extends beyond bandwidth constraints to privacy-sensitive applications (e.g., security footage analysis) where transmitting entire videos to third-party cloud services is prohibited — the on-device model can formulate queries locally, receive only the specific frames needed, and discard them after answering, minimizing data exposure.