Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-in

ArXiv: 2512.14273

🎯 Pitch

Video AI models often hallucinate answers because they miss key details in coarse video frames. Zoom-Zero fixes this by teaching models to first find the right moment, then zoom in on crucial frames to extract fine-grained evidence, boosting grounded accuracy significantly.

---

1. Executive Summary

This paper introduces Zoom-Zero, a coarse-to-fine framework for grounded video question answering that first localizes query-relevant temporal segments and then temporally zooms into the most salient frames for finer-grained visual verification. Evaluated on a suite of GVQA and long-video benchmarks using Qwen2.5-VL-7B as the base model, the method improves over GRPO-based baselines through two named mechanisms: a zoom-in accuracy reward (validating that grounded segments contain the visual evidence required for correct reasoning by re-answering the question on high-resolution cropped frames) and token-selective credit assignment (decoupling multi-faceted reward signals so that temporal-grounding tokens and answer-generation tokens receive separate advantage estimates rather than a single uniform scalar). The approach advances temporal grounding by 5.2% mIoU on NExT-GQA and 4.6% on ReXTime while improving average answer accuracy by 2.4%, and the coarse-to-fine paradigm yields a 6.4% average improvement on long-video benchmarks, establishing that the zoom-in verification mechanism is effective only when the base model already possesses non-trivial temporal grounding capability to produce initial segment predictions that can be refined.

2. Context and Motivation

The Core Problem: Large Video-Language Models Can't Reliably Ground Their Answers in Video Evidence

The fundamental challenge this paper addresses is that large video-language models (LVLMs) exhibit a dangerous disconnect between what they say and what they see. Even state-of-the-art models like Qwen2.5-VL can generate plausible-sounding answers to questions about videos, but those answers are frequently not grounded in the correct temporal segments of the video. The model might answer a question about "what happened after the person poured the liquid" by referencing frames from before the pouring event, or worse, hallucinate details that never occurred anywhere in the video. This is not merely about getting answers wrong — it's about getting them wrong for the wrong reasons, creating a brittle system that cannot be trusted in applications where temporal precision matters.

The paper formalizes this through the task of Grounded Video Question Answering (GVQA). Unlike standard video question answering, which only requires the model to produce a correct answer, GVQA requires the model to both answer the question and localize the temporal segment(s) that provide the evidence for that answer. This dual requirement exposes a critical capability gap: models that score well on answer accuracy often perform poorly when you ask them to point to where in the video their answer came from, revealing that their reasoning process is not faithfully anchored to the visual timeline.

Why This Problem Matters

The significance of temporal grounding extends far beyond benchmark scores. Several real-world implications motivate this work:

Trust and verifiability. In applications where video evidence is used for decision-making — medical procedure review, surveillance footage analysis, sports officiating, industrial inspection — knowing that a model gave the right answer isn't enough. Users need to verify where in the video the evidence exists. A model that says "the worker violated safety protocol at [00:30–00:45]" but points to the wrong clip erodes trust even when the answer happens to be correct by chance.

Hallucination mitigation. Temporal mislocalization is a specific form of hallucination that is particularly pernicious in video understanding. Unlike factual hallucinations (e.g., claiming a non-existent object appears), temporal hallucinations can be subtle — the described event does occur somewhere in the video, just not where the model claims. This makes them harder to detect through simple factuality checks.

Long-video comprehension bottleneck. As videos grow longer, models face a fundamental trade-off: they can either preserve temporal coverage (many frames) or spatial detail (high resolution per frame), but not both under fixed context budgets. The paper explicitly frames this as a context budget problem: "capturing long-range temporal context versus preserving fine-grained visual detail" is a zero-sum game. In long videos (e.g., the 4101-second average duration of LVBench videos), models are forced to downsample aggressively, discarding the fine-grained visual cues needed for precise understanding. This problem is not merely technical — it represents a fundamental scaling challenge that will only intensify as video content continues to proliferate.

The accuracy-grounding trade-off. The paper observes that RL-based approaches explicitly optimized for temporal grounding often sacrifice general question-answering accuracy. This creates a perverse incentive structure where improving localization hurts understanding, or vice versa. A method that can improve both simultaneously — or at minimum, improve one without degrading the other — represents genuine progress.

Prior Approaches and Their Shortcomings

The paper positions itself against three categories of prior work, each with identifiable gaps:

SFT-Based LVLMs with Grounding Capability

Models like TimeChat, VTimeLLM, and Grounded-VideoLLM reformulate temporal grounding as a text generation problem — predicting start and end timestamps in natural language — and are trained via supervised fine-tuning. While these models established the baseline for GVQA, they suffer from a fundamental limitation: SFT alone cannot provide the fine-grained feedback needed to distinguish between answers that are correct-but-ungrounded versus answers that are both correct and faithfully localized. The supervision signal (cross-entropy loss on token predictions) treats all correct-format outputs identically, regardless of whether the predicted temporal span actually contains the evidence needed to answer the question.

Concretely, if a model produces the right answer (e.g., "B") and predicts a plausible-looking timestamp (e.g., [20.3, 30.8]), the SFT loss provides no signal about whether that timestamp genuinely covers the relevant video evidence. The model learns to produce well-formatted outputs, not to verify that its localization is faithful.

GRPO-Based RL Approaches for Video Grounding

Recent work — Time-R1, TVG-R1, TimeZero, VideoChat-R1 — has applied Group Relative Policy Optimization to video temporal grounding. These methods use verifiable rewards (format compliance, IoU with ground truth) to optimize models beyond supervised fine-tuning. VideoChat-R1 further incorporates answer accuracy into the reward.

However, the paper identifies two specific limitations in existing GRPO-based approaches:

Limitation 1: No verification that grounded segments contain the answer. Even when a model achieves high IoU with the ground-truth temporal segment and produces the correct answer, there is no mechanism to verify that the answer actually follows from the visual content of the grounded segment. A model could learn to game the metrics by: (a) producing high-IoU temporal predictions from spurious patterns unrelated to the question's visual evidence; and (b) answering questions correctly from general knowledge or linguistic cues rather than from the localized video content. The paper makes this explicit: "these training objectives still cannot guarantee that localized video segments actually contain the visual evidence required for correct reasoning."

Limitation 2: GRPO's uniform credit assignment under multi-faceted rewards. The standard GRPO algorithm (Equation 1) compresses multiple reward signals — format, IoU, answer accuracy — into a single scalar via simple summation before computing advantages. This means every token in a response receives the same advantage signal, regardless of whether that token contributed to temporal localization, answer generation, or just formatting boilerplate. The paper provides a concrete example in Appendix C, Table 8, demonstrating how this leads to perverse incentives: a response with poor temporal grounding (IoU = 0.0) but correct answer can receive a positive overall advantage under standard GRPO, reinforcing all tokens in that response — including the tokens responsible for the poor localization. Conversely, a response with excellent temporal grounding (IoU = 0.8) but incorrect answer can receive a negative advantage, penalizing the very tokens that performed accurate localization.

This uniform credit assignment is not a minor implementation detail — it fundamentally limits the model's ability to improve specific capabilities independently. The model cannot learn to dissociate "generate accurate timestamps" from "generate correct answers" because both receive the same blended reward signal.

General-Purpose LVLMs Without Grounding

Models like LLaVA-OneVision and InternVL2.5 achieve strong general video understanding but lack temporal grounding capability entirely. They answer questions without localizing evidence, making them unusable for applications requiring verifiable temporal reasoning. Their existence establishes the upper bound for accuracy-only performance but highlights the gap between "knowing the answer" and "knowing where the answer came from."

The Critical Role of Context Budgets and Spatial Resolution

A less obvious but equally important motivation lies in how LVLMs allocate their visual token budgets. Qwen2.5-VL uses dynamic spatiotemporal resolution (Section 3, Equation 3): given a fixed video context budget $L_v$, the model can trade off between sampling more frames (at lower per-frame resolution) or fewer frames (at higher per-frame resolution). The minimum token resolution per frame is $16 \times 28 \times 28$ and the maximum is $768 \times 28 \times 28$.

This creates a practical failure mode that the paper's running example illustrates concretely: in a video where the critical visual evidence is a small number "29%" displayed on screen, if the model has allocated its token budget to cover many frames (to preserve temporal context in a long video), each frame may be encoded at such low resolution that the "29%" is effectively invisible. The model sees the scene, sees the general action, but misses the fine-grained detail necessary for answering. Only by temporally zooming into the relevant segment and re-encoding at higher spatial resolution can the model capture such details.

This is not merely a future scalability concern — it is a present limitation that affects real model behavior. The paper's Figure 1 illustrates exactly this scenario: the coarse pass misses "29%" due to low-resolution tokens; the zoom-in pass, with higher per-frame resolution on the localized segment, captures it.

How This Paper Positions Itself

Zoom-Zero is positioned not as a new base model or a new RL algorithm from scratch, but as an enhancement to the GRPO training framework specifically tailored to the multi-faceted nature of GVQA. The paper makes two distinct contributions that address orthogonal gaps:

The zoom-in accuracy reward (Section 4.2) addresses the verification gap: it ensures that temporal grounding predictions are not merely evaluated by IoU overlap with ground truth, but are actually tested for whether they contain the answer. By cropping the video to the predicted segment, re-encoding at higher resolution, and re-answering the question, the zoom-in reward provides a direct signal about the faithfulness of the localization. This connects the temporal grounding task to the visual verification task that humans perform naturally: "if I only show you this segment of the video, can you still answer the question correctly?"

Token-selective credit assignment (Section 4.3) addresses GRPO's uniform credit problem: by computing separate advantages for each reward type (format, IoU, answer accuracy, zoom-in accuracy) and assigning them to the specific tokens responsible for each aspect (e.g., IoU advantage goes to tokens inside <glue></glue> tags, answer accuracy advantage goes to tokens outside those tags), the model receives disentangled feedback that allows it to improve temporal localization and question answering independently.

The paper explicitly connects these contributions: "In our case, since the outputs for answering and temporal grounding are explicitly formatted with task-specific tokens, it is feasible to distinguish the contribution of corresponding tokens to each aspect." This is a pragmatic insight — the structured output format (with <answer>, <glue>, and thinking response tags) that is already required for the format reward also provides the token-level segmentation needed for selective credit assignment. The method doesn't require additional annotation or architectural changes; it exploits structure that already exists in the training protocol.

The coarse-to-fine paradigm is presented as an inference-time strategy that builds on the temporal grounding capability developed during training. The key insight is that once the model can reliably localize relevant segments, those segments can be re-processed at higher resolution — turning what was previously a context budget limitation into an active mechanism for detail recovery. This mirrors a broader principle in cognitive science (which the paper briefly alludes to): "This hierarchical paradigm resembles human visual cognition: breaking down complex problems, identifying relevant temporal intervals, and then refining focus to extract precise details."

The Accuracy-Grounding Trade-off as a Unifying Concern

Underlying both contributions is the observation that improving temporal grounding and maintaining question-answering accuracy are partially competing objectives under standard GRPO. The paper's ablation (Table 5) quantifies this: baseline GRPO with format + IoU + answer accuracy rewards achieves 69.6% accuracy on NExT-GQA, while adding the zoom-in reward and token-selective credit assignment brings it to 70.7% — a modest but meaningful gain that demonstrates both capabilities can be improved simultaneously when the reward structure is properly designed.

This positions Zoom-Zero as addressing a fundamental tension in GVQA that prior work either ignored (by optimizing only for IoU) or failed to resolve (by naïvely summing rewards). The method doesn't claim to eliminate the trade-off entirely — the paper is careful about its scope — but rather provides a training framework that extracts more signal from multi-faceted rewards than standard GRPO can.

3. Technical Approach

This is primarily an RL-training enhancement paper whose core idea is that GRPO-based optimization for grounded video question answering can be substantially improved by (a) adding a verification step that tests whether grounded segments actually contain answer-relevant visual evidence and (b) decoupling multi-faceted reward signals so that temporal grounding tokens and answer generation tokens receive separate advantage estimates rather than a single blended scalar.

3.1 Reader Orientation

Zoom-Zero is a training framework (not a new model architecture) that teaches a large video-language model to first identify which temporal segments of a video are relevant to a question, and then to verify that those segments actually contain the visual information needed to answer correctly. The framework solves a specific failure mode: existing GRPO-based methods for video grounding produce models that can predict plausible-looking timestamps and correct answers, but there is no mechanism ensuring the answer actually follows from the visual content at the predicted timestamp rather than from general knowledge, linguistic cues, or spurious correlations.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components arranged in a two-pass pipeline:

1. Base LVLM (Qwen2.5-VL-7B) — the pretrained video-language model that serves as the policy to be optimized. It processes video frames and questions to generate structured outputs containing explicit timestamp predictions (in <glue></glue> tags) and answer choices (in <answer></answer> tags).
2. Coarse-Grained Rollout Pass — the policy generates $G$ candidate responses per question, each containing a predicted temporal span and preliminary answer. This pass uses the full video at whatever spatial resolution the dynamic token budget allocates (typically lower resolution per frame to cover the full temporal extent).
3. Zoom-In Verification Module — for each rollout, the predicted temporal span is used to crop the video to the localized frames, which are then re-encoded at higher per-frame spatial resolution (since fewer frames need to be represented under the same context budget). The model re-answers the question on this zoomed-in crop, producing a zoom-in accuracy reward $R_{\mathrm{Zoom}}$ that signals whether the grounded segment genuinely contains the answer.
4. Multi-Faceted Reward Calculator — computes four distinct reward signals per rollout: format compliance ($R_{\mathrm{format}}$), temporal grounding quality ($R_{\mathrm{IoU}}$ via intersection over union with ground truth), coarse answer accuracy ($R_{\mathrm{Acc}}$), and zoom-in verification accuracy ($R_{\mathrm{Zoom}}$).
5. Token-Selective Advantage Estimator (TokenAdv) — rather than summing all rewards into a single scalar and computing one uniform advantage per sequence (standard GRPO), this module computes separate advantages for each reward type and assigns them to the specific tokens responsible for that capability: IoU and zoom-in advantages go to tokens inside <glue>...</glue> tags, answer accuracy and format advantages go to all other tokens.

Information flows as follows: a video-question pair enters → the policy $\pi_{\theta_{\mathrm{old}}}$ generates $G$ candidate responses → each response is parsed into temporal span + answer → the zoom-in module crops and re-encodes the video at the predicted span → the model re-answers on the zoomed crop → four rewards are computed per response → token-selective advantages are estimated per reward type → the policy $\pi_\theta$ is updated via GRPO loss with token-specific advantage weighting → the process repeats.

3.3 Roadmap for the Deep Dive

• First, the coarse-to-fine zoom-in mechanism (Section 4.1): how the model predicts temporal spans, how those spans are used to crop and re-encode the video at higher resolution, and how the zoomed-in frames feed back into answer verification. This is the core algorithmic contribution and must be understood before the rewards make sense.
• Second, the reward design (Section 4.2): the four reward functions — format, answer accuracy, IoU, and zoom-in accuracy — with precise definitions, how they are computed, and why the zoom-in reward provides a signal that standard IoU + accuracy rewards cannot.
• Third, token-selective credit assignment (Section 4.3): the mathematical modification to GRPO's advantage computation, including the per-reward-type advantage normalization, the conditional assignment rule based on token position, and the concrete example from Appendix C that motivates why this matters.
• Fourth, the overall training procedure: the two-stage RL training with data filtering, the hyperparameter choices ($G=8$ rollouts, 8192 token context budget, 1 FPS sampling), and the GRPO objective with KL regularization.

3.4 Detailed, Sentence-Based Technical Breakdown

Coarse-to-Fine Video Understanding via Temporal Zoom-In

The coarse-to-fine mechanism addresses the fundamental tension between temporal coverage and spatial detail that arises under a fixed visual context budget $L_v$. Qwen2.5-VL uses dynamic spatiotemporal resolution (Section 3, Equation 3): when many frames are sampled, each frame gets fewer tokens and hence lower spatial resolution; when fewer frames are sampled, each can be encoded at higher resolution. The zoom-in strategy exploits a model capability the paper has trained — temporal grounding — to convert this from a fixed trade-off into an active detail-recovery mechanism.

The coarse pass. In the first pass, the model processes the full video under the context budget constraint. Since the video may be long (up to thousands of seconds), the per-frame token resolution $V_{\mathrm{res}}$ is typically low — the model trades spatial granularity to maintain broad temporal coverage. From this coarse view, the model generates a structured response that includes predicted temporal spans: a set of (start, end) pairs $(s_1, e_1), (s_2, e_2), \ldots, (s_n, e_n)$ enclosed in <glue></glue> tags, representing where in the video the model believes the answer-relevant evidence resides.

Cropping and re-encoding. Given the predicted spans, the video is cropped to those intervals, yielding $N' < N$ frames (where $N$ is the number of frames in the full video). Because the visual context budget $L_v$ is fixed but the number of frames has decreased, the per-frame token resolution increases:

$V_{\mathrm{res}}' = \frac{L_v}{N'} > \frac{L_v}{N} = V_{\mathrm{res}}$

where $V_{\mathrm{res}}'$ is the new per-frame token resolution after zoom-in and $V_{\mathrm{res}}$ is the original per-frame resolution from the coarse pass.

What this achieves: fewer frames are processed, but each frame receives more tokens from the fixed budget, enabling higher spatial resolution encoding. This means fine-grained visual details — small text, subtle object manipulations, nuanced facial expressions — that were compressed or lost in the coarse pass become visible in the zoom-in pass. The paper's Figure 1 illustrates this concretely: the coarse pass encodes frames at low resolution and misses a small "29%" text element; the zoomed-in frames, at higher per-frame resolution, capture it.

The fine pass. The cropped, higher-resolution frames are fed back to the model along with the original question. The model produces a new answer. If this answer is correct, the model receives the zoom-in accuracy reward $R_{\mathrm{Zoom}} = 1$; otherwise $R_{\mathrm{Zoom}} = 0$. This creates a direct verification signal: the model only gets the zoom-in reward if (a) the predicted temporal span genuinely contains the visual evidence needed to answer the question, AND (b) the model can extract that evidence from the higher-resolution crop.

Why this is a training mechanism, not just an inference trick. During GRPO training, the zoom-in reward is computed for every rollout. Rollouts with accurate spans that crop to the right video segment will (if the model can answer from that crop) receive $R_{\mathrm{Zoom}} = 1$, boosting their advantage. Rollouts with inaccurate spans — even if the coarse answer was correct by chance or from general knowledge — will fail the zoom-in verification (receiving $R_{\mathrm{Zoom}} = 0$), reducing their advantage. This creates a selection pressure that jointly optimizes temporal grounding accuracy and evidence-faithful answering: the model learns that predicting the right timestamp matters because it determines whether the zoom-in verification succeeds.

The dependency on base capability. The paper is careful to note that this mechanism is only effective because Qwen2.5-VL already has non-trivial temporal grounding ability (Table 1 shows the base model achieves 20.2% mIoU on NExT-GQA without any grounding-specific training). If the base model's temporal spans were essentially random, the zoom-in verification would provide no useful signal — no rollout would consistently crop to the right segment, and the reward would be dominated by noise. The method amplifies an existing capability rather than creating it from scratch.

Inference-time zoom-in as a separate benefit. Beyond training, the coarse-to-fine mechanism is applied at inference for long-video understanding (Section 5.3 and Table 4). Even without ground-truth IoU labels, the trained model can predict query-relevant segments, crop to them, and re-answer at higher resolution, improving accuracy on long-video benchmarks by +4.2% on VideoMME (long with subtitles), +1.2% on MLVU, and +1.0% on LVBench over the one-stage Zoom-Zero baseline. This demonstrates that the temporal grounding capability learned during RL training transfers to a practical inference-time detail-recovery mechanism.

Rewards Design

The paper defines four reward functions, each providing a distinct signal to the policy during GRPO training. The rewards are computed per-rollout from the model's structured output.

Format Reward ($R_{\mathrm{format}}$). The model is required to produce output conforming to a specific template. The format prompt specifies:

"Enclose the full analysis in thinking response tags. Then, provide your answer within the <answer></answer> tags, output the corresponding letter of the option. At the same time, in the <glue></glue> tags, include only the precise video segments (in seconds) that strongly support your answer, in the format of [(s1, e1), (s2, e2), …]."

Regular expression matching verifies whether the output conforms to this template. $R_{\mathrm{format}} = 1$ if the format fully matches; otherwise $0$. This reward is binary and applied after both the coarse pass (for the initial response) and contributes to the overall advantage. Its primary function is to enforce the structured output that makes token-selective credit assignment possible — without reliable <answer> and <glue> tag segmentation, the TokenAdv mechanism would not know which tokens correspond to which capability.

Temporal Grounding Reward ($R_{\mathrm{IoU}}$). The temporal grounding quality is evaluated by Intersection over Union between the model-predicted interval $\mathcal{I}_{\mathrm{pred}}$ (extracted from the <glue></glue> tags) and the ground-truth interval $\mathcal{I}_{\mathrm{gt}}$:

$R_{\mathrm{IoU}} = \frac{|\mathcal{I}_{\mathrm{pred}} \cap \mathcal{I}_{\mathrm{gt}}|}{|\mathcal{I}_{\mathrm{pred}} \cup \mathcal{I}_{\mathrm{gt}}|}$

where $|\mathcal{I}_{\mathrm{pred}} \cap \mathcal{I}_{\mathrm{gt}}|$ is the temporal overlap (in seconds) between predicted and ground-truth spans, and $|\mathcal{I}_{\mathrm{pred}} \cup \mathcal{I}_{\mathrm{gt}}|$ is the total time covered by either span.

What it computes: a continuous value in $[0, 1]$ measuring how well the predicted temporal segment matches the ground-truth segment. An IoU of 1.0 means perfect overlap; 0.0 means no overlap. This is the standard metric used in temporal grounding literature (Cobbe et al., 2021; Lightman et al., 2023; Xiao et al., 2024) and is used both as a training reward and as an evaluation metric (mIoU, R@0.3, R@0.5).

Why this form: IoU penalizes both under-coverage (predicted span is too small, missing parts of the ground truth) and over-coverage (predicted span is too large, including irrelevant frames). This is important because the zoom-in mechanism relies on the predicted span being both precise (not too large, or the re-encoded crop will still have low per-frame resolution) and complete (not too small, or the answer-relevant evidence will be cropped out). A reward based only on precision (intersection over prediction) would encourage tiny, high-precision spans that miss evidence; a reward based only on recall would encourage excessively large spans that defeat the purpose of zoom-in.

Answer Accuracy Reward ($R_{\mathrm{Acc}}$). The correctness of the model's answer in the coarse pass is evaluated against the ground-truth answer. For multiple-choice questions, $R_{\mathrm{Acc}} = 1$ if the predicted option letter matches the correct option; otherwise $R_{\mathrm{Acc}} = 0$.

What it computes: a binary signal indicating whether the model can answer the question correctly from the full, coarsely-encoded video. This reward is attributed to tokens outside the <glue></glue> tags (mainly the answer generation and reasoning tokens) under the TokenAdv scheme.

Why it is separate from zoom-in accuracy: the paper explicitly separates coarse answer accuracy from zoom-in verification accuracy because they measure different things. $R_{\mathrm{Acc}}$ measures whether the model can answer correctly given the full video at coarse resolution. $R_{\mathrm{Zoom}}$ measures whether the model's temporal localization is faithful — that is, whether the predicted span, when cropped and shown in isolation at higher resolution, contains enough evidence to answer. A model could achieve $R_{\mathrm{Acc}} = 1$ (correct answer from full video) while getting $R_{\mathrm{Zoom}} = 0$ (the predicted span doesn't actually contain the evidence), or vice versa. By keeping these rewards separate and attributing them to different tokens, the policy receives disentangled feedback.

Zoom-In Accuracy Reward ($R_{\mathrm{Zoom}}$). This is the paper's key innovation. After the coarse pass produces a predicted temporal span from <glue></glue>, the video is cropped to that span, re-encoded at higher per-frame resolution, and the model re-answers the question on the cropped frames. $R_{\mathrm{Zoom}} = 1$ if this zoomed-in answer is correct; otherwise $R_{\mathrm{Zoom}} = 0$.

What it computes: a binary verification signal that tests the hypothesis "the predicted temporal segment contains the visual evidence necessary to answer this question." If the answer on the zoomed-in crop is correct, the hypothesis is supported. If it is incorrect, either the span is wrong (doesn't cover the relevant frames) or the evidence is ambiguous (the model can't extract the answer even from the right frames), both of which indicate poor grounding.

Why this form and not simply re-using IoU: IoU measures spatial/temporal overlap with the ground-truth annotation, but ground-truth annotations may not perfectly align with "the minimal set of frames needed to answer." A span could have high IoU but still miss a critical frame with the answer-relevant detail; conversely, a span with moderate IoU might actually contain all the necessary evidence. The zoom-in reward provides a complementary signal: it tests functional adequacy of the span (can you answer from it?) rather than merely geometric overlap (does it match the annotation?). This is analogous to how the PRM's zoom-in reward in the reference paper tests whether a localized solution step actually enables reaching the correct final answer, rather than just whether the step looks plausible.

The interaction between rewards. The paper demonstrates (Table 5, ablation) that adding $R_{\mathrm{Zoom}}$ to the baseline GRPO (with $R_{\mathrm{format}} + R_{\mathrm{IoU}} + R_{\mathrm{Acc}}$) improves both answer accuracy (+0.8 on NExT-GQA, +1.9 on ReXTime) and temporal grounding (+1.0 mIoU on NExT-GQA, +1.7 mIoU on ReXTime). This mutual improvement — where a verification reward for temporal grounding also improves answer accuracy — is the paper's key empirical finding. It occurs because the zoom-in reward creates a selection pressure that favors rollouts where the temporal span and the answer are internally consistent: the model cannot get $R_{\mathrm{Zoom}} = 1$ without both a correct span AND a correct answer on that span, so the reward structure implicitly couples the two objectives.

Token-Selective Credit Assignment for Advantage Estimation

This is the paper's second major contribution: a modification to GRPO's advantage computation that addresses the uniform credit assignment problem when multiple reward signals are present.

The problem with standard GRPO for multi-faceted rewards. In standard GRPO (Section 3, Equations 1–2), all reward signals are summed into a single scalar $r_i$ before advantage computation:

$A_{i,t} = \frac{r_i - \mathrm{mean}(\{r_i\}_{i=1}^G)}{\mathrm{std}(\{r_i\}_{i=1}^G)}$

This single advantage $A_{i,t}$ is then assigned uniformly to every token $t$ in response $i$. The paper provides a concrete example (Appendix C, Table 8) illustrating why this fails. Consider five responses with two rewards:

In this example, Response 1 has the worst temporal grounding ($R_{\mathrm{IoU}} = 0.0$) but receives a positive overall advantage ($A_{\mathrm{Sum}} = +0.06$) because its correct answer inflates the summed reward. Under standard GRPO, all tokens in Response 1 — including the tokens inside <glue></glue> that produced the completely wrong temporal span — get reinforced with a positive advantage. Conversely, Response 4 achieves the best temporal grounding ($R_{\mathrm{IoU}} = 0.8$) but receives a negative advantage ($A_{\mathrm{Sum}} = -0.58$), penalizing the very tokens that performed accurate localization.

This uniform credit assignment creates a fundamental learning obstacle: the policy cannot disentangle "generate good timestamps" from "generate good answers" because both capabilities receive the same blended signal. The model may learn to improve answer accuracy at the expense of grounding (or vice versa), or may converge to a local optimum where one capability dominates the advantage signal and the other stagnates.

TokenAdv: per-reward-type advantage computation. The paper's solution is to compute advantages separately for each reward type, then assign each token an advantage that is the average of the relevant task-specific advantages. The computation proceeds in two steps:

Step 1: Per-reward advantage normalization. For each reward type $r^k$, compute a separate advantage for each response $i$:

$A_i^k = \frac{r_i^k - \mathrm{mean}(\{r_i^k\}_{i=1}^G)}{\mathrm{std}(\{r_i^k\}_{i=1}^G)}, \quad r^k \in \{R_{\mathrm{format}}, R_{\mathrm{Acc}}, R_{\mathrm{Zoom}}, R_{\mathrm{IoU}}\}$

where $A_i^k$ is the advantage of response $i$ with respect to reward type $k$ only, $r_i^k$ is the raw reward value for response $i$ on reward type $k$, $\mathrm{mean}(\{r_i^k\}_{i=1}^G)$ is the average of that reward across all $G$ rollouts, and $\mathrm{std}(\{r_i^k\}_{i=1}^G)$ is the standard deviation across rollouts.

What this computes: four separate advantage scalars per response, one per reward type. Each advantage represents how much better (or worse) this particular response is relative to the other $G-1$ rollouts on that specific capability. Crucially, this preserves the relative ranking information within each capability. A response with high $R_{\mathrm{IoU}}$ but low $R_{\mathrm{Acc}}$ will receive a positive $A^{\mathrm{IoU}}$ and a negative $A^{\mathrm{Acc}}$, rather than having both signals collapsed into a single ambiguous scalar.

Step 2: Token-level selective assignment. The per-token advantage $A_{i,t}$ is computed by averaging the relevant task-specific advantages based on which structural element of the output the token belongs to:

$A_{i,t} = \begin{cases} \mathrm{mean}(A_i^{\mathrm{format}}, A_i^{\mathrm{Zoom}}, A_i^{\mathrm{IoU}}) & \text{if } o_{i,t} \in \texttt{<glue>}\cdots\texttt{</glue>} \\ \mathrm{mean}(A_i^{\mathrm{format}}, A_i^{\mathrm{Zoom}}, A_i^{\mathrm{Acc}}) & \text{else} \end{cases}$

where $o_{i,t}$ is the $t$-th token of the $i$-th response.

What this computes: two different advantage values depending on token position. For tokens inside <glue></glue> tags (the temporal grounding prediction), the advantage is the average of format, zoom-in, and IoU advantages — these are the reward signals relevant to temporal localization quality. The answer accuracy advantage $A^{\mathrm{Acc}}$ is deliberately excluded because answer tokens do not contribute to temporal grounding. For tokens outside <glue></glue> tags (answer generation, reasoning, formatting boilerplate), the advantage is the average of format, zoom-in, and answer accuracy advantages. The IoU advantage $A^{\mathrm{IoU}}$ is excluded because general reasoning tokens are not directly responsible for the timestamp prediction.

Why include $A^{\mathrm{Zoom}}$ in both branches: the zoom-in reward depends on both the temporal span (did it crop to the right frames?) and the answer on the cropped frames (was the model able to extract the correct answer?). It is therefore relevant to both the grounding tokens (which determine what gets zoomed into) and the answer tokens (which determine whether the model can use the zoomed-in information). Including it in both branches ensures that both sets of tokens receive feedback about the zoom-in verification outcome.

Why this particular grouping and not other combinations: the paper does not exhaustively ablate alternative token assignment schemes. The design follows a straightforward logic: tokens inside <glue></glue> are solely responsible for temporal grounding, so they receive advantages from rewards that measure grounding quality ($R_{\mathrm{IoU}}$, $R_{\mathrm{Zoom}}$) plus the format reward that ensures structural validity. Tokens outside <glue></glue> are responsible for reasoning and answer generation, so they receive advantages from rewards that measure answer quality ($R_{\mathrm{Acc}}$, $R_{\mathrm{Zoom}}$) plus format. The format advantage is included in both because format compliance is a global property of the entire output — the presence of correctly structured <answer> and <glue> tags depends on all tokens in the sequence.

The downstream effect on the policy gradient. The final GRPO objective (Equation 2) remains unchanged in structure but now uses the token-specific advantage $A_{i,t}$ from Equation 5 rather than the uniform advantage from Equation 1. The likelihood ratio term $\frac{\pi_\theta(o_{i,t})}{\pi_{\theta_{\mathrm{old}}}(o_{i,t})}$ is multiplied by $A_{i,t}$ for each token, meaning:

• Tokens in <glue></glue> are reinforced or penalized based on temporal grounding quality (via $A^{\mathrm{IoU}}$ and $A^{\mathrm{Zoom}}$), independent of answer accuracy.
• Tokens outside <glue></glue> are reinforced or penalized based on answer quality (via $A^{\mathrm{Acc}}$ and $A^{\mathrm{Zoom}}$), independent of IoU.
• The zoom-in advantage $A^{\mathrm{Zoom}}$ couples the two: good grounding without good answering (or vice versa) will not receive the full zoom-in benefit, creating pressure for consistent rollouts where the span and answer are mutually supportive.

Empirical validation. Figure 4 shows training curves comparing baseline GRPO with GRPO + TokenAdv. The key observation is that the IoU reward $R_{\mathrm{IoU}}$ converges faster and to a higher value with TokenAdv, while the answer reward $R_{\mathrm{Acc}}$ follows a similar trajectory to baseline. This suggests that TokenAdv primarily benefits temporal grounding (by preventing the answer accuracy signal from contaminating the grounding token advantages) without harming answer accuracy training. Table 5 quantifies this: TokenAdv alone (without zoom-in reward) improves mIoU from 35.3 to 36.9 on NExT-GQA and from 39.2 to 41.5 on ReXTime, while answer accuracy improves marginally (69.6 → 69.9 on NExT-GQA, 58.3 → 59.8 on ReXTime).

Training Procedure and Data Pipeline

The paper describes a two-stage RL training process with data filtering between stages. This is not the main contribution but is necessary to understand the experimental setup.

Stage I training data. The first stage uses three datasets: NExT-GQA (3,358 queries, average video duration 43.9s, average moment length 8.5s), ActivityNet (4,727 queries, 177.3s average duration, 48.35s average moment), and QVHighlights filtered to videos >120 seconds (7,218 queries, 150.0s average duration, 34.1s average moment). Total: approximately 15K examples.

Stage II data preparation and filtering. After Stage I training, the Stage I model is used for offline data filtering on Stage II datasets. The key insight is that not all training examples provide equally useful learning signals for RL. Two filtering criteria are applied:

For question-answering signal quality: examples where all $n=8$ generated responses answer correctly are discarded because they lack discriminative signal — the model already gets them right and there is no variation in answer accuracy to compute meaningful advantages from.

For temporal grounding signal quality: examples with low response variance are filtered using a relative reward signal metric:

$\delta = \max_{1 \leq i \leq n} \mathrm{IoU}_i - \frac{1}{n}\sum_{i=1}^n \mathrm{IoU}_i$

where $\delta$ measures the difference between the best IoU among rollouts and the mean IoU across rollouts. Examples with $\delta < 0.1$ are filtered out.

What this computes: a measure of how much the model's temporal grounding predictions vary for a given example. If all rollouts produce similar IoU values (low $\delta$), the example provides little relative advantage signal — there is no "good" rollout to reinforce versus "bad" rollout to penalize. If the rollouts vary substantially (high $\delta$), the GRPO advantage mechanism can meaningfully distinguish better and worse temporal grounding behaviors.

Stage II data. After filtering, Stage II uses: NExT-GQA (1,004 queries, 50.2s average duration, 7.1s average moment), ActivityNet (1,395 queries, 220.87s average duration, 88.4s average moment), and PLM-Video (5,333 queries, 808.6s average duration, 26.1s average moment). Total: approximately 7K examples. The PLM-Video dataset is further filtered by a quality check: examples that cannot be correctly answered using the full video but CAN be correctly answered when restricted to the cropped segment defined by the clue duration are removed. This ensures the ground-truth clue duration actually contains the information needed to answer, making the temporal grounding task well-defined.

Training hyperparameters. The paper specifies: base model Qwen2.5-VL-7B, maximum video tokens $L_v = 8192$, video sampling rate 1 FPS during training, minimum video frame resolution $16 \times 28 \times 28$ pixels, maximum video frame resolution $768 \times 28 \times 28$ pixels, maximum response length 512 tokens, global batch size 64, $G = 8$ rollouts per prompt (ablated in Table 9, showing consistent improvement from $G=2$ through $G=8$), and training hardware 8× NVIDIA A100 80GB GPUs.

The per-step computational cost. Without temporal zoom-in, each training step takes 13.29 minutes and consumes 551.2 TFLOPs. With temporal zoom-in (the additional forward pass on cropped, re-encoded frames for $R_{\mathrm{Zoom}}$ computation), each step takes 15.30 minutes and consumes 585.4 TFLOPs — a 1.15× increase in training time.

Inference configurations. For short-video benchmarks (NExT-GQA, ReXTime), models are evaluated at 1 FPS with context size 8,192. For long-video benchmarks (CG-Bench, VideoMME, MLVU, LVBench), a maximum of 256 frames are uniformly sampled with context size 16,384.

Why two-stage training: the paper states this is "due to computational resource limitations." The first stage trains on shorter, more abundant examples to establish basic temporal grounding and answering capability. The second stage incorporates longer videos from PLM-Video (average 808.6 seconds) with filtered, high-variance examples that provide stronger learning signals. The data filtering step is particularly important: it ensures that Stage II training focuses on examples where the model's current policy produces diverse rollouts with meaningful variation in both answer accuracy and temporal grounding quality, maximizing the information content of each GRPO update.

The GRPO objective remains standard in form. Despite the modified advantage computation, the overall optimization objective (Equation 2) does not change. The policy $\pi_\theta$ is still updated via:

$\max_{\pi_\theta} \mathbb{E}_{(q,a),\{o_i\}_{i=1}^G \sim \pi_{\theta_{\mathrm{old}}}(\cdot|q)} \left[\frac{1}{G}\sum_{i=1}^G \frac{1}{|o_i|}\sum_{t=1}^{|o_i|} \left(\frac{\pi_\theta(o_{i,t}|q, o_{i,<t})}{\pi_{\theta_{\mathrm{old}}}(o_{i,t}|q, o_{i,<t})} \cdot A_{i,t} - \beta D_{\mathrm{KL}}(\pi_\theta \| \pi_{\mathrm{ref}})\right)\right]$

The only change is that $A_{i,t}$ now comes from the token-selective Equation 5 rather than the uniform Equation 1. The KL-divergence penalty $\beta D_{\mathrm{KL}}(\pi_\theta \| \pi_{\mathrm{ref}})$ remains, preventing the policy from deviating too far from the reference (pretrained) Qwen2.5-VL distribution — this is standard GRPO practice and ensures the model retains general video understanding capabilities while specializing in temporal grounding.

Summary of Design Choices and Their Justifications

• Four separate rewards rather than a single composite: enables TokenAdv's per-reward advantage computation, which requires that each reward type can be normalized independently across rollouts. A composite reward would collapse the distinct signals and reintroduce the uniform credit assignment problem.

• Zoom-in reward as binary rather than continuous: answers are either correct or incorrect (MCQ format), making binary reward natural. Continuous rewards (e.g., soft accuracy based on token probability) would introduce variance without clear benefit for this task.

• TokenAdv's conditional assignment based on <glue> tags rather than learned segmentation: exploits the format reward's structural output constraint to provide reliable token-level segmentation without requiring additional annotation, model components, or training objectives. This is a pragmatic design that makes the method immediately applicable to any model trained with the specified output format.

• Two-stage training with offline data filtering rather than single-stage on all data: maximizes the information content of each GRPO update by focusing on examples where the current policy produces diverse rollouts with meaningful reward variation. The $\delta < 0.1$ filtering threshold is empirically chosen and not ablated, representing a sensitivity parameter that could be tuned.

• $G=8$ rollouts (ablated from $G=2, 4, 8$): Table 9 shows consistent improvement with more rollouts — more comparisons per group provide finer-grained advantage estimates. The paper does not test beyond $G=8$, likely due to computational constraints (each additional rollout requires an additional forward pass for the zoom-in verification).

• 8192 token context budget: the standard setting for Qwen2.5-VL-7B, providing a realistic constraint under which the coarse-to-fine mechanism demonstrates value. Without the context budget constraint, there would be no need for zoom-in — the model could simply encode all frames at maximum resolution.

• 1 FPS sampling during training: balances temporal coverage (enough frames to capture events) with computational efficiency. The paper does not ablate this, but notes that inference uses different sampling strategies per benchmark (1 FPS for short videos, 256-frame uniform sampling for long videos).

4. Key Insights and Innovations

Innovation 1: The Zoom-In Accuracy Reward as a Verification Mechanism That Connects Temporal Grounding to Answer Faithfulness

The paper's most conceptually distinctive contribution is not the zoom-in mechanism itself — cropping and re-encoding video segments at higher resolution is a natural inference-time strategy — but rather the insight that this zoom-in can be converted into a training signal that verifies the faithfulness of temporal grounding predictions. This represents a shift in how the field evaluates grounding quality: from measuring geometric overlap with ground-truth annotations (IoU) to measuring functional adequacy (can you answer the question from this segment?).

What the field did before. Prior GRPO-based methods for video grounding — Time-R1, TVG-R1, TimeZero, VideoChat-R1 — all used IoU as their primary (and often sole) temporal grounding reward. IoU measures how well the predicted timestamp overlaps with a human-annotated ground-truth segment. This is a surface-level metric: it tells you whether the model's prediction matches the annotation, not whether the prediction captures the visual evidence actually needed for reasoning. A model could achieve high IoU by learning to predict segment boundaries from spurious patterns — question phrasing, video length heuristics, or superficial visual cues — without ever verifying that the cropped segment contains answer-relevant information. Conversely, a prediction with moderate IoU might actually contain all necessary evidence but be penalized for not matching the annotation boundaries exactly.

What makes this contribution distinctive. The zoom-in accuracy reward $R_{\mathrm{Zoom}}$ changes the evaluation criterion from does the predicted span look like the annotation? to does the predicted span contain the answer? This is a subtle but fundamental reframing. It connects the temporal grounding objective to the question-answering objective through a verification step: the model only receives $R_{\mathrm{Zoom}} = 1$ if (a) the predicted span crops to frames that genuinely contain the visual evidence, AND (b) the model can extract the correct answer from those frames at higher resolution. This creates a training signal that no prior GVQA method provided — a direct test of whether the grounding is faithful rather than merely geometrically similar to ground truth.

The significance goes beyond the specific reward design. This verification mechanism transforms temporal grounding from a standalone prediction task (where the model outputs timestamps and is scored against annotations) into a coupled task where grounding quality is evaluated by its downstream consequence: does the localized segment enable correct reasoning? This is a more principled objective because it aligns the training signal with the actual purpose of grounding in GVQA — providing the visual evidence needed to answer the question. The paper's Figure 1 concretizes this: in the running example with the "29%" text, a span that covers the general temporal region of the relevant event but misses the specific frame with "29%" would achieve decent IoU (it overlaps with the annotated segment) but fail the zoom-in verification (the cropped frames don't contain the answer). The zoom-in reward penalizes this failure directly, while IoU-based rewards would not.

Empirical evidence for the claim. The ablation in Table 5 shows that adding $R_{\mathrm{Zoom}}$ to the baseline GRPO (with format + IoU + answer accuracy rewards) improves both answer accuracy and temporal grounding. On NExT-GQA, accuracy increases from 69.6% to 70.4% and mIoU from 35.3 to 36.3. On ReXTime, accuracy increases from 58.3% to 60.2% and mIoU from 39.2 to 40.9. This mutual improvement — where a verification reward designed for temporal grounding also improves answer accuracy — is the paper's key empirical signature. It occurs because $R_{\mathrm{Zoom}}$ creates selection pressure for internally consistent rollouts: the model cannot receive the zoom-in reward without both accurate localization AND correct answering on the localized frames. In the rollout group, responses where the predicted span and answer are mutually supportive gain an additional reward component that responses with high IoU but incorrect answers (or correct answers but poor IoU) do not receive.

The dependency on base capability as a boundary condition. The paper is explicit that this mechanism works because Qwen2.5-VL already possesses non-trivial temporal grounding ability (20.2% mIoU on NExT-GQA without grounding-specific training). If the base model's temporal spans were essentially random — if no rollout in the group ever cropped to frames containing the answer — the zoom-in reward would be dominated by noise and provide no useful training signal. This means the zoom-in reward is not a general-purpose temporal grounding training mechanism that can bootstrap from zero; it is an amplification mechanism that sharpens and verifies an existing capability. The paper does not claim otherwise, but this boundary condition is important for understanding the scope of the contribution: the method requires a pretrained model with some baseline grounding ability to be effective.

Why this is more than an incremental engineering trick. Adding a second forward pass to verify predictions is a straightforward idea. What makes this contribution intellectually substantive is the reframing it enables: temporal grounding quality is redefined from geometric overlap to functional adequacy, and the training objective is restructured to optimize for the latter. This mirrors a broader trend in machine learning where evaluation metrics that are easy to compute (IoU, BLEU, ROUGE) are supplemented or replaced by metrics that test functional competence (can the system use its output to accomplish the downstream task?). The zoom-in accuracy reward is an instance of this principle applied to video temporal grounding, and its effectiveness suggests that similar verification-based training signals could benefit other structured prediction tasks where geometric metrics capture only surface-level alignment.

Innovation 2: Token-Selective Credit Assignment as a Diagnosis and Solution to GRPO's Uniform Credit Problem Under Multi-Faceted Rewards

The paper identifies a specific failure mode in standard GRPO when applied to tasks with multiple distinct reward signals — uniform credit assignment — and proposes a structurally simple but conceptually important fix: compute separate advantages per reward type and assign them to the tokens responsible for each capability. This is not merely a hyperparameter tuning or reward weighting contribution; it is a diagnostic insight about GRPO's limitations backed by a concrete counterexample (Appendix C, Table 8) that demonstrates how naive reward summation can produce perverse optimization incentives.

What the field did before. Standard GRPO, as introduced by Shao et al. (2024) and adopted by prior video grounding work (VideoChat-R1, TVG-R1), handles multi-faceted rewards by summing them into a single scalar. The advantage $A_{i,t}$ is then computed from this summed reward and assigned uniformly to every token in the response. This means a response with excellent temporal grounding but poor answer accuracy (or vice versa) receives an ambiguous blended signal — all tokens get reinforced or penalized together, regardless of which tokens contributed to which aspect of performance. Prior work did not identify this as a problem requiring a structural solution; it was simply the default GRPO formulation applied to multi-reward tasks.

What makes this contribution distinctive. The paper provides both a diagnosis (Appendix C, Table 8, with the concrete numerical example showing Response 1 with $R_{\mathrm{IoU}} = 0.0$ receiving positive overall advantage while Response 4 with $R_{\mathrm{IoU}} = 0.8$ receives negative advantage) and a targeted solution (TokenAdv) that exploits the structured output format to disentangle credit assignment. The diagnosis is valuable independently of the solution: it explains why naively applying GRPO to GVQA can lead to suboptimal grounding performance, and it provides a framework for thinking about credit assignment in any multi-capability RL fine-tuning setting.

The solution — computing separate advantages per reward type and assigning them conditionally based on token position — is elegant in its simplicity. It requires no architectural changes, no additional model components, and no new loss terms. It operates entirely within the existing GRPO framework by modifying only the advantage computation step. The key insight that enables this simplicity is that the structured output format (with <glue></glue> and <answer></answer> tags), which is already required for the format reward and for evaluation, provides natural token-level segmentation between capabilities. The paper does not need to learn which tokens correspond to grounding versus answering — the format constraint already enforces this separation, and TokenAdv simply exploits it.

Why this matters beyond performance gains. The significance of this contribution is not primarily the metric improvements (+1.6 mIoU on NExT-GQA from adding TokenAdv to baseline GRPO) but rather the conceptual framework it establishes for RL-based multi-capability training. The paper demonstrates that when multiple reward signals are present, it is not sufficient to treat them as a single composite objective — the credit assignment structure matters. Tokens responsible for different capabilities should receive different advantage signals, because reinforcing or penalizing all tokens uniformly based on blended rewards creates interference between learning objectives. This is a general principle that applies to any RL fine-tuning setting where the model's output has distinguishable structural components with distinct evaluation criteria (e.g., code generation with separate correctness and style rewards, summarization with separate factual accuracy and fluency rewards).

The paper's conditional assignment rule (Equation 5) provides a template for how to structure credit assignment when output format constraints exist. The choice to include $A^{\mathrm{Zoom}}$ in both branches is deliberate and principled: the zoom-in reward depends on both the grounding quality (which determines what frames are cropped) and the answering quality (which determines whether the model extracts the correct answer), so it is relevant to both token groups. This design choice reflects an understanding of the causal structure of the task — which capabilities causally influence which rewards — rather than a heuristic grouping.

Empirical evidence and the training dynamics. Figure 4 provides training curves showing that TokenAdv accelerates and improves IoU reward convergence without degrading answer reward trajectories. This is evidence that the credit assignment structure successfully reduces interference: the grounding tokens receive cleaner signals about grounding quality (uncontaminated by answer accuracy variation), enabling faster improvement in temporal localization. The answer tokens are not harmed because they still receive answer accuracy advantages — the answer quality signal is simply not diluted by IoU variation in tokens that don't affect IoU.

Distinguishing incremental from fundamental. TokenAdv is a refinement of GRPO's advantage computation rather than a new RL algorithm. However, the underlying insight — that uniform credit assignment under multi-faceted rewards creates perverse incentives that can be resolved by reward-type-specific advantage estimation and token-selective assignment — is a fundamental contribution to the RL-for-LLM-fine-tuning methodology. It identifies a structural limitation of standard GRPO that had not been previously characterized and provides a principled solution that generalizes beyond the specific GVQA setting. The paper's concrete counterexample (Table 8) makes this limitation salient and actionable for practitioners.

Innovation 3: The Coarse-to-Fine Zoom-In Paradigm as an Inference-Time Strategy That Converts Context Budget Constraints from Limitations into Active Mechanisms

The paper's third contribution is demonstrating that the temporal grounding capability developed during RL training enables an inference-time strategy — coarse-to-fine zoom-in — that systematically improves long-video understanding by converting the context budget constraint from a fixed limitation into a mechanism for detail recovery. This is not a new training algorithm or reward design, but rather a deployment paradigm that exploits a learned capability (query-conditioned temporal localization) to allocate visual processing resources adaptively.

What the field did before. Prior work on long-video understanding addressed the context budget problem through four general strategies: (1) uniform temporal downsampling (sparsely sampled frames, as in LLaVA-OneVision and Qwen2.5-VL), (2) token pooling or compression (VideoChatGPT, LongVU), (3) memory aggregation across time (MA-LMM), and (4) retrieval-based frame selection (VideoRAG, V-Gent). All of these approaches make a fixed allocation decision — which frames to include at what resolution — before the model processes the query. The allocation is either uniform or based on pre-computed relevance scores, not on the model's own query-conditioned reasoning about which temporal segments matter.

What makes this contribution distinctive. Zoom-Zero's coarse-to-fine paradigm introduces a fundamentally different approach: query-conditioned adaptive allocation via learned temporal grounding. The model first processes the full video at whatever resolution the context budget permits for broad temporal coverage (coarse pass), produces an explicit prediction of which temporal segments are relevant to the specific question, and then re-processes only those segments at higher spatial resolution (fine pass). This is not frame selection in embedding space (which operates on pre-computed frame features and selects a fixed set) — it is temporal reasoning in the output space, where the model articulates which time intervals contain the answer and then verifies by looking more carefully.

The crucial distinction is that the allocation decision is internal to the model's reasoning process and specific to each query. A question about "what color was the car at 2:30?" and a question about "what did the person say at 5:00?" will cause the model to zoom into completely different temporal segments, even for the same video. This contrasts with uniform downsampling (same frames for all queries), fixed retrieval (same retrieved frames for all queries), or learned compression (same compressed representation regardless of query). The coarse-to-fine paradigm makes the model's visual processing query-dependent in a way that prior long-video understanding approaches do not.

The role of trained capability. This inference-time strategy only works because the RL training (with zoom-in accuracy reward and token-selective credit assignment) has taught the model to produce reliable query-conditioned temporal spans. Table 13 quantifies the practical effect: on VideoMME Long videos (2386s average duration), the coarse pass uses 256 frames at 0.1 FPS with 54 tokens/frame, while the fine pass zooms into 136 frames at 1.0 FPS with 76 tokens/frame — fewer total frames but substantially higher per-frame resolution focused on the relevant interval. This is a direct realization of the context budget reallocation principle: the model trades unnecessary temporal breadth for necessary spatial depth.

Empirical evidence. Table 4 shows that the coarse-to-fine inference strategy provides consistent improvements over the one-stage Zoom-Zero baseline across long-video benchmarks: +2.0 points on VideoMME (long with subtitles, from 64.2 to 66.2), +0.6 on MLVU (70.8 to 71.4), +0.6 on LVBench (45.7 to 46.3), and +2.9 on CG-Bench (36.1 to 39.0). The divide-and-conquer strategy — partitioning the video into windows, predicting spans per window, aggregating high-confidence spans, and zooming in — provides even larger gains (+6.4 average across benchmarks), at the cost of 2.3× inference time (Table 6).

The trade-off framing. The paper is transparent about the cost: coarse-to-fine inference increases latency by approximately 1.4× (Table 6), and divide-and-conquer by approximately 2.3×. This is not a free lunch — it is a deliberate compute-accuracy trade-off where additional inference computation is invested in higher-resolution processing of query-relevant frames. The paper frames this not as a drawback but as a deployment choice: one-stage inference provides solid performance with baseline latency; coarse-to-fine provides meaningful gains at moderate cost; divide-and-conquer provides maximum gains at higher cost. This is analogous to test-time compute scaling strategies (best-of-N, beam search) in language models, where additional inference computation buys improved performance, and the user selects the operating point based on their accuracy-latency requirements.

Why this matters as an architectural principle. The coarse-to-fine zoom-in paradigm suggests a broader design principle for video-language models: rather than treating the visual processing budget as a fixed pre-allocation to be optimized once and applied uniformly, models should be designed to make query-conditioned, multi-pass allocation decisions where an initial coarse understanding guides subsequent fine-grained processing. This principle connects to cognitive science (which the paper briefly references): human visual cognition operates via rapid coarse scene understanding followed by focused attention on task-relevant regions. Zoom-Zero implements a computational analog of this attentional mechanism for video understanding, using learned temporal grounding as the attention director. This is a conceptual contribution that extends beyond the specific GVQA setting — it suggests a general architecture for efficient video processing where models learn to allocate their own visual compute budget adaptively based on the query.

A Distinction Worth Making: Incremental vs. Fundamental Contributions

Of the three innovations, the zoom-in accuracy reward (Innovation 1) and the coarse-to-fine inference paradigm (Innovation 3) represent fundamental conceptual shifts in how temporal grounding quality is evaluated and how visual processing budgets are allocated. They change the problem formulation rather than merely improving an existing solution. The token-selective credit assignment (Innovation 2) is best characterized as a methodological refinement with broad applicability — it identifies and solves a structural limitation of standard GRPO that is relevant beyond this specific task, but it operates within the existing GRPO framework rather than replacing it. All three are substantive contributions, but they differ in their level of generality: the zoom-in verification principle and the coarse-to-fine allocation paradigm are likely to influence how future video-language models are designed and trained, while TokenAdv provides a practical fix for a specific GRPO limitation that practitioners can immediately adopt.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary GVQA evaluation uses NExT-GQA (xiao2024can) with its standard test split (Section 5.1). Additional GVQA results are reported on ReXTime (chen2024rextime) and CG-Bench (chen2024cg). Long-video understanding is evaluated on VideoMME (fu2024video), MLVU (zhou2024mlvu), LVBench (wang2024lvbench), and the long-form GQA portion of CG-Bench. All datasets are publicly available. Training data statistics (with video durations and moment lengths) are summarized in Appendix A, Table 7.

• Base model. All experiments use Qwen2.5-VL-7B (bai2025qwen2) as the base model. The paper states this model is chosen because it possesses non-trivial temporal grounding ability (20.2% mIoU on NExT-GQA in zero-shot evaluation, per Table 1) — a necessary condition for the zoom-in mechanism to provide useful training signal — while being of comparable scale (7B) to other open-source baselines (LLaVA-OneVision 7B, InternVL2.5 8B). The model uses dynamic spatiotemporal resolution with a maximum visual token budget of 8192 during training and 16384 for long-video inference.

• Metrics. The paper reports multiple metrics organized by task. For temporal grounding: mean Intersection-over-Union (mIoU), R@0.3 (percentage of predictions with IoU > 0.3), and R@0.5 (IoU > 0.5). For grounded question answering: Acc@GQA (the percentage of questions that are correctly answered AND visually grounded with IoP ≥ 0.5, where IoP is intersection over prediction span). For standard question answering: multiple-choice question (MCQ) accuracy. For CG-Bench specifically: rec.@IoU (average recall across IoU thresholds {0.1, 0.2, 0.3, 0.4, 0.5}) and acc.@IoU (average accuracy across the same thresholds, counting a response as correct only if both answer and IoU threshold are satisfied). Coverage is measured by mean Intersection-over-Ground truth (mIoG), defined as $|\mathcal{I}_{\mathrm{pred}} \cap \mathcal{I}_{\mathrm{gt}}| / |\mathcal{I}_{\mathrm{gt}}|$.

• Baselines. The paper compares against three categories. SFT-based LVLMs with grounding: TimeChat (ren2023timechat), VTimeLLM (huang2023vtimellm), Grounded-VideoLLM (wang2024grounded). RL-based methods: VideoChat-TPO (li2025temporal), TVG-R1 (chen2025datasets), VideoChat-R1 (li2025videochat). General-purpose LVLMs: LLaVA-OneVision (li2024llava), Qwen2.5-VL (bai2025qwen2), InternVL2.5 (chen2024expanding). Proprietary models (Gemini 1.5 Pro, GPT-4o) are reported for context on long-video benchmarks but not treated as direct competitors. All open-source models are of comparable scale (7B or 8B parameters).

• Generation budget / compute accounting. The paper measures computational cost in several ways. For training: per-step time (minutes) and TFLOPs, with and without temporal zoom-in (Section 5.5). For inference: average inference time per video in seconds, measured across three operating points — one-stage inference, coarse-to-fine (1.4× latency), and divide-and-conquer (2.3× latency) — as reported in Table 6. The number of rollouts per prompt during GRPO training is $G=8$, selected via ablation (Appendix D.2, Table 9) showing consistent improvement from $G=2$ through $G=8$.

• Cross-validation / statistical protocol. The paper does not employ cross-validation. For NExT-GQA, RL methods (VideoChat-R1, TVG-R1, and Zoom-Zero) are trained on the validation split and evaluated on the test split; SFT-based models are evaluated zero-shot on the test split. For ReXTime and CG-Bench, all methods are evaluated strictly zero-shot "ensuring a valid and fair comparison across methods" (Section 5.1). For ReXTime mIoG, which requires ground-truth spans not released for the test set, results are reported on the validation split (Table 3). Statistical significance tests (confidence intervals, standard errors) are not reported for any result.

Main Quantitative Results

Grounded Video Question Answering (Table 1)

The headline result on NExT-GQA: Zoom-Zero achieves 29.0% Acc@GQA, 37.6% mIoU, 55.6% R@0.3, and 33.8% R@0.5. This represents improvements of +4.7 percentage points in Acc@GQA, +5.2 in mIoU, +5.4 in R@0.3, and +6.1 in R@0.5 over the strongest RL baseline (VideoChat-R1 at 24.3 / 32.4 / 50.2 / 27.7). The gains over the base Qwen2.5-VL model are even larger: +10.1 Acc@GQA, +17.4 mIoU, +24.0 R@0.3, +15.7 R@0.5.

On ReXTime: Zoom-Zero achieves 62.0% accuracy, 43.2% mIoU, 56.5% R@0.3, and 44.1% R@0.5, improving over VideoChat-R1 (58.1 / 38.6 / 50.6 / 39.0) by +3.9 accuracy, +4.6 mIoU, +5.9 R@0.3, and +5.1 R@0.5.

A notable pattern across both benchmarks: the improvements are larger on stricter metrics. On NExT-GQA, the gain over VideoChat-R1 is 5.2 points on mIoU but 6.1 points on R@0.5, and on ReXTime the gain is 4.6 points on mIoU but 5.1 points on R@0.5. This suggests Zoom-Zero is not merely shifting the IoU distribution upward uniformly but is specifically improving the model's ability to make precise, high-overlap temporal predictions rather than approximately correct ones.

The gap between Zoom-Zero and SFT-based methods is substantial. Grounded-VideoLLM — the strongest SFT baseline — achieves 26.7% Acc@GQA on NExT-GQA compared to Zoom-Zero's 29.0%, but its mIoU (21.1) and R@0.3/R@0.5 metrics (not fully reported for NExT-GQA) are dramatically lower than Zoom-Zero's. TimeChat, despite being explicitly designed for temporal grounding, achieves only 7.6% Acc@GQA with 20.6% mIoU — evidence that SFT alone cannot reliably produce temporally grounded correct answers, even when the model learns to output plausible-looking timestamps.

Temporal Grounding Coverage Analysis (Table 3)

Beyond IoU, the paper evaluates how well predicted spans cover the ground-truth evidence via mean Intersection-over-Ground truth (mIoG) and mean Intersection-over-Prediction (mIoP). On NExT-GQA: Zoom-Zero achieves 94.7% mIoG and 43.2% mIoP, compared to VideoChat-R1's 93.5% mIoG and 39.1% mIoP, and Qwen2.5-VL's 56.8% mIoG and 29.5% mIoP. On ReXTime (validation): Zoom-Zero achieves 67.6% mIoG and 53.5% mIoP versus VideoChat-R1's 64.3% and 52.8%.

The mIoG metric reveals that both Zoom-Zero and VideoChat-R1 achieve very high coverage of the ground-truth span (94.7% and 93.5% on NExT-GQA), meaning their temporal predictions rarely miss large portions of the relevant evidence. The key differentiator is mIoP — how much of the predicted span is actually relevant. Zoom-Zero's higher mIoP (43.2% vs. 39.1%) indicates that its predictions are more precise and include less irrelevant footage. This precision matters for the zoom-in mechanism: tighter predictions mean fewer extraneous frames in the crop, allowing higher per-frame resolution under the fixed context budget.

On CG-Bench, the coverage advantage is dramatic: 26.15% mIoG and 8.25% mIoP for Zoom-Zero versus 18.44% mIoG and 7.34% mIoP for VideoChat-R1, and 10.35% mIoG for base Qwen2.5-VL. The mIoG improvement of 7.71 percentage points over VideoChat-R1 is particularly meaningful for CG-Bench because its videos are extremely long (4101s average) with clues typically occupying ≤1% of total duration — even approximately correct temporal predictions can miss the evidence entirely. Zoom-Zero's substantially higher mIoG indicates that the zoom-in accuracy reward $R_{\mathrm{Zoom}}$ creates effective selection pressure for predictions that genuinely cover the answer-relevant frames, even when those frames constitute a tiny fraction of the video.

Long Video Understanding (Table 2)

On long-video benchmarks, Zoom-Zero (one-stage inference) achieves: VideoMME (long with subtitles) 71.2%, MLVU 64.2%, LVBench 70.8%, CG-Bench accuracy 45.7%. These represent improvements over the base Qwen2.5-VL of +0.5 VideoMME, +2.2 MLVU, +0.6 LVBench, and +0.4 CG-Bench (note: the paper reports Qwen2.5-VL as 70.7 / 62.0 / 70.2 / 45.3; the MLVU 62.0 figure appears in Table 2 as the "w/ sub." column while the main Zoom-Zero comparison is to 62.0).

The more significant comparison in Table 2 is between Zoom-Zero and the RL baselines TVG-R1 and VideoChat-R1. Both RL baselines show degraded general video understanding compared to Qwen2.5-VL: VideoChat-R1 achieves 69.1% on VideoMME (vs. Qwen2.5-VL's 70.7%, a drop of 1.6 points), 62.3% on MLVU (vs. 62.0%, effectively unchanged), 69.5% on LVBench (vs. 70.2%, a drop of 0.7 points), and 43.7% on CG-Bench (vs. 45.3%, a drop of 1.6 points). This quantifies the accuracy-grounding trade-off that the paper identifies as a limitation of prior RL approaches.

Zoom-Zero, by contrast, improves over Qwen2.5-VL on all long-video benchmarks while simultaneously achieving better grounding (Table 1). On VideoMME, Zoom-Zero reaches 71.2% versus Qwen2.5-VL's 70.7% and VideoChat-R1's 69.1%. On MLVU, 64.2% versus 62.0% and 62.3%. On LVBench, 70.8% versus 70.2% and 69.5%. On CG-Bench accuracy, 45.7% versus 45.3% and 43.7%. This is the paper's key demonstration that token-selective credit assignment successfully mitigates the accuracy-grounding trade-off: the model simultaneously improves temporal grounding (Table 1) and maintains or improves general video understanding (Table 2), unlike prior RL methods that sacrifice one for the other.

On CG-Bench's grounding metrics: Zoom-Zero achieves 6.68% mIoU, 9.30% rec.@IoU, and 3.62% acc.@IoU, compared to VideoChat-R1's 5.91 / 8.38 / 2.56 and Qwen2.5-VL's 2.48 / 3.15 / 1.36. The rec.@IoU improvement of +0.92 over VideoChat-R1 and +6.15 over Qwen2.5-VL is substantial in absolute terms for this challenging benchmark where clue intervals are extremely short relative to video duration.

Long Video Understanding via Temporal Zoom-In (Table 4)

The coarse-to-fine inference strategy provides consistent improvements over one-stage Zoom-Zero: VideoMME +2.0 (64.2 → 66.2), MLVU +0.6 (70.8 → 71.4), LVBench +0.6 (45.7 → 46.3), CG-Bench +2.9 (36.1 → 39.0). These gains demonstrate that even when the model has already been trained with the zoom-in reward — and thus has learned to produce reliable temporal spans — there is additional benefit to applying the zoom-in at inference time on long videos. The coarse-to-fine pass enables the model to recover fine-grained visual details that are lost in the one-stage encoding due to the context budget constraint.

The divide-and-conquer strategy yields larger gains at higher cost: VideoMME +4.5 (64.2 → 68.7), MLVU +2.6 (70.8 → 73.4), LVBench +2.4 (45.7 → 48.1), CG-Bench +6.1 (36.1 → 42.2). The average improvement across these four benchmarks is +6.4% over the base Qwen2.5-VL. The divide-and-conquer strategy is particularly effective on CG-Bench (+6.1 points) — given that CG-Bench videos average 4101 seconds with clues occupying ≤1% of duration, scanning windows to identify candidate spans and then zooming into high-confidence ones is well-matched to the task structure. The paper reports (Appendix D.3, Table 10) that a window size of 256 seconds is optimal, and that aggregating 4 predicted spans with the highest answer confidence (Appendix D.3, Table 11) provides the best performance.

An important negative space in these results: Table 2 also reports performance of proprietary models (Gemini 1.5 Pro at 77.4 on VideoMME long w/ sub., GPT-4o at 72.1) that substantially exceed Zoom-Zero. The paper does not claim to match these models; the comparison to Qwen2.5-VL and other open-source 7B-8B models establishes the relevant performance envelope.

Ablation Studies and Robustness Checks

Impact of each component (Table 5): The systematic ablation on NExT-GQA and ReXTime builds from the base Qwen2.5-VL through four configurations. Qwen2.5-VL baseline achieves 53.3% accuracy / 20.2% mIoU / 31.6% R@0.3 / 18.1% R@0.5 on NExT-GQA, and 51.1% / 27.4% / 36.1% / 24.8% on ReXTime. Adding GRPO with format + IoU + answer accuracy rewards (+ GRPO $R_{\mathrm{format}} + R_{\mathrm{IoU}} + R_{\mathrm{Acc}}$) substantially improves all metrics: NExT-GQA accuracy jumps to 69.6% (+16.3) and mIoU to 35.3% (+15.1); ReXTime accuracy to 58.3% (+7.2) and mIoU to 39.2% (+11.8). This confirms that RL-based training is dramatically more effective than the base model's zero-shot grounding. Adding TokenAdv to this configuration (+ GRPO + TokenAdv) improves grounding: mIoU 35.3 → 36.9 on NExT-GQA (+1.6) and 39.2 → 41.5 on ReXTime (+2.3), with marginal accuracy gains (69.6 → 69.9 and 58.3 → 59.8 respectively). Adding the zoom-in reward without TokenAdv (+ GRPO + $R_{\mathrm{Zoom}}$) produces smaller grounding gains than TokenAdv alone: mIoU 35.3 → 36.3 on NExT-GQA (+1.0) and 39.2 → 40.9 on ReXTime (+1.7), but larger accuracy gains: 69.6 → 70.4 (+0.8) and 58.3 → 60.2 (+1.9). The full configuration (+ GRPO + $R_{\mathrm{Zoom}}$ + TokenAdv) achieves the best performance on every metric: NExT-GQA 70.7% accuracy (+1.1 over baseline GRPO) and 37.6% mIoU (+2.3); ReXTime 62.0% accuracy (+3.7) and 43.2% mIoU (+4.0).

The key interaction pattern: TokenAdv provides the largest grounding improvement while $R_{\mathrm{Zoom}}$ provides the largest accuracy improvement, and the combination is additive (or slightly super-additive) — the gains from combining them (e.g., +2.3 mIoU on NExT-GQA) approximately equal the sum of individual contributions (+1.6 from TokenAdv, +1.0 from $R_{\mathrm{Zoom}}$). This supports the paper's claim that the two innovations address orthogonal limitations of standard GRPO.

Number of generated responses G (Appendix D.2, Table 9): Increasing $G$ from 2 to 4 to 8 consistently improves all metrics on both NExT-GQA and ReXTime. On NExT-GQA with $G=2$: 69.6% accuracy, 33.7% mIoU, 50.3% R@0.3, 27.5% R@0.5. $G=4$: 69.8 / 35.2 / 52.6 / 29.6. $G=8$: 70.7 / 37.6 / 55.6 / 33.8. The mIoU gain from $G=2$ to $G=8$ is 3.9 points; from $G=4$ to $G=8$ is 2.4 points. On ReXTime, the pattern is similar: mIoU at $G=2$ is 36.9%, $G=4$ is 40.1%, $G=8$ is 43.2%. The paper adopts $G=8$ for all main experiments, noting it "provides the optimal balance between computational efficiency and performance gains." The diminishing returns from $G=4$ to $G=8$ (2.4 mIoU points on NExT-GQA, 3.1 on ReXTime) versus $G=2$ to $G=4$ (1.5 and 3.2 respectively) suggest that further increases beyond $G=8$ might yield additional gains, though the paper does not test this due to computational constraints.

Divide-and-conquer window size (Appendix D.3, Table 10): Window sizes of 128, 256, and 384 seconds are tested with the divide-and-conquer strategy. The reported results: at 128 seconds — VideoMME long w/ sub. 67.4%, MLVU 72.1%, LVBench 47.6%; at 256 seconds — 68.7 / 73.4 / 48.1; at 384 seconds — 68.4 / 72.4 / 48.5. Window size 256 is optimal on all three benchmarks, suggesting a sweet spot: windows that are too small (128s) may fragment temporal context and miss cross-window events; windows that are too large (384s) may include too many frames per window, reducing per-frame resolution in the coarse pass and degrading span prediction quality. The paper uses 256-second windows for the reported divide-and-conquer results.

Number of aggregated spans in divide-and-conquer (Appendix D.3, Table 11): Aggregating 3, 4, or 5 predicted temporal spans with top answer confidence yields: at 3 spans — VideoMME 73.2%, MLVU 66.8%, LVBench 47.5%; at 4 spans — 73.4 / 68.7 / 48.1; at 5 spans — 73.2 / 68.4 / 47.9. Four spans is optimal, with the key effect visible on MLVU (68.7% vs. 66.8% for 3 spans). Aggregating 5 spans degrades performance slightly, presumably because lower-confidence spans introduce noisy frames that dilute the high-resolution detail in the zoom-in pass.

Temporal grounding robustness across clue durations (Figure 3): The paper analyzes mIoU as a function of ground-truth clue duration and clue proportion (relative to total video duration) on NExT-GQA. The left panel shows that shorter ground-truth durations are harder for all methods — mIoU drops monotonically as clue duration decreases. Zoom-Zero (orange line) consistently outperforms the GRPO baseline (blue line) across all duration ranges, with the gap widening at very short durations (0-5 seconds). The right panel shows the same pattern by clue proportion: Zoom-Zero maintains a lead across all proportions, with the advantage most pronounced when clues occupy a very small fraction of the video (0-10%). This demonstrates robustness to the specific challenge of localizing brief events in longer videos, which is precisely the scenario where the zoom-in mechanism should be most valuable.

Training curve comparison (Figure 4): The training curves plot $R_{\mathrm{IoU}}$ and $R_{\mathrm{Acc}}$ reward trajectories for baseline GRPO versus GRPO + TokenAdv. The IoU reward with TokenAdv converges faster and to a higher final value, while the answer accuracy reward follows a nearly identical trajectory in both conditions. This visually confirms that TokenAdv primarily benefits temporal grounding without degrading answer accuracy training — the key claim of Section 4.3.

One-stage vs. coarse-to-fine inference on short GVQA (Appendix D.4, Table 12): On NExT-GQA, Zoom-Zero achieves 70.7% accuracy; adding coarse-to-fine inference yields 71.4% (+0.7). On ReXTime, Zoom-Zero achieves 62.0%; with coarse-to-fine, 62.8% (+0.8). The paper notes that short videos "can preserve most temporal context and details within the context budget," so the zoom-in provides only marginal benefit — the model already sees most frames at reasonable resolution in the one-stage setting. The zoom-in mechanism's value is most apparent on long videos where the context budget constraint is binding.

Spatial and temporal resolution before and after zoom-in (Appendix D.4, Table 13): On VideoMME Long (2386s average duration), the coarse pass uses 256 frames at 0.1 FPS with 54 tokens/frame, while the fine pass zooms into 136 frames at 1.0 FPS with 76 tokens/frame. On MLVU (651s duration): coarse pass 253 frames at 0.2 FPS with 64 tokens/frame; fine pass 86 frames at 1.0 FPS with 190 tokens/frame. On LVBench (4101s duration): coarse pass 256 frames at 0.06 FPS with 64 tokens/frame; fine pass 154 frames at 1.0 FPS with 62 tokens/frame. The per-frame token increase is most dramatic on MLVU (64 → 190, a 2.97× increase), reflecting that MLVU videos are shorter (651s average) so even in the fine pass, enough budget exists for substantial per-frame resolution. On LVBench, the per-frame token increase is minimal (64 → 62), likely because its extremely long videos (4101s) require even the fine pass to cover many frames, limiting per-frame resolution gains.

Critical Assessment

The experimental results provide strong support for the paper's central claim that zoom-in accuracy reward and token-selective credit assignment improve temporal grounding over standard GRPO. The ablation (Table 5) cleanly isolates the contribution of each component, and the additive pattern — TokenAdv improves grounding, $R_{\mathrm{Zoom}}$ improves accuracy, both improve together — is internally consistent with the paper's mechanistic explanation. The improvements hold across two distinct GVQA benchmarks (NExT-GQA and ReXTime) with consistent effect sizes (+5.2 and +4.6 mIoU respectively), and generalize to long-video grounding on CG-Bench.

The claim that the coarse-to-fine paradigm improves long-form video understanding is supported with an important boundary condition: the benefits are most visible when the divide-and-conquer strategy is applied (Table 4, +6.4% average), while the simple coarse-to-fine strategy provides more modest gains (+1.0% average). The divide-and-conquer strategy's 2.3× inference cost (Table 6) means these gains come at a substantial latency premium, and the paper doesn't ablate whether simpler test-time strategies (e.g., simply sampling more frames at higher resolution and using a longer context) might achieve comparable benefits at similar cost. The comparison to Qwen2.5-VL's one-stage baseline with 16384 token context is the relevant counterfactual, but the paper doesn't report Qwen2.5-VL performance with equivalent total inference compute (e.g., processing the full video at higher resolution with a proportionally larger context budget), making it unclear whether the gains are specifically due to the coarse-to-fine mechanism or to the additional inference computation generally.

The claim that token-selective credit assignment "mitigates the accuracy-grounding trade-off" is supported by the long-video results in Table 2, where Zoom-Zero improves over Qwen2.5-VL on all benchmarks while VideoChat-R1 and TVG-R1 degrade. However, the effect sizes on long-video benchmarks are modest: +0.5 VideoMME, +2.2 MLVU, +0.6 LVBench, +0.4 CG-Bench over Qwen2.5-VL. These improvements, while directionally consistent, are within potential variance ranges given that the paper reports no confidence intervals or statistical tests. The stronger evidence for the trade-off mitigation claim is that Zoom-Zero does not degrade general understanding while substantially improving grounding — a qualitative pattern rather than a large quantitative margin.

Several genuine weaknesses in the experimental design merit attention:

No statistical significance reporting. None of the tables report confidence intervals, standard deviations, or p-values. The test sets — NExT-GQA (500 questions, per the original dataset specification), ReXTime (size not specified), CG-Bench (size not specified) — are moderate in size, and the observed differences between some configurations (e.g., accuracy gains of 0.5-1.0 points in Table 5) could be within sampling noise. The training curve in Figure 4 lacks error bars or shaded regions, making it impossible to assess whether the TokenAdv training trajectory is reliably different from baseline or within normal run-to-run variation. This is a significant omission for a paper reporting metric improvements as evidence of algorithmic superiority.

Single base model family. All experiments use Qwen2.5-VL-7B. The paper does not demonstrate whether the zoom-in reward or TokenAdv transfers to other model architectures (e.g., LLaVA-based LVLMs, InternVL2.5) or other scales. The claim that Qwen2.5-VL is "representative" (implicit in the choice) is untested. The paper's own results show that Qwen2.5-VL has remarkably strong zero-shot temporal grounding (20.2% mIoU on NExT-GQA) compared to SFT-trained models like VTimeLLM (24.4%) and TimeChat (20.6%), which raises questions about whether the base model's strong grounding capability is a prerequisite for the method's success — and whether models with weaker zero-shot grounding would benefit less or not at all.

The data filtering step is critical but under-ablated. Stage II training uses filtered data with $\delta \geq 0.1$ (Equation 6) and removal of examples where all rollouts answer correctly. The paper does not ablate this filtering threshold or compare against training on unfiltered data. Given that filtering discards a substantial fraction of examples (Stage I uses ~15K examples across three datasets; Stage II uses ~7K after filtering), it's possible that the filtering, rather than the algorithmic innovations, drives some of the observed gains. A control experiment showing Stage II performance without filtering would have quantified this contribution.

Two-stage training conflates data distribution shifts with algorithmic improvements. The shift from Stage I (short videos: NExT-GQA 43.9s avg, ActivityNet 177.3s) to Stage II (long videos from PLM-Video: 808.6s avg) is confounded with the algorithmic components. The paper doesn't report Stage I performance on the final test benchmarks, making it impossible to determine whether the improvements come from the second training stage (with PLM-Video data), from the filtering, or from the algorithmic innovations. A fairer design would have compared the full method against itself with only one training stage using the same combined data.

No generation-time compute-matched comparison with baselines. Table 6 reports inference times for Zoom-Zero variants but doesn't provide equivalent measurements for VideoChat-R1 or Qwen2.5-VL. This makes the latency comparisons in Section 5.5 one-sided: we know Zoom-Zero's inference cost but not how it compares to alternatives at equivalent accuracy. For the divide-and-conquer variant specifically — which shows the largest gains but at 2.3× inference cost — an accuracy-versus-latency plot including baselines would have contextualized whether the gains justify the expense or whether a simpler strategy (e.g., running Qwen2.5-VL with best-of-N sampling and majority voting at equivalent total FLOPs) might achieve similar performance.

No ablation on the zoom-in crop quality. The zoom-in mechanism depends on cropping the video to the predicted temporal span. The paper does not investigate how sensitive $R_{\mathrm{Zoom}}$ and the fine-pass performance are to the quality of the predicted span. Would the method work if the base model's spans were substantially noisier (e.g., if evaluated on a model with <10% mIoU)? Does the zoom-in reward provide a useful training signal when most predicted spans fail to crop to answer-relevant frames? These questions are central to the method's generality but remain unexplored.

Missing baseline: zoom-in reward without token-selective credit assignment but with weighted reward summation. The paper compares + GRPO + $R_{\mathrm{Zoom}}$ (Table 5, row 4) against + GRPO + TokenAdv (row 3), but doesn't test whether the zoom-in reward's benefit could be achieved through a simpler mechanism — e.g., weighting $R_{\mathrm{Zoom}}$ more heavily in the summed reward rather than decoupling it entirely. The ablation compares the full method against components added separately, but not against a configuration where rewards are summed with learned or tuned weights, which would have isolated whether the credit assignment structure or simply the additional reward signal drives the gains.

Long-video benchmarks evaluated at different resolutions than GVQA benchmarks. Short-video benchmarks (NExT-GQA, ReXTime) are evaluated at 1 FPS with context size 8192; long-video benchmarks at max 256 frames with context size 16384. This difference in evaluation protocol means the GVQA and long-video results are not directly comparable — the long-video evaluation uses a larger token budget and different frame sampling, which could independently affect grounding quality and answer accuracy. The paper's claim that the method "preserves global context while capturing critical information" should be understood as conditional on the specific evaluation protocol used for each benchmark.

In summary, the experiments convincingly demonstrate that the proposed innovations improve over standard GRPO for GVQA on the tested benchmarks and model. The evidence for the method's generality — across model families, data regimes, and task distributions — is absent. The strongest results (5.2% mIoU gain on NExT-GQA, 4.6% on ReXTime) are measured against RL baselines that also use Qwen2.5-VL as the base model; whether similar gains would materialize with other base architectures or at other scales is an open question that the paper does not address. The ablation design is thorough within the chosen configuration but leaves several confounds (data filtering, two-stage training, evaluation protocol differences) unexplored, which limits the strength of causal attribution to the named innovations.

6. Limitations and Trade-offs

The Zoom-In Mechanism Depends on Base Model Temporal Grounding Capability That Is Not Guaranteed to Generalize

The assumption or constraint. The zoom-in accuracy reward and coarse-to-fine inference strategy both depend on the base model producing temporal span predictions that are non-trivially better than random. The paper explicitly acknowledges this scope constraint: the base Qwen2.5-VL-7B achieves 20.2% mIoU on NExT-GQA in zero-shot evaluation (Table 1), and this non-trivial grounding ability is what allows the zoom-in mechanism to provide a useful training signal. If the base model's temporal spans are essentially random — if no rollout in the group crops to frames containing the answer — the zoom-in reward would be "dominated by noise and provide no useful training signal." The paper does not demonstrate the method's effectiveness with a base model that has weak zero-shot grounding, nor does it test with model families other than Qwen2.5-VL.

The consequence. A practitioner considering this method for a different base architecture (e.g., LLaVA-OneVision, which the paper reports has substantially weaker long-video GQA on CG-Bench with mIoU 1.56 vs. Qwen2.5-VL's 2.48 in Table 2) cannot assume the zoom-in reward will provide similar benefits. If the base model's grounding ability is poor, the zoom-in verification step will consistently fail — predicted spans will not crop to answer-relevant frames, $R_{\mathrm{Zoom}}$ will rarely be 1, and the reward signal will not differentiate between better and worse rollouts. This would turn the zoom-in reward from a training signal into noise that could actively harm optimization. The method is therefore best understood as an amplification mechanism for existing grounding capability, not a bootstrapping mechanism that can create grounding ability from scratch.

What evidence exists in the paper. The paper's own baselines quantify the variation in zero-shot grounding across models. TimeChat, despite being SFT-trained for temporal grounding, achieves only 20.6% mIoU on NExT-GQA (Table 1) — comparable to Qwen2.5-VL's zero-shot performance. Grounded-VideoLLM achieves 21.1% mIoU. VTimeLLM achieves 24.4%. The paper does not test Zoom-Zero with any of these as the base model, and the ablation in Table 5 is built entirely on Qwen2.5-VL. There is no experiment that varies the base model's grounding capability to determine the threshold below which the zoom-in reward ceases to be beneficial. The Figure 3 analysis shows that shorter ground-truth clue durations make temporal grounding harder for all methods, and that Zoom-Zero maintains an advantage across all duration ranges, but this analysis is done on the trained Zoom-Zero model — it does not assess how initial grounding quality affects training dynamics.

Mitigation status. The paper does not attempt to address this limitation, nor does it acknowledge it as a boundary condition for the method's applicability. The choice of Qwen2.5-VL as the base model is justified as "representative of the capabilities of many contemporary LLMs" (Section 1), but this claim is made about video-language understanding generally, not specifically about temporal grounding capability. The paper's own Table 1 shows that Qwen2.5-VL's zero-shot grounding (20.2% mIoU) dramatically exceeds that of some SFT-trained grounding models (TimeChat's 20.6% mIoU was achieved after supervised training on grounding data), which suggests Qwen2.5-VL may be unusually strong at grounding relative to its peers. A practitioner using a different base model would need to independently verify that the model possesses sufficient baseline grounding for the zoom-in mechanism to be effective — the paper provides no diagnostic or threshold for making this determination.

---

Difficulty Estimation for Zoom-In Is Effectively Performed by the Model's Own Temporal Grounding, Creating a Chicken-and-Egg Problem

The assumption or constraint. The zoom-in mechanism requires the model to predict which temporal segments to crop and re-encode at higher resolution. During training, this prediction is what generates the zoom-in accuracy reward. During inference (coarse-to-fine and divide-and-conquer), the predicted spans determine which frames receive high-resolution processing. In both cases, the quality of the zoom-in depends on the quality of the temporal grounding — but the temporal grounding is precisely what the training is trying to improve. This creates a circular dependency: the mechanism used to generate the training signal (temporal span prediction) is the same capability being trained, meaning the quality of the training signal improves as training progresses.

The consequence. In early training, when the model's temporal grounding is poor, the zoom-in reward will be dominated by noise. Most predicted spans will fail to crop to answer-relevant frames, $R_{\mathrm{Zoom}}$ will be 0 for most rollouts, and the relative advantage from $R_{\mathrm{Zoom}}$ will be minimal. The useful training signal from the zoom-in reward only emerges once the model has achieved some baseline grounding quality — but the model achieves that baseline grounding quality through other reward components ($R_{\mathrm{IoU}}$ and $R_{\mathrm{Acc}}$), not through $R_{\mathrm{Zoom}}$ itself. This means the zoom-in reward functions primarily as a refinement signal in later training stages, not as a driver of initial grounding capability. This is a more subtle boundary condition than the previous limitation: even with a base model that has decent zero-shot grounding, the zoom-in reward may contribute little in early GRPO steps and only become useful after $R_{\mathrm{IoU}}$ and $R_{\mathrm{Acc}}$ have improved the policy.

What evidence exists in the paper. The training curves in Figure 4 provide indirect evidence. The IoU reward trajectory for GRPO + TokenAdv (blue vs. orange for $R_{\mathrm{IoU}}$) shows separation from baseline GRPO relatively early in training, suggesting TokenAdv provides immediate benefit. However, the paper does not show training curves comparing GRPO + $R_{\mathrm{Zoom}}$ (without TokenAdv) against baseline GRPO, nor does it show how the zoom-in reward $R_{\mathrm{Zoom}}$ itself evolves during training — does it start near 0 and gradually increase? Does the advantage signal from $R_{\mathrm{Zoom}}$ become stronger over time? These dynamics are not analyzed. The two-stage training protocol further complicates interpretation: Stage I uses short videos without PLM-Video data, and Stage II introduces different data after filtering. The contribution of $R_{\mathrm{Zoom}}$ at each stage cannot be disentangled from the contribution of the data shift and filtering.

Additionally, the data filtering step for Stage II (Section 4, Equation 6) specifically retains examples with high variance in temporal grounding quality ($\delta \geq 0.1$), which are precisely the examples where the model's grounding is uncertain or variable. This filtering may artificially amplify the zoom-in reward's effectiveness by selecting examples where there is room for improvement — but it also means the Stage II training distribution is not representative of the overall data distribution. A practitioner replicating this method will need to implement similar filtering, but the paper does not ablate the filtering threshold to guide this choice.

Mitigation status. The paper partially acknowledges this concern through the data filtering design — by retaining only examples where the Stage I model produces variable rollouts ($\delta \geq 0.1$), Stage II training focuses on examples where the relative advantage signal can differentiate better and worse spans. However, the paper does not frame this as addressing a chicken-and-egg problem, nor does it discuss how the zoom-in reward's effectiveness depends on training stage. The training curves in Figure 4 omit the zoom-in reward trajectory entirely, showing only $R_{\mathrm{IoU}}$ and $R_{\mathrm{Acc}}$.

---

The Difficulty Estimation Cost During Training Is Unaccounted for in Reported Efficiency Claims

The assumption or constraint. The zoom-in accuracy reward requires a second forward pass of the model on cropped, re-encoded frames for every rollout during GRPO training. The paper reports that this increases per-step training time from 13.29 minutes to 15.30 minutes and TFLOPs from 551.2 to 585.4 — a 1.15× increase (Section 5.5). During inference, the coarse-to-fine strategy increases latency by approximately 1.4×, and the divide-and-conquer strategy by approximately 2.3× (Table 6). However, the headline performance numbers in Tables 1, 2, and 4 report accuracy and grounding improvements without normalizing by compute cost.

The consequence. The paper's central claim — that Zoom-Zero "advances temporal grounding by 5.2% on NExT-GQA and 4.6% on ReXTime" — cannot be interpreted as an efficiency gain. These are absolute performance improvements achieved at higher computational cost. A practitioner comparing Zoom-Zero against baselines needs to know whether equivalent or better performance could be achieved by allocating the same additional compute to a simpler strategy: training the baseline GRPO for 1.15× more steps, increasing the rollout count G beyond 8, or using a larger context budget that eliminates the need for zoom-in entirely. The paper provides no compute-matched comparison — e.g., VideoChat-R1 trained for 15.30 minutes/step or with more rollouts — that would isolate whether the zoom-in mechanism specifically or the additional compute generally drives the gains.

This is particularly relevant for the divide-and-conquer inference strategy, which achieves the largest long-video gains (+6.4% average across benchmarks in Table 4) at 2.3× inference cost (Table 6). The paper does not compare this against a Qwen2.5-VL baseline with 2.3× inference compute (e.g., processing all frames at higher resolution with a proportionally larger context budget, or running best-of-N sampling with majority voting at equivalent total FLOPs). Without such comparisons, the reported gains could reflect the general principle that more inference compute improves performance rather than the specific value of the coarse-to-fine mechanism.

What evidence exists in the paper. Section 5.5 and Table 6 provide the necessary cost measurements to compute efficiency metrics, but the paper does not perform the computation. The per-video inference times in Table 6 show that one-stage Zoom-Zero achieves its gains with negligible additional latency (e.g., 25.8s vs. 25.6s for Qwen2.5-VL on VideoMME long), but the coarse-to-fine and divide-and-conquer variants add substantial latency. The paper frames this as a deployment choice — "one-stage inference provides solid performance with baseline latency; coarse-to-fine provides meaningful gains at moderate cost; divide-and-conquer provides maximum gains at higher cost" — but doesn't analyze whether the cost-performance Pareto frontier of Zoom-Zero actually dominates that of simpler alternatives at equivalent compute.

Mitigation status. The paper partially addresses this through transparency about computational costs in Section 5.5 and Table 6, which is commendable. However, the paper does not provide the compute-matched baselines needed to convert cost transparency into efficiency analysis. The statement in the conclusion that the method "improves both temporal grounding and answer accuracy" is true in absolute terms but incomplete without cost normalization. A practitioner reading the headline results should understand them as absolute performance improvements at increased computational cost, not efficiency improvements. The one-stage Zoom-Zero results (Tables 1 and 2) represent the relevant zero-cost-improvement comparison — these gains (+5.2% mIoU on NExT-GQA) come at essentially identical inference cost to the base model.

---

All Results Are on a Single Model Family and Require the Model's Specific Dynamic Resolution Mechanism

The assumption or constraint. All experiments use Qwen2.5-VL-7B as the base model, including all RL baselines that the paper compares against. The zoom-in mechanism exploits Qwen2.5-VL's dynamic spatiotemporal resolution (Section 3, Equation 3), which automatically adjusts per-frame token resolution based on the number of frames and the context budget. When the model crops to fewer frames in the zoom-in pass, Qwen2.5-VL's resolution adjustment is what provides higher per-frame detail — the paper does not implement custom resolution control. Additionally, the TokenAdv mechanism depends on the structured output format with specific <glue> and <answer> tags, which is enforced through the format reward during GRPO training. This format is specific to the paper's training protocol and may not transfer to models with different output conventions.

The consequence. A practitioner using a different base model family — InternVL2.5, LLaVA-OneVision, or any model that does not implement dynamic spatiotemporal resolution — cannot directly apply the zoom-in mechanism. If the model uses fixed per-frame resolution, cropping to fewer frames will not automatically increase detail; the practitioner would need to manually implement re-encoding at higher resolution, potentially exceeding the model's native processing pipeline. If the model does not support the specific output format, the TokenAdv mechanism cannot segment grounding tokens from answer tokens, and the entire credit assignment structure collapses. The paper does not discuss which aspects of the method are model-specific versus generally applicable.

More broadly, the single-model evaluation means that all results are conditional on Qwen2.5-VL-7B's specific capabilities, failure modes, and training data. The paper's Table 1 shows that Qwen2.5-VL has remarkably strong zero-shot temporal grounding (20.2% mIoU) compared to SFT-trained grounding specialists, and its zero-shot answer accuracy (53.3% on NExT-GQA, Table 5) is substantially higher than prior SFT models (TimeChat 7.6% Acc@GQA in Table 1). This suggests Qwen2.5-VL may be unusually receptive to the GRPO-based grounding training, and the observed improvements may not transfer to base models with weaker initial capabilities.

What evidence exists in the paper. The paper's baseline comparisons in Table 1 include models from different families (TimeChat, VTimeLLM, Grounded-VideoLLM), but these are evaluated as fixed baselines, not as base models for the Zoom-Zero training protocol. The RL baselines (VideoChat-TPO, TVG-R1, VideoChat-R1) all use Qwen2.5-VL as their base, so the comparison is between RL training strategies applied to the same starting point, not between base model families. Table 2 includes long-video results for LLaVA-OneVision, LongVA, InternVL2.5, and Qwen2.5-VL, but again these are fixed baselines, not Zoom-Zero training targets.

The paper does not ablate or discuss the dependency on dynamic spatiotemporal resolution. The mechanism is described in Section 3 as part of Qwen2.5-VL's architecture but is never identified as a prerequisite for Zoom-Zero. If a practitioner attempted to apply the method to InternVL2.5 (which the paper evaluates as a baseline in Table 2 but does not train with Zoom-Zero), it is unclear whether the zoom-in would function at all without architectural modifications.

Mitigation status. The paper does not acknowledge or address the model-specificity of its approach. The claim that Qwen2.5-VL-7B is "representative of the capabilities of many contemporary LLMs" (Section 1) is asserted but not tested. The paper does not discuss which components of the method are architecture-dependent (dynamic resolution, output format conventions) and which are transferable. No experiments with alternative base models are proposed as future work.

---

Small Test Sets and Absence of Statistical Significance Reporting Weaken Confidence in Quantitative Comparisons

The assumption or constraint. The paper evaluates on standard benchmarks with moderate test set sizes — NExT-GQA uses 500 test questions per the dataset specification, ReXTime and CG-Bench test set sizes are not explicitly stated in the paper. The two-fold cross-validation used in some experiments results in strategy selection based on approximately 250 questions per fold. None of the results in Tables 1–6 or Figure 4 include confidence intervals, standard deviations, error bars, or any form of statistical significance testing. The training curves in Figure 4 show single-run trajectories without shaded regions indicating variance.

The consequence. Many of the reported improvements are modest in absolute magnitude, particularly on long-video benchmarks. In Table 2, Zoom-Zero improves over Qwen2.5-VL by +0.5 on VideoMME (71.2 vs. 70.7), +0.6 on LVBench (70.8 vs. 70.2), and +0.4 on CG-Bench accuracy (45.7 vs. 45.3). These are differences of less than 1 percentage point. Without confidence intervals, a practitioner cannot determine whether these differences are statistically reliable or within the range of sampling variation from the test set. The same applies to the ablation in Table 5, where the accuracy gain from adding TokenAdv to baseline GRPO is +0.3 on NExT-GQA (69.6 → 69.9) and +1.5 on ReXTime (58.3 → 59.8). The NExT-GQA accuracy difference is small enough that it could be explained by test-set sampling noise alone.

The divide-and-conquer ablation (Appendix D.3, Tables 10 and 11) is particularly vulnerable to this concern. The optimal window size (256 seconds) and optimal number of aggregated spans (4) are selected based on performance differences of 0.3–0.5 percentage points across configurations. With no reported variance, it is impossible to conclude that these hyperparameter choices are reliably better than adjacent values. The reported results may reflect the idiosyncrasies of these particular test sets rather than generalizable hyperparameter optima.

What evidence exists in the paper. The numbers themselves are the evidence. Table 1 reports mIoU improvements of +5.2 (37.6 vs. 32.4) and +4.6 (43.2 vs. 38.6), which are large enough that they are unlikely to be explained by test-set variance alone — but even here, confidence intervals would contextualize the effect size relative to sampling error. The absence of statistical reporting is systematic: no table, figure, or appendix includes variance estimates. The paper's reproducibility statement mentions "full implementation details" and "publicly accessible" datasets but does not discuss statistical reproducibility across random seeds or data splits.

Mitigation status. The paper makes no attempt to address this. It includes no discussion of statistical methodology, no reporting of confidence intervals, and no mention of random seed sensitivity. Given that GRPO training involves stochastic sampling of rollout groups and that RL training trajectories can exhibit substantial run-to-run variance, this is a significant omission for a paper whose primary contribution is quantitative improvement over baselines. A practitioner seeking to determine whether Zoom-Zero provides reliable improvements over VideoChat-R1 for their use case cannot assess whether the reported differences would replicate.

---

The Inference-Time Zoom-In Strategy's Gains May Be Achievable Through Simpler Compute-Scaling Approaches Not Tested

The assumption or constraint. The paper positions the coarse-to-fine and divide-and-conquer inference strategies as evidence that the temporal grounding capability developed during RL training enables a "query-conditioned adaptive allocation" mechanism that improves long-video understanding (Section 5.3, Table 4). The claim is that the improvements (+2.0 on VideoMME, +0.6 on MLVU, +0.6 on LVBench, +2.9 on CG-Bench for coarse-to-fine; +4.5, +2.6, +2.4, +6.1 for divide-and-conquer) are specifically attributable to the model's ability to zoom into query-relevant segments rather than to the additional inference computation generally.

The consequence. The paper does not test whether comparable or better gains could be achieved by simpler compute-scaling strategies that do not require the trained temporal grounding capability. Examples not tested include: (a) running Qwen2.5-VL with a proportionally larger context budget (e.g., 32768 tokens instead of 16384) to encode all frames at higher resolution in a single pass; (b) applying best-of-N sampling with majority voting at equivalent total inference cost; (c) uniform frame upsampling (e.g., 0.5 FPS instead of 0.1 FPS for the entire video) to match the total compute of the two-pass strategy; or (d) a retrieval-based baseline that selects frames based on embedding similarity to the question text rather than learned temporal grounding.

Without these baselines, the paper cannot distinguish between the hypothesis "temporal grounding provides uniquely effective frame selection" and the hypothesis "any mechanism that allocates additional compute to relevant frames will improve performance." This is a critical distinction: if uniform upsampling achieves similar gains, the coarse-to-fine mechanism is unnecessary complexity; if best-of-N voting achieves similar gains, the temporal grounding training is unnecessary. The paper's claim that the method "preserves global context while capturing critical information" (Section 5.3, Table 4 caption) implies a specific mechanism — adaptive allocation based on query understanding — but the experiments do not isolate this mechanism from the general effect of increased inference compute.

What evidence exists in the paper. The paper provides cost measurements in Table 6: coarse-to-fine inference takes 35.2 seconds on VideoMME long versus 25.8 seconds for one-stage Zoom-Zero (1.36× increase), and divide-and-conquer takes 59.7 seconds (2.31× increase). The paper also reports (Appendix D.3) that divide-and-conquer "introduces an average 2.3× increase in inference cost." However, there is no measurement of what Qwen2.5-VL or VideoChat-R1 achieves at 1.4× or 2.3× inference cost using simpler scaling strategies. The comparison is always Zoom-Zero at higher cost versus baselines at baseline cost, never baselines at matched higher cost.

Table 4 shows the improvements over Qwen2.5-VL's one-stage baseline, but Qwen2.5-VL is evaluated at the standard 256-frame, 16384-token context — not at a compute-matched setting. If Qwen2.5-VL with 384 frames and 24576-token context achieves similar or better performance on long-video benchmarks, the specific value of the coarse-to-fine mechanism would be substantially diminished. The paper's own dynamic resolution formula (Equation 3) shows that $L_v$ directly controls the resolution trade-off, suggesting that simply increasing $L_v$ is a natural competing strategy.

Mitigation status. The paper does not acknowledge or address this limitation. The inference-time strategies are presented as demonstrating the value of the trained temporal grounding capability, but the experimental design does not isolate temporal grounding quality as the causal mechanism. A compute-matched comparison against Qwen2.5-VL with proportionally larger context budgets would have provided the necessary control, and the absence of this comparison is a meaningful gap in the experimental design. A practitioner deciding whether to invest in training a Zoom-Zero model versus simply deploying Qwen2.5-VL with a larger context budget does not have the evidence needed to make an informed choice.

7. Implications and Future Directions

How This Work Changes the Landscape

Zoom-Zero introduces a conceptual reframing of temporal grounding quality from geometric overlap (IoU) to functional adequacy (can you answer from this segment?) , rather than a paradigm shift in video-language model architecture. The paper does not propose a new model, a new RL algorithm, or a new dataset — it operates entirely within the existing GRPO framework and Qwen2.5-VL architecture, adding two mechanisms that together reformulate what it means to train for temporal grounding. The significance lies in changing the training objective from surface-level timestamp prediction to evidence-faithful localization verified through downstream task success.

The field of grounded video question answering has been split between two evaluation philosophies that the paper helps reconcile. On one side, temporal grounding research (Gao et al., 2017; Anne et al., 2017; Lei et al., 2021) optimized for IoU overlap with human annotations — a metric that is easy to compute but does not verify whether the localized segment actually contains the visual information needed for reasoning. On the other side, video question answering research optimized for answer accuracy alone, with grounding treated as an optional auxiliary output (Xiao et al., 2024; Li et al., 2025). These two traditions produced models that were either good at timestamp prediction but indifferent to whether those timestamps captured answer-relevant evidence, or good at answering but unable to identify where their evidence came from. Zoom-Zero demonstrates that a training signal connecting these two objectives — the zoom-in accuracy reward $R_{\mathrm{Zoom}}$, which tests whether the localized segment functionally enables correct answering — can simultaneously improve both grounding precision (+5.2% mIoU on NExT-GQA over the strongest RL baseline) and answer accuracy (+2.4% average across benchmarks). This reframing suggests that the apparent tension between grounding and answering quality is partially an artifact of training objectives that treat them as independent tasks rather than coupled ones.

A second reframing concerns how video-language models should allocate their visual processing budgets. Prior work on long-video understanding treats the context budget as a fixed constraint to be optimized once — through uniform downsampling (Zhang et al., 2024; Li et al., 2024), token compression (Shen et al., 2024; Maaz et al., 2023), or one-time retrieval-based frame selection (Ren et al., 2025; Shen et al., 2025). Zoom-Zero demonstrates that the allocation decision can be query-conditioned and multi-pass, where the model first identifies relevant temporal segments at coarse resolution and then re-processes those segments at higher spatial resolution. This is not the first coarse-to-fine approach in vision (hierarchical processing has a long history in object detection and image recognition), but it is the first to tie the coarse-to-fine transition specifically to a learned temporal grounding capability trained with RL. The practical implication in Table 4 — +2.0 on VideoMME long, +2.9 on CG-Bench from the coarse-to-fine inference strategy at 1.4× inference cost — establishes that query-conditioned rezoning is viable with today's models and budgets.

On the RL methodology side, the paper's token-selective credit assignment (TokenAdv) identifies a specific failure mode of standard GRPO — uniform credit collapse under multi-faceted rewards — that has implications beyond video grounding. The diagnosis in Appendix C, Table 8, showing that a response with $R_{\mathrm{IoU}} = 0.0$ can receive positive overall advantage while a response with $R_{\mathrm{IoU}} = 0.8$ receives negative advantage, makes the problem concrete and actionable. This is not a new RL algorithm; it is a structural modification to GRPO's advantage computation that exploits structured output formatting to decouple credit assignment. The principle — that tokens responsible for different capabilities should receive advantages computed from the specific reward signals relevant to those capabilities — generalizes to any RL fine-tuning setting where the model's output has distinguishable structural components with distinct evaluation criteria. The paper's conditional assignment rule (Equation 5) provides a template that practitioners can adapt to their own multi-reward settings.

What this work makes less attractive. The paper's results suggest that pure IoU optimization without functional verification — as practiced in Time-R1, TVG-R1, and TimeZero — is a limited strategy for GVQA. The ablation in Table 5 shows that baseline GRPO with format + IoU + accuracy rewards achieves 35.3% mIoU on NExT-GQA, while adding the zoom-in reward (which verifies functional adequacy) brings this to 36.3% mIoU — an incremental but meaningful gain that signals the value of going beyond geometric metrics. Similarly, the long-video results in Table 2 show that RL methods optimizing primarily for IoU (TVG-R1, VideoChat-R1) degrade in general video understanding compared to the base Qwen2.5-VL, while Zoom-Zero with its decoupled credit assignment does not. This suggests that single-metric RL optimization for grounding is a dead end — the field should either use multi-faceted rewards with structured credit assignment (as this paper does) or accept that grounding improvements come at the cost of general understanding.

Which research directions become more attractive. The paper makes verification-based training signals — where a model's prediction is tested by checking whether a downstream task can be completed using only the information in that prediction — a demonstrated effective strategy for video grounding. This invites extensions to other structured prediction tasks where geometric metrics capture only surface alignment: object grounding in images (does the bounding box contain enough visual information to identify the object?), document retrieval (does the retrieved passage enable answering the query?), and code generation (does the generated function pass the test cases?). The token-selective credit assignment principle also opens a research direction on designing RL credit structures that match a task's causal decomposition — which tokens causally influence which reward signals — rather than treating credit assignment as a uniform or heuristic problem.

Follow-Up Research This Work Enables

Measuring the minimum base model grounding capability required for zoom-in reward effectiveness. The paper demonstrates that the zoom-in mechanism works with Qwen2.5-VL-7B, which has 20.2% zero-shot mIoU on NExT-GQA (Table 1). But the mechanism depends on the base model producing temporal span predictions that are non-trivially better than random — if no rollout crops to answer-relevant frames, $R_{\mathrm{Zoom}}$ provides no useful training signal. A critical unanswered question is: what is the threshold below which the zoom-in reward ceases to be beneficial, and can that threshold be estimated before training? A strong follow-up would take a model family with controllable grounding quality — e.g., Qwen2.5-VL at different sizes (0.5B, 1.5B, 3B, 7B), or the same architecture at different pretraining checkpoints — and apply the full Zoom-Zero training protocol with the same hyperparameters. The key measurement would be the correlation between initial zero-shot mIoU and the gain in mIoU from adding $R_{\mathrm{Zoom}}$ to baseline GRPO. If the gain is zero or negative below some threshold mIoU (e.g., <10%), that establishes the method's applicability boundary. The training curve of $R_{\mathrm{Zoom}}$ itself — does it start near zero and gradually increase, indicating that the reward signal only emerges as grounding improves through $R_{\mathrm{IoU}}$? — would reveal whether $R_{\mathrm{Zoom}}$ is a bootstrapping signal or purely a refinement signal.

Testing whether zoom-in benefits are achievable through simpler compute scaling strategies on the base model. The paper's divide-and-conquer inference strategy achieves +6.4% average improvement over Qwen2.5-VL on long-video benchmarks (Table 4) at 2.3× inference cost (Table 6). But the paper does not test whether the same or better gains could be achieved by simply giving Qwen2.5-VL more compute through standard scaling strategies: using a larger context budget (32768 or 65536 tokens instead of 16384), uniformly sampling frames at higher FPS, or applying best-of-N sampling with majority voting at equivalent total inference FLOPs. A fair comparison would measure Qwen2.5-VL accuracy as a function of total inference FLOPs on VideoMME long, MLVU, and LVBench, then overlay the Zoom-Zero divide-and-conquer operating point on the same cost-accuracy curve. If Zoom-Zero's cost-accuracy point lies on or below the Qwen2.5-VL scaling curve, the specific value of temporal grounding-based frame selection would be called into question — the gains would reflect additional compute rather than intelligent allocation. If Zoom-Zero is substantially above the curve, that quantifies the allocation benefit specifically.

Ablating whether the zoom-in reward adds value beyond weighted reward summation with learned weights. The paper's ablation in Table 5 compares the full method (+ GRPO + $R_{\mathrm{Zoom}}$ + TokenAdv) against configurations where components are added to baseline GRPO. But it never tests whether the benefit of $R_{\mathrm{Zoom}}$ could be captured by a simpler mechanism: summing all rewards with learned per-reward weights optimized by a small hyperparameter sweep or a meta-learning objective, rather than the full decoupled advantage computation of TokenAdv. A controlled experiment would compare three configurations at matched $G=8$: (a) baseline GRPO with summed rewards, (b) baseline GRPO with weighted sum (weights tuned on validation), and (c) TokenAdv with per-reward advantages. If (b) closes most of the gap between (a) and (c), then the credit assignment structure — not the decoupling math — is the active ingredient, and practitioners can achieve similar benefits with simpler engineering. If (c) substantially outperforms (b), the paper's claim that token-selective assignment is structurally necessary would be strengthened.

Extending token-selective credit assignment to tasks where token groups are learned rather than hard-coded. The TokenAdv mechanism in Equation 5 relies on the structured output format (<glue> and <answer> tags) to determine which tokens receive which advantages. This is feasible because the format reward $R_{\mathrm{format}}$ enforces reliable tag generation. However, many multi-capability tasks lack such explicit structural separation — consider dialogue generation where factual accuracy and conversational fluency are both evaluated, but no tags distinguish "factual tokens" from "fluency tokens." A natural extension would learn a soft token-level attribution: train a lightweight attention-based module that, for each token, predicts a probability distribution over reward types based on the token's hidden state, and use these predicted attributions to weight the per-reward advantages in a soft version of Equation 5. The training signal for this attribution module could come from the downstream task performance — if attributing IoU advantage to a token improves the policy's temporal grounding, the attribution module is reinforced. This would convert TokenAdv from requiring hard-coded structural tags to being a general mechanism applicable to any multi-faceted reward setting.

Stress-testing on video domains where temporal annotations are scarce or ambiguous. The paper trains on datasets with clean ground-truth temporal intervals (NExT-GQA, ActivityNet, QVHighlights, PLM-Video). Appendix E suggests that the framework "could also be trained without explicit temporal interval annotations" by replacing $R_{\mathrm{IoU}}$ with a self-verification mechanism — the model learns to verify whether key visual clues exist within its predicted segments based on the zoom-in accuracy reward alone. This is a significant claim that the paper does not test. A follow-up would take a video question-answering dataset without temporal annotations (e.g., portions of EgoSchema, NextQA, or ActivityNet-QA) and train Zoom-Zero using only $R_{\mathrm{format}}$, $R_{\mathrm{Acc}}$, and $R_{\mathrm{Zoom}}$ — removing $R_{\mathrm{IoU}}$ entirely. The key question is whether the zoom-in reward alone can bootstrap temporal grounding: when the model predicts a temporal span, crops to it, and re-answers, does the binary feedback of correct/incorrect zoomed-in answer provide enough signal to improve span quality? If so, this would dramatically expand the applicability of the method to domains where temporal annotations are expensive or impossible to obtain (instructional video, procedural task understanding, long-form documentary analysis). If not, it would establish that IoU supervision remains necessary and the method's scope is limited to well-annotated datasets.

Generalizing the coarse-to-fine paradigm to iterative, multi-round zoom-in with the model deciding when to stop. The paper's zoom-in is a single round: predict a span, crop, re-encode, re-answer. Appendix E acknowledges that "multi-stage zooming could further refine temporal grounding by progressively narrowing the search space." A direct extension would implement iterative zoom-in where each round's predicted span becomes the input for the next round, with progressively tighter crops and higher per-frame resolution. The challenge is designing a stopping criterion — the model should decide when additional zoom-in rounds are unlikely to improve the answer. This could be implemented by training the model to output a confidence score alongside its answer and stopping when confidence exceeds a threshold, or by terminating when two consecutive rounds produce the same answer. The empirical question is whether iterative zoom-in outperforms single-round zoom-in on very long videos (CG-Bench, where average video duration is 4101 seconds and clues occupy ≤1% of duration), where the initial predicted span might still be too broad for maximum-resolution detail recovery.

Practical Applications and Downstream Use Cases

Long-form video content moderation and compliance verification. In settings where human reviewers must verify that specific events occurred at specific times in video footage — insurance claim investigation, workplace safety compliance, content policy enforcement — Zoom-Zero's temporal grounding provides a mechanical check on answer faithfulness. Rather than trusting a model that says "the safety violation occurred at [00:45–01:20]," a reviewer can verify by watching only that 35-second segment. The paper's Table 1 reports 55.6% R@0.3 on NExT-GQA, meaning more than half of model predictions have at least 30% temporal overlap with ground truth — a practical threshold for narrowing down review windows. The mIoG of 94.7% on NExT-GQA (Table 3) indicates that when the model does identify the right general region, it rarely misses large portions of the evidence, reducing the risk that reviewers are directed to segments missing the critical event. The 1.4× inference cost for coarse-to-fine (Table 6) is modest compared to the cost of human review time saved by precise temporal localization.

Educational video indexing and lecture navigation. Long-form educational content (recorded lectures, tutorial videos, conference presentations) requires students to locate specific topics within multi-hour recordings. Zoom-Zero's divide-and-conquer strategy — partitioning the video into windows, identifying candidate segments, zooming into high-confidence spans — maps directly to this use case. On LVBench (4101s average duration, Table 4), divide-and-conquer achieves 48.1% accuracy, a +2.4 point improvement over one-stage Zoom-Zero and +2.8 over base Qwen2.5-VL. For lecture videos where questions like "at what point does the professor explain backpropagation?" require temporal localization followed by content understanding, the method provides both the timestamp and the verified answer. The inference cost (110.5 seconds on LVBench per Table 6) is incurred once per query and can be parallelized across students, making it a one-time indexing cost rather than a per-user expense.

Sports analytics and highlight generation with evidence verification. Sports video analysis requires precise temporal localization of events (goals, fouls, tactical formations) followed by detailed visual analysis. Zoom-Zero's training paradigm — which rewards not just finding the right time window but verifying that the answer can be extracted from that window — aligns with the requirements of sports analytics, where knowing that a goal was scored at [12:30] is less valuable than being able to answer how it was scored (assist type, defensive formation, player positioning) from the localized frames. The paper's strong performance on ReXTime (62.0% accuracy, 43.2% mIoU, Table 1), a dataset designed for temporal reasoning with explicit event descriptions, suggests the method handles fine-grained event localization well. The high mIoG (94.7% on NExT-GQA, Table 3) indicates that localized segments rarely miss relevant visual details, which matters for sports scenarios where the critical action may span only a few seconds within a multi-hour broadcast.

When to Prefer This Method

The paper positions Zoom-Zero against prior GRPO-based RL approaches that use summed multi-faceted rewards with uniform credit assignment, and against SFT-based grounding models that cannot enforce evidence-faithful localization. The decision conditions are:

• Prefer Zoom-Zero with token-selective credit assignment when your task has multiple distinct reward signals that evaluate different structural components of the model's output, and those components can be identified through output format constraints (tags, delimiters, or other parseable structure). The paper demonstrates that when temporal grounding tokens and answer generation tokens are separable — and they receive different reward signals (IoU vs. accuracy) — standard GRPO's uniform credit creates perverse incentives (Appendix C, Table 8) that TokenAdv resolves. This condition holds for any task where the model is required to output both a structured prediction and reasoning in a parseable format.

• Prefer adding a zoom-in accuracy reward when the base model already possesses non-trivial capability on the structured prediction being verified (temporal grounding in this paper's case), AND the task has a downstream verification check that tests functional adequacy rather than geometric similarity. The base Qwen2.5-VL's 20.2% zero-shot mIoU on NExT-GQA (Table 1) is the paper's implicit threshold — if your base model's grounding is substantially worse, the zoom-in reward may provide noise rather than signal. The zoom-in reward is most valuable when geometric metrics (IoU) can be gamed — the model learns to produce spans that look right without actually containing the answer — and the verification step directly tests whether the span serves its intended purpose.

• Prefer the coarse-to-fine inference strategy when dealing with long videos (1000+ seconds) where the context budget necessarily forces a trade-off between temporal coverage and spatial detail, AND latency constraints permit 1.4× increase in inference time (Table 6). The gains are most substantial on CG-Bench (+2.9 points, Table 4), where videos average 4101 seconds and clues occupy ≤1% of duration — the ratio of video length to clue brevity makes query-conditioned zoom-in disproportionately valuable. For shorter videos (NExT-GQA's 43.9s average, ReXTime), the coarse-to-fine inference gain is marginal (+0.7% accuracy per Table 12) because the one-stage pass already captures most details within the context budget, and the additional latency may not be justified.

• Prefer the divide-and-conquer strategy when maximum accuracy is required and inference cost is less constrained (2.3× latency, Table 6), especially for extremely long videos where scanning sliding windows captures events that a single coarse pass might miss. The window size of 256 seconds and aggregation of 4 top-confidence spans (Appendix D.3, Tables 10-11) are empirically optimal on the tested benchmarks but should be recalibrated for new video distributions. The strategy's value is highest on CG-Bench (+6.1 points over one-stage Zoom-Zero, Table 4), where the divide-and-conquer structure mirrors the task's inherent challenge — finding a very short clue in a very long video — and the window-scanning approach provides multiple independent opportunities to localize the relevant segment.