Multimodal2025~99 minarXiv:2507.07999

Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology

score 8Haochen Wang, Xiangtai Li, Zilong Huang +9OpenAI↑ 50★ 75Jul 10, 2025cs.CVhttps://github.com/Haochen-Wang409/TreeVGR

🎯 Pitch

Even GPT‑4o‑class vision models fall below 60% on a new benchmark that verifies reasoning by demanding the model pinpoint objects before answering. The paper then shows that explicitly training a 7B model to generate bounding boxes while reasoning—enforced by a dual‑reward that checks both precision and recall—closes much of the gap, yet the gains fade on questions requiring cognitive leaps beyond locating objects.

1. Executive Summary

This paper introduces TreeBench (Traceable Evidence Evaluation Benchmark), a diagnostic benchmark for evaluating visual grounded reasoning—the ability of large multimodal models to dynamically reference and amplify task-relevant visual regions during reasoning—and proposes TreeVGR (Traceable Evidence Enhanced Visual Grounded Reasoning), a training paradigm that jointly supervises localization and reasoning with reinforcement learning. TreeBench evaluates three capabilities (focused visual perception of subtle targets in complex scenes, traceable evidence via bounding box annotations, and second-order reasoning such as perspective transforms or spatial containment) across 405 rigorously curated VQA pairs drawn from SA-1B, where even the most advanced models fail to reach 60% accuracy (OpenAI-o3 scores only 54.87%). TreeVGR, initialized from Qwen2.5-VL-7B and trained with a novel dual IoU reward that enforces both precision (ensuring each predicted box matches a ground-truth) and recall (ensuring each ground-truth is covered) during reinforcement learning, improves performance by 13.4 points on TreeBench, 16.8 on V* Bench, and 12.6 on MME-RealWorld-Lite, establishing that explicit supervision of intermediate bounding box generation enables transparent reasoning pathways and yields the strongest open-source performance on perception-heavy tasks—though the correlation between localization precision and reasoning performance weakens on tasks requiring second-order cognitive operations beyond precise spatial grounding.

2. Context and Motivation

The Core Problem: We Cannot Evaluate “Thinking with Images”

The fundamental problem this paper addresses is that the community lacks any benchmark capable of evaluating whether a multimodal model can genuinely "think with images" — that is, dynamically identify, reference, and reason over specific visual regions during multi-step inference. This capability has become central to the frontier of multimodal reasoning with the release of models like OpenAI-o3 (April 2025), which explicitly crops and zooms into regions of interest, producing image-text interleaved reasoning pathways. Yet, as the authors state starkly in the opening of Section 1:

"no benchmark exists to evaluate these capabilities holistically."

This gap matters because without a diagnostic benchmark, we cannot answer basic questions that are essential for progress in multimodal AI: Which models genuinely ground their reasoning in visual evidence versus relying on linguistic shortcuts? When a model answers correctly, was it because it actually located the right object, or did it guess from context? When it fails, did the failure occur at the perception stage (couldn't find the right object) or the reasoning stage (found the object but drew the wrong inference)? And critically, does better localization actually translate to better reasoning?

The answer to this last question turns out to be surprisingly nuanced, which the paper discovers only because TreeBench provides the measurement infrastructure to ask it. The paper finds that localization precision correlates strongly with perception performance but weakly with reasoning performance on second-order tasks (Figures 9-10), revealing that some reasoning failures are not fixable by better grounding alone. This distinction—between localization-dependent and localization-independent reasoning failures—is precisely the kind of diagnostic insight that existing benchmarks cannot provide.

Why This Problem Matters

The paper identifies several converging trends that make visual grounded reasoning evaluation urgent:

1. Frontier models are deploying this capability right now — without evaluation. OpenAI-o3, released in April 2025, introduced dynamic image manipulation during reasoning as a core feature. The model internally crops and zooms into regions of interest, producing reasoning chains that interleave visual and textual evidence. This represents a qualitative shift from earlier reasoning models (OpenAI-o1, DeepSeek-R1) that operated purely in text space, even when processing images. Yet no existing benchmark can assess whether this "thinking with images" actually works correctly, whether the model grounds its reasoning in the right visual regions, or whether it exhibits systematic failure modes (e.g., grounding in plausible but incorrect regions when the correct target is subtle).

2. The open-source community is racing to replicate this capability — without evaluation. The paper cites a flurry of recent works from early-to-mid 2025 attempting to build visual grounded reasoning into open models: VGR (Wang et al., 2025e), DeepEyes (Zheng et al., 2025b), GRIT (Fan et al., 2025), Ground-R1 (Cao et al., 2025), Pixel-Reasoner (Su et al., 2025), VisionReasoner (Liu et al., 2025a), and others. These methods all follow a "grounding-then-answering" pipeline — first localize relevant objects, then reason about them. But they evaluate primarily on benchmarks that either don't require grounding at all (MMBench, SEED-Bench) or only test simple localization (V* Bench). The paper explicitly identifies this as a problem:

"These RL-based methods supervise only the final answer. In contrast, our TreeVGR emphasizes traceable evidence during RL training." (Section 2)

Without a benchmark that separately evaluates localization quality and reasoning quality on the same task, it is impossible to know whether these methods are actually improving because they ground better, or for some other reason.

3. Language bias creates a silent failure mode in multimodal reasoning. The paper argues that models relying exclusively on textual reasoning pathways accumulate "language bias" (Section 1). This is the well-documented phenomenon where multimodal models learn to answer questions about images by recognizing linguistic patterns in the question and answer choices rather than by actually parsing the visual content. The classic demonstration is that models can achieve high scores on VQA benchmarks even when given scrambled or irrelevant images, because the language prior carries the signal. Visual grounded reasoning — forcing the model to explicitly localize and reference visual evidence — is proposed as an antidote to this bias, but only if we can verify that the grounding is faithful. TreeBench provides this verification mechanism through its bounding box annotations.

4. The inference-time compute paradigm is shifting toward visual reasoning. Just as the LLM community discovered that test-time compute scaling (chain-of-thought, search, revision) could substitute for pretraining compute on certain tasks, the multimodal community is discovering that test-time visual operations (cropping, zooming, re-examining regions) can amplify a model's effective visual acuity. However, without a benchmark that measures the quality of these intermediate visual operations, we cannot distinguish between methods that genuinely leverage visual computation and those that achieve similar end-task performance through other means (e.g., better language priors, larger models). TreeBench's traceable evidence evaluation enables this distinction by measuring mIoU on intermediate bounding boxes.

Where Existing Benchmarks Fall Short

The paper provides a detailed critique of existing multimodal benchmarks along the three axes that define visual grounded reasoning (Section 2, "Benchmarks for LMMs"):

Classical Benchmarks: No Localization, No Traceability

Standard multimodal evaluation suites — POPE (Li et al., 2023c), MMBench (Liu et al., 2023b), SEED-Bench (Li et al., 2023a), MMMU (Yue et al., 2024) — are designed to assess general visual understanding and reasoning. However, the paper identifies a specific and consequential shortcoming:

"Classical benchmarks like POPE, MMBench, SEED-Bench, and MMMU usually overlook fine-grained localization and verifiable reasoning chains." (Section 1)

These benchmarks ask questions that can often be answered through holistic scene understanding or language priors without requiring the model to identify which specific objects support the answer, or to demonstrate where those objects are in the image. When a model gets a question wrong on POPE, you know it hallucinated or failed to recognize something — but you don't know which object it failed on, or whether it even looked at the right region of the image. The evaluation is a black box: accuracy goes in, accuracy comes out, and nothing is learned about the failure mode.

Spatial and Localization Benchmarks: Too Simple or Contaminated

V* Bench (Wu and Xie, 2024), published at CVPR 2024, was a significant step forward. It specifically evaluates whether models can locate and describe attributes of small objects in images, and it uses bounding boxes to verify localization. However, the paper identifies three specific limitations that motivated TreeBench:

First, the questions are limited to primitive spatial queries. V* Bench primarily asks binary relational questions like "Is object A to the left of object B?" or attribute queries like "What color is the object?" As the paper notes:

"V* Bench is restricted to simple spatial queries (e.g., 'Is A left of B?')" (Section 1)

These questions test first-order visual perception — can the model find the object and read out a property — but they do not test second-order reasoning: what can the model infer from the spatial relationships it has perceived? This is a critical distinction. A model that can locate objects but cannot reason about their interactions (e.g., "From the perspective of person A, what is the relative direction of object B?") has not fully achieved visual grounded reasoning.

Second, COCO-derived images introduce contamination risk. V* Bench uses images from COCO (Lin et al., 2014), which is the most widely used dataset for training vision-language models. The paper raises the concern that models may have memorized features of specific COCO images during pretraining:

"V* ... relies on COCO-derived images, introducing high contamination risk." (Section 2)

While the extent of COCO contamination in modern multimodal models is debated, the concern is legitimate: if a model has seen an image (or a near-duplicate) during training, its performance on that image in evaluation reflects memorization, not generalization. TreeBench mitigates this by sampling images from SA-1B (Kirillov et al., 2023), a more recent and much larger dataset that is less saturated in training pipelines, and by using images with dense, cluttered scenes that are difficult to memorize.

Third, models are approaching saturation. The paper reports in Table 1 and Section 1 that even open-source models are achieving >90% accuracy on V* Bench, which means the benchmark is ceasing to discriminate between good and excellent models. A benchmark that everyone passes is no longer useful for driving progress. TreeBench, by contrast, sees state-of-the-art models like OpenAI-o3 scoring only 54.87% and Qwen2.5-VL-72B at 42.2%, leaving substantial headroom.

High-Resolution Benchmarks: No Traceability, Insufficient Reasoning Depth

MME-RealWorld (Zhang et al., 2024a) and HR-Bench (Wang et al., 2025f) evaluate models on high-resolution images with fine details, addressing the "small object" problem. However, the paper identifies two gaps:

First, they lack traceable evidence. These benchmarks evaluate only the final answer — was it correct or not? They do not provide bounding box annotations that would allow evaluating whether the model actually located the relevant objects. The paper makes this point explicitly:

"MME-RealWorld and HR-Bench support high-resolution inputs but lack traceable evidence, and their questions often become easy when grounded precisely." (Section 2)

The parenthetical observation — "their questions often become easy when grounded precisely" — is an important insight. It suggests that for these benchmarks, the primary difficulty is finding the relevant objects, not reasoning about them. Once an object is precisely localized, answering the question becomes straightforward. This makes these benchmarks primarily tests of visual search, not visual reasoning. TreeBench, by contrast, contains questions that remain difficult even after perfect localization (the second-order reasoning tasks), providing a more comprehensive evaluation.

Second, they lack second-order reasoning. Neither benchmark includes tasks like perspective transformation ("From the perspective of this person, which direction is the door?"), spatial containment reasoning ("Is the cup inside or outside the cabinet?"), or occlusion-based inference ("Which object is in front and which is behind?"). These tasks require the model to not just locate objects but to perform cognitive operations over their spatial relationships — operations that go beyond the primitive spatial queries in V* Bench.

Document Benchmarks: Niche and Missing Visual Reasoning

The paper briefly acknowledges that document understanding benchmarks like DocVQA (Mathew et al., 2021), TextVQA-related works (Biten et al., 2022), and OCRBench (Liu et al., 2023c) share some superficial similarities — they require models to locate and read small text in images, which is a form of fine-grained visual grounding. However, these benchmarks are domain-specific (documents, charts, infographics) and do not test the kind of real-world, open-domain visual reasoning that models like o3 are being deployed for. They also typically lack traceable evidence annotations at the level of individual reasoning steps.

LMM-Generated Benchmarks: Annotation Quality Concerns

The paper also implicitly critiques a growing trend in benchmark construction: using large multimodal models to automatically generate questions and answers at scale. Benchmarks like MMT-Bench (Ying et al., 2024) and aspects of SEED-Bench use LMM-generated labels. The paper argues that this approach "would inevitably introduce significant noise, compromising the quality of the annotations" (Section 1). The counter-argument, which the paper does not fully develop but implies, is that LMMs have known failure modes — hallucination, bias, visual blindspots — and using them to generate evaluation data risks baking those failure modes into the benchmark design. A benchmark built this way might test whether a model shares the same biases as the generating model, rather than testing genuine visual understanding.

TreeBench addresses this through its expert-driven annotation pipeline (8 annotators including 6 PhD candidates and 2 senior research scientists), but notably does use LMMs (o3 and Gemini-2.5-Pro) as proposal generators whose outputs are then manually curated, modified, or replaced. This hybrid approach — using LMMs to reduce the manual annotation burden while maintaining expert oversight — is a pragmatic compromise.

How This Paper Positions Itself

The paper's positioning can be understood through three deliberate design choices that distinguish TreeBench from all prior work:

Positioning Choice 1: Traceability as a First-Class Evaluation Dimension

This is the paper's most distinctive contribution to evaluation methodology. Most benchmarks evaluate only the final answer. TreeBench additionally evaluates the quality of the reasoning pathway by providing ground-truth bounding boxes for every target instance mentioned in every question. This enables computing mIoU (mean Intersection-over-Union) between predicted and ground-truth bounding boxes, quantifying localization precision independently of answer correctness.

The implications go beyond "we have a new metric." Traceability enables failure mode decomposition: by comparing mIoU and accuracy on the same question, you can determine whether a wrong answer was caused by a localization failure (low mIoU) or a reasoning failure (high mIoU but wrong answer). The paper demonstrates exactly this decomposition in Figures 9-10, showing that perception questions (Attributes, Material, Physical State) show strong mIoU-accuracy correlation, while reasoning questions (Perspective Transform, Spatial Containment) show weak correlation — meaning that even with perfect localization, models struggle with the reasoning step.

This is a fundamentally different kind of evaluation than what existed before. It transforms benchmarking from "how often is the model right?" to "when the model is wrong, what went wrong and at which stage?"

Positioning Choice 2: Second-Order Reasoning Beyond Localization

The paper explicitly distinguishes between first-order visual reasoning (locating an object and reading out an attribute — what most existing benchmarks test) and second-order visual reasoning (performing cognitive operations over localized visual evidence — what TreeBench uniquely tests). The Reasoning subset of TreeBench (256 out of 405 questions, or 63% of the benchmark) is deliberately weighted toward second-order tasks.

The five reasoning subtasks reveal the paper's conceptualization of what "thinking with images" should entail:

Perspective Transform: requires the model to adopt an agent-centric frame of reference rather than the viewer-centric frame. For example, "From the perspective of the person in the red shirt, which direction is the door?" This tests whether the model can perform a mental rotation of the coordinate system — a distinctly cognitive operation that goes beyond localization.
Ordering: requires resolving ordinal relationships (leftmost, rightmost, second from the left) among visually similar objects in dense arrangements. This tests sequential reasoning over spatial layouts.
Contact and Occlusion: requires understanding physical interactions — which objects are touching, which are occluding which — by parsing subtle visual cues like boundary fusion, shadow overlap, and depth ordering. This tests causal-physical reasoning from visual evidence.
Spatial Containment: requires understanding hierarchical spatial relationships (inside/outside, on/under, within/outside a region) that cannot be reduced to pairwise relative positions. This tests compositional spatial reasoning.
Comparison: requires comparing attributes (size, distance, color) across multiple co-present objects. This tests relative judgment rather than absolute identification.

None of these capabilities are tested by existing benchmarks. POPE and SEED-Bench don't require any spatial reasoning. V* Bench tests only primitive spatial relations (left/right, above/below). MME-RealWorld and HR-Bench test fine-grained recognition but not reasoning over recognized objects. The paper thus positions TreeBench as filling a specific and previously unoccupied niche in the evaluation landscape.

Positioning Choice 3: Expert-Driven Design with Difficulty as a Feature

The paper is explicit that TreeBench is meant to be hard — and that difficulty is achieved through deliberate design rather than accidental noise. The annotation pipeline incorporates a specific step called "Difficulty Filtering" (Section 3.2):

"Questions deemed insufficiently challenging are removed through model-based consensus screening. Specifically, any question answered correctly by all four state-of-the-art vision-language models (Qwen2.5-VL-72B, InternVL3-78B, GPT-4o, Gemini-2.5-Flash) was excluded."

This is a strong statement about the benchmark's intended role. TreeBench is not designed to rank models across the full spectrum of capability (for which a mix of easy, medium, and hard questions would be appropriate). It is designed to test the frontier — to identify the specific capabilities that even the best models lack, and to provide headroom for future improvement. The fact that no model reaches 60% accuracy, and that even the unprecedented visual reasoning capabilities of o3 leave substantial room for improvement, validates this design choice.

The small size of the benchmark (405 questions) is a consequence of this focus on quality and difficulty. The paper prioritizes annotation correctness and task representativeness over scale, explicitly contrasting with benchmarks that use LMM-generated labels to achieve larger sizes. The trade-off is acknowledged: 405 questions across 10 sub-tasks and 5 difficulty levels means relatively few examples per cell, which could limit statistical power for fine-grained comparisons. But the paper argues that this is the right trade-off for a diagnostic benchmark whose primary purpose is identifying qualitative failure modes rather than producing precise model rankings.

Relationship to Prior Work on Visual Grounded Reasoning Training

The paper positions TreeVGR as both building on and critiquing the emerging line of work on training models to "think with images." It identifies a specific, subtle flaw in prior RL-based approaches (DeepEyes, Pixel-Reasoner, GRIT, Ground-R1, VisionReasoner):

"Critically, these RL-based methods supervise only the final answer. In contrast, our TreeVGR emphasizes traceable evidence during RL training, i.e., supervising generated bounding boxes to ensure precise localization throughout the reasoning process." (Section 2)

The paper's argument is that if you only reward the final answer, the model might learn to produce correct answers through routes that don't involve faithful visual grounding — for instance, by learning spurious correlations between question patterns and answer choices, or by generating plausible-sounding bounding boxes that are not actually used in the reasoning step. The dual IoU reward in TreeVGR is designed to close this gap by explicitly rewarding the model for producing accurate bounding boxes, ensuring that the intermediate grounding steps are faithful, not just plausible.

This positions TreeVGR as a methodological contribution to RL training design (the dual IoU reward formulation) as well as an empirical demonstration that supervising intermediate steps matters (the ablation showing that traceable RL outperforms untraceable RL in Table 6, \circled{3} vs. \circled{4}).

What the Paper Does Not Claim to Solve

The paper's positioning is characterized as much by what it acknowledges it does not address as by what it does. It explicitly notes (Section 6, Limitations) that TreeBench is small (405 questions), that TreeVGR is trained only at the 7B scale, and that expanding both to larger scales and broader domains is future work. It does not claim that TreeVGR achieves state-of-the-art overall performance (open-source SOTA models like InternVL3-78B still outperform it on TreeBench overall, though TreeVGR matches or exceeds them on perception-heavy subsets). And it does not claim that the dual IoU reward solves all grounding problems — the paper shows that even with improved localization, second-order reasoning remains difficult (Figure 10).

This modesty is, in itself, a form of positioning. The paper is not announcing that it has solved visual grounded reasoning; it is announcing that it has built the measurement infrastructure (TreeBench) and the training methodology (TreeVGR) that will enable the community to make progress on it. The combination of benchmark + method is presented as a foundation for future work, not a destination.

3. Technical Approach

3.1 Reader Orientation

TreeVGR is a two-stage training pipeline that teaches a pre-trained multimodal model to perform "grounding-then-answering" — first locating task-relevant objects by predicting their bounding boxes, then reasoning about those objects to produce a final answer. The system addresses the problem that existing methods for visual grounded reasoning train only on final answer correctness, which means the model can learn to produce right answers through spurious shortcuts without ever actually grounding its reasoning in the correct visual regions. The "shape" of the solution is a reinforcement learning framework with a novel dual IoU reward that explicitly supervises the quality of intermediate bounding box predictions, enforcing both precision (every predicted box should match some real object) and recall (every real object should have some matching predicted box), thereby creating transparent, auditable reasoning pathways where you can verify not just that the model answered correctly, but whether it genuinely looked at the right things.

3.2 Big-Picture Architecture (Diagram in Words)

TreeVGR consists of two sequential training stages, each applied to a pre-trained multimodal model (Qwen2.5-VL-7B-Instruct), plus the inference procedure that the trained model executes at test time:

Cold-Start Initialization (Stage 1): A supervised fine-tuning phase that teaches the model the format of visual grounded reasoning — namely, generating bounding boxes for relevant objects as part of its reasoning chain before producing an answer. The training data comprises 35K examples (TreeVGR-SFT-35K), each containing an image, a question, a reasoning trajectory with bounding box annotations, and a final answer. The model is initialized from Qwen2.5-VL-7B-Instruct and fine-tuned on these trajectories.
Reinforcement Learning with Traceable Evidence (Stage 2): A reinforcement learning phase that refines the quality of the grounding and reasoning. Using GRPO (Group Relative Policy Optimization), the model generates candidate reasoning trajectories (including bounding boxes and answers), and receives a composite reward that includes (a) whether the final answer is correct, (b) whether the output follows the required format, and (c) most distinctively, a dual IoU reward that measures how well the predicted bounding boxes align with ground-truth bounding boxes. This reward has two complementary terms — recall (ensuring all ground-truth objects are covered) and precision (ensuring predicted boxes aren't spurious) — that together prevent the model from cheating by enumerating too many boxes or missing key objects. The RL training uses 37K samples (TreeVGR-RL-37K) drawn from V* and VisDrone.
Inference (test-time behavior): At test time, the trained TreeVGR model receives an image and a question. It internally generates a reasoning chain that includes predicted bounding boxes (in absolute pixel coordinates) for task-relevant objects, then produces a final answer. No image cropping or re-feeding is performed — the bounding boxes exist purely in text space, making inference computationally efficient compared to methods that actually crop and re-process image regions (as OpenAI-o3 does internally). This design choice trades some potential accuracy for substantially faster inference.

3.3 Roadmap for the Deep Dive

First, the cold-start initialization phase — why it's needed, how the training data is constructed, the coordinate conversion from normalized to absolute coordinates, the reflective subset design, and the optimization hyperparameters.
Second, the reinforcement learning stage — the reward design (accuracy, format, and dual IoU), how each reward term is computed, and why the dual IoU reward is structured as an average of recall and precision terms rather than a single IoU metric.
Third, the mathematical formulation of the dual IoU reward in detail — the recall term that matches each ground-truth to the best prediction, the precision term that matches each prediction to the best ground-truth, and why both are necessary to prevent degenerate behavior.
Fourth, the RL training data construction — how hard samples are filtered from V*, how VisDrone is reformulated into multiple-choice counting problems, and the optimization configuration (GRPO, hardware setup, reward model serving).
Fifth, the overall training pipeline in terms of what happens sequentially — cold-start SFT followed by traceable RL — and why this two-stage design is more compute-efficient than end-to-end RL from the base model.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methodology paper whose core idea is that visual grounded reasoning can be substantially improved by explicitly supervising the intermediate bounding box predictions during reinforcement learning, rather than only supervising the final answer. The key innovation is the dual IoU reward formulation that simultaneously enforces precision and recall in localization, preventing the model from either missing relevant objects or generating spurious boxes to game the reward.

Cold-Start Initialization

Why a cold-start phase is necessary. The paper identifies a specific practical bottleneck with end-to-end reinforcement learning for visual grounded reasoning: it is computationally prohibitive. The authors cite DeepEyes-7B (Zheng et al., 2025b) as a representative example, which required RL training on 47K samples for 32 episodes using 32 H100 (80GB) GPUs operating continuously for 48 hours. This resource intensity creates what the authors describe as "barriers to iterative experimentation and broader accessibility" (Section 4.1).

More fundamentally, the authors observed a training efficiency problem when applying RL directly to visual grounded reasoning without any initialization:

"models required extensive iterations to autonomously identify task-relevant visual regions before generating answers."

In plain language: when you throw an untrained model into RL and ask it to both learn what to look at and how to reason about it, the exploration space is enormous. The model spends many training episodes essentially flailing — generating random bounding boxes, getting low rewards, and slowly (very slowly) discovering that accurate grounding precedes correct answers. The cold-start phase collapses this exploration problem by providing the model with explicit demonstrations of the desired behavior: here is an image, here is a question, here are the objects you should locate, here are their bounding boxes, and here is the reasoning that leads from those boxes to the answer.

The cold-start phase thus serves as a behavioral prior — it teaches the model the format and intent of grounding-then-answering before RL refines the quality of the grounding and reasoning. This is analogous to how instruction tuning precedes RLHF in LLM training: SFT establishes the basic capability and output format, and RL optimizes for quality within that format.

Data construction (TreeVGR-SFT-35K). The supervised fine-tuning dataset is built on top of VGR-158K (Wang et al., 2025e), which provides pseudo-chain-of-thought annotations paired with bounding boxes for visual reasoning tasks. The paper applies three transformations to create the final SFT dataset:

Coordinate system conversion. VGR-158K was built for LLaVA-NeXT (Liu et al., 2024a), which uses normalized coordinates — bounding boxes specified as fractions of image dimensions, with values ranging from 0 to 1. However, the base model for TreeVGR is Qwen2.5-VL (Bai et al., 2025a), which outputs absolute coordinates in pixel units. This mismatch would make the training data incompatible with the model's native output format. The conversion is straightforward: for each bounding box originally expressed as normalized coordinates $[r_{x_1}, r_{y_1}, r_{x_2}, r_{y_2}]$ , the absolute coordinates are computed as:

$[x_1, y_1, x_2, y_2] = [W r_{x_1}, H r_{y_1}, W r_{x_2}, H r_{y_2}]$

where $W$ and $H$ are the width and height of the input image in pixels. Each normalized coordinate is multiplied by the corresponding image dimension to produce the pixel-level coordinate that Qwen2.5-VL expects.
Filtering for complex reasoning pathways. The paper retains only samples with multiple bounding boxes (more than one box per reasoning trajectory). This filtering is motivated by a specific instructional hypothesis:

"multi-box interactions demand stronger spatial-temporal reasoning compared to single-box tasks."

The intuition is that if a reasoning task involves only one object, the model can succeed by simply detecting that object and reading out its properties — a first-order perceptual task. Tasks involving multiple objects require the model to reason about relationships between objects (spatial, comparative, causal), which is precisely the second-order reasoning capability that TreeBench is designed to evaluate. Filtering for multi-box samples ensures the SFT data emphasizes relational reasoning over simple detection. After this filtering, 35K samples remain.
Reflective subset with synthetic errors. From the 35K multi-box samples, the authors construct a smaller subset of 4.7K samples with an additional training signal: error detection and correction. For each of these samples, the procedure is:
- Inject a synthetic error by inserting a randomly generated incorrect bounding box into the reasoning sequence (e.g., a box that points to an irrelevant region of the image).
- Append the meta-cognitive prompt "Wait, this box seems to be wrong" immediately after the erroneous box.
- Continue with the correct reasoning trajectory from that point.
This design is important because it explicitly teaches the model to detect and recover from grounding errors — a skill that is critical for robust real-world deployment where the model's initial grounding attempts may be imperfect. The model learns not just to produce correct boxes, but to recognize when it has produced an incorrect box and to self-correct. This reflective capability is what the authors refer to as "a critical skill for robust real-world deployment" (Section 4.1).

The resulting dataset, TreeVGR-SFT-35K, thus contains 35K samples total, of which 4.7K include the reflective error-correction pattern. All samples have the structure: image, question, reasoning trajectory with multiple bounding boxes (some potentially erroneous in the reflective subset), and final answer.

Optimization details. The cold-start model, referred to as TreeVGR-7B-CI ("CI" for Cold Initialization), is trained from Qwen2.5-VL-7B-Instruct. The training configuration is:

Hardware: 8 GPUs (exact GPU model not specified, but the RL stage uses H100s, suggesting similar hardware for SFT).
Framework: LLaMA-Factory (Zheng et al., 2024).
Optimizer: AdamW (Loshchilov and Hutter, 2017).
Learning rate: $5 \times 10^{-6}$ .
Global batch size: 256.
Learning rate schedule: Cosine decay (Loshchilov and Hutter, 2016) with a warmup ratio of 0.1.

The warmup ratio of 0.1 means that for the first 10% of training steps, the learning rate linearly increases from 0 to the peak value of $5 \times 10^{-6}$ , then follows a cosine curve down to (near) zero by the end of training. This is standard practice for stabilizing early training when the model weights are being adapted to a new task distribution.

What the cold-start model can do. After this phase, TreeVGR-7B-CI has learned the format of visual grounded reasoning: when given an image and a question, it produces a reasoning chain that includes bounding box predictions (in absolute coordinates) for task-relevant objects, followed by a final answer. However, the quality of the bounding boxes and the reasoning is limited by the quality of the pseudo-labels in VGR-158K. The RL phase is designed to refine this quality.

Reinforcement Learning with Traceable Evidence

Motivation for RL after SFT. The cold-start model knows how to produce grounded reasoning chains, but it does not necessarily produce good ones. The bounding boxes may be imprecise, the reasoning may be incomplete, and the model has no direct training signal that rewards accurate localization. The RL phase addresses this by providing a composite reward signal that jointly evaluates (a) whether the final answer is correct, (b) whether the output follows the required format, and (c) critically, whether the predicted bounding boxes accurately localize the ground-truth objects.

The paper makes a specific argument for why traceable evidence supervision during RL matters beyond just accuracy-based RL:

"Critically, these RL-based methods supervise only the final answer. In contrast, our TreeVGR emphasizes traceable evidence during RL training, i.e., supervising generated bounding boxes to ensure precise localization throughout the reasoning process." (Section 2)

The concern is that if you only reward the final answer, the model can learn to produce correct answers through routes that don't involve faithful visual grounding. For instance, the model might learn to generate plausible-sounding but incorrect bounding boxes, then produce the correct answer by relying on language priors or memorized associations rather than the visual evidence those boxes purportedly represent. The dual IoU reward closes this loophole by requiring that the intermediate grounding steps be verifiably accurate.

Reward design. The total reward $R$ for a generated trajectory is the sum of three components:

$R = R_{\text{acc}} + R_{\text{format}} + R_{\text{IoU}}$

where $R_{\text{acc}} \in \{0, 1\}$ is the accuracy reward, $R_{\text{format}} \in \{0, 1\}$ is the formatting reward, and $R_{\text{IoU}} \in [0, 1]$ is the dual IoU reward.

What it computes: a scalar total reward in the range $[0, 3]$ that the RL algorithm (GRPO) uses to update the policy. The policy model generates a complete reasoning trajectory (bounding boxes + reasoning + answer), and this reward function scores the trajectory on three independent axes. Because the terms are additive (not multiplicative or conditional), each term provides an independent gradient signal — the model can learn to improve its format even when the answer is wrong, and it can learn to improve its localization even when both format and answer are already correct.

Why sum rather than product or minimum: the additive form means the model receives partial credit for partial improvements. If the dual IoU term were multiplied by the accuracy term (product), then the model would receive zero localization gradient whenever the answer was wrong, which would slow learning because most trajectories start with wrong answers. If the minimum were used, the model would be penalized for any weakness regardless of strengths, which could create a harsh learning signal. The sum provides a smooth, decomposable reward landscape.

Accuracy reward ( $R_{\text{acc}}$ ). This term evaluates whether the model's final answer matches the ground-truth answer. The evaluation method depends on the question type:

For multiple-choice questions, exact string matching is used — the model's predicted option letter (A, B, C, D, etc.) is compared directly to the ground-truth option letter.
For open-ended questions (which appear in the V* portion of the RL training data), the paper uses an LLM-based judge: Qwen2.5-72B-Instruct (Team, 2024) serves as an online reward model that receives the question, the ground-truth answer, and the model's predicted answer, and outputs a binary judgment of correctness.

The use of an LLM judge for open-ended questions is a pragmatic choice: exact matching fails when answers are semantically equivalent but lexically different (e.g., "the red one" vs. "the crimson object"), while LLM judges can handle synonymy and paraphrase. However, LLM judges introduce their own noise and biases, which the paper does not extensively analyze.

Format reward ( $R_{\text{format}}$ ). This term is a binary reward (0 or 1) that enforces a specific output structure. The model's reasoning process must be enclosed between thinking and response tags, and the final answer must be enclosed between <answer> and </answer> tags. If these structural constraints are satisfied, the format reward is 1; otherwise, it is 0.

This may seem like a minor implementation detail, but it serves an important function in RL training. Without a format reward, the model's outputs can drift into unstructured text that is difficult to parse automatically — for instance, the model might interleave reasoning and answer in ways that make it impossible to extract the predicted bounding boxes for IoU computation. The format reward anchors the model to a parseable output structure throughout training, ensuring that the other reward components can be reliably computed.

Dual IoU reward ( $R_{\text{IoU}}$ ). This is the core technical contribution of TreeVGR. It measures the alignment between the model's predicted bounding boxes and the ground-truth bounding boxes. The key design choice is that it is dual — composed of two complementary terms that address different failure modes:

$R_{\text{IoU}} = \frac{1}{2}(R_{\text{IoU}}^{\text{R}} + R_{\text{IoU}}^{\text{P}})$

where $R_{\text{IoU}}^{\text{R}}$ is the recall term and $R_{\text{IoU}}^{\text{P}}$ is the precision term.

What it computes: the arithmetic mean of a recall score and a precision score, each in $[0, 1]$ . The recall score measures how completely the predictions cover the ground-truth objects; the precision score measures how many of the predictions are actually matched to ground-truth objects (versus being spurious). The symmetric averaging ensures that the model cannot maximize the reward by either enumerating many boxes (which would increase recall but hurt precision) or predicting very few boxes (which might have high precision but miss some ground-truth objects).

Why average rather than F1 (harmonic mean): the harmonic mean (F1 score) would penalize extreme imbalances more heavily than the arithmetic mean. For an RL training signal, the arithmetic mean provides a smoother gradient landscape — small improvements in either recall or precision translate directly into proportional improvements in the reward, without the nonlinear interaction that the harmonic mean introduces. This is important for stable policy gradient updates, where reward smoothness affects convergence.

The recall term ( $R_{\text{IoU}}^{\text{R}}$ ). This term ensures that every ground-truth object has at least one predicted box that overlaps with it. Formally, for $M$ ground-truth boxes $\{\bm{b}_k\}_{k=1}^{M}$ and $N$ predicted boxes $\{\hat{\bm{b}}_i\}_{i=1}^{N}$ :

$R_{\text{IoU}}^{\text{R}} = \frac{1}{M} \sum_{k=1}^{M} \max_i \text{IoU}(\hat{\bm{b}}_i, \bm{b}_k)$

where $\text{IoU}(\hat{\bm{b}}_i, \bm{b}_k)$ is the standard Intersection-over-Union between predicted box $i$ and ground-truth box $k$ , and $\max_i$ selects the best-matching prediction for each ground-truth.

What it computes: for each ground-truth bounding box $\bm{b}_k$ , find the predicted box $\hat{\bm{b}}_i$ that maximizes the IoU with it (i.e., the prediction that best covers this ground-truth object). Take the IoU of that best match. Average these best-match IoUs across all $M$ ground-truth boxes. The result is a score in $[0, 1]$ where 1 means every ground-truth box has a perfect-matching prediction, and 0 means at least one ground-truth box has no overlapping prediction at all.

Operational behavior: maximizing this term encourages the model to produce at least one predicted box for every ground-truth object. If a ground-truth object is completely missed (no prediction overlaps with it), its contribution to the average is 0, pulling the recall term down. The model can increase recall by either (a) predicting boxes that better cover existing ground-truths, or (b) predicting additional boxes to cover previously missed ground-truths.

The precision term ( $R_{\text{IoU}}^{\text{P}}$ ). This term ensures that predicted boxes are not spurious — each prediction should correspond to some ground-truth object. Formally:

$R_{\text{IoU}}^{\text{P}} = \frac{1}{N} \sum_{i=1}^{N} \max_k \text{IoU}(\bm{b}_k, \hat{\bm{b}}_i)$

where the roles are reversed: for each predicted box, find the best-matching ground-truth box.

What it computes: for each predicted bounding box $\hat{\bm{b}}_i$ , find the ground-truth box $\bm{b}_k$ that maximizes the IoU with it (i.e., the real object that this prediction is most likely referring to). Take the IoU of that best match. Average these best-match IoUs across all $N$ predicted boxes. The result is a score in $[0, 1]$ where 1 means every predicted box perfectly matches some ground-truth object, and 0 means at least one predicted box matches no ground-truth at all.

Operational behavior: maximizing this term discourages the model from generating "junk" boxes — predictions that don't correspond to any real target object. If the model predicts a box in an empty region of the image, that box will have near-zero IoU with every ground-truth, dragging the precision average down. The model can increase precision by either (a) improving the alignment of existing predictions with ground-truths, or (b) reducing the number of predictions to eliminate low-quality ones.

Why both terms are necessary (the paper's key empirical finding). The paper discovers through ablation (Table 6, \circled{3} vs. \circled{5} and \circled{3} vs. \circled{6}) that removing either term causes specific failure modes:

Without the precision term (recall-only IoU): the model learns to enumerate candidate boxes exhaustively. As the authors observe, "the model tends to enumerate candidate boxes to obtain larger IoU recall, and fails to produce final answers." The mean response length grows rapidly during training (Figure 14), because the model discovers that predicting more and more boxes increases the chance that at least one will cover each ground-truth — a degenerate strategy that maximizes $R_{\text{IoU}}^{\text{R}}$ while producing unusable outputs. The model gets stuck generating boxes and never transitions to producing an answer.
Without the recall term (precision-only IoU): the model achieves significant accuracy improvements but with limited localization quality — it "usually grounds incomplete target instances." The model learns to predict a small number of high-confidence boxes (maximizing precision) but misses some ground-truth objects entirely, meaning its reasoning may be based on incomplete visual evidence.

The dual formulation solves both problems simultaneously. To maximize $R_{\text{IoU}}$ , the model must: (a) predict enough boxes to cover all ground-truths (recall pressure), but (b) not predict so many boxes that some are spurious (precision pressure), and (c) make each predicted box as accurate as possible (both terms benefit from higher per-box IoU). The equilibrium is a set of predictions that exactly covers the ground-truth objects with high spatial accuracy, which is precisely the desired behavior for faithful visual grounding.

Relationship to standard object detection metrics. The dual IoU reward is structurally similar to the Average Precision (AP) metric used in object detection, which also balances precision and recall. However, there is an important difference: AP uses a threshold (typically IoU > 0.5) to determine whether a prediction "matches" a ground-truth, turning the continuous IoU into a binary match/no-match decision. The dual IoU reward uses the raw continuous IoU values, which provides a smoother training signal — a prediction with IoU 0.49 receives almost the same reward as one with IoU 0.51, rather than being treated as a complete miss. This smoothness is important for RL, where the policy gradient depends on reward differences between similar outputs.

RL training data construction (TreeVGR-RL-37K). The RL phase requires training data with ground-truth bounding boxes, since the dual IoU reward needs something to compare predictions against. The paper constructs 37K samples from two sources:

V* (30K samples): The original V* training set (Wu and Xie, 2024) contains 191K samples, but these are of varying difficulty. The paper filters for hard samples only, using Qwen2.5-VL-7B-Instruct as a difficulty filter — any sample that the base model can already answer correctly is removed, retaining only those where the model's initial performance is poor. This yields 30K open-ended question-answering samples, each with an image, a question, a ground-truth answer, and bounding box annotations for all target instances. The filtering ensures that the RL phase focuses on samples where there is genuine room for improvement, rather than wasting compute on samples the model already handles well.
VisDrone (7K samples): VisDrone (Zhu et al., 2021) is a dataset originally designed for object detection and tracking in UAV (drone) imagery. It features extremely high-resolution, real-world aerial scenes with large numbers of small objects (people, vehicles, etc.) and comes with bounding box annotations. The paper reformulates the detection task into multiple-choice counting problems: given an image containing many small objects, how many objects of a specified class are present? The answer choices are numerical counts. The paper filters to retain only samples where the ground-truth count is between 5 and 10, yielding 7K samples.

Why VisDrone? The paper's motivation is that VisDrone images are challenging in ways that align with TreeBench's design principles: extremely high resolution, dense clusters of small objects, and the need for precise localization to count accurately. Training on VisDrone counting problems should transfer to the fine-grained counting and localization capabilities that TreeBench evaluates. The constraint on counts (5-10) ensures the counting tasks are non-trivial — too few objects and the task is easy, too many and it becomes prohibitively difficult even for humans — and that the model must engage in systematic localization rather than subitizing (the human ability to instantly recognize quantities up to ~4 without counting).

Optimization details for RL. The RL training uses the following configuration:

Initialization: TreeVGR-7B-CI (the cold-start model).
Algorithm: Group Relative Policy Optimization (GRPO) (Shao et al., 2024b). GRPO is a variant of Proximal Policy Optimization (PPO) that removes the value function critic, instead computing advantages by comparing each generated response to the average reward of a group of responses generated for the same prompt. This reduces memory requirements (no separate value network) and has been shown effective for reasoning tasks.
Hardware: 8 GPUs for training the policy model, plus an additional 8 GPUs for serving the reward model (Qwen2.5-72B-Instruct) using vLLM (Kwon et al., 2023). The reward model is needed to judge answer correctness for open-ended questions; serving it on separate GPUs avoids memory contention with the policy model.
Framework: EasyR1 (Zheng et al., 2025a), described as "a clean fork of veRL" (Sheng et al., 2024).
Epochs: 5 epochs on TreeVGR-RL-37K. The paper explicitly contrasts this with DeepEyes-7B, which used 32 epochs on 47K samples — TreeVGR achieves its results with substantially less training, which the authors attribute to the efficiency gains from cold-start initialization and the dual IoU reward signal.

Why GRPO over alternatives. The paper reports that they also tried DAPO (Yu et al., 2025), another RL algorithm, but found it "unstable compared with GRPO." The instability likely manifests as reward oscillation or policy collapse during training — common failure modes in RL when the advantage estimates are noisy or the policy updates are too aggressive. GRPO's group-based advantage estimation (comparing each response to its group's average rather than a learned value function) tends to be more stable because it doesn't suffer from value function approximation errors, though it requires generating multiple responses per prompt, increasing the per-step compute cost.

Training efficiency comparison. The paper makes a specific claim about training efficiency:

"We train our TreeVGR-7B with 5 epochs on TreeVGR-RL-37K, which is significantly less than DeepEyes-7B (which is trained on 47K samples with 32 epochs)."

The factors contributing to this efficiency difference are: (1) the cold-start initialization means the model doesn't need to discover the grounding-then-answering format from scratch during RL; (2) the dual IoU reward provides a dense training signal (every predicted box gets feedback) rather than just a sparse terminal reward (only the final answer gets feedback); and (3) the VisDrone counting data provides additional challenging localization training that may accelerate learning of precise bounding box prediction.

Inference procedure. At test time, the trained TreeVGR model receives an image and a question and autonomously generates a reasoning chain. The paper makes a specific design choice worth highlighting:

"Notably, our TreeVGR does not require actually replaying cropped images as previous approaches do, as text-space grounding is already effective. It leads to much more efficient training and inference procedures."

This is a significant practical distinction. Models like OpenAI-o3 and some prior open-source approaches (VGR, DeepEyes, Pixel-Reasoner) operate by: (1) generating bounding boxes, (2) cropping those regions from the original image, (3) re-encoding the cropped regions through the vision encoder, and (4) reasoning over the higher-resolution crops. This "re-playing" of image crops is computationally expensive because it requires multiple passes through the vision encoder and LLM for each reasoning step.

TreeVGR, by contrast, performs grounding entirely in text space: the bounding box coordinates are generated as text tokens, and the reasoning proceeds over these text-space representations without ever re-encoding cropped image regions. This means a single forward pass through the model suffices for the entire reasoning chain (assuming autoregressive generation), making inference substantially faster and more memory-efficient. The trade-off is that text-space grounding provides lower visual fidelity than actual image crops — the model can't "zoom in" to see fine details that might be invisible at the original resolution — but the paper's results suggest that for the tasks evaluated, this fidelity loss is outweighed by the efficiency gains.

Design Choices and Their Justifications (Summary)

Two-stage pipeline (SFT + RL) over end-to-end RL: end-to-end RL requires the model to simultaneously discover the grounding format and optimize grounding quality, leading to slow convergence and high compute costs (as demonstrated by DeepEyes-7B's 32-epoch, 48-hour training). The cold-start SFT phase teaches the format efficiently via supervised demonstrations, leaving RL to focus on quality optimization.
Dual IoU over single-term IoU: a recall-only reward causes the model to enumerate spurious boxes; a precision-only reward causes the model to miss ground-truth objects. The dual formulation's symmetric averaging prevents both failure modes.
Arithmetic mean over harmonic mean (F1) for the dual IoU: the arithmetic mean provides a smoother reward landscape for policy gradient optimization, where small improvements in either recall or precision translate linearly into reward improvements.
Absolute coordinates over normalized coordinates: Qwen2.5-VL's native output format uses absolute pixel coordinates, so using normalized coordinates would create a mismatch between the training data format and the model's expected output format, potentially degrading performance.
Reflective subset with synthetic errors: explicitly training the model to detect and correct its own grounding errors builds robustness for real-world deployment where initial grounding attempts may be imperfect.
VisDrone for RL training data: the small-object, high-resolution, dense-scene characteristics of VisDrone align with the challenges TreeBench evaluates, providing complementary localization training to the V* data.
GRPO over DAPO: empirical stability — DAPO was found to be unstable during training, likely due to noisier advantage estimation.
Separate reward model serving: using 8 GPUs to serve Qwen2.5-72B-Instruct via vLLM for open-ended answer judging avoids memory contention with the policy model on the training GPUs, enabling larger batch sizes and faster training.

4. Key Insights and Innovations

Innovation 1: Traceable Evidence as a First-Class Evaluation Dimension

The paper's most fundamental intellectual contribution is the elevation of traceability from a nice-to-have feature to a core evaluation dimension that is co-equal with accuracy. Prior benchmarks (POPE, MMBench, MMMU, V* Bench, MME-RealWorld, HR-Bench) all share the same evaluative structure: a model produces a final answer, and that answer is scored as correct or incorrect. This binary judgment treats the model as an opaque function — accuracy is the only observable, and when the model is wrong, why it was wrong remains invisible.

TreeBench breaks this opacity by providing ground-truth bounding box annotations for every target instance in every question, enabling the computation of mIoU between predicted and ground-truth boxes. This is not merely "adding another metric." It transforms evaluation from a scalar output (accuracy) into a two-dimensional diagnostic space where each question can be located by its (mIoU, correctness) coordinates. The paper exploits this in Figures 9–10: by plotting the histogram of IoU values separately for correct and incorrect predictions, it reveals that low IoU is characteristic of wrong answers on perception questions (Attributes, Material), but that even high-IoU predictions can be wrong on reasoning questions (Perspective Transform, Spatial Containment). This decomposition — which would be impossible without traceable annotations — yields the paper's most important empirical insight: localization precision and reasoning capability are partially decoupled, and the gap between them is where "thinking with images" breaks down.

This is a genuine conceptual advance in multimodal evaluation methodology. It shifts the question from "how accurate is the model?" to "when the model fails, does it fail because it couldn't see the right thing, or because it couldn't think about what it saw?" The field has implicitly recognized this distinction — hence the separate literatures on visual grounding benchmarks and reasoning benchmarks — but no prior benchmark had integrated both into a single evaluation framework where the same question yields both a localization score and a reasoning score. TreeBench makes the interaction between these two capabilities measurable, which opens the door to fine-grained failure analysis that was previously impossible.

The significance goes beyond measurement. Traceability changes what can be claimed about model behavior. Without traceable evidence, a model that answers "What color is the ball?" correctly might have genuinely located the ball and read its color, or it might have inferred the answer from context (e.g., all balls in this dataset are red). With traceable evidence, the bounding box prediction provides an auditable record of whether the model actually attended to the correct object. This transforms claims about visual grounding from unverifiable assertions into empirically testable hypotheses. As the reasoning LMM literature grows and models increasingly claim to "think with images," this kind of verification infrastructure becomes essential, not optional.

Innovation 2: The Dual IoU Reward as a Training Signal That Prevents Reward Hacking

The second distinctive contribution is the dual IoU reward formulation for reinforcement learning, which addresses a specific and previously unexamined pathology in training visual grounded reasoning models. The paper's ablation (Table 6) reveals that naive IoU-based rewards create perverse incentives that degrade, rather than improve, model behavior: a recall-only reward causes the model to enumerate ever-larger sets of bounding boxes, maximizing recall by saturating the image with predictions (Figure 14 shows the mean response length exploding), while a precision-only reward causes the model to predict too few boxes, missing target objects entirely.

The conceptual move here is recognizing that localization reward design is subject to the same reward hacking dynamics that plague RLHF-style training, and that the solution follows the same structural pattern: balance complementary metrics that create opposing pressures. In RLHF, the KL penalty balances reward maximization against distributional drift; in TreeVGR, the precision term balances the recall term, preventing the model from gaming either metric in isolation. This is not a trivial observation — it required the paper to first discover the failure modes empirically and then design the symmetric reward to address them.

Prior RL-based visual grounded reasoning methods (DeepEyes, Pixel-Reasoner, GRIT, Ground-R1) either used no localization reward at all (relying only on answer accuracy) or used simple IoU metrics without the dual structure. The paper's ablation (\circled{3} vs. \circled{4} in Table 6) demonstrates that traceable RL with the dual IoU reward substantially outperforms untraceable (accuracy-only) RL, particularly on out-of-distribution benchmarks like TreeBench and MME-RealWorld-Lite. This establishes a concrete best practice: when training models to produce intermediate grounding outputs, those outputs must be explicitly supervised with a balanced reward, or the model will find ways to satisfy the superficial metric without achieving genuine grounding.

The significance extends to the broader RL-for-reasoning literature. As multimodal models increasingly generate structured intermediate outputs (bounding boxes, crop regions, tool calls, code snippets), the question of how to reward those intermediate outputs faithfully — without creating reward hacking incentives — becomes critical. The dual IoU formulation provides a template: identify the degenerate strategies that optimize each component metric in isolation, then design a composite reward where those strategies are mutually penalizing. This pattern — pair a coverage metric (recall) with a parsimony metric (precision) — is likely applicable to many structured output domains beyond bounding boxes.

Innovation 3: The Cold-Start + Traceable RL Pipeline as a Compute-Efficient Alternative to End-to-End RL

The paper's third contribution is a methodological insight about training efficiency: that a short supervised fine-tuning phase teaching the output format, followed by traceable RL refining the output quality, is substantially more compute-efficient than end-to-end RL that must simultaneously discover both format and quality. The paper provides a concrete comparison point: DeepEyes-7B required 32 epochs of RL on 47K samples using 32 H100 GPUs for 48 hours to achieve its results, while TreeVGR-7B required only 5 epochs on 37K samples using 16 GPUs (8 for training, 8 for reward model serving), plus a cold-start SFT phase that the paper characterizes (by contrast with the RL cost) as computationally modest.

This is not an incidental efficiency gain — it reflects a deeper insight about what makes RL difficult in this domain. The exploration space for visual grounded reasoning is enormous: the model must simultaneously decide which objects to attend to, where they are located (continuous bounding box coordinates), how to reason about them, and what answer to produce. RL from scratch must explore this space with only sparse terminal rewards (answer correctness) to guide it, which is why DeepEyes required 32 full epochs. The cold-start phase collapses the exploration problem by providing supervised demonstrations of the correct output format (what grounding-then-answering looks like) and initial bounding box quality, leaving RL to optimize within a much narrower space of refinements.

This two-stage design pattern — SFT for format, RL for quality — is well-established in text-only LLM training (instruction tuning before RLHF), but the paper demonstrates that it transfers to the multimodal grounding domain and provides specific evidence for why the transfer is particularly important here. The reflective subset of the SFT data (4.7K samples with synthetic errors and self-correction prompts) is a novel addition that explicitly teaches the model to recover from grounding mistakes, a capability that would be extremely expensive to discover through RL exploration alone.

The broader significance is for the growing community of researchers training open-source visual grounded reasoning models. The paper's recipe — cold-start SFT on converted VGR-158K data, followed by 5 epochs of GRPO with dual IoU reward on filtered V* + VisDrone data — is concrete, reproducible, and substantially cheaper than prior approaches. By demonstrating that this recipe achieves strong results (matching or exceeding DeepEyes and Pixel-Reasoner on multiple benchmarks) at a fraction of the training cost, the paper lowers the barrier to entry for research on visual grounded reasoning, which could accelerate progress in the field.

Innovation 4: The Discovery That Localization and Reasoning Are Partially Independent Capabilities, and That the Gap Between Them Defines the Frontier

The paper's fourth contribution is an empirical finding with significant implications for how the field conceptualizes visual reasoning: localization precision (mIoU) correlates strongly with accuracy on first-order perception tasks, but the correlation weakens substantially on second-order reasoning tasks (Figures 9–10). On Perception questions, correct answers cluster at high IoU values and wrong answers at low IoU values — the relationship is approximately what you would expect if visual grounding were the primary bottleneck. On Reasoning questions, the distributions overlap much more: there are wrong answers with high IoU (the model found the right objects but reasoned incorrectly about them) and correct answers with moderate IoU (the model reasoned correctly despite imprecise localization).

This finding is important because it challenges an implicit assumption in the visual grounded reasoning literature: that better grounding necessarily leads to better reasoning. If grounding and reasoning were tightly coupled, then improving localization (through better vision encoders, higher resolution, or RL with IoU rewards) would be sufficient to improve overall task performance. The weak correlation on Reasoning questions suggests otherwise — that there are reasoning capabilities that are bottlenecked by something other than visual acuity, and that these capabilities may require different training strategies than localization-focused ones.

The paper doesn't fully explore what these alternative bottlenecks are, but the breakdown by Reasoning subtask provides suggestive evidence. Perspective Transform questions require the model to perform a coordinate-system rotation (adopting an agent-centric rather than viewer-centric frame), which is a spatial reasoning operation that does not become easier with more precise bounding boxes. Spatial Containment questions require understanding hierarchical relationships (inside/outside, on/under) that depend on 3D scene geometry, not 2D box positions. These capabilities likely require different training data and objectives than localization accuracy.

This finding reframes the challenge of visual grounded reasoning: it is not a single capability to be optimized but a pipeline with at least two partially independent stages (localization and reasoning), each with its own failure modes and improvement strategies. The paper's traceable evaluation framework makes this decomposition visible, but the training methodology (TreeVGR) primarily addresses the localization stage. The future direction this finding suggests — developing training methods that specifically target the reasoning stage, perhaps by providing supervision on reasoning chains rather than just on bounding boxes — is not pursued in this paper but is a natural next step.

5. Experimental Analysis

Evaluation Methodology

Dataset. The primary evaluation benchmark is TreeBench, the paper's newly proposed benchmark consisting of 405 multiple-choice VQA pairs sampled from SA-1B images, annotated by 8 LMM experts with ground-truth bounding boxes for all target instances. The benchmark is split into 10 sub-tasks across two protocols: Perception (Attributes, Material, Physical State, Object Retrieval, OCR-Integrated QA) and Reasoning (Perspective Transform, Ordering, Contact and Occlusion, Spatial Containment, Comparison). Additional evaluation benchmarks include V* Bench (Wu and Xie, 2024), HR-Bench (Wang et al., 2025f), MME-RealWorld-Lite (Zhang et al., 2024a), and several standard multimodal benchmarks (AI2D, MathVista, MMStar, CV-Bench, MMVP, RealWorldQA, MMBench, POPE, HallusionBench, ChartQA).
Base model(s). TreeVGR is initialized from Qwen2.5-VL-7B-Instruct (Bai et al., 2025a), a 7-billion parameter multimodal model that supports arbitrary-resolution inputs via multimodal Rotary Position Embedding (mROPE) and outputs absolute bounding box coordinates natively. The choice of 7B scale is deliberate — the authors explicitly note it leaves room for scaling to larger architectures as future work. For comparison, the paper also evaluates Qwen2.5-VL-32B and Qwen2.5-VL-72B, InternVL3-8B and InternVL3-78B (Zhu et al., 2025), LLaVA-OneVision-7B (Li et al., 2024), and proprietary models including OpenAI-o3-0416, GPT-4o-1120, Gemini-2.5-Pro-0605, and Gemini-2.5-Flash-0520. For visual grounded reasoning baselines, the paper evaluates DeepEyes-7B (Zheng et al., 2025b) and Pixel-Reasoner-7B (Su et al., 2025).
Metrics. The paper uses two complementary metrics evaluated on TreeBench:
- Accuracy (overall score): The percentage of questions where the model's predicted answer matches the ground-truth answer, using exact matching for multiple-choice questions. This is the standard metric reported as "Overall" in Table 2.
- mIoU (mean Intersection-over-Union): For models that generate bounding boxes as part of their reasoning (visual grounded reasoning models), the paper computes the mean IoU between predicted and ground-truth bounding boxes across all target instances in all questions. This metric quantifies localization precision independently of answer correctness and is reported in the "mIoU" column of Table 2. For models that do not produce bounding boxes (general LMMs), mIoU is not computed.
- For other benchmarks (V* Bench, HR-Bench, MME-RealWorld-Lite, etc.), standard accuracy metrics are used as defined by each benchmark.
Baselines. The paper organizes baselines into four tiers:
- Proprietary models: OpenAI-o3-0416, GPT-4o-1120, Gemini-2.5-Pro-0605, Gemini-2.5-Flash-0520 — representing the state-of-the-art in both general multimodal understanding and (for o3) visual grounded reasoning.
- Open-source general LMMs: Qwen2.5-VL series (7B, 32B, 72B), InternVL3 series (8B, 78B), LLaVA-OneVision-7B — representing the best available open multimodal models that do not employ explicit grounding-then-answering pipelines.
- Visual grounded reasoning models: DeepEyes-7B and Pixel-Reasoner-7B — both follow a "grounding-then-answering" paradigm and are trained with RL, making them the most direct methodological comparisons to TreeVGR.
- Ablation baselines (TreeVGR variants): The base Qwen2.5-VL-7B-Instruct, the cold-start model TreeVGR-7B-CI, and RL variants with different reward configurations (accuracy only, recall-only IoU, precision-only IoU, no cold start) — used in ablation studies (Table 6).
Generation budget / compute accounting. The paper reports training budgets rather than inference budgets. For TreeVGR: cold-start SFT uses 8 GPUs (model unspecified but presumably H100-class given the RL stage) on 35K samples; RL uses 16 GPUs total (8 for policy training, 8 for reward model serving via vLLM) with 5 epochs on 37K samples. The paper emphasizes efficiency by contrasting with DeepEyes-7B, which used 32 H100 GPUs for 48 hours across 32 epochs on 47K samples. Inference cost is not quantified but the paper notes that TreeVGR does not require image cropping and re-encoding (as o3 and some prior methods do), operating purely in text space for bounding box generation, which the authors claim leads to "much more efficient training and inference procedures."
Cross-validation / statistical protocol. The paper does not report cross-validation, confidence intervals, or statistical significance testing for its benchmark evaluations. Evaluations on TreeBench are single-run scores for each model. For open-source general models, evaluations are conducted using the VLMEvalKit framework (Duan et al., 2024), while visual grounded reasoning models are evaluated directly by the authors. There is no mention of multiple evaluation runs, standard error, or test-set split validation — the 405 questions are evaluated as a single set. This is a notable omission for a benchmark paper, as it means the reported accuracy differences between models cannot be statistically assessed.

Main Quantitative Results

TreeBench Results: Frontier Models Struggle, and Localization Precision Partially Predicts Success

Overall performance (Table 2). The headline finding is that no model achieves 60% accuracy on TreeBench. The best-performing model is OpenAI-o3-0416 at 54.87%, followed by Gemini-2.5-Pro-0605 at 52.94%. Among open-source general models, InternVL3-78B leads at 45.93%, with Qwen2.5-VL-72B notably underperforming at 42.22% — lower than its 32B counterpart (43.46%), which the paper flags as anomalous. TreeVGR-7B achieves 45.19%, placing it just 0.74 points below the open-source SOTA InternVL3-78B despite being approximately 11× smaller (7B vs. 78B parameters).

The per-category breakdown in Table 2 reveals substantial variation in model rankings. On Perception tasks, proprietary models dominate: o3-0416 achieves 66.46% on Attributes and 71.43% on Physical State, while open-source models cluster 20-30 points lower on the same categories. On Reasoning tasks, the gaps are often narrower — Gemini-2.5-Pro-0605 leads on Perspective Transform at 63.33%, while several open-source models achieve scores within 10-15 points. The hardest category overall appears to be Ordering, where o3-0416 scores only 36.67% and most models score below 30%. Spatial Containment shows a similar pattern: o3-0416 at 38.71% leads a field where most models score 20-30%. These low scores on specific Reasoning sub-tasks — even for frontier models — validate the paper's claim that second-order visual reasoning remains a significant challenge.

Localization-accuracy correlation (Figures 8, 9, 10, and Table 2). The paper's traceable evaluation reveals a positive but task-dependent correlation between mIoU and accuracy. In Table 2, among visual grounded reasoning models, TreeVGR-7B achieves the highest mIoU (0.585) and the highest overall score (45.19), while DeepEyes-7B achieves the lowest mIoU (0.458) and the lowest overall score (31.11), with Pixel-Reasoner-7B in between (mIoU 0.565, score 39.26). This monotonic relationship is the paper's primary evidence that better localization contributes to better overall performance.

However, the histograms in Figures 8–10 reveal critical nuance. Figure 8 aggregates all questions and shows that correct predictions (orange bars) have higher IoU values on average than incorrect predictions (blue bars), with a clear concentration of wrong answers at low IoU values (near 0). Figure 9 (Perception questions) amplifies this pattern — the separation between correct and incorrect IoU distributions is stark, with correct answers clustering at high IoU and wrong answers at low IoU. Figure 10 (Reasoning questions) tells a different story: the two distributions overlap substantially, with a non-trivial number of correct answers at moderate-to-low IoU and a non-trivial number of incorrect answers at moderate-to-high IoU. This is the paper's empirical basis for claiming that "reasoning questions of TreeBench require second-order cognitive capabilities that go beyond precise spatial localization" (Section 5.1).

Difficulty distribution across targets (Figures 5 and 6). The paper quantifies the difficulty of TreeBench along two structural axes. Figure 5 shows that 86.4% of questions require identifying 1-2 target instances, with a long tail extending to 8 targets in one extreme case. Figure 6 shows the distribution of mean target instance area per question, with a sharp peak near 0.0 and a mean area of 0.0305 (i.e., target instances occupy on average only 3.05% of the image). This confirms the paper's design claim that TreeBench emphasizes small-object perception in cluttered scenes.

Results on High-Resolution Benchmarks: TreeVGR Matches or Exceeds 10× Larger Models

V* Bench (Table 3). TreeVGR-7B achieves 94.3% accuracy on V* Bench, representing a +16.8 point improvement over its base model Qwen2.5-VL-7B-Instruct (77.5%). This result is comparable to OpenAI-o3-0416 (96.6%) and surpasses GPT-4o-1120 (83.9%). Among open-source models, TreeVGR-7B achieves the highest score, ahead of DeepEyes-7B (82.3%) and Pixel-Reasoner-7B (92.0%). The authors note that V* Bench contains training data in TreeVGR's RL training set (the 30K filtered V* samples), so some improvement may come from training set overlap — they flag this with a † in Table 6 and acknowledge that "this improvement mainly comes from the training set, as many training samples from V* are included in RL."

HR-Bench (Table 3). TreeVGR-7B achieves 53.01% on HR-Bench-4K and 53.49% on HR-Bench-8K, compared to 48.01% and 48.79% for Qwen2.5-VL-7B-Instruct — improvements of +5.0 and +4.7 points respectively. Pixel-Reasoner-7B scores slightly higher (53.08% and 54.35% on the 4K and 8K splits), and DeepEyes-7B leads at 54.09% and 54.71%. TreeVGR's performance here is strong but not dominant — the paper highlights the significant gains over the base model rather than claiming SOTA on this specific benchmark.

MME-RealWorld-Lite (Table 4). TreeVGR-7B achieves 53.36%, a +12.64 point improvement over Qwen2.5-VL-7B-Instruct (40.72%). This is the highest reported score among all models evaluated, including proprietary models — GPT-4o-1120 scores 51.92%, Gemini-2.5-Flash-0520 scores 38.31%, and both are substantially lower. Compared to other visual grounded reasoning models, TreeVGR-7B leads DeepEyes-7B (49.14%) and Pixel-Reasoner-7B (49.88%). This result is particularly notable because MME-RealWorld-Lite contains diverse real-world scenes with small objects that were not in TreeVGR's training data (the RL data came from V* and VisDrone), suggesting genuine out-of-distribution generalization of the grounding capability.

Results on Other Multimodal Benchmarks: Improvements Are Largest on Vision-Centric Tasks

Table 5 results. TreeVGR-7B is compared to its base model Qwen2.5-VL-7B-Instruct across eight standard benchmarks. The improvements are uneven but consistently positive:

Vision-centric benchmarks show the largest gains. On MMVP (Tong et al., 2024b), which specifically tests visual perception by pairing images where subtle differences change the answer, TreeVGR-7B achieves 75.3 — a dramatic improvement over Qwen2.5-VL-7B-Instruct (40.0, a +35.3 point gain) and even surpassing Qwen2.5-VL-72B (which the paper reports at 68.0). On CV-Bench (Tong et al., 2024a), TreeVGR-7B scores 67.5 vs. 59.1 for the base model (+8.4). On RealWorldQA (xAI, 2024), TreeVGR-7B scores 64.8 vs. 61.1 (+3.7).
General VQA benchmarks show moderate gains. MMBench improves from 82.2 to 84.2 (+2.0), POPE from 87.0 to 88.5 (+1.5), HallusionBench from 55.0 to 57.4 (+2.4). These are all statistically modest but consistently positive.
Document/chart comprehension shows mixed results. AI2D improves from 83.5 to 84.5 (+1.0), while ChartQA is flat at 83.0 vs. 83.1.

The pattern across these benchmarks supports the paper's claim that TreeVGR's grounding-then-answering pipeline is most beneficial for tasks where precise visual localization is the primary bottleneck (MMVP, CV-Bench) and less beneficial for tasks where general language understanding or world knowledge dominates (MMBench, POPE, ChartQA). The MMVP result is particularly striking — a 35.3-point gain over the base model on a benchmark explicitly designed to test visual perception — and represents the strongest single evidence that TreeVGR's localization training transfers effectively to out-of-distribution visual tasks.

Performance Decoupling Analysis: TreeBench Measures Something Distinct

Correlation with other benchmarks (Figures 11, 12, 13). The paper plots model rankings on TreeBench against rankings on AI2D (Figure 11), MathVista (Figure 12), and MMStar (Figure 13). The key finding is that GPT-4o-1120, which "ranks among the top performers on TreeBench," shows substantially lower performance on the other benchmarks — the data point is visually displaced from the main correlation trend. The paper interprets this as evidence that "this dissociation underscores the unique emphasis on 'thinking with images' of our TreeBench" (Section 5.1).

This is an important validation claim: that TreeBench is not simply re-measuring general multimodal capability, but is specifically testing a distinct construct (visual grounded reasoning). However, the paper does not compute correlation coefficients (Pearson or Spearman) or provide statistical tests for this dissociation. The figures show scatter plots of a small number of models (approximately 8-12 points per plot), and the visual evidence of dissociation relies heavily on GPT-4o being an outlier — a single data point. If GPT-4o were removed, the remaining models would likely show a much stronger rank correlation. This is a weak form of validation, and the paper would be strengthened by either (a) computing formal correlation metrics or (b) adding more models to the scatter plots to establish that the dissociation is systematic rather than point-specific.

Ablation Studies and Robustness Checks

Cold-start initialization: Removing the cold-start SFT phase and applying RL directly from Qwen2.5-VL-7B-Instruct (Table 6, \circled{1} vs. \circled{2}) substantially degrades performance. The cold-start model (\circled{1}) achieves 33.09 on TreeBench, while direct RL without cold start (\circled{2}) reaches only 22.47 — a gap of 10.62 points. On MME-RealWorld-Lite, the gap is similarly large (47.63 vs. 38.51). This validates the paper's claim that cold-start initialization is critical for efficient RL training in this domain, though it does not isolate whether the benefit comes from reduced exploration difficulty, better format compliance, or both.

Traceable RL vs. untraceable RL: Starting from the same cold-start checkpoint, adding the dual IoU reward to the RL phase (Table 6, \circled{3} vs. \circled{4}) improves TreeBench from 39.51 to 45.19 (+5.68 points) and MME-RealWorld-Lite from 42.78 to 53.36 (+10.58). Critically, mIoU on TreeBench improves from 0.465 to 0.585 (+0.120), confirming that the dual IoU reward drives better localization, not just better answer accuracy. The V* Bench score remains essentially unchanged (86.4 vs. 86.3), which is expected given that V* data is in the RL training set for both conditions. This is the paper's strongest evidence that traceable evidence supervision during RL is beneficial beyond what accuracy-only RL provides, and that the benefit transfers to out-of-distribution benchmarks (MME-RealWorld-Lite, TreeBench).

Precision term ablation: Removing the precision component from the dual IoU reward (Table 6, \circled{5}) reveals a specific failure mode: the model achieves competitive TreeBench accuracy (47.41, even higher than the full model's 45.19) but with degraded mIoU (0.466 vs. 0.585) and an exploding mean response length (Figure 14). The paper reports that this model "tends to enumerate candidate bounding boxes to obtain larger IoU recall and thus always fails to produce final answers." The higher accuracy is therefore misleading — the model may be generating many boxes, finding one that matches the ground-truth by chance, and then producing the correct answer, but the reasoning pathway is unfaithful and the model's output is practically unusable due to extreme length. Figure 14 quantifies this: without precision, the mean response length grows continuously throughout training, reaching approximately 3-4× the length of the full dual IoU model.

Recall term ablation: Removing the recall component (Table 6, \circled{6}) produces the opposite pattern: TreeBench accuracy improves substantially (44.69 vs. the cold-start baseline of 39.51) but mIoU remains low (0.448), indicating that the model grounds incompletely — it finds some target instances precisely but misses others. The paper interprets this as the model "usually grounding incomplete target instances." Without recall pressure, the model has no incentive to ensure every ground-truth object is covered by at least one prediction, so it converges to a strategy of predicting a small number of high-confidence boxes while ignoring harder-to-find objects.

Text-only RL vs. visual grounded reasoning RL: Running vanilla RL with only accuracy and format rewards, without any grounding-specific training (Table 6, \circled{7}), produces an accuracy of 33.09 on TreeBench — identical to the cold-start model and substantially lower than traceable RL (45.19). This is somewhat of an unfair comparison because \circled{7} uses a different RL training data configuration, but the paper's point is that text-space RL alone, even when effective for text-based reasoning tasks, does not translate to visual grounded reasoning without explicit grounding supervision. The cold-start model's grounding capability (learned during SFT) provides no further improvement when RL only rewards the final answer.

Format and coordinate design choices (cold-start data construction):

The paper does not ablate the coordinate conversion step (normalized to absolute) — we cannot assess from the reported results whether this conversion was necessary or whether Qwen2.5-VL could have learned to interpret normalized coordinates with sufficient training.
The paper does not ablate the reflective subset (the 4.7K samples with synthetic errors) — the contribution of error-correction training to final performance cannot be isolated.
The paper does not ablate the multi-box filtering criterion for SFT data — we cannot assess whether single-box samples would have been harmful or simply less beneficial.

RL training data ablations:

The paper does not ablate the VisDrone component of the RL training data (7K samples) versus using only V* data (30K samples). The contribution of the drone imagery counting task to the final model's performance is unknown.
The paper does not ablate the difficulty filtering for V* (using Qwen2.5-VL-7B-Instruct to select hard samples). Training on the full 191K V* training set might produce different results — potentially better due to more data, or worse due to dilution with easy samples.

Critical Assessment

Claim: TreeBench "reveals the limitations of state-of-the-art models" with no model exceeding 60% accuracy

Assessment: Strongly supported, but saturation may be closer than the headline implies. The empirical evidence is clear: o3-0416 at 54.87% and Gemini-2.5-Pro at 52.94% are the top scores, with substantial headroom to 100%. The per-category breakdown shows even lower scores on specific sub-tasks (Ordering: o3 at 36.67%, Spatial Containment: o3 at 38.71%).

However, two caveats temper this headline. First, the 60% figure is derived from a single evaluation run on 405 questions, and no confidence intervals are reported. With 405 questions, the standard error for a model scoring ~55% is approximately ±2.5 percentage points — so o3's "true" accuracy could be anywhere from roughly 50% to 60%. An o3 score of 57.5% would be within one standard error of 60%, making the "no model reaches 60%" claim less definitive than it appears. Second, the field is moving extremely fast — o3 was released in April 2025, and it is plausible that the next generation of proprietary models (or improved prompting strategies) will push scores above 60% within months. The benchmark's difficulty is real but the precise threshold is fragile.

Claim: "Traceable evaluation demonstrates a positive correlation between localization precision and overall performance"

Assessment: Supported but with a critical qualification that the paper itself documents. Table 2 shows the monotonic relationship mIoU: DeepEyes (0.458) < Pixel-Reasoner (0.565) < TreeVGR (0.585) and overall: DeepEyes (31.11) < Pixel-Reasoner (39.26) < TreeVGR (45.19). This is consistent with a positive correlation.

However, the paper's own analysis in Figures 9-10 demonstrates that this correlation is task-dependent and weak on Reasoning questions. On Perspective Transform and Spatial Containment, there are substantial numbers of correct answers with low IoU and incorrect answers with high IoU. The positive correlation at the aggregate level is driven largely by Perception questions, where the relationship is tight. The paper is transparent about this — Section 5.1 explicitly states that "reasoning performance reveals a weaker correlation with mIoU" — but the qualification is important: the claim that localization precision drives overall performance is true in aggregate but misleading at the sub-task level.

A stronger version of the paper's claim would require demonstrating that the relationship is causal, not merely correlational — that improvements in mIoU cause improvements in accuracy. The ablation showing that traceable RL improves both mIoU and accuracy relative to untraceable RL (Table 6, \circled{3} vs. \circled{4}) is suggestive of causation but not definitive, because traceable RL differs from untraceable RL along multiple dimensions (different reward shape, different training dynamics), and the accuracy improvement could come from those other factors rather than from mIoU improvement per se. An instrumental variables analysis or a mediation analysis would strengthen the causal claim.

Claim: TreeVGR "achieves comparable performance with InternVL3-78B" on TreeBench

Assessment: Supported numerically but the comparison is asymmetric. TreeVGR-7B scores 45.19 vs. InternVL3-78B at 45.93 — a difference of 0.74 points on 405 questions. Given the likely standard error (~2.5 points), these scores are statistically indistinguishable.

However, the comparison is between a 7B model with specialized visual grounding training and a 78B model without such training. InternVL3-78B was not trained to produce bounding boxes or follow a grounding-then-answering format — it was trained as a general-purpose multimodal model. The fact that TreeVGR-7B "matches" InternVL3-78B tells us something about the value of the grounding-then-answering paradigm at the 7B scale, but it does not tell us how a 7B grounding-trained model compares to a 78B grounding-trained model — which would be the more relevant comparison for assessing the scaling properties of this approach. The paper does not train TreeVGR at larger scales, so we cannot assess whether the grounding-then-answering advantage diminishes, persists, or amplifies with model size.

Claim: TreeVGR achieves "+16.8 on V* Bench, +12.6 on MME-RealWorld-Lite, and +13.4 on TreeBench" over the base model

Assessment: Supported, but the V* Bench improvement is confounded by training data overlap. The V* Bench +16.8 improvement (77.5 → 94.3) is the largest gain, but the paper acknowledges in Table 6 footnote that "this improvement mainly comes from the training set, as many training samples from V* are included in RL." The V* training data (30K samples) and V* Bench test data are drawn from the same distribution — images from COCO with similar question formats. This means the +16.8 figure reflects a mix of genuine capability improvement and training set memorization/generalization, and cannot be cleanly interpreted as out-of-distribution transfer.

The MME-RealWorld-Lite (+12.64) and TreeBench (+13.4) improvements are stronger evidence because these benchmarks contain images and question types that were not in TreeVGR's training data (SA-1B images for TreeBench, diverse real-world images for MME-RealWorld-Lite). These figures are more representative of the genuine capability gain from TreeVGR training.

Claim: TreeVGR is "significantly less" training than DeepEyes-7B (5 epochs vs. 32 epochs)

Assessment: The comparison is potentially misleading due to unaccounted SFT cost. TreeVGR's RL phase uses 5 epochs on 37K samples, while DeepEyes-7B used 32 epochs on 47K samples. However, TreeVGR also requires the cold-start SFT phase (35K samples, unknown number of epochs), which DeepEyes did not use. The paper does not report the SFT training cost (GPU hours, wall-clock time, number of SFT epochs), making the total cost comparison incomplete.

Furthermore, the RL phases are not directly comparable: TreeVGR uses a different base model (Qwen2.5-VL vs. DeepEyes' LLaVA-NeXT), different RL data (V* + VisDrone vs. DeepEyes' data), and different reward structures (dual IoU + accuracy + format vs. whatever DeepEyes used). The claim of greater efficiency is plausible — cold-start initialization should reduce the RL burden — but the paper does not provide the controlled experiment that would prove it: training TreeVGR with end-to-end RL (no cold start) on identical hardware and measuring convergence time. The ablation in Table 6 (\circled{2}) shows that end-to-end RL performs poorly, but does not report how many epochs or GPU hours were needed to converge.

What Experiments Would Have Strengthened the Paper

Scaling experiments. Training TreeVGR at 32B and 72B scales would reveal whether the grounding-then-answering advantage compounds with model size or diminishes (as base models get better at implicit grounding). This is the most important missing experiment for understanding the practical value of the approach: if a 72B TreeVGR achieves 60%+ on TreeBench, that is a much stronger result than any 7B finding.

Combined PRM search + revisions. The paper's related work discusses test-time visual computation (cropping, zooming) but TreeVGR does not actually crop and re-encode images. A comparison between TreeVGR's text-space grounding and a variant that actually crops and re-encodes regions at test time would quantify the accuracy cost of the efficiency gain.

Statistical reporting. Reporting standard errors or confidence intervals for benchmark scores, and computing correlation coefficients (Pearson/Spearman) for the cross-benchmark comparisons in Figures 11-13, would substantially strengthen the empirical claims and allow readers to assess whether observed differences are likely to be real or noise.

Ablation of RL training data components. Specifically, ablating the VisDrone component (7K samples) to isolate its contribution, and ablating the difficulty filtering on V* (using full 191K vs. filtered 30K) to understand whether hard-sample selection matters.

Human performance baseline. TreeBench is designed by 8 LMM experts to be challenging, but no human performance ceiling is reported. Without knowing whether humans find these questions easy (95%+) or difficult (70%), we cannot calibrate the difficulty scale. If human accuracy on TreeBench is 80%, then o3's 54.87% represents a substantial gap; if human accuracy is 98%, the gap is much larger. This context matters for interpreting the benchmark's role.

Error analysis on the decoupling phenomenon. The paper's most interesting finding — that mIoU correlates weakly with accuracy on Reasoning tasks — is presented only as aggregate histograms (Figure 10). A qualitative analysis of specific cases where the model had high mIoU but wrong answers (or low mIoU but correct answers) would illuminate why localization and reasoning decouple, providing actionable insights for future work. What kinds of reasoning errors survive perfect localization? What kinds of correct inferences tolerate imprecise grounding? The paper leaves these questions unanswered.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Not Accounted For, Making the Reported Efficiency Gains Aspirational

The assumption or constraint. TreeVGR's compute efficiency narrative implicitly depends on the assumption that the model can be deployed without additional overhead for assessing when and how to ground. While TreeBench does not require a difficulty estimator in the same way the compute-optimal test-time scaling paper did, there is an analogous unaccounted cost here: the paper reports that TreeVGR is "significantly less" training than its closest comparable method, DeepEyes-7B (5 RL epochs vs. 32), and that it achieves "much more efficient training and inference procedures" (Section 4, opening paragraph). The paper explicitly acknowledges in Section 6:

"The current implementation of TreeVGR is based on a 7B parameter model, which may limit scalability compared to larger architectures."

However, this is a limitation of scale, not of accounting. The deeper issue is that the paper does not report the full training cost of the cold-start SFT phase — the number of epochs, GPU hours, or wall-clock time for the 35K-sample SFT stage — making the total efficiency comparison with DeepEyes incomplete.

The consequence. A practitioner deciding whether to adopt the TreeVGR pipeline cannot accurately estimate the total compute cost. The headline efficiency claim — "5 epochs on 37K samples" for RL — omits the SFT phase entirely from the accounting. If the SFT phase takes, say, an additional 10 epochs on 35K samples (reasonable for a language-vision fine-tuning task), the total training cost could be comparable to DeepEyes' 32 RL epochs on 47K samples, not "significantly less." Furthermore, the inference efficiency claim — that text-space grounding avoids re-encoding cropped images — is not quantified: no latency benchmarks, no throughput comparisons, and no FLOPs accounting are reported. Without these, the claim that TreeVGR is "much more efficient" remains qualitative and unverifiable.

What evidence exists in the paper. The paper states that DeepEyes-7B "requested RL training on 47K samples across 32 episodes, a process requiring 32 H100 (80GB) GPUs operating continuously for 48 hours" (Section 4.1). For TreeVGR, the RL phase uses "8 GPUs, with another 8 GPUs serving the reward model" for 5 epochs on 37K samples (Section 4.2), but no GPU-hours or wall-clock time is reported, and the SFT phase is described only as "trained with 8 GPUs" (Section 4.1) without epoch count, time, or exact hardware model. The paper also notes that TreeVGR "does not require actually replaying cropped images" (Section 4), claiming this "leads to much more efficient training and inference procedures," but provides no inference-time measurements.

Mitigation status. Not addressed. The paper does not provide a total training cost for TreeVGR, nor does it benchmark inference latency or throughput against approaches that re-encode cropped regions. These measurements are left as implicit future work.

Second-Order Reasoning Remains Largely Unsolved, and TreeVGR Does Not Target It

The assumption or constraint. TreeVGR's training methodology — cold-start SFT followed by RL with a dual IoU reward — is designed to improve localization precision. The reward signal explicitly supervises bounding box quality (recall and precision) and final answer correctness, but provides no direct supervision on the reasoning chain that connects localized objects to inferences. The paper discovers empirically that this leaves a capability gap. In Section 5.1, analyzing Figure 10:

"reasoning performance reveals a weaker correlation with mIoU, as improvements in localization alone fail to fully translate to complex reasoning tasks. This disconnect suggests that reasoning questions of TreeBench require second-order cognitive capabilities that go beyond precise spatial localization."

The consequence. A practitioner deploying TreeVGR for tasks requiring perspective transforms, spatial containment inference, or occlusion reasoning should expect the model's performance to be bounded not by its ability to locate objects — which the dual IoU reward optimizes — but by its ability to reason about located objects, which the training pipeline does not explicitly address. On Perception tasks, improved localization translates directly to improved accuracy (Figure 9 shows tight mIoU-accuracy correlation). On Reasoning tasks, the relationship is loose (Figure 10) — there are substantial numbers of high-IoU but incorrect predictions, and correct predictions with moderate-to-low IoU. This means that even with perfect bounding boxes (mIoU = 1.0), the model would still fail on a significant fraction of Reasoning questions because the inference step, not the grounding step, is the bottleneck.

What evidence exists in the paper. Table 2 shows TreeVGR-7B achieving 45.19 overall accuracy with mIoU of 0.585. On individual Reasoning sub-tasks, performance remains low: Perspective Transform (43.33%), Ordering (23.33%), Spatial Containment (41.94%). These scores are 15-25 points below what would be expected if localization were the sole bottleneck — and the mIoU on these sub-tasks is not reported separately, so we cannot directly measure how much of the remaining error is attributable to residual localization error versus reasoning failure. Figure 10 provides aggregate evidence that the correlation is weak across all Reasoning questions, but a per-subtask breakdown of the (mIoU, accuracy) relationship is missing.

Mitigation status. The paper explicitly acknowledges this limitation in Section 5.1 and in the conclusion (Section 6): "Scaling up would be future work." However, scaling the model size (the suggested future work) may improve both localization and reasoning simultaneously, providing a path toward addressing the reasoning gap. The paper does not propose or evaluate methods that directly supervise reasoning chains (as opposed to bounding boxes), leaving this as an open problem.

The V* Bench Improvement Is Confounded by Training-Test Overlap, Undermining the Strongest Reported Gain

The assumption or constraint. The RL training data for TreeVGR includes 30K samples filtered from the V* training set (Section 4.2). V* Bench, on which the paper reports a +16.8 point improvement (77.5 → 94.3), is constructed from the same distribution — images from COCO with spatially-oriented questions about small objects. The paper acknowledges this contamination candidly in a footnote to Table 6:

"This improvement mainly comes from the training set, as many training samples from V* are included in RL."

The consequence. The +16.8 V* Bench improvement — the paper's largest reported gain — cannot be interpreted as evidence of genuine out-of-distribution generalization of the grounding capability. A practitioner evaluating TreeVGR for deployment on a novel task domain should heavily discount the V* Bench result and instead focus on the MME-RealWorld-Lite (+12.6) and TreeBench (+13.4) improvements, which are on benchmarks that do not share training data with the RL phase. The V* result conflates memorization of training distribution characteristics with transferable grounding skill.

The problem is particularly acute because V* Bench is the benchmark most similar to TreeVGR's RL training data in terms of image source (COCO), question type (attribute and spatial queries about small objects), and output format (bounding boxes + answer). This makes V* an in-distribution evaluation, while MME-RealWorld-Lite and TreeBench are out-of-distribution. The +16.8 gain likely represents an upper bound on what the method can achieve, with the +12-13 point gains on OOD benchmarks being more representative of genuine capability improvement.

What evidence exists in the paper. Table 3 reports the +16.8 V* improvement alongside the HR-Bench improvements (+5.0 on 4K, +4.7 on 8K). Table 6 footnote explicitly flags the contamination. However, the paper does not report an ablation where TreeVGR is trained on RL data that excludes V* — we cannot isolate how much of the V* improvement comes from distributional overlap versus genuine grounding improvement.

Mitigation status. Partially addressed. The paper is transparent about the contamination (Table 6 footnote), which is better than many papers that silently include test data in training. The paper also provides OOD benchmarks (MME-RealWorld-Lite, TreeBench) that demonstrate meaningful improvements without contamination. However, the +16.8 figure appears prominently in the abstract and introduction without the contamination caveat, which could mislead a casual reader. A V*-excluded RL training ablation would substantially strengthen the paper's claims of genuine grounding improvement.

The 405-Question Benchmark Is Too Small for Fine-Grained Sub-Task Comparisons or Statistical Inference

The assumption or constraint. TreeBench contains 405 questions distributed across 10 sub-tasks and two protocols. The authors explicitly acknowledge this in Section 6:

"TreeBench contains only 405 rigorously curated question-answer pairs. Expanding the benchmark with additional samples across broader domains would further challenge model capabilities."

The consequence. The benchmark's small size creates two practical problems for anyone using TreeBench to evaluate models or compare methods. First, per-subtask sample sizes are tiny. The Reasoning protocol contains 256 questions across 5 sub-tasks, averaging ~51 questions per sub-task. The Perception protocol contains 149 questions across 5 sub-tasks, averaging ~30 questions per sub-task. For a specific sub-task like Perspective Transform, a model scoring 43.33% (TreeVGR-7B in Table 2) achieved this on approximately 30-50 questions — the exact number is not reported per sub-task. With such small sample sizes, the standard error on sub-task accuracy is approximately ±7-9 percentage points, meaning that a reported difference of 10 points between two models on a single sub-task could easily be noise. Second, the overall benchmark score has substantial uncertainty. On 405 questions, a model scoring ~45% has a standard error of approximately ±2.5 percentage points. The paper's key comparison — TreeVGR-7B at 45.19 vs. InternVL3-78B at 45.93 — is a difference of 0.74 points, which is well within one standard error. Without confidence intervals, the claim that TreeVGR "achieves comparable performance" cannot be statistically assessed.

What evidence exists in the paper. Table 2 reports per-category accuracies without standard errors or confidence intervals. The paper does not report the number of questions per sub-task (though it can be estimated from Figure 5's distribution and the 405 total). No statistical tests (t-tests, bootstrap, McNemar's test) are reported for any pairwise model comparison. In Section 5.1, the decoupling analysis (Figures 11-13) plots model scores as scatter points without error bars, making visual assessment of correlation strength unreliable.

Mitigation status. The paper acknowledges the small size in Section 6 and frames expansion as future work. No immediate mitigation (e.g., bootstrapped confidence intervals, per-subtask question counts, or statistical testing) is provided. The paper's defense is implicit in its design philosophy — prioritizing annotation quality and difficulty over scale — but this defense does not address the statistical limitations that small sample sizes create for the benchmark's stated purpose of comparing model capabilities.

Hard Problems (Dense Clutter, Many Targets, Extreme Small Objects) Are Not Separately Analyzed, Leaving the Difficulty Ceiling Uncharacterized

The assumption or constraint. TreeBench is designed around challenging images with small target objects (mean relative area 3.05%) in cluttered SA-1B scenes. The paper provides aggregate difficulty statistics — Figures 5 and 6 show the distribution of target counts and areas — but does not stratify results by difficulty level. There is no difficulty binning (analogous to the compute-optimal test-time scaling paper's quintiles), no breakdown of how model performance varies with target count (1 target vs. 2 vs. 4+), and no analysis of how accuracy scales with target instance area (the hardest tail of Figure 6).

The consequence. A practitioner deploying TreeVGR on a specific application domain — say, counting small objects in dense drone imagery (similar to VisDrone) or locating specific components in cluttered manufacturing scenes — cannot determine from the paper's results how well the model will perform on images of comparable difficulty to their use case. TreeBench's aggregate accuracy of 45.19 for TreeVGR-7B averages together questions with 1 target (41.5% of the benchmark), 2 targets (44.9%), and 3-8 targets (13.6%), as well as questions where targets occupy anywhere from near-0% to 70% of the image. If a practitioner's use case involves primarily 4+ target questions with extremely small objects, TreeVGR's effective accuracy on their distribution could be substantially lower than 45.19, and the paper provides no way to estimate this.

Furthermore, the paper does not report how many of the 405 questions fall into the "extremely hard" tail of the target area distribution (Figure 6). If, say, 50 questions have mean target area below 1%, and models score near-random on these, this subset could be driving a significant fraction of the overall error. Without this stratification, we cannot determine whether future progress on TreeBench will come from improving small-object perception (which the dual IoU reward targets) or from some other capability entirely.

What evidence exists in the paper. Figure 5 shows the distribution of target instances per question (1-8). Figure 6 shows the histogram of mean target area. Table 2 provides per-subtask accuracy but not per-difficulty accuracy. The paper does not bin questions by structural difficulty (target count, target area, clutter level) and report stratified results. The only difficulty-related filtering is in the data construction pipeline (Section 3.2): questions answerable by all four SOTA models were excluded, ensuring a minimum difficulty floor, but no upper-difficulty stratification is provided.

Mitigation status. Not addressed. The paper does not discuss difficulty stratification as either a current feature or a future direction. The benchmark's construction includes a "Difficulty Filtering" step that removes easy questions, but this produces a floor, not bins. Providing binned results by target count and area — even at coarse granularity given the small sample size — would substantially improve the benchmark's diagnostic value and is a natural extension for future work.

The Revision Model Is Not Evaluated for Trustworthiness or Safety — High mIoU Does Not Guarantee Faithful Reasoning

The assumption or constraint. TreeVGR's dual IoU reward supervises bounding box accuracy (spatial overlap with ground-truth) but not bounding box faithfulness — whether the model actually uses the predicted boxes in its subsequent reasoning, or merely generates boxes that happen to be correct while reasoning via an unrelated pathway. The paper's traceable evaluation computes mIoU between predicted and ground-truth boxes but does not verify that the model's textual reasoning chain references or depends on those boxes.

The paper acknowledges a version of this concern implicitly when motivating the dual IoU reward (Section 2):

"Critically, these RL-based methods supervise only the final answer. In contrast, our TreeVGR emphasizes traceable evidence during RL training."

But the traceable evidence supervision is on spatial accuracy, not on reasoning fidelity. A model could learn to generate accurate boxes (rewarded by the dual IoU term) and correct answers (rewarded by the accuracy term) through independent mechanisms — the boxes come from a learned localization pathway, the answers come from a language-prior pathway, and the two never actually interact. The mIoU metric would look good, the accuracy would look good, but the reasoning would be unfaithful.

The consequence. For high-stakes deployment — medical imaging, autonomous navigation, industrial inspection — where understanding why a model made a decision is as important as the decision itself, TreeVGR's traceable evidence provides a weaker guarantee than it appears to. A practitioner might reasonably interpret high mIoU as evidence that the model "looked at the right things," but the paper provides no mechanism to verify that the model reasoned from those things. A post-hoc explanation (the boxes) may be correlated with correct answers without being causally responsible for them.

This is not a hypothetical concern. It is the multimodal analog of the well-documented phenomenon in text-only LLMs where chain-of-thought reasoning can be plausible but unfaithful — the model generates reasoning steps that support the answer but are not causally responsible for the answer the model actually produced. The bounding box equivalent would be: the model generates a box around the correct object (rewarded), then generates the correct answer (rewarded), but the answer was produced from a language shortcut (e.g., the question text contains cues that strongly predict the answer) and the box was produced from visual features independently.

What evidence exists in the paper. The paper does not measure reasoning faithfulness. The traceable evaluation (Figures 8-10) correlates mIoU with answer correctness but does not test whether the model's reasoning chain causally depends on the predicted boxes. There is no ablation where boxes are removed or randomized at inference time to measure the impact on answer accuracy. The paper does not analyze whether the model's textual reasoning (the text between thinking tags) references the predicted box coordinates or the objects they localize. The reflective subset of the SFT data (4.7K samples with synthetic errors and "Wait, this box seems to be wrong") is the paper's only gesture toward faithfulness — it teaches the model to detect obviously wrong boxes — but this does not verify that correct boxes are actually used in reasoning.

Mitigation status. Not addressed and not acknowledged as a limitation. The paper's framing treats traceability (measurable mIoU) as equivalent to faithful reasoning (causal dependence of the answer on the localized evidence), but these are distinct properties. Verifying faithfulness would require causal intervention experiments (e.g., perturbing boxes at inference time and measuring answer change) or attention-based analyses that are beyond the scope of this paper but are important for future work.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not introduce a new model architecture or a radical training paradigm — it introduces measurement infrastructure. TreeBench is, fundamentally, a diagnostic instrument. Its impact on the field will be less about any single result and more about what it makes newly visible: the distinction between seeing the right thing and thinking about what you saw. Before TreeBench, these two capabilities were conflated in evaluation. A model's answer was either right or wrong, and that was the end of the story. TreeBench splits that story into two chapters: localization quality (mIoU) and reasoning accuracy, measured on the same task, from the same model, on the same image. This is not an incremental refinement of existing VQA benchmarks — it is a qualitatively different kind of evaluation, and it changes what researchers can claim about their models.

The evidence for this shift comes from the paper's own diagnostic findings. Figure 9 shows that on Perception questions, wrong answers are overwhelmingly concentrated at low IoU values — when the model can't find the object, it can't answer the question. This is the expected story, and it validates the intuition behind a decade of work on better visual grounding. But Figure 10 tells a different story: on Reasoning questions, correct and incorrect answers overlap substantially in IoU space. There are wrong answers with near-perfect localization, and correct answers with mediocre localization. This means that the reasoning step has failure modes independent of the perception step, and that improving perception alone — through larger models, higher resolution, or better detection — will not close the reasoning gap. The paper makes this finding empirically visible for the first time, and it reframes the challenge of visual grounded reasoning from a single-stage optimization problem (ground better → reason better) to a two-stage diagnostic problem (which stage broke?).

This finding partially resolves a tension in the literature that the paper itself exemplifies. The visual grounded reasoning community — VGR, DeepEyes, Pixel-Reasoner, GRIT, Ground-R1 — has been racing to build models that "think with images," but evaluating them primarily on benchmarks that test perception, not reasoning (V* Bench, HR-Bench, MME-RealWorld). These benchmarks validate that the grounding-then-answering pipeline improves localization, which it does. But they cannot validate whether it improves reasoning over localized evidence, because they don't separately measure the two. TreeBench provides that measurement, and the answer is nuanced: yes, better localization helps (the aggregate mIoU-accuracy correlation in Table 2 is positive), but no, it is not sufficient (Figure 10). This doesn't invalidate the grounding-then-answering paradigm — TreeVGR's strong performance on Perception tasks and on vision-centric benchmarks like MMVP (+35.3 over the base model) demonstrates its value for tasks where perception is the bottleneck. But it redraws the boundary of that paradigm's applicability: for second-order reasoning tasks, grounding is necessary but not sufficient, and future methods will need to explicitly train the reasoning step as well.

The paper also shifts the incentive structure for benchmark construction. The expert-driven, quality-over-quantity approach — 405 questions produced through three rounds of human validation with LMMs serving only as proposal generators — is a direct response to the proliferation of LMM-generated benchmarks where annotation quality is suspect. The paper's "Difficulty Filtering" step, which removes any question answerable by all four SOTA models, is a specific design choice that other benchmark authors will likely adopt: it ensures the benchmark measures frontier capability, not basic competence, and it prevents rapid saturation. The fact that no model reaches 60% on TreeBench, while V* Bench already sees >90% scores from open-source models, is a concrete demonstration of this design philosophy's value.

Finally, the paper makes training efficiency for visual grounded reasoning newly measurable and improvable. By providing a concrete baseline (DeepEyes: 32 H100 GPUs for 48 hours) and a cheaper alternative (TreeVGR: 16 GPUs with a cold-start phase), the paper establishes that the computational cost of teaching models to ground their reasoning is not fixed — it depends on training methodology. The cold-start + traceable RL pipeline is a specific recipe that others can adopt, modify, or beat, but the key contribution is making efficiency a first-class metric in this subfield. Before TreeVGR, the community had one data point (DeepEyes) on what it costs to train a visual grounded reasoner. Now it has two, and the second is substantially cheaper. This creates a baseline that future work will be compared against, which tends to accelerate progress.

Follow-Up Research This Work Enables

Separating perception and reasoning training with a two-stage curriculum. The paper's central diagnostic finding — that mIoU correlates with accuracy on Perception but weakly on Reasoning (Figures 9-10) — implies that these two capabilities may benefit from different training objectives and data. A natural follow-up would train a model with two sequential RL phases: Phase 1 uses the dual IoU reward (as in TreeVGR) on perception-heavy data (V*, VisDrone) to optimize localization precision; Phase 2 uses a reasoning chain reward — perhaps an LLM judge evaluating the logical coherence of the reasoning steps themselves, or a process reward model trained on human-annotated reasoning traces — on reasoning-heavy data (the TreeBench Reasoning subset or synthetically generated perspective-transform and spatial-containment problems) to optimize inference quality without further modifying the localization module. The hypothesis is that Phase 2 would improve Reasoning scores without degrading Perception scores (which are already near-ceiling on some sub-tasks after Phase 1). The key measurement would be whether the mIoU-accuracy correlation on Reasoning questions tightens after Phase 2 — if it does, that would mean the training is successfully linking better localization to better reasoning, which TreeVGR currently does not do.

Verifying whether bounding boxes are causally used in reasoning. The paper's traceable evaluation measures whether predicted boxes overlap with ground-truth boxes, but not whether the model uses those boxes to produce its answer. A direct causal test would run the trained TreeVGR model at inference time with three conditions: (1) normal generation (boxes + reasoning + answer), (2) box removal (the model is forced to skip the bounding box generation step and reason directly, perhaps by masking the box-output tokens), and (3) box perturbation (the model's correctly predicted boxes are replaced with plausible but incorrect boxes — e.g., boxes around semantically similar but spatially wrong objects — and the model continues its reasoning from that point). If the model's accuracy drops substantially in condition 2 and condition 3 compared to condition 1, that would demonstrate causal dependence of the reasoning on the grounding. If accuracy is unchanged, the model is generating boxes that are correlated with correct answers but not causally responsible for them — a form of unfaithful reasoning that would significantly weaken the paper's claim that traceable evidence enables "explainable reasoning pathways." This experiment requires no new training, only inference-time intervention, and could be run on TreeBench using the existing TreeVGR-7B checkpoint.

Scaling TreeVGR to 32B and 72B to measure whether the grounding-then-answering advantage amplifies or diminishes. The paper only trains at 7B, leaving open whether the method's benefits scale with model size. Two competing hypotheses are plausible. Hypothesis A: larger base models have better implicit visual grounding (Qwen2.5-VL-72B already achieves 42.22 on TreeBench without any grounding-specific training), so the marginal benefit of explicit grounding training should diminish — a 72B TreeVGR might show only a small improvement over its base model. Hypothesis B: larger models have more capacity to benefit from structured training, and explicit grounding training teaches a skill (faithful localization-then-reasoning) that even large models haven't acquired from pretraining — a 72B TreeVGR might show gains proportional to or larger than the 7B gains (which were +13.4 on TreeBench, +12.6 on MME-RealWorld-Lite). Training TreeVGR at 32B and 72B using the same pipeline (cold-start SFT on converted VGR-158K, 5 epochs GRPO with dual IoU on filtered V* + VisDrone) and evaluating on TreeBench, MME-RealWorld-Lite, and MMVP would resolve this question. The result would inform whether the community should invest in scaling this specific training paradigm or whether it is primarily useful for making small models competitive with larger ones.

Building a reasoning-specific diagnostic benchmark with the same traceable structure. TreeBench reveals that reasoning is the bottleneck on its Reasoning sub-tasks, but the benchmark was not designed to diagnose which reasoning capability fails. A follow-up benchmark could decompose each second-order task into finer-grained cognitive operations. For Perspective Transform: does the model fail at identifying the reference agent's orientation, computing the relative vector, or mapping that vector to a directional label? For Spatial Containment: does the model fail at detecting the container boundary, determining whether the target's centroid is inside, or applying the correct containment rule (e.g., "on top of" counts as containment for some objects but not others)? Each decomposition would require annotating not just target bounding boxes but also intermediate reasoning steps (the agent's orientation vector, the container's boundary polygon, the relative position label). This is expensive — the paper's 405 questions required 8 experts — but the diagnostic value would be high. The paper's annotation pipeline (LMM-generated proposals + expert curation + cross-verification) could be adapted for this purpose, with the LMMs generating candidate reasoning decompositions and experts validating them.

Training with combined text-space and visual-space grounding to quantify the accuracy cost of text-only grounding. TreeVGR operates purely in text space — bounding boxes are generated as coordinate tokens, and the model never re-encodes cropped image regions. This makes inference efficient but may sacrifice accuracy on tasks where high-resolution visual detail matters (e.g., reading small text, identifying fine material properties). A direct comparison would train two variants of TreeVGR: one with the current text-space grounding, and one where, after generating bounding boxes, the model actually crops those regions from the image, re-encodes them through the vision encoder at higher effective resolution, and incorporates the re-encoded features into subsequent reasoning. This is architecturally similar to what o3 does internally, and to what VGR and DeepEyes do during their "re-playing" steps. Evaluating both variants on TreeBench, particularly on sub-tasks where fine visual detail matters (Attributes, Material, OCR-Integrated QA), would quantify the accuracy-efficiency tradeoff. If the visual-re-encoding variant improves Attributes accuracy by 15 points but increases inference latency by 5×, practitioners can make informed decisions based on their deployment constraints. If the improvement is negligible, the paper's claim that "text-space grounding is already effective" (Section 4) would be strongly validated.

Applying the dual IoU reward to other structured-intermediate-output domains. The dual IoU reward addresses a general problem: when training models to generate structured intermediate outputs (bounding boxes, keypoints, segmentation masks, tool calls, code snippets), a naive reward that only checks the final output creates incentives for spurious intermediate generation. The precision-recall formulation — pairing a coverage metric with a parsimony metric so that degenerate strategies are mutually penalizing — should transfer to any domain where the intermediate output has a set-based structure. A concrete follow-up would apply the dual reward to keypoint-based reasoning (e.g., "identify the joints of the person doing X, then determine their pose"): the recall term ensures all relevant keypoints are predicted, the precision term ensures predicted keypoints match actual anatomical landmarks, and the combined reward prevents the model from predicting hundreds of keypoints to game the recall metric. Evaluating on a keypoint-based VQA benchmark (e.g., a version of MMVP annotated with keypoint locations) would test the transferability of the approach, and failure would identify domain-specific assumptions in the dual IoU formulation (e.g., that objects are well-approximated by axis-aligned boxes, which may not hold for articulated keypoints).

Practical Applications and Downstream Use Cases

Fine-grained visual inspection in manufacturing and agriculture. TreeVGR's combination of small-object localization (mean target area 3.05% in TreeBench, VisDrone counting of 5-10 small objects) and reasoning about localized objects maps directly onto industrial inspection workflows. In electronics manufacturing, an inspector must locate specific components on a circuit board (often <1% of the image area), check their attributes (correct orientation, no visible damage), and reason about their relationships (is component A properly seated in its socket? is there a visible solder bridge between adjacent pins?). A TreeVGR-style model, fine-tuned on domain-specific inspection images with bounding box annotations, could perform this workflow with auditable intermediate outputs — the bounding boxes provide verifiable evidence of what the model looked at, which is essential for regulatory compliance in safety-critical manufacturing. The paper's MME-RealWorld-Lite result (+12.64 over the base model, Table 4) demonstrates that the grounding capability transfers to real-world scenes not seen during training, suggesting domain adaptation would require relatively modest fine-tuning rather than from-scratch training.

Accessibility applications requiring agent-centric spatial reasoning. The Perspective Transform sub-task of TreeBench — where the model must adopt an agent's frame of reference rather than the viewer's — is directly relevant to assistive technology for visually impaired users. A user might ask "From where I'm standing, which direction is the exit?" or "Is the cup to my left or right?" These questions require the model to (1) identify the user in the image, (2) determine their orientation (which way they're facing), and (3) compute the egocentric direction to the target object. TreeVGR-7B achieves 43.33% on Perspective Transform (Table 2), which is far from production-ready but represents a starting point — and the traceable bounding boxes mean that when the model is wrong, you can diagnose whether it failed to find the user, misidentified their orientation, or computed the wrong relative direction. This diagnostic capability is crucial for safety-critical assistive applications where wrong answers have real-world consequences (directing a user into a hazard). The paper's MMVP result (75.3, surpassing Qwen2.5-VL-72B at 68.0) further suggests that TreeVGR's training improves perception of subtle visual differences that matter for distinguishing similar-looking objects in real environments.

Satellite and drone imagery analysis with verifiable object counts. The VisDrone component of TreeVGR's RL training data — reformulated as multiple-choice counting problems with 5-10 targets — directly targets a practical need in remote sensing: counting objects (vehicles, structures, livestock) in high-resolution aerial imagery. Current approaches typically use specialized detection models that output bounding boxes, then count them. TreeVGR offers an alternative: a single model that both localizes objects (producing verifiable bounding boxes) and reasons about them (answering "how many red cars are in this parking lot?"). The verification aspect matters because satellite imagery analysis is often used for decisions with significant consequences (disaster response resource allocation, agricultural subsidy verification), and the ability to inspect the model's intermediate bounding boxes provides accountability. The paper's demonstration that TreeVGR improves on MME-RealWorld-Lite (+12.64, Table 4), which includes diverse real-world scenes, suggests reasonable generalization from the VisDrone drone imagery to other overhead imagery domains, though direct evaluation on a satellite imagery benchmark would be needed to confirm.

Medical image triage with auditable region-of-interest localization. While the paper does not evaluate on medical images, the core capability TreeVGR develops — locating small, subtle targets in complex visual scenes and providing verifiable bounding boxes for those locations — is directly relevant to radiology triage. A model that can examine a chest X-ray, locate potential abnormalities (nodules, opacities), and provide bounding boxes for radiologist review could serve as a prioritization tool: cases with high-confidence abnormality detections get expedited review, while the bounding boxes ensure the radiologist can quickly verify or dismiss the model's findings. The traceable evidence is particularly important here because medical AI faces stringent regulatory requirements for explainability — a "black box" diagnosis is unacceptable, but a diagnosis accompanied by "here is exactly where I'm looking and what I see" is closer to approvable. TreeVGR-7B's strong performance on small-object perception tasks (Table 2: 48.98% on Object Retrieval, where target descriptions are linguistically complex and spatially precise) suggests the model can handle the kind of precise localization medical triage requires, though domain-specific fine-tuning on medical images with expert annotations would be essential and the current 45% overall accuracy on TreeBench indicates substantial room for improvement before clinical deployment.

When to Prefer This Method

The paper does not explicitly articulate a tradeoff matrix between TreeVGR and named alternatives. TreeVGR is positioned as an improvement over the base model (Qwen2.5-VL-7B-Instruct) and as competitive with or exceeding other visual grounded reasoning models (DeepEyes, Pixel-Reasoner) on multiple benchmarks, but it is not contrasted against those alternatives in terms of when a practitioner should choose one over the other. The choice of TreeVGR over a larger general LMM (e.g., InternVL3-78B) is implied by the efficiency results — comparable TreeBench accuracy at ~11× fewer parameters — but the paper does not provide latency, throughput, or deployment-cost comparisons that would make this a concrete decision rule. The choice of TreeVGR over other visual grounded reasoning models is supported by the superior benchmarks (Tables 2-4) but not by a principled analysis of failure mode complementarity or domain specialization. Given this, a prescriptive "prefer A when X, prefer B when Y" section would be imposing a structure the paper does not itself provide.

Dependency tree

Papers this one builds on, walking backwards through the same domain and shared topics.

GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning2025· multimodal← in-domain predecessor
- G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learning2025· training-methods← LLM-judged similar
- MMaDA: Multimodal Large Diffusion Language Models2025· multimodal← in-domain predecessor
- From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning2026· multimodal← similar topic
- Reinforcing Multimodal Reasoning Against Visual Degradation2026· multimodal← similar topic
- Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory2025· multimodal← similar topic
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency2025· multimodal← similar topic
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning2025· multimodal← similar topic
- Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey2024· multimodal← similar topic

What it influenced

Later papers in the same chain. Direction reversed.

GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning2025· multimodal→ LLM-judged similar
- Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale2025· data→ LLM-judged similar
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning2025· multimodal→ LLM-judged similar
- SpatialTree: How Spatial Abilities Branch Out in MLLMs2025· multimodal→ LLM-judged similar
- SAIL-RL: Guiding MLLMs in When and How to Think via Dual-Reward RL Tuning2025· rl-training→ LLM-judged similar
- Unified Reinforcement and Imitation Learning for Vision-Language Models2025· training-methods→ LLM-judged similar
- PairUni: Pairwise Training for Unified Multimodal Language Models2025· training-methods→ LLM-judged similar
- Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning2025· vision→ LLM-judged similar
- Thinking with Drafting: Optical Decompression via Logical Reconstruction2026· reasoning→ LLM-judged similar
- Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey2024· multimodal→ similar topic
- Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory2025· multimodal→ similar topic
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency2025· multimodal→ similar topic
Voxtral2025· multimodal→ in-domain predecessor
- Meta CLIP 2: A Worldwide Scaling Recipe2025· multimodal→ in-domain predecessor
Reinforcing Multimodal Reasoning Against Visual Degradation2026· multimodal→ similar topic
- Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context2026· multimodal→ in-domain predecessor
- Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation2024· multimodal→ similar topic