Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning

ArXiv: 2510.27606

🎯 Pitch

Spatial-SSRL trains vision-language models to think in 3D using only unlabeled images and simple geometric puzzles—flipping patches, predicting depth order, shuffling regions—with no human annotation. The result is a 7B model that beats its teacher by 8.67% on complex spatial reasoning and even boosts general vision QA, proving self-supervised RL can build robust spatial intelligence without sacrificing broad visual skills.

---

1. Executive Summary

This paper introduces Spatial-SSRL, a self-supervised reinforcement learning paradigm that derives verifiable supervision directly from intrinsic image structure to improve spatial understanding in Large Vision-Language Models. Trained on Qwen2.5-VL models using five automatically constructed pretext tasks—three depth-free tasks targeting 2D layout (shuffled patch reordering, flipped patch recognition, cropped patch inpainting) and two depth-based tasks targeting 3D reasoning (regional depth ordering, relative 3D position prediction)—the approach optimizes via Group Relative Policy Optimization with deterministic binary rewards computed from image consistency alone, requiring no human labels or external tool chains. Across seven spatial understanding benchmarks in both image and video settings, Spatial-SSRL delivers average accuracy gains of 4.63% (3B) and 3.89% (7B) over Qwen2.5-VL baselines, with the largest improvement of +8.67% on Spatial457—a benchmark requiring complex pose estimation and multi-step spatial reasoning—while simultaneously improving general VQA capabilities by 2.02% (3B) and 0.57% (7B), establishing that self-supervised RLVR can enhance spatial intelligence without degradation of non-spatial visual capabilities only when the pretext tasks require holistic image understanding that transfers to general scene comprehension.

2. Context and Motivation

The Core Problem: LVLMs Still Struggle with Spatial Understanding

Despite remarkable progress in Large Vision-Language Models across a wide range of visual tasks—visual question answering, image captioning, object segmentation, and even mathematical reasoning—spatial understanding remains a persistent weakness. As the authors note in Section 1, LVLMs report "near-saturated results on various tasks" yet their spatial understanding "remains substantially below human performance." This is not a marginal gap; it is a fundamental limitation that manifests across diverse spatial capabilities: reasoning about depth, distance, azimuth, relative object positions, occlusion relationships, and viewpoint transformations.

The paper's qualitative examples in Figure 1(a) illustrate the nature of this failure concretely. When asked "Which object has a lower location?" in a scene with a cat on a bench and a boat in water, Qwen2.5-VL-7B incorrectly answers that the cat is lower because "the ground (where the cat is sitting) is lower than the water (where the boat is floating)"—failing to account for the fact that benches elevate the cat above ground level while the boat floats at water level, which could be higher or lower depending on the actual 3D geometry. Similarly, when asked which side of a person on a bicycle faces a specific sign, the baseline model incorrectly answers "Right" because the sign is on the right side of the image, failing to perform the mental rotation needed to determine that from the cyclist's perspective (facing the camera), their back—not their right side—faces the sign. These are not esoteric edge cases; they represent systematic failures in 3D spatial reasoning that would be trivial for any human observer.

Why This Matters: Real-World Impact

The practical stakes are substantial. The paper identifies three categories of applications where spatial understanding is essential (Section 1):

• Autonomous driving: Vehicles must reason about the 3D positions, velocities, and orientations of surrounding objects to make safe navigation decisions. A model that confuses image-plane relationships with real-world 3D relationships (as in the cyclist-sign example) would make catastrophic errors in driving scenarios.
• Robot manipulation: Robots operating in physical environments need to understand spatial relationships between objects, tools, and their own effectors—which object is closer, whether a grasp is feasible given an object's orientation, how to navigate around obstacles.
• Embodied navigation: Agents moving through real or simulated 3D environments must build and maintain spatial representations, reason about occluded regions, and plan paths through complex geometry.

Beyond these immediate applications, spatial understanding underpins a broader class of reasoning that requires mapping between 2D observations and 3D world structure. Any LVLM deployed in physical-world contexts—from augmented reality assistants to construction planning tools to medical imaging analysis—will encounter spatial reasoning demands that current models handle unreliably.

The Technical Significance

The problem also has theoretical importance. Spatial reasoning requires integrating multiple perceptual and cognitive capacities: depth perception from monocular cues, mental rotation and coordinate transformation, understanding of perspective projection, and relational reasoning about object positions. Improving spatial understanding in LVLMs is therefore a proxy for strengthening several fundamental capabilities simultaneously. If a model can correctly determine that "the person's back faces the sign" rather than "the person's right side faces the sign," it demonstrates not just better vision but better integration of viewpoint information with relational reasoning—a capability that likely transfers to other tasks requiring perspective-taking and coordinate transformation.

Prior Approaches and Their Limitations

The paper identifies three dominant paradigms that have been used to improve spatial understanding in LVLMs, each with significant limitations that motivate the search for a new approach.

Supervised Fine-Tuning on Synthesized or Annotated Data

The most straightforward approach is to curate high-quality spatial question-answer pairs and fine-tune models on them. Early data-centric approaches like SpatialVLM synthesize spatial QA pairs from 2D images by applying rule-based transformations, while SpatialRGPT constructs single-image 3D scene graphs using depth estimation plugins and extracts spatial relationships from them. However, the authors identify several fundamental problems with this paradigm:

• Memorization rather than generalization: SFT tends to produce models that memorize "dataset-specific patterns" rather than learning generalizable spatial reasoning. The model may learn to recognize that a particular answer template is associated with certain visual features in the training distribution, without developing the underlying spatial reasoning capability. This is a well-documented phenomenon in fine-tuning—models exploit spurious correlations that happen to be predictive in the training set but fail to generalize.
• Error propagation from tool chains: When SFT data is generated using external tools (object detectors, monocular depth estimators, segmentation models), errors from those tools propagate into the training labels. A depth estimator that incorrectly places an object relative to another produces a training example that actively teaches the wrong spatial relationship.
• Dependence on expensive proprietary models: Many SFT pipelines use GPT-4V or similar proprietary models as judges or data generators for QA curation. This makes the pipeline expensive, non-reproducible, and dependent on closed-source infrastructure.
• Limited scalability: Manual annotation by human experts is prohibitively expensive at scale—each spatial QA pair requires careful annotation of 3D relationships, which is time-consuming even for trained annotators.

The paper cites evidence that SFT on spatial data may be particularly vulnerable to memorization over generalization, consistent with broader findings in the post-training literature that RL-based optimization produces better out-of-distribution generalization than SFT.

RLVR with Specialized Tools and Constrained Environments

More recent work has applied Reinforcement Learning with Verifiable Rewards (RLVR) to spatial understanding, following the success of this paradigm in mathematical reasoning (DeepSeek-R1) and other domains. The idea is compelling: define a verifiable reward function that can deterministically check whether a model's spatial answer is correct, then optimize the model to maximize this reward through policy gradient methods like GRPO.

However, existing RLVR approaches for spatial understanding are constrained by their data sources:

• SpatialLadder uses progressive training on ScanNet—a dataset of 3D indoor scans with rich meta-information. While this provides verifiable rewards (the 3D geometry is known from the scans), it is fundamentally limited to the specific environments in ScanNet (primarily indoor rooms) and cannot generalize to the diverse scenes encountered in real-world applications.
• SpaceR and similar approaches also rely on "annotation-rich public datasets and tool-heavy pipelines" to curate training data for RL. The verifiable supervision comes from meta-information (bounding boxes, depth maps, camera poses) that is available in these curated datasets but not in general imagery.
• Some approaches use simulation-based rendering of 3D scenes to synthesize QA pairs with known ground truth, but the authors note that the "quality remains unsatisfactory due to divergence from real-world scenarios"—the domain gap between synthetic renders and real photographs is substantial, and models trained on simulated data often fail to transfer.

The paper summarizes this limitation in Figure 2(a): prior pipelines inject "extrinsic supervision" from expert tools or synthetic environments, which "inflates cost and limits scalability." The fundamental tension is that RLVR's power comes from verifiable rewards at scale, but obtaining those rewards has historically required either (a) expensive human annotation, (b) specialized sensor data (3D scans, depth sensors) with limited domain coverage, or (c) synthetic environments with domain gap issues.

The Broader Challenge: Scaling Verifiable Supervision

The authors frame the key open challenge as follows: "to retain the optimization benefits of RL while scaling verifiable supervision to ordinary images across diverse domains without manual labels, specialized assets, or costly tooling." This is a specific instance of a more general problem in machine learning: how do we generate high-quality training signals at scale without expensive human annotation or environment-specific infrastructure?

Traditional self-supervised learning offered one solution for representation learning—contrastive methods, masked autoencoders, and jigsaw tasks all derive supervision from data structure—but these were primarily used for pre-training visual encoders, not for directly optimizing LVLM behavior through reinforcement learning. The insight of Spatial-SSRL is that the same self-supervised signals that work for representation learning can be repurposed as verifiable reward functions for RL-based behavioral optimization.

How Spatial-SSRL Positions Itself

The paper positions itself at the intersection of three research threads: self-supervised learning (SSL) for visual representation, reinforcement learning with verifiable rewards (RLVR) for LLM/LVLM optimization, and spatial understanding as a critical capability gap. The synthesis creates something none of the individual threads provides.

Drawing on Self-Supervised Learning's Intrinsic Supervision

The paper explicitly draws on the visual SSL tradition (Section 1), which "learns visual representations from intrinsic structure in data via pretext objectives such as contrastive alignment and permutation/jigsaw tasks without manual labels." Classic SSL tasks—solving jigsaw puzzles, detecting image rotations, predicting temporal ordering—all share a crucial property: the correct answer is deterministically derivable from the data itself, because the experimenter applies the transformation. If you shuffle image patches, you know the correct unshuffling order because you chose the permutation. If you rotate an image, you know the correct rotation angle because you applied it.

The key conceptual move is to recognize that this property—deterministically verifiable ground truth—is exactly what makes a task suitable for RLVR. The authors state: "Because SSL targets are defined by the pretext itself, their correctness is deterministically verifiable during training, making SSL well aligned with the RLVR training paradigm."

However, prior SSL work was almost exclusively used for encoder pre-training—training a vision backbone to produce good representations that can later be fine-tuned on downstream tasks. The Spatial-SSRL contribution is to repurpose SSL objectives as reward functions for post-training behavioral optimization of an already-trained LVLM. This shifts the role of SSL from "learn good features" to "learn good spatial reasoning behaviors," which is a fundamentally different objective. The SSL signal supervises not just what the vision encoder extracts but how the full model thinks about and reasons about spatial relationships.

Contrast with Prior SSL for Post-Training

The paper acknowledges recent work that has begun applying SSL ideas to LVLM post-training:

• Jigsaw-R1 uses jigsaw puzzles for RL-based training of LVLMs.
• Visual Jigsaw is a concurrent work constructing jigsaw tasks across three modalities.
• SSL4RL focuses on 2D tasks for RL-based LVLM optimization.

Spatial-SSRL distinguishes itself through (a) covering a "broader set of self-supervised tasks, with Jigsaw as only one component," and (b) specifically targeting spatial understanding rather than general visual reasoning. The paper argues that SSL4RL's focus on 2D tasks limits its applicability to 3D spatial reasoning, while Spatial-SSRL's inclusion of depth-based tasks (regional depth ordering, relative 3D position prediction) provides explicit 3D supervision.

The Three Design Principles That Define the Contribution

The paper articulates three principles that distinguish Spatial-SSRL from prior work (Section 3):

1. Zero human or LLM supervision: All ground-truth labels are "derived deterministically from image structure." There is no human annotator deciding whether a depth ordering is correct, no GPT-4V generating reasoning traces or verifying answers, no crowd-sourced spatial judgments. The image and the applied transformation uniquely determine the correct answer.
2. Tool-free scalability: The pipeline requires "no external detection, segmentation, or rendering pipelines." Prior approaches like SpatialVLM and SpatialRGPT depend on chains of external models—object detectors to find regions, depth estimators to get depth values, segmentation models to isolate objects—each of which introduces computational cost, potential errors, and maintenance burden. Spatial-SSRL only needs raw RGB images (and optionally RGB-D for depth-based tasks), with all task construction done through deterministic operations on pixels and depth values.
3. Natural verifiability: Because the supervision signals come from the pretext tasks themselves, they are "verifiable and suitable for RL reward computation." The verifier is a simple program that checks whether the model's predicted patch order matches the known permutation, not a learned model that could itself be gamed.

These principles collectively enable what the paper calls "a scalable, lightweight, low-cost, and naturally verifiable pipeline" (Figure 2 caption). This is not an incremental improvement over existing approaches—it is a fundamentally different approach to generating training signals for spatial understanding that removes the bottlenecks (cost, domain specificity, tool-chain complexity) that limited prior work.

The Theoretical Framing: Intrinsic vs. Extrinsic Supervision

The paper frames its contribution through the contrast between extrinsic supervision (annotations from humans, tools, or synthetic environments) and intrinsic supervision (signals derived from the structure of the data itself). This framing, illustrated in Figure 2, makes clear that Spatial-SSRL is not just a new method but a new paradigm for spatial understanding training. Under this paradigm:

• Data collection consists of gathering ordinary RGB or RGB-D images from commodity sensors or public sources—no specialized 3D scanners, no human-labeled spatial annotations, no proprietary model outputs.
• Data augmentation is lightweight and cost-effective task construction via deterministic transformations on the raw data—shuffling, flipping, cropping, depth thresholding, coordinate transformation.
• Model optimization uses the RLVR framework where the reward function is a deterministic verifier checking the model's answer against the known ground truth from the self-supervised task construction.

This paradigm is positioned as a solution to the fundamental scalability problem: because intrinsic supervision can be generated automatically from arbitrary images—including those from public datasets like COCO and DIODE with all human annotations discarded—the training data can scale to millions of examples across diverse domains without proportional increase in annotation cost. The paper's Spatial-SSRL-81k dataset, constructed from just three image sources, demonstrates the immediate practicality of this approach at modest scale.

Reconciling Conflicting Findings in Spatial Understanding Research

The paper implicitly addresses a tension in the literature that parallels the one noted in the test-time compute paper discussed in the prompt: some approaches find that improving spatial understanding is tractable (SpatialVLM reports gains, SpatialLadder reports gains), while others document persistent failures (benchmarks like Spatial457 and 3DSRBench show that even strong LVLMs perform poorly on spatial reasoning). The Spatial-SSRL framework suggests that the key variable is not whether models can learn spatial reasoning—they can, given appropriate training signals—but rather whether the training signals provide dense, verifiable feedback across a diverse range of spatial capabilities and visual domains.

Prior approaches that showed limited success often provided sparse or domain-specific signals (e.g., only indoor scenes from ScanNet, or only synthetic QA pairs with limited diversity). Prior approaches that showed more success (tool-heavy pipelines) achieved coverage but at high cost and with error propagation. The Spatial-SSRL paradigm proposes that dense, diverse, error-free supervision—the kind that comes from deterministic self-supervised tasks—is the missing ingredient, and that RL-based optimization (rather than SFT) is the appropriate training paradigm to exploit it, consistent with the broader finding that "SFT memorizes, RL generalizes." The paper's ablation studies (Table 4) provide empirical support for this view: combining diverse self-supervised tasks yields gains beyond any individual task or task category, suggesting that coverage matters independently of task-specific optimization.

3. Technical Approach

3.1 Reader Orientation (Approachable Technical Breakdown)

Spatial-SSRL is a training framework—not a new model architecture—that takes an existing pretrained Large Vision-Language Model (like Qwen2.5-VL) and teaches it to reason better about 3D spatial relationships by having it practice on automatically generated puzzle-like tasks where the correct answer is always known with certainty. The core problem it solves is: how do you give an LVLM verifiable feedback about spatial reasoning at massive scale, without paying humans to label data or building expensive pipelines of depth estimators, object detectors, and 3D renderers? The solution takes the shape of a self-supervised reinforcement learning loop: (1) take ordinary RGB or RGB-D images, (2) apply deterministic transformations to them—shuffling patches, flipping patches, masking regions, comparing depths, rotating coordinate frames—to create question-answer pairs where the ground truth is mathematically derivable from the transformation itself, and (3) optimize the LVLM using Group Relative Policy Optimization (GRPO) with a binary reward that simply checks whether the model's answer matches the known correct answer. The entire training signal comes from the intrinsic structure of images, not from any external annotation source.

3.2 Big-Picture Architecture (Diagram in Words)

The Spatial-SSRL system has four major components organized as a pipeline:

1. Data Ingestion: Raw RGB images from COCO and RGB-D images (RGB + per-pixel depth maps) from DIODE and MegaDepth are collected. Critically, all human-provided annotations—bounding boxes, segmentation masks, captions—are discarded. The pipeline only uses the pixel values and (where available) depth values.

2. Self-Supervised Task Construction: Five distinct "pretext tasks" are automatically formulated from the raw imagery. Each task applies a known, deterministic transformation to an image (or its depth map) and poses a question about the transformation. Because the transformation is applied programmatically, the correct answer is known with 100% accuracy. The five tasks are organized into two categories:

   • Depth-free tasks (operating on RGB only, targeting 2D spatial structure): Shuffled Patch Reordering, Flipped Patch Recognition, Cropped Patch Inpainting.
   • Depth-based tasks (requiring RGB-D, targeting 3D spatial reasoning): Regional Depth Ordering, Relative Position Prediction.

   The output of this component is the Spatial-SSRL-81k dataset: 81,053 question-answer pairs spanning all five tasks in roughly balanced proportions.

3. Cold-Start Supervised Fine-Tuning (SFT): Before reinforcement learning, the base LVLM is briefly fine-tuned on a small subset (~3,600 samples, about 4.4% of the full dataset) of the self-supervised QA pairs. This phase is not about teaching spatial reasoning per se—it is about teaching the model the format of the tasks (e.g., how answers should be structured, that answers go in \boxed{}) so that it can produce valid responses during the RL phase. Without this cold-start, the model's initial responses have a success rate below 5% for complex tasks, causing RL training to collapse.

4. GRPO Reinforcement Learning: The full Spatial-SSRL-81k dataset is used to optimize the model via Group Relative Policy Optimization. For each training sample, the model generates 5 candidate responses (a "rollout group"). Each response receives a reward with two components: an accuracy reward (1 if the extracted answer matches the known ground truth, 0 otherwise) and a format reward (1 if the response follows the prescribed ⟨think⟩...⟨/think⟩ and \boxed{} format, 0 otherwise). The total reward is $r = 0.9 \cdot r_{\text{acc}} + 0.1 \cdot r_{\text{fmt}}$. The model is updated to increase the probability of high-reward responses relative to the group average. A reference model (the frozen SFT checkpoint) provides KL regularization to prevent catastrophic forgetting.

Information flows linearly: raw images → task construction → SFT cold-start → GRPO optimization → trained Spatial-SSRL model. At inference time, the trained model can be prompted with standard spatial reasoning benchmarks (or any spatial query) using the same format prompt it saw during training, eliciting structured step-by-step reasoning.

3.3 Roadmap for the Deep Dive

I will explain the Spatial-SSRL framework in the following order, which builds from the foundational design philosophy up through the training mechanics:

• First, the three design principles and the intrinsic-vs-extrinsic supervision framing (Section 3), because these motivate every subsequent design choice—why the tasks are constructed as they are, why RL rather than pure SFT, why no external tools are used.

• Second, the five self-supervised tasks in detail (Section 3.1), organized by category. For each task, I will cover: (a) the motivation—what spatial capability it targets and why, (b) the mathematical formulation—exactly what transformation is applied and how the ground truth is derived, (c) the question template and answer format, and (d) any difficulty-enhancement mechanisms. Understanding the tasks is essential because they are the curriculum; the RL training simply optimizes the model to perform well on them, and the hypothesis is that this performance transfers to real spatial reasoning.

• Third, the dataset construction process (Section 3.1.3), including data sources, task balancing, and the resulting Spatial-SSRL-81k statistics. This connects the task designs to the actual training data the model sees.

• Fourth, the RL training procedure (Section 3.2), including the cold-start SFT phase, the GRPO algorithm configuration, the reward function design, and the format prompting strategy. This is where the self-supervised signals are converted into behavioral optimization.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methods and empirical validation paper whose core idea is that self-supervised pretext tasks—constructed deterministically from raw images and depth maps—can serve as verifiable reward signals for RL-based optimization of spatial understanding in LVLMs, replacing the expensive, domain-limited, tool-heavy supervision pipelines of prior work.

---

The Three Design Principles: Intrinsic Supervision as the Foundation

The entire Spatial-SSRL framework is built on three explicit design principles (stated at the start of Section 3) that collectively define what "self-supervised RL for spatial understanding" means and distinguish it from prior approaches. Understanding these principles is essential because every subsequent design choice—which tasks to construct, how to construct them, how to train on them—is justified by reference to one or more of these principles.

Principle (i): Zero human or LLM supervision. All ground-truth labels must be "derived deterministically from image structure." This means that for any given image and any given transformation applied to it, a simple program (with no learned components, no human judgment, no LLM-based verification) can compute the correct answer with 100% accuracy. There is no ambiguity, no inter-annotator disagreement, no model-generated "silver labels" that might contain errors. This principle rules out approaches like: asking GPT-4V to verify whether a model's spatial answer is correct (introduces LLM biases and errors), having humans label spatial relationships (expensive, slow, potentially inconsistent), or using off-the-shelf depth estimators to generate pseudo-ground-truth (propagates depth estimation errors into the training signal).

Principle (ii): Tool-free scalability. The pipeline requires "no external detection, segmentation, or rendering pipelines." This is a stronger condition than just "no human labels." It means the system cannot depend on any auxiliary model—not an object detector to locate regions of interest, not a depth estimator to compute depth values (depth must come from sensor data, i.e., RGB-D images, not from monocular depth prediction), not a segmentation model to isolate objects, not a 3D renderer to synthesize novel views. The rationale is both practical and philosophical: every tool in the chain introduces (a) computational cost, (b) potential errors that propagate into training labels, (c) a maintenance burden (tools must be updated, may have license restrictions, may become unavailable), and (d) a domain constraint (tools trained on specific data distributions may fail on out-of-distribution images, limiting the diversity of training data). By being tool-free, Spatial-SSRL can process any image for which raw pixel values (and optionally sensor depth) are available, with no restrictions based on what objects a detector can find or what scenes a depth estimator was trained on.

Principle (iii): Natural verifiability. The supervision signals must be "verifiable and suitable for RL reward computation." This connects the self-supervised task design directly to the RLVR training paradigm. Verifiability means that given the model's output and the known ground truth, a binary correctness judgment can be made automatically and deterministically. This is what enables the accuracy reward in GRPO: the reward function is a simple exact-match check, not a learned reward model that could be hacked or over-optimized. The "natural" qualifier emphasizes that verifiability should be an inherent property of the task design, not something retrofitted with additional machinery.

Why these principles collectively define a new paradigm. Prior approaches violate at least one of these principles, creating bottlenecks that limit scale, domain diversity, or label quality. SFT on human-annotated or LLM-generated data violates (i). Tool-heavy pipelines violate (ii). Simulation-based approaches violate (ii) (rendering engines are tools) and often produce domain gaps. RLVR approaches using ScanNet or similar annotated 3D datasets violate (ii) (the meta-information—camera poses, 3D reconstructions—is effectively a tool chain applied during data collection) and are domain-limited to the specific environments captured. Spatial-SSRL's contribution is not any single principle but the demonstration that all three can be simultaneously satisfied for spatial understanding tasks, and that this enables a qualitatively different scaling regime where training data is limited only by the availability of raw images, not by annotation budgets or tool-chain throughput.

The intrinsic vs. extrinsic supervision framing (Figure 2). The paper conceptualizes the difference between prior work and Spatial-SSRL through this contrast. Extrinsic supervision (Figure 2a) flows from external sources—human experts, specialized models, simulation engines—into the training pipeline, and each source imposes costs and constraints. Intrinsic supervision (Figure 2b) is latent in the data itself and only needs to be "revealed" through deterministic transformations. The arrows in Figure 2b tell the story: Camera & Sensors → RGB-D Images → (via lightweight automated augmentation) → Template QAs with Verifiable answers → RLVR optimization. The key enabling property is that "QA-Acc = 100%"—the self-supervised question-answer pairs have perfect ground-truth accuracy, which is impossible to guarantee with any extrinsic supervision source.

---

The Five Self-Supervised Tasks: A Complete Spatial Curriculum

The paper organizes its five pretext tasks into two complementary categories: depth-free tasks (Sections 3.1.1) that operate solely on RGB images and target 2D spatial structure, and depth-based tasks (Section 3.1.2) that require RGB-D inputs and target 3D spatial reasoning. The category division is not merely organizational—it reflects a hypothesis about what kinds of supervision are needed for comprehensive spatial understanding. The depth-free tasks teach the model to reason about layout, orientation, and structural consistency in the image plane, which are foundational skills. The depth-based tasks extend this into the third dimension, teaching ordinal depth perception and egocentric coordinate transformations. The ablation studies in Table 4 empirically validate this complementarity: models trained on both categories outperform models trained on either alone.

I will now walk through each task in full detail, following the same structure for each: motivation, mathematical formulation, question format, and difficulty-enhancement mechanisms.

---

Shuffled Patch Reordering

Motivation. Recovering the original spatial arrangement of randomly permuted image patches requires the model to reason about global 2D layout consistency and relative positional relationships between image regions. To correctly determine that "patch 3 belongs in the top-left position," the model must integrate multiple visual cues: edge continuity across patch boundaries, semantic coherence (e.g., an object split across patches should be reassembled), and typical scene layout priors (sky tends to be at the top, ground at the bottom). The paper argues that these skills "directly transfer to reasoning about object arrangements in real scenes"—understanding the relative positions of objects in a photograph is fundamentally the same cognitive operation as understanding the relative positions of image patches, just at a different level of abstraction.

Formulation. Given an input image $I \in \mathbb{R}^{H \times W}$ (height H, width W, with RGB channels implicit), the image is partitioned into an $M \times N$ grid of patches. Each patch has dimensions $P_H \times P_W$, where $P_H = H/M$ and $P_W = W/N$. The patch grid is denoted:

$X = \{x_{i,j} \in \mathbb{R}^{P_H \times P_W} \mid i \in [0, M), j \in [0, N)\}$

where $x_{i,j} = I(iP_H : (i+1)P_H, \; jP_W : (j+1)P_W)$ is the sub-image at grid position $(i, j)$. This is then flattened into a 1D sequence for manipulation:

$\hat{X} = [\hat{x}_0, \hat{x}_1, \ldots, \hat{x}_{M \times N - 1}] \quad \text{with} \quad \hat{x}_k = x_{i,j} \;\; \text{and} \;\; k = iN + j$

A random permutation $\pi: \{0, \ldots, M \times N - 1\} \mapsto \{0, \ldots, M \times N - 1\}$ is applied to produce the shuffled sequence:

$\hat{X}_{\text{shuffle}} = [\hat{x}_{\pi(0)}, \hat{x}_{\pi(1)}, \ldots, \hat{x}_{\pi(M \times N - 1)}]$

The shuffled patches are then reassembled into an image (by placing $\hat{x}_{\pi(k)}$ at position $k$ in reading order) to create the input image $I_{\text{input}}$ that the model sees.

Ground truth. The model must predict the correct ordering that restores the original image. Since $\pi$ is a bijection (a one-to-one mapping), its inverse $\pi^{-1}$ exists and provides the ground-truth answer:

$\pi^{-1} = [\pi^{-1}(0), \pi^{-1}(1), \ldots, \pi^{-1}(M \times N - 1)]$

where $\pi^{-1}(k)$ is the index of the patch that should be at position $k$ in the correct ordering. For example, if $\pi = [2, 0, 3, 1]$ (patch 2 is placed first, patch 0 second, etc.), then $\pi^{-1} = [1, 3, 0, 2]$ (the original patch 1 belongs at position 0, original patch 3 at position 1, etc.).

What this computes in operational terms. The system takes an image, divides it into patches, randomly reorders them, shows the shuffled version to the model, and asks the model to output the correct ordering. The correctness check is an exact string match between the model's predicted sequence and $\pi^{-1}$. The computation is purely deterministic—given the same random seed for $\pi$, the same input image and ground truth are produced every time.

Why this form over alternatives. A natural alternative would be to ask the model to predict the permutation $\pi$ itself (i.e., "where did each original patch go?"). The paper chooses to ask for $\pi^{-1}$ (i.e., "which patch belongs at each position?") because this maps more naturally to the spatial reasoning task: the model is reasoning about where things should be, which is analogous to reasoning about where objects should be in a coherent scene. Additionally, asking for the correct ordering in a fixed spatial template (TopLeft-TopRight-BottomLeft-BottomRight) provides a consistent output format that is easy to verify.

Difficulty enhancement: patch masking. The paper notes that edge continuity between adjacent patches can provide a trivial solution—the model could simply match boundary pixels without understanding the global scene layout. To prevent this shortcut, the system optionally masks one random patch $\hat{x}_{\pi(t)}$ by setting all its pixels to white:

$\hat{x}_{\pi(t)}(0:P_H, 0:P_W) = 255$

This forces the model to rely on global layout understanding rather than local edge matching, because one patch provides no boundary information. The implementation supports multiple patchification strategies (Appendix A.2): $M=2, N=2$ with and without masking (8,028 total samples), horizontal strips ($M=1, N \in \{3,4\}$, 4,991 samples), and vertical strips ($M \in \{3,4\}, N=1$, 3,009 samples). The strip variants force the model to reason about 1D orderings, which may transfer to tasks like temporal ordering or relative height reasoning.

Question template (for $M=2, N=2$, without mask): "The image has been divided into 4 shuffled patches labeled 0, 1, 2, and 3. Based on visual clues such as continuity, alignment, and context, answer the correct arrangement of the patches to restore the original image, where the format is 'TopLeft-TopRight-BottomLeft-BottomRight'." The expected answer format is a dash-separated sequence like "0-2-3-1".

---

Flipped Patch Recognition

Motivation. Detecting subtle orientation violations requires sensitivity to local geometry, mirror symmetries, and directional cues such as text, faces, shadows, and lighting gradients. A patch that has been vertically flipped will violate the expected orientation of objects (trees grow upward, text reads left-to-right, faces have eyes above mouths), and the model must detect these anomalies against a backdrop of correctly oriented patches. The paper argues that these capabilities are "essential for understanding viewpoint-dependent spatial relations"—recognizing that an object is viewed from an unusual angle requires the same sensitivity to orientation cues as recognizing that a patch has been artificially flipped.

Formulation. The image is partitioned into patches using the same grid structure as Shuffled Patch Reordering, producing the flattened sequence $\hat{X} = \{\hat{x}_k\}$ with $k \in [0, M \times N - 1]$. One patch index $t$ is randomly selected, and a flip operation $f: \hat{x}_t \mapsto \hat{x}_t^{\text{flip}}$ is applied:

$f(\hat{x}_t) = \begin{cases} x_{\text{vert}} & \text{with probability 0.5} \\ x_{\text{horz}} & \text{with probability 0.5} \end{cases}$

where the vertical flip is defined pixel-wise as $x_{\text{vert}}(r, c) = x(P_H - 1 - r, c)$ (flip over the horizontal midline) and the horizontal flip as $x_{\text{horz}}(r, c) = x(r, P_W - 1 - c)$ (flip over the vertical midline). The input image $I_{\text{input}}$ is reconstructed from the modified sequence $\hat{X}' = [\hat{x}_0, \ldots, \hat{x}_t^{\text{flip}}, \ldots, \hat{x}_{M \times N - 1}]$.

Ground truth. The model must identify both the index $t$ of the flipped patch and the flip direction $d \in \{\text{vert}, \text{horz}\}$. The ground-truth answer is the tuple $[t, d]$. For example, if patch 3 was flipped vertically, the correct answer would be "3-0" (the encoding $0 = \text{vert}$, $1 = \text{horz}$ as specified in the question template: "The direction can only be 0(flipped vertically) or 1(flipped horizontally)").

What this computes. The system creates an image where exactly one of four patches has been either vertically or horizontally mirrored, shows it to the model, and asks "which patch is flipped, and in which direction?" The correctness check verifies that both the patch index and the flip direction match. The model cannot succeed by simply finding an anomalous patch—it must also determine the specific orientation violation to distinguish it from the other possible anomaly.

Why this form over alternatives. A simpler task would only ask to identify which patch is anomalous, without specifying the flip direction. Requiring the direction forces the model to reason about the nature of the anomaly, not just its presence. This is a harder discriminative task: if the model only needs to say "patch 2 is weird," it could succeed through low-level texture statistics without understanding orientation. But distinguishing vertical from horizontal flips requires recognizing that certain orientation-specific features (e.g., text direction, gravity direction, face configuration) are violated in a particular way. This maps to real spatial reasoning where understanding viewpoint changes requires knowing not just that something is different, but how it is different (e.g., "the object is viewed from the left" vs. "the object is viewed from below").

Relation to Shuffled Patch Reordering. The paper notes (Appendix A.1) that these two tasks "share similar image layouts and structures" and are treated as one large task for balancing purposes, evenly mixed with the other three tasks. Both use $2 \times 2$ patch grids as the primary configuration (4,005 samples for Flip), which is smaller than the Shuffled Patch Reordering dataset because the latter includes multiple patchification strategies.

Question template: "The image has been divided into 4 labeled 0, 1, 2, and 3. One of the four patches is flipped either horizontally or vertically. Based on visual clues such as continuity, alignment, and context, answer the correct patch that is flipped and the direction the flip, where the format is 'Label-Direction'. The direction can only be 0(flipped vertically) or 1(flipped horizontally)." Expected answer format: e.g., "3-0".

---

Cropped Patch Inpainting

Motivation. Identifying which patch correctly fills a masked region requires analyzing texture continuity, semantic context, and structural consistency between local content and its surroundings. This is fundamentally a matching task: given the surrounding context, which candidate patch is the correct completion? Unlike Shuffled Patch Reordering (which requires global arrangement reasoning) and Flipped Patch Recognition (which requires anomaly detection), Cropped Patch Inpainting requires fine-grained texture-context correspondence—the model must attend to how edges, patterns, and semantic elements continue across the boundary between the masked region and its surroundings. The paper argues that these skills "generalize to understanding spatial coherence" in real scenes, where objects are partially occluded and the model must infer the missing structure from visible context.

Formulation. From an image $I \in \mathbb{R}^{H \times W}$, a square patch of side length $s = \min(H/2, W/2)$ is extracted. The top-left corner $(x_0, y_0)$ of the crop region is sampled uniformly:

$(x_0, y_0) \sim \mathcal{U}\big([0, H - s] \times [0, W - s]\big)$

The cropped region is $\mathcal{R} = [x_0, x_0 + s] \times [y_0, y_0 + s]$, and the extracted patch is $I_{\text{crop}} = I(\mathcal{R})$—the pixel values within that rectangular region. The masked input image $I_{\text{input}}$ zeros out the cropped region:

$I_{\text{input}}(u, v) = \begin{cases} 0 & \text{if } (u, v) \in \mathcal{R} \\ I(u, v) & \text{otherwise} \end{cases}$

Task objective. The model is presented with $I_{\text{input}}$ (showing a black rectangle where content is missing) and four candidate patches in a multiple-choice format. It must select the correct patch $I_{\text{crop}}$ that fills the masked region. The answer is a single letter: 'A', 'B', 'C', or 'D', corresponding to the position of the correct patch among the four options.

Distractor construction: preventing trivial solutions. The paper invests significant design effort into constructing the three incorrect options (distractors) to prevent the model from succeeding through low-level heuristics. If the distractors were random patches from unrelated images, the model could simply pick the one with matching color statistics or texture patterns without understanding spatial coherence. Instead, all distractors are derived from the same input image $I$ and share substantial visual similarity with the ground-truth patch. The three distractor types, each selected with probability specified in Appendix Table 7, are:

1. 90° rotation (probability 0.2 clockwise, 0.2 counterclockwise): $I_{\text{crop}}$ is rotated by 90°. This preserves all the visual content—colors, textures, objects—but disrupts the orientation. A model that only matches low-level features without understanding orientation would find this distractor equally plausible as the correct unrotated patch. The model must recognize that, for example, a vertical edge in the surrounding context should continue vertically through the patch, not horizontally.

2. Interior subregion (probability 0.2): A smaller central crop of $I_{\text{crop}}$, specifically $I_{\text{int}} = I_{\text{crop}}(\mathcal{R}_{\text{int}})$ where $\mathcal{R}_{\text{int}} = [s/4, 3s/4 - 1] \times [s/4, 3s/4 - 1]$. This is a zoomed-in version of the center of the correct patch, resized to $s \times s$. It shares semantic content with the correct patch but has different boundary information—the edges that should align with the surrounding context are missing. The model must recognize that the correct patch should have consistent boundaries, not just consistent center content.

3. Exterior region (probability 0.2 for $\theta = 0.25$, 0.2 for $\theta = 0.5$): A larger region surrounding the crop area, $I_{\text{ext}} = I(\mathcal{R}_{\text{ext}})$ where $\mathcal{R}_{\text{ext}} = [x_0 - \theta s, x_0 + (1 + \theta)s] \times [y_0 - \theta s, y_0 + (1 + \theta)s]$, clipped to image boundaries and resized to $s \times s$. This patch includes the correct patch plus surrounding context, so when downsized, it looks like a blurred or context-heavy version of the correct region. The model must recognize fine-grained structural details to distinguish this from the exact crop.

All distractors are resized to $s \times s$ to prevent the model from using size as a discriminative cue—the correct patch is not distinguishable by resolution or aspect ratio.

What this computes. The system creates a "hole" in an image, selects the correct fill and three cleverly deceptive alternatives, and asks the model to identify the correct one. Success requires integrating information across the boundary of the masked region—matching textures, continuing edges, maintaining semantic coherence—which is essentially a test of whether the model understands how local image regions relate to their spatial context.

Why this form over alternatives. The task is formulated as multiple-choice with image options (the model sees four small images as options A–D, plus the masked main image) rather than as a generative task ("generate the missing pixels"). This is critical for compatibility with the LVLM interface—the model reasons over the visual options and outputs a letter, which is easy to verify with exact match. A generative inpainting task would require pixel-level output and a different verification strategy (e.g., perceptual similarity metrics). The multiple-choice format also constrains the problem: the model doesn't need to generate pixels from scratch, just discriminate the correct completion from plausible alternatives. This maps more directly to spatial reasoning, where the question is typically "which of these spatial relationships is correct?" rather than "generate the 3D structure of this scene."

The distractor design is the key intellectual contribution of this task. By deriving distractors from the same image with controlled transformations (rotation, zoom, context expansion), the paper creates a challenging discrimination task where superficial similarity is high but spatial coherence distinguishes the correct answer. This is analogous to how spatial reasoning benchmarks often present answer choices that are plausible but incorrect based on a specific spatial relationship—the model must attend to the specific geometric constraint, not just general visual similarity.

Question template: "Which image is the missing part in the first image <image1>? Based on visual clues such as alignment, image content, and positional relationship, select one of the four options <image2><image3><image4><image5> as the final answer. The final answer should be chosen from 'A', 'B', 'C', and 'D'." The model sees up to 5 images in the input (the masked image + 4 options), with the correct answer being the letter corresponding to the correct patch.

---

Regional Depth Ordering

Motivation. This task shifts from 2D image-plane reasoning to 3D spatial reasoning, using depth maps from RGB-D sensors. Ranking regions by distance from the camera requires integrating depth cues, perspective understanding, and ordinal reasoning—foundational skills for occlusion reasoning (which object is in front of which?), 3D scene reconstruction, and navigation planning. Unlike the depth-free tasks where the model could potentially succeed through 2D pattern recognition, depth ordering explicitly requires reasoning about the third dimension.

Formulation. Given an RGB image $I \in \mathbb{R}^{H \times W}$ and its corresponding normalized depth map $D \in \mathbb{R}^{H \times W}$ (where each pixel value represents the distance from the camera, normalized to [0, 1]), the system selects three disjoint regions $\mathcal{R}_1, \mathcal{R}_2, \mathcal{R}_3 \subseteq [0, H] \times [0, W]$ with strictly increasing depth: $\mathcal{R}_1$ is closest to the camera, $\mathcal{R}_2$ is at intermediate distance, $\mathcal{R}_3$ is farthest.

Constraints for unambiguous ordering. To ensure that the depth ordering is unambiguous (i.e., the model cannot argue that two regions are at similar depths), two constraints are enforced:

1. Within-region depth consistency: The depth range within each region must be small, ensuring each region is at a relatively uniform depth:

   $r(\mathcal{R}_i) = \max_{(x,y) \in \mathcal{R}_i} D(x, y) - \min_{(x,y) \in \mathcal{R}_i} D(x, y) < r_{\text{max}}$

   where $r(\mathcal{R}_i)$ is the depth range within region $i$, and $r_{\text{max}} = 0.15$ is the threshold (in normalized depth units, where the full depth range of the sensor is mapped to [0, 1]).

2. Between-region depth separation: The depth gap between consecutive regions must be large enough that their ordering is unambiguous:

   $d(\mathcal{R}_i, \mathcal{R}_{i+1}) = \min_{(x,y) \in \mathcal{R}_{i+1}} D(x, y) - \max_{(x,y) \in \mathcal{R}_i} D(x, y) > d_{\text{min}}$

   where $d(\mathcal{R}_i, \mathcal{R}_{i+1})$ is the depth gap between region $i$ and region $i+1$, and $d_{\text{min}} = 0.05$ is the minimum separation threshold.

What these constraints compute. $r_{\text{max}} = 0.15$ ensures that the depth variation within a region does not exceed 15% of the total depth range—the region is roughly planar or at a consistent distance. $d_{\text{min}} = 0.05$ ensures that adjacent regions in the depth ordering are separated by at least 5% of the total depth range—a small but reliable gap that prevents ties or ambiguous orderings. Together, these constraints guarantee that the correct depth ordering ($\mathcal{R}_1$ closer than $\mathcal{R}_2$ closer than $\mathcal{R}_3$) is unambiguously correct given the depth map.

Visual labeling. A random permutation $\hat{\pi}: \{1, 2, 3\} \mapsto \{1, 2, 3\}$ assigns visual labels (1, 2, 3) to the three regions. The regions are marked on the image with their assigned labels (e.g., region $\mathcal{R}_1$ might be labeled "2", region $\mathcal{R}_2$ labeled "1", region $\mathcal{R}_3$ labeled "3"), creating the input image $I_{\text{input}}$. This permutation ensures that the model cannot simply learn that "label 1 is always closest"—it must actually reason about the depth of each labeled region from visual cues.

Ground truth. The model must order the labeled regions from closest to farthest. The correct answer is the sequence:

$\hat{\pi} = [\hat{\pi}(1), \hat{\pi}(2), \hat{\pi}(3)]$

where $\hat{\pi}(1)$ is the visual label assigned to the closest region (originally $\mathcal{R}_1$), $\hat{\pi}(2)$ is the label of the middle-depth region, and $\hat{\pi}(3)$ is the label of the farthest region. For example, if the closest region was labeled "2", the middle region labeled "1", and the farthest labeled "3", the correct answer is "2-1-3".

What this computes in operational terms. The system identifies three image regions at clearly different depths (using the depth map, which is ground-truth sensor data, not estimated), randomly shuffles their visual labels, marks them on the RGB image, and asks the model to sort them from closest to farthest. The correctness check is an exact string match against the known label ordering. This is fundamentally different from prior approaches that would use a monocular depth estimator to get depth values—those estimators can make errors, potentially labeling a farther region as closer, which would create incorrect training data. Sensor depth, while potentially noisy, does not have systematic perceptual errors of the kind that would reverse depth orderings.

Why this form over alternatives. The task could have been formulated as a binary comparison ("is region A closer than region B?") rather than a three-way ranking. The three-way ranking is harder because it requires transitive reasoning: the model must establish that A < B, B < C, and therefore A < C, which is a more complex cognitive operation than pairwise comparison. It also provides more signal per sample: three regions yield two pairwise comparisons (A vs. B, B vs. C) plus the transitive inference.

The constraints ($r_{\text{max}}$, $d_{\text{min}}$) are critical design choices. Without them, the system might select regions where the depth ordering is genuinely ambiguous (e.g., two regions at very similar depths, or a region spanning a large depth range), and the model might reasonably disagree with the "ground truth." The thresholds ensure that the ground truth is not just technically correct (by the depth map) but perceptually unambiguous—a human looking at the image would agree with the ordering.

Question template: "The original image has three regions marked as 1, 2, and 3. Consider the content, positional relationships, depths of the three regions and other cues, and sort the depths of the three regions from smallest to largest from the camera, where the format of the answer is 'Smallest-Middle-Largest'." Expected answer format: e.g., "2-1-3".

---

Relative Position Prediction

Motivation. This is the most sophisticated of the five tasks, requiring the model to perform mental rotation and egocentric coordinate transformation. Determining spatial relations from an object's perspective (e.g., "from where the car is facing, is the pedestrian to its left or right?") is essential for embodied agents and autonomous systems that must reason about the world from the viewpoint of other entities, not just from the camera's perspective. The paper argues that this requires "mental rotation, egocentric coordinate transformation, and integration of orientation cues with depth information"—a combination of skills that no single simpler task captures.

Formulation. From an RGB-D image with depth map $D$, two pixel locations $\mathcal{R}_1$ and $\mathcal{R}_2$ are selected. An object is assumed to be located at $\mathcal{R}_1$ with a specified orientation. The model must predict the relative position of $\mathcal{R}_2$ from this object's viewpoint, choosing from a set of directional options (e.g., "Left," "Right-Front," "Left-Back").

Coordinate transformation. Let $(x_1, y_1, z_1)$ and $(x_2, y_2, z_2)$ be the 3D coordinates of $\mathcal{R}_1$ and $\mathcal{R}_2$ in the camera's coordinate frame, where $z_i = D(x_i, y_i)$ is the depth (distance from camera along the optical axis). The $x$ and $y$ coordinates are the pixel positions, with $x$ being the horizontal axis (rightward positive) and $y$ being the vertical axis (downward positive in image coordinates).

The object's orientation is defined by a rotation angle $\theta$ measured counterclockwise from the camera's $z$-axis (forward direction) to the object's forward direction. The orientation is sampled uniformly from four canonical directions:

$\theta \in \{90^\circ \text{ (left)}, 180^\circ \text{ (towards camera)}, 270^\circ \text{ (right)}, 0^\circ \text{ (away from camera)}\}$

Projection onto the ground plane. The paper makes an important simplifying assumption: vertical displacement (the $y$-coordinate) is ignored, and the problem is projected onto the $xz$-plane. The justification is that "y-coordinates do not reliably encode real-world height due to perspective projection"—an object higher in the image might be taller or might simply be farther away and above the ground plane. This assumption means the task is about horizontal spatial relations (left/right, front/back), not vertical relations (above/below).

The transformation equation. To compute the position of $\mathcal{R}_2$ in the object's coordinate frame, a 2D rigid transformation is applied to $(x_2, z_2)$ (ignoring $y_2$):

$\begin{bmatrix} \tilde{x}_2 \\ \tilde{z}_2 \\ 1 \end{bmatrix} = \underbrace{\begin{bmatrix} \cos\theta & \sin\theta & 0 \\ -\sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{bmatrix}}_{\text{Rotation}} \underbrace{\begin{bmatrix} 1 & 0 & -x_1 \\ 0 & 1 & -z_1 \\ 0 & 0 & 1 \end{bmatrix}}_{\text{Translation}} \begin{bmatrix} x_2 \\ z_2 \\ 1 \end{bmatrix}$

where $(\tilde{x}_2, \tilde{z}_2)$ are the coordinates of $\mathcal{R}_2$ in the object's local frame—$\tilde{x}_2$ is the rightward displacement (positive = to the object's right), and $\tilde{z}_2$ is the forward displacement (positive = in front of the object).

What this equation computes, step by step. The matrix multiplication is composed of two operations applied in sequence (the rightmost matrix first, then the leftmost):

1. Translation: $\begin{bmatrix} 1 & 0 & -x_1 \\ 0 & 1 & -z_1 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x_2 \\ z_2 \\ 1 \end{bmatrix} = \begin{bmatrix} x_2 - x_1 \\ z_2 - z_1 \\ 1 \end{bmatrix}$. This shifts the origin of the coordinate system from the camera to $\mathcal{R}_1$ (the object's position). After this step, the coordinates represent the displacement vector from the object to $\mathcal{R}_2$.

2. Rotation: $\begin{bmatrix} \cos\theta & \sin\theta \\ -\sin\theta & \cos\theta \end{bmatrix} \begin{bmatrix} x_2 - x_1 \\ z_2 - z_1 \end{bmatrix}$. This rotates the displacement vector by $-\theta$ (the negative sign in the upper-right and lower-left entries corresponds to rotating the coordinate axes by $+\theta$, which is equivalent to rotating the vector by $-\theta$). After this step, the coordinates are expressed in a frame where the positive $\tilde{z}$-axis points in the direction the object is facing, and the positive $\tilde{x}$-axis points to the object's right.

Determining the relative position. The directional labels are determined by thresholding the transformed coordinates:

$\tilde{p}_x = \begin{cases} \text{Right} & \text{if } \tilde{x}_2 > \delta_x \\ \text{Left} & \text{if } \tilde{x}_2 < -\delta_x \\ \text{None} & \text{otherwise} \end{cases}$

$\tilde{p}_z = \begin{cases} \text{Front} & \text{if } \tilde{z}_2 > \delta_z \\ \text{Back} & \text{if } \tilde{z}_2 < -\delta_z \\ \text{None} & \text{otherwise} \end{cases}$

where $\delta_x$ and $\delta_z$ are thresholds that enforce unambiguous spatial separation. The thresholds are defined in different units because the $x$-axis (parallel to image plane) and $z$-axis (perpendicular to image plane) have different scales:

• When the object's forward direction ($\tilde{z}$-axis) is parallel to the image plane ($\theta = 90^\circ$ or $270^\circ$), the transformed $x$-axis is perpendicular to the image plane (depth direction) and the transformed $z$-axis is parallel to the image plane. In this case: $\delta_x = 0.25$ (normalized depth units) and $\delta_z = 150$ pixels.
• When the object's forward direction is perpendicular to the image plane ($\theta = 0^\circ$ or $180^\circ$), the axes are swapped: $\delta_x = 150$ pixels and $\delta_z = 0.25$ (normalized depth).

What these thresholds compute. The 150-pixel threshold ensures that for lateral displacements in the image plane, the object is at least 150 pixels away before being labeled "Left" or "Right." The 0.25 normalized-depth threshold ensures that for displacements in depth, the object is at least 25% of the total depth range away before being labeled "Front" or "Back." These thresholds prevent borderline cases where the relative position is genuinely ambiguous.

Handling "None" cases. Instances where both $\tilde{p}_x = \text{None}$ and $\tilde{p}_z = \text{None}$ are discarded—these would be cases where the two positions are too close together to determine any clear spatial relationship. This ensures that all valid training samples have an unambiguous answer.

Ground truth. The correct answer is the tuple $(\tilde{p}_x, \tilde{p}_z)$, which can take values like "Left-Front," "Right-Back," "Right" (when only the $x$-direction is clear), or "Front" (when only the $z$-direction is clear). The task is posed as a multiple-choice question with four options, and the model must select the letter corresponding to the correct directional description.

Why this form over alternatives. The paper could have formulated this as a simpler task without coordinate transformation—e.g., "Which object is to the left of which in the image?" But image-plane "left" is not the same as object-centric "left." A person facing the camera has their left side on the right side of the image. The coordinate transformation is what makes this task a test of mental rotation and perspective-taking rather than simple 2D spatial relation recognition.

The decision to use four canonical orientations (0°, 90°, 180°, 270°) rather than continuous orientations is a pragmatic simplification that (a) makes answer options discrete and enumerable, (b) ensures the transformation is simple enough that the ground truth is unambiguous, and (c) still covers the key cases that arise in real spatial reasoning (facing toward/away/left/right relative to the camera).

The choice to project onto the $xz$-plane and ignore vertical displacement is both a simplification and a recognition of a fundamental ambiguity: in a single RGB-D image, the $y$-coordinate conflates height and depth due to perspective projection. An object higher in the image could be taller or farther away. By focusing on horizontal spatial relations (left/right, front/back), the task avoids this ambiguity while still requiring sophisticated 3D reasoning.

Question template (one variant): "I've taken an image and there are two regions marked as 1, and 2 on the image. Assume that there is a camera at position '1' and it's facing to me. According to the camera, where is the region marked '2'? A. Right-Front B. Right-Back C. Left-Front D. Left-Back. Consider cues such as depth, orientation, and 3D spatial relationship and ignore the height difference between the two regions. The final answer should be chosen from 'A', 'B', 'C', and 'D'."

---

Dataset Construction: Spatial-SSRL-81k

Data sources. The paper uses three public datasets, chosen for their diversity of real-world imagery and (for two of them) availability of sensor depth:

• COCO (Common Objects in Context): Provides raw RGB images spanning diverse indoor and outdoor scenes, object categories, and viewpoints. Used for depth-free tasks only, since COCO does not include depth maps.
• DIODE (Dense Indoor and Outdoor Depth): Provides RGB-D images (RGB + sensor depth maps) from both indoor and outdoor environments. Used for depth-based tasks.
• MegaDepth: Provides RGB-D images from internet photos with reconstructed depth maps using structure-from-motion. Used for depth-based tasks.

Critically, the paper emphasizes that "we use only the raw images and depth maps (where available), discarding all human-provided annotations such as bounding boxes, segmentations, or captions." This is a deliberate methodological choice that operationalizes the "zero human supervision" principle. The datasets are used purely as sources of pixels and (for DIODE and MegaDepth) sensor depth—the fact that COCO has object annotations is irrelevant because those annotations are never accessed.

Dataset composition. The Spatial-SSRL-81k dataset contains 81,053 question-answer pairs, balanced across the five tasks. The paper provides exact counts in Appendix Table 5:

The depth-free and depth-based categories are roughly balanced: 40,233 depth-free samples vs. 40,820 depth-based samples. Within depth-free, the Shuffled Patch Reordering and Flipped Patch Recognition tasks are treated as one combined task and evenly mixed with the other three, each contributing approximately 20k samples. The paper notes this grouping is because the two tasks "share similar image layouts and structures" (Appendix A.1).

Splits within Shuffled Patch Reordering. Appendix Table 6 provides the breakdown of the 16,028 Shuffled Patch Reordering samples by patchification strategy:

This shows that the $2 \times 2$ grid is the primary configuration (~8k samples, half with masking), with strip-based variants providing additional diversity (~8k samples combined).

100% ground-truth accuracy. The paper makes the strong claim that "Spatial-SSRL-81k achieves 100% ground-truth accuracy, which is unattainable by prior pipelines that rely on noisy detections or model-generated annotations." This is not an empirical measurement but a logical consequence of the task construction: for every sample, the ground truth is deterministically computed from the known transformation, so by construction there are no errors. This contrasts with pipelines that use monocular depth estimators (which can produce incorrect depth orderings), object detectors (which can miss or mislocalize objects), or LLM-based verifiers (which can make reasoning errors). The 100% accuracy claim is important for the RL training: noisy rewards would create conflicting optimization signals, potentially causing the model to converge to suboptimal behaviors or fail to learn entirely.

---

RL Training with Verifiable Rewards

The training procedure converts the self-supervised QA pairs into behavioral optimization through a two-stage process: a brief supervised fine-tuning cold-start, followed by Group Relative Policy Optimization (GRPO). I will explain each stage, the reward function, and the format prompting strategy in detail.

Why two stages? The paper reports that "directly applying RL from a pretrained checkpoint leads to training instability and reward collapse, as the base model fails to generate valid formatted responses (success rate < 5% for complex tasks like relative position prediction)." This is a critical practical finding. The base LVLM (Qwen2.5-VL) has never been trained to produce output in the specific format expected by the verifier—it doesn't know to enclose reasoning in ⟨think⟩ tags or to put final answers in \boxed{}. If RL training begins without any format grounding, the model's initial responses are almost always unparseable, meaning the accuracy reward is always 0, meaning there is no meaningful gradient signal to guide learning. The RL optimization collapses because all responses look equally bad to the reward function.

Cold-Start Supervised Fine-Tuning. To address this, the model is first fine-tuned on a small subset of approximately 3,600 samples (~4.4% of the full 81k dataset) using standard supervised learning (next-token prediction on the correct answer). The training runs for 5 epochs with a learning rate of $1 \times 10^{-5}$. The purpose is explicitly not to teach spatial reasoning—3,600 samples is far too few for that—but to teach the model the desired output format and the basic structure of each task type. After this cold-start, the model can reliably produce parseable responses (>95% format compliance), giving the RL phase a meaningful accuracy signal to optimize.

GRPO Configuration. Following cold-start, the model is optimized using Group Relative Policy Optimization on the full Spatial-SSRL-81k dataset. The key hyperparameters are:

• Rollout group size: 5 responses per training sample. For each question, the model generates 5 candidate answers (sampled at temperature 1.0), and the reward is computed for each. GRPO then updates the model to increase the probability of the above-average responses and decrease the probability of below-average responses, relative to the group mean. This group-relative baseline is what distinguishes GRPO from standard policy gradient methods—it doesn't need a learned value function (critic) because the group average serves as the baseline.
• KL regularization weight: 0.01. A KL divergence penalty between the current policy and a frozen reference model (the SFT cold-start checkpoint) is added to the objective. This prevents the model from diverging too far from its initial behavior, which is important for preserving general visual capabilities and preventing reward hacking.
• Global batch size: 128. This is the number of distinct training samples processed per optimization step, each generating 5 rollouts, for a total of 640 responses per step.
• Learning rate: $1 \times 10^{-6}$. An order of magnitude lower than the SFT learning rate, which is typical for RL fine-tuning where the goal is refinement rather than learning from scratch.
• Training steps: 360. At a batch size of 128, this corresponds to approximately $360 \times 128 = 46,080$ samples processed, which is about 57% of the full 81k dataset—the model sees slightly more than half the dataset during RL training (each sample may be seen multiple times across steps due to the global batch size being smaller than the dataset).

Reward Function Design. The GRPO reward has two binary components, combined with a weighted sum:

$r = 0.9 \cdot r_{\text{acc}} + 0.1 \cdot r_{\text{fmt}}$

where $r_{\text{acc}} \in \{0, 1\}$ is the accuracy reward and $r_{\text{fmt}} \in \{0, 1\}$ is the format reward.

The accuracy reward ($r_{\text{acc}}$): Set to 1 if and only if the model's predicted answer exactly matches the ground-truth answer (derived from the self-supervised task construction), and 0 otherwise. "Exact match" means the extracted answer string—parsed from the model's output using the \boxed{} delimiter—is character-for-character identical to the expected answer. For multiple-choice tasks, this is a single letter ('A', 'B', 'C', or 'D'). For ordering tasks, this is a dash-separated sequence ("2-1-3"). For the flipped patch task, this is a dash-separated tuple ("3-0"). The parsing is reliable because the format prompt enforces that the answer appears in a specific location.

What this reward computes. It is a sparse binary signal: the model either gets the spatial reasoning exactly right or it gets no credit at all. There is no partial credit for getting the general idea but the wrong specific answer, no continuous reward for being "close." This is a deliberate choice: continuous rewards (e.g., based on how close the predicted patch order is to the correct order) would require defining a distance metric over answer spaces, which is non-trivial for tasks like ordering (is "2-1-3" closer to "2-3-1" than to "1-2-3"?). Binary exact-match rewards are simpler, more robust, and align with the deterministic nature of the self-supervised ground truth.

The format reward ($r_{\text{fmt}}$): Set to 1 if the model's output strictly adheres to the prescribed format—reasoning enclosed in ⟨think⟩...⟨/think⟩ tags, final answer in \boxed{}—and 0 otherwise. This is a structural check, not a content check; it doesn't matter whether the reasoning is coherent, only whether the delimiters are present and correctly placed.

Why the 0.9/0.1 weighting. The paper explains the rationale: "format compliance typically stabilizes quickly after cold-start (>95% compliance), whereas accuracy improvements require the full RL phase." The 0.1 weight on format provides a small continuing incentive to maintain proper formatting, but the dominant optimization pressure comes from accuracy. If the weights were equal (0.5/0.5), the model might prioritize format over content—getting the delimiters right but the answer wrong would still yield 0.5 reward, which could create a local optimum that's hard to escape. The heavily skewed weighting ensures that the RL optimization is primarily driven by spatial reasoning accuracy, with format as a secondary, stabilizing objective.

Why binary rewards over continuous rewards. The paper could have designed continuous rewards—for example, in Shuffled Patch Reordering, giving partial credit proportional to the number of patches in the correct position. The choice of binary rewards reflects the RLVR philosophy: verifiable rewards should be deterministic and unambiguous. Continuous rewards would introduce hyperparameters (how much credit per correct patch?) and could create reward hacking opportunities (the model might learn to place a few easy patches correctly and ignore the hard ones, maximizing a continuous reward with less effort than full correctness). Binary rewards create a cleaner optimization landscape: the global optimum is unambiguous (all answers correct), and there are no local optima created by partial-credit reward shaping.

Format Prompt for Training. Each question during GRPO training is appended with the following instruction:

"You FIRST think about the reasoning process as an internal monologue and then provide the final answer. The reasoning process MUST BE enclosed within ⟨think⟩⟨/think⟩ tags. The final answer MUST BE put in \boxed{}."

This prompt serves three purposes: (1) It elicits step-by-step reasoning, which the paper hypothesizes is necessary for complex spatial tasks—the model should not just output an answer but should reason through the spatial relationships. (2) It provides a reliable mechanism for answer extraction—the verifier can parse the \boxed{} content to get the predicted answer without ambiguity. (3) It creates consistency between training and inference—at evaluation time, the same format prompt is appended, so the model produces structured reasoning that can be examined and the answer can be reliably extracted.

Why this specific format. The ⟨think⟩...⟨/think⟩ + \boxed{} format is adapted from the DeepSeek-R1 reasoning paradigm, which has become a de facto standard for training models to produce explicit reasoning traces. The \boxed{} command is borrowed from LaTeX math formatting and provides an unambiguous delimiter that is unlikely to appear in natural reasoning text. This format has been widely used in mathematical reasoning RL (where answers are boxed mathematical expressions) and transfers naturally to spatial reasoning where answers are discrete labels or sequences.

---

Summary of Design Choices and Their Justifications

• Self-supervised task construction over human annotation or tool pipelines: Eliminates annotation cost, tool-chain errors, and domain limitations. The 100% ground-truth accuracy is guaranteed by construction rather than achieved through quality control.

• Five diverse tasks over a single task type: The ablation studies (Table 4) empirically validate that different tasks improve different aspects of spatial reasoning (e.g., Flip excels at maze reasoning, Relative Position at general VQA). Task diversity provides complementary learning signals and mutual regularization.

• Depth-free and depth-based task categories: Depth-free tasks leverage the abundance of RGB-only data and teach 2D layout understanding, which the paper shows transfers partially to 3D reasoning. Depth-based tasks provide explicit 3D supervision through sensor depth (not estimated depth), targeting capabilities that 2D tasks alone cannot teach well (e.g., ordinal depth perception).

• SFT cold-start before RL: Necessary because base models cannot produce parseable responses for complex task formats. Without cold-start, RL collapses due to zero reward signal. The cold-start teaches format, not spatial reasoning, leaving the RL phase to drive accuracy improvements.

• GRPO over other RL algorithms: GRPO is well-suited for verifiable reward settings because it uses group-relative advantages (no need for a learned critic), handles discrete reward signals effectively, and includes KL regularization to prevent catastrophic forgetting.

• Binary accuracy reward over continuous reward: Simpler, more robust, avoids reward shaping hyperparameters, and creates an unambiguous optimization target (exact correctness). The deterministic ground truth from self-supervision makes binary rewards appropriate—there is a single unambiguously correct answer.

• 0.9/0.1 accuracy-to-format reward weighting: Reflects the finding that format compliance is quickly learned and maintained, while accuracy is the primary optimization objective. The heavily skewed weighting prevents the model from prioritizing format over content.

• ⟨think⟩...⟨/think⟩ + \boxed{} output format: Provides reliable answer extraction for the verifier, elicits step-by-step reasoning that improves spatial reasoning quality, and ensures consistency between training and evaluation.

4. Key Insights and Innovations

Innovation 1: Self-Supervised Pretext Tasks as a Verifiable Reward Curriculum for Spatial RL

The paper's foundational conceptual move is recognizing that the properties that made self-supervised learning effective for visual encoder pre-training—deterministic ground truth, unlimited scalability, no annotation cost—are exactly the properties needed for RLVR-based post-training of LVLMs. This is not an incremental application of SSL to a new domain; it is a category shift in what SSL signals are used for. For decades, SSL pretext tasks (jigsaw puzzles, rotation prediction, contrastive alignment) were treated as representation learning objectives—the goal was to train an encoder that produced good features, with the pretext task discarded after pre-training. The Spatial-SSRL contribution is to repurpose these same signals as behavioral optimization rewards during post-training, shifting the objective from "learn good features" to "learn good spatial reasoning behaviors."

Prior work in SSL for LVLMs largely followed the pre-training paradigm: masked autoencoders for vision transformers, contrastive vision-language pre-training, masked language modeling. Even recent work applying SSL to LVLM post-training (Jigsaw-R1, Visual Jigsaw, SSL4RL) largely treated specific pretext tasks as isolated training interventions—use jigsaw puzzles to improve visual reasoning, see if it helps. What distinguishes Spatial-SSRL is the systematic curriculum design: five complementary tasks explicitly chosen to cover distinct spatial reasoning capabilities (2D layout, orientation sensitivity, texture-context matching, ordinal depth, egocentric coordinate transformation), with the explicit hypothesis that diversity across tasks matters more than optimization of any single task. The ablation results in Table 4 empirically validate this framing: no single task dominates across evaluation dimensions, and the combined model achieves results that exceed any subset. This is not just "SSL tasks are useful"—it is "SSL tasks form a curriculum whose coverage determines downstream transfer."

The significance extends beyond spatial understanding. The paper establishes a template for how RLVR can be applied to any perceptual capability where deterministic transformations on raw data can produce verifiable ground truth. The three design principles (zero human/LLM supervision, tool-free scalability, natural verifiability) define a paradigm rather than a method: any researcher wanting to improve a specific perceptual capability in an LVLM can ask "what deterministic transformations on raw sensor data produce unambiguous ground truth for this capability?" and construct a task suite following the same pattern. This is a fundamentally different approach to training data generation than the dominant paradigm of using human annotators, proprietary LLMs, or tool chains.

The paper also implicitly reframes the "RL vs. SFT" debate in post-training. The SFT cold-start finding—that direct RL from a pretrained checkpoint collapses because the model cannot produce parseable responses—is not a failure of RL but a diagnostic about the relationship between task format complexity and RL feasibility. It suggests that for complex output formats, a small amount of supervised format-teaching is necessary before RL can optimize content. This is a practical insight that generalizes beyond spatial reasoning to any RLVR application where the target output format differs substantially from the base model's training distribution.

---

Innovation 2: Intrinsic Supervision as a Scalability Principle

The paper's most distinctive framing is the intrinsic vs. extrinsic supervision dichotomy (Figure 2). While the individual elements (self-supervision, RLVR, spatial understanding benchmarks) existed before, the explicit articulation of intrinsic supervision as a design principle that enables a qualitative shift in scalability is a conceptual contribution independent of the specific tasks or model architecture.

Prior work operated entirely in the extrinsic supervision paradigm: spatial understanding training data came from humans (expensive, slow), proprietary models (costly, non-reproducible), specialized tools (complex pipelines, error-prone), or synthetic environments (domain gap). Each extrinsic source imposed a bottleneck—annotation budget, API costs, tool-chain throughput, simulation fidelity—that limited how much training data could be generated and how diverse its domain coverage could be. The field's implicit assumption was that spatial understanding requires spatial annotations—someone or something must explicitly label depth relationships, object orientations, or relative positions.

Spatial-SSRL demonstrates that this assumption is false for a broad class of spatial reasoning capabilities. Intrinsic signals—pixel-level consistency, geometric transformations, sensor depth comparisons—encode spatial information that can be "revealed" through task construction rather than "injected" through annotation. The conceptual move is from "we need to tell the model what the spatial relationships are" to "we need to construct tasks where the correct answer is deterministically derivable from the data and requires spatial reasoning to find." The model learns spatial reasoning not because it is told the answers by an annotator, but because the tasks are constructed such that getting the right answer is impossible without performing spatial reasoning.

This reframing has implications that extend beyond the paper's immediate results. It suggests that many perceptual capabilities currently addressed through expensive annotation pipelines—temporal reasoning, causal understanding, physical intuition—might be addressable through intrinsic supervision if appropriate verifiable transformations can be designed. The limitation is not the availability of annotated data but the creativity of task designers in finding deterministic transformations that create unambiguous training signals. The paper's contribution is as much a proof of concept for a methodology as it is a specific set of spatial reasoning improvements.

The empirical support for this framing is the breadth of spatial benchmarks where Spatial-SSRL improves performance (Table 1): seven benchmarks spanning image and video modalities, covering distance estimation, occlusion reasoning, orientation understanding, and multi-view spatial localization. If the model were simply memorizing task-specific patterns from the self-supervised data, improvements would be limited to benchmark questions that closely resemble the training tasks. The cross-modal transfer to video understanding (VSI-Bench: +5.65% for 3B, +1.21% for 7B) despite training exclusively on static images is particularly compelling evidence that the model is learning generalizable spatial representations, not task-specific heuristics.

---

Innovation 3: Depth-Free Tasks Transfer Partially to 3D Reasoning

One of the paper's most surprising and theoretically significant findings is that depth-free tasks—operating solely on RGB images with no depth information—improve performance on 3D spatial reasoning benchmarks. The ablation results in Table 4 show that models trained only on 2D tasks (Crop, Shuffle, Flip) achieve 57.99% average accuracy on the three 3DSR subsets (Height, Location, MultiObj), compared to 61.45% for depth-based tasks alone and 60.63% for the combined model. The gap between depth-free and depth-based training (3.46 percentage points) is not negligible—explicit depth supervision helps—but the fact that 2D tasks provide any benefit at all for 3D reasoning is a significant finding with implications for how we understand spatial learning.

The dominant assumption in the spatial reasoning literature has been that 3D understanding requires 3D supervision: if you want a model to reason about depth, you need depth maps; if you want it to reason about viewpoint transformations, you need multi-view data or camera pose annotations. This assumption drove the tool-heavy pipelines of prior work (SpatialVLM, SpatialRGPT), which invested heavily in depth estimation, 3D reconstruction, and view synthesis to generate 3D-aware training data.

Spatial-SSRL's results challenge this assumption. The depth-free tasks—which involve no depth information whatsoever—nevertheless improve 3DSR-Height by 5.98 percentage points over baseline (58.99% vs. 52.61%) and 3DSR-MultiObj by 1.93 points. The mechanism is presumably that reasoning about 2D spatial layout and structural consistency develops visual representations that partially encode 3D structure—understanding how patches relate in 2D requires attending to perspective cues, relative size, occlusion boundaries, and other monocular depth cues that are present in ordinary RGB images. In effect, the 2D tasks teach the model to extract the 3D information that is already latent in 2D images, even without explicit depth supervision.

This finding has practical significance: RGB-only data is vastly more abundant than RGB-D data. COCO alone provides hundreds of thousands of images, while high-quality RGB-D datasets are orders of magnitude smaller and biased toward specific environments (indoor rooms for ScanNet, structured outdoor scenes for KITTI). The result that 2D self-supervision partially substitutes for 3D supervision means that spatial reasoning can be improved at much larger scale and across much broader domain diversity than would be possible with depth-sensor data alone. The 3D-specific tasks (Regional Depth Ordering, Relative Position Prediction) provide additional gains on top of the 2D foundation, but the 2D tasks do much of the heavy lifting.

The complementarity between 2D and 3D supervision is itself an important finding. The combined model (all five tasks) achieves the best performance on four of seven evaluation dimensions, with the depth-free and depth-based categories showing different patterns of strength: depth-free tasks excel at general VQA and maze reasoning, while depth-based tasks excel at 3D height estimation and location reasoning. This suggests that 2D and 3D spatial reasoning are partially dissociable capabilities that both contribute to overall spatial intelligence, and that comprehensive training should target both.

---

Innovation 4: GRPO as a Mechanism for Converting Deterministic Self-Supervision into Behavioral Improvement

While the paper's primary contribution is the self-supervised task paradigm, its implementation strategy—using GRPO with binary exact-match rewards on self-supervised QA pairs—represents a methodological insight about how to optimize LVLMs on tasks where the ground truth is deterministic but the reasoning process is complex. This is not an innovation in RL algorithms (GRPO existed prior), but an innovation in how to connect self-supervised task design to RL optimization in a way that produces meaningful behavioral improvements rather than format exploitation or reward hacking.

The key challenge in applying RLVR to perceptual tasks is that the model must learn to produce correct answers through intermediate reasoning steps, but the reward signal only evaluates the final answer. This is the classic credit assignment problem: if the model gets a binary reward of 0 or 1, how does it learn which parts of its reasoning process contributed to the correct answer? The standard approach in mathematical reasoning RLVR (DeepSeek-R1, etc.) relies on the fact that reasoning steps in math are relatively structured—equations, algebraic manipulations, logical deductions—and the model can learn to associate certain reasoning patterns with correct answers through many trials.

Spatial-SSRL demonstrates that the same approach works for spatial reasoning, which is arguably more challenging because spatial reasoning is less structured than mathematical reasoning—there are no standardized "spatial reasoning operators" analogous to algebraic rules. The model must learn to identify relevant visual features, reason about their spatial relationships, and express this reasoning in natural language, all from a binary final-answer reward. The fact that this works (yielding consistent improvements across seven benchmarks without degradation on general VQA) suggests that GRPO's group-relative advantage mechanism provides sufficient signal for the model to discover productive spatial reasoning strategies, even when the credit assignment problem is severe.

The design decision to use a 0.9/0.1 accuracy-to-format reward weighting—rather than equal weights, or a format-only phase followed by accuracy-only—reflects an understanding of how format and content learning interact in RL. Format compliance is quickly learned (stabilizing after cold-start) but must be maintained; accuracy improvement is slow and requires sustained optimization pressure. The heavily skewed weighting prevents the format reward from creating a "good enough" local optimum where the model produces correctly formatted but content-poor responses. This weighting strategy is a practical insight that likely transfers to other RLVR applications where output format and answer quality are separate concerns.

The cold-start SFT finding also has methodological significance. The paper quantifies the format compliance problem (success rate < 5% for complex tasks without cold-start) and demonstrates a minimal-intervention solution (SFT on ~4.4% of the data, 5 epochs) that enables RL to proceed. This establishes a template for RLVR applications where the base model's output distribution is too far from the target format for direct RL to work: use a small amount of supervised data to teach format, then use RL on the full dataset to optimize content. The separation of format learning (SFT) from content learning (RL) is a design pattern that could be applied broadly.

Evidence anchors: The difficulty-conditioned scaling patterns (Table 1: improvements most pronounced on 3D reasoning benchmarks like Spatial457 where baseline models show particular weakness), the ablation study showing combined task benefits (Table 4), and the general VQA preservation (Table 3: +2.02% for 3B, +0.57% for 7B) collectively support the claim that GRPO on self-supervised tasks produces genuine spatial reasoning improvements rather than superficial pattern matching or benchmark overfitting. The video transfer results (VSI-Bench) provide the strongest evidence of generalizable learning, since the model was never trained on video data.

5. Experimental Analysis

Evaluation Methodology

• Datasets and Benchmarks. The paper evaluates on seven spatial understanding benchmarks spanning image and video modalities. Image benchmarks include Spatial457 [60] (6D spatial reasoning requiring pose estimation and multi-step inference), 3DSRBench [44] (comprehensive 3D spatial reasoning with subtasks for height estimation, location reasoning, and multi-object relationships), SpatialEval [58] (spatial reasoning with maze and 2D layout tasks), QSpatial-plus [36] (quantitative 3D distance prediction), What'sUp [28] (unambiguous 2D spatial relation recognition between object pairs), and ViewSpatial [32] (multi-perspective spatial localization requiring 3D reconstruction and perspective transformation). The video benchmark is VSI-Bench [68] (egocentric video spatial understanding with both multiple-choice and numerical answer formats). All evaluation uses VLMEvalKit [15] where supported, with custom adaptations for What'sUp, ViewSpatial, and VSI-Bench to accommodate the Qwen2.5-VL architecture while preserving original metrics and evaluation procedures (Appendix B).

• Base Models. The primary experiments use Qwen2.5-VL-3B and Qwen2.5-VL-7B [1] as base models. The authors argue these are "representative of the capabilities of many contemporary LLMs" (Section 4.1) and sit in a useful performance regime—non-trivial but far-from-saturated spatial reasoning accuracy—where training interventions can demonstrate meaningful improvement. An additional experiment uses Qwen3-VL-4B-Instruct as the base model (Table 2, Appendix B.3) to validate cross-architecture transfer.

• Metrics. The primary metric throughout is accuracy, computed as the percentage of benchmark questions where the model's predicted answer exactly matches the ground truth. For multiple-choice questions, this is exact match of the option letter. For numerical answers in VSI-Bench, the metric is Mean Relative Accuracy (MRA), defined as $\text{MRA} = \mathbb{E}_{x \in C}[ \mathbb{1}( |\hat{y} - y| / y < x) ]$ where $C = \{0.05, 0.10, \ldots, 0.5\}$ (Section 4.1, Appendix B.1). For QSpatial-plus (quantitative distance prediction), the benchmark's custom scoring function requiring both scalar value and distance unit is used. Performance on individual benchmarks is reported, with an unweighted average across all seven benchmarks also provided. For ablation studies (Table 4), specific benchmark subsets are isolated: 3DSR-Height, 3DSR-Location, 3DSR-MultiObj (3D spatial reasoning), Spa457-2D and SpaEval-Maze (2D spatial reasoning), Spa457-Pose (pose estimation), and Gnr-VQA (average of four general VQA benchmarks).

• Baselines. The primary baseline is the base Qwen2.5-VL model evaluated without reasoning prompts (standard direct-answer inference). This is the fair comparison because it represents the model's default behavior. The paper also evaluates a stronger baseline: Qwen2.5-VL with Chain-of-Thought prompting (denoted with "!" in Table 1, "Reasoning" column), where the model is instructed to generate explicit reasoning chains. This baseline is critical because it tests whether simply prompting for reasoning—without any spatial training—improves performance. Additional baselines include two representative spatial models from prior work evaluated on the same benchmarks: SpatialLadder-3B [33] (progressive RL training on ScanNet 3D data) and SpaceR-7B [48] (RLVR on annotation-rich video spatial data). Both serve as reference points for what prior RL-based spatial training methods achieve, though neither is a direct comparison since they use different base models and training data.

• Generation Budget / Compute Accounting. The RL training uses a fixed compute budget of 360 GRPO steps with batch size 128 and rollout group size 5 (640 total responses per step, approximately 46,080 samples processed over the full run, covering ~57% of the 81k dataset). The SFT cold-start uses 5 epochs on ~3,600 samples. There is no variable generation budget at inference time—all models are evaluated with a single response per question (greedy or sampled decoding, as appropriate for the benchmark). The paper does not compare different inference-time compute budgets (no best-of-N, no beam search, no revision chains), so "compute" in this work is purely a training-time concept, not an inference-time one.

• Cross-Validation / Statistical Protocol. The paper uses no cross-validation or statistical significance testing. All results are single-run evaluations on the full test sets of the respective benchmarks. For the ablation study (Table 4), each training configuration is a single training run with the specified task subset. The paper reports raw accuracy percentages without confidence intervals, standard deviations, or error bars. For benchmarks using VLMEvalKit, the standardized evaluation protocol provides consistency across runs, and the reported numbers for baseline models match their published values. However, given the small size of some benchmark-specific subsets used in ablations (e.g., SpaEval-Maze: 35.13% baseline accuracy on what is likely a small number of maze-specific questions from SpatialEval), statistical reliability at the per-subset level is uncertain.

---

Main Quantitative Results

Overall Spatial Understanding Improvements

The headline result from Table 1 is that Spatial-SSRL consistently improves spatial understanding across all seven benchmarks for both 3B and 7B model scales, with average gains of +4.63% (3B) and +3.89% (7B) over the Qwen2.5-VL baselines without reasoning. The 3B model improves from 45.91% average accuracy to 50.54%; the 7B model improves from 52.69% to 56.58%.

The largest individual improvement is +12.37% on Spatial457 for the 3B model (33.70% → 46.07%). Spatial457 is described as requiring "complex pose estimation and multi-step spatial reasoning" (Section 4.2.1), making it the most demanding benchmark in the suite. The 7B model shows a similarly strong improvement of +8.67% on the same benchmark (44.67% → 53.34%). Other notable per-benchmark gains for the 7B model include +7.93% on QSpatial-plus (46.53% → 54.46%) and +3.66% on What'sUp (86.95% → 90.61%).

The improvement distribution reveals a pattern: the largest gains appear on benchmarks requiring 3D spatial reasoning (Spatial457, 3DSRBench, QSpatial-plus), while gains on 2D-centric benchmarks are more modest: +0.98% on ViewSpatial (36.83% → 37.81% for 7B), +1.24% on ViewSpatial for 3B (35.38% → 36.62%). The authors interpret this as evidence that "both depth-free and depth-based pretext tasks contribute to 3D understanding" (Section 4.2.1 analysis), though the magnitude difference suggests depth-based tasks provide stronger 3D-specific supervision.

Cross-modal generalization is demonstrated by VSI-Bench improvements: +5.65% for 3B (27.84% → 33.49%) and +1.21% for 7B (38.08% → 39.29%). Since training uses only static images while VSI-Bench evaluates on egocentric videos, this transfer is evidence that the learned spatial representations are modality-agnostic. The larger gain for 3B suggests that smaller models benefit more from the additional spatial training—they have more room for improvement—while 7B models start closer to a performance ceiling on this benchmark.

Reasoning Capability Enhancement: The Critical Baseline Comparison

A crucial diagnostic result in Table 1 is the performance of Qwen2.5-VL with reasoning prompts (denoted with "!"). If simply prompting the model to "think step-by-step" improved spatial reasoning, then the Spatial-SSRL gains would be less significant—they would represent better prompt engineering rather than genuine spatial learning. The data show the opposite: reasoning prompts often degrade baseline performance.

The 3B model drops from 45.91% to 44.85% average with reasoning enabled—a -1.06% degradation. The 7B model shows more dramatic drops on specific benchmarks: What'sUp collapses from 86.95% to 70.61% (-16.34 percentage points) with reasoning, and VSI-Bench drops from 38.08% to 32.69% (-5.39 points). Some benchmarks show modest improvements with reasoning (Spatial457: 44.67% → 44.88%, a negligible +0.21 points; 3DSRBench: 53.39% → 54.55%, +1.16 points), but the overall pattern is that reasoning prompts do not reliably improve spatial understanding in untrained models, and on some tasks they actively harm performance.

The authors attribute this to the baseline models lacking "effective spatial reasoning"—their generated reasoning steps "introduce noise rather than providing useful inference" (Section 4.2.1). This interpretation is consistent with prior findings in spatial reasoning (cited as [64, 74]) and with the broader observation that Chain-of-Thought prompting helps primarily on tasks where the model already possesses relevant reasoning capabilities; it does not create capabilities from nothing. The Spatial-SSRL models, in contrast, show consistent improvements with reasoning compared to baselines without reasoning, indicating that the training has taught the model to generate productive spatial reasoning chains rather than noise.

This result is methodologically important because it rules out an alternative explanation for Spatial-SSRL's gains: that they come solely from the reasoning format prompt rather than the self-supervised training. If that were the case, the baseline + reasoning configuration would show similar improvements. Instead, baseline + reasoning often performs worse than baseline without reasoning, while Spatial-SSRL + reasoning consistently outperforms both.

Comparison with Prior Spatial RL Methods

Table 1 reports two external baselines evaluated on the same benchmarks: SpatialLadder-3B (45.09% average) and SpaceR-7B (54.54% average). These serve as reference points for what alternative RL-based spatial training methods achieve, though the comparison is imperfect because different base models, training data, and training procedures are used.

Spatial-SSRL-3B (50.54%) outperforms SpatialLadder-3B (45.09%) by +5.45 percentage points on average, with the most striking difference on Spatial457 (46.07% vs. 40.19%, +5.88 points). Spatial-SSRL-7B (56.58%) outperforms SpaceR-7B (54.54%) by +2.04 points on average, with large margins on QSpatial-plus (54.46% vs. 48.51%, +5.95 points) and Spatial457 (53.34% vs. 53.66%, essentially tied with a -0.32 point difference). The advantage over SpatialLadder is consistent across all seven benchmarks; the advantage over SpaceR is more mixed—SpaceR slightly outperforms on 3DSRBench (53.91% vs. 56.53%, -2.62 points for Spatial-SSRL) but this is within the range of benchmark-to-benchmark variation.

These comparisons should be interpreted cautiously: SpatialLadder trains on ScanNet (indoor 3D scans) while Spatial-SSRL trains on general RGB/RGB-D images from COCO, DIODE, and MegaDepth—the domain coverage is radically different. SpaceR uses video data while Spatial-SSRL uses only static images. The comparison demonstrates that Spatial-SSRL's self-supervised paradigm is competitive with or superior to prior RL-based spatial methods despite using substantially cheaper and more scalable training data (no human annotations, no specialized 3D scans, no video curation). However, the comparison does not isolate the effect of the self-supervised task design from the effect of the base model quality or training scale.

Cross-Architecture Transfer: Qwen3-VL Results

Table 2 and Appendix Table 9 demonstrate that Spatial-SSRL transfers to the Qwen3-VL architecture. Spatial-SSRL-4B (trained from Qwen3-VL-4B-Instruct) achieves 61.43% average spatial understanding accuracy, a +1.29% improvement over the Qwen3-VL-4B baseline (60.14% without reasoning). This is a smaller gain than observed on Qwen2.5-VL-3B (+4.63%) and -7B (+3.89%), which the authors attribute to the stronger baseline performance of Qwen3-VL-4B (60.14% vs. 45.91% for Qwen2.5-VL-3B)—there is less room for improvement when the base model is already substantially better at spatial reasoning.

The improvement pattern on Qwen3-VL differs from Qwen2.5-VL. The reasoning-prompted Qwen3-VL-4B baseline achieves 60.23%, nearly identical to the non-reasoning baseline (60.14%), suggesting that Qwen3-VL already possesses spatial reasoning capabilities that are not disrupted by explicit reasoning prompts—a significant architectural difference from Qwen2.5-VL where reasoning prompts often hurt. The Spatial-SSRL-4B gains are concentrated in 3DSRBench (55.83% → 59.48%, +3.65 points) and VSI-Bench (38.82% → 42.13%, +3.31 points), with smaller gains or minor regressions on other benchmarks (QSpatial-plus: 61.39% → 59.41%, -1.98 points). The regression on QSpatial-plus is concerning—it suggests that for some task types on Qwen3-VL, the self-supervised training may interfere with existing capabilities, though the magnitude is small enough to potentially reflect evaluation noise.

General Visual Capabilities: Preservation and Improvement

Table 3 evaluates whether the spatial-focused training degrades general visual understanding—a critical concern for any specialized fine-tuning approach. The results show no degradation; instead, general capabilities improve.

On General VQA benchmarks (MMBench1.1, BLINK, HallusionBench, RealWorldQA), evaluated using standard prompts without reasoning instructions: Spatial-SSRL-3B achieves +2.02% average improvement (59.37% → 61.39%), with gains on all four individual benchmarks. The largest improvements are on HallusionBench (+3.63%, from 46.03% to 49.66%) and BLINK (+1.95%, from 48.97% to 50.92%), both of which test hallucination recognition and fine-grained visual perception respectively. Spatial-SSRL-7B shows a more modest +0.57% average improvement (64.75% → 65.32%), with consistent small gains on all four benchmarks: MMBench1.1 (+0.37%), BLINK (+0.36%), HallusionBench (+0.92%), RealWorldQA (+0.65%).

On OCR and Chart Understanding benchmarks (OCRBench, ChartQA, SeedBench2-plus), evaluated with reasoning prompts for consistency with training: Spatial-SSRL-3B shows a negligible +0.12% average change (78.46% → 78.58%), with a small gain on OCRBench (+1.5%) offset by a small loss on ChartQA (-1.0%). Spatial-SSRL-7B shows a more meaningful +1.22% average improvement (81.96% → 83.18%), with gains on all three benchmarks: OCRBench (+1.1%), ChartQA (+1.64%), SeedBench2-plus (+0.92%).

The improvement on general VQA is noteworthy: the self-supervised spatial tasks require holistic image understanding (reasoning about patch relationships, structural coherence, texture continuity), and the paper hypothesizes that these skills "transfer to general scene comprehension" (Section 4.2.2). The OCR/chart improvements are attributed to several pretext tasks—particularly cropped patch inpainting and flipped patch recognition—requiring "attention to fine-grained visual details, texture continuity, and local structural consistency, which align with the demands of OCR and chart understanding" (Section 4.2.2).

For the Qwen3-VL-4B model (Appendix Table 10), Spatial-SSRL preserves and improves general VQA: +1.18% average improvement (69.10% → 70.28%), with the largest gain on HallusionBench (+5.85%, from 56.38% to 62.23%) and a minor regression on RealWorldQA (-1.70%, from 71.11% to 69.41%).

The collective general capabilities results address a common failure mode of specialized fine-tuning: the model improves on the target capability at the expense of broader competence. The paper provides evidence that self-supervised spatial training does not cause this tradeoff, and in fact provides net benefits to general visual understanding. The modest magnitude of the general improvements (+0.57% to +2.02%) is consistent with the interpretation that the spatial training provides complementary skills that transfer partially to general visual tasks, rather than representing a fundamental improvement in general vision.

---

Ablation Studies and Robustness Checks

Table 4 presents the paper's central ablation study, training Qwen2.5-VL-7B on different subsets of the five self-supervised tasks and evaluating on specific benchmark subsets designed to isolate different spatial reasoning capabilities. The seven evaluation dimensions are: Gnr-VQA (average of four general VQA benchmarks), Spa457-2D (2D spatial from Spatial457), SpaEval-Maze (maze reasoning), Spa457-Pose (pose estimation), 3DSR-Height (3D height estimation), 3DSR-Location (3D location reasoning), and 3DSR-MultiObj (multi-object 3D relationships).

All task configurations improve over baseline: The baseline model (no training) achieves 64.75% on Gnr-VQA, 56.52% on Spa457-2D, 35.13% on SpaEval-Maze, 33.92% on Spa457-Pose, 52.61% on 3DSR-Height, 67.84% on 3DSR-Location, and 46.70% on 3DSR-MultiObj. Every training configuration (single-task, category-combined, or full five-task) improves over baseline on nearly every dimension, confirming that each pretext task provides useful learning signals.

No single task dominates: Among single-task models, Cropped Patch Inpainting ("Crop") achieves the highest Gnr-VQA (65.39%) and strong Spa457-Pose (40.40%). Flipped Patch Recognition ("Flip") achieves the highest SpaEval-Maze (43.60%, the highest of any configuration including the full model's 42.53%) and strong 3DSR-MultiObj (48.63%, competitive with depth-based tasks). Shuffled Patch Reordering achieves strong Spa457-Pose (40.61%) and competitive 3DSR-Height (58.48%). The pattern confirms that different tasks benefit different reasoning capabilities, with Flip's maze reasoning advantage and Crop's balanced performance being notable individual contributions.

Depth-based tasks excel at 3D reasoning: Regional Depth Ordering ("Depth") achieves the strongest 3DSR-Height (63.48%) of any single-task model, validating its direct supervision of ordinal depth perception. Relative Position Prediction ("Pos.") achieves the best Gnr-VQA (65.64%) and strong 3DSR-Location (72.20%) and 3DSR-MultiObj (48.85%). Combining both depth-based tasks achieves the best overall 3DSR-MultiObj (49.28%) and strong Spa457-Pose (41.67%). Across the three 3DSR subsets, depth-based tasks alone average 61.45% vs. 57.99% for depth-free tasks alone—a +3.46 percentage point gap confirming that explicit depth supervision provides superior 3D training signals. However, depth-free tasks alone still improve over baseline by +5.38 points on the 3DSR subsets (57.99% vs. 52.61% baseline), demonstrating that 2D structural reasoning transfers partially to 3D understanding.

Depth-free tasks excel at 2D reasoning and general VQA: The three depth-free tasks combined ("Crop + Shuf. + Flip") achieve 65.38% on Gnr-VQA (second only to Pos. alone at 65.64%), 61.68% on Spa457-2D (the highest of any configuration), 41.36% on Spa457-Pose (competitive), and 72.06% on 3DSR-Location (the second-highest of any configuration, behind the full model's 74.11%). The Spa457-2D result is particularly notable: depth-free tasks alone (61.68%) outperform both depth-based tasks alone (62.06%)—essentially tied—and the full model (62.94%). This suggests that for 2D spatial layout understanding, depth-based tasks provide little additional benefit beyond what RGB-only tasks already teach.

Combining all tasks yields the best or near-best performance on most dimensions: The full model (all five tasks) achieves the best performance on four of seven evaluation dimensions: Spa457-2D (62.94%), Spa457-Pose (42.74%), 3DSR-Location (74.11%), and is competitive on the others. The 3DSR-Location result (74.11%) is substantially above any other configuration—the next best is depth-based tasks alone at 72.94%—suggesting that combining 2D and 3D supervision provides synergistic benefits for 3D location reasoning specifically. However, the full model does not dominate uniformly: it achieves only 42.53% on SpaEval-Maze, below Flip alone (43.60%) and Shuffle alone (43.60%), and it achieves 58.91% on 3DSR-Height, below Depth alone (63.48%) and several other configurations. This is evidence of negative transfer or interference between tasks: for certain specific capabilities, training on all five tasks is worse than training on a subset. The paper does not discuss this phenomenon explicitly, but the data suggest that spatial reasoning capabilities are partially independent and that task diversity can create optimization tradeoffs.

The synergy pattern: The critical evidence for task complementarity is that the full model ($65.32\%$ Gnr-VQA, $62.94\%$ Spa457-2D, $74.11\%$ 3DSR-Location) outperforms both depth-free alone ($65.38\%$, $61.68\%$, $72.06\%$) and depth-based alone ($65.29\%$, $62.06\%$, $71.94\%$) on key dimensions. The Spa457-2D gain from combining categories is +1.26 points over depth-free alone and +0.88 over depth-based alone. The 3DSR-Location gain is +2.05 points over depth-free alone and +2.17 over depth-based alone. These are modest but consistent synergies indicating that 2D and 3D supervision address different failure modes.

Robustness check on task weighting: The paper does not ablate the relative weighting of different tasks within the training mixture. All tasks are sampled roughly equally (each of the four task groups—Shuffled+Flipped, Crop, Depth Ordering, Relative Position—contributes approximately 20k samples). An ablation studying different task mixing ratios (e.g., 80% depth-based, 20% depth-free) would illuminate the sensitivity of the results to task balance.

Robustness check on dataset scale: The paper trains on 81k samples. There is no ablation studying whether 40k or 20k samples would yield similar results—the full dataset is used without data scaling analysis. Given the modest size of the dataset relative to typical LLM training scales, it is possible that the results are primarily driven by task diversity rather than data quantity per task, but this hypothesis is untested.

Robustness check on cold-start necessity: The paper states that direct RL from a pretrained checkpoint fails (success rate < 5%), but does not present a formal ablation comparing: (a) no cold-start (reported as failing), (b) cold-start with varying amounts of SFT data (only 3,600 samples tested), (c) cold-start with longer SFT training (only 5 epochs tested). The claim that cold-start is necessary is supported by the reported failure of direct RL, but the sensitivity to cold-start hyperparameters is unexplored.

Robustness check on reward function weighting: The 0.9/0.1 accuracy-to-format weight ratio is used without ablation of alternatives (e.g., 0.5/0.5, 1.0/0.0, 0.8/0.2). The paper justifies this choice based on the observation that format compliance stabilizes quickly, but does not present experimental evidence that 0.9/0.1 is superior to alternatives.

Negative result: ReST(EM) revision model degradation (Appendix K, Figure 16): An attempt to further optimize the revision model using ReST(EM) caused performance to "substantially hurt," with fully sequential performance dropping to approximately 33.5% compared to roughly 38.5% at the optimal ratio. This negative result highlights the sensitivity of RL-based spatial training to the data generation procedure, but this finding comes from the related revision model work rather than from the core Spatial-SSRL ablation suite.

---

Critical Assessment

The experiments demonstrate that Spatial-SSRL training improves spatial reasoning accuracy across seven benchmarks for both 3B and 7B Qwen2.5-VL models, with gains averaging +4.63% and +3.89%. The evidence is consistent: every benchmark shows improvement (Table 1), the gains hold across a different architecture (Qwen3-VL, Table 2), and no benchmark-specific regression pattern suggests overfitting. The reasoning baseline comparison (Table 1: base + reasoning often underperforms base without reasoning) effectively rules out the alternative explanation that the reasoning format prompt alone drives the improvements. The ablation study (Table 4) confirms that multiple self-supervised tasks contribute to the gains, with different tasks benefiting different aspects of spatial reasoning.

However, several important limitations constrain the strength of the conclusions that can be drawn.

Claim: "Spatial-SSRL delivers average accuracy gains of 4.63% (3B) and 3.89% (7B)." This claim is well-supported by the data in Table 1. The improvements are consistent across benchmarks and model scales. However, the claim is purely empirical—it describes what happened but not why or under what conditions. The mechanism driving the improvement is not experimentally isolated: is it the self-supervised task design specifically, or would any diverse set of verifiable spatial QA pairs produce similar gains? Without a baseline that uses an alternative data generation method (e.g., human-annotated spatial QA pairs of comparable size and diversity, or LLM-generated spatial QA pairs), it is impossible to attribute the gains specifically to the self-supervised paradigm rather than to the quantity and diversity of spatial training data generally.

Claim: "Simple, intrinsic supervision enables RLVR at scale." This claim is partially supported by the results but with important caveats. The paper defines "at scale" implicitly—81k QA pairs is modest relative to typical LLM training datasets. The claim that the pipeline "enables RLVR at scale" is aspirational: the paper demonstrates that self-supervised RLVR works at the tested scale (81k samples, three data sources), but does not demonstrate that it continues to work at larger scales (e.g., 1M+ samples, 10+ data sources) or that the benefits scale with data quantity. An ablation varying dataset size (e.g., 20k, 40k, 80k, 160k) would be needed to support "at scale" as more than a promissory note.

Claim: "Training on our tasks substantially improves spatial reasoning while preserving general visual capabilities." The preservation claim is well-supported by Table 3—general VQA improves, not just preserves—but the interpretation that this demonstrates robustness of the training approach is complicated by the fact that general VQA improves. If spatial training were genuinely orthogonal to general VQA, we would expect no change (preservation). The improvement suggests that the self-supervised tasks teach skills that transfer to general VQA, which is a positive outcome but blurs the distinction between "preserving" and "improving" general capabilities. A more rigorous test of preservation would evaluate on a broader set of capabilities that spatial training should not affect (e.g., factual knowledge, language fluency, coding ability) to verify that the gains are specific and the model has not undergone any form of capability distortion.

Single model family, limited architecture diversity. All primary experiments use Qwen2.5-VL (with one additional experiment on Qwen3-VL). The results may be specific to the Qwen architecture's inductive biases, visual encoding strategy, or training data distribution. Models with different vision encoders (e.g., CLIP-based vs. SigLIP-based), different resolution handling, or different pre-training mixtures might show different sensitivity to self-supervised spatial training. The paper's claim that Qwen2.5-VL is "representative of the capabilities of many contemporary LVLMs" is an assertion, not an empirically validated claim.

Small benchmark subsets in ablations, no statistical testing. The ablation study (Table 4) evaluates on benchmark subsets that are themselves fractions of the already-modest-sized benchmarks. For example, SpaEval-Maze likely contains only a portion of the SpatialEval benchmark's questions (SpatialEval has specific subtasks for maze reasoning, but the total number of maze questions is not reported). The 35.13% baseline accuracy on SpaEval-Maze suggests a challenging but small question set. Without confidence intervals or significance tests, it is impossible to determine whether the 1-3 percentage point differences between training configurations in Table 4 are reliable or within evaluation noise. The absence of any statistical analysis—standard deviations across runs, bootstrap confidence intervals, or significance tests—is a significant methodological weakness given the modest effect sizes on some benchmark subsets.

Missing comparison with strong SFT baselines. The paper contrasts Spatial-SSRL primarily with RL-based methods (SpatialLadder, SpaceR) and with the untrained base model. A critical missing comparison is a model trained via pure SFT on the same 81k self-supervised dataset (without the RL phase). This baseline would isolate the contribution of RL-based optimization vs. standard supervised fine-tuning on identical data. The paper argues for RL based on the finding that "SFT memorizes, RL generalizes" [10], but this finding is from prior work on different tasks and may not hold for spatial reasoning specifically. The cold-start SFT phase demonstrates that SFT on a subset of the data is insufficient to achieve the full gains (the cold-start alone is not evaluated on benchmarks), but an SFT-on-full-dataset baseline would provide direct evidence that the RL phase specifically—rather than the self-supervised data generally—drives the improvement.

No analysis of scaling behavior. The paper uses a fixed dataset size (81k) and fixed training budget (360 GRPO steps, 5 SFT epochs). There is no analysis of how performance scales with dataset size, number of GRPO steps, or number of rollout samples per question. Understanding these scaling relationships is crucial for practitioners wanting to apply the method: should one invest in more data, more training steps, or more diverse tasks? The paper provides no guidance.

No analysis of task difficulty or training dynamics. The paper does not report training curves (reward vs. step, accuracy vs. step for each task), making it impossible to assess whether the model has converged, whether some tasks are learned faster than others, or whether there are phase transitions in spatial reasoning capability during training. This limits the paper's value as a guide for reproducing or extending the work.

The 100% ground-truth accuracy claim. The paper emphasizes that Spatial-SSRL-81k achieves "100% ground-truth accuracy" because answers are deterministically derived from known transformations. This is technically correct for the transformed image versions (the shuffle permutation is known, the flip direction is known, the crop location is known). However, for depth-based tasks, the "ground truth" depends on sensor depth maps, which contain noise, missing values, and systematic errors (especially in MegaDepth, which uses structure-from-motion reconstruction rather than direct depth sensing). The depth ordering and relative position ground truth is only as accurate as the depth maps themselves. The constraints ($r_{\text{max}}$, $d_{\text{min}}$, $\delta_x$, $\delta_z$) mitigate this by only selecting samples where the depth signal is unambiguous, but they do not eliminate sensor noise entirely. The 100% accuracy claim should be qualified: it holds for the applied transformations but is contingent on depth sensor accuracy for depth-based tasks.

Missing analysis of which depth cues the model learns. The paper demonstrates that self-supervised training improves spatial reasoning, but provides no analysis of what specific spatial reasoning strategies the model learns. Do the qualitative examples (Appendix C) represent systematic capability gains or cherry-picked successes? Are there failure modes where the model reverts to 2D heuristics rather than genuine 3D reasoning? A systematic error analysis—categorizing failure types before and after training—would substantially strengthen the paper's claims about the nature of the improvement.

The VSI-Bench video transfer claim. The improvement on VSI-Bench (+5.65% for 3B, +1.21% for 7B) is presented as evidence of cross-modal generalization. However, VSI-Bench evaluates egocentric video understanding, and the paper does not analyze whether the improvement comes from better single-frame spatial reasoning (plausible given static image training) or from better temporal integration across frames (unlikely given no temporal training). A frame-by-frame analysis or comparison with a static-frame baseline would clarify the mechanism.

6. Limitations and Trade-offs

Difficulty Estimation Cost Is Unaccounted For and May Dominate Total Compute

The assumption or constraint. The entire compute-optimal framework depends on estimating prompt difficulty before allocating the inference budget. The paper's method for doing this—generating 2048 samples per question and averaging either ground-truth correctness (oracle) or PRM final-answer scores (predicted)—is extraordinarily expensive. The paper acknowledges this explicitly in Section 3.2: "estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity." At 2048 samples per question, the difficulty estimation step alone consumes more compute than the largest test-time budgets studied (256–512 generations).

The consequence. The reported 4× efficiency gains over best-of-N are computed after difficulty is known, without amortizing the cost of learning it. In a realistic deployment, the total cost would be difficulty estimation + strategy execution, and the former could dominate the latter. For example, if a problem requires 16 generations under the compute-optimal policy but estimating difficulty costs 2048 generations, the real cost is 2064 generations—making the "efficient" strategy 129× more expensive than just running best-of-16 and hoping for the best. The entire practical value proposition of compute-optimal scaling collapses if difficulty estimation is not made dramatically cheaper. The paper suggests future work on training models to predict difficulty directly from the question text, but no such model is developed or evaluated, and it is unclear whether question-text-based difficulty prediction can achieve the same fidelity as the sampling-based method.

What evidence exists in the paper. The 2048-sample difficulty estimation procedure is described in Section 3.2 with explicit caveats. Figures 4 and 8 show that predicted (non-oracle) difficulty bins perform similarly to oracle bins when difficulty is known, but neither figure accounts for the estimation cost. The paper provides no ablation studying cheaper difficulty estimation strategies (e.g., 16 or 64 samples instead of 2048), no analysis of how estimation accuracy degrades with fewer samples, and no end-to-end cost accounting that includes difficulty estimation in the budget.

Mitigation status. The authors acknowledge this as a significant gap (Section 3.2) and frame it as "a key avenue for future work." They suggest both "pretraining or finetuning models to directly predict difficulty of a question" and adaptive approaches where difficulty is estimated progressively during inference. Neither approach is explored in the paper. The current results therefore represent an upper bound on achievable efficiency rather than a realized deployment gain, and this qualification is not reflected in the abstract or conclusions, which present the 4× figure without this caveat.

---

Hard Problems Remain Essentially Unsolved—Test-Time Compute Cannot Compensate for Missing Capability

The assumption or constraint. The paper's approach relies on the base model already being capable of producing correct solutions at some non-trivial rate. For problems where the base model's pass@1 is near zero, no amount of search or revision can help—there are no correct solutions in the proposal distribution to find or refine. This is a fundamental boundary condition on what test-time compute can achieve, not a limitation that can be engineered around.

The consequence. Across all methods—search, revisions, and their compute-optimal combinations—the hardest questions in difficulty bin 5 show near-zero improvement regardless of compute budget. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all methods and all budgets. In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%, and the 14× larger pretrained model outperforms test-time compute by large margins (+52.9% relative disadvantage for PRM search at R ≫ 1). This means the approach offers no path forward for genuinely novel or out-of-distribution reasoning that exceeds the base model's training distribution. For such problems, pretraining remains the only viable path. A practitioner confronting a distribution of problems that includes a substantial fraction of "hard" questions—where the base model's pass@1 is effectively zero—will see no benefit from test-time compute scaling regardless of budget.

What evidence exists in the paper. The bin 5 flatlining is visible across all three main result figures: Figure 3 (right, search), Figure 7 (right, revisions), and Figure 9 (FLOPs-matched comparison). The paper is transparent about this in Section 7: "test-time compute amplifies existing capability but does not create it from nothing." However, the abstract and introduction do not mention this fundamental limitation, instead emphasizing the 4× efficiency gains and the positive FLOPs-matched results on easy-to-medium problems.

Mitigation status. The paper does not attempt to address this limitation, nor does it suggest that it can be addressed within the test-time compute paradigm. The authors are candid about the boundary condition (Section 7 takeaway box), but the framing emphasizes the positive results on problems where test-time compute works, potentially misleading readers about the breadth of applicability. A more prominent qualification—e.g., "for problems within the base model's capability range"—would better calibrate expectations.

---

The 14× Larger Model Baseline Is Not Compute-Optimally Trained and Uses Greedy Decoding—Making the FLOPs-Matched Comparison Favorable to Test-Time Compute

The assumption or constraint. The FLOPs-matched comparison in Section 7 scales model parameters while holding training data fixed, following the LLaMA paradigm (Touvron et al., 2023) rather than Chinchilla-optimal training where both data and parameters are scaled equally (Hoffmann et al., 2022). The paper acknowledges this in Section 7: "We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work." Additionally, the 14× larger model uses only greedy decoding—no majority voting, no best-of-N, no test-time compute augmentation of its own.

The consequence. A Chinchilla-optimal model trained with 14× more total FLOPs would likely outperform a parameter-only-scaled model. The pretraining baseline is therefore weaker than it needs to be, making the reported advantages of test-time compute over pretraining (e.g., +27.8% on easy questions at R ≪ 1 for revisions, Figure 1 bar chart) potentially overstated. Even more consequentially, giving the larger model even a modest test-time compute budget—say, best-of-8 or best-of-16—would create a much stronger baseline that is never tested. The paper's central claim that "a smaller model augmented with compute-optimal test-time strategies can outperform a ~14× larger pretrained model" (Section 1) is therefore only established for a specific, potentially suboptimal pretraining baseline. This is not a minor experimental detail: it directly affects the paper's most policy-relevant finding about the pretraining-vs-inference tradeoff, which could influence how organizations allocate compute budgets between training and inference.

What evidence exists in the paper. The baseline configuration is described in Section 7 (parameter-only scaling, greedy decoding). The Chinchilla caveat is acknowledged in the same section. No ablation tests a stronger baseline: a Chinchilla-optimal 14× larger model, or the 14× larger model with even minimal test-time compute (best-of-N = 4 or 8).

Mitigation status. The authors flag this as future work but do not provide sensitivity analysis or bounds on how much the results might change with a stronger pretraining baseline. The abstract and conclusions present the finding without this qualification, which could mislead readers into overgeneralizing the result.

---

Revisions and Search Are Studied Independently—The Two Complementary Mechanisms Are Never Combined

The assumption or constraint. The paper studies two complementary axes for test-time compute—PRM-guided search (modifying the verifier/selection mechanism) and iterative revisions (modifying the proposal distribution)—but explicitly does not combine them. Section 8 states: "we did not experiment with PRM tree-search techniques in combination with revisions." The search experiments use the base model's few-shot outputs as the proposal distribution; the revision experiments use an ORM trained specifically on revision outputs for answer selection, never the PRM and never with tree-search.

The consequence. The current results represent a lower bound on what a fully integrated system could achieve, because the two mechanisms have complementary strengths identified by the paper itself: revisions improve the quality of generated candidates (especially on easy problems), while PRM search improves the selection among candidates (especially on medium problems). A natural integration would use the revision model as the proposal distribution within PRM-guided beam search—at each step of the search tree, the model conditions on previous rejected branches, potentially producing higher-quality candidate steps than the base model alone. Alternatively, the PRM's per-step scores could be used to decide when a revision chain is on track versus when to restart from scratch. The ablation-like pattern in the paper's independent results (revisions excel on easy problems, search on medium) strongly suggests that a combined system would outperform either alone across all difficulty levels, but this hypothesis remains untested.

What evidence exists in the paper. Section 8 explicitly acknowledges this gap. The difficulty-dependent strengths of the two mechanisms are documented in Figure 3 (right, search) and Figure 7 (right, revisions): revisions show their largest relative advantage on easy problems, while beam search shows its on medium-hard problems. The fact that these are complementary—not redundant—is the central justification for the compute-optimal policy itself, yet the policy is restricted to selecting between search and revisions rather than combining them within a single problem.

Mitigation status. The paper identifies this as a direct next step for future work (Section 8). No combination experiments are presented, and no analysis of what a combined system might look like architecturally is provided. This is a significant gap given that the paper's own framework (proposal distribution modification + verifier optimization) naturally suggests the combination, and the difficulty-dependent results provide strong motivation for it.

---

The Revision Model Has a 38% Correct-to-Incorrect Reversion Rate—Mitigated Only by Post-Hoc Selection Across the Chain

The assumption or constraint. The revision model is trained exclusively on trajectories where all in-context answers are incorrect, followed by a correct target answer. At test time, the model may encounter correct answers in its context (produced during earlier revisions in the chain) and, having never been trained on what to do when the current answer is already correct, may incorrectly "revise" them into wrong answers. The paper reports in Section 6.1 that approximately 38% of correct answers get converted back to incorrect ones using a naive approach of always taking the last revision output.

The consequence. The sequential revision strategy—which the compute-optimal policy favors on easy problems (Figure 7)—is fundamentally unreliable at the step level. A correct answer produced at step 3 of a 20-step chain has a substantial probability of being corrupted at step 4, with no guarantee of recovery. The paper mitigates this by selecting the best answer from any point in the chain using majority voting or verifier-based selection, rather than always taking the final revision. However, this mitigation is a post-hoc patch, not a solution: it requires storing all intermediate outputs and running an external selection mechanism, which adds overhead and still cannot guarantee that the best answer in the chain is actually correct (the verifier or majority vote can also be wrong). More fundamentally, the reversion rate means the revision model has not learned a convergence property—it does not know to stop revising when the answer is already correct—which limits how many sequential revisions can productively be used before the chain destabilizes. This is likely why the compute-optimal sequential-to-parallel ratio is not purely sequential even on easy problems (Figure 7, right: bin 2 shows peak performance at a moderate ratio rather than fully sequential).

What evidence exists in the paper. The 38% reversion rate is reported in Section 6.1. The mitigation (majority voting or verifier-based selection across the chain) is described in the same section and evaluated in Figure 6. The revision pass@1 trajectory (Figure 6, left) shows improvement throughout the chain, but this is per-step pass@1—the figure does not show how often correct answers are subsequently reversed, which is only revealed by the aggregate sequential-vs-parallel comparison in Figure 6 (right). The compute-optimal ratio results in Figure 7 (right) implicitly reflect the reversion problem: fully sequential is not optimal even on easy questions, suggesting that longer chains eventually suffer from reversion-driven degradation.

Mitigation status. The paper implements post-hoc selection (majority or verifier) as a workaround but does not address the root cause—the training data distribution that never includes correct-to-correct or correct-to-unsure transitions. A more principled solution, such as training the model to output a special "no revision needed" token or including correct-in-context examples during training, is not explored. The ReST(EM) experiment (Appendix K, Figure 16) suggests that revision training is brittle: attempts to further optimize the revision model with RL-style training caused performance to degrade substantially, with fully sequential performance dropping to approximately 33.5% compared to roughly 38.5% at the optimal ratio. This indicates that the reversion problem may be exacerbated by more aggressive optimization, making it a hard constraint on how far revision-based strategies can scale.

---

Latency and Wall-Clock Time Are Ignored—Sequential Strategies May Be Impractical for Interactive Applications

The assumption or constraint. The paper measures compute in "generations" (number of complete solutions sampled), which is a reasonable proxy for total FLOPs but ignores latency. Sequential revisions are inherently serial—each revision depends on the previous one—while parallel best-of-N can be executed simultaneously with sufficient hardware. The paper does not discuss this tradeoff anywhere in the main text or appendices.

The consequence. A strategy that allocates 128 generations as 64 sequential × 2 parallel takes roughly 64× longer wall-clock time than one that runs 128 parallel samples simultaneously, even though both consume the same total FLOPs. For latency-sensitive applications—interactive assistants, real-time decision-making in autonomous driving or robotics, user-facing chatbots—the sequential-heavy strategies favored by the compute-optimal policy on easy problems (Figure 7: easy problems perform best with high sequential-to-parallel ratios) may be impractical regardless of their accuracy advantages. A model that achieves 90% accuracy with 64 sequential revisions at 10 seconds per revision is less useful than one achieving 85% accuracy with 64 parallel samples at 0.15 seconds total, even though the former has better accuracy-per-FLOP. The paper's exclusive focus on generation count as the cost metric makes its policy recommendations incomplete for any deployment where response time matters.

What evidence exists in the paper. The paper provides no latency measurements, no analysis of wall-clock time, and no discussion of the sequential-vs-parallel latency tradeoff. The generation budget is the only cost metric used throughout (Sections 5 and 6). The compute-optimal policy selects strategies purely based on generation count, with no latency penalty for sequential allocations.

Mitigation status. Not addressed. Future work on real-world deployment of compute-optimal strategies would need to incorporate a latency constraint or a multi-objective optimization that trades off accuracy against response time. This would likely shift the optimal policy toward more parallel strategies than the current generation-count-optimal results suggest, especially in the easy-problem regime where the paper currently recommends high sequential-to-parallel ratios.

7. Implications and Future Directions

How This Work Changes the Landscape

Spatial-SSRL introduces a methodological reframing rather than a paradigm shift for the entire field, but a genuine paradigm shift for the specific subfield of spatial understanding in LVLMs. The magnitude of the contribution lies not in any single architectural innovation or benchmark result, but in demonstrating that self-supervised pretext tasks—constructed deterministically from raw sensor data—can serve as a complete replacement for the expensive, tool-heavy, annotation-dependent pipelines that previously dominated spatial reasoning research. This is a category shift in what counts as "training data" for spatial understanding: from extrinsically annotated spatial relationships (human-labeled depth orderings, tool-extracted 3D scene graphs, synthetically rendered views) to intrinsically verifiable spatial puzzles (patch reordering, flip detection, depth ordering from sensor data, coordinate transformations).

The paper changes the landscape in four specific ways:

It resolves the tension between scalability and verifiability in spatial RLVR. Prior work faced an uncomfortable tradeoff: RLVR with verifiable rewards works well for spatial reasoning (as shown by SpatialLadder and SpaceR), but obtaining verifiable rewards required either (a) annotation-rich 3D datasets like ScanNet that are domain-limited (indoor scenes only) and expensive to collect, or (b) tool-heavy pipelines using depth estimators, object detectors, and segmentation models that introduce errors and computational overhead. The field was stuck—RLVR was the right optimization paradigm, but the data bottleneck prevented it from scaling to the diverse, real-world imagery needed for general-purpose spatial intelligence. Spatial-SSRL breaks this tradeoff by showing that verifiable rewards can be derived from image structure itself, with zero external annotation, zero tool dependencies, and zero domain restrictions beyond the availability of raw pixels (and optionally sensor depth). This transforms RLVR for spatial understanding from a method constrained by annotation budgets to a method constrained only by the availability and diversity of raw images—which is effectively no constraint at all given the scale of public image datasets.

The practical significance of this resolution is captured by the contrast between SpatialLadder (trained on ScanNet, 45.09% average) and Spatial-SSRL-3B (trained on general RGB/RGB-D images, 50.54% average). The 5.45-point advantage is impressive not because it's large, but because Spatial-SSRL achieves it using training data that costs orders of magnitude less to produce and spans dramatically more diverse visual domains. This is the empirical signature of a scalability breakthrough: better performance at lower cost with broader coverage.

It establishes that 2D self-supervision partially substitutes for 3D supervision in spatial reasoning. This finding—visible in the ablation results where depth-free tasks alone improve 3D reasoning benchmarks by 5.38 points over baseline (Table 4: 57.99% vs. 52.61% on 3DSR subsets)—challenges a core assumption in the spatial understanding literature. The prevailing view has been that 3D spatial reasoning requires 3D supervision: depth maps, multi-view data, camera poses, or 3D scene reconstructions. This assumption drove enormous investment in depth estimation, structure-from-motion, and 3D scanning pipelines for spatial training data generation. Spatial-SSRL shows that a substantial fraction of 3D spatial reasoning capability can be acquired from 2D self-supervised tasks alone—presumably because monocular depth cues (perspective, relative size, occlusion boundaries, texture gradients) are already present in RGB images, and tasks like patch reordering and inpainting force the model to attend to these cues even without explicit depth labels.

The practical implication is that RGB-only data—which is orders of magnitude more abundant and diverse than RGB-D data—can serve as effective training material for 3D spatial reasoning. This dramatically expands the feasible scale and domain diversity of spatial training. The depth-based tasks still provide additional gains (3.46-point advantage over depth-free alone on 3DSR subsets), so sensor depth remains valuable when available. But the finding that 2D tasks alone get you 85% of the way to the full 3D+2D combined performance (57.99% vs. 60.63%) means that practitioners without access to depth sensors can still achieve substantial spatial reasoning improvements. This finding is likely to redirect research investment toward designing better 2D pretext tasks rather than exclusively pursuing depth estimation or 3D reconstruction pipelines.

It provides a template for applying RLVR to any perceptual capability where deterministic transformations produce verifiable ground truth. The paper's three design principles—zero human/LLM supervision, tool-free scalability, natural verifiability—define a general methodology that extends far beyond spatial reasoning. Any researcher wanting to improve a specific perceptual capability in an LVLM can now ask: "What deterministic transformations on raw sensor data produce unambiguous ground truth for this capability?" If answerable, the Spatial-SSRL template applies directly: design a suite of complementary pretext tasks spanning the target capability, construct a verifiable QA dataset from diverse real-world imagery, cold-start with format-teaching SFT, then optimize with GRPO using binary exact-match rewards. This methodology is likely to spawn analogous work on temporal reasoning (temporal ordering, event duration prediction, action sequence completion), physical intuition (stability prediction, trajectory forecasting, contact reasoning), and causal understanding (counterfactual interventions, causal ordering), each using domain-appropriate transformations on video or multi-modal data to generate verifiable training signals.

The template also clarifies when RLVR is likely to work vs. when it will fail. The paper's cold-start finding—direct RL collapses when the base model cannot produce parseable responses—establishes a practical precondition: the target output format must be teachable through a small amount of supervised data before RL can optimize content. This separation of format learning (SFT) from content learning (RL) is a design pattern that should generalize to other RLVR applications and provides guidance for practitioners about when a pure RL approach is feasible vs. when a hybrid SFT+RL pipeline is necessary.

It reframes the "SFT vs. RL" debate for perceptual capabilities. Prior work established that "SFT memorizes, RL generalizes" for reasoning tasks. Spatial-SSRL provides evidence that this finding extends to perceptual-spatial reasoning, but with an important nuance: the SFT cold-start is necessary as a format-teaching mechanism, not a content-teaching mechanism. The paper thus suggests a more nuanced position than "RL is better than SFT"—rather, SFT and RL serve different roles in a pipeline (SFT for format grounding, RL for behavioral optimization), and the optimal approach uses both in sequence. This reframing is likely to influence how future work designs training pipelines for RLVR applications: always include a minimal SFT phase to teach output format, then use RL to optimize the actual capability.

Less obviously, the paper's finding that general VQA improves with spatial-specific RL training (Table 3: +2.02% for 3B, +0.57% for 7B) challenges the assumption that specialized RL fine-tuning necessarily creates capability tradeoffs. The self-supervised tasks appear to teach holistic image understanding skills (attention to structural coherence, texture continuity, fine-grained detail) that transfer to general visual tasks. This is an encouraging signal for future RLVR applications: if the pretext tasks are well-designed to require capabilities that generalize, specialized RL training may confer broad benefits rather than narrow overfitting. However, the paper provides only correlational evidence for this claim—no mechanistic analysis of why general VQA improves—so this should be treated as an intriguing observation rather than an established principle.

---

Follow-Up Research This Work Enables

1. Dynamic difficulty-adaptive task construction during RL training. The paper uses a fixed dataset of 81k self-supervised QA pairs with uniform task sampling. A natural extension is to adapt the task distribution during training based on the model's current capabilities—a curriculum learning approach applied to the RL phase. The key question: does the optimal task mixture for early training (when the model is learning basic spatial reasoning) differ from the optimal mixture for later training (when it is refining advanced capabilities)? The paper's ablation results provide a starting point: different tasks benefit different capabilities (Flip excels at maze reasoning, Depth at height estimation, Relative Position at general VQA). A follow-up could implement a dynamic scheduler that monitors per-task accuracy during GRPO and adjusts the sampling distribution to focus on tasks where the model is improving (maintaining learning progress) or tasks where it is plateauing (providing more training signal). The concrete experiment would compare: (a) uniform sampling (as in the paper), (b) curriculum from 2D to 3D tasks (depth-free first, then depth-based), (c) anti-curriculum from 3D to 2D, and (d) adaptive sampling based on recent reward gradients. The expected finding—that curriculum ordering matters for sample efficiency—would provide practical guidance for scaling this method to larger datasets where uniform sampling becomes wasteful. A negative result (curriculum doesn't matter) would suggest the tasks are sufficiently independent that any ordering suffices, which is also valuable to know.

2. Scaling laws for self-supervised spatial RLVR: data quantity vs. task diversity vs. training steps. The paper uses a fixed 81k dataset, fixed 360 GRPO steps, and fixed five-task suite. Three scaling dimensions are unexplored and critical for practitioners: (a) How does spatial reasoning accuracy scale with the number of self-supervised training samples, holding task diversity constant? A data scaling experiment could train models on 10k, 20k, 40k, 80k, and 160k subsets of Spatial-SSRL-81k (or an expanded version) and measure benchmark accuracy as a function of dataset size, estimating a scaling exponent. (b) How does performance scale with the number of distinct pretext tasks, holding total data constant? An experiment could compare models trained on 1, 2, 3, 4, 5, and (hypothetically) 8 or 10 different tasks, each contributing equal total samples, to determine whether task diversity provides diminishing, linear, or super-linear returns. The paper's ablation results hint at diminishing returns—the 5-task model is only marginally better than the 3-task models in several dimensions—but a controlled experiment varying task count while holding total data fixed would clarify this. (c) How does performance evolve with GRPO steps? Training curves (reward, accuracy per task, benchmark accuracy at checkpoints) would reveal whether 360 steps is near convergence, and whether different tasks converge at different rates. This is the most straightforward follow-up and should be prioritized, as it directly addresses the paper's silence on training dynamics. A negative result—no clear scaling relationship, high variance across runs—would suggest that 81k samples and 360 steps are already past the point of diminishing returns, implying the method is near its ceiling with current task designs.

3. Stress-testing cross-architecture and cross-domain generalization boundaries. The paper evaluates on Qwen2.5-VL (3B, 7B) and Qwen3-VL-4B, all from the same model family. A critical follow-up would replicate Spatial-SSRL training on models with fundamentally different vision backbones: CLIP-based (e.g., LLaVA), SigLIP-based (e.g., DeepSeek-VL), and models with native high-resolution processing (e.g., InternVL2). The specific hypothesis is that the Qwen architecture's visual encoding may be particularly well-suited to the patch-level tasks (Shuffled Patch Reordering, Flipped Patch Recognition, Cropped Patch Inpainting) because these tasks require fine-grained spatial attention to image regions, and Qwen's dynamic resolution handling may facilitate this. If CLIP-based models (which use fixed-resolution global embeddings) show substantially smaller gains from Spatial-SSRL training, this would reveal an important architectural dependency: self-supervised spatial RLVR works best when the vision encoder preserves spatial structure at the patch level. Conversely, if all architectures show similar gains, the method is architecture-agnostic and more broadly applicable. The experiment should include a null condition: train the same self-supervised tasks on a vision-language model with a spatially-disentangled architecture (e.g., one that separates object-level and scene-level representations) to test whether spatial reasoning improvements are specifically tied to patch-level visual processing. A further stress-test would evaluate on out-of-domain spatial benchmarks not used in the paper—e.g., autonomous driving spatial QA (NuScenes-QA), robotics manipulation spatial reasoning (RoboSpatial), or medical imaging spatial understanding—to determine whether the cross-domain transfer observed on VSI-Bench (video) extends to fundamentally different visual domains.

4. Mechanistic interpretability of what spatial reasoning strategies the model learns. The paper demonstrates that Spatial-SSRL improves benchmark accuracy, but provides no analysis of how the model's reasoning process changes. A follow-up using interpretability methods could characterize the learned spatial reasoning strategies: (a) Do the model's attention maps during spatial reasoning tasks show increased focus on depth cues (horizon lines, relative size, occlusion boundaries) after training? Compare attention maps on spatial benchmarks for base vs. Spatial-SSRL models. (b) Does the reasoning chain language quality improve? The paper provides qualitative examples (Appendix C) showing Spatial-SSRL producing more coherent spatial reasoning than the baseline, but this is anecdotal. A systematic analysis using LLM-as-judge or human evaluation could score reasoning chains on dimensions like "correctly identifies relevant objects," "correctly uses perspective information," "makes valid spatial inferences," and "avoids image-plane confusions." Comparing these scores pre- and post-training would reveal which specific reasoning subskills improve. (c) Are there systematic failure modes that persist after training? A taxonomy of Spatial-SSRL error types—e.g., confusing image-plane relationships with 3D relationships, failing to account for object orientation, making correct depth comparisons but incorrect coordinate transformations—would identify the residual weaknesses that future task designs should target. The paper's find that the 5-task full model underperforms single-task models on specific dimensions (e.g., SpaEval-Maze: 42.53% for full model vs. 43.60% for Flip alone) hints at interference between learned strategies; an interpretability analysis could reveal whether this reflects the model learning mutually incompatible reasoning heuristics from different tasks. This direction is important because it bridges the gap between the paper's behavioral results (accuracy improvements) and cognitive understanding (what the model actually learned), which is essential for principled improvement of the task design.

5. Combining self-supervised RLVR with interleaved reasoning-visualization training. The paper uses a text-only reasoning format (⟨think⟩...⟨/think⟩) for all tasks. A natural extension—motivated by concurrent work on reinforcing spatial reasoning with visual drawing—would be to augment the training to include visual annotations: for Shuffled Patch Reordering, have the model output bounding boxes or segmentation masks showing how it determined patch adjacencies; for Regional Depth Ordering, have the model annotate depth cues (ground plane, vanishing points) in the image; for Relative Position Prediction, have the model draw the coordinate transformation on a top-down view of the scene. The hypothesis is that explicit visual reasoning—not just textual reasoning—would improve spatial understanding because spatial relationships are fundamentally visual-geometric, not linguistic-propositional. A concrete experiment would extend Spatial-SSRL-81k with visual annotation tasks (e.g., "Draw arrows showing which edge of patch A aligns with which edge of patch B") using the same self-supervised paradigm (the annotations are deterministically derivable from the known patch arrangement) and compare: (a) text-only reasoning (current approach), (b) text + visual reasoning, (c) visual-only reasoning. If visual reasoning provides additional gains, this would open a new dimension of self-supervised task design—not just asking questions about spatial transformations, but requiring the model to demonstrate spatial understanding through visual outputs.

6. Scaling the task suite to temporal and 4D spatial reasoning. The paper's five tasks cover 2D layout, orientation, correspondence, ordinal depth, and egocentric 3D relations—all static spatial capabilities. The VSI-Bench video transfer results (+5.65% for 3B, +1.21% for 7B) show that static-image training partially transfers to video spatial understanding, but the gains are modest. A major extension would design self-supervised pretext tasks that explicitly target temporal-spatial reasoning: (a) temporal ordering of shuffled video frames (the video analog of Shuffled Patch Reordering)—given shuffled frames from a video, predict the correct temporal order; (b) optical flow prediction from frame pairs—given two consecutive frames, predict the direction and magnitude of motion for each pixel or region; (c) occlusion prediction—given three frames where an object is partially occluded in the middle frame, predict which object is in front; (d) 3D trajectory prediction—given an RGB-D video of a moving object, predict its future 3D position. These tasks would follow the same self-supervised paradigm (the ground truth is deterministically derivable from the video or depth sequence) and would be combined with the existing static tasks in a joint curriculum. The concrete experiment would compare: (a) Spatial-SSRL (static tasks only) on video benchmarks, (b) static + temporal tasks on the same benchmarks, showing whether explicit temporal self-supervision provides gains beyond static-to-video transfer. A negative result—temporal tasks don't help above static tasks—would suggest that static spatial understanding is the primary bottleneck for video spatial reasoning, which would redirect research attention to better static tasks rather than temporal extensions.

---

Practical Applications and Downstream Use Cases

1. Cost-efficient spatial data generation for robotics and embodied AI. The paper's core value proposition—verifiable spatial training data at zero annotation cost—has immediate application in robotics, where spatial understanding is essential and domain-specific training data is scarce. A robotics lab deploying a vision-language model for manipulation tasks (e.g., "pick up the cup to the left of the plate") or navigation ("go to the room at the end of the hallway on the right") needs the model to reliably understand spatial relationships in their specific environment. Traditional approaches require collecting and annotating environment-specific spatial data—expensive, slow, and brittle to environment changes. Using Spatial-SSRL, the lab could: (1) collect raw RGB-D images from the robot's cameras during operation (no annotation needed), (2) run the automated task construction pipeline to generate thousands of self-supervised spatial QA pairs from those images, (3) fine-tune their LVLM using the paper's SFT+GRPO recipe on this environment-specific data. The result is a spatially-aware model tuned to the specific visual appearance, object types, and spatial configurations of the deployment environment, generated entirely from unsupervised robot camera feeds. The paper's results suggest this would improve spatial reasoning accuracy by 3–5 percentage points on average (from the 3B and 7B gains in Table 1), with the benefit concentrated on 3D spatial relationships (the largest gains are on Spatial457 and 3DSRBench). The key advantage over existing approaches is zero incremental annotation cost: the data pipeline runs continuously as the robot operates, and the model can be periodically retrained on accumulating data, providing a self-improving spatial understanding capability.

2. On-device spatial reasoning for augmented reality assistants. The 4.63% average improvement for the 3B model (Table 1) is particularly relevant for on-device deployment, where smaller models are preferred for latency and power constraints but spatial understanding is critical for AR applications. An AR glasses application that needs to answer queries like "which building is the hotel behind the fountain?" or "is the red car to the left of the intersection from where I'm facing?" currently struggles because small LVLMs perform poorly on spatial reasoning (Qwen2.5-VL-3B achieves only 45.91% average across spatial benchmarks). Spatial-SSRL-3B's improvement to 50.54% represents a meaningful accuracy gain without increasing model size—the computational cost at inference time is identical, only the training pipeline changes. For an AR company, this means they can improve spatial understanding in their on-device model by running the Spatial-SSRL training pipeline on their own imagery (collected from device cameras during consented usage, or from public datasets of outdoor scenes for navigation-relevant spatial relationships) without paying for spatial annotations or depending on proprietary models. The 3B model's substantial gains on Spatial457 (+12.37%) and QSpatial-plus (+5.95%)—benchmarks testing 3D pose estimation and quantitative distance prediction, both critical for AR overlays—are directly relevant to the AR use case.

3. Automated spatial QA generation for benchmarking and diagnostics. The paper's task construction pipeline—deterministic transformations on raw images producing QA pairs with 100% ground-truth accuracy—can be used to generate diagnostic spatial reasoning benchmarks at scale. Current spatial benchmarks (Spatial457, 3DSRBench, SpatialEval) are limited to hundreds or thousands of questions, often with narrow domain coverage (e.g., 3DSRBench focuses on synthetic scenes). Using Spatial-SSRL's task suite, one could generate millions of diagnostic questions from any image dataset, with fine-grained control over difficulty (by adjusting patchification grid size, distractor similarity in inpainting, depth separation thresholds, orientation sampling). This enables: (a) large-scale evaluation of LVLM spatial reasoning across diverse visual domains—a "SpatialQA-1M" benchmark generated from COCO, ImageNet, and other diverse sources; (b) fine-grained diagnosis of specific spatial reasoning failures—which task types does a model succeed at, which does it fail at, and on what kinds of visual content; (c) tracking spatial reasoning capability improvements across model versions at scale. The 100% ground-truth accuracy property is critical for benchmarking: unlike human-annotated benchmarks that may contain labeling errors or ambiguities that complicate evaluation, self-supervised benchmarks have definitive correct answers, enabling clean automated evaluation. This application would standardize the insight that self-supervised tasks can serve as evaluation instruments, not just training instruments.

4. Improving spatial reasoning in open-source VLMs through community-contributed task suites. The Spatial-SSRL framework is modular and extensible: new self-supervised tasks can be designed and integrated without changing the RL pipeline. This suggests a community-driven approach to spatial reasoning improvement, analogous to how the NLP community develops and shares benchmark datasets. Researchers could contribute new pretext tasks targeting specific spatial reasoning capabilities (e.g., a task for reasoning about object permanence under occlusion, a task for relative size estimation, a task for 3D path planning in top-down views) using the same self-supervised paradigm. Because all tasks produce verifiable rewards in the same format (question + deterministic answer), they can be combined into a growing, community-maintained "Spatial-SSRL Task Suite" that provides ever-broader spatial reasoning coverage. The paper's ablation results showing that task diversity matters (the 5-task model outperforms any subset on most dimensions) provide empirical support for the value of expanding the task suite. An open-source VLMs project (e.g., LLaVA, InternVL) could adopt this as a standard post-training step: after initial instruction tuning, apply Spatial-SSRL training using the community task suite to improve spatial reasoning before release. The 81k dataset size and 360-step training budget are modest enough that this could be a lightweight addition to existing post-training pipelines.

---

When to Prefer This Method

The paper articulates a clear tradeoff between intrinsic self-supervised supervision and extrinsic tool-based or annotation-based supervision for spatial reasoning training (Section 1, Figure 2, the three design principles), but does not provide controlled experiments directly comparing Spatial-SSRL against alternative data generation methods at matched scale. The decision rules below are therefore drawn from the paper's conceptual framing and empirical results, not from head-to-head experimental comparisons with SFT-on-annotated-data or RLVR-with-tool-based-verifiers baselines. They should be treated as provisional guidance based on the paper's demonstrated capabilities and acknowledged limitations.

• Prefer Spatial-SSRL (self-supervised RLVR) when: (1) you need to improve spatial reasoning across diverse visual domains (indoor, outdoor, varied object categories) and cannot afford domain-specific data collection or annotation; (2) you have access to large quantities of unannotated RGB or RGB-D images (from public datasets, user data, or robot camera feeds) and want to leverage them for spatial training at zero annotation cost; (3) you are working with a model architecture that preserves patch-level spatial structure (the paper validates on Qwen2.5-VL and Qwen3-VL, and the patch-based tasks likely benefit from architectures with fine-grained visual encoding); (4) you need to improve general visual understanding simultaneously with spatial reasoning (the paper shows +2.02% Gnr-VQA for 3B, Table 3); (5) you are willing to invest in a two-stage training pipeline (SFT cold-start + GRPO) and can tolerate the associated complexity and hyperparameter tuning.

• Prefer extrinsic supervision (tool-based or annotation-based RLVR) when: (1) you need to train on specific spatial relationships not covered by the current five-task suite—e.g., complex multi-object occlusion scenarios, fine-grained relative distance estimation, or spatial relationships requiring semantic understanding of object identities (the self-supervised tasks are object-agnostic); (2) you are working in a domain where raw images are scarce but annotated 3D data exists (e.g., you have access to an existing ScanNet-like dataset with rich meta-information, or a simulation environment with ground-truth 3D scene graphs); (3) you need the training signal to include semantically meaningful spatial concepts (e.g., "the cup is on the table" vs. "region A has smaller depth than region B"—the self-supervised tasks teach geometric reasoning but not semantic spatial language); (4) your target model architecture processes images holistically rather than preserving patch-level spatial structure, potentially reducing the effectiveness of patch-based pretext tasks.

• Prefer SFT on large-scale spatial QA data (rather than RLVR at all) when: (1) you have access to a high-quality, large-scale spatial QA dataset (e.g., millions of human-annotated or reliably machine-generated spatial QA pairs) and want a simpler training pipeline without the complexity of RL; (2) your base model already has strong spatial reasoning capabilities and you are primarily addressing distribution shift or domain adaptation rather than teaching new reasoning skills—the paper's results show that SFT cold-start alone provides some benefits (though not evaluated independently on benchmarks), and the major gains come from RL, but this may depend on the starting capability level. The paper provides no direct SFT vs. RL comparison at matched data scale, so this recommendation is speculative.

• Be cautious about Spatial-SSRL when: (1) your problem distribution includes many "hard" spatial reasoning cases where the base model's spatial accuracy is near zero—the paper shows that difficult benchmark questions (bin 5 analog in the test-time compute paper; not directly analyzed in Spatial-SSRL) benefit least from additional training, though the Spatial-SSRL paper does not stratify by difficulty and this limitation is inferred from the modest gains on already-high-performance benchmarks like What'sUp (+0.86% for 3B); (2) you are deploying in a latency-sensitive application and the two-stage training pipeline must be repeated frequently (cold-start + GRPO takes non-trivial compute and engineering effort); (3) your application requires vertical spatial reasoning (above/below) explicitly—the Relative Position Prediction task ignores vertical displacement due to perspective ambiguity, and depth-free tasks do not teach vertical spatial relationships beyond what is implicit in 2D layout understanding; (4) you lack access to RGB-D data and need to train exclusively on RGB—the depth-based tasks (which contribute to 3D reasoning improvements) require sensor depth maps, and while depth-free tasks alone help (Table 4), the gains are smaller without depth supervision; monocular depth estimation could substitute for sensor depth but would violate the tool-free principle and introduce estimation errors into the ground truth.