Agents2026~126 minarXiv:2601.13976

FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation

score 3Jing Zuo, Lingzhou Mu, Fan Jiang +3Alibaba AMAP CV Lab↑ 22★ 16Jan 20, 2026cs.CVhttps://github.com/Fantasy-AMAP/fantasy-vln

🎯 Pitch

FantasyVLN trains VLN agents to reason with explicit chain-of-thought across text, imagined vision, and multimodal modes, but sheds this token bloat entirely at inference—achieving an order-of-magnitude speedup while surpassing all explicit CoT baselines on long-horizon navigation tasks. By compressing costly visual imaginations into a compact latent space during training, the model internalizes complex reasoning patterns without sacrificing real-time responsiveness.

1. Executive Summary

This paper introduces FantasyVLN, a unified implicit reasoning framework for Vision-and-Language Navigation that trains with explicit Chain-of-Thought reasoning across textual, visual, and multimodal modes but performs inference without generating CoT tokens, eliminating the latency penalty of explicit CoT while preserving reasoning-aware behavior. Using Qwen2.5-VL as the base model and evaluating on the LH-VLN benchmark — a multi-stage, long-horizon navigation dataset — the framework integrates Textual CoT (decomposing instructions into semantic plans), Compact Visual CoT (encoding imagined future observations into a compressed latent space via a pretrained Visual AutoRegressor to avoid token inflation), and Multimodal CoT (joint textual–visual reasoning), all unified under a gating mechanism with a cross-mode alignment constraint that forces CoT-driven and direct action predictions to converge to shared representations. At inference, the model maps instructions directly to actions, achieving 2.44 SR and 11.01 ISR on LH-VLN — surpassing all explicit CoT baselines while delivering 1.03 actions per second (~5× faster than explicit CoT-VLA's 0.19 APS) and reducing inference latency by an order of magnitude, establishing that complex multimodal reasoning patterns can be internalized into compact latent representations and deployed without token overhead during inference.

2. Context and Motivation

The fundamental tension this paper addresses is a classic speed-versus-intelligence tradeoff in embodied AI. Vision-and-Language Navigation (VLN) requires an agent to process natural language instructions while interpreting visual observations and planning actions in real time — the agent cannot pause for seconds to "think" before each step because the physical world (or its simulator) keeps moving. This real-time constraint creates a hard ceiling on how much computation can be expended per decision.

Chain-of-Thought (CoT) reasoning has emerged as a powerful technique for improving complex decision-making in language models, and recent works have extended it to VLN with promising results. Models like NavCoT (Lin et al., 2025b) and NavGPT-2 (Zhou et al., 2024) generate explicit step-by-step textual reasoning — decomposing instructions into subgoals, describing visual observations in natural language, and articulating action justifications — before predicting actions. This produces more interpretable and often more accurate navigation behavior because the intermediate reasoning steps provide structured guidance for action prediction.

However, the paper identifies a critical practical barrier: explicit CoT generation is catastrophically slow for real-time navigation. A typical reasoning step spanning 5–7 actions expands into 3,000–5,000 tokens of generated CoT content, compared to fewer than 500 tokens for purely textual CoTs in non-visual domains. This is not merely a linear slowdown — it represents an order-of-magnitude inflation in sequence length that makes each decision step take seconds rather than milliseconds. The paper quantifies this starkly in Table 4: explicit CoT methods like CoT-VLA achieve only 0.19 actions per second (APS), while implicit methods approach 1 APS. For a long-horizon navigation task requiring hundreds of actions, an explicit CoT agent would be impractically slow even on high-end hardware.

This problem becomes more severe, not less, as CoT methods grow more sophisticated. Multimodal CoT approaches — which the paper argues are necessary for true spatial-semantic integration — generate both textual reasoning and imagined visual observations at each step. CoT-VLA (Zhao et al., 2025), VISTA (Huang et al., 2025), and similar works predict future visual frames as part of their reasoning chain, producing thousands of additional visual tokens per step. The token inflation compounds: textual reasoning tokens plus imagined observation tokens plus action tokens per step, multiplied across dozens or hundreds of steps in long-horizon navigation, creates a sequence length explosion that is fundamentally incompatible with real-time deployment.

Why This Matters: The Gap Between Cognitive and Reactive Systems

This problem is significant for both practical and conceptual reasons. On the practical side, VLN has direct applications in assistive robotics (guiding visually impaired users through unfamiliar buildings), warehouse automation, drone navigation, and service robots. In all these domains, the agent must act in real time — a robot that pauses for 5 seconds to reason before each step is not just inefficient; it is non-functional in dynamic environments with moving obstacles, changing conditions, or time-sensitive tasks.

The LH-VLN benchmark that the paper uses specifically targets multi-stage, long-horizon navigation where these tensions are most acute. The agent must sequentially complete multiple subgoals (e.g., "go to the bedroom, find the toilet paper, then bring it to the bathroom"), maintain awareness of progress through extended trajectories, and avoid the accumulation of small errors across dozens of action steps. These demands make reasoning particularly valuable — you need to plan at a high level to avoid getting lost or forgetting which subgoal you are pursuing — but they also amplify the cost of explicit CoT because each step's reasoning must be generated from scratch.

On the conceptual side, the problem exposes a fundamental limitation in how the field thinks about reasoning in embodied agents. The dominant paradigm — generate explicit reasoning tokens, then condition actions on those tokens — treats reasoning as an explicit sequence generation process that is separable from action prediction. This works for offline tasks (solving math problems, answering questions) where latency is irrelevant, but it breaks down when milliseconds matter. The paper argues that this separation is artificial: a sufficiently trained model should be able to internalize reasoning patterns so that action predictions implicitly encode the same structured decision-making without generating intermediate tokens. This is the conceptual leap from "reasoning as generation" to "reasoning as representation."

Prior Approaches and Their Limitations

The paper organizes prior work into three categories, each with specific weaknesses that FantasyVLN is designed to address:

Purely textual CoT methods (e.g., NavCoT, NavGPT-2, Aux-Think). These methods generate natural language reasoning steps — semantic plans, visual descriptions, action justifications — before predicting actions. The key insight from Aux-Think (Wang et al., 2025) is particularly relevant: it introduced the "train-with-CoT, infer-without-CoT" paradigm, showing that a model trained to generate textual CoT during training can perform direct action prediction at inference while retaining some reasoning benefits. This established the feasibility of implicit reasoning but left two major gaps. First, it operates purely in the textual modality, meaning visual-spatial reasoning (which is central to VLN) must be awkwardly translated into captions and descriptions, losing the rich geometric and spatial information in raw visual observations. Second, it provides no mechanism for multimodal reasoning — the agent cannot imagine future visual states or reason about spatial transformations in a modality-appropriate way. As the paper notes, this limitation is "compounded by the difficulty of annotating CoT supervision in VLN," since human-annotated reasoning traces tend to overfit specific navigation paths and generalize poorly to unseen environments.

Visual CoT methods (e.g., CoT-VLA, WorldVLA, VISTA). These methods generate imagined future visual observations as part of the reasoning chain, typically by predicting pixel-level images of what the agent expects to see after executing candidate actions. This addresses the spatial grounding problem — the model reasons directly in the visual modality — but introduces the token inflation problem in its most severe form. A single imagined observation frame can require thousands of visual tokens to generate, and when this is done at every reasoning step across a long-horizon trajectory, the total sequence length becomes unmanageable. Table 1 quantifies different compression approaches: standard VAE achieves only 64× compression (producing 1,024 tokens for a 256×256 image), while VQ-VAE achieves similar ratios. Even with these compression methods, generating imagined visual observations at each step remains far too expensive for real-time use. Worse, the paper reports in Figure 5 that visual CoT methods like WorldVLA exhibit "slow and unstable convergence" during training, requiring over 10,000 iterations to reach moderate token prediction accuracy. This is attributed to the weak gradient signal from pixel-level reconstruction — the loss function penalizes the model for imperfect visual predictions, but the relationship between pixel accuracy and navigation success is indirect and noisy.

Multimodal CoT methods (emerging but nascent). The paper positions FantasyVLN as the first unified multimodal CoT framework for VLN, but acknowledges that the broader vision-language community has begun exploring joint textual-visual reasoning chains — for example, methods that generate paired textual descriptions and visual sketches as intermediate steps. These approaches face the worst of both worlds: they incur the token cost of both textual reasoning and visual imagination, and they must deal with the alignment problem of ensuring the two modalities are consistent (a textual plan that says "turn right" paired with a visual imagination that shows a leftward view would be actively harmful). The paper cites Visual Thoughts (Cheng et al., 2025) as providing a unified perspective on multimodal CoT, but notes that such methods have not been adapted to the real-time constraints of embodied navigation.

Non-CoT methods (MGDM, GPT-4 + NaviLLM, GLM-4v). Table 2 shows that methods without any explicit reasoning achieve near-zero success rates on LH-VLN. MGDM, a memory-based method, achieves 0 SR and 2.34 ISR. GPT-4 + NaviLLM achieves 2.19 ISR but still 0 SR. Random agents and zero-shot prompted baselines score 0 across all metrics. This establishes that the LH-VLN benchmark is genuinely difficult — pure pattern matching or instruction following without reasoning fails completely — and that some form of structured reasoning is necessary. However, these results also demonstrate that memory mechanisms alone (storing and retrieving past observations) are insufficient; what is needed is reasoning that integrates semantic plans with spatial understanding.

How This Paper Positions Itself

The paper positions FantasyVLN as a unified implicit reasoning framework that resolves the tension between reasoning depth and inference speed by moving the reasoning computation to training time. The key conceptual move is: instead of generating explicit CoT tokens at inference, train the model on diverse CoT reasoning patterns during training and then force those patterns to be absorbed into the model's internal representations through a cross-mode alignment constraint. At inference, the model performs direct instruction-to-action mapping, but the representations driving that mapping have been shaped by CoT supervision across multiple modalities.

This positioning builds on several intellectual threads while making novel contributions:

From Aux-Think, FantasyVLN inherits the train-with-CoT, infer-without-CoT paradigm, but generalizes it from purely textual CoT to a multimodal setting with three distinct CoT modes (textual, visual, multimodal) operating under shared parameters. The paper explicitly cites Aux-Think's approach and extends it.
From visual CoT methods, FantasyVLN retains the idea of imagined visual observations as a reasoning mechanism but eliminates the token inflation problem by compressing imagined observations into a compact latent space (30 tokens vs. thousands) using a pretrained Visual AutoRegressor model. This is the essential technical move that makes multimodal reasoning training tractable.
From multimodal learning, FantasyVLN introduces a gating mechanism and cross-mode alignment constraint that forces the action predictions from different reasoning modes to converge, effectively distilling multimodal reasoning patterns into a single, modality-agnostic policy representation. This is the mechanism by which reasoning becomes implicit — the model learns that whether it reasoned textually, visually, or both, the resulting action should be the same, so the intermediate reasoning becomes a redundant pathway that can be pruned at inference.

The paper's most distinctive claim is that the three reasoning modes (textual, visual, multimodal) are complementary and that training with all three produces better implicit reasoning than any single mode alone. Table 3 directly tests this: adding T-CoT to non-CoT training improves ISR from 2.01 to 8.26; adding V-CoT improves it to 11.19; combining all four modes (non-CoT + T-CoT + V-CoT + MM-CoT) achieves the best ISR of 11.01. This ablation demonstrates that visual reasoning contributes something textual reasoning cannot capture (likely spatial-geometric understanding) and vice versa, and that their combination in MM-CoT provides further benefits. The cross-mode alignment constraint is essential for making this mode combination work — without it (Table 5), the unified framework collapses to near-zero performance, suggesting that without explicit alignment, the different reasoning modes produce conflicting gradient signals that prevent coherent policy learning.

The paper also positions itself relative to the scalability challenge in VLN. The LH-VLN benchmark provides only 18,000 trajectory slices for training (each slice contains 5 action steps as described in Appendix A), which is a relatively small dataset for training complex multimodal reasoning. The paper argues that explicit CoT methods are particularly vulnerable to overfitting under these data constraints because they must learn to generate coherent reasoning chains, not just correct actions. Implicit reasoning sidesteps this by making the CoT supervision a regularizer during training rather than a requirement at inference — the model can benefit from the structured supervision without being penalized for imperfect reasoning generation at test time. This is supported by the explicit-versus-implicit comparison in Table 6: for every CoT modality, implicit inference outperforms explicit inference, with the gap being largest for MM-CoT (2.44 SR implicit vs. 0.98 SR explicit).

In summary, the paper addresses a clear and well-motivated gap: the real-time requirements of embodied navigation are fundamentally incompatible with the token generation overhead of explicit CoT reasoning, and existing attempts to resolve this either sacrifice reasoning quality (by staying in a single modality) or sacrifice speed (by generating pixel-level visual imaginations). FantasyVLN proposes to have it both ways — multimodal reasoning depth during training, real-time action prediction during inference — through the combination of compact visual representations, unified multimodal training, and cross-mode alignment.

3. Technical Approach

3.1 Reader Orientation

FantasyVLN is a training framework — not a novel model architecture — that teaches a vision-language-action model to internalize complex multimodal reasoning patterns so that it can navigate efficiently at inference time without generating explicit reasoning steps. The problem it solves is that explicit Chain-of-Thought reasoning (generating intermediate plans, visual descriptions, and imagined future observations before predicting each action) is far too slow for real-time navigation — explicit CoT methods produce only 0.19 actions per second versus the 1+ APS needed for practical deployment — and purely textual CoT misses crucial spatial-geometric information. The solution's shape is: train with expensive, multimodal CoT supervision across textual, visual, and combined reasoning modes, but use a cross-mode alignment constraint to distill those reasoning patterns into the model's internal representations so that at inference time, the model maps instructions directly to actions in a single forward pass, achieving reasoning-aware behavior with no token overhead.

3.2 Big-Picture Architecture (Diagram in Words)

FantasyVLN consists of five major components, all sharing a single VLM (Qwen2.5-VL) as the underlying neural network:

The Base Vision-Language-Action Model (πθ): A Qwen2.5-VL model fine-tuned with LoRA that takes as input an instruction, a sequence of visual observations (current multi-view images plus history), and two binary gating signals, and produces either: (a) actions directly, (b) textual reasoning then actions, (c) compressed visual reasoning then actions, or (d) multimodal reasoning then actions. At inference, only path (a) is used.
The Gating Mechanism: Two binary switches — gT (textual reasoning on/off) and gV (visual reasoning on/off) — encoded as special tokens in the model's input vocabulary. These gates determine which mode the model operates in for a given training example, and their combinations (0,0), (1,0), (0,1), (1,1) correspond to non-CoT, T-CoT, V-CoT, and MM-CoT modes respectively.
The Visual AutoRegressor (VAR) Compressor: A frozen pretrained model that encodes 256×256 images into a hierarchical latent representation. At scale 4 (the optimal setting identified in Figure 3), this produces only 30 discrete latent tokens while preserving sufficient visual information for reconstruction (as shown in Figure 4). During V-CoT and MM-CoT training, the VLM predicts these 30 latent tokens rather than pixel-level images; during inference, the VAR is not used at all.
The Textual CoT Annotation Pipeline: An external large VLM (Qwen-VL-Max) that pre-processes all training data to generate structured textual reasoning traces — semantic plans, visual descriptions, action justifications, and visual imaginations — following a detailed annotation prompt template. These serve as ground-truth targets for T-CoT training.
The Cross-Mode Alignment Constraint: A training mechanism that computes action predictions from each CoT mode, then penalizes divergence between those predictions and the non-CoT mode's predictions. This forces all reasoning pathways to converge to the same action outputs, enabling the model to eventually skip the reasoning entirely while retaining its benefits.

Information flow during training: A trajectory slice (instruction + historical images + current multi-view images + ground-truth actions + pre-computed T-CoT text + ground-truth future images for V-CoT targets) enters the system → the gating signals are randomly sampled to select a reasoning mode → if gV=1, the target future image is compressed through the frozen VAR encoder to produce a 30-token latent target → the VLM generates the appropriate output sequence (actions only, text+actions, latent tokens+actions, or text+latent tokens+actions) → three loss components are computed: (1) cross-entropy on action prediction, (2) cross-entropy on CoT token prediction (text and/or visual latents), and (3) cross-entropy alignment loss between each CoT mode's action predictions and the non-CoT mode's action predictions (via its soft targets) → parameters are updated via alternating optimization: first a non-CoT-only step, then a joint step with alignment.

Information flow during inference: Instruction + visual observations enter the model → a single forward pass with gT=0, gV=0 → the model autoregressively generates action tokens only (e.g., <|forward|> <|left|> <|forward|> <|right|> <|stop|>) → actions are executed in the simulator → new observations arrive → repeat. The VAR is never invoked, no CoT tokens are generated, and the reasoning is entirely implicit in the model's internal representations.

3.3 Roadmap for the Deep Dive

First, the problem formulation (Section 3.2): how VLN is formalized as a sequential decision problem, what the agent receives as input at each timestep, and what probabilities the model estimates. This establishes the basic vocabulary.
Second, the Compact Visual CoT mechanism (Section 3.3): how the VAR model compresses images into 30 latent tokens and why this is the essential technical fix for the token inflation problem. This must be understood before V-CoT or MM-CoT make sense, because it is the mechanism that makes visual reasoning training tractable.
Third, the three CoT reasoning modes individually (Sections 3.4): what T-CoT, V-CoT, and MM-CoT each generate, how their training targets are constructed, and what role they play. This builds on CompV-CoT for the visual modes.
Fourth, the unified gating mechanism (Section 3.4): how the four modes are combined under shared parameters using binary gating signals and joint training with a data mixture. This is where the framework becomes "unified" — the same model handles all modes.
Fifth, the cross-mode alignment constraint (Section 3.5): the training algorithm that forces action predictions from all CoT modes to match the non-CoT mode's predictions, including the alternating optimization procedure. This is the mechanism that enables implicit reasoning at inference.
Sixth, the inference procedure (Section 3.6): what happens at deployment — why non-CoT mode is used, what tokens are generated, and how the system achieves real-time performance. This closes the loop from training to deployment.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a training methodology paper whose core idea is that multimodal CoT reasoning can be internalized into a single vision-language-action model's latent representations through joint training with cross-mode alignment, enabling the model to perform reasoning-aware navigation without generating explicit reasoning tokens at inference. The technical contribution consists of five integrated mechanisms: (1) compact visual reasoning via VAR latent space prediction, (2) annotation and integration of textual CoT traces, (3) a gating mechanism that unifies four reasoning modes under shared parameters, (4) a cross-mode alignment loss that distills reasoning patterns into implicit representations, and (5) an alternating optimization procedure that prevents mode interference. Below, the VLN problem is formalized, then each mechanism is decomposed in sequence.

Problem Formulation: VLN as a Sequential Decision Process

The paper formalizes Vision-and-Language Navigation as a non-Markovian temporal decision problem. This classification — non-Markovian rather than Markovian — is important because it captures the fact that the agent's optimal action depends not just on the current observation (as in a Markov Decision Process) but on the full history of observations, the instruction, and the agent's implicit understanding of which subgoal it is currently pursuing.

Let $s_0$ denote the initial state, which includes the agent's starting location and orientation in the 3D environment. Let $\mathcal{U}$ denote the action space — the set of discrete navigation primitives the agent can execute. In this paper, the action space consists of four tokens: <|forward|>, <|left|>, <|right|>, and <|stop|>. At each discrete timestep $t$ , the agent receives multi-view visual observations $o_t \in \mathcal{O}$ — specifically, three images corresponding to left, front, and right camera views at the agent's current position and heading. The agent's decision-making policy, denoted $\pi_\theta$ with parameters $\theta$ , conditions on both the instruction $I$ (a natural language description of the task) and the full history of observations $\{o_{\leq t}\}$ up to the current timestep to predict a sequence of future actions $A_t \in \mathcal{U}$ :

The agent $\pi_\theta$ receives multi-view visual observations $o_t \in \mathcal{O}$ and predicts future actions $A_t \in \mathcal{U}$ conditioned on the instruction $I$ and historical observations $\{o_{\leq t}\}$ .

The predicted actions are executed in the environment, which updates the agent's state according to environment dynamics (movement, collision detection, goal proximity checking). This process repeats until either a <|stop|> action is predicted or the maximum number of steps $T$ is reached. In multi-stage navigation (the LH-VLN setting), a <|stop|> triggers a subtask completion check: if the agent is within 1 meter of the current subtask's target location, the next subtask is activated; otherwise, the subtask is marked as failed.

Crucially, the paper trains the model to predict multiple future actions at once — specifically, $k = 5$ actions per prediction step (as described in Appendix A.1). Each training slice consists of the instruction, the historical and current visual observations, and the next 5 ground-truth actions. This means that at each inference step, the model outputs up to 5 action tokens autoregressively, and the environment executes them one by one before the next observation is received. This design choice — predicting a short horizon rather than one action at a time or the full trajectory — balances the need for planning ahead against the accumulating uncertainty of long-range predictions. It also means that the CoT reasoning generated during training provides guidance for 5 actions at once, amortizing the reasoning cost over multiple execution steps.

Compact Visual Chain-of-Thought (CompV-CoT)

This mechanism is the essential technical contribution that makes visual CoT training tractable. Without it, visual reasoning would require generating pixel-level imagined observations at each step, producing thousands of visual tokens and making training both slow and unstable (as the paper demonstrates with WorldVLA in Figure 5).

The token inflation problem quantified. When a model generates an imagined future observation as part of its reasoning chain, it must produce visual tokens autoregressively — one token at a time, with each token conditioned on all previous tokens. For a standard image generation approach, a 256×256 image might require: 256 latent tokens with a VAE (64× compression), 1,024 tokens with VQ-VAE (64× compression), or 16,384 tokens with pixel-level prediction (1× compression, i.e., no compression at all). Each of these tokens requires a forward pass through the transformer decoder, making the generation process $O(N_{\text{tokens}} \times L_{\text{model}})$ in computational cost. When this is done at every reasoning step — and a long-horizon navigation trajectory might involve hundreds of such steps — the total cost becomes prohibitive.

Table 1 compares different compression approaches. RAE-DINOv2-B achieves 256× compression (producing 64 tokens for a 256×256 image) with 0.012 MSE reconstruction error. RAE-SigLIP2-B achieves the same 256× compression with slightly better 0.011 MSE. VAE and VQ-VAE achieve only 64× compression (1,024 tokens) with 0.005 and 0.007 MSE respectively. The critical entry is VAR: 1/2185 compression ratio — meaning a 256×256 image is compressed into approximately 30 tokens — with 0.039 MSE. The MSE is higher than the other methods (0.039 vs. 0.005–0.012), but the compression ratio is more than 8× better than the next-best approach, making it uniquely suitable for real-time applications where sequence length is the dominant cost.

Why VAR specifically. The Visual AutoRegressive model (Tian et al., 2024) introduced a "next-scale prediction" paradigm that differs fundamentally from standard autoregressive image generation. Rather than predicting individual pixels or latent tokens one at a time in raster-scan order, VAR predicts entire feature maps at progressively finer scales. At scale 1, VAR predicts a coarse 1×1 representation; at scale 2, it predicts a 2×2 representation conditioned on scale 1; at scale 4 (the setting used in this paper), it predicts a 4×4 representation; and so on up to the finest scale. The key property is that the early, low-scale representations are extremely compact — scale 4 produces only 16 tokens for that scale, and the total across scales 1–4 is approximately 30 tokens — yet contain sufficient high-level structural information to reconstruct the full image when processed through the VAR decoder.

The paper formulates CompV-CoT as follows. Instead of predicting pixel-level images $\hat{V}_t$ directly, the VLM predicts low-scale latent representations $\hat{H}_t$ of the VAR model:

$[\hat{H}_t, \hat{A}_t] \sim \pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 1)$

where $\hat{H}_t$ is the predicted set of latent tokens corresponding to VAR scales 1 through the chosen scale (scale 4 in the optimal configuration), and $\hat{A}_t$ is the predicted action sequence. The predicted latent tokens are then passed through the frozen VAR decoder $g$ to reconstruct pixel observations if needed:

$\hat{V}_t \sim g(\hat{H}_t)$

where $g$ denotes the VAR generation pipeline that takes the low-scale latents as input and predicts the remaining higher-scale latents via next-scale prediction, then decodes the full latent hierarchy into a pixel image.

What this computes, operationally. During a V-CoT training step, the system takes an instruction and a sequence of visual observations, and compresses the ground-truth future observation (the image the agent will actually see after executing the next 5 actions) through the frozen VAR encoder to produce a ground-truth latent target $V_t$ — a sequence of approximately 30 discrete tokens representing the hierarchical visual information. The VLM is then trained to generate those 30 tokens autoregressively (predicting one token at a time, conditioned on all previous predicted tokens and the full input context) followed by the 5 action tokens. The loss is standard causal cross-entropy on both the latent tokens and the action tokens. At inference in explicit V-CoT mode, the model would generate the 30 latent tokens, optionally decode them through the VAR to visualize the imagined observation (for interpretability), and then condition its action predictions on those tokens. In implicit mode, this entire generation pathway is skipped.

Why this form — the design tradeoffs. The choice of VAR over other compressors is driven by the extreme compression ratio (1/2185 vs. 1/64–1/256 for alternatives). The higher MSE (0.039) is an acceptable tradeoff because the goal is not photorealistic image reconstruction but rather capturing sufficient spatial-semantic information to guide action prediction — the model needs to know "there will be a doorway on the right after turning" not the exact pixel colors of the doorframe. The VAR's hierarchical latent structure is particularly well-suited for this because the low-scale representations capture global scene layout and semantic content (room type, presence of doorways, furniture arrangement) while discarding fine-grained texture details that are irrelevant for navigation decisions. This is visible in Figure 4: at scale 4, the reconstructed image captures the overall room geometry and major objects but lacks texture detail; at higher scales, details progressively fill in, but Figure 3 shows that scale 4 actually achieves the best ISR (7.41) compared to both lower scales (0–3.76) and higher scales (down to 0.90 at scale 10), suggesting that too much visual detail introduces noise that interferes with reasoning.

Freezing the VAR. The VAR model is completely frozen during training — its parameters are never updated. This is a crucial design choice for three reasons: (1) it prevents the VLM's training objective from distorting the VAR's latent space, which would break the compression property; (2) it keeps training stable since the VAR's reconstruction targets (the ground-truth images) provide a fixed reference for latent token prediction; and (3) it means the VAR is not needed at inference time at all — the VLM learns to predict and use latent representations that happen to be decodable by the VAR, but the decoder is only used for qualitative visualization, not for navigation decisions. The training objective is:

$\arg\min_\theta \mathcal{L}_{\text{CE}}(\hat{V}_t, V_t) + \mathcal{L}_{\text{CE}}(\hat{A}_t, A_t)$

where $\mathcal{L}_{\text{CE}}$ is causal cross-entropy loss, $\hat{V}_t$ are the predicted latent tokens (which, when decoded through the frozen VAR, reconstruct imagined observations), $V_t$ are the ground-truth latent tokens (obtained by encoding the actual future observation through the frozen VAR), $\hat{A}_t$ are the predicted actions, and $A_t$ are the ground-truth actions.

Textual Chain-of-Thought (T-CoT)

Textual CoT serves as the semantic planning component of the reasoning framework. While visual CoT captures spatial-geometric information (what the agent expects to see), textual CoT captures high-level task decomposition (what subgoals to pursue, how to assess progress, what landmarks to look for).

Structure of T-CoT annotations. The paper uses Qwen-VL-Max — an external, larger vision-language model — to generate structured textual reasoning traces for all 18,554 training slices. The annotation follows a specific four-part template (detailed in Appendix A.3):

Semantic Planning: Decompose the instruction into precise sub-tasks with spatial goals (e.g., "reach the cabinet table in the living room," "exit the living room," "enter the office," "approach the office table"). The prompt explicitly instructs the annotator to produce "stepwise navigation milestones rather than abstract summaries."
Visual Description: Based on the semantic plan, describe what the historical and current images reveal about completed and upcoming sub-tasks ("The historical images show that I have exited the bedroom and am now in a hallway. The current front view shows a living room entrance approximately 3 meters ahead").
Action Decision-Making: Predict the next action sequence (5 steps) aligned with the semantic plan and supported by the visual description, with a brief justification ("I should move forward three times to approach the living room entrance, then turn slightly right to align with the doorway").
Visual Imagination: Describe the expected scene after executing the predicted actions, focusing on landmarks or key objects that should become visible or reachable ("After moving forward, the living room entrance should fill the front view, and a cabinet table should be visible on the left").

The complete reasoning trace $T_t$ is the concatenation of these four components into a single structured text sequence. During training, when $g_T = 1$ and $g_V = 0$ , the model generates this text autoregressively before predicting actions:

$[\hat{T}_t, \hat{A}_t] \sim \pi_\theta(I, \{o_{\leq t}\}, g_T = 1, g_V = 0)$

where $\hat{T}_t$ is the predicted textual reasoning trace and $\hat{A}_t$ is the predicted action sequence.

Training objective for T-CoT. The loss has two components — one for the reasoning trace quality and one for the action prediction:

$\arg\min_\theta \mathcal{L}_{\text{CE}}(\hat{T}_t, T_t) + \mathcal{L}_{\text{CE}}(\hat{A}_t, A_t)$

where $\mathcal{L}_{\text{CE}}$ is causal cross-entropy, $\hat{T}_t$ are the model's predicted reasoning tokens, $T_t$ is the Qwen-VL-Max-generated ground-truth reasoning trace, $\hat{A}_t$ are the predicted action tokens, and $A_t$ are the ground-truth actions.

What this computes, operationally. During a T-CoT training step, the system takes an instruction and visual observations, feeds them to the VLM with $g_T = 1, g_V = 0$ , and the model autoregressively generates: first, a sequence of text tokens representing the four-part reasoning trace (typically a few hundred tokens, as noted in Section 1: "purely textual CoTs usually <500 tokens"), then a sequence of up to 5 action tokens. The cross-entropy loss is computed across all generated tokens — both text and actions — meaning the model is penalized both for generating poor reasoning and for predicting incorrect actions given that reasoning. This is a standard teacher-forcing setup: the ground-truth reasoning is provided as context for predicting each subsequent token, so the model learns the joint distribution of reasoning and actions.

Why this form — the relationship to pure action prediction. The key insight behind this training approach is that the textual reasoning traces provide structured causal guidance for action prediction. Rather than learning a direct mapping from pixels and instructions to actions — which is a black-box function with many spurious correlations — the model first learns to articulate its understanding of the situation (what subgoal am I on? what do I see? what should happen next?) and then learns to map that articulated understanding to actions. This structure acts as a kind of bottleneck that forces the model to compress the high-dimensional visual and linguistic input into a semantically meaningful intermediate representation before committing to actions. The paper's results (Table 3) confirm that adding T-CoT to non-CoT training improves ISR from 2.01 to 8.26, supporting the claim that the textual reasoning structure provides beneficial inductive bias.

A subtle but important design choice: offline annotation rather than online generation. The paper uses a separate, larger model (Qwen-VL-Max) to pre-generate all textual reasoning traces offline. This is not the same as having the model generate its own reasoning during training (which would be more expensive and potentially less stable, since the model's reasoning quality would evolve during training). The offline approach means that the T-CoT targets are fixed, high-quality references that the model learns to imitate. However, it also means that the model is learning to reason in the style of Qwen-VL-Max, which may differ from its own natural reasoning patterns — a potential source of distribution shift that the cross-mode alignment constraint is designed to mitigate.

Multimodal Chain-of-Thought (MM-CoT)

MM-CoT is the native combination of textual and visual reasoning modes. The agent generates paired textual–visual reasoning steps where each reasoning component provides complementary information: the textual reasoning handles semantic decomposition and progress tracking, while the visual reasoning handles spatial anticipation and geometric verification.

Construction of multimodal reasoning traces. The multimodal reasoning trace $M_t$ is defined as the concatenation of the textual reasoning trace $T_t$ (the four-part annotation from Qwen-VL-Max) and the compressed visual latent tokens $H_t$ (obtained by encoding the ground-truth future observation through the frozen VAR encoder):

$M_t = [T_t, H_t]$

This is a sequential concatenation, not a parallel one — the model first generates the full textual reasoning trace, then generates the 30 VAR latent tokens, then generates the action tokens. The ordering matters because the textual reasoning provides semantic context that can guide the visual imagination: after reasoning textually that "I should see a doorway on the right," the model is better positioned to imagine a visual observation consistent with that expectation.

During training, when $g_T = 1$ and $g_V = 1$ , the model generates:

$[\hat{M}_t, \hat{A}_t] \sim \pi_\theta(I, \{o_{\leq t}\}, g_T = 1, g_V = 1)$

where $\hat{M}_t = [\hat{T}_t, \hat{H}_t]$ is the predicted multimodal reasoning trace. The training objective is:

$\arg\min_\theta \mathcal{L}_{\text{CE}}(\hat{M}_t, M_t) + \mathcal{L}_{\text{CE}}(\hat{A}_t, A_t)$

where $\mathcal{L}_{\text{CE}}(\hat{M}_t, M_t)$ decomposes into cross-entropy on the textual tokens (matching the Qwen-VL-Max annotations) and cross-entropy on the visual latent tokens (matching the VAR-encoded ground-truth observations). The two components are not separately weighted — they are treated as a single sequence and the standard causal language modeling loss is applied uniformly.

What this computes, operationally. During an MM-CoT training step, the system constructs a target sequence by concatenating: (1) the Qwen-VL-Max-generated textual reasoning trace (hundreds of tokens), (2) the 30 VAR latent tokens from the ground-truth future observation, and (3) the 5 ground-truth action tokens. The VLM, conditioned on the instruction and visual observations with both gates set to 1, generates this entire sequence autoregressively. The loss penalizes deviation from the target at every position, so the model learns to: produce coherent textual reasoning, produce visual latent tokens that are consistent with both the textual reasoning and the actual future observation, and produce actions that are consistent with both reasoning modalities.

Why this form — the complementarity hypothesis. The underlying hypothesis, which the ablations in Table 3 support, is that textual and visual reasoning provide complementary information that neither modality alone captures fully. Textual CoT excels at: decomposing complex instructions into subgoals ("first find the toilet paper, then go to the bathroom"), tracking which subgoal is currently active, and describing semantic properties of the environment ("the living room has a blue sofa"). Visual CoT excels at: anticipating spatial layouts ("after turning right, a doorway will appear on the left at approximately 2 meters"), verifying that planned actions are geometrically feasible, and providing dense spatial information that is difficult to articulate in words (the exact visual appearance of a landmark). By combining both, the model can reason about both the "what" (semantic content) and the "where" (spatial geometry) of the navigation task.

However, MM-CoT also incurs the highest training cost: the model must generate both hundreds of text tokens and 30 visual latent tokens before predicting actions, making each MM-CoT training example significantly more expensive than T-CoT or V-CoT alone. This is why the paper studies all four modes in combination — the gating mechanism allows the model to benefit from multimodal reasoning during training without requiring it for every example, amortizing the cost.

The Unified Gating Mechanism

The gating mechanism is what transforms three separate reasoning modes (T-CoT, V-CoT, MM-CoT) plus a non-reasoning mode into a single unified framework. Without it, training on multiple reasoning modes would require either separate models or some form of multi-task learning with mode-specific output heads. The gating mechanism allows a single set of parameters to handle all four modes by conditioning the model's behavior on two binary input tokens.

Gating signals as special vocabulary tokens. The paper introduces four special tokens that serve as the binary gating signals: <textual think> and <no textual think> for $g_T = 1$ and $g_T = 0$ respectively, and <visual think> and <no visual think> for $g_V = 1$ and $g_V = 0$ respectively. These tokens are added to the model's vocabulary (leveraging Qwen2.5-VL's vocabulary extensibility) and are embedded like any other token. They are prepended to the input sequence, so the model "reads" which reasoning mode it should operate in before processing the instruction and observations.

The complete input format for a given mode is: [<gating_tokens>] [instruction] [historical images] [current left image] [current front image] [current right image], where the gating tokens are one of four possible pairs: (<no textual think>, <no visual think>) for non-CoT, (<textual think>, <no visual think>) for T-CoT, (<no textual think>, <visual think>) for V-CoT, or (<textual think>, <visual think>) for MM-CoT.

Formal specification of mode-dependent outputs. The unified policy is defined as:

$[\hat{R}_t, \hat{A}_t] = \pi_\theta(I, \{o_{\leq t}\}, g_T, g_V)$

where $I$ is the instruction, $\{o_{\leq t}\}$ are the visual observations, $g_T, g_V \in \{0, 1\}$ are the gating signals, $\hat{R}_t$ is the predicted reasoning trace (which depends on the mode), and $\hat{A}_t$ is the predicted action sequence. The reasoning trace $\hat{R}_t$ is defined case-wise:

$\hat{R}_t = \begin{cases} \text{None} & \text{if } (g_T, g_V) = (0, 0), \\ \hat{T}_t & \text{if } (g_T, g_V) = (1, 0), \\ \hat{V}_t & \text{if } (g_T, g_V) = (0, 1), \\ \hat{M}_t & \text{if } (g_T, g_V) = (1, 1). \end{cases}$

where $\hat{T}_t$ is the predicted textual reasoning trace, $\hat{V}_t$ (in this notation) refers to the predicted VAR latent tokens (the paper uses $\hat{H}_t$ for latents and $\hat{V}_t$ for the decoded image, but Equation 2 uses $\hat{V}_t$ to mean the latent representation), and $\hat{M}_t = [\hat{T}_t, \hat{V}_t]$ is the concatenated multimodal trace.

What this computes in plain terms. Given an instruction, visual observations, and a mode selection (specified by the two gating tokens), the model generates the appropriate output sequence. If the mode is non-CoT (0,0), it generates only action tokens. If the mode is T-CoT (1,0), it generates a textual reasoning trace followed by actions. If V-CoT (0,1), it generates VAR latent tokens followed by actions. If MM-CoT (1,1), it generates textual reasoning, then VAR latent tokens, then actions. The model learns to interpret the gating tokens and adjust its generation behavior accordingly — essentially, the gating tokens function as a learned prompt prefix that activates different generation pathways.

Data mixture during training. For each training example in the LH-VLN dataset, the gating signals $(g_T, g_V)$ are uniformly sampled from the four possible configurations. This means that over the course of training, the model sees each mode approximately 25% of the time. The training data is organized into five-tuples (Equation 3):

$[I, \{o_{\leq t}\}, T_t, V_t, A_t] \in \mathcal{D}$

where $I$ is the instruction, $\{o_{\leq t}\}$ are the visual observations, $T_t$ is the Qwen-VL-Max-generated textual reasoning trace, $V_t$ is the VAR-encoded ground-truth visual latent representation, and $A_t$ is the ground-truth action sequence. For a given training step, depending on the sampled gating signals, the target sequence is constructed by selecting the appropriate subset of $\{None, T_t, V_t, M_t\}$ concatenated with $A_t$ , as specified by Equation 2.

Why this form — the benefits of shared parameters. Having all four modes share a single set of parameters (rather than being separate models or having mode-specific output heads) is critical for the implicit reasoning goal. When the model processes a T-CoT example, it updates parameters that are also used for non-CoT prediction. When it processes a V-CoT example, it updates the same parameters. This means that the representations learned for reasoning — the intermediate activations that encode semantic plans or spatial anticipations — are available to the non-CoT pathway even though the non-CoT pathway does not explicitly generate them. The cross-mode alignment constraint (described next) explicitly reinforces this sharing by penalizing divergence between modes, but even without it, parameter sharing provides a inductive bias toward learning shared representations.

The uniform sampling strategy is simple but has an important property: it prevents the model from overfitting to any single mode. If the model were trained on MM-CoT 90% of the time and non-CoT 10% of the time, the non-CoT pathway might be poorly optimized despite parameter sharing. Uniform sampling ensures that all modes receive equal optimization pressure.

The Cross-Mode Alignment Constraint

The cross-mode alignment constraint is the mechanism that enables implicit reasoning. Without it, training on multiple modes with shared parameters could lead to mode interference — the T-CoT pathway might learn one action policy, the V-CoT pathway a different one, and the non-CoT pathway yet another, with the shared parameters averaging these conflicting gradients. The alignment constraint forces all pathways to produce the same action predictions, which has two effects: (1) it resolves inter-mode conflicts, and (2) it distills the reasoning patterns from the CoT modes into the non-CoT pathway's internal representations, since the non-CoT pathway must learn to produce the same actions without access to the intermediate reasoning tokens.

The alignment mechanism in detail. The constraint works by treating the non-CoT mode's action predictions as a teacher signal and penalizing the CoT modes when their action predictions diverge from this teacher. Specifically, let:

$\hat{A}_t = \pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 0)$ be the non-CoT mode's action predictions,
$\hat{A}^T_t$ be the action predictions from the T-CoT mode,
$\hat{A}^V_t$ be the action predictions from the V-CoT mode,
$\hat{A}^M_t$ be the action predictions from the MM-CoT mode.

The alignment loss is:

$\mathcal{L}_{\text{Align}} = \mathcal{L}_{\text{CE}}(\hat{A}^T_t, \tilde{A}_t) + \mathcal{L}_{\text{CE}}(\hat{A}^V_t, \tilde{A}_t) + \mathcal{L}_{\text{CE}}(\hat{A}^M_t, \tilde{A}_t)$

where $\tilde{A}_t = \text{sg}(\pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 0))$ and $\text{sg}$ denotes the stop-gradient operator.

What this computes, operationally. First, the non-CoT mode generates action logits $\hat{A}_t$ through a standard forward pass. These logits are converted to soft targets $\tilde{A}_t$ — the probability distribution over action tokens — and the stop-gradient operator is applied, meaning that gradients from the alignment loss do not flow back into the non-CoT pathway's computation of $\tilde{A}_t$ . This is critical: if gradients flowed into $\tilde{A}_t$ , the alignment loss could collapse by making the non-CoT predictions match the CoT predictions (a degenerate solution) rather than the intended direction (CoT predictions matching non-CoT). Then, for each CoT mode, the model generates its own action logits, and the cross-entropy between those logits and the frozen soft targets is computed. This penalizes each CoT mode for predicting actions that differ from what the non-CoT mode would predict.

The stop-gradient operator and why it is essential. The $\text{sg}$ function in the paper is written explicitly in Algorithm 1 as $\tilde{A}_t \leftarrow \text{sg}(\pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 0))$ . The stop-gradient means that $\tilde{A}_t$ is treated as a constant during backpropagation — its value depends on the current parameters $\theta$ , but the gradient of the alignment loss with respect to $\theta$ does not include the gradient through $\tilde{A}_t$ . Without this, the system would have a circular optimization problem: the alignment loss would try to make $\hat{A}^T_t$ match $\tilde{A}_t$ , but simultaneously the non-CoT mode would be pushed to change $\tilde{A}_t$ to match the CoT predictions, potentially leading to oscillation or collapse. The stop-gradient breaks this circularity and establishes a clear optimization hierarchy: the non-CoT mode is the reference; the CoT modes adapt to it.

Why cross-entropy and not a symmetric divergence. The paper uses cross-entropy $\mathcal{L}_{\text{CE}}(\hat{A}^{\text{CoT}}_t, \tilde{A}_t)$ rather than a symmetric loss like KL divergence or mean squared error. Cross-entropy is asymmetric: it penalizes the CoT mode's predictions for deviating from the non-CoT mode's distribution, but does not penalize the non-CoT mode for deviating from the CoT mode's distribution. This asymmetry is intentional — the goal is to make the CoT modes converge to the non-CoT mode, not to find a compromise between them. Additionally, because $\tilde{A}_t$ comes from a non-CoT forward pass, it represents the "distilled" action that the model should produce without explicit reasoning, making it the natural target.

The Alternating Optimization Procedure

The paper does not simply add the alignment loss to the standard training objective and optimize everything jointly. Instead, it uses an alternating optimization procedure (Algorithm 1) that interleaves non-CoT-only optimization steps with joint alignment optimization steps. This design choice is motivated by the need to prevent the non-CoT mode's quality from degrading as the CoT modes evolve.

The two phases of each iteration. Algorithm 1 specifies the following procedure for each training iteration:

Phase 1 — Optimize non-CoT mode independently:

Sample a training example $[I, \{o_{\leq t}\}, T_t, V_t, A_t] \sim \mathcal{D}$ .
Compute the non-CoT action predictions: $\hat{A}_t \leftarrow \pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 0)$ .
Update parameters using only the non-CoT loss: $\theta \leftarrow \theta - \eta \nabla_\theta \mathcal{L}_{\text{CE}}(\hat{A}_t, A_t)$ .

Phase 2 — Optimize all CoT modes with alignment:

Recompute the non-CoT predictions with stop-gradient to obtain soft targets: $\tilde{A}_t \leftarrow \text{sg}(\pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 0))$ .
Compute predictions from each CoT mode: $[\hat{T}_t, \hat{A}^T_t] \leftarrow \pi_\theta(I, \{o_{\leq t}\}, g_T = 1, g_V = 0)$ , $[\hat{V}_t, \hat{A}^V_t] \leftarrow \pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 1)$ , $[\hat{M}_t, \hat{A}^M_t] \leftarrow \pi_\theta(I, \{o_{\leq t}\}, g_T = 1, g_V = 1)$ .
Compute the full joint loss with alignment:

$\mathcal{L}^*_{\text{Joint}} = \underbrace{\mathcal{L}_{\text{CE}}(\hat{A}^T_t, \tilde{A}_t) + \mathcal{L}_{\text{CE}}(\hat{A}^V_t, \tilde{A}_t) + \mathcal{L}_{\text{CE}}(\hat{A}^M_t, \tilde{A}_t)}_{\mathcal{L}_{\text{Align}}} + \underbrace{\mathcal{L}_{\text{CE}}(\hat{T}_t, T_t) + \mathcal{L}_{\text{CE}}(\hat{V}_t, V_t) + \mathcal{L}_{\text{CE}}(\hat{M}_t, M_t)}_{\mathcal{L}_{\text{CoT}}}$

where $\mathcal{L}_{\text{Align}}$ is the cross-mode alignment loss (Equation 8) and $\mathcal{L}_{\text{CoT}}$ is the standard cross-entropy loss on the reasoning trace tokens (Equation 9).

Update parameters: $\theta \leftarrow \theta - \eta \nabla_\theta \mathcal{L}^*_{\text{Joint}}$ .

This alternation continues until both $\mathcal{L}_{\text{non-CoT}}$ and $\mathcal{L}^*_{\text{Joint}}$ converge.

What this computes and why it alternates. The alternation ensures that the non-CoT mode maintains strong standalone performance while the CoT modes learn to align with it. If Phase 1 were omitted and only Phase 2 were used (joint optimization of all modes together), the non-CoT mode's gradient would come only indirectly through the alignment loss — the CoT modes' action predictions being penalized for diverging from the non-CoT mode — but the non-CoT mode itself would receive no direct supervision from ground-truth actions. This could lead to a degenerate solution where all modes collapse to a mediocre compromise. Phase 1 provides the non-CoT mode with direct, unmediated supervision from the ground-truth actions, ensuring that $\tilde{A}_t$ remains a high-quality target for alignment.

Phase 2 then improves the CoT modes along two axes simultaneously: (1) the $\mathcal{L}_{\text{CoT}}$ term pushes them to generate accurate reasoning traces (matching the Qwen-VL-Max annotations for text, matching the VAR latents for vision, matching both for MM-CoT), and (2) the $\mathcal{L}_{\text{Align}}$ term pushes their action predictions to match the (now frozen) non-CoT mode's predictions. The CoT modes are thus encouraged to generate reasoning that is both internally coherent (matching the annotations) and actionally consistent (leading to the same decisions as direct prediction).

The relationship between the separate and joint losses. Note that the non-CoT objective (Equation 5) is:

$\mathcal{L}_{\text{non-CoT}} = \mathcal{L}_{\text{CE}}(\hat{A}_t, A_t)$

where $\hat{A}_t = \pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 0)$ . This is a standard supervised learning loss: predict the ground-truth actions from the instruction and observations. The joint objective (Equation 7) is:

$\mathcal{L}^*_{\text{Joint}} = \mathcal{L}_{\text{Align}} + \mathcal{L}_{\text{CoT}}$

where $\mathcal{L}_{\text{Align}}$ does not reference the ground-truth actions $A_t$ at all — it only references the non-CoT mode's soft predictions $\tilde{A}_t$ . The ground-truth actions appear only in Phase 1 (through $\mathcal{L}_{\text{non-CoT}}$ ) and indirectly in Phase 2 (through $\mathcal{L}_{\text{CoT}}$ , where the CoT reasoning traces guide action prediction, and that action prediction is compared to $\tilde{A}_t$ , which was trained on $A_t$ in Phase 1). This design ensures that the ground-truth actions remain the ultimate supervisory signal — the non-CoT mode learns directly from them, and the CoT modes learn to agree with the non-CoT mode, which has learned from them.

Why stop-gradient is applied in Phase 2. In Phase 2, step 4 recomputes the non-CoT predictions with stop-gradient. This is necessary because the parameters were updated in Phase 1 (step 3), so the non-CoT mode's predictions have changed. The stop-gradiented recomputation provides an up-to-date teacher signal for the CoT modes while preventing the Phase 2 alignment loss from backpropagating into the non-CoT pathway and undoing the Phase 1 update. Conceptually, Phase 1 says "improve the non-CoT mode using ground truth," and Phase 2 says "now make the CoT modes agree with the improved non-CoT mode, and also improve their reasoning quality."

Inference Procedure

At inference, the model operates exclusively in non-CoT mode — $g_T = 0, g_V = 0$ . The input consists of the instruction, historical visual observations, and current multi-view observations. The model performs a single autoregressive forward pass to generate only action tokens:

$\hat{A}_t \sim \pi_\theta(I, \{o_{\leq t}\}, g_T = 0, g_V = 0)$

What happens at each navigation step. The agent receives three images (left, front, right camera views) from the simulator. These are concatenated with a buffer of up to 20 historical images (front views only, as described in Appendix A.2). The instruction and the navigation prompt (which varies depending on whether the task is single-stage or multi-stage) are prepended. In multi-stage tasks, the prompt includes a counter indicating how many <|stop|> actions have been output so far, to help the model track which subgoal it is pursuing. The entire input is processed through the VLM's vision encoder (to produce visual embeddings) and language model (to autoregressively generate output tokens). The model generates up to 5 action tokens — <|forward|>, <|left|>, <|right|>, or <|stop|> — stopping generation either after 5 tokens or when <|stop|> is predicted. These actions are executed sequentially in the simulator: the agent moves forward, turns, or stops, and the environment returns new visual observations. The process repeats until the task is complete or the action budget is exhausted.

Why non-CoT only. There are two reasons the paper uses only non-CoT mode at inference. First and most obviously, latency: explicit CoT generation would require decoding hundreds (T-CoT) or thousands (V-CoT, MM-CoT) of additional tokens per decision step, as quantified in Table 4. The paper reports that explicit CoT-VLA achieves 0.19 APS, meaning each action takes approximately 5.3 seconds to generate. With FantasyVLN's implicit mode achieving 1.03 APS, each action takes approximately 0.97 seconds — a 5.4× speedup that makes real-time navigation feasible.

Second and more subtly, generalization: Table 6 shows that explicit CoT inference actually produces worse results than implicit inference for most modes. For MM-CoT, explicit inference achieves 0.98 SR and 8.62 ISR, while implicit inference achieves 2.44 SR and 11.01 ISR. The paper attributes this to "cumulative reasoning errors across long trajectories" — explicit CoT generates a chain of reasoning tokens at each step, and errors in that chain (e.g., a textual plan that misidentifies the current subgoal, or a visual imagination that hallucinates a doorway) can mislead subsequent action predictions. Over dozens or hundreds of steps, these errors compound. Implicit inference avoids this by not generating the intermediate reasoning at all — the reasoning patterns are encoded in the model's weights and are applied during the forward pass that produces actions, but there is no autoregressive error propagation through reasoning tokens.

The conceptual model of implicit reasoning. During training, the cross-mode alignment constraint forced the non-CoT mode's action predictions to be consistent with the CoT modes' predictions, and the shared parameters caused the non-CoT pathway to internalize the intermediate representations that the CoT modes explicitly generate. At inference, when the model processes an instruction and visual observations, it computes internal activations that — while not decoded into explicit reasoning tokens — encode the same structured information: the current subgoal, the progress made so far, the expected spatial layout ahead, the landmarks to look for. These activations then directly condition the action prediction, producing reasoning-aware behavior without reasoning tokens. This is akin to how a human expert navigates a familiar building: they do not verbally articulate "I need to turn left at the end of this hallway because the office is on the left" — they simply turn left, because the reasoning is compressed into their spatial understanding and procedural memory.

Summary of Design Choices and Their Justifications

VAR compression at scale 4 over pixel-level or VAE-based compression: achieves 1/2185 compression ratio (30 tokens vs. 1,024+), making visual CoT training computationally feasible while preserving sufficient spatial-semantic information. Higher scales introduce redundancy that degrades navigation performance (Figure 3). The hierarchical next-scale prediction structure of VAR naturally captures global scene layout at low scales, which is what matters for navigation.
Offline T-CoT annotation using Qwen-VL-Max over online generation: provides fixed, high-quality reasoning targets that are consistent across training, avoiding the instability of having the model generate its own evolving reasoning traces. The structured four-part template (semantic planning, visual description, action decision-making, visual imagination) ensures that reasoning captures complementary aspects of the navigation task.
Uniform gating signal sampling over weighted or curriculum-based sampling: ensures all four modes receive equal optimization pressure, preventing overfitting to any single mode and ensuring the shared parameters learn representations useful across all modes. This is the simplest strategy that guarantees balanced training.
Stop-gradiented non-CoT soft targets for alignment over joint gradient flow: prevents circular optimization (CoT modes matching non-CoT while non-CoT drifts to match CoT) and establishes a clear hierarchy where the non-CoT mode is the reference. This is essential for the alignment constraint to function as intended.
Alternating optimization (Phase 1 non-CoT-only, Phase 2 joint) over single-phase joint training: ensures the non-CoT mode receives direct ground-truth supervision (not just indirect pressure through alignment), maintaining its quality as a teacher for the CoT modes. Without Phase 1, the non-CoT mode could degrade due to lack of direct supervision.
Predicting 5 actions at once over single-action or full-trajectory prediction: balances planning horizon against prediction uncertainty. Five actions is short enough that the visual consequences are reasonably predictable (enabling V-CoT training) but long enough to amortize reasoning cost across multiple execution steps. Training slices of 5 actions with up to 20 historical images provide sufficient context for the model to learn temporal dependencies.
Implicit inference (non-CoT mode only) at deployment over explicit CoT or mode selection: eliminates token generation overhead (1.03 APS vs. 0.19 APS) and avoids autoregressive error propagation through reasoning chains. Table 6 empirically validates that implicit inference outperforms explicit for all CoT modalities on LH-VLN.

4. Key Insights and Innovations

Innovation 1: Implicit Multimodal Reasoning as a Training-Time Distillation Problem

The paper's most fundamental intellectual move is reframing multimodal Chain-of-Thought reasoning not as a generation problem — the dominant paradigm in the VLN and broader VLM literature — but as a representation learning and distillation problem. The core observation is that the intermediate reasoning tokens (whether textual plans or imagined visual observations) serve their purpose during training by shaping the model's internal representations, and once those representations are adequately shaped, the tokens themselves become redundant. The cross-mode alignment constraint introduced in Section 3.5 operationalizes this as a specific optimization procedure, but the conceptual contribution is deeper: it identifies that the value of CoT in embodied settings is primarily as a training signal for representation learning, not as a runtime computation mechanism.

This is a fundamental reframing, not an incremental improvement. Prior work on CoT for VLN — NavCoT, NavGPT-2, and even Aux-Think (which introduced train-with-CoT, infer-without-CoT) — treated CoT reasoning as a causal scaffold: generate intermediate tokens, then condition actions on those tokens. The thinking was that the reasoning tokens provide structured causal pathways that make action prediction more reliable. Aux-Think's key contribution was showing that the scaffolding could be removed at inference, but even it retained the assumption that the improvement came from having practiced generating those reasoning tokens. This paper argues something different: the improvement comes from learning representations that are invariant across reasoning modalities. The textual, visual, and multimodal reasoning modes are not just different scaffolds — they are different projections of the same underlying task structure (semantic intent, spatial layout, subgoal progress), and forcing the model to produce consistent actions from all three projections while sharing parameters forces it to learn modality-invariant representations that capture that underlying structure directly. The CoT tokens themselves are merely one path to those representations; once learned, they are unnecessary.

Evidence for this reframing comes from two complementary sources. First, Table 6 shows that implicit inference consistently outperforms explicit inference for every CoT modality — including MM-CoT, where explicit reasoning generates both text and visual latents yet achieves 0.98 SR vs. 2.44 SR for implicit. This is the opposite of what the causal-scaffold view predicts: if reasoning tokens provide useful causal pathways, generating them should help, not hurt. The paper's explanation — that explicit reasoning tokens accumulate errors across long trajectories — suggests that the tokens are not just redundant but actively harmful at test time, consistent with the view that their value lies in training rather than runtime. Second, Table 3 shows that combining all four reasoning modes (non-CoT + T-CoT + V-CoT + MM-CoT) during training achieves the best performance (2.44 SR, 11.01 ISR), with each added mode contributing incremental gains beyond what any subset provides. This supports the complementary-representations view: textual reasoning shapes representations one way, visual reasoning another, and their combination shapes a third way, and the model benefits from all three shaping pressures via shared parameters even though it uses none of them at inference.

Innovation 2: Difficulty-Adaptive Compression via Hierarchical Latent Scales

The paper's second distinctive contribution is the discovery that visual reasoning for navigation benefits from a specific, intermediate level of compression granularity, and that both insufficient and excessive compression degrade performance. This is not simply a finding that VAR compression works — many papers use compressed visual representations — but rather a diagnostic result about the information content required for spatial reasoning that the field had not previously characterized.

The key evidence is Figure 3, which sweeps VAR scales from 1 to 10 and measures ISR. Scale 1 and 2 produce 0 ISR — the latent representation at these scales contains essentially no useful visual information. Scale 3 achieves 3.76 ISR, marking the threshold where useful information begins to appear. Scale 4 achieves the peak at 7.41 ISR. Scales 5–10 show a monotonic decline: 6.00, 3.76, 3.76, 2.26, 0.75, 0.90. This inverted-U relationship is striking because it is not what a naive compression-efficiency argument would predict. If more information were always better for reasoning, performance would increase monotonically with scale (since higher scales capture more visual detail with lower reconstruction error). If compression ratio were all that mattered, performance would be highest at the lowest scales. Instead, there is an optimal middle ground.

The paper's interpretation — that lower scales lack sufficient visual information while higher scales introduce redundancy that interferes with reasoning — points to a more specific mechanism. The VAR's hierarchical structure means that scale 4 captures global scene layout (room type, doorway positions, furniture arrangement) while discarding texture details and fine geometry. Higher scales add precisely the kind of information that is useful for reconstruction but irrelevant or misleading for navigation: wall textures, object surface details, lighting variations, small furniture items. When the model is trained to predict these details, it allocates representational capacity to capturing them, potentially at the expense of the global spatial relationships that matter for action prediction. In other words, the V-CoT training objective at high scales becomes dominated by a de facto image reconstruction task rather than a spatial reasoning task, and the gradients from pixel-level reconstruction accuracy compete with — and partially override — the gradients from action prediction accuracy.

This is a novel diagnostic concept: the optimal compression ratio for reasoning is not determined by reconstruction fidelity but by task-relevant information content. Prior work on visual CoT (CoT-VLA, VISTA, WorldVLA) implicitly assumed that better visual reconstruction would lead to better reasoning, since the imagined observations are more accurate. Figure 3 and Figure 4 jointly demonstrate that this assumption is wrong: the reconstruction quality at scale 10 is superior to scale 4 (Figure 4 shows progressively more detailed images), but navigation performance at scale 10 is near zero. This finding has implications beyond VLN — it suggests that visual reasoning in any embodied domain should use task-specific compression levels rather than generic image compression metrics, and that the common practice of using VAE-based compression (64×, producing 1,024 tokens as in Table 1) may be simultaneously too expensive and too detailed for optimal performance.

Innovation 3: Modality-Complementary Reasoning Symmetry as a Unifying Principle

The paper introduces a specific architectural hypothesis — that textual, visual, and multimodal reasoning are complementary symmetry operations on a shared task representation — and validates it through systematic mode ablation. This is a conceptual contribution about the structure of reasoning in VLN, not just a demonstration that "more modes are better."

The complementarity hypothesis asserts that textual CoT provides semantic decomposition (what subgoals exist, what order they should be pursued in, what landmarks indicate progress) while visual CoT provides spatial anticipation (what the environment will look like after actions, where obstacles or affordances will appear), and that these two forms of information are not reducible to each other. Textual reasoning cannot capture the precise visual appearance of a doorway that signals "this is the entrance to the office," and visual reasoning cannot capture the high-level task structure that says "I need to find the toilet paper before going to the bathroom." MM-CoT, which combines both, provides a third reasoning mode that jointly constrains semantic and spatial predictions — the textual plan constrains what visual features to anticipate, and the visual anticipation verifies the feasibility of the textual plan.

The evidence for complementarity rather than mere additive benefit comes from Table 3's mode ablation patterns. If textual and visual reasoning provided redundant information, adding V-CoT to a T-CoT system should produce minimal gains — the model would already have the relevant reasoning patterns from the textual mode. Instead, T-CoT alone achieves 8.26 ISR; adding V-CoT (T-CoT + V-CoT + non-CoT) achieves 11.19 ISR — a 35% relative improvement. V-CoT alone achieves 2.01 ISR (when trained only with non-CoT); adding T-CoT to reach the same three-mode combination produces a much larger jump. This asymmetry — V-CoT's marginal contribution to a T-CoT system is larger than T-CoT's marginal contribution to a V-CoT system — suggests that the two modalities capture different aspects of the task, with visual reasoning providing something that textual reasoning does not fully capture (likely precise spatial-geometric information) and textual reasoning providing something that visual reasoning is poor at (high-level task decomposition and progress tracking). The further gain from adding MM-CoT (reaching 11.01 ISR in the four-mode configuration) is modest compared to the T-CoT → T-CoT+V-CoT jump, suggesting that MM-CoT provides a refinement of already-captured complementary patterns rather than an entirely new information source.

The theoretical significance of this finding is that it provides a decompositional framework for multimodal reasoning tasks. Rather than treating "multimodal CoT" as a monolithic concept, the paper identifies separable dimensions — semantic vs. spatial reasoning — and shows that they can be trained independently (via gating) and combined through parameter sharing and alignment. This framework is more modular than end-to-end multimodal CoT approaches (which generate text and vision jointly without explicit separation) and provides a clearer path for diagnosing which reasoning components are most beneficial for which tasks. For a navigation task where spatial reasoning is paramount, V-CoT might dominate; for a task requiring heavy instruction decomposition, T-CoT might be more important. The framework's modularity allows future work to weight these modes differently or introduce additional reasoning modes (e.g., a temporal reasoning mode that tracks progress over time) without redesigning the architecture.

Innovation 4: The Counterintuitive Superiority of Implicit Over Explicit Reasoning

Perhaps the paper's most counterintuitive finding — and one that challenges a widely held assumption in the CoT literature — is that explicitly generating reasoning tokens at inference produces worse results than not generating them, across all CoT modalities tested. This is not a speed-versus-accuracy tradeoff where explicit reasoning is more accurate but slower; it is a case where implicit reasoning is both faster and more accurate.

Standard CoT intuition, inherited from the LLM literature, holds that generating intermediate reasoning steps improves final answer quality by making the model "think step by step" — the autoregressive generation of reasoning tokens forces the model to commit to intermediate conclusions that constrain and guide subsequent token generation. This has been validated across numerous benchmarks for static QA, math, and code tasks. The VLN setting challenges this intuition because it introduces two factors absent from standard CoT benchmarks: long action horizons with compounding errors and limited training data.

The error compounding mechanism works as follows. At each decision step, explicit CoT generates hundreds of reasoning tokens before predicting actions. Any error in those reasoning tokens — a slightly incorrect subgoal identification, a hallucinated visual feature, a misestimated distance — propagates forward in two ways. First, it biases the subsequent action predictions at the current step. Second, because the agent's actions determine what it sees next, an action error changes the future visual observations, potentially creating a distribution shift relative to what the CoT reasoning anticipated. Third, in long-horizon tasks, these per-step errors compound over dozens or hundreds of steps — a small reasoning error in step 10 can lead to navigation errors that make the task unsolvable by step 50. Implicit reasoning sidesteps this entirely because there are no reasoning tokens to contain errors; the reasoning patterns are encoded in the model's weights and applied during a single forward pass that produces only action tokens, avoiding the autoregressive error propagation pathway.

The limited data factor amplifies this problem. The LH-VLN training set contains only 18,554 trajectory slices, each covering 5 action steps. Training a model to generate coherent textual or visual reasoning traces from this amount of data is challenging — the model must learn not just to predict correct actions (4 possible tokens) but to generate fluent paragraphs of text (hundreds of tokens from a vocabulary of tens of thousands) and hierarchical visual latents. With limited data, the model's reasoning generation is noisy and prone to overfitting, and these noisy reasoning traces then mislead action prediction. Implicit reasoning avoids learning the generation side of CoT entirely — it only needs to learn the action prediction, with the CoT supervision acting as a regularizer that shapes internal representations without requiring the model to produce coherent reasoning at test time.

The evidence for this interpretation comes from Table 6, which shows that for every CoT modality, implicit outperforms explicit: T-CoT 0.98 vs. 0.49 SR (implicit worse for T-CoT alone but note the anomalous pattern — T-CoT explicit at 0.98 SR is actually better than implicit at 0.49 SR, suggesting textual reasoning may be an exception), V-CoT 0.49 vs. 1.46 SR, MM-CoT 0.98 vs. 2.44 SR. The gap is largest for MM-CoT — the mode that generates the most reasoning tokens and is therefore most vulnerable to compounding errors. The paper attributes this explicitly to "the limited training data of LH-VLN (only 18k trajectory slices of five steps each) makes explicit CoT sequences prone to overfitting and error propagation" and notes that "explicit reasoning expands temporal dependencies, causing misaligned textual or visual CoT tokens to accumulate deviations over time."

This finding has significant implications beyond VLN. It suggests that for any embodied or sequential decision-making task where errors compound over time and training data is limited, the standard "generate CoT then act" paradigm may be suboptimal compared to "train with CoT, act without CoT." This is a fundamentally different relationship between reasoning and action than what the LLM CoT literature assumes, and it calls into question the applicability of that literature's findings to interactive, temporally extended domains.

5. Experimental Analysis

Evaluation Methodology

Dataset. All experiments use the LH-VLN benchmark (Song et al., 2025), which consists of multi-stage, long-horizon navigation tasks in continuous 3D environments. The benchmark provides 18,554 trajectory slices for training (each covering 5 action steps, as described in Appendix A.1), with separate validation and test sets. The test set contains unseen tasks and scenes. The benchmark is characterized by requiring agents to sequentially complete multiple subtasks (e.g., "go to the bedroom, find the toilet paper, then bring it to the bathroom") with trajectories substantially longer than standard VLN benchmarks like R2R or VLN-CE.
Base model(s). All FantasyVLN variants use Qwen2.5-VL (7B parameters) as the base vision-language model, fine-tuned using LoRA-based parameter-efficient tuning on both the language layers and vision-language projection modules. The paper states in Appendix C that training is conducted on 64 H20 GPUs (141 GB each), using the AdamW optimizer with learning rate 1×10⁻⁴, weight decay 0.1, cosine schedule with 5% warmup, per-device batch size 4, bfloat16 precision, gradient checkpointing, and DeepSpeed ZeRO-2. The choice of Qwen2.5-VL as the base model is motivated by its vocabulary extensibility (enabling the addition of gating tokens, action tokens, and VAR latent tokens) and its multimodal capabilities (processing interleaved text and images natively).
Metrics. The paper uses four navigation accuracy metrics from the LH-VLN benchmark (Song et al., 2025), plus one inference efficiency metric:
- SR (Success Rate): The fraction of multi-stage navigation tasks where the agent successfully completes all subtasks in sequence. This is the strictest metric — a single subtask failure means the entire task is counted as failed.
- ISR (Independent Success Rate): The success rate of individual subtasks, treating each subtask independently regardless of whether preceding subtasks were completed. This measures subtask-level capability in isolation.
- CSR (Conditional Success Rate): The ISR weighted by the success of preceding subtasks — a subtask only counts if all earlier subtasks in the same task were successfully completed. This captures sequential dependency: you cannot succeed at subtask 3 if you failed subtask 2.
- CGT (CSR weighted by Ground Truth): CSR further weighted by the length of the expert (ground-truth) trajectory for each subtask, giving more weight to subtasks that require more steps. This accounts for task difficulty heterogeneity.
- APS (Actions Per Second): $\text{APS} = N_{\text{act}} / T_{\text{nav}}$ , where $N_{\text{act}}$ is the total number of actions executed during navigation and $T_{\text{nav}}$ is the total navigation time in seconds. This measures inference efficiency — higher APS means faster decision-making.
Baselines. The paper compares against several categories of methods, with all methods trained on the same LH-VLN training set and evaluated using the same online evaluation protocol (the validation set is used only for checkpoint selection):
- Zero-shot / no-CoT methods: Random (a random action policy, achieving 0 on all metrics), GLM-4v prompt (zero-shot prompted GLM-4V, also 0 on all metrics), GPT-4 + NaviLLM (a GPT-4-based agent combined with a navigation LLM, achieving 0 SR, 2.19 ISR, 1.45 CSR, 2.61 CGT), and MGDM (a memory-based method, achieving 0 SR, 2.34 ISR, 1.65 CSR, 2.91 CGT).
- Textual CoT methods: Aux-Think (Wang et al., 2025), which introduces auxiliary CoT supervision to internalize reasoning patterns during training and performs inference without CoT. The authors reimplemented Aux-Think based on the paper's description since training codes were not publicly available. Aux-Think achieves 0.65 SR, 3.16 ISR, 2.04 CSR, 1.47 CGT.
- Visual CoT methods: CoT-VLA (Zhao et al., 2025), a visual CoT method that generates future frames before action prediction, achieving 0 on all metrics; and WorldVLA, which the authors adapted from the official implementation by modifying the preprocessing pipeline to support LH-VLN training, also achieving 0 on all metrics. Both CoT-VLA and WorldVLA are 7B models.
Generation budget / compute accounting. The paper uses Actions Per Second (APS) as its primary compute efficiency metric rather than a generation budget or FLOPs count. This choice reflects the real-time nature of the VLN task: the relevant constraint is not total computation per task but the rate at which the agent can make decisions. The paper does not report total FLOPs or generation token counts per task for the main results. Compared methods are grouped by model size (all main methods are 7B–8B parameters) to control for per-forward-pass cost. For the explicit vs. implicit comparison in Table 6, the paper evaluates the same trained model using two inference pathways — explicit (generating CoT tokens) and implicit (direct action prediction) — controlling for model capacity and training data.
Cross-validation / statistical protocol. The paper explicitly states that the LH-VLN validation set is used to select the best checkpoint for each method. Online evaluation is then performed on the test set (unseen tasks and scenes). No k-fold cross-validation, confidence intervals, or statistical significance tests are reported. The VAR scale sweep in Figure 3 is conducted on "a subset of LH-VLN" rather than the full test set, though the size of this subset is not specified. Given the 500-question test set (implied by the LH-VLN benchmark structure), this is a relatively small evaluation set, and the absence of error bars or statistical testing means the reliability of the exact numerical rankings (e.g., 2.44 SR vs. 0.98 SR) cannot be assessed for statistical significance.

Main Quantitative Results

The headline result is that FantasyVLN achieves 2.44 SR, 11.01 ISR, 9.64 CSR, and 8.99 CGT, substantially outperforming all baselines. The closest baseline is Aux-Think (textual CoT, implicit inference) at 0.65 SR and 3.16 ISR — FantasyVLN achieves 3.8× higher SR and 3.5× higher ISR. The gap is even larger relative to visual CoT methods (CoT-VLA and WorldVLA), which score 0 on all metrics, and non-CoT methods (MGDM at 0 SR, GPT-4 + NaviLLM at 0 SR), which fail entirely on multi-stage completion.

The ISR metric (11.01) is notably higher than SR (2.44), reflecting the sequential dependency captured by CSR and CGT: the agent succeeds at individual subtasks more often than it succeeds at completing all subtasks in sequence. CSR (9.64) weights ISR by preceding subtask success, and CGT (8.99) further weights by trajectory length. The progressive decline from ISR → CSR → CGT is expected for multi-stage tasks and indicates that errors accumulate across subtasks — a subtask failure early in the sequence prevents credit for later subtasks that might have been completed. The relatively small gap between CSR (9.64) and CGT (8.99) suggests that trajectory length weighting does not dramatically alter the ranking.

The zero scores for visual CoT methods (CoT-VLA and WorldVLA) are a striking negative result. These are not methods that failed to train — they are methods whose explicit visual reasoning approach appears fundamentally incompatible with the LH-VLN benchmark's requirements. The paper attributes this to training instability (Figure 5 shows WorldVLA's slow convergence) and the inability of pixel-level visual CoT to generalize to long-horizon scenarios. This is an important empirical finding: visual CoT methods validated on shorter-horizon manipulation or navigation tasks do not directly transfer to multi-stage, long-horizon settings, at least not with the training data scale available on LH-VLN.

The relative performance of MGDM (0 SR, 2.34 ISR) versus Aux-Think (0.65 SR, 3.16 ISR) shows that memory mechanisms alone provide limited benefit compared to reasoning-based approaches. MGDM's positive ISR but zero SR indicates it can complete individual subtasks but fails at sequential multi-stage completion — exactly the pattern that reasoning (tracking which subgoal is currently active) should address.

Inference Efficiency (Table 4)

The paper compares APS across four CoT reasoning methods. FantasyVLN achieves 1.03 APS, which is comparable to other implicit reasoning methods (WorldVLA at 1.02 APS, Aux-Think at 0.97 APS) and dramatically faster than the explicit CoT baseline (CoT-VLA at 0.19 APS). The 5.4× speedup of FantasyVLN over CoT-VLA (1.03 vs. 0.19) quantifies the practical impact of the implicit reasoning approach: the agent makes decisions approximately 5 times faster.

The three implicit methods (FantasyVLN, WorldVLA, Aux-Think) cluster tightly in the 0.97–1.03 APS range, all 7B–8B models. This clustering is expected because implicit inference for all three methods requires only decoding action tokens (a few tokens per step) rather than reasoning tokens (hundreds to thousands of tokens per step). The similarity in APS confirms that the per-forward-pass cost is dominated by model size and input processing (visual encoding of images) rather than output length when output length is minimal. The explicit method CoT-VLA at 0.19 APS — approximately 5.3 seconds per action — would require over 8 minutes for a 100-action trajectory, making it impractical for any real-world deployment.

The paper does not report APS for different FantasyVLN configurations (e.g., V-CoT vs. MM-CoT during explicit inference), which would help quantify the latency penalty of generating visual latents vs. text vs. both. Table 4 only reports the implicit FantasyVLN configuration (non-CoT at inference). It also does not report the total navigation time or total tokens generated for each method, which would provide a more complete picture of the compute-accuracy tradeoff.

Mode Combination Ablation (Table 3)

The paper systematically tests different combinations of reasoning modes during training, evaluating all variants at inference using the non-CoT (implicit) pathway. The results demonstrate clear complementarity between modes:

Non-CoT only (no CoT training): SR 0, ISR 2.01, CSR 1.51, CGT 1.55. This establishes the baseline performance of the fine-tuned Qwen2.5-VL without any CoT supervision — the model can complete some individual subtasks (2.01 ISR) but fails entirely at multi-stage completion (0 SR).

Non-CoT + T-CoT: SR 0.98, ISR 8.26, CSR 6.60, CGT 6.15. Adding textual CoT training more than quadruples ISR (2.01 → 8.26), a 311% relative improvement, and enables multi-stage completion (SR goes from 0 to 0.98). This is the largest single-mode improvement, suggesting that semantic planning and subgoal decomposition — the core of T-CoT — are the most impactful reasoning components for this benchmark.

Non-CoT + V-CoT: SR 1.46, ISR 11.19, CSR 9.66, CGT 8.84. Adding visual CoT training achieves the highest ISR among two-mode configurations (11.19 vs. 8.26 for T-CoT) and substantially higher SR (1.46 vs. 0.98 for T-CoT). This is the paper's key evidence for visual reasoning providing complementary benefits — V-CoT contributes gains beyond what T-CoT alone provides, and the gains are concentrated in metrics that require spatial understanding (higher ISR suggests better individual subtask completion, which requires precise navigation to goal locations).

Non-CoT + MM-CoT: SR 0.49, ISR 7.77, CSR 6.48, CGT 8.89. Adding multimodal CoT training alone (without separate T-CoT or V-CoT) is less effective than T-CoT or V-CoT alone, particularly in SR (0.49) and ISR (7.77). This is somewhat surprising — MM-CoT generates both textual and visual reasoning traces, so one might expect it to provide the benefits of both. The lower performance suggests that training only on MM-CoT (without separate unimodal CoT training) does not allow the model to fully specialize representations for each modality's distinct contribution. The model may be learning a compromised representation that averages across modalities rather than developing clean, modality-specific reasoning patterns that are then aligned.

All four modes (Non-CoT + T-CoT + V-CoT + MM-CoT): SR 2.44, ISR 11.01, CSR 9.64, CGT 8.99. The full configuration achieves the best or near-best results across all metrics. Compared to the best two-mode configuration (Non-CoT + V-CoT at 1.46 SR), adding T-CoT and MM-CoT provides a 67% relative improvement in SR. Compared to Non-CoT + T-CoT (8.26 ISR), adding V-CoT and MM-CoT provides a 33% relative improvement in ISR. The CGT (8.99) is essentially tied with Non-CoT + V-CoT (8.84), suggesting that the trajectory-length-weighted metric is less sensitive to the additional modes.

Interpretation of the mode interaction pattern. The results support a complementarity narrative: T-CoT contributes strongly to ISR improvement alone (8.26), V-CoT contributes to both ISR (11.19, the highest) and SR (1.46, higher than T-CoT's 0.98), and their combination in the full framework provides the best SR (2.44). The fact that V-CoT alone achieves higher SR than T-CoT alone (1.46 vs. 0.98) but lower ISR (11.19 vs. 8.26? No — V-CoT achieves 11.19 ISR vs. T-CoT's 8.26) suggests that visual reasoning provides both better individual subtask execution (higher ISR) and better sequential coordination (higher SR). The further improvement from adding both T-CoT and MM-CoT to V-CoT (SR 1.46 → 2.44, a 67% gain) demonstrates that textual and multimodal reasoning provide incremental benefits beyond visual reasoning alone, likely by improving semantic planning and progress tracking — abilities that pure visual reasoning does not fully capture.

VAR Scale Selection (Figure 3)

The paper sweeps VAR scales from 1 to 10 and measures ISR on a subset of LH-VLN to determine the optimal compression level for V-CoT training. The results show a clear inverted-U pattern:

Scales 1–2: ISR = 0. The latent representation at these scales contains negligible visual information — the reconstruction in Figure 4 confirms this, showing essentially blank or color-blob images at scales 1–2.
Scale 3: ISR = 3.76. This marks the threshold where useful visual information begins to emerge — the reconstruction in Figure 4 shows rough room structure becoming visible.
Scale 4: ISR = 7.41 (the peak). This is the optimal setting and the one used in all other experiments. The reconstruction at scale 4 (Figure 4) captures room layout, major furniture, and doorway positions while lacking fine texture detail.
Scale 5: ISR = 6.00. A decline from the peak, though still substantially above scale 3.
Scale 6: ISR = 3.76. Returns to the scale-3 level, implying that the additional detail at scale 6 provides no net benefit over the coarse scale-3 representation and actively hurts relative to scale 4.
Scales 7–10: ISR declines monotonically (3.76, 2.26, 0.75, 0.90), approaching zero at the highest scales. At scale 10, where the reconstruction is most faithful to the original image, navigation performance is nearly as poor as at scale 1 where reconstruction is impossible.

The paper attributes this pattern to a tradeoff between insufficient information (low scales) and redundancy that interferes with reasoning (high scales). The interpretation — discussed more fully in the Key Insights — is that higher scales force the model to allocate representational capacity to predicting fine visual details (textures, small objects, lighting variations) rather than the global spatial relationships that matter for navigation. The single peak at scale 4, with monotonic decline on both sides, is particularly clean evidence for this interpretation and rules out alternative explanations like training instability at specific scales.

Figure 4 provides qualitative support: at scale 4, the reconstructed image captures the room type, doorway positions, and major furniture — exactly the information needed to verify that "after moving forward, I should see the living room entrance on my left." At scale 10, the reconstruction adds wall textures, furniture details, and lighting nuances that are perceptually realistic but navigationally irrelevant. The paper's claim is that the training gradient from predicting these details competes with the gradient from action prediction, degrading the quality of the learned representations for navigation.

Cross-Mode Alignment Constraint Ablation (Table 5)

The paper compares FantasyVLN trained with and without the cross-mode alignment constraint. Without alignment, performance collapses to near zero: SR 0, ISR 2.39, CSR 1.19, CGT 1.28. With alignment, performance is substantial: SR 2.44, ISR 11.01, CSR 9.64, CGT 8.99.

This is the most dramatic ablation in the paper — the alignment constraint is not merely beneficial but essential for the unified framework to function. The ISR of 2.39 without alignment is only marginally better than the non-CoT-only baseline (2.01 in Table 3), indicating that joint training of multiple modes without alignment pressure yields essentially no benefit over training without any CoT supervision at all. The model's parameters, receiving conflicting gradients from different reasoning modes (each mode optimizing its own action predictions independently while sharing parameters), converge to a representation that performs poorly in all modes.

The SR of 0 without alignment (vs. 2.44 with alignment) is particularly stark — without alignment, the model literally never completes a full multi-stage task. This confirms the paper's claim that the cross-mode alignment constraint is the mechanism by which diverse reasoning patterns are distilled into a coherent implicit policy, and that parameter sharing alone is insufficient to achieve this distillation. The alignment constraint serves two simultaneous functions: (1) it resolves inter-mode gradient conflicts by providing a common action target, and (2) it forces the non-CoT pathway's internal representations to encode information that the CoT pathways explicitly generate, since the non-CoT pathway must produce actions consistent with those generated after explicit reasoning.

Note that the without-alignment variant trains on the same four modes (non-CoT, T-CoT, V-CoT, MM-CoT) with shared parameters — it simply omits the alignment loss and the alternating optimization procedure. The fact that this variant achieves near-zero performance while the individual mode combinations in Table 3 (e.g., Non-CoT + T-CoT at 0.98 SR) perform substantially better suggests that the combination of all four modes without alignment is actively harmful compared to training on fewer modes. The conflicting gradients from four simultaneous objectives appear to be worse than the gradients from any single CoT mode paired with non-CoT. The alignment constraint resolves this by unifying the action prediction objectives across modes.

Training Efficiency Comparison (Figure 5)

The paper compares training convergence between FantasyVLN and WorldVLA, measured by training token accuracy over iterations. FantasyVLN converges rapidly — rising to high token accuracy within approximately 2,000–3,000 iterations and stabilizing. WorldVLA exhibits slow and unstable convergence — requiring over 10,000 iterations to reach moderate accuracy, with substantial fluctuations.

The paper attributes this difference to the CompV-CoT design: by training in the VAR latent space (30 tokens per image) rather than pixel space, FantasyVLN provides "stable supervision and more efficient learning dynamics" — the model's loss is concentrated on a small number of semantically meaningful latent tokens rather than spread across thousands of pixel-level predictions. WorldVLA's pixel-level V-CoT learning "delivers weak gradient signals, as the model must reconstruct high-dimensional visual tokens for each reasoning step." This is consistent with the VAR scale results: at high scales where the model must predict many visual tokens, training becomes less effective.

However, this comparison has a significant confound: FantasyVLN and WorldVLA are different model architectures with different training procedures, and Figure 5 compares training token accuracy (not navigation accuracy) — a metric that measures how well the model predicts the next token in its training sequences. Higher token accuracy for FantasyVLN is at least partially expected because the 30 VAR latent tokens are a much coarser target than pixel-level tokens, and the model may learn to predict them more easily regardless of whether the learned representations are better for navigation. The fact that FantasyVLN achieves higher navigation accuracy is established in Table 2, but Figure 5's training efficiency advantage should be interpreted cautiously — it may partly reflect the inherent difficulty of the prediction task rather than the quality of the learned representations.

Explicit vs. Implicit Inference (Table 6)

For each CoT modality, the paper compares explicit inference (generating CoT tokens then actions) with implicit inference (generating actions directly) using the same trained model. The results are not uniformly in favor of implicit — there is a modality-dependent pattern:

T-CoT: Explicit inference achieves SR 0.98, ISR 8.26. Implicit inference achieves SR 0.49, ISR 6.06. Explicit outperforms implicit for textual CoT — this is the one case where generating reasoning tokens helps. The paper does not comment extensively on this exception, but it is consistent with T-CoT being the mode where reasoning tokens provide genuine causal scaffolding (decomposing the instruction into subgoals, then mapping each subgoal to actions) rather than just representation learning. The relatively low token cost of textual CoT (<500 tokens per step, as stated in Section 1) compared to visual CoT may also mean that error accumulation is less severe.

V-CoT: Explicit inference achieves SR 0.49, ISR 7.34. Implicit inference achieves SR 1.46, ISR 11.19. Implicit substantially outperforms explicit — a 3× improvement in SR (0.49 → 1.46) and 52% improvement in ISR (7.34 → 11.19). This is a striking result: generating the imagined visual observations (via V-CoT latent tokens) at inference time actively degrades performance. The paper attributes this to the limited training data making explicit V-CoT generation unreliable, with errors in the generated visual latents misleading subsequent action predictions.

MM-CoT: Explicit inference achieves SR 0.98, ISR 8.62. Implicit inference achieves SR 2.44, ISR 11.01. Implicit substantially outperforms explicit — a 2.5× improvement in SR (0.98 → 2.44) and 28% improvement in ISR (8.62 → 11.01). This is the largest absolute gap in SR (1.46 percentage points) and confirms that the token overhead and error accumulation of explicit multimodal reasoning are severe enough to outweigh any benefits from explicit reasoning guidance.

The paper's explanation for these results is twofold: (1) limited training data (18K slices) makes explicit CoT generation prone to overfitting — the model learns to produce plausible-looking reasoning traces that are not actually accurate, and these inaccurate traces mislead action prediction; (2) error propagation over long trajectories — explicit reasoning tokens at each step can contain errors that bias subsequent decisions, and these errors compound over dozens or hundreds of steps. Implicit inference avoids both problems by never generating the reasoning tokens at test time.

The modality-dependent pattern (T-CoT explicit better, V-CoT and MM-CoT implicit better) is interesting and underexplored in the paper. It suggests that different reasoning modalities have different "distillability" — textual reasoning may be harder to fully internalize into implicit representations than visual reasoning, perhaps because the structured, symbolic nature of subgoal decomposition benefits from explicit articulation. Alternatively, the annotation quality for T-CoT (generated by Qwen-VL-Max) may be higher than the V-CoT targets (VAR-encoded ground-truth images), making explicit T-CoT generation more reliable than explicit V-CoT generation.

Ablation Studies and Robustness Checks

VAR scale (Figure 3 and Figure 4): Sweeping VAR scales from 1 to 10 shows that scale 4 achieves peak ISR (7.41), with monotonic decline at higher scales down to 0.90 at scale 10. Figure 4 confirms that higher scales improve reconstruction fidelity but degrade navigation performance, supporting the claim that fine visual details interfere with reasoning-relevant representation learning.

Reasoning mode combinations (Table 3): Testing all subsets of {non-CoT, T-CoT, V-CoT, MM-CoT} shows that (1) any CoT mode added to non-CoT training improves performance, (2) T-CoT and V-CoT provide complementary benefits (T-CoT alone: 8.26 ISR; V-CoT alone: 11.19 ISR; both combined in the four-mode setup: 11.01 ISR), and (3) the full four-mode configuration achieves the best SR (2.44) and near-best ISR (11.01). MM-CoT alone added to non-CoT is less effective than T-CoT or V-CoT alone (0.49 SR vs. 0.98 and 1.46, respectively), suggesting that joint multimodal reasoning training without separate unimodal training does not effectively specialize representations.

Cross-mode alignment constraint (Table 5): Removing the alignment constraint collapses SR from 2.44 to 0 and ISR from 11.01 to 2.39, demonstrating that the constraint is essential — shared parameters alone cannot resolve the conflicting gradients from multiple reasoning modes, and the stop-gradiented non-CoT soft targets are necessary to provide a common action target. The ISR of 2.39 without alignment is only marginally better than training with no CoT at all (2.01, Table 3), confirming that four-mode training without alignment provides essentially zero benefit.

Explicit vs. implicit inference (Table 6): Comparing the same trained model under explicit and implicit inference pathways shows that implicit inference outperforms explicit for V-CoT (1.46 vs. 0.49 SR) and MM-CoT (2.44 vs. 0.98 SR), but explicit outperforms implicit for T-CoT (0.98 vs. 0.49 SR). This suggests that visual and multimodal reasoning patterns can be more effectively internalized than purely textual reasoning patterns, or that explicit visual reasoning generation is more error-prone due to higher token counts and more complex targets. The paper does not ablate why the modality-dependent pattern exists.

VAR scale reconstruction quality (Figure 4): Qualitative examples show that scale 4 captures room layout and major objects while lacking texture, consistent with the hypothesis that navigation-relevant information (spatial layout, object positions) is preserved while navigation-irrelevant information (textures, fine details) is discarded. Higher scales progressively add detail that is visually realistic but degrades navigation performance. This is a qualitative robustness check supporting the quantitative results in Figure 3.

Data augmentation (Appendix A.4): The paper describes but does not ablate two data augmentation strategies — uniform subsampling (removing every other historical frame with 50% probability for trajectories with ≥10 historical images) and stochastic history trimming (removing initial or random consecutive frames with 50% probability for trajectories with ≥7 historical frames). No ablation table shows the effect of these augmentations on navigation performance, so their contribution is unquantified. The paper states the goal is to "improve the robustness of the instruction-following model under diverse visual histories" by "forcing the model to rely on stable, task-relevant cues rather than positional biases," but without ablation results, this remains an unverified design choice.

Training convergence (Figure 5): Comparing FantasyVLN and WorldVLA training curves shows FantasyVLN converges within ~2,000–3,000 iterations while WorldVLA requires >10,000 iterations with instability. This supports the claim that CompV-CoT provides more stable and efficient training dynamics, but the confound that token accuracy is an easier target for coarser latent predictions means the comparison should be interpreted cautiously as evidence for better representation learning rather than just easier token prediction.

Negative results from baselines (Table 2): Visual CoT methods (CoT-VLA, WorldVLA) achieve zero on all metrics, demonstrating that existing visual reasoning approaches do not transfer to the LH-VLN benchmark's multi-stage, long-horizon setting. This is an important negative result that validates the difficulty of the benchmark and the non-triviality of FantasyVLN's performance. The paper does not investigate why these methods fail — whether it is a training data insufficiency, an architectural limitation, or an incompatibility with the online evaluation protocol.

Missing ablations. Several experiments would have strengthened the paper but are absent: (1) an ablation of the VAR compressor choice — comparing against VAE-based compression or no compression to isolate the contribution of the VAR's hierarchical latent structure specifically; (2) an ablation of the T-CoT annotation source — comparing Qwen-VL-Max annotations against a smaller annotator or human annotations to assess whether the annotation quality drives results; (3) an ablation of the number of actions predicted per step (k=5) — testing k=1 or k=10 to determine the optimal planning horizon; (4) an ablation of the gating signal sampling strategy — comparing uniform sampling against weighted or curriculum-based sampling; (5) an ablation of the stop-gradient in the alignment loss — testing whether removing stop-gradient causes the circular optimization problem the paper theoretically argues it would; and (6) an ablation of the alternating optimization frequency — testing whether joint training (no alternation) produces the same collapse as removing alignment entirely, or a less severe degradation.

Critical Assessment

What the experiments demonstrate: Table 2 shows FantasyVLN achieving 2.44 SR and 11.01 ISR, which is higher than all listed baselines. The closest baseline (Aux-Think) achieves 0.65 SR and 3.16 ISR — FantasyVLN is 3.75× better on SR and 3.48× better on ISR.

What is convincing: The gap is large in absolute terms, and the pattern holds across all four metrics (SR, ISR, CSR, CGT). The baselines cover a reasonable range of approaches — zero-shot, memory-based, textual CoT, visual CoT — and FantasyVLN outperforms all of them. The complete failure of visual CoT methods (0 across all metrics) provides a clear failure case that FantasyVLN avoids.

What is less convincing: Three concerns limit the strength of this claim. First, the test set size is not reported, but based on the LH-VLN benchmark, it is likely small (on the order of hundreds of tasks). Without confidence intervals, it is impossible to assess whether 2.44 SR is statistically distinguishable from 0.98 (the next-best non-zero SR from a mode-combination ablation in Table 3) or whether the ranking of methods is robust to test-set resampling. Second, the baselines include reimplementations — Aux-Think and CoT-VLA were reimplemented from paper descriptions because training code was not publicly available, and WorldVLA was adapted from official code with modified preprocessing. The performance of these reimplementations may not match what the original authors would achieve, and the paper does not validate the reimplementations on the original benchmarks. Third, FantasyVLN uses a specific base model (Qwen2.5-VL) and extensive annotation pipeline (Qwen-VL-Max for T-CoT data), while the baselines may use different base models — the paper does not fully control for base model capability, making it unclear whether the gains come from the FantasyVLN framework or from using a stronger base model with higher-quality training data.

A stronger experiment would be: (a) report confidence intervals or test-retest variance for all metrics, (b) validate reimplemented baselines on their original benchmarks to ensure fidelity, and (c) ablate the base model choice by running the same framework on the same base model used by baselines where possible.

Claim 2: FantasyVLN reduces inference latency by an order of magnitude compared to explicit CoT methods

What the experiments demonstrate: Table 4 shows FantasyVLN at 1.03 APS vs. CoT-VLA at 0.19 APS — a 5.4× speedup. "An order of magnitude" (10×) slightly overstates the result, but the 5.4× speedup is substantial.

What is convincing: The comparison controls for model size (both 7B) and the metric is straightforward (actions per second during online evaluation). The clustering of implicit methods (FantasyVLN, WorldVLA, Aux-Think) at ~1 APS vs. the explicit method (CoT-VLA) at ~0.2 APS provides convergent evidence that explicit CoT generation is the bottleneck, not differences in model architecture or input processing.

What is less convincing: The comparison is thin — only one explicit CoT baseline (CoT-VLA) is evaluated for APS, and the paper does not report APS for its own models when run in explicit inference mode (which would provide a direct within-model comparison of the latency penalty). The paper also does not report total tokens generated per step or total navigation time, which would contextualize the APS numbers. A 5.4× speedup from 0.19 to 1.03 APS means going from ~5.3 seconds per action to ~0.97 seconds per action. For a 100-action trajectory, that is the difference between ~9 minutes and ~1.6 minutes — a meaningful improvement, but "order of magnitude" (which typically implies 10×) is an overstatement. Additionally, the paper does not discuss whether 1.03 APS is actually "real-time" — for a robot moving at 0.5 m/s, 1 second per decision may still be too slow for smooth navigation.

Claim 3: The unified multi-CoT training strategy enables implicit reasoning that preserves the benefits of CoT supervision

What the experiments demonstrate: Table 3 shows that adding CoT modes during training consistently improves implicit (non-CoT) inference performance: non-CoT-only achieves 2.01 ISR, while adding T-CoT achieves 8.26, adding V-CoT achieves 11.19, and the full four-mode configuration achieves 11.01 ISR. The improvement from CoT training is substantial (4–5.5× on ISR). Table 6 shows that implicit inference outperforms explicit inference for V-CoT and MM-CoT, demonstrating that the reasoning patterns have been internalized rather than merely mimicked.

What is convincing: The mode-combination ablation directly tests the claim: if CoT training provides benefits that transfer to implicit inference, then varying which CoT modes are used during training should systematically affect implicit inference performance, which it does. The ablation of the alignment constraint (Table 5) shows that the mechanism for this transfer — cross-mode alignment — is essential; without it, adding CoT modes to training provides essentially no benefit (2.39 ISR vs. 2.01 without any CoT). This establishes a causal chain: CoT modes → alignment constraint → implicit performance improvement, which is strong evidence for the paper's central mechanism.

What is less convincing: The T-CoT exception in Table 6 complicates the narrative. If implicit reasoning "preserves the benefits of CoT supervision," why does implicit T-CoT underperform explicit T-CoT (0.49 vs. 0.98 SR)? The paper does not address this anomaly directly. It suggests that at least for textual reasoning, the benefits are not fully transferred to the implicit pathway, or that explicit textual reasoning provides benefits (causal scaffolding) that implicit reasoning cannot replicate. This means the claim of "preserving benefits" is modality-dependent in ways the paper does not fully characterize.

Additionally, the paper does not provide direct evidence that the implicit representations actually encode reasoning-like structure. There is no probing analysis, no visualization of internal activations, and no comparison of representations between CoT-trained and non-CoT-trained models. The evidence is entirely behavioral (better actions at inference), which is consistent with the internalization claim but does not directly confirm it. The improvement could alternatively be explained by the CoT training providing useful regularization or data augmentation that improves the action prediction pathway without the model learning anything that resembles "reasoning" internally. A probing experiment — e.g., training a linear classifier to predict subgoal identity from the non-CoT model's intermediate representations and comparing accuracy with vs. without CoT training — would substantially strengthen this claim.

Claim 4: Compressing imagined observations into VAR latent space improves training efficiency while preserving reasoning capacity

What the experiments demonstrate: Figure 5 shows FantasyVLN converging faster than WorldVLA (~3K vs. >10K iterations). Figure 3 shows that scale-4 VAR compression achieves the best ISR (7.41), outperforming both lower scales (0 at scales 1–2) and higher scales (declining to 0.90 at scale 10). Table 1 shows VAR achieves 1/2185 compression vs. 1/64–1/256 for alternatives.

What is convincing: The VAR scale sweep provides clear, internally consistent evidence for an optimal compression level. The inverted-U shape is not what a naive "more information = better" or "more compression = better" hypothesis would predict, and the qualitative reconstructions in Figure 4 provide an intuitive explanation (scale 4 captures layout, scale 10 adds texture noise). The comparison against WorldVLA's convergence demonstrates the practical training benefit.

What is less convincing: Three significant gaps. First, the comparison in Figure 5 confounds model architecture and compression method — WorldVLA is a different model with a different training pipeline, not FantasyVLN with VAE compression substituted for VAR compression. The faster convergence could be partly due to model architecture differences rather than compression method. A proper ablation would compare FantasyVLN with VAR compression against FantasyVLN with VAE compression (at the same compression ratio) and against FantasyVLN with no visual compression. Second, the VAR scale sweep is conducted on "a subset of LH-VLN" of unspecified size, and the resulting ISR numbers (peaking at 7.41) do not match the full-test-set ISR for any configuration in Table 3 (where V-CoT alone achieves 11.19 ISR), suggesting potential distribution shift between the subset and full test set, or that the sweep was conducted in a different configuration (e.g., different number of training steps or different hyperparameters). Third, the paper claims VAR is used because of its "next-scale prediction paradigm" and higher compression ratio, but never ablates against the alternative compressors listed in Table 1 (RAE-DINOv2-B at 1/256, VAE at 1/64) in a navigation experiment. The comparison in Table 1 is only on reconstruction MSE, not on downstream navigation performance. Without this ablation, the claim that VAR is specifically responsible for the performance improvement (rather than just "using compression of roughly the right amount") is not fully substantiated.

Overall Strengths

The experimental section has several genuine strengths. The mode-combination ablation (Table 3) is systematic and informative — testing all subsets of reasoning modes directly addresses the complementarity hypothesis and provides clear evidence that textual and visual reasoning contribute distinct benefits, with their combination providing the best overall performance. The alignment constraint ablation (Table 5) is compelling — the near-total collapse of performance without alignment (0 SR vs. 2.44 SR) demonstrates that the alignment mechanism is not an incremental improvement but a necessary condition for the unified framework to function. The VAR scale sweep (Figure 3) reveals a non-obvious optimal compression level — the inverted-U relationship with a single clear peak at scale 4 is clean evidence that too much visual detail degrades reasoning, a finding with implications beyond this paper. The explicit vs. implicit comparison (Table 6) provides direct evidence for the central implicit-reasoning claim and reveals the interesting modality-dependent exception (T-CoT) that complicates a simplistic "implicit is always better" narrative.

Overall Weaknesses

The most significant weakness is the small scale and limited statistical rigor. The LH-VLN test set size is not reported, but the benchmark's structure and the magnitude of the metrics suggest it is on the order of hundreds of tasks (SR of 2.44 means ~12 successful multi-stage completions if there are 500 tasks). With such small numerators, small changes in agent behavior can produce large relative changes in SR, and without confidence intervals, it is unclear whether the difference between 2.44 SR (FantasyVLN) and 0.98 SR (best mode combination) is statistically reliable. The paper does not report variance across random seeds, training runs, or test-set splits.

A second weakness is confounded comparisons. The training efficiency comparison (Figure 5) compares different models rather than the same model with different compression methods. The baseline comparisons (Table 2) include reimplementations that may not match original performance. The base model (Qwen2.5-VL) and the T-CoT annotation model (Qwen-VL-Max) are both recent, strong models, and the paper does not control for base model capability when comparing against baselines that may use older or weaker base models.

A third weakness is missing ablations. The paper does not ablate: the data augmentation strategies (Appendix A.4), the number of predicted actions per step (k=5), the gating signal sampling strategy (uniform), the T-CoT annotation source or quality, the stop-gradient in the alignment loss, the alternating optimization frequency, or the historical image buffer size. Several of these are important hyperparameters whose sensitivity affects the reproducibility and robustness of the results.

A fourth weakness is the lack of diagnostic experiments for implicit reasoning. The paper claims that CoT training causes the model to learn reasoning-aware internal representations that are used during implicit inference, but provides no direct evidence — no probing, no activation analysis, no ablation of intermediate layers, no comparison of attention patterns. The behavioral improvement could be explained by other mechanisms (regularization, data augmentation, longer effective training due to CoT token prediction objectives) without the model learning anything recognizable as "implicit reasoning." The paper would be stronger with even a simple diagnostic, such as measuring whether the non-CoT model's internal states are predictive of the subgoal structure (which the CoT modes explicitly generate) above and beyond what a non-CoT-only model achieves.

6. Limitations and Trade-offs

1. The Difficulty Estimation Overhead Is Unaccounted for in the Headline Efficiency Gains

The assumption or constraint. The entire compute-optimal framework rests on the ability to estimate problem difficulty before allocating the inference budget. The paper's method for difficulty estimation — generating 2,048 samples per question and computing the pass@1 rate (oracle) or averaging PRM final-answer scores (predicted) — is extraordinarily expensive. The paper acknowledges this explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The consequence. The reported 4× efficiency gains over best-of-N are computed as marginal savings after difficulty is known, without amortizing the cost of learning it. In a real deployment, the total cost would be C_difficulty + C_strategy, and since difficulty estimation requires 2,048 samples per question — more than the largest test-time budgets studied (256–512 generations) — the upfront cost could dominate or completely eliminate the claimed savings. For one-shot queries, the approach would be actively wasteful: spending thousands of generations to estimate difficulty before spending dozens on the actual solution. The only regime where this overhead amortizes is when the same difficulty estimate can be reused across many similar queries, which the paper does not address.

What evidence exists in the paper. Figure 4 and Figure 8 plot compute-optimal scaling curves without including the difficulty estimation cost on the x-axis. The x-axis represents only the generation budget for the strategy execution phase. The 4× claim is derived by comparing points on these curves: e.g., at 16 generations, compute-optimal achieves the same accuracy as best-of-N at 64 generations. If difficulty estimation required even 64 generations per question, the effective cost for compute-optimal would be 16 + 64 = 80 generations — worse than the best-of-N baseline at 64. The paper provides no experiment showing total-cost-normalized results.

Mitigation status. The authors flag this as "a key avenue for future work" in Section 3.2, suggesting that models could be trained to "directly predict difficulty of a question" from the text alone, or that difficulty estimation could be made adaptive (start with a few samples, assess difficulty online, then allocate remaining budget). Neither approach is developed or evaluated. In Section 8, the paper reiterates this as a direction for future research. Until this gap is closed, the 4× figure should be understood as an upper bound on achievable efficiency under the idealized assumption that difficulty is known at zero cost, not as a realized deployment gain.

2. The Method Offers No Path Forward for Problems Outside the Base Model's Capability Range

The assumption or constraint. All test-time compute strategies — search, revisions, and their compute-optimal combination — operate on the proposal distribution of the base model. They can amplify the probability of selecting a correct answer if correct answers exist in the model's output distribution at some non-trivial rate, but they cannot generate correct solutions to problems the base model fundamentally cannot solve.

The consequence. On the hardest difficulty quintile (bin 5), the paper reports near-zero improvement regardless of method or budget. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all search methods across budgets from 4 to 256 generations. In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio at a budget of 128 generations. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%. This means that for problems where the base model's pass@1 is near zero, the compute-optimal framework provides no benefit whatsoever — pretraining a larger model remains the only viable path. This is a fundamental capability ceiling, not a minor limitation: it applies to the hardest 20% of problems in the MATH benchmark (by the model's own capability distribution) and likely extends to any task distribution with a substantial tail of problems beyond the base model's reach.

What evidence exists in the paper. Every difficulty-bin analysis across both search (Figure 3, right) and revisions (Figure 7, right) shows bin 5 as a flat line near zero. The FLOPs-matched comparison in Section 7 explicitly confirms this: "On the hardest questions (bins 4–5), pretraining is almost always more effective." The paper's own characterization in the Section 7 takeaway box is candid:

"test-time compute is powerful when problems are within the base model's reach (it already produces correct solutions at some non-trivial rate), but it cannot compensate for fundamental capability gaps that larger pretraining would address"

Mitigation status. The paper is transparent about this limitation and does not claim otherwise. The practical implication is clear but underexplored: any deployed system using compute-optimal test-time scaling needs a mechanism to detect when a problem is outside the base model's range and escalate to a larger model or flag for human intervention, rather than wasting compute on strategies guaranteed to fail. The difficulty estimator described in Section 3.2 could in principle serve this role — bin 5 problems would be identified as such — but the paper does not report results on an escalation strategy or characterize the tradeoff between false-positives (escalating problems the base model could solve) and false-negatives (wasting compute on problems it cannot).

3. The Revision Model Suffers from a Severe Correct-to-Incorrect Reversion Problem

The assumption or constraint. The revision model is trained exclusively on sequences where all in-context answers are incorrect, followed by a correct target answer. This is a direct consequence of the training data construction procedure described in Section 6.1: the model sees only incorrect-to-correct trajectories during supervised fine-tuning. It never sees examples where the current answer is already correct, nor is it trained to recognize when no revision is needed.

The consequence. At test time, approximately 38% of correct answers produced during a revision chain get converted back to incorrect answers in the subsequent revision step (reported in Section 6.1). This means the revision process is not a monotonic improvement — it is a random walk with a downward bias once a correct answer is found. The paper's mitigation is to apply majority voting or verifier-based selection across the entire chain of revisions rather than taking the final revision output, but this is an imperfect patch. In a chain of length N, a correct answer appearing at step k has a 38% chance of being overwritten at each subsequent step, so the probability that the last revision is correct given that step k was correct is (1 - 0.38)^{N-k}, which decays rapidly. For a long chain, the final revision is almost certainly incorrect even if a correct answer appeared earlier. This makes the sequential revision strategy inherently unreliable as a standalone mechanism and forces dependence on the within-chain selection mechanism, which may itself fail (majority voting across revisions where 38% are incorrect-to-incorrect revisions and some correct answers have been overwritten).

What evidence exists in the paper. The 38% reversion rate is stated explicitly in Section 6.1. The paper also reports in Appendix K (Figure 16) that an attempt to further optimize the revision model using ReST $^{EM}$ "backfires," causing "additional sequential revisions to substantially hurt performance." At 256 generations, fully sequential performance with the ReST $^{EM}$ model drops to approximately 33.5% compared to roughly 38.5% at the optimal ratio, suggesting the reversion problem is exacerbated by on-policy data collection. This negative result indicates that the revision approach is sensitive to training methodology in ways that are not fully understood.

Mitigation status. The paper partially mitigates this with within-chain selection (majority voting or verifier), but acknowledges in Section 8 that a more principled solution — "such as training the model to recognize when no revision is needed" — is not explored. The compute-optimal revision strategy (Figure 7) implicitly mitigates this by preferring balanced sequential-to-parallel ratios on harder problems (where revisions are less reliable) and fully sequential on easier problems (where the initial answer is more likely correct and revisions are refinements rather than corrections). However, the 38% reversion rate represents a fundamental limitation of the training data construction approach and limits the ceiling of purely sequential revision strategies, particularly for long chains.

4. Revisions and Search Are Studied Independently, Not Combined

The assumption or constraint. The paper studies two complementary test-time compute mechanisms — PRM-guided search (Section 5) and iterative revisions (Section 6) — but never combines them. The compute-optimal policy treats them as independent strategy choices (e.g., "use beam search for medium-difficulty problems" or "use 4:1 sequential-to-parallel ratio for revisions") and selects between them per difficulty bin, but never deploys both simultaneously on the same problem.

The consequence. The paper explicitly acknowledges in Section 8:

"we did not experiment with PRM tree-search techniques in combination with revisions"

This is a significant gap because the two mechanisms have complementary strengths that could compound. Revisions improve the proposal distribution — the revision model generates better candidate solutions than the base model by conditioning on previous attempts. PRM search improves candidate selection — beam search with a verifier finds the best solution among generated candidates. Applying beam search to the revision model's outputs — or using the PRM to guide which revisions to pursue (e.g., pruning unpromising revision branches, prioritizing revisions that score well) — could yield gains beyond either method alone. The current results therefore represent a lower bound on what a fully integrated system could achieve, and it is unknown whether the gains from search and revisions are additive (the combined method achieves the sum of individual improvements), subadditive (they overlap and combining them yields diminishing returns), or even superadditive (revision-generated candidates are easier to search over because they are higher quality).

What evidence exists in the paper. Tables 3 and 4 show that compute-optimal strategies for search and revisions each independently achieve 4× efficiency gains over best-of-N baselines, and the FLOPs-matched comparison in Figure 9 shows that revisions generally outperform search in the FLOPs-matched setting. However, there is no experiment that combines both — e.g., a revision model generating multiple candidate chains, with PRM beam search applied to select the best chain and the best answer within each chain. The paper's difficulty-bin analyses (Figures 3 right and 7 right) also suggest a natural combination: on medium-difficulty problems, beam search works well (Figure 3, right, bin 3) and revisions work moderately (Figure 7, right, bin 3); on easy problems, revisions excel (Figure 7, right, bin 1) but beam search over-optimizes (Figure 3, right, bin 1). A combined system could use revisions + beam search on medium problems and revisions + best-of-N on easy problems, potentially outperforming either mechanism alone on both difficulty tiers.

Mitigation status. The paper identifies this as a direction for future work in Section 8 and does not attempt to address it. The compute-optimal framework is described as a meta-strategy that selects among existing methods, not one that synthesizes new combined methods. This is a scope limitation rather than a failure of the existing experiments, but it means that the reported performance numbers should not be interpreted as the ceiling of what the framework could achieve with further engineering.

5. All Results Are on a Single Benchmark with a Single Model Family, and the Test Set Is Small

The assumption or constraint. Every experiment in the paper uses the MATH benchmark (500 test questions, Section 4) with PaLM 2-S* as the base model. The paper states in Section 4 that it "believe[s] this model is representative of the capabilities of many contemporary LLMs," but this claim is untested. All quantitative claims — the 4× efficiency gain, the difficulty-dependent scaling curves, the optimal sequential-to-parallel ratios, the FLOPs-matched tradeoff boundaries — are conditional on this specific model-benchmark combination.

The consequence. Several aspects of the findings could be model-specific or benchmark-specific in ways that would substantially change the practical recommendations:

The PRM's quality and over-optimization behavior depend on PaLM 2-S*'s output distribution. A model with different calibration properties could exhibit different difficulty-dependent scaling curves — the threshold at which beam search begins to over-optimize (currently bin 1–2) could shift, changing the compute-optimal strategy.
The revision model's ability to learn from incorrect in-context examples depends on the base model's in-context learning capabilities, which vary substantially across model families (GPT-4 vs. Claude vs. LLaMA vs. PaLM).
The MATH benchmark consists exclusively of competition-level math problems requiring symbolic reasoning and exact-answer matching. The difficulty-dependent patterns — e.g., beam search hurting easy problems, revisions helping easy problems — may not generalize to coding benchmarks (where execution feedback is available), open-ended generation tasks (where correctness is ambiguous), or knowledge-intensive QA (where factuality rather than reasoning is the bottleneck).
The optimal VAR compression scale (scale 4, achieving 7.41 ISR), the best sequential-to-parallel ratio for each difficulty bin, and the specific difficulty thresholds that trigger strategy switches are all tuned to this specific setup and would need re-estimation for any new model or benchmark.

What evidence exists in the paper. All experiments are on MATH with PaLM 2-S*. The paper does not report results on any other benchmark (e.g., GSM8K for math reasoning, HumanEval for code, MMLU for knowledge) or with any other model family. The test set of 500 questions, split into five difficulty quintiles of approximately 100 questions each, then further split by two-fold cross-validation, means the compute-optimal policy is selected based on roughly 50 questions per fold per bin. The paper does not report confidence intervals, standard errors, or statistical significance tests for any metric. With SR in the 2–3% range for the hardest problems and numerator sizes of perhaps 2–3 successful completions, small changes in individual question outcomes could produce large relative changes in reported metrics.

Mitigation status. The paper does not claim generalization beyond MATH or PaLM 2-S*, but it also does not discuss this as a limitation. The authors suggest in Section 8 that future work could extend to "code generation (e.g., HumanEval, MBPP), logical reasoning (e.g., ARC, FOLIO), scientific QA, and open-ended generation tasks." However, several of these extensions would require fundamentally different verifier training approaches — coding tasks have natural execution-based verifiers (unit tests), but open-ended generation lacks clean correctness signals entirely. The paper provides no guidance on how the compute-optimal framework would adapt to such settings.

6. The `14×` Larger Model Baseline in the FLOPs-Matched Comparison Is Not Compute-Optimally Trained

The assumption or constraint. The FLOPs-matched comparison in Section 7 uses a 14× larger model (parameter-scaled only, holding training data fixed) following the LLaMA paradigm (Touvron et al., 2023). The paper acknowledges in Section 7 that this departs from compute-optimal pretraining (Hoffmann et al., 2022), where both model parameters and training data would be scaled equally:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

Additionally, the larger model is evaluated using only greedy decoding — no test-time compute augmentation of its own (no majority voting, no best-of-N, no search, no revisions). This stacks the comparison in favor of test-time compute by giving the smaller model sophisticated inference strategies while giving the larger model none.

The consequence. The reported advantages of test-time compute over pretraining — e.g., +27.8% relative improvement on easy questions at R ≪ 1 for revisions, as shown in Figure 1's bar charts — may shrink or reverse against a properly compute-optimal larger model. A Chinchilla-optimal model trained with 14× more total FLOPs (scaling both parameters and data by roughly √14 ≈ 3.7× each) would likely outperform a parameter-only-scaled model, making the pretraining baseline stronger. Similarly, giving the larger model even a modest test-time compute budget — say, best-of-8 or best-of-16 — would create a much fairer comparison, since the 14× model costs 14× more per token at inference, but its total inference budget in the comparison could still accommodate a few parallel samples. The paper's conclusion that "a smaller model augmented with compute-optimal test-time strategies can outperform a ~14× larger pretrained model" (Section 7) is therefore conditional on the larger model being trained suboptimally and evaluated with a weak inference strategy.

What evidence exists in the paper. The paper is transparent about the parameter-only scaling choice, citing the LLaMA paradigm as justification. It acknowledges the departure from compute-optimal pretraining and flags it as future work. However, it does not discuss the greedy-decoding baseline for the larger model as a limitation — this is simply presented as the default configuration. The FLOPs accounting in Section 7 is careful about matching total FLOPs, but the allocation of those FLOPs is asymmetric: the smaller model gets their full budget optimized across search/revision strategies, while the larger model gets none of its budget allocated to inference-time optimization.

Mitigation status. The paper identifies the compute-optimal pretraining comparison as future work in Section 7. However, the greedy-decoding baseline is not flagged as a limitation, and no experiment shows results with the larger model receiving even a small test-time compute budget. A simple extension — running best-of-4 or best-of-8 on the larger model and comparing against the compute-optimal smaller model at equivalent total FLOPs — would substantially strengthen the comparison. The paper's current conclusions about the pretraining-vs-inference tradeoff should be interpreted as lower bounds on pretraining's competitiveness — a stronger pretraining baseline would shift the crossover points in Figure 9 to the left (making test-time compute less favorable at higher R values and harder difficulty bins).

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a conceptual reframing rather than a paradigm shift. It does not propose a fundamentally new model architecture or learning algorithm — all components (Qwen2.5-VL, VAR compression, gated multi-task training, knowledge distillation via alignment) are drawn from existing toolkits. The reframing is this: multimodal Chain-of-Thought reasoning for embodied tasks is primarily valuable as a training-time representation learning mechanism, not as a runtime computation strategy. Prior work (NavCoT, NavGPT-2, CoT-VLA, WorldVLA) treated CoT reasoning as a cause — generate intermediate tokens, and those tokens causally improve downstream action predictions. Aux-Think took a step toward reframing by showing that textual CoT training benefits could persist at inference without CoT generation. This paper completes that reframing and extends it to the multimodal case: textual, visual, and multimodal CoT are three different projections of shared task structure (semantic plans, spatial layouts, subgoal progress), and training a model to produce them forces it to learn representations that capture that structure. The CoT tokens themselves are scaffolding that can be discarded once the representations are built. The cross-mode alignment constraint is the mechanism that operationalizes this — it says "learn to produce the same actions whether you reasoned textually, visually, both, or neither," which is functionally equivalent to saying "learn action-determining representations that are invariant to reasoning modality."

This reframing resolves a latent tension in the embodied CoT literature. On one side, works like VISTA, CoT-VLA, and DreamVLA argue that visual imagination is essential for spatial grounding — you need to explicitly predict what you will see to plan movements in 3D space. On the other side, the real-time constraint of embodied tasks makes explicit visual generation (thousands of tokens per step) completely impractical, and Table 2 shows that existing visual CoT methods score zero on the long-horizon LH-VLN benchmark, confirming that the approach does not scale. The tension is: visual reasoning seems necessary but explicit visual generation is impossible. FantasyVLN's reframing dissolves this tension by separating the reasoning function (anticipating spatial layouts, verifying action feasibility) from the generation process (autoregressively decoding visual tokens). The VAR latent space provides a compact medium where the model can learn to anticipate future visual states during training (via the 30-token V-CoT target) without needing to generate those states at inference. The spatial reasoning happens in the model's internal representations, shaped by the V-CoT training objective, but it never surfaces as explicit tokens. This is conceptually similar to how a chess player visualizes future board positions without needing to draw them — the representation is internal and action-determining, not externally rendered.

The practical consequence of this reframing is that it makes the entire visual CoT research direction more tractable for real-time embodied systems. Prior visual CoT work faced a hard ceiling: no matter how clever the reasoning mechanism, generating thousands of visual tokens per step will always be too slow for real-time control. This paper demonstrates that the ceiling is artificial — you can achieve the representational benefits of visual reasoning without the token generation overhead. This shifts the research agenda from "how do we make visual CoT generation faster?" (a hardware or model-compression problem) to "what is the optimal visual representation for reasoning during training, and how do we best distill it into implicit policies?" (a representation learning and knowledge distillation problem). The latter is a richer and more tractable research space, with connections to self-supervised learning, information bottleneck theory, and modular policy architectures that the generation-centric view did not engage.

A secondary contribution is what the paper reveals about the information content required for spatial reasoning in navigation. The VAR scale sweep (Figure 3) produces a clean inverted-U relationship: scale 4 (30 tokens, capturing room layout and major furniture but not textures) achieves 7.41 ISR, while scale 10 (most faithful reconstruction, with textures and fine details) drops to 0.90 ISR — worse than scale 3 (3.76 ISR, where reconstruction is barely above color blobs). This is not an efficiency tradeoff (more tokens = higher quality but more cost) — it is a quality inversion where more visual information produces strictly worse navigation. The paper's interpretation — that fine-grained visual prediction gradients compete with and partially override action prediction gradients — suggests that the optimal training signal for spatial reasoning is a specific, intermediate level of abstraction. This finding has implications beyond VLN: it suggests that visual pretraining objectives for embodied tasks should not aim for photorealistic reconstruction (the dominant paradigm in computer vision) but for task-relevant structural abstraction. The VAR's hierarchical latent structure happens to provide this abstraction naturally because low scales capture global layout while discarding texture, but the principle — use the coarsest representation that preserves task-relevant spatial information — could be operationalized through other architectures.

The paper also redirects attention from search algorithm sophistication to verifier robustness as the primary bottleneck for any approach that uses a learned scoring function. While this contribution is primarily from the earlier sections (5.3 identifies over-optimization as the ceiling for PRM search, and Appendix K shows that ReST-based revision model optimization backfires due to spurious correlations), FantasyVLN's implicit reasoning approach sidesteps the verifier problem entirely by not using a learned verifier at inference. The cross-mode alignment constraint is a form of self-verification during training — the non-CoT mode acts as a teacher that the CoT modes must agree with — but this is architecturally simpler and more robust than training a separate PRM and searching against it. This suggests that for tasks where explicit search over a reward model is fragile (as Sections 5.3 and Appendix K demonstrate for MATH), implicit distillation approaches may be more reliable.

The paper makes visual CoT in VLN newly viable and makes explicit multimodal CoT at inference newly obsolete for this task. Prior to this work, a researcher wanting to incorporate visual reasoning into a VLN system faced two bad options: generate pixel-level visual imaginations and accept impractical latency (CoT-VLA at 0.19 APS), or abandon visual reasoning entirely and use only textual CoT (Aux-Think at 0.97 APS, but missing spatial grounding). FantasyVLN provides a third option: train with compressed visual reasoning and deploy implicitly. The 5.4× speedup over explicit CoT-VLA (1.03 vs. 0.19 APS) and the 3.75× SR improvement over Aux-Think (2.44 vs. 0.65 SR) together demonstrate that this option dominates both alternatives — it is simultaneously faster than explicit visual CoT and more accurate than textual-only implicit CoT. This makes explicit visual CoT at inference a dispreferred paradigm for VLN going forward, not because visual reasoning is unimportant but because it can be internalized more effectively than it can be generated.

Follow-Up Research This Work Enables

Probing the internal representations of CoT-trained models to verify implicit reasoning. The paper's central claim — that CoT training causes the model to internalize reasoning patterns that are applied during implicit inference — is supported only by behavioral evidence (better actions). A direct test would probe whether the non-CoT model's internal activations encode information that the CoT modes explicitly generate. Specifically: take the trained four-mode FantasyVLN model, run it in non-CoT mode on LH-VLN test tasks, and extract intermediate layer activations at the point where the model would be generating CoT tokens in the explicit modes. Train linear probes to predict (a) the current subgoal identity from the textual CoT annotations, (b) the spatial layout (room type, doorway positions) from the VAR latent tokens, and (c) the action sequence from the ground truth. Compare probe accuracy against two baselines: a non-CoT-only model trained without any CoT supervision, and a model trained with CoT but without the alignment constraint. If FantasyVLN's activations are more predictive of subgoal and spatial information than both baselines, this confirms that CoT training + alignment causes the model to learn richer internal representations. If the alignment-ablated model shows similar or higher probe accuracy (despite producing worse actions), it would suggest that the representations are learned but the alignment constraint is needed to make them action-determining. This experiment is straightforward to implement (requires only saving activations during evaluation) and would convert a behavioral claim into a mechanistic one.

Ablating the stop-gradient in the cross-mode alignment constraint to test the circular optimization hypothesis. The paper argues theoretically that removing stop-gradient from the alignment loss would cause a "circular optimization problem" where the non-CoT mode and CoT modes chase each other rather than converging. This is never tested experimentally. A clean ablation would train three variants of the four-mode FantasyVLN: (a) the standard alternating optimization with stop-gradient (the paper's method), (b) alternating optimization without stop-gradient (gradients flow from the alignment loss back through the non-CoT pathway), and (c) joint single-phase optimization without alternation (all modes trained together in a single loss, with and without stop-gradient). Measure SR, ISR, and training stability (loss curves, gradient norms). The prediction from the paper's argument is that (b) and (c) should degrade or destabilize relative to (a). If (b) performs comparably to (a), the stop-gradient is unnecessary and the circular optimization concern is empirically unfounded — an important negative result that would simplify the training procedure. If (c) performs well, the alternating optimization is also unnecessary. This experiment would clarify which components of the training algorithm are essential versus incidental.

Replacing VAR compression with VAE-based compression at matched compression ratios to isolate the VAR architecture's contribution. The paper claims that VAR's "next-scale prediction paradigm" and hierarchical latent structure are responsible for the effectiveness of CompV-CoT, but the only comparison against alternative compressors is a reconstruction MSE table (Table 1) that does not measure downstream navigation performance. A controlled experiment would compare three visual CoT training configurations: (a) the standard FantasyVLN V-CoT with VAR at scale 4 (30 tokens, 1/2185 compression), (b) V-CoT with VAE compression truncated to produce approximately 30 tokens (aggressive downsampling of the VAE latent grid), and (c) V-CoT with VQ-VAE at a codebook size producing approximately 30 discrete tokens per image. Train all three on the same LH-VLN data with the same non-CoT + V-CoT configuration, and measure ISR. If VAR significantly outperforms VAE and VQ-VAE at matched token counts, the hierarchical structure is specifically beneficial — perhaps because low-scale latents naturally capture global layout. If all three perform similarly, any compression to ~30 tokens suffices, and the VAR choice is incidental. This experiment would determine whether future work should invest in VAR-specific architectures or can use simpler, more widely available compressors.

Evaluating FantasyVLN on a benchmark with ground-truth execution feedback to test whether implicit reasoning benefits persist with stronger supervision. A natural concern is that FantasyVLN's advantage over explicit CoT may be specific to the LH-VLN setting, where training data is limited (18K slices) and CoT annotations are model-generated (Qwen-VL-Max T-CoT data) rather than verified through environment interaction. A stronger test would apply the same implicit reasoning framework to a VLN benchmark where the agent receives execution feedback — e.g., RL-based fine-tuning where the agent can try actions in the simulator and learn from success/failure signals. Train FantasyVLN variants with and without CoT modes on this richer supervision signal, and compare against explicit CoT baselines. If implicit reasoning maintains its advantage even with abundant environment feedback, the internalization mechanism is robust and not an artifact of data scarcity. If explicit CoT catches up or surpasses implicit reasoning under stronger supervision (because the feedback corrects CoT reasoning errors, solving the error-propagation problem), then the implicit advantage is contingent on data-limited regimes and the paper's conclusions should be scoped accordingly. This experiment would be particularly informative for the video-based VLN benchmarks (NaVid, Uni-NaVid) where environment interaction is integral to the training pipeline.

Extending the gating mechanism to dynamically adjust reasoning depth during inference based on uncertainty. The paper uses uniform gating during training but fixed non-CoT mode at inference. An adaptive inference strategy would use the model's own uncertainty to decide when to invoke explicit reasoning: if the non-CoT action predictions have low entropy or high confidence, proceed implicitly; if uncertainty is high, activate T-CoT, V-CoT, or MM-CoT generation for that specific step. This could be implemented by training the model to also predict a confidence score for each action, thresholding that score to trigger explicit reasoning, and measuring the speed-accuracy tradeoff as the threshold varies. The paper's infrastructure already supports this — the gating tokens can be changed at each inference step rather than fixed for the entire trajectory. The key question is whether "occasional explicit reasoning" (on hard steps only) achieves accuracy close to full explicit reasoning with latency close to full implicit reasoning, a Pareto improvement over either extreme. The existing data in Table 6 suggests this could work: T-CoT explicit outperforms T-CoT implicit (0.98 vs. 0.49 SR), so for textual reasoning at least, selectively generating explicit CoT on difficult steps could capture the explicit reasoning benefit while avoiding the latency cost on easy steps. This would require an uncertainty estimation mechanism that the paper does not currently provide — a concrete experiment would train a confidence head jointly with the action head and evaluate whether confidence-calibrated step selection improves the speed-accuracy Pareto frontier.

Testing the complementarity hypothesis on tasks where spatial and semantic reasoning demands can be independently varied. The paper argues that textual and visual CoT provide complementary benefits — semantic decomposition from text, spatial anticipation from vision — but LH-VLN conflates both because all tasks require both semantic planning (multiple subgoals) and spatial navigation (moving through environments). A cleaner test would use a benchmark where these dimensions can be separately manipulated: tasks with complex instructions but simple environments (high semantic demand, low spatial demand) versus tasks with simple instructions but complex environments (low semantic demand, high spatial demand). If the complementarity hypothesis is correct, T-CoT training should provide disproportionate benefits on the former and V-CoT on the latter, and the optimal training mode combination should depend on the task distribution. This could be tested by constructing such a benchmark from existing VLN datasets (e.g., R2R for complex instructions in simple environments, VLN-CE for complex environments with simple instructions) or by systematically perturbing LH-VLN tasks. This experiment would validate the paper's theoretical framework and provide practical guidance for which CoT modes to prioritize given a task distribution.

Practical Applications and Downstream Use Cases

Real-time assistive navigation for visually impaired users. An assistive robot guiding a visually impaired person through an unfamiliar building must continuously interpret natural language directions ("take me to the restroom near the cafeteria"), visually perceive the environment, and issue movement commands in real time — exactly the VLN task. FantasyVLN's implicit reasoning architecture (1.03 actions per second) makes this feasible where explicit CoT approaches (0.19 APS) would introduce ~5-second pauses between each action, breaking the fluidity of guidance. The multi-stage navigation capability (2.44 SR on LH-VLN) means the system could handle complex instructions with multiple waypoints ("first go to the reception desk, then to room 302"), and the compact VAR-based visual reasoning training means the model can learn spatial anticipation without requiring impractical hardware for real-time visual generation. The key deployment advantage is that the full framework — including the VAR model and the CoT annotation pipeline — is needed only during training; the deployed model is a standard fine-tuned Qwen2.5-VL running implicit inference, making it compatible with edge deployment on a robot's onboard GPU.

Warehouse and logistics robot navigation with dynamic task assignment. In a warehouse setting, robots receive continuously updated task instructions ("pick up item A from shelf 7B, deliver to packing station 3, then return to charging dock") and must navigate among moving obstacles, other robots, and changing inventory layouts. The LH-VLN setting directly models this: multi-stage tasks requiring sequential subgoal completion in continuous environments. FantasyVLN's APS of 1.03 means the robot can make navigation decisions at roughly 1 Hz — sufficient for warehouse speeds (typically ~1 m/s for autonomous mobile robots) where a decision is needed approximately every meter of travel. The 4-mode training strategy is particularly valuable here because the robot benefits from semantic planning (decomposing pick-and-place sequences from textual instructions) and spatial reasoning (anticipating the visual appearance of shelf labels and aisle markers for localization), which are precisely the complementary modalities FantasyVLN trains jointly. The system's ability to internalize reasoning means that even when task instructions change frequently, the robot does not slow down to "think" explicitly about each new task — the reasoning patterns from training generalize to novel instruction compositions.

Data generation for self-improving navigation policies through implicit reasoning distillation. A common approach in embodied AI is to use a capable but slow "teacher" policy (e.g., an explicit CoT model running on a server-grade GPU) to generate training trajectories for a fast "student" policy (a lightweight model deployed on the robot). FantasyVLN inverts this: the same model serves as both teacher and student, with the CoT modes generating rich reasoning traces during offline training and the implicit mode providing fast inference at deployment. This architecture is particularly suited for iterative self-improvement loops: deploy the implicit policy to collect new trajectories in novel environments, use a larger annotation model (like Qwen-VL-Max) to generate CoT traces on those trajectories offline, fine-tune the model on the expanded dataset with all four reasoning modes, and redeploy the improved implicit policy. This avoids the distribution-shift problem that typically plagues student-teacher distillation (the teacher's output distribution differs from the student's) because the same parameters process both modes. The LH-VLN results provide a concrete baseline: starting from 2.44 SR, each self-improvement iteration could expand the training set with trajectories from environments where the current policy succeeds, gradually improving coverage of diverse spatial layouts and instruction phrasings. The key metric to track would be whether SR on held-out environments improves with each iteration without catastrophic forgetting of previously mastered environments.

Sim-to-real transfer for drone navigation with visual reasoning. Drones operating in GPS-denied indoor environments must navigate using only onboard cameras and natural language instructions ("fly through the corridor, turn left at the second doorway, land on the table in the third room"). The real-time constraint is even stricter than for ground robots — a drone moving at 2 m/s needs decisions at 5–10 Hz, making explicit CoT generation (0.19 APS) completely infeasible. FantasyVLN's implicit inference at 1.03 APS is still below ideal drone control rates, but the approach of training with visual CoT in compressed latent space could be adapted to faster hardware or smaller models to reach higher frequencies. The key transfer vector is the CompV-CoT design: training the model to anticipate future visual observations in a compact latent space forces it to learn spatial relationships (distance to walls, presence of doorways, room geometry) that are invariant to visual domain shift between simulation and reality. A sim-to-real experiment would train FantasyVLN entirely in a high-fidelity indoor simulator (with photorealistic rendering), then deploy the implicit policy on a real drone with onboard RGB cameras, measuring the SR degradation from sim to real. If the degradation is modest, it would validate that the latent spatial representations learned through V-CoT training transfer across the visual domain gap — the model learns "room geometry" rather than "simulator textures."

When to Prefer This Method

The paper explicitly positions FantasyVLN against two classes of alternatives: (a) explicit CoT methods that generate reasoning tokens at inference (CoT-VLA, WorldVLA, and the explicit inference variants in Table 6), and (b) single-modality implicit methods that train with one CoT mode (Aux-Think with textual CoT only, and the mode-combination ablations in Table 3). The decision rules are clear from the experimental results:

Prefer FantasyVLN (multimodal implicit) over explicit CoT when the deployment task requires real-time action rates above ~0.5 APS (i.e., decisions must be made in under 2 seconds). The 5.4× speedup over CoT-VLA (1.03 vs. 0.19 APS) is the primary justification, and Table 6 shows that implicit inference also achieves higher accuracy for visual and multimodal CoT (1.46 vs. 0.49 SR for V-CoT, 2.44 vs. 0.98 SR for MM-CoT). The only exception is purely textual CoT where explicit outperforms implicit (0.98 vs. 0.49 SR in the T-CoT-only configuration), but the full four-mode FantasyVLN (2.44 SR) substantially exceeds even the best explicit configuration. The speedup is most consequential for long-horizon tasks (100+ actions) where explicit CoT's ~5 seconds per action would produce unworkable total navigation times.
Prefer FantasyVLN over single-modality implicit methods (e.g., Aux-Think, non-CoT + V-CoT only) when the task involves both semantic complexity (multi-stage instructions requiring subgoal decomposition) and spatial complexity (navigation in geometrically complex environments). Table 3 shows that the full four-mode training achieves 2.44 SR vs. 1.46 for non-CoT + V-CoT (the best two-mode configuration) and 0.65 for Aux-Think — a 67% and 275% relative improvement respectively. The complementarity evidence (T-CoT contributing semantic planning, V-CoT contributing spatial anticipation, their combination providing the best results) means tasks that skew heavily toward only one type of complexity might suffice with single-mode training, but tasks with both require the full framework.
The cross-mode alignment constraint and alternating optimization are non-optional — Table 5 shows that removing alignment collapses SR from 2.44 to 0, making the framework no better than random for multi-stage completion. Any replication or adaptation of this approach must include both the stop-gradiented alignment loss and the alternating optimization procedure. The VAR compression at scale 4 (30 tokens) is also essential if visual CoT is included — Figure 3 shows that higher scales degrade performance. However, the specific compressor choice (VAR vs. VAE at matched compression) has not been ablated, so VAE-based compression at ~30 tokens may be substitutable.
Do not use this method (in its current form) when (a) training data is below ~10K trajectory slices — the paper's 18K slices are already at the low end, and Table 6 shows explicit T-CoT outperforms implicit T-CoT (0.98 vs. 0.49 SR), suggesting that with even less data, explicit reasoning may be more reliable because the implicit pathway cannot fully internalize the reasoning patterns; or (b) the base VLM does not support vocabulary expansion for gating tokens, action tokens, and VAR latent tokens — FantasyVLN's unified framework depends on these special tokens to control mode switching and generate compressed visual representations, and adapting to a VLM without extensible vocabulary would require architectural modifications not addressed in the paper; or (c) the navigation task requires sub-500ms decision latency — 1.03 APS (~970ms per action) may be insufficient for high-speed drone flight or dynamic obstacle avoidance, and further speed improvements (model quantization, smaller architectures, speculative decoding) would be needed.

Dependency tree

Papers this one builds on, walking backwards through the same domain and shared topics.

Agentic Reasoning for Large Language Models2026· agents← in-domain predecessor
- Large Language Model Agent: A Survey on Methodology, Applications and Challenges2025· agents← LLM-judged similar
- Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text2026· agents← in-domain predecessor
- Decoding ML Decision: An Agentic Reasoning Framework for Large-Scale Ranking System2026· agents← similar topic
- GeoVista: Web-Augmented Agentic Visual Reasoning for Geolocalization2025· agents← similar topic
Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalization2025· training-methods← topic thread
- ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration2025· training-methods← in-domain predecessor
VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory2026· agents← similar topic
- MolmoAct: Action Reasoning Models that can Reason in Space2025· uncategorized← LLM-judged similar
- ExpSeek: Self-Triggered Experience Seeking for Web Agents2026· agents← in-domain predecessor
Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis2026· agents← similar topic
- DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Search2025· llm-systems← LLM-judged similar
- Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification2026· agents← in-domain predecessor

What it influenced

Later papers in the same chain. Direction reversed.

EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience2026· agents→ in-domain predecessor
- Computer Environments Elicit General Agentic Intelligence in LLMs2026· agents→ in-domain predecessor
- Reflexion: Language Agents with Verbal Reinforcement Learning2023· agents→ similar topic
- Agent Learning via Early Experience2025· agents→ similar topic
- Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents2025· agents→ similar topic
The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoning2026· reasoning→ topic thread
- Lost in the Noise: How Reasoning Models Fail with Contextual Distractors2026· reasoning→ in-domain predecessor
- Monte Carlo, Puppetry and Laughter: The Unexpected Joys of Prompt Engineering2022· reasoning→ similar topic
- Self-Consistency Improves Chain of Thought Reasoning in Language Models2022· reasoning→ similar topic
- STaR: Bootstrapping Reasoning With Reasoning2022· reasoning→ similar topic
VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory2026· agents→ similar topic
- Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning2026· agents→ in-domain predecessor
Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis2026· agents→ similar topic
- CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery2026· agents→ in-domain predecessor
- Scaling Agent Learning via Experience Synthesis2025· agents→ similar topic
- ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training2026· agents→ similar topic