RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models

ArXiv: 2603.21341

🎯 Pitch

Improving an MLLM's embodied reasoning often degrades its VLA's robot control, rather than improving it. RoboAlign solves this by directly aligning the model's chain-of-thought reasoning with low-level action accuracy through reinforcement learning, yielding massive VLA gains—up to 106.6% on real-world tasks—with minimal extra data.

---

1. Executive Summary

This paper introduces ROBOALIGN, a two-stage training framework that aligns multimodal-large-language models (MLLMs) with low-level action policies to reliably improve vision-language-action model (VLA) performance. The framework first applies supervised fine-tuning (SFT) to equip the MLLM with the ability to generate discretized action tokens (FAST tokens) through explicit chain-of-thought reasoning, then refines this reasoning process using Group Relative Policy Optimization (GRPO) with a reward based on action-token prediction accuracy. Evaluated across LIBERO, CALVIN, and real-world robotic benchmarks using Qwen2.5VL-7B-Instruct as the backbone, ROBOALIGN achieves performance improvements of 17.5%, 18.9%, and 106.6% over SFT baselines respectively, while using less than 1% additional data for the RL stage. These gains are most pronounced on long-horizon and goal-conditioned tasks, establishing that direct alignment between embodied reasoning and low-level actions transfers to VLA performance gains that language-only or action-only training fails to deliver, particularly on complex manipulation sequences.

2. Context and Motivation

The Core Problem: Embodied Reasoning Improvements Don't Reliably Translate to Better Robot Control

The fundamental problem this paper tackles emerges from a surprising empirical observation: making MLLMs better at embodied reasoning does not consistently make VLAs better at controlling robots. This is a deeply counterintuitive finding that challenges the dominant assumption in the field — namely, that if you improve an MLLM's ability to answer spatial questions, predict object affordances, reason about temporal sequences, and decompose high-level tasks, the downstream VLA that uses this MLLM as its backbone will naturally perform better at generating precise robot actions.

The paper opens with a striking demonstration of this disconnect. Figure 1 shows VLAs built on top of several open-source MLLMs specifically fine-tuned for embodied reasoning — including RoboBrain 2.0, which achieved the highest reasoning scores among evaluated models and even outperformed GPT-4o on major benchmarks. Yet this same model yielded the lowest VLA performance among all tested backbones. As stated explicitly by the authors:

"Although RoboBrain 2.0 achieved the highest reasoning scores among evaluated MLLMs and even outperformed GPT-4o on major benchmarks (see Table 9), it yielded the lowest VLA performance."

This is not an isolated finding. The paper cites VLM4VLA (Zhang et al., 2026), which systematically studied this relationship and "revealed that the correlation between embodied reasoning capability and VLA performance is inconsistent and highly task-dependent, sometimes even leading to performance degradation." This means the problem is not that a particular MLLM fine-tuning recipe fails, but that the entire paradigm of improving VLAs indirectly through language-based embodied reasoning has a fundamental flaw.

The authors attribute this discrepancy to what they call the modality gap between language and low-level actions. In their framework:

"Optimizing embodied reasoning purely through language supervision does not guarantee improvements in actual action generation."

Let's unpack what this means. When researchers train MLLMs to improve embodied reasoning, the supervision signal is entirely linguistic — the model learns to answer questions about "what action should the robot take next?" or "where is the object located?" in natural language. But the VLA's actual task is to output precise continuous control signals — joint positions, end-effector velocities, gripper commands. These are fundamentally different modalities. A model can learn to describe "move the gripper to the red cup" perfectly without learning the precise sequence of Cartesian coordinates, orientations, and gripper states needed to actually execute that motion. The linguistic reasoning exists in a symbolic, coarse-grained space, while the action generation requires continuous, fine-grained control.

This is not merely a theoretical concern — it manifests as real performance degradation. The paper's experiments show that language-only SFT (training on embodied VQA without any action data) yields mixed results: improvements on relatively simple tasks but drops on complex, long-horizon sequences (Table 3 shows CALVIN length-5 success dropping from 18.1% to 15.3%). The representations learned through language supervision capture high-level task semantics but miss the fine-grained spatial and temporal precision needed for precise manipulation.

Why This Problem Matters: The Promise and Current Limitation of VLAs

The significance of this problem becomes clear when we consider why VLAs are so promising in the first place. VLAs represent a convergence of two powerful ideas in modern robotics:

Leveraging pretrained knowledge. MLLMs like Qwen2.5VL, GPT-4V, and PaLM-E are trained on internet-scale data encompassing visual perception, language understanding, common-sense reasoning, and world knowledge. By using these models as backbones for robotic policies, VLAs can tap into this vast reservoir of knowledge, potentially enabling generalization to novel objects, environments, and instructions that would be impossible with robot-data-only training. This matters enormously for real-world deployment, where robots face open-world variability that cannot be fully captured in any finite demonstration dataset.

End-to-end learning from demonstrations. Rather than requiring hand-engineered perception pipelines, state estimators, and controllers, VLAs learn directly from demonstrations to map raw sensor inputs (images, language instructions) to low-level actions. This simplifies the engineering pipeline and allows the system to learn behaviors that emerge from the interaction of perception and control, rather than being decomposed by human designers into separate modules.

The field has converged on two dominant architectures for VLAs (as the paper notes in Section 2):

1. Discrete action token prediction: The MLLM's output vocabulary is extended with special tokens representing discretized action chunks, and the model generates actions autoregressively — e.g., RT-2 (Brohan et al., 2023), OpenVLA (Kim et al., 2024), FAST (Pertsch et al., 2025).

2. Continuous action experts: The MLLM processes visual and language inputs into latent representations, and a separate action head (often a diffusion policy or transformer) maps these latents to continuous action trajectories — e.g., π0 (Black et al., 2024), GR00T N1 (Bjorck et al., 2025), Octo (Team et al., 2024).

In both architectures, the MLLM's internal representations serve as the bridge between perception and action. The quality of these representations — how well they encode object locations, spatial relationships, task progress, affordances, and manipulation constraints — directly determines how well the action head can produce precise movements.

This is why the disconnect between reasoning improvements and VLA performance is so critical. If improving an MLLM's embodied reasoning doesn't improve its internal representations in ways that help action generation, then the entire pipeline of "improve the MLLM → better VLA performance" is broken. Researchers and practitioners investing in better MLLM reasoning may be optimizing a proxy metric that doesn't translate to the real goal.

Where Prior Approaches Fall Short

The paper identifies several specific limitations in existing work, which we can group into three categories:

1. Language-Only Supervision Cannot Bridge the Modality Gap

The vast majority of prior work on improving MLLMs for embodied tasks uses linguistic supervision signals. The paper catalogs the diversity of these approaches:

• High-level action QA: Asking "which action should the robot take?" from a set of discrete options (Chen et al., 2025; Lynch et al., 2023). The model learns to classify actions verbally but never produces the actual motor commands.

• Spatial reasoning QA: Training on questions about object relationships, bounding box coordinates, or affordance points (Chen et al., 2024a; Xu et al., 2025; Yuan et al., 2024; Song et al., 2025a). The model learns to output coordinates as text tokens — e.g., "[0.350, 0.100, 0.600, 0.400]" — but these are linguistic tokens, not spatial representations that an action head can directly use.

• Point trajectory prediction: Asking the model to predict future 2D trajectories of the end-effector or objects in image space (Ji et al., 2025; Yuan et al., 2025a; Huang et al., 2025a). This gets closer to action-like outputs but remains in pixel coordinates, not robot coordinates, and discards critical information like gripper state, orientation, and temporal dynamics.

• Visual trajectory forecasting: Generating future image frames or feature trajectories (Yang et al., 2025; Ranasinghe et al., 2024). Again, this is image-space prediction, not action-space control.

All of these approaches share the same limitation: the supervision signal is in a different modality and representational space than the actual control outputs the VLA needs to produce. The MLLM learns to be good at answering questions, predicting points, or generating language, but these skills don't automatically transfer to generating precise, continuous action trajectories. The representations optimized for linguistic outputs may not encode the fine-grained geometric and dynamic information that action heads need.

The paper's own experiments make this limitation concrete. When comparing RL alignment with different targets (Table 6), they find that language-based RL (predicting high-level actions as text) improves the LIBERO Goal category substantially (59.0% → 90.0%) but actually degrades Long-horizon performance (65.6% → 58.2%). Similarly, visual-based RL (predicting 2D trajectories) helps Goal tasks (59.0% → 87.8%) but provides only marginal gains on Long tasks (65.6% → 64.6%). Only direct action-based alignment improves both categories simultaneously, including the critical long-horizon tasks (65.6% → 70.0%).

2. SFT-Based Alignment Causes Forgetting and Poor Transfer

A natural attempt to fix the modality gap is to directly train the MLLM to generate actions through supervised fine-tuning. This is what the paper calls "Action-Only SFT" — fine-tuning the MLLM on FAST token prediction from robot demonstration data. The idea is that by learning to output action tokens directly, the model's representations will align with the control task.

This approach does produce large in-domain gains. Table 2 shows Action-Only SFT improving LIBERO Goal from 42.4% to 82.8% — a dramatic improvement. The model clearly learns to map from visual observations and instructions to action tokens for the tasks it was trained on.

However, the paper identifies two critical failure modes:

Catastrophic forgetting of general capabilities. On the zero-shot CALVIN benchmark (Table 3), Action-Only SFT causes a severe degradation: length-1 success drops from 77.8% to 66.1%, and average sequence length drops from 2.16 to 1.26. This is substantially worse than the untuned base model. The authors hypothesize:

"While FAST token training strengthens alignment between instructions and low-level actions in-domain, it also induces forgetting of general MLLM capabilities, leading to reduced zero-shot generalization."

The model overfits to the action prediction task, losing the visual understanding, common-sense reasoning, and generalization abilities that made the pretrained MLLM valuable in the first place. This is a classic catastrophic forgetting problem in fine-tuning — the model specializes to the narrow training distribution at the expense of broad capabilities.

Poor transfer to complex tasks. Even for the LIBERO Long category, where the training and evaluation tasks are from the same domain, Action-Only SFT actually degrades performance (63.2% → 57.6%). The model can handle short, simple tasks but fails on long-horizon sequences requiring sustained reasoning, adaptation, and precise sequencing of multiple subtasks.

The paper further demonstrates the limitations of SFT-based alignment through a comparison with ECoT (Zawalski et al., 2024), which jointly trains reasoning and low-level actions through SFT. Table 7 shows that this approach reduces performance across all categories compared to the base ROBOALIGN SFT model, with Long dropping from 65.6% to 45.6% and average from 78.7% to 67.7%. The authors diagnose:

"We attribute this to the limited generalization of SFT, where knowledge aligned on BridgeV2 transfers poorly to LIBERO, as well as to forgetting effects introduced during SFT."

This is crucial: SFT forces the model to memorize action patterns from a specific dataset (BridgeV2), but these patterns don't generalize to new environments (LIBERO, CALVIN) because the action distributions, object appearances, and task structures differ. The model overfits to the training distribution's action patterns rather than learning generalizable control principles.

3. Existing RL Approaches for Embodied Reasoning Still Operate in the Language Modality

More recent work has applied reinforcement learning (specifically GRPO, inspired by DeepSeek-R1) to improve embodied reasoning in MLLMs. The idea is compelling: rather than memorizing reasoning patterns through SFT, let the model explore diverse reasoning trajectories and optimize those that lead to correct answers. This has shown significant improvements on embodied reasoning benchmarks (Kim et al., 2025a; Song et al., 2025b; Huang et al., 2025a; Azzolini et al., 2025).

However, these approaches still define correctness in terms of language outputs: did the model select the correct multiple-choice answer? Did it predict the right high-level action label? Did it output accurate bounding box coordinates? The RL objective is aligned with improving linguistic reasoning quality, not action generation quality.

The paper's key insight is that this represents a missed opportunity. If we're going to use RL to explore diverse reasoning trajectories, we should optimize those trajectories for the actual downstream task — producing accurate low-level actions — not for proxy linguistic metrics. The reasoning process should be evaluated by whether it leads to correct FAST token sequences, not by whether it correctly answers an embodied question.

This is precisely what ROBOALIGN does differently: it applies GRPO with a reward defined on action token prediction accuracy (Section 4.2). The format reward ensures the model structures its thinking properly, but the accuracy reward directly measures how many FAST tokens match the ground truth:

$r_a = \frac{1}{m} \max\{i \in \{1, \ldots, m\} : T^{\text{gen}}_{1:i} = T^{\text{target}}_{1:i}\}$

This reward function encourages the model to explore reasoning chains that lead to precise action generation, not just plausible-sounding descriptions. Because GRPO samples multiple reasoning trajectories per prompt and reinforces those with high action accuracy, the model discovers which reasoning patterns genuinely help with control versus which ones merely sound good but don't translate to precise movements.

How This Paper Positions Itself

ROBOALIGN positions itself at the intersection of three research threads, synthesizing their strengths while addressing their individual limitations:

From VLA training: The paper adopts the architecture of attaching an action head to a frozen MLLM backbone (following GR00T-N1.5/GEAR, 2025, and the broader VLA paradigm). But rather than accepting the MLLM's representations as fixed, it actively optimizes them for the action task — not by fine-tuning on action data alone (which causes forgetting), but by using RL to align the reasoning process with action accuracy while preserving general capabilities.

From embodied reasoning training: The paper incorporates the rich VQA and reasoning datasets that prior work has developed for spatial, temporal, and task-level understanding. But rather than treating these as the end goal (improving QA accuracy), it uses them as auxiliary signals to maintain and enhance general capabilities while the primary optimization target is action generation.

From reasoning-centric RL: The paper adopts the DeepSeek-R1/GRPO paradigm of incentivizing explicit chain-of-thought reasoning through RL. But rather than defining rewards on linguistic answer correctness, it defines rewards on action token prediction accuracy, directly bridging the modality gap that prior RL-for-embodied-reasoning work left unaddressed.

The paper frames its contribution not as a new architecture or a new dataset, but as a training framework that orchestrates existing components (SFT on diverse data, RL with GRPO, FAST tokenization) around a novel objective: aligning the MLLM's reasoning process with low-level action generation. The two-stage design (SFT → RL) is deliberate:

• Stage 1 (SFT) solves the cold-start problem. Off-the-shelf MLLMs cannot generate FAST tokens or structured reasoning about actions, so they need supervised exposure to this capability. The SFT stage also injects diverse embodied reasoning knowledge, spatial understanding, and general VQA capabilities to preserve the model's broad competence.

• Stage 2 (RL) solves the alignment problem. With the basic capability established, RL explores the space of reasoning trajectories and reinforces those that produce accurate actions, sharpening the model's internal representations for control without the forgetting that would occur from SFT on action data alone.

The paper's empirical positioning is clear: it demonstrates (Figure 1, Tables 2–4) that prior approaches — language-only fine-tuning, action-only fine-tuning, and specialized embodied reasoning MLLMs — all fail to consistently improve VLA performance, particularly on the long-horizon tasks that matter most for real-world deployment. ROBOALIGN is proposed as the solution that reliably bridges this gap.

3. Technical Approach

3.1 Reader Orientation

ROBOALIGN is a two-stage training framework that transforms a general-purpose multimodal-large-language model (MLLM) into a backbone whose internal representations are directly optimized for generating precise low-level robot actions. The core problem it solves is the modality gap — the disconnect between learning to reason about actions in natural language and actually producing accurate continuous control signals — by using reinforcement learning to align the model's chain-of-thought reasoning process with the accuracy of its predicted action tokens, rather than with linguistic answer correctness.

3.2 Big-Picture Architecture (Diagram in Words)

The ROBOALIGN framework consists of five major components connected through a sequential two-stage pipeline:

1. Base MLLM (Qwen2.5VL-7B-Instruct) — the pretrained vision-language model that processes robot camera images and task instructions into latent representations. It serves as the frozen backbone during VLA training and the trainable foundation during MLLM alignment.

2. FAST Tokenizer — a learned compression module that converts continuous action chunks (sequences of 7-DoF end-effector states over a horizon $H$) into discrete token sequences using Discrete Cosine Transform (DCT) followed by byte-pair encoding (BPE). This bridges the continuous action space to the discrete token vocabulary of the MLLM.

3. Stage 1: Supervised Fine-Tuning Data Mixture — a curated collection of 2.28M question-answer pairs spanning four categories: (a) general vision-language understanding from LLaVA-OneVision, (b) embodied reasoning VQA datasets (RefSpatial, RoboPoint, EgoPlan-IT, etc.) and a custom ROBOALIGN VQA dataset, (c) specialized zero-shot reasoning data distilled from a GRPO-trained model, and (d) FAST token prediction data from BridgeV2 where the answer is a sequence of action tokens. This mixture equips the MLLM with three simultaneous capabilities: general multimodal understanding, embodied reasoning, and basic action token generation with explicit chain-of-thought.

4. Stage 2: GRPO-Based Reinforcement Learning — a policy optimization loop that takes the SFT-trained model, samples multiple reasoning-and-action responses per robotic manipulation prompt, evaluates each response using a composite reward (format correctness + FAST token prefix accuracy), and updates the policy to favor reasoning trajectories that lead to accurate action sequences. This stage uses only 12.8K samples — less than 1% of the SFT data.

5. VLA Conversion Pipeline — a post-alignment procedure that attaches a newly-initialized diffusion-based action head to the frozen MLLM backbone and trains only the action head on target robot datasets (LIBERO, CALVIN, real-world) while keeping the MLLM fixed. This tests the quality of the aligned representations without further modifying them.

Information flows as follows: robot images and task instructions enter the base MLLM → the MLLM generates a reasoning trace enclosed in thinking... tags followed by a sequence of FAST tokens enclosed in <answer> tags → during SFT, the model is trained to match these tokens to ground truth via next-token prediction → during RL, multiple such reasoning-token sequences are sampled, scored by how many leading FAST tokens match ground truth, and the best-performing generation patterns are reinforced → after MLLM training is complete, the frozen model processes new robot observations into latent representations, and a separately trained action head maps these latents to continuous action trajectories.

3.3 Roadmap for the Deep Dive

• First, the FAST tokenization scheme — how continuous action chunks are converted into discrete tokens the MLLM can generate. This is necessary to understand before anything else because it defines the output space that both SFT and RL operate on.

• Second, the Stage 1 SFT data mixture and training procedure — what data goes in, why each component is included, and how the MLLM is fine-tuned to gain the initial capability of generating FAST tokens with reasoning. This establishes the foundation that RL builds upon.

• Third, the Stage 2 GRPO-based RL procedure — the reward function, the policy optimization objective, and the training configuration. This is the core novel contribution.

• Fourth, the VLA conversion pipeline — how the trained MLLM is frozen and combined with an action head to produce the final robot policy. This connects the MLLM training to the actual evaluation metric (robot task success rate).

3.4 Detailed, Sentence-Based Technical Breakdown

This is a training methodology paper whose core idea is that aligning an MLLM's chain-of-thought reasoning directly with low-level action prediction accuracy, using reinforcement learning with action-token rewards, produces internal representations that transfer substantially better to downstream VLA training than either language-only or action-only supervised fine-tuning alone.

---

FAST Action Tokenization

The first technical challenge ROBOALIGN must solve is how to represent continuous robot actions in a form that an MLLM can generate. MLLMs operate on discrete tokens — they produce sequences from a fixed vocabulary through autoregressive sampling. Robot actions, however, are continuous-valued vectors: the end-effector's Cartesian position $(x, y, z)$, orientation $(\text{roll}, \text{pitch}, \text{yaw})$, and gripper state (open/close). These are fundamentally different representational spaces, and the choice of how to bridge them has significant consequences for what the MLLM can learn.

ROBOALIGN adopts FAST tokenization (Pertsch et al., 2025), which compresses action sequences into compact discrete token sequences through a three-stage pipeline. The paper justifies this choice by noting that FAST "has been shown to be effective not only for end-to-end policy learning but also for representation learning," citing subsequent work that uses FAST-derived representations for various robotics tasks (Black et al., 2025; Driess et al., 2025).

Stage 1: Action Chunking and DCT Compression. Rather than tokenizing individual action vectors one at a time, FAST operates on action chunks — sequences of actions over a horizon of $H$ timesteps. A chunk is defined as:

$a_{1:H} = [[a_{1,1}, a_{1,2}, \ldots, a_{1,D}], \ldots, [a_{H,1}, a_{H,2}, \ldots, a_{H,D}]]$

where $D$ is the dimensionality of each action (7 in the paper's setup: 3 for position, 3 for orientation, 1 for gripper) and $H$ is the chunk length (the number of future timesteps predicted at once). Each action vector $a_t \in \mathbb{R}^D$ represents the desired end-effector state at timestep $t$.

What this captures: the full trajectory of the robot's end-effector over a short future window, including position, orientation, and whether the gripper should be open or closed at each step. This is the complete control signal that a robot controller needs to execute.

This chunk is then transformed into the frequency domain using a Discrete Cosine Transform (DCT) (Ahmed et al., 2006). The DCT converts the time-domain sequence of action vectors into a sequence of frequency coefficients. The key property that makes DCT valuable here is energy compaction: for smooth trajectories (which robot motions typically are), most of the signal's information concentrates in the first few low-frequency coefficients, while high-frequency coefficients capture fine-grained jitter that matters less for task success. This means the DCT representation can be aggressively compressed with minimal information loss.

Why DCT over alternatives: The paper doesn't explicitly compare DCT to alternatives like FFT or learned autoencoders, but the cited FAST work established that DCT provides better compression ratios for robot trajectories than direct temporal encoding because of the smoothness prior. A learned VAE-based compression would require training yet another model and might not generalize across robot embodiments. DCT is a fixed, deterministic transform with well-understood properties.

Stage 2: Quantization and BPE Tokenization. The DCT coefficients are continuous-valued, so they must be discretized. FAST quantizes them into a finite set of values, then flattens the 2D (timestep × coefficient) array into a 1D sequence. This flattened sequence is then compressed using byte-pair encoding (BPE) (Gage, 1994) — the same subword tokenization algorithm used for natural language. BPE iteratively merges the most frequent pairs of consecutive symbols in the training corpus, building a vocabulary of subword units that can efficiently represent common patterns.

The result is a sequence of $T_k$ discrete tokens:

$T_k = \text{FAST}(a_{1:H})$

where each token is drawn from a vocabulary of $2K$ special tokens that are added to the MLLM's existing vocabulary. The parameter $K$ controls the vocabulary size: with $K=512$ (typical for FAST), there are 1024 possible action tokens. This is small relative to the MLLM's text vocabulary (typically 100K+ tokens) but large enough to represent the necessary action diversity.

Stage 3: Vocabulary Extension. The MLLM's tokenizer is extended with the $2K$ FAST tokens plus two special marker tokens: <ACTION_START> and <ACTION_END>. These delimiters let the model clearly separate reasoning text from action token generation and allow downstream parsers to extract the action sequence from the model's output. During inference, the MLLM generates natural language reasoning first, then emits <ACTION_START>, then the FAST token sequence, then <ACTION_END>.

What this enables: the MLLM can now generate robot actions using exactly the same autoregressive next-token prediction mechanism it uses for text. The action tokens are just additional entries in the output vocabulary. This means all the MLLM's pretrained capabilities — visual understanding, instruction following, common-sense reasoning — can potentially influence action generation through the same transformer architecture, without requiring a separate action prediction module during the MLLM training phase.

Design choice — why tokenize actions at all: The alternative would be to have the MLLM output continuous action vectors directly through a regression head, or to use the MLLM's latent representations as input to a separate action policy without ever generating actions as tokens. Tokenization enables the MLLM's reasoning and action generation to be expressed in a unified autoregressive framework, which means RL algorithms designed for language (like GRPO) can be applied directly to action generation. It also means the model's internal reasoning process — the thinking... chain — can directly condition the action token predictions, since both are produced by the same autoregressive sequence model.

---

Stage 1: Supervised Fine-Tuning with a Multi-Objective Data Mixture

The first stage of ROBOALIGN solves a cold-start problem: an off-the-shelf MLLM like Qwen2.5VL-7B-Instruct has never seen FAST action tokens and has never been trained to produce structured chain-of-thought reasoning before generating actions. If we attempted RL directly, the model would have zero probability of producing valid action sequences, so no reward signal could propagate. SFT provides the initial capability that RL then refines.

The SFT stage has a more subtle objective than just enabling action token generation, however. The paper explicitly states that the SFT stage must accomplish three things simultaneously:

"equip the MLLM with the ability to generate FAST action tokens... while simultaneously preserving its general vision-language capabilities and enhancing its embodied reasoning skills"

This multi-objective requirement drives the design of the SFT data mixture, which the paper constructs from four distinct data sources totaling 2.28M question-answer pairs.

Data Component 1: General Vision-Language Understanding (100K samples)

To prevent catastrophic forgetting of the MLLM's broad capabilities, the paper includes 100K single-view samples from LLaVA-OneVision (Li et al., 2024). This is a large-scale VQA dataset covering diverse topics: object recognition, scene description, visual reasoning, OCR, and common-sense QA about images. By continuing to train on this data alongside the new robotics-specific data, the model maintains its ability to understand general visual scenes and follow natural language instructions — capabilities that the VLA will need when deployed in novel environments.

The paper notes that it uses only the single-view portion of LLaVA-OneVision, since robot manipulation environments in the evaluated benchmarks provide single-camera views (confirmed by Figure 3, which shows single RGB images as observations). Multi-view data would add unnecessary training cost without matching the deployment setting.

Data Component 2: Embodied Reasoning VQA (1.2M samples)

This is the largest component of the SFT mixture, designed to enhance the MLLM's understanding of spatial relationships, object affordances, task structures, and robot-specific knowledge. The paper draws from several existing open-source datasets and contributes one custom dataset:

Existing datasets (650K samples):

• RefSpatial (Zhou et al., 2025): 300K samples. A dataset for spatial referring expression understanding — given an image and a natural language description of a spatial relationship, the model must identify the correct object or region. This teaches the model to ground spatial language in visual observations.
• RoboPoint (Yuan et al., 2024): 200K samples. Trains models to predict 2D points or bounding boxes corresponding to object affordances — e.g., "where should the gripper grasp this mug?" The spatial grounding task teaches fine-grained coordinate prediction relevant to manipulation.
• EgoPlan-IT (Chen et al., 2023): 50K samples. A dataset for egocentric task planning from video — the model must predict the next step in a procedural task given the current visual observation. The paper converts videos to single frames by extracting the last frame, matching the single-image input format used in VLA training.
• ShareRobot (Ji et al., 2025): 100K samples. Robot manipulation data with QA annotations about high-level actions and task progress. Again, video frames are converted to single-image inputs.
• RobotVQA (Sermanet et al., 2024): 100K samples. Long-horizon reasoning questions about robot manipulation sequences from the RoboVQA dataset, converted to single-image inputs.

Custom multi-view instruction dataset (500K samples): The paper mentions this dataset without providing extensive detail about its construction in the main text. Appendix A.3 indicates it is a "multi-view instruction dataset" generated by the authors, presumably using robot data with multiple camera angles to produce instruction-following examples. The inclusion of multi-view data suggests the authors want the model to develop viewpoint-invariant spatial understanding.

Custom ROBOALIGN VQA dataset (150K samples): This is a novel contribution of the paper. The authors note that existing VQA datasets "focus on high-level QA tasks, lacking the fine-grained, spatial-temporal information needed for low-level control." To address this gap, they construct a data generation pipeline:

1. Input data: Robot images from the BridgeV2 and Droid datasets (Walke et al., 2023; Khazatsky et al., 2024), along with associated metadata — bounding boxes of objects, end-effector states, high-level and low-level action labels.
2. Generator model: Gemini-2.5-Pro (Hassabis et al., 2025), a powerful commercial LLM with vision capabilities.
3. Output: Automatically generated VQA pairs, captioning examples, and grounding QA pairs that combine the visual observations with the metadata to create questions about object locations, action progress, spatial relationships, and task requirements.

The key design choice here is using metadata-augmented generation rather than pure image-based generation. By providing the LLM with ground-truth bounding boxes, end-effector states, and action labels alongside the image, the generated QA pairs can be more precise and factually grounded than if the model generated questions based only on the image. This matters because errors in the training data (hallucinated object positions, incorrect action descriptions) would teach the MLLM wrong spatial relationships.

Robot QA from BridgeV2 and Droid (300K samples): In addition to the VQA format, the paper includes 300K samples that directly query the model about robot state and movement predictions:

• Predicting movements such as "move right" or "move left" from the current observation
• Predicting the current 7-DoF end-effector state as a text answer
• Predicting a future sequence of 10 end-effector states

These queries are still formatted as natural language QA — the model outputs text descriptions, not FAST tokens — but they require the model to develop internal representations that can predict precise spatial and kinematic quantities. This bridges the gap between pure visual understanding and the action prediction that comes later.

Data Component 3: Zero-Shot Reasoning Distillation (76K samples)

This component is motivated by a critical observation the paper makes: without explicit training on structured reasoning, the MLLM produces only minimal, unhelpful reasoning when asked to think before generating actions. Table 1 illustrates this dramatically:

Trained without reasoning data: thinkingGo to the cup. response Trained with reasoning data: thinkingTo pick up the cup from the table, the robot needs to move its gripper towards the cup, position it correctly, close the gripper to secure the cup, and then lift it up. Given the current state of the gripper being open and positioned above the cup, the immediate next action would be to move down towards the cup to prepare for grasping. response

The short reasoning ("Go to the cup.") provides essentially no useful information for guiding action generation — it doesn't decompose the task, consider the current state, or identify subgoals. The detailed reasoning breaks down the manipulation into phases, relates them to the current gripper state, and identifies the immediate next sub-action. For the RL stage to be effective, the model needs to produce diverse, detailed reasoning trajectories that can be evaluated and differentially reinforced. Minimal reasoning means there's nothing for RL to optimize.

To inject this reasoning capability, the paper constructs a specialized distillation pipeline:

1. Train a reasoning teacher model using GRPO on spatial and robot-related embodied multiple-choice QA tasks, following the procedure from Robot-R1 (Kim et al., 2025a). This teacher learns to generate step-by-step chain-of-thought reasoning before selecting the correct multiple-choice answer.

2. Generate reasoning data by feeding 50K multiple-choice QA prompts (converted from the ROBOALIGN VQA dataset) plus another 50K prompts derived from SAM2 (Ravi et al., 2024) — which query spatial relations among key objects detected by the segmentation model — to the teacher model. For each prompt, the teacher generates multiple reasoning trajectories.

3. Filter the generated trajectories using a combination of rule-based rewards (checking that the reasoning follows the required format) and correctness checks (verifying that the reasoning leads to the correct multiple-choice answer). Only trajectories that are both well-formatted and correct are kept.

4. Final dataset: After augmentation and filtering, 76K high-quality reasoning trajectories remain. Of the original 100K prompts, 30K were used specifically to train the teacher model; the remaining were used for data generation.

A nuance worth highlighting: the teacher model is trained specifically for embodied reasoning QA, not for action generation. The reasoning patterns it learns — decomposing tasks, identifying spatial relationships, reasoning about object states — are expected to transfer to the action generation context because they involve the same underlying cognitive skills (spatial understanding, task decomposition, state estimation).

Data Component 4: FAST Token Generation Dataset (400K samples)

This is the component that directly teaches the MLLM to output action tokens. The training data is constructed from the BridgeV2 dataset (Walke et al., 2023) in a QA format:

• Input: A robot image paired with a fixed task instruction (e.g., "pick up the cup from the table")
• Output: The corresponding sequence of FAST tokens, enclosed in the <action_start> and <action_end> delimiter tokens

The prompt template (shown in Appendix Figure 5) is: "Your current task is instruction. Output the robot's actions to perform this task through FAST tokens."

Why BridgeV2: The BridgeV2 dataset contains diverse manipulation demonstrations across many objects and tasks, collected with a WidowX robot arm in tabletop environments. It is one of the largest open-source robot manipulation datasets and has become a standard for training generalist manipulation policies. Crucially, it is collected in a different domain than the evaluation benchmarks (LIBERO uses a Franka Panda arm, CALVIN uses a different Franka setup, the real-world experiments use a different robot entirely). This means the FAST token training data is domain-disjoint from the evaluation data, which tests whether the learned representations generalize rather than merely memorizing action patterns for specific environments.

The 400K samples represent a subset of the full BridgeV2 dataset. The paper doesn't specify the selection criteria, but it's presumably chosen to balance coverage of diverse skills with training computational cost.

SFT Training Configuration

The SFT stage trains the Qwen2.5VL-7B-Instruct model on the full 2.28M-sample mixture. Key configuration details from the paper (Section 5.1):

• Vision encoder: Frozen. Only the language model backbone (the transformer layers that process the combined visual and text tokens) is trained. This preserves the pretrained visual features while allowing the model to learn new mappings from those features to action-relevant representations.

• Optimizer and schedule: The paper states it follows the official Qwen2.5VL training repository, using a cosine learning rate scheduler with learning rate $2 \times 10^{-5}$, warmup ratio of 0.03, and training for 1 epoch. The relatively high learning rate (compared to the RL stage's $1 \times 10^{-6}$) reflects the need to learn entirely new capabilities (action token generation, structured reasoning) from the initialized model.

• Training duration: Appendix A.1 reports that SFT takes approximately 30 hours on 8×H200 GPUs.

The cross-dataset balancing act: A crucial implicit design choice is how the different data components are weighted during training. The paper doesn't specify sampling ratios (e.g., whether general VQA data is upsampled or downsampled relative to action data), but the total sizes (100K general, 1.2M embodied reasoning, 76K reasoning, 400K action) provide a rough prior over the training distribution. The embodied reasoning component dominates by volume, followed by action data, with general capabilities and structured reasoning as smaller components. This distribution reflects the paper's goal: the primary purpose of SFT is to enhance embodied understanding, while preserving enough general capability and injecting enough reasoning structure to make the subsequent RL stage effective.

---

Stage 2: Aligning Reasoning with Actions via GRPO

The second stage is where ROBOALIGN's core contribution lies. Building on the SFT model's ability to generate FAST tokens with reasoning, RL is used to directly optimize the reasoning process for action prediction accuracy. The paper states this plainly:

"We optimize the model's embodied reasoning process to directly improve the accuracy of FAST action token generation."

This is a fundamentally different objective from prior RL-for-reasoning work in embodied AI. Rather than asking "does the reasoning lead to the correct high-level answer?", ROBOALIGN asks "does the reasoning lead to the correct low-level action sequence?"

Training Data Construction for RL

The RL stage uses a 12.8K subset of the BridgeV2 FAST token prediction data — less than 1% of the 2.28M SFT samples. This dramatic reduction in data volume is possible because RL is not trying to teach new capabilities from scratch; it's refining an existing capability (the SFT model can already generate plausible action sequences) by exploring variations in the reasoning process.

The key modification from the SFT FAST token data is the prompt augmentation. Each sample's instruction is augmented with a requirement for explicit reasoning within structured tags:

"The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within thinking response and <answer> </answer> tags, respectively."

This augmented prompt template is shown in Appendix Figure 5 and enforces a structured output format that makes the reasoning trace explicit and separable from the action tokens. This is essential for RL because it makes the reasoning process observable and evaluable — the reward function can check whether the model followed the format (format reward) and whether the final action tokens are correct (accuracy reward), even though the reasoning content itself is not directly supervised.

The Reward Function

The reward is defined as the arithmetic mean of two components:

$r = \frac{r_f + r_a}{2}$

where $r_f \in \{0, 1\}$ is the format reward and $r_a \in [0, 1]$ is the accuracy reward.

Format reward $r_f$: A binary indicator of whether the model's output correctly follows the required structure — the output must contain properly formatted thinking... and <answer>...</answer> blocks. This encourages the model to maintain the reasoning habit even as it explores diverse reasoning strategies. Without this reward, the model might learn to skip the reasoning entirely and output action tokens directly, which would be faster but would prevent the reasoning process from being shaped by the accuracy signal.

Accuracy reward $r_a$: Measures how many leading FAST tokens in the generated sequence match the ground truth. Formally:

$r_a = \frac{1}{m} \max\left\{ i \in \{1, \ldots, m\} : T^{\text{gen}}_{1:i} = T^{\text{target}}_{1:i} \right\}$

where $T^{\text{gen}}_{1:n}$ is the generated FAST token sequence of length $n$, $T^{\text{target}}_{1:m}$ is the target sequence of length $m$, and $T^{\text{gen}}_{1:i} = T^{\text{target}}_{1:i}$ means the first $i$ tokens match exactly.

What this computes: The maximum prefix length (in tokens) where the generated and target sequences are identical, divided by the total target length. For example, if the target has 5 FAST tokens [a486, a265, a268, a116, a269] and the model generates [a486, a265, a268, a116, a269], the longest matching prefix is 5, so $r_a = 5/5 = 1.0$. If the model generates [a486, a265, a268, a116, a619], the match breaks at position 5, so the longest prefix is 4, giving $r_a = 4/5 = 0.8$. This is illustrated directly in Figure 2's reward computation panel.

Why prefix accuracy rather than exact match: An exact-match reward (1 if all tokens match, 0 otherwise) would be extremely sparse — for a sequence of 5 tokens with vocabulary size $2K$, the chance of random exact match is $(1/2K)^5 \approx 10^{-15}$. The model would almost never receive a non-zero reward, making learning impossible. Prefix accuracy provides a dense, graded signal: even if the model gets the later tokens wrong, it still receives partial credit for getting the early tokens right. This smooths the optimization landscape and lets the model gradually extend the length of correct action sequences.

Why prefix rather than arbitrary subsequence: The prefix constraint reflects the sequential nature of the action trajectory. The first FAST token encodes the earliest part of the motion; the second token encodes the next part; and so on. Getting later tokens right but early tokens wrong would correspond to a trajectory that starts in the wrong place but somehow corrects — this is physically implausible for smooth robot motion. The prefix metric appropriately penalizes early errors more heavily than late errors, which matches the causal structure of the task.

Why normalize by target length: Without normalization, longer target sequences would naturally have higher expected prefix lengths, biasing the reward toward actions with more tokens. Normalizing by $m$ makes the reward scale-invariant and ensures the model isn't incentivized to prefer tasks with longer or shorter action sequences.

GRPO Policy Optimization

The RL algorithm is Group Relative Policy Optimization (GRPO) (Shao et al., 2024), the same algorithm used in DeepSeek-R1. GRPO is a variant of policy gradient methods designed for language model fine-tuning with the following structure:

For a given prompt $q \sim P(Q)$ (sampled from the training distribution), the current policy $\pi_{\theta_{\text{old}}}$ generates $G$ responses $[o_1, \ldots, o_G] \sim \pi_{\theta_{\text{old}}}(q)$. Each response is the full model output — the reasoning trace plus the FAST token sequence. Each response receives a reward $r_i = R(q, o_i)$ from the reward function defined above.

The key insight of GRPO (relative to standard PPO) is in how it computes advantages. Rather than learning a separate value function to estimate baselines, GRPO normalizes the rewards within each group of $G$ responses to compute a relative advantage:

$A_i = \frac{r_i - \text{mean}(r)}{\text{std}(r)}$

where $\text{mean}(r)$ and $\text{std}(r)$ are the mean and standard deviation of rewards across the $G$ responses for prompt $q$. A response that scores above the group average gets a positive advantage; one that scores below average gets a negative advantage.

What this does: This normalization eliminates the need for a critic network (value function), which is expensive to train and can itself be a source of instability. Instead, the baseline is defined implicitly by the group's average performance. If all responses to a prompt are poor (e.g., the task is hard), the advantages are still centered around zero — the model isn't penalized for failing on hard tasks, only for doing worse than other attempts on the same task. This makes the training more robust to variation in task difficulty across the dataset.

The policy is then optimized using a clipped surrogate objective with a KL penalty against a reference policy:

$J_{\text{GRPO}}(\theta) = \mathbb{E}\left[ \frac{1}{G} \sum_{i=1}^{G} \min\left( \rho_i A_i, \text{clip}(\rho_i, 1-\varepsilon, 1+\varepsilon) A_i \right) - \beta D_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}}) \right]$

where $\rho_i = \frac{\pi_\theta(o_i|q)}{\pi_{\theta_{\text{old}}}(o_i|q)}$ is the importance sampling ratio, $\varepsilon$ is the clipping hyperparameter, and $\beta$ controls the strength of the KL penalty.

Breaking this down piece by piece:

The importance sampling ratio $\rho_i$: This is the ratio of the probability the current policy assigns to response $o_i$ divided by the probability the old policy (the one that generated the responses) assigned to it. If the current policy makes a response more likely than the old policy did, $\rho_i > 1$; if it makes it less likely, $\rho_i < 1$. This ratio corrects for the fact that we're evaluating actions generated by an old policy using the current policy's probability distribution.

The clipped objective $\min(\rho_i A_i, \text{clip}(\rho_i, 1-\varepsilon, 1+\varepsilon)A_i)$: This is the standard PPO clipping mechanism. If a response has a positive advantage (good response), the objective encourages increasing its probability, but only up to a factor of $1+\varepsilon$ above the old probability — beyond that, further increases aren't rewarded. If a response has a negative advantage (bad response), the objective encourages decreasing its probability, but only down to a factor of $1-\varepsilon$ below the old probability. This clipping prevents destructively large policy updates that could cause the model to collapse.

The KL penalty $-\beta D_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}})$: This term penalizes the current policy for diverging too far from a reference policy (typically the SFT model or the policy from the start of training). The KL divergence $D_{\text{KL}}$ measures how different two probability distributions are — a high value means the current policy assigns very different probabilities to sequences than the reference policy. By penalizing this divergence, the objective prevents the model from drifting into regions of the policy space where it produces nonsensical outputs or forgets its pretrained capabilities. The hyperparameter $\beta$ controls how strongly to enforce this closeness constraint.

Why this objective form: The combination of clipping and KL penalty provides stability. Without clipping, a single high-advantage response could cause a massive policy update that overfits to that one example and degrades performance on everything else. Without the KL penalty, the policy could drift arbitrarily far from the well-behaved SFT initialization, potentially losing the general language and reasoning capabilities that make the MLLM useful. Together, these mechanisms allow the model to explore diverse reasoning strategies and reinforce effective ones while staying within a trustworthy region of the policy space.

GRPO Training Configuration

The paper uses the EasyR1 repository for implementation, with the following hyperparameters (Section 5.1):

• Rollout batch size: 512 prompts per training step
• Samples per prompt: $G = 5$ — for each prompt, 5 diverse reasoning-and-action responses are generated
• Update batch size: 128 — the number of (prompt, response, reward) tuples used in each gradient update
• Learning rate: $1 \times 10^{-6}$, constant (no schedule)
• Training duration: 1 epoch over the 12.8K samples
• Compute: Approximately 1 hour on 8×H200 GPUs (Appendix A.1)

The 5-sample design choice: Sampling 5 responses per prompt creates meaningful variance in the group for advantage normalization. With only 2 samples, the advantage would be determined entirely by which of the two is better, providing minimal signal about how much better. With 5, the model can distinguish between slightly better-than-average and dramatically better-than-average responses, providing a more nuanced training signal. The computational cost scales linearly with $G$ (since each sample requires a full forward pass), so 5 represents a balance between signal quality and training efficiency.

The learning rate and training duration: The $1 \times 10^{-6}$ learning rate is 20× smaller than the SFT learning rate ($2 \times 10^{-5}$), reflecting the different nature of the two stages. SFT needs to teach entirely new capabilities (action token vocabulary, reasoning structure) from a model that has never seen these output formats. RL only needs to refine the quality of outputs that the model already knows how to produce. A small learning rate prevents the RL updates from destroying the SFT-acquired capabilities while still allowing gradual improvement. The 1-epoch training duration (approximately 25 gradient steps given 12.8K samples and update batch size of 128) further limits the total change, keeping the model close to its SFT initialization.

Training dynamics: Appendix Figures 6a and 6b show the evolution of training metrics during RL. The accuracy reward increases steadily from approximately 0.10 to 0.18 over the 25 training steps, while the mean response length remains stable between 76 and 86 tokens. The stable response length is significant: it means the model is not simply learning to output longer reasoning (which might correlate with higher accuracy by chance) but is genuinely improving the content of the reasoning while maintaining a consistent verbosity. If response length were increasing dramatically, it could indicate that the model is learning to "pad" reasoning with irrelevant text that happens to correlate with correct actions in the training distribution but wouldn't generalize.

What Changes During RL: A Qualitative Example

Appendix Figure 7 provides a qualitative before/after comparison that illustrates what RL does to the reasoning:

SFT model reasoning: "To perform the task of moving the orange cloth to the top of the right burner, the robot needs to first approach the cloth, then grasp it, and finally move it to the right burner. The actions should be sequential and purposeful, focusing on the cloth and the burner in question." Action: wrong FAST tokens (reward 0.0).

RL-trained model reasoning: "To perform the task of moving the orange cloth to the top of the right burner, the robot needs to first approach and align its gripper with the cloth. Once aligned, it will need to close the gripper to pick up the cloth, lift it, and then move it to the right burner before releasing it. The robot's current position suggests it is already aligned with the cloth, ready to pick it up." Action: correct FAST tokens (reward 1.0).

The RL-trained reasoning differs in several concrete ways:

• More specific sub-actions: "approach and align" instead of just "approach" — recognizing that alignment is a distinct phase
• State awareness: "The robot's current position suggests it is already aligned with the cloth" — this shows the model is conditioning its reasoning on the visual observation, not just the task instruction
• Complete action sequence: It describes the full grasp-lift-move-release cycle, not just the initial approach
• Grepper-specific detail: It explicitly mentions closing and releasing the gripper, which are critical for task success

These improvements emerge from the RL process because reasoning trajectories that include such awareness and specificity are more likely to lead to correct action token predictions. The model doesn't receive direct supervision on the reasoning content — it discovers through exploration that certain reasoning patterns correlate with action accuracy and amplifies those patterns.

Why RL Over SFT for Alignment: The ECoT Comparison

The paper provides a direct ablation (Table 7) comparing RL-based alignment against SFT-based alignment using ECoT (Zawalski et al., 2024). Both approaches start from the same ROBOALIGN SFT model and train on the same 12.8K samples. The ECoT approach jointly trains reasoning and action generation through standard next-token prediction SFT — the model learns to exactly reproduce reasoning traces that led to correct actions.

The results are striking: ECoT degrades performance from 78.7% to 67.7% average, while RL improves it to 86.8%. The degradation is worst on Long tasks (65.6% → 45.6%, a 30% relative drop).

The authors diagnose two causes:

"We attribute this to the limited generalization of SFT, where knowledge aligned on BridgeV2 transfers poorly to LIBERO, as well as to forgetting effects introduced during SFT."

The generalization problem: SFT on reasoning traces from BridgeV2 teaches the model to replicate specific reasoning patterns that worked for specific BridgeV2 scenarios. When deployed on LIBERO (different robot, different objects, different camera angle, different task structure), these memorized patterns may not apply, and the model lacks the flexibility to adapt because SFT encourages exact reproduction. RL, by contrast, rewards outcomes (action accuracy) rather than process (matching a reference reasoning trace). This means the model can develop reasoning strategies that generalize because they're optimized for the objective (accurate actions) rather than the surface form (matching BridgeV2 reasoning text).

The forgetting problem: SFT on 12.8K reasoning-action pairs represents a focused optimization on a narrow data distribution. Even though the model is initialized from the diverse SFT checkpoint, additional SFT can erode the broad capabilities if the new data distribution is narrow. RL's KL penalty against the reference policy directly mitigates this — the policy is constrained to stay near the well-rounded SFT initialization, preventing the narrow specialization that causes forgetting.

---

VLA Conversion Pipeline

After MLLM training is complete (either after SFT only or after both SFT and RL), the model must be converted into a functional VLA — a system that takes in images and instructions and outputs continuous robot actions. This conversion is done through a standard architecture that the paper adopts from GR00T-N1.5 (GEAR, 2025).

Architecture: A diffusion-based action head is attached on top of the frozen MLLM backbone. Specifically, the action head takes as input the hidden states from the 18th layer of the Qwen2.5VL-7B-Instruct transformer. This is a design choice: rather than using the final layer outputs (which are optimized for next-token prediction in language space), the 18th layer (out of 28 total layers for a 7B model) provides intermediate representations that may encode more general visual and semantic features before they're specialized for language output. The paper doesn't ablate this choice, but it's consistent with prior work showing that intermediate layer features often transfer better to downstream tasks than final-layer features.

The action head is a diffusion policy — a generative model that produces continuous action trajectories by iteratively denoising random noise into structured outputs, conditioned on the MLLM's latent representations. Diffusion policies have become popular in robotics because they can model multimodal action distributions (multiple valid ways to accomplish a task) and produce smooth, physically plausible trajectories.

Training: Only the action head is trained; the MLLM backbone is completely frozen. The training data comes from the target benchmark environments:

• LIBERO: The provided dataset covering all 40 tasks (10 per category × 4 categories), training for 60K steps
• CALVIN: 100K steps of data from environments A, B, and C, training for 100K steps
• Real-world: 60 demonstrations per task (4 tasks), training for 30K steps per task

The batch size is 32 across all environments (Appendix A.2). Hyperparameters follow the official GR00T-N1.5 implementation unless otherwise specified.

Why freeze the MLLM: Freezing the backbone during VLA training is essential for clean evaluation. If the MLLM were fine-tuned alongside the action head, it would be impossible to attribute performance differences to the MLLM training method versus the VLA fine-tuning. By keeping the backbone frozen, the VLA conversion becomes a pure readout of the quality of the MLLM's internal representations — a better MLLM will produce better features, which will enable the action head to learn more accurate policies with the same training data and architecture.

What the action head learns: The diffusion policy learns to map from the MLLM's visual-linguistic representations to continuous action trajectories. Because the MLLM representations encode information about objects, spatial relationships, task progress, and manipulation affordances (shaped by the SFT and RL training), the action head doesn't need to learn these from scratch — it can focus on the relatively simpler problem of converting rich semantic features into precise motor commands.

Compute requirements: Appendix A.1 reports that VLA training takes approximately 1 hour per 10K steps on 2×A100 GPUs. So LIBERO (60K steps) takes about 6 hours, CALVIN (100K steps) about 10 hours, and real-world (30K steps per task, 4 tasks) about 12 hours total.

The evaluation protocol: After VLA training, the model is evaluated on held-out tasks:

• LIBERO: 50 trials per task, 500 trials per category, measuring success rate (%)
• CALVIN: 1,000 chains of 5 consecutive instructions in environment D (zero-shot — never seen during training), measuring success rate at each chain length and average number of consecutive successful tasks
• Real-world: 24 trials per object, 96 trials per task, measuring success rate (%)

Since the VLA conversion is identical for all MLLM training methods (same architecture, same training data, same hyperparameters, same evaluation protocol), any differences in VLA performance directly reflect differences in the quality of the MLLM's internal representations as shaped by the training method. This controlled comparison is what makes the paper's central claim testable: ROBOALIGN's representations are better for action generation than those produced by alternative MLLM training approaches.

4. Key Insights and Innovations

Innovation 1: Reframing the Embodied Reasoning Problem from Improving QA Accuracy to Closing the Modality Gap

The paper's most fundamental conceptual contribution is the diagnosis of a specific failure mode in the VLA training pipeline and the reframing of what it means to make MLLMs useful for robot control. Prior work operated under the implicit assumption that improved embodied reasoning — measured by accuracy on spatial QA, high-level action prediction, affordance grounding, and trajectory forecasting — would naturally cascade into improved VLA performance because the same model processes the same visual inputs for both tasks. This assumption made intuitive sense: if a model can correctly answer "where is the red cup relative to the gripper?" and "what action should the robot take next?", it must have developed internal representations that encode the right spatial and semantic information for action generation.

The paper demonstrates that this intuition is wrong in practice, and more importantly, it provides a specific mechanistic explanation for why it fails. The language-action modality gap is not merely a statement that text and motor commands are different output spaces — that much is obvious. The deeper insight is that optimizing representations for linguistic output creates a misalignment in what information the model's hidden states encode. When an MLLM is trained to answer embodied questions in natural language, its internal representations are shaped to be good at symbolic, discrete, and coarse-grained reasoning — identifying object categories, naming spatial relationships, selecting among high-level action options. But VLAs need representations that support continuous, fine-grained, and geometrically precise control — exact gripper positions, smooth trajectory shaping, sub-centimeter spatial accuracy. These two objectives pull the representations in genuinely different directions, and the paper's experiments show that optimizing for the former can actively degrade the latter.

The evidence for this reframing is stark. Figure 1 shows that RoboBrain 2.0 — the strongest embodied reasoning model, outperforming GPT-4o on major benchmarks — produces the worst VLA performance among all tested backbones. This is not a small degradation at the margins; it's a qualitative reversal of the expected relationship. The paper's own ablation (Table 2, "Language-Only SFT") shows that training on 1.88M embodied VQA samples improves LIBERO Goal from 42.4% to 67.8% — a real gain — but leaves Long essentially flat (63.2% → 65.0%) and actually degrades Spatial (95.2% → 91.0%). On CALVIN (Table 3), the same language training improves length-1 success (77.8% → 87.4%) but reduces length-5 (18.1% → 15.3%). The pattern is consistent: language-only training helps on simple, short-horizon tasks where high-level understanding suffices, but fails or backfires on tasks requiring sustained precise control.

This reframing has direct implications for research prioritization. If the goal is to build better VLAs, investing in better embodied VQA benchmarks and training recipes is a misaligned proxy objective. Resources should instead target methods that directly bridge the gap between linguistic reasoning and continuous control. The paper positions ROBOALIGN as the first training framework designed explicitly around this diagnosis rather than treating embodied reasoning improvement as an end in itself.

Innovation 2: Action-Token Accuracy as a Reward Signal for Reasoning Improvement

The paper's second conceptual contribution is the introduction of a new RL reward definition that fundamentally changes what it means to "improve reasoning" in the context of embodied AI. In the GRPO/DeepSeek-R1 paradigm that the paper builds on, reasoning is optimized by rewarding the model for producing correct final answers — in math, the right number; in coding, passing unit tests; in embodied QA, selecting the correct multiple-choice option. The reasoning trace itself is not directly supervised; it's shaped indirectly through the outcome reward.

ROBOALIGN applies this same principle — reinforce reasoning trajectories that lead to good outcomes — but defines "good outcome" in a way that is both more demanding and more directly aligned with the downstream task than prior work. Instead of rewarding linguistic correctness ("did you pick option C?"), it rewards action-token prefix accuracy ("did your first 3 FAST tokens match the ground truth?"). This seems like a small change — swap one reward function for another — but it has profound implications for what the model learns to value in its reasoning process.

Why this is non-obvious: Prior RL-for-reasoning work in robotics (Robot-R1, ThinkAct, Cosmos-Reason1) all defined rewards on language outputs — answering questions correctly, predicting trajectories in pixel space, or selecting high-level actions. These approaches implicitly assume that the reasoning patterns that produce correct linguistic answers are the same patterns that produce good control. The paper's results suggest this assumption is false. By switching to action-token rewards, ROBOALIGN forces the model to discover reasoning patterns that causally contribute to precise action generation, not just plausible-sounding descriptions.

The prefix-accuracy formulation (reward = fraction of leading FAST tokens that match ground truth) is itself a design choice with conceptual weight. An exact-match reward would be too sparse to learn from (the probability of randomly generating a correct 5-token sequence is astronomically low). A token-level cross-entropy loss (standard SFT) would teach the model to reproduce specific action token sequences but wouldn't encourage exploration of diverse reasoning strategies — it would reward matching the reference, not discovering better approaches. The prefix-accuracy reward provides a dense, graded signal that credits partial correctness while preserving the exploration benefits of RL: the model is rewarded for getting the early part of the trajectory right, incentivizing it to figure out which reasoning patterns help with the crucial initial phases of a manipulation (approaching the object, aligning the gripper) rather than only the final outcome.

The qualitative evidence in Appendix Figure 7 illustrates what this reward signal produces. The SFT model's reasoning describes a sequence of high-level actions ("approach, grasp, move") without specific detail about the robot's current state or what precise adjustment is needed. The RL-trained model's reasoning adds spatial specificity ("align its gripper with the cloth"), state awareness ("the robot's current position suggests it is already aligned"), and the complete action cycle including gripper control. These are not elaborations that happen to sound better — they are reasoning patterns that causally enable more accurate FAST token prediction because they require the model to extract precise spatial and state information from the visual input. The reward function discovers these patterns through trial and error across thousands of rollouts, without any human annotation of what constitutes "good" reasoning.

Innovation 3: The Two-Stage Cold-Start + Alignment Paradigm for Vision-Language-Action Models

The paper's third contribution is the two-stage training paradigm itself — not as a novel architectural innovation (two-stage training is common), but as a specific solution to a specific set of constraints that prior approaches handled poorly. The constraints are: (1) off-the-shelf MLLMs cannot generate action tokens, so zero-shot RL is impossible; (2) SFT on action data alone causes catastrophic forgetting of general capabilities essential for generalization; (3) SFT on diverse data including actions enables action generation but doesn't optimize reasoning for action accuracy; (4) SFT on reasoning+actions jointly (ECoT-style) causes overfitting to the training domain's action patterns.

The two-stage design — SFT to establish capabilities, RL to refine reasoning — is the minimal architecture that satisfies all constraints simultaneously. Stage 1 solves the cold-start problem while injecting diverse embodied knowledge and preserving general capabilities through a carefully balanced data mixture. Stage 2 addresses the alignment problem that neither language-only nor action-only SFT could solve, using orders of magnitude less data (12.8K vs 2.28M samples) because it's refining rather than building from scratch.

What makes this distinctive relative to prior two-stage approaches: The standard two-stage recipe in VLA training is pretrain-then-finetune: take a pretrained MLLM, fine-tune it on robot data (with or without action heads), and deploy. This paper's key addition is the intermediate alignment stage between general pretraining and task-specific VLA training, where the model learns to think in ways that help with action generation before it's frozen and connected to an action head. This intermediate stage is what prevents the tradeoff that the paper documents: language-only MLLM training helps simple tasks but not complex ones (Tables 2-3); action-only MLLM training helps in-domain but destroys generalization (Table 3, Action-Only SFT drops CALVIN length-1 from 77.8% to 66.1%). ROBOALIGN's SFT+RL combination improves both simple and complex tasks simultaneously while maintaining generalization — the LIBERO Long category jumps from 63.2% (base model) to 70.0% (ROBOALIGN), and CALVIN length-5 rises from 18.1% to 22.2%, while shorter-horizon performance also improves.

The significance of this paradigm extends beyond the specific datasets and models used in the paper. It suggests a general principle for building VLAs: the MLLM backbone should be explicitly trained to reason about the action space before being frozen and connected to an action head, and this reasoning training should use outcome-based RL rather than imitation of reference reasoning traces. If this principle generalizes, it would reshape the standard VLA training pipeline across model architectures and robot embodiments.

The paper's comparison with ECoT (Table 7) is particularly revealing about why the RL stage matters. ECoT applies SFT on reasoning+action data — essentially a stronger version of Stage 1 that includes reasoning traces. Yet it degrades performance across all LIBERO categories. The authors attribute this to limited generalization and forgetting, but the deeper point is that SFT cannot discover new reasoning strategies — it can only reproduce what's in the training data. If the training data's reasoning patterns are specific to BridgeV2 and don't transfer to LIBERO, SFT locks the model into strategies that work in the wrong domain. RL, by exploring the space of possible reasoning traces and reinforcing those that produce accurate actions on BridgeV2 data, discovers strategies that happen to generalize better — not because BridgeV2 is similar to LIBERO (it isn't), but because the strategies that work well on diverse BridgeV2 tasks capture something fundamental about manipulation that transfers across embodiments.

Innovation 4: Positive Transfer from Action Alignment to General MLLM Capabilities

The paper's fourth contribution is an unexpected positive finding: aligning an MLLM with low-level actions through RL does not degrade its general visual understanding and embodied reasoning capabilities — it actually improves them. This is not an obvious outcome. The standard concern with domain-specific fine-tuning is catastrophic forgetting: the model specializes to the narrow training distribution and loses the broad capabilities that made the pretrained model valuable. The paper's Action-Only SFT baseline demonstrates exactly this failure mode (Table 3: CALVIN performance collapses from 2.16 to 1.26 average sequence length).

The natural expectation would be that RL alignment on action tokens — an even more specialized objective than action SFT — would cause even worse forgetting. Instead, Table 9 shows the opposite: ROBOALIGN achieves state-of-the-art performance on embodied reasoning benchmarks (Robot-R1 Bench: 1.38 vs. 1.17 for RoboBrain2.0, 1.19 for Cosmos-Reason1) while also improving on general image understanding (MMStar: 62.80 vs. 60.30 for the base Qwen2.5VL-7B) and spatial reasoning (RoboSpatial: 50.86 vs. 36.29 for the base model; Where2Place: 54.49 vs. 11.35; BLINK depth: 87.90 vs. 55.64).

Why this is significant beyond raw performance: This finding challenges the assumption that alignment to a specific downstream task necessarily narrows a model's capabilities. The paper's explanation, though not fully spelled out, seems to be that the RL process reinforces reasoning patterns that generalize across tasks — patterns like "look carefully at spatial relationships," "consider the current state before planning," "decompose the task into phases" — because these are the patterns that causally improve action token prediction. If such patterns are broadly useful for visual understanding and reasoning (which seems plausible — they're essentially meta-cognitive strategies), then training that reinforces them for one objective can fortuitously improve other capabilities that benefit from the same strategies.

This stands in contrast to SFT-based alignment (ECoT), which the paper shows degrades general MLLM performance — confirmation that the benefit is specific to the RL approach, not to alignment in general. SFT forces the model to memorize specific reasoning traces; RL allows it to discover principles. The principles transfer; the memorized traces don't.

The practical implication is substantial: it means the ROBOALIGN framework can be applied without needing to carefully balance domain-specific and general training data to prevent forgetting. The RL stage's KL penalty against the reference policy, combined with the exploration-driven nature of the optimization, appears to naturally preserve and even enhance broad capabilities while sharpening action-specific representations. This removes a major practical barrier to deploying such methods — practitioners don't need to trade off general competence for task-specific performance.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary dataset for MLLM training (both SFT and RL stages) is the BridgeV2 dataset (Walke et al., 2023), which contains diverse tabletop manipulation demonstrations using a WidowX robot arm. For SFT, 400K samples from BridgeV2 are used for FAST token prediction training, while 12.8K samples are used for the RL stage. The evaluation of the downstream VLA is conducted on three separate benchmarks: LIBERO (Liu et al., 2023) using its provided dataset covering 40 tasks across four categories (Spatial, Object, Goal, Long), with 50 evaluation trials per task for a total of 500 trials per category; CALVIN (Mees et al., 2022) using 100K steps of data from environments A, B, and C for training and evaluating zero-shot on environment D over 1,000 chains of five consecutive instructions; and real-world experiments using a Franka Research 3 arm across four pick-and-place tasks with 60 demonstrations per task for training and 24 trials per object (96 total per task) for evaluation. The SFT stage additionally incorporates 1.88M samples from general VQA (LLaVA-OneVision, 100K), embodied reasoning datasets (RefSpatial, RoboPoint, EgoPlan-IT, ShareRobot, RobotVQA, custom multi-view data, and the ROBOALIGN VQA dataset totaling 1.2M), robot QA data (300K from BridgeV2 and Droid), and zero-shot reasoning distillation data (76K).

• Base model(s). All experiments use Qwen2.5VL-7B-Instruct (Bai et al., 2025), a 7-billion-parameter vision-language model, as the primary MLLM backbone. The paper states this model is selected because it is a representative open-source MLLM with strong general capabilities but without specialized embodied reasoning training, making it a suitable testbed for evaluating whether the ROBOALIGN framework can improve downstream VLA performance. For the compatibility experiment, Qwen3VL-8B-Instruct (Team, 2025) is used as an alternative backbone to test generalizability across architectures. The VLA conversion uses the hidden states from the 18th layer of the MLLM as input to the diffusion-based action head.

• Metrics. The primary metric across all robot evaluations is task success rate (%) — the fraction of evaluation trials in which the robot successfully completes the specified task. For LIBERO, success is measured per task (50 trials each) and averaged within each of the four categories (Spatial, Object, Goal, Long) and overall. For CALVIN, success is reported as the percentage of chains where the robot successfully completes 1, 2, 3, 4, and 5 consecutive instructions, along with the average number of consecutive successful tasks completed. For real-world experiments, success rate is reported per task and averaged. For MLLM benchmark evaluation (Table 9), the paper uses standard metrics from each benchmark: MMStar uses the benchmark's accuracy metric; Robot-R1 Bench uses a 0–3 scoring scale; RoboSpatial, Where2Place, and BLINK (relative depth component) use their respective official accuracy metrics.

• Baselines. The paper defines several baselines, all built on the same Qwen2.5VL-7B-Instruct backbone to isolate the effect of the training method:

  • Qwen2.5VL-7B-Ins (base model): The off-the-shelf pretrained MLLM with no additional fine-tuning. Converted directly to a VLA by attaching and training the action head.
  • Language-Only SFT: The base model fine-tuned for one epoch on 1.88M embodied reasoning and general VQA samples (the same SFT mixture used in ROBOALIGN Stage 1 but without the 400K FAST token data). Tests whether improving embodied reasoning through language supervision alone transfers to VLA performance.
  • Action-Only SFT: The base model fine-tuned for one epoch on 1.88M FAST token prediction samples from the full BridgeV2 dataset (the same size as Language-Only SFT but with only action data). Tests whether direct action prediction training transfers to VLA performance, and whether it causes catastrophic forgetting.
  • ROBOALIGN w/o RL (SFT only): The model after Stage 1 SFT on the full 2.28M mixture (including general VQA, embodied reasoning, reasoning distillation, and action token data). This isolates the contribution of the RL stage by serving as the initialization for RL and as a standalone baseline.
  • SFT-based Alignment (ECoT): An ablation that replaces Stage 2 RL with SFT on the same 12.8K samples, jointly training reasoning traces and FAST token predictions following the ECoT methodology (Zawalski et al., 2024). Tests whether the improvement comes from alignment in general or specifically from RL-based alignment.
  • Language-based RL: An ablation that replaces the action-accuracy reward in Stage 2 with a reward based on predicting high-level language actions (movement descriptions like "move right" in multiple-choice format). Tests whether any RL alignment helps, or whether the action-specific reward is crucial.
  • Visual-based RL: An ablation that replaces the Stage 2 reward with one based on predicting 2D end-effector trajectories in pixel space, following the reward formulation from ThinkAct (Huang et al., 2025a). Tests a different modality of alignment that is closer to actions than language but still not identical to low-level control.
  • Reference VLAs: For contextual comparison, the paper reports published results for Diffusion Policy (no citation in the paper, but likely Chi et al.), Octo (Team et al., 2024), OpenVLA (Kim et al., 2024), TraceVLA (Zheng et al., 2024), CoT-VLA (Zhao et al., 2025), and ThinkAct (Huang et al., 2025a). These are not trained by the authors; they are reproduced from prior publications to situate the absolute performance levels.

• Generation budget / compute accounting. The paper does not use a generation budget in the traditional sense (e.g., number of sampled responses at inference). Instead, the primary units of compute accounting are training data volume (2.28M samples for SFT, 12.8K samples for RL) and training steps (60K for LIBERO VLA, 100K for CALVIN VLA, 30K per real-world task). The emphasis on the 12.8K RL samples being "less than 1% of the data" used in the SFT stage highlights that the majority of the framework's benefit comes from a data-efficient refinement process. For the RL stage, the compute budget is further specified by the GRPO configuration: 5 samples are generated per prompt, with a rollout batch size of 512 prompts (producing 2,560 total responses per training step) and an update batch size of 128. Appendix A.1 reports the wall-clock compute: SFT takes approximately 30 hours on 8×H200 GPUs, RL takes approximately 1 hour on 8×H200 GPUs, and VLA training takes roughly 1 hour per 10K steps on 2×A100 GPUs.

• Cross-validation / statistical protocol. The paper does not report formal cross-validation or statistical significance testing for the main VLA results. The LIBERO evaluation uses 50 trials per task (500 per category), which provides meaningful statistics for success rates, but no confidence intervals or standard errors are reported. The CALVIN evaluation uses 1,000 chains, providing stable estimates. The paper uses two-fold cross-validation only implicitly through its design: the MLLM is trained on BridgeV2 data (a different domain from the evaluation benchmarks), making the evaluation a test of out-of-domain generalization rather than in-distribution performance. For the MLLM benchmark evaluations in Table 9, the paper follows the official protocols of each benchmark. The representation analysis (Table 8) uses 20 trajectories from a single LIBERO task, which is a small sample — the paper acknowledges this is a qualitative probe rather than a rigorous statistical comparison.

Main Quantitative Results

VLA Performance Across Benchmarks

The central experimental claim — that ROBOALIGN consistently and substantially improves VLA performance across diverse robotic environments — is evaluated through three main benchmark comparisons and one generalization test.

LIBERO results (Table 2, Figure 4). The full ROBOALIGN framework (SFT + RL) achieves an average success rate of 86.8% across all four LIBERO categories, compared to 73.9% for the untuned Qwen2.5VL-7B-Instruct backbone — a 17.5% relative improvement. This places it above all reference VLAs reported in the paper, including CoT-VLA (83.9%) and ThinkAct (84.4%), which incorporate chain-of-thought reasoning but use different training methodologies. The per-category breakdown reveals where the gains concentrate:

• Spatial: 93.8% (base: 95.2%, slight decrease, statistically likely within noise given the already-ceiling performance)
• Object: 96.0% (base: 95.0%, marginal improvement near ceiling)
• Goal: 87.2% (base: 42.4%, a 106% relative improvement, the largest single-category gain)
• Long: 70.0% (base: 63.2%, an 11% relative improvement, but — critically — the only training method besides CoT-VLA and ThinkAct that achieves 70% on this hardest category)

The baselines reveal a stark pattern: Language-Only SFT improves Goal (42.4% → 67.8%, a substantial 60% relative gain) but provides essentially zero improvement on Long (63.2% → 65.0%, +2.8%). Action-Only SFT dramatically improves Goal (42.4% → 82.8%, +95%) but actually degrades Long (63.2% → 57.6%, −8.9%) — the model becomes worse at long-horizon tasks than the untuned base model. ROBOALIGN w/o RL (SFT only) shows a similar pattern: Goal improves marginally (42.4% → 59.0%) but Long stays flat (63.2% → 65.6%). Only the full ROBOALIGN (SFT+RL) improves both categories simultaneously, with the Long gain being unique among all non-reference baselines.

This pattern is the paper's strongest empirical evidence for its central thesis: language training helps on tasks requiring high-level understanding (Goal — the model must interpret which instruction applies from among multiple possibilities in the same environment), while action training helps within its trained distribution but hurts generalization. Only the SFT+RL combination, which aligns reasoning with action accuracy while the KL penalty preserves general capabilities, delivers gains on both axes simultaneously.

CALVIN results (Table 3, Figure 4). ROBOALIGN achieves an average consecutive task completion of 2.57 (meaning the robot successfully completes 2.57 instructions in a row on average before failing), compared to 2.16 for the base model — an 18.9% relative improvement. The per-length breakdown shows consistent improvement across all chain lengths:

• Length-1: 87.6% (base: 77.8%, +12.6%)
• Length-2: 67.2% (base: 55.0%, +22.2%)
• Length-3: 47.1% (base: 38.6%, +22.0%)
• Length-4: 32.8% (base: 26.6%, +23.3%)
• Length-5: 22.2% (base: 18.1%, +22.7%)

The baselines again tell a revealing story. Language-Only SFT improves short-horizon performance (length-1: 77.8% → 87.4%) but degrades long-horizon (length-5: 18.1% → 15.3%, a 15.5% relative decline). Action-Only SFT catastrophically fails across the board: length-1 drops to 66.1%, and length-5 collapses to 3.2% — worse than random performance and dramatically worse than the untuned base model. This is the clearest evidence of catastrophic forgetting in the paper: training exclusively on action token prediction from BridgeV2 causes the model to lose the visual understanding and generalization capabilities essential for the zero-shot CALVIN evaluation. ROBOALIGN w/o RL also underperforms the base model (1.89 vs. 2.16 average length), confirming that SFT alone, even with diverse data, does not improve the representational quality for action generation in zero-shot settings.

The fact that ROBOALIGN improves all chain lengths — not just short ones like Language-Only SFT, and without the catastrophic degradation of Action-Only SFT — directly supports the claim that RL-based alignment bridges the modality gap while preserving general capabilities.

Real-world results (Table 4). ROBOALIGN achieves an average success rate of 66.7% across the four pick-and-place tasks, compared to 32.3% for the base model — a 106.6% relative improvement. The per-task breakdown:

• Box to bowl: 87.5% (base: 16.7%, +424%)
• Box to plate: 58.3% (base: 70.8%, a decrease — the model underperforms the base on this specific task)
• Basket to bowl: 70.8% (base: 20.8%, +240%)
• Plate to basket: 50.0% (base: 20.8%, +140%)

The ROBOALIGN w/o RL (SFT only) baseline achieves 55.2% average — a substantial improvement over the base model (71% relative gain), but the full RL stage adds another 20.8% relative improvement (55.2% → 66.7%). The real-world results are particularly notable because they test generalization to a completely different robot embodiment (Franka Research 3) and visual environment than the training data (BridgeV2, collected with a WidowX arm). The "Box to plate" task showing a degradation with ROBOALIGN alerts us to task-specific variance — no single method uniformly dominates every scenario.

Compatibility with different backbones (Table 5). Applying ROBOALIGN to Qwen3VL-8B-Instruct, an entirely different model architecture from Qwen2.5VL, shows consistent patterns:

• Average: 92.5% (base: 85.2%, +8.6% relative)
• Spatial: 95.6% (base: 94.2%)
• Object: 99.6% (base: 96.4%)
• Goal: 95.2% (base: 90.0%, +5.8%)
• Long: 78.6% (base: 60.0%, +31.0%)

The Long category sees the largest relative improvement, consistent with the Qwen2.5VL results. The ROBOALIGN w/o RL baseline also improves substantially (89.5%), suggesting the SFT data mixture transfers well across model families. The RL stage adds an additional 3.0 percentage points on average. This experiment provides evidence that the framework's benefits are not tied to a specific model architecture, though it should be noted that both backbones are from the same model family (Qwen), and testing on entirely different architectures (e.g., LLaMA-based VLMs) would provide stronger evidence of generalizability.

Alignment Strategy Comparisons

RL alignment target ablation (Table 6). When comparing different objectives for the Stage 2 RL alignment (all starting from the same ROBOALIGN w/o RL checkpoint and using the same GRPO configuration), the action-based reward (ROBOALIGN) achieves the highest overall performance at 86.8%:

• ROBOALIGN w/o RL (baseline): 78.7%
• w/ Language-based RL: 83.6% (+4.9% over SFT baseline)
• w/ Visual-based RL: 85.1% (+6.4% over SFT baseline)
• w/ Action-based RL (ROBOALIGN): 86.8% (+8.1% over SFT baseline)

The category-level results are more informative. Language-based RL dramatically improves Goal (59.0% → 90.0%, +52.5%) — the largest single-category gain of any method — but degrades Long (65.6% → 58.2%, −11.3%). Visual-based RL improves Goal substantially (59.0% → 87.8%, +48.8%) but provides only a marginal gain on Long (65.6% → 64.6%, −1.5%). Only Action-based RL simultaneously improves both Goal (59.0% → 87.2%, +47.8%) and Long (65.6% → 70.0%, +6.7%).

This ablation directly supports the paper's core thesis: language-based alignment reinforces reasoning patterns that help with high-level task interpretation (Goal tasks require understanding which of multiple possible instructions applies) but doesn't help — and can hurt — the fine-grained control needed for sustained long-horizon execution. The action-based reward is the only objective that produces reasoning patterns transferable to both types of tasks.

SFT vs. RL alignment (Table 7). When replacing the Stage 2 RL with SFT on the same 12.8K samples using the ECoT methodology (jointly training reasoning traces and action tokens through next-token prediction), performance degrades from the ROBOALIGN w/o RL baseline across all categories:

• ROBOALIGN w/o RL (SFT baseline): 78.7% average
• w/ SFT-based Alignment (ECoT): 67.7% average (−14.0%)
• w/ RL-based Alignment (ROBOALIGN): 86.8% average (+10.3%)

The degradation is most severe on Long (65.6% → 45.6%, −30.5%) and Goal (59.0% → 49.6%, −15.9%). This is a critical negative result: it demonstrates that simply training the model on reasoning-action pairs through SFT — even with exactly the same data that RL uses — is actively harmful. The authors attribute this to limited generalization (reasoning patterns memorized from BridgeV2 don't transfer to LIBERO) and forgetting (additional SFT on a narrow distribution erodes the broad capabilities established in Stage 1). The RL approach avoids both failure modes: the exploration-driven optimization discovers generalizable reasoning strategies rather than memorizing specific traces, and the KL penalty prevents the policy from drifting into a narrow specialized regime.

Representation Analysis (Table 8)

To investigate the mechanism behind ROBOALIGN's improvements, the paper probes whether the MLLM's internal representations become more discriminative for the underlying robot state. Using K-Nearest Neighbor (KNN) classification (k=5) on 20 trajectories from a single LIBERO long-horizon task, where timesteps are assigned to 32 classes based on Dynamic Time Warping over robot states, the accuracy of classifying the correct state class from MLLM hidden states alone (without access to the robot's proprioceptive state) is:

• Qwen3VL-8B-Instruct (base): 39.06%
• w/ ROBOALIGN w/o RL (SFT only): 43.23%
• w/ ROBOALIGN (SFT+RL): 69.79%

This nearly 1.8× improvement in state discriminability from the untuned model, and a 1.6× improvement from the SFT-only model, provides a mechanistic explanation for the VLA performance gains. The RL alignment stage is not just improving action token accuracy in isolation — it is restructuring the MLLM's internal representations to encode fine-grained robot state information that the downstream action head can leverage. This is consistent with the paper's hypothesis that the RL reward (prefix accuracy of FAST tokens) forces the model to attend to precise spatial and kinematic information in the visual input, producing representations that are useful for control even when the action head is trained separately.

The small scale of this experiment (20 trajectories from one task) limits its statistical power and generalizability, but its purpose is qualitative: to show that the representations change in a direction consistent with better action generation, not to establish a quantitative relationship between KNN accuracy and task success.

MLLM Benchmark Performance (Table 9)

Beyond VLA performance, the paper evaluates whether ROBOALIGN preserves or improves the MLLM's general and embodied reasoning capabilities:

• MMStar (general image understanding): 62.80 (base: 60.30, +4.1%). ROBOALIGN slightly improves over the base model and is competitive with GPT-4o (65.10).
• Robot-R1 Bench (embodied reasoning, 0–3 scale): 1.38 (base: 1.02, +35.3%). ROBOALIGN outperforms all comparison models including GPT-4o (1.55), RoboBrain2.0 (1.17), and Cosmos-Reason1 (1.19).
• RoboSpatial (spatial reasoning): 50.86 (base: 36.29, +40.2%). Substantially improved, though still below RoboBrain2.0 (54.23).
• Where2Place (spatial affordance): 54.49 (base: 11.35, +380%). Enormous improvement from a very low base, approaching RoboBrain2.0 (63.59).
• BLINK (relative depth): 87.90 (base: 55.64, +58.0%). ROBOALIGN achieves the highest score among all evaluated models, exceeding RoboBrain2.0 (83.95) and GPT-4o (77.90).

The ROBOALIGN w/o RL (SFT only) model already performs competitively with specialized embodied reasoning MLLMs on most benchmarks, confirming that the Stage 1 data mixture is effective at injecting embodied knowledge. The RL stage then provides additional gains across the board — the benchmark performance uniformly improves rather than exhibiting tradeoffs. This is noteworthy because it contradicts the common concern that task-specific RL alignment narrows a model's capabilities. Instead, the action-aligned representations appear to be beneficial for spatial understanding, depth perception, and embodied reasoning tasks more broadly.

The most striking comparison is with RoboBrain2.0: despite that model having the best embodied reasoning scores among all evaluated specialized MLLMs (and outperforming GPT-4o), it yielded the worst VLA performance (Figure 1, 57.35% on LIBERO). ROBOALIGN achieves competitive or superior embodied reasoning scores while also dramatically improving VLA performance. This directly supports the paper's diagnosis that the modality gap — not inadequate reasoning capability — is the bottleneck. RoboBrain2.0 reasons excellently in language but this reasoning doesn't transfer to action; ROBOALIGN reasons well and explicitly aligns that reasoning with action generation.

Ablation Studies and Robustness Checks

RL alignment target modality: Comparing language-based, visual-based, and action-based RL rewards (Table 6) reveals that only the action-based reward improves both Goal and Long tasks simultaneously. Language-based RL improves Goal dramatically (59.0% → 90.0%) but degrades Long (65.6% → 58.2%), suggesting it teaches high-level task discrimination skills that are useful for interpreting instructions but simultaneously encourages reasoning patterns that are detrimental to sustained precise control. Visual-based RL shows a similar but less extreme pattern, improving Goal (59.0% → 87.8%) while Long stays flat (65.6% → 64.6%). The paper does not ablate whether combining multiple reward types (e.g., language + action) would outperform action-only reward — a natural extension that could potentially capture the Goal improvement of language-based RL while preserving the Long improvement of action-based RL.

SFT-based vs. RL-based alignment: The comparison with ECoT (Table 7) shows that SFT-based alignment on the same 12.8K samples degrades performance across all categories (average 78.7% → 67.7%), while RL-based alignment improves it (78.7% → 86.8%). This is a decisive ablation: it demonstrates that the benefit of Stage 2 comes specifically from the RL optimization process (exploration, advantage-weighted reinforcement, KL-constrained updates) rather than from simply exposing the model to reasoning-action pairs. The paper also notes, without a dedicated table, that this SFT-aligned model shows degraded performance on general MLLM benchmarks — evidence of catastrophic forgetting that the RL approach avoids through its KL penalty.

Contribution of the RL stage relative to SFT: Across all benchmarks, the ROBOALIGN w/o RL (SFT only) baseline provides moderate improvements over the untuned base model: LIBERO 73.9% → 78.7% (+6.5%), CALVIN 2.16 → 1.89 (−12.5%, a degradation), real-world 32.3% → 55.2% (+71%). The RL stage then adds substantial further gains: LIBERO 78.7% → 86.8% (+10.3% over SFT), CALVIN 1.89 → 2.57 (+36.0%), real-world 55.2% → 66.7% (+20.8%). On CALVIN, the SFT-only model actually underperforms the base model, making the RL stage entirely responsible for the positive result. This highlights that SFT alone, even with a diverse data mixture, is insufficient for improving representations in zero-shot settings — the RL alignment is essential.

Contribution of reasoning data to SFT: The paper demonstrates through qualitative examples (Table 1) that including the 76K zero-shot reasoning distillation data during SFT enables detailed, structured reasoning in the model's outputs, while omitting it results in minimal reasoning ("Go to the cup."). However, no quantitative ablation of VLA performance with and without this reasoning data is provided. The paper states that "the absence of such data results in limited zero-shot reasoning" and that this "reduces the diversity of reasoning samples and hinders RL," but does not include a LIBERO or CALVIN result for an SFT model trained without the reasoning data. This is a missing ablation: it would directly quantify whether the reasoning capability injected during SFT is causally necessary for the RL stage's effectiveness, or whether the RL stage could discover effective reasoning patterns from scratch given enough exploration.

Data efficiency of RL stage: The paper emphasizes that the RL stage uses only 12.8K samples (less than 1% of the 2.28M SFT samples). However, no ablation varies the amount of RL data (e.g., 6.4K, 25.6K, 51.2K samples) to characterize the scaling behavior — does performance continue to improve with more RL data, or does it saturate at 12.8K? Is there an optimal ratio of RL to SFT data for this framework? These questions are not addressed.

Compatibility across model architectures: The experiment with Qwen3VL-8B-Instruct (Table 5) demonstrates that the framework generalizes to a different Qwen model (larger, newer generation), with the same pattern of improvements — particularly strong on Long tasks (60.0% → 78.6%). However, the Qwen3VL experiment uses only 5K RL samples compared to 12.8K for the Qwen2.5VL experiments, and all other hyperparameters are kept the same. The paper does not discuss whether the reduced RL data was a deliberate choice (perhaps reflecting that larger models need less RL data) or a computational constraint. Testing on a non-Qwen architecture (e.g., LLaVA, InternVL, or a LLaMA-based VLM) would provide stronger evidence of architectural generalizability.

Freezing strategy in VLA conversion: The paper freezes the MLLM backbone and trains only the action head during VLA conversion. While this is necessary for cleanly evaluating the MLLM's representational quality (preventing the VLA training from confounding the comparison), it leaves open the question of whether joint fine-tuning of the MLLM and action head would yield further gains. The paper does not include an ablation where the MLLM is unfrozen during VLA training — this could show whether ROBOALIGN's representations are already near-optimal or whether end-to-end fine-tuning would provide additional benefits (and potentially erase the differences between MLLM training methods).

Choice of MLLM layer for action head: The action head uses hidden states from the 18th layer of the Qwen2.5VL-7B-Instruct transformer. No ablation is provided for this choice — would earlier or later layers provide better representations? The 18th layer is approximately 2/3 of the way through a 28-layer model, which might balance semantic richness (later layers) with generality (earlier layers), but without an ablation this remains speculation.

Effect of KL penalty strength: The GRPO objective includes a KL penalty with hyperparameter β against a reference policy, which the paper argues is crucial for preventing catastrophic forgetting. However, no ablation varies β to show sensitivity to this hyperparameter, or demonstrates that removing the KL penalty leads to degradation. This is a relevant ablation because much of the paper's argument about why RL works better than SFT hinges on the KL penalty constraining the policy update.

Critical Assessment

The paper makes several central claims. I examine each against the experimental evidence.

Claim: ROBOALIGN consistently and substantially improves VLA performance over SFT baselines. The evidence strongly supports this claim for the specific models, benchmarks, and training configurations tested. On LIBERO, the improvement is 17.5% relative (73.9% → 86.8%, Table 2). On CALVIN, it is 18.9% (2.16 → 2.57, Table 3). On real-world tasks, it is 106.6% (32.3% → 66.7%, Table 4). These are large effect sizes across three qualitatively different evaluation domains (simulation with in-domain training, simulation with zero-shot generalization, and real-world transfer). However, the claim's generality is constrained by several untested dimensions:

• Single model family: Both tested backbones (Qwen2.5VL-7B, Qwen3VL-8B) are from the same architectural lineage. The paper does not test whether the benefits transfer to VLMs with different pretraining distributions, visual encoders, or architectural choices (e.g., LLaVA's vision-language connector, InternVL's dynamic resolution).
• Single action tokenization scheme: All experiments use FAST tokenization. The paper doesn't test whether the approach works with alternative discretization methods (e.g., RT-2 style binning, VQ-VAE based compression) or with continuous action spaces (which would require a different reward formulation).
• Scale: The experiments use 7B–8B parameter models. Whether the approach scales to larger models (13B, 34B, 70B+) where pretrained capabilities are stronger and catastrophic forgetting may be less severe, or to smaller models (1–3B) where RL might be more challenging due to limited capacity, is unknown.

Claim: The improvement comes specifically from RL-based alignment with action-token rewards, not from alignment in general. The evidence here is strong and well-controlled. The RL alignment target ablation (Table 6) shows that language-based and visual-based RL rewards provide partial improvements (Goal tasks) but fail on Long tasks, while only action-based rewards improve both. The ECoT comparison (Table 7) shows that SFT-based alignment on the same data degrades performance while RL improves it. These are clean, well-designed ablations that isolate the specific contribution claimed. Two missing pieces would further strengthen this claim: (1) an ablation showing what happens if the KL penalty is removed from the RL objective (does it then cause forgetting like SFT?), and (2) an ablation showing whether the improvement persists if the reasoning tags are removed (does the model need explicit chain-of-thought, or does the action-accuracy reward alone reshape representations beneficially?).

Claim: ROBOALIGN bridges the modality gap between language reasoning and low-level actions. The evidence supports this claim indirectly and mechanistically. The representation analysis (Table 8) shows that ROBOALIGN produces substantially more discriminative state representations (KNN accuracy 69.79% vs. 39.06% for the base model). The benchmark results (Table 9) show that embodied reasoning capability improves alongside VLA performance, contradicting the pattern seen with RoboBrain2.0 (excellent reasoning, poor VLA). The differential performance on Goal vs. Long tasks across alignment targets (Table 6) is consistent with the modality gap hypothesis: language-aligned models succeed at tasks requiring instruction interpretation but fail at tasks requiring sustained precise control. However, the paper does not provide a direct causal test of the modality gap hypothesis — for example, by measuring how well different MLLM training methods predict action-relevant quantities (gripper position, object pose) from vision, or by analyzing attention patterns to see whether the MLLM attends more to task-relevant spatial regions after ROBOALIGN. The evidence is correlational, not interventional.

Claim: ROBOALIGN preserves and even improves general MLLM capabilities. This claim is supported by the MLLM benchmark results (Table 9), which show improvements or maintenance across all tested benchmarks (MMStar, Robot-R1 Bench, RoboSpatial, Where2Place, BLINK). However, the benchmark coverage is limited — five benchmarks, with only one (MMStar) measuring general visual understanding. Testing on a broader suite (e.g., MMBench, MME, SEED-Bench, POPE for hallucination) would provide more comprehensive evidence. The paper's report that ECoT-based alignment degrades general capabilities (mentioned in text without a dedicated table) is an important contrast, but the absence of quantified benchmark results for the ECoT model limits the ability to assess the magnitude of this degradation.

Genuine weaknesses in the experimental design:

1. No baseline that combines language-based and action-based RL: Given that language-based RL dramatically improved Goal tasks on LIBERO (59.0% → 90.0%, the best single-category result in the entire paper) while action-based RL improved Long tasks, a natural question is whether a multi-objective RL reward combining both signals would outperform either alone. This experiment is not present.

2. Limited statistical reporting: No confidence intervals, standard errors, or significance tests are reported for any VLA result. For LIBERO, with 500 trials per category, a 1–2% difference between methods near ceiling (e.g., Spatial: 95.2% vs. 93.8%) may not be statistically distinguishable. For the real-world experiments with only 96 trials per task, variability could be substantial particularly for tasks with middling success rates. The "Box to plate" real-world task shows ROBOALIGN underperforming the base model (58.3% vs 70.8%) — without error bars, it's impossible to determine whether this is a genuine negative effect or sampling noise.

3. The CALVIN SFT-only negative result: ROBOALIGN w/o RL underperforms the base model on CALVIN (1.89 vs. 2.16). This means the SFT stage, despite incorporating diverse embodied reasoning data, actually degraded zero-shot generalization. The paper uses this to argue for the necessity of the RL stage, but it also raises a concern: if the SFT model is worse than the base model, and RL improves it, is RL recovering capabilities that SFT damaged, or is it genuinely pushing beyond the base model's capabilities? The fact that ROBOALIGN (2.57) significantly exceeds the base model (2.16) suggests genuine improvement, but the negative SFT result warrants investigation into why diverse SFT hurts zero-shot performance — is it the BridgeV2 action data causing interference, or the sheer volume of data causing forgetting, or something else?

4. Missing difficulty-stratified analysis: Unlike the reference example paper (which stratified all results by prompt difficulty), ROBOALIGN reports only aggregate and category-level results. Within each LIBERO category, there are likely easier and harder tasks — does ROBOALIGN help uniformly, or does it show the same difficulty-dependent patterns seen in the reference paper (helping most on medium tasks, not helping on very easy or very hard tasks)? Such an analysis would provide richer insight into when the method works.

5. Real-world evaluation limited to pick-and-place: The four real-world tasks are all variants of pick-and-place with different objects and containers. This tests generalization to a different robot and visual environment, but not to qualitatively different task types (e.g., articulated object manipulation, tool use, deformable object handling, bimanual coordination). The claim that ROBOALIGN improves VLA performance in "real-world settings" should be qualified as applying specifically to tabletop pick-and-place with a single arm.

6. No comparison to simply scaling the VLA training data: A pragmatic alternative to MLLM alignment is to collect more robot demonstration data and train the VLA action head for longer. The paper doesn't include an experiment showing whether the gain from ROBOALIGN (e.g., +8.1% on LIBERO over the SFT baseline) could be matched by, say, training the baseline VLA for 2× more steps or with 2× more demonstration data. This would help practitioners decide whether investing in MLLM alignment or in data collection is more cost-effective.

7. Single RL algorithm tested: The paper uses GRPO exclusively. Whether the benefits are specific to GRPO's group-normalized advantage estimation and KL penalty formulation, or whether other RL algorithms (PPO with a learned value function, DPO, REINFORCE with baseline) would achieve similar results, is not tested.

6. Limitations and Trade-offs

6.1 The Difficulty Estimation Cost Is Excluded from the Headline Efficiency Numbers

The assumption or constraint. The training framework assumes access to 12.8K BridgeV2 samples for RL that are pre-paired with ground-truth FAST token sequences — the accuracy reward $r_a$ requires a known target $T^{\text{target}}_{1:m}$ to compute the prefix match. For the domains where ROBOALIGN is evaluated (LIBERO, CALVIN, real-world), this ground-truth pairing exists because BridgeV2 provides demonstration data with action labels. However, the paper does not provide any mechanism for generating or adapting this reward signal when expert demonstrations with action labels are unavailable or when the action space differs from the training distribution (different robot kinematics, different control frequencies, different gripper types). The paper does not acknowledge this dependency as a limitation explicitly — it is an implicit constraint of the pipeline.

The consequence. A practitioner wanting to apply ROBOALIGN to a new robot or task domain must first collect expert demonstrations with action labels and tokenize them via FAST, which requires (a) running the full DCT + BPE pipeline on the action data, (b) training or adapting the BPE tokenizer if the action distribution differs from BridgeV2, and (c) generating enough paired (image, instruction, FAST token sequence) examples for the RL stage. The paper provides no guidance on the minimum number of demonstrations needed, whether fewer than 12.8K samples suffice, or whether demonstrations collected on one robot can serve as reward supervision for a different robot through domain adaptation. The headline claim that "less than 1% additional data" is required for the RL stage is true only relative to the SFT data volume, not relative to the effort of collecting and tokenizing robot demonstrations. For a new domain where no action-labeled dataset exists, the "additional data" requirement is effectively 100% of the RL training data, collected from scratch.

What evidence exists in the paper. The paper's experiments use BridgeV2 — an existing large-scale dataset with action labels — for both the SFT and RL stages. It does not ablate the size of the RL dataset (e.g., testing with 1K, 5K, 12.8K, 25K samples) to characterize whether the 12.8K figure is a minimum, an optimum, or an arbitrary choice. The Qwen3VL experiment uses 5K RL samples (Table 5), but this is presented as a cheaper variant rather than as a systematic sweep. No experiment tests training with action labels from a different robot embodiment than the evaluation environment to assess whether the reward signal transfers across kinematic differences.

Mitigation status. The paper does not address this limitation, propose alternative reward formulations that do not require paired action demonstrations (e.g., using task success as a sparse reward, or using a learned forward dynamics model to evaluate action proposals), or suggest strategies for adapting the FAST tokenizer to new embodiments.

6.2 Single Action Tokenization Scheme with No Alternative Tested

The assumption or constraint. The entire ROBOALIGN framework — both the SFT action generation capability and the RL accuracy reward — depends on FAST tokenization (Pertsch et al., 2025), which converts action chunks into discrete tokens via DCT followed by BPE compression. The reward function $r_a$ measures exact token-level prefix matches, which is only defined because FAST produces discrete tokens from a finite vocabulary of size $2K$. The paper does not test whether ROBOALIGN works with alternative action representations: continuous action vectors (which would require a different reward, e.g., L2 distance in action space), RT-2 style discretization (uniform binning of action dimensions), VQ-VAE based compression, or raw action chunk prediction without tokenization. Additionally, the paper does not ablate FAST-specific hyperparameters — the vocabulary size $K$, the chunk length $H$, or the DCT compression ratio — to determine whether the method's success depends on particular configurations of the tokenizer.

The consequence. A practitioner using a different action representation (e.g., a VLA that predicts continuous actions through a diffusion head without an intermediate tokenization step, or one that uses a different discretization scheme) cannot directly apply ROBOALIGN's RL reward formulation. The prefix-matching reward assumes (a) actions are represented as token sequences, (b) prefix matching is a meaningful measure of action quality (i.e., early tokens encode temporally earlier parts of the motion), and (c) the token vocabulary is small enough that exact token matches occur with non-negligible probability. If a different tokenizer produces tokens that are not temporally ordered, or if the vocabulary is too large for prefix matches to provide a dense learning signal, the reward function may become ineffective. The paper provides no evidence that ROBOALIGN's benefits stem from the alignment principle rather than from specific properties of FAST tokenization.

What evidence exists in the paper. All experiments — LIBERO, CALVIN, real-world, Qwen3VL compatibility — use FAST tokenization with the same configuration. The paper does not include an ablation comparing FAST against an alternative tokenization scheme (e.g., uniform binning with a comparable vocabulary size) to test whether the representation matters, or whether any discrete action encoding that enables prefix-matching rewards would work. The representation analysis (Table 8) shows that ROBOALIGN improves MLLM hidden state quality for state classification, which is independent of the tokenization scheme — this hints that the benefits might transfer, but it is not a direct test.

Mitigation status. The paper does not acknowledge this as a limitation or discuss how the framework would need to be modified for continuous action spaces or alternative discretizations. The reliance on FAST is presented as an implementation choice rather than a constraint.

6.3 The Method Is Evaluated on Only a Single Model Family

The assumption or constraint. All VLA experiments use MLLM backbones from the Qwen family: Qwen2.5VL-7B-Instruct for the main experiments (Tables 2–4, 6–7, 9) and Qwen3VL-8B-Instruct for the compatibility test (Table 5). Both models share the same architectural lineage, pretraining data distribution, vision encoder design, and training methodology (Bai et al., 2025; Team, 2025). The paper does not evaluate whether ROBOALIGN transfers to MLLMs with fundamentally different architectures — for example, LLaMA-based VLMs (LLaVA), models with different vision-language connectors (InternVL's dynamic resolution, Cambrian-1's vision-centric design), or models trained on different data distributions (proprietary models like GPT-4V, Claude). The paper states in its compatibility section that "ROBOALIGN effectively generalizes across different MLLM architectures" (Section 5.3), but the evidence supporting this claim is limited to two models from the same organization with highly similar designs.

The consequence. The paper's central claim — that ROBOALIGN is "an effective and generalizable approach for advancing VLA training" (Section 6) — overstates the evidence. A practitioner using a non-Qwen MLLM (e.g., LLaVA-1.6, InternVL2, Pixtral, or a proprietary API model) has no empirical basis to expect similar gains. The method could fail for several architecture-specific reasons: (a) the hidden states at the 18th layer of Qwen2.5VL may have different properties than equivalent-depth representations in other architectures (the paper does not ablate layer choice, making the 18th layer an architecture-specific heuristic), (b) the vision encoder's feature resolution and quality may affect how well action-relevant spatial information is preserved through the alignment process, (c) different pretraining data mixtures may produce initial representations that respond differently to the SFT and RL stages, and (d) the specific prompt templates and reasoning format ( thinking...) may interact differently with models that have different instruction-tuning styles.

What evidence exists in the paper. The compatibility experiment (Table 5) shows that ROBOALIGN improves Qwen3VL-8B-Instruct on LIBERO (85.2% → 92.5%), with the same pattern of largest gains on Long tasks (60.0% → 78.6%). This demonstrates generalization within the Qwen family, supporting the claim of architectural transfer, but it does not test a qualitatively different model design. The paper does not cite or discuss whether the method has been attempted on non-Qwen architectures. Additionally, the Qwen3VL experiment uses only 5K RL samples (vs. 12.8K for Qwen2.5VL) and does not report CALVIN or real-world results, making it a weaker test than the main experiments.

Mitigation status. The paper does not acknowledge the restricted model family as a limitation. The compatibility section is presented as positive evidence of generalization, without qualification about the architectural similarity of the two tested models.

6.4 No Measurement of Inference-Time Latency or Computational Overhead of Reasoning

The assumption or constraint. ROBOALIGN's Stage 2 RL trains the MLLM to generate explicit chain-of-thought reasoning (within thinking... tags) before outputting FAST action tokens. The qualitative examples (Table 1, Appendix Figure 7) show that this reasoning can be substantial — multiple sentences describing task decomposition, spatial relationships, and state assessment. The paper's RL training metrics (Appendix Figure 6b) show that mean response length stabilizes between 76 and 86 tokens. However, the paper evaluates only the offline representational quality of the MLLM (by freezing it and training a separate action head), never measuring the online inference cost of generating the reasoning trace before producing actions. The VLA setup bypasses this because the action head reads the MLLM's hidden states directly without autoregressive decoding — the reasoning is never actually generated during VLA inference. This creates a disconnect between how the MLLM is trained (to autoregressively generate reasoning + actions) and how it is deployed (as a frozen feature extractor).

The consequence. The paper's framework produces a model that is trained to be a good feature extractor, but the training procedure itself incurs inference-time costs that are not accounted for in the headline VLA results — and that would be incurred if the model were ever used for autoregressive action generation (e.g., in a discrete-action VLA like RT-2 or OpenVLA that generates action tokens online). For practitioners who want to deploy the MLLM directly as an action generator (rather than as a frozen backbone for an action head), the latency penalty of generating 80+ tokens of reasoning before every action chunk would be substantial. At typical MLLM inference speeds (~20–50 tokens/second on A100-class hardware for a 7B model), the reasoning alone adds 1.5–4 seconds of latency per action chunk. For a control loop running at 10–30 Hz, this is unacceptable. The paper's evaluation strategy sidesteps this issue entirely by using the MLLM as a frozen feature extractor, but this architectural choice is not positioned as a tradeoff — the paper never discusses when one would prefer autoregressive action generation vs. frozen-backbone VLA conversion.

What evidence exists in the paper. No latency measurements, FLOP counts, or wall-clock inference times are reported for any model. Appendix A.1 reports training compute (30 hours for SFT, 1 hour for RL, 1 hour per 10K VLA steps), but these are training-time costs, not inference-time costs. The VLA architecture description (Section 5.1) notes that the action head reads hidden states from the 18th layer, confirming that the reasoning trace is never generated during VLA evaluation, but the paper does not discuss the latency implications if the model were used differently.

Mitigation status. The paper does not acknowledge the training-inference deployment mismatch as a tradeoff, discuss scenarios where autoregressive action generation (with its reasoning latency) would be preferred over the frozen-backbone action head approach, or suggest methods to reduce reasoning length while preserving representational quality (e.g., distilling the RL-trained model's representations into a smaller or faster architecture).

6.5 Hardest Tasks Show Diminishing or Zero Gains from Alignment

The assumption or constraint. The paper's results reveal a consistent pattern: ROBOALIGN provides large gains on some task categories but marginal or zero gains on tasks that are either already near performance ceiling or fundamentally beyond the model's capability. On LIBERO (Table 2), the Spatial and Object categories — where the base model already achieves ~95% success — show marginal improvements or slight regressions (Spatial: 95.2% → 93.8%; Object: 95.0% → 96.0%). On the real-world experiments (Table 4), the "Box to plate" task shows a degradation from 70.8% (base) to 58.3% (ROBOALIGN). On CALVIN (Table 3), the average consecutive task completion of 2.57 means the robot still fails to complete chains longer than 2–3 instructions on average — ROBOALIGN improves the failure rate but does not solve the long-horizon problem. The paper's reference example discussion of difficulty-dependent scaling (where test-time compute strategies help on medium-difficulty problems but fail on very hard ones) provides a framework for interpreting this: ROBOALIGN likely improves tasks within the base model's capability range but cannot overcome fundamental capability gaps. The paper does not analyze its results through this lens.

The consequence. A practitioner deploying a VLA built with ROBOALIGN should expect improvements on tasks where the base model already has non-trivial but sub-ceiling performance (LIBERO Goal: 42.4% → 87.2%; Long: 63.2% → 70.0%), but should not expect improvements — and may see regressions — on tasks where the base model is already near-perfect (ceiling effects) or where the base model fundamentally lacks the visual understanding or physical reasoning needed for the task. The paper provides no principled way to predict which tasks will benefit and which will not, beyond the coarse category labels (Goal, Long, Spatial) that are specific to the LIBERO benchmark. For a new task, there is no pre-deployment diagnostic to estimate whether ROBOALIGN alignment will help or hurt.

What evidence exists in the paper. The pattern is visible across all three evaluation domains: LIBERO Spatial (no gain), real-world Box-to-plate (degradation), and CALVIN where length-5 success improves but remains low (22.2%). The paper does not stratify results by task difficulty within categories, perform error analysis on failure cases to understand why certain tasks don't improve, or propose a difficulty estimation mechanism (analogous to the reference paper's pass@1-based binning) that could predict alignment benefit.

Mitigation status. The paper does not discuss the ceiling effects, the task-specific regressions, or the need for difficulty-conditioned deployment strategies. The real-world Box-to-plate regression (70.8% → 58.3%) is reported without comment or analysis, leaving the reader uncertain whether this is a statistically meaningful degradation or sampling noise in a 96-trial evaluation.

6.6 The Method Has Not Been Tested on Tasks Requiring Contact-Rich or Dynamic Manipulation

The assumption or constraint. All evaluation tasks across LIBERO, CALVIN, and the real-world experiments involve quasi-static pick-and-place manipulation: the robot grasps objects from tabletops and moves them to target locations or containers. The tasks do not require dynamic motions (throwing, swinging, catching), contact-rich manipulation (peg insertion, assembly, screwing, door opening with latches), bimanual coordination, deformable object handling (folding cloth, manipulating ropes), or force-sensitive control (pushing with specified force, sliding objects along surfaces). The BridgeV2 training data (Section 4.1, Appendix A.3) is similarly composed of tabletop pick-and-place demonstrations. The paper's claims about "substantial gains in embodied reasoning performance within MLLM tasks as well as in the VLA domain across both short and long horizon tasks" (Section 6) implicitly generalize to all manipulation, but the evidence covers only a narrow slice of the manipulation spectrum.

The consequence. The paper provides no evidence that ROBOALIGN's alignment strategy — which optimizes for FAST token prediction accuracy using a prefix-match reward — transfers to manipulation skills that depend on contact dynamics, force feedback, or rapid motion. FAST tokenization compresses action chunks via DCT, which assumes smooth trajectories — this assumption holds for pick-and-place but may break for tasks requiring sharp force transients (e.g., snapping two parts together) or rapid direction changes (e.g., flipping an object). The prefix-match reward measures whether the model's predicted action tokens match the expert's trajectory early in the sequence, which is appropriate for quasi-static reaching and grasping where small spatial errors early in the motion compound into task failure. For contact-rich tasks, the critical phase may be in the middle or end of the trajectory (e.g., the moment of insertion, the force profile during tightening), and a prefix-based reward that weights early tokens equally to later tokens may not capture task success. A practitioner working on assembly, tool use, or dynamic manipulation has no empirical basis to expect ROBOALIGN to help, and may find that the assumptions embedded in the tokenization and reward design are actively detrimental for their task type.

What evidence exists in the paper. None — all evaluated tasks are pick-and-place. The real-world tasks (Table 4) are explicitly described as "pick-and-place tasks" with objects (teddy bear, box, cup, sponge) moved between bowls, plates, and baskets. The LIBERO tasks include some variation (opening drawers, turning knobs) but remain within the quasi-static manipulation regime. CALVIN tasks include sliding doors and operating switches, but the paper does not separately analyze performance on contact-rich vs. pick-and-place subtasks. There is no ablation testing alternative reward formulations that might be more appropriate for contact-rich tasks (e.g., rewarding trajectory-level similarity via Dynamic Time Warping, or using a sparse task-success reward for the entire sequence).

Mitigation status. The paper does not acknowledge the narrow task distribution as a limitation or discuss how the framework would need to be adapted for contact-rich or dynamic manipulation. The conclusion's statement about "both short and long horizon tasks" is presented without qualification about task type.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a diagnostic reframing rather than a paradigm shift. It does not propose a new architecture, a new dataset, or a new RL algorithm. Instead, it identifies a specific, previously underappreciated failure mode — the modality gap between language-based embodied reasoning improvements and actual VLA performance — and provides a training framework that directly addresses it. The conceptual contribution is the claim that optimizing MLLM representations for linguistic reasoning tasks and optimizing them for action generation are not just different objectives — they can be actively in tension, and that bridging this tension requires a reward signal defined in the action modality itself, not in language.

The magnitude of this reframing is moderate but practically significant for the VLA research community. The paper provides compelling evidence that the dominant assumption of the past several years — "make the MLLM better at embodied QA, and the VLA will naturally improve" — is unreliable enough to warrant a change in training methodology. Figure 1, with RoboBrain 2.0 as the strongest embodied reasoner producing the weakest VLA, serves as a memorable anchoring example that should shift researcher intuition about what proxy metrics to trust.

Several prior contradictions are resolved by this reframing:

• Why VLM4VLA (Zhang et al., 2026) found inconsistent correlations between reasoning and VLA performance. The modality gap explains it: language-based improvements help on tasks where high-level instruction understanding is the bottleneck (LIBERO Goal) but fail on tasks where fine-grained spatial and temporal precision dominates (LIBERO Long, CALVIN long-horizon). The inconsistent correlation across tasks in VLM4VLA's analysis likely reflected a mixture of these two task types.

• Why Action-Only SFT improves in-domain performance but catastrophically degrades zero-shot generalization. The paper demonstrates this directly (Tables 2–3): the model overfits to the narrow action prediction distribution, losing the general visual and semantic capabilities that the pretrained MLLM contributed. This is a classic catastrophic forgetting pattern, but its interaction with the modality gap is new: the action-only model has good in-domain alignment but no generalization; the language-only model has good generalization but poor alignment for complex control.

• Why SFT-based alignment (ECoT) degrades performance while RL-based alignment improves it, on the same data. The SFT approach memorizes reasoning traces from BridgeV2 that don't transfer to new environments; the RL approach discovers generalizable reasoning principles through exploration and reward shaping. This resolves the tension between prior work that advocated for SFT on reasoning traces (Zawalski et al., 2024) and the DeepSeek-R1 paradigm that advocates for RL-based reasoning optimization (Guo et al., 2025), at least for the embodied domain.

The paper redirects research attention in several concrete ways:

Become more attractive: (1) Developing better action-aligned reward functions for RL — not just prefix accuracy, but trajectory-level metrics, task-success signals, or learned reward models that capture manipulation quality beyond token matching. (2) Investigating why RL-based alignment preserves general capabilities while SFT-based alignment does not, potentially leading to better understanding of the KL penalty's role in preventing forgetting. (3) Exploring intermediate alignment objectives between language and actions — the paper's ablation (Table 6) shows a gradient from language (helps Goal, hurts Long) to visual trajectory (helps Goal, neutral on Long) to low-level actions (helps both), suggesting that alignment targets exist on a spectrum of action-specificity.

Become less attractive: (1) Investing heavily in better embodied VQA benchmarks and training recipes as a primary path to better VLAs — the paper shows this is a misaligned proxy. (2) Scaling up SFT on reasoning-action pairs as a substitute for RL — the ECoT comparison (Table 7) is a clear negative result. (3) Assuming that improvements in MLLM reasoning benchmarks will automatically cascade to downstream robotics tasks without explicit action-space evaluation.

Follow-Up Research This Work Enables

Characterizing the scaling behavior of RL data volume for action alignment. The paper uses 12.8K RL samples for Qwen2.5VL and 5K for Qwen3VL, but provides no sweep over RL dataset size. A systematic ablation varying RL data from 1K to 100K samples on a fixed SFT checkpoint, measuring both VLA performance (LIBERO, CALVIN) and MLLM benchmark scores (Table 9 suite), would reveal whether performance saturates at 12.8K, continues to improve with more RL data, or exhibits an inverse-U (too much RL causes forgetting despite the KL penalty). This would directly inform practitioners about the cost-benefit tradeoff of scaling the RL stage. A strong follow-up would also measure whether the optimal RL data volume depends on the SFT data mixture — does a model with less diverse SFT data need more or less RL?

Testing whether the action-alignment benefit transfers across robot embodiments with different kinematics. The paper trains on BridgeV2 (WidowX arm, 7-DoF) and evaluates on LIBERO/CALVIN (Franka Panda, 7-DoF) and a real Franka Research 3 — all single-arm manipulators with similar kinematic structure. A decisive stress test would train the SFT+RL alignment on BridgeV2 and evaluate VLA performance on a benchmark with qualitatively different kinematics: a bimanual setup, a mobile manipulator with a different end-effector (suction gripper vs. parallel jaw), or a robot with significantly different workspace dimensions (e.g., a small tabletop arm vs. a large industrial arm). If the benefit transfers, it suggests ROBOALIGN learns general manipulation principles; if it doesn't, the alignment may be overfitting to the kinematic priors implicit in the BridgeV2 action distribution. The FAST tokenizer's DCT compression assumes smooth trajectories — an embodiment with jerky dynamics or different control frequencies could break this assumption. Measuring the cross-embodiment transfer would directly bound the generality claim.

Combining language-based and action-based RL rewards in a multi-objective framework. Table 6 shows a striking pattern: language-based RL achieves 90.0% on LIBERO Goal (the best single-category result in the paper) but degrades Long to 58.2%, while action-based RL achieves 87.2% on Goal and 70.0% on Long. The complementary strengths are obvious but untested. A natural experiment would train with a weighted sum of both rewards — e.g., r = α * r_action + (1-α) * r_language — sweeping α from 0 to 1, and measure whether an intermediate α achieves the best of both worlds (Goal near 90%, Long near 70%) or whether the objectives are fundamentally in tension (any mixture underperforms the better single-objective result). The paper's theoretical framing of the modality gap would be refined by understanding whether language and action alignment can coexist in the same model or whether optimizing for one inevitably degrades the other.

Probing whether the reasoning trace is causally necessary or merely correlational for the representation improvement. The paper shows that ROBOALIGN-trained models produce better representations (Table 8) and better VLA performance, but does not demonstrate that the explicit chain-of-thought reasoning — generating thinking... tokens autoregressively — is the mechanism. An ablative experiment would train an RL-aligned model with the same action-accuracy reward but without the format reward r_f and without requiring reasoning tags, allowing the model to either skip reasoning entirely or discover its own internal computation patterns. If this model achieves comparable VLA performance, the reasoning trace is a training artifact rather than a mechanism — the RL reward alone restructures the representations beneficially. If it underperforms, the explicit reasoning is functionally necessary. A deeper variant would analyze the hidden states at the 18th layer (where the action head reads) during reasoning generation vs. during direct action prediction, measuring whether the reasoning process produces richer representations or whether equivalent information is present regardless. This would clarify whether the paper's method works because of chain-of-thought or despite it.

Testing on contact-rich and dynamic manipulation tasks to determine the limits of prefix-accuracy rewards and DCT-based tokenization. The paper's evaluation is entirely quasi-static pick-and-place. A follow-up that applies ROBOALIGN to a benchmark requiring peg insertion, gear assembly, or deformable object manipulation (e.g., RLBench with contact-rich tasks, or the FMB benchmark for food manipulation) would test whether FAST tokenization's smoothness prior and the prefix-accuracy reward transfer to tasks where (a) force/torque sensing matters, (b) the critical success-determining phase is mid-trajectory rather than early-trajectory, and (c) trajectories may be non-smooth (impact, snapping, vibration). A negative result here — ROBOALIGN fails on contact-rich tasks — would motivate alternative tokenization schemes (e.g., learning a task-adaptive compression rather than fixed DCT) or alternative reward functions (e.g., Dynamic Time Warping distance in action space, sparse task-success reward with a learned transition model, or a reward that weights trajectory phases by their importance to task success).

Comparing ROBOALIGN against simply scaling the downstream VLA training data or training steps. The paper's VLA training uses fixed budgets: 60K steps for LIBERO, 100K for CALVIN. A pragmatic ablation would train the baseline (Qwen2.5VL-7B-Ins, no MLLM alignment) VLA for 2×, 4×, and 8× the number of steps, or with 2× and 4× the demonstration data, and measure at what point it matches ROBOALIGN's performance. If the baseline catches up with 2× more VLA training, the benefit of MLLM alignment is modest and perhaps not worth the complexity of the two-stage pipeline. If the baseline never catches up (saturates below ROBOALIGN's performance), the alignment provides a genuine representational advantage that data scaling alone cannot replicate. This experiment would directly inform the cost-benefit tradeoff between investing in MLLM alignment vs. investing in robot data collection, which is a practical decision every VLA practitioner faces.

Practical Applications and Downstream Use Cases

Improving open-source VLA backbones for long-horizon manipulation. For robotics labs and companies building on open-source VLAs (OpenVLA, Octo, GR00T-N1.5), the immediate application is to replace the default pretrained MLLM backbone with a ROBOALIGN-trained version. The paper's headline numbers — 17.5% improvement on LIBERO, 18.9% on CALVIN, 106.6% in real-world settings — are directly applicable because the VLA conversion pipeline is identical regardless of the MLLM training method. A group deploying a VLA for warehouse pick-and-place or lab automation could apply ROBOALIGN to their MLLM backbone using existing BridgeV2 data (for the SFT and RL stages) without collecting any new robot demonstrations, since the alignment training data is domain-disjoint from the deployment data. The key practical decision is whether the ~30 GPU-hours for SFT and ~1 GPU-hour for RL (on 8×H200) is justified by the expected performance gain. For long-horizon tasks (the paper's strongest result domain), the answer appears to be yes; for short-horizon tasks near performance ceiling, the marginal benefit may not warrant the compute.

Data-efficient adaptation to new robot embodiments. The paper demonstrates that ROBOALIGN's SFT+RL alignment transfers across robot embodiments (BridgeV2 → LIBERO/CALVIN/real Franka). A practitioner deploying a VLA on a custom robot platform with limited demonstration data would benefit from this transfer: they can align the MLLM on the abundant BridgeV2 data (which uses a different robot), then train only the action head on their limited in-domain demonstrations. Since the aligned MLLM representations encode finer-grained state information (KNN accuracy improving from 39% to 70%, Table 8), the action head needs fewer examples to learn the mapping from enriched representations to the new robot's specific action space. The paper does not directly measure sample efficiency of the VLA stage, but the mechanism is plausible and could be validated by a follow-up that sweeps VLA demonstration counts (e.g., 25%, 50%, 100% of the full dataset) and compares ROBOALIGN vs. baselines.

Serving as a reasoning-aware feature extractor for hybrid VLA architectures. The paper evaluates by freezing the MLLM and training a diffusion action head on its hidden states, never generating the reasoning trace at inference. But the aligned MLLM could serve a dual purpose: (1) as a frozen feature extractor for a fast reactive action head (the primary deployment mode tested in the paper), and (2) as an autoregressive reasoning module that generates explicit chain-of-thought for debugging, failure analysis, or human-interpretable planning when latency constraints are relaxed. The qualitative examples in Appendix Figure 7 show that the RL-trained reasoning is detailed and state-aware — a deployed system could surface this reasoning to a human operator when the action head's confidence is low or when the robot encounters a novel situation, enabling informed teleoperation or debugging. This dual-use capability — fast action generation and on-demand interpretable reasoning from the same backbone — is enabled by ROBOALIGN's design of training the MLLM to generate reasoning even though the evaluation bypasses it.

When to Prefer This Method

The paper does not position ROBOALIGN against a specific named alternative method and declare explicit conditions under which each should be preferred. The comparisons are against ablative baselines (Language-Only SFT, Action-Only SFT, SFT-based Alignment) that represent variants of the training pipeline rather than competing deployment strategies. The paper's contribution is a training methodology, not a policy architecture or inference-time strategy that would create deployment-time tradeoffs against alternatives. The practical choice for a practitioner is not "ROBOALIGN vs. Method X" but rather "should I invest in the two-stage ROBOALIGN training, or should I just use the base MLLM / language-only fine-tuned MLLM / action-only fine-tuned MLLM as my VLA backbone?" The paper's results suggest that the full ROBOALIGN pipeline is preferable when: (1) the target tasks include long-horizon manipulation sequences where the base model's performance is non-trivial but sub-ceiling, (2) the deployment environment differs from the training environment (requiring generalization), and (3) the additional ~31 GPU-hours of MLLM training is acceptable. When tasks are short-horizon and already near performance ceiling, or when zero-shot generalization is not required (the VLA will be trained and deployed in identical environments), the simpler baselines may suffice. However, these conditions are inferred from the paper's results rather than explicitly articulated by the authors as a decision framework.