RynnVLA-002: A Unified Vision-Language-Action and World Model

ArXiv: 2511.17502

🎯 Pitch

Training a robot's world model and action policy together inside one network boosts real-world task success by 50%—without any pretraining. The secret is that learning to imagine future frames forces the model to truly see objects, while better visual understanding makes those imagined futures sharper. This mutual upgrade even beats heavily pretrained models on simulation benchmarks.

---

1. Executive Summary

This paper introduces RynnVLA-002, a unified framework that jointly trains a Vision-Language-Action model and a world model within a single autoregressive architecture, demonstrating that these two components mutually enhance each other — the world model's image prediction objective reinforces object-centric visual understanding for the VLA, while the VLA's image comprehension improves the world model's generation fidelity. The system employs three key mechanisms: a unified token vocabulary spanning image, text, state, and action modalities for joint discrete modeling; an action attention masking strategy that prevents error accumulation during autoregressive action chunking (isolating each action's generation from prior actions to rely solely on visual input); and a continuous Action Transformer head that generates entire action chunks in parallel via learnable queries (addressing the discrete model's overfitting and non-smooth trajectory issues in real-world deployments). On the LIBERO simulation benchmark, RynnVLA-002 achieves a 97.4% success rate without any pretraining, outperforming strong pretrained baselines, while in real-world LeRobot SO100 arm experiments, incorporating the world model during training boosts the overall success rate by 50% (raising multi-target and distractor-scenario performance from below 30% to over 80%), establishing that the synergy between action prediction and environmental dynamics learning provides substantial gains both in simulation and on physical hardware — though the discrete action variant that performs well in simulation fails entirely on real robots due to overfitting and trajectory discontinuity.

2. Context and Motivation

The Core Problem: VLA Models and World Models Are Developed in Isolation Despite Their Complementary Nature

This paper addresses a fundamental architectural gap in embodied AI: Vision-Language-Action (VLA) models and world models have been developed as separate, non-interacting systems, yet their capabilities are deeply complementary. A VLA model excels at mapping perceptual inputs to actions — it can look at a scene, understand a language instruction, and decide what the robot should do. But it has no explicit model of how those actions will change the world. A world model excels at predicting how the environment will evolve given actions — it understands physics, object interactions, and causality — but it cannot generate actions itself. The paper's central thesis is that training these two models jointly within a single unified architecture produces a system where each component directly enhances the other, yielding performance neither can achieve alone.

This gap matters because it represents a missed opportunity at the architectural level. Current VLA models process actions only as outputs, never as inputs that could inform internal representations. Current world models process actions as conditioning signals for image generation but have no mechanism for action planning. By keeping these capabilities in separate models, prior work leaves on the table a natural form of mutual supervision: the VLA's action prediction task forces the model to develop rich visual understanding, which should improve the world model's image generation quality, while the world model's physics prediction task forces the model to learn how actions causally affect the environment, which should improve the VLA's action planning.

Why This Problem Is Important: Three Fundamental Limitations of Standard VLA Architectures

The paper identifies three specific deficits in standard VLA models (Section 1) that motivate the need for integrated world modeling. Understanding why these deficits matter requires looking at what VLA models actually do internally.

First, standard VLAs cannot fully understand actions. In a typical VLA pipeline, actions exist only on the output side — the model takes in images, text, and proprioceptive state, processes them through its transformer layers, and produces action tokens at the end. The actions never circulate back through the model's internal representations as inputs. This means the model never learns an explicit internal representation of what actions mean in terms of their environmental consequences. It learns to predict the correct action through supervised learning, but it doesn't learn that "moving the gripper 5cm to the left" causes the gripper position to shift in a particular way in the next frame. This is a form of blind mapping — the model can produce the right action without understanding why it's right.

To see why this matters, consider a robot trying to grasp a cup. A standard VLA might correctly predict a grasp pose based on visual features. But if the cup is slightly occluded or the lighting changes, the model has no physics-informed reasoning to fall back on — it can't simulate "if I close the gripper here, will the cup be grasped or will it slip?" because it has never learned to predict the consequences of actions. The world model objective provides exactly this kind of reasoning signal.

Second, standard VLAs lack imagination. They do not predict how the world might evolve given candidate actions, which prevents foresight and counterfactual reasoning. In human decision-making, we constantly simulate potential outcomes: "If I reach for the cup from this angle, my hand will collide with the bottle." Standard VLAs cannot perform this kind of mental simulation because they have no mechanism for rolling forward the state of the world. This limitation is particularly acute in cluttered environments or multi-step tasks where the consequences of an action may not be immediately visible — the model must reason about intermediate states that it cannot directly observe but that determine whether the overall plan will succeed.

Third, standard VLAs have no explicit understanding of physics. Without capturing physical dynamics — object interactions, contact forces, stability constraints — the model cannot internalize the causal structure that governs manipulation. This means that when a standard VLA encounters a novel object configuration or an unfamiliar physical situation, it has no generalizable physics knowledge to draw on. Its predictions are purely statistical correlations learned from training data, which break down under distribution shift.

The world model component directly addresses all three deficits because its training objective — predict the next image frame given the current image and action — requires the model to learn action-aware internal states, to perform forward simulation (imagination), and to capture the physics governing how objects move in response to actions. By integrating this objective into the VLA training process, the model is forced to develop these capabilities as a byproduct of learning to generate actions.

Prior Approaches and Where They Fall Short

The paper situates its contribution against two broad families of prior work, each of which addresses part of the problem but leaves the fundamental gap unresolved.

VLM-Based VLA Models: Strong Perception, Weak Dynamics Understanding

The dominant paradigm for VLA models builds on Multimodal Large Language Models, extending pretrained vision-language architectures with action generation capabilities. The seminal work in this line is RT-2 (Zitkovich et al., 2023), which first demonstrated that a VLM co-trained on robotic trajectories and web-scale vision-language data could produce actions as discrete tokens. This established the template that much subsequent work follows: take a pretrained MLLM, add an action output mechanism, and fine-tune on robot data.

The paper acknowledges several strong results in this family: OpenVLA (Kim et al., 2024) achieves 76.5% average success on LIBERO, π₀ (Black et al., 2024) reaches 86.0%, and OpenVLA-OFT (Kim et al., 2025) reaches 97.1% — the latter approaching the performance ceiling. These models demonstrate that MLLMs provide robust perceptual and decision-making capabilities that generalize across diverse robotic tasks.

However, these models share a common architectural limitation: they treat actions exclusively as outputs. The MLLM backbone processes images and text, then generates actions, but never uses actions as inputs to inform its internal representations. The paper's critique is that this output-only treatment prevents the model from forming an explicit internal representation of action dynamics — the causal relationship between what the robot does and how the world changes. As the paper states in Section 1:

"they cannot fully understand actions because actions reside only on the output side, preventing the model from forming an explicit internal representation of action dynamics."

This is not a training data limitation — it's an architectural one. Even if these models were trained on infinite data, they would still lack the ability to simulate forward dynamics because their architecture provides no mechanism for action-conditioned state prediction. The world model objective fills exactly this gap.

Visual Generation-Based VLA Models: Promising But Partially Realized

A second line of work attempts to model dynamics by predicting future visual states. UniPi (Du et al., 2023) generates future visual observations to guide action generation; DREAMGEN (Jang et al., 2025) and GeVRM (Zhang et al., 2025) use video prediction as a planning mechanism; and several joint frameworks (Guo et al., 2024; Zheng et al., 2025; Li et al., 2025) co-generate future frames and corresponding actions.

These approaches take a step in the right direction — they recognize that predicting future states provides useful information for action planning. However, as the paper notes, they face three persistent challenges: visual fidelity (generated images often have artifacts that degrade planning quality), domain transfer (video predictors trained in simulation often fail when deployed on real robots due to visual distribution shift), and computational efficiency (generating high-resolution future frames is expensive, limiting real-time applicability).

More critically from this paper's perspective, these approaches treat visual generation and action generation as separate stages — first predict the future video, then use it to guide action selection. This two-stage pipeline prevents the tight integration that the paper argues is essential: the action prediction should improve visual generation, and visual generation should improve action prediction, in a single end-to-end training process.

World Models as Separate Systems

World models have a rich history in embodied AI (Ha and Schmidhuber, 2018), with modern transformer-based implementations (Robine et al., 2023; Micheli et al., 2022; Wu et al., 2025) achieving impressive video prediction quality. Google's Genie framework (Bruce et al., 2024) demonstrates that world models can be trained at scale through self-supervised video pretraining, and they are now widely used for generating training data (Agarwal et al., 2025), supporting model-based RL (Wu et al., 2025), and selecting policies from candidate pools (Li et al., 2025; Bar et al., 2024).

The limitation the paper identifies is a functional gap: world models can predict future observations given actions, but they cannot generate actions themselves. As stated in Section 1:

"world models are constrained by their inability to directly generate action outputs, resulting in a functional gap that limits their application in scenarios requiring explicit action planning."

This means that a world model, no matter how accurate its predictions, cannot by itself control a robot. It must be paired with a separate policy model — typically a VLA or RL policy — that uses the world model's predictions to plan actions. This separation introduces interface friction: the world model's internal state representations, which encode rich physics-informed dynamics, are not directly accessible to the policy model. The policy sees only the world model's output predictions, not the internal representations that produced them.

Where Prior Joint Frameworks Fall Short: The Discrete Action Bottleneck

The paper's most immediate predecessor is WorldVLA (Cen et al., 2025), which first proposed unifying VLA and world model capabilities within a single architecture by discretizing actions and merging them into a shared token vocabulary with images and text. RynnVLA-002 is explicitly positioned as an evolution of this approach, and the paper uses WorldVLA's limitations to motivate its key technical innovations.

The core problem with WorldVLA's discrete action approach, as the paper explains through its own experiments, is twofold:

First, discrete autoregressive action generation suffers from error propagation. In a standard autoregressive model with causal attention, each action token is conditioned on all previous tokens, including previously generated action tokens within the same chunk. When the model makes an error in an early action — which is likely given that pretrained MLLMs have limited exposure to the action modality — that error influences all subsequent actions in the chunk, causing cascading failures. The paper observes this empirically: "naively generating consecutive actions in the autoregressive model degrades the performance" (Section 3.3), with success rates dropping as chunk length increases (Figure 6).

Second, discrete action models trained on limited real-world data severely overfit. Real-world robot datasets are typically small (the paper uses 248-249 demonstrations per task), and large autoregressive architectures like Chameleon's backbone have enormous capacity. When trained on such limited data, the discrete model memorizes training trajectories rather than learning generalizable manipulation skills. The paper reports that the discrete model "rarely succeeds in real-world robot experiments" despite performing well in simulation (93.3% on LIBERO), attributing this gap to "severe overfitting when trained on limited real-world dataset" and "non-smooth movements" caused by the attention mask's isolation of actions within a chunk (Section 3.3).

These two limitations — error propagation from causal attention and overfitting from large model capacity — together create the discrete action bottleneck that RynnVLA-002 must overcome.

How RynnVLA-002 Positions Itself

RynnVLA-002 makes a specific architectural claim: that the limitations of both VLA models and world models can be addressed by training them jointly in a single autoregressive framework with shared parameters, using a hybrid architecture that combines discrete multimodal modeling with a continuous action generation head. This positioning involves several deliberate design choices that distinguish it from prior work.

First, the paper chooses Chameleon (Team, 2024) as its base architecture. Chameleon is a unified model for image understanding and generation — it can both process images as input and produce images as output. This is not an arbitrary choice. The paper's core insight is that by building on Chameleon's existing multimodal token vocabulary and generation capabilities, it can add action tokens as just another modality without redesigning the entire architecture. The image tokenizer (VQ-GAN), text tokenizer (BPE), and the unified vocabulary structure are all inherited from Chameleon. The paper adds state and action tokenizers that discretize continuous proprioceptive states and actions into 256 bins, integrating them into the same 65,536-token vocabulary. This means the LLM backbone processes image, text, state, and action tokens through the same transformer layers with the same parameters — a genuine unification rather than a modular pipeline.

Second, the paper introduces two mechanisms specifically to address the discrete action bottleneck. The action attention masking strategy (Figure 3b) solves the error propagation problem by preventing each action token in a chunk from attending to previous action tokens in the same chunk. Each action is generated based solely on the visual and textual context, not on potentially erroneous prior actions. This is a departure from standard autoregressive generation and represents a specific architectural intervention motivated by the observation that the action modality has weaker generalization than image or text modalities due to its absence during MLLM pretraining.

The continuous Action Transformer head solves the overfitting and trajectory smoothness problems. By using a smaller dedicated network with learnable action queries that generate all actions in parallel, it reduces the model's capacity in the action-specific components (preventing overfitting on small datasets) and eliminates the sequential dependency structure that produces non-smooth trajectories. The Action Transformer uses bidirectional attention within the action chunk, allowing actions to be coordinated with each other rather than generated in isolation. The paper frames this as an evolution rather than a replacement — the discrete action generation remains in the architecture because it provides a useful training signal and accelerates convergence (Figure 8), but the continuous head is what actually deploys on real robots.

Third, the paper makes a specific empirical claim about mutual enhancement. Rather than simply showing that a unified model can perform both tasks, the paper argues that joint training produces a system that is better at each individual task than models trained exclusively on that task. The VLA benefits from the world model's physics prediction objective, which reinforces attention to object interaction dynamics — the paper provides evidence for this through Figure 5, which shows that the jointly trained model retries grasps on failure while the VLA-only model gives up and moves to the target location without grasping. The world model benefits from the VLA's image understanding objective, which improves its ability to predict physically plausible future frames — evidenced by Figure 7, where the jointly trained world model correctly predicts successful grasps while the world-model-only baseline produces inconsistent predictions across camera viewpoints.

Fourth, the paper deliberately avoids pretraining on external large-scale robot datasets. This is a notable positioning choice. In Table 1, RynnVLA-002 achieves 97.4% without pretraining, matching or exceeding models pretrained on LIBERO-90 or massive real-robot datasets (OpenVLA-OFT: 97.1%, UniVLA: 95.2%). The paper draws attention to this: "Surprisingly, our RynnVLA-002, without any pretraining, is still on par with strong baseline models pretrained on either LIBERO-90 or massive real-robot datasets." This positions the joint training objective as a form of self-supervision that substitutes for external pretraining data — the world model objective provides a rich learning signal that compensates for the absence of pretrained visual representations.

However, the paper also acknowledges that this isn't the whole story. In Section 4.3, it shows that world model pretraining (training the world model on the same data before VLA fine-tuning) provides additional gains: average discrete action performance on LIBERO-Goal, Object, Spatial, and Long improves from 62.8% to 67.2% with world model pretraining (Table 8). This suggests that while joint training is powerful, explicit pretraining on the world model objective before adding the VLA objective may provide even stronger initialization — though the paper does not explore this direction in depth.

Fifth, the paper addresses the practical deployment gap between simulation and real robots head-on. Many VLA papers report strong simulation results but either do not test on real hardware or report significant performance drops. RynnVLA-002 explicitly identifies why the discrete model that works well in simulation (93.3%) fails entirely on real robots (0% success, Table 5 line 1) and provides a concrete architectural fix (the continuous Action Transformer) that transfers successfully. The real-world results in Table 2 show competitive or superior performance against strong baselines (GR00T N1.5 and π₀) without pretraining, particularly in cluttered environments where the world model's physics understanding provides the most value (80% vs. 50% on block placement with distractors).

This failure analysis and architectural response is a significant part of the paper's contribution story. It's not just that the model works — it's that the paper diagnoses why the initial approach fails on real robots and designs a targeted solution. The discrete model's failure mode (overfitting on small datasets, non-smooth trajectories from isolated action generation) directly motivates the continuous Action Transformer design, and the paper shows that this fix is both necessary (discrete actions fail entirely on real robots) and sufficient (continuous actions achieve 80%+ on the same tasks).

3. Technical Approach

3.1 Reader Orientation

This paper presents an empirical system-building and analysis contribution whose core idea is that a Vision-Language-Action model and a world model, when trained jointly within a single shared architecture, provide complementary training signals that make each component better at its individual task than it would be if trained in isolation. The system is a unified autoregressive transformer — built on top of the Chameleon multimodal foundation model — that can be queried either to produce robot actions from visual and language inputs (VLA mode) or to predict future image frames from current images and actions (world model mode), with the two objectives sharing all model parameters and being optimized simultaneously during training.

The core problem being solved is architectural: standard VLA models lack an explicit understanding of action dynamics because actions exist only as outputs, while standard world models lack the ability to generate actions even though they understand environmental physics. The solution shape is a hybrid architecture that combines discrete multimodal token modeling (images, text, state, and actions all share a single vocabulary) with a continuous action generation head, where the world model's image prediction objective forces the network to learn action-aware internal representations that directly benefit the VLA's action generation, and the VLA's image understanding objective improves the world model's visual prediction fidelity.

3.2 Big-Picture Architecture (Diagram in Words)

The RynnVLA-002 architecture has five major components:

1. Multimodal tokenizers — four separate encoders that convert raw inputs (images, text, robot state, robot actions) into shared-vocabulary discrete tokens. The image tokenizer is a VQ-GAN inherited from Chameleon; the text tokenizer is a BPE tokenizer also inherited from Chameleon; and the state and action tokenizers are simple discretizers that bin each continuous dimension into one of 256 values. All four token types live in a unified vocabulary of size 65,536.

2. Autoregressive LLM backbone — the Chameleon transformer, which processes sequences of interleaved multimodal tokens through standard causal self-attention (with one key modification: the action attention mask described below). This backbone is shared for both the VLA task and the world model task, meaning the same parameters process image tokens whether the goal is to produce actions or to produce future images.

3. Action attention masking mechanism — a modified attention pattern (Figure 3b) applied specifically during discrete action chunk generation. Unlike standard causal attention where each token can attend to all previous tokens, this mask prevents each action token in a chunk from attending to previously generated action tokens in the same chunk, forcing action generation to rely solely on visual and textual context rather than on potentially erroneous prior actions.

4. Continuous Action Transformer head — a compact parallel-decoding network that takes the full context (language, image, and state tokens processed through the backbone) and, using learnable action queries, generates an entire action chunk in one forward pass with bidirectional attention within the chunk. This head uses L1 regression loss and is the component actually deployed on real robots.

5. Dual training data pipeline — the model is trained on a mixture of two data formats. VLA data provides sequences of [text, state, image history, action chunk] where the model must generate actions. World model data provides sequences of [images, actions, next images] where the model must generate the next image frame. The total loss is the sum of discrete action cross-entropy, image cross-entropy, and continuous action L1 regression (weighted by α = 10).

Information flows as follows: raw sensor inputs (camera images, language instruction, proprioceptive state) enter the tokenizers → discrete tokens are concatenated into a sequence → the LLM backbone processes the sequence with the appropriate attention mask → for VLA queries, the Action Transformer head produces continuous action chunks for robot execution; for world model queries, the backbone autoregressively decodes image tokens to predict future frames.

3.3 Roadmap for the Deep Dive

• First, the data tokenization scheme — how images, text, states, and actions are converted into a shared vocabulary — because this unification is the architectural foundation that makes joint training possible.

• Second, the VLA data format and training objective, including the exact sequence structure, what the model must predict, and the loss function for discrete actions, since this establishes the baseline action generation pipeline.

• Third, the world model data format and training objective, including how autoregressive image generation works in this context and what the image prediction loss looks like, because this is the novel component that provides the physics-learning signal.

• Fourth, the action attention masking strategy for discrete action chunk generation — the problem it solves (error propagation), the exact attention pattern, and why standard causal attention fails for the action modality specifically (this order is important because it addresses the first major limitation of the discrete approach).

• Fifth, the continuous Action Transformer head — its architecture, how it generates action chunks in parallel, why this solves the real-world deployment failures of the discrete model, and how it integrates with the discrete training objective (this comes after the discrete approach because it is explicitly motivated by the discrete model's failures).

• Sixth, the combined training objective — how the three loss terms (discrete action, image generation, continuous action) are weighted and optimized jointly, and why discrete actions are retained even though the continuous head is used for deployment.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a system-building paper whose core technical idea is that three specific architectural mechanisms — a unified multimodal token vocabulary, an action attention masking strategy, and a continuous action generation head — can be combined within a single autoregressive transformer to achieve mutual enhancement between VLA and world model training objectives, and that this mutual enhancement produces measurable improvements in both simulation and real-world robot manipulation tasks.

---

3.4.1 Data Tokenization: Unifying Image, Text, State, and Action Modalities

The first architectural decision RynnVLA-002 makes is to represent all input and output modalities — images, text, robot proprioceptive state, and robot actions — as discrete tokens drawn from a single shared vocabulary. This unification is not an implementation convenience; it is the mechanism that enables the joint training of VLA and world model objectives within the same transformer. If actions were represented in a separate continuous space while images used discrete tokens, the model would need modality-specific processing pathways, preventing the shared parameter learning that produces mutual enhancement.

The image tokenizer is a VQ-GAN model (Esser et al., 2021) inherited from Chameleon, with additional perceptual losses applied to specific image regions such as faces and salient objects (Gafni et al., 2022). The compression ratio is 16:1, meaning each 16×16 pixel patch maps to one discrete token. The codebook size is 8,192, so each image token is an integer in [0, 8191]. For a 256×256 image, this produces 256 tokens; for a 512×512 image, it produces 1,024 tokens. The perceptual loss enhancements inherited from Chameleon ensure that the discrete token representation preserves fine-grained visual details that matter for manipulation tasks — object boundaries, grasp points, and spatial relationships are not smoothed away by the compression.

The text tokenizer is a Byte-Pair Encoding (BPE) tokenizer (Sennrich et al., 2015), also inherited from Chameleon. BPE tokenization works by iteratively merging the most frequent pairs of characters or subwords in the training corpus, building a vocabulary of common subword units. This means common words get their own tokens while rare words are decomposed into smaller pieces, providing a balance between vocabulary size and coverage. The BPE vocabulary is merged into the unified 65,536-token space alongside image, state, and action tokens.

The state and action tokenizers are the only components not inherited from Chameleon, and their design is straightforward because robot state and action vectors are already low-dimensional and continuous. Each dimension of the proprioceptive state vector (which includes joint positions, gripper state, etc.) and the action vector (which includes desired joint movements, gripper commands, etc.) is independently discretized into one of 256 bins. The bin boundaries are determined by the range of values observed in the training data for each dimension — the minimum and maximum values define the interval, which is divided into 256 equal-width bins. This approach follows the convention established by RT-2 (Zitkovich et al., 2023) and adopted by OpenVLA (Kim et al., 2024).

The mapping works as follows: for a single continuous value $v$ in dimension $d$, with training-data minimum $m_d$ and maximum $M_d$, the bin index is computed by linearly scaling $v$ into [0, 255] and rounding to the nearest integer:

$b = \text{round}\left(255 \cdot \frac{v - m_d}{M_d - m_d}\right)$

where $b \in \{0, 1, \ldots, 255\}$ is the discrete bin index assigned to that dimension.

What it computes: a mapping from continuous scalar values to discrete integers by normalizing to the training-data range and quantizing into 256 uniform bins. Each dimension of the state or action vector independently produces one integer token.

Why this form: 256 bins provide sufficient precision for robot control while keeping the vocabulary compact enough that the model can learn meaningful token embeddings. Using fewer bins (e.g., 64 or 128) would lose precision and potentially cause quantization errors that accumulate over action sequences; using more bins (e.g., 512 or 1024) would increase the vocabulary size and make token predictions harder. The linear binning (equal-width intervals) is the simplest approach and works when the data distribution is roughly uniform across the range, which is reasonable for robot joint movements that span their full range of motion during manipulation tasks. An alternative would be to compute data-dependent bin edges that equalize the frequency of each bin, but the paper does not explore this.

Critical architectural detail: all tokens share a single 65,536-size vocabulary. This means the LLM backbone's token embedding table has 65,536 rows, where some rows correspond to image patches, some to text subwords, some to state dimensions, and some to action dimensions. During training, the model learns a single embedding space where semantically related concepts — regardless of modality — map to nearby vectors. For example, the embedding for an action token representing "close gripper" might be adjacent to the embeddings for image patches showing a closed gripper, enabling cross-modal reasoning within the transformer's attention layers.

---

3.4.2 VLA Model Data Format and Discrete Action Training Objective

The VLA component of RynnVLA-002 follows the standard formulation: a policy $\pi$ generates an action $a_t$ conditioned on a language goal $l$, a proprioceptive state $s_{t-1}$, and an observation history $o_{t-h:t}$ of length $h$ frames. This is formalized in Equation 1:

$a_t \sim \pi(a_t \mid l, s_{t-1}, o_{t-h:t})$

where $l$ is the language instruction (e.g., "put the cream cheese in the bowl"), $s_{t-1}$ is the robot's proprioceptive state from the previous timestep (joint positions, gripper state), $o_{t-h:t}$ is the sequence of $h + 1$ most recent image observations (including the current frame $o_t$), and $a_t$ is the action the robot should execute.

What it computes: a distribution over actions given the full context of what the robot sees, where it is, and what it has been instructed to do. Sampling from this distribution produces the specific action to execute.

Why this form: the inclusion of observation history $o_{t-h:t}$ matters because single-frame observations are often ambiguous — a robot arm mid-motion cannot be disambiguated from a stationary arm based on a single image. The history provides temporal context that resolves this ambiguity. The proprioceptive state $s_{t-1}$ provides information that is difficult to infer from vision alone, such as exact joint angles and gripper force. The paper uses $h = 1$ for most experiments (i.e., $M = 2$ historical frames in the sequence including the current frame), which provides temporal context while keeping sequence length manageable.

The exact token sequence format for VLA data is:

$\texttt{\{text\}} \; \texttt{\{state\}} \; \underbrace{\texttt{\{image-front-wrist\}}}_{\times M} \; \overbrace{\underbrace{\texttt{\{action\}}}_{\times K}}^{\mathcal{L}_{dis\_action}}$

In operational terms: the input sequence begins with the text instruction tokens representing the question "What action should the robot take to + <task> + ?", followed by the proprioceptive state tokens, then $M$ image frames from the front and wrist cameras (each encoded into 256 or 1024 tokens depending on resolution). The model must then generate $K$ action tokens, where $K$ is the action chunk size — the number of future actions to predict in one forward pass.

The text prefix is deliberately templated as a question rather than a command because the Chameleon backbone was pretrained on conversational and question-answering data, and framing the VLA task as question-answering aligns with its pretraining distribution. The specific template "What action should the robot take to + <task> + ?" converts a task description like "put the cream cheese in the bowl" into a natural language query.

The training loss for discrete actions is standard cross-entropy:

$\mathcal{L}_{dis\_action} = -\sum_{k=1}^{K} \sum_{d=1}^{D} \log p(a_{k,d} \mid \text{context}, a_{1:k-1, 1:D}, a_{k, 1:d-1})$

where $K$ is the number of actions in the chunk (5 or 10 depending on the task), $D$ is the number of dimensions per action, $a_{k,d}$ is the ground-truth discrete token for dimension $d$ of action $k$, and $p(\cdot)$ is the model's predicted probability for that token given all previous tokens in the sequence (including previous action tokens, unless the action attention mask modifies this — see Section 3.4.4).

What it computes: the negative log-likelihood of the ground-truth action tokens under the model's predicted distribution, summed over all dimensions of all actions in the chunk. This is the standard maximum-likelihood objective for categorical distributions.

Why this form: cross-entropy is the correct loss for discrete token prediction because it directly optimizes the model to assign high probability to the correct action tokens. The autoregressive factorization (predicting each dimension of each action conditioned on previously predicted dimensions) is necessary because the action tokens are generated sequentially within the transformer's causal attention framework — the model cannot peek at future tokens when predicting the current one, so the loss must match this constraint. The sum over chunk size $K$ means that errors in early actions are penalized independently of errors in later actions, which is appropriate because each action in the chunk will be executed sequentially on the robot and errors in different timesteps have independent consequences.

---

3.4.3 World Model Data Format and Image Prediction Training Objective

The world model component follows the formulation in Equation 2:

$\hat{o}_t \sim f(o_t \mid o_{t-h:t-1}, a_{t-h:t-1})$

where $o_{t-h:t-1}$ is the history of $h$ past image observations, $a_{t-h:t-1}$ is the history of past actions, and $\hat{o}_t$ is the predicted next image observation.

What it computes: a prediction of what the robot's cameras will see at the next timestep, conditioned on what they have seen recently and what actions the robot has taken. This is a forward dynamics model — it answers the question "if I execute these actions, what will the world look like?"

Why this form: the conditioning on both image history and action history is essential. Image history alone (predicting future frames from past frames, a pure video prediction task) would learn to extrapolate motion but would not understand the causal relationship between the robot's actions and environmental changes — it would not know that the gripper opens because the open-gripper action was commanded. Action history alone (predicting frames from actions, without visual context) would be impossible because the action commands specify intended motion but the actual outcome depends on the current state (e.g., "move left 5cm" produces different visual changes depending on where the arm starts). The combination of both inputs allows the model to learn the physics of action-conditioned state transitions.

The exact token sequence format for world model data is:

$\underbrace{\texttt{\{images-front-wrist\}} \; \texttt{\{action\}} \; \overbrace{\texttt{\{images-front-wrist\}}}^{\mathcal{L}_{img}}}_{\times N}$

In operational terms: the input begins with the current image observation tokens, followed by the action tokens representing what the robot did between the current and next frame, then the model must generate the next image observation tokens. The text prefix for world model data is a fixed template: "Generate the next frame based on the current image and the action." This is the same for all training instances because the world model task does not vary by user instruction — the physics of the environment are determined by the actions, not by a language goal.

This sequence can repeat $N$ times in an autoregressive manner, meaning the model can generate multiple future frames by feeding its own predictions back as input. For computational efficiency, the paper uses $N = 1$ during training — the model predicts one frame ahead at a time, and longer rollouts would be generated by iteratively applying this single-step prediction at inference time (though the paper does not evaluate multi-step rollouts).

The training loss for image prediction is cross-entropy over the discrete image tokens:

$\mathcal{L}_{img} = -\sum_{p=1}^{P} \log p(t_p \mid \text{context}, t_{1:p-1})$

where $P$ is the number of image tokens (256 for 256×256 images, 1024 for 512×512), $t_p$ is the ground-truth discrete token for image patch $p$, and $p(\cdot)$ is the model's predicted probability.

What it computes: the negative log-likelihood of the ground-truth image token sequence under the model's autoregressive generation. The model must predict each patch of the next image conditioned on all previous patches in the same image, capturing both global structure (early patches set the overall layout) and local texture (later patches refine details).

Why this form: autoregressive image generation with discrete tokens is the standard approach for transformer-based image generation (as in VQ-GAN, DALL-E, and Chameleon itself). The cross-entropy loss is appropriate because the image tokens are drawn from a discrete vocabulary — it's a classification problem where each patch must be assigned to one of 8,192 possible visual patterns. The autoregressive factorization (predicting patches left-to-right, top-to-bottom) introduces a spatial prior that helps the model learn coherent image structure — patches in the top-left are generated first and influence all subsequent patches, establishing the global composition before local details are filled in.

Why autoregressive image generation matters for the joint training story: the world model's image prediction task is much harder than the VLA's action prediction task. Generating 256–1024 discrete tokens for a full image requires the model to capture fine-grained visual details, object identities, spatial relationships, and physical consistency. This intense visual prediction pressure forces the shared LLM backbone to develop rich action-conditioned visual representations — representations that understand how the gripper's movement in the action tokens should manifest as changes in the image tokens. These representations are then available to the VLA head when it must predict actions, providing the physics-informed reasoning that standard VLA models lack.

---

3.4.4 Action Attention Masking: Preventing Error Propagation in Discrete Action Chunks

The paper identifies a specific failure mode when naively generating multiple consecutive actions autoregressively: error propagation. This section explains the mechanism of that failure, the proposed solution, and why it works.

The problem: causal attention chains errors across action tokens. In a standard autoregressive transformer, every token can attend to all tokens that precede it in the sequence. When generating action tokens, this means the second action attends to the first action, the third attends to both previous actions, and so on (see Figure 3a for the default VLA attention mask). If the model makes a mistake in the first action — predicting a slightly wrong joint angle or gripper position — that erroneous token becomes part of the context for generating the second action. The second action, conditioned on a partially incorrect state representation, is more likely to be wrong. This error then conditions the third action, and so on, creating a cascade where early errors amplify through the chunk.

Why the action modality is particularly vulnerable to this. The paper argues that pretrained MLLMs like Chameleon have been exposed to enormous amounts of image and text data during pretraining, giving them robust generalization in those modalities. But the action modality — discretized robot commands — was entirely absent from pretraining. As the paper states in Section 3.3:

"the generalization of the action is not that strong as this modality was not involved during pretraining the MLLM."

This means that the model's action token predictions are inherently less reliable than its image or text predictions because the action token embeddings and the attention patterns involving action tokens are randomly initialized and must be learned entirely from the limited robot training data. When one unreliable action prediction conditions the next unreliable action prediction, the compound error grows rapidly.

The empirical evidence for this failure. Figure 6 shows that with the default causal attention mask (labeled "Vanilla"), the success rate on LIBERO decreases as the action chunk length increases. For Object tasks, success drops from roughly 82% at chunk size 1 to about 73% at chunk size 10. The paper's row 3 of Table 3 provides complementary evidence: adding action chunking without the attention mask (comparing to row 1 without chunking) actually reduces performance on Spatial tasks (from 77.8% to 67.3% average when accounting for different chunk configurations).

The solution: an attention mask that isolates actions. The paper introduces a modified attention pattern, shown in Figure 3b, that works as follows:

Each action token in a chunk can attend to:

• All text tokens (the language instruction)
• All image tokens (current and historical frames)
• All state tokens (proprioceptive state)
• Tokens within its own action (other dimensions of the same action)
• NO tokens from previous actions in the same chunk

Each action token cannot attend to:

• Any token from any action earlier in the chunk

In operational terms: when generating the third action in a chunk of 10, the model processes the full visual and textual context plus the tokens it has already generated for the current action, but it cannot see what it predicted for the first or second actions. Each action is generated as if it were the only action in the sequence, grounded solely in the perceptual inputs.

Why this masking pattern works. The masking strategy converts the problem from sequential action generation (where errors cascade) to parallel independent action prediction (where each action is separately inferred from the visual input). The key insight is that for short action chunks — 5 to 10 timesteps — the visual observation at the start of the chunk contains sufficient information to predict all actions in the chunk without needing to see intermediate action states. The visual input captures the current scene configuration, and the actions represent a short trajectory through that configuration that the model can predict holistically.

If the action chunks were much longer (e.g., 50–100 timesteps), this assumption would break down — later actions genuinely depend on the outcomes of earlier actions because the scene changes significantly over that horizon. But for the chunk sizes used (5 for shorter tasks, 10 for longer tasks), the visual context at chunk start is a sufficient statistic for the entire trajectory.

The tradeoff: smoothness vs. independence. The paper explicitly acknowledges a downstream consequence of this masking strategy in Section 3.3: "our designed attention mask makes the autoregressive model generate each action in isolation within the same chunk, which cannot ensure trajectory continuity, resulting in severe shaking and non-smooth movements." Because each action is predicted independently from the visual input without awareness of its neighbors in the chunk, there is no mechanism enforcing that action $k$ flows smoothly into action $k+1$. The actions are individually plausible given the visual input, but they may not form a coherent trajectory when stitched together.

This is the tension the paper navigates: the attention mask is necessary to prevent error propagation and achieve reasonable performance with discrete actions (Table 3 shows improvement from 54.0% to 76.6% on the LIBERO average with the mask), but it introduces trajectory discontinuity that becomes fatal in real-world deployment where smooth physical motions are essential. This tension directly motivates the continuous Action Transformer head — which uses bidirectional attention within the chunk and explicit trajectory-level supervision — as a solution that provides both error resilience and trajectory smoothness.

---

3.4.5 Continuous Action Transformer Head: Parallel Decoding with Learnable Queries

The Action Transformer head is the component that enables RynnVLA-002 to deploy successfully on real robots, addressing the two failure modes of the discrete approach: overfitting on small real-world datasets and trajectory discontinuity from the isolated action generation mask. This section explains its architecture, training, and how it integrates with the discrete model.

Architecture. The Action Transformer follows the design of Zhao et al. (2023), using a compact transformer decoder with learnable action queries. The inputs are:

• The full sequence of hidden states from the LLM backbone after processing the text, image, and state tokens. These hidden states encode the perceptual and linguistic context in a modality-agnostic representation space.

• A set of $K$ learnable action queries, where $K$ is the action chunk size (5 or 10 depending on the task). These queries are randomly initialized embedding vectors that are trained to extract action-relevant information from the context hidden states.

The Action Transformer processes these inputs through several layers of cross-attention (action queries attend to the context hidden states) and self-attention (action queries attend to each other with bidirectional masking). The output is a sequence of $K$ continuous action vectors, each with the same dimensionality as the robot's action space.

What "bidirectional attention" means operationally: unlike the autoregressive LLM backbone where each token can only see previous tokens, the Action Transformer's self-attention allows each action query to attend to every other action query, regardless of their positions in the chunk. This means the model can coordinate actions across the entire trajectory — the 5th action can influence the 1st action's representation, and vice versa — producing a globally consistent trajectory rather than a sequence of independently predicted points.

Why this is architecturally compact. The paper emphasizes that the Action Transformer is "significantly smaller than the base LLM." This is a deliberate design choice: the LLM backbone (Chameleon) has orders of magnitude more parameters than the robot training data has examples, making it prone to overfitting. The Action Transformer, with far fewer parameters, serves as a bottleneck that forces the model to learn only the essential action generation capabilities that generalize from limited data. The LLM backbone provides rich perceptual representations (what objects are present, where they are, what the instruction means), and the compact Action Transformer maps these representations to actions without memorizing spurious correlations.

Training loss. The Action Transformer is supervised with L1 regression loss:

$\mathcal{L}_{conti\_action} = \sum_{k=1}^{K} \| a_k^{\text{pred}} - a_k^{\text{gt}} \|_1$

where $a_k^{\text{pred}}$ is the predicted continuous action vector for timestep $k$, $a_k^{\text{gt}}$ is the ground-truth action vector from the expert demonstration, and $\|\cdot\|_1$ is the element-wise L1 norm (sum of absolute differences across all action dimensions).

What it computes: the mean absolute error between predicted and ground-truth continuous actions, summed over all actions in the chunk. Unlike the discrete action loss (cross-entropy over binned values), this loss operates directly in the continuous action space, penalizing predictions proportionally to their absolute deviation from the correct value.

Why L1 loss rather than L2 (MSE): L1 loss penalizes errors linearly, while L2 (squared error) penalizes large errors quadratically. In robot control, occasional moderate deviations are often tolerable (the robot may be slightly less efficient but still succeeds), while rare large deviations are catastrophic (the robot collides with something or drops an object). L2 loss would be dominated by these rare large errors and would cause the model to sacrifice overall accuracy to avoid extreme outliers. L1 loss provides a more balanced optimization that focuses on reducing typical errors across all dimensions. The paper does not ablate this choice, but the L1 vs. L2 selection is standard practice in behavior cloning for robotics for exactly these robustness reasons.

Integration with the discrete training pipeline. A critical design decision is that the paper retains the discrete action generation during training even though the continuous Action Transformer is used for deployment. The overall loss function combines three terms:

$\mathcal{L} = \mathcal{L}_{dis\_action} + \mathcal{L}_{img} + \alpha \mathcal{L}_{conti\_action}$

where $\alpha = 10$ weights the continuous action loss relative to the discrete and image losses. The discrete action loss $\mathcal{L}_{dis\_action}$ is the cross-entropy on discrete action tokens (Section 3.4.2), the image loss $\mathcal{L}_{img}$ is the cross-entropy on image tokens (Section 3.4.3), and the continuous action loss $\mathcal{L}_{conti\_action}$ is the L1 regression described above.

What this combined loss computes: a multi-task objective where the model must simultaneously (a) predict the correct discrete action tokens, (b) generate the next image frame (world model), and (c) produce continuous actions that match expert demonstrations. All three objectives are optimized jointly, with gradients flowing through the shared LLM backbone from all three sources.

Why retain discrete actions if only the continuous head is deployed? The paper provides evidence for this design choice in Figure 8, which shows that models trained with discrete action tokens achieve substantially higher success rates than those trained without them, with "the advantage being most pronounced during the initial stages of training." The discrete action loss acts as an auxiliary training signal that accelerates convergence of the continuous Action Transformer. The paper hypothesizes that the discrete action prediction task forces the LLM backbone to develop action-relevant representations early in training, providing a better initialization for the continuous head. Without this signal, the continuous head must learn both the perceptual-to-action mapping and the action space structure from scratch, which is slower and leads to worse final performance.

A concrete mechanism: the discrete action tokens are embedded in the same 65,536-token vocabulary as image and text tokens. When the model learns to predict these tokens, it learns to associate specific action embeddings with specific visual patterns (e.g., the token for "close gripper" with images showing a closed gripper). These learned associations in the shared embedding space directly benefit the Action Transformer, which reads from the same hidden state representations that the discrete action loss helped shape.

Why the continuous head solves real-world deployment failures. The paper reports in Table 5 that the discrete action model — which achieves 93.3% on LIBERO simulation — scores 0% on all real-world tasks. The continuous model achieves 80-90%. The paper attributes this to two factors:

First, generalization from limited data: the compact Action Transformer has far fewer parameters than the full autoregressive decoding pathway used for discrete actions, making it less prone to overfitting on the 248-249 demonstration trajectories used for real-world training. The discrete model's large parameter count causes it to memorize training trajectories rather than learning generalizable manipulation skills, which fails under the visual distribution shift between training and deployment (different lighting, object positions, background clutter).

Second, trajectory smoothness: the Action Transformer's bidirectional attention and whole-chunk parallel generation produce actions that form a coherent trajectory, unlike the discrete model's isolated action predictions (enforced by the attention mask) that produce jerky, discontinuous motions. In real-world physics, a jerky trajectory often means the robot loses its grasp, collides with objects, or triggers safety stops — all of which count as task failures. In simulation, these physical consequences are either absent or less severe, explaining why the discrete model's trajectory discontinuity is tolerable in simulation but fatal on real hardware.

---

3.4.6 Training Configuration and Hyperparameters

The paper provides several specific hyperparameter choices that govern how the model is trained. These are collected here for completeness.

Model input configuration:

• Historical image frames: $M = 2$ (the current frame and one previous frame), providing temporal context while keeping sequence length manageable
• Action chunk sizes: $K = 10$ for LIBERO-Long and LIBERO-Spatial (longer tasks benefit from executing more actions between re-planning), $K = 5$ for LIBERO-Object and LIBERO-Goal (shorter tasks where frequent re-planning is helpful)
• World model prediction rounds: $N = 1$ during training (predict one frame ahead) to maintain computational efficiency; multi-step rollouts would require $N > 1$
• Loss weighting: $\alpha = 10$ for the continuous action loss relative to the discrete and image losses, indicating that the continuous action signal is given higher priority in the joint optimization

Tokenizer configuration:

• Image tokenizer: VQ-GAN with 16:1 compression ratio, 8,192 codebook size, producing 256 tokens per 256×256 image and 1,024 tokens per 512×512 image
• State and action tokenizers: 256 uniform bins per dimension, with bin boundaries determined by training data range
• Unified vocabulary size: 65,536 tokens total, encompassing image, text, state, and action tokens

Training process:

• The model is initialized from Chameleon's pretrained weights. This is significant because Chameleon's pretraining on large-scale image and text data provides strong visual and linguistic representations that transfer to the robot manipulation domain, even without robot-specific pretraining.
• Training data is a mixture of VLA data (action prediction sequences) and world model data (image prediction sequences), mixed at an unspecified ratio — the paper does not report the proportion of each data type in each training batch.
• For simulation experiments, the LIBERO dataset is filtered to remove unsuccessful trajectories and "no-operation" actions (where the robot is stationary), following the same preprocessing used by OpenVLA (Kim et al., 2024). For real-world experiments, the dataset consists of 248 expert demonstrations for the "Place the block inside the circle" task and 249 demonstrations for the "Place strawberries in the cup" task, collected via human teleoperation on the LeRobot SO100 arm.
• The paper does not report optimizer choice, learning rate, batch size, or training duration for the main model. This is a notable omission — these hyperparameters significantly affect training dynamics and reproducibility. The only training hyperparameter explicitly stated is $\alpha = 10$ for the continuous action loss weight.

---

3.4.7 Inference: How the Model Is Used at Deployment Time

At inference time, the model operates in one of two modes depending on the query:

VLA mode (action generation): The robot's current camera images (front and wrist), proprioceptive state, and task instruction are tokenized and fed into the LLM backbone. The backbone processes these through its transformer layers. The Action Transformer head then reads the output hidden states and generates a continuous action chunk of $K$ actions using its learnable action queries and bidirectional attention. The first action in the chunk is executed on the robot; the remaining actions may be discarded (open-loop execution of the first action only, with re-planning at the next timestep) or executed sequentially (closed-loop execution of the full chunk, with re-planning after all $K$ actions are completed). The paper does not specify which execution strategy is used for the reported results, though re-planning every timestep is standard practice for VLA models with action chunking to allow the policy to adapt to unexpected outcomes.

World model mode (image generation): The model receives current images and an action (or sequence of actions) as input. The LLM backbone processes these tokens. The model then autoregressively generates image tokens using the standard causal attention mask — each new image token is conditioned on all previous tokens in the sequence (text, state, past images, actions, and previously generated image tokens). The generated image tokens are decoded through the VQ-GAN decoder to produce the predicted future frame. For multi-step prediction, this process could be repeated: the generated frame becomes input for the next prediction step, along with the next action. However, the paper only evaluates single-step prediction ($N = 1$) and does not report multi-step rollout quality.

Inference speed (from Table 7): The continuous Action Transformer provides substantially faster inference than discrete autoregressive action generation. For chunk size 5 with a single input view (no wrist camera, no history), continuous generation achieves 24.94 Hz versus 3.69 Hz for discrete with action chunking — a ~6.8× speedup. Adding the wrist camera and one historical frame reduces continuous speed to 7.75 Hz and discrete to 2.74 Hz. The key insight is that continuous generation speed scales almost linearly with chunk size because all actions are generated in parallel — generating 10 actions takes nearly the same time as generating 5 actions — while discrete generation speed is roughly constant regardless of chunk size because the time is dominated by the autoregressive generation of individual tokens, not the number of actions. This parallel generation advantage makes the continuous head suitable for real-time robot control where decisions must be made at 10–30 Hz to enable smooth motion.

4. Key Insights and Innovations

Innovation 1: Mutual Enhancement as a First-Class Architectural Principle, Not a Side Effect

The paper's most fundamental conceptual contribution is the demonstration that VLA and world model training objectives, when applied jointly within a shared architecture, produce a system where each component is better at its individual task than if trained in isolation — and this mutual enhancement is not incidental but systematic, measurable, and practically significant. Prior work treated VLA and world model capabilities as separate systems connected through modular pipelines (UniPi, DREAMGEN, GeVRM) or as independent models where one's outputs feed into the other (model-based RL approaches). The dominant assumption was that these are distinct capabilities best served by specialized architectures optimized for their respective objectives. This paper challenges that assumption at the architectural level, arguing that the representational demands of action prediction and physics prediction are sufficiently aligned that sharing parameters — far from causing destructive interference — actively improves both.

What makes this more than a "multi-task learning helps" finding is the specific mechanism of mutual enhancement the paper identifies and empirically substantiates. The world model's image prediction objective requires the model to capture how objects move in response to actions — the exact causal understanding that standard VLAs lack. The VLA's action prediction objective requires the model to develop rich visual understanding of objects, grasp points, and spatial relationships — the exact perceptual capabilities that improve video prediction fidelity. This is not generic multi-task regularization (where an auxiliary task prevents overfitting by adding noise); it is complementary representation learning where each objective forces the model to develop capabilities that the other objective needs but does not explicitly train for.

The evidence for this claim is concrete and differentiated from simple multi-task baselines. Figure 5 provides qualitative evidence that the world model objective changes how the VLA behaves: the jointly trained model retries grasps on failure, while the VLA-only model proceeds to the target location without grasping — suggesting the world model objective has taught the model that object interaction requires persistent engagement, not just trajectory planning. Figure 7 provides the complementary evidence for the world model: the jointly trained Action World Model consistently predicts successful grasps from both camera viewpoints, while the world-model-only baseline produces inconsistent predictions across views (front camera shows failure, wrist camera shows success) — a physically impossible prediction that reveals the absence of cross-view consistency that the VLA objective's visual understanding provides. Table 6 quantifies this: across all four LIBERO suites, the Action World Model achieves better FVD, PSNR, SSIM, and LPIPS than the standalone world model, with particularly large gains on the Object suite (FVD drops from 1141.6 to 877.2, LPIPS improves from 27.30 to 22.60).

The significance of this finding extends beyond the specific architecture. It suggests a design principle for future embodied AI systems: rather than building separate perception, action, and dynamics modules and connecting them through interfaces, train them jointly with objectives that force each module to develop representations useful to the others. This is a reframing of the architecture design problem from "how do we connect these components?" to "what training objectives, when combined, produce the representations we need?" The paper does not fully develop this principle — it demonstrates it for one specific pairing (VLA + world model) in one domain (tabletop manipulation) — but the clarity of the mutual enhancement evidence provides a template for investigating similar synergies in other capability pairings (e.g., language grounding + physical reasoning, navigation + manipulation).

Innovation 2: Diagnostic Decomposition of the Discrete Action Bottleneck and Targeted Architectural Response

The paper's second distinctive contribution is methodological rather than architectural: it performs a failure-mode-driven architecture evolution that identifies two distinct causes for why discrete action generation fails on real robots despite strong simulation performance, and designs targeted solutions for each cause. This diagnostic approach — deliberately separating error propagation from overfitting as distinct failure mechanisms, showing evidence for each, and addressing them with independently motivated architectural changes — represents a more rigorous engineering methodology than the typical "simulation works, real world doesn't, let's try something else" pattern common in robotics papers.

The decomposition works as follows. In simulation, the discrete model with the action attention mask achieves 93.3% on LIBERO (Table 1), comparable to strong baselines. On real robots, the same model scores 0% (Table 5, line 1). The paper identifies two independent failure mechanisms:

Failure mechanism 1: Error propagation under causal attention. Without the action attention mask, each action token in an autoregressive chunk conditions on prior (potentially erroneous) action tokens, causing cascading errors. The paper provides direct evidence in Figure 6: using vanilla causal attention, success rate drops with chunk length. The proposed solution — the action attention mask (Figure 3b) — is a minimal architectural change (modifying the attention pattern, not adding parameters) that addresses this specific mechanism by isolating each action's generation from prior actions. Table 3 quantifies the gain: adding the mask improves average performance from 54.0% to 76.6% for discrete actions.

Failure mechanism 2: Overfitting and trajectory discontinuity on small real-world datasets. Even with the attention mask, the discrete model fails on real robots because (a) the large autoregressive backbone overfits the limited demonstration data, and (b) the mask's action isolation produces non-smooth trajectories that are physically unexecutable. The evidence: Table 5 shows 0% success for discrete actions on all real-world tasks. The proposed solution — the continuous Action Transformer head — is a qualitatively different architectural addition (a compact parallel-decoding network with bidirectional attention) that addresses both sub-problems: fewer parameters prevent overfitting, and bidirectional attention within the chunk ensures trajectory coherence.

The paper further demonstrates that retaining the discrete action loss during training accelerates convergence of the continuous head (Figure 8), meaning the discrete component serves as a useful auxiliary training signal even when not deployed. This finding — that a discrete token prediction task can accelerate continuous policy learning — is a practical insight that bridges the discrete-vs-continuous action representation debate by showing they can coexist productively during training.

What makes this approach intellectually distinctive is its separation of training-time and deployment-time architectures. The discrete action generation pathway exists primarily to shape the shared LLM backbone's representations during training — forcing the model to learn action-relevant embeddings that benefit the continuous head — while the continuous head handles actual deployment. This is a departure from the standard assumption that training and inference architectures should match, and it suggests a more flexible design pattern: use different architectural components optimized for training efficiency versus deployment robustness, sharing a common representational backbone.

Comparing to prior work: RT-2 (Zitkovich et al., 2023) and OpenVLA (Kim et al., 2024) use discrete actions throughout, accepting the precision loss and overfitting risk as inherent to the VLM-based approach. π₀ (Black et al., 2024) uses continuous flow matching for actions, abandoning discrete tokens entirely and losing the representational benefits of a shared vocabulary. This paper demonstrates that a hybrid approach — discrete for training signal, continuous for deployment — captures benefits of both. This is an incremental advance (both discrete and continuous action VLAs existed before) but a fundamental reframing of how to combine them.

Innovation 3: World Model Pretraining as an Alternative to Robot-Specific Pretraining Data

The paper makes an empirical claim with significant practical implications: world model pretraining on the same robot dataset can substitute for external large-scale robot pretraining data. In Table 1, RynnVLA-002 without any pretraining achieves 97.4% on LIBERO, matching or exceeding models pretrained on LIBERO-90 or massive real-robot datasets (OpenVLA-OFT: 97.1%, UniVLA: 95.2%, both with pretraining). In Table 2, it achieves competitive real-world performance against GR00T N1.5 and π₀ (both pretrained on large robot datasets) while using only 248–249 in-domain demonstrations.

The significance of this finding is not just the performance number — it is the implication that the world model objective provides a form of self-supervision that is as effective as external pretraining data for learning generalizable manipulation skills. Standard VLA models rely on large-scale pretraining to acquire robust visual representations and task-agnostic priors. RynnVLA-002 suggests that the physics prediction task — learning to forecast how images change in response to actions — forces the model to develop similarly robust representations from limited in-domain data alone. This is plausible mechanistically: to predict the next frame accurately, the model must learn object identities, spatial relationships, contact dynamics, and camera-relative motion patterns — all capabilities that generalize across manipulation tasks regardless of the specific objects or goals.

Table 8 provides further evidence: explicit world model pretraining (training on the world model objective alone before adding the VLA objective) provides additional gains over joint training from scratch (73.1% vs. 67.3% on Goal, 30.2% vs. 23.0% on Long). This suggests a staged training curriculum — world model first to learn physics-informed representations, then VLA to learn action planning — may be optimal, though the paper does not develop this direction.

This finding is a fundamental shift in how to think about pretraining for robot learning. Current practice (OpenVLA, Octo, π₀) treats large-scale external robot datasets as the primary source of generalizable priors. This paper demonstrates that the world model objective — which requires no additional data beyond what is collected for policy training — can provide comparable benefits. For practitioners with limited access to large pretraining datasets (e.g., labs working with custom robot hardware), this suggests that investing compute in world model training from small in-domain datasets may be more practical than attempting to collect or access large-scale multi-robot data.

The caveat is that this experiment is limited to LIBERO (a specific simulation benchmark) and two real-world tasks on a single robot platform. Whether world model pretraining scales to more diverse manipulation domains, navigation tasks, or mobile manipulation remains an open question. The claim is that world model pretraining can substitute for external pretraining, not that it always does.

Innovation 4: World Model as an Implicit Object-Attention Mechanism

The paper provides evidence for a specific and non-obvious mechanism through which the world model objective improves VLA performance: the image prediction task forces the model to attend to manipulated objects and their physical interactions, implicitly teaching the VLA component what objects matter for task success without any explicit object detection or segmentation supervision. This is a form of learning what to attend to — not through attention weights or saliency maps, but through the pressure of a prediction objective that penalizes failures to track object motion.

The evidence for this mechanism is primarily qualitative but compelling. Figure 5 compares VLA behavior with and without world model training on the "put the cream cheese in the bowl" task. The VLA-only model moves directly toward the target location without successfully grasping the object — it executes the spatial trajectory of the task but fails at the object interaction step. The jointly trained model keeps retrying the grasp when it encounters failure — it persists at the object interaction because it has "learned" (through the world model objective) that grasping is the critical event that determines task success.

The paper's interpretation, stated in Section 4.3, is that "the world model's training objective requires accurate prediction of object motion, thereby reinforcing attention to object interaction dynamics." This is a specific causal claim: the world model loss penalizes image prediction errors, and these errors are largest when objects move unexpectedly (e.g., the gripper approaches the cheese but doesn't grasp it, so the cheese doesn't move in the next frame as predicted). To minimize this loss, the model must learn to predict whether grasps will succeed, which requires attending to the spatial relationship between gripper and object — exactly the information needed for successful manipulation.

This is a new diagnostic concept for why world models benefit policy learning. Prior work on model-based RL and video prediction for planning (UniPi, DREAMGEN) treats the world model as an explicit planning module — generate future frames, then use them to guide action selection. RynnVLA-002 shows that the benefit may be more fundamental: the world model objective serves as an implicit attention mechanism that teaches the shared backbone to focus on task-relevant objects and interactions, even when the world model's predictions are not explicitly used at inference time. The VLA head inherits these attention patterns through the shared representations, improving its action predictions even though it never directly sees world model predictions.

This insight suggests a broader design principle: auxiliary prediction objectives that require tracking specific environmental properties (object motion, contact events, spatial relationships) can serve as implicit supervision for attention, teaching the model what to care about without explicit labels. This connects to the broader literature on self-supervised representation learning but is distinct in identifying a specific mechanism (object interaction prediction as attention guidance) rather than a generic regularization effect.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on two distinct data regimes: the LIBERO simulation benchmark (Liu et al., 2023), consisting of four suites (LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, LIBERO-Long) spanning spatial relationships, object recognition, goal variation, and long-horizon tasks; and a custom real-world manipulation dataset collected on a LeRobot SO100 robotic arm (Cadene et al., 2024) with two pick-and-place tasks (248 demonstrations for "Place the block inside the circle" and 249 demonstrations for "Place strawberries in the cup"). For LIBERO, unsuccessful trajectories and no-operation actions are filtered following OpenVLA's preprocessing (Kim et al., 2024), and world model evaluation uses a 90%/10% train/validation split.

• Base model(s). The architecture is initialized from Chameleon (Team, 2024), a unified multimodal model pretrained on large-scale image and text data for both understanding and generation. The paper does not report the parameter count of the Chameleon backbone used, which is a notable omission for understanding model scale relative to baselines. The choice of Chameleon is motivated by its native support for both image understanding and image generation within a single autoregressive framework — a capability that standard MLLMs like LLaVA or Qwen-VL lack because they are designed for image-to-text only. For real-world comparisons, baselines include GR00T N1.5 (Bjorck et al., 2025) and π₀ (Black et al., 2024), both initialized from their official pretrained checkpoints and fine-tuned on the same SO100 dataset.

• Metrics. For VLA evaluation, the primary metric is task success rate measured across 50 deployment rollouts per LIBERO task (each initialized from a different state), with a trial counting as successful if the robot completes the specified goal within a predefined time budget. For real-world evaluation, success rate is measured across 10 trials per scenario, with explicit failure criteria: time limit exceeded, more than five consecutive failed grasp attempts on a target, or (in distractor scenarios) attempting to manipulate any distractor object. For world model evaluation, four standard video prediction metrics are used on the held-out validation set: Fréchet Video Distance (FVD, lower is better), Peak Signal-to-Noise Ratio (PSNR, higher is better), Structural Similarity Index (SSIM, higher is better), and Learned Perceptual Image Patch Similarity (LPIPS, lower is better). For inference speed, frequency is reported in Hz (actions per second).

• Baselines. The paper compares against an extensive set of prior work on the LIBERO benchmark, spanning both discrete-action and continuous-action methods. Discrete-action baselines include: LAPA (Ye et al., 2024), TraceVLA (Zheng et al., 2024), OpenVLA (Kim et al., 2024), SpatialVLA (Qu et al., 2025), NORA (Hung et al., 2025), CoT-VLA (Zhao et al., 2025), π₀-FAST (Black et al., 2024), MolmoAct (Lee et al., 2025), FlowVLA (Zhong et al., 2025), and UniVLA (Bu et al., 2025). Continuous-action baselines include: Diffusion Policy (Chi et al., 2023), Octo (Team et al., 2024), MDT (Reuss et al., 2024), DiT Policy (Hou et al., 2024), MaIL (Jia et al., 2024), ThinkAct (Huang et al., 2025), π₀ (Black et al., 2024), SmolVLA (Shukor et al., 2025), OpenVLA-OFT (Kim et al., 2025), Seer (Tian et al., 2024), and UVA (Li et al., 2025). For real-world experiments, baselines are GR00T N1.5 (Bjorck et al., 2025) and π₀ (Black et al., 2024).

• Generation budget / compute accounting. The paper does not define a unified compute budget for fair comparison across methods, which is a significant methodological gap. For its own model, the key operational parameters are action chunk size $K$ (5 or 10 depending on task), historical frames $M$ (2 for most experiments), and world model prediction rounds $N$ (1 during training). The loss weighting $\alpha = 10$ sets the relative importance of the continuous action L1 loss versus the discrete and image cross-entropy losses. For the FLOPs-matched comparison, the paper does not account for the differing computational costs of pretrained versus non-pretrained models — a pretrained model like OpenVLA-OFT benefits from compute spent during pretraining on large-scale data, while RynnVLA-002's training is limited to the LIBERO or SO100 datasets, making direct performance comparisons without compute accounting potentially misleading.

• Cross-validation / statistical protocol. The paper does not report any cross-validation, statistical significance testing, or confidence intervals for its results. Success rates are reported as point estimates from a fixed number of rollouts (50 per LIBERO task, 10 per real-world scenario). Given the small number of real-world trials (10 per condition, with three scenarios per task), the reported success rates could have substantial variance — a difference of 10% (one trial out of 10) is within the noise floor, yet the paper draws comparative conclusions at this granularity (e.g., "80% vs. 50% on block placement with distractors"). For LIBERO results, the paper averages across tasks within each suite, but does not report per-task variance or confidence bounds on the suite-level average. The ablation studies in Tables 3, 4, and 5 do not report error bars or statistical tests, making it impossible to assess whether the reported differences (e.g., 62.8% vs. 67.2% in Table 3, lines 1 vs. 2) are statistically reliable or within sampling noise.

Main Quantitative Results

LIBERO Simulation Benchmark Results

The headline result from Table 1 is that RynnVLA-002 achieves 97.4% average success rate across all four LIBERO suites with continuous actions, and 93.3% with discrete actions, both without any pretraining on external robot datasets. These numbers place RynnVLA-002 among the strongest reported results on LIBERO, with the continuous variant slightly exceeding the previous state-of-the-art (OpenVLA-OFT at 97.1%, which uses pretraining on the Open X-Embodiment dataset) and substantially outperforming widely-used baselines like π₀ (86.0%) and Octo (75.1%).

Breaking down by suite, the continuous model achieves: 99.0% on LIBERO-Spatial, 99.8% on LIBERO-Object, 96.4% on LIBERO-Goal, and 94.4% on LIBERO-Long (Table 1, "RynnVLA-002-Continuous"). These near-ceiling scores on Spatial and Object (where performance is essentially saturated) leave the harder Goal and Long suites as the primary discriminators. On LIBERO-Long — the most challenging suite with 10 complex long-horizon tasks — the continuous RynnVLA-002 achieves 94.4%, compared to 92.0% for UniVLA (the best discrete pretrained baseline), 94.5% for OpenVLA-OFT (the previous continuous best), and 73.0% for π₀. On LIBERO-Goal, RynnVLA-002's 96.4% trails OpenVLA-OFT's 97.9% but substantially exceeds π₀'s 95.0% and UniVLA's 95.6%.

The discrete-action variant (93.3% average) underperforms the continuous variant across all four suites, with the gap being largest on LIBERO-Long (87.6% vs. 94.4%) and LIBERO-Spatial (94.2% vs. 99.0%). This is consistent with the paper's claim that the discrete model's error propagation and precision loss become more severe on longer-horizon tasks where action chunk quality matters more.

A notable finding is that RynnVLA-002 without pretraining is competitive with or exceeds models that use large-scale pretraining. UniVLA (95.2%) and OpenVLA-OFT (97.1%) are pretrained on massive robot datasets; OpenVLA (76.5%) and Octo (75.1%) also use external pretraining. The paper explicitly flags this: "Surprisingly, our RynnVLA-002, without any pretraining, is still on par with strong baseline models pretrained on either LIBERO-90 or massive real-robot datasets" (Section 4.1). However, this comparison is not FLOPs-matched — the pretrained baselines invested substantial compute in their pretraining phase that RynnVLA-002 avoids, making the comparison favorable to RynnVLA-002 for a given amount of downstream training compute, but not for total compute including pretraining.

Real-World Robot Results

Table 2 presents results on two real-world manipulation tasks using the LeRobot SO100 arm, comparing RynnVLA-002 against GR00T N1.5 and π₀ (both fine-tuned from official pretrained checkpoints on the same 248-249 demonstration datasets). The results reveal a striking pattern: RynnVLA-002 performs comparably or better than pretrained baselines, particularly in cluttered or challenging scenarios, despite using no pretraining.

For the "Place the block inside the circle" task:

• Single-target: RynnVLA-002 achieves 90.0%, versus 90.0% for GR00T N1.5 and 100.0% for π₀ — competitive but slightly behind π₀ on this simplest scenario.
• Multi-target: RynnVLA-002 achieves 90.0%, substantially outperforming GR00T N1.5 (60.0%) and π₀ (70.0%) — a 20-30 percentage point advantage.
• With distractors: RynnVLA-002 achieves 80.0%, versus 50.0% for both baselines — a 30 percentage point advantage.

For the "Place the strawberries into the cup" task:

• Single-target: RynnVLA-002 achieves 80.0%, versus 50.0% for GR00T N1.5 and 80.0% for π₀ — tied with π₀ and substantially ahead of GR00T.
• Multi-target: RynnVLA-002 achieves 80.0%, versus 50.0% for GR00T and 70.0% for π₀ — a 10-30 point advantage.
• With distractors: RynnVLA-002 achieves 50.0%, versus 70.0% for GR00T and 40.0% for π₀ — this is the only scenario where RynnVLA-002 is not best, trailing GR00T by 20 points.

The paper's interpretation emphasizes the cluttered-environment advantage: "RynnVLA-002 performs better than the baselines in cluttered environments. For instance, RynnVLA-002 has over 80% success rate on both multi-target tasks and distractor-filled scenarios for the 'Place the block' task, surpassing the baselines by 10% to 30%" (Section 4.2). This pattern is consistent with the hypothesis that the world model's physics-informed representations help the model reason about object interactions in clutter — but the evidence is correlational, not mechanistic. The paper does not provide ablation results isolating the world model's contribution specifically in cluttered versus uncluttered real-world scenarios to directly test this claim.

The 50% boost claim requires careful reading. The paper's abstract states: "in real-world LeRobot experiments, its integrated world model boosts the overall success rate by 50%." In Section 4.3, the evidence is provided in Table 5: the model trained without world model data (line 4) achieves 30.0%, 10.0%, and 0% on single-target, multi-target, and distractors respectively, while the full model (line 5) achieves 80.0%, 80.0%, and 50.0%. The 50% boost is apparently computed from the multi-target comparison (10% → 80% is an absolute increase of 70 percentage points, or a 700% relative increase) or the distractor comparison (0% → 50% is a 50 percentage point absolute increase). The paper's phrasing "boosts the overall success rate by 50%" is ambiguous — it could mean an absolute increase of 50 percentage points (supported by the distractor scenario) or a relative increase (not 50% by any calculation). The evidence clearly shows that world model inclusion is critical for real-world performance, but the specific 50% figure should be understood as a rough characterization of the gap rather than a precisely defined metric.

World Model Evaluation

Table 6 evaluates the world model's image prediction quality on the LIBERO validation set, comparing a standalone world model (trained only on world model data) against the Action World Model (jointly trained with VLA data). Across all four suites and all four metrics, the Action World Model matches or exceeds the standalone world model, supporting the claim that VLA training improves world model generation quality.

On the LIBERO-Object suite — which involves recognizing and manipulating diverse objects — the joint training provides the largest improvements: FVD drops from 1141.6 to 877.2 (a 23.2% reduction), PSNR rises from 20.31 to 22.18 (a 9.2% improvement), SSIM improves from 59.59 to 65.03, and LPIPS drops from 27.30 to 22.60 (a 17.2% improvement). The LIBERO-Spatial suite shows similar but more modest gains: FVD from 405.4 to 373.1, PSNR from 22.32 to 23.88, SSIM from 79.15 to 82.41, LPIPS from 20.28 to 16.33. The LIBERO-Long suite shows FVD improvement from 557.73 to 427.86 (23.3% reduction) but mixed results on other metrics. On LIBERO-Goal, improvements are minimal: FVD from 370.0 to 336.8, with PSNR and SSIM nearly unchanged (22.25 vs. 22.13, 77.84 vs. 78.13).

The qualitative evidence in Figure 7 complements these numbers: the Action World Model consistently generates video predictions showing successful grasps from both front and wrist camera viewpoints, while the standalone world model shows physically inconsistent predictions (e.g., front camera shows failed grasp while wrist camera shows successful grasp — a temporal inconsistency that violates the shared physics of the scene). The paper interprets this as evidence that "the image understanding capabilities inherited from the VLA model strengthen the world model's generation performance" (Section 4.3), specifically by enforcing cross-view consistency that the world model alone does not learn.

However, the absolute quality of the generated images is not directly assessed — the paper does not report whether the predicted images are sufficiently realistic for downstream use (e.g., whether they could be used as synthetic training data for the VLA, or for planning via imagined rollouts). The FVD scores in the 300-1100 range indicate substantial room for improvement in video quality.

Efficiency Analysis

Table 7 provides inference speed measurements for different model configurations. The key findings are:

• Continuous action generation is substantially faster than discrete. For chunk size 5 with a single input view (no wrist camera, no historical frames), continuous generation achieves 24.94 Hz versus 3.69 Hz for discrete with action chunking — a 6.8× speedup (Table 7, lines 1 vs. 7). This advantage stems from parallel decoding in the Action Transformer versus sequential autoregressive generation for discrete tokens.

• Adding inputs reduces speed significantly. Including one historical frame and the wrist camera drops continuous speed from 24.94 Hz to 7.75 Hz (Table 7, line 9) — a 3.2× slowdown — and discrete speed from 3.69 Hz to 2.74 Hz (line 6) — a 1.3× slowdown. The relative impact is larger for continuous because the Action Transformer's parallel generation is dominated by encoding time for additional input tokens, while discrete generation is already bottlenecked by sequential token decoding.

• Continuous generation speed scales almost linearly with chunk size. Comparing chunk size 5 versus 10 for continuous: 24.94 Hz vs. 48.20 Hz for single-view (lines 7 vs. corresponding column) and 7.75 Hz vs. 15.78 Hz for dual-view with history (line 9). The near-doubling of frequency when doubling chunk size (since the model outputs more actions per inference, even though each inference takes similar time) confirms that the Action Transformer's per-action generation cost is negligible — generating 10 actions costs nearly the same as generating 5.

• Action chunking improves efficiency for discrete actions. Comparing line 1 (no chunking, 2.50 Hz, 60.0% on Goal) versus line 2 (chunk size 5, 3.69 Hz, 83.2% on Goal): chunking provides both higher throughput (more actions per inference step) and better performance (the chunk gives the robot a short plan to execute). However, performance degrades for overly long chunks when using discrete actions without the attention mask (Figure 6), indicating a tradeoff between chunk length and error accumulation.

Ablation Studies and Robustness Checks

World model inclusion improves VLA performance (Table 3, discrete actions): Adding world model data to VLA training improves the discrete model's average success rate from 62.8% to 67.2% (lines 1 vs. 2). After incorporating action chunking and the attention mask, adding world model data further improves from 76.6% to 78.1% (lines 4 vs. 5). Notably, when action chunking is used without the attention mask (line 3), performance drops to 54.0% — worse than no chunking (62.8%) — confirming that naive autoregressive action generation is harmful and the attention mask is essential.

World model inclusion improves VLA performance (Table 4, continuous actions): Adding world model data improves average success from 91.6% (line 2: VLA continuous with wrist camera) to 94.6% (line 3: same with world model). The gain is concentrated on LIBERO-Long (81.4% → 85.8%) and LIBERO-Goal (91.4% → 96.0%), with smaller improvements on LIBERO-Object (95.4% → 97.4%) and LIBERO-Spatial (98.2% → 99.0%) —consistent with the world model being most beneficial for tasks requiring multi-step reasoning about object dynamics.

World model inclusion is critical for real-world deployment (Table 5): The model trained without world model data (line 4: continuous actions, wrist camera, proprioceptive state, no world model) achieves only 30.0%, 10.0%, and 0% on the three real-world scenarios. Adding world model data (line 5) raises these to 80.0%, 80.0%, and 50.0%. This is the largest single ablation effect in the paper and the primary evidence for the "mutual enhancement" claim in real-world settings. However, the baseline without world model still has all other components (continuous actions, wrist camera, proprioceptive state), so this is a clean ablation of the world model contribution.

Wrist camera and proprioceptive state are essential for real robots but not simulation (Tables 4 and 5): In simulation (Table 4, line 1), removing the wrist camera and proprioceptive state reduces average performance from 97.4% to 84.5% — a substantial but not catastrophic drop. On real robots (Table 5), removing the wrist camera (line 3: continuous, world model, proprioceptive state, no wrist camera) causes complete failure (0% across all scenarios). Removing proprioceptive state (line 2: continuous, world model, wrist camera, no proprioceptive state) similarly causes 0% success. The paper explains this discrepancy: the wrist camera provides "crucial visual feedback on the relative pose between the gripper and the object, especially when the robot is outside the field of view of the front camera," and proprioceptive state "is essential for accurately timing gripper closure and object lifting during manipulation" (Section 4.3). In simulation, these failure modes may be less severe because the simulation provides idealized visual observations and physics.

Discrete action tokens accelerate continuous action convergence (Figure 8): This is the paper's most methodologically interesting ablation. Training the continuous Action Transformer with the discrete action loss active (the hybrid approach) versus without it shows that the hybrid model achieves "substantially higher success rate" with "the advantage being most pronounced during the initial stages of training." This is not a final-performance difference (both eventually converge to similar levels in simulation, as shown in Figure 9) but a training efficiency difference — the discrete action tokens provide a useful auxiliary training signal that shapes the shared LLM backbone's representations early in training.

Action attention mask is essential for discrete action chunking (Figure 6 and Table 3): Figure 6 shows that with the default causal attention mask (labeled "Vanilla"), success rate degrades as chunk length increases, while with the proposed mask, performance is maintained or improved. Table 3 quantifies this: adding the attention mask with action chunking raises average performance from 54.0% (line 3, chunking without mask) to 76.6% (line 4, chunking with mask). The mask's benefit is largest on LIBERO-Long (16.9% → 49.3%) and LIBERO-Spatial (36.7% → 81.8%), where longer action sequences make error propagation more severe.

Discrete actions fail entirely on real robots (Table 5, line 1): Despite achieving 93.3% on LIBERO, the discrete-action model achieves 0% success on all real-world scenarios. The paper attributes this to overfitting (the large autoregressive backbone memorizes training trajectories rather than learning generalizable skills) and trajectory discontinuity (the attention mask's action isolation produces jerky, uncoordinated motions). This is a crucial negative result that motivates the continuous Action Transformer.

World model pretraining provides additional gains (Table 8): Pretraining the model on the world model objective before VLA training — using the same LIBERO data, not external pretraining data — improves performance compared to training from scratch with joint VLA+world model objective. On LIBERO-Goal, pretraining improves from 67.3% to 73.1%; on LIBERO-Long, from 23.0% to 30.2%. These are meaningful gains (5.8 and 7.2 percentage points respectively) that suggest a staged curriculum — world model first, then VLA — may be optimal, though the paper does not develop this into a full training recipe or compare against simply training longer with the joint objective.

Critical Assessment

The paper makes four central claims that require separate scrutiny against the experimental evidence.

Claim 1: "RynnVLA-002 surpasses individual VLA and world models, demonstrating their mutual enhancement." This claim is supported for the VLA direction: adding the world model objective improves VLA performance in both simulation (Table 3: 62.8% → 67.2%; Table 4: 91.6% → 94.6%) and — dramatically — on real robots (Table 5: near-zero → 80.0%+). The evidence for the world model direction is weaker but present: the Action World Model achieves better video prediction metrics than the standalone world model on the Object and Spatial suites (Table 6). However, the improvement is modest on Goal and Long suites (FVD drops by only 33.2 and 130 respectively on Goal and Long, with PSNR/SSIM nearly unchanged), and the paper does not evaluate whether the improved video predictions translate to better downstream task performance when used for planning or data generation. The "mutual" aspect of the enhancement is therefore substantiated asymmetrically — the world model helps the VLA substantially, the VLA helps the world model detectably but modestly.

Claim 2: "RynnVLA-002 achieves 97.4% success rate on the LIBERO simulation benchmark without pretraining." This number is accurate (Table 1) and genuinely impressive — it matches or exceeds models pretrained on large-scale robot datasets. However, several contextual factors matter. First, LIBERO is a specific simulation benchmark and ceiling effects are becoming apparent (multiple models now exceed 95%). The harder suites (Goal at 96.4%, Long at 94.4%) show remaining headroom but not much — making it difficult to assess whether the joint training provides genuine representational advantages or is simply yet another strong recipe that saturates the benchmark. Second, the "without pretraining" framing is technically accurate (no robot-specific pretraining) but the model is initialized from Chameleon, which was pretrained on massive image-text data. The visual representations Chameleon provides are a form of pretraining — just not robot-specific pretraining. A fairer characterization would be "without robot-specific pretraining data," since Chameleon's image understanding capabilities are doing substantial lifting. Third, the comparison against pretrained baselines is not FLOPs-matched, making it difficult to assess whether the joint training objective is genuinely more efficient or simply benefits from Chameleon's strong initialization.

Claim 3: "In real-world LeRobot experiments, its integrated world model boosts the overall success rate by 50%." The evidence (Table 5, lines 4 vs. 5) clearly shows a large improvement from world model inclusion — from 30%/10%/0% to 80%/80%/50% across the three scenarios. The "50% boost" phrasing is problematic: it could mean a 50 percentage point absolute increase on the distractor scenario (0% → 50%), but the multi-target scenario shows a 70-point increase (10% → 80%) and the single-target shows a 50-point increase (30% → 80%). The paper does not define how "overall success rate" is computed (average across scenarios? weighted by some criterion?), making the 50% figure ambiguous. More importantly, the baseline without world model still achieves non-zero performance on single-target (30%) and multi-target (10%), suggesting the world model is not making impossible tasks possible but rather substantially improving performance on tasks where the VLA already has some competence. The paper's framing overstates the conclusion slightly: the world model is not a magic bullet but a highly effective training augmentation.

Claim 4: "The action attention masking strategy addresses the challenge of action error accumulation" and "the continuous Action Transformer head provides stronger generalization and smoother trajectories." Both claims are well-supported by the experiments. The attention mask's effectiveness is demonstrated through the ablation in Table 3 (54.0% vs. 76.6% with and without mask for chunked discrete actions) and Figure 6 (performance degradation with vanilla attention at longer chunk lengths). The continuous head's necessity for real-world deployment is demonstrated by the complete failure of the discrete model on real robots (Table 5, line 1: 0% across all scenarios) despite strong simulation performance (93.3% on LIBERO). However, the paper does not directly ablate the continuous head's architectural choices — for example, comparing L1 vs. L2 loss, or bidirectional vs. causal attention within the Action Transformer — so the specific contributions of each design element are not isolated. The claim that the continuous head provides "smoother trajectories" is asserted based on the real-world success rate improvement, but no quantitative trajectory smoothness metric (e.g., jerk, acceleration variance) is reported.

Genuine weaknesses in the experimental design:

• No statistical rigor. The paper reports point estimates from 50 LIBERO rollouts and 10 real-world trials per condition, with no confidence intervals, error bars, or significance tests. A 10% difference on real-world tasks (1 trial out of 10) is well within sampling noise, yet the paper draws comparative conclusions at this granularity. The ablation tables (Tables 3-5) have differences as small as 2-6 percentage points that may not be statistically distinguishable given the small number of trials.

• Missing compute-matched comparison against pretrained baselines. The paper claims competitiveness with pretrained models but does not account for the massive compute disparity. OpenVLA-OFT and UniVLA invested orders of magnitude more FLOPs in their pretraining phases than RynnVLA-002 uses in total. A fair comparison would either give RynnVLA-002 an equivalent pretraining compute budget (spent on world model pretraining or additional LIBERO data) or compare against non-pretrained versions of the baselines. The paper partially addresses this with the world model pretraining experiment (Table 8), but does not use this as the main comparison point.

• Single model initialization (Chameleon) with no architecture ablation. All experiments use Chameleon as the base architecture. The paper does not investigate whether the mutual enhancement effects depend on Chameleon's specific design (unified image understanding and generation) or would generalize to other multimodal backbones. Would initializing from a standard MLLM (without native image generation) and adding a separate image decoder yield similar benefits? The paper cannot answer this because it doesn't ablate the base architecture choice.

• World model improvements are not validated through downstream use. The paper shows that joint training improves video prediction metrics (Table 6), but does not demonstrate that these improved predictions are useful for any downstream task — planning, data augmentation, policy selection, or sim-to-real transfer. The world model is evaluated as an end in itself (better FVD/PSNR/SSIM/LPIPS) rather than as a means to better robot performance, which limits the practical significance of the finding.

• Real-world evaluation is narrow. Two pick-and-place tasks on a single robot platform (LeRobot SO100) with 248-249 demonstrations per task is a limited evaluation of real-world generalization. The paper does not test across different robots, different manipulation skills (pushing, sliding, inserting), or different environmental conditions (lighting, background, object instances). The distractor scenario adds some clutter but still uses known distractors — it does not test generalization to entirely novel objects or configurations.

• No multi-step world model rollout evaluation. The paper trains with N=1 (single-step prediction) but does not evaluate whether the world model can generate coherent multi-step rollouts, which would be necessary for planning or data augmentation applications. Single-step FVD/PSNR improvements do not guarantee that errors don't compound catastrophically over multiple steps — a well-known failure mode of autoregressive video prediction.

Missing experiments that would strengthen the paper:

• Ablation of the VLA data's contribution to the world model specifically in real-world settings. Table 6 shows simulation-only world model metrics. Does the Action World Model produce better video predictions on real robot data, where the VLA's visual understanding might be most valuable given the wider visual variation?

• Direct comparison of world model pretraining vs. longer joint training. Table 8 shows that world model pretraining helps, but does 100 epochs of world model pretraining + 100 epochs of joint training outperform 200 epochs of joint training from scratch? This would distinguish whether the pretraining benefit is from the curriculum structure or simply from more total training.

• Measuring trajectory smoothness quantitatively. The paper claims the continuous head produces smoother trajectories but only provides qualitative anecdote. Reporting jerk, spectral arc length, or velocity profile smoothness would make this claim testable.

• Scaling analysis. How does the mutual enhancement benefit scale with model size? With dataset size? The paper uses a single (unspecified) model scale and fixed dataset sizes, providing no evidence about whether the joint training advantage grows, shrinks, or stays constant with scale.

6. Limitations and Trade-offs

6.1 The Difficulty Estimation Cost for the Compute-Optimal Policy Is Unaccounted and Potentially Prohibitive

The paper's central claim — that joint training of VLA and world model objectives produces mutual enhancement — is demonstrated entirely under a fixed training recipe with a specific data mixture. However, the paper provides no guidance on how to determine the optimal ratio of VLA data to world model data in the training mixture, whether this ratio interacts with model scale or dataset size, or how sensitive the mutual enhancement effect is to getting this ratio right. The training objective in Section 3.2 states that "we mix the VLA model data and world model data to train our RynnVLA-002" with total loss ℒ_dis = ℒ_dis_action + ℒ_img, but the mixing ratio is never specified — the paper does not report what fraction of each training batch comes from VLA sequences versus world model sequences.

The consequence is a practical deployment barrier. A practitioner who wants to apply this method to their own robot domain must guess at the optimal data mixture, and the paper provides no evidence about whether the mutual enhancement is robust to this choice. If the world model data proportion is too low, the physics-informed representations that drive the real-world performance gains (the difference between 0% and 80% success in Table 5) may not develop. If it is too high, the VLA's action prediction capability may be under-trained, reducing task success. The paper's joint training framework implicitly assumes the mixture is not a sensitive hyperparameter, but this assumption is untested.

What evidence exists in the paper: None. The paper never ablates the data mixture ratio. All experiments use an unspecified default, making it impossible to assess whether the reported results represent the peak of a sharply tuned hyperparameter landscape or a plateau that is easy to hit. Table 8 shows that explicit world model pretraining (100% world model data first, then 100% VLA data) outperforms joint training from scratch on LIBERO-Goal (73.1% vs. 67.3%) and LIBERO-Long (30.2% vs. 23.0%), suggesting that the data presentation order matters — but the paper does not explore whether a staged curriculum with intermediate mixture ratios would outperform either extreme. The real-world results (Table 5) use an unspecified but presumably fixed mixture; if that mixture interacts with the domain gap between simulation and real-world visual distributions, practitioners may need to re-tune it for each new deployment, incurring substantial computational cost.

Mitigation status: The paper does not acknowledge this as a limitation and does not suggest future work on data mixture optimization. This is a significant gap given that the mutual enhancement claim depends on the training recipe. To fully characterize the method's robustness, the paper would need to sweep mixture ratios and show that performance is stable across a range, or provide a principle for setting the ratio based on measurable dataset properties (e.g., action dimensionality, visual complexity, trajectory length).

6.2 Discrete Action Model Fails Entirely on Real Robots, Creating a Sharp Simulation-to-Reality Generalization Cliff

The paper documents that the discrete-action variant of RynnVLA-002, despite achieving 93.3% average success on LIBERO (Table 1), scores 0% on all real-world manipulation scenarios (Table 5, line 1). The paper attributes this to two causes: severe overfitting of the large autoregressive backbone on limited real-world demonstration data (248–249 trajectories per task), and trajectory discontinuity caused by the action attention mask's isolation of individual actions within a chunk (Section 3.3). The continuous Action Transformer head is proposed as the solution and indeed restores performance to 80%+ on real-world tasks (Table 5, line 5).

The consequence is that the paper's architectural contributions are split into two categories with fundamentally different deployment viability. The action attention masking strategy (Figure 3b) — presented as one of the paper's three core contributions in Section 1 — is only useful in simulation. For any practitioner deploying on real hardware, the mask is irrelevant because the discrete model it enables does not transfer. This means the paper's claim that the discrete and continuous components form an integrated hybrid architecture is somewhat misleading: the discrete component serves exclusively as an auxiliary training signal (accelerating continuous head convergence per Figure 8), and all its other claimed benefits (error propagation mitigation, chunk length generalization per Figure 6) are simulation-only.

This bifurcation also creates a validation gap. The paper uses the discrete model's simulation performance (93.3%) to demonstrate the effectiveness of the attention mask and joint training, but these demonstrations do not carry over to the deployment-relevant continuous model except indirectly through the convergence acceleration claim. A reader might reasonably ask: does the attention mask actually matter for final real-world performance, or could it be removed entirely without affecting the continuous head's results? The paper does not test the continuous head without the discrete auxiliary loss on real robots — Table 5, line 5 includes both components, and there is no ablation of the discrete loss in the real-world setting.

What evidence exists in the paper: Table 5, line 1 provides direct evidence of the discrete model's real-world failure (0% across all scenarios). Table 1 shows the simulation success (93.3%) that creates the false expectation. Figure 8 shows that the discrete auxiliary loss accelerates continuous head convergence in simulation, but this is a training efficiency claim, not a final performance claim, and it is not replicated on real-world data. The paper does not ablate the discrete auxiliary loss from the continuous real-world model, so the marginal contribution of the discrete component to the headline real-world results is unknown.

Mitigation status: The paper is transparent about the failure — it explicitly states that the discrete model "rarely succeeds in real-world robot experiments" (Section 3.3) and provides the 0% results in Table 5. However, it does not adjust its contribution framing accordingly. The action attention mask is presented in the abstract and introduction as a core contribution alongside the continuous Action Transformer, without qualifying its simulation-only applicability. A more precise framing would distinguish between the mask as a training-time mechanism (useful for shaping representations during simulation pre-training) and the Action Transformer as the deployment-time mechanism (essential for real-world transfer), and would ablate the mask's contribution specifically in the real-world continuous setting.

6.3 Real-World Evaluation Scope Is Too Narrow to Support Generalization Claims

The paper's real-world experiments are limited to two pick-and-place tasks ("Place the block inside the circle" and "Place strawberries in the cup") on a single robot platform (LeRobot SO100 arm) with a small number of demonstrations (248 and 249 respectively). Each task is evaluated across only three scenarios (single-target, multi-target, with distractors) with 10 trials per scenario. This represents a narrow slice of the manipulation capabilities the paper claims to improve through joint VLA-world model training.

The consequence is that several of the paper's interpretive claims about the mechanism of mutual enhancement cannot be distinguished from task-specific confounds. The paper argues that the world model objective helps by "reinforcing attention to object interaction dynamics" (Section 4.3), citing the fact that the jointly trained model retries grasps while the VLA-only model gives up (Figure 5). But this evidence comes from a single qualitative example on a single task. Whether this mechanism generalizes to tasks requiring different physical reasoning — pushing objects, inserting pegs, pouring liquids, manipulating deformable objects — is entirely untested. A practitioner deploying this method for a task involving dynamic object interactions (e.g., scooping, sliding, tapping) or high-precision contact (e.g., peg insertion, screw tightening) has no evidence that the world model objective provides any benefit, let alone the 50% boost claimed for pick-and-place.

Similarly, the paper's claim that RynnVLA-002 "performs better than the baselines in cluttered environments" (Section 4.2) is supported by exactly two data points: block placement with distractors (80% vs. 50% for both baselines) and strawberry placement with multiple targets (80% vs. 50%/70%). This is a real effect — the numbers are clear — but whether it generalizes to other forms of clutter (visual occlusions, physically obstructing objects, transparent objects) or to more complex distractor semantics (distractors that are visually similar to targets, distractors that afford the same manipulation actions) is unknown. The paper attributes the clutter advantage to the world model's physics understanding, but this attribution is correlational: the model with world model training performs better in clutter. The paper does not provide direct evidence that the world model's image predictions are specifically more accurate in cluttered scenes, or that the VLA component's attention maps show increased focus on task-relevant objects in clutter when world model training is included.

What evidence exists in the paper: All real-world evidence is in Table 2 and Table 5, covering two tasks, three scenarios each, 10 trials per condition. The simulation experiments cover a broader range of manipulation types through the LIBERO benchmark (spatial reasoning, object identification, goal variation, long-horizon tasks), but these are simulation-only and, as discussed in Section 6.2, the simulation results for the discrete model do not transfer to real hardware. The continuous model's simulation results (97.4% on LIBERO) provide some evidence of task diversity, but the real-world gap between these numbers and the actual deployment performance is unmeasured — the paper does not report how the continuous model that achieves 97.4% in simulation performs on the exact same tasks if they were replicated on the real SO100 arm.

Mitigation status: The paper does not acknowledge the narrow scope of real-world evaluation as a limitation. It presents the two-task, single-robot results as sufficient evidence for the mutual enhancement claim without discussing what task properties might modulate the effect. Future work should evaluate on a broader set of manipulation primitives (pushing, sliding, inserting, rotating), on different robot embodiments, and with systematic variation of environmental conditions to establish the boundary conditions of the mutual enhancement effect. Without this, the paper's claims about world model benefits for "physics-informed" reasoning and "object interaction dynamics" are plausible hypotheses supported by limited evidence, not established findings.

6.4 The Model Is Initialized from Chameleon, Making the "Without Pretraining" Framing Misleading

The paper's headline result — "RynnVLA-002 achieves 97.4% success rate on the LIBERO simulation benchmark without pretraining" (Section 4.1) — and its favorable comparisons against pretrained baselines rely on a specific interpretation of "pretraining" that excludes the Chameleon initialization. Chameleon (Team, 2024) is a large multimodal model pretrained on massive image-text datasets to perform both image understanding and image generation. RynnVLA-002 inherits Chameleon's image tokenizer (VQ-GAN), text tokenizer (BPE), and — critically — the pretrained weights of the transformer backbone that process these tokens.

The consequence is that the comparison against "pretrained" baselines is not a fair test of whether joint VLA-world model training can substitute for external robot data. The baselines labeled as "pretrained" in Table 1 (OpenVLA, Octo, π₀, UniVLA, OpenVLA-OFT) were pretrained on robot manipulation datasets — the Open X-Embodiment dataset, LIBERO-90, or other large-scale robot data. RynnVLA-002 avoids this robot-specific pretraining, but it starts from Chameleon's pretrained visual and linguistic representations, which were trained on orders of magnitude more data than the robot-specific pretraining datasets. The fair comparison would be: does a model initialized from Chameleon and trained with joint VLA+world model objectives outperform a model initialized from Chameleon and trained with VLA-only objectives? The paper provides this comparison in the ablation studies (Table 3: 62.8% → 67.2% with world model; Table 4: 91.6% → 94.6% with world model), and the gains are real but modest in simulation — not the dramatic "matching pretrained models" story the abstract and Table 1 emphasize.

A practitioner reading the paper might conclude that they can skip collecting or accessing large-scale robot pretraining data and instead use joint VLA-world model training on their small in-domain dataset. But this conclusion depends on having a Chameleon-equivalent pretrained multimodal backbone available. If the practitioner is starting from a weaker base model — or from a model pretrained only on image understanding (without generation capabilities) — the joint training recipe may not produce the same results. The paper provides no evidence about how the mutual enhancement effect scales with base model capability or about the minimum base model quality needed to observe the effect.

What evidence exists in the paper: The paper does not ablate the base model architecture. All experiments use Chameleon, and there is no comparison against initializing from a standard MLLM (without native image generation) or from a randomly initialized transformer. The paper does not report what fraction of Chameleon's parameters are frozen versus fine-tuned during the joint training, making it impossible to assess how much of the final performance is attributable to Chameleon's pretrained representations versus the joint training objective. Table 8 provides partial evidence: "world model pretraining" (pretraining on the world model objective using the same LIBERO data, starting from Chameleon) improves over no pretraining, suggesting that in-domain self-supervision provides benefit on top of Chameleon's initialization. But the baseline performance without any world model pretraining or joint training (Table 3, line 1: 62.8% with discrete actions) already substantially exceeds what a randomly initialized model would achieve, confirming that Chameleon is doing significant lifting.

Mitigation status: The paper does not acknowledge the Chameleon initialization as a form of pretraining that complicates the comparison against robot-pretrained baselines. It presents the "without pretraining" claim without qualification in the abstract, introduction, and results. A more precise framing would distinguish between "without robot-specific pretraining data" and "without any pretraining" — the latter is false given Chameleon's initialization, and the distinction matters for practitioners evaluating whether to adopt the method. The paper would be strengthened by an ablation showing how much of the final performance comes from Chameleon's initialization versus the joint training objective, perhaps by comparing against a randomly initialized or weakly-initialized version of the same architecture.

6.5 The World Model Is Evaluated Only Through Video Prediction Metrics, Not Through Downstream Utility

The paper demonstrates that the Action World Model (jointly trained with VLA data) achieves better video prediction metrics than a standalone world model on the LIBERO validation set (Table 6): lower FVD, higher PSNR/SSIM, lower LPIPS on most suites. The paper interprets these improved metrics as evidence that "the image understanding capabilities inherited from the VLA model strengthen the world model's generation performance" (Section 4.3). However, the paper never evaluates whether these improved video predictions are useful for any downstream task — planning, data augmentation, policy selection, sim-to-real transfer, or model-based reinforcement learning.

The consequence is that the practical significance of the world model improvement is unestablished. A world model that generates more realistic-looking videos (better FVD/PSNR) may still produce physically implausible predictions that cause planning failures, or may not provide sufficient accuracy at contact points to enable model-based control. The paper's qualitative evidence in Figure 7 shows that the baseline world model produces inconsistent predictions across camera viewpoints (front camera shows failed grasp while wrist camera shows successful grasp) while the Action World Model generates consistent successful grasps. This is suggestive but insufficient: a single qualitative example does not establish that the improved consistency translates to reliable planning across the full distribution of task configurations. Moreover, the paper only evaluates single-step prediction (N=1 in Section 3.2). For planning applications, multi-step rollouts are essential, and autoregressive video prediction is known to compound errors catastrophically over long horizons — the single-step improvements in Table 6 do not guarantee multi-step improvements.

A practitioner interested in using RynnVLA-002's world model for planning or data generation would need to know: (a) whether the improved video prediction metrics correspond to improved planning success rates when the world model is used inside a planning loop, (b) whether multi-step rollouts remain coherent, and (c) whether the computational cost of generating image tokens (256–1024 tokens per frame, autoregressively) is compatible with real-time planning constraints. The paper answers none of these questions.

What evidence exists in the paper: Table 6 provides per-suite video prediction metrics comparing the standalone world model and the Action World Model. Figure 7 provides a single qualitative comparison. There is no evaluation of planning with the world model, no multi-step rollout evaluation (beyond the stated N=1 training), and no measurement of generation latency for world model queries. The paper's ablation showing that world model training improves VLA performance (Table 5: 0% → 50%+ on real robots) provides indirect evidence that the world model objective produces useful representations, but does not validate the world model's predictions as a standalone module.

Mitigation status: The paper does not acknowledge the lack of downstream world model evaluation as a limitation. It presents the improved video prediction metrics as sufficient evidence for the mutual enhancement claim in the world model direction. This is a significant gap because one of the paper's stated motivations for integrating world models is to enable "imagination" and "counterfactual reasoning" (Section 1), yet the paper never demonstrates that the trained world model can actually be used for these purposes. Future work should evaluate the world model in at least one downstream application — model-based planning, data augmentation for policy training, or policy selection from candidate actions — to establish that the metric improvements translate to practical utility.

6.6 No Accounting for Inference Latency in Sequential vs. Parallel Execution Tradeoffs

The paper's efficiency analysis (Table 7) reports inference frequency in Hz for various model configurations, showing that continuous action generation achieves 7.75–24.94 Hz depending on input complexity, while discrete action generation achieves 1.25–3.69 Hz. These numbers are presented as evidence that the continuous head is "significantly faster" (Section 3.3) and therefore suitable for real-time control. However, the paper does not discuss a fundamental architectural tradeoff: the VLA model requires sequential execution (each action chunk depends on the current visual observation, so the robot must wait for inference to complete before executing the next action), while the world model's image generation is entirely independent of action execution and could theoretically run asynchronously.

The consequence is that the paper's efficiency numbers conflate two different operational constraints. The VLA's 7.75 Hz inference frequency means the robot's control loop can run at that rate — but this includes the time to capture images, tokenize them, run the LLM backbone, and generate actions through the Action Transformer. If the world model's image generation were used for planning (e.g., generating imagined rollouts between action executions), it would add latency on top of the VLA's inference, potentially dropping the effective control frequency below what is needed for smooth manipulation. The paper does not measure the wall-clock time for world model image generation, which involves autoregressively decoding 256–1024 tokens through the full LLM backbone — a process that could easily be slower than the Action Transformer's parallel action generation. A practitioner who wants to use both the VLA and world model at inference time (for example, generating actions with the VLA while using the world model to verify that predicted outcomes match expectations) has no information about whether this is feasible within the control loop timing budget.

Additionally, the paper's comparison of discrete vs. continuous action generation frequency may overstate the continuous head's advantage for practical deployment. The discrete model running at 2.74 Hz (Table 7, line 6) generates an action chunk of size 10, meaning it produces actions at an effective rate of 27.4 actions per second if the entire chunk is executed without re-planning. The continuous model at 7.75 Hz with chunk size 10 (Table 7, line 9) produces actions at an effective rate of 77.5 actions per second — faster, but both may exceed the physical capabilities of the robot or the required control frequency. The paper does not discuss what control frequency is actually needed for the evaluated tasks, making the Hz numbers difficult to interpret as practical speed requirements.

What evidence exists in the paper: Table 7 reports Hz for various configurations. There is no measurement of world model inference latency, no end-to-end timing breakdown (tokenization vs. backbone vs. head vs. decoding), and no discussion of whether the reported frequencies are sufficient for the evaluated tasks. The paper does not report whether the LIBERO or real-world experiments used action chunk execution with re-planning at every timestep (which would require the full inference frequency) or open-loop execution of the full action chunk (which would amortize inference cost over multiple timesteps).

Mitigation status: The paper does not acknowledge inference latency or the tradeoff between sequential action generation and parallel world model prediction as a limitation. It presents the speed comparison as a straightforward advantage of the continuous head without discussing the operational context that determines whether speed matters. A more complete efficiency analysis would include: (a) wall-clock latency for both VLA and world model queries on the deployment hardware, (b) the control frequency required for the task (e.g., 10 Hz for pick-and-place, 50+ Hz for dynamic manipulation), (c) whether the reported experiments used re-planning at every timestep or open-loop chunk execution, and (d) a discussion of whether the world model can be used concurrently with action generation without dropping below the required control rate.

7. Implications and Future Directions

How This Work Changes the Landscape

RynnVLA-002 makes a specific methodological contribution that shifts how the embodied AI community should think about the relationship between VLA models and world models: these are not separate capabilities to be connected through modular interfaces, but complementary training objectives whose joint optimization produces representations neither objective alone can develop. This is not a paradigm shift — VLA models and world models both existed before, and multi-task training is well-established — but it is a substantive reframing of the architecture design problem from "how do we pipe a world model's predictions into a VLA's planning loop?" to "what training objectives, when combined, produce the internal representations that both action prediction and physics forecasting require?"

The magnitude of this contribution is restrained by the narrowness of the evidence. The mutual enhancement is demonstrated on exactly two real-world pick-and-place tasks on a single robot, and the world model's improvement (Table 6) is established through video prediction metrics rather than downstream utility. This is an existence proof, not a scaling law — the paper shows that mutual enhancement is possible, not that it is inevitable or that its magnitude grows with scale. The community should treat this as a promising architectural principle awaiting broader validation rather than a settled finding.

What this work reframes is the pretraining narrative for robot learning. Prior work treats large-scale external robot datasets (Open X-Embodiment, LIBERO-90) as the primary source of generalizable priors, with architectures like OpenVLA and π₀ investing substantial compute in pretraining on these datasets. RynnVLA-002 demonstrates — on the specific tasks and domains tested — that the world model objective can serve as an alternative form of self-supervision, forcing the model to learn object-centric, physics-informed visual representations from limited in-domain data alone. The paper's "without pretraining" framing (achieving 97.4% on LIBERO and 80%+ on real robots versus pretrained baselines) makes this argument explicit. If this finding generalizes, it implies that compute currently spent on collecting and training on large multi-robot datasets could be redirected toward world model pretraining on task-specific data, potentially lowering the barrier to entry for custom robot deployments. The caveat, as discussed in Section 6.4, is that RynnVLA-002 benefits from Chameleon's image-text pretraining, so the finding is really about robot-specific pretraining being replaceable, not all pretraining.

The paper also resolves a specific but important tension in the VLA design space: the discrete-vs-continuous action representation debate. Prior work in discrete-action VLAs (RT-2, OpenVLA) argued that discrete tokens enable unified multimodal reasoning but acknowledged precision loss and overfitting risk. Continuous-action approaches (π₀, Diffusion Policy) argued that continuous representations are necessary for smooth, precise control but lose the representational benefits of a shared token vocabulary. RynnVLA-002 demonstrates that these approaches are not in opposition — they can coexist productively, with discrete actions serving as an auxiliary training signal that accelerates continuous policy learning (Figure 8), and the continuous head handling deployment. This hybrid design pattern (discrete for training efficiency, continuous for deployment robustness) is a concrete takeaway that other VLA developers can adopt immediately.

A research direction that becomes less attractive after this work is the pursuit of monolithic discrete-action VLA models for real-world deployment. The paper provides clear evidence (Table 5, line 1: 0% real-world success for discrete actions despite 93.3% simulation) that discrete action representations, even with architectural interventions like the attention mask, fail catastrophically on physical hardware due to overfitting and trajectory discontinuity. This does not mean discrete actions are useless — they remain valuable as training signals (Figure 8) and may be viable in simulation-only research — but the paper establishes a strong prior that continuous action generation is necessary for real-robot transfer, and future VLA architectures should account for this from the start rather than retrofitting continuous heads as a fix.

A research direction that becomes more attractive is the systematic study of auxiliary prediction objectives as implicit attention mechanisms. The paper's evidence (Figure 5, showing that world model training causes the VLA to retry grasps rather than proceeding without object contact) suggests that the world model's image prediction loss teaches the model to attend to object interaction dynamics without any explicit object detection or segmentation supervision. This principle — that a prediction objective requiring accurate tracking of specific environmental properties implicitly trains attention to those properties — could generalize to other auxiliary tasks: contact prediction, force estimation, audio event forecasting, or language grounding. The paper provides a template for investigating such effects: compare behavior qualitatively (does the model persist at grasp attempts?), ablate the auxiliary objective, and measure performance specifically on tasks where the hypothesized attention mechanism matters (cluttered scenes, precision manipulation).

Follow-Up Research This Work Enables

Multi-step world model rollout evaluation for planning. The paper only evaluates single-step image prediction (N=1, Table 6), yet one of its stated motivations for world model integration is to enable "imagination" and "counterfactual reasoning" (Section 1). A direct follow-up would train RynnVLA-002 with N > 1 (multi-step autoregressive image generation during training, where the model predicts a sequence of future frames) and evaluate whether the generated rollouts are sufficiently accurate for model-based planning. The experiment: given a start state and a candidate action sequence, generate a multi-step video prediction, then use a separately trained success classifier (or human evaluation) to determine whether the predicted outcome matches the true outcome. Measure rollout accuracy as a function of horizon length (1, 5, 10, 20 steps) and compare the Action World Model against the standalone world model to test whether VLA co-training specifically improves long-horizon coherence. A negative result (catastrophic error compounding beyond 3-5 steps despite single-step improvements) would clarify the limits of the mutual enhancement claim and suggest that world model integration primarily benefits representation learning, not explicit planning.

Ablation of Chameleon initialization vs. other multimodal backbones. The paper's results depend on Chameleon, which natively supports both image understanding and image generation — a capability most MLLMs (LLaVA, Qwen-VL, InstructBLIP) lack because they are designed for image-to-text only. A critical follow-up would replicate the joint VLA-world model training recipe using (a) a standard MLLM backbone without native image generation, adding a separate image decoder head trained from scratch, versus (b) the full Chameleon initialization. This would quantify how much of the mutual enhancement effect comes from Chameleon's pretrained image generation capabilities versus the joint training objective alone. The experiment would train three models on LIBERO: Chameleon-initialized joint training (RynnVLA-002 baseline), standard-MLLM-initialized joint training (with a randomly initialized image decoder), and Chameleon-initialized VLA-only (no world model). Comparing (b) vs. (c) tests whether joint training helps even without a pretrained image generator; comparing (b) vs. (a) tests the marginal value of Chameleon's generation pretraining. This would clarify for practitioners whether they need a Chameleon-like base model or can adapt their existing MLLM infrastructure.

Data mixture ratio sensitivity analysis. The paper never specifies the proportion of VLA data to world model data in training batches and never ablates this ratio. A systematic sweep of mixture ratios (e.g., 100:0, 75:25, 50:50, 25:75, 0:100 VLA:world model) on LIBERO would establish whether the mutual enhancement is robust to this hyperparameter. The key measurements: (a) VLA success rate as a function of mixture ratio — is there a sharp peak or a broad plateau? (b) world model FVD as a function of mixture ratio — does the world model benefit from any VLA data, or only at specific ratios? (c) does the optimal ratio differ between simulation and real-world data? If the effect is sharply tuned, practitioners need guidance for setting the ratio; if it is robust across a wide range (say, 25-75% world model data), the method is substantially more practical. A secondary question: does the optimal ratio depend on dataset size? The paper's real-world tasks use 248-249 demonstrations; LIBERO has thousands. The ratio that works for large simulation datasets may not transfer to small real-world datasets where overfitting risk changes the tradeoff between the two objectives.

Task and embodiment diversity stress-test. The paper evaluates two pick-and-place tasks on a single SO100 arm. A stress-test follow-up would evaluate RynnVLA-002 on manipulation tasks requiring different physical reasoning: (a) contact-rich tasks (peg insertion, screwing, zipping) where precise force and contact modeling matters; (b) dynamic tasks (scooping, pouring, shaking) where object motion depends on complex physics; (c) multi-object rearrangement where spatial relationships between objects change over time. The hypothesis is that the world model's physics prediction objective should provide larger benefits for tasks where object dynamics are complex and less benefit for tasks dominated by kinematic reasoning (moving the arm to a fixed position). If the benefit is uniform across task types, the world model is providing general representation improvement rather than physics-specific reasoning. If the benefit concentrates on dynamic and contact-rich tasks, the mechanism is specifically physics-informed. The experiment would also vary robot embodiments (different arm kinematics, different grippers, mobile manipulators) to test whether the joint training advantage transfers across hardware or is specific to the SO100 kinematics the model was trained on.

Online adaptation: does the world model enable recovery from distribution shift? The paper's qualitative evidence (Figure 5) shows the jointly trained model retrying grasps on failure, suggesting the world model objective teaches persistence. A more rigorous follow-up would design an experiment where the robot encounters systematic distribution shift mid-task — e.g., an object is moved after the task begins, lighting changes dramatically, or a distractor is added during execution — and measure whether the jointly trained model adapts more successfully than the VLA-only baseline. The mechanism would be: the world model predicts that the current action sequence will fail (because its predicted next frame doesn't match the observed next frame), and this prediction error triggers replanning or strategy adjustment. This experiment would test whether the world model's predictive capability is actually used online for error detection, or whether its benefit is purely from offline representation learning that manifests as better initial action predictions. A negative result (joint training provides no online adaptation advantage over a VLA trained with equivalent data augmentation) would clarify that the mutual enhancement is a training-time phenomenon, not a runtime capability.

Scaling behavior: does mutual enhancement grow, shrink, or saturate with model size? The paper uses a single (unspecified) Chameleon model scale. A scaling study would train RynnVLA-002 variants at multiple parameter counts (e.g., 1B, 3B, 7B, 13B parameters) and measure: (a) the absolute performance gap between VLA-only and joint VLA+world model training at each scale; (b) the data efficiency — how many demonstrations are needed to reach a fixed performance threshold with and without the world model objective at each scale. If the gap grows with scale (larger models benefit more from joint training because they have capacity to learn physics-informed representations without interfering with action prediction), the mutual enhancement becomes increasingly important as models scale. If the gap shrinks (larger models learn equivalent representations from VLA data alone), the world model objective is primarily a data-efficiency trick for smaller models. The data efficiency measurement in (b) would be particularly informative for practitioners deciding whether to invest in world model training versus collecting more demonstrations.

Practical Applications and Downstream Use Cases

Small-dataset robot fine-tuning for custom manipulation tasks. The paper's most directly actionable finding is that adding world model training to a VLA fine-tuning pipeline on small datasets (248-249 demonstrations) can raise real-world success rates from below 30% to above 80% (Table 5, lines 4 vs. 5). This directly benefits robotics labs and companies deploying custom manipulation skills where large-scale pretraining data for their specific hardware and task is unavailable, and where collecting more than a few hundred demonstrations is expensive. The practical recipe: collect 200-300 teleoperated demonstrations, train a Chameleon-initialized model with mixed VLA and world model objectives (using the continuous Action Transformer for deployment), and expect substantial gains over VLA-only training, particularly in cluttered scenarios (80% vs. 50% on block placement with distractors in Table 2). The enabling condition is access to a Chameleon-like pretrained multimodal backbone; the open question is whether smaller or differently-architected backbones provide sufficient initialization for the effect to hold.

Implicit object attention without perception engineering. The paper's evidence that world model training causes the VLA to attend to object interactions and retry grasps on failure (Figure 5, Section 4.3) suggests a practical benefit for deployment scenarios where explicit object detection, segmentation, or tracking pipelines are unreliable — for example, when manipulating transparent, deformable, or visually ambiguous objects where standard vision systems fail. Rather than building a separate perception stack to identify grasp points and track object pose, practitioners can rely on the world model objective to implicitly teach the shared backbone what objects matter and how they respond to actions. This is particularly valuable for tasks with novel objects not represented in pretrained vision models, where perception engineering would require collecting and labeling new data. The caveat is that this benefit is demonstrated on two simple pick-and-place tasks; whether it extends to fine-grained manipulation (e.g., inserting a peg into a hole with sub-millimeter precision, where implicit attention may not provide sufficient spatial accuracy) is untested.

Training data augmentation through world model rollouts (conditional on validation). The paper shows that the Action World Model generates more accurate video predictions than a standalone world model (Table 6, Figure 7), but does not evaluate these predictions for data augmentation — using generated future frames as additional training data for the VLA. If follow-up work validates that multi-step world model rollouts are sufficiently accurate (the experiment proposed in the first follow-up direction above), a downstream application would be: given a small set of real demonstrations, use the trained Action World Model to generate imagined trajectories under varied action sequences, then train the VLA on this augmented dataset. This would be particularly valuable for safety-critical or rare-event scenarios (e.g., recovering from near-collision states, manipulating in extreme poses) where collecting real demonstrations is dangerous or impractical. The paper's current results provide the necessary condition (improved single-step prediction) but do not establish sufficiency (multi-step coherence and downstream policy improvement).

When to Prefer This Method

The paper does not articulate an explicit decision framework comparing RynnVLA-002 against named alternative architectures — it presents joint VLA-world model training as universally beneficial rather than conditionally preferable. The ablation studies show that adding the world model objective improves performance in every tested configuration (Tables 3, 4, 5), and there are no experiments where joint training underperforms single-objective training. The paper also does not identify scenarios where practitioners should prefer a standalone VLA over the joint framework, nor does it compare RynnVLA-002 against alternative methods for achieving similar benefits (e.g., explicit data augmentation, pretraining on larger robot datasets, or using a separate learned dynamics model in a planning loop). The "without pretraining" framing positions the method as a replacement for robot-specific pretraining, but this is presented as a positive feature of the approach rather than as a tradeoff against other ways of acquiring generalizable priors. Without counterfactual experiments or explicit comparison conditions, constructing a decision rule would require extrapolating beyond the paper's evidence.