MolmoAct2: Action Reasoning Models for Real-world Deployment

ArXiv: 2605.02881

🎯 Pitch

An open VLA model, MolmoAct2, outperforms proprietary systems like π0.5 while running on low-cost hardware, but its bigger surprise is MolmoAct2-Think, which selectively re-predicts depth only for moving scene regions, slashing reasoning latency without sacrificing location accuracy.

---

1. Executive Summary

This paper introduces MolmoAct2, a fully open action reasoning model for real-world robot deployment that advances its predecessor along five axes: a spatially specialized VLM backbone (Molmo2-ER), new multi-embodiment datasets, an open action tokenizer (MolmoAct2-FAST Tokenizer), a new VLA architecture with per-layer KV conditioning (where each layer of a DiT-style flow-matching action expert cross-attends to the corresponding VLM layer's keys and values rather than final hidden states), and MolmoAct2-Think, an adaptive-depth reasoning variant that re-predicts depth tokens only for scene regions that change between timesteps. Across 7 simulation and real-world benchmarks—the most extensive empirical study of any open VLA to date—MolmoAct2 outperforms strong baselines including π0.5, achieving 97.2% average success on LIBERO (a 10.6-point gain over the prior MolmoAct-7B-D) and 87.1% zero-shot on DROID real-world tasks, while MolmoAct2-Think yields further gains concentrated on the hardest task suites (+2.2% on LIBERO-Long). The FLOPs-matched backbone study establishes that Molmo2-ER surpasses GPT-5 and Gemini Robotics ER-1.5 on 9 of 13 embodied-reasoning benchmarks (63.8% overall average), demonstrating that specialized spatial pretraining transfers directly into action-learning improvements, though zero-shot deployment remains tied to the three embodiments with large-scale training data—bimanual YAM, SO-100/101, and DROID Franka—requiring fine-tuning for new robot platforms.

2. Context and Motivation

The Core Problem: VLA Models Don't Meet Real-World Deployment Demands

The paper addresses a fundamental tension in vision-language-action (VLA) models for robotics: the gap between research-visible capabilities and deployable reality. Today's VLA landscape presents a frustrating paradox for practitioners. On one side, frontier models demonstrate impressive capabilities in controlled settings—they can follow natural language instructions, manipulate diverse objects, and even exhibit emergent spatial reasoning. On the other side, when you try to actually deploy these systems on real robots for practical tasks, you encounter a cascade of obstacles: the best models are proprietary and inaccessible, open alternatives are tied to expensive hardware most labs can't afford, reasoning-augmented policies add prohibitive latency that makes real-time control impossible, and even after task-specific fine-tuning, success rates fall below the threshold required for dependable operation.

The paper crystallizes this gap in its opening paragraph: existing systems "are difficult to adapt or extend due to limited openness" and their "reasoning processes... impede real-time interaction." This framing is important because it shifts the conversation from "can VLAs work?"—a question largely answered in the affirmative—to "can VLAs be deployed, adapted, and reasoned with in ways that are practically useful?" MolmoAct2 is positioned as an answer to this second, harder question.

Why This Problem Matters

The motivation operates on multiple levels, each with distinct stakes for different communities.

For the robotics research community, the opacity of frontier VLA systems represents a serious barrier to scientific progress. When models like the π series or Gemini Robotics release weights alone—withholding training data, recipes, and procedures—researchers cannot reproduce results, diagnose failure modes, or build incrementally on prior advances. The paper frames this explicitly: "This opacity both impedes scientific progress and prevents practitioners from adapting these models to their own robots or fine-tuning on in-house demonstrations." The consequence is that progress in VLA research becomes concentrated in a handful of industrial labs with the resources to train these systems from scratch, while the broader academic community is relegated to consuming black-box outputs.

For practitioners seeking real-world deployment, the hardware economics are prohibitive. Open-weights VLAs that can run out-of-the-box are "tied to expensive or specialized robot platforms beyond the reach of most academic labs and independent researchers." The paper notes that even when models are available, they typically support a single platform—often a Franka arm costing tens of thousands of dollars—rather than the low-to-medium cost range ($6,000 USD for the complete bimanual YAM setup) that would democratize access to generalist robot policies. This constrains not only who can use these models, but also the diversity of settings in which they can be evaluated and improved, creating a feedback loop where model development optimizes for the narrow distribution of affordances available in well-funded labs.

For deployment reliability, the paper identifies a critical failure mode: "zero-shot performance remains brittle, and even after task-specific fine-tuning, success rates on realistic tasks fall well below the threshold required for dependable deployment." This is not merely an accuracy problem—it's a trust problem. A policy that succeeds 60% of the time on a pick-and-place task creates more problems than it solves in a real workflow, since failures must be detected and recovered, often requiring human intervention that negates the automation benefit. The threshold for "dependable" is task-dependent but generally high—the paper's own evaluation targets 15–50 trials per task, implicitly setting a bar where policies must demonstrate consistency, not just occasional success.

For the architectural design space, there is an unresolved tension between reasoning and reactivity. Recent work has demonstrated that grounded spatial reasoning—predicting goal images, point trajectories, depth maps, or full world-model rollouts before acting—improves both action quality and interpretability. The paper cites MolmoAct, CoT-VLA, and emerging world-model approaches as examples. However, it identifies a critical bottleneck: "hundreds of tokens or entire predicted frames must be generated before a single action is emitted." The mechanism intended to make policies more reliable thus "renders them too slow for closed-loop control." This tension—that reasoning improves quality but destroys responsiveness—creates a design challenge that cannot be solved by simply adding more reasoning, because the marginal benefit of additional reasoning tokens is overwhelmed by the latency cost.

Where Prior Approaches Fall Short

The paper systematically identifies specific limitations in four categories of prior work, building a case that no existing system adequately addresses the full deployment challenge.

Frontier VLA Models Are Effectively Closed Systems

The paper names the dominant proprietary models directly: Gemini Robotics (Team et al., 2025b), GR00T N1 (NVIDIA et al., 2025), and the π series (Black et al., 2024; Intelligence et al., 2025). These are characterized as "effectively closed systems: their training data, recipes, and model weights are proprietary." The paper acknowledges that some open-weight alternatives exist—it cites π0 and OpenVLA as examples—but notes that these "release weights alone, withholding the data and training procedures needed to reproduce or extend them." Even the existing open FAST action tokenizer weights are flagged as problematic because "their training data mixture is not fully specified," making the tokenizer a black-box component within an otherwise open pipeline.

This opacity matters for two distinct reasons that the paper distinguishes. First, scientific reproducibility: without training data and recipes, results cannot be independently verified, and the sources of model behavior cannot be traced. Second, practical extensibility: practitioners cannot adapt these models to new embodiments or fine-tune on in-house demonstrations without understanding the training distribution and procedures that produced the base model. The paper's own prior work, MolmoAct, is positioned as the notable exception: a fully open system that released "full training code, data, and model checkpoints."

Open VLAs Are Tied to Inaccessible Hardware

The paper argues that openness alone is insufficient if the resulting models cannot be used on affordable hardware. The specific problem: "The few open-weights VLAs that can be run out-of-the-box are tied to expensive or specialized robot platforms." While the paper doesn't enumerate exact costs for competing platforms, it contrasts this implicitly with the bimanual YAM setup, which it emphasizes costs "under $6,000 USD" for the complete rig—a figure that is achievable for academic labs and independent researchers. The DROID Franka platform, while open, requires a Franka arm costing significantly more, and the SO-100/101 platform represents the low-cost extreme at a few hundred dollars.

The hardware constraint is not merely economic—it shapes the entire research ecosystem. When models are developed for expensive platforms, the tasks, environments, and failure modes that get studied are those accessible to well-funded labs. The paper's decision to release datasets across three platforms spanning "the low-to-medium cost range" is a deliberate intervention in this dynamic, intended to broaden the set of researchers who can both use and improve VLA systems.

Reasoning-Augmented Policies Have Prohibitive Latency

This is where the paper makes its most technically specific critique of prior work. The argument proceeds in two steps. First, it acknowledges that reasoning works: "Recent systems including MolmoAct and others have shown that grounded spatial reasoning, predicted goal images, point trajectories, or full world-model rollouts improve both action quality and interpretability." Second, it identifies the latency cost that makes this reasoning impractical: "In current implementations, however, this reasoning dominates inference latency: hundreds of tokens or entire predicted frames must be generated before a single action is emitted."

The paper's framing of this as a tension rather than a failure is important. It is not arguing that reasoning should be abandoned—quite the opposite, it proposes MolmoAct2-Think as a reasoning variant. Instead, it argues that prior reasoning approaches are inefficient because they fail to exploit temporal redundancy. The paper states this explicitly in the MolmoAct2-Think motivation: "The main distinction from MolmoAct is that MolmoAct2-Think makes depth prediction adaptive across time. Robot trajectories contain substantial temporal redundancy: many cells in a scene-level depth grid remain unchanged from one control step to the next."

The prior approach that the paper aims to improve upon is the dense, per-step reasoning of MolmoAct and similar systems, where depth tokens or other spatial representations are regenerated in full at every timestep, "repeating nearly identical computation across largely static scenes." The latency numbers in Section 6.8 quantify the improvement: MolmoAct2-Think achieves 12.71 Hz with CUDA Graph optimizations, up from 8.04 Hz in the original implementation—a 1.58× speedup that directly addresses the closed-loop control bottleneck.

Fine-Tuned Success Rates Remain Below Deployment Thresholds

The paper argues that even when models are open and reasonably fast, their raw performance is insufficient: "after task-specific fine-tuning, success rates on realistic tasks fall well below the threshold required for dependable deployment." This claim is supported quantitatively in the experimental results. On the real-world bimanual YAM evaluation (Table 19), the best prior method (OpenVLA-OFT) achieves only 35.5% average success across 8 tasks, while even the best method on individual tasks rarely exceeds 60–80% (with π0.5 reaching 85.9% on Cup Storing but 2.0% on Pegboard). The paper's own MolmoAct2 achieves 50.6% average—a substantial improvement but still far from the 90–95%+ that would constitute dependable deployment for most practical applications.

The paper is transparent about this limitation, describing it as a threshold problem: current methods operate in a regime where performance is statistically distinguishable from chance but practically unreliable. The robustness study (Section 6.5) demonstrates that performance degrades further under distribution shift, with MolmoAct2 achieving only 26.25% on spatial variation perturbations and 54.10% on distractors, even as the best-performing model. This establishes that the deployment reliability problem is compounded by brittleness under the natural variation that characterizes real-world settings.

How This Paper Positions Itself

MolmoAct2 is positioned as addressing all four limitations simultaneously—a deliberately ambitious scope that distinguishes it from prior work that optimizes for a single axis (e.g., openness, performance, or speed). The paper states this explicitly: "MolmoAct2 is fully open in every respect, and beyond that, is capable of supporting real-world deployment for practical tasks."

The positioning relative to specific prior systems is carefully constructed:

Relative to MolmoAct (the direct predecessor): The paper identifies five axes of improvement, each addressing a limitation of the prior system. (1) The VLM backbone is upgraded from Molmo to Molmo2-ER, providing stronger embodied reasoning (46.8% → 63.8% on 13 benchmarks). (2) The data pipeline is expanded with new datasets, particularly the bimanual YAM data that was absent from MolmoAct. (3) The action tokenizer is made fully open and multi-embodiment. (4) The architecture adds per-layer KV conditioning rather than simpler hidden-state conditioning. (5) The reasoning variant shifts from dense per-step depth prediction to adaptive depth reasoning.

Relative to π0.5: The paper treats π0.5 as the primary competitive baseline, comparing against it across simulation and real-world benchmarks. The key differentiators are openness (π0.5 is "open-weights" but not fully open), hardware accessibility (π0.5 targets Franka while MolmoAct2 spans low-to-medium cost platforms), and reasoning capability (π0.5 does not provide interpretable intermediate representations).

Relative to closed-source systems (Gemini Robotics, GPT-5): The comparison operates on two levels. For embodied reasoning (the VLM backbone benchmarks), the paper directly competes, showing that Molmo2-ER surpasses these systems on 9 of 13 benchmarks. For the full VLA deployment, the comparison is structural rather than performance-based: closed systems cannot be reproduced, adapted, or deployed on arbitrary hardware, making them unsuitable for the research and practical deployment goals the paper targets.

The overarching positioning: MolmoAct2 is framed as more than "an academic robotics foundation model"—the paper explicitly states it wants the model to be "deployed in real-world workflows and deliver meaningful social impact." This ambition shapes the entire evaluation strategy: out-of-the-box deployment across embodiments, efficient fine-tuning, robustness under perturbations, trajectory quality beyond raw success rates, and inference speed optimizations. Each element of the evaluation is designed to answer not "does this work in principle?" but "can this be used in practice?"

3. Technical Approach

3.1 Reader orientation (approachable technical breakdown)

MolmoAct2 is a three-stage training pipeline that transforms a vision-language model into a deployable robot policy. The system solves the problem of producing precise, continuous robot arm movements from camera images and language instructions, while maintaining real-time control speed and open-source reproducibility. The "shape" of the solution is a modular, staged architecture: first teach the VLM to understand robots and predict discretized actions using standard language modeling, then attach a separate continuous-motion expert that learns to denoise action trajectories while reading the VLM's internal attention state at every layer, and finally provide an optional reasoning module that predicts spatial depth information only where the scene has changed, avoiding redundant computation.

3.2 Big-picture architecture (diagram in words)

The system has seven major components arranged in a sequential training pipeline:

1. Molmo2-ER Backbone — A 4B-parameter vision-language model specialized for embodied and spatial reasoning (trained in Section 2), which encodes camera images and language instructions into a sequence of hidden states using a SigLIP2 vision transformer and an autoregressive language model.

2. MolmoAct2-FAST Tokenizer — An open-weight, open-data action tokenizer that compresses one second of continuous 32-dimensional robot joint commands into a compact sequence of discrete tokens from a 2048-token vocabulary, enabling the VLM to predict actions using the same next-token objective it uses for text.

3. Pre-training Stage (MolmoAct2-Pretrain) — Adapts the VLM backbone into a discrete autoregressive robot policy by interleaving robot trajectory data (images, states, setup descriptors, action tokens) with standard multimodal data, training everything end-to-end with a next-token prediction loss for 200K steps.

4. Continuous Action Expert — A 621M-parameter DiT-style transformer with 36 layers (matching the VLM depth) that learns to generate smooth continuous action trajectories via flow matching, transforming Gaussian noise into executable robot commands through a learned velocity field.

5. Per-Layer KV Conditioning — The interface between the VLM and the action expert: at each of 36 layers, the action expert cross-attends to the keys and values from the corresponding VLM layer's self-attention, after lightweight learned projections map the VLM's attention state into the expert's cross-attention space. This gives the continuous controller access to the full hierarchical visual-language reasoning state rather than a single compressed hidden vector.

6. Post-training Stage (MolmoAct2) — Attaches the continuous action expert to the pre-trained VLA and co-trains both discrete and continuous action objectives for 100K steps, using knowledge insulation to prevent continuous-action gradients from destabilizing the VLM, producing the general continuous-control base model.

7. MolmoAct2-Think (optional reasoning variant) — Adds an autoregressive depth-token interface before action prediction, where the model predicts a 10×10 grid of quantized depth codes (128 possible values per cell) to make spatial structure explicit. At inference, only depth cells where the RGB image has changed (cosine similarity below 0.996) are re-predicted; unchanged cells replay cached values.

Information flows through the deployed system as follows: camera images → ViT encoder → visual tokens → LLM with language instruction and robot state tokens → VLM hidden states (36 layers of keys and values) → per-layer KV projections → action expert cross-attention → flow-matching integration (10 Euler steps from noise to action) → 32-dimensional continuous action chunk → unnormalized to embodiment-specific joint commands. If MolmoAct2-Think is active, an intermediate step inserts adaptive depth-token prediction before the action expert conditions on the VLM state.

3.3 Roadmap for the deep dive

• First, the three-stage training philosophy — why pre-training, post-training, and fine-tuning are separated, and what problem each stage solves, because this staged approach is the central architectural insight that enables stable optimization across heterogeneous data.
• Second, the pre-training stage and action tokenizer — how MolmoAct2-FAST Tokenizer compresses continuous trajectories into discrete tokens, and how these tokens are integrated into the VLM's next-token prediction objective alongside state tokens, setup descriptors, and multimodal data, because this is the foundation that makes the subsequent continuous expert possible.
• Third, the post-training architecture and per-layer KV conditioning — how the DiT-style action expert is constructed, how flow matching works for action generation, and why conditioning on keys and values at every VLM layer outperforms final-hidden-state conditioning, because this is the novel architectural contribution.
• Fourth, the post-training recipe — how discrete and continuous objectives are co-trained, how packing and masking work, and why knowledge insulation matters, because the training dynamics determine whether the architecture's potential is realized.
• Fifth, MolmoAct2-Think — how adaptive depth tokens are predicted, cached, and reused across timesteps, and how the depth gate modulates the influence of depth tokens on the action expert, because this addresses the latency bottleneck that makes prior reasoning systems impractical.
• Sixth, embodiment-specific fine-tuning and inference optimization — how the general post-trained model is rapidly adapted to specific robots, and how CUDA Graphs and caching optimize the inference pipeline for real-time control, because deployment practicality depends on both accuracy and speed.

3.4 Detailed, sentence-based technical breakdown

This is primarily a systems and architecture paper whose core idea is that a VLA can be constructed by first teaching a VLM to predict discretized actions through standard autoregressive training, then attaching a continuous flow-matching expert that conditions on the full depth of the VLM's attention state rather than its final output, with an optional adaptive spatial reasoning module that exploits temporal redundancy to make geometric grounding fast enough for real-time control.

---

The Three-Stage Training Philosophy: Why Separate Pre-Training, Post-Training, and Fine-Tuning?

The paper frames its training pipeline around a practical tension: "we want to preserve the scaling properties and general visual-language competence of a pretrained VLM, while producing precise continuous robot actions across heterogeneous embodiments." Directly training a VLM and a continuous action expert together from the beginning "makes optimization unnecessarily difficult: the model must simultaneously learn a robot-aware token interface, align new state/action embeddings, and fit a flow-matching controller."

The three-stage design decomposes this complex optimization into three sequentially easier sub-problems:

Stage 1: Pre-training turns Molmo2-ER into an action-aware VLA using only autoregressive supervision. The VLM predicts discrete action tokens (produced by MolmoAct2-FAST Tokenizer) using the same next-token objective it was pre-trained with for text. This gives the backbone aligned robot, state, and action-token embeddings under a familiar loss landscape, without introducing the additional complexity of a continuous action head. The key insight is that the VLM's existing next-token machinery is perfectly suited to learning robot action distributions if those actions can be expressed as tokens.

Stage 2: Post-training attaches the continuous action expert to this already action-aware backbone. Because the backbone has already learned to process robot observations, state tokens, and action semantics, the flow-matching controller and its VLM conditioning interface can "align quickly on a broad robot-data mixture" rather than fighting a multi-front optimization war. The backbone provides rich visual-language grounding from the start; the expert only needs to learn the continuous refinement.

Stage 3: Fine-tuning adapts the general post-trained model to specific embodiments, environments, and use cases. The paper explains the separation from post-training: "the post-training mixture is intentionally diverse across datasets, robots, control rates, camera layouts, and task families, so extending it until the model is directly deployment-ready for every target embodiment would require a long and expensive training stage." Fine-tuning is therefore kept fast (50K–100K updates) and embodiment-specific, while post-training produces a general continuous-control base that can be rapidly adapted.

An important practical nuance: during post-training, the paper uses knowledge insulation (from Driess et al., 2025), where the flow-matching loss updates only the action expert and its adapter projections, with gradients detached from the VLM. During fine-tuning, this insulation is removed—gradients from the flow loss propagate back through the VLM conditioning path. The paper states this choice is empirical: "we did not observe a consistent performance gain from detaching this path at this stage" during fine-tuning, while during post-training, the insulation prevents the newly attached expert from destabilizing the backbone's representations.

---

Pre-Training: Turning a VLM into a Discrete Action Model

MolmoAct2-FAST Tokenizer: From Continuous Trajectories to Discrete Tokens

The fundamental challenge for using a language model as a robot policy is that robot actions are continuous, embodiment-specific, and produced at different control rates, while language models operate on discrete tokens. MolmoAct2-FAST Tokenizer bridges this gap by mapping one second of robot motion into a compact discrete sequence, building on the FAST framework (Pertsch et al., 2025) but making the training data fully transparent.

Input standardization. Before tokenization, all raw robot telemetry is converted to a common format. Each training sequence corresponds to one second of robot motion, so the number of raw actions in a chunk is set by the control frequency of the source dataset (30 Hz for YAM/SO-100/101, 15 Hz for DROID, 10 Hz for LIBERO). Each action vector is padded to 32 dimensions to create a shared input space across embodiments with different action dimensionalities (e.g., bimanual arms with 14+2 gripper dimensions, single Franka arms with 7+1 dimensions). Continuous dimensions are normalized using 1–99 percentile statistics computed per dataset, which "limits the effect of outliers while preserving the useful dynamic range of each control dimension." Gripper commands are handled separately from this continuous normalization because they are "typically binary or narrow-range open/close signals" that would be distorted by percentile scaling.

Frequency-domain compression. The tokenizer follows the FAST approach of representing the trajectory with a frequency-domain transform (Discrete Cosine Transform, or DCT), quantizing the resulting coefficients through vector quantization, and applying byte-pair encoding (BPE) to produce tokens from a 2048-token action vocabulary. The DCT step is critical because it compresses temporal information: instead of representing every raw action sample (which would require 30 × 32 = 960 values per second at 30 Hz), the frequency-domain representation captures the trajectory shape with far fewer coefficients by exploiting the fact that robot motions are typically smooth and low-frequency.

Training data transparency. Unlike the prior FAST tokenizer whose released weights are not paired with a fully specified training distribution, MolmoAct2-FAST Tokenizer is trained from a fully documented mixture of one million action sequences across five embodiments (Table 2). The mixture balances the main deployment platforms (YAM: 30%, SO-100/101: 30%, DROID Franka: 30%) with smaller sources that broaden control-mode coverage (Fractal: 3.33%, BC-Z: 3.33%, Bridge: 3.33%). Critically, this mixture includes both absolute joint control and delta end-effector control conventions—the tokenizer does not require a unified coordinate frame. The paper notes: "During training, the VLM backbone learns to associate these disparate action representations with the specific embodiment mentioned in the task prompt or visual context." In other words, the tokenizer is a shared vocabulary, but the meaning of each token is context-dependent, and the VLM learns this context sensitivity from the setup descriptors and visual observations.

Multi-embodiment vector format. The 32-dimensional padded vector follows a structured convention. For single-arm embodiments, the format is: [A₁, ..., Aₙ, G₁, 0, ..., 0] where A represents n arm joints and G₁ is the gripper state. For bimanual embodiments: [AL₁, ..., ALₙ, GL, AR₁, ..., ARₙ, GR, 0, ..., 0] representing left and right arms respectively. This structured padding lets the model learn which dimensions are active for each embodiment without explicit masking—the zero-padded dimensions naturally carry no information.

Pre-Training Data Construction: How Robot Examples Enter the VLM

The pre-training mixture combines multimodal examples (10% sampling weight) with robot trajectories (90% sampling weight). Within the robot portion, YAM, SO-100/101, and DROID each receive 30% of the robot sampling weight, with the remaining 10% split across smaller BC-Z, BridgeData V2, RT-1, and MolmoAct Dataset sources. This allocation reflects the deployment priorities: the three main embodiments get equal weight, while the academic datasets provide additional diversity without dominating the distribution.

Visual processing. For all visual inputs during pre-training, the model uses a single resized crop rather than the high-resolution tiled crops used in Molmo2's general vision tasks. This is a deliberate efficiency choice: robot control requires processing multiple camera views at moderate resolution, and the tile-based approach would multiply sequence length prohibitively. Each camera observation is encoded by the SigLIP2 ViT, pooled by the vision-language connector (which reads features from the third-to-last and ninth-from-last ViT layers, pools 2×2 patch windows for images or 3×3 windows for videos into compact tokens using multi-head attention with the mean patch feature as query), and projected into the LLM embedding space.

For multi-camera robot episodes, the camera order is randomized at the episode level during pre-training. This prevents the model from relying on a fixed camera slot (e.g., always attending more to the first camera) and instead forces it to use the image-index text to identify which view it is processing—a form of view-invariant training that benefits downstream deployment where camera positions may vary.

For videos, the model samples at most 8 frames at up to 2 FPS. This is a significant temporal downsampling that the paper justifies implicitly: robot actions are predicted over one-second chunks, and the key visual information for action prediction (object positions, gripper state, spatial relationships) changes slowly relative to the control frequency, so dense frame sampling provides diminishing returns while dramatically increasing sequence length and compute cost.

State tokenization. Proprioceptive state (joint positions, gripper status) is represented separately from actions. After normalization using the same 1–99 percentile statistics, each scalar state value is uniformly discretized into one of 256 state tokens. Specifically, the continuous value is binned into 256 equal-width intervals spanning the normalized range, and the bin index is used as the token ID. These state tokens are appended to the prompt before the action output marker, giving the model explicit knowledge of the current robot configuration before it predicts the next action. The 256-bin discretization is a pragmatic choice: it provides approximately 8 bits of precision per joint, which is sufficient for robot control (joint positions rarely need sub-degree precision), while keeping the state vocabulary compact relative to the action vocabulary.

Setup and control descriptors. Each robot example includes setup and control strings wrapped with special tokens: <setup_start>bimanual yam robotic arms in molmoact2<setup_end> and <control_start>absolute joint pose<control_end> or <control_start>delta end-effector pose<control_end>. These descriptors make embodiment and action semantics explicit in the prompt, so the same tokenizer and backbone can support different robots without forcing all datasets into a single physical coordinate convention. The model learns to condition its action predictions on these descriptors—for example, interpreting the same action token differently when the control mode is "absolute joint pose" versus "delta end-effector pose."

Image augmentation. The paper applies a consistent augmentation pipeline to all images during robot pre-training, post-training, and fine-tuning (disabled at inference). The recipe includes: RandomCrop to 95% of original size followed by Resize back to original dimensions (providing mild translation invariance), RandomRotation up to 5 degrees (providing mild rotation invariance for camera roll), ColorJitter with brightness=0.2, contrast=(0.8, 1.2), saturation=(0.8, 1.2), hue=0.05 (providing robustness to lighting variation), and GaussianBlur applied with 20% probability (kernel_size=5, sigma=(0.1, 1.0), providing robustness to motion blur and focus variation). The augmentation is applied to all images and videos, not limited to robot data, which means the VLM backbone also sees augmented inputs during its vision-language training.

Packing. Examples vary dramatically in length: a text-only example may use only a few hundred tokens, while multi-camera robot examples carry images, states, and action targets. To maximize GPU utilization, the paper uses on-the-fly packing: each data-loader worker maintains a pool of 48 preprocessed examples, and a dynamic-programming solver selects the subset that maximizes total text tokens plus a weighted image-crop bonus, subject to per-sequence caps (maximum 4200 text tokens for pre-training). Packed examples share the same forward pass but have their attention masks separated, so the model cannot condition one example on another. The solver objective is:

$T + w_i I \quad \text{subject to} \quad T \leq T_{\text{max}}, \quad I \leq I_{\text{max}}$

where $T$ is the total number of text tokens, $I$ is the total number of image crops, and $w_i = 30$ balances token and image utilization.

What it computes: a simple linear utility function that the dynamic programming solver maximizes by selecting which examples from the pool to pack together. The weight $w_i = 30$ means one image crop is considered as valuable as 30 text tokens for packing purposes, which roughly reflects the relative computational cost of processing image tokens versus text tokens in the transformer.

Why this form: the linear utility is computationally efficient for the dynamic programming solver (subset-sum-like structure), and the fixed weight ratio avoids the need to learn or tune packing preferences. The value 30 was likely chosen empirically to balance visual and text utilization—too low would starve visual examples, too high would waste text capacity.

Pre-Training Objective and Training Configuration

The pre-training loss is the standard autoregressive next-token prediction loss applied to all tokens: text tokens from VLM examples, discrete action tokens from robot examples, and special marker tokens. The training runs for 200K steps with a maximum sequence length of 4200 tokens, a global batch size of 128 across 64 H100 GPUs (approximately 5,760 GPU hours total).

The optimization uses separate learning rates per component: the vision encoder and connector use a learning rate of $5 \times 10^{-6}$, while the language model uses $1 \times 10^{-5}$. This two-tier schedule reflects the fact that the vision components have already been extensively trained in Molmo2-ER and need smaller updates, while the language model must learn substantial new capabilities (robot state interpretation, action token prediction) and benefits from a higher learning rate. All components use AdamW with betas (0.9, 0.95), epsilon $10^{-6}$, weight decay following standard practice, and cosine learning rate decay to 10% of the initial value over the training horizon, with 200 linear warmup steps for each component.

---

Post-Training Architecture: The Continuous Action Expert and Per-Layer KV Conditioning

Why Flow Matching for Action Generation?

The paper chooses flow matching (Lipman et al., 2022) over diffusion as the continuous generation framework for the action expert. Flow matching defines a deterministic transformation between a simple base distribution (Gaussian noise) and the target data distribution (normalized action trajectories) through a time-dependent velocity field. The key advantage over diffusion for robot control is that flow matching provides straighter integration paths—fewer steps are needed during inference to go from noise to a clean action, which reduces the latency of each action generation cycle. The paper uses 10 Euler integration steps at inference.

The flow-matching objective for a single action chunk is constructed as follows. Given a normalized target action chunk $\mathbf{a} \in \mathbb{R}^{H \times D_{\text{max}}}$ (where $H$ is the action horizon—30 for post-training, 10 for LIBERO—and $D_{\text{max}} = 32$ is the padded action width), Gaussian noise $\boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})$, and a sampled time $t \in [0,1]$, the interpolated trajectory and target velocity are:

$\mathbf{x}_t = (1 - t)\boldsymbol{\epsilon} + t\mathbf{a}$

$\mathbf{u}^\star = \mathbf{a} - \boldsymbol{\epsilon}$

where $\mathbf{x}_t$ is the noisy action at time $t$ (linearly interpolating between pure noise at $t=0$ and the clean action at $t=1$), and $\mathbf{u}^\star$ is the target velocity field that the model must predict—the direction and magnitude from noise to data.

What it computes: the first equation constructs the forward noising process: at $t=0$, $\mathbf{x}_0 = \boldsymbol{\epsilon}$ (pure noise); at $t=1$, $\mathbf{x}_1 = \mathbf{a}$ (clean action); at intermediate $t$, the trajectory is a convex combination. The second equation defines the constant velocity that transforms noise into data: subtracting the noise from the clean data gives the vector that, if followed at constant speed from $t=0$ to $t=1$, would produce the observed data.

Why this form: the linear interpolation schedule $(1-t)\boldsymbol{\epsilon} + t\mathbf{a}$ makes the target velocity $\mathbf{a} - \boldsymbol{\epsilon}$ constant in time—it does not depend on $t$. This is the key property that makes flow matching more efficient than diffusion: the velocity field is simpler (constant direction) rather than time-dependent, which means fewer integration steps suffice. An alternative like DDPM would require the model to predict time-dependent scores $\nabla_{\mathbf{x}_t} \log p_t(\mathbf{x}_t)$, which vary nonlinearly with $t$ and require more steps for accurate integration.

The flow-matching loss is:

$\mathcal{L}_{\text{flow}} = \mathbb{E}_{\mathbf{a}, \boldsymbol{\epsilon}, t} \left[ \| \mathbf{m} \odot (f_\theta(\mathbf{x}_t, t, \mathbf{c}) - \mathbf{u}^\star) \|_2^2 \right]$

where $f_\theta$ is the action expert network, $\mathbf{c}$ is the VLM context (task, observations, setup/control descriptors, state tokens), $\mathbf{m}$ is a binary mask that zeros out padded action steps and padded action dimensions, and $\odot$ denotes element-wise multiplication.

What it computes: the squared L2 distance between the predicted velocity and the target velocity, but only over valid (non-padded) dimensions, averaged over random noise samples, time samples, and training action chunks. The mask $\mathbf{m}$ ensures that embodiments with fewer than 32 action dimensions do not receive spurious loss from padded dimensions, and that action chunks with fewer than $H$ steps do not receive loss on padded temporal positions.

Why this form: the masked MSE is the standard regression objective for flow matching—it estimates the conditional expectation of the velocity field. The mask is essential because the shared 32-dimensional action space is an artificial construction for batch processing; without it, the model would be forced to predict specific (normalized) values for padded dimensions, which would create conflicting gradients across embodiments with different native dimensionalities.

Multiple flow samples per chunk. During post-training, for each robot action chunk, the paper evaluates the flow objective at $K = 4$ independently sampled noise-time pairs $\{(\boldsymbol{\epsilon}_i, t_i)\}_{i=1}^K$, reusing the same VLM context. The average loss is:

$\mathcal{L}_{\text{flow}}(\mathbf{a}, \mathbf{c}) = \frac{1}{K} \sum_{i=1}^{K} \| \mathbf{m} \odot (f_\theta(\mathbf{x}_{t_i}, t_i, \mathbf{c}) - (\mathbf{a} - \boldsymbol{\epsilon}_i)) \|_2^2$

What it computes: four points on the same flow trajectory from noise to data, providing denser supervision per training sample than a single noise-time pair would. Since the VLM context is invariant across the four samples (same scene, same instruction), this effectively quadruples the number of flow-matching training examples without quadrupling the expensive VLM forward pass.

Why this form: the VLM forward pass dominates training compute, and reusing one VLM forward pass for multiple flow samples amortizes this cost while providing richer training signal. The paper uses $K=4$ during post-training (constrained by GPU memory) and increases to $K=8$ during fine-tuning where memory is less constrained (smaller datasets, fewer GPUs). The alternative of $K=1$ would waste the VLM context that has already been computed, and the ablation (Table 12) confirms that increasing $K$ improves performance—$K=8$ achieves 95.90% average LIBERO versus 94.15% for $K=1$.

Inference-time integration. At inference, the action expert starts from Gaussian noise $\mathbf{z}_0 \sim \mathcal{N}(0, \mathbf{I})$ and iteratively applies:

$\mathbf{z}_{i+1} = \mathbf{z}_i + \Delta t \, f_\theta(\mathbf{z}_i, t_i, \mathbf{c}), \quad t_i = \frac{i}{N}, \quad \Delta t = \frac{1}{N}$

where $N = 10$ is the number of Euler steps used in the released checkpoints. The final normalized trajectory $\mathbf{z}_N$ is sliced to the embodiment-specific action width and unnormalized with dataset statistics.

What it computes: a first-order Euler integration of the predicted velocity field from $t=0$ (noise) to $t=1$ (clean action). At each step, the current noisy trajectory $\mathbf{z}_i$ is fed to the expert along with the current time $t_i$ and the VLM context $\mathbf{c}$, the predicted velocity is scaled by the step size $\Delta t$, and the trajectory is updated.

Why this form: Euler integration with 10 steps is a pragmatic balance between accuracy and speed. More steps would produce more accurate trajectories (closer to the true flow ODE solution) but increase latency linearly; fewer steps would be faster but might produce trajectories that deviate from the data distribution. The choice of 10 steps is likely tuned empirically—the paper does not ablate this number, but the 55.79 Hz control rate with CUDA Graph optimization (Section 6.8) suggests 10 steps is fast enough for real-time control while maintaining action quality.

The Action Expert Architecture

The action expert is a DiT-style transformer with $L = 36$ layers, matching the VLM depth. This depth-matching is deliberate: each expert layer will condition on the corresponding VLM layer, so having the same number of layers creates a natural one-to-one mapping. The expert has 621M parameters versus the VLM's 4.0B, making it substantially smaller while still being a full transformer in its own right.

Input embedding. A noisy action chunk $\mathbf{x}_t \in \mathbb{R}^{H \times 32}$ is first projected from 32 continuous dimensions to the expert hidden width of 768 through a learned linear projection. A sinusoidal time embedding (standard transformer positional encoding applied to the scalar time $t$) is passed through a small MLP to produce dimension 256, which will modulate each expert block.

Block structure. Each expert block $\ell$ applies three operations in sequence, each wrapped with DiT-style adaptive layer normalization and gating. Let $\mathbf{h}_\ell$ be the input to block $\ell$, and let $\mathbf{t}$ be the time embedding. The block computes:

$\mathbf{h}'_\ell = \mathbf{h}_\ell + g^\text{sa}_\ell \, \text{SA}(\text{AdaRMS}^\text{sa}_\ell(\mathbf{h}_\ell, \mathbf{t}))$

$\bar{\mathbf{h}}_\ell = \mathbf{h}'_\ell + g^\text{ca}_\ell \, \text{CA}(\text{AdaRMS}^\text{ca}_\ell(\mathbf{h}'_\ell, \mathbf{t}), \tilde{\mathbf{K}}_\ell, \tilde{\mathbf{V}}_\ell)$

$\mathbf{h}_{\ell+1} = \bar{\mathbf{h}}_\ell + g^\text{ff}_\ell \, \text{MLP}(\text{AdaRMS}^\text{ff}_\ell(\bar{\mathbf{h}}_\ell, \mathbf{t}))$

where SA is self-attention over the action chunk (bidirectional, using RoPE for action-step position encoding), CA is cross-attention to the VLM context (using projected keys and values from VLM layer $\ell$), MLP is a SwiGLU feedforward network, and $\text{AdaRMS}$ is RMS normalization followed by time-conditioned shift and scale:

$\text{AdaRMS}(\mathbf{x}, \mathbf{t}) = \text{RMSNorm}(\mathbf{x}) \odot (1 + \gamma(\mathbf{t})) + \beta(\mathbf{t})$

where $\gamma(\mathbf{t})$ and $\beta(\mathbf{t})$ are learned linear projections of the time embedding, producing per-dimension scale and shift parameters.

What it computes: a residual block where each sub-layer (self-attention, cross-attention, MLP) receives a time-conditioned normalization that adapts the layer's behavior based on where in the denoising process the model currently is. The gates $g^\text{sa}_\ell$, $g^\text{ca}_\ell$, $g^\text{ff}_\ell$ are learned scalars (initialized to zero) that control the initial contribution of each sub-layer—they allow the network to start training close to the identity function and gradually activate sub-layers as needed.

Why this form: the DiT (Diffusion Transformer) design separates the time conditioning from the main transformer computation through adaptive normalization rather than concatenating time embeddings to the input. This has been shown to scale better than concatenation-based conditioning for diffusion and flow models because it gives the time signal direct multiplicative control over the normalization statistics at every layer, which is crucial for denoising at different noise levels (early denoising steps need coarse structure, later steps need fine detail). The zero-initialized gates are a training stability trick: at initialization, the expert is effectively the identity function, predicting near-zero velocity everywhere. The flow loss gradient then gradually activates the sub-layers, avoiding the training instability that can occur when a newly attached module produces large random outputs. The final projection layer is also initialized to produce near-zero velocity predictions for the same reason.

The expert uses query-key normalization (QK-Norm) in both self-attention and cross-attention layers. This normalizes the query and key vectors before computing attention scores, which has been shown to stabilize training in large transformers by preventing attention logit drift. Rotary position embeddings (RoPE) are applied to the action sequence dimension in self-attention, giving the denoising transformer an explicit ordering over the predicted trajectory steps so that step $t$ and step $t+1$ have a structured positional relationship.

Per-Layer KV Conditioning: The Core Architectural Innovation

The central question the paper's architecture addresses is: how should the continuous action expert receive context from the VLM? A simple approach—conditioning on the VLM's final hidden state—"compresses the backbone into a single residual-stream representation" and discards the rich hierarchical information present at intermediate layers. MolmoAct2 instead conditions the expert on the VLM at every layer, giving the continuous controller "access to the same attention state used by the VLM itself."

The mechanism works as follows. For VLM layer $\ell$, the self-attention operation produces key and value tensors $\mathbf{K}^\text{vlm}_\ell$ and $\mathbf{V}^\text{vlm}_\ell$ for every token in the context (prompt text, image tokens, state tokens, markers). In the released architecture, the VLM has 8 KV heads with head dimension 128, so each token contributes a 1024-dimensional key and a 1024-dimensional value (8 heads × 128 dimensions, concatenated). These tensors are then passed through learned adapter projections:

$\tilde{\mathbf{K}}_\ell = \text{reshape}(\mathbf{P}_K \mathbf{K}^\text{vlm}_\ell)$

$\tilde{\mathbf{V}}_\ell = \text{reshape}(\mathbf{P}_V \mathbf{V}^\text{vlm}_\ell)$

where $\mathbf{P}_K$ and $\mathbf{P}_V$ are learned linear projections (matrices) that map the 1024-dimensional VLM key/value space to the 768-dimensional action expert width, and the reshape operation organizes the projected vectors into 8 attention heads of 96 dimensions each for the expert's cross-attention.

What it computes: for each token in the VLM context, its per-head key and value vectors (produced by VLM self-attention) are linearly transformed to match the action expert's dimensionality and head structure. The projected keys and values then serve as the cross-attention context for the action expert at the same layer depth.

Why this form: the key design choice is that the adapter projections $\mathbf{P}_K$ and $\mathbf{P}_V$ are separate from the VLM self-attention projections. The VLM computes its own keys and values for its own self-attention; the adapter reads these and projects them into a different space optimized for the expert's cross-attention. This modular separation means the VLM can be trained (or kept frozen) without considering the expert's needs, while the adapter learns to extract from the VLM's attention state whatever information is most useful for continuous control.

The cross-attention in expert block $\ell$ is then:

$\text{CA}(\mathbf{Q}_\ell, \tilde{\mathbf{K}}_\ell, \tilde{\mathbf{V}}_\ell) = \text{softmax}\left( \frac{\mathbf{Q}_\ell \tilde{\mathbf{K}}_\ell^\top}{\sqrt{d_h}} \right) \tilde{\mathbf{V}}_\ell$

where $\mathbf{Q}_\ell$ is the action expert's query (computed from the current noisy action hidden state), $\tilde{\mathbf{K}}_\ell$ and $\tilde{\mathbf{V}}_\ell$ are the projected VLM keys and values, and $d_h = 96$ is the expert head dimension.

What it computes: standard scaled dot-product attention where the action expert's queries attend over all VLM context tokens (images, text, state) to aggregate relevant information for denoising the current action chunk. The softmax produces attention weights that sum to 1 over the context tokens, and the weighted sum of values $\tilde{\mathbf{V}}_\ell$ is the output.

Why this form: per-layer KV conditioning gives the action expert access to the VLM's attention state at the same representational depth. Early VLM layers encode low-level visual features (edges, textures, simple spatial layouts); middle layers encode object-level semantics and spatial relationships; late layers encode task-level reasoning and language grounding. By conditioning the expert at each corresponding depth, the action expert can draw on whatever abstraction level is most relevant for the current denoising step—coarse spatial layout early in denoising, fine geometric details later in denoising. This is a richer conditioning signal than final hidden states, which collapse all levels of abstraction into a single vector.

Ablation evidence. The paper's ablation (Table 11) directly compares three conditioning sources: hidden-state conditioning (condition on VLM final layer output only), per-head per-layer KV conditioning (preserve head structure in the projection), and the standard per-layer KV conditioning. On LIBERO, the standard per-layer KV conditioning achieves 95.9% average success, outperforming per-head per-layer (94.8%) and hidden-state (94.0%). The largest gaps appear on Object (99.0% vs. 95.4% vs. 97.2%) and Long (89.8% vs. 89.6% vs. 87.0%), suggesting that per-layer KV conditioning is particularly helpful for object-centric manipulation (where fine-grained visual features matter) and long-horizon tasks (where hierarchical reasoning matters). Interestingly, hidden-state conditioning is competitive on Spatial (97.0% vs. 96.2%), perhaps because spatial reasoning is well-captured in the VLM's final representations.

The standard variant outperforms the per-head variant because flattening the keys and values before projection (rather than keeping them separated by head) gives the adapter more flexibility in how it redistributes the VLM's attention state across the expert's heads—the adapter can learn to recombine information from different VLM heads in task-appropriate ways.

Knowledge insulation during post-training. During post-training, the VLM keys and values are detached before they enter the expert's cross-attention, meaning $\mathcal{L}_{\text{flow}}$ updates only the action expert parameters $f_\theta$ and the adapter projections $\mathbf{P}_K$, $\mathbf{P}_V$, but does not back-propagate gradients into the VLM. The VLM is still updated by the discrete autoregressive loss $\mathcal{L}_{\text{LLM}}$. This prevents the newly attached expert's loss from destabilizing the VLM's carefully learned visual-language representations during the critical post-training stage.

---

Post-Training Recipe: Co-Training Discrete and Continuous Action Objectives

The post-training objective combines the autoregressive next-token prediction loss from pre-training with the flow-matching loss:

$\mathcal{L}_{\text{post}} = \mathcal{L}_{\text{LLM}} + \mathcal{L}_{\text{flow}}$

What it computes: the sum of two losses computed on the same batch (though on different subsets—see below). The LLM loss applies to text tokens in multimodal examples and discrete action tokens in robot examples. The flow loss applies only to continuous robot action chunks. The two losses are simply added without learned weighting, implicitly giving them equal importance.

Why this form: the equal weighting reflects the paper's philosophy that discrete and continuous action prediction are complementary rather than competitive. The discrete loss keeps the VLM grounded in the correct action semantics and token vocabulary; the continuous loss teaches the expert to produce smooth, precise trajectories. The paper's ablation (Table 13) shows that removing discrete co-training during fine-tuning shifts performance across suites—Long improves (95.0% vs. 93.2%) but Spatial and Object degrade, and the overall average is slightly lower (96.95% vs. 97.20%). This suggests the discrete loss provides regularization that is particularly important for precision-critical tasks (Object) while the continuous loss alone suffices for long-horizon planning (Long).

Masking the discrete action target from the expert. A critical implementation detail: for robot examples, the discrete action-token span (the target tokens the LLM predicts) is masked out of the expert's VLM conditioning path. This prevents the expert from "cheating" by conditioning on the ground-truth action tokens it is supposed to predict, which would make the flow-matching task trivial and prevent the expert from learning to use visual and state information for action generation. The expert conditions only on the task instruction, visual observations, setup/control descriptors, and state tokens—everything except the action tokens.

Packing for the continuous expert. The packing procedure from pre-training is extended to handle continuous action chunks. A packed sequence can contain multiple robot examples, each with its own action chunk. The expert attends only to the VLM context corresponding to its own sub-example, not to other examples in the pack. To keep memory stable when packed batches contain different numbers of robot chunks, the paper pads the packed action-chunk axis to a small fixed cap of five chunks per packed sequence. Padded chunk rows are marked invalid and excluded from the flow loss; sequences with more than five chunks are still used but processed without this fixed-cap padding.

Sequence length split. During post-training, robot batches use a sequence length of 2100 tokens, while non-robot VLM batches keep the 4200-token context from pre-training. This split exists because robot batches additionally run the action expert and four flow samples per action chunk, which approximately doubles the memory footprint per example relative to VLM-only batches. Halving the sequence length for robot batches keeps memory usage balanced across the two dataloaders.

Training configuration. Post-training runs for 100K updates with global batch size 128 and device batch size 2 across 64 H100 GPUs (approximately 2,304 GPU hours). Learning rates match pre-training for VLM components ($5 \times 10^{-6}$ for ViT and connector, $1 \times 10^{-5}$ for LLM) and use a higher $5 \times 10^{-5}$ for the action expert, which starts from random initialization and needs faster adaptation. All other optimizer settings (AdamW, betas, epsilon, warmup, cosine decay) match pre-training.

---

MolmoAct2-Think: Adaptive Depth Reasoning

The Spatial Reasoning Problem

Robot manipulation depends on spatial information that is "only indirectly supervised by action imitation: object distance, free space, occlusion, and surface layout all affect the action, but standard behavior-cloning objectives do not ask the model to make this structure explicit before acting." MolmoAct (the predecessor) solved this by having the model predict a discretized depth map as an intermediate reasoning step before action prediction—essentially, "first understand where things are in 3D, then decide what to do." The depth representation was a 10 × 10 grid of quantized depth codes (128 possible values per cell), represented as autoregressive tokens that the VLM predicts before the action tokens.

The problem with this approach is that it regenerates all 100 depth tokens at every timestep, even though "robot trajectories contain substantial temporal redundancy: many cells in a scene-level depth grid remain unchanged from one control step to the next." In a static camera setup (common for tabletop manipulation), the background, table surface, and stationary objects occupy most depth cells and do not change across consecutive frames. Regenerating their depth codes every step wastes computation and increases latency.

Adaptive Depth Data Construction

For every robotics dataset, the paper attaches depth annotations to the policy observation stream. The pipeline uses monocular depth estimation (Depth Anything V2; Yang et al., 2024) to estimate a dense depth map for each RGB frame, then quantizes it using the pretrained depth VQ-VAE from MolmoAct (which operates on 320×320 depth images with a downsampling factor of 32, producing a 10×10 grid of codebook indices from {0, ..., 127}). This produces the full depth codes $\mathbf{d}_t \in \{0, \ldots, 127\}^{100}$ for each frame, flattened in raster order.

The adaptive-depth side channels are then constructed. For each frame $t$:

• A depth buffer $\mathbf{b}_t$ stores the carried-forward codes: values that have not changed from the previous frame are replayed from the buffer; values that have changed are taken from the current frame's full depth codes.

• An update mask $\mathbf{m}_t \in \{0, 1\}^{100}$ marks which cells changed.

The mask is computed by comparing RGB patches between consecutive frames. The RGB image is resized to 320×320 and divided into a 10×10 grid of 32×32 patches. For each cell $i$, the cosine similarity between the patch at frame $t$ and frame $t-1$ is computed:

$m_{t,i} = \mathbf{1}\left[ \cos(\mathbf{x}_{t,i}, \mathbf{x}_{t-1,i}) < 0.996 \right]$

$b_{t,i} = \begin{cases} d_{t,i}, & \text{if } m_{t,i} = 1 \\ b_{t-1,i}, & \text{if } m_{t,i} = 0 \end{cases}$

where $\mathbf{x}_{t,i}$ is the flattened 32×32×3 = 3072-dimensional RGB patch vector for cell $i$ at frame $t$, $\cos(\cdot, \cdot)$ is cosine similarity, and $\mathbf{1}[\cdot]$ is the indicator function.

What it computes: a binary decision per grid cell: if the RGB appearance has changed enough (cosine similarity below 0.996), the depth code is updated from the current frame's estimation; otherwise, the previous buffer value is replayed. The threshold 0.996 is very strict—cosine similarity measures angle between vectors, and 0.996 corresponds to approximately 5 degrees of angular difference, meaning even small lighting changes or sub-pixel motions can trigger an update.

Why this form: the cosine similarity threshold on RGB patches is a lightweight proxy for "has the geometry changed in this image region." The intuition is that depth changes are almost always accompanied by visible appearance changes (an object moved, an occlusion changed), so regions with near-identical RGB patches can safely reuse their previous depth. The strict threshold (0.996) errs on the side of recomputing depth rather than risking stale depth codes, which is appropriate because using incorrect depth is worse than incurring extra computation. The patch-based comparison (32×32 regions) matches the spatial resolution of the depth grid (10×10 over 320×320 = 32×32 per cell), making the mapping between RGB patches and depth cells exact.

The model is supervised on the buffer codes $\mathbf{b}_t$, not the full codes $\mathbf{d}_t$, matching the representation it will maintain during adaptive inference. For the first frame of each episode, all cells are marked updated and $\mathbf{b}_1 = \mathbf{d}_1$.

Training MolmoAct2-Think

MolmoAct2-Think starts from the same 200K-step MolmoAct2-Pretrain checkpoint as standard MolmoAct2 post-training. It adds depth-related special tokens to the vocabulary: <depth_output>, <depth_start>, <depth_end>, and <depth_0> through <depth_127> for the 128 codebook values.

During post-training, robot data uniformly samples three output styles:

1. Action-only: the standard action prediction task (identical to MolmoAct2 post-training).
2. Depth-only: the model predicts the 100 depth-buffer tokens autoregressively given the current observation, learning the depth-geometry mapping.
3. Depth-and-action: the model first predicts depth tokens, then predicts discrete action tokens autoregressively, while the action expert conditions on the input context plus predicted depth tokens to generate continuous actions. The target action-token span is masked from the expert conditioning path (as in standard post-training), so the expert uses task, observations, state, and depth tokens, but not the ground-truth discrete action targets.

All three styles use the same data and the same 100K training updates on 64 H100 GPUs with global batch size 128.

Fine-tuning differences for MolmoAct2-Think. During embodiment-specific fine-tuning (e.g., for LIBERO), the paper makes three targeted changes:

1. Style sampling: only action and depth-and-action examples are used (uniformly sampled), removing pure depth-prediction examples. At deployment, the model always uses the depth-and-action style, so training should focus on this mode.

2. Depth-token noise injection: because inference conditions on predicted depth tokens (which may contain errors), the training pipeline injects noise into the teacher-forced depth prefix—10% of depth-code input tokens are randomly replaced with uniformly sampled depth codes, while the prediction targets remain the correct codes. This teaches the model to be robust to imperfect depth predictions, preventing a train-test mismatch where the model learns to rely on perfect depth codes that it cannot produce at inference.

3. Per-layer depth gate: a learned mechanism that controls how strongly each action-expert layer uses the depth prefix. For action-expert layer $\ell$, let $\mathbf{M}_t = 1$ denote positions belonging to depth tokens (output trigger, delimiters, depth codes), and let $\mathbf{A}_t$ denote valid context positions. The gate is computed from the non-depth context of the corresponding VLM layer:

$\mathbf{c}_\ell = \frac{\sum_t \mathbf{A}_t (1 - \mathbf{M}_t) \mathbf{V}^{\text{vlm}}_{\ell, t}}{\sum_t \mathbf{A}_t (1 - \mathbf{M}_t)}$

$g_\ell = \sigma(\mathbf{w}_\ell^\top \mathbf{c}_\ell + b_\ell)$

where $\mathbf{c}_\ell$ is the mean value vector over all non-depth context tokens at VLM layer $\ell$, $\mathbf{w}_\ell$ and $b_\ell$ are learned parameters, and $\sigma$ is the sigmoid function (output in [0, 1]).

The gate is then applied to depth-token keys and values before they enter the expert:

$\bar{\mathbf{K}}^{\text{vlm}}_{\ell, t} = (1 - \mathbf{M}_t + \mathbf{M}_t g_\ell) \mathbf{K}^{\text{vlm}}_{\ell, t}$

$\bar{\mathbf{V}}^{\text{vlm}}_{\ell, t} = (1 - \mathbf{M}_t + \mathbf{M}_t g_\ell) \mathbf{V}^{\text{vlm}}_{\ell, t}$

What it computes: for non-depth tokens ($\mathbf{M}_t = 0$), the key/value is unchanged (multiplied by 1). For depth tokens ($\mathbf{M}_t = 1$), the key/value is scaled by $g_\ell \in [0, 1]$, where $g_\ell$ is computed from the mean non-depth context—essentially, the model learns from the non-depth information (image, text, state) how much it should trust the depth tokens at each layer.

Why this form: the gate provides a learned mechanism for the model to modulate its reliance on depth information. Depth predictions may be useful for some aspects of action generation (e.g., obstacle avoidance, free-space reasoning) but irrelevant or noisy for others (e.g., grasp force, precise joint angles). The per-layer gate lets different expert layers use depth to different degrees. The bias is initialized to $-4$, giving $g_\ell \approx \sigma(-4) \approx 0.018$ at initialization—the model starts nearly ignoring depth and gradually learns to incorporate it. This prevents the depth pathway from dominating training early when depth predictions are poor.

The ablation (Table 14) confirms the importance of these fine-tuning choices. Removing noise injection and the depth gate reduces average LIBERO performance from 98.10% to 97.65%, primarily through a 1.8-point drop on Goal. Removing mixed training as well (using only depth-and-action examples) reduces average to 97.50%. The depth pathway is most useful when the model is regularized for imperfect inference-time depth and retains a strong action-only path.

Adaptive Depth Inference

At inference, MolmoAct2-Think uses only the depth-and-action output style. The pipeline proceeds as follows:

1. Prefill: the VLM processes the prompt (task, images, state tokens) and produces KV caches for all context tokens through a single forward pass.

2. Depth prediction (adaptive):

   • If no depth cache exists (start of rollout or after reset), the model autoregressively predicts all 100 depth tokens: <depth_start>, 100 code tokens, <depth_end>.
   • If a depth cache exists, the current first observation image is compared to the cached previous image using the same 10×10 RGB-patch cosine threshold (0.996) described above. Updated cells are generated by argmax decoding from depth-token logits; unchanged cells are replayed from the previous predicted depth buffer. Consecutive unchanged spans are replayed together (a single operation), while changed spans are decoded token by token. After all 100 cells are filled, the model emits <depth_end> and stores the current image and newly filled depth buffer as the cache for the next step.

3. Action generation: the action expert conditions on the VLM keys and values for the prompt plus the filled depth prefix, and runs 10 Euler flow-matching steps to generate the continuous action chunk. The action interface is identical to the depth-and-action training objective.

What it computes: a selective recomputation strategy that exploits temporal coherence. In a typical tabletop manipulation scene, perhaps 10–30% of the depth grid cells change between consecutive frames (the robot arm and manipulated objects), while 70–90% remain static (background, table, stationary objects). Adaptive depth re-predicts only the changed cells, reducing the autoregressive decoding time from 100 tokens to approximately 10–30 tokens in typical scenes.

Why this form: the efficiency gain comes from two sources. First, autoregressive decoding of the large VLM (4B parameters) is the inference bottleneck—each new token requires a full forward pass through the LLM. Reducing 100 tokens to 10–30 tokens proportionally reduces this cost. Second, unchanged spans can be replayed in bulk rather than token-by-token, further reducing the number of decoding steps. The tradeoff is that adaptive depth requires computing RGB patch similarities and maintaining a depth cache, but these operations are extremely cheap relative to VLM decoding.

The inference systems challenge is that "each frame can contain a different mixture of regenerated depth cells and replayed cached cells, so the overall decode schedule is data-dependent." Capturing the full adaptive loop in a CUDA Graph (which requires fixed-shape computation) would require a separate graph for each possible update pattern. The paper instead keeps the adaptive scheduler eager (Python-level control flow) for the variable part, while using CUDA Graphs for the fixed-shape transformer work (post-attention through next layer's pre-attention) during regenerated depth token decoding. Attention itself remains eager because its effective KV length changes throughout decoding.

The speedup numbers in Section 6.8 quantify the practical impact: MolmoAct2-Think with CUDA Graphs achieves 12.71 Hz, a 1.58× speedup over the original 8.04 Hz. This is more modest than the 2.42× speedup of standard MolmoAct2 (23.02 Hz → 55.79 Hz) because MolmoAct2's fixed-shape flow-matching steps benefit more from graph capture, while Think's adaptive autoregressive depth stage remains partially eager.

---

Embodiment-Specific Fine-Tuning and Inference Optimization

Fine-Tuning Recipe

Embodiment-specific fine-tuning starts from the MolmoAct2-Post checkpoint and keeps the same VLM–action-expert architecture. The recipe differs from post-training in four specific ways, each justified empirically:

1. Robot-only data: no multimodal VLM mixture or separate VLM dataloader. The model is being specialized to a specific robot and task distribution; general vision-language data would interfere with this specialization.

2. Increased flow samples: $K = 8$ instead of $K = 4$, providing denser supervision per action chunk. Fine-tuning datasets are smaller (hundreds to thousands of demonstrations), so extracting more training signal per sample is valuable.

3. No knowledge insulation: gradients from the flow loss propagate through the VLM conditioning path. The paper states this choice is empirical—no consistent gain from detaching—and reflects that by fine-tuning, the VLM representations are stable enough to benefit from flow-loss gradients rather than being destabilized by them.

4. No added-token embedding tuning: the token embeddings for special robot tokens (action tokens, state tokens, depth tokens) are frozen, assuming they are "already well placed" from pre-training and post-training. Only the language-model output head and final normalization are updated for these tokens. This is standard practice in VLM fine-tuning to prevent catastrophic forgetting of token semantics.

Inference Optimization

For standard MolmoAct2, inference is dominated by the continuous action expert's flow-matching integration. Within one action chunk, the VLM context is invariant across flow steps—only the noisy action state and flow time change. The optimization strategy therefore caches reusable action-expert intermediates and uses CUDA Graphs.

Reusable intermediates. Three categories are cached: (a) context-dependent cross-attention states (the VLM keys and values after projection by $\mathbf{P}_K$, $\mathbf{P}_V$—these are the same for all flow steps), (b) fixed position-dependent terms (positional embeddings for the action sequence, which don't change across steps), and (c) the VLM KV cache itself (computed once during prefill and reused). This means that each flow step only needs to recompute: the noisy action embedding (which changes), the time embedding (which changes), and the action self-attention and cross-attention using cached keys/values. The VLM does zero additional computation during flow integration.

CUDA Graphs. MolmoAct2's flow-matching steps form a regular, repeated pattern: 10 iterations of the same computation with different noisy action inputs and time embeddings. The fixed-shape nature of this loop (same tensor shapes every iteration, same sequence of operations) makes it ideal for CUDA Graph capture. The graph records the GPU operations for one flow step and replays them for subsequent steps, eliminating Python overhead and kernel-launch latency. The 2.42× speedup (23.02 Hz → 55.79 Hz) reported in Section 6.8 comes predominantly from this optimization, since the per-step compute is relatively small and kernel-launch overhead dominates.

For MolmoAct2-Think, the adaptive depth stage performs autoregressive decoding with variable-length execution, which is less amenable to graph capture. The paper captures only the fixed-shape transformer work from post-attention through the next layer's pre-attention as CUDA Graph stages, while keeping attention eager (since KV length changes) and the adaptive scheduler eager (since the update pattern varies). The resulting 1.58× speedup is more modest but still meaningful for real-time control.

4. Key Insights and Innovations

Innovation 1: The Three-Stage Training Pipeline as a Decomposition of Optimization Difficulty

The paper's most consequential architectural contribution is not any single component but the decomposition strategy itself—the deliberate separation of VLA training into pre-training (discrete action tokens only), post-training (attach continuous expert with knowledge insulation), and fine-tuning (remove insulation, specialize to embodiment). This reframes VLA training from a monolithic optimization problem into three sequentially easier sub-problems, each building on the stable representations learned in the previous stage.

What the field did before. Prior VLA systems largely fell into two camps. The first camp—including RT-2 (Brohan et al., 2022), OpenVLA (Kim et al., 2024), and MolmoAct (Lee et al., 2025)—used purely discrete action tokenization throughout, avoiding the continuous control problem entirely at the cost of precision and smoothness. The second camp—including π0 (Black et al., 2024) and π0.5 (Intelligence et al., 2025)—trained continuous action experts jointly with the VLM from the start, which the paper argues "makes optimization unnecessarily difficult: the model must simultaneously learn a robot-aware token interface, align new state/action embeddings, and fit a flow-matching controller." GR00T N1 (NVIDIA et al., 2025) conditioned its action expert on final hidden states, a simpler interface but one that discards hierarchical information. No prior system had systematically separated the learning to tokenize actions problem from the learning to produce continuous trajectories problem while maintaining a modular interface between them.

Why this is fundamental rather than incremental. This is not merely a training schedule tweak—it is a conceptual claim about the nature of the optimization landscape. The paper argues implicitly that discrete action token prediction and continuous flow matching have incompatible early-training dynamics: the former benefits from stable VLM representations (since action tokens are just new vocabulary items in a familiar next-token objective), while the latter would destabilize those same representations if trained jointly from scratch (since flow-matching gradients through the VLM would compete with the language-modeling objective). The three-stage design resolves this by establishing the VLM's robot-awareness first (Stage 1), then attaching the expert without backpropagating through the VLM (Stage 2, via knowledge insulation), and only then allowing joint optimization when representations are mature (Stage 3, removing insulation during fine-tuning).

Evidence that supports the claim. The ablation in Table 10 shows that the embodied-reasoning backbone alone (Molmo2-ER with discrete actions, no continuous expert) improves LIBERO-Long from 77.6% to 83.6%—a 6-point gain from the first stage alone. Table 11 shows that the choice of per-layer KV conditioning (Stage 2's architectural choice) matters substantially: 95.9% average versus 94.0% for hidden-state conditioning. Table 13's fine-tuning ablation confirms that removing discrete co-training and knowledge insulation shifts performance in task-dependent ways (Long improves but Spatial Object degrades), validating that the three stages serve different purposes. The overall trajectory—from 83.6% (discrete-only) to 97.2% (full MolmoAct2) to 98.1% (with Think)—demonstrates that each stage adds non-overlapping capability.

The broader implication. This decomposition suggests a general principle for building VLAs: separate the representation alignment problem (teaching the VLM what robot actions look like in its native token space) from the control precision problem (teaching a dedicated expert to produce smooth trajectories). The three-stage recipe is reusable: any VLM backbone can be adapted via Stage 1 (discrete action pre-training), then upgraded with an embodiment-specific continuous expert in Stage 2 (post-training), and finally specialized in Stage 3 (fine-tuning). This modularity is what makes the system "fully open" in a practical sense—practitioners can swap the backbone, add new embodiments at Stage 2, or fine-tune for novel tasks at Stage 3 without retraining the entire pipeline.

---

Innovation 2: Per-Layer KV Conditioning as Access to Hierarchical Visual-Semantic State

The paper's second distinctive contribution is the architectural claim that a continuous action expert should condition on the VLM's full-depth attention state rather than its compressed final output. The mechanism—per-layer KV conditioning—is described in Section 3. What matters here is why this design represents a conceptual shift in how VLAs integrate vision-language understanding with continuous control.

What the field did before. The dominant design pattern in VLA architectures conditioned continuous action experts on the VLM's final hidden states. GR00T N1 used final-layer features as conditioning input. π0 and π0.5 used a similar approach, where the action expert receives a single summary vector representing the entire visual-language context. Some systems avoided the problem entirely by using purely discrete actions, eliminating the continuous expert altogether. The implicit assumption was that the VLM's final layer contains all task-relevant information, compressed into a single residual-stream representation—and that this compression is lossless enough for precise continuous control.

Why the per-layer design is a conceptual reframing. MolmoAct2 challenges this assumption by arguing that different layers encode different types of information relevant to different aspects of action generation. Early VLM layers encode low-level visual features—edges, textures, spatial frequencies—that matter for fine-grained manipulation (e.g., precisely aligning a peg with a hole). Middle layers encode object-level semantics and spatial relationships—what objects are present, where they are relative to each other—that matter for task planning (e.g., deciding which object to grasp). Late layers encode task-level reasoning and language grounding—what the instruction means, what sub-goals remain—that matter for long-horizon coordination. Collapsing all these into a single final vector forces the action expert to disentangle them from a lossy summary, while per-layer conditioning gives the expert direct access to each level of abstraction at the corresponding depth.

This is not merely an architectural detail—it is a claim about where in the VLM the information needed for continuous control resides. The ablation evidence (Table 11) supports this claim with specificity: hidden-state conditioning is competitive on LIBERO-Spatial (97.0% vs. 96.2% for per-layer KV), suggesting spatial reasoning is well-captured in final representations, but falls substantially behind on LIBERO-Object (95.4% vs. 99.0%) and LIBERO-Long (87.0% vs. 89.8%), suggesting that object-centric manipulation and long-horizon coordination require intermediate-layer information that final states discard.

Comparison to cross-attention alternatives. The paper also tests a "per-head per-layer" variant that preserves the VLM's head structure during projection (Table 11). This variant underperforms the standard per-layer design (94.8% vs. 95.9%), which is itself informative: the standard variant flattens keys and values across heads before learned projections remap them into the expert's attention space. This gives the adapter more flexibility to recombine information from different VLM attention heads—essentially, the adapter learns which VLM heads carry information relevant to each expert head. The per-head variant constrains this recombination, forcing expert heads to inherit the VLM's head partitioning. The fact that relaxing this constraint improves performance suggests that the VLM's head structure is optimized for language and vision tasks, not for robotics, and the adapter benefits from the freedom to reorganize that structure for control.

Significance beyond raw performance. The 1.9-point average gain from per-layer KV over hidden-state conditioning (95.9% vs. 94.0%) is solid but not transformative. The deeper significance is that this design establishes a modular interface between the VLM and the action expert: the VLM can be trained, replaced, or upgraded independently, as long as it exposes per-layer keys and values. This modularity is what enables the three-stage training decomposition to work—the action expert can be attached and detached without modifying the backbone. It also creates a clean research path: improving the VLM backbone (as in Innovation 3 below) automatically improves the expert's conditioning signal, without any architectural changes to the expert itself.

---

Innovation 3: Specialized Embodied Reasoning Pretraining as a Transferable Capability for Action Models

The paper demonstrates that specialized spatial and embodied reasoning training for the VLM backbone—producing Molmo2-ER—transfers directly to action-learning performance, even before any continuous action expert is attached. This is a claim about what kind of pretraining matters for robotics, and it challenges the prevailing assumption that general-purpose vision-language pretraining is sufficient.

What the field did before. Most open VLA systems initialized from general-purpose VLMs: OpenVLA used a Llama-based backbone pretrained on web data, MolmoAct used the general Molmo VLM, and π0 used a proprietary VLM with unspecified pretraining. The implicit assumption was that web-scale pretraining provides sufficient visual and linguistic competence, and that robot-specific skills (spatial reasoning, metric depth understanding, cross-view correspondence) would be acquired during robot-specific fine-tuning. The alternative—training a specialized embodied-reasoning VLM from scratch—was largely unexplored in the open-source VLA literature, in part because the training data and recipes for such specialization were not publicly available.

The conceptual move. Molmo2-ER is not just a stronger VLM—it is a VLM trained on a different distribution of skills. The 3.3M-sample embodied-reasoning corpus (Table 1) deliberately covers capabilities that general VLMs rarely encounter during web-scale pretraining: metric distance estimation, free-space reasoning, cross-view object tracking, pixel-accurate pointing, and video-based temporal reasoning. The paper's claim is that these skills are causal prerequisites for action learning—that a model which cannot point to objects, estimate distances, or track objects across views will learn actions more slowly and less accurately, because it must simultaneously acquire these spatial competencies while learning to map them to motor commands.

Evidence for transfer. The most compelling evidence is Table 10: replacing the general Molmo2 backbone with Molmo2-ER, using only discrete action prediction and no continuous expert, improves LIBERO-Long from 77.6% to 83.6%. This is a 6-point gain from the backbone alone, with no architectural changes to the action prediction pipeline. The improvement comes purely from better visual and spatial representations—the model can more accurately identify objects, understand their spatial relationships, and predict appropriate actions because it sees the scene in richer geometric terms.

The full VLM benchmark results (Table 3) contextualize this: Molmo2-ER achieves 63.8% average across 13 embodied-reasoning benchmarks, a 17-point improvement over Molmo2's 46.8%, and surpasses GPT-5 (57.9%) and Gemini Robotics ER-1.5 Thinking (61.3%). The specific benchmarks where gains are largest are informative: RoboSpatial-Point (13.9% → 32.0%), Where2Place (18.0% → 54.0%), and VSI-Bench (26.1% → 74.5%) all measure spatial reasoning capabilities—exactly the skills needed for robot manipulation. The improvement on BLINK (50.8% → 72.5%) measures fine-grained visual perception.

The broader implication for VLA research. This finding suggests that backbone quality is a first-order determinant of VLA performance, not merely a scaling detail. The field's focus on action architectures (discrete vs. continuous, diffusion vs. flow matching, conditioning interfaces) may be over-indexed relative to the importance of the visual representations feeding those architectures. A 6-point gain from backbone improvement (Table 10) is larger than many architectural innovations achieve, and it requires no changes to the downstream pipeline. This implies that investment in embodied-reasoning pretraining—curating spatial reasoning datasets, designing specialize-then-rehearse recipes, and releasing open backbones—may yield higher returns than further architectural refinements to action experts.

---

Innovation 4: Adaptive Spatial Reasoning as a Practical Solution to the Latency-Grounding Tradeoff

The paper identifies and partially resolves a tension that has constrained reasoning-augmented VLAs: spatial reasoning before acting improves action quality, but the computation required for that reasoning destroys closed-loop reactivity. MolmoAct2-Think's adaptive depth prediction reframes this from an irreconcilable tradeoff into an efficiency problem solvable through temporal redundancy exploitation.

The tension as the field understood it. Prior reasoning-augmented VLAs demonstrated that intermediate spatial representations—depth maps (MolmoAct), point trajectories (ThinkAct), goal images (CoT-VLA), or full world-model rollouts (Cosmos Policy)—improve action quality and interpretability. However, each of these approaches regenerated the full reasoning output at every control step, creating a linear relationship between reasoning fidelity and latency: more detailed reasoning meant more tokens to generate, which meant slower control rates. MolmoAct, the paper's direct predecessor, predicted 100 depth tokens per step, requiring a full autoregressive decode through a 4B-parameter LLM before any action could be emitted. The paper frames this as a failure of efficiency, not of concept: "The very mechanism intended to make policies more reliable thus renders them too slow for closed-loop control."

The conceptual reframing. MolmoAct2-Think's key insight is that the cost of spatial reasoning should scale with scene change, not with scene complexity. In a typical manipulation task with a fixed camera, most of the scene is static: the table, background, and stationary objects occupy 70–90% of the depth grid and do not change between consecutive frames. Prior systems paid the full reasoning cost for every cell at every step, effectively recomputing answers they already knew. Adaptive depth reasoning pays only for cells where the RGB evidence has changed (cosine similarity < 0.996), which typically amounts to 10–30% of the grid—the robot arm, the manipulated object, and their immediate vicinity. This is not a new reasoning mechanism but a new resource allocation strategy for an existing mechanism.

Why this reframing matters beyond MolmoAct2. The principle of adaptive reasoning—recompute only what has changed, cache and replay what hasn't—generalizes beyond depth tokens. Any intermediate representation that is spatially or temporally structured (point trajectories, object bounding boxes, segmentation masks, future image predictions) can be partially cached and selectively recomputed. The depth case is particularly clean because the 10×10 grid provides a natural spatial decomposition with a cheap RGB-comparison heuristic for detecting change, but the same idea could apply to denser representations with appropriate similarity metrics.

The performance-latency tradeoff in evidence. The results show that adaptive depth reasoning improves both axes simultaneously: MolmoAct2-Think achieves 98.1% average LIBERO (vs. 97.2% for MolmoAct2), with the gains concentrated on the hardest suite (+2.2% on Long, Table 8), while running at 12.71 Hz (vs. 8.04 Hz for the original dense implementation, Section 6.8). The latency improvement (1.58×) is more modest than the 2.42× achieved for standard MolmoAct2 because autoregressive depth decoding cannot be fully graph-captured, but it is sufficient to move from "too slow for real-time control" (~8 Hz) to "marginally acceptable for many manipulation tasks" (~13 Hz). The finding that adaptive depth improves performance (not just maintains it at lower cost) is notable—the paper attributes this to reduced overfitting from dense full-grid prediction, but does not deeply investigate the mechanism.

Limitations that qualify the contribution. The adaptive depth approach requires monocular depth estimation (Depth Anything V2) for training data preparation, which adds a preprocessing cost and ties the quality of depth supervision to the quality of the depth estimator. It also requires maintaining a depth cache and computing RGB patch similarities at inference—cheap operations relative to VLM decoding, but additional system complexity. Most importantly, the 12.71 Hz control rate, while improved, is still below the 30 Hz that many robot control loops require for reactive manipulation. The paper's approach partially resolves the latency-grounding tradeoff but does not eliminate it; further speedups would require either a smaller backbone or more aggressive model compression.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The evaluation spans 7 benchmarks across simulation and the real world. Simulation benchmarks include LIBERO (Liu et al., 2023) with four suites—Spatial, Object, Goal, and Long—each containing 10 tasks and 500 demonstrations; RoboEval (Wang et al., 2025b) with 8 bimanual Franka tasks; MolmoSpaces (Kim et al., 2026b) and MolmoBot (Deshpande et al., 2026), both targeting single-cycle pick-and-place tasks in the DROID setup. Real-world evaluations use the bimanual YAM setup (8 tasks across static lab, in-the-wild, and mobile settings, 50 trials each), the DROID Franka setup (5 tasks, 15 trials each), and the SO-100/101 setup (5 tasks, 15 trials each). The embodied reasoning benchmarks for the backbone comprise 13 established VLM benchmarks including Point-Bench, RefSpatial, RoboSpatial, Where2Place, BLINK, CV-Bench, ERQA, EmbSpatial, MindCube, SAT, OpenEQA, and VSI-Bench.

• Base model(s). All VLA experiments use the Molmo2-ER backbone, a 4B-parameter VLM specialized for embodied and spatial reasoning, trained via the specialize-then-rehearse recipe described in Section 2. For the FLOPs-matched backbone study, the comparison spans multiple model families: Molmo2-ER vs. Molmo2 (both 4B, same architecture), GPT-5 and GPT-5-mini (proprietary, scales undisclosed), Gemini Robotics ER-1.5 and ER-1.5 Thinking (proprietary), and several open-weight VLMs at 4B–8B scale including Qwen3-VL, InternVL3.5, and LLaVA-OneVision. The VLA baselines span π0 (Black et al., 2024), π0.5 (Intelligence et al., 2025), GR00T N1.7 (NVIDIA et al., 2025), OpenVLA (Kim et al., 2024), and MolmoAct-7B-D (Lee et al., 2025), among others.

• Metrics. The primary metric is task success rate (%), computed as the fraction of evaluation rollouts where the task's completion criteria are met. For RoboEval, additional trajectory-quality metrics are reported: completion time (s), trajectory length (steps), joint path length (rad), Cartesian path length (m), Cartesian and joint jerk, self-collision count, environment collision count, slip count, and bimanual coordination metrics (vertical end-effector height discrepancy ∆z, inter-arm velocity divergence ∆v). For the partial-credit SO-100/101 tasks, scoring uses milestones: 0.25 for reaching, 0.5 for pickup, 1.0 for successful placement. For LIBERO pick-and-place tasks, both oracle success (success conditions fulfilled at any timestep) and success at end (conditions fulfilled at final timestep) are reported. For embodied reasoning benchmarks, accuracy is computed as the fraction of correct answers under each benchmark's standard evaluation protocol. Inference latency is measured as end-to-end action-generation time on a single H100 GPU, and control rate is reported as action horizon divided by this latency (Hz).

• Baselines. For embodied reasoning (Section 6.1): Gemini Robotics ER-1.5 and ER-1.5 Thinking (Team et al., 2025b), Gemini 2.5 Pro (Comanici et al., 2025), GPT-5 and GPT-5-mini (Singh et al., 2025), Qwen3-VL-4B and Qwen3-VL-8B (Yang et al., 2025a), LLaVA-OV-7B (Li et al., 2024a), InternVL3.5-4B and InternVL3.5-8B (Wang et al., 2025a), and Molmo2 (Clark et al., 2026). For out-of-the-box deployment (Section 6.2): π0-DROID (Black et al., 2024), π0.5-DROID (Intelligence et al., 2025), StereoVLA (Deng et al., 2025), LAP-VLA (Zha et al., 2026), X-VLA (Zheng et al., 2025), MolmoBot (Deshpande et al., 2026), SmolVLA (Shukor et al., 2025), and π0-SO100/101 (a π0 variant fine-tuned by the authors on the SO-100/101 dataset). For fine-tuning evaluation (Section 6.3): on LIBERO—TraceVLA (Zheng et al., 2024), OpenVLA (Kim et al., 2024), SpatialVLA (Qu et al., 2025), CoT-VLA (Zhao et al., 2025), π0, ThinkAct (Huang et al., 2025), MolmoAct-7B-D, GR00T N1.7, π0.5, and NORA-1.5 (Hung et al., 2025); on RoboEval—Diffusion Policy, GR00T N1.5, X-VLA, and π0.5-DROID; on real-world bimanual YAM—Cosmos Policy (Kim et al., 2026a), X-VLA (Zheng et al., 2025), OpenVLA-OFT (Kim et al., 2025), and π0.5-DROID.

• Generation budget / compute accounting. For VLA experiments, compute is measured in training updates (50K–100K steps) and GPU hours, with batch sizes and hardware specified per experiment (e.g., 64 H100 GPUs for post-training, 32 H100 GPUs for most fine-tuning runs). For the backbone embodied reasoning benchmarks, each model is evaluated under its standard inference protocol with no additional test-time computation—all results reflect a single inference pass per question. For inference speed measurements (Section 6.8), latency is measured on a single H100 GPU at batch size 1, with the control rate computed as action horizon divided by end-to-end generation time.

• Cross-validation / statistical protocol. For simulation benchmarks (LIBERO, RoboEval, MolmoSpaces, MolmoBot), each task is evaluated over the standard number of rollouts specified by the benchmark: 1000 episodes per LIBERO task, 500–1000 per RoboEval task (exact count varies by task with standard errors reported), and protocol-specified counts for MolmoSpaces and MolmoBot. For real-world evaluations, each task uses 15–50 trials; for the bimanual YAM tasks, each is evaluated under three spatial variants with 50 total trials per task. For the robustness study, 4 tasks × 4 perturbation types × 5 trials per perturbation = 80 total trials. Standard errors and 95% confidence interval half-widths are reported for simulation results; real-world results report raw success counts and percentages. Fine-tuning checkpoints are selected based on best validation performance within the training run (e.g., LIBERO checkpoint at 40K updates within a 50K-update run; MolmoAct2-Think-LIBERO checkpoint at 30K updates).

Main Quantitative Results

Embodied Reasoning Backbone (Section 6.1)

Molmo2-ER achieves the highest overall average of 63.8% across 13 embodied reasoning benchmarks, outperforming all open-weight and proprietary baselines (Table 3). The runner-up, Gemini Robotics ER-1.5 Thinking, averages 61.3%—a 2.5-point gap. The improvement over the base Molmo2 model is 17 points (46.8% → 63.8%), demonstrating the substantial gains from specialized embodied pretraining.

The win distribution is broad: Molmo2-ER achieves the best score on 9 of 13 benchmarks. The strongest proprietary model, GPT-5, averages 57.9% across all benchmarks. Gemini 2.5 Pro achieves 57.1% and Gemini Robotics ER-1.5 (non-Thinking) 55.0%. Open-weight alternatives trail with Qwen3-VL-8B at 61.0%, Qwen3-VL-4B at 59.0%, and InternVL3.5-8B at 52.4%.

The benchmarks where Molmo2-ER achieves the largest absolute gains over Molmo2 merit attention: RoboSpatial-Point (13.9% → 32.0%, +18.1 points), Where2Place (18.0% → 54.0%, +36 points), BLINK (50.8% → 72.5%, +21.7 points), and VSI-Bench (26.1% → 74.5%, +48.4 points). These benchmarks specifically target metric spatial understanding, affordance prediction, fine-grained visual perception, and spatial video reasoning—exactly the skills the 3.3M-sample embodied corpus was designed to cultivate. RoboSpatial-Point in particular shows that general-purpose VLMs near floor performance on pixel-accurate pointing tasks, and that deliberate oversampling of pointing and object detection data (780K pointing examples, 100K detection examples in the corpus) directly addresses this gap.

On 4 of 13 benchmarks, Molmo2-ER is not the best model. On Point-Bench, Qwen3-VL-8B edges Molmo2-ER 64.2% to 77.3% (Molmo2-ER actually wins at 77.3% vs. Qwen's 64.2%—correction: Molmo2-ER wins Point-Bench 77.3% vs. Qwen3-VL-4B's 63.8%). On RefSpatial, Molmo2-ER ties Qwen3-VL-4B at 52.5%. On OpenEQA, GPT-5 leads at 64.4% vs. Molmo2-ER's 44.7%. On CV-Bench, GPT-5-mini scores 85.9% vs. Molmo2-ER's 87.8% (Molmo2-ER wins). The OpenEQA gap is notable—it is an embodied question-answering benchmark requiring both spatial and semantic scene understanding, and Molmo2-ER's 44.7% vs. GPT-5's 64.4% suggests that web-scale pretraining on diverse semantic knowledge still provides advantages for certain types of scene reasoning that the 3.3M-sample embodied corpus does not fully replicate.

Out-of-the-Box Deployment (Section 6.2)

Simulation: MolmoSpaces. MolmoAct2-DROID achieves an average success rate of 37.7% across four manipulation skill categories (Pick, Pick & Place, Open, Close), surpassing the strongest baseline π0.5-DROID (34.5%) by +3.2 points (Table 4). The gains are concentrated on contact-rich multi-stage tasks: Pick improves from 36.4% to 43.7% (+7.3), Pick & Place from 13.6% to 26.7% (+13.1). On Close, MolmoAct2-DROID also sets a new best at 70.8% (π0.5-DROID: 65.1%, +5.7). On Open, MolmoAct2-DROID trails at 9.5%, behind π0.5-DROID (22.7%) and LAP-VLA (11.4%)—the paper notes this suggests articulated-object interaction remains a direction for improvement.

Simulation: MolmoBot. MolmoAct2-DROID achieves 87.1% average success across five tasks (Table 6), a +38.7-point absolute gain over the runner-up MolmoBot (48.4%) and +41.9 over π0.5-DROID (45.2%). On individual tasks, the margins are dramatic: Knife in Box (93.3% vs. 40.0% vs. 26.7%), Red Cube in Tape Roll (93.3% vs. 33.3% vs. 53.3%), Pipette in Tray (86.7% vs. 53.3% vs. 33.3%). Apple on Plate reaches 100% (vs. 86.7% and 66.7%). Objects in Bowl—the only partial-credit task—achieves 62.0% vs. 28.6% and 46.2%.

Simulation: Held-out environments (Table 5). On seven held-out environment configurations, MolmoAct2-DROID averages 20.6% success, doubling π0.5-DROID's 10.0% and substantially exceeding LAP-VLA (4.8%), X-VLA (1.2%), and StereoVLA (limited evaluation scope). The gap between oracle success (first number) and success at end (second number) in pick-and-place tasks reveals interesting failure modes: for Pick & Place Next-To, MolmoAct2-DROID achieves 20.9% oracle vs. 14.4% at end—a 6.5-point differential indicating that placement stability and task-completion detection remain challenges even when the grasp and transport succeed.

Real-world: DROID Franka zero-shot. Under challenging out-of-distribution conditions (random camera poses, unseen objects, novel environments), MolmoAct2-DROID achieves 87.1% average success across 5 tasks (Table 18, 6)—figures need reconciliation with Table 6 which reports the same 87.1% for MolmoBot benchmark. On the DROID real-world tasks (Table 18): MolmoAct2 achieves 100% on Apple on Plate, 86.7% on Pipette in Tray and Red Cube in Tape Roll, 93.3% on Knife in Box, and 62.0% (partial credit) on Objects in Bowl. π0.5-DROID trails substantially: 66.7%, 33.3%, 53.3%, 26.7%, and 46.2% respectively. MolmoBot is competitive only on Apple on Plate (86.7%) but falls to 53.3% or below on all other tasks.

Real-world: SO-100/101 zero-shot. MolmoAct2-SO100/101 achieves 56.7% average success (partial credit) across 5 tasks, exceeding π0-SO100/101 (45.3%) by +11.4 points and dramatically outperforming SmolVLA (2.3%) (Table 7). The per-task pattern reveals mixed strengths: MolmoAct2 leads on Fork on Plate (70.0% vs. 30.0% vs. 3.3%), Tissues in Basket (73.3% vs. 20.0% vs. 0.0%), and Stack Blocks (20.0% vs. 6.7% vs. 5.0%). However, on Block in Box, MolmoAct2 scores only 33.3% vs. π0-SO100/101's 90.0%—a 56.7-point deficit that the paper does not discuss. This single-task reversal suggests that MolmoAct2's pre-training mixture may not cover certain action primitives or object interactions as thoroughly as the π0 variant fine-tuned specifically on SO-100/101 data.

Effective Fine-Tuning (Section 6.3)

LIBERO. MolmoAct2 achieves 97.2% average success across the four LIBERO suites, the highest among all compared methods (Table 8). The per-suite breakdown: Spatial 97.8%, Object 100.0%, Goal 97.8%, Long 93.2%. The closest competitors are π0.5 at 96.9% (98.8%, 98.2%, 98.0%, 92.4%) and GR00T N1.7 at 97.0% (97.7%, 97.5%, 98.5%, 94.4%). MolmoAct2's advantage comes primarily from Object (100% vs. 98.2% for π0.5) and Long (93.2% vs. 92.4% for π0.5), with GR00T N1.7 actually leading on Goal (98.5% vs. 97.8%) and Long (94.4% vs. 93.2%). The 10.6-point improvement over the prior MolmoAct-7B-D (97.2% vs. 86.6%) represents the cumulative effect of all five axes of improvement from Section 1.

MolmoAct2-Think further improves to 98.1% average, with gains concentrated on Long (93.2% → 95.4%, +2.2) and more modest improvements on Spatial (97.8% → 98.8%) and Goal (97.8% → 98.5%), while Object is essentially saturated (100.0% → 99.8%). The paper notes this pattern as evidence that "the adaptive-depth pipeline yields a real, non-incidental gain rather than noise around saturation"—the largest improvement appears on the hardest suite where the baseline leaves the most headroom.

RoboEval. MolmoAct2 achieves 44.3% average success rate across 8 tasks, surpassing π0.5-DROID (40.5%?—exact figure not in Table 17; the abstract mentions 3.8% over the second-best model which is π0.5) by 3.8 percentage points (Table 17, Figure 6A). The per-task results show particularly strong performance on long-horizon tasks: Pack Box 62% vs. π0.5 54%, Rotate Valve 72% vs. 72% (tie), Stack Two Blocks 52% vs. 40%. The radar plot (Figure 6B) shows MolmoAct2 dominating across nearly all trajectory-quality metrics—completion time, trajectory length, joint path length, Cartesian path length, Cartesian jerk, joint jerk, self-collisions, and slip count—indicating improvements not only in task success but in efficiency, stability, and smoothness.

Real-world bimanual YAM. MolmoAct2 achieves 50.6% average success across 8 real-world tasks, 15.1 points above the runner-up OpenVLA-OFT at 35.5% (Table 19). This evaluation is described as "the most extensive... of any open VLA to date" and spans "static laboratory settings, in-the-wild environments (kitchen, study room, pantry, and wet labs), and mobile manipulation." Per-task performance varies dramatically: Cup Storing reaches 100% (vs. OpenVLA-OFT 96.0%), Linearbot 64.0% (vs. 24.4%), Store Test Tube 67.0% (vs. 42.0%), and Prepare Pipette 33.0% (vs. 32.0% for Cosmos Policy, 28.5% for OpenVLA-OFT). On Cup Stacking, MolmoAct2 (54.0%) trails both Cosmos Policy (64.5%) and π0.5 (62.0%), while on Pegboard, MolmoAct2's 14.0% (vs. 4.5% for OpenVLA-OFT) shows that fine-grained insertion tasks remain challenging for all methods. The 15.1-point average margin is substantial, but the 50.6% absolute success rate underscores the paper's own framing that "success rates on realistic tasks fall well below the threshold required for dependable deployment."

MolmoAct2-Think Performance (Section 6.4)

The adaptive-depth variant improves over the standard MolmoAct2 on three of four LIBERO suites and matches on the fourth where the baseline is already at ceiling (Table 8). The average improvement is +0.9 percentage points (97.2% → 98.1%), with the gain entirely concentrated on Long-horizon tasks (+2.2). The paper interprets this concentration as evidence that adaptive depth reasoning helps most when the baseline leaves substantial headroom—the easier suites (Spatial, Object, Goal) are near saturation for all methods—and that the Long suite's multi-stage coordination particularly benefits from explicit spatial structure.

Robustness Under Distribution Shift (Section 6.5)

MolmoAct2 achieves 50.69% average success across four perturbation types (Spatial Variance, Lighting, Language Rephrasing, Distractors), 10.80 points above OpenVLA-OFT (39.89%) and 23.68 points above π0.5 (27.01%) (Table 9). The per-perturbation breakdown reveals which shifts are hardest: Spatial Variance is the most challenging for all methods, with MolmoAct2 at 26.25% (vs. 15.00% for π0.5, 13.75% for OpenVLA-OFT). This aligns with the intuition that spatial generalization—manipulating objects in positions far from training distribution—tests geometric understanding that is only indirectly supervised by action imitation. Lighting perturbation is relatively easier (MolmoAct2 62.05%), suggesting that the image augmentation pipeline (color jitter, blur) provides meaningful robustness to illumination changes. Language rephrasing (60.35%) shows that the model's instruction grounding generalizes beyond exact training phrases. Distractor objects (54.10%) reveal that the policy correctly attends to task-relevant objects even when visual clutter increases, though the 5.8-point margin over OpenVLA-OFT (48.30%) is narrower than on other perturbation types.

The absolute performance levels—even for the best model—remain far from dependable: 50.69% average means roughly half of all perturbations cause failure. For real-world deployment where perturbations are the norm rather than the exception, this represents a significant reliability gap.

Trajectory Quality (Section 6.6)

Beyond raw success rates, MolmoAct2 demonstrates consistently better trajectory quality metrics across the RoboEval benchmark (Table 17, Figure 6B). On Stack Two Blocks, MolmoAct2 reduces completion time from 5.87s (π0.5) and 7.27s (Diffusion Policy) to 4.70s, while reducing joint path length from 2.16 rad to 1.04 rad—approximately 2× shorter. On Rotate Valve, MolmoAct2 achieves the lowest completion time (8.51s vs. 9.69s for π0.5) and the lowest trajectory length (481.94 steps vs. 527.64). The radar plot (Figure 6B) shows MolmoAct2 operating near the outer edge (best normalized value) on Cartesian and joint-space jerk metrics, indicating smoother trajectories with fewer abrupt accelerations. Self-collision count and slip count are also near-minimal. These metrics collectively indicate that MolmoAct2's trajectories are not only more successful but more efficient, stable, and safer—properties that matter for real-world deployment beyond binary task completion.

Inference Speed (Section 6.8)

Under optimized CUDA Graph inference on a single H100 GPU, MolmoAct2 achieves 55.79 Hz continuous control rate—a 2.42× speedup over the original implementation at 23.02 Hz (Figure 8). Intermediate caching alone improves from 23.02 Hz to 27.39 Hz; CUDA Graphs provide the larger gain. MolmoAct2-Think reaches 12.71 Hz (1.58× speedup over 8.04 Hz original). The discrete action path runs at 14.17 Hz for standard MolmoAct2 and 6.82 Hz for Think—3.94× and 1.86× slower than the continuous path, respectively, due to the VLM decoding overhead per action token. The paper uses the continuous path as the default deployment option.

Ablation Studies and Robustness Checks

Backbone comparison: Molmo2 vs. Molmo2-ER for discrete action prediction: Replacing Molmo2 with Molmo2-ER, using only discrete action tokens and no continuous expert, improves LIBERO-Long from 77.6% to 83.6%—a 6.0-point gain (Table 10). This confirms that embodied-reasoning specialization transfers directly to action-token prediction, independent of the continuous expert architecture. Both variants are trained for 60K steps with identical architecture (MolmoAct2-FAST Tokenizer, no action expert).

VLM-to-expert conditioning source: Three conditioning methods are compared starting from the same MolmoAct2-Pretrain checkpoint (Table 11). Per-layer KV conditioning achieves 95.9% average LIBERO, outperforming per-head per-layer KV (94.8%) and hidden-state conditioning (94.0%). The gap is largest on Object (99.0% vs. 95.4% vs. 97.2%) and Long (89.8% vs. 89.6% vs. 87.0%). Hidden-state conditioning is competitive on Spatial (97.0% vs. 96.2%), suggesting spatial reasoning is well-captured in final-layer representations. The per-head variant (which preserves VLM head structure during projection) underperforms the standard variant (which flattens and then reprojects), indicating that the flexibility to recombine information from different VLM attention heads matters for control performance.

Number of flow samples per action chunk: Increasing the number of independently sampled noise-time pairs per action chunk from 1 to 8, while reusing the same VLM context, improves average LIBERO performance (Table 12). K=1 achieves 94.15%, K=2 achieves 95.05%, K=4 achieves 95.15%, and K=8 achieves 95.90%. The effect is not monotonic on every suite (e.g., K=2 achieves 92.2% on Long vs. 87.6% for K=1 and 89.8% for K=8), but the aggregate trend favors denser flow supervision. The paper uses K=4 for post-training (memory-constrained) and K=8 for fine-tuning.

Fine-tuning design choices: Table 13 ablates three axes of the final fine-tuning recipe. Removing discrete action co-training (keeping only flow-matching loss) drops the average from 97.20% to 96.95%—Long improves (95.0% vs. 93.2%) but Spatial and Object degrade. Adding knowledge insulation (detaching flow gradients from the VLM) yields similar average (97.05%) but slightly below the final recipe. Using LoRA instead of full fine-tuning achieves 96.25% average—competitive on Spatial (98.0% vs. 97.8%) and Object (99.4% vs. 100.0%) but substantially weaker on Long (90.4% vs. 93.2%). Tuning only the action expert is the clearest failure mode at 93.05% average. The final recipe—full fine-tuning with discrete and continuous co-training, no knowledge insulation, frozen added-token embeddings—provides the best overall outcome.

Depth-aware fine-tuning for MolmoAct2-Think: Removing depth-token noise injection and the per-layer depth gate reduces MolmoAct2-Think average from 98.10% to 97.65%, primarily through a 1.8-point drop on Goal (Table 14). Removing mixed training as well (using only depth-and-action examples) reduces average to 97.50%. The ablation confirms that the depth pathway benefits from regularization for imperfect inference-time depth predictions (noise injection) and from retaining a strong action-only path (mixed training) rather than making depth the sole action-conditioning signal.

Backbone benchmarks at scale: The embodied reasoning results include a scale dimension through comparison with proprietary models of undisclosed size (GPT-5, Gemini families), 8B open-weight models (Qwen3-VL-8B, InternVL3.5-8B), and 4B models (Qwen3-VL-4B, InternVL3.5-4B, Molmo2, Molmo2-ER). Molmo2-ER's 4B scale achieving 63.8% average vs. GPT-5's 57.9% and Gemini ER-1.5 Thinking's 61.3%—both at unknown but certainly larger scales—is a significant finding suggesting that deliberate spatial pretraining can compensate for raw parameter count on embodied reasoning tasks.

Critical Assessment

The experiments broadly support the paper's central claims, but several qualifications merit attention.

The claim that MolmoAct2 is "built for real-world deployment" and "deployable out-of-the-box on multiple embodiments" is supported for the three specific embodiments with large-scale training data (bimanual YAM, DROID Franka, SO-100/101). The out-of-the-box results show strong performance: 87.1% on DROID real-world tasks, 56.7% on SO-100/101. However, these results are conditional on using the exact hardware configuration the model was trained for. The paper is explicit that "deployment on robots outside this set requires effective fine-tuning," and the fine-tuning results demonstrate this works (50.6% on novel bimanual YAM tasks, 97.2% on LIBERO). But the zero-shot results are not truly "out-of-the-box" in the universal sense—they are out-of-the-box for three specific embodiments. A practitioner with a different robot (e.g., a Kinova arm, a different bimanual platform) must fine-tune, and the paper does not evaluate how many demonstrations are needed for such cross-embodiment transfer. The term "out-of-the-box" in the title and abstract overstates the generality relative to what is actually demonstrated.

The claim of "the most extensive empirical study of any open VLA to date" (covering 7 benchmarks across simulation and real-world) is well-supported. The evaluation spans 13 embodied reasoning benchmarks for the backbone, 3 simulation benchmarks for zero-shot deployment, 2 simulation benchmarks plus 1 large real-world suite for fine-tuning, separate robustness and trajectory-quality analyses, and an inference speed study. The diversity of baselines is strong: MolmoAct2 is compared against π0, π0.5, GR00T N1.7, GR00T N1.5, OpenVLA, OpenVLA-OFT, MolmoAct, MolmoBot, SmolVLA, TraceVLA, SpatialVLA, CoT-VLA, ThinkAct, NORA-1.5, X-VLA, StereoVLA, LAP-VLA, Cosmos Policy, and Diffusion Policy—spanning discrete and continuous action VLAs, world-model policies, and traditional imitation learning. The statistical rigor varies: LIBERO and the simulation benchmarks use large trial counts (500–1000 episodes) with confidence intervals; real-world evaluations use 15–50 trials per task, which is standard for robot manipulation but limits the precision of reported percentages (a 1/15 difference is 6.7 percentage points).

The claim that MolmoAct2-Think "retains geometric grounding at a fraction of prior latency" is supported: 12.71 Hz vs. 8.04 Hz (1.58× speedup) with performance remaining at or above the non-reasoning baseline (98.1% vs. 97.2%). However, the "fraction" is modest—12.71 Hz is still below the 30 Hz control rate common in robot manipulation, and substantially below the 55.79 Hz of standard MolmoAct2. The adaptive depth approach reduces but does not eliminate the reasoning latency bottleneck. The paper does not ablate the inference step count for flow matching (fixed at 10) or explore whether fewer flow steps could further improve speed with acceptable quality loss—a natural experiment given that flow matching's straight paths should degrade more gracefully than diffusion.

The claim that "fine-tuned success rates remain below the threshold for dependable use" (acknowledged as a limitation of the field in Section 1) is confirmed even for MolmoAct2: 50.6% on real-world bimanual YAM tasks, 50.69% under perturbations, and 97.2% on LIBERO simulation (where "simulation success" is easier than real-world success). The paper is transparent about these absolute numbers, but a reader should note that even the system's best case (LIBERO at 97.2%) represents a setting with limited visual diversity, known object sets, and controlled lighting—conditions unlikely to hold in deployment. The paper's own robustness study shows that adding spatial variation drops success from the base-task rates to 26.25%, suggesting that real-world deployment reliability degrades sharply when the visual or geometric conditions deviate from training.

A significant missing experiment is the combination of PRM-guided search with MolmoAct2-Think. The paper introduces depth reasoning as an intermediate step before action generation, but never explores whether multiple candidate depth predictions (or multiple action predictions conditioned on different depth hypotheses) could improve robustness through verification or consensus. Since the depth tokens are autoregressive and the action expert is flow-matching, a natural extension would be to sample multiple depth grid predictions, verify them against the observed image (e.g., via the cosine-similarity comparison already used for adaptive updates), and select the most consistent one before action generation. This would be expensive (multiple autoregressive decodes) but could improve the reliability of geometric grounding for critical manipulation steps.

The zero-shot SO-100/101 result on Block in Box (33.3% vs. π0-SO100/101's 90.0%) is a striking anomaly that the paper does not analyze. A 56.7-point deficit on a specific task within the same embodiment suggests that MolmoAct2's pre-training distribution may have coverage gaps: perhaps the action primitive (placing a block precisely into a container) is underrepresented relative to the π0 variant's fine-tuning data. Without investigating this failure, the paper cannot claim uniform out-of-the-box capability even within the supported embodiments.

The embodied reasoning backbone comparison (Table 3) includes a range of proprietary and open models, but the comparison is fundamentally unequal: GPT-5 and Gemini models are evaluated via API calls only, preventing any adaptation of their inference protocol or prompt format to the specific benchmarks. Molmo2-ER, being open, can be evaluated with benchmark-specific prompting and formatting. The 2.5-point advantage over Gemini ER-1.5 Thinking might partially reflect this evaluation asymmetry rather than a genuine capability gap. Conversely, if GPT-5 and Gemini benefit from API-level optimizations unavailable to open models, Molmo2-ER's win is even more impressive. The paper cannot resolve this confound but also does not acknowledge it.

The FLOPs-matched comparison that appears in the abstract's framing ("Molmo2-ER surpasses GPT-5 and Gemini Robotics ER-1.5") is a comparison of benchmark scores, not a FLOPs-matched study in the style of Section 7 of the reference paper. The actual FLOPs of GPT-5 and Gemini are undisclosed; they are almost certainly larger models than Molmo2-ER's 4B parameters. The claim that Molmo2-ER "surpasses" these models is therefore a performance claim at lower compute, which is stronger than just a performance claim—but the paper cannot quantify the compute differential, making the FLOPs-efficiency framing imprecise.

The difficulty estimation and adaptive allocation framework that characterized the reference paper is entirely absent from MolmoAct2. The paper does not estimate task difficulty, does not adapt its inference strategy per-task, and does not explore whether different tasks or scenes benefit differentially from depth reasoning. The adaptive depth mechanism adapts within a trajectory (recomputing depth for changed regions) but does not adapt across tasks (e.g., skipping depth reasoning entirely for tasks where it doesn't help). Given the paper's focus on practical deployment, a difficulty-conditioned allocation policy—using depth reasoning only when the scene is geometrically complex, or using more flow-matching steps only for fine-grained manipulation—would be a natural extension.

Finally, the robustness study (Section 6.5) tests 4 tasks × 4 perturbation types, but the 5 trials per perturbation (80 total trials) provides limited statistical power. A 1/5 difference is 20 percentage points, meaning the reported numbers have substantial uncertainty that the paper does not quantify with confidence intervals. The conclusion that MolmoAct2 is "substantially more robust" is qualitatively supported but the magnitude of difference relative to evaluation noise is unclear.

6. Limitations and Trade-offs

Assumption: Out-of-the-Box Deployment Requires Embodiment-Specific Data at Scale

MolmoAct2's zero-shot deployment capability is fundamentally bounded by the embodiments for which large-scale training data exists: bimanual YAM, DROID Franka, and SO-100/101. The paper is explicit about this constraint:

"Deployment on robots outside this set (e.g., other bimanual platforms, dexterous hands, mobile manipulators with different kinematics, or humanoids) requires effective fine-tuning on target-embodiment demonstrations, as shown in our fine-tuning studies in Sec 6.3." (Appendix E)

This is not a temporary limitation but a structural one: the pre-training and post-training mixtures concentrate 90% of robot sampling weight on these three platforms (30% each), with the remaining 10% split across smaller academic datasets that broaden control-mode coverage but do not provide the scale needed for zero-shot transfer. The model learns to map visual-language context to actions through the specific kinematics, camera layouts, and action representations of these three embodiments. A new robot with different joint configurations, a different number of arms, or different control conventions cannot be used without collecting demonstrations and fine-tuning.

Consequence. The term "out-of-the-box" in the paper's title and framing overstates generality relative to what is actually demonstrated. A practitioner with a robot outside the three supported platforms—which includes the majority of research robots in academic labs (Kinova, UR5, Panda with different end-effectors, mobile manipulators beyond the tested mobile YAM variant)—faces the same cold-start problem that VLAs were supposed to solve: collect demonstrations, fine-tune, evaluate. The paper does not characterize how many demonstrations are needed for cross-embodiment transfer, creating uncertainty about whether MolmoAct2 reduces the data requirement or merely shifts it.

Evidence. The fine-tuning results in Table 19 show that even with the best embodiment-specific fine-tuning recipe, real-world bimanual YAM tasks achieve only 50.6% average success, and individual tasks range from 14.0% (Pegboard) to 100% (Cup Storing). This is with the target embodiment's data at scale (50 trials per task × 8 tasks, plus three spatial variants). Cross-embodiment transfer to a completely novel platform would presumably require more data or achieve lower performance, but the paper provides no such experiment. The zero-shot SO-100/101 result of 33.3% on Block in Box vs. π0-SO100/101's 90.0% (Table 7, Table 20) suggests that even for supported embodiments, certain action primitives or object interactions may be inadequately covered by the pre-training distribution.

Mitigation status. The paper is transparent in Appendix E that MolmoAct2 "is not a universal controller that transfers without adaptation to arbitrary new embodiments" and frames the released datasets and training recipe as "a foundation for that effort rather than a finished product." Fine-tuning is presented as the adaptation path (Section 4.3.1, Section 6.3), and the fine-tuning results demonstrate that adaptation works—50.6% on novel bimanual YAM tasks, 97.2% on LIBERO, 44.3% on RoboEval. However, the paper does not provide cross-embodiment few-shot learning experiments (e.g., fine-tuning on a new robot with 5, 20, or 100 demonstrations), which would characterize the data efficiency of transfer and establish the practical cost of adopting MolmoAct2 on a new platform. This is partially addressed in Appendix E as a matter of "scaling data collection along the same axes we have begun here," i.e., the assumption is that the community will collect more multi-embodiment data rather than that the model will transfer with few examples.

---

Assumption: Fixed-Horizon Open-Loop Action Chunks Are Sufficient for Reactive Control

MolmoAct2 predicts fixed-horizon action chunks (30 steps at 30 Hz for YAM/SO-100/101, 15 steps at 15 Hz for DROID, 10 steps at 10 Hz for LIBERO) and executes them open-loop before re-querying the policy. The paper acknowledges this limitation in Appendix E:

"Motion smoothness across chunk boundaries is not enforced anywhere in the pipeline: the flow-matching expert is trained to denoise each chunk independently, with no continuity loss tying the end of chunk t to the start of chunk t + 1. In practice this can produce visible velocity or acceleration discontinuities at the seams, particularly when the VLM context shifts between queries (new observation, slightly different attention state)."

More critically: "the policy commits to a full chunk before observing its consequences, it cannot react within-chunk to perturbations, contact events, or its own tracking error—the 55.79 Hz number in Sec 6.8 is amortized chunk throughput, not closed-loop reactivity."

Consequence. The 55.79 Hz control rate reported in Section 6.8 is misleading as a reactivity metric. The model emits actions at an amortized rate of 55.79 Hz, but the effective closed-loop bandwidth is 1/30s ≈ 33 Hz for YAM/SO-100/101, 1/15s ≈ 67 ms for DROID, and 1/10s = 100 ms for LIBERO—the time between re-querying the policy with new observations. This means the policy is blind to the consequences of its actions for 33–100 ms at a time, during which it cannot adjust to unexpected contacts, object slippage, or trajectory deviations. For tasks requiring reactive precision—insertion, peg-in-hole, dynamic object interaction—this open-loop window may be too long. The lack of inter-chunk continuity enforcement also means that two independently denoised chunks may produce a velocity discontinuity at the boundary, which could excite vibrations in the robot's controller or cause the end-effector to jerk.

Evidence. The paper's own trajectory quality metrics (Table 17) partially capture this issue: the mean Cartesian jerk (MCJ) and mean joint jerk (MJJ) metrics measure acceleration changes, and while MolmoAct2 performs well relative to baselines, the absolute jerk values (e.g., 12.91 m/s³ for Stack Two Blocks) indicate non-trivial acceleration changes that chunk boundaries likely contribute to. However, the paper does not specifically measure chunk-boundary discontinuities (e.g., velocity difference between the last step of chunk t and first step of chunk t+1), which would directly quantify this limitation. The LIBERO success rates (97.2%) and real-world results (50.6% on bimanual YAM) suggest the open-loop assumption is workable for many tasks, but the tasks where it breaks down—those requiring continuous contact adjustment—are not specifically isolated or studied.

Mitigation status. The paper acknowledges this as an explicit limitation in Appendix E but does not propose a solution. Possible mitigations exist: overlapping action chunks with temporal blending at the boundaries, adding a continuity loss term between consecutive chunks during training (penalizing velocity discontinuities), or using a shorter action horizon (at the cost of more frequent VLM re-queries and higher total inference cost). The architectural design—with the VLM processing images and the action expert denoising trajectories—does not preclude these extensions, but the current implementation does not include them. The paper frames the 55.79 Hz number as "amortized chunk throughput" but does not clarify this distinction in the main text, where it could mislead readers unfamiliar with action-chunking architectures.

---

Assumption: Monocular Depth Estimation Provides Sufficient Ground-Truth for Spatial Reasoning Training

MolmoAct2-Think's adaptive depth reasoning depends on dense depth annotations for its training data, generated by Depth Anything V2 (Yang et al., 2024), a monocular depth estimation model. The paper treats these depth estimates as supervision targets for the depth VQ-VAE quantization and the autoregressive depth-token prediction objective. However, monocular depth estimation is inherently metric-ambiguous—it produces depth maps that are consistent in relative ordering but may have systematic scale errors, particularly for scenes with unfamiliar camera intrinsics, object scales, or viewpoints not well-represented in the estimator's training data.

Consequence. If Depth Anything V2 produces systematically biased depth estimates for certain scenes—e.g., consistently underestimating the distance to small objects, or mis-scaling depth for tabletop scenes viewed from atypical camera angles—these biases become baked into the depth token targets that MolmoAct2-Think learns to predict. At inference, the model will reproduce these biases in its predicted depth grid, and the action expert will condition on geometrically inaccurate spatial information. The consequence is not necessarily catastrophic for task success (since the depth signal is one of many conditioning inputs, and the depth gate in fine-tuning can learn to down-weight it), but it bounds the quality of geometric reasoning to the quality of the depth estimator. For tasks requiring precise metric reasoning—placing an object exactly 5 cm behind another, inserting a peg into a hole with sub-centimeter tolerance—monocular depth estimation errors could mislead the policy.

Evidence. The paper does not evaluate the accuracy of Depth Anything V2 on its specific robot datasets, nor does it compare training with monocular depth vs. ground-truth depth (e.g., from structured-light sensors or stereo cameras) where available. The LIBERO suite provides ground-truth depth from simulation, and DROID provides stereo depth from calibrated camera pairs—both potential sources of higher-quality depth supervision that the paper does not exploit or compare against. The MolmoAct2-Think results (Table 8: 98.1% LIBERO average, +2.2% on Long) are in simulation where the depth estimator likely performs well given the controlled rendering, but real-world performance (where the only real-world Think evaluation is on the bimanual YAM robustness study, Table 9, showing 50.69% average but without a non-Think real-world comparison on the same tasks) may degrade if monocular depth estimation is less reliable. The paper's ablation (Table 14) shows that removing depth-token noise injection and the depth gate drops performance from 98.10% to 97.65%, indicating that the depth pathway is sensitive to prediction quality, but does not isolate the contribution of depth estimator accuracy specifically.

Mitigation status. The paper does not address this limitation—it does not evaluate depth estimation quality, compare against ground-truth depth where available, or study how robust the action expert is to depth errors of varying magnitudes. The depth-token noise injection during fine-tuning (10% of depth-code inputs randomly replaced) is a form of robustness training against inference-time depth prediction errors, but it assumes random uniform errors rather than the systematic, scene-dependent errors that monocular depth estimators produce. The per-layer depth gate (learned modulation of depth token influence) provides some protection by allowing the expert to down-weight unreliable depth information, but this is a learned response to depth quality rather than an improvement in depth estimation itself. A natural mitigation—using stereo depth or structured-light sensors where available to provide metric-scale supervision—is not explored.

---

Assumption: Hardware-Agnostic Control Does Not Require Explicit Kinematic or Dynamic Modeling

MolmoAct2 treats robot control as a direct mapping from pixels and language to joint-space or end-effector actions, with embodiment identity communicated through setup descriptors (e.g., <setup_start>bimanual yam robotic arms in molmoact2<setup_end>) and control-mode strings (<control_start>absolute joint pose<control_end>). The model does not receive explicit kinematic parameters (link lengths, joint limits, maximum velocities), dynamic parameters (masses, inertias, friction coefficients), or collision geometry (self-collision meshes, workspace boundaries).

Consequence. The model must learn implicit representations of its own embodiment's physical constraints from demonstration data. This creates several failure modes: (a) the model may predict actions that exceed joint limits or self-collide, requiring the low-level controller to clamp or reject commands, producing jerky or incomplete motions; (b) the model may not learn the precise kinematic relationship between joint angles and end-effector pose for embodiments with complex geometry (e.g., 7-DOF arms with redundancy), leading to inefficient trajectories or failure to reach target poses; (c) the model's understanding of its own workspace boundaries—where the arm can and cannot reach—is only as good as the coverage of those boundaries in training data, and extrapolation to novel configurations may produce infeasible commands.

Evidence. The RoboEval trajectory quality metrics (Table 17) show that MolmoAct2 has non-zero self-collision counts (e.g., 1.10 on Cube Handover, 0.48 on Lift Pot)—the fact that self-collisions occur at all indicates imperfect implicit kinematic modeling. The per-task real-world bimanual YAM results (Table 19) show 54.0% on Cup Stacking vs. 64.5% for Cosmos Policy, and the paper's robustness study (Table 9) shows 26.25% on spatial variance perturbations—both suggesting that the model's understanding of its own reach and kinematics degrades when object positions deviate from training. The LIBERO results (97.2%) are in simulation with a single well-characterized embodiment, so the kinematic model is easier to learn implicitly from dense demonstration coverage; real-world performance on novel embodiments would likely suffer more from the lack of explicit kinematic knowledge.

Mitigation status. The paper does not discuss this limitation explicitly. The use of setup descriptors and control-mode strings is a form of explicit embodiment conditioning, but it provides categorical identification rather than parametric kinematic information. The approach assumes that behavioral cloning from demonstration data with diverse spatial coverage is sufficient to learn safe, kinematically-feasible trajectories—an assumption that is reasonable for the tabletop manipulation tasks studied (where reachable workspaces are compact and well-covered by training data) but likely breaks down for tasks requiring precise near-kinematic-limit operation, whole-body manipulation, or dynamic motions. Incorporating explicit kinematic parameters as additional conditioning inputs (e.g., joint limit tokens, link-length normalizations) would be a natural extension that the architecture could accommodate without structural changes, since state tokens are already part of the input sequence.

---

Tradeoff: Training Data Scale vs. Quality Creates Uneven Zero-Shot Capability Within Supported Embodiments

MolmoAct2's training data strategy combines three large-scale robot datasets with fundamentally different collection methodologies: MolmoAct2-BimanualYAM Dataset is centrally collected with strict quality protocols (limited failure retries, maximum no-op duration), MolmoAct2-DROID Dataset is a quality-filtered subset of a large academic dataset with re-annotated language instructions, and MolmoAct2-SO100/101 Dataset is community-contributed data filtered through a multi-stage pipeline with automatic quality scoring (TOPReward). These datasets differ not only in size and embodiment but in task diversity, spatial coverage, action distribution, and language-instruction quality—creating an uneven capability surface within the model's supposed area of competence.

Consequence. The model may exhibit strong performance on task types well-represented in its training data and weak performance on superficially similar tasks that differ in coverage. This is most starkly illustrated by the SO-100/101 zero-shot evaluation (Table 7, Table 20): MolmoAct2 achieves 86.7% on Pen on Notebook but 33.3% on Block in Box—a 53.4-point gap between two pick-and-place tasks on the same embodiment. The likely explanation is that the community-contributed SO-100/101 data over-represents certain action types (placing flat objects on surfaces) and under-represents others (placing blocks accurately into containers). The filtered nature of the data means that quality-gating removed the worst trajectories but did not balance the task distribution. Similarly, the DROID real-world evaluation (Table 18) shows that Objects in Bowl—the only partial-credit multi-object task—achieves 62.0%, substantially lower than the single-object tasks (86.7–100%), suggesting that the model's training data provides weaker coverage of multi-object reasoning or cluttered-scene manipulation.

Evidence. The SO-100/101 Block in Box result (Table 7: 33.3% vs. 90.0% for π0-SO100/101) is the clearest evidence of this uneven capability. The paper does not analyze this failure or break down model performance by task characteristics (cluttered vs. uncluttered, single vs. multi-object, precise vs. coarse placement), making it impossible for a practitioner to predict which of their own tasks will succeed out-of-the-box and which will require fine-tuning. The DROID real-world results (Table 18) show some variation: Apple on Plate (100%) vs. Pipette in Tray (86.7%) vs. Objects in Bowl (62.0%), but the task set is too small to identify systematic patterns. The MolmoSpaces simulation results (Table 4) show a similar pattern: Close (70.8%) and Pick (43.7%) are strong, but Open (9.5%) is extremely weak, with the paper noting this suggests "articulated-object interaction remains a direction for further improvement"—a post-hoc observation that does not help practitioners anticipate which task families will succeed.

Mitigation status. The paper's language re-annotation pipeline (Section 3.4, Appendix D.2) attempts to address instruction quality differences across datasets by using Qwen3.5-27B to re-annotate instructions for diversity and accuracy. This improves instruction coverage (Table 24: 71,121 → 146,485 unique instructions, 22% → 46% unique-instruction ratio) but does not address task-type imbalance—the underlying action distribution remains whatever the original datasets contained. The paper does not report per-task-type statistics for any dataset, does not analyze the coverage of different manipulation primitives (grasping from above vs. from the side, pushing vs. picking, inserting vs. placing), and does not study whether the model's failures correlate with under-represented action types in training data. A practitioner adopting MolmoAct2 for a specific manipulation domain (e.g., assembly, sorting, tool use) has no way to determine whether the pre-training distribution covers their needs without conducting their own evaluation—at which point they may as well collect fine-tuning data.

---

Assumption: Continuous Action Supervision Can Be Blended with Discrete Action Supervision Without Destabilizing the VLM's Multimodal Competence

The post-training recipe (Section 4.2.2) co-trains two objectives: the autoregressive next-token prediction loss (which includes discrete action tokens for robot examples and text tokens for VLM examples) and the flow-matching loss (which trains the continuous action expert). During post-training, knowledge insulation prevents flow-matching gradients from backpropagating into the VLM, but during fine-tuning, this insulation is removed. The VLM backbone is therefore updated by gradients from both losses, which have fundamentally different structure: the autoregressive loss is a per-token categorical cross-entropy over a discrete vocabulary, while the flow loss is a continuous regression over 32-dimensional normalized trajectories. The paper does not study whether this dual supervision causes the VLM to forget its general vision-language capabilities, or whether the quality of its visual representations degrades over the course of fine-tuning as it adapts to the control objective.

Consequence. If the VLM's general visual-language competence degrades during fine-tuning, the model becomes less robust to novel language instructions, unusual camera viewpoints, or out-of-distribution objects—precisely the conditions that matter for real-world deployment. The VLM is the substrate for all generalization: its ability to understand rephrased instructions, recognize unseen objects by their visual appearance and linguistic descriptions, and reason about spatial relationships in unfamiliar configurations depends on representations acquired during web-scale pretraining and the embodied-reasoning specialization. If fine-tuning for control overwrites or distorts these representations, the model's zero-shot and few-shot capabilities degrade even as its task-specific success rates improve. This is a classic catastrophic forgetting problem, well-documented in continual learning but rarely studied in VLA fine-tuning at the scale of a full VLM backbone.

Evidence. The paper does not measure VLM capability degradation. The evaluation strategy tests only robot-task success rates (LIBERO, RoboEval, real-world manipulation tasks), not general vision-language benchmarks post-fine-tuning. The robustness study (Table 9) does test language rephrasing (60.35% success) and visual distractors (54.10%), which partially probe VLM robustness, but these are tested on the fine-tuned checkpoint and compared against other fine-tuned baselines—there is no comparison to a "VLM-frozen" ablation where only the action expert is fine-tuned, which would isolate VLM degradation. The ablation in Table 13 does include an "action expert only" variant where the VLM is frozen during fine-tuning, achieving 93.05% average LIBERO—substantially below the 97.20% of full fine-tuning. This suggests that full VLM fine-tuning provides genuine control benefits, but it does not address whether those benefits come at the cost of degraded general capabilities. The LoRA variant (96.25% average) partially updates the VLM via low-rank adaptation and might preserve more general capability, but no VLM benchmark evaluation is provided for any of these variants.

Mitigation status. The paper does not address this limitation. The specialize-then-rehearse recipe used for Molmo2-ER training (Section 2.2) explicitly includes a rehearsal phase where embodied data is interleaved with general multimodal data to prevent catastrophic forgetting of vision-language skills, suggesting the authors are aware of this issue at the pretraining stage. However, the VLA fine-tuning recipe (Section 4.3.2) is robot-only—no multimodal VLM data is included during fine-tuning—and there is no rehearsal mechanism to preserve general capabilities while acquiring embodiment-specific control skills. This is a deliberate choice for efficiency (robot-only fine-tuning is faster), but it means the model's general competence is unguarded during the adaptation stage. A natural mitigation would be to include a small fraction of multimodal data during fine-tuning (similar to the 8% NLP data in the specialize-then-rehearse recipe) or to evaluate the fine-tuned model on a subset of the embodied-reasoning benchmarks to detect representation drift. Neither approach is attempted.

7. Implications and Future Directions

How This Work Changes the Landscape

MolmoAct2 does not introduce a single paradigm-shifting algorithm or a new theoretical framework. Instead, it changes the landscape through demonstration at scale: it shows that a fully open VLA—with transparent training data, reproducible recipes, and released model weights—can match or exceed the performance of closed and semi-closed alternatives across a wide range of benchmarks, while supporting deployment on hardware affordable to academic labs. This is a methodological shift rather than an algorithmic one, but its implications for how VLA research is conducted are substantial.

The primary reframing is from "can we build a VLA that works?" to "can we build a VLA that the community can own, inspect, adapt, and improve?" Prior to MolmoAct2, the landscape was stratified: frontier models (π series, Gemini Robotics, GR00T) demonstrated strong performance but were inaccessible for reproduction or extension; open-weight alternatives either lagged substantially in performance or were tied to expensive hardware. MolmoAct2 collapses this stratification by providing a system that is simultaneously open (weights, code, data, recipes), performant (97.2% LIBERO vs. 96.9% for π0.5), and hardware-accessible ($6,000 bimanual YAM setup, DROID Franka, SO-100/101). The paper demonstrates that these three properties are not in tension—that openness does not require sacrificing performance, and that performance does not require expensive hardware.

This reframing has both positive and negative consequences for the field. On the positive side, it establishes a new baseline for what "open" means in VLA research. Releasing weights alone is no longer sufficient; the paper implicitly argues that the community should expect (and demand) training data, code, and reproducible recipes. This raises the bar for future open VLA releases and makes it harder for industrial labs to claim openness while withholding the artifacts needed for reproduction. On the negative side, MolmoAct2's very comprehensiveness creates a barrier to entry for smaller labs: training the full pipeline requires approximately 10,368 GPU hours on H100s (5,760 for pre-training + 2,304 for post-training + 2,304 for bimanual YAM fine-tuning), plus additional compute for the backbone's 13-benchmark evaluation. The paper's openness means that others can inspect these costs, but it does not reduce them—and the scale of compute required may paradoxically reinforce the advantage of well-resourced labs while claiming to democratize access.

The paper reconciles a specific tension in the VLA architecture literature: whether to use purely discrete action tokenization (RT-2, OpenVLA, MolmoAct) or purely continuous action generation (π0, π0.5). MolmoAct2 demonstrates that both can coexist productively in a staged pipeline: discrete pre-training provides stable, scalable alignment of the VLM with robot action semantics; continuous post-training provides smooth, precise trajectory generation; and the per-layer KV conditioning interface lets the continuous expert benefit from the VLM's full hierarchical representations without requiring the VLM to be optimized for continuous control. This is not a theoretical resolution—it is a practical demonstration that the discrete-vs-continuous debate was a false dichotomy. Future VLA architectures can adopt this staged approach as a template, adapting the specific tokenizer, expert architecture, and conditioning interface to their own backbones and embodiments.

The paper's specialized embodied-reasoning backbone (Molmo2-ER) changes the calculus for what kind of pretraining matters for VLAs. The 6-point gain on LIBERO-Long from backbone improvement alone (Table 10: Molmo2 77.6% → Molmo2-ER 83.6%, discrete actions only, no continuous expert) demonstrates that backbone quality is a first-order determinant of VLA performance, not merely a scaling detail to be washed out by more robot data. This redirects research attention from action-expert architectures (which the ablation in Table 11 shows have a ~2-point window) toward visual representation quality (which the backbone comparison shows has a ~6-point window). Specifically, it suggests that investment in embodied-reasoning pretraining—curating datasets that teach metric depth, cross-view correspondence, and pixel-accurate pointing—may yield higher returns than further refinements to diffusion schedules, flow-matching step counts, or conditioning architectures. The paper does not prove this conclusively (a single backbone comparison on a single task suite is suggestive, not definitive), but it provides the strongest evidence to date that what the VLM sees shapes what the VLA can do more than the details of how actions are generated.

The adaptive depth reasoning (MolmoAct2-Think) introduces a design principle that generalizes beyond this paper: the cost of intermediate reasoning should scale with scene change, not with scene complexity. This is a specific instance of a broader idea—temporal redundancy exploitation—that has been underexplored in robot learning, where most systems either avoid intermediate reasoning entirely (for speed) or regenerate it fully at every step (for accuracy). The paper demonstrates that selective recomputation can achieve both (98.1% accuracy, 12.71 Hz), opening a research direction on efficient spatial reasoning that applies to any structured intermediate representation: point trajectories, object masks, future image predictions, scene graphs. The specific mechanism (cosine similarity on RGB patches with a 0.996 threshold) is simple enough to be replicated and extended with more sophisticated change-detection heuristics, learned update policies, or hierarchical spatial decompositions at multiple resolutions.

Perhaps most importantly, the paper establishes a comprehensive evaluation suite and protocol that future open VLAs can adopt for apples-to-apples comparison. The combination of 13 embodied-reasoning benchmarks (for the backbone), 3 simulation deployment benchmarks (MolmoSpaces, MolmoBot, held-out environments), 2 simulation fine-tuning benchmarks (LIBERO, RoboEval), 3 real-world embodiments (bimanual YAM, DROID, SO-100/101), a robustness perturbation suite (4 perturbation types × 4 tasks), a trajectory-quality analysis (8 metrics beyond success rate), and an inference-speed study provides a template for what "comprehensive VLA evaluation" means. This is a methodological contribution: it raises the standard for what constitutes a thorough empirical study in the VLA literature, making it harder for future papers to claim state-of-the-art based on a single benchmark or a handful of cherry-picked tasks.

---

Follow-Up Research This Work Enables

Cross-embodiment few-shot transfer characterization. The paper demonstrates zero-shot deployment on three embodiments (bimanual YAM, DROID Franka, SO-100/101) with large-scale training data, and full fine-tuning on novel tasks within those embodiments. But the critical practical question—how many demonstrations are needed to adapt MolmoAct2 to a completely new embodiment?—is unanswered. A follow-up study would freeze the post-trained MolmoAct2 checkpoint and fine-tune on 5, 20, 50, 100, and 500 demonstrations from a novel robot (e.g., a Kinova Gen3 arm, a UR5 with a different gripper, or a mobile manipulator variant of YAM), measuring success rates on a standardized task suite and also evaluating on the embodied-reasoning benchmarks from Table 3 to quantify VLM capability retention. The hypothesis (suggested but untested by the paper) is that MolmoAct2's discrete action pre-training provides a useful action prior that transfers across embodiments sharing the same tokenizer vocabulary, reducing the data requirement relative to training a VLA from scratch. The experiment would produce a data-efficiency curve per embodiment, which is the single most actionable piece of information for practitioners deciding whether to adopt MolmoAct2 or build their own VLA.

Verifier-guided depth reasoning for improved geometric grounding. MolmoAct2-Think predicts a single depth grid deterministically at inference (argmax decoding), then conditions the action expert on that grid. But autoregressive depth prediction can produce errors—a single incorrect depth code for a critical region (e.g., the object to be grasped) could mislead the action expert. A natural extension is to sample multiple depth grid hypotheses (using temperature-based sampling rather than argmax), verify each against the observed RGB image using a consistency metric (the cosine similarity already computed for adaptive updates could serve as a per-cell verification score), and either select the most consistent grid or ensemble the action predictions conditioned on different grids. This would trade additional inference compute for improved robustness—a worthwhile tradeoff for high-stakes manipulation steps (final grasp alignment, peg insertion) where geometric errors are costly. The experiment would measure how depth-grid diversity correlates with action quality, whether verification can identify incorrect grids before they cause failures, and whether the depth gate (Table 14 ablation) learns to down-weight uncertain depth tokens automatically—essentially, whether the model already does implicit verification or whether explicit verification adds value.

Joint optimization of the discrete-continuous action interface through reinforcement learning. The paper's three-stage pipeline uses purely supervised objectives: next-token prediction for discrete actions, flow matching for continuous actions. But the ultimate metric—task success—is only indirectly optimized. A follow-up would take the fine-tuned MolmoAct2 checkpoint and apply online reinforcement learning (e.g., RLPD or DrEureka-style reward-weighted regression) to further optimize the continuous action expert, using sparse task-success rewards from the environment. This is tractable because MolmoAct2 already achieves non-trivial success rates (50.6% on real-world bimanual YAM, 97.2% on LIBERO), providing a strong initialization for RL—the exploration problem is substantially easier when the policy already succeeds some fraction of the time. The key question is whether RL can close the gap between fine-tuned performance and deployment-ready reliability (e.g., pushing the 50.6% real-world average above 80%), and whether the per-layer KV conditioning interface transfers well to RL (can the VLM's representations support the credit assignment needed for policy-gradient updates?). The experiment would also test whether joint optimization of discrete and continuous actions under a task reward causes the two action paths to diverge or remain complementary—a stress test of the co-training design.

Scaling laws for embodied-reasoning pretraining data. The paper shows that Molmo2-ER (3.3M embodied-reasoning samples) substantially outperforms Molmo2 (no embodied specialization) on both VLM benchmarks (Table 3: +17 points) and downstream action learning (Table 10: +6 points on LIBERO-Long). But the shape of the scaling curve is unknown: would 10M samples provide proportionally more benefit? Is there a saturation point beyond which additional embodied data yields diminishing returns? Does the specialize-then-rehearse recipe (Section 2.2) change the scaling exponent relative to training on the embodied data from scratch? A scaling study would train Molmo2-ER variants at 1M, 3.3M, 10M, and 30M embodied samples (controlling for data mixture composition), evaluate each on the 13 embodied-reasoning benchmarks and on downstream LIBERO fine-tuning (same protocol as Table 10), and fit a power-law or sigmoid to the resulting curves. This would produce embodied-reasoning scaling laws analogous to neural scaling laws for language modeling, providing principled guidance for how much spatial pretraining data to curate. The study would also answer whether the benchmark saturation observed on some tasks (e.g., Point-Bench at 77.3% for Molmo2-ER) reflects a capability ceiling or a data ceiling—i.e., would 10× more pointing data push Molmo2-ER above 85% or is 77% near the achievable maximum for a 4B-parameter model?

Depth-quality ablation to quantify the cost of monocular depth supervision. MolmoAct2-Think's training depends on monocular depth estimates from Depth Anything V2, but the paper never evaluates depth estimation quality or compares against ground-truth depth where available. A controlled experiment would train three MolmoAct2-Think variants on the same LIBERO data but with different depth supervision: (a) Depth Anything V2 monocular depth (the paper's default), (b) ground-truth simulation depth, and (c) no depth (standard MolmoAct2). The comparison would quantify the depth-quality tax—how much performance is lost due to monocular depth estimation errors versus ground-truth depth—and establish an upper bound for what improved depth estimators could achieve. The experiment would also test whether the depth-token noise injection (10% random replacement during fine-tuning) partially compensates for depth errors (by comparing with and without noise injection across depth-quality conditions), and whether the per-layer depth gate's learned gating values correlate with per-cell depth error, indicating that the model learns to identify unreliable depth predictions. This is a negative-result-tolerant experiment: if monocular depth performs nearly as well as ground-truth (within 1% on LIBERO), that validates the paper's approach and suggests depth estimator quality is not a bottleneck; if ground-truth depth provides substantial gains (3-5%+), that motivates investment in better depth sensors or stereo pipelines for VLA training.

Dynamic task-conditional depth reasoning for selective geometric grounding. MolmoAct2-Think applies depth reasoning uniformly to all tasks, but depth information is likely more valuable for some manipulation phases than others: crucial for precise grasping and insertion, less important for coarse transport or approach. A follow-up would replace the fixed output-style selection (always using depth-and-action during inference) with a learned policy that decides per-timestep whether to invoke depth reasoning, trained via reinforcement learning with a reward that penalizes both task failure and depth-computation cost. The decision would be conditioned on the current VLM context (what the model sees and the instruction it is following), allowing the policy to learn that certain visual cues (e.g., the gripper being close to an object, the presence of clutter) predict high depth utility. This converts the adaptive depth idea from a per-cell efficiency optimization (which cells changed?) to a per-timestep necessity optimization (do I need depth right now?), potentially further reducing average latency while focusing geometric reasoning on the timesteps where it matters most. The experiment would measure both task success and average depth-token generation per episode, producing a Pareto frontier of reasoning cost vs. manipulation performance that characterizes when spatial reasoning is worth its latency cost.

---

Practical Applications and Downstream Use Cases

Academic robotics labs adopting a common VLA baseline. The most immediate practical application is MolmoAct2 serving as a standardized, reproducible baseline for VLA research. A lab with a Franka arm can download the MolmoAct2-DROID checkpoint and deploy it out-of-the-box (87.1% zero-shot on DROID real-world tasks, Table 18), then compare new methods (novel architectures, training objectives, data augmentation strategies) against this known baseline with controlled fine-tuning protocols. A lab building a bimanual YAM setup ($6,000 total cost, Figure 3) can use the MolmoAct2-BimanualYAM checkpoint for their own task development, fine-tuning on in-house demonstrations using the recipe in Section 4.3.2. A lab with an SO-100 arm ($300–500) can deploy MolmoAct2-SO100/101 for community-sourced tasks. The key practical benefit is reproducibility: because the training data, code, and recipes are fully released, results can be independently verified and compared across labs, which has been nearly impossible with prior VLA systems where training data was withheld or training procedures were undocumented. The 10,368 GPU-hour training cost is substantial but not prohibitive for well-resourced academic groups, and the staged training design means that individual stages (e.g., fine-tuning only, which costs ~1,152 GPU hours) can be reproduced without re-running the entire pipeline.

Cost-efficient robot data collection through targeted fine-tuning. The paper's fine-tuning results demonstrate that MolmoAct2's post-trained checkpoint adapts effectively to new tasks with modest data requirements: 500 demonstrations per LIBERO suite (reaching 97.2%), 8-task bimanual YAM evaluation with 50 trials per task (reaching 50.6%), and 4-task robustness evaluation with 20 trials per task (reaching 50.69% under perturbations). This establishes a data-efficiency baseline for VLA fine-tuning that practitioners can use to budget data collection efforts. A company or lab deploying a robot for a specific application (e.g., warehouse picking, lab automation, food service) can estimate that 50-500 demonstrations per task, collected via teleoperation on the target embodiment, will yield success rates in the 50-97% range depending on task complexity. The cost of data collection dominates the cost of fine-tuning compute (1,152 GPU hours on 32 H100s, approximately $2,000-4,000 at current cloud rates), making the economic calculus straightforward: collect demonstrations, fine-tune MolmoAct2, evaluate, and decide whether additional data or algorithmic improvements are needed. This is substantially more predictable than training a VLA from scratch, where the relationship between data quantity and performance is unknown and initial results may be near zero.

Open-source robot foundation models for assistive and service robotics. The paper's emphasis on real-world deployment and practical tasks—washing dishes, bussing tables, scanning groceries, packing medication, wetlab automation, pouring tea (Figure 1, right panel)—positions MolmoAct2 as a candidate base model for assistive and service robotics applications that require manipulation in human environments. These domains share characteristics with the paper's evaluation: diverse objects, variable scene configurations, natural language instructions, and the need for robustness to lighting, distractors, and spatial variation. The robustness study (Table 9: 50.69% average under perturbations) establishes a quantitative baseline for how MolmoAct2 degrades under realistic distribution shift, which is directly informative for system integrators assessing deployment readiness. The key practical advantage over prior systems is the low hardware cost: a complete bimanual YAM setup at $6,000 makes it feasible for small companies, research hospitals, or university labs to deploy and customize MolmoAct2 for their specific assistive tasks, whereas prior VLAs required$30,000+ Franka arms or were entirely proprietary. The released bimanual YAM dataset (720 hours, 34.5K demonstrations) covers household, factory, and coffee-shop settings, providing broad initial coverage that fine-tuning can specialize.

Evaluating and improving spatial reasoning in VLMs through embodied benchmarks. Molmo2-ER's strong performance on the 13 embodied-reasoning benchmarks (Table 3: 63.8% average, best on 9 of 13) makes it a useful diagnostic tool for spatial reasoning capabilities in VLMs. A team developing a new VLM can evaluate it against Molmo2-ER on these benchmarks to identify specific spatial reasoning weaknesses: poor performance on RoboSpatial-Point indicates weak pixel-accurate pointing; poor performance on Where2Place indicates weak affordance reasoning; poor performance on VSI-Bench indicates weak spatial video understanding. Because Molmo2-ER is fully open (weights, data, code), the team can inspect its training data mixture (Table 1) to understand which capability pillars are supported by which datasets, and can ablate their own training data or architecture against this known reference point. This transforms the 13 benchmarks from an abstract leaderboard into a practical development tool: a researcher can identify that their VLM scores 32.0% on RoboSpatial-Point vs. Molmo2-ER's 32.0% (Table 3—note: Molmo2 scores 13.9%, Molmo2-ER scores 32.0%), then examine whether adding pointing data (as Molmo2-ER did with RoboPoint's 700K pointing examples) closes this gap. The benchmarks thus become actionable rather than merely evaluative.