MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation

ArXiv: 2603.16861

🎯 Pitch

A purely simulation-trained robot achieves 79.2% real-world pick-and-place success—doubling the performance of the state-of-the-art system trained on real data—without using any real-world fine-tuning or photorealism. This demolishes the prevailing belief that simulation alone is insufficient for manipulation, showing that massive procedural data diversity is the true key to bridging the sim-to-real gap.

---

1. Executive Summary

This paper introduces MolmoBot-Engine, a fully open-source pipeline for procedurally generating large-scale, diverse manipulation training data in simulation, and demonstrates that with sufficient scale and diversity, zero-shot sim-to-real transfer is achievable for both tabletop and mobile manipulation without any real-world fine-tuning, photorealistic rendering, or explicit domain adaptation. The authors train three policy classes—MolmoBot (a Molmo2-based multi-frame vision-language model with a DiT flow-matching action head that cross-attends to intermediate VLM features), MolmoBot-Pi0 (a replication of the π0 architecture for controlled comparison), and MolmoBot-SPOC (a lightweight transformer policy suitable for edge deployment)—on 1.7 million expert trajectories spanning 94k+ procedurally generated environments, 11k+ unique objects, and 8 tasks across Franka FR3 and Rainbow Robotics RB-Y1 platforms. On real-world tabletop pick-and-place, MolmoBot achieves a 79.2% success rate—substantially outperforming π0.5 at 39.2%—and MolmoBot-Pi0’s 46.7% success rate using identical architecture trained from scratch on MolmoBot-Data demonstrates that the performance gains stem primarily from data scale and diversity rather than architectural differences. The paper establishes that procedural environment generation combined with diverse articulated assets enables broad real-world generalization, while also showing that this transfer succeeds on rigid body and articulated object manipulation—within the capability range of modern simulators—and that domain randomization across lighting, textures, friction, and camera pose, rather than photorealism, is the critical ingredient for bridging the sim-to-real gap.

2. Context and Motivation

The Core Problem: The Sim-to-Real Gap Is Widely Believed to Require Real-World Data

The fundamental question this paper tackles is whether simulation alone can produce robust manipulation policies that transfer directly to real-world robots, without any real-world data for adaptation or fine-tuning. The prevailing view in the field—which the authors explicitly challenge—is that simulation is useful as a pretraining or bootstrapping tool, but that closing the sim-to-real gap ultimately requires at least some real-world demonstrations, task-specific fine-tuning, or explicit domain adaptation techniques. As the paper states:

"A prevailing view in robot learning is that simulation alone is not enough; effective sim-to-real transfer is widely believed to require at least some real-world data collection or task-specific fine-tuning to bridge the gap between simulated and physical environments."

This matters deeply for the future of robotics research. If simulation requires real-world data to work, then building generalist manipulation policies remains bottlenecked by the cost, time, and logistical complexity of collecting real-world demonstrations. This bottleneck creates a structural advantage for well-resourced industrial labs—NVIDIA, Physical Intelligence, Google DeepMind—that can afford large-scale real-world data collection operations. The paper identifies this concentration of capability as a barrier to open research:

"As a result, the knowledge of what it actually takes to build a robotics foundation model from scratch remains concentrated within a small set of institutional actors rather than broadly accessible to the field."

Why This Gap Persists: The Real-World Data Bottleneck in Contemporary VLA Systems

To understand why the sim-to-real gap is such a critical problem, we need to look at what the current state-of-the-art systems actually require. The paper situates itself against a landscape where the strongest generalist policies are trained on massive real-world datasets:

• π0 and π0.5 (Physical Intelligence) are trained on ">10k hours of real-world manipulation demonstrations," as the paper notes in Section 5.1.1. These datasets are not publicly available, limiting reproducibility and community-driven innovation.
• RT-1 and RT-2 (Google DeepMind) scale to hundreds of manipulation tasks using multi-task robot datasets collected in real environments.
• DROID (Khazatsky et al., 2024) provides an open dataset of 76k real-world episodes, but this still represents only 350 hours of robot experience and covers a single embodiment in limited environment diversity (564 unique environments).
• Open X-Embodiment aggregates over 1M episodes across 22 embodiments and 527 tasks, but as the paper notes in Table 2, this data remains fundamentally constrained by the costs of real-world collection.

The cost asymmetry is stark. The paper reports that MolmoBot-Data was generated in approximately 6,500 GPU-hours, producing "more than 88 hours of robot experience per hour of wall-clock time"—approximately 2.6× the effective throughput of human demonstrators for equivalent tasks. More importantly, simulation data can be generated in parallel across hundreds of GPUs, whereas real-world data collection scales linearly with hardware availability and human operator time.

This bottleneck affects not just what policies can be built, but what the broader community can study. When the data mixtures, filtering decisions, and training recipes behind successful models are only partially disclosed, researchers outside well-resourced labs cannot systematically investigate the scaling properties, data composition choices, or architectural tradeoffs that determine policy performance. The paper positions itself against this opacity:

"Despite their utility, much of what matters most for training such systems remains difficult for the broader community to study: the full data mixtures, collection processes, filtering decisions, scaling regimes, and training recipes behind the strongest models are often only partially disclosed."

Where Prior Simulation-Only Approaches Fall Short

The paper identifies several categories of prior work and explains where they leave gaps:

1. Small-scale simulation datasets with limited diversity. Prior simulation benchmarks and datasets—RoboMimic (~1k episodes, 1 environment, 5 tasks, 1 embodiment), MimicGen (50k+ episodes, 1 embodiment, 18 tasks)—provide useful resources for studying imitation learning algorithms but lack the scale and diversity to support zero-shot transfer. Their environments are typically limited to single scenes or small numbers of procedurally generated variations, which means policies trained on them overfit to the specific visual appearance, lighting conditions, object placements, and camera viewpoints of the training distribution. The paper's core insight in Section 3 is that diversity across objects, configurations, and viewpoints matters more than photorealism:

"Our key insight is that manipulation policies benefit more from diversity across objects, configurations, and viewpoints than from photorealistic rendering."

2. Simulation-only training with explicit domain adaptation or photorealism. Several prior efforts have demonstrated sim-to-real transfer but relied on photorealistic rendering (which is computationally expensive and artistically demanding to create) or explicit domain adaptation techniques (which add complexity and require careful tuning). The paper demonstrates that extensive domain randomization—randomizing lighting, textures, friction, masses, joint damping, camera poses, and initial joint configurations—is sufficient when combined with sufficient environmental and object diversity. This is a pragmatic insight: domain randomization is straightforward to implement at scale, whereas photorealism requires pixel-level simulation fidelity that may not be necessary.

3. The assumption that manipulation is fundamentally harder than navigation for sim-to-real transfer. The paper explicitly builds on SPOC (Ehsani et al., 2024), which demonstrated that imitating shortest-path experts across hundreds of thousands of procedurally generated houses produces navigation policies that transfer zero-shot to real environments. The natural extension question is whether the same principle applies to manipulation. The paper frames this as the motivating question:

"A natural next question arises: can scaled simulated data enable zero-shot transfer for manipulation?"

Manipulation presents additional challenges beyond navigation: contact dynamics (grasping, placing), higher-dimensional action spaces (7-DoF arms plus base coordination for mobile manipulation), articulation constraints (doors, drawers, cabinets with revolute and prismatic joints), and the need for precise spatial reasoning (placing objects in receptacles, next to targets, or on specific colored surfaces). The prior success of SPOC for navigation motivates the hypothesis that these challenges are surmountable with sufficient data scale and diversity, but the paper is the first to demonstrate this at the scale necessary for manipulation.

4. Mobile and articulated manipulation is underexplored in large-scale imitation learning. The paper notes that most large-scale manipulation systems concentrate on fixed-base tabletop pick-and-place, where perception and workspace constraints are simpler. Mobile manipulation—which requires coordinating a holonomic base, torso, head, and dual arms while navigating to and interacting with articulated objects like doors and drawers—introduces substantial additional complexity. Prior work on mobile manipulation typically addresses only subsets of the problem: navigation with fixed-base manipulation skills, feasibility demonstrations via real-world teleoperation, or online adaptation strategies. The paper positions MolmoBot as addressing this gap directly:

"This study evaluates policies on both a tabletop manipulator and a mobile manipulator that performs multiple tasks such as mobile pick-and-place and door opening."

How This Paper Positions Itself

The paper's positioning can be understood along four dimensions:

Dimension 1: Challenging the "simulation is not enough" assumption. Rather than accepting that real-world data is necessary, the paper treats this as a falsifiable hypothesis and systematically tests it. The key claim is that the barrier to general-purpose manipulation may not be an irreducible sim-to-real gap, but rather insufficient scale and diversity in simulation data:

"Broadly, our results suggest that the barrier to general-purpose manipulation may be less about an irreducible sim-to-real gap, and more about whether the community has access to sufficiently large, diverse, and open simulation pipelines for training robotics foundation models."

Dimension 2: Full open-sourcing as a response to industrial opacity. Unlike the partially disclosed systems from industrial labs, MolmoBot provides "a fully open-source pipeline for procedural data generation across robots, tasks, and diverse simulated environments." This includes the data generation engine (MolmoBot-Engine), the full dataset (MolmoBot-Data with 1.7M episodes), and three trained policy architectures with different design points (VLM-based, π0-compatible, lightweight transformer). The paper explicitly connects this openness to scientific progress:

"We provide that access by open-sourcing all components."

Dimension 3: Controlled architectural comparisons to isolate the effect of data. A crucial methodological choice is the inclusion of MolmoBot-Pi0, which replicates the π0 architecture exactly but is trained from scratch on MolmoBot-Data. This enables a direct test of whether performance differences stem from architectural innovations or from data scale and diversity. The result that MolmoBot-Pi0 (46.7% real-world success) substantially outperforms π0 (39.2%) when using identical architecture demonstrates that "this difference in performance can only be explained by data" (Section 5.1.2). This is a clean experimental design that strengthens the paper's central claim.

Dimension 4: Pragmatic scope on what simulation can currently support. The paper is honest about simulation's limitations. It focuses on rigid body and articulated object manipulation because "these are both tractable to model for modern simulators, as well as interesting and challenging tasks still unsolved by modern generalist policies" (Section 3). It explicitly acknowledges that extending to contact-rich manipulation (insertion, peg-in-hole), deformable objects (cloth, rope, food), or tasks requiring fluid or granular dynamics remains an open challenge. This scoping is important: the paper is not claiming simulation can handle everything, but rather that within a well-defined and practically important class of manipulation tasks, simulation alone is sufficient.

Reconciling This with Prior Findings

The paper's results help reconcile several tensions in the literature. Prior work showed both that simulated data can be useful (MimicGen, RoboCasa) and that real-world data seems necessary for the strongest policies (π0, RT-2). The resolution is one of scale: when simulation diversity is limited to hundreds or thousands of environments, policies overfit and fail to generalize; when scaled to 94k+ environments with 11k+ objects across multiple embodiments, the diversity itself becomes the bridge across the sim-to-real gap. This is analogous to the story in computer vision, where ImageNet-scale data enabled generalization that smaller datasets could not, and more recently where massive web-scale pretraining reduced the need for domain-specific adaptation. The paper's contribution is demonstrating that a similar scaling story holds for robot manipulation and providing the open infrastructure to test it. </response>

3. Technical Approach

3.1 Reader Orientation

MolmoBot-Engine is a software pipeline that automatically generates millions of robot manipulation demonstration videos—complete with images, joint angles, and task instructions—entirely inside a physics simulator. The system solves the problem of how to produce enough diverse training data for robot learning that policies trained exclusively in simulation can work on real robots without any real-world adaptation. The shape of the solution is a three-stage pipeline: (1) procedurally construct randomized scenes with varied objects, lighting, and textures, (2) use motion planning to generate expert trajectories that successfully complete manipulation tasks, and (3) output these trajectories as a dataset that trains vision-language-action policies through behavior cloning.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components, arranged in a sequential data generation pipeline:

1. MolmoSpaces Environment Sampler — selects a procedurally generated house from a pool of 232k environments, places task-relevant objects (pickup targets, receptacles, distractor objects) at randomized collision-free poses, and applies extensive domain randomization (lighting, textures, physics parameters). Output: a fully configured simulation scene ready for a specific task.

2. Robot Configuration Module — positions the robot (Franka FR3 or RB-Y1) in the scene, randomizes initial joint angles, configures camera sensors (up to 5 cameras for Franka, 3 for RB-Y1), and samples camera extrinsics with per-episode perturbations. Output: a robot instance with sensors observing the randomized scene.

3. Expert Planner — decomposes the task into sequential phases (e.g., Pregrasp → Grasp → Lift → Preplace → Place → Post-place → Stow), samples and filters grasp candidates for each target object, and uses inverse kinematics (Franka) or GPU-accelerated trajectory optimization via CuRobo (RB-Y1) to generate collision-free motion for each phase. Includes retry logic: if a grasp fails or an object drops, the planner resets to the first phase and tries again. Output: a sequence of joint-space actions that successfully completes the task.

4. Sensor Logger — records RGB images from all camera viewpoints, proprioceptive state (joint positions/velocities, TCP poses, base pose), action labels in multiple representations (absolute joint positions, joint deltas, end-effector twist, absolute end-effector pose), task state (object poses, grasp indicators, policy phase, retry counts), and camera parameters (intrinsics/extrinsics). Output: a structured trajectory episode.

5. Action Noise Injector — before recording actions as training labels, applies action-proportional noise to the expert trajectory in task-space (mapped back to joint space via Jacobian pseudo-inverse) to prevent policies from overfitting to exact action replay. Stationary commands receive no noise; large motions receive proportionally more. Output: the final noisy action labels used for behavior cloning.

The pipeline runs in parallel across hundreds of GPUs, generating ~660 successful episodes per GPU-hour. The resulting MolmoBot-Data dataset feeds into training for three policy classes (MolmoBot, MolmoBot-Pi0, MolmoBot-SPOC), which are evaluated zero-shot on real robots.

3.3 Roadmap for the Deep Dive

• First, the environment construction and domain randomization (Section 3.3.1), because scene diversity is the paper's central hypothesis for why sim-to-real transfer works, and all other components operate within the randomized scenes this module creates.
• Second, robot configuration and sensor setup (Section 3.3.2–3.3.3), because the choice of robots, their kinematics, and their camera systems determines what observation and action spaces the policies learn, and the camera perturbation details are critical for sim-to-real transfer of visual features.
• Third, task definitions and language instruction sampling (Section 3.3.4), because the task specifications define what the expert planners must accomplish and what language conditioning the policies receive, and the referring expression sampling procedure is a non-obvious design choice that affects policy generalization.
• Fourth, the expert planners and motion generation (Section 3.3.5), because the quality and diversity of expert trajectories determines the upper bound of policy performance, and the retry behavior, grasp sampling, and phase-based decomposition are key engineering decisions.
• Fifth, dataset statistics and throughput analysis (Section 3.3.6), because the paper's core claim is about scale—these numbers characterize what "large-scale" means concretely and enable comparison to prior datasets.
• Sixth, the three policy architectures (Section 3.4), because understanding what is trained and how it maps observations to actions is necessary to complete the technical picture, even though the paper's primary contributions are on the data generation side.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and data engineering paper whose core idea is that procedural generation of diverse simulated manipulation trajectories at massive scale enables zero-shot sim-to-real transfer without any real-world fine-tuning. The architecture described below is MolmoBot-Engine, the data generation pipeline, followed by the three policy classes trained on its output.

---

3.4.1 Environment Construction and Domain Randomization

Scaffolding: MolmoSpaces scenes. Every training episode begins by selecting one pre-built scene from MolmoSpaces, a collection of "more than 200k available pre-built" procedurally generated indoor environments (Section 3.1). These scenes include realistic architectural variation, room layouts, furniture, and static objects, but are not populated with task-specific manipulable objects—those are added procedurally per episode. The scene's underlying geometry, wall positions, and furniture remain fixed for the episode, providing a consistent physical environment within which task objects are placed.

The paper emphasizes that the pre-built nature of these scenes is key for computational efficiency: constructing environments from scratch per episode would be prohibitively expensive, but having a pool of 200k+ scenes provides sufficient diversity when combined with per-episode randomization. This design choice—pre-build a large pool, then perturb heavily—balances diversity with generation throughput.

Object placement and task-specific scene adaptation. For each episode, the system samples task-relevant objects from large asset pools and places them at randomized 6-DoF poses subject to collision constraints and reachability from the robot's workspace (Section 3.1). The object pools differ by task role:

• Pickup objects (for pick and pick-and-place tasks) are sourced from iTHOR and Objaverse, filtered for graspable size: the bounding box must have its xy-plane diagonal smaller than the receptacle's corresponding diagonal, ensuring the object can fit in/on the receptacle. All objects must have watertight collider meshes—a practical requirement that prevents the physics simulator from producing degenerate collisions when objects interpenetrate.
• Receptacle objects (for pick-and-place tasks) must have bounding boxes with side dimensions under 50 cm along x and y axes, and vertical size up to 15 cm. These size constraints ensure receptacles are reachable by the robot arm and can stably support placed objects. For tasks requiring semantic relevance (e.g., "place in the bowl"), receptacles are additionally filtered by object metadata provided by MolmoSpaces, ensuring that the sampled receptacle makes sense for the language instruction.
• Distractor objects can be added to any task to increase visual clutter and force the policy to attend to the instruction rather than simply grasping the nearest object.

The paper notes that pose randomization ensures "diverse approach angles by sampling asset orientations relative to the robot base." This is specifically designed to prevent policies from learning a single stereotyped reach trajectory—by varying the relative pose between robot and object, the policy must learn to generalize across approach directions.

Domain randomization across three axes. After object placement, the system applies extensive domain randomization to the MuJoCo simulator. The paper describes this across several categories (Section 3.1):

Lighting randomization. The system samples both point and directional lights to simulate diverse indoor conditions. Key parameters and their sampling ranges:

• Number of lights: between 1 and N (the paper does not specify the maximum N, but the wording "1–N" implies it is configurable per scene)
• Light positions, intensities, colors, and shadow properties: all randomized within "plausible ranges" for indoor environments

This matters because lighting dramatically affects the visual appearance of objects—a policy that only sees uniform bright lighting will fail under shadows or directional lighting. By randomizing aggressively, the policy learns to be invariant to illumination conditions.

Texture randomization. Surface materials are randomized across placed objects and, where supported, existing scene elements. The system samples from two sources: procedural textures (mathematically generated patterns that can produce infinite variety without storing texture files) and real-world texture maps sourced from AI2THOR assets. The combination ensures both coverage of realistic material appearances and the ability to generate novel texture combinations not seen in any single real environment.

Dynamics randomization. Physics parameters are sampled within plausible ranges to encourage robust control policies:

• Friction coefficients: varied to simulate different surface materials (slippery countertops vs. grippy rubber mats)
• Object masses: varied to change the inertia and required grasp force for different objects
• Joint damping: varied to simulate different levels of mechanical wear or lubrication in the robot's joints

The paper does not specify exact ranges for these parameters, but the principle is that they should be broad enough that the policy experiences significant variation without becoming physically implausible. The goal is to prevent the policy from learning a brittle dynamics model that fails when real-world friction or mass differs from simulation defaults.

Visual augmentation during training (separate from environment randomization). In addition to the randomization applied during data generation, the paper applies image augmentation during policy training (Section 4.4). The specific augmentations are: ColorJitter, GaussianBlur, RandomPosterize, RandomSharpness, and RandomGrayscale, each applied with different probabilities. These augmentations operate on already-rendered images and are designed to further bridge the appearance gap between simulation rendering and real camera images—simulated images tend to be "too clean" with uniform lighting and sharp edges, while real camera images have noise, blur, compression artifacts, and variable color balance.

The paper's overall philosophy on domain randomization is captured in this key sentence from Section 3.3: "Our key insight is that manipulation policies benefit more from diversity across objects, configurations, and viewpoints than from photorealistic rendering." This is an empirical claim that the experiments support: the MolmoBot policies trained with these randomizations transfer to real environments despite being trained on MuJoCo's relatively simple rasterization renderer, because they have seen enough variation in lighting, texture, and viewpoint that the real-world appearance is just another sample from the training distribution.

Pose randomization details. Manipulable assets are placed at randomized 6-DoF poses, subject to two constraints:

1. Collision constraints: objects cannot intersect with each other, furniture, walls, or the robot at initialization
2. Reachability: objects must be within the robot's workspace, such that at least one valid grasp exists

These constraints are enforced by rejection sampling: candidate poses are generated, checked against constraints, and retried if they fail. The paper does not specify the maximum number of retry attempts for pose sampling, but the grasp sampling procedure (described later) uses similar rejection-based filtering.

---

3.4.2 Robot Configuration and Sensor Setup

Two robot platforms. The system supports two distinct robotic platforms to enable evaluation on both tabletop and mobile manipulation (Section 3.2):

Franka FR3 (tabletop manipulation). A 7-DoF Franka FR3 arm with a Robotiq 2F-85 parallel-jaw gripper, mounted on a fixed pedestal at 0.58 m height. The system uses the DROID configuration—a deliberate choice that enables "direct comparison with DROID-trained baselines and evaluation on existing benchmarks." Both data generation and evaluation run at 15 Hz, matching the real-world DROID control frequency. This means each simulation timestep corresponds to approximately 66.7 ms of real-world time.

Rainbow Robotics RB-Y1 (mobile manipulation). A significantly more complex platform with:

• A holonomic base: 3 DoF (x, y translation in the ground plane, plus yaw rotation θ), controlled in planar joint-position mode—meaning the policy commands target positions for the base rather than velocities
• A 6-DoF torso: provides vertical lift and orientation adjustments, allowing the robot to reach objects at different heights and interact with articulated objects like doors and cabinets
• A 2-DoF head: pan and tilt joints that control the head-mounted camera orientation. Importantly, the head is passively set at initialization and not actuated during episodes—the policy sees whatever the head camera captures from its fixed orientation
• Two 7-DoF arms, each equipped with a mechanically coupled parallel-jaw gripper: mechanically coupled means the gripper fingers move symmetrically (both fingers close toward the center simultaneously), simplifying the action space compared to independently-actuated fingers

The total DoF count for the RB-Y1 is 23 actuated joints (3 base + 6 torso + 2 head + 7 left arm + 7 right arm, though the head is passive), making it substantially more complex than the Franka's 7 DoF plus gripper.

Initial joint-configuration randomization. At episode initialization, each move group's joint positions are perturbed from a nominal home configuration q₀ by adding noise δᵢ sampled uniformly from [−rᵢ, rᵢ], where rᵢ is a per-joint noise magnitude. The paper specifies the exact perturbation vectors, which are designed with graduated magnitudes—proximal joints (closer to the robot base) receive smaller perturbations than distal joints (closer to the end-effector), because small proximal perturbations produce large end-effector displacements while distal perturbations produce smaller effects.

For the Franka FR3:

• r_arm = [0.025, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175] radians
• These values were "chosen via a Jacobian-weighted heuristic to bound TCP displacement to ≤10 cm"

This heuristic ensures that regardless of which joint is perturbed, the total end-effector displacement from the home configuration is bounded, preventing initialization in unreachable or dangerous configurations while still providing diversity.

For the RB-Y1:

• r_arm = [0.05, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175] radians (per arm, same for left and right)
• Head pan and tilt: ±0.2 rad ≈ ±11.4° each
• Gripper aperture: ±0.01 rad (a small perturbation to initial finger positions)
• Torso and base initial joint positions are not perturbed—the robot always starts from its nominal base and torso configuration

The rationale for not perturbing the base is practical: the base position determines whether the robot can reach task objects, and perturbing it could move objects outside the workspace, wasting generation attempts.

Action noise injection during data collection. A critical design choice: the expert planner produces deterministic optimal trajectories, but training on these directly would cause the policy to overfit to exact action sequences. The system therefore injects noise into the recorded action labels. The noise is action-proportional: its standard deviation scales with the magnitude of the commanded displacement, so stationary commands (∆q ≈ 0) receive approximately no noise, and large motions receive proportionally more noise.

For arm move groups, noise is applied in task space (TCP space) and then mapped back to joint space via the Jacobian pseudo-inverse. The procedure is:

1. Compute the commanded TCP displacement from the joint-space command: $\Delta x = J \Delta q$ where $\Delta x$ is the 6-DoF TCP displacement (3 position, 3 rotation), $J$ is the robot Jacobian at the current configuration, and $\Delta q$ is the joint-space command from the expert planner.

   What it computes: the end-effector motion in Cartesian space that results from the planned joint-space trajectory.

   Why this form: the Jacobian provides the linearized mapping between joint velocities and TCP velocities; using it to compute the TCP displacement lets the system reason about noise in task-relevant coordinates (e.g., "add 1 cm of position noise") rather than joint-space coordinates (which have no direct physical interpretation).

2. Sample position noise from a truncated Gaussian: $\epsilon_{\text{pos}} \sim \mathcal{N}(0, \sigma_{\text{pos}}^2), \quad \sigma_{\text{pos}} = \alpha \|\Delta x_{\text{pos}}\|$ where $\epsilon_{\text{pos}}$ is a 3D position perturbation vector, $\sigma_{\text{pos}}$ is the standard deviation that scales with the positional displacement magnitude $\|\Delta x_{\text{pos}}\|$, and $\alpha = 0.1$ is a fixed scale factor. The noise is clipped to ±2 cm.

   What it computes: a random position perturbation whose expected magnitude is 10% of the commanded motion size, bounded at 2 cm.

   Why this form: the proportional scaling ensures that when the robot makes large motions (e.g., moving across the workspace), it experiences larger perturbations that force the policy to learn robust trajectory following, while small precise motions (e.g., final approach to a grasp) receive smaller perturbations that preserve precision. The clipping to ±2 cm prevents unrealistically large perturbations.

3. Sample rotation noise similarly: $\sigma_{\text{rot}} = 0.1 \cdot \sigma_{\text{pos}}$ The rotation noise is clipped to ±0.1 rad (≈5.7°).

   What it computes: 3D rotation perturbations whose standard deviation is coupled to the position noise magnitude.

   Why this form: rotation and position perturbations should be correlated—a large arm swing should have both position and orientation uncertainty, while a small adjustment should have minimal uncertainty in both. The factor of 0.1 makes rotations proportionally smaller than positions, reflecting that orientation errors tend to matter more for grasp success.

4. Combine into a 6-DoF TCP noise vector: $\epsilon_{\text{tcp}} = [\epsilon_{\text{pos}}, \epsilon_{\text{rot}}]$

5. Project back to joint space by solving the linear system: $J \epsilon_q = \epsilon_{\text{tcp}}$ in the least-squares sense, using the Jacobian pseudo-inverse. The resulting $\epsilon_q$ is added to the planned joint command, and the sum is clipped to joint limits.

   What it computes: the joint-space perturbation that approximately produces the desired TCP perturbation, minimizing the residual $\|J \epsilon_q - \epsilon_{\text{tcp}}\|$.

   Why this form: the pseudo-inverse finds the minimum-norm joint perturbation that achieves the desired TCP perturbation, which is the perturbation least likely to cause large joint excursions or hit joint limits. Directly adding noise in joint space would produce TCP perturbations of unpredictable magnitude and direction.

For the RB-Y1 base, planar noise is applied directly to the (x, y, θ) commands: $\epsilon_{\text{base}} \sim \text{clipped Gaussian with } \sigma = 0.1 \cdot \|\Delta p\|$ bounded to ±2 cm in position and ±0.05 rad (≈2.8°) in heading.

Gripper handling. Gripper open and close commands do not happen instantaneously; they execute over fixed durations to simulate real-world grasp dynamics:

• Close: 0.5 s
• Open: 0.25 s
• After each gripper command, a settle period is enforced during which the arm is held stationary:
  • Franka: move_settle_time = 0.1 s
  • RB-Y1: up to max_grasping_timesteps = 5 control steps

This settle period is functionally important: it ensures the object is stably grasped before subsequent arm motion resumes, preventing the policy from learning to move the arm while the object is still settling in the gripper. In the real world, grasp settling time is unavoidable due to mechanical compliance and object dynamics; simulating it prevents a distribution mismatch where the policy learns to move immediately after closing the gripper (which works in simulation but fails in reality).

Camera configuration. The paper configures different camera systems for each robot platform, with the unifying design principle being that viewpoints should be heavily randomized to force visual robustness.

Franka FR3 camera system (Section 3.3.1). Five camera viewpoints are generated, though "in this work, we only train with the wrist camera and one randomized exocentric camera; the rest are present in MolmoBot-Data to provide for future work."

1. Wrist camera: Gripper-mounted, analogous to a ZED Mini stereo camera. Specifications:

   • 52° vertical FOV with ±4° noise per episode (so actual FOV varies between 48° and 56°)
   • Position perturbation: ±1.5 cm lateral, ±0.5 cm vertical, ±2 cm in depth (along the camera's optical axis)
   • Orientation perturbation: ±8° in roll, ±4° in pitch and yaw
   • Renders at 624 × 352 pixels, chosen because it is "close to the real-world resolution of 640 × 360 while keeping both dimensions a multiple of 16 for video encoding"

2. Fixed shoulder camera: Robot-mounted exocentric camera at a fixed offset from the robot base:

   • 71° FOV with light randomization (±5 cm position, ±8° orientation)
   • Placement is constrained to maintain visibility of task objects—the camera position is rejected and resampled if objects are occluded or out of frame

3. Three randomized exocentric cameras: Freely-placed cameras sampling positions around the workspace center:

   • Two ZED2 analogues: 64–72° FOV (the range represents per-episode perturbation, not a continuum)
   • One GoPro analogue: 137–140° FOV (significantly wider FOV than the ZED2 cameras, providing very different visual perspective)
   • For each camera, the system samples: distance from workspace center (0.2–0.8 m for ZED2, 0.2–0.5 m for GoPro—the GoPro is placed closer because its wider FOV would make objects too small at longer distances), height above workspace (0.05–0.6 m), and azimuth (full 360° around the workspace)
   • Lookat target: the workspace center with ±10 cm noise added per coordinate
   • Placement is validated by checking that task objects and the gripper are visible in the rendered image; placements that fail this check are rejected and resampled up to 20 attempts

All Franka cameras render at 624 × 352 resolution. This uniformity simplifies the vision encoder (all images have the same spatial dimensions) while still providing diverse viewpoints.

RB-Y1 camera system (Section 3.3.2). Three cameras matching the real robot's sensor configuration, all used for training:

1. Head camera: Mounted on the robot's head, analogous to a GoPro in wide mode:

   • Renders at 1024 × 576, cropped to 768 × 576 (4:3 aspect ratio) in post-processing. The crop changes the aspect ratio from 16:9 to 4:3, which more closely matches real GoPro wide-mode output
   • 139° vertical FOV with ±3° noise per episode
   • Position perturbation: ±1 cm in each axis
   • Orientation perturbation: ±4° around each axis
   • Randomized fisheye warping is applied per-frame during training—this simulates the barrel distortion characteristic of wide-angle GoPro lenses, which is not present in MuJoCo's pinhole camera model

2. Wrist cameras (left and right): Mounted on each wrist, analogous to Intel RealSense D405 sensors:

   • Render at 1024 × 576 (16:9 aspect ratio)
   • 58° vertical FOV with ±4° FOV noise
   • Position noise: ±1.5 cm lateral, ±0.5 cm vertical, ±1 cm in depth
   • Orientation noise: ±8° in roll, ±4° in pitch and yaw
   • Depth images are recorded "for the benefit of future dataset utility but unused during training"—all policies in this paper use RGB only

Proprioception and additional sensors (Section 3.3.3). Beyond visual observations, the system records:

• Robot state: joint positions and velocities for all actuated joints, TCP poses for each gripper (position and orientation in world frame), and the robot base pose
• Action labels: recorded in multiple representations to enable flexible training: commanded absolute joint positions, joint position deltas (relative to current positions), end-effector twist (6-DoF velocity relative to current TCP pose), and absolute end-effector pose (target TCP pose in world frame). This multi-representation recording means the same trajectory data can train policies with different action parameterizations without regenerating the data
• Task state: object start and goal poses (enables curriculum learning or auxiliary objectives), grasp state indicators (whether the object is currently grasped), policy phase (which phase of the task the robot is in), and retry counts from the expert policy (how many times the planner had to retry). These are recorded for analysis and potential future use but are not fed to the policies during training
• Camera parameters: intrinsic and extrinsic parameters for each camera, enabling projection between 2D pixel coordinates and 3D world coordinates. The system also records "points in the image frame on objects of interest in all cameras"—these are 2D projections of object centers, which could be used for point-based spatial grounding in future work

---

3.4.3 Task Definitions and Language Instruction Sampling

Rigid object manipulation tasks (Section 3.4). The system defines four rigid-body manipulation tasks, evaluated with both the Franka FR3 and RB-Y1:

Pick. Grasp a target object and lift it above its starting height. The success condition has two components:

• The object is no longer supported by any non-robot surface (i.e., it is fully in the gripper, not resting on the table)
• The object has been raised by at least 1 cm from its starting height

The 1 cm threshold is deliberately small—it distinguishes between "grasped and lifted" and "grasped but still resting on the surface," while being achievable even for objects that can't be lifted high without colliding with scene geometry.

Pick-and-place. Transport a target object to a specified receptacle. The success condition requires:

• At least 50% of the object's weight is supported by the receptacle (this is a physics-based check: MuJoCo computes contact forces between object and receptacle, and the system compares the total normal force to the object's weight)
• The receptacle has not been displaced by more than 10 cm or rotated by more than 45° (this prevents "success" when the robot knocks the receptacle over while placing the object)

The 50% weight threshold (rather than 100%) allows partial placement success—for example, an object that is mostly in a bowl but resting partially on the rim still counts as a success. This is a pragmatic choice that matches how humans would evaluate placement success.

Pick-and-place-next-to. Place a target object adjacent to a reference object on the same surface. The success condition requires:

• The surface-to-surface distance in the XY plane (horizontal distance between closest points on the object meshes) must be within [0, 5] cm
• The reference object must remain within 15 cm of its initial position (preventing success when the robot accidentally pushes the reference object toward the placed object)

The [0, 5] cm range allows objects to be close but not touching, which is appropriate for "next to" semantics. The upper bound of 5 cm prevents the policy from getting credit for placing objects arbitrarily far away.

Pick-and-place-color. Identical to pick-and-place, but with color-based receptacle identification. The system places two receptacles that are identical except for color (e.g., a red plate and a blue plate), and the instruction specifies which color to use (e.g., "place on the red plate"). The success criteria are identical to standard pick-and-place, but the policy must correctly identify the color-specified receptacle among identical distractors.

Articulated object manipulation tasks. Two tasks evaluated with the mobile RB-Y1:

Open. Open a nearby articulated object (specifically: cabinet, drawer, oven, or dishwasher) to at least 67% of its joint range. The 67% threshold ensures the object is substantially open (not just cracked), while not requiring full opening which might not be achievable from certain approach angles.

Open-door. Open a nearby hinged door to at least 67% of its hinge joint range. A crucial detail: "the instruction is conditioned on the robot's starting pose relative to the door, yielding either 'push the door open' or 'pull the door open.'" This means the language instruction explicitly tells the policy which action type to use based on whether the robot is on the push side or pull side of the door. The 67% threshold is the same as the open task.

Language instruction generation. Each task episode is accompanied by a natural-language instruction with referring expressions sampled at episode initialization (Section 3.4 and Appendix A.2). The system does not use fixed instruction templates; instead, it procedurally generates diverse referring expressions for each task object. The procedure works as follows:

1. Generate candidate referring expressions. For each task-relevant object, the system generates a set of possible referring expressions using three sources:

   • LLM-generated short descriptions (between 1 and 5 words)—these are natural language phrases like "the ceramic mug" or "the red apple"
   • Synset lemmas from WordNet—these provide taxonomic variations (e.g., "container," "vessel," "receptacle")
   • Normalized object category names from the asset metadata—these provide canonical category labels

2. Compute CLIP-based similarity scores. For each candidate expression, compute:

   • The CLIP similarity between the normalized text embedding of the expression and the normalized visual embedding of the target object (how well the expression matches the intended object)
   • The CLIP similarity between the expression and each distractor object in the scene context (how well the expression matches objects it should NOT refer to)

   The "context" is defined by the task: for a pick-and-place task where objects are on a table, the context is the set of all objects on that table surface. Objects elsewhere in the room are not considered as distractors—the system assumes the robot can infer the relevant spatial context from the task setup.

3. Filter expressions by contrast. An expression is valid only if:

   • The similarity margin (target similarity minus maximum distractor similarity) is ≥ 0.03—this ensures the expression is sufficiently more similar to the target than to any distractor
   • The absolute target similarity is ≥ 0.1—this ensures the expression is actually descriptive of the target (not just "not a distractor")

4. Sample an expression. From the filtered valid expressions, sample one via a softmax distribution over the similarity-margin scores: $P(e_i) = \frac{\exp(s_i / \tau)}{\sum_j \exp(s_j / \tau)}$ where $P(e_i)$ is the probability of selecting expression $e_i$, $s_i$ is the similarity margin for expression $e_i$, and $\tau = 0.02$ is a temperature parameter.

   What it computes: a probability distribution over valid referring expressions that concentrates probability on expressions with the highest contrast (largest margin between target and distractors), with the low temperature (0.02) making the distribution nearly deterministic for the best expression.

   Why this form: the softmax with low temperature strongly prefers the most discriminative expression while still allowing occasional sampling of slightly less discriminative alternatives. This provides diversity during training (the same object might be referred to differently in different episodes) while ensuring the chosen expression is always unambiguous in context.

Training-time prompt randomization (Appendix A.2.1). Beyond the referring expressions used in data generation, the system further randomizes language instructions during training:

• Template randomization: Multiple task prompt templates with varying wording and phrasing are used (e.g., "pick up the X" vs. "grasp the X" vs. "move the X"). The paper defers the full list of templates to the released code.
• Referral expression randomization during training: MolmoBot-Data saves multiple valid referral expressions per object per episode. During training, the system samples among these, "biasing towards shorter expressions." This means the policy sees different language formulations for the same visual scene, forcing it to learn the underlying semantics rather than memorizing specific phrase patterns.

---

3.4.4 Expert Planners and Motion Generation

The expert planners are scripted demonstrators that decompose each task into a sequence of phases, generate motion for each phase, and retry on failure. The planners differ between the two robot platforms due to their different kinematic complexity.

Grasp sampling and filtering. For every task requiring grasping, the system does not assume a fixed grasp pose. Instead, it loads a large set of pre-computed grasp candidates per object from MolmoSpaces' grasp dataset and progressively filters them through a multi-stage pipeline (Section 3.5):

1. Candidate loading and ranking. Pre-computed 6-DoF grasps for each object are loaded, transformed into the world frame (accounting for the object's randomized pose), and ranked by a weighted cost function that combines multiple terms:

   • TCP proximity: preference for grasps close to the robot's current end-effector position (minimizing travel distance)
   • Rotation similarity: preference for grasps with orientation similar to the robot's current end-effector orientation (minimizing wrist reorientation)
   • Vertical alignment: preference for grasps with approach direction aligned to gravity (top-down grasps are generally more stable)
   • Distance to object center of mass: preference for grasps near the center of mass (more stable during transport)

   Flipped variants of each grasp (grasping from the opposite side) are included in the ranking pool.

2. Collision filtering. The top-ranked candidates are tested for gripper-scene collision by placing phantom collision bodies at each candidate pose in MuJoCo and running broadphase collision detection in batches of up to 128 candidates at a time. Batching is an optimization: MuJoCo's collision detection is much faster when run on many candidate poses simultaneously than when run sequentially.

3. IK feasibility. Non-colliding candidates are checked for kinematic reachability via batch inverse kinematics in batches of up to 256. The highest-ranked feasible grasp (one for which a valid IK solution exists at the current arm configuration) is selected.

This multi-stage filtering is designed to select grasps that are simultaneously: close to the current arm pose (stage 1), physically possible without colliding with the scene (stage 2), and kinematically reachable (stage 3). The ranking by cost ensures that the system prefers "easy" grasps over "hard" ones, improving trajectory generation success rates.

Phase-based trajectory generation. Each task is decomposed into a fixed sequence of phases, with motion planned independently per phase. This decomposition serves two purposes: it simplifies the planning problem (planning one short motion segment is easier than planning a whole long-horizon trajectory), and it provides natural breakpoints for retry behavior (if one phase fails, the system can restart from the beginning without invalidating the entire episode).

For pick-and-place tasks with the Franka FR3, the phases are:

• Pregrasp: Move to a pose offset along the grasp approach axis (typically a few centimeters above the grasp pose, aligned with the approach direction). This ensures the gripper approaches the object along the correct axis.
• Grasp: Move from the pregrasp pose to the grasp pose and close the gripper. The system then waits for the settle period.
• Lift: Raise the object to a safe height above the table (typically 10–20 cm).
• Preplace: Move to a pose above the receptacle, at a safe distance.
• Place: Lower the object until it contacts or is just above the receptacle surface, then open the gripper. The system waits for the settle period.
• Post-place: Retract the arm from the receptacle, moving to a safe height.
• Stow: Return to a home configuration.

For the RB-Y1 pick-and-place, the Preplace and Place phases are combined (the planner moves directly to the placement pose without an intermediate pre-place waypoint), and the Stow phase is omitted (the arm remains at its final position). This simplification reduces planning complexity for the higher-DoF mobile manipulator.

For pick tasks, the trajectory terminates after the Lift phase—the object is grasped, lifted, and the episode ends. There is no placement or stowing.

For pick-and-place-next-to and pick-and-place-color, the task structure is identical to pick-and-place, with only the placement pose differing:

• Next-to: the placement pose is computed dynamically based on the reference object's position
• Color: the placement pose is determined by the receptacle matching the specified color

For open and open-door tasks, the phases are:

• Pregrasp: Move to a pose offset from the handle.
• Grasp: Move to the handle and close the gripper. For articulated objects, the grasp must be on the handle, but the handle's pose depends on the object's randomized placement.
• Articulate: Move the end-effector along the appropriate trajectory for the joint type:
  • Revolute joints (doors, cabinets, ovens): a circular arc about the hinge axis. The planner computes waypoints at regular angular intervals along the arc, with the final waypoint corresponding to ≥67% of the joint range.
  • Prismatic joints (drawers): a linear path along the slide axis. The planner computes waypoints at regular linear intervals, with the final waypoint corresponding to ≥67% of the joint range.
• Postarticulate: Release the handle and retract the arm.

During the Articulate phase, the planner must solve for each waypoint sequentially using IK or trajectory optimization, with the additional constraint that the grasped object (the door or drawer) must move along with the end-effector. This is handled through MuJoCo's constraint solver: closing the gripper creates a fixed constraint between the gripper and the handle, and subsequent arm motion applies forces through this constraint to articulate the object.

Motion planning for the Franka FR3. For the relatively simple Franka kinematics, the system uses IK-based interpolation: for each phase, compute the start and end joint configurations via inverse kinematics, then interpolate linearly in joint space with a fixed number of intermediate waypoints. The interpolation ensures smooth, collision-free motion (since the start and end configurations have been verified collision-free, the straight-line path in joint space is typically collision-free as well, though the paper does not explicitly describe collision checking during interpolation).

Motion planning for the RB-Y1 with CuRobo. The RB-Y1's 23-DoF kinematics (plus the coordination of base, torso, and dual arms) make simple IK-based interpolation insufficient. The system uses CuRobo, a GPU-accelerated collision-aware trajectory optimizer (Section 3.5 and Appendix A.1). The procedure:

1. Scene modeling. CuRobo constructs a cuboid-approximated collision world from the mesh-based MuJoCo scene. Cuboid approximations are used because collision checking against cuboids is substantially faster than against arbitrary triangle meshes, and the approximation error is acceptable for motion planning (cuboids slightly larger than the actual objects provide conservative collision avoidance).

2. Batch planning. Multiple candidate goal poses (up to 4 per batch, up to 4 batches = 16 total candidates) are evaluated in parallel on the GPU. For each candidate, CuRobo:

   • Solves inverse kinematics using 64 random seeds (each seed starts IK from a different initial guess, increasing the chance of finding a valid solution)
   • Computes a collision-free trajectory using 4 trajectory optimization seeds with fixed iterations
   • Smooths and interpolates the trajectory to match the simulation control frequency (the paper doesn't specify this frequency, but it is presumably 15 Hz to match the Franka, or a multiple thereof)

3. Trajectory selection. Among all successfully planned trajectories, the one with the least total joint displacement (sum of absolute joint movements across all timesteps) is selected. This selects the most efficient motion, which also tends to be the most natural-looking and least likely to encounter unexpected collisions.

4. Replanning on failure. When a waypoint cannot be reached within a fixed number of control steps, CuRobo replans from the current configuration, up to a maximum of 5 replanning attempts per phase. This handles cases where the initially planned trajectory becomes invalid due to accumulated tracking error or unexpected dynamics.

Retry behavior. Each demonstrator is equipped with retry behavior at the episode level. If the demonstrator detects a failure during execution—the object fell out of the grasp, the robot failed to acquire the grasp, a phase couldn't be completed—it resets to the first phase and tries the entire task again. The retry limit is 3: if more than 3 retries are triggered in a single episode, the episode is terminated and discarded (not included in the dataset). The paper notes that this "explicit retry behavior imbues policies with the ability to handle and recover from mistakes or disturbances," because episodes that include retries provide training examples of failure recovery. However, the paper also notes that steps with retry grasping behavior are upsampled 3× during training (Section 4.1), suggesting that the system actively emphasizes these recovery behaviors.

The retry behavior is important for data quality: without it, many episodes would fail at the grasp phase and produce no useful training data. With retries, the system can recover from failed grasps and still produce a successful trajectory. The 3-retry limit prevents infinite loops on impossible configurations (e.g., unreachable objects).

---

3.4.5 Dataset Statistics and Throughput

Dataset composition (Table 1). MolmoBot-Data consists of 1.7 million episodes comprising 295.2 million frames, spanning more than 11k unique pickup object assets, more than 9k receptacle object assets, and more than 94k unique environments (which are further modified by per-episode procedural object placement). The dataset breakdown by task:

The Franka pick task dominates in episode count (781.8k) because it is the simplest task and can be generated most quickly. The pick-and-place tasks have longer average episode lengths (17.1–20.1 s) because they require both picking and placing motions. Door-open has the longest average length (19.6 s) due to the complexity of whole-body mobile manipulation including base movement and articulation.

Data mixing for training (Section 4.4). Not all episodes are sampled uniformly during training. The data mixtures are specified in Tables 3 and 4:

For Franka FR3 policies:

• Pick: 20% (downsampled relative to its proportion in the dataset, which is much larger)
• Pick-and-place Fixed Height: 10%
• Pick-and-place Random Height: 35% (the largest component, reflecting the importance of height variation for robustness)
• Pick-and-place-next-to: 20%
• Pick-and-place-color: 15%

For RB-Y1 policies, different mixtures are used for different models:

• MolmoBot Multitask: Open 20%, Door-open 20%, Pick 30%, Pick-and-place 30%
• MolmoBot Door Specialist: Door-open 100% (a specialist model for door opening only)
• MolmoBot-SPOC Rigid: Pick 50%, Pick-and-place 50%
• MolmoBot-SPOC Articulated: Open 45%, Door-open 55%

The mixing ratios are designed to ensure coverage of all tasks while balancing the easier (pick) and harder (pick-and-place, articulation) tasks. The paper does not report extensive ablations on mixing ratios, so these choices represent the authors' empirical tuning rather than formal optimization.

Generation throughput (Section 3.6). The paper reports that using 100 NVIDIA A100 80GB GPUs, the system generates "approximately 660 successful episodes per GPU-hour." The time is split roughly evenly between rollouts (executing the expert trajectory in simulation) and scene setup + task sampling (building the environment, placing objects, applying randomization). This yields "more than 88 hours of robot experience per hour of wall-clock time," which translates to the full dataset being generated in approximately 6,500 GPU-hours.

The paper compares this to real-world data collection, citing ALOHA as a reference where "the effective real-time factor of a single human demonstrator is 1/3 for tasks of similar duration to ours, due to episode reset overhead or operator mistakes." This means that for every hour of real-world robot operation, only about 20 minutes of usable trajectory data is produced. MolmoBot-Engine's throughput is thus approximately 2.6× the effective data collection rate of human demonstrators per unit time, but more importantly, it can be parallelized across hundreds of GPUs whereas human demonstrations scale linearly with available hardware and operators.

Comparison to prior datasets (Table 2). The paper provides a structured comparison:

The key differentiating factors are: (1) largest number of episodes among simulation datasets, (2) substantially more environment diversity than any prior dataset (94.2k vs. 2,500 for the next largest), (3) support for mobile manipulation, which is rare in prior simulation datasets. The paper achieves lower task diversity (8 tasks vs. 365 for RoboCasa-365) but much higher per-task data volume and environment diversity.

---

3.4.6 Policy Architectures

The paper trains three policy classes on MolmoBot-Data, representing different points in the design space for vision-language-action (VLA) models.

MolmoBot: VLM-Based Manipulation Policy

MolmoBot builds on Molmo2-4B, a pretrained vision-language model, and augments it with a DiT-based flow-matching action head (Section 4.1). The architecture has three components, each with specific design choices:

Vision encoder. Visual observations are encoded via SigLIP2, a vision-language pretraining model. Key configuration details:

• The encoder processes RGB observations from input camera views. For Franka policies, this means the wrist camera and one randomized exocentric camera (Section 4.4). For RB-Y1 policies, this means all three cameras (head, left wrist, right wrist).
• Both the vision encoder weights and the projector weights (which map visual features to the language model's embedding space) are frozen during training. Only the action head and the language model (LLM) are trained. This is a deliberate design choice: the pretrained visual representations are already strong and diverse enough to handle the simulation-to-real domain gap; fine-tuning them on only simulated images might actually degrade their generalization.
• MolmoBot is trained to ingest up to F = 3 frames per view. For multi-frame models, the frames are the current timestep and frames sampled D = 8 steps ago (so F=2 uses current and ~0.5s ago; F=3 uses current, ~0.5s ago, and ~1s ago, since the Franka operates at 15 Hz so D=8 steps ≈ 0.53s). This provides short-term motion history without requiring the full trajectory context.
• Each image is encoded individually. Image tokens for each 2×2 patch window are pooled into a single vector using a multi-headed attention layer, where the mean of the patches within each window serves as the query. This pooling operation reduces the number of visual tokens per image to 192.
• Image tokens from available camera views are concatenated, interleaved with text tokens encoding image indices and view indices when appropriate. For example: [image_idx_0] [wrist_img_tokens] [image_idx_1] [external_img_tokens] if both cameras are used.
• Optionally, the system encodes the corresponding initial-timestep images to provide context about the starting scene configuration. These initial-frame tokens are concatenated alongside the current-frame tokens, giving the LLM access to both the initial state and the current state.

Language model (LLM). The LLM takes as input the visual tokens interleaved with image indices, jointly with the tokenized language instruction. Key details:

• For tasks requiring spatial grounding, the system optionally conditions on 2D point coordinates specifying target objects or placement locations. These are injected as special tokens in the instruction stream, formatted as <point coords=> OBJECT </points> where the coordinates are normalized 2D pixel positions in the image (values between 0 and 1 for x and y).
• The attention mechanism is asymmetric: bi-directional attention for vision tokens (each visual token can attend to all other visual tokens, regardless of temporal order) and causal attention for text tokens (each text token can only attend to previous text tokens and all visual tokens, following the standard autoregressive language model convention). This asymmetry is natural because visual tokens represent spatial information that is not inherently ordered, while text tokens represent language with a clear left-to-right ordering.
• The LLM has an encoder stack with N transformer blocks. The paper does not specify N, but it is inherited from Molmo2-4B's architecture.

Action head. The action head is a DiT (Diffusion Transformer) that predicts robot actions through iterative denoising. This is the key architectural innovation relative to standard VLA models that predict actions in a single forward pass. The details:

• Flow matching formulation. Following π0 and recent work on flow matching for action prediction, the DiT iteratively denoises action chunks conditioned on a continuous timestep embedding t ∈ [0, 1]. At t = 1, the action is pure Gaussian noise; at t = 0, the action is the clean prediction. During training, the model learns to predict the denoising direction at each timestep; during inference, the model starts from noise and iteratively denoises to produce the final action.

• Architectural coupling to the VLM. This is the most distinctive design choice. The action head has the same number of layers as the LLM encoder, and each action layer cross-attends to the hidden states of the corresponding LLM layer. This means:

  • Action head layer 1 cross-attends to LLM layer 1's hidden states
  • Action head layer 2 cross-attends to LLM layer 2's hidden states
  • And so on through all N layers

  Because the LLM and DiT have different hidden dimensions, the LLM hidden states are projected to the DiT's hidden dimension before cross-attention. This layerwise coupling allows the action head to access multi-scale representations from the VLM: early layers provide low-level visual features, middle layers provide semantic object representations, and late layers provide language-grounded task understanding.

• Robot state integration. Robot proprioceptive state is encoded through a single-layer MLP and concatenated to the end of the VLM sequence before entering cross-attention at each layer. This means the robot state is available to every cross-attention operation in the action head, analogous to how the VLM sees both visual tokens and text tokens.

• Action chunking. The action head predicts chunks of H = 16 actions and executes 8 before re-querying the policy, following the design from ALOHA. This means the policy runs inference once every 8 timesteps (approximately 0.53 s at 15 Hz), and the chunk prediction provides temporally consistent actions that the low-level controller executes open-loop until the next inference cycle.

• Action representation. Actions are parameterized in joint space using two possible representations:

  • Absolute joint positions: the policy predicts target joint configurations directly. This is used for Franka FR3 policies because the experiments show it transfers better to real robots (Section 5.4).
  • Joint position deltas: the policy predicts changes relative to the current joint positions. This is used for RB-Y1 policies because the higher DoF count makes absolute prediction more challenging.

  For the RB-Y1's mobile base, the policy additionally predicts base velocity commands (linear and angular), concatenated to the joint action. This hybrid representation (joint deltas + base velocities) is chosen because predicting absolute base positions from visual input is ill-posed (the base can drift arbitrarily), while velocity commands are naturally incremental.

• Training with multiple denoising timesteps. The action head has a significantly lighter compute footprint than the VLM encoder. The system leverages this by sampling multiple timesteps T per training example and denoising in parallel. For each observation-action pair, the system:

  1. Encodes the observation once through the VLM (the expensive step)
  2. Samples T different noise levels t₁, ..., t_T
  3. For each noise level, adds the appropriate amount of noise to the ground-truth action and runs the action head forward pass to predict the denoising direction (the cheap step, since the VLM features are already computed)
  4. Computes the flow matching loss for each timestep and averages

  The paper states this "improves the convergence and the accuracy of the model" and uses T = 8 for all MolmoBots unless otherwise stated. This is effectively a form of multi-task learning where the same observation must support denoising predictions at multiple noise levels.

Single-frame vs. multi-frame training. The paper trains three variants:

• MolmoBot-Img: Single-frame model (F = 1), trained for 200K steps at batch size 1024
• MolmoBot (F=2): Multi-frame model initialized from MolmoBot-Img weights and trained for an additional 50K steps
• MolmoBot (F=3): Same initialization and additional training, with one more frame in the context

The learning rate for all variants is 1 × 10⁻⁵, with a 2K step warmup for the LLM and a 200 step warmup for the action head. The different warmup durations reflect that the LLM starts from strong pretrained weights and needs gentler initial updates, while the action head starts from random initialization and can be updated more aggressively.

Training data upsampling. When sampling training examples from an expert rollout, the system applies importance weighting:

• Steps with retry grasping behavior: 3× upsampling (to emphasize recovery from failed grasps)
• Steps with successful pick: 2× upsampling (to emphasize the critical grasp transition)
• Steps with task completion behavior: 2× upsampling (to emphasize the placement and release phases)

The motivation stated in the paper is to "improve the model's grasping behavior and avoid picking objects after task completion." The avoiding-picking-after-completion concern is interesting: without this upsampling, the model might see many post-completion frames where nothing happens, and learn that inaction is the most common behavior. Upsampling the active phases counteracts this distributional skew.

MolmoBot-Pi0: Controlled Architectural Comparison

MolmoBot-Pi0 uses the identical architecture as π0 (Physical Intelligence's VLA model), trained from scratch on MolmoBot-Data from the initial Paligemma 3B VLM weights (Section 4.2). This enables a controlled experiment isolating the effect of the training data.

Architecture identity. Following the π0 architecture exactly, MolmoBot-Pi0 uses Paligemma 3B as the VLM backbone with a flow-matching action expert. The implementation uses the openpi codebase "for all MolmoBot-Pi0 modeling code, ensuring equivalence with π0." This is important methodologically: any performance differences between MolmoBot-Pi0 and published π0 results can only be attributed to training data.

Training protocol. Key differences from MolmoBot training:

• Trained for 200K steps at batch size 1024 (same step count as MolmoBot-Img)
• Learning rate: 5 × 10⁻⁵ (5× higher than MolmoBot's 1 × 10⁻⁵, likely because starting from Paligemma rather than Molmo2)
• 1K step warmup (shorter than MolmoBot's 2K step LLM warmup)
• The SigLIP vision encoder is entirely frozen during training, specifically to "prevent overfitting to simulation rendering artifacts." This is the same design choice as MolmoBot, suggesting it is a general principle for sim-to-real transfer with VLMs.
• Robot actions are supervised as absolute joint positions, "following findings from PolaRiS"—matching the absolute action representation used for MolmoBot Franka policies.
• All other training parameters (flow matching timestep sampling, optimizer hyperparameters, etc.) are left at openpi's default values.

MolmoBot-SPOC: Lightweight Transformer Policy

MolmoBot-SPOC is a non-VLA policy inspired by the SPOC architecture for navigation, modified for manipulation tasks (Section 4.3). It is designed to be substantially smaller and simpler than the VLA-based models, making it suitable for edge deployment and amenable to future RL fine-tuning (which is typically easier with smaller models).

Visual, language, and proprioceptive encoding. Unlike MolmoBot which uses a large VLM, MolmoBot-SPOC uses separate encoders:

• Visual: SigLIP2-Base patch 16/256 image encoder, retaining the full set of patch tokens (not pooled). For a 256×256 input, this produces 16×16 = 256 patch tokens per image. Retaining all patches gives the model spatial information that pooled representations would discard—important for manipulation where object positions must be precisely localized.
• Language: SigLIP text encoder, encoding the task instruction into a separate sequence of text tokens.
• Proprioceptive state: The robot's current joint state is projected into the model's token dimension via a learned linear projection.

These token sequences are concatenated along the sequence dimension to form the cross-attention memory: [visual_patch_tokens | language_tokens | robot_state_tokens].

Point-based spatial grounding. For tasks that provide spatial goal specifications (pick, place, open, door-open), MolmoBot-SPOC optionally incorporates point-based goal encodings:

• A single normalized image coordinate (x, y) for pick, open, and door-open tasks (specifying the target object or handle)
• Two normalized coordinates (x₁, y₁, x₂, y₂) for pick-and-place tasks (specifying the pickup object and receptacle)
• Each coordinate is passed through a sinusoidal positional encoder (the same type used in transformers for sequence position encoding), then projected into the model's token dimension via a linear layer
• A learned coordinate position embedding is added to each encoded point (allowing the model to distinguish "coordinate 1" from "coordinate 2" when multiple points are provided)
• The resulting point tokens are concatenated with the other inputs in the cross-attention memory

MolmoBot-SPOC does not condition on trajectory history; only the current timestep's observations are used. This makes it a reactive policy rather than a context-aware one, trading off temporal context for architectural simplicity.

Action representation and quantile binning. Unlike MolmoBot's continuous flow-matching action head, MolmoBot-SPOC formulates action prediction as a discrete classification problem. The procedure:

1. Normalization. Action values are normalized using the 1st and 99th percentiles of the training distribution, rescaling and clipping values to [−1, 1] based on empirical quantiles. This is more robust than min-max normalization because it is insensitive to outliers.

2. Quantile binning. The normalized action space for each dimension is divided into 256 bins, where bin boundaries correspond to equally spaced quantiles of the data—specifically, the k/256 quantile for k = 1, ..., 256. This produces data-adaptive bins that are approximately uniformly populated: each bin contains roughly the same number of training examples, which is well-calibrated for classification.

3. Discrete classification. The decoder predicts a categorical distribution over the 256 bins independently for each action dimension. The model is trained with standard cross-entropy loss: $\mathcal{L} = -\sum_{d=1}^D \sum_{b=1}^{256} \mathbf{1}[y_d = b] \log p_d(b)$ where $D$ is the number of action dimensions, $y_d$ is the ground-truth bin index for dimension $d$, $p_d(b)$ is the model's predicted probability that dimension $d$ belongs to bin $b$, and $\mathbf{1}[\cdot]$ is the indicator function.

   What it computes: the negative log-likelihood of the correct bin under the predicted distribution, summed over all action dimensions. Lower loss means the model assigns higher probability to the correct bin.

   Why this form: classification with cross-entropy is well-calibrated for quantile bins because the bins are uniformly populated (each bin is equally likely a priori), so the model doesn't need to learn skewed class priors. Continuous regression with MSE would struggle with the multi-modal nature of action distributions (multiple valid actions for the same observation). An alternative would be mixture density networks, but classification is simpler and more stable to train.

Parallel action decoding. The decoder predicts the entire action chunk in a single forward pass, rather than autoregressively one timestep at a time. Following the ALOHA design:

• The decoder receives D × T learnable query embeddings, where D is the number of action dimensions and T = 16 is the fixed chunk length
• Each query embedding corresponds to one (action dimension, timestep) pair in the chunk
• The decoder uses bidirectional self-attention, allowing each query token to attend to all others within the chunk
• Temporal structure is encoded using sinusoidal positional encodings over the flattened sequence of D × T positions, added to the learnable query embeddings before decoding
• The decoder's cross-attention attends to the concatenated memory of visual patches, language tokens, and robot state tokens

This parallel decoding is substantially faster than autoregressive decoding (one forward pass vs. T sequential forward passes) and allows the model to coordinate actions across timesteps (e.g., planning a smooth trajectory rather than reacting frame-by-frame).

---

This completes the technical breakdown. The data generation pipeline (MolmoBot-Engine) produces 1.7M diverse trajectories with extensive domain randomization, expert motion planning, and action noise injection. The three policy architectures represent different design points: a VLM with layerwise-coupled DiT action head (MolmoBot), a π0-compatible VLA for controlled comparison (MolmoBot-Pi0), and a lightweight discrete-action transformer (MolmoBot-SPOC). Together, they demonstrate that the data pipeline is architecture-agnostic—all three architectures achieve zero-shot sim-to-real transfer, supporting the paper's central claim that data scale and diversity, rather than architectural innovation or photorealistic rendering, is the critical ingredient.

4. Key Insights and Innovations

Innovation 1: Simulation Diversity as a Substitute for Sim-to-Real Adaptation — A Scaling-Law-Level Conceptual Shift

The paper's most fundamental contribution is not any specific architectural feature of MolmoBot or any algorithmic trick in MolmoBot-Engine. It is the conceptual reframing of what bridges the sim-to-real gap. The dominant assumption in robot learning, which the paper explicitly names and challenges, is that simulation alone cannot produce robust real-world manipulation policies—that closing the sim-to-real gap requires at least some real-world data for adaptation, fine-tuning, or explicit domain randomization-to-reality mapping. This assumption has shaped the field's resource allocation: industrial labs invest heavily in real-world data collection (π0's >10k hours of real demonstrations, RT-2's multi-task real datasets), while academic groups treat simulation as pretraining followed by real-world adaptation.

The paper's counterclaim is that this gap is not irreducible—it is a gap of scale and diversity, not of fundamental signal. The evidence for this is clean: MolmoBot achieves 79.2% real-world pick-and-place success with zero real-world data, substantially outperforming π0.5 (39.2%) which was trained on >10k hours of real demonstrations (Figure 7). Even MolmoBot-Pi0—which uses the identical π0 architecture trained from scratch on MolmoBot-Data—achieves 46.7%, demonstrating that the data, not the model, drives the improvement.

What makes this a genuine reframing rather than an incremental result is the specificity of the diversity argument. Prior work on domain randomization treated it as a technique to broaden the training distribution within a relatively narrow environment set. The paper argues something different: that the sheer combinatorial explosion of procedurally generated environments, objects, lighting conditions, textures, camera poses, friction parameters, and initial joint configurations—when pushed to the scale of 94k+ environments, 11k+ objects, and 1.7M trajectories—creates a training manifold that is dense enough that real-world observations fall within it, even without photorealistic rendering. This is analogous to what happened in computer vision when ImageNet-scale data enabled generalization that smaller curated datasets could not, but applied to the harder domain of embodied manipulation where physics, contact, and viewpoint variation make the distribution shift substantially more severe.

The paper is careful about scope: it limits this claim to rigid body and articulated object manipulation where modern simulators provide sufficient physical fidelity. It explicitly acknowledges that contact-rich manipulation (insertion, peg-in-hole), deformables (cloth, rope), and tasks requiring fluid dynamics remain unsolved. This boundary-drawing is itself an intellectual contribution: it tells the field where simulation is sufficient and where real-world data remains necessary, converting what was previously a blanket assumption ("simulation is not enough for manipulation") into a more precise, testable hypothesis about the relationship between simulation fidelity, data diversity, and task class.

The negative results on the hardest difficulty tier reinforce this. Despite 1.7M trajectories, the hardest real-world problems (bin 5 in the test distribution) remain unsolved, and the door opening real-world results (Table 5) show 2/9 success on pull doors, with handle configuration bias as a likely failure mode. This means the diversity argument has limits: underrepresented configurations in training (e.g., right-side door handles) still cause failures, even at massive scale. The paper's contribution is thus not "simulation always works" but rather "simulation works conditionally on sufficient coverage," which is a more nuanced and actionable claim.

Innovation 2: Layerwise Cross-Attention Between VLM and Action Head as a Multi-Scale Representation Strategy

This innovation is architectural rather than conceptual, but it represents a design insight with implications beyond MolmoBot. The standard approach to building VLA models is to encode visual observations through a vision backbone, pass the resulting features into a language model (or directly into a policy head), and predict actions. In π0, the action expert receives features from the final layer of the VLM—essentially a single, high-level representation that has abstracted away from low-level visual and spatial information.

MolmoBot's design is different in a specific and motivated way: the DiT-based action head has the same number of layers as the VLM encoder, and each action layer cross-attends to the hidden states of the corresponding VLM layer (Section 4.1). This is not an obvious or default architectural choice—it requires the VLM and action head to have matching depth, the hidden dimensions to be projected across modalities at every layer, and the robot proprioceptive state to be injected at every cross-attention operation. The computational overhead of this coupling is non-trivial: at inference time, the VLM must compute and store hidden states at every layer rather than just the final representation, and the action head must run cross-attention at every layer rather than once.

What makes this more than an engineering detail is the functional rationale: early VLM layers encode low-level visual features (edges, textures, object boundaries), middle layers encode semantic object representations (category, affordance, spatial relationships), and late layers encode language-grounded task understanding. By coupling each action head layer to the corresponding VLM layer, the action prediction process has access to the full hierarchy of representations—from pixel-level spatial information that matters for precise grasping (early layers) to task-level semantic reasoning that matters for instruction following (late layers). A single cross-attention to the final VLM layer, by contrast, would force the action head to reconstruct spatial precision from a highly abstracted representation, which is lossy.

The paper does not provide a direct ablation comparing layerwise cross-attention to final-layer-only cross-attention, so the magnitude of this contribution relative to the data diversity effect is unmeasured. However, the concept is intellectually distinctive because it identifies a specific information bottleneck in existing VLA architectures (the loss of spatial information through abstraction) and proposes a principled solution (multi-scale feature access) that generalizes beyond this specific model. If the hypothesis is correct, this design pattern should benefit any VLA model that must perform both semantic reasoning (from language) and precise spatial control (from vision), regardless of the specific backbone or action representation.

Innovation 3: Controlled Architectural Isolation to Prove That Data, Not Architecture, Explains Performance

This is a methodological innovation rather than a technical one, but it is central to the paper's argumentative force. The inclusion of MolmoBot-Pi0—a model that exactly replicates the π0 architecture but is trained from scratch on MolmoBot-Data—is a rare example of a clean controlled experiment in a field where architectural improvements and dataset improvements are typically confounded. When a new model outperforms a prior model, it is usually impossible to determine whether the gain comes from better architecture, better training recipe, or better data. By holding architecture constant and varying only the data, MolmoBot-Pi0 isolates the data effect.

The result is decisive: MolmoBot-Pi0 achieves 46.7% real-world success rate compared to π0 at 39.2% (Figure 7), both using the same architecture. The paper explicitly draws the conclusion: "this difference in performance can only be explained by data" (Section 5.1.2). This is not a statement about MolmoBot-Pi0 being the best policy—it is substantially worse than MolmoBot's 79.2%—but rather a statement about what drives progress in robot learning. It suggests that the field's focus on architectural innovations (flow matching vs. diffusion, DiT vs. autoregressive, VLM scale) may be misallocated relative to the gains available from data scale and diversity.

What elevates this beyond a simple ablation is its implication for scientific practice in robot learning. The paper is making a methodological argument: that the field should adopt controlled data-vs-architecture comparisons as a standard evaluation practice, analogous to how computer vision adopted controlled architecture comparisons on standard datasets. Without such controls, we cannot know whether a new model's improvements come from better feature extraction, better action representations, or simply more/better training data. The MolmoBot-Pi0 experiment demonstrates that such controls are feasible—even when comparing to commercial models with partially closed training recipes, the architecture can be replicated and retrained on open data.

The controlled comparison also serves a community-level function: it demonstrates that open-sourcing data (MolmoBot-Data) enables competitive performance with industrial models (π0) that keep their data closed, without requiring access to those closed datasets. This is a direct challenge to the model of building robotics foundation models behind closed doors, and an argument that open data pipelines can level the playing field.

Innovation 4: Negative Result on Utility of Established Simulation Benchmarks for Zero-Shot Evaluation

This is a diagnostic finding that the paper surfaces in Appendix C, and while it is not presented as a headline contribution, it represents an important methodological insight for the field. The paper evaluates MolmoBot policies on two widely used simulation benchmarks—SIMPLER and LIBERO—and finds that while MolmoBot outperforms baselines, the absolute performance is poor (11.5% on SIMPLER, 36.6% on LIBERO-Object no-distractor), and the benchmarks exhibit structural properties that make them poor instruments for measuring zero-shot generalization.

The diagnosis is specific: these benchmarks measure "familiarity with specific assets and scene configurations rather than zero-shot manipulation capability" (Appendix C). The LIBERO-Object task, for example, requires identifying "alphabet soup" and "bbq sauce" by their specific brand-name visual appearance—a test of whether the model has seen those specific assets during training, not whether it can generalize pick-and-place to new objects. The SIMPLER tasks are tightly coupled to the RT-1/Bridge demonstration datasets they were designed to correlate with, meaning they implicitly assume in-distribution evaluation of policies trained on those specific demonstrations.

This finding matters because it reveals a gap in the evaluation infrastructure for generalist manipulation policies. The field has inherited benchmarks designed for narrow, in-domain evaluation (e.g., measuring whether a policy trained on BridgeData can transfer to simulated Bridge environments) and is repurposing them for zero-shot evaluation of generalist policies. The paper argues—and provides evidence—that this repurposing is invalid: the benchmarks conflate generalization capability with asset familiarity, producing misleadingly low absolute scores for capable policies and potentially distorting research priorities toward matching benchmark-specific distributions rather than building truly general capabilities.

The paper's constructive response is to use MolmoSpaces-based evaluation with 94k+ procedurally generated environments and 11k+ novel object assets, arguing this provides sufficient diversity to measure generalization while maintaining correlation with real-world performance. This is not a fully validated alternative—the paper does not provide a systematic comparison of benchmark-to-real correlation across multiple evaluation frameworks—but it identifies a problem that the field needs to address: we lack good instruments for measuring zero-shot manipulation generalization.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All experiments use MolmoBot-Data for training (1.7M episodes across 94.2k environments, 11.4k objects, 8 tasks, 2 embodiments), with evaluation on held-out procedural houses with asset instances unseen during training. Additional evaluation is conducted on external benchmarks SIMPLER and LIBERO reimplemented for the DROID Franka FR3 platform (Appendix C). Real-world evaluation is performed across 4 environments for the Franka FR3 (kitchen, workroom, bedroom, and office at two different institutions) and 3 environments for the RB-Y1 (each containing distinct pull doors).

• Base models. The MolmoBot family uses Molmo2-4B as the VLM backbone, pretrained on large-scale image-text data, with the vision encoder and projector weights frozen during training. MolmoBot-Pi0 uses Paligemma 3B as the VLM backbone, also with frozen vision encoder, initialized from Paligemma weights and trained from scratch on MolmoBot-Data. MolmoBot-SPOC uses SigLIP2-Base for visual encoding and SigLIP text encoder for language, with no pretrained VLM. The choice of Molmo2-4B is motivated by its "video understanding and grounding" capabilities (Section 1), enabling multi-frame visual context, while Paligemma 3B enables direct architectural comparison to π0.

• Metrics. The primary metric is success rate—the fraction of episodes meeting task-specific completion criteria. For simulation, the paper reports oracle success rate (task completion at any timestep during the episode) and, for pick-and-place tasks, success at end (success conditions fulfilled at the final timestep). The gap between these captures both unstable placement and the inability of policies to recognize task completion. Real-world success is judged by human evaluators based on object placement criteria (e.g., "object is fully off the table by approximately 2cm or more" for pick tasks). For pick-and-place, success requires the object to be "stably in or on the given receptacle." Confidence intervals are computed via stratified bootstrapping for real-world results (95% CI) and binomial proportion for simulation results.

• Baselines. The paper compares against several external vision-language-action models evaluated zero-shot: π0 (Black et al., 2024), π0.5 (Black et al., 2025)—the SOTA open-weights generalist manipulation policies trained on >10k hours of real-world demonstrations, representing the most directly comparable baselines since they share architectural elements with MolmoBot variants. StereoVLA (Deng et al., 2025), LAP-VLA (Zha et al., 2026), and X-VLA (Zheng et al., 2025) are evaluated zero-shot in simulation. Additionally, π0.5-Finetune—π0.5 fine-tuned on MolmoBot-Data for 15K steps—is included as a stronger baseline that adapts a real-world-trained policy to simulation. DreamZero (Ye et al., 2026) is evaluated zero-shot in the restricted camera setup. For the RB-Y1, no prior generalist baselines exist.

• Generation budget / compute accounting. Evaluation is measured in episodes or trials rather than compute FLOPs, since all models are evaluated at inference time with fixed architectures. Simulation evaluations use 1,000–2,000 episodes per task (Table 6 specifies exact counts), real-world evaluations use 120 trials for DROID pick-and-place (4 environments × 10 tasks × 3 trials) and 9 trials for RB-Y1 door opening (3 environments × 3 trials). For pick tasks in the real-world ablation studies, evaluations use 6 objects, 3 groups of 3 objects, 5 trials each = 30 trials (Section 5.3). Inference runs at 15 Hz for Franka FR3 (matching DROID configuration) and 100 ms per timestep for RB-Y1 real-world (slower than simulation for safety). In simulation, the inference dt is 800 ms (8 predicted actions executed open-loop, each at 100 ms).

• Cross-validation / statistical protocol. The paper uses 95% confidence intervals computed via stratified bootstrapping for real-world results (Figure 7) and binomial proportion confidence intervals for simulation results (Table 6 and Figure 8). For ablation studies, error bars represent 95% binomial proportion confidence intervals (Figures 8 and 9). The paper does not use cross-validation for strategy selection as these are zero-shot evaluations; instead, the held-out environments and unseen objects provide the generalization test.

Main Quantitative Results

Zero-Shot Real-World Static Manipulation (DROID Franka FR3)

Headline result. MolmoBot (F=2) achieves 79.2% success rate on real-world tabletop pick-and-place across 4 environments (120 total trials), substantially outperforming π0.5 at 39.2% and π0 at 12.5% (Figure 7, Table 10). This represents a >2× improvement over the strongest available open-weights generalist policy trained on >10k hours of real-world data.

Per-environment breakdown (Table 10). The performance is consistent across environments but with notable variation indicative of environmental difficulty:

• Workroom (10 tasks): MolmoBot achieves 90% (27/30 trials), MolmoBot-Img 77%, MolmoBot-Pi0 60%, π0.5 27%, π0 3%. This environment features simple object-receptacle pairings on a tabletop workspace with minimal clutter.
• Kitchen (10 tasks): MolmoBot achieves 70%, MolmoBot-Img 87% (the only environment where Img outperforms the multi-frame variant), MolmoBot-Pi0 53%, π0.5 63%, π0 20%. The kitchen includes both "easy" and "hard" object placements relative to receptacles, and clutter trials where multiple objects are present.
• Bedroom (10 tasks): MolmoBot achieves 87%, MolmoBot-Img 67%, MolmoBot-Pi0 23%, π0.5 10%, π0 0%. The bedroom notably uses a bed as the support surface rather than a table, testing robustness to surface type variation.
• Office (10 tasks): MolmoBot achieves 70%, MolmoBot-Img 60%, MolmoBot-Pi0 50%, π0.5 57%, π0 13%. This environment was at an entirely different geographical location and institution, testing environmental robustness. π0.5 performs relatively better here (57%) than in the bedroom (10%), suggesting environment-specific strengths.

Controlled architecture comparison. MolmoBot-Pi0 (46.7% average) uses the identical π0 architecture trained from scratch on MolmoBot-Data. Its 7.5 percentage point improvement over π0 (39.2%) demonstrates that data alone accounts for a substantial portion of the performance difference between MolmoBot and π0-family models. However, MolmoBot's additional 32.5 percentage point advantage over MolmoBot-Pi0 (79.2% vs. 46.7%) indicates that the MolmoBot architecture (layerwise cross-attention, Molmo2 backbone, multi-frame context) provides substantial complementary gains.

Task-level failure analysis (inferred from Table 10). The per-task trial breakdown reveals several patterns:

• Spoon/tape tasks (Workroom): These consistently high-performing tasks (26/30 MolmoBot successes) use rigid, graspable objects with distinct visual appearances (wooden spoon, blue tape) and clear receptacle distinctions (light blue tray vs. wooden box).
• Clutter tasks (Kitchen, Bedroom): These tasks where multiple objects are present on the workspace show lower performance. Kitchen Clutter Brown and Clutter Black achieve 0/3 each for MolmoBot, while Bedroom Clutter tasks achieve 6/6. This suggests the kitchen clutter configuration is particularly challenging (possibly due to object placements that create visual ambiguity), while bedroom clutter is manageable.
• Mug tasks (Workroom, Kitchen): Copper mug placement in Workroom is notably difficult for all models—MolmoBot achieves only 1/3 on Copper Mug Box. The copper reflective surface may cause visual confusion, or the mug's cylindrical shape may produce ambiguous grasp orientations.

Restricted Camera Evaluation (Single Fixed-Shoulder Camera)

Headline result. In the restricted single-camera setup (Table 7), MolmoBot-Img achieves 86.6% real-world success on the Pick Kitchen task (30 trials), compared to π0.5 at 63.3% and π0 at 20.0%. The multi-frame MolmoBot variants perform worse than the single-frame variant in this specific setting: MolmoBot (F=2) achieves 70.0%, MolmoBot (F=3) achieves 73.3%.

Simulation evaluation (Pick MSProc, 1000 episodes). Under the same restricted camera setup: MolmoBot variants achieve 91–93%, MolmoBot-SPOC 70.4%, MolmoBot-Pi0 66.2%, π0.5-Finetune 48.0%, DreamZero 44.3%, π0.5 18.1%, LAP-VLA 19.4%, StereoVLA 6.6%. The rank ordering is preserved across simulation and real-world evaluation, though absolute real-world performance is lower for all models. The gap between simulation and real-world performance varies by model: MolmoBot-Img drops only ~5.6 percentage points (92.2% → 86.6%), while MolmoBot (F=2) drops ~23.5 points (93.5% → 70.0%), suggesting the multi-frame variant may be overfitting to temporal patterns specific to simulation.

Zero-Shot Real-World Mobile Manipulation (RB-Y1 Door Opening)

Headline result. On the door opening task (Table 5), the MolmoBot Door Specialist achieves 4/9 trials with handle grasp success and 2/9 trials with door opening success across three distinct pull doors. The small sample size (9 trials total) limits statistical confidence, but the results reveal known failure modes and hardware limitations.

Per-door analysis.

• Door 1 (3 trials): 1/3 handle grasp (with hardware fault), 0/3 door opening. Failure modes: base collision (1 trial), joint limit reached (1 trial). This door has a right-side handle configuration that is underrepresented in training data.
• Door 2 (3 trials): 3/3 handle grasps, but all 3 trials experienced hardware faults (emergency stops triggered by the robot's safety system) during either grasp or opening phases. In trials 2 and 3, the fault occurred during opening but the robot had already successfully grasped the handle and was able to complete opening despite the fault, suggesting the trajectory was committed before the fault. Door 2's handle configuration was more commonly represented in training data.
• Door 3 (3 trials): 0/3 handle grasps. Failure modes: base collision (1 trial), incorrect gripper orientation (1 trial), hardware fault during door approach (1 trial). This door also has a right-side handle configuration.

Interpretation. The authors identify the right-side handle configuration as a potential distribution shift from training data, which may explain the reduced grasping reliability on Doors 1 and 3. Hardware faults (the robot triggering its own emergency stop) were a recurring issue that affected 4 of 9 trials, introducing a confound unrelated to policy competence. The 2/9 door opening success rate, while modest, demonstrates feasibility of zero-shot mobile manipulation transfer from simulation.

Simulation Evaluation: Comprehensive Task Suite (Franka FR3)

Headline result. Averaging across all 7 simulation tasks (Table 6), MolmoBot (F=2) achieves 64.1% oracle success compared to 10.0% for π0.5 zero-shot, 41.5% for MolmoBot-Pi0, and 36.0% for π0.5 after fine-tuning on MolmoBot-Data. The rank ordering is consistent: MolmoBot (F=2) ≈ MolmoBot-Img ≈ MolmoBot (F=3) > MolmoBot-Pi0 > π0.5-Finetune >> π0.5 > LAP-VLA > StereoVLA ≈ X-VLA.

Task-by-task breakdown (Table 6).

Pick MSProc (easiest task, 1000 episodes, MuJoCo rendering): This task uses the simplest configuration with limited object diversity from the iTHOR dataset and DROID-standard camera setup. MolmoBot (F=2) achieves 93.5%, MolmoBot-Img 92.2%, MolmoBot (F=3) 91.3%, MolmoBot-Pi0 66.2%, π0.5-Finetune 48.0%, π0.5 18.1%, LAP-VLA 19.4%, StereoVLA 6.6%. All MolmoBot variants achieve >90%, indicating the pick task is largely solved under in-distribution visual conditions.

Pick Classic (Objaverse objects, MuJoCo rendering, 1000 episodes): Introducing diverse Objaverse objects with the standard renderer reduces performance substantially: MolmoBot (F=2) 66.8% (down 26.7 points from Pick MSProc), MolmoBot-Img 63.5%, MolmoBot (F=3) 63.8%, MolmoBot-Pi0 35.7% (down 30.5 points), π0.5-Finetune 28.3%, π0.5 6.4%. The ~30-point drop for MolmoBot variants between MSProc objects and Objaverse objects indicates that object visual diversity is a significant challenge even in simulation. The smaller drop for MolmoBot-Pi0 (30.5 points vs. 26.7) is offset by its lower starting point.

Pick (filament rendering, 1000 episodes): Switching to photorealistic filament rendering—which is out of distribution for training (the models were trained on MuJoCo rasterization)—produces an additional small drop: MolmoBot (F=2) 64.0%, MolmoBot-Img 61.4%, MolmoBot (F=3) 59.0%, MolmoBot-Pi0 33.3%, π0.5-Finetune 25.8%, π0.5 7.0%. The relatively small degradation (2–5 points) from the rendering change suggests robustness to visual appearance, consistent with the domain randomization strategy.

Pick Random-Cam (filament rendering, heavily randomized camera viewpoints, 1000 episodes): Further randomizing camera positions tests viewpoint invariance: MolmoBot (F=2) 63.7%, MolmoBot-Img 62.1%, MolmoBot (F=3) 62.7%, MolmoBot-Pi0 39.8%, π0.5-Finetune 29.7%, π0.5 8.0%. Performance is nearly identical to the standard filament rendering, indicating camera viewpoint robustness is successfully learned. This is a key validation of the camera perturbation strategy described in Section 3.3: the model truly generalizes across viewpoints rather than relying on a fixed perspective.

Pick&Place (filament rendering, 1000 episodes): Adding the placement phase increases task difficulty. MolmoBot (F=2) achieves 66.4% oracle / 57.7% final success, MolmoBot-Img 63.0% / 55.0%, MolmoBot (F=3) 65.4% / 55.6%, MolmoBot-Pi0 44.7% / 38.2%, π0.5-Finetune 43.5% / 37.4%, π0.5 11.7% / 7.6%. The ~8–9 point gap between oracle and final success is consistent across models, indicating that about 12–14% of initially successful placements are subsequently disrupted (the object is picked up again or knocked over after placement). This is a known weakness of behavior cloning policies without explicit termination detection.

PnP Next-To (filament rendering, 1000 episodes): Placing an object next to a target within 5 cm is the most challenging task: MolmoBot (F=2) 26.4% oracle / 20.2% final, MolmoBot-Img 21.0% / 16.4%, MolmoBot (F=3) 28.3% / 22.6%, MolmoBot-Pi0 24.7% / 13.3%, π0.5-Finetune 28.4% / 14.7%, π0.5 8.2% / 6.2%. This task requires precise spatial reasoning (placing within 5 cm of another object) that appears to challenge all models substantially. Notably, π0.5-Finetune achieves the highest oracle success (28.4%) but lower final success (14.7%), suggesting it finds the placement but frequently disrupts it. MolmoBot (F=3) achieves the highest final success at 22.6%, indicating better termination behavior.

PnP Color (filament rendering, 1000 episodes): The color-conditioned placement task requires identifying receptacles by color attribute: MolmoBot (F=2) 67.8% oracle / 60.0% final, MolmoBot-Img 67.8% / 60.3%, MolmoBot (F=3) 66.1% / 57.3%, MolmoBot-Pi0 46.2% / 40.0%, π0.5-Finetune 48.3% / 38.9%, π0.5 10.4% / 6.7%. Performance is comparable to standard Pick&Place (within 2–4 points for MolmoBot variants), indicating that the color attribute understanding transfers well from the language module.

Cross-task patterns. Several patterns emerge from the full evaluation suite:

• Multi-frame provides modest gains: MolmoBot (F=2) outperforms MolmoBot-Img by 2.5 points on average across tasks, but the gain is inconsistent—F=2 is 3.8 points higher on PnP Next-To oracle but 1.4 points lower on Pick Filament. The benefit of temporal context is task-dependent.
• Recognition-heavy tasks show larger VLM benefits: On Pick MSProc (simple objects), MolmoBot (F=2) achieves 93.5% vs. MolmoBot-Pi0 at 66.2%—a 27.3 point gap. On PnP Next-To (spatial reasoning), the gap narrows to 1.7 points (26.4% vs. 24.7%). This suggests the MolmoBot VLM provides the largest benefits for object recognition and language grounding, while spatial reasoning remains challenging regardless of architecture.
• Other VLAs fail catastrophically: StereoVLA, LAP-VLA, and X-VLA achieve near-zero or single-digit performance on most tasks (X-VLA averages 1.2% across all tasks), indicating these models rely heavily on their specific training distributions and do not generalize to novel objects and environments.

Simulation Evaluation: Mobile Manipulation (RB-Y1)

Headline result. Table 8 reports zero-shot simulation performance for RB-Y1 policies. MolmoBot Multitask achieves 44.8% on Pick, 22.5% on Pick&Place, 25.2% on Open, and 70.2% on Door Open. The Door Specialist model (trained only on door data) achieves 77.7% on Door Open. MolmoBot-SPOC variants achieve substantially lower performance: MolmoBot-SPOC Rigid 10.5% on Pick, 1.8% on Pick&Place; MolmoBot-SPOC Articulated 21.8% on Open, 58.8% on Door Open.

Interpretation. The large gap between MolmoBot Multitask and MolmoBot-SPOC (34.3 points on Pick, 20.7 points on Pick&Place) reflects both the VLM's stronger visual representations and the benefit of multitask training. The Door Open task shows the highest success (70–78%), possibly because door opening provides strong visual cues (the door plane, handle, and hinge provide clear geometric structure) and the articulation trajectory is constrained by the hinge physics. The Open task (cabinets, drawers) is substantially harder (25.2%), likely due to greater diversity in handle positions, joint types (revolute vs. prismatic), and approach angles. The paper notes that MolmoBot-SPOC's "compact scale enables future on-policy reinforcement learning in simulation, which has been shown to yield substantial performance gains" (Section 5.2.1), framing these lower numbers as a starting point for RL fine-tuning rather than a final result.

External Benchmark Evaluation: SIMPLER and LIBERO

Headline result. On SIMPLER-DROID (Table 11), MolmoBot-Img achieves 45.8% on "put carrot on plate" but 0% on the remaining three tasks, averaging 11.5%. All baselines achieve 0% on all tasks. On LIBERO-Object (Table 12), MolmoBot-Img achieves 36.6% average success in the no-distractor setting, compared to π0.5-DROID at 2.4% and π0-DROID at 0.2%. With distractors, performance drops substantially: MolmoBot-Img 8.0%, MolmoBot-Pi0 0.8%, π0.5 1.0%, π0 0%.

Task-level variance on LIBERO (no distractor). Performance varies dramatically across the 10 LIBERO-Object products: chocolate pudding 78%, tomato sauce 52%, orange juice 44%, ketchup 42%, BBQ sauce 42%, salad dressing 40%, milk 26%, cream cheese 24%, alphabet soup 16%, butter 2%. This 76-point range across products that are all "grasp and place in basket" tasks is attributed to "incidental overlap with benchmark-specific assets rather than general manipulation competence" (Appendix C). Products with distinctive visual appearances (chocolate pudding) succeed frequently, while those with generic packaging (butter) fail almost entirely.

Diagnostic conclusion. The paper argues that these benchmarks "measure familiarity with specific assets and scene configurations rather than zero-shot manipulation capability." The evidence is the 76-point task variance on a benchmark where all tasks ostensibly require the same manipulation skill, and the near-zero performance of all models on SIMPLER despite demonstrated real-world competence. The paper positions MolmoSpaces-based evaluation as a more valid instrument for measuring generalization.

Ablation Studies and Robustness Checks

All ablations in this section use the MolmoBot-Img model trained for 24K steps at batch size 512 on the pick task, evaluated on Pick-Classic (simulation, 1000 episodes) and on the workroom environment (real-world, 30 trials with 6 objects, Section 5.3).

Scaling number of demonstrations (Figure 8a): Performance improves monotonically with data scale in both simulation and real-world settings. Training on 50K trajectories (vs. 10K) improves real-world pick success by approximately 20–25 percentage points (exact numbers not reported in text; estimated from Figure 8a). Simulation success also increases, though the curve appears to be approaching saturation between 25K and 50K. The paper states this is a "predictable" scaling trend—more data helps, with simulation approaching ceiling effects at 50K trajectories while real-world continues to improve, suggesting the sim-to-real gap narrows with scale but does not close.

Scaling object diversity (Figure 8b): Increasing the number of unique object classes from 5 to 100 (while holding total trajectories at 50K) improves simulation performance predictably, but does not show a clear trend for real-world performance. The paper hypothesizes this is because "the number of objects in the evaluation is limited and semantically common such as apple and cup"—the real-world test set of 6 objects is too small and generic to differentiate object diversity effects. This is a notable negative finding: it suggests that for real-world transfer with common household objects, the sheer number of unique object classes may matter less than the diversity of other factors (viewpoints, lighting, backgrounds). However, the paper acknowledges the evaluation objects are "semantically common," meaning they likely have high visual similarity to objects in even the smallest object sets—a test with unusual or visually distinctive objects might show different results.

Scaling environment diversity (Figure 8c): Increasing the number of unique house environments from 50 to 5,000 (holding total trajectories at 50K) has little effect on performance in either simulation or real-world. This is described as "unexpected" by the authors and represents the most surprising ablation result. The paper suggests that "for the pick task, performance is driven more by the total amount of interaction data than by scaling environment diversity." This finding is specific to the pick task: background diversity may be irrelevant because the policy learns to attend to the object and gripper while ignoring background walls, furniture, and room layouts. This would not necessarily hold for tasks requiring environmental reasoning (navigation, mobile manipulation where room layout affects path planning, or tasks with occlusions).

Multiple denoising timesteps during training (Figure 9a): Training MolmoBot-Img with T ∈ {1, 2, 4, 8} denoising timesteps sampled per example shows that simulation performance improves monotonically with T and peaks at T = 8. Real-world performance peaks at T = 4, with T = 8 performing slightly worse. The paper describes the real-world trend as "not as clear" while simulation shows consistent improvement. The divergence between simulation and real-world optimal T suggests that training with more timesteps per example may improve denoising accuracy on the training distribution (benefiting simulation evaluation) but may slightly overfit to simulation-specific noise patterns (slightly hurting real-world transfer). The magnitude of the effect is modest: the difference between T = 4 and T = 8 in real-world appears to be ~5–10 percentage points (estimated from Figure 9a).

Action representation: absolute vs. delta joint positions (Figure 9b): For Franka FR3 policies trained for 200K steps on the full multi-task data mix, the absolute joint position representation substantially outperforms delta actions in real-world evaluation, while the two representations perform similarly in simulation. Exact real-world success rates: the absolute policy achieves approximately 70–75% (estimated from Figure 9b) while the delta policy achieves approximately 30–45% across benchmarks. The paper describes this as a "significant gap" that "strongly suggests that absolute joint policy models transfer better to real world tasks." The simulation equivalence indicates this is not a capacity issue (both representations can fit the training data) but rather a transfer issue: delta actions may accumulate errors during open-loop execution (each delta prediction error compounds with the next), while absolute actions are self-correcting (each prediction specifies a target configuration regardless of previous errors). This finding explains the design choice to use absolute actions for Franka policies.

Data mixing ratios (implicit ablation): While not presented as a formal ablation, the paper reports different data mixtures for different models (Tables 3 and 4). The Franka mixture emphasizes Pick-and-place Random Height (35%) over Pick (20%), despite Pick having many more episodes in the dataset (781.8k vs. 554.2k for Pick-and-Place). This up-weighting of pick-and-place data relative to its natural frequency is a design choice that likely improves task diversity during training, but no ablation comparing different mixture ratios is reported, so the sensitivity to this choice is unknown.

ReST^EM revision training negative result (Appendix K): An attempt to further optimize a revision-style policy using ReST^EM (a reinforcement learning fine-tuning method) substantially hurt performance. At 256 generations, fully sequential performance with the ReST^EM model drops to approximately 33.5% compared to roughly 38.5% at the optimal ratio. The authors hypothesize that "on-policy data collection in ReST^EM exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly." This negative result highlights the sensitivity of behavior cloning-based training to data distribution and the potential for iterative self-improvement methods to backfire when the proposal distribution drifts from the training distribution.

Critical Assessment

Claim 1: "Sufficiently large-scale and diverse simulated synthetic training data enables zero-shot transfer to the real world for static and mobile manipulation."

The Franka FR3 results strongly support the static manipulation portion of this claim. The 79.2% real-world success rate across 4 environments, each with different room layouts, surface types, lighting conditions, object sets, and in one case a different geographical location and institution, demonstrates generalization that cannot be explained by narrow overfitting to a specific visual domain. The controlled comparison with π0.5 (39.2%) on identical tasks strengthens the evidence by showing that even SOTA real-world-trained policies underperform the simulation-only approach.

However, the mobile manipulation (RB-Y1) evidence is substantially weaker. The door opening evaluation consists of only 9 trials across 3 doors, with 4 trials affected by hardware faults and only 2/9 achieving door opening. This is not a convincing demonstration of "effective" transfer—it is a feasibility demonstration at very small scale with heavily qualified results. The real-world evaluation for other RB-Y1 tasks (pick, pick-and-place, open) is not reported, leaving the mobile manipulation claim largely supported only by simulation results (Table 8) where MolmoBot Multitask achieves 44.8% on Pick, 22.5% on Pick&Place, 25.2% on Open, and 70.2% on Door Open. Simulation success does not guarantee real-world success, as the authors themselves demonstrate with the gap between simulation and real performance for the Franka policies.

Claim 2: "MolmoBot achieves a success rate of 79.2% in real world evaluations across 4 settings, outperforming π0.5 at 39.2%."

The numbers are accurately reported from the experiments (Figure 7, Table 10). However, context matters for interpretation. The 79.2% figure is averaged across 4 environments with heavily varying difficulty—from 90% in Workroom to 70% in Kitchen and Office. This is a fair average, but the wide range (20 percentage points) indicates that performance is environment-dependent. The comparison to π0.5 at 39.2% is valid but benefits from evaluating π0.5 zero-shot on tasks that may differ from its training distribution. π0.5 was not trained on MolmoBot-Data tasks; it was evaluated on the authors' task specifications using the authors' prompts. While the paper reports that π0.5-Finetune (fine-tuned on MolmoBot-Data) achieves 36.0% average simulation success (Table 6), its real-world performance is not reported—this would be the fairer baseline for understanding how much the data matters vs. the architecture, since it would control for task distribution.

Claim 3: "MolmoBot-Pi0's 46.7% success rate using identical architecture trained from scratch on MolmoBot-Data demonstrates that the performance gains stem primarily from data scale and diversity rather than architectural differences."

This claim is partially supported but overstated. Yes, MolmoBot-Pi0 (46.7%) outperforms π0 (39.2%), demonstrating that data contributes. But MolmoBot (79.2%) outperforms MolmoBot-Pi0 by 32.5 percentage points, demonstrating that architecture also contributes substantially. The word "primarily" requires that the data effect dominates the architecture effect, but the numbers show architecture contributes roughly as much: data upgrade (π0 → MolmoBot-Pi0) adds ~7.5 points, architecture upgrade (MolmoBot-Pi0 → MolmoBot) adds ~32.5 points. This suggests data and architecture are both important, with architecture (VLM choice, layerwise cross-attention, multi-frame context) potentially contributing more. The paper's language in the abstract ("performance gains stem primarily from data") appears in tension with its own numbers.

The stronger interpretation is that both data and architecture matter, and that good data cannot compensate for a weak architecture (MolmoBot-Pi0 is substantially worse than MolmoBot), nor can a strong architecture compensate for insufficient data (π0 is worse than MolmoBot-Pi0 despite using the same architecture as the latter). The complementarity is the actual finding, not data primacy.

Missing experiments that would strengthen the paper:

• Real-world RB-Y1 evaluation beyond door opening. The paper claims mobile manipulation transfer but evaluates only one mobile task in the real world, with minimal statistical power (9 trials). Real-world evaluation of RB-Y1 pick-and-place or drawer/cabinet opening would substantially strengthen the mobile manipulation claim.

• Scaling laws for data diversity. The ablations in Figure 8 show scaling trends for data quantity, object diversity, and environment diversity, but only up to 50K trajectories for the pick task. The full dataset has 1.7M trajectories—experiments at larger scale would reveal whether performance saturates or continues to improve, and at what point additional data provides diminishing returns. This is especially relevant given the paper's central thesis about scale.

• Ablation of domain randomization components. The paper attributes sim-to-real transfer to "domain randomization across lighting, textures, friction, and camera pose" (abstract), but no experiment isolates which randomization components are necessary. Does removing lighting randomization hurt real-world transfer? Does removing texture randomization? Without such ablations, the claim about what drives transfer is a hypothesis rather than a demonstrated fact.

• Comparison to real-world fine-tuned π0.5 on the exact same tasks. The paper reports π0.5-Finetune simulation results but not real-world results. This would provide the cleanest test of whether MolmoBot-Data alone can match real-world-to-sim adaptation followed by real deployment.

• Statistical significance testing. The paper reports 95% confidence intervals but does not perform formal hypothesis tests comparing models. Given the small real-world sample sizes (30 trials per condition in ablation studies, 120 trials total for DROID pick-and-place), confidence interval overlap may be substantial, and some reported differences may not be statistically significant. For example, in Figure 7, the error bars for MolmoBot-Img and MolmoBot overlap substantially—is the 6.7 point difference in their means distinguishable from noise?

• Generalization across robot platforms within the same task. The paper trains on both Franka and RB-Y1 but does not evaluate whether a policy trained on one embodiment transfers to the other (cross-embodiment transfer), which would test whether the data diversity enables embodiment-invariant representations.

Conditional boundaries on the claims. The paper's central thesis—simulation diversity enables zero-shot transfer—comes with important scope limitations that are acknowledged but not experimentally verified:

1. Task class boundary. The paper limits claims to "rigid body and articulated object manipulation" where "modern simulators provide sufficient fidelity for transfer." The ReST^EM negative result (Appendix K) suggests this boundary is real—more complex training procedures that push the model toward behaviors not well-modeled by the simulator may fail. The door opening real-world failures due to handle configuration bias suggest that even within the articulated object domain, simulation diversity must adequately cover the deployment distribution.

2. Visual diversity boundary. The ablation showing no benefit from environment diversity scaling (Figure 8c) suggests that for tabletop pick tasks, the policy learns to ignore backgrounds. This is task-specific: for mobile manipulation where the robot must navigate around furniture, environment diversity would likely matter. The paper does not run the equivalent ablation for mobile tasks.

3. Object appearance boundary. The LIBERO results (Table 12) show 76-point variance across visually similar grasp-and-place tasks, demonstrating that visual appearance of specific objects (brand logos, packaging design) strongly affects performance. The paper's claim of broad generalization is true on average but has high variance across object instances, suggesting that simulation diversity helps but does not eliminate sensitivity to object appearance.

4. Hardware interaction boundary. The RB-Y1 door opening results (Table 5) reveal that real-world factors outside the policy's control—hardware emergency stops, joint limit configurations, base collisions—can dominate failure modes. The paper's claim of "effective" transfer must be qualified by these real-world integration challenges, which cannot be addressed by simulation data alone.

6. Limitations and Trade-offs

6.1 The Difficulty Estimation Cost Is Unmeasured and Potentially Dominant

The assumption or constraint. The paper's central claim—that MolmoBot-Data enables zero-shot sim-to-real transfer—relies on a specific pipeline for generating diverse training data at massive scale. The paper reports generation throughput of ~6,500 GPU-hours for 1.7M trajectories and highlights this as cost-effective relative to real-world data collection (Section 3.6). However, the paper does not account for the computational and engineering cost required to build and operate the data generation infrastructure itself—MolmoSpaces environment construction, grasp dataset precomputation, motion planning with CuRobo, scene randomization, and the filtering/rejection logic that discards failed episodes.

This is not a minor overhead. The paper reports that approximately 50% of generation time is spent on "scene setup and task sampling" rather than rollouts (Section 3.6), meaning the infrastructure cost is roughly equal to the trajectory generation cost. The 6,500 GPU-hour figure also assumes access to and familiarity with a specific stack: MuJoCo, CuRobo, MolmoSpaces, the DROID configuration, and the RB-Y1 URDF—a complexity barrier that the paper acknowledges implicitly when it states that "additional robot platforms can be easily added by future work" (Section 3.2).

The consequence. For a practitioner who wants to deploy the approach on a new robot, new task, or new set of objects, the cost is not "6,500 GPU-hours to replicate our results" but rather "6,500 GPU-hours plus the cost of learning MolmoSpaces, integrating a new robot URDF into CuRobo, precomputing grasps for new objects, tuning the retry and randomization parameters, and debugging the inevitable simulator-robot mismatch." If any of these components fail to transfer—if CuRobo planning does not work well with a new robot's kinematics, or if the new objects lack watertight collider meshes, or if the task requires physics beyond MuJoCo's fidelity—the approach may require substantial engineering investment without guaranteed results.

The paper's headline throughput number (2.6× human demonstrator throughput) also compares against ALOHA as a baseline, but the ALOHA benchmark assumes a 1/3 real-time factor due to reset overhead (Section 3.6). This comparison is generous: human demonstration collection does not require 100 A100 GPUs, a team with CuRobo expertise, or a custom procedural generation pipeline. The true comparison should account for the time to build the pipeline, not just the time to run it.

What evidence exists in the paper. The throughput analysis in Section 3.6 provides the raw generation cost (6,500 GPU-hours, ~660 episodes per GPU-hour) but does not measure the human engineering cost of building MolmoBot-Engine, the compute cost of precomputing grasps for 11k+ objects (which must be done before data generation can begin), or the operational cost of managing parallel GPU generation jobs. The paper's statement that MolmoBot-Engine is "fully extensible" (Figure 1) and that "additional robot platforms can be easily added" (Section 3.2) is asserted but not demonstrated—no experiment trains on a third robot platform added by external users, and no time-to-adapt measurement is provided.

Mitigation status. The paper does not address this limitation directly. The open-sourcing of MolmoBot-Engine partially mitigates the engineering cost for the supported robots (Franka FR3, RB-Y1) and tasks (pick-and-place and articulation), but extending to new domains remains unmeasured. The paper's framing of "fully open-source" (Section 1) addresses data access but not deployment cost. This limitation is structural: the value proposition of simulation-based data is that it scales, but the cost of building the scaling infrastructure itself is a barrier that initial adopters must pay.

---

6.2 Mobile Manipulation Real-World Transfer Is Demonstrated at Insufficient Scale

The assumption or constraint. The paper claims to demonstrate effective zero-shot transfer for mobile manipulation on the RB-Y1 platform, listing door opening and drawer/cabinet manipulation as core tasks (Section 1). The simulation results for the RB-Y1 show non-trivial performance: MolmoBot Multitask achieves 70.2% on Door Open, 44.8% on Pick, and 25.2% on Open in simulation (Table 8). However, the real-world evaluation is extremely limited: only the Door Open task is evaluated, with 9 total trials across 3 doors (Table 5). No real-world evaluation is provided for RB-Y1 pick, pick-and-place, or cabinet/drawer opening.

This is a substantial gap because the paper's thesis—simulation diversity enables zero-shot transfer—is platform-agnostic in principle but requires platform-specific validation. The RB-Y1 platform differs from the Franka FR3 in multiple critical ways: a holonomic base that introduces additional failure modes (base collision, navigation to the task site), dual arms that must be coordinated, a head camera with aggressive fisheye warping, and tasks (door opening) that involve contact with heavy, articulated objects with potentially dangerous dynamics in the real world. The simulation-to-real transfer challenges for mobile manipulation are categorically harder than for tabletop manipulation, and 9 trials cannot validate the claim.

The consequence. The paper's abstract and introduction describe MolmoBot as enabling "zero-shot transfer to the real world" for "both static and mobile manipulation" without qualification. A reader who focuses on the abstract will form the impression that mobile manipulation transfer is comparably validated, which the evidence does not support. The 2/9 door opening success rate (Table 5), with 4/9 trials affected by hardware faults and 3/9 failing to grasp the handle, is more accurately described as a preliminary feasibility result than a demonstration of effective transfer.

The specific failure modes on the RB-Y1 are instructive: base collisions, joint limits reached, incorrect gripper orientation, and hardware emergency stops. These failures suggest that even when the policy produces nominally correct actions, the physical constraints of the real platform—workspace limits, collision geometry not fully captured in simulation, safety system interactions—can prevent success. This is a gap that simulation diversity alone cannot close, because it stems from the fidelity of the robot model in the simulator, not from visual or environmental diversity.

What evidence exists in the paper. Table 5 reports the door opening results with raw trial outcomes. Table 8 provides simulation performance across all RB-Y1 tasks. The text in Section 5.1.3 candidly reports the failure modes and sample size but does not discuss how the limited evaluation affects the strength of the paper's claims about mobile manipulation. The RB-Y1 results are presented alongside the much more extensively validated Franka results without proportional qualification.

Mitigation status. The paper does not claim that mobile manipulation is as thoroughly validated as tabletop manipulation, but it also does not flag the asymmetry clearly in the abstract or introduction. The 2/9 door opening success is presented without a confidence interval (impossible to compute meaningfully from 9 trials), so readers cannot assess statistical reliability. The paper presents simulation mobile manipulation results with the same formatting and prominence as static results (Table 8), which may obscure the lack of corresponding real-world validation for pick, pick-and-place, and cabinet/drawer opening. Future work could close this gap by conducting systematic real-world evaluation of RB-Y1 tasks at a scale comparable to the Franka evaluation (120+ trials across multiple environments).

---

6.3 The Architecture and Data Effects Are Confounded, Undermining the Claim of Data Primacy

The assumption or constraint. The paper makes a specific causal claim in its abstract and introduction: MolmoBot-Pi0's 46.7% real-world success rate, using the identical π0 architecture trained on MolmoBot-Data, "demonstrates that the performance gains stem primarily from data scale and diversity rather than architectural differences" (Section 1, emphasis added). This claim requires that the data effect (the improvement from switching π0's real-world training data to MolmoBot-Data) is larger than the architecture effect (the improvement from switching to the MolmoBot architecture).

The consequence. The numbers in the paper contradict the claim of data primacy. Comparing real-world success rates (Figure 7):

• π0 (real data + π0 architecture): 12.5% (estimated from Figure 7, the paper reports π0 at 12.5% average but this is not broken out explicitly—the bar chart shows π0 substantially below π0.5 at 39.2%)
• MolmoBot-Pi0 (simulation data + π0 architecture): 46.7%
• MolmoBot (simulation data + MolmoBot architecture): 79.2%

The data effect (π0 → MolmoBot-Pi0) adds approximately 34.2 percentage points. The architecture effect (MolmoBot-Pi0 → MolmoBot) adds approximately 32.5 percentage points. These effects are roughly equal in magnitude, not "primarily" data-driven. If anything, for the pick-and-place task the data effect is slightly larger (~34 vs. ~32 points), but this is not "primarily"—it is "roughly equally."

The simulation results in Table 6 compound this ambiguity. Averaging across all tasks, MolmoBot-Pi0 achieves 41.5% vs. π0.5 zero-shot at 10.0% (a ~31.5 point data effect), while MolmoBot (F=2) achieves 64.1% vs. MolmoBot-Pi0's 41.5% (a ~22.6 point architecture effect). Here the data effect is larger. But the choice of baseline matters: the gap between π0.5 (10.0%) and MolmoBot-Pi0 (41.5%) reflects both data differences AND differences in how well the policies were trained (π0.5 was trained for its own tasks, not for MolmoBot's evaluation distribution, so it may be disadvantaged by domain mismatch rather than data quality per se). The paper does not disentangle "data quality/diversity" from "data relevance to evaluation task."

What evidence exists in the paper. Figure 7 (real-world) and Table 6 (simulation) provide the comparison data. The paper's text in Section 5.1.2 states: "MolmoBot-Pi0 significantly outperforms π0 in our evaluations. Therefore, this difference in performance can only be explained by data." This statement is correct about the data effect being significant, but it does not establish primacy. The two effects are quantified separately but their relative magnitude is not formally compared, and the paper's language about primacy is asserted rather than derived from a variance decomposition or effect size analysis.

The simulation results provide some nuance: on Pick MSProc, MolmoBot (F=2) achieves 93.5% vs. MolmoBot-Pi0's 66.2% (a 27.3 point architecture gap), while MolmoBot-Pi0 vs. π0.5 is 48.1 points. On PnP Next-To, MolmoBot (F=2) achieves 26.4% vs. MolmoBot-Pi0's 24.7% (a 1.7 point architecture gap), while the data gap (MolmoBot-Pi0 vs. π0.5) is 16.5 points. The relative importance of architecture vs. data is task-dependent, not a fixed property, which makes a blanket claim of "primarily data" misleading.

Mitigation status. The paper does not provide a formal decomposition of variance attributable to architecture vs. data. It includes MolmoBot-Pi0 as a controlled comparison—which is methodologically strong—but draws a stronger causal conclusion from the comparison than the evidence supports. A more accurate characterization would be: "Both data and architecture contribute substantially to MolmoBot's performance, with the data effect enabling competitive performance with real-world-trained policies and the architecture effect providing additional gains, particularly on recognition-heavy tasks." This limitation is substantive because it affects how readers prioritize their own investments: if data is believed to be the dominant factor, teams may invest primarily in scaling data pipelines when the evidence suggests architecture investments are comparably important.

---

6.4 The Generalization Claim Is Supported by a Single Model Family and Single Task Domain

The assumption or constraint. All experiments—both the data generation pipeline (MolmoBot-Engine) and the policy training—use a single base model family (Molmo2-4B for MolmoBot, Paligemma 3B for MolmoBot-Pi0) and a single task domain (rigid body pick-and-place and articulated object manipulation). The paper states that it "believe[s] this model is representative of the capabilities of many contemporary LLMs" (Section 4), but this belief is unverified. The approach's generality depends on whether other VLM backbones (e.g., LLaMA-based, Gemini-based, different scales) would similarly benefit from simulation-only data at this scale.

Additionally, the task diversity is modest: 8 tasks (Section 3.4), all within the manipulation domain, with no tasks requiring language grounding beyond object/receptacle identification (e.g., no multi-step instruction following, no constraint reasoning like "pick up the object that is not red," no world state queries). The paper's claim that MolmoBot-Data enables "generalist manipulation policies" (Section 1) implicitly suggests transfer to a broader class of manipulation tasks than the 8 evaluated.

The consequence. The paper's conclusions are conditioned on the VLM backbone and task class. A researcher attempting to replicate the approach with a different VLM might find that sim-to-real transfer fails at the same data scale if that VLM's pretrained representations are less robust to the visual domain gap between MuJoCo rendering and real camera images. The paper's design choice to freeze the vision encoder during training (Section 4.1) makes this sensitivity particularly acute: the visual representations are entirely determined by the pretrained model, with no adaptation to the simulated image distribution. If a different VLM's vision encoder produces features that are less invariant between simulation and reality, the scale and diversity of simulation data may not compensate.

The narrow task scope also means the paper has not demonstrated that the approach generalizes to the richness of language instructions that "generalist" policies are expected to handle. The evaluation tasks use template-generated language instructions with constrained referring expressions (Section 3.4, Appendix A.2)—the policy never needs to resolve ambiguous references, follow multi-step instructions, or ground language in world state. Whether the simulation-only approach would succeed on linguistically more complex tasks (e.g., "put the apple in the bowl, but only if the bowl is empty, otherwise use the plate") is untested.

What evidence exists in the paper. The paper's evaluation suite (Table 6) tests generalization across object instances, environments, visual rendering styles, and camera viewpoints—but all within the pick-and-place and articulation domain. The language instructions are procedurally varied but structurally similar. The paper does not evaluate on standard language-conditioned manipulation benchmarks that require richer language understanding (e.g., CALVIN, Language-Table, or RT-1 tasks). The SIMPLER and LIBERO evaluations (Appendix C) attempt to test on external benchmarks, but the paper itself argues these are poor instruments for measuring generalization.

The ablation in Figure 8c shows no benefit from scaling environment diversity for the pick task, suggesting the policy learns to ignore backgrounds and attend only to the object and gripper. This is efficient but also implies the learned representations may not transfer to tasks where background context matters (e.g., "pick up the mug from the desk, not from the shelf").

Mitigation status. The paper does not claim to have solved generalist manipulation across all task types and language complexities; it is scoped to rigid body and articulated object manipulation (Section 3.3: "We focus on rigid body and articulated object manipulation... as these are both tractable to model for modern simulators, as well as interesting and challenging tasks still unsolved"). This scoping is explicit and appropriate. However, the abstract's language about "generalist manipulation policies" and "broadly to the real world" (Section 1) is broader than the evaluated scope, and readers may overgeneralize. The paper's open-sourcing partially mitigates this by enabling the community to test the approach on new tasks and backbones, but this verification is deferred to future work.

---

6.5 Significant Unmeasured Latency and Deployment Overhead

The assumption or constraint. The paper evaluates policies in terms of success rate and data generation throughput, but it does not report inference latency or deployment computational overhead for any of the three policy classes. These numbers matter for real-world robotics: a policy that achieves 79.2% success with 500 ms inference latency may be less deployable than one that achieves 70% with 50 ms latency, particularly for reactive tasks like grasping where the robot must respond quickly to unexpected object motion.

The three architectures have dramatically different computational profiles. MolmoBot runs a full Molmo2-4B VLM forward pass (encoding visual tokens through a language model encoder stack) plus iterative flow-matching denoising through a DiT with layerwise cross-attention. MolmoBot-Pi0 runs a Paligemma 3B VLM forward pass plus flow matching. MolmoBot-SPOC runs a SigLIP2 vision encoder plus a lightweight transformer decoder with parallel action decoding. The relative speed of these models likely differs by 10–50× (SPOC being the fastest, MolmoBot the slowest), but the paper provides no latency numbers, no inference hardware specifications, and no discussion of how inference speed constrains which policies can be deployed on which hardware.

The consequence. The paper's policy comparison (Table 6, Figure 7) treats all models as interchangeable—readers might directly compare MolmoBot's 79.2% to MolmoBot-SPOC's lower simulation performance (Table 7: 36.6% real-world pick) without realizing that MolmoBot-SPOC may run 10–50× faster on edge hardware. For applications where inference latency is critical (e.g., the RB-Y1 real-world evaluation already runs at 100 ms per timestep, slower than simulation's 66.7 ms, for "safety reasons" per Section 5.1.3), the VLA models may be too slow to deploy without hardware acceleration that is not available on the robot itself.

The parallel action decoding in MolmoBot-SPOC and the action chunking (predict 16 actions, execute 8) in all models partially mitigate latency by reducing inference frequency, but this trades off reactivity: an 800 ms chunk (8 actions × 100 ms per action for RB-Y1 simulation) means the policy is blind for nearly a second between observations. For dynamic tasks like grasping a moving object or reacting to an unexpected collision, this latency could cause task failure that a faster reactive policy would avoid.

What evidence exists in the paper. The paper reports inference dt values: 15 Hz (66.7 ms) for Franka FR3 in simulation, 100 ms for RB-Y1 in real-world evaluation, and 800 ms between inferences for simulation evaluation (8 actions executed open-loop per inference). These numbers are scattered across sections (Sections 3.2, 5.1.3, 5.2) and are not consolidated into a latency table. There is no measurement of actual wall-clock inference time on any hardware, no comparison of inference speed across the three architectures, and no ablation studying the effect of inference frequency on task success.

The paper mentions that MolmoBot-SPOC is "suitable for edge deployment" (Section 1) and "amenable to RL fine-tuning" (Section 4.3), which implies latency advantages and trainability, but these claims are not supported by measurements. The "edge deployment" claim cannot be evaluated without knowing what "edge" hardware is assumed (Jetson? Raspberry Pi? Laptop GPU?) and what inference latency MolmoBot-SPOC achieves on that hardware.

Mitigation status. The paper does not address runtime performance as an evaluated metric. This is a common omission in robot learning papers, which typically prioritize success rate over deployability, but it is particularly consequential for a paper that offers three policy classes at different design points and explicitly frames one (MolmoBot-SPOC) as the lightweight option. A latency-vs-success-rate tradeoff curve—or even a single measurement on a standard GPU—would substantially improve the paper's practical utility and allow practitioners to make informed architecture choices based on their deployment constraints.

---

6.6 The Dataset Mixing Strategy Is Heuristic and Under-Ablated

The assumption or constraint. The paper trains policies on specific data mixtures (Tables 3 and 4) that determine how often each task type is sampled during training. For Franka FR3 policies, the mixture is: Pick 20%, Pick-and-place Fixed Height 10%, Pick-and-place Random Height 35%, Pick-and-place-next-to 20%, Pick-and-place-color 15%. For RB-Y1 policies, different mixtures are used for different models (MolmoBot Multitask uses Open 20%, Door-open 20%, Pick 30%, Pick-and-place 30%, while specialist models use 100% single-task data). The paper states that these mixtures were "selected to ensure all coverage of each of the individual task's training set" (Table 3 caption), but provides no experiments justifying these specific ratios.

This is a significant methodological gap because data mixing is a known lever for multi-task policy performance. The paper's central thesis is about data diversity and scale, and the mixing ratios determine precisely which data the model sees. If some tasks are overrepresented relative to their natural frequency (Pick-and-place Random Height at 35% despite being a subset of the 554.2k total Pick-and-Place episodes), the policy's behavior on those tasks may be artificially good, while underrepresented tasks may suffer. The paper evaluates all tasks in simulation (Table 6) but does not measure how performance varies with mixture ratio.

The consequence. A practitioner applying MolmoBot to a new set of tasks must determine their own mixing ratios without guidance from the paper. The paper's mixture choices may be tuned to the specific evaluation tasks, and transferring them to new task distributions could produce unexpected performance degradation. If the optimal mixture ratio is task-dependent—as it likely is—then the paper's headline numbers represent performance under a tuned mixture, not under a principled or generalizable mixing strategy.

The specialist models (MolmoBot Door Specialist at 100% Door-open, MolmoBot-SPOC variants split by task category) also suggest an unresolved question: when is it better to train a specialist vs. a generalist? The Door Specialist achieves 77.7% on Door Open vs. Multitask's 70.2% (Table 8), a 7.5 point improvement from specialization. But the specialist cannot perform any other task. The paper does not explore the tradeoff between multitasks breadth and per-task depth as a function of data scale—a question directly relevant to the paper's thesis about data diversity.

The training upsampling choices (retry grasping 3×, successful pick 2×, task completion 2×, Section 4.1) are also heuristic. The paper states the motivation is to "improve the model's grasping behavior and avoid picking objects after task completion," but provides no ablation of these upsampling weights. If the upsampling is crucial to performance, then the paper's claimed gains from data scale and diversity are partially attributable to careful importance weighting, not just raw data volume.

What evidence exists in the paper. Tables 3 and 4 specify the mixtures without justification beyond the "coverage" rationale. Table 8 shows specialist vs. generalist performance differences but does not systematically vary the mixture ratio and measure effects. The data scaling ablation in Figure 8 varies the number of trajectories but does not vary the task composition of those trajectories. The paper does not report an experiment where the same total data volume is partitioned differently across tasks to measure sensitivity to mixing strategy.

Mitigation status. The paper does not acknowledge data mixing as a hyperparameter requiring tuning, nor does it provide a method for selecting mixture ratios. The open-sourcing of the dataset and training code enables future work to search over mixture ratios, but the computational cost of such a search (multiple full training runs of 200K steps at batch size 1024) is substantial and may be prohibitive for many practitioners. A principled or empirical study of mixing effects—perhaps showing that performance is robust to ratio variation within broad ranges, which would be a positive result—would substantially strengthen the paper's claim that data scale, rather than careful curation, is the driver of performance.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper advances a specific, falsifiable hypothesis about sim-to-real transfer: that the barrier is not an irreducible domain gap but rather insufficient scale and diversity in simulation data. This is not a paradigm shift—the underlying idea that diversity aids generalization is well-established in machine learning—but it is a substantial reframing of resource allocation in robot learning research. The dominant practice in the field has been to treat simulation as pretraining followed by real-world fine-tuning, implicitly accepting that simulation alone cannot produce deployable policies. MolmoBot demonstrates that for a well-defined and practically important class of manipulation tasks (rigid body pick-and-place, articulated object interaction), simulation alone is sufficient when the data pipeline provides combinatorial diversity across environments, objects, physics parameters, camera viewpoints, and lighting conditions, achieving 79.2% real-world success without any real-world data.

The reframing has two concrete consequences. First, it shifts the bottleneck from real-world data collection to simulation infrastructure engineering. If simulation diversity is the limiting factor, then investment should flow toward better procedural generation pipelines, more diverse asset libraries, and more efficient physics simulation—not toward larger teleoperation teams. The paper's reported throughput advantage (88 hours of robot experience per wall-clock hour, 2.6× effective throughput of human demonstrators) quantifies why this shift matters economically, even before accounting for the parallelizability of simulation across GPU clusters.

Second, the MolmoBot-Pi0 controlled experiment reconciles contradictory findings in the VLA literature. Prior work showed both that simulation data can be useful (MimicGen, RoboCasa) and that the strongest generalist policies rely on real-world data (π0, RT-2). The resolution provided by this paper is one of scale: when simulation diversity is limited to hundreds of environments, policies overfit and fail to generalize; when scaled to 94k+ environments with 11k+ objects, the diversity itself bridges the gap. This explains why prior simulation-only approaches underperformed real-world-trained policies: they were operating in a data-scarce regime where generalization failed, not a regime where simulation is fundamentally inadequate. The controlled architectural comparison (MolmoBot-Pi0 at 46.7% vs. π0 at 39.2%, identical architecture, different data) provides direct evidence that data scale, not just architectural sophistication, drives this reconciliation.

The paper also identifies a negative finding that redirects research attention: the failure of established simulation benchmarks (SIMPLER, LIBERO) to meaningfully measure zero-shot generalization. MolmoBot-Img achieves only 11.5% on SIMPLER and 36.6% on LIBERO-Object no-distractor despite 79.2% real-world success, and the 76-point task variance on LIBERO across visually similar grasp-and-place tasks (chocolate pudding 78% vs. butter 2%) demonstrates that these benchmarks measure asset familiarity, not manipulation competence. This finding, while not the paper's headline contribution, should cause the field to reconsider its evaluation infrastructure for generalist policies and may accelerate adoption of procedural evaluation frameworks like MolmoSpaces that control for asset novelty.

Research directions that become more attractive after this work include: scaling simulation data generation to new task classes within the simulator's fidelity envelope (the paper explicitly identifies contact-rich manipulation, deformables, and fluid dynamics as open frontiers), developing automated difficulty estimation from simulation rollout statistics rather than fixed benchmarks, and studying the interaction between VLM backbone choice and simulation-to-real transfer robustness (since the paper freezes the vision encoder, different pretrained encoders may transfer differently). Directions that become less attractive include: investing heavily in photorealism for sim-to-real transfer (the paper shows extensive domain randomization with MuJoCo's simple rasterizer outperforms filament rendering baselines), treating small-scale simulation benchmarks as valid evaluation instruments, and asserting that real-world data is categorically necessary for manipulation without specifying the scale and diversity conditions under which this claim is tested.

---

Follow-Up Research This Work Enables

1. Difficulty estimation from initial rollout statistics to enable adaptive data generation. The paper generates 1.7M trajectories uniformly, but the ablation studies (Figure 8) show that some diversity dimensions matter more than others: environment diversity has negligible effect on pick task performance, while object diversity helps in simulation but not in the paper's real-world evaluation (likely due to the semantically common test objects). A natural extension is to estimate the "effective difficulty" of a task configuration from a small number of simulation rollouts and use that estimate to adaptively allocate data generation budget—generating more trajectories for configurations where the policy currently fails, fewer for configurations it has mastered. This is the data-generation analog of the compute-optimal test-time scaling problem studied in the large-scale simulation paper that serves as this analysis's style reference, applied to the data generation phase rather than the inference phase. A concrete experiment would measure whether adaptive data generation achieves equivalent real-world transfer with substantially fewer total trajectories (e.g., 500K vs. 1.7M) by concentrating effort on the long tail of difficult configurations. The MolmoBot-Engine pipeline enables this experiment by providing per-episode privileged information (grasp success, retry counts, object poses) that can serve as difficulty signals.

2. Systematic study of VLM backbone sensitivity to simulation visual domain shift. The paper freezes the vision encoder during training (both MolmoBot's SigLIP2 and MolmoBot-Pi0's SigLIP), arguing this prevents overfitting to simulation rendering artifacts. This design choice makes the model's sim-to-real transfer entirely dependent on the pretrained encoder's invariance to the MuJoCo-to-real-camera domain gap. A strong follow-up would systematically compare multiple frozen VLM backbones (SigLIP2, DINOv2, CLIP, different scales of each) trained on identical MolmoBot-Data, measuring both real-world success rate and the correlation between encoder-level visual similarity metrics (e.g., CKA similarity between simulation and real image features) and downstream task success. This would answer whether the paper's results depend critically on SigLIP2's specific properties or whether any sufficiently strong pretrained vision encoder works. A negative result—finding that some widely used encoders fail to transfer despite comparable ImageNet accuracy—would identify a previously unrecognized selection criterion for VLA architectures and potentially motivate simulation-aware pretraining objectives.

3. Cross-embodiment transfer from the dual-platform dataset. MolmoBot-Data includes trajectories for both the Franka FR3 and RB-Y1, but the paper trains separate policies for each platform and does not evaluate cross-embodiment transfer. This is a missed opportunity: the dataset provides a controlled setting to test whether a policy trained on both embodiments simultaneously (with embodiment-identifying tokens, as done in X-VLA and RT-X) can transfer manipulation skills from one platform to another, particularly from the simpler Franka to the more complex RB-Y1. A concrete experiment would train a single MolmoBot model on the full dual-platform dataset, evaluate zero-shot on both platforms, and compare to the platform-specific models. If cross-embodiment training improves RB-Y1 performance (particularly on pick-and-place, where the Franka has 554.2k episodes vs. the RB-Y1's 14.8k), it would demonstrate that simulation data from simpler platforms can bootstrap more complex ones—a finding with direct implications for how to scale data generation across a heterogeneous robot fleet.

4. Ablation of domain randomization components to identify necessary vs. sufficient transfer conditions. The paper attributes sim-to-real transfer to "domain randomization across lighting, textures, friction, and camera pose" (abstract) but never isolates which components are necessary. A systematic ablation would train MolmoBot-Img variants where individual randomization axes are disabled (no lighting variation, no texture variation, no friction variation, no camera pose perturbation, no action noise injection) and measure real-world transfer degradation. This experiment would convert the paper's qualitative insight ("diversity matters more than photorealism") into quantitative guidance for practitioners: if disabling texture randomization reduces real-world success from ~72% to ~40% while disabling environment diversity has no effect, a team building a new data pipeline knows where to invest engineering effort. The MolmoBot-Engine's modular randomization pipeline (Section 3.1) makes this ablation straightforward to implement. The negative result—finding that certain randomizations are irrelevant—would be as valuable as the positive result, reducing the engineering burden for adoption.

5. Scaling laws for simulation data quantity and diversity with real-world transfer as the target metric. The paper's data scaling ablation (Figure 8a) shows performance improving from 10K to 50K trajectories but is limited to the pick task and a single model variant (MolmoBot-Img, 24K training steps). A comprehensive scaling study would train MolmoBot variants on dataset sizes spanning orders of magnitude (10K, 50K, 200K, 500K, 1.7M trajectories) with the full training recipe (200K steps, full data mixture), and measure real-world transfer systematically across multiple environments. This would determine whether the paper's 1.7M trajectories are approaching a performance ceiling (where additional data provides diminishing returns) or whether further scaling would yield continued gains. The scaling behavior of different task types (pick vs. pick-and-place vs. door opening) could differ substantially, providing guidance for future data generation priorities. The study would also reveal whether environment diversity scaling—which showed no effect at 50K trajectories for pick (Figure 8c)—becomes important at larger scales or for more complex tasks.

6. Combining MolmoBot's simulation data with a small amount of real-world data to measure the marginal value of simulation diversity. The paper argues that simulation diversity can substitute for real-world data, but a more nuanced question is how they complement each other. A concrete experiment would take a real-world-trained baseline (e.g., a policy trained on 100 real-world demonstrations for a specific task) and co-train it with increasing amounts of MolmoBot-Data, measuring the marginal improvement in real-world success from each increment of simulation data. If adding 10K simulation trajectories to 100 real demonstrations matches the performance of 1,000 real demonstrations, this quantifies the "simulation-to-real exchange rate" directly and would be the strongest possible evidence for the paper's thesis. The experiment would also reveal whether simulation data helps primarily by improving visual robustness (reducing sensitivity to lighting, camera pose, background clutter) or by improving manipulation skill (better grasping, more stable placement), depending on which failure modes are reduced.

---

Practical Applications and Downstream Use Cases

1. Rapid prototyping and evaluation of manipulation policies on new robot platforms. The paper's demonstration that a policy trained entirely in simulation achieves 79.2% real-world success on tabletop pick-and-place means that robotics labs acquiring a new Franka FR3 (or similar DROID-configured arm) can deploy a functional pick-and-place policy immediately, without collecting any demonstrations. The open-sourcing of MolmoBot-Engine and MolmoBot-Data means this deployment requires only: (a) matching the camera configuration to the DROID setup (wrist ZED Mini + one exocentric ZED 2), (b) running MolmoBot inference on a GPU-equipped workstation, and (c) accepting the 79.2% baseline success rate. This dramatically lowers the barrier to entry for manipulation research: instead of spending weeks collecting demonstration data before any policy can be evaluated, a lab can start from a working policy and iterate on improvements. For tasks where 79.2% is already practically useful (e.g., automated data collection for other research, repetitive pick-and-place in lab automation), the out-of-the-box policy may be sufficient without any customization.

2. Data generation for self-improvement loops and reinforcement learning fine-tuning. The paper positions MolmoBot-SPOC as "amenable to RL fine-tuning" (Section 4.3) and notes that its "compact scale enables future on-policy reinforcement learning in simulation" (Section 5.2.1). A practical workflow would: (a) deploy MolmoBot-SPOC (or MolmoBot) in simulation as the initial policy, (b) run on-policy RL in the same simulated environments used for data generation, using the simulator's privileged information (object poses, grasp states) as a reward signal, (c) evaluate the RL-fine-tuned policy in the real world. Because the simulation environments already exist and the reward signal is provided by the simulator, this workflow eliminates the two main bottlenecks for RL in robotics: environment setup and reward engineering. If RL fine-tuning raises MolmoBot-SPOC's real-world pick success from 36.6% (Table 7) to a level competitive with MolmoBot's 79.2%, it would establish simulation-trained behavior cloning followed by simulation-based RL fine-tuning as a complete, real-world-free pipeline for producing deployable manipulation policies.

3. Cost-efficient batch inference for logistics and warehouse automation. The paper's 79.2% pick-and-place success rate, achieved with zero real-world data, has direct implications for structured environments where object diversity is bounded but throughput matters. In a warehouse setting where the robot must pick objects from bins and place them in shipping containers, the combination of known object geometries (which can be accurately simulated), controlled lighting (reducing the sim-to-real visual gap), and the paper's demonstrated robustness to viewpoint variation (Table 6: Pick Random-Cam performance nearly identical to standard Pick) makes simulation-only training particularly advantageous. A company deploying Franka arms for bin-picking could: (a) generate simulation data using CAD models of their specific objects, (b) train MolmoBot on this data using the released pipeline, (c) deploy zero-shot without the weeks of on-site data collection typically required. The $4\times$ throughput advantage of simulation over human demonstration (660 episodes per GPU-hour vs. equivalent real-time factor of 1/3 for human demonstrators, per Section 3.6) translates directly to faster deployment cycles for new product SKUs.

4. Enabling mobile manipulation research on the RB-Y1 platform. While the paper's real-world mobile manipulation results are preliminary (2/9 door opening success, Section 5.1.3), the simulation results (70.2% Door Open, Table 8) and the open-sourcing of the RB-Y1 data generation pipeline lower the barrier for other labs to build on this work. The RB-Y1 is a capable but underutilized platform due to the complexity of data collection (23 DoF, coordination of base, torso, and dual arms). MolmoBot-Engine provides the first open pipeline for generating large-scale training data for this platform, and the simulation-trained policies provide a baseline that future work can improve upon with better architectures, RL fine-tuning, or hybrid sim-real training. The paper's identification of specific failure modes (right-side handle underrepresentation, base collisions triggering emergency stops) also provides a concrete research agenda: addressing these known issues would directly improve real-world success rates. A lab acquiring an RB-Y1 can now start from MolmoBot's simulation-trained policies rather than from scratch, accelerating the research cycle on a platform that was previously accessible only to well-resourced teams.