World Simulation with Video Foundation Models for Physical AI

ArXiv: 2511.00062

🎯 Pitch

A video generation model post-trained with reinforcement learning for physical accuracy can produce synthetic training data that boosts a robot policy from 5/30 to 24/30 task successes, yet standard open-source video models fail catastrophically on the same data augmentation task because they systematically hallucinate object interactions and violate physics.

---

1. Executive Summary

This paper introduces Cosmos-Predict2.5, a flow-based video world foundation model for Physical AI that unifies Text2World, Image2World, and Video2World generation in a single architecture, and its companion Cosmos-Transfer2.5, a control-net style framework for Sim2Real and Real2Real visual translation. Trained on 200M curated video clips and refined with a reinforcement learning post-training pipeline—incorporating domain-specific supervised fine-tuning followed by model merging and RL-based alignment—Cosmos-Predict2.5 achieves substantial gains over its predecessor across the PAI-Bench benchmark, with the 2B post-trained model reaching a 0.768 Overall Score in Text2World and 0.810 in Image2World, while the 14B model is preferred over Wan 2.1 14B in human evaluations by a 48.6% vs. 31.8% win rate. Downstream, Cosmos-Transfer2.5—despite being 3.5× smaller than Cosmos-Transfer1—delivers higher control-signal fidelity and substantially reduced error accumulation in long-horizon autoregressive generation (quantified via a proposed Relative Normalized Dover Score metric), while enabling practical applications including visual data augmentation for robot policy learning (24/30 task successes vs. 5/30 for standard augmentation baselines), multi-view driving simulation from world scenario maps, and camera-controllable multi-view robotic manipulation, establishing that high-fidelity world simulation can serve as a synthetic data engine for embodied intelligence only when the generation model is trained on domain-curated physical data with task-aligned control conditioning.

2. Context and Motivation

The Core Problem: Video Generation Models Understand Pixels, Not Physics

The fundamental issue this paper tackles sits at the intersection of two rapidly advancing fields that have largely developed in isolation: video generative modeling and embodied AI. Modern video generation models—Sora, Kling, Veo, Wan, CogVideoX—can produce visually stunning outputs conditioned on text prompts. But these models have been overwhelmingly optimized for what the paper terms "digital world" content creation: movies, artistic visualizations, and entertainment media. Their training objectives optimize for perceptual quality, aesthetic appeal, and semantic alignment with prompts, not for the physical accuracy, temporal consistency, and controllability that embodied agents require.

This gap manifests concretely in several failure modes that the paper identifies through its experiments:

• Object permanence violations: In action-conditioned video prediction, objects flicker in and out of existence across frames (visible in the comparison between Cosmos-Predict2.5 and the prior model in Figure 20, where the baseline loses track of manipulated objects).
• Hallucinated scene elements: In driving simulation, prior models insert phantom vehicles, misplace pedestrians as trucks, or render cars driving on the wrong side of the road—errors directly documented in the Cosmos-Transfer2.5 driving comparison (Figure 15).
• Error accumulation over long horizons: When generating videos autoregressively (one chunk feeding into the next), standard models exhibit compounding artifacts that make closed-loop simulation impossible—quantified by the paper's proposed RNDS metric showing the prior Cosmos-Transfer1-7B's scores degrading far faster than Cosmos-Transfer2.5-2B's (Figure 10).
• Poor adherence to control signals: When conditioned on edge maps, segmentation masks, or depth inputs, prior models generate outputs that deviate from the specified structure—Tab. 12 shows Cosmos-Transfer1-7B achieving only 0.20 Edge F1 on its edge-conditioned variant versus 0.49 for Cosmos-Transfer2.5-2B.

These are not cosmetic issues. For a robot learning a manipulation policy in simulation, an object that disappears mid-motion breaks the correspondence between actions and outcomes, making the learned policy untransferable to reality. For an autonomous vehicle validating behavior in simulated traffic, a hallucinated vehicle in the wrong lane trains the system on dangerous patterns that don't exist in deployment.

Why Physical AI Needs Purpose-Built Generative Models

The paper's motivation is anchored in a practical bottleneck that pervades embodied AI research: the cost, risk, and slowness of real-world data collection. Training Physical AI systems—robots, autonomous vehicles, smart space agents—directly in the physical world carries compounding challenges. The paper lists them concisely in Section 1:

"Training such systems directly in the real world, however, is often slow, costly, and risky. This is particularly true in the early stages, when system imperfections may lead to unsafe actions that damage either the agent, the environment, or both."

This is not hypothetical. A bimanual robot learning to grasp and place objects will inevitably drop items, collide with surfaces, or execute dangerous motions during exploration. An autonomous vehicle testing new policy behaviors on public roads faces safety constraints that make certain edge cases (near-collision recovery, extreme weather handling) prohibitively dangerous to evaluate. The paper's solution is explicit: a world simulator that generates high-quality, diverse visual environments can serve as a safe proxy for the physical world, enabling agents to acquire skills entirely in silicon before deployment.

But for such a proxy to be useful, the generated environments must satisfy several stringent requirements:

1. Physical plausibility: Objects must obey basic mechanics—they shouldn't float, teleport, or interpenetrate.
2. Action-conditioned control: The simulation must respond causally to agent actions, not just passively play out a predetermined sequence.
3. Multi-view consistency: For embodied agents with multiple cameras (vehicle surround-view systems, humanoid head-plus-wrist cameras), generated frames across viewpoints must be geometrically coherent.
4. Domain-specific fidelity: A kitchen manipulation scene requires different visual details, motion patterns, and object interactions than a highway driving scene.
5. Long-horizon stability: Closed-loop simulation—where generated frames feed back as conditioning for the next generation step—must not diverge into visual chaos.

Standard video generation models, trained on internet-scale video data dominated by entertainment content, fail these requirements because their training distribution fundamentally differs from the physical-task distribution. A movie camera pans artistically; a robot camera remains fixed but must capture precise object-gripper relationships. A nature documentary contains flowing water and swaying trees; it rarely contains the specific visual signature of a Kinova arm executing position-controlled joint trajectories.

What Prior Approaches Offer—And Where They Fall Short

The paper positions itself against three broad categories of prior work, each with well-defined limitations:

General-purpose video generation models (closed-source and open-source). Systems like Sora (OpenAI, 2024), Kling (KuaiShou, 2024), Gen-3 (Runway, 2024), MovieGen (Polyak et al., 2024), and open-source alternatives like Wan (Wan et al., 2025), CogVideoX (Yang et al., 2024), and Hunyuan (Kong et al., 2024) represent a rapidly advancing frontier in video synthesis quality. However, the paper identifies a critical tradeoff: the closed-source models achieve impressive quality but provide no access to weights or training code, making them impossible to fine-tune, extend, or adapt for specialized Physical AI domains. The open-source models enable customization but are "optimized for general-purpose content creation" and "often fall short in domains requiring precise, fine-grained control over object dynamics, interactions, and physical consistency."

This limitation is not superficial. When the paper benchmarks these models on PAI-Bench's Domain Score—which specifically measures Physical AI task performance through VQA-based evaluation across domains including robotics, physics, and driving—the gap becomes quantitative. From Tab. 10, Wan2.1-1.3B achieves a Domain Score of 0.786 and Wan2.1-14B reaches 0.794. These are reasonable numbers, but they reflect the models' reliance on prompt-text alignment without grounding in physical dynamics. The benchmark's Quality Score (0.726 and 0.727, respectively) captures perceptual quality but not the structural requirements of embodied tasks.

Latent-representation world models. The other major paradigm, exemplified by Dreamer-style architectures (Ha and Schmidhuber, 2018; Hafner et al., 2019) and more recent approaches like V-JEPA (Assran et al., 2025), compresses sensory inputs into learned latent states and performs planning and prediction in that abstract space. This approach has been highly successful for policy learning because it operates efficiently and can model long horizons. However, the paper identifies a key limitation: these models "aim to compress high-dimensional sensory inputs into compact, learned state representations," which means the generated outputs are latent codes, not pixel-space videos. For applications requiring visual inspection, human evaluation, or integration with vision-based downstream modules (object detectors, segmentation models, inverse-dynamics predictors), latent-space generation is insufficient. The paper's approach of modeling dynamics "directly in pixel space through high-fidelity video prediction" preserves all visual information, making it compatible with existing perception stacks.

The first-generation Cosmos models. The immediate predecessor, Cosmos-Predict1 (NVIDIA, 2025), was the "first open-source video generative model explicitly tailored for physical AI applications." It established the paradigm—train on domain-relevant data, support Text2World and Video2World modes, release model weights—but had several limitations that this paper systematically addresses:

• Diffusion architecture: Cosmos-Predict1 used EDM-style diffusion with preconditioning coefficients designed to normalize network inputs and outputs. While stable, this formulation proved less effective than the flow-matching velocity prediction approach that Cosmos-Predict2.5 adopts, as the paper notes in Section 3.1: flow matching "not only provides a more direct training target but also tends to yield smoother optimization and improved sample quality in practice."

• T5 text encoder: The predecessor used T5, a general-purpose encoder-only language model, for text conditioning. Cosmos-Predict2.5 replaces this with Cosmos-Reason1, a decoder-only VLM specifically trained for Physical AI tasks, providing "richer text representations and enabling finer-grained control over world generation" (Section 1). The concatenation of multi-layer activations (Sec. 3.2) rather than single-layer output gives the model access to both local and global linguistic context.

• Data pipeline quality: Cosmos-Predict1's filtering pipeline retained approximately 30% of input clips. Cosmos-Predict2.5's far stricter filtering—incorporating a VLM-based final pass for precision rejection—retains only 4%, producing 200M clips from over 6 billion candidates (Section 2.1). The paper explicitly frames this as trading "volume for quality control": processing 35 million hours of raw video (vs. 20 million) but keeping a much smaller, cleaner subset.

• Limited Physical AI specialization: While Cosmos-Predict1 incorporated some domain data, Cosmos-Predict2.5 systematically curates five distinct Physical AI domains (robotics, autonomous driving, smart spaces, human dynamics, physics) with domain-specific filtering, captioning, and metadata injection (Sections 2.2.1–2.2.5).

• No reinforcement learning post-training: The predecessor stopped at supervised fine-tuning. Cosmos-Predict2.5 introduces an RL phase using VideoAlign reward models (Section 4.2.2), which the human evaluation results in Fig. 5 confirm produces videos preferred 40.0% vs. 18.9% (pre-trained + RL vs. pre-trained alone) and 46.7% vs. 16.3% (merged + RL vs. merged alone).

The Cosmos-Transfer1 control framework. The predecessor control-net model, Cosmos-Transfer1-7B, could condition on edge, blur, depth, and segmentation inputs. But as Tab. 12 documents, its control adherence was weak—Edge F1 scores of 0.38 on edge-conditioned variants, compared to 0.49 for Cosmos-Transfer2.5-2B—and its long-horizon error accumulation (Figure 10) made closed-loop simulation unreliable. At 7B parameters, it was also computationally heavier than the 2B Cosmos-Transfer2.5, meaning it required more resources for worse results.

Reconciling the Tension Between Visual Quality and Physical Utility

The paper's central intellectual move is to argue that visual quality and physical utility are not the same optimization target. A video generation model that produces aesthetically pleasing 4K cinematic outputs might still be useless for robotics if it hallucinates object permanence or fails to follow action-conditioning signals. Conversely, a model that looks less polished on standard visual quality metrics might be far more valuable for embodied AI if it maintains geometric consistency across views, respects control inputs, and generates temporally stable rollouts.

This tension explains why the paper prioritizes Physical AI-specific benchmarks (PAI-Bench's Domain Score) over generic video quality metrics, and why it develops new evaluation protocols like the RNDS metric for long-horizon consistency and Sampson error for cross-view geometric alignment. The PAI-Bench results in Tab. 10 and Tab. 11 are structured to reflect this dual evaluation: Domain Scores and Quality Scores are reported separately and averaged, rather than collapsing everything into a single number that might mask a model that scores well on one but poorly on the other.

The DreamGen results (Tab. 18) make this tension concrete. Cosmos-Predict2.5-14B achieves 91.8% and 69.4% instruction-following scores (GPT and Qwen evaluators, respectively) on object generalization tasks—meaning the model generates videos where robots correctly manipulate unseen objects specified in the prompt. General-purpose models like WAN2.1 score 72.0% and 58.0% on the same tasks. The gap is not primarily about visual quality; it's about whether the model faithfully renders the specified interactive scenario, which is what matters for downstream VLA training.

How This Paper Positions Itself

The paper's positioning can be understood through three strategic choices, each with clear implications (Section 7):

Open-source as a deliberate research infrastructure play. Unlike Sora, Veo, or MovieGen, the Cosmos family is released under an open model license with code and checkpoints. The paper frames this not as an afterthought but as a core design goal: "by providing these open resources, we aim to lower the barrier for practitioners to adapt and specialize the pre-trained models for their own targeted Physical AI setups." Tab. 1 provides a detailed release map showing exactly which model variants, capabilities, and conditioning modes are available, from the base pre-trained checkpoints to domain-specialized versions for driving simulation, robot action-conditioning, and multi-view AgiBot data.

Domain-specific post-training as a scaling strategy. Rather than training one model to do everything, the paper adopts a train-separately-then-merge approach (Section 4.2.1). Five domains—object permanence, high motion, complex scenes, driving, and robotic manipulation—each get their own supervised fine-tuning run on curated data (Tab. 5 shows volumes: 10.4M videos for object permanence, 3.1M for driving, 730K for robotic manipulation). These specialized models are then merged into a unified checkpoint using model soup averaging (Fig. 4 demonstrates this preserves domain performance while maintaining general-domain capability, with win rates of 36.7% over the base model on the general domain). This strategy sidesteps the problematic need to balance mixture ratios across heterogeneous domains with different data volumes and learning dynamics.

Reinforcement learning as a post-training alignment mechanism, not just a quality boost. The paper's RL approach (Section 4.2.2) is noteworthy for being applied to a flow-based diffusion model—a relatively unexplored combination. Using VideoAlign's three reward dimensions (text alignment, motion quality, visual quality), GRPO-style advantage normalization within rollout groups, and a data-regularized training objective to prevent reward hacking, the RL phase produces measurable gains in both automated rewards (Tab. 6: merged model Text2World sum reward rises from 1.23 to 1.74) and human preference (Fig. 5). The paper's explicit acknowledgment of reward hacking—and its mitigation through diffusion loss regularization—is important because it anticipates a failure mode that typically emerges when optimizing learned reward models for generative outputs.

The Scaling Philosophy: Smaller, Cleaner, More Controllable

An implicit argument running through the paper is that scale alone is insufficient for Physical AI generation. Cosmos-Transfer2.5-2B outperforms Cosmos-Transfer1-7B despite being 3.5× smaller (Tab. 12). Cosmos-Predict2.5-2B achieves comparable or better PAI-Bench scores than Wan2.2-5B (0.768 vs. 0.764 in T2W Overall Score from Tab. 10) despite having 2.5× fewer parameters. Cosmos-Predict2.5-14B is preferred over Wan 2.1 14B by a 48.6% vs. 31.8% win rate in human evaluation (Fig. 7) at the same parameter count.

These results suggest that for Physical AI applications, the quality and domain relevance of training data, the specificity of control conditioning, and the alignment of the training objective with physical task requirements matter more than raw parameter count. The paper's explicit design choices—strict 4% filtering survival rate, domain-specific captioning with metadata injection, replacement of T5 with Cosmos-Reason1, RL alignment on Physical AI-aware reward dimensions—all point toward this philosophy: build for the task, not for the benchmark.

3. Technical Approach

3.1 Reader Orientation

Cosmos-Predict2.5 is a video world foundation model — a neural network that takes text, images, or video as conditioning input and generates physically plausible video sequences of future world states. It solves the problem that existing video generation models, while visually impressive, cannot reliably serve as simulators for embodied AI systems because they hallucinate object permanence failures, ignore control signals, and accumulate errors over long horizons. The "shape" of the solution is a flow-matching latent diffusion model trained on domain-curated Physical AI data, post-trained with supervised fine-tuning followed by reinforcement learning on a video quality reward model, and extended with control-net branches for conditional world translation tasks.

3.2 Big-Picture Architecture (Diagram in Words)

The Cosmos-Predict2.5 system has six major components, each with a distinct responsibility:

1. Visual Tokenizer (WAN2.1 VAE) — a causal variational autoencoder that compresses raw video frames (pixel space) into a compact latent representation (4× temporal, 8× height, 8× width compression). Its job is to reduce the computational cost of operating on high-resolution video while preserving spatiotemporal structure.

2. Text Encoder (Cosmos-Reason1) — a decoder-only vision-language model specialized for Physical AI that converts text prompts (and optionally visual inputs) into conditioning embeddings. Unlike the T5 encoder used in Cosmos-Predict1, Cosmos-Reason1 concatenates activations from multiple transformer blocks per token and projects them into a 1024-dimensional space, providing richer linguistic context.

3. Denoising Network (DiT backbone) — a transformer-based architecture operating in latent space that takes noisy latent tokens, timestep embeddings, text conditioning (via cross-attention), and optional visual conditioning frames, and predicts the velocity of the flow-matching trajectory. The architecture uses 3D Rotary Position Embeddings (RoPE) and adaptive layer normalization modulated by timestep.

4. Control-Net Branches (Cosmos-Transfer2.5 only) — four additional control blocks inserted at evenly spaced intervals (every 7 blocks) in the main DiT branch. Each block conditions on a spatial control input (edge map, depth map, segmentation mask, or blurred video), enabling the model to generate outputs that respect structural constraints from simulation engines or real sensor data.

5. Post-Training Pipeline — a three-stage process: (a) domain-specific supervised fine-tuning on five curated Physical AI domains, (b) model merging via weight averaging (model soup) to combine specialized capabilities, and (c) reinforcement learning using a VLM-based video quality reward model to align generated videos with human preferences.

6. Timestep Distillation Module — an optional post-training step that compresses the 20-step diffusion sampling process into just 4 steps using a hybrid forward-reverse joint distillation framework (rCM), enabling faster inference without substantial quality degradation.

Information flow: A prompt (text + optional image/video) enters the system → the visual tokenizer encodes any conditioning frames into latents → Cosmos-Reason1 encodes the text into multi-layer embeddings → noise is sampled and progressively denoised through the DiT, with cross-attention to the text embeddings at every block → if control branches are active (Transfer2.5), spatial control signals are processed through separate control blocks and integrated gradually → the final denoised latent is decoded by the VAE back to pixel space → the output is a 93-frame video at 16 fps (approximately 5.8 seconds).

3.3 Roadmap for the Deep Dive

• First, the flow-matching formulation (Section 3.1) — because it defines the core training objective and sampling procedure that everything else builds on. Understanding the velocity prediction target and the shifted logit-normal timestep distribution is prerequisite to understanding the network architecture's design choices.

• Second, the network architecture (Section 3.2) — the DiT backbone, the visual tokenizer, the text encoder, and the multi-mode conditioning strategy (Text2World, Image2World, Video2World), along with the frame-replacement mechanism that enables flexible conditioning. This section also covers the architecture of the Cosmos-Transfer2.5 control-net branches.

• Third, the pre-training procedure (Section 4.1) — the multi-stage progressive training strategy that escalates from Text2Image at 256p to full Text2World/Image2World/Video2World at 720p, including the critical timestep sampling modification (5% of samples from the highest 2% noise region) that fixes temporal artifacts.

• Fourth, the post-training pipeline (Section 4.2) — domain-specific SFT with model merging, reinforcement learning with VideoAlign rewards, and timestep distillation. These stages build on the pre-trained base and are what give Cosmos-Predict2.5 its Physical AI specialization and alignment quality.

• Fifth, the training infrastructure (Section 4.3) — FSDP2 with hybrid sharding, Ulysses-style context parallelism with dynamic enable/disable, selective activation checkpointing, and the elastic reward service for RL. These matter because video model training at 720p with 93 frames pushes GPU memory and communication limits.

• Sixth, the Cosmos-Transfer2.5 control-net architecture and training — because it is a structurally distinct extension (adding control blocks, training on modality-specific data) that deserves separate treatment.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and empirical scaling paper whose core idea is that high-fidelity world simulation for Physical AI requires three ingredients: (1) a flow-matching generative model trained on aggressively filtered, domain-curated video data, (2) a multi-stage post-training pipeline that specializes the model for physical task domains and aligns outputs with human preferences, and (3) a control-net extension that enables precise spatial conditioning for simulation-to-reality and reality-to-reality translation tasks.

---

Flow Matching Formulation

The paper adopts flow matching as its generative modeling framework, replacing the Elucidated Diffusion Model (EDM) formulation used in Cosmos-Predict1. The switch is motivated by two properties: (1) flow matching provides a more direct training target (velocity prediction vs. denoising), which empirically yields smoother optimization, and (2) while EDM and flow matching are mathematically equivalent in their forward and backward processes, they differ in how the denoising network is parameterized, with flow matching's velocity-based formulation tending to produce better sample quality in practice (Section 3.1, citing Gao et al., 2025).

The training objective is defined through three equations that together specify the entire forward diffusion process, the target prediction, and the loss function.

Forward interpolation: $x_t = (1 - t)x + t\epsilon$

where $x$ is a clean data sample (an image or video latent), $\epsilon \sim \mathcal{N}(0, I)$ is Gaussian noise, and $t \in [0, 1]$ is a timestep drawn from a logit-normal distribution.

What it computes: a linearly interpolated noisy latent that smoothly transitions from the clean data at $t = 0$ (where $x_t = x$) to pure noise at $t = 1$ (where $x_t = \epsilon$). At intermediate $t$ values, the latent contains a mixture of signal and noise weighted by $(1-t)$ and $t$, respectively. This interpolation defines a straight-line path in data space from the clean sample to the noise vector, which is the simplest possible continuous trajectory connecting the data distribution to the noise distribution.

Why this form: the linear interpolation is the defining characteristic of flow matching, as opposed to the stochastic differential equation (SDE) trajectories used in diffusion models. A straight path means the model only needs to learn to predict a constant velocity along that path, which is an easier regression target than the curved trajectories that arise from SDE-based diffusion. The logit-normal distribution for $t$ places higher probability mass in the middle range of $[0, 1]$ (as shown in Esser et al., 2024), meaning the model sees more intermediate-noise examples during training, which is where the most informative learning signal exists—at very low noise the task is trivial, and at very high noise there is too little signal to learn meaningful structure.

Target velocity: $v_t = \epsilon - x$

where $v_t$ is the ground-truth velocity vector that the model is trained to predict.

What it computes: the difference between the noise vector and the clean data. Geometrically, this is the constant velocity that, if applied uniformly over the unit time interval, would transport the data point $x$ to the noise point $\epsilon$ along the straight line defined by the interpolation. In other words, $x + v_t \cdot 1 = x + (\epsilon - x) = \epsilon$.

Why this form: predicting the velocity rather than the noise (as in standard diffusion) or the clean data (as in some score-based formulations) has a subtle but important advantage. The velocity $v_t = \epsilon - x$ is independent of the timestep $t$ — it is the same constant vector for all $t$ along the interpolation path. This means the model's prediction target does not change with $t$, which simplifies the learning problem: the model always predicts the same direction and magnitude, regardless of how noisy the input is. By contrast, predicting $\epsilon$ (denoising) or $x$ (clean data reconstruction) requires the model to implicitly account for the current noise level. This time-independence is what the paper means by "a more direct training target."

Flow matching loss: $\mathcal{L}(\theta) = \mathbb{E}_{x, \epsilon, c, t} \| u(x_t, t, c; \theta) - v_t \|^2$

where $u(\cdot; \theta)$ is the velocity prediction network with parameters $\theta$, $x_t$ is the noisy latent at timestep $t$, $c$ denotes conditioning information (text embeddings, reference frames, control signals), and $v_t$ is the ground-truth velocity from the previous equation.

What it computes: the mean squared error between the model's predicted velocity and the constant velocity $v_t$. The expectation is taken over the data distribution ($x$), the noise distribution ($\epsilon$), the conditioning distribution ($c$), and the timestep distribution ($t$). At each training step, a clean sample and a noise vector are drawn, a timestep is sampled, the noisy latent is constructed via the interpolation, and the loss encourages the model to output the vector that would transport the clean data to noise.

Why this form: MSE is the standard regression loss for continuous targets, and the velocity is a continuous vector in the same space as the latent data. The expectation over all sources of randomness ensures the model learns to predict velocities for any combination of data, noise, conditioning, and noise level. The conditioning variable $c$ appears as an input to the network, meaning the velocity prediction is conditional on text prompts, reference frames, etc. — the model learns to predict different velocities for different conditioning signals even when the noisy latent $x_t$ looks similar, which is how text prompts control generation.

Shifted timestep distribution. The paper identifies a specific failure mode during training: at high resolutions, nearby pixels in video frames are highly correlated. If the injected noise level is too small, the noise fails to "break apart" these correlations, meaning the model sees inputs where adjacent pixels are nearly identical and learns little meaningful structure. To address this, the paper biases training toward higher noise levels by applying a monotone transformation to the sampled timesteps:

$t_s = \frac{\beta t}{1 + (\beta - 1)t}$

where $\beta \geq 1$ is a shift hyperparameter controlling how aggressively the distribution is skewed toward larger timestep values. When $\beta = 1$, the transformation is identity ($t_s = t$). As $\beta$ increases, the mapping compresses small $t$ values toward zero and expands large $t$ values toward one, effectively shifting probability mass toward the high-noise regime.

What it computes: a remapped timestep $t_s$ that is always larger than or equal to the original $t$ (for $\beta > 1$). The denominator $1 + (\beta - 1)t$ grows with $t$, so the mapping is nonlinear—small $t$ values are pushed down proportionally more than large $t$ values are pushed up.

Why this form: the logit-normal distribution naturally concentrates probability in the middle range. Applying this shift with increasing $\beta$ progressively moves that concentration toward the high-noise end. The paper uses a progressive schedule: $\beta = 1$ at 256p resolution, gradually increasing to $\beta = 5$ at 720p resolution. Higher resolutions have more pixel redundancy (more highly correlated adjacent pixels), so they need proportionally more high-noise training examples to force the model to learn to reconstruct structure from heavily corrupted inputs. The specific functional form is a rational function that maps the $[0,1]$ interval onto itself monotonically, preserving the ordering of timesteps while redistributing their density.

Critical training fix: explicit high-noise sampling. Even with the shift, the paper observed "abrupt and unnatural transitions between frames" in generated videos. The hypothesized cause is that the model received too few training examples in the highest-noise region, leaving it underexposed to conditions where it must reconstruct coherent motion from near-pure noise. The fix is a targeted sampling strategy: 5% of training samples are drawn explicitly from the highest 2% of the noise distribution. This means in 1 out of every 20 training iterations, the timestep $t$ is forced to be in $[0.98, 1.0]$, ensuring the model practices reconstructing clean video structure from almost completely destroyed inputs. The paper reports this "significantly reduced the transition artifacts and improved temporal consistency."

---

Network Architecture

The architecture is built on a latent diffusion framework: raw video is compressed by a VAE into a latent space, the diffusion process operates in that latent space, and the final latent is decoded back to pixels. This two-stage design (encode → diffuse in latent → decode) is standard in modern video generation because operating directly on pixels at 720p resolution with 93 frames would be computationally prohibitive.

Visual tokenizer: WAN2.1 VAE. The paper adopts a causal variational autoencoder that compresses video along all three axes: 4× temporal compression, 8× height compression, and 8× width compression. The term "causal" means the encoder processes frames sequentially—a frame at time $t$ can only attend to frames at times $\leq t$, not future frames. This preserves the temporal ordering structure needed for video prediction tasks (where future frames are unknown during encoding) and enables autoregressive rollout (where generated chunks can be decoded frame-by-frame without lookahead).

The compression ratios translate to concrete numbers: a 93-frame input sequence at 720p spatial resolution (approximately 1280 × 704 pixels, per Tab. 4) would have shape $(93, 3, 704, 1280)$ in pixel space (frames × channels × height × width). After VAE encoding, the temporal dimension becomes $93 / 4 \approx 23$ (the paper specifies 24 latent frames, suggesting padding or rounding), and the spatial dimensions become $704 / 8 = 88$ and $1280 / 8 = 160$. So the latent representation has shape approximately $(24, C, 88, 160)$, where $C$ is the VAE latent channel dimension (typically 4 or 8 in Wan-style VAEs).

Patchification. On top of the VAE compression, the paper applies an additional $1 \times 2 \times 2$ patchification strategy. This means each $1 \times 2 \times 2$ block of latent features (1 in time, 2 in height, 2 in width) is flattened into a single token embedding. With 24 temporal frames, 88 height, and 160 width in latent space, the total number of tokens is $24 \times (88/2) \times (160/2) = 24 \times 44 \times 80 = 84,480$ tokens. These tokens are the sequence elements fed into the DiT transformer, with attention operating across all tokens regardless of spatial or temporal position.

DiT backbone. The denoising network is a transformer with repeated blocks, each containing three sub-layers modulated by adaptive layer normalization (AdaLN):

1. Self-attention — tokens attend to all other tokens across both space and time, using 3D Rotary Position Embeddings (RoPE) to encode relative positions. The removal of absolute positional embeddings (used in Cosmos-Predict1) is architecturally significant: absolute embeddings provide a fixed spatial or temporal reference point, but they prevent the model from generalizing to resolutions or sequence lengths not seen during training. By using only relative embeddings (RoPE), the model can handle longer videos and higher resolutions during post-training without architectural changes, because relative positions generalize naturally (the distance between token $i$ and token $i+k$ is the same regardless of the total sequence length).

2. Cross-attention — tokens attend to the text embeddings from Cosmos-Reason1. This is the mechanism by which text prompts direct video generation: at every transformer block, each spatial-temporal token queries the text embeddings and aggregates relevant linguistic information. The multi-block concatenation strategy for text embeddings (see below) means the cross-attention can access both local syntactic patterns and global semantic context.

3. Feed-forward MLP — a position-wise fully connected network with GELU activation and an intermediate hidden dimension 4× the model dimension.

Each sub-layer is gated and modulated: AdaLN produces scale, shift, and gate parameters from the timestep embedding, which control how the sub-layer's output is transformed. Scale and shift normalize and bias the input, while the gate controls how much of the sub-layer's output is added to the residual stream. This conditioning on timestep $t$ is essential because the denoising behavior must differ dramatically at different noise levels—at high noise, the model needs to make large, structural corrections; at low noise, it needs to refine fine details.

Model scaling configurations. Tab. 3 specifies two model sizes:

The head dimension of 128 is held constant across model sizes, meaning the 14B model achieves its larger model dimension by increasing the number of attention heads (from 16 to 40) rather than making each head wider. This design choice preserves the computational efficiency of attention (cost scales with head dimension) while increasing model capacity through more parallel attention patterns. The AdaLN-LoRA dimension of 256 refers to a low-rank adaptation applied to the AdaLN layers—the timestep embedding is projected through a small bottleneck to produce the modulation parameters, reducing parameter count while maintaining expressiveness.

Cosmos-Reason1 text encoder. Unlike the T5 encoder in Cosmos-Predict1 (which used the output of a single transformer layer), Cosmos-Reason1 concatenates activations across multiple transformer blocks for each token and projects them into a 1024-dimensional space. This design choice is motivated by the observation that different layers in a transformer capture different levels of linguistic abstraction: early layers encode local syntactic patterns, middle layers capture semantic roles and relations, and late layers encode global context and discourse structure. By concatenating across layers, the model provides the DiT with a richer representation that simultaneously encodes fine-grained word-level information and broad contextual meaning.

The concatenation-then-project approach (inspired by Wang et al., 2025) produces a single sequence of embedding vectors, one per token, each of dimension 1024. These embeddings are integrated into the DiT via cross-attention, as described above. The paper notes that Cosmos-Reason1's vision encoder can also support visual conditional inputs (images and videos) for style control, though this capability is "left as an exciting direction for future exploration."

Multi-mode conditioning: Text2World, Image2World, Video2World. A single Cosmos-Predict2.5 model operates in three modes, differentiated only by the number of conditioning frames provided:

• Text2World: zero conditioning frames. The model generates all 93 frames (24 latent frames) purely from text. This is the most challenging mode because the model must invent initial visual content from scratch before generating motion.

• Image2World: one conditioning frame. The model receives a single reference image, which it uses as the first frame. The remaining 92 frames are generated conditioned on both the text prompt and this image. This anchors the generated video in specific visual content, enabling controlled generation where the starting state is known (e.g., "given this photo of a kitchen, generate a video of a robot picking up the red cup").

• Video2World: up to five conditioning frames (the paper mentions randomly sampling 1 or 5 conditioning frames during training). The model receives the first few frames of a video and generates the remaining frames. This enables temporally coherent continuation of existing video clips—for example, given the first second of a driving scene, generate the next 4.8 seconds.

Frame-replacement strategy. During training for Image2World and Video2World, the initial frames of the generated sequence are consistently substituted with the conditioned frames before computing the loss. This serves two purposes: (1) it provides flexibility, since the number of conditioned frames can be adjusted depending on the task (1 for image conditioning, 5 for video conditioning), and (2) it strengthens temporal consistency by ensuring the model never learns to reconstruct the conditioning frames—it only learns to generate frames that follow naturally from them. The denoising loss is applied only to the designated generated frames, not the conditioning frames.

Masking scheme for variable conditioning. To specify which inputs are conditional, the paper applies a masking scheme: each input token is formed by concatenating the original latent token with a mask token, where the mask is a binary flag indicating whether the input is a conditioning frame or a frame to be generated. The mask tokens are learned embeddings, meaning the model learns to interpret the binary flag and adjust its behavior accordingly—attending more strongly to conditioning frames and treating generated frames as uncertain.

Training-time conditioning sampling. During the final pre-training stage (720p, with all three modes active), the paper samples 0, 1, or 2 conditioning frames with probabilities 0.5, 0.25, and 0.25, respectively. This biased sampling toward Text2World (zero conditioning) ensures the model maintains strong text-only generation capability even as it learns to leverage visual conditioning. The paper explicitly notes that at this stage, the model samples "0, 1, or 2 condition frames" (Section 4.1), which differs from the 1-or-5 scheme used earlier—this progression reflects increasing flexibility as training advances.

Cosmos-Transfer2.5 control-net architecture. The Cosmos-Transfer2.5 models extend the base architecture with control-net branches that condition generation on spatial control inputs: edge maps, depth maps, segmentation maps, and blurred video. The key architectural difference from Cosmos-Transfer1-7B is in the placement of control blocks. Cosmos-Transfer1-7B inserted all four control blocks sequentially at the start of the main branch—this concentrated all control influence early in the network, which the paper found suboptimal. Cosmos-Transfer2.5-2B distributes its four control blocks by inserting one after every 7 blocks in the main branch (since the base 2B model has 32 layers, the control blocks appear at layers 7, 14, 21, and 28). This "integrates conditioning information more gradually throughout the network" (Section 6.1), allowing control signals to influence both low-level feature extraction (early blocks) and high-level semantic generation (late blocks).

Each control branch is trained independently: a separate training run for edge conditioning, another for depth conditioning, and so on. Each run uses 100,000 iterations with an effective batch size of 64. The independent training strategy allows each branch to specialize fully in extracting useful representations from its specific modality without competition from other control signals. During inference, the full model integrates all four control modalities with spatially uniform weights (each 0.25), meaning the control signals are combined additively with equal influence.

Training data for control branches. The paper curates control-conditioned data from the pre-training video dataset, with specific volumes:

• Depth maps: generated using Video Depth Anything (Chen et al., 2025) on 10 million videos
• Segmentation maps: generated using SAMv2 (Ravi et al., 2024) on 3 million videos
• Edge maps and blurred video: curated from 14 million videos following the Cosmos-Transfer1-7B pipeline

The asymmetry in data volumes (10M depth, 3M segmentation, 14M edge/blur) reflects the difficulty of generating high-quality annotations for each modality—depth estimation is more mature and reliable than instance-level video segmentation, allowing larger-scale depth data curation.

---

Pre-Training Strategy

The pre-training follows a multi-stage progressive schedule, escalating along two axes simultaneously: spatial resolution and task diversity (Tab. 4):

Stage 1 (256p, Text2Image only): Resolution of 320×192 pixels. Only the Text2Image task with a single generated frame. This stage allows the model to learn high-quality individual frame generation—essentially, learning what objects, scenes, and textures look like—before addressing the much harder problem of temporal consistency. Starting with images rather than videos is a standard strategy in video generation because it separates the problem of "what should the world look like?" from "how should it move?"

Stage 2 (256p, Text2Image + Video2World): Same resolution, but now both the single-image task and the 93-frame video prediction task are trained jointly. The Video2World task at this stage uses 1 or 5 conditioning frames. This introduces the temporal dimension gradually while maintaining image generation capability as a regularizer.

Stage 3 (480p, Text2Image + Video2World): Resolution increased to 832×480. The model must now handle significantly more spatial detail while maintaining motion quality. The paper notes that training advances to the next stage "once the model converges and visual quality plateaus."

Stage 4 (720p, Text2Image + Video2World): Resolution increased to 1280×704. This is the target resolution for final models.

Stage 5 (720p, Text2Image + Video2World + Text2World): Same resolution, but the Text2World task (zero conditioning frames) is added. This is when the model learns to generate from text alone at full resolution. The conditioning frame sampling probabilities at this stage are 0.5 for 0 frames, 0.25 for 1 frame, and 0.25 for 2 frames—slightly different from the 5-frame maximum used in earlier stages, reflecting a focus on the most common practical use cases (pure text-to-video and single-image conditioning).

Optimization hyperparameters: The paper uses AdamW with $\beta_1 = 0.9$, $\beta_2 = 0.999$, and weight decay of 0.001. Learning rates differ by model size: $3 \times 10^{-5}$ for the 2B model, $1.3 \times 10^{-5}$ for the 14B model. The lower learning rate for the larger model is standard practice—larger models have more parameters, making them more sensitive to gradient variance and requiring more conservative updates. A linearly decaying learning rate schedule with a 2000-iteration warmup phase stabilizes early training.

Progressive timestep shift: The paper applies increasing $\beta$ values (the shift hyperparameter in the timestep transformation) as resolution increases: $\beta = 1$ at 256p, gradually increasing to $\beta = 5$ at 720p. This progressive shift accounts for the increasing pixel redundancy at higher resolutions—more correlated pixels require more aggressive noise injection to create informative training examples.

The explicit high-noise sampling fix: As noted in the flow matching section, the paper observed frame transition artifacts and hypothesized they arose from insufficient training in the highest-noise regime. The solution—5% of training samples drawn from the top 2% of the noise distribution ($t \in [0.98, 1.0]$)—is implemented as a scheduler modification rather than a change to the underlying distribution. In practice, this means that for 5% of training iterations, the normal logit-normal sampling (with or without shift) is bypassed entirely, and $t$ is sampled uniformly from $[0.98, 1.0]$. The paper reports this "significantly reduced the transition artifacts and improved temporal consistency across generated sequences."

Motivation for progressive training: The staged approach is motivated by curriculum learning principles: easier tasks (low resolution, single frames) provide better gradient signals early in training, allowing the model to learn foundational visual concepts before confronting the joint challenges of high resolution, long sequences, and open-ended text-to-video generation. The alternative—training all tasks at full resolution from the start—would waste compute on high-resolution gradients that are noisy and uninformative when the model has not yet learned basic visual structure.

---

Post-Training Pipeline

Post-training is where Cosmos-Predict2.5 acquires its Physical AI specialization. The pre-trained model is a capable general video generator, but it has not been optimized for the specific domains (robotics, driving, industrial scenes) or the specific quality criteria (physical plausibility, control adherence, temporal consistency) that Physical AI applications demand. The post-training pipeline comprises three stages, applied sequentially.

---

Stage 1: Domain-Specific Supervised Fine-Tuning (SFT)

The paper trains separate specialized models for each of five Physical AI domains, rather than a single model trained on the combined dataset. This domain-specific strategy is justified by a practical concern: "enables us to fully leverage the available data without the need to balance mixture ratios across a combined dataset." Different domains have vastly different data volumes (10.4M videos for object permanence vs. 388K for 4K enhancement, per Tab. 5), and naive mixing would either under-train on small domains or over-train on large ones.

Domain categorization: A multi-head classifier trained on InternVideo2 embeddings (Wang et al., 2024) assigns videos to five domains:

The InternVideo2 classifier provides automatic domain assignment, meaning no manual labeling is required—the classifier is trained to recognize visual patterns characteristic of each domain.

Training procedure: Each domain-specific model is fine-tuned from the pre-trained checkpoint for 30,000 iterations with a batch size of 256, using the same hyperparameters as the final pre-training stage. The paper evaluates these models through human preference studies (Fig. 3), comparing each SFT model against the pre-trained baseline on its target domain. All SFT models achieve significantly higher win rates: from 21.4% improvement (high motion, 34.7% SFT win vs. 21.3% base win) to 53.6% improvement (4K, 72.6% SFT win vs. 19.0% base win, with only 8.3% ties).

A critical observation: domain-specific SFT "substantially improves performance on specialized domains, while causing only minimal degradation on general-domain tasks." This is important because it means the specialization does not catastrophically forget general video generation capability—the model retains its ability to generate diverse content while gaining Physical AI expertise.

Cooldown stage: In addition to the five domain-specific models, the paper applies a "cooldown" stage to the pre-trained model using the curated 4K video set, where the learning rate is linearly decayed to zero. This stage "enhances fine-grained visual detail and produces smoother motion" by allowing the model to converge more precisely on a high-quality data manifold. The 388K 4K videos provide the highest spatial resolution training signal, enabling the model to refine textures, edges, and fine motion patterns that were learned only approximately at lower resolutions.

---

Stage 2: Model Merging

The six models (pre-trained + cooldown + five domain-specific SFTs) must be unified into a single model that combines their complementary capabilities. The paper experiments with four merging methods, each operating on model weights directly:

• Model Soup (Wortsman et al., 2022): simple weight averaging—for each parameter, compute the arithmetic mean across all six model checkpoints. This is the simplest approach and works surprisingly well when models share a common pre-trained initialization (as they do here, since all SFT models start from the same checkpoint and differ only in the fine-tuning trajectories).

• TIES (Yadav et al., 2023): Trim, Elect Sign, and Merge. This method first trims small-magnitude parameter changes (noise reduction), then resolves sign conflicts across models (if one model increased a parameter and another decreased it, the method determines the majority direction), and finally merges using the resolved signs and magnitudes.

• DARE-Linear (Yu et al., 2024): randomly drops (sets to zero) a fraction of the delta parameters (the difference between the fine-tuned and pre-trained weights) before merging, then rescales the remaining deltas. This sparsification reduces interference between models by zeroing out parameters where models made conflicting changes.

• DARE-TIES: combines DARE's random dropping with TIES's sign resolution, applying both sparsification and conflict resolution.

Hyperparameter search and selection: For each method, the paper runs hyperparameter sweeps generating more than 20 merged models. The best model is selected based on "quality assessments over a small, hand-picked set of challenging examples" and then validated on a larger set using human preference voting. The paper notes an interesting empirical finding: "simple grid search over hyperparameters consistently outperforms heuristic selection based on individual fine-tuned models' win rates." This means that naively weighting models by their individual-domain performance does not produce optimal merged behavior—there are complex interactions between models that require empirical optimization.

Results (Fig. 4): All methods achieve comparable performance, with the notable exception of DARE-Linear, which underperforms. Model Soup is selected as the final approach "given its effectiveness and simplicity." The merged model achieves win rates of 36.7–48.1% against the base model across domains (object permanence, high motion, complex scene, driving, robotic manipulation) while maintaining a 41.3% general-domain win rate (meaning it is better than the base model on general content 41.3% of the time, tied 17.3%, worse 41.4%—roughly equal).

---

Stage 3: Reinforcement Learning with VideoAlign

The merged model is further refined through reinforcement learning using VideoAlign (Liu et al., 2025), a VLM-based reward model that evaluates generated videos along three dimensions:

• Text Alignment: how well the generated video matches the text prompt semantics.
• Motion Quality: how natural, smooth, and physically plausible the motion is.
• Visual Quality: overall aesthetic and technical quality (absence of artifacts, sharpness, color fidelity).

RL formulation: Following the GRPO (Guo et al., 2025) paradigm, the paper treats the flow-matching denoising trajectory as a sequence of actions (each denoising step is an action). For each input condition, 8 outputs are generated using 20 diffusion steps. The reward for each output is computed by VideoAlign as a scalar sum of the three dimension scores. The advantage of each output is computed by normalizing rewards within the rollout group—this group-relative normalization (subtract the group mean, divide by the group standard deviation) is the GRPO mechanism that eliminates the need for a separately learned value function.

Memory-efficient trajectory decomposition: Computing the probability of the full 20-step denoising trajectory would require storing 20 intermediate latent states and their associated log-probabilities, which exceeds GPU memory constraints. The paper decomposes the trajectory into the sum of conditional probabilities at each step: $\log p(\text{trajectory}) = \sum_{i=1}^{20} \log p(\text{step}_i | \text{steps}_{<i})$. In practice, gradients are accumulated over every two consecutive steps (10 gradient accumulation steps total before one parameter update), keeping memory usage tractable while preserving the full trajectory gradient.

Data-regularized training: A critical technique borrowed from Ye et al. (2025) prevents reward hacking—the phenomenon where the model learns to generate videos that score highly on the reward model but are not actually better. The regularization consists of adding the standard diffusion loss (MSE velocity prediction on the fine-tuning dataset) to the RL objective. This ensures the model's outputs remain within the distribution of realistic videos, preventing it from exploiting blind spots in the reward model (e.g., generating extremely simple, high-contrast scenes that score well on motion quality metrics because they contain minimal challenging motion).

Training details: The RL phase runs for 256 steps with a batch size of 32. The model is updated using the GRPO-style advantage-weighted gradient, with the EMA (exponential moving average) of the weights released as the final checkpoint.

Quantified improvements (Tab. 6): For the pre-trained 2B model:

• Text2World total reward: 1.08 → 1.69 (56% increase)
• Image2World total reward: 0.23 → 0.42 (83% increase)

For the merged model:

• Text2World total reward: 1.23 → 1.74 (41% increase)
• Image2World total reward: 0.24 → 0.45 (88% increase)

The per-dimension breakdown reveals that the motion quality component improves most dramatically (from -0.43 to -0.19 for the pre-trained model's Text2World motion quality, a 56% reduction in negativity), suggesting the RL phase particularly improves temporal coherence. The visual quality component flips from negative to positive in all cases (e.g., -0.05 → 0.19 for pre-trained Text2World), indicating the model learns to reduce artifacts that the reward model penalizes.

Human validation (Fig. 5): Human preference studies confirm the reward improvements translate to perceived quality. Comparing videos before and after RL: the pre-trained + RL model is preferred 40.0% vs. 18.9% (RL win vs. Before-RL win, with 41.1% ties), and the merged + RL model is preferred 46.7% vs. 16.3% (with 37.0% ties). The high tie rates suggest many improvements are subtle (fine motion smoothing, slight artifact reduction) rather than dramatically transformative, which is consistent with fine-grained RL alignment rather than capability acquisition.

---

Timestep Distillation

To accelerate inference, the paper applies a hybrid forward-reverse joint distillation framework called rCM (Zheng et al., 2025), which combines continuous-time consistency distillation with distribution matching distillation. The goal is to reduce the 20-step diffusion sampling process to just 4 steps while maintaining output quality.

Framework components:

• Continuous-time consistency distillation: teaches the student model to map any point along the diffusion trajectory directly to the endpoint, rather than requiring sequential denoising steps. This is done by enforcing that the student's output from a noisy latent at timestep $t$ matches the teacher's output from the same noisy latent after full denoising.
• Distribution matching distillation: adds a loss term that matches the overall distribution of student-generated samples to teacher-generated samples, using discriminator-based or moment-matching objectives. This prevents the student from collapsing to a mean or mode of the teacher's distribution, preserving sample diversity.

Infrastructure requirements: The distillation framework requires fused flash attention with Jacobian–vector product (JVP) support (needed for the consistency distillation gradient computation, which involves differentiating through the denoising function multiple times) and adaptations for FSDP2 and context parallelism. The JVP support is specifically needed because consistency distillation backpropagates through the teacher model's denoising trajectory, which is a computation graph involving sequential function evaluations.

Results (Tab. 7, Tab. 8): The distilled 2B model achieves nearly identical scores to the teacher model:

• Text2World Overall Score: 0.768 (teacher) → 0.764 (distilled), a 0.5% decrease
• Image2World Overall Score: 0.810 (teacher) → 0.816 (distilled), a 0.7% increase (within noise)

The Image2World score actually improves slightly after distillation, which the paper does not explain but is plausibly a regularization effect—the distillation process may average out some small imperfections in the teacher's outputs. The 4-step generation represents a 5× inference speedup (20 steps → 4 steps), making the model practical for applications requiring real-time or near-real-time generation.

---

Training Infrastructure

Training video models at 720p resolution with 93 frames (producing 84,480 latent tokens per sequence) pushes the limits of current GPU hardware in memory, communication bandwidth, and computational throughput. The infrastructure section describes the distributed training system that makes this feasible.

FSDP2 with hybrid sharding. FSDP2 (Fully Sharded Data Parallelism, version 2) is the primary distributed training framework, chosen over FSDP1 for GPU-specific reasons. Unlike FSDP1's bucket-based sharding (which groups parameters into fixed-size buckets and shards at bucket granularity), FSDP2 performs per-parameter sharding. This finer granularity "enables more efficient memory management by releasing memory promptly"—when a parameter is no longer needed for the current computation, its memory can be freed immediately rather than waiting for the entire bucket to be released. This is especially critical for video models where a single sequence produces 84K+ tokens, consuming massive activation memory.

The paper incorporates optimizations from TorchTitan (Liang et al., 2025): asynchronous distributed checkpointing (saving model states without blocking training) and meta-device initialization (creating the model graph on a "meta" device first, then materializing parameters on actual GPUs only as needed, avoiding peak memory spikes during initialization).

Ulysses-style context parallelism. When training on 720p video with 93 frames, the 84K-token sequence length exceeds what can fit on a single GPU's attention computation. Context parallelism distributes the sequence across multiple GPUs, with each GPU computing attention for its local chunk and then communicating to reconstruct the full attention output. The paper adopts Ulysses-style parallelism (Rasley et al., 2020) over the ring-attention approach used in Cosmos-Predict1 for two reasons: (1) it is simpler to implement, and (2) it is more communication-efficient on NVIDIA GPUs because it leverages intra-node all-to-all collectives rather than point-to-point ring communication.

A key architectural decision: the paper dynamically enables and disables context parallelism. During image iterations (Text2Image tasks with a single frame), context parallelism is disabled because the sequence length is short enough to fit on a single GPU. During video iterations (93 frames), it is re-enabled. This dynamic switching avoids unnecessary communication overhead for image training while providing the necessary parallelism for video sequences.

The Ulysses approach also supports advanced mechanisms that would be difficult with ring attention: NATTEN sparse attention (Hassani et al., 2025), which computes attention only within spatial neighborhoods, and fused flash attention with JVP support for distillation, which requires specialized kernel implementations that assume particular parallelism patterns.

Selective activation checkpointing. To balance memory usage with computational efficiency, the paper applies selective activation checkpointing using a fine-grained policy: lightweight operators (element-wise functions, normalization layers) are prioritized for recomputation because they introduce minimal overhead (they are computationally cheap to re-execute) while yielding significant memory savings (their activations consume substantial memory). For large-scale video workloads, checkpointing is extended to portions of linear layers once all memory-intensive but computation-light operators have been covered. This staged approach (light operators first, then partial linear layers) provides a memory-computation tradeoff curve that can be tuned for specific hardware configurations.

Elastic reward service for RL post-training. The RL phase introduces a unique infrastructure challenge: the VideoAlign reward model must evaluate generated videos during training, but the reward computation (decoding VAE latents, running a VLM, aggregating scores) is expensive and cannot happen synchronously on the training GPUs without creating a bottleneck. The paper's solution is an elastic, pipelined external service:

1. Producer-consumer architecture: A decode stage converts VAE latents to pixel-space videos, while an inference stage runs multiple reward models simultaneously on different videos. The two stages operate in a pipeline: while the inference stage processes video $i$, the decode stage processes video $i+1$.

2. Separate processes: Each stage runs in its own process with its own environment and GPU allocation, enabling independent scaling and avoiding environment conflicts between the VAE decoder and the VLM reward model.

3. Zero-copy CUDA IPC: Data sharing between stages uses CUDA inter-process communication, meaning video tensors are passed by reference (shared GPU memory) rather than by copy (CPU transfer). This eliminates the dominant latency source in multi-process video pipelines.

4. Asynchronous reward computation: When training sends a video for reward evaluation, a task UUID is returned immediately. The video is enqueued, and training proceeds with other work. A Redis server stores computed rewards, which are retrieved later using the UUID. This decouples training speed from reward computation speed—training never waits for reward results.

5. Batch processing: Each task can contain multiple videos, enabling the reward service to batch evaluations for throughput efficiency.

Training efficiency (Tab. 9). With 4096 NVIDIA H100 GPUs, the 2B model achieves 36.49% Model FLOPs Utilization (MFU) with context parallelism size 2, while the 14B model achieves 33.08% MFU with context parallelism size 8. MFU measures the ratio of actual FLOPs executed to theoretical peak FLOPs of the hardware—36.49% means the system achieves roughly one-third of the H100s' maximum computational throughput, with the rest lost to communication, memory bottlenecks, and imperfect load balancing. The drop from 36.49% to 33.08% is attributed to "large context parallelism, which introduces more communication cost"—with 8-way parallelism, more GPU time is spent on all-to-all collectives for attention computation rather than on actual matrix multiplications.

---

Putting It All Together: The Training Trajectory

The complete training pipeline for Cosmos-Predict2.5-2B proceeds through the following sequence:

1. Compute video latents: The WAN2.1 VAE encodes all pre-training videos into latent space. This is a one-time preprocessing step; the VAE weights are frozen throughout training.

2. Multi-stage pre-training: The DiT backbone trains progressively from 256p Text2Image to 720p full multi-mode, consuming the 200M curated clips. The model learns to predict flow-matching velocities conditioned on text (via Cosmos-Reason1 cross-attention) and optional visual frames (via frame replacement and masking).

3. Domain-specific SFT: Five separate fine-tuning runs specialize the model for object permanence, high motion, complex scenes, driving, and robotic manipulation, using InternVideo2-classified domain data. A cooldown stage on 4K videos refines fine details.

4. Model merging: The six checkpoints are averaged using model soup, producing a single model that combines all specialized capabilities without catastrophic forgetting of general video generation.

5. Reinforcement learning: The merged model is further trained using GRPO-style policy optimization with VideoAlign rewards, regularized by diffusion loss to prevent reward hacking. The EMA checkpoint is released as the final model.

6. Timestep distillation (optional): The final model can be distilled to 4-step sampling using rCM, trading a small quality decrease for a 5× speedup.

The resulting model generates 93 frames (approximately 5.8 seconds at 16 fps) in either Text2World, Image2World, or Video2World mode, with the Cosmos-Transfer2.5 variant additionally supporting spatial control conditioning for Sim2Real and Real2Real translation.

---

Key Design Decisions and Their Justifications

Flow matching over EDM diffusion: The paper explicitly chooses flow matching for its "more direct training target" (time-independent velocity) and empirical sample quality improvements. The shift from denoising to velocity prediction changes the network's output semantics—rather than predicting what noise to remove, it predicts the constant direction from data to noise—which the paper argues leads to smoother optimization.

Cosmos-Reason1 over T5: The multi-block activation concatenation provides richer text representations than single-layer T5 outputs, enabling finer-grained control. This matters for Physical AI prompts that specify precise object interactions, spatial relationships, and action sequences—a generic text encoder might lose the specificity that distinguishes "the robot picks up the red cup" from "the robot picks up the blue cup."

Removal of absolute positional embeddings: This is forward-looking: absolute embeddings fix the model to a specific resolution and sequence length, while relative embeddings (3D RoPE) allow generalization to longer videos and higher resolutions during post-training without architectural changes. The paper explicitly notes this enables the post-training phase to operate at extended contexts.

Domain-specific SFT rather than joint SFT: The paper avoids the mixture ratio problem—different domains have vastly different data volumes and learning dynamics. Training separately and merging decouples these concerns: each domain can be trained to convergence on its own data schedule, without one domain dominating gradients or another being undertrained.

RL on top of SFT: The SFT stage adds capability (knowledge of Physical AI domains). The RL stage adds alignment (outputs that match human preferences for quality, motion, and text adherence). The paper's data-regularization technique is essential because without it, the model would optimize for reward model scores rather than actual video quality—a classic reward hacking failure mode.

WAN2.1 VAE over alternative tokenizers: The causal VAE design (temporal causality in encoding) is necessary for autoregressive video prediction, where future frames should not influence the encoding of current frames. This preserves the temporal information flow needed for closed-loop simulation.

4. Key Insights and Innovations

Innovation 1: Physical AI Generation Is a Data-Curation Problem, Not a Scale Problem

The dominant narrative in generative AI over the past several years has been that scaling models and datasets produces better outputs—more parameters plus more data equals higher quality. Cosmos-Predict2.5 challenges this assumption head-on by demonstrating that, for the specific requirements of Physical AI simulation, curation quality dominates curation volume. The paper does not merely claim this; it encodes the claim in its most extreme design choice: a filtering pipeline that rejects 96% of input clips, keeping only 200M from over 6 billion candidates, compared to Cosmos-Predict1's 30% survival rate.

This is not a marginal tightening of quality standards. It is a philosophical stance: for world simulation that must support robot policy learning and autonomous driving validation, the presence of one semantically incorrect video (a physics-defying animation, a video game cutscene, a clip with text overlay that gets mis-captioned) can be more damaging than the absence of a hundred clean but redundant clips. The paper explicitly identifies the filtering targets—motion artifacts, distortion, overlay text, semantic artifacts, video-in-video, and content depicting "physically unrealistic phenomena such as video games, synthetic visual patterns, animations, or cartoons"—and each of these targets corresponds to a specific failure mode that would propagate into downstream Physical AI applications. A policy trained on synthetic data where objects flicker in and out of existence learns to ignore object permanence violations rather than react to them.

The evidence for this claim is distributed across the paper rather than concentrated in a single ablation. The Cosmos-Transfer2.5-2B model, at 2B parameters, outperforms Cosmos-Transfer1-7B across every control modality (Tab. 12), despite being 3.5× smaller. The driving simulation comparison (Fig. 15) shows the predecessor hallucinating phantom vehicles and misrendering pedestrian locations—failures that are not parameter-count problems but training-data problems. The robotic policy learning results (Tab. 13) demonstrate that augmentation with Cosmos-Transfer2.5-generated data enables 24/30 task successes on out-of-distribution scenarios versus 5/30 for standard augmentation—a gain that comes from the augmented data being physically coherent, not from it being generated by a larger model. This insight reframes the problem from "how do we scale video generation?" to "how do we identify and retain only the data that teaches physical fidelity?"

---

Innovation 2: Post-Training as a Multi-Stage Capability-Acquisition Pipeline

Prior video generation models typically follow a linear training trajectory: pre-train on large-scale data, then optionally fine-tune on a target domain. The paper introduces a fundamentally different post-training philosophy: capability acquisition is decomposed into distinct, sequential stages, each with its own objective, data regime, and optimization procedure, and these stages are composable through weight-space merging rather than joint optimization.

The three post-training stages each serve a different purpose that would conflict if trained jointly. Domain-specific SFT (Section 4.2.1) acquires specialized knowledge—the model learns what robotic manipulation looks like, how driving scenes evolve, what object permanence means in video. But the paper explicitly acknowledges that domain-specific SFT causes "minimal degradation on general-domain tasks," which would not be guaranteed under joint training where a dominant domain could overwrite useful representations. Model merging (Section 4.2.2) then combines these independently acquired capabilities into a single model using weight averaging, with the unexpected finding that "simple grid search over hyperparameters consistently outperforms heuristic selection based on individual fine-tuned models' win rates"—the interactions between fine-tuned models are complex enough to require empirical optimization rather than theoretical weighting. Finally, RL (Section 4.2.3) aligns the merged model's outputs with human preferences, using a VL-based reward model and GRPO-style advantage normalization to improve motion quality, text alignment, and visual quality without changing what the model knows, only how it expresses that knowledge.

This staged approach is conceptually related to the pre-training → SFT → RLHF pipeline that has become standard in large language models, but the paper adapts it to video generation with domain-specific innovations: the model merging step has no direct LLM analog (LLMs typically do not merge domain-specific checkpoints via weight averaging), and the RL phase is one of the first applications of GRPO to flow-matching video diffusion, requiring the trajectory decomposition technique and diffusion-loss regularization described in Section 4.2.3. The result is a model that is simultaneously specialized across five Physical AI domains while maintaining general video generation capability—a property that would be difficult to achieve through any single-stage training procedure.

---

Innovation 3: Error Accumulation as a Quantifiable, Architecturally Addressable Property

Long-horizon video generation has largely been evaluated through qualitative inspection: does the video look plausible after 30 seconds, 60 seconds, 120 seconds? The paper introduces error accumulation as a quantitative, measurable property of autoregressive video generation through the proposed Relative Normalized Dover Score, and demonstrates that it is not merely a function of model size but is addressable through data curation and control architecture.

The RNDS metric (Section 6.1.2) is elegant in its design. By normalizing each chunk's perceptual quality score against both the first chunk (to measure relative degradation) and the ground-truth video (to account for content-dependent quality variations), it isolates the error accumulation signal from content difficulty confounds. A video of a static scene will naturally have a flatter RNDS curve than a video of complex motion, but this is not informative about model quality. By normalizing against ground truth, the metric says: given how much quality should degrade across this specific video, how much more does the model degrade it?

The results (Fig. 10) are striking. Cosmos-Transfer2.5-2B maintains RNDS values close to 1.0 across 18 chunks (approximately 100 seconds of video) for all four control modalities, while Cosmos-Transfer1-7B's RNDS drops to 0.75–0.85 for blur control, 0.8–0.9 for depth control, and shows high variance for segmentation control (dropping as low as 0.4–0.6 by chunk 18). This means the 7B model accumulates errors 3–4× faster than the 2B model on some modalities—a result that directly contradicts the "bigger is better" scaling assumption and points to data quality and architecture as the controlling factors.

The paper attributes this improvement to two design choices that are not obviously connected to long-horizon performance: the stricter data filtering (which removes clips with artifacts that would compound autoregressively) and the distributed control block placement in Cosmos-Transfer2.5 (which integrates conditioning information gradually rather than all at the start). This insight—that error accumulation is a trainable property, not an inevitable consequence of autoregressive generation—opens a new axis for video model evaluation and development that did not previously exist as a formalized metric.

---

Innovation 4: Reinforcement Learning for Diffusion Models with Data Regularization as a Reward-Hacking Mitigation

Applying RL to generative models—particularly diffusion-based video models—is relatively unexplored territory. The paper's contribution is not the application itself but the identification and mitigation of a specific failure mode: when a flow-matching video model is optimized against a learned reward model without regularization, it learns to produce outputs that maximize the reward signal rather than outputs that are genuinely better. This is the reward hacking problem familiar from RLHF, but it manifests differently in continuous video generation than in discrete token generation, and the paper's solution—adding the standard diffusion loss as a regularization term—is a conceptually clean fix that keeps the model's outputs within the natural video distribution.

The evidence that this regularization matters is implicit in the training design rather than ablated in a controlled experiment, which is a limitation. The paper notes that the technique is taken from Ye et al. (2025) and that it "effectively alleviates the reward hacking phenomenon," but does not show what happens without it. However, the human evaluation results (Fig. 5) provide indirect evidence that the RL phase produces genuine quality improvements rather than reward-score gaming: human raters prefer RL-trained outputs at rates of 40.0% vs. 18.9% (pre-trained + RL) and 46.7% vs. 16.3% (merged + RL), with substantial tie rates (41.1% and 37.0%) suggesting the improvements are fine-grained and not obviously gaming a particular metric. If the model were simply exploiting VideoAlign's reward function, the human preference signal would likely be weaker or negative, since human raters would detect unnatural artifacts that a VLM-based reward model misses.

This innovation is more methodological than conceptual—it provides a practical recipe for applying RL to video diffusion models—but it matters because it establishes that the RL → SFT → RL pipeline, which has been transformative for language models, can be productively adapted for video generation when the regularization challenge is addressed. The trajectory decomposition technique (computing probabilities over two-step intervals rather than full trajectories) is also a practical contribution that makes RL training feasible under GPU memory constraints that would otherwise be prohibitive.

5. Experimental Analysis

Evaluation Methodology

Dataset. All main generation quality benchmarks use PAI-Bench (Zhou et al., 2025), a recently proposed benchmark specifically designed to assess physical AI generation and understanding capabilities. The benchmark evaluates on two dimensions: the Domain Score (obtained through VQA-based evaluation across seven domains: autonomous vehicles, common scenes, human activities, industrial settings, miscellaneous, physics, and robotics) and the Quality Score (derived from eight text-to-video and image-to-video metrics adapted from VBench). The Overall Score is the average of the Domain and Quality scores. The paper evaluates on PAI-Bench's predict task in both Text2World (T2W) and Image2World (I2W) settings. Transfer models are additionally evaluated on PAIBench-Transfer, which contains 600 videos spanning driving and robotics domains, measuring both adherence to control inputs and overall video quality. The DreamGen benchmark (Jang et al., 2025) is used for VLA training evaluation, measuring instruction-following across object, behavior, and environment generalization axes using GPT-4o and Qwen-VL-2.5 as evaluators. For action-conditioned video prediction, 100 episodes from the official Bridge dataset test set are used. The driving multi-view evaluation uses a 1000 multi-view clip dataset in RDS-HQ with human-labeled lanes and cuboids, disjoint from training data.

Base model(s). The Cosmos-Predict2.5 family includes models at two scales: Cosmos-Predict2.5-2B (32 DiT layers, model dimension 2048, 16 attention heads) and Cosmos-Predict2.5-14B (36 layers, model dimension 5120, 40 attention heads). Both use the WAN2.1 VAE for latent encoding, Cosmos-Reason1 for text encoding, and flow-matching training with 3D RoPE positional embeddings. The paper releases pre-trained and post-trained variants of both scales, plus domain-specialized versions for autonomous driving, robotic action-conditioning, and multi-view AgiBot manipulation. The predecessor Cosmos-Predict1-7B and Cosmos-Transfer1-7B serve as internal baselines. The choice of two model scales is deliberate: the 2B model demonstrates efficiency (competitive with much larger models), while the 14B model pushes the quality frontier.

Metrics. The paper uses a heterogeneous set of metrics organized by evaluation type:

• PAI-Bench Predict Metrics: Domain Score (VQA-based, seven-domain coverage of Physical AI tasks), Quality Score (eight VBench-adapted T2V and I2V metrics), Overall Score (average of Domain and Quality).
• PAI-Bench Transfer Metrics: Blur Alignment (SSIM ↑ between generated and blurred-conditioned video), Edge Alignment (F1 score ↑ for edge map consistency), Depth Alignment (si-RMSE ↓ between generated and depth-conditioned depth maps), Segmentation Alignment (mIoU ↑ for mask consistency), Overall Quality Score ↑.
• Human Evaluation: Win ratios from pairwise comparisons where annotators assess realism, visual quality, temporal consistency, and alignment with conditioning inputs. Ties are reported explicitly.
• Long-Horizon Error Accumulation: Averaged Relative Normalized Dover Score (RNDS), a curve over chunk indices defined as RNDS[𝑖] = (DOVER[𝑖] / DOVER_GT[𝑖]) / (DOVER[1] / DOVER_GT[1]), where DOVER[𝑖] is the perceptual quality score of chunk 𝑖 and DOVER_GT[𝑖] is the corresponding ground-truth score. Values close to 1.0 indicate minimal degradation.
• Multi-View Driving Metrics: FVD StyleGAN ↓ and FVD I3D ↓ (video quality), FID ↓ (frame quality), TSE ↓ (temporal Sampson error for temporal consistency), CSE ↓ (cross-camera Sampson error for multi-view geometric consistency). Cuboid detection: LET-AP ↑, LET-APL ↑, LET-APH ↑. Lane detection: F1 ↑, x-error far ↓, Category Accuracy ↑.
• Multi-View Robotic Metrics: Translation Error ↓ and Rotation Error ↓ (camera trajectory accuracy from ViPE), Sampson Error ↓ (cross-view geometric consistency).
• Action-Conditioned Video Prediction: PSNR ↑, SSIM ↑, Latent L2 ↓, FVD ↓ comparing generated rollouts against ground-truth Bridge dataset videos.
• Reinforcement Learning Rewards: Text Alignment, Motion Quality, Visual Quality (three VideoAlign dimensions), reported individually and as a sum.

Baselines. The paper compares against multiple model families, both open-source and proprietary, spanning different scales:

• Cosmos-Predict1-7B (NVIDIA, 2025): the direct predecessor using EDM diffusion with T5 text encoder, serving as the primary internal baseline for quality improvements.
• Cosmos-Transfer1-7B (NVIDIA, 2025): predecessor control-net model with 7B parameters, with both single-control variants (Blur, Edge, Depth, Seg) and a uniform-weights multi-modal variant.
• Wan2.1-1.3B, Wan2.1-14B (Wan et al., 2025): open-source video generation models at two scales.
• Wan2.2-5B, Wan2.2-27B-A14B: larger Wan family variants, with the 27B-A14B being a Mixture-of-Experts architecture.
• Hunyuan (Kong et al., 2024) and CogVideoX (Yang et al., 2024): evaluated on the DreamGen GR1 Instruction Following benchmark.
• Cosmos-Predict2-14B/robot/gr00tdream-gr1: an earlier Cosmos version without Cosmos-Reason1 text embeddings, included in the DreamGen comparison.
• For robot policy learning: a base policy trained solely on 100 teleoperation demonstrations, and a baseline policy trained with standard image-based data augmentation (random adjustments of brightness, contrast, saturation, hue; gamma correction; salt-and-pepper noise; histogram equalization; random blurring or sharpening).
• For driving simulation: real videos serve as reference for detection metrics and multi-view consistency.
• For action-conditioned generation: Cosmos-Predict1-7B-Video2World-Sample-ActionCond as baseline.

Generation budget / compute accounting. For the main generation quality benchmarks, the paper does not report generation budgets in terms of inference FLOPs or wall-clock time; instead, models are compared at their default inference configurations. The RL post-training uses 8 outputs per condition with 20 diffusion steps (Section 4.2.2), while timestep distillation compresses this to 4 steps. For Cosmos-Transfer2.5 training control experiments, each control branch is trained for 100,000 iterations with an effective batch size of 64. For long-horizon video evaluation, chunks of 93 frames are generated autoregressively with RNDS measured across chunk indices. For action-conditioned video prediction, videos are generated autoregressively in chunks conditioned on the last generated frame. The MFU analysis (Tab. 9) reports achieved utilization on 4096 H100 GPUs: 36.49% for the 2B model (context parallelism 2) and 33.08% for the 14B model (context parallelism 8), with the drop attributed to increased communication costs. For robot policy experiments, each of the 100 original demonstration videos is augmented with five synthetic variants, and a single real-robot evaluation consists of three trials per condition.

Cross-validation / statistical protocol. The paper does not report cross-validation for the main benchmark evaluations, which are standard test-set evaluations on fixed benchmarks. For human evaluation, the paper reports win ratios with tie rates, indicating pairwise comparisons with a third "tie" option, but does not report confidence intervals or significance tests on these ratios. For model merging, more than 20 merged models are generated through hyperparameter sweeps, with the best selected based on "quality assessments over a small, hand-picked set of challenging examples" and then validated on a larger evaluation set using human preference voting. The RL training uses group-relative advantage normalization (GRPO), where advantages are computed within each rollout group of 8 outputs, providing a form of variance reduction. The driving detection metrics use RDS-HQ evaluation clips that are "disjoint from the prior two datasets used in training." For DreamGen evaluation, the paper uses two different VLM evaluators (GPT-4o and Qwen2.5-VL) as a form of evaluator triangulation, though agreement between them varies substantially. For the robot policy learning evaluation, each of the 10 test scenarios is evaluated with exactly 3 trials with fixed object poses and environment configurations, for a total of 30 trials per policy.

---

Main Quantitative Results

PAI-Bench Text2World and Image2World Generation Quality

The headline results establish that Cosmos-Predict2.5 models achieve leading performance on Physical AI-specific benchmarks, with post-trained models significantly outperforming their pre-trained baselines and matching or exceeding larger competing models.

Text2World results (Tab. 10): The post-trained Cosmos-Predict2.5-2B achieves an Overall Score of 0.768 (Domain 0.804, Quality 0.732), improving over its pre-trained counterpart's 0.751 (Domain 0.782, Quality 0.720). The post-trained Cosmos-Predict2.5-14B achieves an identical Overall Score of 0.768 (Domain 0.803, Quality 0.732), with a slightly higher pre-trained baseline of 0.757 (Domain 0.791, Quality 0.722). These scores place both Cosmos models at parity with the much larger Wan2.2-27B-A14B (MoE) which achieves 0.769 Overall (Domain 0.810, Quality 0.728), despite the Cosmos models having 13.5× fewer parameters than the 27B-A14B variant. The Cosmos models outperform Wan2.1-14B (0.761 Overall), Wan2.2-5B (0.764 Overall), and Wan2.1-1.3B (0.756 Overall). A notable pattern: Cosmos models achieve higher Quality Scores than all Wan variants (0.732 vs. 0.726–0.730), while the Wan2.2-27B-A14B edges ahead on Domain Score (0.810 vs. 0.803–0.804). Post-training provides consistent gains: the 2B model improves from 0.751 to 0.768 (+0.017), primarily through Domain Score improvement (0.782 → 0.804, +0.022) with a smaller Quality gain (0.720 → 0.732, +0.012). The 14B model shows a similar pattern (0.757 → 0.768, +0.011 Overall), with Domain Score improving from 0.791 to 0.803 (+0.012) and Quality from 0.722 to 0.732 (+0.010).

Image2World results (Tab. 11): The post-trained Cosmos-Predict2.5-2B achieves an Overall Score of 0.810 (Domain 0.840, Quality 0.779), which is the best-performing model on this benchmark, tied with the post-trained 14B at 0.810 Overall (Domain 0.838, Quality 0.781). Pre-trained baselines are 0.799 for the 2B model and 0.806 for the 14B model. The competing models score lower: Wan2.2-27B-A14B at 0.806 (Domain 0.841, Quality 0.772), Wan2.2-5B at 0.804 (Domain 0.834, Quality 0.774), Wan2.1-14B at 0.797 (Domain 0.827, Quality 0.768). Here, the pattern flips somewhat: Cosmos models are behind on Domain Score (0.838–0.840 vs. Wan2.2-27B-A14B's 0.841) but substantially ahead on Quality Score (0.779–0.781 vs. 0.772). This suggests the Cosmos models generate perceptually higher-quality videos from image conditioning, even if the largest Wan model has a marginal advantage in domain-specific physical understanding. Post-training gains are larger in I2W than T2W: the 2B model improves 0.011 Overall (0.799 → 0.810) and the 14B improves 0.004 Overall (0.806 → 0.810), with both showing the largest absolute improvement in Domain Score.

Distillation results (Tab. 7, Tab. 8): The 4-step distilled Cosmos-Predict2.5-2B model achieves a T2W Overall Score of 0.764 (Domain 0.797, Quality 0.731), compared to the teacher model's 0.768, representing a 0.5% degradation. In I2W, the distilled model surprisingly achieves 0.816 (Domain 0.842, Quality 0.790), marginally exceeding the teacher's 0.810—a 0.7% improvement that the paper does not explain but may reflect regularization effects during distillation. This is significant because it demonstrates that the 5× inference speedup (20 steps → 4 steps) comes with essentially no quality penalty, and possibly a small quality gain in the image-conditioned setting.

---

Human Evaluation Results

Human evaluations provide a complementary assessment that captures aspects of video quality difficult to quantify automatically, and reveal quality differences that automated metrics partially obscure.

2B model comparisons (Fig. 6): Despite being 60.0% smaller than Wan2.2-5B and 85.7% smaller than Wan2.1-14B, the post-trained Cosmos-Predict2.5-2B achieves more wins than Wan2.2-5B (30.0% vs. 26.2%, with 43.8% ties) and is comparable to Wan2.1-14B (33.0% vs. 34.8%, with 32.2% ties). The high tie rates (43.8% and 32.2%) suggest the models produce outputs of broadly similar quality on many prompts, with Cosmos winning slightly more often against the smaller Wan model and losing slightly more often to the larger Wan model. The comparison is across "a diverse set of prompts" but the paper does not specify the exact number of prompts or annotators.

14B model comparisons (Fig. 7): The post-trained Cosmos-Predict2.5-14B demonstrates clearer superiority at the same parameter scale. Against Wan2.1-14B, it achieves a win rate of 48.6% vs. 31.8% (19.6% ties)—a substantial 16.8 percentage point advantage. Against the much larger Wan2.2-27B-A14B (approximately 2× the parameter count), it achieves a competitive 38.1% vs. 35.9% (26.0% ties), effectively at parity despite having half the parameters. This represents a meaningful improvement over the 2B model's performance: comparing against Wan2.1-14B, the win rate increases from 33.0% (2B) to 48.6% (14B), a 15.6 percentage point jump. The paper notes this explicitly: "the benefit of our 14B model over our 2B model is more evident in the human evaluation results." This suggests that while automated PAI-Bench scores show near-identical performance between the 2B and 14B post-trained models (0.768 Overall for both in T2W, 0.810 for both in I2W), human raters perceive qualitative differences that the automated metrics fail to capture.

RL pre-training vs. post-training human evaluation (Fig. 5): Comparing videos generated by models before and after reinforcement learning: the pre-trained model with RL is preferred 40.0% vs. 18.9% (41.1% ties), and the merged model with RL is preferred 46.7% vs. 16.3% (37.0% ties). The preference for RL-trained models is clear (roughly 2:1 win ratios), but the high tie rates (37–41%) indicate the improvements are often subtle rather than transformative, consistent with RL alignment being a fine-tuning of output characteristics rather than a capability leap. The merged model benefits more from RL than the pre-trained model (46.7% win vs. 40.0% win), suggesting RL is particularly effective at polishing the outputs of the already-domain-specialized merged model.

---

Domain-Specific SFT and Model Merging

SFT effectiveness (Fig. 3): Every domain-specific SFT model achieves a significantly higher win rate than the pre-trained baseline on its target domain:

• Object Permanence: 50.9% SFT vs. 21.4% base (21.3% ties)
• High Motion: 34.7% SFT vs. 21.3% base (44.0% ties)
• Complex Scenes: 35.4% SFT vs. 22.0% base (42.6% ties)
• Driving: 28.8% SFT vs. 23.3% base (47.9% ties)
• Robotic Manipulation: 8.3% SFT vs. 19.0% base (72.6% ties)

The robotic manipulation domain shows the most interesting (and initially counterintuitive) result: the SFT model wins only 8.3% of comparisons vs. 19.0% for the base model, with 72.6% ties. This does not mean the SFT model is worse—it means the specific comparisons in robotic manipulation are overwhelmingly judged as ties, suggesting that in this domain the base model already performs reasonably well and the SFT improvements are subtle. The driving domain similarly shows a high tie rate (47.9%), though with a clearer SFT advantage (28.8% vs. 23.3%). The 4K cooldown model (not shown in Fig. 3 but described in Section 4.2.1) achieves a 72.6% win rate with only 8.3% base wins and 19.0% ties, indicating the most dramatic improvement among all domains—high-resolution fine-tuning produces visually obvious quality enhancements.

Merged model performance (Fig. 4): The merged model (model soup) achieves balanced performance across domains, with win rates ranging from 36.7% to 66.8% against the base model across domains. On the general domain, the merged model achieves 41.3% wins vs. 41.4% base wins with 17.3% ties—essentially equal, confirming the paper's claim that merging domain-specific SFT models does not catastrophically degrade general video generation capability. The comparison between TIES and DARE-TIES shows both perform comparably to model soup, while DARE-Linear underperforms (the paper does not provide exact win rates for DARE-Linear but explicitly states it is the exception, with Fig. 4 showing all methods "comparable" except it). The key insight: the merged model "gets the best of all the worlds" without requiring domain identification at inference time.

---

RL Post-Training Reward Improvements

Quantitative reward scores (Tab. 6): For the pre-trained 2B model, RL improves the T2W total reward from 1.08 to 1.69 (+0.61, 56% increase), driven by improvements across all three dimensions: Text Alignment (1.55 → 1.69, +0.14), Motion Quality (-0.43 → -0.19, +0.24), and Visual Quality (-0.05 → 0.19, +0.24). The I2W total reward improves from 0.23 to 0.42 (+0.19, 83% increase), with Text Alignment (1.48 → 1.57, +0.09), Motion Quality (-0.76 → -0.70, +0.06), and Visual Quality (-0.49 → -0.45, +0.04).

For the merged model, the improvements are larger: T2W total reward from 1.23 to 1.74 (+0.51, 41% increase), with Text Alignment (1.69 → 1.75, +0.06), Motion Quality (-0.46 → -0.18, +0.28), and Visual Quality (-0.01 → 0.18, +0.19). I2W total reward from 0.24 to 0.45 (+0.21, 88% increase), with Text Alignment (1.57 → 1.57, +0.00), Motion Quality (-0.82 → -0.68, +0.14), and Visual Quality (-0.52 → -0.44, +0.08).

Several patterns emerge: (1) Motion Quality shows the most dramatic improvement across all settings, flipping from clearly negative to near-zero or positive, indicating the RL phase particularly addresses temporal coherence issues; (2) Visual Quality flips from negative to positive in T2W settings for both models but remains negative in I2W settings (though substantially less negative), suggesting image-conditioned generation remains more challenging for visual quality even after RL; (3) Text Alignment improves modestly but consistently, confirming the RL phase enhances prompt adherence; (4) the merged model starts with higher pre-RL rewards than the pre-trained model across most dimensions (e.g., T2W total: 1.23 vs. 1.08), confirming that the SFT + merging stage already improves output quality beyond the pre-trained baseline.

---

Cosmos-Transfer2.5 Control Adherence and Quality

Quantitative control evaluation (Tab. 12): Cosmos-Transfer2.5-2B substantially outperforms Cosmos-Transfer1-7B across all control modalities and metrics, despite being 3.5× smaller. The uniform-weights multi-modal variant (integrating all four control modalities) illustrates the aggregate advantage: Overall Quality Score of 9.31 for Transfer2.5-2B vs. 9.24 for Transfer1-7B (a small but consistent edge). The single-modality comparisons are more dramatic:

• Blur conditioning: Cosmos-Transfer2.5-2B achieves Blur Alignment (SSIM) of 0.90 vs. 0.89 for Transfer1-7B, with a much larger Overall Quality advantage (9.75 vs. 6.56, a 49% improvement).
• Edge conditioning: Edge Alignment (F1) improves from 0.38 to 0.49 (29% relative improvement), with Overall Quality from 6.76 to 8.73 (29% improvement).
• Depth conditioning: Depth Alignment (si-RMSE ↓) improves from 0.85 to 0.70 (18% reduction in error), Overall Quality from 6.89 to 8.85 (28% improvement).
• Segmentation conditioning: Segmentation Alignment (mIoU) slightly degrades from 0.73 to 0.71, but Overall Quality jumps from 6.02 to 8.81 (46% improvement).

The segmentation mIoU anomaly—where the 2B model is slightly worse at mask alignment but produces vastly higher quality videos—is interesting and suggests a tradeoff: the smaller model may prioritize visual quality over strict mask adherence, or the semantic segmentation evaluation protocol may be sensitive to factors other than pure alignment fidelity. The uniform-weights model improves Edge F1 from 0.26 (Transfer1) to 0.41 (Transfer2.5, 58% improvement) and Blur SSIM from 0.82 to 0.87, while maintaining comparable Depth si-RMSE (0.70 vs. 0.70) and Segmentation mIoU (0.74 vs. 0.76).

Long-horizon error accumulation (Fig. 10): The RNDS curves across all four control modalities demonstrate that Cosmos-Transfer2.5-2B maintains quality far better than Transfer1-7B over extended autoregressive generation. For edge control, Transfer2.5-2B's RNDS stays close to 1.0 across all 18 chunk indices (approximately 100 seconds), while Transfer1-7B's RNDS drops below 0.90 by chunk 12 and below 0.85 by chunk 18. For blur control, the degradation is even more pronounced: Transfer1-7B drops to approximately 0.75 by chunk 18, while Transfer2.5-2B stays near 1.0. For depth control, Transfer2.5-2B maintains RNDS around 0.95–1.05 with low variance, while Transfer1-7B shows high variance and a downward trend (0.8–1.0 range). For segmentation control, Transfer1-7B's RNDS is highly unstable, dropping as low as approximately 0.4–0.6 by chunk 18, while Transfer2.5-2B maintains 1.0 ± 0.1 throughout. The paper attributes this improvement to the stronger base model and more carefully curated Physics-AI-focused training data, but the architectural change—distributed control block placement—likely also contributes by preventing control signal degradation over many denoising steps.

---

Cosmos-Transfer2.5 for Robot Policy Learning

Quantitative policy evaluation (Tab. 13): The Cosmos-Transfer2.5-augmented policy achieves 24/30 successes across 10 test scenarios (3 trials each), dramatically outperforming both the base policy (1/30 successes) and the standard-augmentation baseline policy (5/30 successes). The test scenarios are explicitly designed to test visual generalization under adversarial perturbations:

• Base setting (matching training): Cosmos-augmented policy succeeds on all 3 trials; base policy succeeds on 1/3; baseline policy succeeds on 3/3. The base policy's poor performance on what should be the easiest setting (only 1/3 success) is attributed to "subtle, human-imperceptible scene variations" that differ from the training distribution—an important finding suggesting brittleness even in-domain.
• Object appearance changes (mangosteen replacing apple, orange bowl replacing gray bowl): Cosmos-augmented achieves perfect 3/3 and 3/3; base policy fails completely (0/3, 0/3); baseline policy achieves 0/3 and 2/3. The baseline policy's partial success on the orange bowl suggests hue-based augmentation can help with color generalization to some degree, but it fails completely on the more radical mangosteen substitution.
• Scene background changes (beige tablecloth, black tablecloth, spotlight, distractors, black cabinet, open drawers): Cosmos-augmented achieves 1/3, 1/3, 2/3, 3/3, 2/3, and 3/3, respectively. The baseline policy fails on all of these (0/3 across the board), except the distractors where it also achieves 0/3. The base policy fails on all. The lower success rates on tablecloth changes (1/3 each) suggest these perturbations are genuinely challenging—the model struggles to disentangle table surface appearance from object identities.
• Composite condition (mangosteen + orange bowl + beige tablecloth simultaneously): Cosmos-augmented achieves 3/3—a perfect score on the hardest test, suggesting the augmentation strategy generalizes compositionally rather than overfitting to single-factor perturbations. Both baselines achieve 0/3.

The 24/30 vs. 5/30 comparison represents a 4.8× improvement over the standard augmentation baseline, and a 24× improvement over the base policy. These are dramatic gains, but the small trial count (3 per condition) means the success rate estimates have high variance—a single failure changes the rate by 33 percentage points. The paper acknowledges failures are due to "imprecise grasps" in some cases, suggesting the policy learns the right high-level behavior but struggles with fine motor control, which is consistent with visual augmentation improving perception but not action precision.

---

Cosmos-Transfer2.5 for Driving Simulation

Multi-view visual quality (Tab. 14): Cosmos-Predict2.5-2B/auto/multiview achieves substantially better visual quality than the predecessor across all metrics:

• FVD StyleGAN: 23.060 vs. 63.685 for Predict1-7B-Sample-AV (2.8× improvement)
• FVD I3D: 25.308 vs. 69.613 (2.8× improvement)
• FID: 12.095 vs. 25.341 (2.1× improvement)
• TSE (temporal Sampson error): 0.948 vs. 0.930 (comparable, slightly worse)
• CSE (cross-camera Sampson error): 1.903 vs. 2.631 (38% improvement)

For the control-net transfer variant, Cosmos-Transfer2.5-2B/auto/multiview achieves:

• FVD StyleGAN: 24.222 vs. 56.606 for Transfer1-7B-Sample-AV (2.3× improvement)
• FVD I3D: 25.692 vs. 60.660 (2.4× improvement)
• FID: 20.022 vs. 22.633 (12% improvement)
• CSE: 2.310 vs. 1.835 (26% worse)

The CSE degradation for the transfer variant (2.310 vs. 1.835) is notable and suggests a tradeoff: the control conditioning improves visual quality and detection metrics but may slightly compromise cross-view geometric consistency compared to the predecessor. Real videos achieve CSE of 1.832 and TSE of 1.193, serving as an upper bound for what perfect geometric consistency would look like.

Detection-based control adherence (Tab. 15): Cosmos-Transfer2.5-2B/auto/multiview dramatically improves object detection metrics over the predecessor:

• LET-AP (cuboid detection): 0.394 vs. 0.243 (62% improvement)
• LET-APL: 0.254 vs. 0.154 (65% improvement)
• LET-APH: 0.383 vs. 0.236 (62% improvement)
• Lane F1: 0.637 vs. 0.604 (5.5% improvement)
• Lane x-error far ↓: 0.487 vs. 0.524 (7.1% reduction)
• Lane Category Accuracy: 0.904 vs. 0.899 (0.6% improvement)

The cuboid detection improvements are substantial (60%+), while lane detection improvements are more modest (5–7%). This asymmetry makes sense given the new world scenario map design: dynamic objects (vehicles, pedestrians) are represented as color-coded bounding boxes that are "occlusion-aware and heading-aware," providing stronger control signals for object placement, while lane line types are rendered with fine-grained colors and geometry patterns, but lane detection may be inherently limited by the 720p resolution and the challenge of rendering thin lane markings that are detectable by monocular lane detectors. The generated videos approach but do not fully match real video detection performance: LET-AP of 0.394 vs. 0.476 for real (83% of real), LET-APH of 0.383 vs. 0.462 (83%), lane F1 of 0.637 vs. 0.637 (100%), lane Category Accuracy of 0.904 vs. 0.905 (essentially equal). The model perfectly matches real lane detection F1 and category accuracy, suggesting the lane rendering is highly faithful, while the gap in cuboid detection indicates room for improvement in dynamic object placement and rendering fidelity.

---

Multi-View Robotic Camera Control

Quantitative evaluation (Tab. 17): Cosmos-Transfer2.5-2B/robot/multiview demonstrates superior cross-view consistency compared to a single-view baseline, while maintaining comparable camera trajectory accuracy:

• Translation Error: 0.08 vs. 0.08 (identical)
• Rotation Error: 0.20 rad vs. 0.19 rad (slightly worse, 5% relative)
• Sampson Error: 19.73 px vs. 26.61 px (35% improvement in cross-view geometric consistency)

The key insight is that generating multiple views simultaneously (multiview) rather than independently (singleview) dramatically improves geometric consistency across views with negligible impact on per-view camera accuracy. The Sampson error reduction from 26.61 to 19.73 represents a meaningful improvement in the model's ability to produce views that are geometrically coherent—essential for robotic applications where stereo correspondence or multi-camera triangulation might be used downstream. The evaluation uses 80 validation videos with 16 camera trajectories each, providing a reasonably sized test set.

---

Synthetic Data Generation for VLA Training

DreamGen benchmark results (Tab. 18): Cosmos-Predict2.5-14B/robot/gr00tdream-gr1 achieves the highest instruction-following scores across all evaluation axes:

• Object Generalization: 91.8% (GPT) / 69.4% (Qwen), compared to WAN2.1 at 72.0% / 58.0% and Cosmos-Predict2 (without Cosmos-Reason1) at 90.0% / 62.0%
• Behavior Generalization: 70.2% (GPT) / 59.6% (Qwen), compared to WAN2.1 at 72.3% / 55.3% and CogVideoX at 44.0% / 28.0%
• Environment Generalization: 69.0% (GPT) / 69.0% (Qwen), compared to WAN2.1 at 48.3% / 65.5% and Huanyuan at 27.6% / 27.6%

The GPT-4o and Qwen2.5-VL evaluators show substantial disagreement—often 10–20 percentage point gaps—suggesting that VLM-based evaluation of instruction following is sensitive to the choice of evaluator and that the absolute percentages should be interpreted cautiously. Nevertheless, the relative ordering is consistent: Cosmos-Predict2.5-14B ranks first or second on every metric, and the improvement over the earlier Cosmos-Predict2 (without Cosmos-Reason1) is most visible in Qwen-evaluated object generalization (69.4% vs. 62.0%, +7.4 points) and GPT-evaluated behavior generalization (70.2% vs. 59.6%, +10.6 points), suggesting Cosmos-Reason1 meaningfully improves text-to-video alignment for robotic tasks.

---

Action-Conditioned Video Generation

Bridge dataset evaluation (Tab. 19): Cosmos-Predict2.5-2B/robot/action-cond consistently outperforms the predecessor baseline:

• PSNR: 24.95 vs. 21.14 (18% improvement)
• SSIM: 0.85 vs. 0.82 (3.7% improvement)
• Latent L2: 0.28 vs. 0.32 (12.5% reduction)
• FVD: 146 vs. 190 (23% improvement)

The improvements are most dramatic in PSNR and FVD, which measure reconstruction fidelity and video-level perceptual quality respectively. SSIM improvement is modest (0.82 → 0.85), suggesting both models capture coarse structural information similarly, while the finer pixel-level accuracy (PSNR) and temporal dynamics (FVD) are substantially better in the 2.5 model.

Action conditioning architecture ablation (Tab. 20): The paper compares three methods for incorporating action conditioning:

• Time embedding (proposed): PSNR 24.95, SSIM 0.85, Latent L2 0.28, FVD 146
• Cross-attention: PSNR 24.41, SSIM 0.84, Latent L2 0.28, FVD 159
• Channel concatenation: PSNR 23.11, SSIM 0.78, Latent L2 0.35, FVD 267

Time embedding conditioning (adding action features to timestep embeddings) clearly dominates, with cross-attention a close second and channel concatenation substantially worse. The performance gap between time embedding and channel concatenation is large—PSNR drops from 24.95 to 23.11 (7.4% decrease), FVD nearly doubles from 146 to 267 (83% increase)—suggesting that concatenating action features to the latent channel dimension disrupts the learned latent structure, while adding them to timestep embeddings preserves it. The cross-attention approach performs reasonably well but is slightly worse than time embedding on all metrics, possibly because cross-attention introduces additional parameters that require more training data or because the action sequence length (8 timesteps × 7 action dimensions = 56 values per chunk) is too short for effective attention-based integration.

---

Ablation Studies and Robustness Checks

Cosmos-Reason1 vs. T5 text encoder: While not presented as a formal ablation table, the comparison between Cosmos-Predict2.5-14B/robot/gr00tdream-gr1 and the earlier Cosmos-Predict2-14B/robot/gr00tdream-gr1 on DreamGen (Tab. 18) provides evidence for the text encoder upgrade. The 2.5 version (with Cosmos-Reason1) outperforms the 2.0 version (without Cosmos-Reason1) on 5 out of 6 sub-metrics, with the only exception being GPT-evaluated behavior generalization where 2.0 scores higher (59.6% vs. 70.2%—actually 2.5 wins here too, wait: 2.5 scores 70.2% vs. 2.0's 59.6%, so 2.5 wins on all metrics). The improvements range from +1.8 points (GPT object, 91.8% vs. 90.0%) to +10.6 points (GPT behavior, 70.2% vs. 59.6%). This is a weak ablation since many other factors changed between Predict2 and Predict2.5, but it provides directional evidence.

Removal of absolute positional embeddings: No explicit ablation is provided. The paper states that removing absolute embeddings and keeping only relative RoPE "gains greater flexibility for handling higher-resolution content and longer video sequences during post-training," but does not compare models with and without absolute embeddings. This is a missing ablation that would have strengthened the architectural justification.

Model merging method comparison (Fig. 4): Model Soup, TIES, DARE-TIES all perform comparably, with DARE-Linear as the exception that underperforms. The paper does not provide exact numbers for each method, making it a qualitative rather than quantitative comparison. The finding that grid search over merging hyperparameters outperforms heuristic selection based on fine-tuned model win rates is interesting but presented without supporting data.

Domain-specific SFT vs. joint SFT: The paper argues that domain-specific SFT followed by merging is preferable to joint training, but does not ablate against a jointly trained model. The justification is practical ("without the need to balance mixture ratios") rather than empirical, representing a missing comparison.

RL with vs. without diffusion loss regularization: The paper states that regularization "effectively alleviates the reward hacking phenomenon" (Section 4.2.2) and cites Ye et al. (2025), but does not show what happens without it. This is a significant missing ablation, since one of the paper's claimed innovations is the application of RL to flow-matching video diffusion with this specific regularization technique. The human evaluation results (Fig. 5) provide indirect evidence that the RL phase does not produce reward-hacked outputs (since human raters prefer RL outputs), but cannot isolate the causal effect of the regularization term.

Control block placement (sequential vs. distributed): The paper states that Cosmos-Transfer2.5 distributes control blocks "by inserting one after every 7 blocks" rather than placing all four at the start as in Transfer1. This architectural change is not ablated—no comparison is provided between distributed and sequential control block placement at the same model size. The performance improvements in Tab. 12 and Fig. 10 are confounded with the base model upgrade, data curation improvements, and control block placement, making it impossible to attribute gains to any single factor.

Context parallelism size scaling (Tab. 9): The MFU drops from 36.49% (2B, CP=2) to 33.08% (14B, CP=8), representing a 9.3% relative efficiency decrease. This provides indirect evidence for communication overhead scaling with context parallelism degree, but no sweep of different CP sizes is reported, so the relationship between CP size and MFU cannot be characterized.

Action conditioning modality (Tab. 20): This is a genuine ablation comparing time embedding, cross-attention, and channel concatenation for the same model and data. The results clearly show time embedding as the superior approach, with channel concatenation substantially worse (FVD 146 vs. 267). The ablation is clean—same base model, same data, different conditioning mechanisms—and directly supports the architectural choice.

Single-view vs. multi-view camera control (Tab. 17): Comparing multi-view generation against a single-view baseline reveals that generating multiple views simultaneously reduces cross-view Sampson error from 26.61 to 19.73 (35% improvement) with negligible impact on per-view camera trajectory accuracy. This is a well-designed ablation that isolates the benefit of joint multi-view generation over independent per-view generation.

Timestep distribution shift and high-noise sampling: The paper observes frame transition artifacts and hypothesizes they arise from insufficient high-noise training. The fix—5% of samples drawn from the top 2% of noise distribution—is described as "significantly reducing" the artifacts, but no quantitative comparison (e.g., FVD with and without the fix) is provided. This is a qualitative claim without quantitative support.

Real video reference for driving metrics (Tab. 14, Tab. 15): Including real video metrics as an upper bound is a thoughtful addition that contextualizes the generation quality numbers. The gaps between generated and real video metrics quantify how far current models are from photorealism: FVD StyleGAN of 23.060 vs. ideally near zero, LET-AP of 0.394 vs. 0.476 for real videos, etc.

---

Critical Assessment

Claim 1: Cosmos-Predict2.5 achieves substantial improvements over Cosmos-Predict1 in video quality and instruction alignment.

This claim is strongly supported by the PAI-Bench results (Tab. 10, Tab. 11), human evaluations (Fig. 6, Fig. 7), and the DreamGen benchmark (Tab. 18). However, the comparison is not head-to-head on identical metrics for all settings: Cosmos-Predict1's PAI-Bench scores are not reported, so the "improvement over predecessor" claim relies on the Cosmos-Transfer comparison (Tab. 12) and the multi-view driving comparison (Tab. 14, Tab. 15) rather than a direct Predict1 vs. Predict2.5 evaluation. The PAI-Bench results compare Cosmos-Predict2.5 against Wan models, showing it leads the field, but the claim of improvement over the predecessor is primarily evidenced through the transfer and driving experiments. The paper would be strengthened by reporting Cosmos-Predict1's PAI-Bench scores in the same tables.

Claim 2: Cosmos-Transfer2.5 is 3.5× smaller than Cosmos-Transfer1 yet delivers higher fidelity and robust long-horizon generation.

This is the paper's most cleanly supported claim. Tab. 12 provides unambiguous evidence: Cosmos-Transfer2.5-2B outperforms Cosmos-Transfer1-7B on Overall Quality scores across all modalities (9.75 vs. 6.56 for blur, 8.73 vs. 6.76 for edge, 8.85 vs. 6.89 for depth, 8.81 vs. 6.02 for segmentation). The RNDS curves (Fig. 10) clearly demonstrate the long-horizon error accumulation advantage. However, several confounds weaken causal attribution: the base model improved (Predict2.5 vs. Predict1), the training data was more carefully curated, and the control block placement changed. No ablation isolates which of these factors is responsible for the improvement. The claim that the model is 3.5× smaller is a fact, but the claim that the smaller size enables the better performance is not established—it could have improved despite being smaller, with other factors dominating.

Claim 3: These advances enable reliable synthetic data generation for VLA training.

Supported by the DreamGen benchmark (Tab. 18), where Cosmos-Predict2.5-14B achieves leading instruction-following scores. However, DreamGen measures whether generated videos depict the correct scenario, not whether those videos are useful for training. The robot policy learning experiment (Tab. 13) directly tests utility for training—the Cosmos-Transfer2.5-augmented policy achieves 24/30 successes—but this is a single-task, single-robot, single-environment result with only 100 base demonstrations. It demonstrates feasibility but not reliability: we don't know if the gains generalize to other tasks, robots, or data regimes. The 3-trial evaluation per condition means success rate estimates have wide confidence intervals (a 1/3 vs. 2/3 difference could be noise). A stronger evaluation would include multiple tasks, multiple seeds, and more trials.

Claim 4: These advances enable policy evaluation and closed-loop simulation for robotics and autonomous systems.

Partially supported. The action-conditioned video generation results (Tab. 19) show improved fidelity over the predecessor, which is necessary for policy evaluation, but the paper does not actually demonstrate closed-loop policy evaluation—it shows open-loop video prediction conditioned on recorded action sequences. True closed-loop evaluation would require an agent to select actions based on generated observations, feed those actions back into the model, and demonstrate that the resulting trajectories are useful for assessing policy quality. For driving, the multi-view generation results (Tab. 14, Tab. 15) and Fig. 15 show improved visual quality and control adherence, but again, no closed-loop driving policy evaluation is demonstrated. The long-horizon RNDS curves (Fig. 10) suggest autoregressive stability, but stability does not equal utility—a stable but physically inaccurate simulator would produce misleading policy evaluations.

Claim 5: RL post-training with data regularization effectively improves video quality and alignment.

Supported by the reward improvements (Tab. 6) and human evaluations (Fig. 5). However, the missing ablation—RL without data regularization—means we cannot assess how critical the regularization is. The paper attributes the technique's importance to preventing reward hacking, but this is asserted rather than demonstrated. Given that reward hacking is a well-known failure mode in RLHF, the absence of a "no regularization" baseline is a significant gap. The human evaluation shows RL-trained outputs are preferred at roughly 2:1 ratios, but with high tie rates (37–41%), suggesting the improvements are real but fine-grained. A more rigorous evaluation would report inter-annotator agreement and statistical significance.

General weaknesses in the experimental design:

1. Absence of error bars or confidence intervals uniformly across all results. Win rates, success rates, benchmark scores, and detection metrics are all reported as point estimates without any measure of uncertainty. For the robot policy evaluation with 3 trials per condition, a single failure changes the success rate by 33 percentage points—confidence intervals would reveal how fragile these estimates are.

2. Reliance on automated metrics whose correlation with downstream utility is unvalidated. PAI-Bench scores, VideoAlign rewards, and detection metrics are proxies for what we actually care about (does the synthetic data improve policy performance? do the generated driving scenes enable safe behavior validation?). The robot policy experiment directly tests utility for one case, but this is the exception. For driving, VLA training, and most other applications, the paper relies entirely on proxy metrics. The DreamGen benchmark partly addresses this by measuring instruction following (a more direct utility proxy), but still doesn't close the loop to actual policy improvement.

3. Missing head-to-head comparisons on equal footing. The paper compares Cosmos-Predict2.5 against Wan models using PAI-Bench, but these models were trained on different data, with different architectures, and for different purposes. A fairer comparison would give competing models the same Physical AI fine-tuning data and post-training pipeline, to isolate whether the Cosmos architecture is genuinely better or whether the data and training recipe are the primary differentiators.

4. The DreamGen evaluator disagreement problem. GPT-4o and Qwen2.5-VL produce substantially different scores (e.g., for CogVideoX on Object Generalization: 72.0% GPT vs. 38.0% Qwen—a 34-point gap). This raises the question: which score reflects true instruction-following quality? Without human validation of the DreamGen scores, the absolute numbers are hard to interpret. Relative rankings are more reliable, but the paper does not compute rank correlations between evaluators to quantify agreement.

5. The 4K cooldown model's dramatic win rate (72.6%) is presented in text but not plotted in Fig. 3. This is a curious omission—the most dramatic SFT improvement is described but not visualized alongside the other domains, making it harder to contextualize.

6. The distillation "improvement" in I2W (0.810 → 0.816, Tab. 8) is unexplained. When a student model outperforms its teacher, it suggests either (a) the distillation process provides beneficial regularization, (b) the metric is noisy and the difference is within noise, or (c) there's a methodological issue. The paper mentions the result without analysis, missing an opportunity to provide insight.

7. No ablation on the data filtering survival rate. The paper's central claim—that strict 4% filtering is critical—is never tested. We don't know how model quality would change at 10% survival, 20% survival, or 30% survival (the Cosmos-Predict1 rate). Given that this 4% rate is one of the paper's most distinctive design choices and is repeatedly emphasized as the key differentiator, the absence of a filtering-rate ablation is a substantial gap.

8. Single trial of the robot policy evaluation protocol. Each policy is trained once and evaluated on 3 trials per condition. Policy learning is stochastic (initialization, data ordering, etc.), so a single training run does not establish reliability. Training 3–5 independent policies per condition and reporting mean and standard deviation of success rates would substantially strengthen the robotics results.

What would have strengthened the paper:

• A direct Predict1 vs. Predict2.5 comparison on PAI-Bench, with both models evaluated in the same setting.
• An ablation varying data filtering survival rate (e.g., 4%, 10%, 20%, 30%) to quantify the impact of data curation strictness on downstream Physical AI task performance.
• An RL regularization ablation (RL with vs. without diffusion loss) to demonstrate the importance of the technique and characterize reward hacking behavior.
• Multiple training seeds for robot policy learning, with confidence intervals on success rates.
• Closed-loop policy evaluation for driving simulation (e.g., train a driving policy using generated data, evaluate its behavior in simulation or on real data, compare to policy trained on real data alone).
• A control block placement ablation (sequential at start vs. distributed) to isolate the architectural contribution to long-horizon error reduction.
• Human validation of DreamGen evaluator scores to establish which VLM evaluator (GPT-4o or Qwen2.5-VL) better correlates with human judgment of instruction following.
• Wall-clock timing or FLOPs counting for generation, to contextualize the efficiency claims with actual computational cost rather than parameter count alone.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Not Accounted for in Efficiency Claims

The assumption or constraint. The entire Cosmos-Predict2.5 framework—domain-specific SFT, model merging, RL post-training, control-net conditioning—is designed around the principle that simulated environments must be high-fidelity for Physical AI tasks. The paper quantifies this fidelity through automated benchmarks (PAI-Bench, DreamGen, detection metrics) and human evaluations, and claims substantial improvements over predecessors and competing models. However, the numbers that establish these improvements—PAI-Bench Domain and Quality scores, Tab. 12 control adherence metrics, Tab. 15 cuboid detection AP—are computed on outputs from models that have already undergone the full post-training pipeline. The cost of constructing that pipeline—the compute spent on domain-specific SFT, model merging hyperparameter sweeps, RL training with VideoAlign reward evaluation, and timestep distillation—is not amortized into any efficiency metric.

What is missing is an accounting of the meta-cost: the total FLOPs or GPU-hours consumed to produce a ready-to-deploy Cosmos-Predict2.5 model, compared to training a larger general-purpose model on internet-scale data without specialized post-training. Tab. 9 reports training MFU (36.49% for 2B, 33.08% for 14B) and the number of GPUs (4096 H100s), but does not report total training time or FLOPs for any stage. Section 4.2.1 mentions each domain-specific SFT model is trained for 30,000 iterations with a batch size of 256, Section 4.2.2 mentions the RL phase runs for 256 steps with a batch size of 32, and Section 4.2.3 notes the distillation framework is separate. None of these component costs are aggregated into a total post-training budget.

The consequence. A practitioner deciding whether to adopt Cosmos-Predict2.5 for their Physical AI application cannot answer the question: "Is it more cost-effective to post-train Cosmos-Predict2.5 on my domain, or to simply train a larger general-purpose model on my data from scratch?" The paper argues that curation quality dominates scale (a central claim in Section 4), but this claim is tested only in the forward direction—small, curated models outperform larger, less curated ones on benchmarks. The reverse comparison is never made: what if the compute spent on curation and post-training were instead spent on scaling the base model? If domain-specific SFT on five domains (each 30,000 iterations at batch size 256), plus a 256-step RL phase, plus model merging hyperparameter sweeps (20+ merged models evaluated), plus timestep distillation, collectively cost as much as training a 7B model from scratch, then the efficiency advantage of the 2B model over a 7B baseline is illusory—the fair comparison would be 2B + post-training vs. 7B from scratch, and the paper provides no basis for making that comparison.

A secondary consequence is that the paper's implicit recommendation—train on aggressively filtered, domain-curated data—cannot be calibrated. How much filtering is enough? The 4% survival rate is presented as a key differentiator from Cosmos-Predict1's 30% rate (Section 2.1), but the absence of a filtering-rate ablation means the marginal benefit of each additional filtering stage is unknown. The VLM-based final filtering pass is described as applying "at the very end of filtering because it is computationally more expensive." If that VLM pass (run on all 6 billion+ clips) dominated the filtering cost but contributed only a small fraction of the quality improvement, a practitioner could skip it and allocate the saved compute elsewhere. The paper provides no evidence to guide such decisions.

What evidence exists in the paper. The paper contains no aggregate FLOPs or GPU-hour accounting for any training stage. Tab. 9 reports MFU for pre-training of the base model only, with no comparable efficiency metrics for SFT, RL, or distillation. Section 4.2.1 states SFT is done with "the same hyperparameter settings as the final stage of pretraining" but does not specify how many GPUs or what wall-clock time this requires. Section 4.2.3 mentions the elastic reward service processes reward evaluations asynchronously, but does not quantify the total reward computation budget (how many VLM forward passes, on what hardware). Section 4.3 describes the infrastructure in detail—FSDP2, Ulysses context parallelism, selective activation checkpointing, the elastic reward service—but all of these are described as mechanisms to make training feasible, not as cost estimates.

Mitigation status. The paper does not acknowledge this as a limitation. There is no discussion of total post-training cost, no suggestion that future work should benchmark the cost-benefit tradeoff of curation pipelines, and no mention of amortized cost in the conclusion or future work sections. The paper frames itself as a scaling contribution ("trained on 200M curated video clips"), but the efficiency claim is about model size at inference time (3.5× smaller, 2B vs. 7B), not about total resource consumption over the model lifecycle. For a paper that makes "smaller is better if curated" its central implicit argument, the absence of a curation-cost accounting is a substantive gap.

---

Closed-Loop Policy Evaluation Is Claimed but Never Demonstrated

The assumption or constraint. The paper's introduction and applications sections repeatedly frame Cosmos-Predict2.5 as enabling "closed-loop simulation" for robotics and autonomous systems. Section 1 states that world simulators "enable agents to acquire perception and control skills entirely in silicon before deployment in the real world." Section 6.6 frames action-conditioned world generation as "particularly well-suited for policy evaluation." Section 6.3 describes the driving simulation as generating scenes from "world scenario maps" with the implication that these could be used to evaluate driving policies.

What the paper actually demonstrates is open-loop video prediction conditioned on pre-recorded action sequences, not closed-loop policy evaluation where an agent selects actions based on generated observations and those actions feed back into the model. In Section 6.6, the Bridge dataset evaluation takes "a single conditional image together with a sequence of robot actions, and generates a chunk of future frames that follow the provided action sequence." The action sequence is from the recorded dataset, not from a policy making decisions based on the generated frames. The evaluation metrics—PSNR, SSIM, Latent L2, FVD—measure reconstruction fidelity against ground-truth videos, which is the correct evaluation for open-loop prediction quality but says nothing about whether a policy trained with this model in the loop would exhibit realistic behavior or develop corrigible errors.

The driving evaluation (Section 6.3) follows the same pattern: the model generates multi-view videos from world scenario maps, and the quality is assessed via detection metrics (cuboid detection AP, lane detection F1) and visual quality metrics (FVD, FID). These measure whether the generated scenes look like real driving scenes and contain correctly placed objects, but they do not measure whether a driving policy deployed in this simulated environment would make safe decisions or whether the simulation would maintain physical fidelity under adversarial or edge-case policy actions.

The consequence. The gap between open-loop prediction and closed-loop simulation is fundamental, not cosmetic. In open-loop prediction, the model generates frames conditioned on ground-truth actions; any errors in the generated frames (a slightly misplaced object, an imprecise gripper position) do not propagate because the next chunk's conditioning still uses ground-truth frames and ground-truth actions. In closed-loop simulation, the model generates frame t, a policy observes frame t and selects action a_t, and then the model must generate frame t+1 conditioned on frame t (not the ground-truth frame) and action a_t (not the ground-truth action). Errors in frame t now cascade: a misplaced object at time t means the policy's action a_t may be inappropriate for the true state, the model generates frame t+1 from an erroneous starting state with a potentially erroneous action, and divergence compounds.

The RNDS metric (Section 6.1.2) partially addresses this concern by measuring visual quality degradation in autoregressive generation, but even RNDS assumes the ground-truth actions and ground-truth conditioning frames—it measures open-loop autoregressive stability, not closed-loop interaction stability. A model could have perfect RNDS (stable video quality over long horizons) but still be useless for closed-loop simulation if small initial errors in object positions cause the policy to select actions that the model cannot plausibly render.

For the specific claim about closed-loop policy evaluation, the absence of a demonstrated closed-loop experiment means a practitioner cannot assess whether Cosmos-Predict2.5 is suitable for the use case the paper most strongly advocates. The robot policy experiment in Section 6.2 comes closest: it uses Cosmos-Transfer2.5 for data augmentation, trains a policy on the augmented data, and evaluates the policy on a real robot. This demonstrates that the synthetic data is useful for policy training, but it does not evaluate the policy in simulation—the policy is evaluated on the real robot, and the model is used only for data generation, not for closed-loop rollouts. The distinction is important because generating training data (where occasional artifacts are averaged out over many episodes) is a much more forgiving task than providing a reliable interactive simulation.

What evidence exists in the paper. The RNDS curves (Fig. 10) show autoregressive stability for up to 18 chunks (approximately 100 seconds), but these curves are for the Cosmos-Transfer2.5 model in an open-loop setting with ground-truth conditioning. The action-conditioned generation results (Tab. 19, Fig. 20) are strictly open-loop reconstruction evaluations. The multi-view driving evaluation (Tab. 14, Tab. 15) measures visual quality and detection metrics, not closed-loop policy performance. There is no experiment anywhere in the paper where an agent (learned policy or otherwise) selects actions based on generated observations, feeds those actions back into the model, and the resulting trajectories are evaluated for task success or safety.

Mitigation status. The paper does not acknowledge the open-loop vs. closed-loop gap explicitly. Section 6.6 states the model "is particularly well-suited for policy evaluation" without qualifying that only open-loop policy evaluation is supported by the presented evidence. The abstract claims "these capabilities enable more reliable synthetic data generation, policy evaluation, and closed-loop simulation"—conflating capabilities that are demonstrated (synthetic data generation) with capabilities that are only gestured at (closed-loop simulation). This is not a hidden limitation; it is an overclaim that a careful reader can identify by noting that no closed-loop results are reported. The paper would be substantially stronger if it either (a) demonstrated a closed-loop experiment, even on a simple toy task, or (b) explicitly scoped its claims to open-loop prediction and data generation, acknowledging closed-loop evaluation as future work.

---

Hard Problems—Scenes Outside the Curation Distribution—Remain Fundamentally Unaddressed

The assumption or constraint. The Cosmos-Predict2.5 pipeline is built on the premise that high-quality, domain-curated data enables high-fidelity Physical AI simulation. The paper curates five specific domains—robotics, autonomous driving, smart spaces, human dynamics, and physics—with domain-specific filtering, captioning, and metadata injection (Sections 2.2.1–2.2.5). Domain-specific SFT further specializes the model for object permanence, high motion, complex scenes, driving, and robotic manipulation (Section 4.2.1). This is a data-centric strategy: the model learns to simulate scenarios that are well-represented in the training distribution.

What happens when the model encounters scenarios that are not well-represented—scenes with novel object configurations, unusual physical interactions, robot embodiments not seen during training, or driving conditions (extreme weather, rare road geometries, unusual agent behaviors) that fall outside the distribution of the curated data? The paper provides no systematic evaluation of out-of-distribution (OOD) generalization for the generation model itself. The robot policy experiment (Section 6.2) evaluates policy generalization to unseen visual conditions (mangosteen instead of apple, black tablecloth, open drawers), but this tests whether data augmentation helps a policy generalize—it does not test whether the generation model can produce plausible videos of those unseen conditions. The prompts for augmentation are generated by an LLM filling in templated slots (COLOR_APPLE, COLOR_BOWL, SENTENCE_BACKGROUND), and Cosmos-Transfer2.5 is used to render the specified variations. But the visual plausibility of those renderings is not evaluated—the paper does not report whether the generated scenes with, say, a mangosteen being manipulated by a robot look physically realistic, or whether the model has seen enough mangosteens during training to render them correctly.

The physics domain curation (Section 2.2.5) attempts to address this by including "a taxonomy of visually observable physical phenomena spanning core domains such as classical mechanics and fluid mechanics," with examples like "shattering glass, colliding rolling balls, or flowing water." But a taxonomy-driven curation strategy can only cover phenomena that curators have anticipated and included. Truly novel physical interactions—a robot interacting with an object that has unusual material properties, a vehicle navigating a road geometry never seen in training—fall outside the coverage of even a well-designed taxonomy.

The consequence. The failure mode is silent degradation: the model generates videos that look visually plausible but are physically wrong in ways that a human unfamiliar with the specific scenario might not notice, and a downstream policy using those videos for training would learn incorrect behaviors. This is the analog of the hallucination problem in language models, but with higher stakes because physical actions have irreversible consequences. A robot trained on synthetic data that incorrectly renders the dynamics of a novel object will learn a grasping strategy that fails when deployed; a driving policy validated on simulated edge cases that the model renders incorrectly will be certified as safe under conditions it cannot actually handle.

The paper's own qualitative comparisons (Fig. 15) show the predecessor model hallucinating a "distorted black car behind the silver vehicle" and rendering "the vehicle in the central lane driving on the wrong side of the street with an incorrect orientation, as well as a truck instead of a pedestrian close to the sidewalk." These are dramatic, visually obvious hallucinations in a domain (driving) for which the model has abundant training data (3.1M clips). If the model hallucinates this strongly in-domain, the risk of physically consequential but visually subtle errors in out-of-distribution scenarios is substantial, and the paper provides no framework for detecting or mitigating such errors.

The multi-view generation results (Fig. 16, Fig. 17, Fig. 19) show qualitatively impressive outputs, but all examples appear to be from training-like scenarios (standard kitchen tabletop manipulation, standard driving views). There is no demonstration of the model's behavior on edge cases: what happens when the robot's arm occludes the object being manipulated? When the lighting changes dramatically mid-sequence? When two objects collide in ways not well-represented in training? The paper's reliance on curated data means these edge cases are systematically underrepresented, and the model's behavior on them is unknown.

What evidence exists in the paper. The RNDS curves (Fig. 10) measure visual quality degradation over long horizons, but this is degradation from a known starting point on evaluation videos that were curated. The detection metrics on driving scenes (Tab. 15) show a gap between generated (LET-AP 0.394) and real (0.476) performance, but this gap is measured on RDS-HQ clips that are drawn from the same distribution as the training data—it does not reveal whether the gap would widen under distribution shift. The DreamGen benchmark (Tab. 18) explicitly tests generalization to unseen objects, behaviors, and environments, and Cosmos-Predict2.5-14B achieves strong scores (91.8% GPT, 69.4% Qwen on object generalization). But DreamGen measures instruction following—does the video show the specified action with the specified object?—not physical fidelity. A video could show a robot correctly identifying and moving toward a mangosteen while rendering the mangosteen's physical dynamics (how it deforms under grasping, how it slides on the table) completely incorrectly, and still score perfectly on DreamGen.

The robot policy experiment (Tab. 13) provides the strongest OOD evidence: a policy trained with Cosmos-Transfer2.5 augmentation succeeds on novel visual conditions. But this tests the policy's robustness, not the model's OOD generation quality. The augmented training data may include physically imperfect renderings that nevertheless provide useful visual variation—the policy learns to be invariant to certain visual features, which is helpful even if the renderings are not perfectly realistic. This is a success for data augmentation but does not validate the model as a reliable simulator of OOD physics.

Mitigation status. The paper does not frame OOD generation quality as a limitation, nor does it propose mechanisms for detecting when the model is operating outside its reliable regime. The physics domain curation (Section 2.2.5) and the object permanence SFT domain (Section 4.2.1) are implicit attempts to broaden the coverage, but they address coverage breadth rather than OOD reliability. The RL phase's data regularization (Section 4.2.3) prevents reward hacking by keeping outputs near the training distribution, which is a form of conservatism, but it does not provide a signal for when the model should refuse to generate or flag uncertainty. The timestep distillation (Section 4.2.3) preserves the teacher model's output distribution but does not add robustness to novel inputs.

A practitioner deploying Cosmos-Predict2.5 for safety-critical Physical AI applications needs to know: when will this model produce plausible-looking but physically incorrect simulations? The paper provides no answer, and the data-centric paradigm—the model is only as good as its training distribution coverage—implies that the answer is "whenever the scenario is sufficiently different from the training data," which is a large and practically unavoidable set of scenarios for any real-world deployment.

---

The Robot Policy Evaluation Uses 3 Trials per Condition—Success Rate Estimates Have Unreported Variance

The assumption or constraint. Section 6.2 reports the paper's most direct evidence that Cosmos-Transfer2.5 enables practical Physical AI applications: a real-robot policy trained with augmentation achieves 24/30 successes across 10 test scenarios (3 trials each), dramatically outperforming a baseline policy (5/30) and a base policy (1/30). These numbers are reported as point estimates without confidence intervals, error bars, or statistical significance tests. The evaluation protocol—single policy training run, 3 trials per condition, fixed object poses and environment configurations—is described in Section 6.2.3: "For each test scenario, we perform three trials and fix the object pose and environment configuration to ensure fair comparisons."

The sample size of 3 trials per condition is extremely small for estimating a Bernoulli success rate. With 3 trials, the possible observed success rates are 0/3, 1/3 (33%), 2/3 (67%), or 3/3 (100%). The binomial confidence interval for 3/3 successes is approximately 29–100% (using the Clopper-Pearson exact method at 95% confidence). In other words, if the true underlying success rate were 50%, observing 3/3 successes in 3 trials would occur with probability 0.5³ = 12.5%—far from statistically significant evidence that the true rate exceeds 50%. For conditions where the Cosmos-augmented policy achieves 1/3 successes (beige table, black table), the interval is approximately 1–91%, meaning the true success rate could be anywhere from near-zero to near-perfect.

The fixed object poses and environment configurations eliminate a source of variance (trial-to-trial variability in initial conditions) but at the cost of generalization: the 3 trials become repeated measurements of the same scenario rather than independent samples from a distribution of scenarios. If the policy succeeds on a particular object pose configuration but fails on a slightly different one, this evaluation protocol cannot detect that sensitivity. The paper's Table 13 caption notes "we perform three trials and fix the object pose and environment configuration to ensure fair comparisons"—fairness is achieved by making conditions identical across policies, but the cost is that the estimated success rates are conditional on exactly those poses and configurations, not on the distribution the policy would encounter in deployment.

The consequence. A practitioner reading "Cosmos-augmented policy achieves 24/30 successes vs. 5/30 for baseline" might reasonably conclude that Cosmos-Transfer2.5 augmentation provides a robust, large-magnitude improvement. But the statistical uncertainty is so large that several of the per-condition comparisons are uninterpretable. For the beige table condition, both policies achieve 1/3—we literally cannot distinguish whether the true underlying success rates are 33%, 50%, or 80%, because the confidence intervals for both estimates nearly entirely overlap. For the black table condition, the same issue applies. For the spotlight condition (2/3 vs. 0/3), the difference appears larger but still has a p-value of approximately 0.10 by Fisher's exact test—conventionally not significant, and entirely dependent on what would have happened in a fourth trial.

The aggregate 24/30 vs. 5/30 comparison pools across conditions. With 10 conditions and 3 trials each, the effective sample size for detecting a difference between policies depends on how correlated successes are across conditions (which we cannot estimate from the reported data). If the policies' performances are uncorrelated across conditions, the 30-trial aggregate is meaningful; if successes cluster by condition (e.g., both policies succeed on easy conditions, both fail on hard conditions), the effective sample size is closer to the number of conditions (10) than the number of trials (30). The paper provides no information about condition-level correlations.

This limitation matters beyond statistical pedantry because the robot policy experiment is the paper's strongest claim to practical impact. The PAI-Bench scores, detection metrics, and DreamGen results are benchmarks—useful for comparing models but not direct evidence of utility. The 24/30 success rate is the paper's only demonstration that Cosmos models improve downstream task performance. If that demonstration is statistically fragile, the paper's central claim—that these models enable practical Physical AI applications—rests on weaker evidence than the headline numbers suggest.

What evidence exists in the paper. The paper reports exactly what was measured: success counts out of 3 trials for each of 10 conditions and 3 policies, totaling 90 real-robot trials (Table 13). No statistical analysis—confidence intervals, hypothesis tests, effect sizes, or power calculations—is reported. The evaluation protocol description (Section 6.2.3) mentions the trial count and fixed conditions but does not discuss statistical power or uncertainty. The paper's Fig. 13 shows qualitative rollout images for each condition, which provides anecdotal evidence of policy behavior but does not address the quantitative reliability of the success rates.

Mitigation status. The paper does not acknowledge the small sample size as a limitation. The 24/30 and 5/30 numbers are reported as point estimates without qualification, and the discussion (Section 6.2.3) treats them as definitive: "The [Cosmos-Transfer2.5-2B]-augmented policy achieves 24 successes out of 30 trials, clearly outperforming both baselines. It demonstrates markedly higher robustness and generalization to novel test-time object and environment changes." This language implies a level of certainty that the experimental design does not support. The paper would benefit from either (a) more trials per condition (10+ would substantially narrow confidence intervals), (b) multiple training seeds to assess policy training stability, or (c) explicit reporting of confidence intervals and a more measured interpretation of the results. The limitation is partially mitigated by the fact that the qualitative trends are directionally consistent (the Cosmos-augmented policy outperforms baselines on every condition except the base setting where the baseline policy also achieves 3/3), but the magnitude of the improvement and its reliability remain uncertain.

---

Single-Task, Single-Robot, Single-Environment Evaluation Affords No Claims About Generalization of the Augmentation Approach

The assumption or constraint. The robot policy learning experiment (Section 6.2) is conducted on exactly one task (bimanual pick-and-place of an apple into a bowl), with exactly one robot platform (semi-humanoid with two 7-DoF Kinova Gen3 arms, Robotiq 2F-140 grippers, Intel RealSense D455 egocentric camera), in exactly one environment (a fixed kitchen setup with specific cabinetry, countertops, and background). The evaluation tests generalization to nine novel visual conditions, but all nine conditions are variations of the same task in the same environment—changing object colors, adding a tablecloth, modifying lighting, adding distractors, altering background elements. The underlying task structure (reach, grasp apple, grasp bowl, place apple in bowl, set bowl down) is identical across all conditions.

The paper frames this experiment as demonstrating that Cosmos-Transfer2.5 "can be used as a visual synthetic data generator to augment robot policy training and enable generalization to unseen visual scenarios" (Section 6.2). This framing implies generality across tasks, robots, and environments, but the experimental evidence supports at most the claim that the approach works for this specific task-robot-environment combination.

The consequence. The primary risk is that the specific choice of task and environmental variations is favorable to the augmentation approach in ways that would not generalize. The task involves two rigid objects (apple and bowl) with simple geometries and well-defined grasping affordances. The augmentation prompts (Section 6.2.2) modify surface-level appearance properties: COLOR_APPLE, COLOR_BOWL, SENTENCE_LIGHT (lighting), SENTENCE_BACKGROUND (background elements). Cosmos-Transfer2.5, with its edge-control conditioning, is well-suited to preserving underlying scene structure while modifying surface textures and colors—exactly the type of variation the prompts request. If the task had involved deformable objects, objects with complex articulated parts, or variations in object geometry rather than just appearance, the augmentation approach might perform very differently.

The fixed robot platform means we cannot assess whether the approach transfers across embodiments. The egocentric camera viewpoint, the specific arm kinematics, and the gripper design all influence what visual features are relevant for the policy. A different robot with a different camera placement or different end-effector might require different augmentation strategies that Cosmos-Transfer2.5 may or may not support equally well.

The fixed environment (kitchen with specific countertops, cabinetry, dishwasher, black curtain) means the policy's demonstrated robustness to background changes is tested only within the range of variations the prompt template can express. The template (Section 6.2.2) allows changes to TABLE, COLOR_APPLE, COLOR_BOWL, SENTENCE_LIGHT, and SENTENCE_BACKGROUND, but all variations operate within the semantic space of "kitchen scenes." The policy is never tested in a completely different environment—a warehouse, an outdoor setting, a laboratory bench—and we do not know whether Cosmos-Transfer2.5 could generate plausible augmentations for such radical domain shifts, or whether the policy would generalize to them.

What evidence exists in the paper. The paper reports exactly one task, one robot, one environment. Table 13 shows 10 test conditions, all within the same kitchen environment. Figure 13 shows rollout images for all 10 conditions, confirming they are visual variations of the same scene. The augmentation strategy (Section 6.2.2) uses a single prompt template with five variable slots, further indicating that all variations are within a constrained semantic space. The paper does not claim multi-task, multi-robot, or multi-environment generalization—it simply does not address these dimensions. But the absence of evidence is not acknowledged: the discussion in Section 6.2.3 concludes that "the results indicate that [Cosmos-Transfer2.5] provides a promising, lightweight, and effective pipeline for synthetic data generation in robotics" without scoping this conclusion to the specific task and setting tested.

Mitigation status. The paper does not acknowledge the single-task limitation as a constraint on the conclusions. The phrase "a promising, lightweight, and effective pipeline for synthetic data generation in robotics" is a general claim about robotics, not a specific claim about bimanual pick-and-place in a fixed kitchen setting. The paper would be strengthened by either (a) results on additional tasks, robots, or environments, or (b) explicit scoping of the claim to visual-augmentation-for-single-task-policy-learning, with multi-task and cross-embodiment generalization identified as future work. The DreamGen benchmark (Tab. 18) provides complementary evidence that Cosmos-Predict2.5 can generate diverse robotic manipulation videos across objects, behaviors, and environments, but DreamGen evaluates video generation quality for VLA training, not the effectiveness of Cosmos-Transfer2.5 augmentation for policy learning specifically. These are related but distinct capabilities.

---

The RL Post-Training Pipeline Lacks a "No Regularization" Baseline, Leaving the Contribution of Data Regularization Unquantified

The assumption or constraint. Section 4.2.2 describes the RL post-training phase in detail, including the use of data-regularized training to prevent reward hacking: "As discussed in Ye et al. (2025), we use diffusion loss on the fine-tuning dataset for regularization, which effectively alleviates the reward hacking phenomenon." The regularization consists of adding the standard flow-matching diffusion loss (MSE on velocity prediction) to the RL objective, ensuring the model's outputs stay within the natural video distribution. The paper presents this as an important technique and credits it with preventing reward hacking, but provides no empirical evidence for its necessity or effectiveness.

The missing evidence is an ablation: RL training without the diffusion loss regularization term. Without this ablation, we cannot answer several questions that matter for practitioners who might want to apply this technique to their own models:

• Does the RL phase require regularization to avoid catastrophic quality collapse, or does it merely provide a modest improvement in training stability?
• Would the same RL budget (256 steps, batch size 32) without regularization produce outputs that score poorly on human evaluation (indicating reward hacking) or outputs that are merely noisier but still improved?
• How sensitive is the optimal regularization weight? If the diffusion loss dominates the RL objective, the RL phase reduces to standard fine-tuning with a weak RL perturbation; if the RL objective dominates, reward hacking may still occur.

The consequence. The paper's contribution to the methodology of RL for diffusion models is partially unvalidated. The claim that data regularization is important and effective is supported only by citation (Ye et al., 2025) and the assertion that it "effectively alleviates the reward hacking phenomenon." For a paper that presents RL post-training as a key innovation (Section 1 lists it as one of three "key improvements" over Cosmos-Predict1), the absence of evidence for the specific technique that makes it work is a gap. A practitioner reading Section 4.2.2 cannot determine whether to adopt the regularization technique because they have no comparative data—the paper shows RL with regularization works, but not that it works better than RL without regularization.

A subtler consequence: the human evaluation results (Fig. 5) compare "Before RL" vs. "RL" outputs, but "RL" always means "RL with data regularization." If the regularization is doing most of the work—keeping outputs near the training distribution while the RL gradient provides a weak alignment signal—then the benefit is primarily from the regularization, not the RL, and the method would be better described as "regularized fine-tuning with a RL-derived perturbation" rather than "RL post-training." The paper's framing implies the RL objective is the primary driver of improvement, but the missing ablation means this framing cannot be verified.

What evidence exists in the paper. Tab. 6 shows RL improves VideoAlign rewards, and Fig. 5 shows RL-trained outputs are preferred in human evaluation. These results demonstrate that the full RL pipeline (including regularization) improves quality. The paper does not report any experiment varying the regularization weight, comparing with and without regularization, or measuring any metric that would isolate the regularization's contribution. The statement about regularization effectiveness cites Ye et al. (2025) and asserts it "effectively alleviates the reward hacking phenomenon," but this is an attribution of mechanism without experimental support within this paper.

Mitigation status. The paper does not acknowledge the missing ablation as a limitation. The regularization technique is presented as a known solution from prior work (Ye et al., 2025), implying that ablation is unnecessary because the prior work already established its importance. However, the prior work operated on different models, different data, and possibly different reward models—the effectiveness of regularization is not guaranteed to transfer. The paper would be strengthened by either (a) a minimal ablation (RL with vs. without regularization, at least on a subset of prompts, evaluated by human preference or reward scores), or (b) explicit acknowledgment that the regularization's contribution is assumed from prior work and not independently validated in this setting.

7. Implications and Future Directions

How This Work Changes the Landscape

Cosmos-Predict2.5 shifts the conversation around video generation for embodied AI from a scaling-centric narrative to a curation-and-control narrative. The paper does not merely argue this position—it encodes it in the architecture and training pipeline, and supports it with empirical results that cannot be explained by scale alone: a 2B model matching or exceeding 7B and 27B-MoE models on PAI-Bench (Tab. 10, Tab. 11), a 2B control-net delivering 60% higher cuboid detection AP than a 7B predecessor (Tab. 15), and a robot policy trained with 2B-augmented data achieving 24/30 successes vs. 5/30 from standard augmentation (Tab. 13). These results collectively argue that for Physical AI simulation, what the model is trained on and how it is controlled matters more than how many parameters it has.

This is not a paradigm shift in the sense of introducing a fundamentally new generative modeling framework—flow matching, DiT architectures, and control-net conditioning are all established techniques. Rather, it is a reframing of the problem statement. The dominant approach in video generation has been to treat Physical AI as a downstream application of general-purpose models: pre-train on internet-scale data, then fine-tune or prompt-engineer for robotics or driving tasks. The paper argues that this direction of transfer is backwards—that the requirements of Physical AI (object permanence, action-conditioned control, multi-view geometric consistency, long-horizon stability) are sufficiently different from general video generation that they must be designed into the training pipeline from the start, not bolted on afterward. The evidence for this reframing is distributed: the 4% filtering survival rate (Section 2.1), the domain-specific SFT with model merging (Section 4.2.1), the Cosmos-Reason1 text encoder replacing T5 (Section 3.2), and the RL phase's reward dimensions explicitly targeting motion quality alongside visual quality and text alignment.

A concrete consequence: this work makes it harder to justify training a single, monolithic video model for all purposes. If strict data curation, domain-specific post-training, and task-aligned control conditioning consistently produce better Physical AI results than scaling a general-purpose model, then the field should expect specialized video models for different Physical AI domains—a driving-specific world model, a manipulation-specific world model, a human-dynamics-specific world model—rather than one model to simulate them all. The model merging results (Fig. 4) suggest these specialists can be recombined without catastrophic interference, which provides a practical path toward unified deployment without requiring unified training.

The paper also reconciles a tension in prior work that has been largely implicit rather than debated. General-purpose video generation papers report impressive qualitative results and benchmark scores on aesthetic quality metrics, while robotics and autonomous driving papers report that naively applying those models to generate training data produces physically incoherent outputs. The field has lacked a framework for understanding why this gap exists. This paper provides that framework: general-purpose models are optimized for perceptual quality and text alignment, which correlates poorly with the physical consistency, multi-view coherence, and control adherence that embodied AI requires. By developing metrics that specifically measure these Physical AI properties—the PAI-Bench Domain Score, RNDS for long-horizon error accumulation, cross-camera Sampson error, and cuboid/lane detection metrics on generated driving scenes—the paper gives the field tools to quantify the gap that previously could only be described qualitatively.

The RL post-training results (Tab. 6, Fig. 5) have implications beyond this specific model. Applying GRPO-style reinforcement learning to flow-matching video diffusion—with trajectory decomposition to manage GPU memory and diffusion-loss regularization to prevent reward hacking—is a methodological contribution that other video generation researchers can adopt. If this technique generalizes, we should expect to see a wave of RL-fine-tuned video models optimized for task-specific reward dimensions (physics plausibility, action adherence, viewpoint consistency) rather than trained purely with supervised objectives.

However, the paper does not resolve the fundamental tension between data curation and OOD robustness. The core strategy—aggressively filter training data to keep only physically plausible, domain-relevant examples—improves in-distribution performance but provides no mechanism for graceful degradation on out-of-distribution scenarios. The physics taxonomy (Section 2.2.5) and object permanence SFT (Section 4.2.1) broaden coverage but do not address the core problem: the model can only simulate what it has seen during training, and will silently produce plausible-looking but physically incorrect outputs when operating outside its training distribution. This is not a limitation the paper claims to solve, but it is the natural next frontier that the field must address.

---

Follow-Up Research This Work Enables

Directly quantify the contribution of data filtering strictness to downstream Physical AI task performance. The paper's most distinctive design choice—the 4% filtering survival rate, down from Cosmos-Predict1's 30%—is presented as critical but never ablated. A controlled experiment would train Cosmos-Predict2.5 models at multiple filtering thresholds (e.g., 4%, 10%, 20%, 30%, 50% survival) on otherwise identical data sources, keeping architecture, training budget, and post-training pipeline constant. The key measurement would be PAI-Bench Domain and Quality scores at each threshold, plus downstream task metrics: cuboid detection AP for driving, policy success rate for robotic manipulation using the Section 6.2 protocol, and DreamGen instruction-following scores. If the marginal benefit of filtering diminishes sharply after some threshold (e.g., Domain Score plateaus at 10% survival), the paper's implicit claim that "stricter is better" would need qualification—practitioners could save the compute cost of VLM-based final-pass filtering. If the benefit continues to improve down to 4% and beyond, it validates the paper's curation-first philosophy and suggests even more aggressive filtering might help. A strong follow-up would also track the VLM filtering pass's FLOPs cost to characterize the quality-per-FLOP curve of curation.

Demonstrate closed-loop policy evaluation, not just open-loop video prediction, with Cosmos-Predict2.5 in the loop. The paper claims Cosmos-Predict2.5 enables closed-loop simulation (Section 6.6, abstract), but all demonstrated video generation is open-loop—conditioned on pre-recorded ground-truth actions. A direct test: take the action-conditioned model (Cosmos-Predict2.5-2B/robot/action-cond), deploy it in a simulated control loop with a trained policy, and measure whether task success rate in the simulated loop correlates with real-robot task success rate. Use the same Bridge dataset setup from Section 6.6, train a diffusion policy on real Bridge data, evaluate that policy (a) on the real robot, (b) in open-loop video prediction from recorded actions, and (c) in closed-loop simulation where the model generates frame t, the policy selects action a_t from frame t, and the model generates frame t+1 from frame t and action a_t. The correlation between (a) and (c) success rates—and whether it exceeds the correlation between (a) and (b)—would directly test whether the claimed closed-loop capability is real. A negative result (low correlation, rapid divergence in closed-loop) would be equally valuable, as it would characterize the open-loop-to-closed-loop gap and motivate new architectures or training objectives to close it.

Train and evaluate a Verifier model for detecting physically implausible generated frames. The paper identifies silent physical errors—hallucinated vehicles, misrendered object dynamics—as a fundamental risk for simulation-based Physical AI, but provides no mechanism for detecting these errors automatically. A natural extension: train a binary classifier (or fine-tune a VLM like Cosmos-Reason1) on generated-vs-real frame pairs, with labels indicating whether each generated frame contains a physical implausibility. Training data could be constructed by deliberately introducing physical errors into generated videos (e.g., removing objects between frames, violating object permanence, misplacing dynamic agents in driving scenes) and labeling those frames as "implausible." At inference time, this verifier could run alongside Cosmos-Predict2.5, flagging frames or sequences where physical plausibility drops below a threshold, enabling the system to either regenerate those segments or alert downstream policies that the simulation is unreliable. The evaluation would measure (a) verifier accuracy on held-out physical errors, (b) whether verifier scores correlate with downstream policy failure rates (do policies fail more often on generated videos the verifier flags?), and (c) whether using the verifier to filter training data (discarding implausible synthetic episodes) improves policy performance compared to training on unfiltered synthetic data.

Test the model merging approach across fundamentally different embodiments, not just visual domains within the same embodiment class. The paper's domain-specific SFT + model soup merging (Section 4.2.1) is demonstrated on five visual domains (object permanence, high motion, complex scenes, driving, robotic manipulation) but all within the same underlying architecture and base model. An ambitious extension: fine-tune separate Cosmos-Predict2.5 models on manipulation data from different robot embodiments (e.g., single-arm Franka, bimanual Kinova, mobile manipulator, quadcopter with gripper), then merge them and evaluate whether the merged model can generate plausible videos for all embodiments—including combinations not seen during any individual fine-tuning run (e.g., a Franka arm in a Kinova environment). The key metric would be the merged model's per-embodiment PAI-Bench Domain Score compared to (a) the embodiment-specific SFT model and (b) the pre-trained base model. If merging preserves embodiment-specific knowledge (Domain Score close to the specialist) while generalizing to novel embodiment-scene combinations, it would validate model merging as a scalable strategy for multi-embodiment Physical AI simulation without requiring joint training on all combinations. A negative result (merging degrades per-embodiment performance or fails on novel combinations) would suggest that embodiment-specific representations interfere in weight space, favoring alternative approaches like mixture-of-experts or embodiment-conditioned routing.

Quantify the sensitivity of the entire pipeline—from data curation through post-training—to domain shift in the training data distribution. The paper's approach is data-centric: every stage from pre-training data filtering (Section 2.1) through domain-specific SFT (Section 4.2.1) through RL alignment (Section 4.2.2) depends on the specific composition and quality of the training data. A systematic stress test would shift the training data distribution along controlled axes and measure the impact on downstream Physical AI capabilities. For example: (a) replace 10%, 25%, or 50% of the curated driving data with internet-scraped driving videos that pass only basic quality filters but not the domain-specific curation pipeline, and measure the impact on cuboid detection AP (Tab. 15) and cross-camera Sampson error (Tab. 14); (b) remove the physics domain data entirely (Section 2.2.5) and measure the impact on DreamGen object-permanence scores; (c) replace Cosmos-Reason1 text embeddings with T5 embeddings while keeping all other factors constant, and measure the DreamGen instruction-following gap. This stress test would characterize how robust the pipeline's advantages are to imperfect curation—a practical concern for any organization that cannot replicate NVIDIA's proprietary data collection infrastructure.

Apply the RL post-training pipeline to a general-purpose video model (e.g., Wan2.1-14B) with the same Physical AI reward dimensions, to test whether the curation advantage can be recovered through alignment. The paper shows that Cosmos-Predict2.5-14B with RL outperforms Wan2.1-14B in human evaluation (48.6% vs. 31.8% win rate, Fig. 7), but Cosmos-Predict2.5 differs from Wan2.1 in architecture, training data, pre-training recipe, and text encoder—the factors are fully confounded. A targeted experiment would take the publicly released Wan2.1-14B checkpoint and apply exactly the same domain-specific SFT (on Cosmos's curated domain data) and RL pipeline (with VideoAlign reward dimensions and data regularization) that Cosmos-Predict2.5-14B receives. Compare the resulting model to Cosmos-Predict2.5-14B on PAI-Bench Domain and Quality scores, DreamGen instruction following, and human evaluation. If the post-trained Wan model achieves comparable or better performance, it suggests the curation and alignment pipeline is the primary differentiator and the base architecture differences (flow-matching vs. EDM, Cosmos-Reason1 vs. T5) are secondary. If the Cosmos model still substantially outperforms, it confirms that architecture choices and pre-training data curation have non-recoverable effects that post-training cannot compensate for. Either outcome would clarify which of the paper's many design decisions actually matter for downstream Physical AI performance.

---

Practical Applications and Downstream Use Cases

Visual data augmentation for robot policy learning under domain shift. The Section 6.2 results provide the most concrete deployment scenario: a practitioner with 100 teleoperated demonstrations of a manipulation task can use Cosmos-Transfer2.5-2B to generate 500 augmented training episodes (5 per original demonstration) covering diverse object appearances, lighting conditions, and background changes, then train a diffusion policy on the combined real + synthetic dataset. The paper's specific numbers—24/30 successes on out-of-distribution test scenarios vs. 5/30 with standard image augmentation (Tab. 13)—provide a quantitative anchor for expected improvement, though the single-task, single-robot evaluation means the absolute success rate should be treated as indicative rather than guaranteed. The augmentation prompt template (Section 6.2.2) provides a directly reusable recipe: caption the scene with a VLM, mark variable components, use an LLM to generate candidate variations, and use Cosmos-Transfer2.5 with global edge control (to preserve scene structure) and robot-localized blur control (to preserve task semantics) to render the variations. The key deployment consideration is that this approach is most effective when the domain shift is primarily visual (appearance changes, lighting, backgrounds) rather than kinematic (different robot kinematics, different object geometries)—the edge-control conditioning preserves underlying structure, which is valuable for appearance generalization but limiting for embodiment transfer.

Driving scenario generation from HD maps for perception model training and validation. The Cosmos-Transfer2.5-2B/auto/multiview model, conditioned on world scenario maps (HD map elements + color-coded dynamic object bounding boxes), can generate multi-view driving videos that approach real-video fidelity on lane detection (F1 0.637 generated vs. 0.637 real, Tab. 15) and within 83% of real-video performance on cuboid detection (LET-AP 0.394 vs. 0.476). This makes the model viable as a data engine for training or augmenting perception models (3D object detectors, lane detectors, occupancy networks) in scenarios where real data collection is expensive, dangerous, or requires coverage of rare events. The concrete workflow: take an HD map of a target geographic region, specify traffic scenarios (agent trajectories, densities, behaviors) as world scenario map sequences, generate multi-view video with Cosmos-Transfer2.5, extract 3D bounding box and lane ground truth from the control inputs, and use the synthetic paired data to train or fine-tune perception models. The detection metrics in Tab. 15 quantify the sim-to-real gap that remains, enabling practitioners to estimate whether that gap is acceptable for their application or whether domain adaptation techniques are needed. The RNDS curves (Fig. 10) provide evidence that the generation remains stable for at least 100 seconds of video, which is sufficient for typical traffic scenario durations, though the paper does not test at longer horizons.

Synthetic VLA training data generation for instruction-following tasks. The DreamGen benchmark results (Tab. 18) demonstrate that Cosmos-Predict2.5-14B/robot/gr00tdream-gr1 achieves 91.8% (GPT) / 69.4% (Qwen) instruction-following accuracy on object generalization tasks—meaning it can generate videos of robots correctly manipulating unseen objects specified in text prompts. When combined with an inverse-dynamics model to extract pseudo-action sequences from generated videos (the paradigm described in Section 6.5), this enables generation of synthetic (video, language instruction, action) triplets for training vision-language-action models. The specific advantage over general-purpose models: Cosmos-Predict2.5-14B's 91.8% object-generalization score is ~20 percentage points higher than WAN2.1's 72.0% (GPT evaluation) and ~7 points higher than CognVideox's 72.0%. For a practitioner building a VLA training pipeline, this means fewer generated episodes will contain physically incorrect or instruction-violating behavior, reducing the need for manual filtering or the risk of training on corrupted data. The 4-step distilled model (Tab. 7, Tab. 8) maintains generation quality with 5× faster inference, making large-scale synthetic data generation practical—a key consideration when generating the millions of episodes that VLA training typically requires.