MoRight: Motion Control Done Right

ArXiv: 2604.07348

🎯 Pitch

Motion-controlled video generators treat all movement as pixel dragging, but MoRight can predict physical consequences from a single action or figure out what action caused a desired outcome—all while you orbit the camera freely. It’s the first model to disentangle camera and object motion while learning causal action–reaction relationships from video data.

---

1. Executive Summary

MoRight introduces a unified video generation framework that achieves disentangled camera-object motion control (specifying object trajectories in a canonical static view and transferring them to arbitrary target camera viewpoints via temporal cross-view attention) and motion causality reasoning (decomposing motion into active user-driven actions and passive consequence components, then training with motion dropout to internalize action–reaction relationships), enabling both forward reasoning—predicting scene outcomes from user actions—and inverse reasoning—recovering plausible driving actions from desired outcomes—all while freely adjusting the camera viewpoint. Evaluated on DynPose-100K, WISA, and a Cooking benchmark using a Wan2.1-14B backbone, MoRight achieves state-of-the-art performance in generation quality and motion controllability—matching or exceeding baselines that rely on privileged future-frame 3D trajectories while using only first-frame reprojected sparse tracks—and is preferred in human evaluations 53.5–55.9% of the time across controllability, motion realism, and photorealism, establishing that explicit causal decomposition and view-disentangled motion representations enable interaction-aware video generation without requiring per-frame dense tracking supervision or external physics simulators.

2. Context and Motivation

The Core Problem: Two Entangled Limitations in Motion-Controlled Video Generation

The fundamental gap this paper addresses is that existing controllable video generation methods lack two capabilities simultaneously: disentangled camera-object motion control and motion causality reasoning. While these are distinct technical challenges, the paper argues they are practically inseparable—a video generation system that offers only one of them fundamentally limits how users can interact with and reason about dynamic scenes.

The paper frames this around how humans naturally interact with the physical world (Section 1):

"Humans interact with the physical world as active agents: we move our viewpoint, manipulate objects, and reason about how actions lead to consequences. Yet existing video generation models lack this unified capability."

A desirable system would let a user (1) specify that a hand pushes a cup while simultaneously orbiting the camera around the scene, and (2) have the model automatically generate the cup sliding forward and potentially colliding with other objects, without the user needing to specify the cup's trajectory pixel-by-pixel. Existing methods cannot do both. They either entangle camera and object motion into a single control signal that breaks when the viewpoint changes, or they treat user-specified motion as simple kinematic displacement—pixels being dragged from one position to another—without reasoning about what should happen to other objects in the scene as a consequence.

Why This Problem Matters

The paper identifies three downstream application domains where these limitations are critical (Section 1):

Embodied AI agents. Robots and AI agents operating in physical environments must predict how their actions will affect the world—what the paper calls "forward reasoning"—and must also infer what actions produced observed outcomes, which is essential for learning from demonstration and planning. Current video generation models, if used as world simulators for training embodied agents, would hallucinate physically implausible consequences because they lack causal understanding of object interactions. The paper cites work on navigation world models (Bar et al., 2025), interactive world simulators (Yang et al., 2023), and autonomous driving world models (Wang et al., 2024) as application areas where this capability is essential.

Immersive content creation. In applications like 3D scene generation from images (WonderPlay, PhysGen3D), users want to freely navigate around a scene (changing the camera viewpoint) while also manipulating objects within it. Existing pipelines treat camera control and object manipulation as separate stages, requiring users to work through awkward multi-step workflows. A unified system that lets users draw object motion trajectories directly on the image plane while independently specifying camera trajectories would be dramatically more intuitive for creative workflows.

World models for physical simulation. The paper connects to video generation models as world simulators (Brooks et al., 2024; Sora-style models). For these to serve as useful simulators, they need to faithfully reproduce causal physical interactions—pushing causes sliding, lifting causes objects to rise, collisions cause scattering. Without explicit modeling of action–consequence relationships, these simulators produce visually plausible but physically incoherent outputs, limiting their utility for downstream tasks like model-based planning or physics-informed content creation.

The theoretical significance is equally important: these two limitations represent a gap between how existing methods represent motion (as pixel displacements) and how motion actually occurs in the physical world (as causally connected events involving multiple interacting objects, observed from a freely moving viewpoint). Bridging this gap requires rethinking what it means to "control" video generation, moving from a rendering paradigm (given all pixel trajectories, reproduce them faithfully) toward a reasoning paradigm (given sparse user intent, generate physically consistent dynamics).

Where Existing Approaches Fall Short

The paper identifies four categories of prior work, each with specific limitations (Section 2):

Trajectory-Based Motion Control: The Entanglement Problem

The dominant paradigm for motion-controlled video generation—exemplified by Motion Prompting (Geng et al., 2024), ATI (Wang et al., 2025), and WanMove (Chu et al., 2025)—treats motion as per-pixel trajectories defined in pixel space. The user (or an upstream model) specifies where every pixel should move over time, and the video model generates frames that respect those trajectories.

The paper identifies a fundamental limitation that is inherent to this pixel-space representation:

"Because trajectories are defined in pixel space, they inevitably entangle object and camera motion: any viewpoint change alters all trajectories, making joint control ill-posed without explicit foreground–background decomposition."

To understand why this is a problem, consider a concrete example. Suppose you want to generate a video of a cup sliding rightward across a table, while the camera simultaneously orbits leftward. In the resulting 2D video, the cup's pixel trajectory is the sum of its physical motion (rightward) and the camera motion (which shifts everything leftward). If you provide the model with the cup's 2D pixel trajectory that combines both effects, there is no way for the model to know which part of the trajectory comes from physical motion versus camera motion—the control signal is ambiguous. Existing methods attempt to work around this by providing separate foreground and background trajectory signals (so the model sees that foreground pixels move differently from background pixels), but this requires either manual foreground–background segmentation or privileged 3D information (per-frame depth, 3D object tracks) that is not available in image-to-video settings where only a single reference frame exists.

The paper notes that these methods "work as renderers: given displacements for all pixels, they generate a visually realistic video that adheres to the displacements." They treat motion as a rendering target rather than as a phenomenon to be understood and completed.

Camera–Object Motion Disentanglement Methods: The Privileged Information Assumption

A separate line of work (Section 2.2) explicitly attempts to decouple camera motion from object motion. Methods like Perception-as-Control (Chen et al., 2025), Diffusion as Shader (Gu et al., 2025), and Motion-I2V (Shi et al., 2024) treat camera and object motion as independent control signals.

However, the paper identifies a critical dependency shared across these approaches:

"They typically rely on privileged information such as per-frame depth, 3D object trajectories, or foreground–background segmentation masks, and pre-warp all signals to their anticipated future locations."

The "pre-warping" step is the key mechanism and the key limitation. To separate camera from object motion, these methods need to know—frame by frame—what the scene's 3D structure looks like (depth), which pixels belong to foreground vs. background, and where objects are in 3D space throughout the video. With this information, they can computationally "undo" the camera motion by warping each pixel according to the known camera transform, isolating the residual motion as object movement. The problem is that this information requires the full video sequence to be known in advance—these are effectively video-to-video or multi-view methods, not single-image-to-video methods. For the practical use case of "I have one photo of a scene and I want to generate a video where I move the camera and move an object," depth at future frames and 3D object tracks at future frames simply do not exist. The paper argues this assumption "severely limits applicability in image-to-video settings."

Physics-Engine-Based Causal Reasoning: The Generality Problem

One approach to modeling interaction causality (Section 2.3) is to incorporate external physics simulators into the video generation pipeline. Methods like PhysGen (Liu et al., 2024), PhysGen3D (Chen et al., 2025), and MotionCraft (Montanaro et al., 2024) use physics engines to simulate rigid-body dynamics, fluid flow, or collision response, and condition the video generator on the simulation output.

The limitation is scope: these methods are "tailored to particular interaction types"—rigid bodies, fluids, specific force interactions—and require a simulation module in the loop that must be configured for each new scene. A video of a hand kneading dough, a ball bouncing on grass, and water pouring from a teapot would require three different physics solvers (deformable body, rigid body with friction, fluid dynamics). This limits generality and adds engineering complexity, making these approaches impractical for general-purpose video generation systems that need to handle arbitrary interactions in arbitrary scenes.

LLM-and-VLM-Based Causal Reasoning: The Error Propagation Problem

Another family of methods (Section 2.3) delegates causal reasoning to language models. Approaches like LLM-Grounded Video Diffusion (Lian et al., 2023) and GPT4Motion (Lv et al., 2024) use large language models or vision-language models to first predict interaction outcomes in text ("the cup will slide to the right and stop at the edge of the table"), then translate these text predictions into intermediate visual representations—optical flow fields, edge maps, or depth maps—that finally condition the video generator.

The paper identifies a cascading failure mode: "This two-stage pipeline is prone to error propagation: imprecise textual predictions are further degraded during cross-modal conversion, yielding spatially inaccurate dynamics." A language model might correctly predict that "the cup slides rightward," but translating that to a precise pixel-level flow field that respects the cup's exact shape, the table's surface geometry, and occlusion boundaries introduces errors at the text→geometry stage. These errors then propagate into the video generator, producing outputs where the cup slides in approximately the right direction but penetrates the table, changes shape, or interacts incorrectly with nearby objects. The spatial precision required for physically plausible interaction is lost in the language bottleneck.

How MoRight Positions Itself Relative to Existing Work

MoRight's position is that the two limitations—entangled camera-object control and absent causal reasoning—are not independent problems requiring separate solutions, but rather symptoms of a deeper representational issue: existing methods represent motion in ways that are fundamentally not aligned with how motion works in the physical world.

For the disentanglement problem, the paper's key insight is that object motion is "intrinsically unambiguous when expressed in a canonical camera" (Section 3.1). If a user specifies that an object moves 50 pixels to the right in the source image plane (where the camera is fixed), that trajectory has a clear physical meaning. The camera motion can then be handled separately as a viewpoint transformation. The paper's dual-stream architecture—one stream for canonical object motion, one stream for target camera motion, sharing information through cross-view attention—is the mechanism that operationalizes this insight without requiring any of the privileged information (depth, 3D tracks, segmentation masks) that prior disentanglement methods depend on. The canonical stream becomes a "virtual anchor" for motion, transferring dynamics to arbitrary viewpoints through learned attention rather than through explicit geometric warping.

This positions MoRight as more practical and general than prior disentanglement methods: it works from a single image, requires no explicit 3D reconstruction, and handles arbitrary camera trajectories without pre-computed scene decomposition. The trade-off is that the model must learn the geometric correspondence implicitly through training data rather than through explicit geometry—a bet that modern large-scale video diffusion models have sufficient capacity to internalize this mapping.

For the causality problem, the paper's key insight is that causal relationships can be learned from data without external simulators or language-model intermediaries, by decomposing motion into active and passive components during training and using asymmetric supervision (motion dropout) to force the model to predict the missing component. This is a fundamentally different approach from physics engines (which hard-code physical laws) and LLM pipelines (which reason in language space). MoRight learns pixel-level causal structure from raw video data, operating entirely in the visual domain.

The positioning is that this approach combines the generality of learning-based methods (it can handle any interaction type present in the training data) with the precision of pixel-level control (it reasons about motion in the same space where it generates). The active/passive decomposition—identified automatically using a VLM during data curation—provides a weak supervision signal that is sufficient for the model to internalize causal relationships without requiring any domain-specific engineering or hand-crafted physics models.

Critically, the paper does not position MoRight as replacing trajectory-based control methods, but as operating with fundamentally weaker, more practical inputs. While Motion Prompting, ATI, and WanMove require dense per-pixel trajectories for all objects across all frames (including both foreground and background, and both active and passive motion), MoRight requires only sparse trajectories on the first frame for either the active or passive component alone. The model handles the rest—viewpoint transformation, passive consequence prediction, or active cause inference—through learned reasoning. This is a substantial reduction in the input burden placed on users, and the paper's experiments are designed to test whether this weaker supervision can still produce competitive or superior results compared to methods that receive privileged future-frame information.

3. Technical Approach

3.1 Reader Orientation

MoRight is a unified video generation system that, given a single image, produces a video where objects move according to user-specified trajectories and the scene is rendered from user-specified camera viewpoints, while automatically reasoning about the causal consequences of those actions on other objects in the scene. The paper is primarily an architectural and training methodology paper whose core idea is that video generation models should represent object motion in a canonical static reference frame (separating it from camera motion) and should learn action–consequence relationships from data by being forced to predict missing motion components during training.

3.2 Big-Picture Architecture

The system has five major components, organized around a dual-stream video diffusion model:

1. Input conditioning pipeline: The user provides (a) a single reference image, (b) sparse object motion trajectories drawn on that image's plane (the canonical view), and (c) a sequence of target camera poses. A VLM decomposes objects into active and passive categories during data curation; a tracker extracts dense trajectories; a depth estimator and camera pose estimator provide geometric information.

2. Dual-stream DiT backbone: A shared-weight diffusion transformer processes two latent video streams simultaneously—one representing the canonical static-view video (object motion only) and one representing the target-view video (object motion plus camera motion). The streams exchange information through cross-view self-attention at every transformer block, enabling motion learned in the canonical stream to transfer to arbitrary camera viewpoints.

3. Condition encoders: A lightweight trajectory encoder converts sparse pixel tracks into per-pixel spatiotemporal embeddings; a camera encoder warps the first frame through the target camera trajectory, encodes the warped frames via a VAE, and produces camera-condition latents. Both are injected additively into the DiT features at every block.

4. Motion causality mechanism: During training, object motion is decomposed into active (agent-driven) and passive (consequence) components. A random dropout masks one component from the input conditioning, forcing the model to predict the missing dynamics from partial motion information. During inference, this enables forward reasoning (drop passive, predict consequences) and inverse reasoning (drop active, predict causes).

5. Training data curation pipeline: A multi-stage process extracts camera poses, depth maps, dense tracks, and active/passive segmentation from raw video data using foundation models (ViPE, AllTracker, Qwen3-VL, SAM2). Paired multi-view training videos are synthesized via camera-controlled video-to-video generation applied to static-camera source videos, supplemented with single-view real-world data and rendered graphics data.

Information flows as follows: a single image and user-drawn sparse trajectories enter the system → the trajectory encoder converts sparse tracks to a dense trajectory map → the camera encoder generates warped-frame latents along the target camera path → both are injected into the canonical and target streams of the dual-stream DiT → the DiT jointly denoises both latent videos, with cross-view attention transferring motion information from the canonical anchor to the target view → the target-stream latent is decoded into the final output video.

3.3 Roadmap for the Deep Dive

• First, the dual-stream formulation (Section 3.1), which establishes the mathematical and architectural framework for disentangling camera and object motion—why it works, what each stream contains, how they interact through cross-view attention, and how conditions are injected.
• Second, the motion causality modeling mechanism (Section 3.2), which builds on the dual-stream architecture by decomposing object motion into active and passive components, introducing motion dropout training, and explaining how this enables forward and inverse reasoning without external simulators.
• Third, the data curation pipeline (Section 3.3), which is essential to understanding how the training data is constructed—motion extraction and canonicalization, active/passive decomposition via VLMs, synthetic paired-view generation, and mixed-training strategies that leverage real-world single-view data.
• Fourth, the training and inference procedures (Section 3.4), covering the flow matching objective, multi-granularity trajectory dropout strategies, occlusion augmentation, and how users interact with the system at test time.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an architectural and training methodology paper whose core idea is that disentangling camera–object motion through a canonical static-view anchor and decomposing motion into active/passive components with asymmetric supervision together enable interaction-aware video generation that requires substantially weaker inputs than prior work while producing physically more coherent outputs.

---

Disentangled Camera-Object Motion Control

The paper's central architectural innovation is a dual-stream latent video diffusion model that processes two videos simultaneously: one in a canonical static-camera view (where object motion is controlled directly) and one in the target user-specified camera view (where the camera may move arbitrarily). The streams share all DiT weights and exchange information through self-attention, allowing the model to learn a geometric correspondence between the two views implicitly, without requiring explicit 3D reconstruction or depth-based warping.

Why this approach over alternatives. Prior methods represent motion as per-pixel trajectories in the target video's pixel space. When a user specifies that an object moves rightward and the camera orbits leftward, the resulting 2D trajectory is a vector sum of two independent motions, providing an ambiguous control signal—the model cannot determine which portion of the displacement comes from physical object motion versus camera motion. The paper's key insight is that this ambiguity disappears when object motion is specified in a canonical reference frame. If the user draws trajectories on the first frame of the video (where the camera position is, by definition, the identity pose), those trajectories have a clear physical meaning independent of whatever camera motion will be applied later. The dual-stream architecture provides a mechanism for transferring this canonical motion into arbitrary target viewpoints without requiring the model to deduce object motion from entangled 2D observations.

Preliminaries: latent video diffusion with flow matching. The generative backbone builds on a DiT-based (Peebles & Xie, 2023) latent video diffusion framework. A pretrained VAE encoder, denoted $\mathcal{E}$, compresses the video $\mathbf{x} \in \mathbb{R}^{T \times H \times W \times 3}$ into a lower-dimensional latent space:

$\mathbf{z}_0 = \mathcal{E}(\mathbf{x}) \in \mathbb{R}^{\hat{T} \times \hat{H} \times \hat{W} \times d}$

where $\mathbf{z}_0$ is the clean latent representing the video, $\hat{T}, \hat{H}, \hat{W}$ are the temporally and spatially downsampled latent dimensions (with temporal downsampling by $4\times$ to match the Wan latent resolution), and $d$ is the latent channel dimension (5120 for the Wan2.1-14B backbone).

The diffusion process uses flow matching (Lipman et al., 2022) rather than the more common DDPM formulation. In flow matching, the forward process interpolates linearly between the clean latent and Gaussian noise:

$\mathbf{z}_t = (1 - t)\mathbf{z}_0 + t\boldsymbol{\epsilon}, \quad \boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$

where $t \in [0, 1]$ is the continuous time parameter (0 = clean, 1 = pure noise), $\mathbf{z}_t$ is the noisy latent at time $t$, and $\boldsymbol{\epsilon}$ is a sample from a standard Gaussian distribution.

The DiT $\mathcal{G}_\theta$ is trained to regress the velocity field—the direction from noise toward data—rather than the noise itself:

$\mathcal{L} = \mathbb{E}_{\mathbf{z}_0, t, \boldsymbol{\epsilon}}\left[ \|\mathcal{G}_\theta(\mathbf{z}_t, t, \mathbf{c}) - (\boldsymbol{\epsilon} - \mathbf{z}_0)\|^2 \right]$

where $\mathbf{c}$ denotes conditioning signals (text, camera poses, motion tracks), $\mathcal{G}_\theta(\mathbf{z}_t, t, \mathbf{c})$ is the model's predicted velocity at time $t$ given the noisy latent and conditions, and $(\boldsymbol{\epsilon} - \mathbf{z}_0)$ is the target velocity—the vector pointing from the clean latent to the noise sample. The expectation is taken over clean latents, time steps, and noise samples drawn from the training distribution.

What this loss computes. At each training step, the model sees a noisy latent $\mathbf{z}_t$ that is a $t$-weighted blend of a real video latent and random noise. It must predict the velocity vector $(\boldsymbol{\epsilon} - \mathbf{z}_0)$—the instantaneous direction and magnitude of change that would transport $\mathbf{z}_t$ toward the clean latent $\mathbf{z}_0$ (equivalently, toward the noise $\boldsymbol{\epsilon}$, depending on sign convention). The squared $\ell_2$ distance between the predicted and true velocity vectors is the training loss. At inference, an ODE solver (UniPC, Zhao et al., 2023) integrates the learned velocity field from pure noise ($t=1$) backward to a clean latent ($\hat{\mathbf{z}}_0$), which is then decoded by the VAE decoder $\mathcal{D}$ to produce the output video $\hat{\mathbf{x}} = \mathcal{D}(\hat{\mathbf{z}}_0)$.

Why flow matching over DDPM. The paper does not explicitly motivate the choice, but flow matching offers two practical advantages that are relevant here. First, the linear interpolation path $(1-t)\mathbf{z}_0 + t\boldsymbol{\epsilon}$ is a straight line in latent space, which means the velocity field is constant along each trajectory—this makes the regression target simpler and more stable than predicting noise at different noise levels (where the optimal prediction changes with $t$). Second, flow matching supports deterministic sampling with fewer steps via ODE solvers, which is important because the dual-stream architecture already doubles the per-step computation relative to a single-stream model.

Dual-stream formulation. The core architectural mechanism: instead of generating one video, the model generates two simultaneously, with the two latent videos concatenated along the temporal dimension and processed by the same DiT.

The two streams receive different conditioning signals:

$\mathbf{c}^{\text{can}} = \{ I, \mathbf{C}_1, \{\boldsymbol{\tau}^{\text{can}}_i\} \}$ $\mathbf{c}^{\text{tar}} = \{ I, \{\mathbf{C}_i\}, \emptyset \}$

where $\mathbf{c}^{\text{can}}$ conditions the canonical stream, $\mathbf{c}^{\text{tar}}$ conditions the target stream, $I$ is the input reference image, $\mathbf{C}_1$ is the identity camera matrix (the first frame's camera, always identity in the canonical view), $\{\boldsymbol{\tau}^{\text{can}}_i\}$ is the set of object motion trajectories defined in the canonical image plane, $\{\mathbf{C}_i\}$ is the sequence of target camera poses for each frame, and $\emptyset$ denotes the empty motion condition—the target stream receives camera poses but no explicit object motion trajectories.

The key asymmetry: the canonical stream knows what moves (via the canonical trajectories) but sees no camera change. The target stream knows where the camera goes but has no direct information about object motion. The cross-view attention layers must transfer object dynamics from the canonical stream to the target stream.

Noise and concatenation detail. Independent noise is sampled for each stream, but at the same timestep $t$:

$\mathbf{z}^{\text{can}}_t = (1-t)\mathbf{z}^{\text{can}}_0 + t\boldsymbol{\epsilon}^{\text{can}}$ $\mathbf{z}^{\text{tar}}_t = (1-t)\mathbf{z}^{\text{tar}}_0 + t\boldsymbol{\epsilon}^{\text{tar}}$

These two noisy latents are concatenated along the temporal dimension to form a single tensor $[\mathbf{z}^{\text{can}}_t; \mathbf{z}^{\text{tar}}_t]$ with twice the original temporal length ($2\hat{T}$ frames). Slightly modified positional embeddings distinguish tokens from the two streams—the paper states they "use different temporal indices" within the RoPE (Rotary Position Embedding) scheme while sharing spatial RoPE embeddings. This temporal-index differentiation is critical: without it, the self-attention layers would not know which tokens belong to the canonical view versus the target view, making cross-view information transfer impossible. The precise modification is described in the supplementary material (Appendix A.1) but not enumerated in the main text.

Condition injection mechanism. Both camera and motion conditions are injected into the DiT at every transformer block through additive modulation. The process has three sub-components: camera encoding, motion encoding, and the injection formula.

Camera encoding. The camera condition is generated by following the approach of Gen3C (Ren et al., 2025). For each frame $i$ in the target camera trajectory, the first-frame image $I$ is warped according to the relative camera pose $\mathbf{C}_i$ (the transformation from the first frame's camera to frame $i$'s camera) and an estimated depth map. The paper uses Depth Anything 3 (Lin et al., 2025) for monocular depth estimation. Each warped frame is then encoded through the same VAE encoder $\mathcal{E}$ used for video encoding, producing a camera-condition latent:

$\mathbf{z}^{\text{cam}} \in \mathbb{R}^{\hat{T} \times \hat{H} \times \hat{W} \times d}$

For the canonical stream, this warping uses the identity matrix for all frames—no camera motion—so the warped frames are simply the first frame repeated. This provides the canonical stream with a static-background signal that anchors its object motion representations.

Motion encoding. Following Motion Prompting (Geng et al., 2024), the object trajectories are encoded into a per-pixel trajectory map. Each pixel $(u, v)$ that lies on a trajectory is assigned a temporal-correspondence embedding that identifies which trajectory it belongs to across frames—essentially, pixels that correspond to the same physical point across time share an embedding vector. Pixels not on any trajectory receive a zero embedding. This trajectory map is then processed by a lightweight temporal encoder consisting of RMSNorm, SiLU activation, and two $3 \times 1 \times 1$ Conv3D layers that downsample the temporal dimension by $4\times$ to match the Wan latent temporal resolution. The output is:

$\mathbf{e}^{\text{trk}} \in \mathbb{R}^{\hat{T} \times \hat{H} \times \hat{W} \times d}$

The trajectory embedding dimension is 64 (input to the encoder), and the camera encoder uses 32 channels. After projection to the Wan hidden dimension, both reach $d = 5120$. For the target stream, the motion encoding is simply set to zero since the target stream receives no explicit object motion condition.

Injection formula. At each DiT block $i$, the latent feature $\mathbf{f}^i$ is augmented by learned linear projections of the camera and trajectory encodings:

$\mathbf{f}^i \leftarrow \mathbf{f}^i + \mathbf{W}_{\text{cam}} \mathbf{z}^{i,\text{cam}} + \mathbf{W}_{\text{trk}} \mathbf{e}^{i,\text{trk}}, \quad i \in \{\text{can}, \text{tar}\}$

where $\mathbf{W}_{\text{cam}}$ and $\mathbf{W}_{\text{trk}}$ are learned linear projection matrices (mapping from the encoding dimension to the DiT hidden dimension 5120), $\mathbf{z}^{i,\text{cam}}$ is the camera encoding for stream $i$, and $\mathbf{e}^{i,\text{trk}}$ is the trajectory encoding for stream $i$ (zero for the target stream). The $\leftarrow$ notation indicates that the injection modifies the feature in place before self-attention.

What this injection achieves. Before self-attention at each block, every token in the canonical stream carries explicit information about (a) what the static scene looks like from the canonical viewpoint (via the warped-identity camera encoding) and (b) which pixels belong to moving objects and where they should move (via the trajectory encoding). Tokens in the target stream carry information about the scene's appearance under the target camera (via the warped-target camera encoding) but no explicit motion information. The self-attention layer that follows must therefore use the canonical stream's motion-rich tokens to inform the target stream's tokens, learning an implicit geometric correspondence between views.

Cross-view self-attention. After condition injection, the features from both streams are concatenated and passed through a standard multi-head self-attention layer:

$[\mathbf{f}^{\text{can}}; \mathbf{f}^{\text{tar}}] := \text{SelfAttn}([\mathbf{f}^{\text{can}}; \mathbf{f}^{\text{tar}}])$

The $:=$ notation indicates that the output overwrites the concatenated features. This operates as a single attention operation over the union of canonical and target tokens—every token can attend to every other token, including across streams. Target-stream tokens can query canonical-stream tokens to learn where objects are moving, and canonical-stream tokens can query target-stream tokens to understand the viewing transformation.

Why this works (the implicit geometric reasoning hypothesis). The paper does not provide a formal proof of the geometric correspondence learned by cross-view attention, but the mechanism is best understood through an information-flow argument. The canonical stream tokens encode "this pixel (or patch) at this position is part of an object that follows trajectory $\boldsymbol{\tau}$." The target stream tokens encode "this pixel at this position corresponds to scene point $\mathbf{P}$ viewed from camera pose $\mathbf{C}$." Through self-attention, a target token can compute a weighted sum of canonical tokens. If the attention weights reflect geometric correspondence—i.e., a target token attends most strongly to the canonical token representing the same physical scene point—then the target token can reconstruct the full motion trajectory of its corresponding point by aggregating trajectory information from the canonical stream. The training objective (predicting the full target video, which includes correct object motion) provides the supervisory signal that shapes these attention weights into geometrically meaningful correspondences.

This is a learned approximation to explicit epipolar geometry: rather than computing $\mathbf{F}$ matrices and performing triangulation, the model learns to route information across views through a high-dimensional attention mechanism that generalizes across scene geometries and camera trajectories present in the training data. The dual-stream design with weight sharing ensures that the geometric reasoning capacity is built into every layer of the DiT, not relegated to a separate warp-and-inpaint stage.

Inference behavior. At test time, both streams are jointly denoised. The canonical stream serves as a "virtual anchor"—it never appears in the output—while the target stream's decoded latent $\hat{\mathbf{x}} = \mathcal{D}(\hat{\mathbf{z}}^{\text{tar}}_0)$ is presented to the user. The user never sees the canonical video; it exists solely to provide motion information to the target stream through cross-view attention. This means the canonical stream's generation quality (artifacts, texture fidelity) does not directly affect the output, though poor canonical-stream generation could degrade target-stream quality through attention-based information transfer.

Training data assumptions for disentanglement. The dual-stream formulation requires paired training data: one video with object motion only (canonical view) and a corresponding video with the same object motion plus camera motion (target view). Real-world data rarely provides such pairs. Section 3.3 and Section 3.4 detail how the paper creates this training data through synthetic camera-controlled video-to-video generation and mixed-training strategies that can leverage single-view real-world videos. The central training challenge is that the paired subset is synthetic and limited in diversity, while real-world data is abundant but single-view. The mixed-training approaches described later are designed to bridge this gap.

---

Motion Causality Modeling

The second major technical contribution is a mechanism for learning action–consequence relationships directly from training data, without external physics simulators or language-model intermediaries. The key mechanism is motion dropout—randomly removing one component of object motion from the input conditioning and forcing the model to predict the full video, which induces the model to internalize the causal structure that connects actions to their consequences.

Active/passive decomposition. The paper defines two categories of object motion (Section 3.2, Fig. 3):

• Active motion ($\boldsymbol{\tau}^{\text{act}}_i$): intentional, agent-driven motion representing actions applied to the scene. Examples include a hand pushing, a foot kicking a ball, or a robotic gripper lifting an object. The active entity "moves by its own power or actuation" (Appendix A.2).

• Passive motion ($\boldsymbol{\tau}^{\text{pas}}_i$): consequential motion representing reactions from other objects. Examples include a cup sliding after being pushed, a ball flying after being kicked, or cloth deforming as a hand pulls it. The passive entity "moves only because of other objects or forces."

The total object motion for frame $i$ is the union of these two components:

$\boldsymbol{\tau}_i = \boldsymbol{\tau}^{\text{act}}_i \cup \boldsymbol{\tau}^{\text{pas}}_i$

During data curation (Section 3.3), a VLM (Qwen3-VL) identifies which objects in the video are active versus passive, and a segmentation model (SAM2) produces masks that assign each trajectory to its category. This assignment is done once per training video during the offline data curation stage.

Motion dropout training. The core training mechanism is asymmetric supervision: randomly drop one motion component from the conditioning input, but supervise on the full video containing both components:

$\tilde{\boldsymbol{\tau}}_i = \begin{cases} \boldsymbol{\tau}^{\text{act}}_i, & \xi < p \\ \boldsymbol{\tau}^{\text{pas}}_i, & \text{otherwise} \end{cases}$

where $\xi \sim \mathcal{U}(0, 1)$ is a random variable sampled uniformly, $p$ is the dropout probability (0.8 for active, 0.2 for passive—meaning active motion is provided 80% of the time and passive 20% of the time, as stated in Appendix A.3), and $\tilde{\boldsymbol{\tau}}_i$ is the partial trajectory set used as the conditioning signal to the video model.

What this does operationally. During training, 80% of the time, the model receives only the active motion trajectories (e.g., the hand's movement) and must generate the full video including the passive consequences (e.g., the object being pushed, cloth deforming, water pouring). The remaining 20% of the time, the model receives only the passive motion trajectories (e.g., the cup's sliding path) and must generate the full video including the active cause (e.g., the hand movement that produced that sliding). In both cases, the model is supervised against the ground-truth video containing all motion—both active and passive. The model never receives both active and passive motion simultaneously during training (assuming dropout is always applied), yet it must produce both in its output.

Why asymmetric supervision induces causal reasoning. The model cannot solve this task by simply replaying the provided trajectories—it must generate additional motion that is causally consistent with what was provided. When given a hand trajectory moving toward a cup, the model must generate the cup moving in response because that is what the ground-truth video shows. When given a cup sliding across a table, the model must generate a plausible hand motion (or other cause) because that is what the ground-truth video contains. The model is forced to learn the statistical relationship $\text{pas} \approx f(\text{act})$ that maps active motion to passive motion (and vice versa) because those relationships are consistently present in the training data. The dropout prevents the model from simply memorizing trajectory associations—it must learn to predict consequences from causes, which is the operational definition of causal reasoning in this context.

Asymmetry in dropout probabilities. The bias toward providing active motion (80% vs. 20%) reflects the asymmetric nature of physical causality: most interactions have clear agent-initiated actions (hands, people, vehicles) that cause unambiguous consequences. The "inverse" case—deducing an action from its outcome—is typically underdetermined (many different hand motions could produce the same cup trajectory). Training with heavy bias toward the forward direction ensures the model learns the more reliable forward mapping well while still developing some capacity for inverse inference. At inference, this asymmetry persists: the forward reasoning mode (user provides active motion, model predicts passive) is generally more reliable than inverse reasoning, though the paper demonstrates both capabilities qualitatively.

Inference modes enabled by motion dropout. At test time, the learned causality supports two complementary applications:

• Forward reasoning (action → reaction): The user specifies active motion trajectories (e.g., drawing a path for a hand to move along). The model receives $\tilde{\boldsymbol{\tau}} = \boldsymbol{\tau}^{\text{act}}$ as the motion condition and generates the full video, which includes the passive consequences it has learned to predict from similar active motions in the training data.

• Inverse reasoning (reaction → action): The user specifies desired passive motion trajectories (e.g., drawing the path they want a ball to follow). The model receives $\tilde{\boldsymbol{\tau}} = \boldsymbol{\tau}^{\text{pas}}$ as the motion condition and generates a video that includes a plausible active cause—a hand, foot, or other agent moving in a way that would produce the specified ball trajectory. This is a form of affordance inference: given a desired outcome, what action achieves it?

Critically, the model does not distinguish active from passive motion in its architecture—the same trajectory encoder and condition injection pathway handle both. The distinction exists only in the data annotations and the dropout procedure. The model learns to treat any set of sparse input trajectories as a partial specification of scene dynamics, completing the missing dynamics (whether causal or consequential) based on patterns in the training data. This means the model does not need to be told at inference time whether the provided trajectories are active or passive—it automatically completes whatever is missing.

Text conditioning and information leakage prevention. To prevent the text prompt from leaking information about the dropped motion component, the paper uses Qwen3 to rewrite video captions during data curation, decomposing object motion into active-only or passive-only descriptions. For example, a caption like "A hand pushes a cup, and the cup slides to the right" would be rewritten to "A hand pushes forward" (active-only) or "A cup slides to the right" (passive-only). During training, the original caption and the rewritten caption are randomly sampled with equal probability (Appendix A.2). When the training sample uses active trajectories as motion condition, an active-only caption is used; when it uses passive trajectories, a passive-only caption is used. This ensures the text condition does not "give away" the answer by describing both the action and its consequence.

---

Training Data Curation

Training the dual-stream model with motion causality requires paired videos depicting the same scene dynamics under different camera viewpoints, with active/passive motion annotations. The paper constructs a three-stage data pipeline using foundation models for extraction and a video generation model for synthetic paired-view creation.

Stage 1: Motion extraction and canonicalization. Given a raw video $\mathbf{x}$, the pipeline extracts geometric and motion information using off-the-shelf foundation models:

• Depth estimation: MoGe (Wang et al., 2024) or Depth Anything 3 (Lin et al., 2025) estimates per-frame depth maps $\{D_i\}$.

• Camera pose estimation: ViPE (Huang et al., 2025) estimates per-frame camera poses $\{\mathbf{C}_i\}$ and intrinsic matrix $\mathbf{K}$. ViPE is described as a "video pose engine for 3D geometric perception," providing relative camera motion between frames.

• Dense tracking: AllTracker (Harley et al., 2025) extracts dense pixel trajectories $\mathcal{T} = \{\boldsymbol{\tau}_i\}_{i=1}^T$ across all frames.

Each trajectory is then canonicalized—transformed from the moving-camera pixel space to the first frame's image plane. This uses the estimated depth, camera poses, and intrinsics to unproject 2D points to 3D, transform them by the inverse of the camera motion, and reproject them back to the first frame:

$\boldsymbol{\tau}^{\text{can}}_i = \pi\left( \mathbf{K}, \mathbf{C}_0 \mathbf{C}^{-1}_u \pi^{-1}(\mathbf{K}, \boldsymbol{\tau}_i, D_i) \right)$

where $\pi^{-1}(\mathbf{K}, \boldsymbol{\tau}_i, D_i)$ lifts the 2D trajectory point $\boldsymbol{\tau}_i$ (at frame $i$, specified in pixel coordinates) into a 3D point using depth $D_i$ and intrinsics $\mathbf{K}$; $\mathbf{C}_0 \mathbf{C}^{-1}_u$ transforms this 3D point from frame $i$'s camera coordinate system back to the first frame's camera coordinate system; and $\pi(\mathbf{K}, \cdot)$ projects the resulting 3D point back onto the first frame's image plane. The subscript $u$ on the inverse camera pose appears to index the specific trajectory or frame—the notation is slightly ambiguous in the paper but the intent is clear: undo the camera motion from frame $i$ back to frame 0.

What this canonicalization achieves. After this transformation, the trajectory $\boldsymbol{\tau}^{\text{can}}_i$ represents where the physical 3D point (that was at pixel $\boldsymbol{\tau}_i$ in frame $i$) projects to in the first frame, in pixel coordinates. This isolates object motion from camera motion: any change in $\boldsymbol{\tau}^{\text{can}}_i$ across $i$ represents genuine physical movement of the object, not apparent movement due to the camera moving. This canonicalized trajectory set is what the user will eventually specify at inference time (by drawing trajectories on the first frame), and what the canonical stream of the dual-stream model is conditioned on.

The paper assumes constant intrinsics $\mathbf{K}$ across the video—a reasonable assumption for most consumer video where zoom is rare, though it would break for videos with variable focus or digital zoom.

Stage 2: Active and passive motion decomposition. Given the canonicalized trajectories, the pipeline must categorize each trajectory as belonging to active or passive motion. This is done using a two-step process:

1. Object identification via VLM: Qwen3-VL (Bai et al., 2025) is prompted with a structured query (shown in Fig. 10 of the appendix) to identify all moving objects in the video and classify each as active_dynamic (objects that "move by their own power or actuation" such as people, hands, animals, robots, vehicles) or passive_dynamic (objects that "move only because of other objects or forces"). The prompt includes an explicit rule: "If a person/hand is visible manipulating an item, that item is passive_dynamic."

2. Segmentation via SAM2: The first frame is segmented using SAM2 (Ravi et al., 2024) with the VLM-identified objects as prompts, yielding masks $M^{\text{act}}$ and $M^{\text{pas}}$ for active and passive objects, respectively.

Trajectories are then assigned to the active or passive category by checking whether their starting pixel (in the first frame) falls within the corresponding mask:

$\boldsymbol{\tau}^{\text{can}} = \boldsymbol{\tau}^{\text{can,act}} \cup \boldsymbol{\tau}^{\text{can,pas}}$

Why a VLM for decomposition rather than heuristics. Prior work on physics-based video generation either relies on manual annotation of active/passive objects or restricts evaluation to scenes where the active object is trivially identifiable (e.g., always a visible human hand). The VLM-based approach enables automatic annotation at scale across diverse video sources (Panda-70M, Wild-SDG-1M), though it introduces noise from VLM misclassifications. The paper does not analyze VLM annotation accuracy, but the downstream training results suggest the annotations are sufficiently reliable for the motion dropout mechanism to learn meaningful causal structure.

Stage 3: Paired multi-view synthesis. The canonicalized trajectories and active/passive decomposition provide supervision for the canonical stream. The target stream requires corresponding videos with the same object motion but different camera motion. Since real-world paired multi-view videos of arbitrary dynamic scenes are extremely rare, the paper synthesizes them.

The first step is to identify source videos with nearly static cameras—these provide the canonical-stream supervision. The paper checks camera pose displacement from ViPE and requires a maximum rotation of $0.5^\circ$ and translation of $5\text{ mm}$ (Appendix A.2). From the collected Internet videos (Panda-70M, Wild-SDG-1M), 76K static-camera videos are identified as suitable sources.

From these static-camera videos, the pipeline synthesizes corresponding moving-camera videos using a camera-control video-to-video model (Plenoptic Video Generation, Fu et al., 2026). Given a static-camera video as input and a target camera trajectory as control, this model generates a new video where the camera moves along the specified trajectory while maintaining the original scene dynamics. This produces 43K paired (static-camera, moving-camera) video pairs, each depicting the same object interactions from different viewpoints.

Camera trajectory diversity. To prevent the model from overfitting to a narrow set of camera motions, the paper augments the synthetic data with:

• Basic camera operations: orbit, pan, zoom
• Dynamic camera trajectories extracted from real videos (likely from videos in the training set that already have camera motion, where the trajectory is extracted via ViPE and applied to different source videos)

An additional 3.4K synthetic interaction videos are incorporated from SyncCamMaster (Bai et al., 2025), which provides rendered multi-view data with known camera poses and object interactions.

Single-view real-world data for mixed training. The synthetic paired videos inevitably contain visual artifacts from the video-to-video generation process—blurring, texture degradation, temporal inconsistency. To mitigate this, the paper incorporates abundant single-view real-world data through two mixed-training strategies (Section 3.3):

1. Static-camera videos: For real videos where the camera is static (only object motion present), the video is duplicated and treated as both the canonical and target video. In this case, the canonical and target streams are identical—both show the same static-camera view. The model learns to transfer motion information from the canonical stream (which receives explicit trajectory conditioning) to the target stream (which receives no trajectory condition) when the camera is static. This teaches the cross-view attention to pass motion information through even when no viewpoint change is needed.

2. Dynamic-camera videos: For real videos that exhibit both camera and object motion (no paired canonical view available), only the target stream is supervised—the loss on the canonical stream is set to zero. The canonical stream still receives trajectory conditioning, and the cross-view attention still operates, but the model is not penalized for canonical-stream errors. This exposes the model to diverse real-world camera motions and object interactions without requiring paired data.

Why mixed training matters. The dual-stream architecture's cross-view attention is trained to perform implicit geometric reasoning. If trained only on synthetic paired videos (43K examples from a video-to-video model), the attention mechanism would learn the geometric correspondence patterns present in those synthetic videos—which may include systematic artifacts (e.g., the video-to-video model may produce slightly inaccurate object placements under large camera motions). The mixed-training strategies expose the cross-view attention to real-world camera-object motion correlations (from the dynamic-camera real videos) and teach robust motion transfer even when the canonical and target views are identical (from the static-camera real videos). The paper controls the sampling ratio to ensure single-view data is "sampled at a lower rate to prevent the model from overfitting to single-view motion patterns" (Appendix A.3), though the exact ratio is not specified.

Rendered graphics data. An additional data source is SyncCamMaster (Bai et al., 2025), which provides rendered multi-view videos of synthetic scenes with known camera trajectories, object motions, and interaction dynamics. The 3.4K examples from this source provide clean, geometrically perfect paired supervision that complements the noisy synthetic paired videos from the video-to-video pipeline.

---

Training and Inference

The training procedure combines the flow matching objective with three complementary augmentation strategies that encourage the model to learn motion causality and improve robustness.

Training objective. The model is trained with the flow matching loss from the preliminary formulation, applied to both streams:

$\mathcal{L} = \mathbb{E}_{\mathbf{z}_0, t, \boldsymbol{\epsilon}}\left[ \|\mathcal{G}_\theta(\mathbf{z}_t, t, \mathbf{c}) - (\boldsymbol{\epsilon} - \mathbf{z}_0)\|^2 \right]$

For dual-stream training with paired data, the loss is computed on both the canonical and target streams. For single-view data with dynamic cameras, only the target stream contributes to the loss. For static-camera single-view data, both streams contribute (since both are the same video).

Training hyperparameters (verbatim from Section 4.1 and Appendix A.3). The model is initialized from the pretrained Wan2.1-14B I2V checkpoint. Only the camera encoder, trajectory encoder, and self-attention layers are fine-tuned; all other DiT weights are frozen. Training runs for 15K iterations on 64 GPUs with a global batch size of 16, using AdamW optimizer (Loshchilov & Hutter, 2017) with learning rate $3 \times 10^{-5}$ and weight decay $0.001$. The trajectory dropout rate is $0.1$ and the text-conditioning dropout rate is $0.2$. All training videos are processed at 480p resolution.

Multi-granularity motion dropout. In addition to the active/passive dropout mechanism (Eq. 4), the paper applies a second form of trajectory dropout that varies the spatial granularity of the provided trajectories. Per-pixel dense trajectories are averaged within each image patch (where "patch" refers to the spatial patches of the DiT). During training, the granularity is randomly selected—sometimes providing fine-grained per-pixel trajectories, sometimes providing coarse patch-averaged trajectories. This enables the model to handle both precise pixel-level control (when the user provides detailed trajectories) and coarse object-level manipulation (when the user draws a simple stroke for an entire object).

Occlusion and track dropout. To simulate the imperfect tracking that occurs at inference time (where off-the-shelf trackers may lose points due to occlusion, motion blur, or out-of-frame movement), the paper randomly masks a subset of trajectories during training. Specifically, visible tracks are randomly dropped with probability 0.2. Additionally, tracks are randomly truncated after a sampled middle frame, simulating the scenario where a tracker loses a point mid-video. During training, the number of tracks is randomly sampled between 500 and 2000 per iteration (Appendix A.3). At inference, the number of input tracks is fixed to 1500 for consistent evaluation.

Sampling ratios for data sources. The paper controls the relative frequency of different data sources during training (Appendix A.3):

• Active motion trajectories are provided with probability 0.8, passive with probability 0.2 in the motion dropout.
• Trajectories are simplified to coarse segment-level (all pixels of an object sharing a single trajectory) with probability 0.5.
• Text captions are rewritten to contain only one motion component (active or passive, matching the provided trajectories) with equal probability to the original full caption.
• Single-view data is sampled at a lower rate than multi-view paired data to prevent overfitting to single-view patterns.

Inference pipeline. At test time, the user interaction proceeds as follows (Section 3.4 and Fig. 11):

1. The user provides a single input image and optionally a text prompt.

2. The user specifies object motion by drawing sparse trajectories directly on the first image. The paper's GUI supports two modes: coarse control (drawing a trajectory for an entire selected object region, where all pixels of that object share the same motion path) and fine-grained control (drawing independent trajectories for sparse keypoints, enabling articulated motion). For hand interactions, the GUI provides motion primitives such as push, pull (along a specified direction), and reach (toward a target location).

3. The user specifies target camera poses $\{\mathbf{C}_i\}_{i=1}^T$ independently—these define how the camera moves through the scene. The first frame is treated as the identity pose $\mathbf{C}_1 = \mathbf{I}$.

4. Occlusion-aware masking is performed using the first-frame depth (estimated by MoGe) to approximate visibility ordering. Points that would be occluded from the target camera's viewpoint are handled appropriately, though the exact mechanism is not detailed in the paper.

5. Both streams are jointly denoised with 35 diffusion steps using an ODE solver (UniPC). The target stream's decoded output is presented to the user. Generating one video takes approximately 15 minutes on a single A100 GPU.

Why 35 steps instead of more or fewer. The paper does not provide an ablation on diffusion steps. Based on flow matching's typical behavior (deterministic ODE sampling allows fewer steps than stochastic DDPM sampling), 35 steps represents a moderate quality-speed tradeoff. More steps would improve generation quality at the cost of longer inference, while fewer steps risk introducing artifacts, especially for the cross-view attention which must converge to geometrically consistent correspondences across the two streams during the denoising trajectory.

4. Key Insights and Innovations

Innovation 1: The Canonical-View Anchor as an Architectural Separation Principle for Motion–Camera Disentanglement

The dominant assumption in motion-controlled video generation—implicitly baked into every trajectory-based method from Motion Prompting (Geng et al., 2024) to WanMove (Chu et al., 2025)—is that object motion and camera motion live in the same pixel space and should be handled by the same conditioning mechanism. These methods provide per-pixel trajectories as a unified control signal and expect the video model to sort out which part of the displacement comes from physical motion versus viewpoint change. MoRight makes the conceptual move of recognizing that this sorting-out is inherently ill-posed when the control signal lives in the target camera's pixel space: the same 2D trajectory could be produced by infinitely many combinations of object motion and camera motion, and no amount of model capacity resolves this ambiguity without auxiliary geometric supervision.

The intellectual shift is to treat camera motion as a viewpoint transformation applied to object dynamics, not as a component of them. By defining object motion in a canonical reference frame (the first frame's image plane, where camera pose is identity by definition) and handling camera motion as a separate, parallel stream that transfers dynamics rather than composing with them, the paper reframes the control problem from "disentangle an entangled signal" to "never entangle in the first place." This is not merely a different architecture—it's a different decomposition of the problem that eliminates the ambiguity at the representation level rather than trying to resolve it through more powerful conditioning.

Compare with prior camera–object disentanglement methods like Perception-as-Control (Chen et al., 2025) and Diffusion as Shader (Gu et al., 2025), which do explicitly separate camera and object signals but require per-frame depth, 3D object trajectories, or foreground–background masks to perform geometric warping that undoes the camera motion. These methods assume privileged access to the full 3D structure of the scene at every frame—information that does not exist in the image-to-video setting where only a single reference frame is available. MoRight's innovation is architectural rather than geometric: instead of computing explicit correspondences via depth-based warping, it learns implicit geometric correspondences through cross-view attention, using the dual-stream formulation as a training mechanism that forces the model to internalize the mapping between canonical and target views.

This is a fundamental conceptual shift from explicit 3D reconstruction to learned geometric reasoning. The canonical stream serves as a "virtual anchor"—a video that exists only during generation, never seen by the user, whose sole purpose is to provide geometrically clean motion information that the target stream can query through attention. The fact that this works (as demonstrated by motion control accuracy in Table 1, where MoRight achieves the best EPE on DynPose-100K and outperforms all baselines on Cooking despite using only first-frame reprojected trajectories) validates the hypothesis that a sufficiently expressive attention mechanism can learn epipolar geometry implicitly from paired multi-view training data, without requiring knowledge of F matrices or triangulation at inference time.

The significance extends beyond practical performance: it establishes the canonical-view anchor as a general design pattern for controllable generation tasks where the control signal and the rendering viewpoint are naturally separable. Any problem where a user wants to manipulate content and independently control viewpoint—3D scene editing, novel view synthesis of dynamic scenes, interactive world models—can adopt this pattern: define control in a canonical space, transfer to target viewpoints through cross-attention, use synthetic paired data for supervision, and supplement with real single-view data through mixed training.

Innovation 2: Motion Dropout as a Self-Supervised Causal Reasoning Objective for Video Generation

Prior work on causal reasoning in video generation falls into two camps, both of which MoRight departs from. The first camp—exemplified by PhysGen (Liu et al., 2024), PhysGen3D (Chen et al., 2025), and MotionCraft (Montanaro et al., 2024)—incorporates external physics engines that hard-code physical laws (rigid-body dynamics, fluid simulation, collision response). These methods know what they know: gravity makes things fall, collisions transfer momentum, fluids deform under pressure. But their scope is limited to the physics they explicitly model, and they require engineering a simulation module for each new scene type. The second camp—exemplified by GPT4Motion (Lv et al., 2024) and LLM-Grounded Video Diffusion (Lian et al., 2023)—delegates causal reasoning to language models that predict outcomes in text, then translates those text predictions to visual conditioning signals. These methods suffer from a precision bottleneck: "the cup slides rightward" is a correct but imprecise description of exactly where the cup's pixels should be in each frame, and the text→geometry conversion introduces errors that propagate through the pipeline.

MoRight's innovation is to treat causal reasoning as a self-supervised learning objective rather than an externally imposed constraint or a language-mediated prediction. The motion dropout mechanism (Eq. 4) is deceptively simple: randomly withhold one motion component from the conditioning input and supervise on the full video. But conceptually, it reframes causal reasoning from "apply known physical laws" or "predict consequences in language" to "learn the statistical relationship pas ≈ f(act) from data by being forced to predict missing dynamics." The model is never told what physics is—it discovers action–consequence correlations in the training distribution by being required to complete partial motion observations into full interaction videos.

What makes this a genuine conceptual contribution rather than a training trick is the asymmetry of the supervision signal. The model receives only active motion 80% of the time and only passive motion 20% of the time, yet it must always produce both in its output. This is fundamentally different from standard conditional generation (where the model learns the joint distribution p(video | condition) and the condition is always complete). Here, the condition is explicitly incomplete, and the model must learn to infer the missing component. This is a form of masked modeling in the motion domain, analogous to masked language modeling in BERT or masked image modeling in MAE, but applied to the causal structure of physical interactions rather than to language or appearance tokens. The key difference is that in masked language modeling, the mask is random and the model learns statistical co-occurrence; here, the mask is semantically meaningful (active vs. passive), and the model learns causal directionality.

The bidirectional capability this enables—forward reasoning (provide active, predict passive) and inverse reasoning (provide passive, predict active)—is a direct consequence of the bidirectional dropout, not a separately engineered feature. The model does not have separate forward and inverse pathways; the same architecture, trained with the same objective, learns both mappings because it has seen both directions during training. This is a form of amortized causal inference: the model internalizes the bidirectional mapping between actions and consequences, amortizing what would otherwise require explicit optimization (e.g., inverse dynamics or trajectory optimization in robotics) into a single forward pass of the neural network.

The results in Table 2 provide empirical validation: MoRight achieves the highest Physical Commonsense (PC) score on WISA (0.76 vs. 0.73–0.75 for baselines) and the best video quality (FID 52.95, FVD 876.03) while using only a single active motion description, compared to baselines that receive full prompts describing both actions and outcomes. The PC improvement specifically validates that the model has learned physically meaningful action–consequence relationships, not just visual pattern matching. The human evaluation (Fig. 8) further reinforces this: MoRight is preferred 54.6% of the time on motion realism, despite baselines having access to privileged 3D trajectories with full active+passive information.

A subtle but important point: the motion dropout mechanism works within the pixel-level motion representation, not in an abstract feature space. This means the causal reasoning is spatially precise—the model predicts where consequence objects should move in pixel coordinates—rather than operating in a compressed representation that might lose spatial fidelity. This directly addresses the error propagation problem that plagues text-mediated causal reasoning pipelines: the model reasons about motion in the same space where it generates, eliminating the precision loss from cross-modal conversion.

Innovation 3: The Weak-Strong Input Asymmetry as a Practical Deployment Enabler

The paper's experimental design reveals an insight that is as much about what a system needs as input as about what it can produce. The baseline methods—Motion Prompting, ATI, and WanMove—all operate on a "strong input" assumption: they require dense per-pixel trajectories across all frames, including separate foreground and background signals, and typically use privileged 3D information (per-frame depth, 3D object tracks) to construct these trajectories. In the paper's evaluation, this includes full interaction trajectories—both active and passive motion—projected from 3D to pixel-aligned per-frame tracks.

MoRight operates on a fundamentally weaker input: first-frame reprojected trajectories (no future-frame tracking), sparse rather than dense (1500 tracks vs. all pixels), and only one motion component (active or passive, not both). The user draws simple curves or strokes on the first frame—the model handles everything else: completing the motion across frames, inferring missing interaction dynamics, transferring the dynamics to arbitrary camera viewpoints.

This weak-strong asymmetry is not an accident of experimental design—it is the practical deployment enabler that makes the conceptual innovations usable. The canonical-view formulation means users specify motion where it makes intuitive sense (on the static first frame), not where it's computationally convenient (in the target camera's pixel space). The motion dropout training means users only need to specify what they care about (the action or the outcome), and the model fills in the rest. Together, these design choices dramatically reduce the input burden: from "specify where every pixel goes in every frame" to "draw a stroke showing where you want the hand to move."

The significance of this is practical rather than theoretical, but it is no less important. A video generation system that requires dense per-pixel trajectories for all objects across all frames is not usable by humans—it requires either automated trajectory generation (which in turn requires full 3D scene understanding) or laborious manual annotation. MoRight's weak input requirement makes interactive video generation feasible: a user can draw a few strokes, specify camera motion, and get a plausible video in ~15 minutes on a single GPU. The fact that MoRight matches or exceeds the strong-input baselines (Table 1: best EPE on DynPose-100K, best overall on Cooking; Table 2: best FID/FVD/PC on WISA) while using substantially weaker inputs demonstrates that the architectural innovations are not just conceptually elegant but practically transformative—they convert a control problem that was fundamentally ill-posed (disentangling entangled signals) or prohibitively expensive (requiring full 3D annotation) into one that is both well-posed and tractable with lightweight user input.

Innovation 4: Mixed Training as a Solution to the Synthetic-Real Distribution Gap in Multi-View Supervision

The dual-stream architecture requires paired multi-view videos for training—showing the same scene dynamics under different cameras—but such data is essentially nonexistent in the real world at scale. The paper's response is a synthetic data generation pipeline (camera-controlled video-to-video generation applied to static-camera source videos) that produces 43K paired examples, but synthetic video-to-video translation inevitably introduces artifacts: texture degradation, temporal flickering, imperfect object boundary preservation under viewpoint change.

The conventional response to synthetic data artifact problems is either (a) improve the synthetic data quality, (b) domain-adapt the model to real data, or (c) accept the artifacts and hope the model learns to ignore them. MoRight takes a fundamentally different approach: mixed training that leverages the dual-stream architecture's inherent flexibility to absorb supervision from heterogeneous data sources without requiring them to be mutually consistent.

The key insight is that the dual-stream architecture does not always require paired data. When a real video has a static camera (object motion only), the model can duplicate it as both streams: the canonical stream receives explicit trajectory conditioning, the target stream receives none, and the cross-view attention must learn to transfer motion information between what are now identical views. When a real video has both camera and object motion (no paired canonical view), the model can train only the target stream, setting the canonical stream's loss to zero—the cross-view attention still operates, exposing the model to real-world camera-object motion correlations. These mixed-training strategies (Section 3.3, "Single-view real-world data for mixed-training") allow the model to learn from real data with real visual quality for the majority of its training, while still using the synthetic paired data to learn the geometric correspondence that enables camera–object disentanglement.

The ablation in Table 3 ("w/o mixed training") validates this: removing mixed supervision degrades camera accuracy (rotation error increases from 2.16 to 2.22, translation from 10.11 to 12.80), confirming that the synthetic paired subset alone has insufficient camera motion diversity. The mixed training strategy is an elegant solution to a problem that is pervasive in 3D and multi-view learning: how to learn geometric reasoning when real data provides visual quality but no geometric correspondence, and synthetic data provides geometric correspondence but poor visual quality. The dual-stream architecture's ability to selectively supervise streams depending on data availability makes this possible—a design choice that is not merely convenient but is the enabling mechanism for training on real-world data at scale.

5. Experimental Analysis

Evaluation Methodology

• Dataset. MoRight is evaluated on three datasets spanning diverse interaction scenarios. DynPose-100K (Rockwell et al., 2025) is an in-the-wild dataset with highly dynamic camera motion; the paper manually selects 50 videos exhibiting strong viewpoint changes and clear object interactions. WISA (Wang et al., 2025) is a large-scale physical-dynamics dataset; 50 videos are selected from categories including collision, deformation, elasticity, liquid, and rigid-body motion. Cooking is a benchmark of 50 real-world cooking videos collected from YouTube, featuring complex hand-object interactions such as pushing, cutting, and picking.

• Base model(s). All experiments build on the pretrained Wan2.1-14B image-to-video diffusion model (Wan et al., 2025), chosen for its state-of-the-art video generation quality and the availability of its 14B-parameter backbone as a strong starting point for fine-tuning. For fair comparison, all baseline methods (Motion Prompting, ATI, WanMove, Gen3C) are trained on the same Wan2.1-14B backbone; the paper reimplements and retrains Gen3C and Motion Prompting in their setup (indicated by asterisks in tables).

• Metrics. Four categories of metrics are used across experiments. Video quality: PSNR and SSIM against reference videos (pixel-level fidelity), and FID (Heusel et al., 2017) and FVD (Unterthiner et al., 2018) for distribution-level similarity. Camera accuracy: rotation error and translation error between reference poses and poses estimated from generated videos using ViPE (Huang et al., 2025); median errors across frames are reported to mitigate estimation noise. Motion accuracy: end-point error (EPE), the ℓ₂ distance between ground-truth object tracks and predicted tracks extracted with AllTracker; median EPE is reported to reduce the impact of outlier tracks. Motion realism: Physical Commonsense (PC) and Semantic Adherence (SA) from VideoPhy (Bansal et al., 2024), both 5-point scores normalized to [0, 1]; PC measures whether generated videos follow real-world physical behaviors, while SA evaluates semantic consistency with the input text prompt. All evaluations are conducted at 480p resolution.

• Baselines. Five methods are compared: (1) Wan2.1 (Wan et al., 2025), the base model without any motion control, providing a lower bound on generation quality; (2) Gen3C (Ren et al., 2025), which supports camera control only (no object motion), reimplemented and retrained in the paper's setup; (3) Motion Prompting (MP) (Geng et al., 2024), a trajectory-based method that takes dense per-pixel tracks as input, reimplemented and retrained; (4) ATI (Wang et al., 2025), which takes dense pixel tracks including privileged 3D information as input; and (5) WanMove (Chu et al., 2025), which uses latent trajectory guidance for motion control with privileged future-frame tracking information.

• Generation budget / compute accounting. The paper does not standardize generation budgets across methods in the way typical for scaling-law analyses (e.g., fixed number of function evaluations). Instead, all methods generate videos of the same length and resolution (480p, number of frames implied by the dataset), with generation cost measured implicitly through inference time—MoRight takes approximately 15 minutes per video on a single A100 GPU using 35 diffusion steps. The computational comparison favors baselines in terms of input requirements (they receive privileged future-frame dense tracks while MoRight uses only first-frame reprojected sparse tracks), making the reported accuracy comparisons conservative with respect to MoRight.

• Cross-validation / statistical protocol. No cross-validation or multiple-random-seed protocol is reported for the quantitative experiments. The ablation studies in Table 3 are single-run results on the Cooking benchmark. The human perceptual evaluation collects 330 responses from 11 participants across 30 randomly selected examples, with videos presented in randomized order to avoid positional bias, and an option for ties or "None" to reduce forced-choice artifacts. Participants evaluate on three criteria: Controllability, Motion Realism, and Photorealism.

Main Quantitative Results

Disentangled Camera-Object Motion Control (Table 1)

The experiments comparing motion-controlled video generation on DynPose-100K and Cooking establish that MoRight achieves competitive or superior motion control accuracy while using fundamentally weaker inputs than all baselines.

Input asymmetry between methods. The baseline methods—Motion Prompting, ATI, and WanMove—all receive "privileged signals of both foreground and background tracks of all the pixel trajectories" including per-frame dense pixel tracks that encode the combined effect of object and camera motion. MoRight uses only "reprojected trajectories defined on the canonical frame, without access to future-frame pixel trajectories." The column "Full info" in Table 1 marks which methods receive this privileged information (✓) versus which do not (×).

DynPose-100K results. On this dataset with highly dynamic camera motion, WanMove achieves the best overall numbers with PSNR 13.91, SSIM 0.521, rotation error 4.12°, translation error 3.56, and EPE 8.05. MoRight obtains PSNR 12.30, SSIM 0.457, rotation error 4.55°, translation error 4.61, and EPE 7.64—the best object motion accuracy (lowest EPE) among all methods. The paper notes that MoRight's photometric scores (PSNR/SSIM) are slightly behind WanMove because "errors in camera pose estimation and trajectory reprojection can degrade the input control signals" under highly dynamic camera motion. ATI achieves PSNR 13.18 and SSIM 0.493 with rotation error 5.62° and EPE 8.43; Motion Prompting achieves PSNR 11.72 and SSIM 0.455 with rotation error 6.76° and EPE 7.56. Gen3C (camera-only) achieves PSNR 12.45 and rotation error 5.46°, and the base Wan2.1 (no control) reaches PSNR 11.23 and SSIM 0.435.

Cooking benchmark results. On this dataset featuring complex hand-object interactions, MoRight achieves the best performance across all metrics: PSNR 16.44, SSIM 0.594, rotation error 2.16°, translation error 10.11, and EPE 4.27. This outperforms WanMove (PSNR 16.42, SSIM 0.589, rotation error 2.93°, translation error 13.27, EPE 5.47), ATI (PSNR 15.93, SSIM 0.582, rotation error 4.25°, translation error 16.94, EPE 5.87), and Motion Prompting (PSNR 15.68, SSIM 0.564, rotation error 2.50°, translation error 12.24, EPE 4.25). The paper attributes baseline degradation on this dataset to the fact that ATI and WanMove "couple camera and object motion in a single tracking signal" and "tend to favor the dominant motion mode... sometimes sacrificing camera accuracy or object tracking fidelity" under the complex interaction patterns of cooking videos. Gen3C achieves PSNR 15.37 and rotation error 1.97°, and the base Wan2.1 reaches PSNR 14.23.

A critical observation about the comparison. MoRight's motion control accuracy (EPE) is evaluated in "future-frame pixel space" for all methods—meaning trajectory predictions are compared in the target camera's pixel coordinates, which is the same space where baselines receive their privileged inputs. That MoRight achieves competitive or superior EPE despite never receiving future-frame pixel trajectories is the key validation of the canonical-view design: the cross-view attention successfully transfers object dynamics from the canonical stream to arbitrary target viewpoints.

Interactive Motion Generation with Causality Reasoning (Table 2)

The experiments on WISA and Cooking evaluate generation quality and motion realism when the model must infer interaction outcomes from partial motion specifications.

Input protocol difference. Baseline methods (MP, ATI, WanMove) receive "detailed motion captions with full interaction descriptions" and their original prompts containing "both motion descriptions and expected consequences." MoRight receives "only a single active motion description, without specifying passive outcomes, and must infer the resulting interactions."

WISA results. MoRight achieves the best video quality metrics (FID 52.95, FVD 876.03) and the highest Physical Commonsense score (PC 0.76). Baseline results: Motion Prompting (FID 57.29, FVD 975.94, PC 0.75, SA 0.82), ATI (FID 69.80, FVD 990.82, PC 0.75, SA 0.83), and WanMove (FID 61.34, FVD 1088.23, PC 0.73, SA 0.83). For Semantic Adherence (SA), MoRight achieves 0.82—slightly lower than ATI (0.83) and WanMove (0.83), but the paper explains this difference explicitly: "we rewrite the input prompt to remove passive motion descriptions and avoid information leakage; consequently, our score is slightly lower than methods that use full prompts containing both actions and outcomes." The SA score thus reflects the model's generation quality under a stricter input condition rather than a failure of semantic consistency.

Cooking results. MoRight achieves the best performance across all metrics: FID 39.94, FVD 730.46, PC 0.88, and SA 0.89. All baselines trail: Motion Prompting (FID 43.49, FVD 759.53, PC 0.87, SA 0.89), ATI (FID 55.80, FVD 881.94, PC 0.85, SA 0.90), and WanMove (FID 53.51, FVD 882.90, PC 0.84, SA 0.87). The PC improvement (+0.01 over the best baseline on Cooking, +0.01 on WISA) is modest in absolute terms but is achieved while using only active motion descriptions versus baselines that receive full interaction prompts—the model's physical commonsense reasoning must compensate for missing explicit passive motion specification.

Human Perceptual Evaluation (Figure 8)

The human study directly compares MoRight against ATI and WanMove, with participants evaluating generated videos on three criteria.

Setup details. From 330 responses by 11 participants across 30 randomly selected examples, MoRight is preferred in the majority of cases across all three criteria:

• Controllability (alignment with input object and camera motion): MoRight 53.5%, WanMove 25.0%, ATI 18.8%, None 2.7%
• Motion Realism (physical plausibility of interactions): MoRight 54.6%, WanMove 25.7%, ATI 18.2%, None 1.5%
• Photorealism (visual quality): MoRight 55.9%, WanMove 23.1%, ATI 17.4%, None 3.6%

Interpretation of the preference gap. The baselines ATI and WanMove "rely on pixel-aligned per-frame tracks projected from privileged 3D trajectories, including both foreground/background and full interaction (active and passive) motion," while MoRight uses "only first-frame active trajectories." Despite this substantial input asymmetry favoring the baselines, MoRight is preferred roughly 2:1 over them across all criteria. The paper attributes this to baselines' lack of "explicit interaction reasoning" and their entanglement of "camera and object motion," which lead to inferior controllability and less physically plausible dynamics even when given perfect trajectory information.

Ablation Studies and Robustness Checks

All ablations are conducted on the Cooking benchmark and reported in Table 3.

Cascaded pipeline (row 1): A naive two-stage approach—first generating a motion-controlled video under a static camera, then applying a Gen3C-style camera controller—is tested as a baseline for camera-object disentanglement. This achieves FID 41.74, FVD 728.80, PSNR 15.98, SSIM 0.569, rotation error 2.69°, translation error 11.50, and EPE 5.05. Compared to the full MoRight model (FID 39.94, FVD 730.46, rotation error 2.16°, translation error 10.11, EPE 4.27), the cascaded approach shows degraded motion control accuracy (higher EPE and camera errors), which the paper attributes to "error accumulation between two stages." This validates that the dual-stream joint denoising is superior to sequential processing, likely because cross-view attention allows the target stream to access clean motion information throughout the denoising trajectory rather than receiving a potentially artifact-ridden intermediate video.

Without fixed-view branch (row 2): Removing the canonical static-view anchor by training only with dynamic camera views and jointly encoding reprojected tracks and camera embeddings into a single stream yields substantially degraded performance: FID 51.17, FVD 997.83, PSNR 14.15, SSIM 0.515, rotation error 3.36°, translation error 14.57, and EPE 14.30 (versus 4.27 for the full model). This is the largest single degradation in the ablation table and confirms that the canonical-view anchor is the critical enabler for camera-object disentanglement—without a separate stream providing geometrically clean motion information, the model cannot reliably separate object dynamics from camera motion.

Without motion reasoning (row 3): Disabling the active/passive motion decomposition during training (no motion dropout, providing all trajectories during training) yields FID 44.04, FVD 784.19, PSNR 15.55, SSIM 0.562, rotation error 2.88°, translation error 12.49, EPE 5.05, PC 0.87, and SA 0.88. Compared to the full model (FID 39.94, PC 0.88, SA 0.89), the increased FID and FVD and reduced PC score indicate that motion dropout training improves both generation quality and physical commonsense reasoning. The EPE remains unchanged at 5.05, suggesting the motion dropout primarily affects interaction quality rather than trajectory-following accuracy—expected, since trajectory following does not require causal reasoning.

Without mixed training (row 4): Training only on the synthetically generated paired multi-view data (removing the real-world single-view data with mixed supervision strategies) yields FID 41.94, FVD 808.96, PSNR 16.29, SSIM 0.583, rotation error 2.22°, translation error 12.80, and EPE 4.09. Camera translation error increases notably (from 10.11 to 12.80) while rotation error is nearly unchanged (2.22 vs. 2.16), confirming the paper's claim that "the paired subset contains limited camera motion diversity." The model overfits to the narrow camera motion distribution of the synthetic paired data and fails to generalize to the more diverse camera trajectories in the Cooking test set.

Coarse motion granularity (row 5): Using coarse segment-level trajectories (all pixels of an object sharing a single trajectory) instead of fine-grained pixel tracks yields FID 39.83, FVD 725.88, PSNR 16.45, SSIM 0.594, rotation error 2.21°, translation error 10.98, EPE 4.37, PC 0.88, and SA 0.88. The model remains robust to reduced trajectory granularity with only a slight EPE increase (4.37 vs. 4.27), demonstrating that the training-time multi-granularity motion dropout successfully enables handling of coarse user inputs.

Passive motion input (row 6): Providing only passive motion trajectories (the inverse reasoning direction) yields FID 44.20, FVD 838.67, PSNR 15.99, SSIM 0.588, rotation error 2.21°, translation error 11.04, EPE 7.27, PC 0.87, and SA 0.88. The substantially degraded EPE (7.27 vs. 4.27 for active motion) mirrors the training-time asymmetry: the model sees active motion 80% of the time and passive motion 20% of the time, making forward reasoning (active → all dynamics) a stronger capability than inverse reasoning (passive → all dynamics). This is expected given the dropout probability ratio, but it quantifies the asymmetry in practical terms: inverse reasoning is functional but notably less accurate than forward reasoning for trajectory control.

Active motion input (row 7): This is the full model's standard configuration, achieving the best overall results: FID 39.94, FVD 730.46, PSNR 16.44, SSIM 0.594, rotation error 2.16°, translation error 10.11, EPE 4.27, PC 0.88, and SA 0.89.

Critical Assessment

The paper makes three central claims: (1) disentangled camera-object motion control enables independent control of object trajectories and camera viewpoints from a single image; (2) motion causality reasoning via active/passive decomposition and motion dropout enables forward and inverse reasoning about interactions; and (3) the system outperforms existing methods that rely on privileged future-frame dense tracking information while using only first-frame reprojected sparse tracks.

Claim 1 (disentangled control) is demonstrated but with important boundary conditions. Table 1 shows that MoRight achieves the best EPE (object motion accuracy) on both DynPose-100K (7.64) and Cooking (4.27), confirming that the canonical-view formulation successfully transfers object dynamics to arbitrary camera viewpoints. However, the camera accuracy metrics reveal a more nuanced picture: on DynPose-100K with highly dynamic camera motion, MoRight's rotation error (4.55°) and translation error (4.61) are mid-range—worse than Gen3C's camera-only control (rotation 5.46° is actually higher; MoRight is better rotation-wise but worse translation-wise than WanMove's 3.56). The paper itself acknowledges that "errors in camera pose estimation and trajectory reprojection can degrade the input control signals" under highly dynamic camera motion. This means the disentanglement quality depends on the accuracy of upstream geometric estimation (ViPE camera poses, depth estimation, trajectory canonicalization)—a dependency chain that is not directly tested in ablation. The experiments demonstrate that the disentanglement works, but not that it is robust to geometric estimation errors.

A missing experiment would have directly measured disentanglement quality: generating the same object motion under progressively more extreme camera motions and measuring how well object trajectories are preserved. The current evaluation compares against reference videos, which confounds generation quality with control accuracy. Without such a controlled experiment, it remains unclear whether the cross-view attention is learning genuine viewpoint invariance or interpolating within the limited camera motion distribution seen during training (orbits, pans, zooms, and real trajectories from the training data).

Claim 2 (causal reasoning) is supported by PC scores but the evidence is modest in absolute magnitude. On WISA, MoRight achieves PC 0.76 versus the best baseline of 0.75 (Motion Prompting and ATI)—a 0.01 improvement. On Cooking, MoRight achieves PC 0.88 versus 0.87 (Motion Prompting)—again 0.01. While these differences are directionally consistent, the absolute gap is small. This could indicate that (a) the VideoPhy PC metric has limited sensitivity to interaction quality improvements, (b) the baselines already achieve reasonable physical commonsense from their full-interaction inputs, or (c) the motion dropout training provides a genuine but modest improvement in causal reasoning that requires more challenging evaluation scenarios to clearly demonstrate.

The qualitative examples in Figure 7 are compelling—the model generates plausible cloth deformation from hand motion (forward) and plausible hand motion from ball trajectories (inverse)—but qualitative success on selected examples does not establish that the capability generalizes reliably. The human evaluation provides stronger evidence: MoRight's 54.6% preference on motion realism (versus 25.7% WanMove and 18.2% ATI) suggests human raters perceive interaction quality differences that the PC metric may not fully capture. However, the human study has only 11 participants and 30 examples, which is a limited sample for drawing strong statistical conclusions.

Claim 3 (outperforming privileged-input baselines) holds but the comparison is asymmetric in complex ways. The baselines receive "privileged 3D trajectories" with "both foreground/background and full interaction (active and passive) motion" projected to per-frame pixel tracks, while MoRight receives only first-frame active trajectories. On its face, MoRight matching or exceeding these baselines is impressive. But the comparison raises several questions:

First, the baselines are designed to follow provided trajectories—they are trajectory-conditioned video generators, optimized to reproduce the trajectories they receive. The evaluation asks them to generate videos where those trajectories are the ground truth. MoRight is given weaker trajectories but compensated by inference-time reasoning that completes the missing dynamics. The fact that MoRight achieves comparable EPE means its inferred trajectories are as accurate as the baselines' provided trajectories—a surprising result that suggests either the baselines are not perfectly following their privileged inputs, or MoRight's inference is remarkably accurate.

Second, the "privilege" of baselines may be a double-edged sword: providing dense per-pixel trajectories including background motion could be overly constraining, forcing the model to reproduce trajectories that may be visually inconsistent with generated content. MoRight's sparse first-frame active trajectories leave more room for the model to generate internally consistent dynamics. The experiments do not distinguish between "the baselines cannot follow their privileged inputs" versus "following privileged inputs perfectly produces worse-looking videos"—these are fundamentally different failure modes with different implications.

Genuine experimental weaknesses:

• Single backbone model family. All experiments use Wan2.1-14B. The paper's claims about disentanglement and causal reasoning could be specific to this architecture's attention mechanisms and training data distribution. No experiments with alternative backbones (e.g., CogVideoX, SVD, or a smaller Wan variant) are reported.

• Small test sets. DynPose-100K evaluation uses 50 manually selected videos, WISA uses 50 selected videos, and Cooking uses 50 collected videos. With test sets of 50 examples, performance differences of a few EPE points or 0.01 PC may not be statistically robust. The paper does not report confidence intervals or significance tests for any quantitative comparison.

• No cross-validation or seed analysis. All ablations in Table 3 are single-run results. Without multiple random seeds or cross-validation folds, it is impossible to distinguish genuine architectural improvements from training noise. The two-fold cross-validation protocol mentioned in the main methodology applies to strategy selection for compute-optimal approaches but is not implemented here.

• Difficulty estimation cost is unaccounted for. The motion extraction and canonicalization pipeline (ViPE pose estimation, depth estimation, AllTracker dense tracking) requires substantial computation per video. The paper's inference pipeline does not account for this cost, which is significant—pose estimation and dense tracking are comparable in cost to video generation itself. In a deployment setting, this pre-processing cost would need to be amortized or optimized.

• The 15-minute per-video inference time. Generating one video on a single A100 GPU requires approximately 15 minutes with 35 diffusion steps. This is substantially slower than single-stream video generation and limits interactive applications despite the paper's interactive GUI. The paper does not report inference time for baselines, making latency comparisons impossible.

• The synthetic training data introduces systematic biases. The 43K paired multi-view videos are generated by applying a camera-control video-to-video model to static-camera source videos. This means the target-stream videos contain artifacts from the video-to-video translation process, and the camera trajectories are limited to what the video-to-video model can handle. The paper does not analyze whether model failures correlate with the synthetic data's artifacts—are the remaining errors in Table 1 due to fundamental model limitations or to artifacts inherited from training data? The mixed-training strategies mitigate but do not eliminate this concern.

Missing experiments that would have strengthened the paper:

• Disentanglement stress test: Generate the same object motion under increasingly extreme camera motions (large rotations, translations, zoom factors well beyond the training distribution) to measure when cross-view attention breaks down. This would characterize the model's generalization envelope for viewpoint change.

• Causal reasoning ablation with varying dropout rates: The paper uses p=0.8 for active and p=0.2 for passive dropout. Systematically varying this ratio would reveal how much asymmetric supervision is needed for effective causal reasoning, and whether the 80/20 split is optimal or arbitrary.

• Comparison with a physics-engine-based approach on the same scenes: A head-to-head comparison with PhysGen or MotionCraft on a subset of the test data would directly test the paper's claim that learning-based causal reasoning matches or exceeds explicit physics simulation.

• Failure mode analysis correlated with geometric estimation quality: The paper acknowledges that pose estimation errors degrade results. Stratifying test examples by estimated pose quality and measuring performance would quantify this dependency and provide practical guidance on when the method is reliable.

• Multi-view consistency evaluation: Since the dual-stream architecture generates both a canonical and target view, evaluating the geometric consistency between these views (e.g., via epipolar constraints on corresponding points) would directly measure whether the cross-view attention is learning genuine geometric correspondence or statistical appearance matching.

Conditional validity of claims. The disentanglement claim holds for camera trajectories within the distribution of the training data (orbits, pans, zooms, real camera trajectories from Internet videos). It likely degrades for out-of-distribution camera motions or extreme viewpoint changes not seen in training. The causal reasoning claim holds for interaction types present in the training data (hand-object interactions, rigid-body collisions, deformable object manipulation). It would not transfer to novel physical phenomena or interaction categories absent from the training distribution. The outperformance claim holds for the Cooking and WISA test sets with their specific interaction patterns; generalization to arbitrary user-specified interactions in arbitrary scenes remains an open question not addressed by the experiments.

6. Limitations and Trade-offs

Reliance on Off-the-Shelf Geometric Estimation Models Creates a Fragile Pre-Processing Bottleneck

The assumption or constraint. The entire canonicalization pipeline—the mechanism that makes camera–object disentanglement possible—depends on three external foundation models operating in sequence: ViPE for camera pose estimation, a monocular depth estimator (Depth Anything 3 or MoGe) for per-frame depth, and AllTracker for dense pixel trajectories. The canonicalization formula (Eq. 5) unprojects 2D trajectory points to 3D using estimated depth, transforms them by estimated camera poses, and reprojects them to the first frame. This stacking of estimations means errors compound: a depth error at frame $i$ produces an incorrect 3D position, which the camera pose transformation (itself estimated with error) then transforms incorrectly, yielding a reprojected trajectory $\boldsymbol{\tau}^{\text{can}}_i$ that does not accurately represent the object's physical motion.

The paper partially acknowledges this in Section 4.3, noting that on DynPose-100K with highly dynamic camera motion, "errors in camera pose estimation and trajectory reprojection can degrade the input control signals." However, the paper does not quantify how large these errors are, how they correlate with downstream generation quality, or whether certain failure modes in the generated videos (unnatural motion, object disappearance, hallucinated content as in Fig. 9) are caused primarily by geometric estimation errors rather than by fundamental limitations of the video model.

The consequence. In deployment scenarios where the upstream geometric estimators produce inaccurate output—low-texture scenes where depth estimation fails, videos with motion blur where tracking drifts, rapid camera motion that exceeds ViPE's reliable operating range—the canonical trajectories provided to the model are geometrically inconsistent with the true scene dynamics. The model then receives a contradictory conditioning signal: the trajectories say the object moved along path $P$, but the visual evidence in the input image (lighting, occlusions, object pose) may be inconsistent with that path given the scene geometry. The model must reconcile this contradiction, and the paper's limitation analysis (Fig. 9) shows that this can produce physically inconsistent dynamics—objects disappearing, merging, or moving unnaturally.

More subtly, even when geometric estimation is approximately correct, small errors in canonicalization mean the "canonical" trajectories are not truly canonical—they contain residual camera motion that the cross-view attention must implicitly correct for. The paper's ablation removing the fixed-view branch (Table 3, row 2) shows that when the canonical anchor is removed entirely, EPE jumps from 4.27 to 14.30, suggesting the disentanglement mechanism is highly sensitive to the quality of the canonical representation. It is plausible that even moderate canonicalization errors would produce intermediate degradation between these two extremes, but no experiment measures this.

What evidence exists in the paper. The sensitivity is visible indirectly. On DynPose-100K (highly dynamic cameras), MoRight's rotation error (4.55°) and translation error (4.61) trail WanMove's camera accuracy (rotation 4.12°, translation 3.56), despite MoRight achieving better object motion accuracy (EPE 7.64 vs. 8.05). This pattern—worse camera accuracy with better object tracking—is consistent with canonicalization errors confusing the model's ability to precisely reconstruct the camera viewpoint while still transferring object dynamics approximately correctly. The limitation examples in Fig. 9 (unnatural motion from occlusion-induced track sparsity, physically unrealistic dynamics such as the disappearing soccer ball) are not attributed to root causes, leaving open the possibility that many or most of these failures originate in the geometric estimation pipeline rather than in the generative model.

Mitigation status. The paper does not directly address geometric estimation robustness. The occlusion and track dropout augmentations during training (Section 3.4) simulate missing or unreliable trajectories, which improves robustness to tracking failures, but these augmentations operate on trajectories after canonicalization—they do not simulate the errors introduced by incorrect depth or pose estimation during the canonicalization step itself. The paper does not propose or evaluate alternative approaches to canonicalization that might be more robust (e.g., using relative camera poses between adjacent frames rather than all relative to the first frame, or using optical flow for canonicalization instead of depth-based unprojection). The paper does not report the accuracy of ViPE pose estimates or depth estimates on the training or evaluation data, making it impossible to gauge how much headroom exists for improvement through better geometric estimation alone.

---

Motion Causality Reasoning Is Trained with a Fixed 80/20 Active/Passive Split, But There Is No Evidence This Ratio Is Optimal or That the Learned Causal Mappings Generalize Beyond the Training Distribution

The assumption or constraint. The motion dropout mechanism (Eq. 4) uses a fixed probability: active motion is provided 80% of the time and passive motion 20% of the time during training (Appendix A.3). This asymmetry reflects the intuition that forward reasoning (action → consequence) is the more common and more well-posed direction—most interactions have clear agent-initiated actions with unambiguous consequences, while inverse reasoning is often underdetermined (many different hand motions could produce the same cup trajectory).

The paper treats this 80/20 split as a fixed hyperparameter and does not ablate it. The consequences of this choice are visible in the ablation study (Table 3, rows 6–7): when the model receives only passive motion at inference (inverse reasoning), EPE degrades from 4.27 (active input) to 7.27 (passive input), and FID increases from 39.94 to 44.20. This asymmetry is expected given the training distribution, but it raises a deeper question: is the model learning genuine causal directionality (actions cause consequences, and that directional relationship is reflected in the learned mapping), or is it simply reflecting the training data frequency—doing well on the more common input pattern (active trajectories) and poorly on the less common one (passive trajectories)?

The consequence. A practitioner deploying MoRight for inverse reasoning tasks—recovering plausible actions from desired outcomes, which is essential for affordance learning, imitation from observation, and planning—would encounter substantially worse trajectory accuracy than for forward reasoning. The 7.27 vs. 4.27 EPE gap is large enough that inverse-reasoned trajectories may not be reliable for downstream tasks requiring precise spatial localization. Moreover, because the model does not distinguish active from passive motion architecturally (both go through the same trajectory encoder and condition injection), there is no mechanism for the user to indicate which reasoning direction is intended. The model's behavior depends implicitly on whether the input trajectories happen to resemble active motion patterns or passive motion patterns from the training distribution, which is not a controllable property.

More fundamentally, the paper provides no characterization of what causal relationships the model has actually learned. Does it learn a generalizable physical principle (e.g., "objects in contact with a moving hand translate in the direction of hand motion") or does it learn dataset-specific correlations (e.g., "when a hand-shaped region moves rightward and a cup-shaped region is nearby, the cup-shaped region often moves rightward too")? The difference matters enormously for generalization. A model that has learned physical principles would handle novel object shapes, unusual viewpoints, or interaction types not seen in training. A model that has learned co-occurrence statistics would fail on out-of-distribution scenes. The evaluation datasets (WISA, Cooking) are within the same domain as the training data (Panda-70M, Wild-SDG-1M, both Internet video datasets), so they do not test out-of-distribution generalization. The PC metric measures whether generated videos look physically plausible to a learned classifier—not whether the underlying causal structure generalizes.

What evidence exists in the paper. The ablation in Table 3 (rows 3 vs. 7) isolates the effect of motion dropout training: without it, PC drops from 0.88 to 0.87 (a 0.01 change) and FID increases from 39.94 to 44.04. This confirms that motion dropout provides a modest improvement in generation quality and physical commonsense, but the 0.01 PC difference—against a baseline that already achieves 0.87—suggests the learned causal structure provides only a marginal benefit over whatever physical understanding the base Wan2.1-14B model already possesses from its large-scale pretraining. The human evaluation (Fig. 8) shows MoRight preferred 54.6% of the time on motion realism, which is a stronger signal, but this evaluates the full system (disentanglement + causal reasoning + generation quality) rather than causal reasoning in isolation.

Mitigation status. The paper does not ablate the active/passive ratio, does not evaluate generalization to novel interaction types, and does not propose a mechanism for users to specify the reasoning direction at inference time. The asymmetry is treated as an implementation detail rather than as a design choice with significant downstream consequences. Future work that the paper does not explicitly propose but that follows naturally would include: (1) characterizing what causal structures the model learns by evaluating on systematically varied interactions (same action with different objects, same objects with different actions, same interaction from different viewpoints); (2) exploring whether the active/passive distinction is the right decomposition or whether a more fine-grained causal graph (agent, instrument, patient, secondary effect) would produce more robust reasoning; and (3) developing training schemes that balance forward and inverse reasoning performance without the current 4:1 asymmetry.

---

The 43K Synthetic Paired Multi-View Training Videos Inherit Systematic Biases from the Video-to-Video Generation Model Used to Create Them

The assumption or constraint. The dual-stream architecture requires paired training data showing the same object dynamics under different camera viewpoints. Since such data is extremely rare in the wild, the paper synthesizes 43K paired videos by taking static-camera source videos (76K identified from Panda-70M and Wild-SDG-1M) and applying a camera-control video-to-video model (Plenoptic Video Generation, Fu et al., 2026) to generate corresponding moving-camera videos.

This synthetic data is not real. The video-to-video model introduces artifacts—texture degradation, temporal flickering, imperfect object boundary preservation under viewpoint change, and potentially incorrect handling of occlusions when the camera moves to reveal previously hidden scene regions. More subtly, the video-to-video model may produce systematically biased dynamics: if it tends to slightly misposition objects under large camera motions, the cross-view attention in MoRight learns to associate those mispositionings with the correct canonical-to-target mapping, encoding the video-to-video model's errors as if they were geometric truths.

The paper acknowledges this limitation explicitly in Section 3.3: "The generated paired videos inevitably contain visual artifacts." The mixed-training strategies (using real single-view data alongside synthetic paired data) are designed to mitigate this, but they cannot eliminate it—the geometric correspondence necessary for camera–object disentanglement can only be learned from the paired data, and if that paired data is systematically biased, the learned correspondence will be biased as well.

The consequence. The model's ability to disentangle camera and object motion is upper-bounded by the quality of the synthetic paired training data. If the video-to-video model introduces a consistent offset between canonical and target views—for instance, placing objects 2 pixels left of where they should be under a rightward camera orbit—MoRight's cross-view attention will learn to compensate for this offset, producing generated videos where object positions are internally consistent but systematically misaligned with the specified camera trajectory. The evaluation metrics in Table 1 (PSNR, SSIM, rotation error, translation error, EPE) compare against reference videos, so this systematic bias would appear as degraded accuracy—but the metrics cannot distinguish "the model got the geometry wrong" from "the model learned incorrect geometry from biased training data."

A second consequence is that camera trajectories in the synthetic data are limited by what the video-to-video model can handle. If the Plenoptic Video Generation model produces artifacts or fails for extreme camera motions (large rotations, rapid translations, extreme zoom), those camera trajectories are underrepresented or absent from the training data, and MoRight will not learn to handle them. The paper augments the synthetic data with "basic camera operations" and "dynamic camera trajectories extracted from real videos," but these are sampled from the same overall training distribution—they are not designed to stress-test the disentanglement mechanism.

What evidence exists in the paper. The ablation removing mixed training (Table 3, row 4) provides indirect evidence: training only on synthetic paired data degrades camera translation error from 10.11 to 12.80, confirming that the synthetic data alone has insufficient diversity or quality. The "w/o fixed view" ablation (row 2) is even more revealing: when the canonical anchor is removed and the model must disentangle camera from object motion without it, EPE spikes to 14.30—the model essentially cannot recover clean object motion. This suggests that the learned geometric correspondence is fragile and heavily dependent on the specific data it was trained on. However, the paper does not directly analyze synthetic data quality—there is no measurement of video-to-video model accuracy on the specific static-camera videos used for training, no characterization of artifact types or frequencies, and no ablation comparing different video-to-video models or different synthetic data generation strategies.

Mitigation status. The mixed-training strategies (Section 3.3) partially address this by exposing the model to real-world single-view videos with natural visual quality. However, real single-view data can only train the canonical stream (when cameras are static, the video is duplicated as both streams) or the target stream alone (when cameras move, the canonical stream loss is zeroed). Neither case provides the paired multi-view supervision necessary for learning the geometric correspondence that enables disentanglement. The mixed training is therefore a regularizer—it prevents overfitting to synthetic data artifacts—but it cannot teach geometric reasoning that the synthetic data fails to encode. The paper does not propose or evaluate alternative approaches to generating paired training data (e.g., using 3D-reconstructed scenes, leveraging multi-view capture rigs, or employing test-time optimization to refine canonicalization), limiting the ceiling on disentanglement accuracy to whatever the Plenoptic Video Generation model provides.

---

The Difficulty Estimation Pipeline Is Computationally Expensive and Its Cost Is Not Accounted for in the Reported 15-Minute Inference Time

The assumption or constraint. At inference time, generating a controllable video requires the user to provide object motion trajectories drawn on the first frame. However, the paper's evaluation protocol—and the implied deployment workflow—includes a substantial pre-processing step that is not accounted for in the reported "15 minutes on a single A100 GPU" inference time. To canonicalize trajectories (for evaluation against reference videos) or to set up the input conditioning, the system must estimate camera poses via ViPE, estimate per-frame depth, and optionally run dense tracking. For the user-facing application, the user draws trajectories directly on the first frame (no canonicalization needed), but for any evaluation or benchmarking scenario where ground-truth or reference trajectories must be canonicalized, this pipeline runs on every video.

The paper does not report the computational cost of this pre-processing. ViPE (a neural network for camera pose estimation), monocular depth estimation (Depth Anything 3 or MoGe), and dense tracking (AllTracker) are each substantial models in their own right. Processing a single video through this pipeline likely requires significant GPU time—potentially comparable to or exceeding the 15-minute generation time. In Section 3.2, the paper explicitly states that difficulty estimation using 2048 samples per question is computationally prohibitive and calls it "a key avenue for future work." The same concern applies here: the pre-processing cost to set up the input conditioning is never factored into the reported inference budget.

The consequence. The true end-to-end latency for a user to go from "I have a video and I want to generate a new video with different camera motion and object trajectories" to receiving the output is 15 minutes (generation) plus the cost of running ViPE, depth estimation, and optionally AllTracker on the input video. For the Cooking benchmark evaluation, where reference videos exist and canonicalized trajectories must be computed for scoring, this pre-processing cost is incurred per test example. The headline "15 minutes per video" is therefore a lower bound that only captures the generative portion, not the full pipeline.

More significantly, if the system is deployed in an interactive setting—a user draws trajectories, specifies a camera path, and waits for the result—the pre-processing cost affects perceived latency. Camera pose estimation and depth estimation must run on the input image (and potentially on the target camera trajectory to validate or adjust it). While these are faster than running on a full video, they still add latency to what is already a 15-minute generation process. The paper's interactive GUI (Fig. 11) does not report total wall-clock time from input to output.

What evidence exists in the paper. The paper specifies the inference configuration (35 diffusion steps, 15 minutes on a single A100 GPU) and the models used in the data curation pipeline (ViPE, Depth Anything 3/MoGe, AllTracker, Qwen3-VL, SAM2), but provides no timing for any of these components. The data curation pipeline description (Fig. 4) suggests this processing is done offline for training data, not at inference time for user inputs. However, the evaluation protocol for motion accuracy (EPE) requires canonicalized trajectories, and the evaluation of the Cooking benchmark likely required running this pipeline on the 50 test videos. None of this evaluation-time pre-processing cost is reported or factored into comparisons with baseline methods, which also require trajectory extraction (via AllTracker or similar) for their privileged input signals—making the cost comparison roughly equal but still unquantified.

Mitigation status. The paper does not address this as a limitation and does not report pre-processing costs. For the user-facing application with hand-drawn trajectories on the first frame, the pre-processing is minimal (no canonicalization needed—the user's strokes are the trajectories). For evaluation and benchmarking, the cost is unavoidable but symmetric across methods (all require trajectory extraction). The paper does not propose or discuss lighter-weight alternatives to the full geometric estimation pipeline, such as using the video model itself to estimate canonical trajectories, amortizing canonicalization into the generation process, or using optical flow for canonicalization instead of depth-based unprojection.

---

The Method Is Evaluated Exclusively on Relatively Small, Hand-Selected Test Sets (50 Videos Each) with No Reported Statistical Significance, and Results May Not Be Robust

The assumption or constraint. All quantitative evaluations in the paper use test sets of 50 videos: 50 manually selected from DynPose-100K, 50 selected from WISA, and 50 collected for Cooking. The paper does not report confidence intervals, standard deviations, or statistical significance tests for any metric comparison. The human evaluation involves 11 participants evaluating 30 examples (330 responses), which is a limited sample for drawing conclusions about perceptual preference.

The selection criteria for the test sets are described qualitatively but not quantitatively. DynPose-100K videos are selected for "strong viewpoint changes and clear object interactions." WISA videos are selected "from categories including collision, deformation, elasticity, liquid, and rigid-body motion." Cooking videos are collected from YouTube "containing rich hand-object interactions." These selection criteria introduce an unspecified degree of curation bias—videos are chosen because they exhibit the phenomena the method is designed to handle, which likely overestimates performance relative to a random or representative sample from the same datasets.

The consequence. With 50 test examples, individual outlier videos can significantly affect aggregate metrics. A single video where geometric estimation fails catastrophically (producing a large EPE) can shift the mean considerably. Without confidence intervals, observed differences between methods—for instance, MoRight's EPE of 4.27 versus Motion Prompting's 4.25 on Cooking—cannot be distinguished from sampling noise. The paper's claim that MoRight "achieves the best overall performance" on Cooking (Table 1) is based on a pattern of small numerical advantages across multiple metrics, none of which is individually tested for significance.

The human evaluation sample size (11 participants, 30 examples) is below typical standards for rigorous perceptual studies. With 11 raters, inter-rater reliability metrics (e.g., Fleiss' kappa) would have wide uncertainty, and the 53.5–55.9% preference rates—while clearly above the 25% baseline competitors—may not generalize beyond this specific set of raters and examples. The paper does not report whether participants had relevant expertise (computer vision researchers, graphics artists, lay users), which can substantially affect preference patterns.

What evidence exists in the paper. All metric tables (Tables 1–3) report point estimates without uncertainty quantification. Table 3's ablation study has seven rows with differences as small as 0.10 on FID (e.g., 39.94 for the full model vs. 39.83 for coarse granularity), 0.01 on SSIM, and 0.01 on PC. Without any measure of variance, it is impossible to determine whether these differences are meaningful or fall within the noise floor of the evaluation pipeline (which itself involves stochastic generation, ViPE pose estimation noise, and AllTracker tracking noise). The human evaluation (Fig. 8) reports aggregate percentages but no confidence intervals, no breakdown by example, and no inter-rater agreement statistics.

Mitigation status. The paper does not address this limitation. The evaluation protocols are standard for the video generation literature—50-example test sets and reporting point estimates without significance tests is common practice in papers evaluating on DynPose-100K and similar benchmarks. However, for a paper whose contributions include claiming superiority over methods that use "privileged" inputs, the lack of statistical rigor weakens the strength of those claims. Reproducing the evaluation with larger test sets, reporting confidence intervals (via bootstrap or multiple random seeds), and conducting the human study with more participants would substantially strengthen the evidence. The paper does not release the specific 50-example subsets used, making independent replication or re-evaluation with uncertainty quantification difficult.

---

Inference Latency (~15 Minutes Per Video on an A100) Makes Interactive Use Impractical Despite the Interactive GUI, and Latency Scaling with Video Length Is Not Characterized

The assumption or constraint. The paper presents an interactive GUI (Fig. 11) that lets users draw motion trajectories, specify camera paths, and generate videos. However, generating one video requires approximately 15 minutes on a single A100 GPU using 35 diffusion steps. The paper does not report latency for the baselines, does not ablate the number of diffusion steps versus quality, and does not characterize how latency scales with video length (number of frames) or with the complexity of the camera trajectory.

The dual-stream architecture inherently doubles the per-step computation relative to a single-stream video diffusion model: two latent videos are concatenated along the temporal dimension (producing a $2\hat{T}$-frame latent) and processed by the same DiT. The cross-view self-attention operates over twice as many tokens as single-stream attention, with quadratic complexity in the total token count. This means the 15-minute latency is not simply the cost of a high-quality video generation model—it is structurally higher than what a single-stream model with comparable per-frame quality would require, because it is simultaneously generating two videos.

The consequence. An "interactive" system that requires 15 minutes of GPU computation per output is not interactive in any conventional sense. A user who draws trajectories, clicks "generate," and receives a result 15 minutes later cannot iteratively refine their input—the feedback loop is too slow for creative exploration. If the generated video has an error (incorrect interaction, unnatural motion, hallucinated content as in Fig. 9), the user must wait another 15 minutes for a corrected version. This fundamentally limits the practical utility of the GUI to batch-oriented, non-interactive use cases—a user might queue up several generations and review them later—which undermines the paper's framing of the system as enabling "interactive motion generation" (Fig. 1 caption).

The latency also complicates comparison with baseline methods. If baselines (ATI, WanMove, Motion Prompting) generate videos faster because they are single-stream models, then MoRight's quality advantages come at a latency cost that is not discussed. A practitioner choosing between methods would need to weigh the improved controllability and interaction quality against the 2× or more increase in generation time.

What evidence exists in the paper. The paper states the 15-minute figure in Section 4.1: "generating one video takes approximately 15 minutes on a single A100 GPU." No other latency measurements are reported. The 35 diffusion steps are specified as the inference configuration but not ablated—the paper does not test whether 25 steps would produce acceptable quality at lower latency, or whether 50 steps would improve quality enough to justify the additional time. The dual-stream architecture's computational overhead relative to single-stream generation is not quantified.

Mitigation status. The paper does not discuss latency as a limitation, does not propose strategies for reducing it (e.g., distillation, fewer diffusion steps, staggered denoising where the canonical stream is denoised with fewer steps), and does not compare latency across methods. The 15-minute figure appears as an implementation detail rather than as a design tradeoff to be analyzed. Future work that follows naturally would include: measuring how generation quality degrades with fewer diffusion steps to characterize the quality–latency Pareto frontier; exploring whether the canonical stream can be denoised with fewer steps or at lower resolution without degrading target-stream quality; and investigating whether the cross-view attention can be made more efficient (e.g., sparse attention, separate attention for cross-view and within-view tokens) to reduce the per-step cost of dual-stream generation.

7. Implications and Future Directions

How This Work Changes the Landscape

MoRight introduces a representational reframing of motion-controlled video generation rather than an incremental improvement to existing trajectory-based methods. The conceptual shift is from treating motion control as a "render pixels along specified paths" problem to treating it as a "transfer causally-structured dynamics across viewpoints" problem. This reframing has three concrete consequences for how the field thinks about controllable generation.

First, the canonical-view anchor establishes a new default assumption for motion specification. Every prior trajectory-based method—Motion Prompting, ATI, WanMove—operates on the assumption that motion control signals should live in the output video's pixel space. MoRight demonstrates that this assumption creates an unnecessary and fundamentally ill-posed disentanglement problem: the model must separate object motion from camera motion when both are encoded in the same 2D displacement signal. By moving motion specification to a canonical reference frame (the first frame, where camera pose is identity) and handling camera motion as a viewpoint transformation applied to those dynamics, the paper eliminates the ambiguity at the representation level rather than trying to resolve it through more powerful conditioning. This is not a new architecture bolted onto the existing paradigm—it is a different decomposition of the problem that makes the dual-stream architecture the natural implementation rather than an arbitrary design choice.

The evidence that this is a genuine conceptual shift rather than an engineering convenience comes from the ablation removing the fixed-view branch (Table 3, row 2): when object and camera motion are jointly encoded without the canonical anchor, EPE spikes from 4.27 to 14.30—the model essentially cannot recover clean object dynamics. This 3.3× degradation is not a marginal accuracy loss; it indicates that the canonical-view formulation is doing the heavy lifting for disentanglement, and the cross-view attention is the mechanism that transfers dynamics between views, not a mechanism that discovers dynamics from entangled signals.

Second, the work reframes causal reasoning in video generation away from external simulators and toward self-supervised learning from partial motion observations. The two dominant paradigms for incorporating physics into video generation—external physics engines (PhysGen, PhysGen3D, MotionCraft) and language-model-mediated reasoning (GPT4Motion, LLM-Grounded Video Diffusion)—both treat causal structure as something that comes from outside the video model. Physics engines hard-code physical laws and require domain-specific simulation modules. Language model pipelines reason about consequences in text and suffer from a precision bottleneck when translating text to pixel-level dynamics.

MoRight's motion dropout mechanism shows that causal structure can be learned from video data alone, through a training objective that is conceptually simple: randomly withhold one motion component (active or passive) and supervise on the full video. The model is never told what gravity is, what collision does, or what "pushing" means—it discovers action–consequence correlations by being forced to predict missing dynamics from partial observations. This is a bidirectional, pixel-level causal reasoning capability that emerges from the training objective rather than being engineered into the architecture, and it operates in the same visual space where generation occurs, eliminating the text-to-geometry precision loss that plagues language-mediated approaches.

The evidence for this claim is nuanced. The PC score improvement (0.76 vs. 0.75 on WISA, 0.88 vs. 0.87 on Cooking) is directionally consistent but modest in absolute magnitude—these are 0.01 differences that could fall within metric noise without statistical significance testing. The stronger evidence is the human evaluation (Fig. 8), where MoRight is preferred 54.6% of the time on motion realism despite baselines receiving full active+passive privileged 3D trajectories while MoRight receives only first-frame active trajectories. This gap is large enough to be unlikely under a null hypothesis of equal quality, and it specifically measures human perception of physical plausibility rather than pixel-level fidelity. The qualitative results (Figs. 7, 13) showing plausible cloth deformation from hand motion (forward reasoning) and plausible hand motion from ball trajectories (inverse reasoning) provide existence proofs, though not statistical guarantees.

Third, and perhaps most practically significant, the work demonstrates that weak inputs can outperform strong inputs when the architecture is designed to complete rather than replay. Every baseline receives privileged information that MoRight does not: dense per-pixel trajectories across all frames, separate foreground and background signals, full active and passive motion, and (in the case of ATI and WanMove) 3D trajectories projected to pixel-aligned per-frame tracks. MoRight receives first-frame sparse trajectories (1500 tracks, single motion component). The fact that MoRight matches or exceeds these baselines (best EPE on DynPose-100K, best overall on Cooking, best FID/FVD/PC on WISA) while operating on dramatically weaker inputs establishes that architectural design—the canonical-view anchor for disentanglement, motion dropout for causal completion—can substitute for privileged geometric supervision.

This has a concrete implication for the field's research priorities: future work on controllable video generation should invest more heavily in architectures that complete underspecified inputs rather than in methods that require increasingly dense and privileged control signals. The trajectory toward methods that demand per-frame 3D trajectories with full scene decomposition is a path toward systems that are accurate but unusable—requiring inputs that no human can provide and no practical pipeline can reliably extract. MoRight's trajectory toward sparse, first-frame, single-component inputs is a path toward systems that are actually deployable, and the results show that this usability does not come at the cost of accuracy.

What directions become more or less attractive. The paper makes three research directions substantially more attractive: (1) canonical-space control for any generative task where the control signal and rendering viewpoint are separable (3D scene editing, novel view synthesis of dynamic scenes, interactive world models); (2) self-supervised causal reasoning through asymmetric supervision, applied to other domains where action–consequence relationships exist in training data but are not explicitly labeled (robotics video, egocentric video, instructional video); and (3) weak-to-strong input generalization, where models are deliberately trained with partial or degraded conditioning to learn completion capabilities that make them usable with lightweight user inputs.

The paper makes two directions less attractive: (1) trajectory-based control methods that require per-frame dense pixel tracks as a universal control representation, since the paper demonstrates that this representation entangles camera and object motion in ways that are fundamentally limiting—no amount of model capacity resolves the inherent ambiguity of a control signal that does not separate viewpoint change from physical dynamics; and (2) purely language-mediated causal reasoning pipelines for spatially precise tasks, since the text→geometry precision loss documented in the paper's related work (Section 2.3) is a structural limitation of language as an intermediate representation for pixel-level dynamics, and MoRight's purely visual causal reasoning demonstrates a viable alternative.

Reconciling prior contradictions. The paper helps reconcile a tension in the video generation literature between methods that achieve strong control accuracy (trajectory-based methods like ATI and WanMove, which follow provided trajectories faithfully) and methods that produce physically plausible dynamics (physics-engine-based methods and LLM-mediated methods, which reason about interactions but with spatial imprecision). Prior to MoRight, these were treated as largely independent axes—you could have accurate trajectory following or physically plausible interactions, but not both simultaneously without privileged information and external simulators. MoRight shows that both can be achieved within a single learned model, by separating the control problem (disentangled camera-object trajectories via canonical anchor) from the reasoning problem (causal completion via motion dropout) at the architectural level while training them jointly. The canonical anchor provides the accuracy (faithful trajectory following by design, since object motion is defined in a clean reference frame), and the motion dropout provides the physical plausibility (causal dynamics learned from data). This resolves the apparent tradeoff by showing it was an artifact of trying to solve both problems with a single representation.

Follow-Up Research This Work Enables

Disentanglement stress-testing with systematic camera motion scaling. The paper evaluates on DynPose-100K (highly dynamic cameras) and Cooking (moderate camera motion), but provides no controlled experiment measuring how disentanglement accuracy degrades as camera motion becomes more extreme. A natural follow-up would take a fixed set of object motions (e.g., 20 hand-object interactions from the Cooking dataset) and systematically apply camera trajectories of increasing magnitude—rotations from 5° to 90°, translations from 10 cm to 2 m equivalent, zoom factors from 1.1× to 4×—and measure EPE and camera accuracy at each level. This would characterize the generalization envelope of the cross-view attention: does it degrade gracefully with camera motion magnitude (suggesting it has learned geometric correspondence principles) or does it fail catastrophically beyond some threshold (suggesting it has memorized a finite set of camera trajectories from the training data)? A negative result—catastrophic failure at moderate camera motions outside the training distribution—would indicate that the canonical-view formulation, while effective, does not truly learn viewpoint-invariant dynamics and would motivate hybrid approaches combining learned attention with explicit geometric constraints.

Adversarial evaluation of causal reasoning on counterfactual interactions. The motion dropout mechanism trains the model to complete missing motion components, but it is unclear whether the model has learned genuine physical principles or statistical co-occurrence patterns in the training data. A strong follow-up would construct a counterfactual interaction benchmark: take existing interaction videos (from WISA or Cooking) and modify them to create physically impossible scenarios—a hand pushing leftward but the object sliding rightward (violating contact dynamics), a ball striking a stack of blocks but the blocks moving upward (violating gravity), a teapot tilting but no liquid pouring (violating fluid dynamics). Testing whether MoRight's causal reasoning module "corrects" these impossible scenarios (generating physically plausible dynamics despite the impossible conditioning) versus faithfully reproducing them would distinguish between learned physical principles and statistical pattern matching. If the model generates physically correct dynamics even when conditioned on impossible trajectories, it has internalized causal constraints that override surface-level trajectory conditioning—a strong test of genuine causal understanding. If it faithfully reproduces the impossible trajectories, its causal reasoning is superficial and would motivate fundamentally different training objectives (e.g., explicit physical constraint satisfaction, contrastive training on possible vs. impossible interactions).

Scaling laws for cross-view attention as a function of paired training data volume. The paper trains on 43K synthetic paired videos, supplemented with real single-view data. It is unclear whether this is sufficient, barely sufficient, or overkill for learning geometric correspondence. A scaling study would train MoRight variants on 1K, 5K, 10K, 43K, and (if feasible) 100K+ paired videos, measuring camera accuracy and object motion EPE on a held-out test set. The key question is whether disentanglement accuracy follows a power-law scaling trend (suggesting more paired data would yield continued improvements and the current performance is data-limited) or plateaus at 43K (suggesting the model saturates its capacity to learn geometric correspondence from this data source and improving synthetic data quality is a higher priority than quantity). This would directly inform resource allocation for future work: should effort go toward generating more paired data, better paired data, or better architectures that require less paired supervision?

Active/passive dropout ratio sweep with inverse reasoning accuracy as the target metric. The paper uses a fixed 80/20 active/passive dropout ratio and observes that inverse reasoning (passive input → active inference) is substantially worse than forward reasoning (EPE 7.27 vs. 4.27, Table 3). A systematic sweep of this ratio—from 50/50 (equal forward and inverse) to 95/5 (nearly all forward)—would characterize the tradeoff: does improving inverse reasoning (by training with more balanced dropout) degrade forward reasoning, or are the two capabilities complementary? The key measurement would be EPE and PC on a bidirectional test set where each interaction is evaluated in both directions. If a 50/50 split achieves EPE of ~5.5 in both directions (trading ~1.3 EPE in forward for ~1.8 improvement in inverse), the optimal ratio depends on the application. If inverse reasoning never approaches forward reasoning accuracy regardless of the ratio, the model architecture has a fundamental asymmetry that dropout alone cannot overcome, and dedicated inverse-reasoning architectures (e.g., explicit inverse dynamics models, amortized inference over action distributions) would be necessary.

Cross-view attention interpretability through attention map analysis. The paper hypothesizes that cross-view attention learns implicit geometric correspondence—target-stream tokens attend to canonical-stream tokens representing the same physical scene point, thereby transferring motion information across views. This hypothesis is untested. A follow-up would extract attention maps from the cross-view self-attention layers during generation, project both canonical and target tokens back to their respective image planes, and measure whether high-attention token pairs correspond to geometrically consistent point correspondences under the known camera transformation. For scenes where ground-truth depth and camera poses are available (from the rendered SyncCamMaster data or from the synthetic paired videos where the camera trajectory is known), one could compute the epipolar error: for a target token at pixel $(u_t, v_t)$ attending to a canonical token at pixel $(u_c, v_c)$, what is the distance from $(u_t, v_t)$ to the epipolar line of $(u_c, v_c)$ under the known relative pose? If attention maps consistently respect epipolar geometry, the model has learned genuine geometric correspondence. If attention is diffuse or respects semantic rather than geometric similarity (e.g., "cup tokens" attend to "cup tokens" regardless of 3D position), the cross-view transfer is operating through appearance matching rather than geometric reasoning, which would motivate incorporating explicit geometric constraints (epipolar attention bias, depth-conditioned attention masking) into the architecture.

Extension to multi-object causal graphs beyond binary active/passive decomposition. The paper decomposes motion into exactly two categories: active (one agent) and passive (everything else). Real-world interactions often involve chains of causality—a hand pushes a cup, the cup strikes a plate, the plate slides and knocks over a salt shaker. The current binary decomposition would label the hand as active and everything else as passive, but this collapses a multi-step causal chain into a single consequence step. A natural extension would introduce a finer-grained causal graph: identify the primary agent (hand), the direct patient (cup), secondary patients (plate), and tertiary patients (salt shaker), and train the model to predict causal chains with varying levels of dropout (e.g., provide only the hand trajectory and require prediction of cup + plate + salt shaker, or provide hand + cup and require plate + salt shaker). This would test whether motion dropout as a training mechanism scales to multi-step causal reasoning, and whether the quality of predicted consequences degrades with causal distance (does the salt shaker's predicted trajectory become less accurate the further it is from the causal source?). The WISA dataset, with its categories including collision chains, would be a natural testbed. A negative result—prediction quality degrading rapidly with causal distance—would indicate that the binary active/passive decomposition is not an arbitrary simplification but a fundamental limitation of single-step dropout training.

Practical Applications and Downstream Use Cases

Data augmentation for robot manipulation policies through forward and inverse reasoning. Training visuomotor policies for robotic manipulation requires large amounts of interaction data showing diverse actions and their consequences. MoRight's forward reasoning (active → consequence) can generate synthetic training data from sparse human demonstrations: given a single frame of a robot scene and a trajectory for the gripper, the model generates a video of the full interaction including how objects in the scene respond. The inverse reasoning (consequence → active) can generate plausible action trajectories from observed outcomes, useful for learning from passive observation (e.g., watching human videos of tasks and inferring what robot actions would produce similar object motion). The 15-minute inference time per video limits real-time applications, but for offline data generation—generating thousands of interaction variants overnight from a small set of seed demonstrations—the latency is not a bottleneck. The key practical value is reducing the need for expensive real-world robot interaction data: a single human demonstration video of "push cup to the right" could generate 100 synthetic variants with different camera angles, different initial cup positions, and different push speeds, each with physically plausible object dynamics, for less cost than collecting 100 real demonstrations. The paper's results on Cooking (EPE 4.27, PC 0.88) suggest the generated dynamics are accurate enough for policy training, though a dedicated sim-to-real evaluation would be needed to confirm transfer.

Interactive storyboarding and pre-visualization for film and animation. In film pre-production, directors and cinematographers need to explore how scenes will look from different camera angles before committing to expensive physical production or full 3D animation. MoRight's disentangled camera–object control is directly applicable: an artist takes a single concept image (e.g., a character reaching for an object on a table), draws a rough trajectory for the character's hand motion, and then explores multiple camera angles—orbit left, zoom in, pan right—each generating a video of the scene from that viewpoint with consistent object dynamics. The 15-minute generation time per 2–5 second clip (at 24–30 fps, a 5-second clip is 120–150 frames, though the paper's frame counts are not specified) is acceptable for pre-visualization workflows where directors review options between shots rather than in real time. The key advantage over existing 3D pre-vis pipelines is removing the need for full 3D scene reconstruction and rigging: MoRight works from a single 2D image with hand-drawn trajectories, bypassing the time-consuming modeling, texturing, and animation setup that makes traditional pre-vis expensive. The human evaluation preference rates (53.5% controllability, 55.9% photorealism) suggest the generated outputs are visually competitive for this use case, though frame-to-frame consistency and temporal artifacts (visible in the limitation examples, Fig. 9) would need to be addressed for production-quality applications.

Interactive physics education and science communication through explorable video demonstrations. Educational content explaining physical principles—collisions, fluid dynamics, mechanical interactions—benefits from showing the same phenomenon from multiple viewpoints and with different initial conditions. MoRight enables an interactive educational tool where an instructor takes a single image of a physics demonstration (e.g., a pendulum, a collision setup, a fluid container), draws the active motion (e.g., the pendulum's initial swing, a ball's trajectory toward a target), and the system generates the full interaction including all consequence dynamics (the pendulum's subsequent swings, the collision aftermath) from any camera angle. Students could explore "what if" scenarios by modifying the active motion (what if the ball is struck harder? what if the pendulum starts from a higher position?) and immediately generating the consequences. The 15-minute latency limits classroom interactivity, but for asynchronous learning materials—generating a library of pre-computed demonstrations covering a range of initial conditions and viewpoints—the latency is acceptable. MoRight's PC score improvements over baselines (0.76 vs. 0.73–0.75 on WISA) suggest the generated physics is more reliable than alternative video generation methods, though it is not at the level of a dedicated physics simulator and would need accuracy disclaimers or verification against known physical laws for formal educational use.

When to Prefer This Method

The paper explicitly positions MoRight against trajectory-based methods (Motion Prompting, ATI, WanMove) that require privileged dense per-pixel trajectories across all frames, and against cascaded approaches that handle camera and object motion in separate stages. The tradeoffs are clear from the experimental results in Tables 1–3, and they translate into a concrete set of deployment conditions:

• Prefer MoRight over trajectory-based methods when the user can provide only sparse first-frame trajectories (drawn as strokes on the input image) and the application requires independent camera viewpoint control. The canonical-view formulation eliminates the need for per-frame dense tracking inputs—MoRight achieves best EPE on Cooking (4.27) and best on DynPose-100K (7.64) using first-frame reprojected tracks, matching or exceeding methods that require privileged future-frame signals. The human evaluation preference rates (53.5% controllability, 54.6% motion realism) further indicate that users perceive better control and interaction quality from the weaker-input system.

• Prefer MoRight over cascaded (generate-then-camera-control) pipelines when the application requires joint optimization of object dynamics and camera viewpoint. The ablation comparing MoRight's dual-stream joint denoising against a cascaded approach (Table 3, row 1) shows dual-stream achieves better camera accuracy (rotation error 2.16° vs. 2.69°, translation 10.11 vs. 11.50) and better object tracking (EPE 4.27 vs. 5.05), attributed to error accumulation in the two-stage pipeline. Dual-stream joint denoising allows the target stream to access clean canonical-stream motion information throughout the denoising trajectory rather than receiving a potentially artifact-ridden intermediate video.

• Prefer MoRight over physics-engine-based methods when the interaction types are diverse (spanning rigid-body, deformable, fluid, and articulated dynamics) and building dedicated simulators for each type is impractical. Motion dropout learns causal structure from data without requiring domain-specific physics models, making it applicable to any interaction present in the training distribution. The WISA results (PC 0.76 across collision, deformation, elasticity, liquid, and rigid-body categories) demonstrate this generality, though the 0.01 PC improvement over baselines is modest and the model's accuracy on out-of-distribution interaction types is untested.

• Prefer trajectory-based or physics-engine-based methods over MoRight when latency is critical (single-stream generation is faster than dual-stream, and physics simulators are real-time) or when the application requires guaranteed physical accuracy rather than learned statistical plausibility. MoRight's 15-minute inference time on an A100 is substantially slower than single-stream video generation (exact baseline latencies are not reported but would be approximately half, since they process one video stream rather than two), and the limitation examples (Fig. 9: objects disappearing, merging, unnatural motion) show that learned causal reasoning does not provide the reliability guarantees of a verified physics simulator for safety-critical applications.