Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation

ArXiv: 2512.17040

🎯 Pitch

Forget depth maps—InfCam achieves state-of-the-art camera-faithful video generation by feeding pre-computed, noise-free rotational warps directly into the diffusion model, bypassing the cascading errors that cripple reprojection-based approaches. It slashes rotation error by half compared to prior work, proving that precise view synthesis is possible without any depth estimation at all.

---

1. Executive Summary

This paper introduces InfCam, a depth-free camera-controlled video-to-video generation framework that achieves high pose fidelity by decomposing novel-view synthesis into known rotation and unknown translation components. The system integrates an infinite homography warping module (which encodes 3D camera rotations directly in the 2D latent space of a video diffusion model, conditioning on noise-free rotational information while the residual parallax term is learned end-to-end) and a data augmentation pipeline (which transforms existing synthetic multiview datasets into sequences with diverse trajectories and focal lengths). Experimental results on the AugMCV and WebVid datasets using the Wan2.1 backbone demonstrate that InfCam outperforms both reprojection-based baselines (TrajectoryCrafter, GEN3C) and trajectory-conditioned baselines (ReCamMaster) in camera-pose accuracy and visual fidelity, reducing rotation error to 3.162° and translation error to 0.438 on real-world videos while establishing that such improvements are achievable without relying on external depth estimation—whose inaccuracies fundamentally constrain reprojection-based approaches.

2. Context and Motivation

The Core Problem: Camera-Controlled Novel-View Video Generation Remains Unreliable

The fundamental challenge this paper addresses is deceptively simple: given a single source video and a target camera trajectory, generate a new video that faithfully follows that trajectory as if the original scene had been filmed from the specified moving camera. This task — known as camera-controlled novel-view video generation — has attracted growing interest because it promises to democratize cinematic camera control in post-production, eliminating the need for costly reshoots and enabling creative visual effects that would otherwise require specialized equipment or multiple camera rigs.

The task is genuinely hard because it requires the model to reason about occluded 3D geometry from limited observations. When you move a camera through a scene, objects that were hidden behind foreground elements become visible, lighting interactions change with viewpoint, and parallax effects shift the apparent positions of objects at different depths. A model must therefore implicitly or explicitly reconstruct the 3D structure of the scene to synthesize plausible novel views — all while maintaining temporal consistency across frames and preserving the appearance of the original video.

The paper frames this as a problem where existing solutions are fundamentally constrained, with each approach hitting a different wall:

• Reprojection-based methods hit the wall of depth estimation accuracy.
• Trajectory-conditioned methods hit the wall of dataset diversity and bias.
• Both walls exist for structural reasons, not as issues that can be fixed by simply scaling up existing approaches.

Why This Problem Matters

The paper is motivated by practical and scientific considerations:

Practical impact for content creation. The ability to change camera trajectories in post-production is described as "a highly sought-after video editing technique" (Section 1). In filmmaking, this capability would let directors experiment with different camera movements without reshooting scenes. In virtual production, it would enable dynamic viewpoint changes for previsualization. In consumer applications, it could power features akin to Apple's cinematic mode or advanced video stabilization, but generalized to fully synthetic camera trajectories rather than just subtle adjustments.

A stress test for video generation models. Camera-controlled novel-view synthesis serves as a rigorous test of a model's 3D understanding. A model that can synthesize correct novel views must have internalized the geometric structure of the scene — depth ordering, occlusion relationships, and projective geometry. This makes the task both an application and a diagnostic for whether video diffusion models truly understand 3D scenes versus merely memorizing 2D appearance patterns.

Complementarity to other video control tasks. Unlike style transfer, object manipulation, or text-conditioned generation, camera control requires precise geometric reasoning. Progress here contributes to a broader goal of making video generation models controllable along multiple axes simultaneously — style, content, motion, and viewpoint — each of which stresses different model capabilities.

Existing Approaches and Their Structural Limitations

The paper identifies two dominant paradigms in the literature, each with fundamental weaknesses:

Reprojection-Based Approaches (e.g., GEN3C, TrajectoryCrafter)

These methods follow a three-stage pipeline: (1) estimate per-frame depth from the source video using an external depth predictor, (2) unproject the video frames into a 3D point cloud and reproject them along the target camera trajectory to create a "warped" conditioning signal, and (3) feed this reprojected signal into a video diffusion model that inpaints the newly exposed (unprojected) regions.

The paper's critique of this approach is sharp and centers on a cascading error argument (Section 3, "Reprojection-based Approach"):

"any inaccuracy in the predicted depth results in an unreliable condition being provided to the generative model, leading to artifacts in the synthesized image"

The problem is not just that depth estimation is imperfect — it's that the errors are non-recoverable. Because depth estimation runs as a pre-processing step with no gradient flow back through the depth predictor during training, incorrect reprojection positions become fixed conditions that the diffusion model must somehow compensate for. The paper explicitly states:

"because no gradients flow through the depth estimation network, the inaccurate reprojection position x′ remains fixed as a condition throughout training. Although the model can be trained to compensate for these errors, the challenge lies in handling noisy conditions, where it is unclear which parts are corrupted and to what extent"

This is a crucial insight: it's not just that reprojection errors exist, but that the model faces an ill-posed denoising problem — it must learn to identify which regions of its conditioning signal are reliable and which are corrupted, without any explicit signal about where the errors occurred. The paper argues this constraint places a hard ceiling on reprojection-based performance that cannot be overcome by better depth estimators alone, because depth will never be perfect in practice.

The evidence for this ceiling is visible in the paper's own results: TrajectoryCrafter, a reprojection-based method, shows artifacts due to inaccurate reprojection (Figure 5, orange box highlights), and GEN3C fails to transform the viewpoint of the initial frame entirely (Section 5.2, Figure 5), which the authors attribute to a training bias in its image-to-video backbone toward preserving the first frame — an observation consistent with findings in GCD (Van Hoorick et al., 2024).

Trajectory-Conditioned Approaches (e.g., SynCamMaster, ReCamMaster)

These methods train video generation models directly on paired datasets of camera trajectories and corresponding videos, learning the mapping from trajectory specifications to viewpoint changes. The model is conditioned on camera pose parameters at training time, and at inference time, it should generate novel views consistent with the specified trajectory.

The paper's critique focuses on dataset-induced bias as a structural limitation (Section 3, and more concretely in Section 4.2.1):

"when the training data contain biased input–output video pairs, the network is likely to internalize these biases, leading to degraded generalization performance"

The specific bias the paper identifies is insidious and dataset-dependent. In the MultiCamVideo dataset (which ReCamMaster uses for training), all 10 cameras within each scene share identical starting positions. This means that during training, the source and target videos always have the same first frame. The model learns a shortcut: when asked to generate a novel view from a different camera pose, it can simply reproduce the source video's first frame rather than performing genuine viewpoint transformation, because it has never seen a training example where the first frame of the source and target differ.

The paper provides direct empirical evidence for this bias (Section 5.2):

"Across both datasets, ReCamMaster consistently preserves the initial frame of the source video, which we attribute to a bias in its training data, where all paired source-target trajectories originate from the same initial frame"

Even when the authors attempt to patch this by prepending interpolated frames to guide ReCamMaster toward a different initial viewpoint (the "ReCamMaster w/ Interp" variant), the results "still exhibit alignment errors, indicating that interpolation alone cannot fully address its trajectory bias" (Section 5.2).

A subtler bias exists in the intrinsic parameters as well. In MultiCamVideo, each scene uses a fixed focal length across all 10 cameras. This means the model never sees examples where source and target videos have different fields of view, so it learns to simply copy the source's focal length rather than adjusting to target specifications.

A Deeper Limitation: Depth as a Bottleneck in Both Paradigms

The paper's motivation goes beyond critiquing individual methods. It identifies a shared vulnerability: the reliance on explicit 3D geometry (via depth estimation) as a conditioning signal.

In reprojection-based methods, the vulnerability is direct — depth estimation errors propagate to the final output. But even trajectory-conditioned methods may be forced to learn implicit depth representations from limited trajectory data, making them sensitive to the diversity of depth structures in their training set. Neither paradigm provides a way to condition on known, noise-free geometric transformations while learning the unknown residual from data.

This motivates the paper's central technical question: Can we decompose the novel-view synthesis problem so that the geometric components we know with certainty (camera rotation, from the specified trajectory) are provided as clean conditioning, while the uncertain components (translation-induced parallax, which depends on unknown scene depth) are learned end-to-end from data?

How This Paper Positions Itself

The paper introduces infinite homography warping as a principled decomposition that addresses the limitations of both prior paradigms simultaneously (Section 3, "Infinite Homography-based Approach").

The key insight comes from Eq. (2): the reprojection of a pixel from a source camera to a target camera can be expressed as:

$x' = \underbrace{K_tRK_s^{-1}}_{\text{infinite homography } H_\infty} x \;+\; \underbrace{K_t t / Z}_{\text{parallax term}}$

The infinite homography $H_\infty = K_t R K_s^{-1}$ is the homography induced by the plane at infinity — it depends only on camera rotation and intrinsics, not on scene depth or translation. It is noise-free because rotation and intrinsics are specified exactly by the user-provided camera trajectory. The parallax term $K_t t / Z$ captures the translation-induced displacement, which depends on the unknown per-pixel depth $Z$.

The paper's positioning is that by conditioning the diffusion model on images warped by the clean $H_\infty$, and learning the residual parallax end-to-end, the model can:

1. Avoid dependency on external depth estimation — depth is never explicitly computed; instead, the model implicitly learns to predict the parallax needed to reconcile the $H_\infty$-warped image with the target view.
2. Exploit geometric constraints — because parallax is restricted to lie on epipolar lines between the epipole $e'$ and the $H_\infty$-mapped point $x_\infty$, the model has a spatially bounded search space (the "yellow segment on the epipolar line $l'$" in Figure 2a). This is not just a computational convenience; the paper argues it "helps the model to achieve higher camera pose fidelity with reduced search space."
3. Benefit from end-to-end learning — unlike the reprojection pipeline where depth errors are frozen during training, the end-to-end framework "enables the network to implicitly refine the predicted depth $Z$ toward the ground truth as it learns to produce accurate parallax" (Section 3).

This positions InfCam as occupying a middle ground between the two paradigms: it is geometry-aware (like reprojection methods) but depth-free (avoiding their primary failure mode), and it is learned end-to-end (like trajectory-conditioned methods) but conditioned on noise-free rotational information (avoiding the need to learn rotation from biased data).

The second positioning axis is the data augmentation strategy. The paper explicitly frames its trajectory and intrinsic augmentation (Section 4.2) as a solution to the dataset biases that cripple trajectory-conditioned methods. By constructing augmented sequences where source and target videos have different starting frames (through trajectory reversal and concatenation) and different focal lengths (through resizing and center cropping), the model is forced to learn genuine novel-view synthesis rather than first-frame copying. The AugMCV dataset expands from 13,600 to 47,600 videos through this augmentation, providing the scale and diversity needed for robust generalization.

Reconciling with Prior Observations

The paper's findings implicitly reconcile conflicting signals in the literature. GEN3C's failure to transform initial frames (Section 5.2) aligns with GCD's observation of first-frame preservation bias in image-to-video backbones. ReCamMaster's dataset biases explain why trajectory-conditioned models may perform well on training-like trajectories but degrade on out-of-distribution starting poses. The paper's framework provides a unified explanation: both paradigms fail when the geometric information needed for novel-view synthesis is either corrupted (reprojection errors) or absent from training (dataset biases). InfCam addresses both by separating clean geometric conditioning from learned residual estimation, and by constructing training data that eliminates the shortcuts that caused prior methods to overfit.

3. Technical Approach

3.1 Reader Orientation

InfCam is a video diffusion model that takes a source video and a target camera trajectory as input, and generates a new video showing the same dynamic scene as if it had been filmed from the moving target camera. The system solves the problem of camera-controlled novel-view synthesis by decomposing the geometric transformation into two components: a noise-free rotational warp (which the model receives as clean conditioning) and a depth-dependent parallax displacement (which the model learns to predict end-to-end from data), thereby avoiding the cascade of errors that plagues methods relying on explicit depth estimation while simultaneously eliminating the dataset biases that cause trajectory-conditioned methods to collapse to first-frame copying.

3.2 Big-Picture Architecture (Diagram in Words)

The InfCam system has five major components, all built around the frozen pretrained Wan2.1 text-to-video diffusion model:

1. 3D VAE Encoder — compresses source and target video frames into a lower-dimensional latent space where the diffusion process operates.
2. Warping Module — takes the source latent of the first frame, applies the infinite homography derived from the target camera's rotation and intrinsics, and produces a warped latent that encodes what the first frame would look like under pure rotation (no translation). Camera embeddings encoding translation and intrinsics are then added.
3. Camera Encoder — a learned linear layer that maps 16-dimensional camera parameter vectors (flattened rotation matrix, translation vector, focal lengths, principal point) to a d-dimensional embedding space matching the diffusion model's hidden dimension. This encoder is shared across all DiT blocks.
4. Homography-Guided Self-Attention Layers — inserted into each DiT block of the frozen Wan2.1 backbone. These layers take three concatenated latent streams (source, target, and warped), each augmented with their corresponding camera embeddings, and perform per-frame attention to enable rotation-aware reasoning while maintaining temporal alignment between corresponding timestamps in source and target.
5. Data Augmentation Pipeline (offline preprocessing) — transforms the MultiCamVideo dataset by reversing and concatenating trajectory pairs (to break first-frame identity bias) and randomly augmenting focal lengths (to break intrinsic-copying bias), producing the AugMCV training set.

Information flows as follows: a source video and target trajectory are provided → the 3D VAE encodes the source video into latents; in parallel, the warping module computes infinite-homography-warped latents from the first frame → camera parameters are encoded into embeddings via the shared camera encoder → at each DiT block, source, target (noised), and warped latents are concatenated with their camera embeddings and processed by the homography-guided attention layer → the warped latent is discarded, and source and target latents proceed through the frozen Wan2.1 self-attention and cross-attention layers → the diffusion model iteratively denoises the target latent → the 3D VAE decoder reconstructs the final output video.

3.3 Roadmap for the Deep Dive

• First, the core mathematical decomposition in Eq. (2) — infinite homography plus parallax — because every architectural decision flows from this equation. Understanding why this decomposition is geometrically meaningful is prerequisite to understanding the warping module.
• Second, the warping module itself — how it instantiates the decomposition in latent space, what it warps, what camera embeddings it adds, and why the zero-initialized convolutional residual connection matters.
• Third, the homography-guided self-attention layer — how the three latent streams (source, target, warped) interact, why per-frame attention is necessary for temporal alignment, and what happens to the warped latent after attention.
• Fourth, the camera encoder and conditioning mechanism — what information is encoded, how embeddings are broadcast and applied, and why sharing the encoder across DiT blocks is important.
• Fifth, the training procedure — how the diffusion model is fine-tuned (what is frozen, what is trained, what loss is used), and the Wan2.1-specific details (VAE compression, patchification, rectified flow formulation).
• Sixth, the data augmentation strategy — trajectory augmentation (reversal and concatenation to break first-frame bias) and intrinsic augmentation (focal length resizing and center cropping to learn focal length effects), including the video pair selection protocol.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a method paper whose core idea is that camera-controlled video generation can be made robust by conditioning a diffusion model on the noise-free infinite homography warp and learning the residual parallax end-to-end, rather than relying on explicit (and error-prone) depth estimation.

The Core Geometric Decomposition: Why Infinite Homography?

The entire architecture rests on a single equation from multi-view geometry (Eq. 2). The paper begins with the standard plane-induced homography relationship between two cameras and then takes a limit to derive the infinite homography.

For two cameras with centers $C$ (source) and $C'$ (target), consider a 3D point that projects to pixel $x$ in the source image. If we know the depth $Z$ of that point from the source camera, the corresponding pixel $x'$ in the target image is given by the plane-induced homography for a plane at distance $d$ with normal $n$:

$H = K_t(R - tn^T/d)K_s^{-1}$

where $K_s, K_t \in \mathbb{R}^{3 \times 3}$ are the source and target intrinsic matrices, $R \in \mathbb{R}^{3 \times 3}$ is the rotation matrix from source to target camera, $t \in \mathbb{R}^3$ is the translation vector, $n$ is the normal of the plane, and $d$ is the distance to the plane.

Why this form matters: This equation says that if all 3D points lie on a single plane, the mapping from source to target pixels is a simple $3 \times 3$ homography. But real scenes are not planar — different points have different depths, so no single homography works for all pixels.

The key insight comes from taking the limit as the plane distance $d$ approaches infinity:

$H_\infty = \lim_{d \to \infty} K_t(R - tn^T/d)K_s^{-1} = K_t R K_s^{-1}$

where $H_\infty$ is the infinite homography — the homography induced by the plane at infinity ($\pi_\infty$).

What this computes: $H_\infty$ maps pixels from the source image to the target image as if all points were infinitely far away. At infinite distance, translation has no effect (parallax vanishes), so the mapping depends only on camera rotation and the ratio of intrinsics.

Why this form: The crucial property is that $H_\infty$ depends only on rotation and intrinsics — it is completely independent of scene depth and camera translation. Since the user specifies the target camera trajectory, $R$, $K_s$, and $K_t$ are all known exactly. Therefore, $H_\infty$ provides a noise-free, perfectly accurate geometric transformation that the model can rely on unconditionally.

Now, for a real point at finite depth $Z$, the actual reprojection is (Eq. 2):

$x' = K_t R K_s^{-1} x + K_t t / Z = H_\infty x + K_t t / Z$

where $x'$ is the reprojected pixel position in the target image, $x$ is the source pixel, and $Z$ is the depth of the 3D point measured from the source camera.

What this computes: The full reprojection decomposes into two additive terms:

• $H_\infty x$: the infinite homography mapping — where the pixel would go if it were at infinity (rotation-only warp), which is known exactly.
• $K_t t / Z$: the parallax displacement — the additional shift caused by finite depth and camera translation, which is unknown because $Z$ is unknown.

Why this form matters operationally: This decomposition cleanly separates the problem into what we know with certainty (rotation and intrinsics, encoded in $H_\infty$) and what we must learn from data (depth-dependent parallax). The model receives the $H_\infty$-warped image as clean conditioning — an image showing exactly what the scene would look like under pure rotation, with no depth errors — and then learns to predict the residual parallax displacement needed to account for translation and scene depth. The paper argues this is fundamentally more robust than reprojection-based methods because the conditioning signal is free of depth-induced errors.

The paper also notes a geometric constraint that aids learning (Section 3, Figure 2a): parallax is restricted to lie on the epipolar line between the epipole $e'$ and the point $x_\infty$ (where $x_\infty = H_\infty x$). This "yellow segment on the epipolar line $l'$" provides the model with strict boundary conditions — the search space for parallax displacement is one-dimensional along the epipolar line, not the full 2D image. This spatial constraint "helps the model to achieve higher camera pose fidelity with reduced search space."

The Warping Module: Instantiating the Decomposition in Latent Space

The warping module (Section 4.1, Figure 3b) translates the mathematical decomposition of Eq. (2) into a neural network operation. Importantly, the warping is performed in the latent space of the VAE (not in pixel space), which the paper states is done "for computational efficiency."

The module operates as follows:

Step 1: Homography warp of the first frame's source latent. The module takes as input $z_s^{\text{init}} \in \mathbb{R}^{b \times 1 \times h \times w \times d}$, the source latent of the initial (first) frame. It applies the infinite homography $H_\infty$ derived from the target camera's rotation and intrinsics relative to the source camera's first frame pose. The output is a warped latent $z_w^{\text{init}} \in \mathbb{R}^{b \times 1 \times h \times w \times d}$ where spatial positions have been transformed according to $H_\infty$. The warping is applied only to the first frame because "target camera poses are defined relative to the source video's first frame" (Section 4.1, "Warping Module").

Step 2: Residual connection with zero-initialized convolution. The warped latent is added to the original $z_s^{\text{init}}$ through a convolutional layer initialized to zero, functioning as a residual connection. This zero initialization means that at the start of training, the warped latent contributes nothing — the model begins by using the unwarped source latent and gradually learns to incorporate the warped signal. This is a standard technique from ControlNet-style architectures to preserve pretrained behavior at initialization.

Step 3: Camera embedding addition for translation. Camera embeddings encoding the target's translation vector and intrinsics are added to the warped-and-residual-combined latent. This step "reflects the second term in Eq. (2)" — the parallax term $K_t t / Z$ — by providing the model with the translation parameters it needs to estimate the depth-dependent displacement.

The paper states that this design "simplifies the reprojection estimation under target camera poses by reducing it to parallax relative to $\pi_\infty$ estimation, helping the model achieve higher camera-pose fidelity."

Key design choice: warping in latent space. The paper warps VAE latents rather than RGB images. This is important for two reasons: (1) it avoids the computational cost of encoding the warped image through the VAE at every training step, and (2) it keeps the warping operation within the same representational space as the diffusion model's inputs, allowing the homography-guided attention to directly compare warped and non-warped latent features. However, this choice means the warping operation must account for the spatial downsampling factor of the VAE — the homography matrix must be adjusted for the reduced spatial resolution of the latent grid.

Why this module design over alternatives: The paper implicitly contrasts this with the reprojection-based approach (Section 3, Figure 2b). In reprojection methods, the entire transformation (rotation + translation + depth) is computed using estimated depth and provided as a single conditioning image. If the depth is wrong, the entire conditioning is corrupted in ways the model cannot identify. By contrast, InfCam's warping module provides the rotation component cleanly (through $H_\infty$) and lets the model learn the translation component end-to-end, with gradients flowing through the entire network to refine the implicit depth representation. The paper states: "end-to-end training enables the network to implicitly refine the 3D geometry, correcting inaccuracies in X" (Figure 2a caption, where "X" refers to the 3D point position that depends on depth).

A crucial input detail: the source camera pose is unknown for real videos. The paper handles this by forming a camera input vector for the source by "concatenating the source intrinsics with an identity pose $[I|0]$" and replicating this across frames (Section 4.1, "Homography-Guided Attention Layer"). This encodes the assumption that the source video defines the world coordinate origin — all target poses are expressed relative to the source's initial frame.

The Homography-Guided Self-Attention Layer

The homography-guided attention layer (Section 4.1, Figure 3a) is inserted into each DiT block of the Wan2.1 backbone. It is the mechanism through which the model integrates information from the warped latent with the source and target latents to perform rotation-aware generation.

The layer processes three latent streams for each frame index $i$:

1. Source latent $z_s^i \in \mathbb{R}^{b \times 1 \times h \times w \times d}$ — the encoded representation of frame $i$ from the source video.
2. Target latent $z_t^i \in \mathbb{R}^{b \times 1 \times h \times w \times d}$ — the current (noised) estimate of frame $i$ in the target video being generated.
3. Warped latent $z_w^i \in \mathbb{R}^{b \times 1 \times h \times w \times d}$ — the infinite-homography-warped version of $z_s^i$, computed by applying $H_\infty$ relative to the target pose at frame $i$. For $i > 0$, this uses the homography derived from the target camera's rotation at frame $i$ relative to the source's first frame.

Camera embedding integration. Before attention, each latent is combined with its corresponding camera embedding. Specifically:

• For the target latent, the user-specified target intrinsics $K_t$, rotation $R_t$, and translation $t_t$ (at frame $i$) are concatenated into a 16-dimensional vector, fed through the shared camera encoder, and added to $z_t^i$.
• For the source latent, since source poses are unknown, the source intrinsics $K_s$ are concatenated with the identity pose $[I|0]$ (a $3 \times 4$ matrix representing zero rotation and zero translation), flattened, and fed through the same camera encoder. This vector is "replicated across frames" (Section 4.1).
• For the warped latent, the camera embedding encodes the translation and intrinsics of the target camera (the rotation is already geometrically applied via the warp itself, so the embedding focuses on the parallax-related parameters).

Spatial concatenation and per-frame attention. The three latent streams (each augmented with camera embeddings) are spatially concatenated along the spatial dimension, producing:

$z_c \in \mathbb{R}^{(b \times f) \times (3hw) \times d}$

Frames are treated as individual batch items within the attention mechanism — the batch dimension becomes $b \times f$. This is the crucial design choice: by "treating frames as individual batch items," the attention operates per-frame, meaning each frame attends only to spatial positions within its own three concatenated latents (source, target, warped) and does not attend across different timestamps. This "ensures temporal alignment by allowing the target frames to reference corresponding source frames at the same timestamps" (Section 4.1).

Why per-frame attention is necessary: If attention were applied across both spatial and temporal dimensions jointly, the model could mix information from different timesteps, potentially breaking the temporal synchronization between source and target videos. Since the task is video-to-video generation with aligned timestamps (frame $i$ in the source corresponds to the same moment in time as frame $i$ in the target), restricting attention to per-frame ensures the model learns view transformations without confusing temporal dynamics.

Post-attention splitting. After the self-attention operation on $z_c$, the output is split and reshaped back into three separate tensors, each with shape $\mathbb{R}^{b \times (fhw) \times d}$. The warped latent $z_w^i$ is then discarded — it served only as contextual information during attention. Only the source and target latents proceed into the subsequent layers of the Wan2.1 DiT block (the pretrained self-attention, cross-attention, and feed-forward layers). Within these pretrained layers, the paired source and target latents are "treated as a unified batch," meaning they flow through the frozen Wan2.1 weights together without any architectural modification to the backbone.

Why concatenate rather than cross-attend: The paper could have used separate cross-attention layers where source and target latents attend to the warped latent as key-value pairs. The concatenation approach means that source, target, and warped representations are treated symmetrically in the attention operation — each can attend to any spatial position across all three streams. This is important because the model needs to learn correspondences: which region of the warped latent corresponds to which region of the source, and how both relate to the target being generated. A symmetric attention mechanism allows the model to discover these relationships without architectural prescription.

Initialization of the new attention layers. The paper states that "the new attention layers are initialized using weights from the corresponding pretrained transformer blocks" (Section 4.1). This means the homography-guided attention starts from a position of already knowing how to process video latents, and only needs to learn the additional functionality of incorporating warped information.

The Camera Encoder and Conditioning Mechanism

The camera encoder (Section 4.1) is a small learned module that maps camera parameter vectors into the embedding space of the diffusion model.

Architecture. The camera encoder consists of a single linear layer with:

• Input dimension: 16, comprising a flattened $3 \times 3$ rotation matrix (9 values), three translation parameters (3 values), focal lengths $f_x$ and $f_y$ (2 values), and principal point coordinates $c_x$ and $c_y$ (2 values). Total: $9 + 3 + 2 + 2 = 16$ dimensions.
• Output dimension: $d$, the hidden dimension of the self-attention layers in Wan2.1.

Why a linear layer: The paper does not use a more complex encoder (e.g., MLP, transformer, or Fourier features) for camera parameters. A linear layer is sufficient because the relationship between camera parameters and their geometric effects is largely linear (the camera parameters appear linearly in the projection equations), and the model's capacity is better allocated to the attention layers that reason about the geometric consequences of these parameters.

Broadcasting and addition. For a latent representation $z \in \mathbb{R}^{b \times f \times h \times w \times d}$, the corresponding camera embedding $e_c \in \mathbb{R}^{b \times f \times d}$ is computed and then broadcast along the spatial dimensions ($h$ and $w$). This means every spatial position in every frame receives the same camera embedding, and the embedding is added (not concatenated) to the latent features.

Why addition rather than concatenation: Concatenation would change the dimensionality of the latent, requiring architectural modifications to the attention layers. Addition preserves the pretrained feature dimension, allowing the frozen Wan2.1 backbone to process the camera-informed latents without modification. This is standard practice in diffusion model conditioning (e.g., timestep embeddings in standard diffusion, class conditioning in classifier-free guidance).

Encoder sharing across DiT blocks. The paper explicitly states: "To ensure consistent camera conditioning, we share the camera encoder across each DiT block for different camera encodings" (Section 4.1). This means the same linear layer is used at every DiT block where camera information is needed, rather than learning separate encoders per block. Sharing reduces parameters and ensures that the camera embedding space has a consistent interpretation throughout the network.

Why 16 dimensions for the camera vector: The paper includes the full $3 \times 3$ rotation matrix (9 parameters) rather than a more compact representation like Euler angles (3 parameters) or a quaternion (4 parameters). Using the full rotation matrix avoids the discontinuities and singularities (e.g., gimbal lock) that plague minimal rotation parameterizations, and provides the encoder with the exact linear transformation applied to 3D points. The translation vector and intrinsics complete the geometric specification.

The source camera encoding deserves special attention. Since source videos from real-world data (e.g., WebVid) have unknown camera poses, the paper encodes the source with an identity pose $[I|0]$ — equivalent to a camera at the world origin with no rotation and no translation. This is not an approximation of the true source pose; rather, it defines the coordinate system in which target poses are expressed. All target trajectories are specified relative to the source video's first frame, so encoding the source as identity and the targets as relative poses is geometrically consistent.

The Training Procedure and Loss

InfCam is built on the Wan2.1 text-to-video diffusion model (Wan et al., 2025), and the training procedure follows Wan2.1's framework with task-specific adaptations.

Base model and weight freezing. The paper uses the pretrained Wan2.1 model as its backbone. Wan2.1 is a transformer-based diffusion model operating in the latent space of a 3D VAE. Crucially, all pretrained weights of Wan2.1 are frozen during training. The paper states: "we freeze the pretrained weights of Wan2.1 and train only the newly introduced camera encoder and the homography-guided self-attention layers" (Section 4.1). This preserves the video generation capabilities learned from Wan2.1's extensive pretraining while adding camera controllability.

The trainable components are therefore:

1. The camera encoder (one linear layer shared across blocks).
2. The homography-guided self-attention layers (one per DiT block).
3. The warping module's zero-initialized convolutional layer.
4. (Implicitly) The zero convolution in the warping module's residual connection.

VAE compression and patchification. Wan2.1 compresses the spatio-temporal dimensions of input videos using a 3D VAE (Supplementary Material, Section 9.1). A video $V \in \mathbb{R}^{B \times (1+F) \times H \times W \times 3}$ (where $F$ is the number of generated frames, plus one context frame) is encoded to a latent with temporal downsampling factor 4 and spatial downsampling factor 8, producing $z \in \mathbb{R}^{B \times (1+F/4) \times H/8 \times W/8 \times d}$. Subsequent patchification further reduces spatial resolution by a factor of 2 along each spatial dimension, yielding:

$z \in \mathbb{R}^{B \times (f \times h \times w) \times d}$

where $f = 1 + F/4$, $h = H/16$, and $w = W/16$. For the paper's main experiments at $480 \times 832$ resolution and 81 frames: $F = 80$ (81 frames includes the context frame), so $f = 1 + 80/4 = 21$, $h = 480/16 = 30$, $w = 832/16 = 52$, giving $z \in \mathbb{R}^{B \times (21 \times 30 \times 52) \times d}$.

The paper also conducts ablation studies at lower resolution ($320 \times 544$, 41 frames) for efficiency: $f = 1 + 40/4 = 11$, $h = 320/16 = 20$, $w = 544/16 = 34$.

Rectified flow formulation. Wan2.1 uses a rectified flow (Liu et al., 2023; Esser et al., 2024) formulation rather than standard diffusion. The key difference from standard DDPM-style diffusion is that rectified flow defines a linear interpolation between noise and data:

Given a clean video latent $z_1$ (the encoded target video), random noise $z_0 \sim \mathcal{N}(0, I)$, and a sampled timestep $t \in [0, 1]$, the intermediate latent is:

$z_t = t z_1 + (1 - t) z_0$

What this computes: $z_t$ is a straight-line interpolation between pure noise (at $t = 0$) and the clean latent (at $t = 1$). At intermediate $t$, it is a noisy version of the target video with the noise level proportional to $(1 - t)$.

Why rectified flow over DDPM: The key property is that the "ground truth velocity" — the direction the model should predict to move $z_t$ toward $z_1$ — is constant along the entire trajectory:

$v_t = \frac{dz_t}{dt} = z_1 - z_0$

This means the model predicts a single velocity vector field that transports noise to data along straight paths, rather than predicting the noise as in DDPM. Rectified flow has been shown to enable faster sampling and more stable training in recent large-scale image and video models (Stable Diffusion 3, Flux, Wan2.1).

The training loss (Eq. 3 in supplementary):

$\mathcal{L} = \mathbb{E}_{z_0, z_1, c_{\text{txt}}, t} \left[ \| u(z_t, c_{\text{txt}}, t; \theta) - v_t \|^2 \right]$

where $c_{\text{txt}}$ is the text embedding sequence (derived from LLaVA captioning of the source video), $t \in [0, 1]$ is the sampled timestep, $\theta$ represents the model weights (including both frozen Wan2.1 weights and the newly trained camera/attention parameters), $u(z_t, c_{\text{txt}}, t; \theta)$ is the velocity predicted by the model, and $v_t = z_1 - z_0$ is the ground-truth velocity.

What this computes: The mean squared error (MSE) between the model's predicted velocity and the true velocity that moves from noise to data. At each training step, a clean target latent $z_1$ is sampled, random noise $z_0$ is drawn, a timestep $t$ is sampled, $z_t$ is constructed via linear interpolation, and the model predicts the velocity field at $z_t$. The loss encourages the model to learn a velocity field that transports any point on the trajectory toward the clean latent.

Why MSE: The velocity prediction task is a regression problem — the target is a continuous vector in $\mathbb{R}^{(fhw) \times d}$. MSE is the natural choice for regression with Gaussian error assumptions and is standard in rectified flow training. Alternative losses (e.g., L1, Huber) could be used but are not explored.

Training hyperparameters. The paper provides specific training configurations:

• Main experiments: 81-frame videos at $480 \times 832$ resolution, trained for 15,000 iterations using 4 H100 GPUs with batch size 8, taking approximately one week. Optimizer: Adam with weight decay 0.01 and learning rate $1 \times 10^{-5}$.
• Ablation experiments: 41-frame videos at $320 \times 544$ resolution, trained for 20,000 iterations using 4 H100 GPUs with batch size 32, taking approximately four days. Same optimizer and hyperparameters.

Text conditioning. Since Wan2.1 is a text-to-video model that requires a text description of the generated video, the authors use LLaVA (a vision-language model) to extract descriptive text from the source video, which is then used as the text prompt during training and inference. This is a practical bridge: InfCam does not require users to provide text descriptions of their videos; the text is automatically extracted from the visual content.

Why LLaVA for captioning: Using an off-the-shelf vision-language model to generate captions allows the system to operate without manual annotation, and LLaVA's captions provide sufficient semantic guidance for the Wan2.1 backbone to leverage its pretrained text-to-video knowledge.

Inference. At inference time, the same rectified flow framework is used for sampling. Starting from random noise $z_0$, the model iteratively applies the learned velocity field to transport the noise toward the data distribution, conditioned on the source video, warped latents, camera embeddings, and text description.

The Data Augmentation Strategy

The data augmentation strategy (Section 4.2) is not just a preprocessing step — it is essential for breaking the dataset biases that cause trajectory-conditioned methods to fail. The paper describes two complementary augmentations (trajectory and intrinsic) and a video pair selection protocol.

Trajectory Augmentation

The MultiCamVideo dataset (Bai et al., 2025) provides 10 synchronized camera trajectories and their corresponding videos for each of 13,600 scenes. A critical property of this dataset is that "all 10 cameras within each scene share an identical starting position" (Section 4.2.1). This means that during standard training — where the source and target videos are two different trajectories from the same scene — the first frame is always identical between source and target. The model never sees an example where it must generate a first frame from a viewpoint different from the source's first frame.

The consequence is a learned shortcut: when presented with a source video, the model reproduces the first frame as-is rather than performing genuine viewpoint transformation. The paper confirms this empirically: "models trained on this dataset exhibit a bias toward reproducing the source video's first frame, even when conditioned to generate it from a different viewpoint" (Section 4.2.1).

The trajectory augmentation procedure exploits a key property: "while all trajectories share the same starting frame, their remaining frames diverge across different videos." The augmentation works as follows (Section 4.2.1, Figure 8b):

1. Randomly sample two distinct videos (Trajectory A and Trajectory B) from the same scene.
2. Reverse Trajectory A: take the first video and play it backwards. The final frame of the reversed video becomes the original first frame — which, crucially, is identical to the first frame of Trajectory B (since all trajectories in the scene share the same start).
3. Concatenate the reversed Trajectory A with Trajectory B. Since the last frame of reversed A matches the first frame of B, the concatenation is seamless at that boundary.
4. Remove the redundant first frame of B: because the last frame of reversed A is already identical to B's first frame, keeping B's first frame would create a duplicate. Removing it yields a video of 161 frames ($81 + 80$, where 81 is the length of each original video: Trajectory A reversed = 81 frames, Trajectory B without first frame = 80 frames).
5. Crop to 81 frames: to match the Wan2.1 sequence length, a random contiguous 81-frame segment is extracted from this 161-frame augmented video.
6. This 81-frame segment is used as one video in the training pair; another trajectory from the same scene (potentially also augmented) is used as the paired video.

What this achieves: After augmentation, when two videos from the same scene are selected as source and target, their first frames are no longer guaranteed to be identical. The reversal operation introduces temporal offset: the first frame of the augmented video corresponds to some frame in the middle of the original Trajectory A (played backwards), while the first frame of another augmented video would correspond to a different point in a different trajectory. This forces the model to learn that the first frame of the target may show a different viewpoint than the first frame of the source, breaking the identity-copying shortcut.

Figure 4 visualizes this transformation: the original MultiCamVideo shows trajectories all converging at the same start point (Figure 4b), while AugMCV shows trajectories with varied starting positions (Figure 4c) — starting poses that differ across videos, created through the reversal and concatenation procedure.

The 161-frame construction rationale: The original videos are 81 frames long. Reversing one gives 81 frames. Concatenating with another (minus the redundant first frame) gives $81 + 80 = 161$. Cropping a random 81-frame window from this 161-frame sequence ensures variety in which temporal segment is used and makes the augmentation stochastic.

Intrinsic (Focal Length) Augmentation

The MultiCamVideo dataset uses four focal lengths — 18mm, 24mm, 35mm, and 50mm — but each scene uses a single, fixed focal length across all 10 camera trajectories. This means that during standard training, source and target videos always have the same focal length. The model never learns how changing focal length affects the field of view, and consequently "demonstrates a bias toward generating videos that preserve the focal length of the source video" (Section 4.2.2).

The intrinsic augmentation addresses this by simulating focal length changes through image resizing and cropping (Algorithm 1 in supplementary):

1. Given a scene with original focal length $f_{\text{scene}}$, randomly sample a new focal length $f_{\text{new}}$ from the set $\{18\text{mm}, 24\text{mm}, 35\text{mm}, 50\text{mm}\}$ with the constraint $f_{\text{new}} > f_{\text{scene}}$ (i.e., only augment in the increasing direction).
2. Resize each frame: compute the resize ratio $r = f_{\text{new}} / f_{\text{scene}}$ and scale the frame dimensions from $(W_{\text{orig}}, H_{\text{orig}})$ to $(r \cdot W_{\text{orig}}, r \cdot H_{\text{orig}})$.
3. Center crop back to the original resolution $(W_{\text{orig}}, H_{\text{orig}})$.

Why only augment in the ascending direction ($f_{\text{new}} > f_{\text{scene}}$): When scaling up (longer focal length → narrower field of view), the resized image is larger than the original, and center cropping discards peripheral regions. This means the cropped result is a subset of the originally observed field of view — no unseen regions are introduced. If the paper augmented in the descending direction ($f_{\text{new}} < f_{\text{scene}}$, shorter focal length → wider field of view), the resized image would be smaller than the original, and center cropping would need to pad or hallucinate content outside the original frame. By restricting to ascending augmentation, the procedure remains a pure geometric transformation of observed content.

How decreasing focal length is still covered: The paper notes that even though augmentation only produces longer-focal-length versions, the video pair selection stage (Section 4.2.3) randomly assigns augmented and non-augmented videos as source or target. When an augmented (long-focal-length) video serves as source and an original (shorter-focal-length) video serves as target, the pair represents a focal length decrease from the model's perspective — the model must generate a wider field of view from a narrower input. Both directions ($f_{\text{source}} < f_{\text{target}}$ and $f_{\text{source}} > f_{\text{target}}$) are covered through this randomized pairing.

Probabilistic application: Intrinsic augmentation is applied with probability 0.5 to each video independently during video pair selection, ensuring that approximately half the training examples involve focal length changes and half do not.

Video Pair Selection Protocol

The final step constructs source–target video pairs from the augmented dataset (Section 4.2.3):

1. For a given scene after trajectory augmentation, there are 10 original trajectories (each 81 frames) and potentially 10 trajectory-augmented versions (reversal-and-concatenation derived 81-frame segments).
2. Each trajectory may have intrinsic augmentation applied with probability 0.5, producing focal-length-augmented versions.
3. Two distinct trajectories are randomly sampled from this pool.
4. One is assigned as the source video, the other as the target (ground truth) video.
5. The corresponding camera trajectories (rotation, translation per frame, focal length) are paired accordingly — the model receives the source's camera parameters and the target's camera parameters as conditioning, and must generate the target video.

This random pairing covers arbitrary combinations of trajectory patterns and intrinsic configurations, ensuring the training distribution is diverse in both camera motion and field-of-view variation.

Dataset scale. The augmentation expands the dataset from 13,600 scenes (in the original MultiCamVideo) to 47,600 videos total. Of these, 47,432 are used for training and 168 for testing, with no scene overlap between train and test splits. This is a roughly 3.5× expansion in total video count, but more importantly, the structural diversity (in starting poses and focal length combinations) increases combinatorially due to the random pairing.

Putting It All Together: The Training and Inference Flow

Training step (one iteration):

1. A scene is sampled from the AugMCV training set.
2. Two trajectory-augmented and potentially intrinsic-augmented videos are randomly selected as source ($V_s$) and target ($V_t$) from that scene.
3. LLaVA generates a text description from $V_s$, which is tokenized and embedded as $c_{\text{txt}}$.
4. The 3D VAE encodes both videos: $z_s = \text{VAE}(V_s)$, $z_1 = \text{VAE}(V_t)$ (the clean target latent).
5. The warping module computes $z_w$ by applying $H_\infty$ to $z_s^{\text{init}}$ (the first frame of the source latent), using the target camera's rotation at each frame relative to the source's first frame pose. Camera embeddings are added for translation.
6. Random noise $z_0 \sim \mathcal{N}(0, I)$ is sampled.
7. A timestep $t \in [0, 1]$ is sampled uniformly.
8. The intermediate latent $z_t = t z_1 + (1 - t) z_0$ is constructed.
9. Camera embeddings are computed for source (using identity pose), target (using target pose at each frame), and warped (using target translation) via the shared camera encoder.
10. $z_s$, $z_t$, and $z_w$ (all augmented with camera embeddings) pass through the stacked DiT blocks. At each block: homography-guided self-attention operates on the concatenated triple, discards $z_w$, then source and target proceed through Wan2.1's pretrained self-attention, cross-attention (with $c_{\text{txt}}$), and feed-forward layers.
11. The model outputs a predicted velocity $u(z_t, c_{\text{txt}}, t; \theta)$.
12. The MSE loss $\| u - (z_1 - z_0) \|^2$ is computed and backpropagated through only the trainable parameters (camera encoder, homography-guided attention layers, warping module convolution).

Inference (generation):

1. A source video $V_s$ and target camera trajectory are provided by the user.
2. If the source video is real-world (e.g., WebVid) and intrinsics are unknown, UniDepth estimates the focal length and principal point to construct $K_s$.
3. LLaVA generates a text description from $V_s$.
4. The 3D VAE encodes $V_s$ to $z_s$.
5. The warping module computes $z_w$ using $H_\infty$ derived from the target trajectory relative to the source's first frame.
6. Random noise $z_0 \sim \mathcal{N}(0, I)$ is initialized.
7. The rectified flow sampler iteratively denoises: starting at $t = 0$, the model predicts the velocity, and an ODE solver (e.g., Euler) steps $z_t$ toward $t = 1$, conditioned on $z_s$, $z_w$, camera embeddings, text, and the current timestep.
8. At $t = 1$, the denoised latent is decoded by the 3D VAE to produce the final output video $V_t$.

Why this training strategy works: By freezing the Wan2.1 backbone, the model retains the strong video prior learned from large-scale pretraining. The new components are lightweight (a linear camera encoder, per-block attention layers initialized from pretrained weights, a zero-initialized warping residual) and learn to inject camera control without disrupting the base model's generative capabilities. The infinite homography warp provides clean geometric conditioning that doesn't suffer from depth estimation errors, and the end-to-end training with rectified flow allows gradients to flow through the entire network, enabling the model to learn implicit depth representations that produce accurate parallax displacements.

4. Key Insights and Innovations

Innovation 1: Decomposing Novel-View Synthesis into Known Rotation and Learned Parallax — a Diagnostic Reframing, Not Just a New Module

The paper's deepest conceptual contribution is not the warping module itself — it's the reframing of what makes camera-controlled generation fragile. Prior work implicitly treated the problem as a monolithic transformation: estimate depth, reproject everything, and hope the errors are small enough that the diffusion model can compensate. InfCam instead asks: What part of the geometric transformation do we know with absolute certainty, and what part must we learn from data?

The answer — embodied in Eq. (2): $x' = H_\infty x + K_t t / Z$ — is that the infinite homography $H_\infty = K_t R K_s^{-1}$ is computable exactly from the user-specified camera trajectory, with zero noise. It captures the rotation-induced view change perfectly. The parallax term $K_t t / Z$ is depth-dependent and therefore unknown, but it obeys strict geometric constraints: it lies on the epipolar line between the epipole and the $H_\infty$-mapped point, confining the search space to a one-dimensional segment (Figure 2a, "yellow segment on the epipolar line $l'$").

This decomposition is intellectually distinctive because it diagnoses the shared failure mode of both dominant paradigms. Reprojection-based methods (GEN3C, TrajectoryCrafter) collapse the entire transformation into a single conditioning image computed from estimated depth. When depth is wrong — and it always is, to some degree — the entire conditioning signal is corrupted, including the rotation component that could have been provided cleanly. The model faces an ill-posed denoising problem: it must distinguish which regions of its conditioning are reliable without any explicit uncertainty signal. The paper's critique in Section 3 is precise on this point: "because no gradients flow through the depth estimation network, the inaccurate reprojection position $x'$ remains fixed as a condition throughout training." This is not a limitation that better depth estimators can fully solve — it is structural, because any depth estimator will have errors, and those errors will always be fixed conditioning artifacts.

Trajectory-conditioned methods (ReCamMaster, SynCamMaster) face a different but equally structural problem: they must learn rotation, translation, and depth disentanglement purely from data, without any geometric prior. When the training data contains biases — such as all source-target pairs sharing identical first frames — the model learns shortcuts (first-frame copying) rather than genuine viewpoint transformation. The paper demonstrates this empirically in Section 5.2: ReCamMaster "consistently preserves the initial frame of the source video," and even trajectory interpolation cannot fully rescue it.

InfCam's decomposition sidesteps both failures simultaneously. By providing rotation as clean, noise-free conditioning (through the infinite homography warp) and letting the model learn only the residual parallax end-to-end, the model never sees corrupted rotation information and never has the opportunity to learn rotation shortcuts from biased data. The paper explicitly connects this to a geometric constraint: "parallax variations are constrained between the epipole $e'$ and $x_\infty$, providing the model with strict boundary conditions" that "help the model to improve camera-pose fidelity by reducing the search space" (Section 3).

Significance beyond performance. This is fundamentally a diagnostic reframing, not an incremental module. It tells the field why existing methods fail rather than simply outperforming them. The decomposition implies that future camera-controlled generation systems should identify which geometric quantities are known with certainty and condition on them explicitly, rather than forcing models to rediscover them from data or estimate them through error-prone preprocessing. This principle generalizes beyond infinite homography: any task where part of the geometric transformation is specifiable in closed form (e.g., known object motion, known lighting changes) could benefit from a similar known-residual decomposition.

Evidence. The ablation in Table 3 demonstrates that the warping module is necessary for the strongest performance: adding it to the augmented-data baseline improves PSNR from 22.807 to 24.412 on shared intrinsics and from 21.866 to 24.311 on different intrinsics — a jump that is larger than the gain from intrinsic augmentation alone (from ~20.820 to ~22.807). Figure 6 shows qualitatively that without the warping module, the model "fails to capture both the target trajectory and intrinsics," producing results that are only "consistent with the target trajectory" after augmentation but still "inaccurate when compared to the ground truth."

The paper does not claim this decomposition is theoretically novel in multi-view geometry — $H_\infty$ is a standard concept (Hartley & Zisserman, 2003, cited in the paper). The innovation is recognizing that this standard geometric decomposition solves a pressing empirical failure mode in learned video generation, and building a training framework that exploits it.

---

Innovation 2: Dataset Bias as the Root Cause of Failure in Trajectory-Conditioned Methods — and a Targeted Augmentation Cure

The paper's second major conceptual contribution is the identification and systematic elimination of specific, named dataset biases that cause trajectory-conditioned models to fail. This goes beyond the standard observation that "more diverse data helps" — the paper pinpoints exactly which patterns in the training data create shortcuts, and designs augmentations that surgically remove each bias.

First-frame identity bias. In MultiCamVideo, all trajectories in a scene share the same starting camera pose. The consequence is that during training, the source and target videos always have identical first frames. The model learns that the correct output for the first frame is simply a copy of the input's first frame — a shortcut that works perfectly on the training distribution but catastrophically on any test input where the target trajectory starts from a different viewpoint. The paper shows this bias in action (Section 5.2): GEN3C "fails to transform the viewpoint of the initial frame" (attributed to its image-to-video backbone's bias), and ReCamMaster "consistently preserves the initial frame of the source video."

Focal-length identity bias. Because each scene uses a fixed focal length across all cameras, source and target videos always share the same field of view during training. The model learns to ignore the target focal length specification, simply reproducing the source's field of view. The paper states the model "demonstrates a bias toward generating videos that preserve the focal length of the source video" (Section 4.2.2).

What makes this contribution intellectually distinctive is the specificity of the diagnosis and the precision of the cure. The trajectory augmentation (Section 4.2.1) does not just add more data — it exploits a structural property of the dataset (that frames diverge after the first, even when the first frames match) to construct sequences where first frames genuinely differ. Reversing one trajectory and concatenating it with another creates a temporally coherent video whose first frame comes from a different original trajectory than another augmented video from the same scene, forcing the model to confront first-frame viewpoint changes.

Similarly, the intrinsic augmentation (Section 4.2.2) does not randomly perturb focal lengths — it uses the ascending-only constraint ($f_{\text{new}} > f_{\text{scene}}$) to ensure that the resizing and cropping operation never introduces unseen regions, and then relies on randomized video pair selection to cover both focal-length-increase and focal-length-decrease scenarios during training. This is a careful, geometrically-motivated design choice, not a generic data augmentation heuristic.

Comparison to prior work. Trajectory-conditioned methods (SynCamMaster, ReCamMaster) trained on datasets with these biases implicitly or explicitly suffered from them, and the paper's results suggest that scaling up such datasets without addressing the structural biases would be ineffective — the model would continue to exploit shortcuts, just with higher confidence. The paper's joint training experiment (Table 4) reinforces this: training on the mixture of MultiCamVideo and SynCamVideo (which has different initial frames) underperforms training on AugMCV alone, suggesting that simply mixing datasets with different biases does not eliminate them as effectively as targeted augmentation.

Evidence. Table 3 shows that adding trajectory augmentation alone improves PSNR from 19.228 to 20.820 on shared intrinsics — a meaningful gain that is visible qualitatively in Figure 6, where the trajectory-augmented model produces "a movement consistent with the target trajectory." But it's the combination with intrinsic augmentation that pushes PSNR to 22.807, and the further addition of the warping module to 24.412. Table 4's comparison to MCV+SCV joint training shows AugMCV achieving 3.368 RotErr vs. 4.158 and 0.839 TransErr vs. 1.441, demonstrating that targeted augmentation outperforms naive dataset mixing.

Scale of impact. The augmentations expand the dataset from 13,600 scenes to 47,600 videos — a roughly 3.5× increase — but the structural diversity gain is combinatorial because of the random video pair selection protocol. The paper is clear that this is not just a scale story; it's a bias-removal story. The ~20% relative reduction in rotation error on WebVid (Table 5 vs. Table 6 combined) between FF-Sync and FF-Async settings for competing methods (e.g., ReCamMaster degrades from 9.673 to 7.076 on rotation error in the harder setting, while TrajectoryCrafter improves from 5.595 to 4.418) compared to InfCam's consistent performance (3.605 to 2.718) suggests that InfCam's training data genuinely teaches viewpoint generalization rather than fitting to dataset-specific correlations.

---

Innovation 3: Latent-Space Infinite Homography as a Practical Mechanism for Noise-Free Geometric Conditioning in Diffusion Models

The third innovation is architectural but with a conceptual edge: the paper demonstrates that performing infinite homography warping directly in the VAE latent space, combined with a zero-initialized residual connection, is a viable and effective way to inject noise-free geometric information into a frozen pretrained video diffusion model. This is not an obvious design choice — warping in latent space requires accounting for the VAE's spatial downsampling factor, and latent features are not pixel-aligned in the same way that RGB values are.

The conceptual advance is the recognition that the clean geometric signal (rotation) should bypass the learned representations entirely and be injected as a direct spatial transformation, while the learned components (the residual parallax, the implicit depth estimation) can operate on the transformed latents in the model's native representational space. This is a form of architectural inductive bias: the model does not need to learn what a rotation looks like, because the rotation is already geometrically applied. It only needs to learn the depth-dependent corrections.

Comparison to prior work. Reprojection-based methods also apply geometric transformations to conditioning images — but in RGB space, with estimated depth, and as a preprocessing step with no gradient flow. The combination of (1) latent-space warping, (2) noise-free rotation (no depth needed), and (3) end-to-end gradient flow through the network that receives the warped conditioning is unique to InfCam.

The zero-initialized convolutional residual connection is a standard technique from ControlNet-style architectures, but the paper's use of it serves a specific purpose: at initialization, the warped latent contributes nothing, so the model begins by using the unwarped source latent and gradually learns to incorporate the warped signal. This preserves the pretrained Wan2.1 behavior at the start of fine-tuning and lets the model discover how to use the geometric conditioning without disrupting existing video generation capabilities.

Why this matters beyond the specific implementation. The paper demonstrates that diffusion models can be conditioned on exact geometric transformations computed at inference time from user-specified parameters, without needing to learn those transformations from data. This opens the door to similar geometric conditioning for other controlled generation tasks — for instance, conditioning on known object rotations, known lighting directions, or known non-rigid deformations that can be expressed as spatial transformations in latent space. The key insight is that if you can compute the geometric transformation exactly, you should apply it directly rather than asking the model to infer it.

Evidence. The ablation in Table 3 isolates the warping module's contribution: it provides the largest single-component gain, improving PSNR by 1.605 on shared intrinsics (22.807 → 24.412) and 2.445 on different intrinsics (21.866 → 24.311). The warping module is what enables the model to achieve accurate intrinsic matching (the "different intrinsics" PSNR of 24.311 is nearly as high as the shared-intrinsics PSNR of 24.412, indicating the model has genuinely learned focal length adaptation rather than copying the source).

The homography-guided self-attention design — where warped, source, and target latents are concatenated and attend symmetrically, then the warped latent is discarded — is also noteworthy. It means the warped latent serves as a geometric reference during the attention computation but does not directly contribute features to the output. This is a clean separation: the warped latent provides spatial correspondence information (through the attention mechanism's ability to compare query and key positions across the three streams) without polluting the target latent's feature content.

---

Innovation 4: End-to-End Learning of Depth Through Parallax Prediction — an Implicit Alternative to Explicit Depth Estimation

The paper's fourth contribution is a training paradigm where depth is never explicitly estimated, yet the model learns to produce depth-appropriate parallax through end-to-end velocity prediction in a rectified flow framework. This is a fundamentally different approach to incorporating 3D information than either reprojection-based methods (which estimate depth as preprocessing) or trajectory-conditioned methods (which may learn implicit depth representations but without geometric grounding).

The paper argues, and provides evidence through its performance, that this implicit approach is more robust than explicit depth estimation because the depth representation is learned in service of the generation task, with gradients flowing from the pixel-level reconstruction loss through the entire network. In the paper's words: "end-to-end training enables the network to implicitly refine the predicted depth $Z$ toward the ground truth as it learns to produce accurate parallax" (Figure 2a caption).

This matters because it eliminates the depth estimation bottleneck entirely. External depth estimators (like UniDepth, which the paper uses only to estimate source intrinsics for real-world videos, not for generation) are trained on different data distributions than the video generation task. They may be accurate on average but fail in ways that are catastrophic for viewpoint transformation — for instance, misestimating depth at occlusion boundaries, where parallax effects are largest and most visually salient. By learning depth implicitly, the model can allocate representational capacity to depth accuracy precisely where it matters for the generation task.

Comparison to prior work. Reprojection-based methods tie their fate to a fixed depth estimator. Trajectory-conditioned methods learn depth implicitly but without the geometric guidance of the infinite homography — they must discover the rotation-depth-translation relationship from scratch. InfCam occupies a middle ground: the model knows the rotation exactly (through $H_\infty$), receives camera embeddings specifying the translation, and must learn only the depth $Z$ that modulates the parallax term $K_t t / Z$. The learning problem is reduced from "discover the entire geometric relationship between two views" to "estimate the depth that explains the residual parallax after rotation is accounted for" — a substantially more constrained and therefore more learnable problem.

Significance beyond the specific architecture. This implicit-depth paradigm suggests a broader principle for controlled generation: when a geometric quantity (depth, surface normals, correspondence fields) is needed for a task but difficult to estimate reliably, consider whether it can be learned implicitly as a latent variable through end-to-end training, with known geometric constraints providing the supervision signal. The rectified flow velocity prediction loss provides the supervision: to predict the correct velocity field that transports noise to the target video, the model must internally represent the depth structure that explains the parallax between the $H_\infty$-warped source and the ground-truth target.

Evidence. The quantitative results on AugMCV (Table 1) and WebVid (Table 2) show InfCam outperforming reprojection-based methods (GEN3C, TrajectoryCrafter) by substantial margins — on WebVid, InfCam's rotation error of 3.162° is 37% lower than TrajectoryCrafter's 5.007° and 62% lower than GEN3C's 9.588°. While these gains reflect the entire system, the warping module is the component that enables implicit depth learning, and its ablation impact (Table 3) confirms it is essential for achieving the highest accuracy — particularly on the challenging "different intrinsics" setting, where the warping module improves PSNR by 2.445 points.

A subtle point: the paper never explicitly validates that the model has learned accurate depth. This is a limitation — we know the model produces better novel views, but we don't know whether its internal depth representation is geometrically accurate or merely sufficient for the generation task. This leaves open the possibility that the model learns a "good enough" depth proxy that works for video generation but would not generalize to other 3D tasks. This is not a flaw in the paper's contributions, but it bounds the claim: the implicit depth is task-specific, not a general-purpose 3D reconstruction.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on two datasets. (1) The AugMCV test split (Section 5.1, "Evaluation Set"): 168 scenes from the augmented MultiCamVideo dataset introduced in Section 4.2, each containing one static-camera video and 10 dynamic-trajectory videos. Using the static-camera clip as source, one video is generated per target trajectory, yielding 1,680 generated videos total. Within these 168 scenes, 96 have source and target videos with identical focal lengths ("shared intrinsics") and 72 have different focal lengths ("different intrinsics"). Ground-truth target videos exist for all trajectories, enabling full-reference metrics. (2) The WebVid dataset (Bain et al., 2021): a random subset of 100 real-world source videos. For each source video, 20 synthetic videos are generated using different camera trajectories — 10 trajectories share the same initial camera pose as the source's first frame ("FF-Sync"), and 10 start from different initial poses ("FF-Async"), yielding 2,000 generated videos. No ground-truth target videos exist, so only no-reference metrics and pose accuracy metrics are applicable. For WebVid, source intrinsics are unknown and are estimated at inference time using UniDepth (Piccinelli et al., 2024).

• Base model. The system uses Wan2.1 (Wan et al., 2025), an open-source text-to-video diffusion model based on a transformer architecture with a 3D VAE and rectified flow formulation. All pretrained Wan2.1 weights are frozen; only newly introduced parameters (camera encoder, homography-guided self-attention layers, warping module's zero-initialized convolution) are trained. The choice is justified implicitly: Wan2.1 provides strong video generation capabilities from large-scale pretraining, and freezing its weights preserves these capabilities while the added components learn camera controllability. The paper does not experiment with other base models, making this a single-backbone study — a limitation to note, though consistent with the paper's goal of demonstrating the conditioning mechanism rather than comparing backbones.

• Metrics. Four categories of metrics are used across the two evaluation datasets.

  • Full-reference visual fidelity (AugMCV only): PSNR, SSIM, and LPIPS (Zhang et al., 2018). These compare generated videos against ground-truth target videos pixel-by-pixel and perceptually. Higher PSNR and SSIM indicate better reconstruction; lower LPIPS indicates better perceptual similarity. Their presence is only possible because AugMCV provides ground truth; for WebVid, no such reference exists.

  • No-reference visual fidelity (WebVid only): FID (Fréchet Inception Distance, Heusel et al., 2017) for frame-level quality and FVD (Fréchet Video Distance, Unterthiner et al., 2019) for video-level quality. Both measure the distributional distance between generated and real video features extracted from pretrained networks. Lower values indicate higher fidelity. These are standard but imperfect — they measure distributional similarity rather than per-sample quality, and can be gamed by mode-collapsing to high-probability regions.

  • Camera pose accuracy (WebVid only): Rotation error (RotErr, in degrees) and translation error (TransErr, in meters), computed following CameraCtrl (He et al., 2025). The generated video's camera trajectory is extracted using ViPE (Video Pose Engine, Huang et al., 2025), and compared against the ground-truth target trajectory. Specifically:

    $\text{RotErr} = \arccos\left( \left| \text{tr}(R_{\text{pred}} R_{\text{gt}}^T) - 1 \right| / 2 \right)$

    $\text{TransErr} = \| t_{\text{pred}} - t_{\text{gt}} \|_2$

    To ensure ViPE references the correct initial camera pose, the first frame of each source video is prepended to every generated video before ViPE extraction; the pose estimated for this prepended frame is discarded. These metrics directly measure how faithfully the generated video follows the specified camera trajectory — the core claim of the paper.

  • Relative performance across difficulty settings (implicit): The paper separately reports results for "shared intrinsics" vs. "different intrinsics" on AugMCV (Table 1) and "FF-Sync" vs. "FF-Async" on WebVid (Table 5, Table 6 in supplementary), effectively measuring robustness to distribution shift in both intrinsic parameters and initial viewpoint.

• Baselines. The paper compares against three state-of-the-art methods spanning both dominant paradigms:

  • ReCamMaster (Bai et al., 2025, ICCV): a trajectory-conditioned video generation approach trained on synthetic trajectory–video pair datasets. Following official code and checkpoints, videos are generated at 480 × 832 resolution and 81 frames. The paper also introduces a variant ReCamMaster w/ Interp. (Section 5.2) to mitigate ReCamMaster's first-frame preservation bias: eight identical frames are prepended to the source video with camera poses linearly interpolated between the identity pose and the target pose of the first frame, and the first eight frames of the output are removed before evaluation.

  • TrajectoryCrafter (YU et al., 2025, ICCV): a reprojection-based method that estimates depth, reprojects along the target trajectory, and conditions a video inpainting model on the reprojection results. Following official code and checkpoints, videos are generated at 384 × 672 resolution and 49 frames. For the 81-frame comparison, the paper generates a 49-frame segment, then generates an additional 33-frame segment using the last frame of the first output as the first frame of the second reprojection condition (to enforce temporal consistency at the segment boundary), discards the overlapping frame, and concatenates, yielding 81 frames. This inference-level extension is an architectural workaround necessary because TrajectoryCrafter's native sequence length is shorter than the paper's evaluation setting.

  • GEN3C (Ren et al., 2025, CVPR): a reprojection-based method that builds a spatio-temporal 3D cache via depth estimation and unprojection, renders along the target trajectory, and conditions a video diffusion model on these renderings. Following official code and checkpoints, videos are generated at 704 × 1280 resolution and 121 frames. For the 81-frame comparison, the input sequence is extended by duplicating its final frame 40 times and appending these to the end; GEN3C is conditioned on this extended input, and the first 81 frames of the output are extracted.

  The paper acknowledges that baseline resolution and sequence length differences are handled by inference-time adaptations (Section 9.2 in supplementary), but does not discuss whether these adaptations could introduce artifacts or disadvantage baselines relative to InfCam, which natively generates at the target resolution and length.

• Generation budget / compute accounting. The paper does not use "generation budget" as a primary axis of comparison — unlike the reference example paper, which systematically sweeps generation budgets. Instead, comparisons are at fixed, comparable output specifications (81 frames, comparable resolutions), with each method using its own native inference procedure. Training compute is reported (15k iterations on 4 H100 GPUs, ~1 week for main experiments; 20k iterations on 4 H100 GPUs, ~4 days for ablations), but inference compute is not compared across methods. This is a notable gap: the paper does not establish whether InfCam is more or less computationally expensive at inference than the baselines it outperforms. The warping module and homography-guided attention add per-step computation, but the frozen Wan2.1 backbone means the base model's cost is unchanged. Without FLOPs or wall-clock comparisons, the reader cannot assess the accuracy-efficiency tradeoff.

• Cross-validation / statistical protocol. The paper does not employ cross-validation, bootstrap confidence intervals, or statistical significance testing. Results are reported as single scalar values per metric per method (Tables 1, 2, 3, 4, 5, 6). The AugMCV test set contains 168 scenes (1,680 generated videos); the WebVid test set contains 100 source videos (2,000 generated videos). These are moderate sample sizes, and the lack of variance reporting means the reader cannot assess whether observed differences (e.g., InfCam's RotErr of 3.162 vs. TrajectoryCrafter's 5.007 in Table 2) are statistically reliable or potentially within noise. The ablation study (Table 3) progressively adds components to the same base configuration, which controls for architecture but does not address whether results are stable across random seeds or data splits. This is a weakness in experimental rigor — particularly for a paper claiming state-of-the-art performance, where confidence intervals or at minimum standard deviations would help contextualize the magnitude of improvements.

Main Quantitative Results

Results on AugMCV: Comparison Against Ground-Truth Target Videos

The AugMCV test set provides the cleanest comparison because ground-truth target videos exist, enabling full-reference metrics (Table 1). The headline numbers are clear:

• Shared intrinsics (96 scenes, source and target have identical focal lengths): InfCam achieves PSNR 22.677, SSIM 0.718, LPIPS 0.246. The next-best method is TrajectoryCrafter at PSNR 21.228, SSIM 0.660, LPIPS 0.296. InfCam's improvement over TrajectoryCrafter is +1.449 PSNR, +0.058 SSIM, −0.050 LPIPS — a consistent margin across all three metrics. Over ReCamMaster, the advantage is larger: +1.547 PSNR, +0.101 SSIM, −0.174 LPIPS. Over GEN3C, the gap is dramatic: +5.786 PSNR, +0.239 SSIM, −0.302 LPIPS.

• Different intrinsics (72 scenes, source and target have different focal lengths): InfCam achieves PSNR 22.261, SSIM 0.699, LPIPS 0.270 — remarkably close to its shared-intrinsics performance (PSNR drops only 0.416, SSIM drops only 0.019), indicating genuine focal length adaptation. TrajectoryCrafter achieves PSNR 19.557, SSIM 0.586, LPIPS 0.390 — a much larger drop from its shared-intrinsics performance (−1.671 PSNR, −0.074 SSIM), suggesting it struggles when focal lengths differ. ReCamMaster achieves PSNR 19.665, SSIM 0.558, LPIPS 0.510 — barely better than TrajectoryCrafter and substantially worse than InfCam (+2.596 PSNR advantage for InfCam). GEN3C, interestingly, improves slightly on different intrinsics (PSNR 17.449, up from 16.891 on shared) but remains the weakest method overall.

What these numbers reveal: InfCam's advantage is not uniform — it is largest against GEN3C (which fails to transform viewpoints at all) and most meaningful against TrajectoryCrafter (which is the strongest reprojection-based competitor). The near-identical shared vs. different intrinsics performance for InfCam (22.677 vs. 22.261 PSNR) versus the large gap for TrajectoryCrafter (21.228 vs. 19.557) is perhaps the most informative single comparison: it demonstrates that InfCam has genuinely learned to condition on focal length, while TrajectoryCrafter — which relies on depth reprojection — degrades when intrinsics change, likely because depth estimation and reprojection become less reliable under different field-of-view configurations. This directly supports the paper's claim that the infinite homography decomposition, which explicitly encodes intrinsics in $H_\infty$, provides robustness to focal length variation that reprojection-based methods lack.

Three caveats temper the AugMCV results:

• Synthetic data only. AugMCV is derived from MultiCamVideo, which is rendered in Unreal Engine 5. The strong performance here establishes that InfCam works on synthetic data, but does not guarantee real-world generalization — a separate claim tested on WebVid.

• Resolution mismatches across baselines. TrajectoryCrafter generates at 384 × 672 (vs. InfCam's 480 × 832), which could disadvantage it in pixel-level metrics (PSNR, SSIM) because lower-resolution outputs must be upsampled or compared at native resolution. The paper does not describe how resolution differences were handled for metric computation — were baselines evaluated at their native resolution, or upsampled to match InfCam? This detail matters for the fairness of PSNR/SSIM comparisons.

• Training data overlap. ReCamMaster was trained on MultiCamVideo (the same dataset AugMCV is derived from), so any advantage InfCam shows is not from access to more data but from the augmentation and architectural innovations. However, the augmentation expands the effective training distribution — if ReCamMaster was not trained with similar augmentation, the comparison is fair but should be understood as a method comparison, not a pure data comparison.

Results on WebVid: Real-World Generalization and Pose Fidelity

The WebVid results (Table 2, aggregated across FF-Sync and FF-Async settings; detailed in Tables 5 and 6 of the supplementary) test generalization to real-world videos where no ground-truth target views exist. The headline numbers:

• Camera pose accuracy: InfCam achieves RotErr 3.162°, TransErr 0.438. TrajectoryCrafter, the strongest baseline on this metric, achieves RotErr 5.007°, TransErr 0.735. InfCam's rotation error is 37% lower than TrajectoryCrafter's and 62% lower than GEN3C's (9.588°). Translation error is 40% lower than TrajectoryCrafter's and 85% lower than GEN3C's (3.012). ReCamMaster achieves RotErr 8.375°, TransErr 1.027 — substantially worse than TrajectoryCrafter despite being a trajectory-conditioned method, which the paper attributes to its first-frame preservation bias. ReCamMaster w/ Interp. actually underperforms the standard ReCamMaster on rotation error (9.016° vs. 8.375°) but shows some improvement on translation error (1.590 vs. 1.027 — a degradation, not improvement; the paper's text in Section 5.2 says "exhibit alignment errors, indicating that interpolation alone cannot fully address its trajectory bias").

• Visual fidelity: InfCam achieves FID 29.702, FVD 286.952. TrajectoryCrafter achieves FID 30.877, FVD 289.879 — a marginal advantage for InfCam (FID −1.175, FVD −2.927). ReCamMaster achieves FID 39.930, FVD 302.025. GEN3C achieves FID 43.790, FVD 331.768. The FID/FVD margins are narrow between InfCam and TrajectoryCrafter, suggesting that while InfCam's pose accuracy is substantially better, its visual quality (as measured by distributional metrics) is only modestly improved over the strongest baseline.

The FF-Sync vs. FF-Async breakdown (Tables 5 and 6 in supplementary) reveals important nuance:

• FF-Sync (source and target share first-frame pose): InfCam achieves RotErr 3.605°, TransErr 0.510, FID 32.906, FVD 282.703. TrajectoryCrafter is close: RotErr 5.595°, TransErr 0.502, FID 32.220, FVD 287.805. The pose accuracy gap is smaller here (3.605 vs. 5.595) compared to the FF-Async setting.

• FF-Async (source and target have different first-frame poses — the more challenging setting): InfCam achieves RotErr 2.718°, TransErr 0.365, FID 26.497, FVD 291.202. TrajectoryCrafter achieves RotErr 4.418°, TransErr 0.969, FID 34.476, FVD 313.036. The pose accuracy gap widens substantially: InfCam's rotation error is now 39% lower than TrajectoryCrafter's (2.718 vs. 4.418), and translation error is 62% lower (0.365 vs. 0.969). ReCamMaster shows its bias clearly: it achieves RotErr 7.076° in FF-Async — much worse than in FF-Sync (9.673°) — and actually performs worse in the harder setting despite sharing first-frame pose in the easier one, suggesting the model's first-frame copying strategy fails when the test distribution shifts. ReCamMaster w/ Interp. shows RotErr 8.360° in FF-Async — worse than the original ReCamMaster (7.076°), confirming that interpolation is not an effective fix.

Why InfCam's performance actually improves on the harder FF-Async setting (RotErr 2.718° vs. 3.605° in FF-Sync) is a non-obvious result that the paper does not comment on. Possible explanations: (1) the FF-Async trajectories may, by chance, involve simpler camera motions or more favorable scene geometry, (2) ViPE's pose estimation may be more accurate when the initial viewpoint differs (since there is more visual change to track), or (3) random variation from the 100-video sample. Without confidence intervals, we cannot determine whether this apparent improvement is statistically significant or noise.

Interpreting the WebVid results critically:

• Pose accuracy metrics rely on ViPE, which is itself imperfect. The paper uses ViPE to extract camera trajectories from generated videos and compares against ground-truth target trajectories. But ViPE's accuracy on generated (potentially artifact-containing) videos is not validated — errors in ViPE's estimation will inflate the reported RotErr and TransErr for all methods, potentially masking or exaggerating true differences. This is a fundamental limitation of the evaluation protocol for real-world videos where ground-truth camera poses are unavailable. The paper acknowledges this implicitly by using ViPE for all methods, making it a fair comparison, but the absolute error values (e.g., "3.162°") should be understood as upper bounds on true pose error that include ViPE's estimation error.

• FID and FVD are distributional metrics that capture mode coverage and quality jointly. The narrow margins between InfCam and TrajectoryCrafter (FID 29.702 vs. 30.877) could indicate either genuine visual quality parity or that both methods produce videos within the same distributional neighborhood, with InfCam's advantage manifesting primarily in geometric accuracy rather than visual fidelity. The qualitative results in Figure 5 support this interpretation: TrajectoryCrafter's failures are geometric (artifacts from reprojection errors, highlighted in the orange box) rather than textural.

• The 100-video WebVid subset is small and randomly selected. Without knowing the content distribution of these 100 videos, we cannot assess whether they represent a diverse or skewed sample of real-world scenes. If they happen to contain mostly static scenes with limited depth variation, the task is easier for all methods and differences may be compressed. If they contain complex dynamic scenes, the test is more stringent.

Training Data Strategy Comparison (Table 4)

Table 4 compares InfCam trained on AugMCV against a variant trained on a naive mixture of MultiCamVideo (MCV) and SynCamVideo (SCV), evaluated on WebVid. This is not a main result but is strategically important because it isolates the effect of the augmentation strategy from the architectural innovations:

• AugMCV-trained InfCam: RotErr 3.368°, TransErr 0.839, FID 40.384, FVD 235.563.
• MCV+SCV-trained InfCam: RotErr 4.158°, TransErr 1.441, FID 47.949, FVD 235.958.

The AugMCV-trained model shows a 19% reduction in rotation error and a 42% reduction in translation error compared to naive dataset mixing. FID improves substantially (40.384 vs. 47.949), while FVD is essentially identical (235.563 vs. 235.958). This directly supports the paper's claim that targeted augmentation is more effective than simply combining datasets with different biases — the biases don't cancel; they persist, and the model learns an average that is suboptimal on real data.

Why this comparison matters: SynCamVideo has different initial frame poses for source and target videos, which might be expected to mitigate the first-frame bias in MultiCamVideo. However, the paper shows that joint training on both datasets underperforms the augmentation strategy, suggesting that the model does not automatically disentangle the conflicting biases — it may learn separate "modes" for each dataset and struggle to interpolate between them. The augmentation strategy, by contrast, creates a single distribution where first-frame diversity is explicitly present.

Ablation Studies and Robustness Checks

The paper presents one systematic ablation study (Table 3, Figure 6) that progressively adds the three proposed components: trajectory augmentation, intrinsic augmentation, and the warping module. All ablations use the lower-resolution setting (41 frames, 320 × 544) for efficiency, trained for 20k iterations.

Baseline (no augmentations, no warping module): This configuration achieves PSNR 19.228, SSIM 0.562, LPIPS 0.427 on shared intrinsics, and PSNR 18.480, SSIM 0.507, LPIPS 0.525 on different intrinsics. These are the weakest results by substantial margins, and Figure 6 shows qualitatively that this model "fails to capture both the target trajectory and intrinsics" — the generated frames show viewpoint and field-of-view similar to the source rather than the target specification. This confirms that the frozen Wan2.1 backbone alone, without any of the proposed components, is insufficient for camera-controlled generation — not surprising, since Wan2.1 was not designed for this task and receives no explicit camera conditioning without the new modules.

Adding trajectory augmentation only: PSNR improves to 20.820 (+1.592) on shared intrinsics and 18.865 (+0.385) on different intrinsics. The large gain on shared intrinsics indicates trajectory augmentation effectively breaks the first-frame identity bias. However, the minimal improvement on different intrinsics (only +0.385 PSNR) reveals that trajectory augmentation alone does not teach the model to handle focal length changes — it learns viewpoint variation but still copies the source's field of view when intrinsics differ. Figure 6 confirms this: the trajectory-augmented model shows "a movement consistent with the target trajectory" but "still fails to reflect the target intrinsics."

Adding both trajectory and intrinsic augmentation: PSNR jumps to 22.807 (+1.987 over trajectory-only) on shared intrinsics and 21.866 (+3.001) on different intrinsics. The dramatic improvement on different intrinsics (+3.001 PSNR) confirms that intrinsic augmentation is essential for focal length generalization — the model can now handle scenes where source and target have different fields of view. SSIM improves from 0.523 to 0.649, and LPIPS drops from 0.499 to 0.293. Figure 6 shows results that better match the ground truth in terms of both viewpoint and field of view, though "performance remains inaccurate when compared to the ground truth" — the model understands what changes are needed but lacks the precision to execute them perfectly.

Adding the warping module (full InfCam): This provides the final boost: PSNR 24.412 (+1.605 over augmentation-only) on shared intrinsics, and 24.311 (+2.445) on different intrinsics. The near-parity between shared and different intrinsics (24.412 vs. 24.311, a gap of only 0.101 PSNR) is the key signal: the warping module enables the model to achieve high accuracy regardless of intrinsic configuration, suggesting it has genuinely learned to use the infinite homography to decouple rotation and intrinsics from depth-dependent parallax. SSIM reaches 0.733 (shared) and 0.720 (different), LPIPS drops to 0.198 and 0.203 — both metrics show strong absolute performance and minimal degradation across intrinsic settings.

Non-obvious finding: the warping module's impact is larger on different intrinsics than on shared intrinsics (+2.445 vs. +1.605 PSNR improvement over the augmentation-only baseline). This suggests that the infinite homography warp provides the most value precisely when the geometric transformation is complex (different intrinsics + rotation + translation). When intrinsics are shared, the augmentation-only model can partially compensate through learned representations; when intrinsics differ, the explicit geometric conditioning becomes critical. This validates the paper's central thesis: clean geometric conditioning is most important in exactly the scenarios where learned or estimated approaches are most error-prone.

Negative result: the ReST-EM-style revision doesn't apply here. Unlike the reference paper, InfCam does not involve iterative self-improvement, so there are no negative RL fine-tuning results. However, the paper does contain an implicit negative result: ReCamMaster w/ Interp. does not fix first-frame bias (Tables 2, 6). Despite the reasonable-sounding idea that interpolating camera poses toward the target would help the model transition to a different initial viewpoint, the interpolated variant actually shows worse rotation error than the original in the FF-Async setting (8.360° vs. 7.076°), and only marginal improvement in visual fidelity (FID 30.828 vs. 39.248, but this is still worse than InfCam's 26.497). This negative result is informative because it demonstrates that dataset biases cannot be patched at inference time with simple heuristics — they are baked into the learned model weights and require training-time intervention (the trajectory augmentation strategy) to address.

Missing ablation: the per-frame attention design choice. The paper describes the homography-guided self-attention as performing per-frame attention by treating frames as individual batch items, but does not ablate this choice against alternatives (e.g., joint spatio-temporal attention that allows cross-frame information flow). Would cross-frame attention improve temporal consistency at the cost of breaking temporal alignment? The paper does not investigate.

Missing ablation: the zero-initialized convolution in the warping module. The paper mentions that the warped latent is added to the original through a zero-initialized convolution, following ControlNet practice, but does not ablate this initialization strategy. Would random initialization cause training instability? Would removing the residual connection entirely and using only the warped latent degrade performance? These questions are unanswered.

Missing ablation: latent-space vs. RGB-space warping. The paper warps in VAE latent space "for computational efficiency" but does not compare against RGB-space warping followed by VAE encoding. Given that warping in latent space requires accounting for the VAE's spatial downsampling and may introduce approximation errors (the homography is exact in pixel space but uses adjusted coordinates in latent space), an ablation comparing the two would clarify whether the efficiency gain comes at an accuracy cost.

Missing ablation: the number of homography-guided attention layers. The paper inserts homography-guided attention into every DiT block of Wan2.1. Would inserting it into only a subset (e.g., the first few blocks, the last few blocks, or alternating blocks) perform comparably with fewer parameters? This would clarify whether the geometric conditioning is primarily a low-level feature integration task (early blocks) or a high-level refinement task (late blocks).

Critical Assessment

Claim 1 (from Executive Summary): "InfCam outperforms both reprojection-based and trajectory-conditioned baselines in camera-pose accuracy and visual fidelity."

Supported with qualifications. The AugMCV results (Table 1) clearly show InfCam outperforming all three baselines (GEN3C, ReCamMaster, TrajectoryCrafter) on all metrics under both shared and different intrinsics settings. The WebVid results (Table 2) show InfCam achieving the best rotation error (3.162°), translation error (0.438), FID (29.702), and FVD (286.952). No baseline exceeds InfCam on any metric in either dataset.

However, two qualifications are necessary:

• The visual fidelity margin over TrajectoryCrafter is narrow on WebVid: FID 29.702 vs. 30.877 (a difference of 1.175) and FVD 286.952 vs. 289.879 (a difference of 2.927). Without confidence intervals, it is unclear whether these FID/FVD differences are statistically meaningful. The claim of "outperforming in visual fidelity" is technically true but potentially overstates the practical significance — the main advantage is in camera-pose accuracy, not visual quality. On AugMCV (Table 1), the fidelity margins are larger (e.g., LPIPS 0.246 vs. 0.296 on shared intrinsics), but these are synthetic data where ground-truth comparison is possible; on real-world WebVid data, the fidelity advantage is marginal.

• GEN3C and ReCamMaster have known failure modes that may make them weak baselines for the specific evaluation settings chosen. GEN3C fails to transform the initial frame (consistent with GCD's findings about image-to-video backbone bias), and ReCamMaster preserves the first frame due to training data bias. The paper's evaluation emphasizes these failure modes — the FF-Async setting on WebVid explicitly tests first-frame pose changes, where both baselines are known to struggle. While this is a fair evaluation (real-world use cases do involve different initial poses), the choice of evaluation settings may amplify InfCam's apparent advantage. A more balanced evaluation would weight FF-Sync and FF-Async equally, or would include baselines that do not share these known biases. The paper does not compare against GCD (Van Hoorick et al., 2024) or other methods that may handle first-frame changes better.

Claim 2 (from Executive Summary): "Such improvements are achievable without relying on external depth estimation, whose inaccuracies fundamentally constrain reprojection-based approaches."

Supported, but the mechanism is inferred rather than directly demonstrated. The paper shows that InfCam, which does not use external depth estimation, outperforms reprojection-based methods (TrajectoryCrafter, GEN3C) that do. The ablation (Table 3) shows that the warping module provides the largest single-component gain, and the paper's geometric argument (Section 3) explains why conditioning on noise-free rotation and learning residual parallax should be more robust than conditioning on depth-estimate-based reprojections.

However, the paper never directly demonstrates that depth estimation inaccuracy is the causal factor limiting baselines. To establish this, the paper would need to show, for example: (a) that TrajectoryCrafter's performance degrades as depth estimation error increases (a correlation analysis), (b) that providing TrajectoryCrafter with ground-truth depth (available for AugMCV synthetic data) substantially improves its performance, closing the gap with InfCam, or (c) that InfCam's implicit depth representations are more accurate than the external depth estimator used by baselines. Without such experiments, the claim that depth estimation inaccuracy "fundamentally constrains" reprojection-based approaches remains a plausible and well-motivated hypothesis that is consistent with the results, but is not experimentally proven. The performance gap could arise from other factors — architectural differences, training data differences, or optimization differences — rather than depth error specifically.

Claim 3 (from Executive Summary): "InfCam reduces rotation error to 3.162° and translation error to 0.438 on real-world videos."

Numerically accurate but the evaluation protocol introduces uncertainty. The numbers 3.162° and 0.438 appear in Table 2 and represent averages across the 2,000 generated WebVid videos (100 source videos × 20 trajectories each). However, these metrics are computed using camera trajectories extracted by ViPE, which itself has estimation error. The paper does not validate ViPE's accuracy on generated (potentially artifact-containing) videos, nor does it report ViPE's error on ground-truth videos from AugMCV to calibrate expectation. The reported errors should be interpreted as upper bounds on true pose error that include both the model's geometric inaccuracy and ViPE's estimation inaccuracy. If ViPE's estimation error is, say, 1.5° on average, then InfCam's true rotation error could be substantially lower than 3.162°, and the relative comparison against baselines (which also include ViPE's error) would still be valid. But the absolute numbers should be treated cautiously.

Additionally, the aggregation of FF-Sync and FF-Async results (each 1,000 videos) into a single average (Table 2) masks the setting-dependent performance. InfCam's rotation error is 3.605° on FF-Sync and 2.718° on FF-Async — the aggregate 3.162° is a weighted average that does not reflect either setting individually. Reporting setting-specific numbers (as in Tables 5 and 6) is more informative than the aggregate.

Claim 4 (implicit in the abstract and Section 3): "Conditioning on infinite homography warping enables the model to achieve higher camera-pose fidelity by providing clean rotation information and constraining parallax to epipolar lines."

Supported by ablation but the causal chain has gaps. The ablation in Table 3 shows that adding the warping module improves PSNR by 1.605 on shared intrinsics and 2.445 on different intrinsics over the augmentation-only baseline — a substantial and consistent gain. Figure 6 qualitatively shows improved alignment with the ground truth. These results support the claim that the warping module is beneficial.

However, the paper does not experimentally verify the proposed mechanism — that the benefit comes specifically from (a) providing noise-free rotation and (b) constraining parallax to epipolar lines. Alternative explanations exist: the warping module adds parameters (the zero-initialized convolution), which could provide general capacity benefits; the warping operation provides a form of data augmentation by transforming the latent; or the camera embeddings added after warping provide useful conditioning regardless of the homography's geometric accuracy. To isolate the mechanism, the paper could ablate: (a) using the warping module with random (incorrect) homographies to test whether geometric accuracy matters, (b) using a simpler spatial transformer that warps without the infinite homography formulation, or (c) testing whether the model actually respects epipolar constraints in its implicit parallax predictions. None of these ablations are performed, so the precise mechanism remains a well-supported hypothesis rather than a demonstrated fact.

Concerns About Experimental Design

Single model family (Wan2.1 backbone). All experiments use the frozen Wan2.1 model as the backbone. The paper does not test whether the infinite homography conditioning mechanism generalizes to other diffusion model architectures (e.g., Stable Video Diffusion, CogVideo, or image-based video generation models). This is understandable given compute constraints, but it limits the claim that InfCam is a general framework — it may be that Wan2.1's specific architecture (transformer-based, rectified flow, specific VAE) is particularly amenable to latent-space warping, and results might differ with other backbones.

Single dataset for training (AugMCV). While InfCam is tested on both synthetic (AugMCV test split) and real-world (WebVid) data, it is trained only on AugMCV, which derives from the synthetic MultiCamVideo dataset. The strong WebVid results demonstrate some zero-shot generalization, but the domain gap between synthetic (Unreal Engine 5 rendered scenes) and real-world videos is substantial. Training on a mixture of synthetic and real data (with ground-truth or pseudo-ground-truth camera poses) might improve real-world performance further, but this is not explored. The paper does not analyze whether InfCam's WebVid failures (if any) cluster on scene types underrepresented in MultiCamVideo.

Resolution and frame-length discrepancies across baselines. TrajectoryCrafter generates at 384 × 672 (vs. 480 × 832 for InfCam), and GEN3C generates at 704 × 1280. For AugMCV, pixel-level metrics (PSNR, SSIM) are resolution-dependent — lower-resolution outputs will score worse even if content is correct, simply due to the downsampling loss. The paper does not describe whether baseline outputs were resized to a common resolution for metric computation. For WebVid, FID and FVD are typically computed at a fixed resolution, but the paper does not specify whether all methods' outputs were brought to a common resolution before feature extraction. If not, resolution differences could confound the FID/FVD comparisons.

The AugMCV test set construction may advantage InfCam. InfCam is trained on AugMCV with trajectory and intrinsic augmentation. The AugMCV test set is constructed from the same underlying MultiCamVideo scenes (though with no scene overlap between train and test). If the augmentation strategies introduce patterns that are easier for InfCam's architecture to learn than for baselines, the test set comparison may favor InfCam. This is not a data leakage issue (since scenes are disjoint), but an evaluation bias issue: the test distribution is more similar to InfCam's training distribution than to the baselines' training distributions, because the augmentations define the test set construction. Testing on a completely independent dataset (e.g., RealEstate10K with ground-truth camera poses, or user-captured videos with known trajectories) would provide a cleaner comparison.

Missing experiments that would strengthen the paper:

• Ground-truth depth oracle for reprojection baselines on AugMCV. Since AugMCV is synthetic, ground-truth depth maps are available. Running TrajectoryCrafter and GEN3C with ground-truth depth (instead of estimated depth) would establish an upper bound on reprojection-based performance and directly test the paper's claim that depth estimation inaccuracy is the bottleneck. If the baselines close the gap with InfCam when given perfect depth, the paper's central thesis is strongly validated. If they don't, other factors are at play.

• Component swapping experiment. Take InfCam's architecture but replace the infinite homography warp with a standard reprojection-based conditioning (using UniDepth-estimated depth), and compare. This would isolate whether the architectural innovations (homography-guided attention, end-to-end training) or the geometric decomposition is responsible for the gains.

• Scaling analysis. How does InfCam's performance vary with the number of training scenes, video length, or resolution? The paper uses a fixed dataset size and two resolutions (ablation and main), but a scaling curve would reveal whether performance is saturating or would benefit from more data.

• Failure case analysis. The paper provides qualitative success cases (Figures 1, 5, 6, 9, 10, 11) but does not systematically analyze failure modes — when does InfCam produce incorrect novel views? Are failures clustered on certain camera motions (large translations, extreme rotations), scene types (thin structures, transparency, specular surfaces), or depth ranges (very near or very far objects)? Understanding failure modes would clarify the practical deployability and guide future work.

• User study or perceptual evaluation. The metrics (PSNR, SSIM, LPIPS, FID, FVD, RotErr, TransErr) are all automated. A user study asking human raters to judge which generated video better matches the specified camera trajectory, or which looks more visually natural, would provide complementary evidence — particularly for WebVid where no ground truth exists and distributional metrics may not capture perceptual quality.

Summary of experimental support. The experiments convincingly demonstrate that InfCam, as a complete system, outperforms the specific baselines tested (GEN3C, ReCamMaster, TrajectoryCrafter) on the specific datasets and metrics reported. The ablation shows that each proposed component (trajectory augmentation, intrinsic augmentation, warping module) contributes additively to performance. The paper's central geometric insight — decomposing novel-view synthesis into noise-free rotation conditioning and learned residual parallax — is consistent with the results and well-motivated theoretically, but is not experimentally isolated from alternative explanations. The lack of confidence intervals, the single-backbone evaluation, the resolution discrepancies across baselines, and the absence of a ground-truth depth oracle experiment for reprojection methods are the most significant limitations in experimental rigor.

6. Limitations and Trade-offs

Limitation 1: Single Backbone Architecture — The Infinite Homography Mechanism Has Only Been Validated on Wan2.1

All experiments in Sections 5.2–5.4 and the supplementary use the frozen Wan2.1 text-to-video diffusion model as the backbone. The paper never tests whether the infinite homography warping module, homography-guided self-attention, or camera conditioning strategy transfers to other video diffusion architectures (e.g., Stable Video Diffusion, CogVideo, or image-to-video models that use different VAE compression ratios or attention patterns).

The consequence. It is unknown whether the performance gains — particularly the 4× reduction in rotation error over TrajectoryCrafter on WebVid (Table 2: 3.162° vs. 5.007°) — are attributable to the geometric decomposition generally or to a favorable interaction with Wan2.1's specific design (transformer-based, rectified flow formulation, 3D VAE with 8× spatial compression). Several aspects of the method could be backbone-dependent: (a) the latent-space homography warp assumes the VAE's spatial downsampling preserves sufficient geometric structure for accurate warping — a property that may not hold for VAEs with higher compression ratios or different architectural inductive biases; (b) the homography-guided self-attention layer is inserted into every DiT block and initialized from pretrained Wan2.1 weights — a backbone with different block structure (e.g., U-Net-based) would require architectural adaptation whose effectiveness is unproven; (c) Wan2.1's rectified flow training objective may interact favorably with the velocity prediction task under geometric conditioning in ways that DDPM-style diffusion would not.

The paper's results on WebVid (Section 5.3) demonstrate generalization from synthetic AugMCV training data to real-world videos, but this is a domain generalization test, not an architectural generalization test. A practitioner considering deploying this approach with a different video generation backbone (for reasons of licensing, inference cost, or task specialization) has no evidence about whether the infinite homography conditioning would be equally effective.

What evidence exists in the paper. None. The paper makes no claim of architectural generality and does not ablate the backbone choice. The statement "We adopt the text-to-video model Wan2.1 as our base architecture" (Section 4.1) is descriptive, not comparative. There is no experiment swapping Wan2.1 for an alternative backbone while keeping the camera conditioning components fixed.

Mitigation status. Not addressed. The paper does not discuss this as a limitation, nor does it suggest future work on multi-backbone validation. The choice to freeze Wan2.1 weights is motivated by preserving pretrained video generation capabilities (Section 4.1: "we freeze the pretrained weights of Wan2.1 and train only the newly introduced camera encoder and the homography-guided self-attention layers"), but this does not imply the method is backbone-agnostic.

---

Limitation 2: No Accounting for Inference-Time Cost — the Computational Overhead of Homography-Guided Attention and Warping Is Unmeasured

The paper does not report any measure of inference-time computational cost — no FLOPs, no wall-clock time, no parameter counts for the added modules, and no memory footprint comparisons against baselines.

The consequence. A practitioner cannot assess the accuracy-efficiency tradeoff. The homography-guided self-attention layer is inserted into every DiT block of Wan2.1 (Section 4.1, Figure 3a) and processes three concatenated latent streams (source, target, warped) — this roughly triples the attention computation per DiT block compared to the base Wan2.1 model, since the attention operates on a spatially concatenated tensor with 3× the token count along the spatial dimension ($z_c \in \mathbb{R}^{(bf) \times (3hw) \times d}$, as described in Section 4.1). The warping module adds a per-frame homography warp and a convolutional operation. The camera encoder adds a small linear layer, but its cost is negligible relative to the attention overhead.

Whether this additional cost is justified depends on the application. If InfCam's inference is 3× more expensive than a faster baseline (e.g., a reprojection-based method that uses lightweight depth estimation and conditions only on reprojected images), the accuracy gains documented in Tables 1–2 may or may not represent a favorable tradeoff. The paper frames its contribution as accuracy-focused (abstract: "high pose fidelity"; Section 1: "robust conditioning"), but without cost data, the reader cannot evaluate whether the method is practical for latency-sensitive applications like interactive video editing, or whether the cost scales acceptably with video resolution and frame count.

The baselines themselves have different computational profiles: TrajectoryCrafter requires external depth estimation as preprocessing, GEN3C builds a 3D cache, and ReCamMaster uses a standard trajectory-conditioned diffusion model. Without FLOPs or timing measurements across methods at matched output specifications, the comparison is incomplete.

What evidence exists in the paper. The paper reports training cost — 15,000 iterations on 4 H100 GPUs, approximately one week for main experiments; 20,000 iterations on 4 H100 GPUs, approximately four days for ablations (Section 5.1) — but reports zero inference-time measurements. The paper does not provide parameter counts for the added modules, nor does it compare the number of DiT blocks, attention heads, or latent dimensions between the base Wan2.1 and the augmented InfCam model.

Mitigation status. Not addressed. The paper makes no mention of inference cost as a concern or limitation, and does not suggest efficiency improvements as future work. This omission is particularly notable because several design choices (latent-space warping "for computational efficiency," per-frame rather than spatio-temporal attention, discarding the warped latent after attention) suggest efficiency was a consideration during development, yet no measurements are reported to validate these choices.

---

Limitation 3: Difficulty Estimation for Real-World Videos Relies on an External Depth Estimator (UniDepth) for Source Intrinsics

For real-world videos from WebVid, the source camera intrinsics ($K_s$: focal lengths $f_x$, $f_y$ and principal point $c_x$, $c_y$) are unknown and must be estimated. The paper uses UniDepth (Piccinelli et al., 2024) to estimate these parameters at inference time (Section 5.1, "Evaluation Set"). However, the accuracy of the infinite homography $H_\infty = K_t R K_s^{-1}$ depends directly on the accuracy of $K_s$ — if the estimated source intrinsics are incorrect, the homography warp will be geometrically wrong, and the model's clean rotation conditioning becomes corrupted.

The consequence. The paper's central claim — that InfCam provides "noise-free" rotational conditioning — only holds to the extent that the source intrinsics are accurate. For synthetic data (AugMCV), $K_s$ is known exactly from the dataset, so the claim is valid. For real-world data, the conditioning is only as noise-free as UniDepth's intrinsic estimation. The paper does not quantify UniDepth's intrinsic estimation error on WebVid videos, nor does it analyze how sensitive InfCam's performance is to errors in $K_s$.

This creates an asymmetry: the method is depth-free (it does not estimate depth for reprojection), but it is not intrinsic-free — and inaccurate intrinsics can produce errors that propagate similarly to depth errors in reprojection-based methods. If UniDepth misestimates the focal length by 10%, the infinite homography warp will incorrectly scale the rotation-induced view change, and the model must compensate for this error in its learned parallax prediction — falling into the same "noisy conditioning" trap that the paper criticizes in reprojection-based approaches (Section 3). The paper does not acknowledge this dependency explicitly when discussing the robustness of its approach.

What evidence exists in the paper. The WebVid results (Table 2, Tables 5–6 in supplementary) show InfCam achieving strong pose accuracy (RotErr 3.162°, TransErr 0.438), which suggests that UniDepth's intrinsic estimates are accurate enough on the WebVid subset. However, the paper does not report UniDepth's intrinsic error statistics, ablate InfCam's performance with perturbed intrinsics to test sensitivity, or compare against an oracle where ground-truth WebVid intrinsics (if obtainable via structure-from-motion) are used. The 100-video WebVid subset may contain videos that are favorable for UniDepth (e.g., scenes with clear geometric structure, minimal motion blur); videos where UniDepth fails (e.g., textureless regions, extreme close-ups, non-pinhole camera distortions) may produce worse InfCam results, but this failure mode is unexplored.

Mitigation status. Partially acknowledged. The paper notes that source intrinsics must be estimated for real-world videos (Section 5.1: "we estimate the source intrinsics at inference time using UniDepth") but does not discuss this as a limitation or dependency. The method does not include a mechanism for refining intrinsic estimates during generation or for being robust to intrinsic errors. Future work could involve learning to predict intrinsics directly from video content (removing the external dependency) or training the model to be robust to intrinsic perturbations (by augmenting intrinsics during training, though the paper's intrinsic augmentation only covers focal length changes, not errors).

---

Limitation 4: Failure Mode Analysis Is Absent — It Is Unknown When or Why InfCam Produces Incorrect Novel Views

The paper presents extensive qualitative success cases (Figures 1, 5, 6, 9, 10, 11) but provides no systematic analysis of failure modes. There is no categorization of when InfCam's generated videos deviate from the target trajectory, produce visual artifacts, or fail to maintain appearance consistency with the source.

The consequence. A practitioner evaluating deployment cannot assess risk. The paper shows that InfCam sometimes works well — but does not characterize when it works poorly. This matters for several reasons: (a) in a production video editing tool, occasional failures may be acceptable if they are detectable, but the paper provides no signal about failure detectability (e.g., do failures correlate with high reconstruction loss, unnatural motion, or verifier disagreement?); (b) certain camera motions or scene types may be systematically harder — for instance, large translations that expose large occluded regions, thin structures where parallax is ambiguous, reflective or transparent surfaces that violate the Lambertian assumption implicit in the homography warp — but without failure analysis, a user has no guidance on which trajectories or videos are safe to use; (c) the strong quantitative results (Tables 1–2) aggregate performance across diverse trajectories and scenes, potentially masking catastrophic failures on a subset that would make the method unreliable for certain applications.

The paper implicitly suggests the method handles a wide range of camera motions — Figure 11 in the supplementary shows results under eight different trajectories — but does not report per-trajectory-type metrics or analyze which trajectory categories (arc, pan, tilt, translation, random) are most challenging. The AugMCV test set contains 1,680 generated videos (168 scenes × 10 trajectories each), providing sufficient data for per-trajectory-type analysis that is not performed.

What evidence exists in the paper. Only qualitative successes. The paper does not include a single figure or example demonstrating a failure case, nor does it discuss limitations in the qualitative results section (Section 5.2). The quantitative metrics aggregate all trajectories and scenes into single numbers, providing no distributional information about worst-case performance.

Mitigation status. Not addressed. The paper does not acknowledge the lack of failure analysis as a limitation, nor does it suggest future work on characterizing failure modes. This is perhaps the most significant gap for practical deployment — a method that works 90% of the time but fails catastrophically 10% of the time may be less useful than a method with slightly lower average accuracy but more predictable failure behavior. The absence of failure cases also limits scientific understanding: without knowing why the method fails when it does, it is difficult to identify which component (warping module, homography-guided attention, training data coverage) is the bottleneck.

---

Limitation 5: The AugMCV Dataset Augmentation Assumes the Availability of Multi-View Synchronized Video with Ground-Truth Poses — a Severely Constrained Data Regime

The entire training pipeline depends on the MultiCamVideo dataset, which provides 10 perfectly synchronized camera trajectories with ground-truth SE(3) poses for each of 13,600 synthetic scenes (Section 4.2). The trajectory augmentation (reversal and concatenation) and intrinsic augmentation (focal length resizing and center cropping) both assume this synchronized multi-view structure. The paper does not train on any real-world video data, and does not demonstrate that InfCam can be trained (or fine-tuned) on monocular video with pseudo-ground-truth poses estimated via structure-from-motion.

The consequence. The training data regime is fundamentally synthetic and multi-view. This has two implications:

(a) Domain gap. AugMCV is rendered in Unreal Engine 5 (Section 4.2, citing the MultiCamVideo dataset from Bai et al., 2025). While InfCam generalizes to WebVid (Table 2), the 100-video subset is small, and the paper does not analyze whether generalization failures correlate with scene properties that differ between Unreal Engine and real-world footage (lighting complexity, material diversity, motion blur, sensor noise, non-Lambertian surfaces, dynamic range). The WebVid results demonstrate some generalization, but the paper cannot claim robustness to arbitrary real-world videos — the generalization gap may be larger than the WebVid results suggest if the 100-video subset happens to favor scenes that render well in Unreal Engine (e.g., outdoor scenes with diffuse surfaces, limited occlusions).

(b) Data availability barrier to replication and extension. Practitioners who want to apply InfCam to a new domain (e.g., egocentric video, drone footage, medical imaging) would need to construct a dataset of multi-view synchronized video with ground-truth camera poses — an expensive and in many cases infeasible requirement. The paper does not provide evidence that InfCam can be trained with monocular video and estimated poses (e.g., using COLMAP or particleSfM for pose estimation, and monocular depth for geometric supervision), which would dramatically expand the range of applicable training data. Without this, InfCam's training recipe is restricted to settings where multi-view capture or high-quality rendering is available.

What evidence exists in the paper. The paper uses only AugMCV for training. The WebVid evaluation (Section 5.3) demonstrates zero-shot generalization, but there is no experiment where InfCam is fine-tuned on real videos, nor is there an analysis of how performance degrades as a function of the domain gap (e.g., comparing AugMCV scenes that are visually similar to WebVid vs. those that are dissimilar). The paper does not report the content distribution of MultiCamVideo/AugMCV (e.g., indoor vs. outdoor ratio, presence of humans, dynamic object categories), making it impossible to assess coverage of real-world scenarios.

Mitigation status. Not addressed. The paper does not discuss this as a limitation, nor does it suggest training on real-world video data or using estimated poses as future work. The abstract claims "generalizing well from synthetic to real-world data," which the WebVid results partially support, but the claim should be qualified by noting that training remains exclusively synthetic.

---

Limitation 6: Lack of Statistical Rigor — No Confidence Intervals, Error Bars, or Significance Tests Are Reported

All quantitative results in Tables 1–6 and the ablation (Table 3) are reported as single scalar point estimates without any measure of variance — no standard deviations, no confidence intervals, no statistical significance tests between methods, and no indication of whether results are averaged over multiple random seeds.

The consequence. The reader cannot assess whether observed performance differences are statistically reliable or potentially within sampling noise. Several comparisons raise this concern:

• The visual fidelity margin over TrajectoryCrafter on WebVid (Table 2) is narrow: FID 29.702 vs. 30.877 (difference 1.175), FVD 286.952 vs. 289.879 (difference 2.927). On a 100-video test set, these differences may fall within one standard deviation of the metric's sampling distribution, especially given that FID and FVD are known to exhibit high variance on small sample sizes. Without error bars, the claim that InfCam "outperforms" on visual fidelity is technically correct directionally but its practical significance is unquantified.

• The ablation study (Table 3) adds components incrementally to a single base configuration. If run with a single random seed for training and a single train/test split, the observed improvements (e.g., +1.592 PSNR from trajectory augmentation, +1.605 from the warping module) could be partially attributable to random variation in optimization or data ordering. The paper does not report seed sensitivity or provide multiple training runs to establish variance.

• The AugMCV test set contains 168 scenes (1,680 generated videos), and the WebVid set contains 100 source videos (2,000 generated videos). These are moderate sample sizes where metric variance is non-negligible. Per-scene or per-video variance in PSNR, SSIM, and LPIPS could be substantial (e.g., some scenes may be intrinsically harder for all methods, inflating variance), but the paper aggregates everything into scene-level means without reporting distributions.

• The WebVid metric computation depends on ViPE for camera trajectory extraction (Section 9.3 in supplementary). ViPE's estimation error contributes to the reported RotErr and TransErr, but the paper does not report ViPE's own accuracy on ground-truth trajectories from AugMCV (where ground truth is available) to calibrate this error. If ViPE's rotation error on ground-truth AugMCV videos is, say, 1.5°, then the true pose error of all methods is lower than reported, and the relative comparisons may compress or expand depending on whether ViPE's error is correlated with video quality or method.

What evidence exists in the paper. The paper reports only point estimates. No error bars appear in any figure or table. No mention is made of statistical testing, cross-validation folds (beyond the train/test split), or multiple random seeds. The ablation study (Table 3) implies additive improvements but without variance estimates, the reader cannot assess whether, for example, the difference between "trajectory augmentation only" and "trajectory + intrinsic augmentation" (PSNR 20.820 vs. 22.807 on shared intrinsics) is robust or fragile.

Mitigation status. Not addressed. The paper does not discuss this as a limitation. In fairness, the field of video generation often reports single-point estimates without variance, and the paper's metrics follow conventions (PSNR, SSIM, LPIPS, FID, FVD, RotErr, TransErr) that are standard in the camera-controlled generation literature. However, for a paper claiming state-of-the-art performance with specific numerical margins, the absence of variance reporting weakens the strength of the conclusions — particularly for comparisons where the margins are narrow (FID/FVD vs. TrajectoryCrafter) or where the evaluation pipeline introduces unquantified noise (ViPE-based pose estimation).

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a conceptual reframing of camera-controlled video generation — shifting the core challenge from "how do we estimate depth and reproject?" to "how do we separate the geometric transformations we know with certainty from those we must learn?" The specific mechanism (infinite homography warping in latent space) is novel, but the more important shift is diagnostic: the paper demonstrates that both dominant paradigms — reprojection-based and trajectory-conditioned methods — share a structural vulnerability to noisy geometric conditioning, and that this vulnerability can be addressed by providing what is known (rotation, expressed via H∞) as a clean conditioning signal while learning what is unknown (depth-dependent parallax) end-to-end.

The magnitude of this shift is best characterized as a diagnostic reframing with practical consequences, not a paradigm overthrow. Reprojection-based methods (GEN3C, TrajectoryCrafter) are not rendered obsolete — they remain viable when depth estimation is reliable — but the paper establishes that their performance ceiling is fundamentally bounded by depth accuracy in a way that a depth-free decomposition can circumvent. The 37% reduction in rotation error over TrajectoryCrafter on WebVid (3.162° vs. 5.007°, Table 2) and the near-identical performance across shared vs. different intrinsics on AugMCV (PSNR 22.677 vs. 22.261, Table 1) are not just accuracy improvements — they are evidence that the failure mode has shifted. InfCam degrades gracefully when intrinsics change, while reprojection-based methods do not. This shift from "the method fails when depth fails" to "the method fails when the learned model fails to predict parallax" is subtle but important: the former failure mode is determined by an external, fixed estimator; the latter is determined by the model itself, which can be improved with more data, better architectures, or longer training.

The paper also reconciles a tension in the trajectory-conditioned literature that was previously puzzling. ReCamMaster (Bai et al., 2025) and SynCamMaster (Bai et al., 2025) showed promising results on their training distributions but struggled with out-of-distribution starting poses — a limitation that could be dismissed as "not enough data" or "insufficient model capacity." This paper provides a specific, testable explanation: the dataset bias of identical first frames across all source-target pairs creates a shortcut that the model learns, and no amount of additional data with the same structural bias will fix it. The trajectory augmentation strategy (Section 4.2.1) — reversing and concatenating trajectories to break first-frame identity — is not just a data augmentation trick; it is a targeted surgical removal of a specific shortcut. Table 4 confirms this: naive dataset mixing (MCV + SCV, which provides different first frames but without the temporal coherence of the reversal strategy) underperforms AugMCV (RotErr 4.158° vs. 3.368°), showing that diversity alone is insufficient — the diversity must be constructed to directly counter the learned shortcut.

Research directions that become more attractive:

• Geometry-aware conditioning for diffusion models. The paper demonstrates that exact geometric transformations can be injected into frozen pretrained video diffusion models through latent-space warping and dedicated attention layers. This opens the door to similar conditioning for other controlled generation tasks where part of the transformation is specifiable in closed form: known object rotations (e.g., generating views of an object from specified azimuth/elevation, where the object's rotation relative to the camera is known), known lighting changes (where the direction of a point light source can be encoded geometrically), or known non-rigid deformations (where a warp field is computable from user controls). The key design pattern — freeze the generative backbone, add lightweight conditioning layers initialized from pretrained weights, provide exact geometric warps as additional attention inputs — is generalizable beyond the specific infinite homography formulation.

• Training data bias diagnosis as a methodology. The paper's approach to identifying and surgically removing specific dataset biases (first-frame identity, focal-length copying) is methodologically significant. Rather than vaguely calling for "more diverse data," the paper asks: what pattern in the training data would allow a model to achieve low loss without learning the intended task? The answer (identical first frames, identical focal lengths) leads directly to targeted augmentations. This diagnostic methodology — identify the shortcut, construct training examples that break it, verify that the shortcut is eliminated — could be applied to other controlled generation tasks where models exhibit unexplained failure modes.

Research directions that become less attractive:

• Improving depth estimation for reprojection-based camera control. The paper's core argument — that reprojection errors are non-recoverable because depth estimation runs without gradient flow — suggests that incremental improvements to depth estimators will yield diminishing returns for camera-controlled video generation. Even a perfect depth estimator would not solve the problem that reprojection-based methods cannot refine depth during training. Unless future work integrates depth estimation as a differentiable module within the generation pipeline (allowing gradients to flow back through depth prediction), the reprojection paradigm faces a structural ceiling that InfCam circumvents.

• Pure trajectory-conditioned methods trained on synthetic data without bias mitigation. The paper's evidence (ReCamMaster failing to transform initial frames, Section 5.2; AugMCV outperforming MCV+SCV joint training, Table 4) suggests that training trajectory-conditioned models on datasets with structural biases — even large-scale, high-quality synthetic datasets — will produce models that learn shortcuts rather than genuine viewpoint transformation, unless the training data is explicitly constructed to eliminate those shortcuts.

Follow-Up Research This Work Enables

Ground-truth depth oracle for reprojection baselines on AugMCV. The paper argues that depth estimation inaccuracy is the root cause of reprojection-based methods' failures, but never directly tests this claim. A high-priority follow-up would run TrajectoryCrafter and GEN3C on the AugMCV test set using ground-truth depth maps (available because AugMCV is synthetic, rendered in Unreal Engine 5) instead of their standard estimated depth. If the baselines close the performance gap with InfCam when given perfect depth — e.g., TrajectoryCrafter's PSNR rises from 21.228 to near InfCam's 22.677 on shared intrinsics (Table 1) — the paper's central diagnostic claim is strongly validated. If the gap persists, factors beyond depth accuracy (architectural limitations, training data mismatches, inference-time heuristics) are at play, which would refine our understanding of why InfCam outperforms. A negative result here — reprojection baselines with perfect depth still underperforming InfCam — would imply that InfCam's end-to-end parallax learning confers benefits beyond just avoiding depth errors, such as implicit refinement of camera pose alignment or better handling of occlusion boundaries.

ViPE calibration on AugMCV ground-truth trajectories. The WebVid pose accuracy metrics (RotErr, TransErr in Table 2) depend on ViPE for camera trajectory extraction from generated videos, but ViPE's own estimation error on generated (potentially artifact-containing) videos is unknown. A critical calibration experiment would run ViPE on the ground-truth AugMCV test videos (where ground-truth poses are available) and report its rotation and translation errors on clean, synthetic, artifact-free videos. If ViPE's error is, say, 1.5° rotation on AugMCV ground-truth, then the reported 3.162° for InfCam on WebVid should be interpreted as containing approximately 1.5° of ViPE-induced noise, and the true pose error is lower. If ViPE's error varies systematically with video quality (e.g., higher error on videos with artifacts), the relative comparisons between InfCam and baselines could be biased — a method that produces fewer visual artifacts (like InfCam, based on FID/FVD in Table 2) could appear to have better pose accuracy simply because ViPE estimates its trajectory more accurately, not because its geometry is truly better. This calibration would establish confidence bounds on the WebVid results and is essential before the pose accuracy numbers can be used for method comparison.

Cross-backbone validation: infinite homography conditioning on Stable Video Diffusion or CogVideo. InfCam's architectural components — latent-space infinite homography warp, homography-guided self-attention, camera encoder — are validated only on the frozen Wan2.1 backbone. A follow-up study would port these components to a different video diffusion model (e.g., Stable Video Diffusion, which uses a U-Net rather than a transformer, or CogVideo, which uses a different VAE and attention pattern) and measure whether the performance gains transfer. The key question: is the infinite homography decomposition beneficial because it provides useful geometric information to any video diffusion model, or because Wan2.1's specific architecture (transformer with per-token attention, rectified flow formulation, 3D VAE with 8× spatial compression) is particularly well-suited to leveraging latent-space geometric warps? A negative result — the method failing to improve over reprojection baselines when using a non-Wan2.1 backbone — would bound the method's generality and suggest that backbone-specific adaptations (e.g., VAE compression ratio, attention pattern) are necessary. This experiment would require retraining InfCam's added modules on the new backbone using the AugMCV dataset and evaluating on both AugMCV and WebVid, following the paper's protocol.

Failure mode taxonomy: per-scene and per-trajectory-type analysis on WebVid. The paper reports aggregate metrics over all 100 WebVid source videos and 20 trajectories each, but never analyzes which scenes or trajectory types cause InfCam to fail. A systematic follow-up would categorize the 2,000 generated WebVid videos by: (a) trajectory type (pan, tilt, translation, arc, random), (b) scene content (indoor vs. outdoor, static vs. dynamic objects, presence of thin structures, depth range), (c) translation magnitude (small, medium, large relative to scene depth), and (d) rotation magnitude. Per-category RotErr and TransErr would reveal systematic failure modes — for instance, large translations that expose large occluded regions may produce higher error because the model must hallucinate content it has never observed, or thin structures (fences, hair, tree branches) may cause failure because the homography warp produces ambiguous correspondences. This analysis would use the existing WebVid generated videos (no new model runs needed) and would provide actionable guidance for practitioners: which trajectories and scenes are safe to use InfCam on, and which require manual intervention or alternative methods.

Perturbation analysis: sensitivity of InfCam to source intrinsic estimation errors. For real-world videos, InfCam relies on UniDepth to estimate source camera intrinsics (focal lengths, principal point), and the infinite homography H∞ depends on these estimates. A controlled experiment would take AugMCV test scenes (where ground-truth source intrinsics are known), perturb the source intrinsics by known amounts (e.g., ±5%, ±10%, ±20% in focal length; ±10, ±20 pixels in principal point), and measure the degradation in InfCam's PSNR/SSIM/LPIPS as a function of perturbation magnitude. This would establish the method's tolerance to intrinsic estimation errors — a practical question for deployment on real-world videos where UniDepth's accuracy varies with scene content. A flat degradation curve (small performance drop up to ±20% focal length error) would indicate that the end-to-end training learns to compensate for intrinsic inaccuracies, validating InfCam's robustness claim. A sharp degradation curve would indicate that clean intrinsics are as critical to InfCam as clean depth is to reprojection methods, narrowing the paper's claimed advantage. The experiment would also reveal which intrinsic parameter (focal length vs. principal point) the model is more sensitive to, guiding deployment: if focal length errors hurt more, practitioners should prioritize accurate focal length estimation (e.g., from EXIF data when available) over other intrinsics.

Differentiable depth estimation within the reprojection pipeline — can it close the gap? The paper's central critique of reprojection-based methods is that depth estimation runs without gradient flow, making errors non-recoverable during training. A natural follow-up would test whether making depth estimation differentiable and jointly training it with the diffusion model (as a single end-to-end pipeline) allows reprojection-based methods to match or exceed InfCam's performance. The experiment would take TrajectoryCrafter's architecture, replace its frozen depth estimator with a lightweight learnable depth prediction network (initialized from UniDepth or a similar pretrained model), allow gradients to flow from the diffusion loss back through the depth predictor, and train on AugMCV using the same protocol as InfCam (frozen backbone, trainable conditioning layers). If the jointly-trained reprojection pipeline matches InfCam's performance, it would suggest that the key innovation is end-to-end gradient flow rather than the infinite homography decomposition specifically — and that reprojection-based methods could be rehabilitated by incorporating differentiable depth modules. If it underperforms, the infinite homography decomposition provides benefits beyond just gradient flow (e.g., the epipolar constraint on parallax search space, the clean separation of rotation from translation), further validating the paper's approach.

Practical Applications and Downstream Use Cases

Post-production camera trajectory editing for film and video. The most direct application is a tool that lets video editors specify a target camera trajectory and generate a new video that follows it, using only a single source video as input. Based on the WebVid results (Table 2), InfCam achieves rotation error of 3.162° and translation error of 0.438 on real-world videos with diverse content. In a post-production setting, an editor could film a scene once with a static or simple camera motion, then in post apply complex camera trajectories — arcs, dollies, pans — to create cinematic sequences without reshoots. The 37% reduction in rotation error over the best baseline (TrajectoryCrafter: 5.007°) translates directly to more visually coherent results: a 3.2° rotation error is roughly the angular deviation noticeable when comparing two views of a scene at typical viewing distances, while a 5.0° error would produce visibly misaligned geometry. The key practical advantage over reprojection-based tools is robustness: InfCam's near-identical performance across shared and different intrinsics (Table 1: PSNR 22.677 vs. 22.261) means editors can simultaneously change camera trajectory and field of view (e.g., zooming in while dollying), which would cause reprojection-based methods to degrade. The main deployment barrier is inference cost, which the paper does not measure — for interactive editing where latency matters, this is a critical unknown.

Synthetic data generation for training 3D vision models. Camera-controlled video generation can produce training data for tasks that require paired multi-view video — for instance, training depth estimation models, novel-view synthesis models, or 3D object detectors. InfCam's AugMCV training procedure (Section 4.2) generates 47,600 videos from 13,600 scenes through augmentation, and the method can generate additional novel views at inference time by specifying arbitrary target trajectories. A practitioner could take an existing dataset of monocular videos (e.g., driving footage, egocentric video), estimate source intrinsics with UniDepth, and use InfCam to generate synchronized multi-view versions — effectively converting monocular data into multi-view data. The strong pose accuracy on WebVid (TransErr 0.438, Table 2) suggests the generated views would be geometrically consistent enough to serve as training signal for downstream 3D tasks. Compared to using reprojection-based methods, InfCam's depth-free approach avoids systematic geometric biases that could cause downstream models to learn incorrect 3D representations. However, the AugMCV training data is synthetic (Unreal Engine), so InfCam's generalization to specialized domains (medical imaging, satellite imagery, microscope footage) is untested and likely poor — domain-specific fine-tuning or training data would be needed.

Consumer video enhancement with virtual camera motion. A consumer-facing application could enable smartphone users to capture a video and apply cinematic camera motions — subtle dolly zooms, parallax-revealing pans, or arc shots — as a post-capture effect. The key advantage over existing consumer video stabilization or cinematic modes (e.g., Apple's Cinematic Mode, which only adjusts focus depth) is that InfCam synthesizes genuinely novel viewpoints, not just 2D transformations of existing frames. The WebVid results on 100 real-world videos (FID 29.702, FVD 286.952, Table 2) indicate that generated videos maintain visual quality comparable to the strongest baseline (TrajectoryCrafter: FID 30.877, FVD 289.879), meaning the synthetic camera motions would not introduce visible artifacts beyond what current video generation methods produce. The practical constraint is inference time: generating a video with novel camera motion on a smartphone would likely require cloud inference, making it more suitable for a cloud-based editing app than on-device real-time processing. The paper's lack of inference cost measurement makes this assessment speculative — if the homography-guided attention triples per-step compute (as suggested by the 3× concatenation), mobile deployment is infeasible without model distillation or architectural optimization.