ArXiv: 2512.15702

🎯 Pitch

An autoregressive video model trained from scratch can match distilled quality on long videos—but only if it learns from its own flawed previous outputs. Resampling Forcing simulates inference-time error during training by degrading history frames with the model's current denoising trajectory, eliminating the need for a bidirectional teacher.

1. Executive Summary

This paper introduces Resampling Forcing, an end-to-end, teacher-free training framework for autoregressive video diffusion models that mitigates exposure bias — the train-test mismatch that causes error accumulation during inference — without requiring a bidirectional teacher or online discriminator. Experiments use the WAN2.1-1.3B architecture evaluated on VBench, comparing against recent autoregressive baselines including Self Forcing, CausVid, and LongLive. The framework’s core mechanism is a self-resampling scheme (autoregressively degrading history frames with the online model’s own denoising trajectory to simulate inference-time errors) paired with history routing (a parameter-free, head-wise top-k retrieval of the most relevant history frames for each query token, maintaining near-constant attention complexity). The method achieves generation quality comparable to distillation-based baselines while achieving 75% attention sparsity with negligible quality loss and exhibits superior temporal causality — establishing that end-to-end trained autoregressive video models can match distilled counterparts in long-horizon visual quality only when the history conditioning distribution accurately reflects the model’s own inference-time error mode.

2. Context and Motivation

The Core Problem: Exposure Bias in Autoregressive Video Diffusion

This paper addresses a fundamental training-inference mismatch in autoregressive video diffusion models: exposure bias. The problem, formalized in Section 3.1, arises from a simple but devastating discrepancy in how these models are trained versus how they are deployed.

During training, autoregressive video models are almost universally trained with teacher forcing. The model learns to predict frame ii conditioned on the ground truth history frames x<i\boldsymbol{x}^{<i} — the actual, clean video frames from the training dataset. The training objective in Equation 4 makes this explicit: the velocity prediction network vθ(xti,x<i,t,c)\boldsymbol{v}_{\theta}(\boldsymbol{x}^{i}_{t}, \boldsymbol{x}^{<i}, t, c) receives the pristine, noise-free ground truth history as its conditioning signal. This is computationally elegant — a causal mask enables parallel training across all frames simultaneously, as shown in Figure 3(b,c) — and it works perfectly during training because the model always sees perfect inputs.

The problem emerges at inference time. Once deployed, the model must generate each frame xi\boldsymbol{x}^{i} conditioned on its own previously generated history frames x~<i\tilde{\boldsymbol{x}}^{<i}, as shown in Equation 5. These self-generated frames are inevitably imperfect — they contain artifacts, subtle distortions, and statistical deviations from the ground truth distribution. The authors refer to this discrepancy as model error. The model has never seen degraded histories during training, so it treats these imperfections as genuine input signals and propagates them forward through the autoregressive loop. Small errors in early frames become larger errors in later frames, which in turn corrupt the conditioning for even later frames, creating a compounding cascade that the paper visualizes conceptually in Figure 2 (top) and empirically demonstrates in Figure 1 (top): what starts as minor quality degradation in the first few seconds of video generation escalates into complete visual collapse — color shifts, texture degradation, loss of object coherence — as the video extends to 10–15 seconds.

This is not merely a theoretical concern. It represents a hard ceiling on the practical utility of autoregressive video models. No matter how good the base model architecture is, if every generated video eventually collapses as it gets longer, the model cannot fulfill the core promise of autoregressive video generation: open-ended, temporally coherent simulation of dynamic scenes.

Why This Problem Matters: World Simulation and Beyond

The paper situates autoregressive video generation within a broader vision articulated in the introduction (Section 1): world simulation. The idea is that video generation models that can faithfully predict future frames conditioned on past observations are, in effect, learning to approximate physical dynamics, object interactions, and causal relationships in the visual world. This vision is not speculative — it builds on demonstrated capabilities of bidirectional video models like SoRA (Brooks et al., 2024) for approximating physical behavior.

Autoregressive generation is the necessary paradigm for world simulation because the physical world is fundamentally causal: the future depends on the past, but the past does not depend on the future. Bidirectional models that can attend to both past and future frames during generation inherently violate this causal structure — they can "peek" at future information when generating earlier frames, which makes them unsuitable for true forward prediction. Only autoregressive factorization, where each frame is generated conditioned solely on its history as expressed in Equation 1, respects temporal causality.

Beyond world simulation, the paper identifies several applications where exposure bias directly undermines practical deployment. Game simulation (Valevski et al., 2024; Alonso et al., 2024) requires stable frame-by-frame generation where player actions at each step produce coherent, predictable visual outcomes — error accumulation would cause game states to drift into visual nonsense. Interactive content creation (Lin et al., 2025; Shin et al., 2025) demands that user inputs at any point in a generated video produce meaningful, persistent changes without degrading the overall visual quality. Temporal reasoning tasks (Wiedemer et al., 2025) rely on maintaining consistent object identities and relationships across extended sequences.

The problem also has economic and scalability implications. If addressing exposure bias requires expensive post-training pipelines with auxiliary teacher models or online discriminators, then training competitive autoregressive video models becomes prohibitively complex for all but the most well-resourced organizations. An end-to-end solution that works from scratch would democratize access to this technology.

Prior Approaches and Where They Fall Short

The paper surveys four families of prior attempts to address exposure bias in autoregressive video generation, each with distinct limitations that motivate the need for the proposed approach.

1. Noise Injection into History Frames (Section 2, "Autoregressive Video Generation")

The simplest approach, adopted by works like GameNGen (Valevski et al., 2024) and ART-V (Weng et al., 2024), is to add small amounts of Gaussian noise to the ground truth history frames during training. The intuition is straightforward: if the model learns to be robust to slightly noisy inputs, it will better handle the noisy outputs it encounters during inference.

The paper identifies this as a distributional mismatch problem. Gaussian noise does not resemble the actual error patterns of a diffusion model's outputs. As the ablation in Table 2 demonstrates, noise augmentation ("noise augmentation" row) achieves the worst performance among error simulation strategies (Temporal: 87.15, Visual: 61.90) compared to approaches that use actual model resampling. The authors attribute this to the qualitative difference between unstructured additive noise and the structured, frequency-dependent errors that real diffusion models produce — diffusion errors primarily affect high-frequency details (Falck et al., 2025), not uniform pixel-level noise.

2. Diffusion Forcing (Section 2)

Diffusion Forcing (Chen et al., 2025a) and its video extensions (Song et al., 2025; SkyReels-V2 by Chen et al., 2025b) take a different approach: during training, each frame is assigned an independent, randomly sampled noise level. This means the model learns to condition on history frames at arbitrary stages of the denoising process, which in principle should allow it to handle the partially-denoised frames it encounters during autoregressive rollout.

However, this approach has a subtle limitation: it trains for robustness to arbitrary noise levels, not specifically to the noise levels that actually occur during autoregressive inference. During inference, the previous frame is fully denoised (noise level 0) before the next frame begins generation — the model conditions on clean (but imperfect) outputs, not noisy intermediates. Diffusion Forcing's training distribution is broader than the inference distribution, so while it provides some robustness, it does not target the specific failure mode of compounding artifacts across clean frames. The paper does not directly ablate against Diffusion Forcing, but the logic of Resampling Forcing — simulating the exact inference trajectory — implicitly argues for specificity over generality.

3. Relaxed Causality Approaches (Section 2)

Several works circumvent strict autoregressive factorization by relaxing temporal causality. The rolling denoising framework (Ruhe et al., 2024; MAGI-1 by Teng et al., 2025; AR-Diffusion by Sun et al., 2025a; Xie et al., 2025) maintains non-decreasing noise levels across a sliding window — the model begins denoising the next frame before the current frame is fully generated, creating overlap between the generation processes. The clip-level autoregressive approach (SkyReels-V2) generates entire 5-second video segments autoregressively rather than individual frames, reducing the number of autoregressive steps where error can accumulate. The plan-interpolate strategy (Zhang and Agrawala, 2025) generates sparse keyframes and interpolates between them.

The paper's qualitative analysis in Figure 5 (upper panel) demonstrates that these relaxed approaches do achieve better long-horizon visual quality than strict autoregressive baselines. MAGI-1 and SkyReels-V2 maintain more stable colors and textures over 15 seconds compared to the progressive degradation seen in Pyramid Flow, CausVid, and Self Forcing.

But the paper argues this comes at a fundamental cost: compromised temporal causality. For world simulation and interactive applications, the ability to condition each frame strictly on its genuine past is non-negotiable — it is what enables the model to serve as a forward simulator of physical dynamics. Relaxed causality breaks this property, and the paper's position is that solving exposure bias within a strictly causal framework is therefore the harder but more important problem.

4. Post-Training Distribution Matching (Self Forcing and Its Descendants)

The most directly competitive approach, and the one the paper engages with most deeply, is Self Forcing (Huang et al., 2025a). Self Forcing operates in two stages:

  1. Rollout: The autoregressive model generates a complete video sequence autoregressively, simulating inference conditions.
  2. Distribution matching: The generated video is compared against real data using either an adversarial loss with an online discriminator (Goodfellow et al., 2020) or distillation loss against a pre-trained bidirectional teacher model (e.g., WAN2.1-14B).

This directly addresses the train-test mismatch because the training signal computed in step 2 reflects the model's actual inference-time behavior. LongLive (Yang et al., 2025), a concurrent work, extends this by training on longer rollouts and applying sub-clip distillation — breaking the long generated video into 5-second segments and matching each against the teacher's output on the corresponding real video segment.

The paper identifies three specific failure modes of this distillation paradigm:

Failure mode 1: Scalability and training-from-scratch impracticality. Self Forcing requires a pre-trained bidirectional teacher model (14B parameters for WAN2.1-14B, as noted in Table 1) or an online discriminator network. This creates a chicken-and-egg problem: to train an autoregressive model that doesn't need a teacher, you need a teacher. The paper's Resampling Forcing eliminates this dependency entirely, enabling end-to-end training with no auxiliary models — a point emphasized in both the abstract and the comparison table (Table 1, where "Teacher Model" column is blank for the proposed method).

Failure mode 2: Future information leakage through the bidirectional teacher. This is the most theoretically interesting critique, demonstrated in Figure 5 (lower panel). Even when the student architecture is strictly causal (attends only to past frames), distillation from a bidirectional teacher can transfer non-causal behaviors. The paper provides a concrete example: in a "milk pouring" scenario, LongLive's generated video shows a liquid level that rises and then falls despite continuous pouring — a physical impossibility. The authors attribute this to the bidirectional teacher's attention mechanism allowing future frames to influence earlier generations, and this information leaking to the student during distillation.

The paper traces this to two mechanisms: (1) the teacher's bidirectional attention directly mixes future context into earlier frame representations, and (2) sub-clip distillation emphasizes local appearance quality over global temporal consistency. By contrast, Resampling Forcing's training never accesses future frames for conditioning, so there is no pathway for future information to contaminate the learned dynamics. The liquid level in their generated video monotonically increases while the source container empties — correct physical causality.

Failure mode 3: Degradation on longer videos due to short-teacher training. Figure 1 (middle) illustrates this pattern conceptually. Self Forcing is distilled from a teacher trained on short videos (5 seconds in the WAN2.1-14B case). When the student generates longer videos (15+ seconds), the quality degrades because the teacher's supervision was only ever applied to short clips. LongLive attempts to address this by rolling out longer videos and applying sub-clip distillation, but this introduces the causality leakage problem described above. Resampling Forcing, by training natively on long videos (the paper trains on 15-second sequences), avoids this extrapolation problem entirely (Figure 1, bottom).

How This Paper Positions Itself

The paper frames its contribution through a clear conceptual innovation rather than an incremental improvement. The key insight, articulated in Section 3.2 and visualized in Figure 2, is to reframe the problem from "eliminating model errors" to "training robustness against model errors."

The traditional teacher forcing paradigm implicitly assumes that if the model is good enough, it will produce perfect outputs during inference, and the train-test mismatch will vanish. This is an asymptotic argument — as model capacity and training data increase, the gap should close. The paper argues this is fundamentally misguided: with finite capacity and finite data, model errors are irreducible. The goal should not be to eliminate them (impossible) but to ensure they don't compound.

The proposed solution — Resampling Forcing — operationalizes this reframing by modifying the training procedure rather than the model architecture or the post-training pipeline. During training, the model conditions on degraded history frames that were generated by its own online weights (the "self-resampling" in Section 3.2), but its prediction target remains the clean, ground truth frame. This teaches the model an error correction capability: it learns to produce correct frames even when its input history is imperfect. Figure 2 (bottom) illustrates the intended effect — errors still exist but no longer amplify across the autoregressive loop, instead stabilizing at a near-constant level.

The paper explicitly positions this as analogous to Scheduled Sampling from the sequence modeling literature (Bengio et al., 2015; Mihaylova and Martins, 2019), where autoregressive language models are trained by occasionally replacing ground truth tokens with model-predicted tokens. However, the analogy is imperfect because video diffusion models have an additional dimension of intra-frame generation (the denoising process) that has no equivalent in discrete token prediction. The paper's self-resampling mechanism addresses both intra-frame errors (via partial denoising trajectory resampling) and inter-frame accumulation (via autoregressive conditioning on degraded histories), which Scheduled Sampling does not cover.

The history routing mechanism (Section 3.3) positions itself as a solution to a secondary but practically critical problem: the quadratic growth in attention complexity as more history frames accumulate. While sliding-window attention is the common heuristic (used by Self Forcing and LongLive), the paper argues that fixed, localized windows cannot distinguish between relevant and irrelevant history frames — a recent frame showing an occluded object may be less useful than an older frame showing the object clearly. The parameter-free routing mechanism, inspired by Mixture of Block Attention (Lu et al., 2025; MoBA) from the LLM literature, makes this relevance determination dynamic and content-aware.

Overall, the paper's positioning is: end-to-end, teacher-free training of strictly causal autoregressive video diffusion models is not only possible but achieves quality competitive with distillation-based methods while providing stronger causal guarantees and better scaling to long videos. This is presented not as a rejection of distillation-based approaches but as establishing a new, simpler paradigm that avoids their inherent limitations — particularly the reliance on expensive teachers and the risk of causal information leakage.

3. Technical Approach

3.1 Reader Orientation

Resampling Forcing is an end-to-end training algorithm for autoregressive video diffusion models that teaches the model to generate future video frames correctly even when its input history is imperfect. The core problem it solves is error accumulation — the tendency of small artifacts in early generated frames to compound across the autoregressive loop until the entire video collapses visually — and it solves it by modifying the training procedure so that the model practices recovering from its own mistakes rather than only ever seeing perfect ground-truth histories. The "shape" of the solution is a training loop that degrades history frames using the model's current denoising behavior, then trains the model to predict clean target frames from those degraded inputs, all without any auxiliary teacher model or discriminator network.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components that interact during each training iteration:

  1. Base Autoregressive Video Diffusion Model ($\boldsymbol{v}_\theta(\cdot)$): A Diffusion Transformer (DiT) that takes a noisy current frame $\boldsymbol{x}^i_t$, clean history frames $\boldsymbol{x}^{<i}$ (or degraded copies $\tilde{\boldsymbol{x}}^{<i}$), a timestep $t$, and conditioning $c$, and outputs a velocity prediction $\mathrm{d}\boldsymbol{x}^i_t/\mathrm{d}t$. This is the model being trained.

  2. Self-Resampling Module (Section 3.2): A procedure that runs during training, before the main gradient update. It takes clean video frames, adds noise to a sampled timestep $t_s$, then uses the model's current weights (with gradients disabled) to autoregressively complete the denoising, producing degraded history frames $\tilde{\boldsymbol{x}}^{1:N}$ that simulate inference-time errors.

  3. Parallel Training with Causal Mask (Section 3.2, Algorithm 1): After the degraded histories are produced, the model conditions on them and computes a standard frame-level diffusion loss against the clean ground-truth frames, using a sparse causal mask so all frames can be trained in parallel while respecting temporal causality.

  4. History Routing Mechanism (Section 3.3): An optional, parameter-free component that replaces dense causal attention with a dynamic top-$k$ retrieval of the most relevant history frames for each query token, maintaining near-constant attention complexity as the video grows longer.

Information flows through these components in a strict sequence during each training step:

  • Step 1 — Simulation timestep sampling: Sample $t_s$ from a logit-normal distribution, optionally shifted.
  • Step 2 — History degradation (teacher-free): The clean video $\boldsymbol{x}^{1:N}$ is corrupted to $t_s$, then resampled autoregressively using the online model (no gradient flow) to produce $\tilde{\boldsymbol{x}}^{1:N}$.
  • Step 3 — Main training: Sample a training timestep $t_i$, corrupt the clean frames to $\boldsymbol{x}^{i}_{t_i}$, feed the noisy frame $\boldsymbol{x}^{i}_{t_i}$ and the degraded history $\tilde{\boldsymbol{x}}^{<i}$ into $\boldsymbol{v}_\theta$, and compute the velocity regression loss against the clean target.
  • Step 4 — Update: Backpropagate through the main training loss only (the resampling step is detached).

3.3 Roadmap for the Deep Dive

The technical explanation follows the logical order of the problem decomposition, building from the core failure mode outward to the full training algorithm and its efficiency optimizations:

  • First, we will formalize the mathematical framework of autoregressive video diffusion — the factorization in Equation 1, the teacher forcing objective in Equation 4, and the causal mask architecture — because the exposure bias problem only makes sense against this formal backdrop.
  • Second, we will analyze why exposure bias causes catastrophic error accumulation, using the conceptual model in Figure 2 to motivate why the solution must involve training on degraded histories with clean prediction targets.
  • Third, we will break down the self-resampling mechanism (Equation 5, Algorithm 1) — how the simulation timestep $t_s$ is sampled, how the autoregressive resampling loop works with KV caching, why gradients are detached, and how the timestep shifting factor $s$ controls the history-freshness tradeoff.
  • Fourth, we will examine the teacher forcing warmup phase and its necessity — why self-resampling from random weights produces meaningless signals and how the transition to Resampling Forcing is managed.
  • Fifth, we will detail the history routing mechanism (Equations 8 and 9, Figure 4) — the frame descriptor computation, the top-$k$ selection metric, the two-branch attention fusion via log-sum-exp, and how head-wise and token-wise routing collectively yields a receptive field much larger than $k$ frames.
  • Finally, we will cover the practical implementation specifics — the model architecture modifications, training schedule, hyperparameters, and inference configuration — because these details are critical for understanding the experimental results and for reproducibility.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a training methodology paper whose core idea is that autoregressive video diffusion models fail during inference because they have never been trained to condition on imperfect histories, and that exposing them to their own degraded outputs during training — while maintaining clean prediction targets — teaches an error correction capability that stabilizes long-horizon generation.

Formal Framework: Autoregressive Video Diffusion

The paper operates within a specific mathematical framework that factorizes video generation into two nested processes: inter-frame autoregression across time and intra-frame diffusion within each frame.

Joint distribution factorization. The probability of generating an $N$-frame video sequence $\boldsymbol{x}^{1:N}$ given conditioning information $c$ (e.g., text prompt, initial frame) is expressed as the product of conditional distributions:

p(x1:N∣c)=∏i=1Np(xi∣x<i,c)p(\boldsymbol{x}^{1:N}|c) = \prod_{i=1}^{N} p(\boldsymbol{x}^{i}|\boldsymbol{x}^{<i}, c)

where $\boldsymbol{x}^{i}$ is the $i$-th frame and $\boldsymbol{x}^{<i}$ denotes all frames with indices strictly less than $i$.

What it computes: The joint probability is decomposed into $N$ conditional terms, where each term represents the probability of frame $i$ given all previously generated frames and the condition. This is a standard autoregressive factorization.

Why this form: This factorization enforces strict temporal causality — frame $i$ depends only on its past, never on its future. This mirrors the causal structure of the physical world and is what makes autoregressive video models suitable for world simulation and interactive applications. The alternative — bidirectional generation where all frames can attend to each other — would violate this causal constraint and allow future information to leak into earlier frames.

Intra-frame generation via diffusion. Each conditional distribution $p(\boldsymbol{x}^{i}|\boldsymbol{x}^{<i}, c)$ is itself generated through a diffusion process. Using the flow matching formulation (Lipman et al., 2022), the model learns a velocity field $\boldsymbol{v}_\theta(\cdot)$ parameterized by a neural network with weights $\theta$. During inference, frame $\boldsymbol{x}^{i}$ (denoted $\boldsymbol{x}^{i}_0$ to emphasize it is the clean frame at diffusion timestep $t = 0$) is synthesized by solving the reverse-time ordinary differential equation (ODE):

xi=x1i+∫10vθ(xti,x<i,t,c) dt\boldsymbol{x}^{i} = \boldsymbol{x}^{i}_{1} + \int_{1}^{0} \boldsymbol{v}_{\theta}(\boldsymbol{x}^{i}_{t}, \boldsymbol{x}^{<i}, t, c) \, \mathrm{d}t

where $\boldsymbol{x}^{i}_{1} \sim \mathcal{N}(\boldsymbol{0}, \boldsymbol{I})$ is the initial Gaussian noise at diffusion timestep $t = 1$, $\boldsymbol{x}^{i}_{t}$ is the intermediate noisy frame at time $t$, and $\boldsymbol{v}_{\theta}(\boldsymbol{x}^{i}_{t}, \boldsymbol{x}^{<i}, t, c)$ is the model's velocity prediction conditioned on the noisy current frame, the clean history frames, the diffusion timestep, and the global condition.

What it computes: Starting from pure noise at $t = 1$, we integrate the velocity field backward in time (from $t = 1$ down to $t = 0$) to progressively denoise the frame. At each infinitesimal step, the velocity $\mathrm{d}\boldsymbol{x}^{i}_{t}/\mathrm{d}t$ tells us how to update the frame to move it closer to the data distribution. In practice, this is solved numerically using an Euler sampler with a finite number of steps (32 steps in the paper's inference configuration).

Why this form: Flow matching with an ODE formulation provides a deterministic mapping from noise to data, which is more stable and requires fewer sampling steps than the stochastic diffusion formulation (Ho et al., 2020). The velocity field $\boldsymbol{v}_\theta$ directly regresses to the difference between noise and clean data ($\boldsymbol{\epsilon}^{i} - \boldsymbol{x}^{i}$), which is a simpler target than the score function in standard diffusion. Conditioning on $\boldsymbol{x}^{<i}$ means the generation of each frame is informed by all previous frames, enabling temporal consistency.

Teacher forcing objective. During standard training with teacher forcing, the model is trained to predict the velocity for frame $i$ given its ground-truth history. The training sample at diffusion timestep $t$ is constructed via linear interpolation:

xti=(1−t)⋅xi+t⋅ϵi\boldsymbol{x}^{i}_{t} = (1 - t) \cdot \boldsymbol{x}^{i} + t \cdot \boldsymbol{\epsilon}^{i}

where $\boldsymbol{\epsilon}^{i} \sim \mathcal{N}(\boldsymbol{0}, \boldsymbol{I})$ is Gaussian noise, $\boldsymbol{x}^{i}$ is the clean ground-truth frame, and $t \in [0, 1]$ is the diffusion timestep. At $t = 0$, $\boldsymbol{x}^{i}_{0} = \boldsymbol{x}^{i}$ (clean); at $t = 1$, $\boldsymbol{x}^{i}_{1} = \boldsymbol{\epsilon}^{i}$ (pure noise).

What it computes: A linear blend between the clean frame and noise, controlled by $t$. The interpolation weight $(1 - t)$ on the clean frame decreases linearly from 1 to 0 as $t$ goes from 0 to 1, while the weight $t$ on the noise increases from 0 to 1. This creates a direct path from data to noise that the model learns to reverse.

Why this form: Linear interpolation is the simplest possible path and is the standard in rectified flow (Esser et al., 2024). It means the velocity $\mathrm{d}\boldsymbol{x}^{i}_{t}/\mathrm{d}t = \boldsymbol{\epsilon}^{i} - \boldsymbol{x}^{i}$ is constant along the path — a property that simplifies the regression target and improves training stability compared to variance-preserving paths where the velocity changes with $t$.

The training loss is the expected squared error between the model's predicted velocity and the true velocity:

L=Ei,t,x,ϵ[∥(ϵi−xi)−vθ(xti,x<i,t,c)∥22]\mathcal{L} = \mathbb{E}_{i, t, \boldsymbol{x}, \boldsymbol{\epsilon}} \left[ \lVert (\boldsymbol{\epsilon}^{i} - \boldsymbol{x}^{i}) - \boldsymbol{v}_{\theta}(\boldsymbol{x}^{i}_{t}, \boldsymbol{x}^{<i}, t, c) \rVert^{2}_{2} \right]

where the expectation runs over frame indices $i$, diffusion timesteps $t$, video clips $\boldsymbol{x}$, and noise samples $\boldsymbol{\epsilon}$.

What it computes: For each frame in each training video, at a randomly sampled timestep, we create the noisy frame $\boldsymbol{x}^{i}_{t}$, feed it to the model along with the clean history $\boldsymbol{x}^{<i}$, and compute the mean squared error between the model's velocity output and the target velocity $(\boldsymbol{\epsilon}^{i} - \boldsymbol{x}^{i})$. The loss is averaged over all frames in the video.

Why this form: Mean squared error on the velocity is the standard flow matching objective (Lipman et al., 2022). It directly encourages the model to learn the constant velocity field that transports noise to data. The parallel training across all frames — enabled by the causal mask that prevents each frame from attending to its future — is what makes teacher forcing computationally efficient: a single forward pass processes all frames simultaneously.

Causal mask architecture. The model architecture uses a sparse causal attention mask (Figure 3c) where each frame $i$ can attend to its own tokens and to tokens from all previous frames $1, \dots, i-1$, but not to tokens from future frames $i+1, \dots, N$. This is implemented using torch.flex_attention() with no additional parameters required beyond the standard DiT architecture. During inference, this enables KV caching: once a frame is generated, its keys and values can be stored and reused for all subsequent frames without recomputation. The number of attention queries stays constant (only the current frame's tokens), while the number of keys and values grows linearly with the video length.

Chunking for efficiency. In practice, the autoregressive unit is not a single video frame but a chunk of 3 latent frames. This means each autoregressive step generates 3 frames simultaneously, conditioning on all previously generated chunks. This balances the tradeoff between autoregressive granularity (more steps = more opportunities for error accumulation) and generation parallelism (fewer steps = less computational cost per second of video).

The Exposure Bias Problem: Why Teacher Forcing Fails

The preceding framework is mathematically clean and computationally efficient — but it contains a hidden assumption that breaks during inference. The teacher forcing objective in Equation 4 conditions every frame on its ground-truth history $\boldsymbol{x}^{<i}$. During training, these are the actual clean frames from the dataset, so the model always sees perfect conditioning.

During inference, the model must condition each frame on its own previously generated outputs $\tilde{\boldsymbol{x}}^{<i}$. These self-generated frames contain imperfections — subtle statistical deviations from the ground-truth distribution — which the authors term model error. The model has never been trained on degraded histories, so it treats these imperfections as legitimate input signals and propagates them forward.

The paper identifies a compounding mechanism (Section 3.1, Figure 2 top): when frame $i-1$ is generated with a small error $\delta_{i-1}$, this error is present in the history conditioning for frame $i$. The model's prediction for frame $i$ will therefore contain not only its own generation error $\delta_i$ but also a component amplifying $\delta_{i-1}$, because the model's mapping from history to output is sensitive to history perturbations. This amplified error then becomes part of the history for frame $i+1$, creating a cascade: $\delta_i = f(\delta_{i-1}) + \epsilon_i$ where $f$ is some amplification function and $\epsilon_i$ is new error. If $|f'| > 1$ on average, the total error grows exponentially with video length.

The failure modes observed are concrete: progressive color shifts, texture degradation, loss of object coherence, and eventually complete visual collapse (qualitatively demonstrated in Figure 1 top, Table 1 showing temporal quality drops from 91.20 to 89.74 over 15 seconds even for the best method, and sharper drops for baselines like Pyramid Flow from 81.90 to 84.27 — noting that higher numbers in VBench are worse for some metrics, the trend of quality degradation with length is consistent).

The key insight that motivates the entire approach: with finite model capacity and finite training data, model errors are irreducible — no practical model will produce exactly ground-truth outputs during inference. Therefore, training under the assumption of perfect histories (teacher forcing) is training for a scenario that never occurs during deployment. The correct approach is not to try to eliminate errors (impossible) but to train the model to be robust to them — to produce correct predictions even when its input history is degraded.

Self-Resampling: Simulating Inference-Time Errors During Training

The core technical contribution is the self-resampling mechanism (Section 3.2, Figure 3a, Algorithm 1), which generates degraded history frames for training without requiring an external teacher model.

The conceptual goal. During training, we want to present the model with history frames $\tilde{\boldsymbol{x}}^{<i}$ that approximately match what the model would produce during inference — not ground-truth frames and not arbitrary noise — so that the model learns to condition on its own typical errors. At the same time, we want the prediction target to remain the clean ground-truth frame $\boldsymbol{x}^{i}$, so the model learns to correct these errors rather than reproduce them.

Simulating intra-frame errors via partial denoising trajectory resampling. Each generated frame $\boldsymbol{x}^{i}$ during inference is the result of a complete denoising trajectory from $t = 1$ to $t = 0$. The model error primarily manifests in high-frequency details that are synthesized during the latter portion of this trajectory (the paper cites Falck et al., 2025 on this point: "intra-frame generation errors that come from imperfect score estimation and discretization, which mainly affect high-frequency details"). To simulate this, the paper does not resample from pure noise at $t = 1$ — that would discard all information about the ground-truth frame and produce outputs too different from what the model would actually generate during inference. Instead, it:

  1. Corrupts the clean frame $\boldsymbol{x}^{i}$ to an intermediate timestep $t_s \in (0, 1)$ using the same linear interpolation as Equation 3: $\boldsymbol{x}^{i}_{t_s} = (1 - t_s) \cdot \boldsymbol{x}^{i} + t_s \cdot \boldsymbol{\epsilon}$.
  2. Completes the denoising from $t_s$ down to $0$ using the model's own velocity predictions, producing a degraded clean frame $\tilde{\boldsymbol{x}}^{i}$.

What $\tilde{\boldsymbol{x}}^{i}$ represents at different $t_s$ values. When $t_s$ is small (close to 0), only a small amount of noise is added, and the resampling primarily affects high-frequency details — $\tilde{\boldsymbol{x}}^{i}$ is nearly identical to $\boldsymbol{x}^{i}$ except for subtle texture differences. This simulates a model that makes only minor errors. When $t_s$ is large (close to 1), most of the frame's content is destroyed and regenerated — $\tilde{\boldsymbol{x}}^{i}$ may deviate substantially from $\boldsymbol{x}^{i}$ in structure, identity, and semantics. This simulates a model that makes major errors, but also risks content drifting where the resampled frame no longer represents the same scene as the original.

Simulating inter-frame error accumulation via autoregressive conditioning. The intra-frame resampling alone is insufficient because it does not capture the propagation of errors across frames. During inference, frame $i$ is generated conditioned on the degraded frame $\tilde{\boldsymbol{x}}^{i-1}$, not the clean frame $\boldsymbol{x}^{i-1}$. This means the degradation of frame $i-1$ directly affects the generation of frame $i$, creating a chain of error propagation.

To simulate this, the paper resamples frames sequentially (autoregressively) rather than in parallel (Algorithm 1, lines 9-12):

x~i=xtsi+∫ts0vθ(xti,x~<i,t,c) dt\tilde{\boldsymbol{x}}^{i} = \boldsymbol{x}^{i}_{t_s} + \int_{t_s}^{0} \boldsymbol{v}_{\theta}(\boldsymbol{x}^{i}_{t}, \tilde{\boldsymbol{x}}^{<i}, t, c) \, \mathrm{d}t

where $\tilde{\boldsymbol{x}}^{<i}$ are the already resampled and degraded history frames, not the clean ground-truth frames.

What this equation computes: For each frame $i$ from $1$ to $N$ (in order), we start from the noisy version $\boldsymbol{x}^{i}_{t_s}$ at the sampled simulation timestep, then run the ODE integration using the model's velocity predictions (with gradient computation disabled — "detached from gradient backpropagation to avoid shortcut learning"). Crucially, the velocity prediction at each step conditions on $\tilde{\boldsymbol{x}}^{<i}$, which are the outputs of the same autoregressive resampling process applied to previous frames. This means if frame $i-1$'s resampling introduces an error, that error is present in the history when resampling frame $i$, propagating the degradation forward through the autoregressive chain.

Why this sequential (autoregressive) resampling is essential: The ablation in Table 2 provides direct empirical evidence. "Resampling - parallel" (where all history frames are resampled independently, conditioning on clean history) achieves Temporal: 88.01, Visual: 62.51, while "Resampling - autoregressive" achieves Temporal: 90.46, Visual: 64.25. The authors attribute this to parallel resampling capturing "only per-frame degradation while neglecting autoregressive accumulation across time." Table 2 also compares against "noise augmentation" (Temporal: 87.15, Visual: 61.90), which is even worse because additive Gaussian noise does not match the structured, frequency-dependent error patterns of diffusion model outputs.

Using online model weights and detaching gradients. A critical design choice: the resampling uses the model's current, online weights $\boldsymbol{v}_\theta$, not fixed weights from a previous checkpoint. This ensures that as the model improves during training, the simulated error distribution evolves to match the model's current inference-time behavior — if the model gets better (produces fewer errors), the degradation becomes subtler; if the model is still making large mistakes, the degradation remains substantial.

However, gradients are explicitly detached from this resampling process (Algorithm 1, "with gradient disabled"). If gradients were allowed to flow through the resampling, the model could learn a "shortcut": it could deliberately produce heavily degraded histories during resampling to make the main training loss trivially easy (since the model would learn to simply ignore heavily degraded inputs rather than correct them). Detaching forces the model to learn genuine error correction purely from the final loss, without being able to manipulate the difficulty of its own conditioning.

Practical implementation with KV cache. The autoregressive resampling loop (Algorithm 1, lines 10-12) is computationally efficient because it leverages the same KV caching mechanism used during inference. After resampling frame $i$, its clean features are cached as keys and values. When resampling frame $i+1$, the model only needs to compute queries for the new frame; keys and values for all previous frames are reused from the cache. This avoids quadratic recomputation of the entire history for each successive frame.

The timestep $t_s$ distribution: logit-normal with shifting. The choice of $t_s$ controls a fundamental tradeoff: history faithfulness vs. error correction flexibility. The paper formalizes this through the sampling distribution of $t_s$:

logit(ts)∼N(0,1)\text{logit}(t_s) \sim \mathcal{N}(0, 1)

where $\text{logit}(t_s) = \log(t_s / (1 - t_s))$ maps $t_s \in (0, 1)$ to the real line.

What it computes: First, a standard normal variable $z \sim \mathcal{N}(0, 1)$ is drawn. Then $t_s = \text{sigmoid}(z) = 1 / (1 + e^{-z})$. This transformation concentrates probability mass in the intermediate range of $t_s$ (around 0.5) while suppressing extremes near 0 and 1, because the sigmoid maps the high-probability region of the normal distribution (near 0) to the high-slope region of the sigmoid (near 0.5).

Why this distribution: Small $t_s$ values (close to 0) produce degraded frames nearly identical to ground truth — the paper says this "encourages the model to stay faithful to the history frames and risks error accumulation." This is essentially teacher forcing in the limit $t_s \to 0$. Large $t_s$ values (close to 1) almost completely resample the frame from scratch — the paper says this "grants greater flexibility for error correction but raises the chance of content drifting, as the model is permitted to deviate significantly from the historical context." The logit-normal concentrates density on intermediate values where both effects are balanced: enough degradation to simulate realistic errors and teach correction, but not so much that the history becomes unrelated to the target.

Timestep shifting for model quality adaptation. The paper introduces an additional mechanism to adjust the $t_s$ distribution based on model quality, adopting the shifting formula from Esser et al. (2024):

ts←s⋅ts1+(s−1)⋅tst_s \leftarrow \frac{s \cdot t_s}{1 + (s - 1) \cdot t_s}

where $s$ is the shifting factor. After sampling $t_s$ from the logit-normal distribution, this transformation is applied.

What it computes: This is a rational function that maps $t_s \in (0, 1)$ to another value in $(0, 1)$. When $s < 1$, the mapping contracts $t_s$ toward 0 (more low-noise emphasis). When $s > 1$, it expands $t_s$ toward 1 (more high-noise emphasis). At $s = 1$, it is the identity.

Why this form: The paper states: "Generally, stronger models induce fewer errors, allowing for a greater emphasis on low resampling strength, and vice versa." The shifting factor provides a single knob to manually adjust the overall level of degradation applied during training. In the experiments, the paper uses $s = 0.6$, which shifts probability mass toward lower $t_s$ values (less aggressive degradation). This makes sense for a model initialized from WAN2.1-1.3B pretrained weights — it starts with reasonable generation capability, so extreme degradation would produce resampled frames too different from what the model would actually generate during inference. The ablation in Figure 6 confirms this: very small $s$ causes error accumulation (too little degradation), very large $s$ causes content drifting (too much degradation), and moderate $s$ balances both.

The complete loss function after resampling. Once the degraded history $\tilde{\boldsymbol{x}}^{1:N}$ has been generated, the main training step proceeds identically to teacher forcing, except that the model conditions on $\tilde{\boldsymbol{x}}^{<i}$ instead of $\boldsymbol{x}^{<i}$:

L=1N∑i=1N∥(ϵi−xi)−vθ(xtii,x~<i,ti,c)∥22\mathcal{L} = \frac{1}{N} \sum_{i=1}^{N} \lVert (\boldsymbol{\epsilon}^{i} - \boldsymbol{x}^{i}) - \boldsymbol{v}_{\theta}(\boldsymbol{x}^{i}_{t_i}, \tilde{\boldsymbol{x}}^{<i}, t_i, c) \rVert^{2}_{2}

What it computes: For each frame, we sample a fresh training timestep $t_i$ (independent of the simulation timestep $t_s$), create the noisy frame $\boldsymbol{x}^{i}_{t_i}$, feed it to the model with the degraded history $\tilde{\boldsymbol{x}}^{<i}$, and compute the velocity regression loss against the clean target $(\boldsymbol{\epsilon}^{i} - \boldsymbol{x}^{i})$. The model therefore sees degraded inputs but is asked to predict as if it had clean inputs — this is the mechanism that teaches error correction.

Why this decoupling of degradation and prediction targets is the key innovation: If the model were trained to predict the degraded frame $\tilde{\boldsymbol{x}}^{i}$ from degraded history $\tilde{\boldsymbol{x}}^{<i}$, it would learn to reproduce errors, not correct them. If it were trained to predict the clean frame $\boldsymbol{x}^{i}$ from clean history $\boldsymbol{x}^{<i}$, it would remain vulnerable to inference-time errors (teacher forcing). By combining degraded input with clean target, the model learns: "even when your history is imperfect, still aim for the correct frame." This is visualized in Figure 2 (bottom): the model's predictions are still imperfect, but they no longer drift further from the ground-truth distribution with each autoregressive step — errors remain bounded rather than compounding.

Teacher Forcing Warmup: Why Resampling from an Untrained Model Is Harmful

The transition from teacher forcing to Resampling Forcing is not immediate. The paper identifies a cold-start problem (Section 3.2, "Teacher Forcing Warmup"): at the beginning of training, the causal attention architecture has not yet converged, and the model produces essentially random outputs when generating autoregressively.

Why this matters for self-resampling. If self-resampling were applied from the very first training step, the degraded histories $\tilde{\boldsymbol{x}}^{<i}$ would be nearly random — dominated by initialization noise rather than by the structured, frequency-dependent errors that characterize a partially-trained model. The training signal would be uninformative: the model would be asked to correct random input perturbations, which teaches nothing about actual inference-time error patterns. The paper states this would "hinder convergence."

The warmup procedure. The model first trains with standard teacher forcing for 10K steps on 5-second videos (81 frames at the working resolution). During this phase, it conditions on clean ground-truth histories exactly as in Equation 4. The causal mask architecture is learned from scratch (starting from the pre-trained WAN2.1-1.3B bidirectional weights, which are modified to support per-frame noise levels and causal attention). Once the model acquires basic autoregressive capabilities — it can generate coherent frames when conditioned on clean history, even if not perfectly — the training transitions to Resampling Forcing.

Why 10K steps is sufficient: The exact number is an empirical choice driven by the observation that the model converges to the causal attention structure relatively quickly when initialized from a pre-trained bidirectional DiT. The key capability that must emerge is the ability to produce meaningful outputs when generating autoregressively (even if imperfect), so that the resampled histories contain model-typical errors rather than initialization noise.

Training schedule after warmup: Following warmup, Resampling Forcing training proceeds on 5-second videos for 15K steps, then on 15-second videos (249 frames) for an additional 5K steps. The transition to longer videos is crucial because Resampling Forcing's error correction capability must scale to the video lengths where error accumulation would otherwise be most severe. The fine-tuning on long videos is done natively — the model is trained on full 15-second sequences, not sub-clips — which the paper argues is what enables superior long-term consistency compared to distillation methods trained on short teacher outputs.

History Routing: Dynamic Sparse Attention for Efficient Long-Horizon Generation

The second major technical contribution addresses the computational scaling problem of autoregressive generation: as the video grows longer, each new frame must attend to an ever-growing history of previous frames. With dense causal attention, the per-token attention cost is $\mathcal{O}(L)$ where $L$ is the number of history frames, and the total cost for generating an $N$-frame video is $\mathcal{O}(N^2)$ — prohibitive for long videos.

The limitation of sliding-window attention. The naive solution — restricting each frame to attend only to the most recent $W$ frames (a sliding window) — compromises long-term dependency. The paper illustrates this in Figure 7: sliding-window attention of size 1 (equivalent sparsity to top-1 routing) fails to maintain appearance consistency in a fish generation example, because the model cannot refer back to early frames that establish the object's correct appearance. Fixed, localized windows are blind to the varying relevance of different history frames: a recent frame showing a partially occluded object may be less useful than an older frame showing the object clearly.

The proposed solution: head-wise, token-wise dynamic routing via frame descriptors. The history routing mechanism (Section 3.3, Figure 4) replaces dense causal attention with a two-stage process: for each query token, first select a small subset of relevant history frames, then attend only to those selected frames. The selection is dynamic — it depends on the specific content of the query and the history — and operates independently for each attention head and each spatial token.

The selection metric. For a query token $\boldsymbol{q}_i$ from frame $i$ (where $\boldsymbol{q}_i$ is a single token — one spatial position within the frame's patchified representation), the relevance of history frame $j$ (where $j < i$) is computed as:

score(qi,frame j)=qi⊤ϕ(Kj)\text{score}(\boldsymbol{q}_i, \text{frame } j) = \boldsymbol{q}_i^{\top} \phi(\boldsymbol{K}_j)

where $\phi(\cdot)$ is a frame descriptor function applied to the keys of frame $j$, and $\boldsymbol{K}_j$ is the set of all key vectors for tokens in frame $j$.

What $\phi(\boldsymbol{K}_j)$ computes: Following Mixture of Block Attention (Lu et al., 2025; MoBA), the paper uses mean pooling as the descriptor: $\phi(\boldsymbol{K}_j) = \frac{1}{|\boldsymbol{K}_j|} \sum_{\boldsymbol{k} \in \boldsymbol{K}_j} \boldsymbol{k}$, the average of all key vectors in frame $j$. This produces a single vector per frame that summarizes its content from the perspective of the key space. The dot product $\boldsymbol{q}_i^{\top} \phi(\boldsymbol{K}_j)$ therefore measures how relevant frame $j$'s content is to query token $\boldsymbol{q}_i$.

Why mean pooling: The paper states it "adheres to the attention score computation and is parameter-free." The "adheres" justification is that the attention score between $\boldsymbol{q}_i$ and individual tokens $\boldsymbol{k} \in \boldsymbol{K}_j$ would be $\boldsymbol{q}_i^{\top} \boldsymbol{k}$; the mean-pooled version $\boldsymbol{q}_i^{\top} \bar{\boldsymbol{k}}_j$ approximates the average attention score over all tokens in frame $j$, which is a natural summary of the frame's overall relevance. Being parameter-free means no additional learnable parameters are introduced — the routing decision is purely a function of the existing key and query representations.

The top-$k$ selection. Given all history frames $j = 1, \dots, i-1$, the router selects the $k$ frames with the highest relevance scores:

Ω(qi)=arg⁡max⁡Ω∗∑j∈Ω∗(qi⊤ϕ(Kj))\Omega(\boldsymbol{q}_i) = \arg\max_{\Omega^*} \sum_{j \in \Omega^*} \left( \boldsymbol{q}_i^{\top} \phi(\boldsymbol{K}_j) \right)

where $\Omega(\boldsymbol{q}_i)$ is the set of selected frame indices for query token $\boldsymbol{q}_i$, and $\Omega^*$ ranges over all subsets of size $k$ from the history frames. In practice, this is simply: compute the score for each history frame, sort, take the top $k$.

Attention with selected frames only. Once the $k$ frames are selected, attention proceeds identically to standard attention but restricted to the selected frames:

Attention(qi,K<i,V<i)=Softmax(qiKΩ(qi)⊤d)⋅VΩ(qi)\text{Attention}(\boldsymbol{q}_i, \boldsymbol{K}_{<i}, \boldsymbol{V}_{<i}) = \text{Softmax}\left( \frac{\boldsymbol{q}_i \boldsymbol{K}^{\top}_{\Omega(\boldsymbol{q}_i)}}{\sqrt{d}} \right) \cdot \boldsymbol{V}_{\Omega(\boldsymbol{q}_i)}

where $\boldsymbol{K}_{\Omega(\boldsymbol{q}_i)}$ and $\boldsymbol{V}_{\Omega(\boldsymbol{q}_i)}$ are the keys and values only from the $k$ selected history frames (concatenated across all tokens in those frames), and $d$ is the head dimension.

What it computes: The standard scaled dot-product attention, but with the key-value set restricted to the routed frames. This reduces the per-query attention cost from $\mathcal{O}(L \cdot T)$ (where $T$ is tokens per frame) to $\mathcal{O}(k \cdot T)$ — constant with respect to the total number of history frames $L$.

The effective receptive field is much larger than $k$ frames. A critical nuance: the routing operates head-wise and token-wise. This means:

  • Different attention heads can route to different history frames — head 1 might select frames {1, 3, 5} while head 2 selects {2, 4, 6}, giving the full multi-head attention block a composite view of many more than $k$ frames.
  • Different spatial tokens within the same frame can route to different history frames — a token corresponding to a moving object's current position might route to frames where that object was previously visible, while a token corresponding to static background might route to earlier frames that establish the scene layout.

The paper quantifies this: the attention sparsity is $1 - k/L$. With $k = 5$ and $L \approx 80$ history chunks (at 3 frames per chunk, for a 15-second video at 249 frames), this gives approximately $1 - 5/80 = 93.75\%$ sparsity, but the effective information accessed across all heads and tokens is substantially richer than 5 frames would suggest.

Intra-frame attention branch. The routing mechanism does not route within a frame — each query token always attends to all other tokens in its own frame (the intra-frame branch). This ensures that spatial coherence within each frame is preserved without any sparsity penalty.

Two-branch fusion via log-sum-exp. Following MoBA (Lu et al., 2025), the intra-frame and history branches are combined efficiently using a global log-sum-exp trick, implemented with the flash_attn_varlen_func() interface from FlashAttention (Dao et al., 2022; Dao, 2024). The idea: compute attention for the intra-frame branch and the history branch separately (each producing attention outputs with associated log-sum-exp normalization terms), then combine them by aligning the normalization terms, yielding a result equivalent to a single softmax over the union of keys from both branches. This avoids materializing the full attention matrix while maintaining exactness.

Training with history routing. The routing mechanism is fine-tuned: after the main Resampling Forcing training is complete, the model trains with sparse history routing enabled for 1.5K iterations on 15-second videos. This short fine-tuning phase allows the model to adapt to the sparsity pattern — learning which frames to route to for different types of queries — without the computational cost of routing from scratch throughout training.

Empirical routing patterns. Figure 8 visualizes the selection frequency of each history frame when generating a target frame (the 21st frame), for different values of $k$ (1, 3, 5, 7). The pattern shows a hybrid of:

  1. "Attention sink" behavior: The very first frames (the "sink") are selected with extremely high frequency — far more than random chance. The maximum bar in Figure 8 is truncated for readability and labeled with its exact value, indicating that the first few frames dominate the routing distribution.
  2. Recency bias: The most recent few frames before the target are also selected with elevated frequency, creating a sliding-window-like pattern near the current timestep.
  3. Intermediate frames: As $k$ increases from 1 to 7, the selection becomes more distributed, encompassing a broader range of intermediate frames. With $k = 1$ (95% sparsity), the routing is extremely concentrated on initial and recent frames. With $k = 7$, more intermediate frames are included.

This hybrid pattern provides empirical justification for heuristic attention designs that combine "frame sinks" with sliding windows (LongLive by Yang et al., 2025, explicitly cited by the paper). However, the paper argues that learned routing is more principled: it discovers this pattern automatically from data rather than imposing it as a fixed architectural constraint, and it can adapt to content-dependent variations that a static mask cannot capture.

Practical performance. Table 1 shows that the model with 75% sparsity ($k = 5$ out of approximately 20 history chunks at 10-15 seconds) achieves Temporal: 90.18, Visual: 63.95, Text: 24.12 — nearly identical to the dense attention baseline (Temporal: 91.20, Visual: 64.72, Text: 25.79). The qualitative comparison in Figure 7 shows top-5 routing producing fish appearance indistinguishable from dense attention, top-1 routing showing only minor degradation, and sliding-window attention (size 1) losing appearance consistency entirely.

Practical Implementation Details

Model architecture modifications from WAN2.1-1.3B. The base model is WAN2.1-1.3B (Wan et al., 2025), a bidirectional DiT that generates 5-second videos at 480×832 resolution with 81 frames. Two modifications are made:

  1. Per-frame noise level conditioning: The standard DiT conditions on a single global timestep. For autoregressive diffusion, each frame needs its own noise level (the training timestep $t_i$ is independently sampled per frame in Equation 4). The timestep conditioning mechanism in the DiT blocks is modified to accept per-frame noise levels.
  2. Causal attention mask: The bidirectional self-attention is replaced with a sparse causal mask (Figure 3c) using torch.flex_attention(), which implements the pattern without additional learnable parameters.

The autoregressive unit is a chunk of 3 latent frames, following prior work (Self Forcing by Huang et al., 2025a; LongLive by Yang et al., 2025; Self Forcing++ by Cui et al., 2025). This choice balances autoregressive granularity against computational efficiency — more frames per chunk means fewer autoregressive steps (less error propagation opportunity) but larger attention operations per step.

Training hyperparameters. The training configuration is specified in Section 4:

  • Batch size: 64.
  • Optimizer: AdamW with learning rate $5 \times 10^{-5}$.
  • Teacher forcing warmup: 10K steps on 5-second videos.
  • Resampling Forcing phase 1: 15K steps on 5-second videos.
  • Resampling Forcing phase 2: 5K steps on 15-second videos (249 frames).
  • History routing fine-tuning: 1.5K iterations on 15-second videos.

Resampling hyperparameters.

  • Simulation timestep shifting factor: $s = 0.6$ — shifts the logit-normal distribution toward lower $t_s$, meaning less aggressive degradation. The paper states this manual setting biases toward "low-noise part" because the model initialized from strong pretrained weights already generates reasonable outputs.
  • Resampling solver: 1-step Euler solver — for computational efficiency, the autoregressive resampling uses only a single Euler step rather than the full 32-step solver used for final inference. The paper notes this is "For efficiency" and implies that even a coarse approximation of the model's inference-time errors is sufficient for the training signal.

Inference configuration.

  • Sampler: Euler with 32 steps.
  • Timestep shifting factor: 5.0 — applied during inference only, not during training. This shifts the diffusion timestep sampling toward higher values, which has been shown to improve sample quality in flow matching models.
  • Classifier-free guidance scale: 5.0 for all frames.

4. Key Insights and Innovations

Innovation 1: Reframing Exposure Bias as an Error Robustness Problem, Not an Error Elimination Problem

The dominant assumption in autoregressive video generation — whether implicit in teacher forcing or explicit in post-training distribution matching — has been that the goal is to make the model's outputs indistinguishable from ground truth, at which point the train-test mismatch vanishes. This paper makes a fundamental conceptual break from that assumption. It argues that model errors are irreducible given finite capacity and data, and therefore the correct objective is not to eliminate errors but to prevent errors from compounding across the autoregressive loop.

This reframing is what makes the entire Resampling Forcing framework coherent. If you believe errors can be driven to zero, then teacher forcing with increasingly powerful models is the asymptotically correct strategy — just make the model good enough, and the problem goes away. But the paper's diagnostic (Section 3.1, Figure 2) shows that even a model with non-zero error on each frame will suffer catastrophic degradation in long rollouts if the errors amplify rather than stabilize. The difference between "model makes small mistakes" and "model makes small mistakes that cause progressively larger mistakes" is not captured by standard per-frame metrics — it's a dynamical systems property of the autoregressive process itself.

This is a genuinely different way of thinking about the problem. Prior work on exposure bias in diffusion models (Ning et al., 2023; Schmidt, 2019) analyzed it as a distributional shift to be corrected through post-hoc alignment. Noise augmentation (GameNGen, ART-V) treated it as a robustness problem but used the wrong noise distribution (Gaussian rather than model-typical errors, as the Table 2 ablation demonstrates). Diffusion Forcing (Chen et al., 2025a) addressed it by broadening the training distribution to include arbitrary noise levels. Self Forcing (Huang et al., 2025a) addressed it by simulating inference and matching distributions. All of these, in different ways, implicitly assume that if the training and inference distributions are aligned, the problem is solved.

Resampling Forcing's reframing suggests something deeper: the problem is not alignment, it's dynamics. You can align training and inference distributions perfectly (Self Forcing does this), but if the model hasn't learned to actively correct upstream errors rather than propagate them, errors will still compound because the mapping from degraded history to output is inherently amplifying. The key shift is from "train on the distribution you'll see at inference" to "train a mapping that contracts errors toward zero regardless of input distribution." This is a control-theoretic perspective rather than a statistical one, and it represents a fundamental conceptual advance over the distribution-matching paradigm.

Evidence for this reframing's validity comes from the qualitative patterns in Figure 5 (upper panel): even Self Forcing, which explicitly simulates inference during training, exhibits progressive visual degradation over 15 seconds. The distribution is matched — the model saw self-generated histories during post-training — but the dynamics of error propagation were not addressed because the distillation objective optimized for per-frame appearance quality, not for contraction of upstream errors. Resampling Forcing's training, by conditioning on degraded inputs with clean targets (Equation 5 → Equation 4 with $\tilde{\boldsymbol{x}}^{<i}$), directly optimizes for the property "correct output despite imperfect input" — which is precisely the contraction property needed to stabilize the autoregressive dynamics.

Innovation 2: Teacher-Free Self-Supervision via Online Model Resampling as an Inference Simulator

The second conceptual innovation is the specific mechanism by which Resampling Forcing achieves error robustness without external supervision: using the model's own online weights as an inference simulator during training. This is a qualitatively different class of solution than the dominant post-training paradigm, and it carries implications beyond video generation.

Prior to this work, the only demonstrated path to strong long-horizon autoregressive video quality — as measured by the baselines in Table 1 (Self Forcing, CausVid, LongLive) — required a pre-trained bidirectional teacher model. These teachers are massive: WAN2.1-14B, a 14-billion-parameter model, serves as the distillation target for all three baselines. This creates what might be called a teacher dependency trap: to train a good causal model, you need a good bidirectional model; to train that bidirectional model, you need massive compute; to improve the causal model beyond what the teacher can teach, you hit a quality ceiling imposed by the teacher's own limitations. The field was converging on a paradigm where autoregressive video models were necessarily parasites on bidirectional hosts.

Resampling Forcing breaks this dependency entirely. The key intellectual move is recognizing that the model's own generation process contains sufficient signal about inference-time errors to serve as a training signal, provided you can prevent the optimization from collapsing into trivial solutions (via gradient detachment) and that the error distribution is appropriately calibrated (via the logit-normal timestep sampling and shifting factor $s$). This is not obvious a priori — one might reasonably assume that a model learning from its own outputs would simply reinforce its own biases, a well-documented failure mode in self-training (the paper itself documents this with the ReSTEM^{\text{EM}} failure in its cited discussion of revision model training, Appendix K of the prior work). The fact that it works — and works competitively with distillation-based methods, as Table 1 shows (Resampling Forcing achieves Temporal: 91.20 vs. Self Forcing's 90.03 at 0-15s, and outperforms it at longer horizons) — is a significant empirical discovery about the structure of the learning problem.

The online weight aspect is crucial and distinguishes this from simpler self-distillation approaches. By using the current (evolving) model weights for resampling, the error distribution automatically tracks the model's improving capabilities. If the model gets better, the simulated errors become subtler; if the model has regressed in some capability (e.g., during a distribution shift when transitioning to longer videos), the errors become larger, providing appropriately calibrated training signal. This creates a self-regulating feedback loop that fixed-weight or teacher-based approaches cannot replicate. It is, in effect, a form of curriculum learning where the curriculum is generated by the model's own competence.

The significance extends beyond video generation. Any autoregressive generative model — language models generating long sequences, audio models, robotic trajectory predictors — faces some form of exposure bias. The teacher-free self-resampling paradigm suggests a general recipe: (1) degrade conditioning inputs by running the model's own generation process partway from a controlled starting point, (2) train with clean targets on degraded inputs, (3) detach gradients through the degradation step. Whether this transfers to discrete domains (where "partial degradation" is less naturally defined than in continuous diffusion) is an open question, but the conceptual template is clear.

Innovation 3: Dynamic, Content-Aware History Routing as a Learned Alternative to Fixed Sparse Attention Heuristics

The third innovation addresses a secondary but practically critical problem — the quadratic growth of attention cost with video length — but does so in a way that reveals something interesting about what information autoregressive video models actually use from their history. This transforms what could have been an engineering optimization into a diagnostic tool.

The standard approach to attention sparsity in long video generation is fixed-pattern sparsity: sliding windows (Self Forcing, LongLive), strided attention, or spatial-temporal decay masks (Radical Attention). These are architectural choices — they hard-code an assumption about which history frames are relevant, typically "recent frames matter most, with maybe some global tokens." They are not learned, not adaptive to content, and not capable of discovering that, for example, the very first frame of a video (which establishes object appearance and scene layout) might be more relevant to frame 100 than frame 99 is.

The paper's history routing mechanism makes the sparsity pattern learned and dynamic. The routing is parameter-free in the sense that no new weights are introduced — the relevance score uses the existing key and query representations — but it is learned in the sense that the attention weights themselves are trained to route effectively under the sparsity constraint. The model discovers its own sparse attention pattern through end-to-end training.

The diagnostic value comes from Figure 8: the visualization of which frames get routed to reveals a hybrid "attention sink + recency" pattern that the model discovered automatically. This is significant for two reasons. First, it empirically validates the heuristic used by LongLive (combining frame sinks with sliding windows) — but it shows that this pattern emerges from data rather than needing to be architecturally imposed. Second, it suggests that the heuristic is an approximation to a richer underlying dynamic: as $k$ increases from 1 to 7, the selection becomes more distributed across intermediate frames, indicating that the model wants to access a diverse temporal sampling of history when given the capacity to do so.

This is a diagnostic insight as much as a methodological one. The paper is implicitly arguing that fixed sparse attention masks are not merely computationally suboptimal — they are theoretically insufficient because they preclude the model from discovering its own routing strategy. The fact that the routing pattern looks interpretable (attention sink + recency) is reassuring — it suggests the model is learning something sensible, not exploiting some obscure sparsity artifact — but the key claim is that letting the model choose is better than imposing a choice, even when the model's choice qualitatively resembles a known heuristic.

The practical significance is clear: 75% sparsity with negligible quality loss (Table 1, "Ours (75% sparsity)" row) means the attention cost for long videos can be dramatically reduced without the degradation that sliding-window approaches exhibit (Figure 7, where sliding window size 1 loses appearance consistency that top-1 routing preserves). The conceptual significance is that this provides a template for moving from fixed to learned sparsity in any autoregressive transformer with growing context.

Innovation 4: The Discovery That Distillation from Bidirectional Teachers Transfers Non-Causal Behaviors

This is a negative result with positive implications — a diagnostic finding that reveals a previously unacknowledged failure mode of teacher-student distillation for causal models, and that simultaneously validates the strictness of Resampling Forcing's causal training.

The paper could have simply argued that teacher-free training is more elegant or more scalable than distillation-based approaches. Instead, it provides concrete evidence that distillation-based approaches have a qualitative failure mode that is not a matter of degree (slightly worse metrics) but a matter of kind (physically impossible outputs). The milk-pouring example in Figure 5 (lower panel) is the key exhibit: LongLive produces a liquid level that rises and then falls during continuous pouring. This is not slightly inaccurate — it violates conservation of mass. The paper attributes this to the bidirectional teacher's attention allowing future frame information to influence earlier frame generations, and this causal contamination being transferred to the student during distillation.

This finding matters because it challenges the assumption — implicit in the distillation literature — that a causal student architecture is sufficient to guarantee causal behavior, even when the teacher is non-causal. The paper argues it is not: architectural causality is necessary but not sufficient when the training signal comes from a non-causal source. The teacher's outputs encode non-causal dependencies (future frames affecting earlier ones), and the student learns to reproduce these outputs, thereby internalizing non-causal patterns even though its own architecture cannot directly implement bidirectional attention.

This is a fundamental insight about knowledge distillation that extends beyond video generation. Any domain where causal structure matters — world models, decision-making, physical simulation — and where distillation from a more powerful but non-causal teacher is tempting (because bidirectional models are often easier to train or more performant on short horizons) should take this as a warning. The student may achieve good short-horizon metrics (LongLive matches or exceeds Resampling Forcing on visual quality in Table 1) while being subtly wrong in ways that compromise its fitness for causal reasoning tasks.

Resampling Forcing, by training with a strict causal mask and no access to future information at any point (the resampling is itself autoregressive and causal), provides an existence proof that competitive quality is achievable without causal compromise. This transforms the tradeoff from "bidirectional teachers give better quality but with some causal leakage" to "teacher-free training gives comparable quality with strictly better causal properties" — a stronger position that reframes the burden of proof for future distillation-based methods.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. All experiments use the VBench benchmark suite (Huang et al., 2024). The paper generates 15-second videos prompted from VBench and partitions them into three temporal segments (0–5s, 5–10s, 10–15s) for separate evaluation, enabling assessment of quality degradation over time.

  • Base model(s). The primary model is built on WAN2.1-1.3B (Wan et al., 2025), a 1.3-billion-parameter bidirectional Diffusion Transformer that generates 5-second videos at 480×832 resolution with 81 frames. The authors modify this architecture to support per-frame noise level conditioning and replace bidirectional self-attention with a sparse causal mask using torch.flex_attention(). The pretrained weights are loaded to speed convergence. For the distillation-based baselines (Self Forcing, CausVid, LongLive), the teacher model is WAN2.1-14B, a 14-billion-parameter bidirectional model also trained on 5-second videos. The paper does not train models from scratch — the Resampling Forcing training starts from the pretrained WAN2.1-1.3B checkpoint, making this a fine-tuning paradigm rather than scratch training.

  • Metrics. The paper uses the automatic evaluation metrics from VBench (Huang et al., 2024), which provide three aggregate scores: Temporal quality (motion smoothness, temporal consistency), Visual quality (aesthetic quality, imaging quality, color, etc.), and Text alignment (how well the generated video matches the text prompt). VBench computes these by running pre-trained assessment models on generated videos, producing scores where higher numbers indicate better quality for each dimension. The paper evaluates each 5-second segment independently to track quality degradation over time. There is no human evaluation reported, no FVD (Fréchet Video Distance), and no FID-style distributional metrics — the paper relies exclusively on VBench's automated assessments.

  • Baselines. Seven baselines are compared in Table 1:

    • SkyReels-V2 (Chen et al., 2025b): 1.3B parameters, clip-level autoregressive model generating 5-second segments sequentially. Uses Diffusion Forcing training.
    • MAGI-1 (Teng et al., 2025): 4.5B parameters, relaxes strict causality via rolling denoising framework where next-chunk denoising begins before the current chunk is fully generated.
    • NOVA (Deng et al., 2024): 0.6B parameters, autoregressive video generation without vector quantization.
    • Pyramid Flow (Jin et al., 2024): 2.0B parameters, pyramidal flow matching for efficient generation.
    • CausVid (Yin et al., 2024c): 1.3B parameters, distilled from WAN2.1-14B (a 5-second bidirectional teacher).
    • Self Forcing (Huang et al., 2025a): 1.3B parameters, distilled from WAN2.1-14B using autoregressive rollout followed by distillation loss.
    • LongLive (Yang et al., 2025): 1.3B parameters, concurrent work that extends Self Forcing by rolling out longer videos and applying sub-clip distillation from WAN2.1-14B.

    Notably, SkyReels-V2 and MAGI-1 do not enforce strict per-chunk causality, while CausVid, Self Forcing, and LongLive all use distillation from the same WAN2.1-14B teacher. The proposed method (Resampling Forcing) is the only strictly causal, teacher-free approach. The paper also reports results for its own method under two configurations: dense attention ("Ours") and sparse attention with 75% sparsity ("Ours (75% sparsity)").

  • Generation budget / compute accounting. The paper does not standardize generation budget across methods in a FLOPs-matched or latency-matched sense. Each baseline uses its own published inference configuration. For the proposed method, inference uses 32 Euler steps with classifier-free guidance scale 5.0 and a timestep shifting factor of 5.0. The computational cost of the self-resampling training procedure is not compared to the cost of training the distillation teacher (WAN2.1-14B) or running the distillation process. The paper's efficiency claims are primarily about training-time simplicity (no teacher model needed) rather than inference-time compute parity.

  • Cross-validation / statistical protocol. No cross-validation, statistical significance testing, or error bars are reported. The paper evaluates on the standard VBench prompt set without specifying the exact number of prompts used. All numbers in Table 1 are single-run results without confidence intervals. The qualitative comparisons in Figure 5 are individual cherry-picked examples. The difficulty-binning approach from the prior sections (two-fold cross-validation within difficulty bins) is not used in this paper.

Main Quantitative Results

Aggregate Comparison Against Baselines (Table 1)

The paper's headline result is that Resampling Forcing achieves temporal quality scores competitive with or exceeding distillation-based baselines while operating without a teacher model. The dense-attention variant ("Ours") achieves Temporal: 91.20, Visual: 64.72, Text: 25.79 on the full 0–15s segment — compared to Self Forcing (Temporal: 90.03, Visual: 67.12, Text: 25.02) and CausVid (Temporal: 89.35, Visual: 65.80, Text: 23.95).

The temporal quality advantage is the key claim. On the 5–10s segment, Resampling Forcing achieves Temporal: 90.44 versus Self Forcing's 84.27 — a gap of 6.17 points that widens further at 10–15s (89.74 vs. 84.26). This shows that while Self Forcing's temporal quality drops substantially between the first and second segments (from 90.03 to 84.27, a 5.76-point drop), Resampling Forcing's temporal quality remains relatively stable (91.20 → 90.44 → 89.74, only a 1.46-point drop). The paper interprets this as evidence of superior temporal stability at long horizons.

Visual quality tells a more nuanced story. Self Forcing achieves Visual: 67.12 versus Resampling Forcing's 64.72 on the full 0–15s segment — a 2.4-point advantage for the distillation-based method. This gap persists across segments (66.18 vs. 64.03 at 5–10s; 63.04 vs. 63.99 at 10–15s, where Resampling Forcing slightly edges ahead). The paper does not emphasize this difference, but it indicates that the distillation teacher provides a visual quality advantage that Resampling Forcing does not fully close, particularly on earlier segments — presumably because the bidirectional teacher (WAN2.1-14B) was trained with substantially more compute and data, and the distilled models inherit some of that quality.

Text alignment favors teacher-based methods slightly. CausVid achieves Text: 23.95 versus Resampling Forcing's 25.79 on the full segment, but Self Forcing achieves 25.02 and LongLive achieves 24.41 — all in a relatively tight range. The differences are small (within ~2 points) and the ordering changes across segments, suggesting text alignment is not a strongly differentiating metric among these methods.

Comparison with relaxed-causality approaches. SkyReels-V2 (clip-level autoregressive) achieves Temporal: 81.93, lower than Resampling Forcing's 91.20, but maintains stable visual quality (60.25 → 59.71 → 58.52) that doesn't degrade as sharply as strict autoregressive baselines. MAGI-1 (rolling denoising, 4.5B parameters) achieves Temporal: 87.09, lower than Resampling Forcing, but with the best visual stability across segments (59.79 → 59.33 → 59.03) — essentially flat. This pattern supports the paper's argument that relaxed causality improves temporal stability at the cost of strict causal guarantees, but it also shows that Resampling Forcing achieves even better temporal quality (91.20) while maintaining strict causality.

Sparse attention results. The 75% sparsity variant (top-5 routing out of approximately 20 history chunks at 10–15 seconds) achieves Temporal: 90.18, Visual: 63.95, Text: 24.12 on the full segment — a 1.02-point temporal drop, 0.77-point visual drop, and 1.67-point text drop relative to dense attention. The paper presents this as "negligible," which is reasonable for temporal quality (the 90.18 still exceeds all baselines) but more debatable for visual quality (63.95 is below all baselines except NOVA and SkyReels-V2).

Error Simulation Strategy Ablation (Table 2)

The paper isolates the contribution of autoregressive resampling by comparing three strategies for simulating inference-time errors during training, all evaluated on 0–15s segments:

StrategyTemporalVisualText
Noise augmentation87.1561.9021.44
Resampling - parallel88.0162.5124.51
Resampling - autoregressive90.4664.2525.26

Noise augmentation — the strategy used by GameNGen and ART-V, where small Gaussian noise is added to history frames — achieves the lowest scores across all three metrics. The 3.31-point temporal gap to autoregressive resampling (87.15 vs. 90.46) is substantial. This confirms the paper's hypothesis that additive Gaussian noise does not match the structured error patterns of actual diffusion model outputs.

Parallel resampling — where all history frames are resampled independently (conditioning each frame's resampling on clean history) rather than autoregressively — improves over noise augmentation but falls short of autoregressive resampling by 2.45 temporal points and 1.74 visual points. The paper attributes this to parallel resampling capturing "only per-frame degradation while neglecting autoregressive accumulation across time." This is the direct evidence that inter-frame error propagation matters — resampling each frame independently of the errors in previous frames provides a weaker training signal than simulating the full autoregressive error cascade.

What the ablation does NOT test: The paper does not compare against (a) using a fixed checkpoint for resampling rather than online weights, (b) using a different simulation timestep distribution, (c) varying the number of Euler steps used during resampling, or (d) the contribution of gradient detachment (there is no "with gradients enabled" ablation, which would likely show training collapse but would confirm the necessity of this design choice).

Simulation Timestep Shifting Ablation (Figure 6)

The paper sweeps the shifting factor $s$ that controls the $t_s$ distribution, using extreme values to visualize the impact: a "small $s$" (shifting toward low-noise, weak resampling), a "moderate $s$" (the default $s = 0.6$), and a "very large $s$" (shifting toward high-noise, aggressive resampling). Results are shown qualitatively rather than with VBench metrics.

  • Small $s$: The model exhibits error accumulation and quality degradation, visible as progressive visual artifacts. This is the "insufficient degradation" regime — the model's training histories are too close to ground truth, so it never learns robust error correction.
  • Very large $s$: The model shows initial content drifting, where frames deviate significantly from the intended scene because the resampling is so aggressive that the degraded history frames are semantically different from the ground truth. The model learns to ignore history rather than correct it — a failure mode of over-aggressive degradation.
  • Moderate $s$ (0.6): Balances both effects. The paper states "model performance is robust to the choice of $s$" and uses extreme values in Figure 6 to demonstrate the boundaries.

What is not shown: Quantitative VBench metrics for different $s$ values, a systematic sweep showing the performance plateau, or results at intermediate values (0.3, 0.4, 0.5, 0.7, 0.8). The claim of "robustness" is asserted but not quantified.

Sparse History Strategies (Figure 7)

The paper compares three attention patterns for history context, shown qualitatively for a fish generation example:

  • Dense causal attention: Full attention to all history frames. Serves as the upper bound.
  • Top-5 routing (75% sparsity, $k = 5$ out of 20 history chunks): Quality "comparable to dense attention."
  • Top-1 routing (95% sparsity, $k = 1$): "Only minor quality degradation."
  • Sliding window, size 1 (equivalent sparsity to top-1): The fish's appearance becomes inconsistent — color and shape drift compared to earlier frames.

The critical finding is not the absolute numbers but the comparison between equal-sparsity methods: top-1 routing preserves appearance consistency where sliding window (size 1) fails, despite both attending to exactly one history frame per query. The paper attributes this to routing's dynamic selection — each query token can choose a different frame, and across all heads and spatial positions, the collective receptive field is much richer than a fixed sliding window.

What is not quantified: VBench scores for each sparsity strategy, the specific attention sparsity percentages and their correspondence to wall-clock speedup, or memory usage comparisons. The qualitative example is a single video — the robustness of the routing advantage across diverse video content is not established.

History Routing Frequency Analysis (Figure 8)

The paper visualizes which history frames get routed to when generating the 21st frame, for $k = 1, 3, 5, 7$. The key observations are:

  • Attention sink behavior: The very first frames are selected with extremely high frequency. The maximum bar is truncated and labeled with its exact value, indicating the first few frames dominate.
  • Recency bias: The most recent few frames before the target are also selected with elevated frequency.
  • Distributed intermediate selection: As $k$ increases, the selection spreads to include more intermediate frames. With $k = 1$, the pattern is highly concentrated on initial and recent frames. With $k = 7$, a broader range of history is accessed.

This provides empirical justification for the "frame sink + sliding window" heuristic used by LongLive, but also shows that the learned routing discovers this pattern automatically rather than having it architecturally imposed. The paper frames this as evidence that dynamic routing can serve as a diagnostic tool — revealing what the model actually uses from history, which can inform future architectural designs.

Ablation Studies and Robustness Checks

  • Teacher vs. teacher-free training: The central comparison running through Table 1 is Resampling Forcing versus the four distillation-based baselines (CausVid, Self Forcing, LongLive, and implicitly NOVA/Pyramid Flow which use other strategies). This is not a controlled ablation — the baselines differ in architecture, training data, and optimization procedure — but it establishes that teacher-free training achieves competitive quality. The "teacher model" column in Table 1 explicitly indicates that all high-performing baselines except SkyReels-V2 and MAGI-1 (which relax causality) require WAN2.1-14B, while Resampling Forcing does not.

  • Error simulation strategy: Table 2 ablates noise augmentation vs. parallel resampling vs. autoregressive resampling. This directly tests the paper's central hypothesis that the specific structure of the simulated error matters — not just the presence of degradation, but whether it accurately reflects the model's autoregressive error accumulation pattern. The monotonic improvement from noise → parallel → autoregressive supports this hypothesis.

  • Timestep shifting factor $s$: Figure 6 demonstrates that the choice of $s$ affects the history-freshness tradeoff, with extremes causing either error accumulation (small $s$) or content drifting (large $s$). The paper's claim that performance is "robust" to $s$ within a moderate range is stated but not quantitatively verified across intermediate values.

  • Sparsity level: The comparison of dense attention, top-5 routing (75% sparsity), and top-1 routing (95% sparsity) in Table 1 and Figure 7 establishes that (a) 75% sparsity incurs minimal quality loss, and (b) top-1 routing outperforms equal-sparsity sliding window attention. The fact that the sparse variant is fine-tuned for only 1.5K iterations after full Resampling Forcing training means the model learns its routing patterns post-hoc rather than from scratch — an important detail for reproducibility.

  • Teacher forcing warmup duration: The paper uses 10K warmup steps but does not ablate this number. Insufficient warmup could produce poor resampled histories (dominated by initialization noise), while excessive warmup delays the transition to error-robust training. This is an unexamined hyperparameter.

  • Negative result: Sliding window attention degradation: Figure 7 shows that sliding window (size 1) loses appearance consistency compared to top-1 routing at equal sparsity. This is a clean negative result that demonstrates the insufficiency of fixed sparsity patterns, even at equivalent computational cost.

  • What is NOT ablated:

    • Gradient detachment: No experiment compares training with gradients flowing through the resampling process versus detached gradients. The paper asserts this prevents "shortcut learning" but provides no empirical evidence that such shortcuts actually occur.
    • Online vs. fixed checkpoint resampling: Using online weights for resampling is argued to provide a self-regulating feedback loop, but no comparison against using a fixed checkpoint shows whether this matters in practice.
    • Number of Euler steps in resampling: The paper uses 1-step Euler for efficiency during training resampling, but does not test whether more accurate resampling (e.g., 4 or 8 steps) improves the quality of the training signal.
    • Resampling timestep distribution shape: Only the shifting factor $s$ is ablated, but the choice of logit-normal (mean 0, std 1) is not compared to alternatives like uniform, beta, or a learned distribution.
    • Training data scale: All training uses the same dataset without scaling curves. It is unknown whether Resampling Forcing's advantage over distillation grows, shrinks, or saturates with more training data.
    • Chunk size: The autoregressive unit of 3 latent frames is adopted from prior work but not varied. Larger chunks reduce the number of autoregressive steps (potentially reducing error accumulation directly) but increase per-step computational cost.

Critical Assessment

Primary claim: Resampling Forcing achieves generation quality comparable to distillation-based baselines.

The evidence in Table 1 partially supports this claim, but with important qualifications. The temporal quality comparison strongly favors Resampling Forcing (91.20 vs. 90.03 for Self Forcing on 0–15s, with a much smaller degradation over time). However, visual quality tells a different story: Self Forcing scores 67.12 versus 64.72, and CausVid scores 65.80 — both exceed Resampling Forcing's 64.72. This is a 2.4–2.8 point gap in visual quality that the paper does not adequately discuss. If the claim is "comparable quality," the visual quality dimension shows a non-trivial deficit that might matter for applications where appearance fidelity is paramount.

More importantly, the claim of "comparable" is made against baselines that were themselves trained with a 14B-parameter teacher model trained on massive compute. The paper argues that matching these baselines without a teacher is the achievement, which is a reasonable framing. But a reader might ask: is a 2.4-point visual quality gap "comparable," or is it a measurable disadvantage that the teacher-free approach has not yet closed? The paper does not report confidence intervals, so we cannot assess whether this gap is statistically significant.

The most convincing evidence for the primary claim is temporal stability: Resampling Forcing's temporal score drops only 1.46 points from 0–5s to 10–15s (91.20 → 89.74), while Self Forcing drops 5.77 points (90.03 → 84.26). This difference in degradation rate — not absolute quality — is where Resampling Forcing's advantage is clearest and most robust. However, this is a within-method comparison (each method evaluated against itself at different lengths), and the VBench temporal metric may reward stability in ways that don't perfectly correlate with human perception of quality. Human evaluation or alternative metrics (FVD, optical flow consistency) would strengthen this finding.

Claim: The method exhibits superior temporal causality compared to distillation-based baselines.

The evidence for this claim is a single qualitative example (Figure 5, lower panel) showing LongLive producing physically impossible liquid dynamics (rising-then-falling level during continuous pouring) while Resampling Forcing produces monotonic liquid level increase. This is a compelling demonstration of a specific failure mode, but it does not constitute systematic evidence of superior causality. The paper does not:

  • Evaluate causal consistency across a test suite of physics-violation scenarios.
  • Quantify the frequency of causal violations in generated videos across methods.
  • Compare against the non-distilled baselines (Pyramid Flow, CausVid with a different distillation strategy, etc.) for the same causal violation pattern.
  • Test whether LongLive's failure is reproducible or a cherry-picked worst case.

The argument that bidirectional teacher distillation leaks future information is theoretically sound — a bidirectional teacher's outputs encode non-causal dependencies by construction, and distillation transfers these to the student. But the empirical demonstration is thin. One example does not establish that this is a systematic problem with distillation or that Resampling Forcing is systematically better. It establishes existence of the failure mode, not its prevalence.

Claim: History routing achieves 75% sparsity with negligible quality loss.

Table 1 supports this for temporal quality (90.18 vs. 91.20 dense, a 1.02-point drop). For visual quality, the gap is 0.77 points (63.95 vs. 64.72), and for text alignment it is 1.67 points (24.12 vs. 25.79). Whether these gaps count as "negligible" depends on the application. The temporal gap is small enough to be practically irrelevant. The text alignment gap is modest but represents a ~6.5% relative reduction, which might matter for text-heavy generation tasks.

More critically, the paper does not report wall-clock speedup or memory savings from sparsity. "75% sparsity" means each query attends to 25% of history frames, but the routing computation itself (computing $\boldsymbol{q}_i^{\top} \phi(\boldsymbol{K}_j)$ for all history frames) has overhead. The actual end-to-end inference speedup may be less than 4×. Without latency and memory measurements, the practical benefit of sparsity is asserted rather than demonstrated.

Claim: Autoregressive resampling is essential for error simulation quality.

Table 2 provides clean evidence for this claim: autoregressive > parallel > noise in all three metrics. This is a well-controlled ablation with clear monotonic improvement, and it directly supports the paper's central hypothesis that inter-frame error accumulation must be simulated during training. The magnitudes are meaningful (3.31 temporal points from noise to autoregressive, 2.45 from parallel to autoregressive). This is the strongest empirical contribution in the experimental section.

However, note that the ablation is reported only for the full 0–15s segment. The paper does not show whether the advantage of autoregressive resampling over parallel resampling grows with video length (as the error accumulation hypothesis would predict) — this would be a stronger test of the mechanism.

What experiments would have strengthened the paper:

  1. Compute-matched comparison: The paper emphasizes that Resampling Forcing eliminates the need for a 14B teacher model, but does not account for the total training compute. A fair comparison would match total FLOPs: Resampling Forcing's training cost (including the self-resampling forward passes, which use the full model with no gradient but still consume compute) versus the combined cost of training a bidirectional teacher and running distillation. Without this, the efficiency argument is qualitative rather than quantitative.
  2. FVD or distributional metrics: VBench provides reference-free quality assessment, but reference-based metrics like FVD (which measures distributional distance between generated and real videos) are standard in the video generation literature and would provide a complementary signal about sample fidelity.
  3. Human evaluation: Automated metrics can miss perceptual quality differences. A human preference study comparing Resampling Forcing against Self Forcing on visual quality, temporal consistency, and physical plausibility would substantially strengthen the claims.
  4. Systematic causal violation testing: A benchmark of physically-constrained scenarios (pouring, falling, occlusions, conservation laws) with automated violation detection would transform the causal superiority claim from anecdotal to quantitative.
  5. Scaling with video length: All experiments generate 15-second videos. The paper's central claim about error accumulation implies that Resampling Forcing's advantage over teacher forcing (and possibly over distillation) should grow with video length. Testing at 30 seconds, 60 seconds, or longer would test this prediction. The absence of longer-horizon results is a significant gap given the paper's framing around world simulation and long-horizon generation.
  6. Training from scratch: The experiments start from pre-trained WAN2.1-1.3B weights. While the paper claims the method "enables training autoregressive video models from scratch and at scale" in the abstract, no scratch-training experiment is reported. The warmup phase with 10K teacher forcing steps, followed by Resampling Forcing, is a fine-tuning pipeline — not scratch training. Whether Resampling Forcing would converge from random initialization (without the teacher forcing warmup) is untested, and the paper's own argument about cold-start problems suggests it would not.
  7. Sensitivity to hyperparameters: Only $s$ and sparsity level $k$ are ablated. The learning rate, batch size, warmup duration, number of resampling Euler steps, and video length curriculum are all fixed without justification. Given the novelty of the training procedure, understanding sensitivity to these choices would be valuable.

Overall assessment: The experiments convincingly demonstrate that Resampling Forcing works — it produces long-horizon videos with temporal stability competitive with distillation-based methods — and the autoregressive resampling ablation (Table 2) cleanly validates the core mechanism. However, the evidence base is narrower than the paper's claims suggest. The superiority claims (better causality, negligible sparsity loss, competitive quality) rest on thin quantitative support (a single qualitative example for causality, modest metric gaps without significance testing, no compute-matched comparisons, no human evaluation). The paper demonstrates a promising new training paradigm with clear advantages in simplicity and temporal stability, but it falls short of establishing that this paradigm is superior to distillation-based approaches rather than competitive with tradeoffs. The strongest results — temporal stability over time, effectiveness of autoregressive over parallel resampling — are well-supported. The broader claims about causality, scalability, and practical efficiency remain suggestive rather than proven.

6. Limitations and Trade-offs

6.1 The Warmup Requirement Precludes True Training from Scratch

The assumption or constraint. Resampling Forcing is presented in the abstract as a framework that "enables training autoregressive video models from scratch and at scale." However, Section 3.2 explicitly acknowledges a cold-start problem: "In the initial training phase, the model has not yet converged to the causal architecture and is incapable of generating meaningful content autoregressively. The model errors at this stage are dominated by random initialization rather than specific intra-frame imperfections or inter-frame accumulation. Therefore, performing history self-resampling can lead to uninformative learning signals and will hinder convergence." The solution is a teacher forcing warmup: "Once the model acquires basic autoregressive capabilities (though imperfect), we transition to Resampling Forcing and continue training."

The consequence. The method does not actually train from scratch in the sense of starting from random weights. It requires a two-phase pipeline where the model first learns autoregressive generation under teacher forcing (10K steps on 5-second videos, per Section 4), then transitions to Resampling Forcing. This means the model must already be capable of coherent autoregressive generation before the core innovation — self-resampling — can be applied. The paper does not demonstrate that Resampling Forcing would converge if applied from random initialization, and its own analysis suggests it would not. This limits the claim of "training from scratch" — the method is more accurately described as a fine-tuning or continued-training paradigm applied after an initial teacher-forcing phase. For practitioners seeking to train causal video models without any bidirectional pretraining, the warmup requirement means teacher forcing is still necessary, and the question of whether Resampling Forcing can replace teacher forcing entirely (rather than augment it after a burn-in period) remains open.

What evidence exists in the paper. The paper does not ablate the warmup duration or test convergence from random initialization with Resampling Forcing. Section 4 specifies the warmup length (10K steps) as a fixed hyperparameter without sensitivity analysis. There is no experiment showing that a model trained with Resampling Forcing from step zero would fail — the claim about "uninformative learning signals" is asserted based on reasoning, not demonstrated empirically.

Mitigation status. The paper does not propose a solution to the cold-start problem. The warmup is treated as a practical necessity, and the paper does not position it as a limitation. Future work on initialization strategies, auxiliary objectives during early training, or progressive transition schedules could potentially eliminate the warmup requirement, but none are explored.

6.2 Self-Resampling Training Cost Is Unaccounted for in Efficiency Claims

The assumption or constraint. The headline efficiency claim is that Resampling Forcing achieves quality comparable to distillation-based methods without requiring a 14B-parameter bidirectional teacher model. The "Teacher Model" column in Table 1 is blank for the proposed method, emphasizing this as a practical advantage. However, Resampling Forcing incurs its own training overhead: during each training iteration, the model performs an autoregressive resampling pass over the full video sequence (Algorithm 1, lines 9–12), which requires running the forward model N times (once per frame chunk) with gradient computation disabled but full model evaluation. While the paper notes that gradient detachment and KV caching make this efficient, the actual computational cost relative to standard teacher forcing is not quantified anywhere in the paper. The resampling uses a 1-step Euler solver for efficiency, but this still represents an additional forward pass through the entire model for every frame in every training video.

The consequence. The efficiency argument compares Resampling Forcing's training cost without a teacher against distillation methods' training cost with a teacher, but this is an apples-to-oranges comparison: the distillation methods do not incur per-iteration resampling overhead. A practitioner evaluating whether to adopt Resampling Forcing over Self Forcing needs to know the total training FLOPs or wall-clock time, not just the absence of a teacher model. If the resampling overhead makes each training iteration 2–3× more expensive than standard teacher forcing (one forward pass for the noisy frame plus N forward passes for autoregressive resampling of the history), then the method's training cost advantage depends on how many iterations are needed relative to distillation. The paper provides no data to answer this question.

What evidence exists in the paper. Section 4 states that a 1-step Euler solver is used for history resampling "for efficiency" and that KV caching is employed, but no throughput measurements, FLOP counts, or training time comparisons are reported. The batch size is 64, the model is 1.3B parameters, and training runs for 31.5K total steps (10K warmup + 15K Resampling Forcing on 5s video + 5K on 15s video + 1.5K routing fine-tuning), but the per-iteration cost multiplier relative to teacher forcing is never stated.

Mitigation status. Not addressed. The paper acknowledges that "our training entails processing dual sequence (diffusion samples and clean history)" in the Limitations section (Section 5), but frames this as an architectural concern ("could be improved with architecture optimizations") rather than a compute-accounting one. The efficiency claim in the abstract and conclusion rests solely on the absence of a teacher model, not on empirical measurements of total training resource requirements.

6.3 Visual Quality Gap Relative to Distillation Methods Is Unexplained and Potentially Significant

The assumption or constraint. The paper claims Resampling Forcing "achieves generation quality comparable to state-of-the-art distilled models" (Section 1). However, the quantitative results in Table 1 show a consistent visual quality deficit: on the full 0–15s segment, Self Forcing scores Visual: 67.12 versus Resampling Forcing's 64.72 — a gap of 2.4 points. CausVid scores 65.80. LongLive scores 66.56. All three distillation baselines exceed Resampling Forcing on visual quality, with gaps ranging from roughly 1.1 to 2.4 points. The paper does not report confidence intervals, so whether these gaps are statistically significant is unknown, but the consistency across baselines suggests a systematic rather than random difference.

The consequence. For applications where visual fidelity — aesthetic quality, imaging quality, color accuracy — is the primary requirement (e.g., cinematic content creation, advertising), the 2.4-point deficit relative to Self Forcing may be practically meaningful. The paper's narrative emphasis on temporal stability and causality leaves this gap undiscussed, but a practitioner choosing between methods needs to know whether the improved temporal consistency comes at a cost to per-frame appearance quality. The gap could arise from the distillation teacher (WAN2.1-14B) having been trained with substantially more compute and on a larger dataset than the WAN2.1-1.3B base model, meaning Resampling Forcing inherits the visual quality ceiling of its 1.3B base model while distillation methods can partially inherit the teacher's superior visual quality. If so, this is a fundamental tradeoff: teacher-free training avoids causal contamination but cannot access the visual quality boost that a larger, better-trained teacher provides.

What evidence exists in the paper. Table 1 shows the visual quality gap across all three temporal segments: at 0–5s, Self Forcing leads 67.12 vs. 64.72; at 5–10s, 66.18 vs. 64.03; at 10–15s, 63.04 vs. 63.99 (where Resampling Forcing slightly surpasses). The gap narrows with video length, suggesting Resampling Forcing's visual quality degrades more slowly than Self Forcing's. But the paper never discusses why the gap exists, whether it is fundamental, or whether scaling the base model or training data would close it.

Mitigation status. Not addressed. The paper's discussion and conclusion emphasize the temporal and causal advantages without acknowledging the visual quality tradeoff. No ablation tests whether longer Resampling Forcing training, a larger base model, or different hyperparameters would close the gap.

6.4 Causal Superiority Evidence Is Anecdotal, Not Systematic

The assumption or constraint. One of the paper's central claims is that distillation-based methods produce videos that violate temporal causality — specifically, that the bidirectional teacher leaks future information to the student — while Resampling Forcing's strictly causal training prevents this. The evidence for this claim is a single qualitative example in Figure 5 (lower panel): a "milk pouring" video where LongLive produces a liquid level that rises and then falls during continuous pouring, while Resampling Forcing produces a monotonically increasing liquid level.

The consequence. The causal superiority claim is the primary justification for preferring teacher-free training over distillation — if distillation methods achieve similar or better quality while maintaining acceptable causal behavior in practice, the motivation for an entirely new training paradigm weakens. A single cherry-picked example does not establish whether causal violations are (a) frequent or rare in distillation-based methods, (b) specific to LongLive or common across all distillation approaches, (c) detectable by human viewers or only visible under careful frame-by-frame inspection, or (d) present at all in the other baselines (Self Forcing, CausVid) that use different distillation procedures. Without systematic evaluation, the causal superiority argument remains an existence proof of a potential failure mode, not a demonstrated practical advantage.

What evidence exists in the paper. Figure 5 (lower panel) shows one comparison with one baseline (LongLive) on one prompt. No quantitative metric for causal consistency is proposed or evaluated. The paper does not test whether Self Forcing or CausVid exhibit the same failure, whether the violation is reproducible across multiple seeds, or whether Resampling Forcing itself sometimes produces causal violations on other prompts.

Mitigation status. The paper does not acknowledge the thinness of this evidence. The causal superiority is presented as a demonstrated property rather than a hypothesized advantage supported by a suggestive example. The Limitations section (Section 5) does not mention this as a limitation of the current evaluation.

6.5 Generalization Is Tested on a Single Benchmark, Single Model Family, Single Video Length

The assumption or constraint. All experiments use a single base architecture (WAN2.1-1.3B), a single evaluation benchmark (VBench), and a single generation length (15 seconds). The paper states in Section 4 that it "builds upon WAN2.1-1.3B architecture and loads its pre-trained weights to speed up convergence." No experiments test whether Resampling Forcing transfers to different base architectures (e.g., other DiT variants, UNet-based diffusion models, non-diffusion autoregressive video generators), different video domains (e.g., real-world vs. synthetic, different resolutions, different frame rates), or substantially different video lengths (e.g., 30 seconds, 60 seconds, open-ended generation).

The consequence. Several aspects of the approach could be model-specific or domain-specific. The logit-normal timestep distribution and shifting factor $s = 0.6$ were tuned for a model initialized from WAN2.1-1.3B — a stronger base model might require different degradation levels, and a weaker one might fail entirely under the same settings. The VBench benchmark captures a specific distribution of prompts and visual content that may not represent the difficulty profile of other video generation tasks. Most critically, the paper's central argument about error accumulation implies that Resampling Forcing's advantage should grow with video length relative to teacher forcing or distillation methods that degrade over time — but this prediction is untested beyond 15 seconds. If the temporal stability curves in Table 1 were extrapolated, Resampling Forcing might dominate at 30 seconds, but this is speculation without empirical data.

What evidence exists in the paper. All quantitative results in Table 1 and Table 2 are on VBench with 15-second videos from WAN2.1-1.3B. The paper does not report results on other standard video generation benchmarks (e.g., UCF-101, Kinetics, MSR-VTT) or other evaluation protocols. The qualitative comparison in Figure 5 includes other baselines but does not span different video lengths or domains.

Mitigation status. The Limitations section (Section 5) does not discuss the single-benchmark, single-model, single-length scope as a limitation. The paper's claims about "world simulation" and "long-horizon generation" implicitly assume generalization, but this remains undemonstrated.

6.6 Inference Latency from Iterative Denoising Is Not Addressed

The assumption or constraint. As a diffusion-based approach, Resampling Forcing requires iterative denoising at inference time — 32 Euler steps per frame with classifier-free guidance, as specified in Section 4. This is orthogonal to the training methodology but has direct practical consequences: each autoregressive step involves running the full model forward 32 times (or 64 times with classifier-free guidance, depending on implementation) for the current frame chunk, plus attending to all accumulated history frames. The paper acknowledges this in Section 5: "As a diffusion-based approach, our model necessitates iterative denoising steps for inference. Achieving real-time latency may require post-hoc acceleration, such as few-step distillation or improved samplers."

The consequence. For interactive applications — game simulation, real-time content creation, user-guided video generation — the latency of generating each frame (or chunk of 3 frames) is critical. The paper does not report inference latency, frames per second, or GPU memory usage for any video length. A 15-second video at the reported configuration (249 frames, 32 Euler steps per chunk of 3 frames, 480×832 resolution, 1.3B parameters) likely requires minutes of GPU time rather than seconds. The history routing mechanism reduces attention complexity but does not reduce the number of denoising steps. A practitioner evaluating Resampling Forcing for an interactive use case has no latency data to work with, and the paper's suggestion of post-hoc distillation as a solution means the model would require additional training (and possibly a teacher model) to become practical for real-time applications — somewhat undermining the teacher-free advantage.

What evidence exists in the paper. No latency measurements, throughput numbers, or memory profiles are reported. The paper uses a 1-step Euler solver during training resampling for efficiency, which suggests awareness of computational constraints, but no inference-time cost analysis is provided.

Mitigation status. The paper acknowledges the limitation explicitly in Section 5 and suggests future work on few-step distillation or improved samplers. This is a fair acknowledgment, but it means the method as presented is not suitable for latency-sensitive deployment without additional optimization — a tradeoff that should inform practitioners' adoption decisions.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the conceptual framing of exposure bias in autoregressive generative models from a distribution matching problem to a dynamical stability problem. The dominant paradigm — whether through noise augmentation, Diffusion Forcing, or post-training distillation — has treated exposure bias as a train-test mismatch to be minimized: align the training distribution with the inference distribution, and error accumulation goes away. Resampling Forcing argues, both conceptually (Section 3.1, Figure 2) and empirically (the Self Forcing baseline in Table 1 still degrades over time despite explicit inference simulation), that distribution matching is not sufficient. Even when the training and inference distributions are perfectly aligned, the autoregressive mapping from history to output can be amplifying — small input errors produce larger output errors, which compound. The paper demonstrates that what is needed is not just alignment but contraction: a learned mapping that actively suppresses input errors rather than propagating them.

This reframing matters because it changes what researchers should optimize for. Under the distribution-matching view, the goal is to make the model's self-generated histories indistinguishable from ground truth — more powerful teachers, better discriminators, more accurate inference simulation. Under the dynamical stability view, the goal is to train the model so that conditioning on degraded inputs produces clean outputs, regardless of whether the degradation perfectly matches inference-time statistics. The self-resampling mechanism in Algorithm 1 is a direct instantiation of this shifted objective: degrade with the model's own errors, but train with clean targets. The fact that a 1-step Euler approximation of the resampling trajectory suffices (Section 4) suggests the training signal does not require high-fidelity inference simulation — it requires exposing the model to structured degradation paired with clean prediction goals.

The strongest evidence for this reframing's validity is the difficulty-dependent behavior of the baselines in Table 1. Self Forcing achieves comparable or better visual quality on short horizons (67.12 vs. 64.72 at 0–5s) — its distribution matching works well locally. But its temporal quality drops sharply as the video lengthens (90.03 → 84.27 over 15 seconds, a 5.76-point degradation), while Resampling Forcing's temporal quality remains nearly stable (91.20 → 89.74, a 1.46-point degradation). This pattern — competent short-horizon performance with progressive temporal collapse — is exactly what the dynamical stability view predicts for a model that matches distributions but does not learn contraction. The distillation baselines learn to produce frames that look individually correct (good visual scores) but do not learn to correct upstream errors (poor temporal stability). Resampling Forcing's training, by conditioning on degraded inputs with clean targets, optimizes directly for error correction, and the result is a flatter temporal degradation curve.

The paper also delivers a negative-result-with-implications that challenges the distillation paradigm: the milk-pouring demonstration in Figure 5 (lower panel) shows that a bidirectional teacher can transfer non-causal behaviors to a causally-architected student. This is not obvious a priori — a reasonable assumption would be that a causal attention mask prevents the student from learning non-causal patterns regardless of the teacher's behavior. The paper's evidence suggests otherwise: the teacher's outputs encode future-dependent features that the student learns to reproduce, effectively smuggling non-causal information through the distillation loss even though the student's own architecture cannot directly attend to the future. If this result generalizes — and the paper does not systematically test it — it means architectural causality is necessary but insufficient for causal behavior when the training signal comes from a non-causal source. This has implications beyond video generation for any domain where causal structure matters (physical simulation, decision-making, world models) and where distillation from more powerful but non-causal models is an attractive shortcut. It shifts the burden of proof onto distillation methods to demonstrate that their students have not internalized causal violations.

In terms of research direction attractiveness, this work makes teacher-free training of autoregressive models a viable and principled alternative to the distillation-heavy status quo. Prior to this paper, the only demonstrated path to competitive long-horizon autoregressive video quality ran through bidirectional teacher models (Table 1: Self Forcing, CausVid, and LongLive all depend on WAN2.1-14B). Resampling Forcing establishes that a model can train itself into robustness using only its own online weights as an error simulator, without ever accessing a teacher's outputs. This does not make distillation obsolete — the visual quality gap of 2.4 points on the 0–5s segment suggests teachers still provide something valuable — but it does mean that teacher dependence is no longer a hard requirement. For resource-constrained settings, for domains where no suitable teacher exists, or for applications where causal fidelity is paramount, teacher-free training is now a demonstrated pathway rather than a speculative hope.

It also elevates learned sparse attention from an engineering optimization to a diagnostic and design tool. The history routing frequency visualization in Figure 8 — showing that the model autonomously discovers an "attention sink + recency" pattern — provides empirical grounding for heuristic sparse attention designs while simultaneously demonstrating that learned routing can outperform fixed heuristics at equal sparsity (Figure 7: top-1 routing preserves appearance consistency where size-1 sliding window fails). This suggests a general principle: let the model discover its own sparsity pattern rather than imposing one architecturally, and use the discovered pattern to inform future architecture design.

Follow-Up Research This Work Enables

Systematic evaluation of causal fidelity in video generation models. The milk-pouring example in Figure 5 is a compelling anecdote, but it establishes existence of a failure mode, not its prevalence. A strong follow-up would construct a benchmark of 50–100 physically-constrained scenarios — pouring, falling objects, collisions, occlusions, conservation of liquid volume, object permanence — with automated violation detectors (e.g., object trackers that flag physically impossible trajectories, segmentation masks that detect disappearances, optical flow consistency checks). Run Resampling Forcing, Self Forcing, CausVid, and LongLive on this benchmark across multiple seeds and measure (a) frequency of causal violations per minute of video, (b) correlation between violation frequency and video length, and (c) whether violations are perceptible to human raters. The hypothesis from this paper's argument is that distillation-based methods should exhibit higher violation rates that grow with video length, while Resampling Forcing's violations (if any) should remain constant. This would transform the causal superiority claim from anecdotal to quantitative and establish whether bidirectional teacher contamination is a first-order or second-order problem in practice.

Scaling Resampling Forcing to truly long horizons (30–120 seconds) to test the error contraction hypothesis. The central claim of the paper is that Resampling Forcing teaches error correction, which should lead to bounded rather than compounding error growth. The current experiments test only up to 15 seconds, where the temporal degradation of Self Forcing is visible (90.03 → 84.26) but not catastrophic. The contraction hypothesis makes a strong prediction: Resampling Forcing's temporal quality should asymptote to some stable level as video length increases, while teacher forcing and distillation methods should eventually diverge to complete collapse. A follow-up would train on 30–60 second videos (requiring architectural adjustments for memory, potentially using the history routing mechanism more aggressively) and measure temporal quality at 15, 30, 45, 60 seconds. If Resampling Forcing maintains Temporal > 85 at 60 seconds while baselines fall below 70, this would strongly validate the dynamical stability framing. If all methods degrade similarly beyond some length, the contraction benefit may saturate, and the practical advantage of Resampling Forcing would be limited to intermediate horizons.

Ablation of gradient detachment to test for shortcut learning. The paper asserts that gradients must be detached from the self-resampling process to "prevent shortcut learning" (Section 3.2), but provides no empirical evidence that such shortcuts would occur. A direct ablation would train two identical models — one with gradients detached during resampling (as in Algorithm 1) and one with gradients flowing through the resampling process — and compare both training dynamics and final generation quality. The hypothesis from the paper is that the non-detached model would learn to produce deliberately degraded histories during resampling to make the main loss trivially small, resulting in a model that ignores history rather than correcting it. If this hypothesis holds, we would expect the non-detached model to achieve lower training loss but worse temporal consistency at inference, particularly on long videos where history dependence matters most. If it does not hold — if the non-detached model trains fine — then the gradient detachment is an unnecessary complexity and the method could be simplified. Either outcome is informative.

Online vs. fixed checkpoint resampling: does the self-regulating feedback loop matter? The paper argues that using online model weights for resampling creates a self-regulating curriculum: as the model improves, simulated errors become subtler; if the model degrades (e.g., during domain shift to longer videos), errors become larger, providing appropriately calibrated training signal. This is a theoretically appealing property, but whether it matters empirically is untested. A comparison would train three variants: (a) online weights (the current method), (b) a periodically updated frozen checkpoint (e.g., snapshot every 5K steps), and (c) an initial fixed checkpoint (the warmup model, never updated). Measure training convergence speed and final generation quality. If online weights provide a clear advantage, this validates the self-regulating curriculum. If the frozen checkpoint performs comparably, then the online aspect is unnecessary and the method could be simplified to pre-compute resampled histories offline, dramatically reducing per-iteration training cost.

Transfer of Resampling Forcing to discrete autoregressive domains (language, code, audio). The mechanism of partial denoising trajectory resampling is specific to continuous diffusion models, but the conceptual template — degrade conditioning inputs using the model's own generation process from a controlled starting point, train with clean targets on degraded inputs — is domain-agnostic. For autoregressive language models, a potential analog would be: given a ground-truth text sequence, randomly select a prefix length, generate the remaining tokens autoregressively using the online model, then train the model to predict the ground-truth continuation conditioned on the self-generated (potentially erroneous) prefix. This is similar to Scheduled Sampling but with a crucial difference: the degradation uses the model's own online generation rather than randomly replacing tokens with model samples at a fixed probability. A follow-up would implement this "online self-resampling" for a small language model (e.g., 1B parameters) on a task where exposure bias is known to matter — long-form text generation, code completion with multi-line coherence, or mathematical proof generation — and compare against standard teacher forcing and Scheduled Sampling. The key question is whether using the model's own online autoregressive degradation (rather than scheduled token replacement) provides a better training signal for long-sequence coherence. A negative result (online resampling performs no better than Scheduled Sampling) would clarify the boundary conditions of the method — perhaps the mechanism is specific to continuous diffusion models where partial degradation is naturally defined via noise level, and does not transfer to discrete token prediction.

Scaling laws for Resampling Forcing: how does the advantage over teacher forcing change with model size and data? The paper uses a single model size (1.3B parameters) trained on a fixed dataset. The dynamical stability framing suggests that larger models with lower per-frame error rates would benefit less from error correction training — if the base error is already small, the compounding effect is weaker to begin with. Conversely, larger models might learn more effective error correction when trained with Resampling Forcing, widening the gap. A scaling study training models at 300M, 1.3B, and 3B+ parameters with both teacher forcing and Resampling Forcing, each trained to convergence on the same data, would map how the temporal stability advantage scales with model capacity. If the advantage shrinks with model scale, then Resampling Forcing is most valuable for smaller, resource-efficient models — consistent with the broader theme of trading test-time or training-time computation for model size. If the advantage grows, it is a universally beneficial technique. This is directly analogous to the scaling analysis in the reference paper's FLOPs-matched comparison (Section 7), which mapped how test-time compute advantages varied with model scale and problem difficulty.

Practical Applications and Downstream Use Cases

Training video world models for physical simulation without large bidirectional teachers. The primary practical value proposition of Resampling Forcing is enabling small teams or organizations with limited compute to train causally-faithful autoregressive video models without first training or accessing a massive bidirectional model. Table 1 shows that a 1.3B-parameter model trained with Resampling Forcing matches or exceeds the temporal quality of models distilled from a 14B-parameter teacher — a >10× reduction in the model capacity required in the training pipeline. For applications like robotics simulation (where predicting future visual states conditioned on actions is essential), game environment modeling, or physical science visualization, this means viable autoregressive video models can be built with a fraction of the total training compute previously assumed necessary. The 15-second training length already demonstrated is sufficient for many short-horizon simulation tasks, and the history routing mechanism (75% sparsity with negligible quality loss per Table 1) provides a template for extending to longer horizons under memory constraints.

Interactive video generation with per-frame user control and guaranteed causal consistency. Figure 5 demonstrates that distillation-based methods can produce physically impossible outputs (liquid level rising then falling during continuous pouring). For applications where users interact with a generated video frame by frame — inserting objects, changing camera angles, applying forces — causal fidelity is non-negotiable: the model must respond to user inputs in a way that respects the established scene state without retroactively altering earlier frames. Resampling Forcing's strictly causal training, with no pathway for future information to contaminate current-frame generation, provides a stronger guarantee of this property than distillation-based alternatives. The autoregressive structure with KV caching (Section 3.1) means new frames can be generated conditioned on the entire interaction history without recomputing earlier frames, and the history routing mechanism keeps the attention cost bounded as the interaction sequence grows. The main practical barrier is inference latency (32 denoising steps per autoregressive chunk), which the paper acknowledges (Section 5 Limitations) and which would require distillation or improved samplers for real-time interactivity. For non-real-time interactive editing — where users specify changes and wait seconds rather than milliseconds for results — the current latency may already be acceptable.

Data generation pipelines for self-improving video models. A natural extension of the paper's approach is to use Resampling Forcing-trained models to generate high-quality synthetic training data for further improving the model or training downstream systems. The temporal stability advantage (temporal quality degrading only 1.46 points over 15 seconds versus 5.77 for Self Forcing) means the generated videos maintain coherence over longer horizons, making them more suitable as training targets for tasks that require long-range temporal reasoning — action recognition, event detection, future frame prediction. Because Resampling Forcing requires no teacher model, the data generation pipeline is self-contained: the model generates videos, the videos are filtered or scored (potentially using the model's own velocity prediction error as a quality signal), and the best videos are fed back into training. This avoids the cost and potential causal contamination of routing through a bidirectional teacher for data generation. The paper's cited work on revision models and self-improvement loops (Section 8 of the reference paper, discussing ReSTEM^{\text{EM}} and self-distillation) provides a template: generate long videos with Resampling Forcing, use a verifier to select high-quality segments, and fine-tune the model on those segments. The sparse history routing mechanism is particularly relevant here — training on long synthetic videos would benefit from efficient attention to distant history, exactly what the routing mechanism provides.

Budget-constrained academic research on autoregressive video models. Prior to this work, a research group wanting to experiment with high-quality autoregressive video generation faced a chicken-and-egg problem: you need a bidirectional teacher to train a good causal model, but training a competitive bidirectional teacher requires industrial-scale compute. Resampling Forcing, by eliminating the teacher requirement, lowers the barrier to entry. The full training pipeline described in the paper — 31.5K steps on a 1.3B-parameter model initialized from public pretrained weights — is feasible on a modest academic GPU cluster (the paper does not report exact GPU-hours, but the batch size of 64 at 480×832 resolution with a 1.3B model suggests a few hundred GPU-days). The history routing mechanism further reduces the memory requirements for training on longer videos. This democratization effect — making a previously teacher-gated technique accessible — is a practical impact that extends beyond the specific quality improvements reported in the paper.