Self-Forcing++: Towards Minute-Scale High-Quality Video Generation

ArXiv: 2510.02283

🎯 Pitch

Autoregressive video models collapse beyond 10 seconds due to error accumulation, but Self-Forcing++ breaks this barrier without any long-video training data. By having the student model learn to recover from its own mistakes on self-generated long rollouts—using corrective guidance from a short-horizon teacher—it generates coherent videos up to 4 minutes 15 seconds, over 50× longer than the baseline.

---

1. Executive Summary

This paper introduces Self-Forcing++, a training framework that mitigates quality degradation in autoregressive long-video generation without requiring supervision from long-video teachers or retraining on long datasets. Using the Wan2.1-T2V-1.3B model as its base and evaluating on VBench and an extended MovieGen prompt set, the method leverages the teacher model's rich corrective knowledge through two complementary mechanisms — Backward Noise Initialization (re-injecting noise into self-generated long rollouts to create temporally consistent initial states) and Extended Distribution Matching Distillation (a sliding-window distillation procedure that trains the student to match the teacher on uniformly sampled segments of self-generated long videos) — to scale video length up to 20× beyond the teacher's 5-second capability while avoiding over-exposure and error accumulation. When scaling training computation, Self-Forcing++ generates videos up to 4 minutes and 15 seconds, equivalent to 99.9% of the base model's positional embedding capacity and more than 50× longer than the baseline, establishing that sustained high-fidelity long-horizon generation is achievable through corrective distillation from short-horizon teachers only when the student is explicitly trained to recover from its own accumulated autoregressive errors.

2. Context and Motivation

The Core Problem: Autoregressive Video Generation Collapses Beyond the Training Horizon

The fundamental challenge this paper confronts is a specific failure mode in autoregressive video generation: when a model trained to generate short video clips (e.g., 5 seconds) is asked to generate longer sequences at inference time, its output quality degrades catastrophically—often collapsing into static scenes, over-exposed frames, or pure noise. This phenomenon is not merely a gradual decline in visual fidelity; it represents a fundamental breakdown of the autoregressive mechanism where accumulated prediction errors compound over time, eventually destroying the semantic coherence of the generated video entirely.

To understand why this matters, we need to appreciate the state of video generation in 2024-2025. Diffusion-based video models—Sora, Wan, Veo, Hunyuan Video—have achieved remarkable visual quality for short clips. However, they are architecturally constrained: the underlying Diffusion Transformer (DiT) uses bidirectional (non-causal) attention, meaning every frame can attend to every other frame during denoising. While this produces high-quality short videos, it has two crippling limitations for long-form generation:

1. Quadratic computational cost: Bidirectional attention scales as $O(L^2)$ with sequence length $L$, making naive extension to minutes-long videos computationally prohibitive.

2. Non-streaming architecture: A bidirectional model cannot generate frame-by-frame in a streaming fashion—it must denoise the entire sequence jointly, which requires all frames to fit in memory simultaneously.

This sets up the central tension in the field: bidirectional models produce high-quality short videos but cannot scale temporally; autoregressive models can scale temporally but suffer severe quality degradation when doing so. The paper's mission is to close this gap—making autoregressive models generate long videos that maintain the quality of their bidirectional teachers.

Why This Problem Is Important

Practical significance. Most real-world videos exceed 5-10 seconds. Films, user-generated content, virtual environments, and game cinematics routinely span minutes or hours. A model that can only produce 5-second clips is, in practical terms, a toy—useful for demos but not for production content creation. Closing the temporal scaling gap would unlock applications in filmmaking, advertising, simulation, and interactive media that are currently served by expensive human production pipelines.

Economic significance. Training a bidirectional model directly on long videos is prohibitively expensive due to the quadratic attention cost. If an autoregressive approach can achieve comparable quality with $O(L)$ cost during inference (via KV caching), the economic case for autoregressive long-video generation becomes compelling. The paper's FLOPs argument is implicit but clear: distill a bidirectional teacher's knowledge into an efficient autoregressive student, then scale that student's horizon far beyond what the teacher could ever handle directly.

Theoretical significance. The paper identifies a specific technical phenomenon—dual training-inference mismatch—that explains exactly why autoregressive video generation fails at long horizons. This characterization is theoretically valuable because it provides a precise target for intervention: (1) temporal mismatch (training on short clips but running long at inference) and (2) supervision mismatch (the teacher never sees the student's accumulated errors during training, so the student never learns to recover from them). By naming and analyzing these failure modes, the paper advances our understanding of why autoregressive generation degrades and what corrective mechanisms work.

Methodological significance for the field. The paper's approach—using a short-horizon teacher to correct errors in a student's self-generated long rollouts—represents a broader principle: you don't need a long-horizon supervisor if you can teach the student to recover from its own mistakes. This principle extends beyond video generation to any autoregressive generative modeling domain where training data is limited to short sequences but inference demands long ones (e.g., long-form text generation, extended musical compositions, lengthy robotic trajectory planning).

Prior Approaches and Where They Fall Short

The paper positions itself against a landscape of prior attempts to extend video generation length. Understanding each approach's failure mode is essential because the paper's method is designed to address precisely the shortcomings it identifies.

Bidirectional Direct Generation (LTX-Video, Wan2.1)

The most straightforward approach is to simply train a bidirectional DiT on longer videos. This avoids the autoregressive error accumulation problem entirely—every frame gets bidirectional context, so there is no compounding of prediction errors. However, as noted above, the $O(L^2)$ attention cost makes this approach computationally infeasible for videos beyond ~5-10 seconds at scale. LTX-Video and Wan2.1 represent the state of this approach: excellent within their temporal window, structurally unable to exceed it.

The gap: Bidirectional models physically cannot generate long videos at practical cost.

Training-Free Positional Encoding Tricks (RIFLEx)

RIFLEx (Zhao et al., 2025) takes a clever shortcut: it observes that certain positional encoding values induce repetitive motion patterns in DiT-generated videos, and by carefully selecting which position encodings to use during inference, it can roughly double generation length without retraining. This is elegant and practical but fundamentally bounded—it extends the architectural envelope slightly but doesn't address the underlying autoregressive error accumulation that limits quality.

The gap: Training-free methods can only push the existing architecture so far; they don't learn to correct errors.

Diffusion Forcing (SkyReels-V2, MAGI-1)

Diffusion Forcing (Chen et al., 2024) proposes a hybrid approach: apply different noise levels to different frames according to a predefined schedule, enabling partially noised future frames to serve as a "soft" autoregressive context. The key idea is that by keeping some frames noisy while others are clean, the model can maintain long-term memory without fully committing to the errors that plague pure autoregressive approaches.

However, the paper identifies a critical practical problem: the combinatorial explosion of possible noise schedules creates severe training instability. As the authors note:

"the combinatorial complexity of noise scheduling often leads to training instability and has proven difficult to scale"

This is a fundamental issue with the approach—the very mechanism that gives Diffusion Forcing its flexibility (variable noise levels across frames) also makes the training objective enormously complex. SkyReels-V2 and MAGI-1 attempt to scale this approach but, as the paper's results show (Tables 1, 2; Figures 4, 8, 9), they still suffer from:

• SkyReels-V2: Preserves structure better than most baselines but exhibits moderate to severe over-exposure at long horizons, resembling CausVid's failure pattern.
• MAGI-1: Initially avoids over-exposure (likely due to the diffusion forcing mechanism) but rapidly deteriorates into heavy over-exposure and structural collapse at extended lengths.

The gap: Diffusion Forcing methods are unstable to train and still degrade at long horizons, albeit with different failure patterns than pure autoregressive approaches.

CausVid: The First Streaming Distillation Approach

CausVid (Yin et al., 2025) represents the baseline most directly ancestral to this paper. Its approach is genuinely clever: distill a bidirectional teacher into an autoregressive student by introducing block-causal attention and training the student to replicate the teacher's ODE trajectories. At inference time, the model uses a KV cache to efficiently generate new frames while attending to prior context, enabling streaming generation with $O(L)$ cost.

However, CausVid has two critical failure modes that the paper characterizes in detail:

1. Over-exposure from overlapping frame recomputation. CausVid's inference procedure involves recomputing overlapping frames at each autoregressive step to maintain temporal consistency. The authors observe that this "relies on recomputing overlapping frames and suffers from a severe over-exposure problem" (Section 3.2). The over-exposure is visible in Figure 4 and the Gemini-2.5-Pro evaluations: CausVid videos at 50-100 seconds are consistently rated as having "Noticeable Exposure Problems" with "persistent clipping in highlights or shadows."

The over-exposure is not a superficial artifact—it fundamentally distorts the visual content. As the Gemini evaluation in Figure 8 notes for one CausVid video: "The large celestial body in the background is significantly overexposed, with large areas blown out to pure white, causing a complete loss of surface detail." This is a consistent, not occasional, failure.

2. Training-inference mismatch. CausVid trains with a fixed attention pattern but infers with a rolling pattern, creating a distribution shift between what the model sees during training and what it encounters at deployment. This mismatch exacerbates error accumulation.

The gap: CausVid can generate long videos (Tables 1, 2 show it producing 50s, 75s, 100s outputs), but the quality is compromised by systematic over-exposure and its associated visual degradation.

Self-Forcing: Closing the Over-Exposure Gap at Short Horizons

Self-Forcing (Huang et al., 2025) directly addresses CausVid's over-exposure by aligning training and inference distributions: it incorporates the KV cache during training (so the model learns to condition on its own cached representations) and uses techniques like Distribution Matching Distillation (DMD) loss on self-generated rollouts. This produces high-quality short videos (5 seconds) that avoid CausVid's over-exposure—a genuine improvement.

However, Self-Forcing introduces a new and arguably worse failure mode at long horizons: quality collapse through error accumulation. The paper is explicit about this:

"its [Self-Forcing's] capacity remains bottlenecked by the fixed-duration teacher model. Consequently, when tasked with generating content beyond this intrinsic temporal window (e.g., >10 seconds), the model's visual quality degrades precipitously."

In Table 2, Self-Forcing's 100-second generation scores confirm this: dynamic degree drops to 26.41 (compared to the paper's 54.12), visual stability falls to 32.03 (vs. 84.22), and text alignment drops to 22.00 (vs. 26.04). The videos don't just look worse—they stop moving. As Figure 4 visually demonstrates (labeled "Self Forcing"), the video collapses to near-darkness by the 75-100 second mark, a failure mode the authors describe as "global darkening and stagnation."

Why does this happen despite Self-Forcing's training-inference alignment? The paper identifies a second, deeper mismatch that Self-Forcing doesn't address:

"a temporal mismatch occurs: during training, models generate short clips of up to 5 seconds—the maximum horizon of the teacher model—whereas at inference, they must generate videos of significantly greater length."

And crucially:

"error accumulation caused by supervision misalignment during long-horizon generation. In training, the teacher model provides abundant supervision for every frame within the short clip. This intensive guidance, however, means the student model is rarely exposed to the compounding errors that naturally arise in long rollouts, leaving it ill-equipped to handle them."

In other words: Self-Forcing teaches the student to generate clean 5-second videos, but it never teaches the student what to do when it's 50 seconds into a rollout and errors have already accumulated. The student has never seen its own degraded state during training, so when it encounters one at inference, it doesn't know how to recover.

The gap that Self-Forcing++ targets: Self-Forcing fixes CausVid's over-exposure but introduces error-accumulation collapse at long horizons. The student needs to be explicitly trained on its own error-accumulated states so it learns to correct rather than compound mistakes.

LongLive: A Concurrent Approach

In the Discussion (Section 7), the paper acknowledges LongLive (Yang et al., 2025) as a concurrent work that also builds on Self-Forcing and applies DMD to long self-rolled sequences. LongLive uses attention sink frames to mitigate error accumulation—a different mechanism than the paper's backward noise initialization and sliding-window distillation. The paper notes that its method "avoids reliance on attention sink frames to counter error accumulation, which was shown to be a key design of LongLive," positioning Self-Forcing++ as a simpler approach that achieves similar or better results.

The Broader Training-Inference Mismatch Pattern

Stepping back, the progression from CausVid → Self-Forcing → Self-Forcing++ reveals a pattern that the paper implicitly identifies: each fix for autoregressive video generation introduces a new failure mode at a longer horizon. CausVid solved "can we generate at all?" but introduced over-exposure. Self-Forcing solved over-exposure but hit an error-accumulation wall at ~10 seconds. Self-Forcing++ targets that wall directly.

This pattern is not unique to video generation—it echoes challenges in language model autoregressive decoding (where exposure bias causes quality degradation on long sequences) and in reinforcement learning (where distributional shift between training and deployment policies causes compounding errors). The paper's solution—explicitly training on self-generated error states—connects to techniques like DAGGER (Dataset Aggregation) in imitation learning, where the learner is iteratively exposed to and trained on its own mistakes.

How This Paper Positions Itself

Self-Forcing++ positions itself as the first method that can generate high-quality videos beyond 5 seconds without long-video supervision and without architectural hacks like attention sinks. The key positioning claims are:

1. A novel failure analysis that enables targeted intervention. The paper doesn't just propose a new method—it diagnoses why Self-Forcing fails at long horizons (dual training-inference mismatch: temporal + supervision) and designs its approach to address both mismatches simultaneously. The backward noise initialization addresses temporal consistency (ensuring the noise schedule is applied to temporally coherent sequences), while the extended DMD addresses supervision misalignment (exposing the student to its own accumulated errors during training).

2. Simplicity as a virtue. Compared to Diffusion Forcing (complex noise schedules) and LongLive (attention sink frames), Self-Forcing++ is architecturally minimal: it uses the same base model, same distillation framework, and same autoregressive mechanism as Self-Forcing, adding only the backward noise initialization and the sliding-window extended DMD. This simplicity makes the method easier to implement, faster to train (despite the paper noting slower training than teacher-forcing in limitations), and more interpretable.

3. Empirical dominance at all horizons with widening margins. Table 1 shows that Self-Forcing++ is competitive with Self-Forcing at 5 seconds (total score 83.11 vs. 83.00) but pulls dramatically ahead at 50 seconds (visual stability 90.94 vs. 40.12). This pattern—parity at the training horizon, dominance beyond it—is exactly what a method targeting horizon extension should demonstrate. The widening gap at 75s and 100s (Table 2) further validates the approach.

4. Scaling properties that suggest a new paradigm. Section 4.4 and Figure 6 demonstrate something the field hasn't seen before: training budget scaling produces monotonic improvements in video generation length and quality, from incoherent ODE initialization → 5-second coherence (1× budget) → 50-second stability (20×) → 255-second high-fidelity (25×). This scaling property is analogous to how language model performance scales with compute (the "scaling laws" observation) and suggests that long-video generation may be more a matter of sufficient corrective training than of architectural innovation.

5. A new evaluation protocol that exposes benchmark failures. The paper's critique of VBench for long-video evaluation (Section 3.4, Figure 3) is a significant meta-contribution. By showing that VBench's image quality and aesthetic quality metrics actually prefer degraded and over-exposed frames—"VBench tends to overrate degraded and over-exposed frames rendering these two metrics unreliable"—the paper identifies a systematic evaluation failure that has likely led the field to overestimate the quality of prior long-video methods. The Visual Stability metric (using Gemini-2.5-Pro as judge) is proposed as a more reliable alternative, supported by high Spearman correlation (94.2-100%) with human annotators.

6. Explicit boundaries on applicability. The paper is transparent about limitations (Section 6): slower training than teacher-forcing, lack of long-term memory (objects that leave the frame for extended periods may not be faithfully reconstructed upon return), and the capacity ceiling of the underlying Wan2.1-T2V-1.3B model. This honesty strengthens the positioning by making clear what the method doesn't claim to solve.

In summary, Self-Forcing++ enters a field where autoregressive long-video generation seemed stuck at a quality ceiling imposed by training-inference mismatch. Prior approaches either couldn't scale (bidirectional), scaled with severe artifacts (CausVid), scaled with over-exposure fixed but error accumulation unfixed (Self-Forcing), or required complex mechanisms with training stability issues (Diffusion Forcing). The paper argues that the missing piece was explicit, corrective supervision on the student's own error-accumulated states, delivered through a simple sliding-window distillation procedure that requires no new data, no long-video teachers, and no architectural changes beyond what Self-Forcing already introduced.

3. Technical Approach

This is primarily a training methodology paper whose core idea is that autoregressive video generation models collapse at long horizons because they are never trained on their own error-accumulated states, and that adding corrective supervision on self-generated long rollouts—using only a short-horizon teacher for guidance—can extend generation length by up to 50× while maintaining visual quality.

3.1 Reader Orientation

Self-Forcing++ is a training framework that takes an existing autoregressive video generation model (specifically, a distilled streaming version of Wan2.1-T2V-1.3B built on the Self-Forcing architecture) and teaches it to generate minutes-long videos by repeatedly exposing it to its own degraded long-horizon outputs during training and using a short-horizon bidirectional teacher to show it how to correct those degradations. The problem it solves is the precipitous quality collapse that occurs when autoregressive video models generate sequences longer than their training horizon—a collapse driven by the model never having practiced recovering from its own accumulated prediction errors—and the solution shape is a self-supervised corrective loop where the student rolls out long sequences, the teacher provides "what good looks like" on randomly sampled windows, and the student learns to bridge its degraded states back toward the teacher's distribution.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components connected in a training loop that extends beyond the inference pipeline itself:

1. Bidirectional Teacher Model (Wan2.1-T2V-1.3B, frozen): A standard DiT-based video diffusion model trained on 5-second video clips. It cannot generate long videos directly, but it possesses rich knowledge of what realistic video frames should look like—knowledge acquired from training on a vast video corpus. It serves as the "quality oracle" that provides corrective gradients to the student.

2. Autoregressive Student Model (distilled Wan2.1-T2V-1.3B, trained): A few-step generative model converted to autoregressive operation via block-causal attention and KV caching. It is initialized through ODE trajectory distillation from the teacher (producing short, low-quality videos) and then trained to generate progressively longer sequences through the Self-Forcing++ procedure. At inference time, it generates videos frame-by-frame using its rolling KV cache.

3. Self-Rollout Engine with Rolling KV Cache: The mechanism that runs the student model autoregressively for $N$ frames (where $N$ far exceeds the 5-second teacher horizon, e.g., 100 seconds' worth of frames). During training, this engine produces long video trajectories that inevitably contain accumulated autoregressive errors—and it is precisely these error-containing trajectories that become the training data for corrective distillation.

4. Extended Distribution Matching Distillation (Extended DMD) Module: The training objective that computes distributional discrepancy between the student and teacher. It operates by: (a) sampling a random contiguous window of length $K$ (5 seconds) from the student's $N$-frame self-rollout, (b) re-injecting noise into that window according to the diffusion noise schedule to create a temporally consistent but perturbed starting state, (c) having both the student and teacher denoise from that state, and (d) minimizing the KL divergence between the resulting distributions. This teaches the student to correct its own errors across all temporal positions in the long rollout.

Information flows cyclically during training: the student generates a long rollout → a random window is extracted and noise-injected → both student and teacher process this window → their distributional discrepancy produces gradients → the student updates to better match the teacher on this window → the improved student generates better (but still error-containing) long rollouts in the next iteration. Over training, the student learns to maintain quality across the entire long horizon.

3.3 Roadmap for the Deep Dive

• First, the mathematical foundation: the Distribution Matching Distillation (DMD) objective and the ODE trajectory initialization procedure, since these form the substrate on which all further mechanisms are built and explain how the bidirectional teacher's knowledge is initially transferred to the autoregressive student.

• Second, the core innovation—Backward Noise Initialization—since it enables temporally consistent long-horizon training by creating noise-initialized states that preserve the temporal dependencies of previously generated frames, without which the extended distillation would operate on contextually incoherent noise patterns.

• Third, the Extended Distribution Matching Distillation procedure itself, which is the primary training mechanism: how the sliding-window sampling works, what the extended DMD loss computes, and how it differs from the short-horizon DMD used in CausVid and Self-Forcing. This is where the paper's key insight—teaching the student to recover from its own errors—is operationalized.

• Fourth, the Rolling KV Cache mechanism and why eliminating train-inference cache mismatch matters for long-horizon consistency. This ties together the backward noise initialization and extended DMD into a unified training procedure (Algorithm 1).

• Fifth, the optional GRPO-based temporal smoothness enhancement, which addresses remaining long-term memory issues that the core training procedure doesn't fully solve. This is a complementary mechanism rather than a core requirement.

• Sixth, the new Visual Stability evaluation metric, since the paper's empirical claims depend on a benchmark that correctly penalizes the degradation modes (over-exposure, error accumulation) that VBench systematically under-penalizes.

3.4 Detailed, Sentence-Based Technical Breakdown

Distribution Matching Distillation (DMD) Foundation

The paper builds on Distribution Matching Distillation (DMD), a technique introduced in prior work (Yin et al., 2024) for distilling multi-step diffusion models into few-step generators. Understanding DMD is essential because both the baseline methods (CausVid, Self-Forcing) and the paper's extended version use it as the core training objective.

What DMD accomplishes. A standard diffusion model generates videos by iteratively denoising random Gaussian noise over many steps (typically 50-1000 steps) along a predefined noise schedule. This is computationally expensive—each video requires dozens of forward passes through a large transformer. DMD compresses this process into a few-step generator that can produce videos in, say, 4 denoising steps instead of 50, achieving 10-15× speedup while maintaining quality. The key insight is that you can train a few-step student by minimizing the reverse KL divergence between the student's generated distribution and the teacher's distribution, rather than by forcing the student to exactly replicate the teacher's multi-step denoising trajectory.

The DMD objective. The paper states the DMD loss in Equation (3):

$\nabla_\theta \mathcal{L}_{\text{DMD}} = \mathbb{E}_t \left[ \nabla_\theta \text{KL}(p_{\text{fake},t} \parallel p_{\text{real},t}) \right] = -\mathbb{E}_t \left[ \int \left( s_{\text{real}}(\Phi(G_\theta(z), t), t) - s_{\text{fake}}(\Phi(G_\theta(z), t), t) \right) \frac{dG_\theta(z)}{d\theta} dz \right]$

where $G_\theta$ is the student generator with parameters $\theta$, $z$ is the initial noise latent, $\Phi(G_\theta(z), t)$ is the transformation process that adds noise to the generated output at diffusion timestep $t$, $s_{\text{real}}$ is the teacher's score function (which points toward realistic data), $s_{\text{fake}}$ is the student's score function (or a learned approximation thereof), and the expectation $\mathbb{E}_t$ is taken over diffusion timesteps.

What it computes (operational English): The gradient update points in the direction that makes the student's generated distribution more similar to the teacher's distribution, as measured by reverse KL divergence. For each training step: (1) the student generator produces a video from random noise, (2) noise is added back to this generated video at a randomly sampled diffusion timestep $t$, (3) the teacher's score function evaluates "how to make this noised video more realistic" and the student's score function evaluates the same, (4) the difference between these two score directions is backpropagated through the student generator. Intuitively, the teacher tells the student "when you see a video of this quality (noised to level $t$), here's how to improve it," and the student updates to incorporate that guidance.

Why this form: Reverse KL is preferred over forward KL for distillation because it penalizes the student for generating samples that are unlikely under the teacher's distribution (mode-seeking behavior) rather than penalizing the student for failing to cover all modes of the teacher (which would encourage blurry, averaged outputs). In video generation, mode-seeking is desirable—we want the student to produce crisp, high-fidelity videos that lie in high-probability regions of the teacher's distribution, not to average over all possible variations. The score difference formulation (also known as Fisher divergence) is computationally tractable because it only requires evaluating score functions at generated samples, not integrating over the entire data space.

How DMD relates to the autoregressive conversion. The paper uses DMD for two purposes that occur at different stages: (1) initially distilling the bidirectional Wan2.1-T2V-1.3B teacher into a few-step generator (still bidirectional), and (2) later, during the Self-Forcing++ training, applying DMD to the autoregressive student's self-generated long rollouts to provide corrective supervision. The first stage is standard model compression; the second stage is where the paper's novelty lies. Critically, the DMD objective remains the same mathematical form in both cases—only the data fed to it changes (short teacher-generated videos vs. long student-generated videos).

ODE Trajectory Initialization: Converting Bidirectional to Autoregressive

Before Self-Forcing++ training can begin, the distilled few-step generator must be converted from a bidirectional model (all frames attend to all frames) into an autoregressive model (each frame attends only to previous frames) that can generate streaming video with KV caching. This conversion uses ODE trajectory distillation, which is formally distinct from the DMD loss.

What needs to happen. A bidirectional diffusion model generates all frames jointly: given a noise latent and a text prompt, it denoises the entire spatio-temporal volume simultaneously. An autoregressive model generates frames sequentially: it produces new frames one at a time (or in small chunks), conditioning each new frame on the clean, previously generated frames through a KV cache. The architectural change involves replacing full bidirectional attention with block-causal attention, where each frame's query can attend to keys and values from all previous frames but not from future frames.

The ODE training procedure (Equation 4). The paper formalizes this initialization as:

$\mathcal{L}_{\text{ode}} = \mathbb{E}_{x,t} \left[ \left\| G_\phi \left( \{x^{(i)}_{t_i}\}_{i=1}^N, \{t_i\}_{i=1}^N \right) - \{x^{(i)}_{\text{teacher}}\}_{i=1}^N \right\|^2 \right]$

where $G_\phi$ is the autoregressive student (with block-causal attention, parameterized by $\phi$), $\{x^{(i)}_{t_i}\}_{i=1}^N$ is a sequence of teacher-generated frames at their corresponding diffusion timesteps (an ODE trajectory sampled from the bidirectional teacher), $\{x^{(i)}_{\text{teacher}}\}_{i=1}^N$ is the fully denoised ground-truth video from that trajectory, $N$ is the number of frames, and the expectation is over training examples and diffusion trajectories.

What it computes (operational English): The bidirectional teacher is used to generate complete training trajectories: starting from random noise, the teacher runs its full multi-step denoising process, and at each step we record both the partially denoised state and the final clean output. The autoregressive student is then trained to map from these intermediate states to the final clean output, but with causal attention—meaning each frame's prediction can only use information from earlier frames and earlier denoising steps. The loss is simple mean squared error (MSE) between the student's predicted clean video and the teacher's actual clean video. This teaches the student to approximate the teacher's denoising trajectory under the autoregressive constraint.

How much data is needed and what quality is achieved. The paper specifies (Section 8.1, Implementation details) that the model is "initialized with sampled 16K ODE training trajectories." This is the warm-up phase, distinct from the main Self-Forcing++ training. At this stage, the model "exhibits only a nascent ability to generate short, low-fidelity clips" (Section 4.4, Figure 6, labeled "ODE"). The ODE initialization establishes the basic autoregressive mechanism—the model can generate sequentially with a KV cache—but the output quality is poor, with significant temporal inconsistency and visual artifacts. This is the starting point for Self-Forcing++ training.

Design choice: why ODE trajectories rather than random noise initialization. An alternative would be to train the autoregressive student directly from random noise, as is done for standard diffusion model training. However, the ODE trajectory approach provides a much stronger initialization because the intermediate states in an ODE trajectory are on the "path" from noise to data—they represent partially denoised states that the teacher would naturally encounter during generation. Training on these states teaches the student the teacher's preferred denoising trajectory, which produces better initial quality than starting from scratch. The 16K trajectory count is a practical balance: enough to establish the autoregressive mechanism across diverse prompts and video content, but not so many that it becomes a computational bottleneck before the main training begins.

Why this stage is critical for what follows. The ODE initialization is not the main contribution, but it establishes the substrate on which Self-Forcing++ operates. Without it, the student would not have the basic capability to generate autoregressively with a KV cache, and the extended DMD training would have nothing to build upon. The paper's insight is that this initialization is sufficient to get the autoregressive mechanism working, but insufficient for quality beyond the training horizon—which is exactly what Self-Forcing++ addresses.

Backward Noise Initialization: The Key Enabler for Long-Horizon Training

This is the first of two core innovations in Self-Forcing++ and the mechanism that makes long-horizon training possible without access to long real videos or long-video teachers. The idea is conceptually simple but has deep implications for how training and inference align.

The fundamental problem it solves. In standard short-horizon DMD training (as used in CausVid and Self-Forcing), the student generates a video from random Gaussian noise, and the DMD loss is computed on that entire video at once. This works for short clips because the noise schedule is designed to map from pure Gaussian noise to a clean video—the noise at timestep 0 is maximally noisy (pure Gaussian), and the denoising process progressively removes it. But when the student generates a long video autoregressively, frames at temporal position 100 have already been conditioned on clean frames 1-99 through the KV cache. If you were to initialize frame 100 from pure Gaussian noise, it would be contextually incoherent with the clean preceding frames—the noise would contain no information about the established scene, objects, or motion patterns. Computing DMD on such a noise-initialized frame would measure distributional divergence on an input that could never arise during actual autoregressive inference (where frame 100 is generated from a clean KV cache, not from noise).

The backward noise initialization procedure (Equation 1). The paper formalizes this as follows:

Given a clean trajectory $\{x_S^t\}_{t=1}^N$ generated by the student (where $N$ far exceeds the teacher's horizon), for each frame at temporal position $t$, the perturbation is:

$x_t = (1 - \sigma_t) x_0 + \sigma_t \epsilon$

where $x_0 = x_{t-1} - \sigma_{t-1} \hat{\epsilon}_\theta(x_{t-1}, t-1)$, $\sigma_t$ is the noise level at diffusion timestep $t$ according to the prescribed diffusion noise schedule, $\epsilon \sim \mathcal{N}(0, \mathbf{I})$ is randomly sampled Gaussian noise, and $\hat{\epsilon}_\theta$ is the noise prediction network (the student itself, parameterized by $\theta$).

What it computes (operational English): For each frame in the student's self-generated long rollout, we "add noise back" to the clean frame, but the amount and type of noise addition depends on the diffusion timestep $t$. At early denoising steps (large $\sigma_t$), we add a lot of noise—the frame becomes mostly random Gaussian—while at late denoising steps (small $\sigma_t$), we add little noise, keeping the frame close to its clean state. Crucially, $x_0$ is computed from the previous frame using the student's own noise prediction, not from an independent random initialization. This means the noise-injected frame at position $t$ is derived from the clean preceding frame at position $t-1$, preserving the temporal dependency that would exist during actual autoregressive inference.

Why "backward"? The name "backward noise initialization" comes from the direction of the operation: during standard diffusion generation, we go forward in denoising time (from noisy to clean). Here, we go backward—starting from a clean frame and adding noise according to the reverse of the denoising process. But this backward step uses the student's own learned noise prediction to determine what the pre-denoising state would have been, ensuring that the noise we add is consistent with the student's internal representation of the diffusion trajectory.

How this enables long-horizon training. With backward noise initialization, the DMD loss is always computed on noise-initialized states that are temporally consistent with the preceding video context. For any window extracted from the long rollout, the noise at the start of that window is derived from the clean frames that preceded it—just as it would be during inference. This eliminates the context misalignment that would otherwise make long-horizon DMD training incoherent. The student is evaluated (and trained) on exactly the kind of noise-initialized states it encounters during actual autoregressive generation.

Relationship to prior techniques. The paper explicitly acknowledges that "similar techniques of re-injecting noise have been employed in prior work [21, 67, 69]" but distinguishes the motivation:

"Whereas they used this for short-video distillation, primarily to enhance single-shot quality or circumvent the need for real training data, we leverage it as a mechanism to enforce temporal consistency across long videos."

In CausVid and Self-Forcing, noise re-injection is used on short (5-second) clips to improve the student-teacher distribution matching by providing more diverse training states. The paper's innovation is recognizing that the same technique, when applied to self-generated long rollouts, solves a fundamentally different problem—context alignment across extended temporal horizons.

What would go wrong without backward noise initialization? If the paper had instead initialized frames for DMD training from independent random noise (as is standard for short-horizon DMD), the teacher would be asked to denoise a state that bears no relationship to the preceding video context. The teacher might produce a high-quality 5-second clip in isolation, but that clip would have no temporal continuity with the frames that came before it. The student would learn to generate good isolated clips, not to maintain coherence over long sequences. The error accumulation problem would remain unsolved because the student would never practice recovering from states where errors have already accumulated.

Extended Distribution Matching Distillation (Extended DMD)

This is the second core innovation—the training procedure that actually teaches the student to maintain quality across long horizons. It operationalizes the paper's key insight: the teacher, despite only being able to generate 5-second videos itself, can evaluate and correct any 5-second window within a longer sequence because its training on a vast corpus has given it knowledge of what realistic video looks like at any temporal scale.

The core observation that motivates the approach. The paper states:

"although the bidirectional teacher model is trained exclusively on short, five-second clips, it implicitly captures the underlying data distribution of the 'world' from its training data. From this perspective, any short, contiguous video segment can be viewed as a sample from the marginal distribution of a valid, longer video sequence."

This is the key conceptual leap: the teacher's knowledge is not limited to videos that it can generate. A teacher trained on millions of diverse 5-second clips has seen enough of the visual world to recognize whether any 5-second window looks realistic, regardless of where that window falls within a longer video. This is analogous to how a human who has only ever seen 5-second movie trailers can still tell you whether a mid-movie 5-second clip looks well-shot and coherent—they don't need to have seen the whole movie to evaluate a snippet.

The extended DMD training procedure (Algorithm 1). The paper formalizes the training loop:

1. Long rollout generation. The student model $G_\theta$ is run autoregressively for $N$ frames (where $N \gg 5$ seconds, e.g., $N$ corresponding to 100 seconds) using the rolling KV cache: V ← Rollout(G_θ, N, L), where $L$ is the cache size (21 latent frames in the default configuration).

2. Random window sampling. A starting index $i$ is sampled uniformly from $\{1, \ldots, N-K+1\}$, and a contiguous window $W$ of length $K$ is extracted: W ← V[i : i+K-1], where $K$ is typically 5 seconds (the teacher's training horizon).

3. Diffusion timestep sampling. A denoising timestep $t$ is sampled from the set of discrete timesteps $\{t_1, \ldots, t_T\}$, where $T$ is the number of denoising steps used during training (the paper uses 4 denoising steps at training time with noise schedules of 1000, 750, 500, and 250).

4. Backward noise initialization. The window $W$ is noise-initialized according to Equation (1), producing a partially noised state $x_t(W)$ that preserves temporal context from preceding frames while being appropriately noisy for the sampled diffusion timestep.

5. DMD loss computation. The distributional divergence between the student and teacher is computed on the noise-initialized window: L_DMD ← DMD(G_θ(x_t(W), t), T_ϕ(x_t(W), t)). Both models process the same noised input; the teacher produces its distribution via multi-step denoising, the student produces its distribution via few-step generation, and the loss measures how different these distributions are.

6. Gradient update. The student parameters are updated: θ ← θ - η∇_θ L_DMD.

The mathematical form of the extended DMD loss (Equation 2). The paper writes:

$\nabla_\theta \mathcal{L}^{\text{extended}}_{\text{DMD}} = \mathbb{E}_t \mathbb{E}_z \left[ \nabla_\theta \text{KL} \left( p^S_{\theta,t}(z) \parallel p^T_t(z) \right) \right] \approx -\mathbb{E}_t \mathbb{E}_{i \sim \text{Unif}\{1,\ldots,N-K+1\}} \left[ \int \left( s_T \left( \Phi(G_\theta(z_i), t), t \right) - s^S_\theta \left( \Phi(G_\theta(z_i), t), t \right) \right) \frac{dG_\theta(z_i)}{d\theta} dz_i \right]$

where $G_\theta(z)$ is the student generator rollout given latent $z$, $p^S_{\theta,t}$ and $p^T_t$ represent the student and teacher distributions at diffusion time $t$, $s^S_\theta$ and $s^T$ are the corresponding score functions, $\Phi(\cdot, t)$ is the noise transformation at timestep $t$, and $i \sim \text{Unif}\{1, \ldots, N-K+1\}$ is the uniformly sampled window start index.

What it computes (operational English): The core operation is identical to standard DMD—measuring and minimizing the reverse KL divergence between student and teacher distributions at a given noise level—but with one critical difference: the expectation is now taken not just over diffusion timesteps and initial latents, but also over window positions within long self-generated rollouts. For each training iteration, a random 5-second segment is extracted from anywhere in the student's long video (which could be 20× longer than the teacher's horizon), noise is re-injected to a random timestep, and both student and teacher distributions are evaluated on that noise-initialized segment. Over many training iterations, the student sees windows from every possible temporal position—early in the video (where errors are minimal), mid-video (where errors have begun to accumulate), and late in the video (where errors are severe).

Why this form works where Self-Forcing fails. In Self-Forcing, DMD is computed only on the first $T$ frames (5 seconds). The student never sees what happens at frame 50 or frame 100 during training. At inference time, when errors have compounded by frame 50, the student encounters a KV cache state that is outside its training distribution—the cached keys and values represent a degraded video, not a clean one—and the student has no learned behavior for how to generate good frames from a degraded cache. Extended DMD solves this by explicitly training on windows from all temporal positions, including those where the cache contains accumulated errors. The teacher provides corrective signal: "given this degraded state (plus appropriate noise), here's how to denoise toward a realistic 5-second clip." Through repeated exposure, the student learns to recover—it learns to generate high-quality frames even when its KV cache contains suboptimal representations.

The sliding-window intuition. The paper includes a remark that captures the essence:

"Bi-directional diffusion can be seen as a process to gradually restore a degraded target in different denoising time-steps. Our method adapts the idea to autoregressive video generation regime by having a short-horizon teacher gradually restore student's degraded rollouts at different temporal time-frames and then distills these correction knowledge back into the student model."

The parallel is elegant: in standard diffusion, a single denoising trajectory restores a single instance from noise to data across denoising time. In extended DMD, the teacher restores the student's rollout across video time—each temporal window is a "degraded target" (containing accumulated autoregressive errors at that point in the sequence), and the teacher shows how to restore it. The student then internalizes this correction ability, learning to maintain quality across the entire temporal extent.

Training hyperparameters and configuration. The paper specifies (Implementation details, Section 8.1):

• Batch size: 8 training examples per iteration.
• Denoising steps: 4 discrete timesteps with noise schedules of 1000, 750, 500, and 250. These are the noise levels at which DMD is evaluated for each training window. Using 4 steps means the student is trained to generate in 4 denoising steps (a few-step generator), while the teacher still uses its full multi-step denoising for score computation.
• Generator learning rate: $2 \times 10^{-6}$.
• Critic learning rate: $4 \times 10^{-7}$ (the critic is a separate network that approximates the student's score function $s^S_\theta$, needed for the DMD gradient computation as shown in Equation 2).
• Generator-to-critic update ratio: 5:1 (the generator is updated 5 times for every 1 critic update).
• Optimizer: AdamW for both generator and critic, with $\beta_1 = 0$ and $\beta_2 = 0.999$.
• EMA (Exponential Moving Average): Applied to the student generator starting at epoch 200. The paper notes that the model "can also generate long high quality videos" without EMA, but "the EMA version performs better."
• KV cache window size: 21 latent frames in all cases except the ablation study (Section 3.2, Training with rolling KV Cache, and Section 4.3.1).

Design choices and their justifications:

• Uniform random window sampling rather than sequential or importance-weighted sampling: Uniform sampling ensures that every temporal position in the long rollout receives equal training attention. This is important because the nature of errors changes across the video—early windows contain minimal errors, while late windows contain severe errors—and the student needs practice at all levels. A sequential scheme (always training on the last window) would risk overfitting to severe-error states while neglecting the cleaner early states that are closer to the teacher's training distribution.

• Window size $K$ matching the teacher's training horizon: The 5-second window size is not arbitrary—it matches what the teacher was trained on and therefore what the teacher can most reliably evaluate. Using a shorter window (e.g., 2 seconds) would give the teacher less context for evaluating temporal coherence, while using a longer window (e.g., 10 seconds) would push the teacher outside its training distribution, potentially leading to unreliable score estimates.

• Four denoising steps rather than more steps: Fewer steps means faster training and faster inference. The trade-off is between generation quality and computational cost—prior work (Yin et al., 2024) has shown that 4-step DMD generators can achieve quality close to multi-step models while being much faster. The specific noise levels (1000, 750, 500, 250) represent a coarse discretization of the full noise schedule, covering the range from highly noisy (1000) to nearly clean (250).

• Beta parameters $\beta_1 = 0, \beta_2 = 0.999$: Setting $\beta_1 = 0$ in AdamW disables momentum (the first-moment estimate), which is an unusual choice. The standard Adam uses $\beta_1 = 0.9$. The paper adopts this from Self-Forcing's configuration. Disabling momentum can improve training stability when the loss landscape changes rapidly (as it does when training on self-generated data that evolves over training) because momentum would carry parameter updates in directions that may no longer be appropriate given the current data distribution.

• 5:1 generator-to-critic update ratio: The DMD loss requires a critic (an approximation of the student's own score function) to compute the gradient. The critic must be reasonably accurate for the gradient to be useful, so updating it frequently is important. However, training the critic is also expensive. The 5:1 ratio balances critic accuracy against computational efficiency.

Training data and the lack of real data requirement. A notable aspect of Self-Forcing++ is that it does not require real video data for training. The paper states:

"In the training phase, since we utilize backward noise initialization, we don't need real data for training. We utilize the same Wan2.1-T2V-1.3B as the teacher model."

This is possible because: (1) the student generates its own training data (self-rollouts), (2) the teacher provides supervision without accessing real data (using its learned score function), and (3) the backward noise initialization ensures the noise states are consistent with the autoregressive process. The only external input is text prompts (from the filtered and LLM-extended VidProM dataset, as used in Self-Forcing). This is a significant practical advantage—acquiring large datasets of long, high-quality videos is expensive and often legally constrained, but Self-Forcing++ sidesteps this entirely.

Remark on the training speed trade-off. The paper acknowledges in Section 6 (Limitations) that the method has "slower training speed compared to teacher-forcing." Teacher-forcing (the standard approach where the model is trained to predict the next frame given ground-truth previous frames) is fast because all frames can be processed in parallel during training—there's no autoregressive rollout needed. Self-Forcing++ must actually generate long rollouts autoregressively to create training data, which is inherently sequential and therefore slower. This is a real computational cost that the paper identifies as a target for future optimization ("we will explore parallelizing the training process").

Training with Rolling KV Cache: Eliminating Train-Inference Mismatch

The KV cache mechanism is central to autoregressive video generation with transformers, and how it's handled during training versus inference is a critical design decision that drove the gap between CausVid and Self-Forcing.

What a KV cache is and why it matters for video generation. In transformer architectures, each attention layer computes queries ($Q$), keys ($K$), and values ($V$) from the input tokens. During autoregressive generation, when producing token $t$, the model needs to attend to all previous tokens $1$ through $t-1$. Without a cache, the keys and values for all previous tokens would be recomputed from scratch at every step—an $O(t^2)$ cost. A KV cache stores the pre-computed keys and values for previously generated tokens, so that at step $t$, only the new token's query needs to be computed and dotted with the cached keys. This reduces the cost to $O(t)$ per step.

For video generation, each "token" is typically a latent frame (or a patch of a latent frame), and caching across frames enables efficient streaming generation where each new frame only needs to attend to stored representations of prior frames.

The CausVid approach and its over-exposure problem. CausVid introduced block-causal attention and KV caching for video generation but did not use the rolling cache during training. At inference time, CausVid recomputes overlapping frames—each new chunk of frames overlaps with previously generated frames, and these overlapping frames are recomputed (with slight variations due to the stochastic generation process) to maintain consistency. The paper identifies this as the source of over-exposure:

"CausVid still relies on recomputing overlapping frames and suffers from a severe over-exposure problem."

The recomputation means that certain frames are denoised multiple times with slightly different noise draws, and over time these differences accumulate as a systematic brightening of the video—hence "over-exposure."

The Self-Forcing approach and its residual mismatch. Self-Forcing improved on CausVid by incorporating the KV cache during training—the model learns to generate frames conditioned on its own cached representations, aligning training and inference more closely. However, Self-Forcing used a fixed cache during training: the cache stores a fixed number of clean frames (matching the 5-second training horizon), and the model generates within that fixed window. At inference, Self-Forcing uses a rolling cache: as new frames are generated, older frames are evicted from the cache to keep its size constant, while new frames are added. The paper identifies this discrepancy:

"Self-Forcing attempts to address this but introduces a train-inference mismatch by using a fixed cache during training and a rolling cache at inference. Although this is partially mitigated by masking the first latent frame, the mismatch still leads to substantial error accumulation and temporal flickering in long videos."

The mismatch is this: during training, every frame in a 5-second clip is generated with a cache that started from clean, teacher-generated context. During inference, frame 60 is generated with a cache that contains 20+ frames of student-generated (and potentially error-containing) context. The cache representations differ in distribution, and the model has never been trained to handle the inference-time version.

The Self-Forcing++ solution: rolling KV cache during both training and inference. The paper's approach is straightforward:

"In contrast, our method naturally eliminates this mismatch by employing a rolling KV cache during both training and inference. At training time, this cache is used to roll out sequences far beyond the teacher's supervisory horizon to compute the extended DMD as detailed above. Consequently, our approach greatly simplifies the entire process, requiring neither the recomputation of overlapping frames nor latent frame masking."

During training, the student model generates the full $N$-frame rollout using a rolling KV cache—just as it would at inference. As the rollout progresses, the cache is updated with newly generated frames and old ones are evicted. The extended DMD loss is then computed on random windows from this rollout. This means the student is trained on exactly the cache states it encounters at inference: frames 1-21 are in the cache when generating frame 22, frames 22-42 are in the cache when generating frame 43, and so on. The distribution of cache contents during training matches the distribution during inference by construction.

Cache size specification. The default KV cache window size is 21 latent frames (Section 8.1), which corresponds to approximately 5 seconds of video at the model's frame rate. This matches the teacher's training horizon—the cache always contains the most recent 5 seconds of generated video, providing the autoregressive context for generating the next frame. The ablation study (Section 4.3.1) tests smaller window sizes (15, 12, 9 latent frames) and finds that while smaller windows bring modest improvements in visual stability (from 40.12 to 52.50 at window size 9), they come at the cost of "increased inconsistency, since the model now relies on much less context compared to the original 21-frame history." The 21-frame default is thus a balance between visual stability (more context helps) and error recovery (less context forces the model to be more robust to cache variations).

Why this solution is "simple." The paper emphasizes that the rolling KV cache during training simplifies the entire process—no overlapping frame recomputation (eliminating CausVid's over-exposure source), no latent frame masking (eliminating a Self-Forcing hack), no attention sink frames (distinguishing from LongLive's approach). The architectural changes from Self-Forcing are minimal: change the cache from fixed to rolling during training, and extend the rollout length beyond 5 seconds. The complexity is in the training procedure (extended DMD, backward noise initialization), not in the model architecture.

GRPO-Based Temporal Smoothness Enhancement (Optional)

The paper introduces Group Relative Policy Optimization (GRPO) as a complementary mechanism to address a remaining limitation: long-term memory degradation. Even with extended DMD training, videos can exhibit temporal inconsistencies—objects abruptly appearing or vanishing, or unnaturally rapid scene transitions—due to the rolling window's limited context.

What GRPO is and how it's adapted for video generation. GRPO is a reinforcement learning technique originally developed for language models (Shao et al., 2024). In the standard RL formulation for generative models, the model (policy) generates outputs, a reward function scores them, and the policy is updated to increase the probability of high-reward outputs. GRPO specifically addresses the high variance problem in policy gradient methods by computing advantages within groups of outputs: generate $G$ outputs for the same prompt, compute rewards for all of them, and use the relative ranking (z-score normalized rewards) as the advantage signal.

The paper adapts GRPO to the autoregressive video generation setting by defining:

$\mathcal{J}(\theta) = \mathbb{E}_{\{o_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}(\cdot|c)} \mathbb{E}_{a_{t,i} \sim \pi_{\theta_{\text{old}}}(\cdot|s_{t,i})} \left[ \frac{1}{G} \sum_{i=1}^G \frac{1}{T} \sum_{t=1}^T \min \left( \rho_{t,i} A_i, \text{clip}(\rho_{t,i}, 1-\epsilon, 1+\epsilon) A_i \right) \right]$

where $\pi_\theta$ is the student policy (the autoregressive generation process), $c$ is the text prompt conditioning, $\{o_i\}_{i=1}^G$ are $G$ generated videos (a group), $a_{t,i}$ is the action (noise prediction) at autoregressive step $t$ for video $i$, $s_{t,i}$ is the state (KV cache + current noise) at step $t$, $\rho_{t,i} = \pi_\theta(a_{t,i}|s_{t,i}) / \pi_{\theta_{\text{old}}}(a_{t,i}|s_{t,i})$ is the importance weight (probability ratio between current and old policy), $A_i$ is the advantage for video $i$, and $\epsilon$ is the clipping hyperparameter (standard in PPO-style objectives).

What it computes (operational English): For each text prompt, the model generates a group of $G$ videos. Each video's quality is scored by a reward function. The advantage $A_i = (r_i - \text{mean}(\{r_j\})) / \text{std}(\{r_j\})$ measures how much better (or worse) video $i$ is compared to the group average, in units of standard deviation. The policy gradient update then increases the probability of actions (noise predictions) that led to above-average videos and decreases the probability of actions that led to below-average videos, with clipping to prevent overly large updates (the PPO-style clipped objective). The expectation over autoregressive steps $t$ means the update considers every step of the generation process, not just the final output.

How the generation probability is computed for GRPO (Equation 7). Since video generation involves sampling Gaussian noise predictions at each denoising step, the log-probability of a generated video is:

$\log p(x_{1:N}) = \sum_{n=1}^N \log p(x_n | x_{<n}) = \sum_{n=1}^N \sum_{t=1}^T \sum_{i=1}^D \left[ -\frac{\left(x^{(n)}_{t,i} - (1-\sigma_t) x^{(n)}_{0,i}\right)^2}{2\sigma_t^2} - \log \sigma_t - \frac{1}{2} \log(2\pi) \right]$

where $x_{1:N}$ is the full video of $N$ frames, $x_n$ is frame $n$, $x^{(n)}_{t,i}$ is the $i$-th dimension of the latent at denoising step $t$ for frame $n$, $x^{(n)}_{0,i}$ is the predicted clean latent (computed following Equation 1, conditioned on previously generated frames), $\sigma_t$ is the noise level at denoising step $t$, $D$ is the latent dimension size, $T$ is the number of non-terminal denoising steps, and $\log p(x_n|x_{<n})$ is the autoregressive conditional log-probability for generating frame $n$ given all previous frames.

What it computes (operational English): This is the log-likelihood of the generated video under the autoregressive Gaussian diffusion process. Each frame's generation involves $T$ denoising steps, where at each step the model predicts the clean latent (via Equation 1) given the noisy input and the KV cache of previous frames, then adds Gaussian noise with variance $\sigma_t^2$. The log-probability sums over all frames ($N$), all denoising steps per frame ($T$), and all latent dimensions ($D$), computing the Gaussian log-density at each point. The three terms are: (1) the squared error between the actual noisy sample and the predicted mean (scaled by $1/2\sigma_t^2$), (2) a normalization term $-\log \sigma_t$, and (3) the Gaussian normalization constant $-\frac{1}{2}\log(2\pi)$.

Why this form: The Eq. (1) predictor-corrector formulation ($x_0$ computed from the previous step's noise prediction) means the generation process is a sequence of Gaussian transitions, each with mean $(1-\sigma_t)x_0$ and variance $\sigma_t^2$. The log-probability under a Gaussian with these parameters has the standard form shown. This decomposition is necessary for GRPO because the importance weight $\rho_{t,i}$ requires evaluating $\pi_\theta(a|s)$—the policy's probability of taking a specific action (noise prediction) in a specific state—which is exactly what this log-probability computation provides.

The optical flow reward function. To guide GRPO toward temporally smooth outputs, the paper uses the relative magnitude of optical flow between consecutive frames as a reward proxy:

"To guide the optimization process towards temporally smooth outputs, we follow prior work [4, 42] and use the relative magnitude of optical flow between consecutive frames as a proxy for motion continuity."

Optical flow measures the apparent motion of pixels between consecutive frames—it captures how much each pixel moves from one frame to the next. Smooth, natural motion produces relatively stable optical flow magnitudes across frames. Abrupt transitions (objects suddenly appearing/disappearing, rapid scene cuts) produce spikes in optical flow magnitude because large portions of the frame change suddenly. By penalizing these spikes, GRPO encourages the model to generate videos with more gradual, natural motion transitions.

How GRPO is integrated into the training pipeline (Algorithm 1, line 9). The paper adds GRPO as a separate optimization step after the DMD training: R ← OpticalFlowReward(G_θ); θ ← GRPO_update(θ, R). This is a post-processing refinement rather than a replacement for DMD. The paper states:

"Our method can already generate consistent high quality long videos such as videos up to 4 minute 15 seconds before GRPO, in the ablation study, we show that it's possible to further boost the model's performance with properly designed rewards."

This means GRPO is not required for the core long-video capability—it enhances temporal smoothness beyond what extended DMD alone achieves.

Evidence for GRPO's effectiveness (Figure 5). The paper compares videos generated with and without GRPO by plotting optical flow magnitude over frame index. Without GRPO, the optical flow trace shows sharp spikes (e.g., at frame indices where abrupt scene transitions occur), and the variance over an 8-frame window is 24.52. With GRPO, the spikes are suppressed, and the variance drops to 20.82. The mean optical flow also decreases (from 6.89 to 2.00), indicating smoother motion overall. The paper attributes the spiking artifact to the rolling window mechanism:

"These transitions manifest as sharp spikes in the optical flow magnitude, an artifact that is exacerbated by the rolling window mechanism used during inference. By promoting smoother temporal transitions, our GRPO method effectively suppresses these spikes."

The interpretation is that the rolling KV cache creates a form of "attention window boundary effect"—when the cache slides to include a completely new set of frames, the model loses some long-range context that was maintaining smooth transitions, and this manifests as an abrupt change. GRPO trains the model to produce smoother transitions even when the cache composition changes significantly.

Relationship to concurrent work on RL for video generation. The paper cites several works that apply RL (specifically GRPO and DPO variants) to video diffusion models, including Flow-GRPO (Liu et al., 2025) and DanceGRPO (Xue et al., 2025). The paper's contribution is not the RL technique itself but its application to autoregressive video generation with a specifically designed optical-flow reward that targets the temporal smoothness failure mode characteristic of rolling-cache architectures.

New Metrics for Long Video Evaluation: The Visual Stability Protocol

The paper's experimental claims depend on demonstrating that Self-Forcing++ maintains quality at long horizons while baselines degrade. This requires an evaluation protocol that correctly measures the failure modes—over-exposure and error accumulation—that actually occur. The paper argues that existing benchmarks systematically fail at this task.

The VBench problem (Figure 3). VBench (Huang et al., 2024) is the standard benchmark for short-video generation quality, evaluating videos across 16 dimensions including image quality, aesthetic quality, temporal consistency, motion smoothness, and text alignment. The paper identifies a critical flaw when VBench is applied to long videos:

"We find, however, that outdated evaluation models make the benchmark favor over-exposed videos (e.g., CausVid) and degraded long videos (e.g., Self-Forcing), leading to inaccurate scores."

The problem is visualized in Figure 3: when VBench's image quality and aesthetic quality metrics are evaluated on early (clean) vs. late (degraded) frames of the same video, the scores paradoxically increase for degraded and over-exposed frames. For example, image quality jumps from 59.05 (regular) to 64.01 (degraded) and 67.29 (over-exposed). Aesthetic quality shows a similar pattern: 43.03 (regular) → 50.88 (degraded) → 53.64 (over-exposed).

Why VBench fails. The paper explains that the evaluation models underlying VBench are "outdated"—they were trained on primarily clean, high-quality images and videos, and they associate certain visual characteristics of degradation (high brightness, washed-out colors) with stylistic choices rather than with quality failures. Specifically, over-exposed frames (like those produced by CausVid) appear "brighter" and "more vibrant" to a model that hasn't been trained to recognize over-exposure as a defect. Similarly, darkened frames (like those from Self-Forcing's error accumulation) may be interpreted as "moody" or "dramatic" lighting rather than as degradation. This is a classic out-of-distribution evaluation problem: the metric models have never seen systematically degraded long videos during their training, so they don't know that these visual characteristics are undesirable.

The Visual Stability protocol. The paper proposes using Gemini-2.5-Pro (Comanici et al., 2025), a state-of-the-art video-capable multimodal large language model (MLLM), as a judge. The key advantage is that Gemini-2.5-Pro has "strong reasoning ability" (the paper cites its performance on Chatbot Arena and VideoReasonBench) and can be prompted with explicit definitions of the failure modes to evaluate. The protocol:

1. Define key failure modes explicitly in the prompt. The Gemini evaluator is given a detailed rubric that defines exposure quality on a 0-5 scale, from "Catastrophic Exposure" (0: nearly entire frame blown out or crushed, scene unreadable) to "Well-Exposed" (5: balanced lighting, no distracting artifacts, highlights and shadows retain detail). Importantly, the prompt instructs Gemini not to attribute exposure issues to artistic style unless the generation prompt explicitly calls for it.

2. Rate videos along these axes. Gemini provides both a numeric rating (0-5) and a free-text reasoning explanation for each video, making the evaluation interpretable.

3. Aggregate onto a 0-100 scale. The paper maps the 0-5 scale onto 0-100 (presumably by multiplying by 20) to produce the "visual stability" metric for consistent comparison with VBench's 0-100 scales.

4. Validate against human judgment. The paper reports a manual verification study: 20 randomly sampled MovieGen videos were independently annotated by two authors, and their averaged scores were compared with Gemini-2.5-Pro. Spearman's rank correlation reached 100% for the top three methods and 94.2% across all six baselines for 50-second videos. Similar results were observed for 75-second and 100-second videos.

The prompt structure. The full evaluation prompt (Section 8.5) defines a clear 6-point scale with explicit criteria for each level. For example:

• Level 2: "Noticeable Exposure Problems. Persistent clipping is present in highlights or shadows. Significant areas lose detail, though the frame remains viewable."
• Level 4: "Minor Exposure Flaws. Small regions are occasionally too bright or too dark, but these do not meaningfully disrupt overall visibility."

The prompt also includes a crucial instruction:

"Do not claim that the observations in any video are of a specific artistic style or scene transitions unless the prompt explicitly states so."

This prevents Gemini from rationalizing exposure failures as intentional artistic choices, which is exactly the failure mode the paper identifies in VBench's evaluation models.

What the Gemini evaluations reveal (Figures 8, 9). The paper provides representative Gemini evaluations that illustrate the protocol's diagnostic power. For a CausVid video of an astronaut on the moon, Gemini rates it 2/5: "The video consistently displays significant exposure issues, primarily with high contrast that leads to a loss of detail... These exposure issues are not fleeting; they are constant throughout the entire video." For the same prompt, Self-Forcing++ receives 5/5: "The video is well-exposed, demonstrating excellent handling of a high-contrast scene... Detail is well-preserved throughout the tonal range."

The evaluations capture not just the presence of exposure problems but their temporal persistence—a critical dimension for long videos. CausVid's over-exposure is described as "constant throughout the entire video," while MAGI-1's degradation is characterized as a temporal progression: "the video begins well-exposed for the first 8-10 seconds... after approximately the 15-second mark, the image quality degrades catastrophically."

Why this evaluation innovation matters beyond this paper. The paper's critique of VBench has implications for the entire long-video generation field. If the standard benchmark systematically underestimates the severity of degradation in long videos, then published results claiming good long-video performance may be over-optimistic. The Visual Stability protocol, while specific to exposure and degradation evaluation, demonstrates a general principle: for new failure modes that emerge at extended generation horizons, evaluation protocols must be explicitly designed to detect and quantify those failure modes. Simple extension of short-video metrics does not suffice.

Summary of Design Choices and Their Justifications

• Backward noise initialization over standard random noise: Ensures temporal consistency in the noise states used for DMD training, which is essential for the student to learn from its own error-accumulated rollouts. Standard random noise would produce contextually incoherent states that never occur during inference.

• Uniform random window sampling over sequential or importance-weighted schemes: Provides balanced training exposure across all temporal positions in the long rollout, from minimally-degraded early windows to severely-degraded late windows. This prevents overfitting to any particular error regime.

• Window size matching teacher's training horizon (5 seconds / 21 latent frames): Leverages the teacher's most reliable evaluation capability—the teacher was trained on exactly this temporal extent and can most accurately score it. Shorter windows lose temporal coherence context; longer windows push the teacher outside its training distribution.

• Rolling KV cache during both training and inference: Eliminates the residual training-inference mismatch that plagued Self-Forcing. The student learns to generate from exactly the cache states it encounters at deployment, including error-containing caches from earlier autoregressive steps.

• Four denoising steps (1000, 750, 500, 250 noise levels): Balances generation speed against quality. The DMD distillation framework has been validated at this step count in prior work; using more steps would improve quality marginally but increase training and inference cost proportionally.

• AdamW with $\beta_1 = 0$: Disables momentum to improve training stability under the rapidly changing data distribution that self-generated training data creates. Standard momentum could carry parameter updates in outdated directions.

• Optical flow reward for GRPO: Directly targets the temporal smoothness failure mode (abrupt scene transitions) that the rolling window mechanism can induce. The reward is computationally cheap (optical flow computation is fast compared to video generation) and provides a continuous, interpretable signal.

• Gemini-2.5-Pro as evaluator rather than traditional metrics: Addresses the systematic failure of standard benchmarks to correctly penalize over-exposure and error accumulation. The MLLM's reasoning capability enables it to follow detailed rubrics and distinguish between intentional style and quality degradation—a distinction that automated metrics trained on short, clean videos cannot make.

• No real video data requirement: The entire training pipeline uses only text prompts and the frozen teacher model. Student generates its own data (self-rollouts); teacher provides supervision (score function). This avoids the practical and legal challenges of acquiring large long-video datasets.

4. Key Insights and Innovations

Innovation 1: Error Accumulation in Autoregressive Video Generation Is a Correctable Training Deficiency, Not an Architectural Inevitability

The dominant assumption in the field prior to this work—implicit in the design of CausVid, Self-Forcing, and Diffusion Forcing approaches—was that the quality collapse of autoregressive video models at long horizons was largely an architectural or inference-time problem. CausVid attempted to solve it by recomputing overlapping frames at inference. Self-Forcing attempted to solve it by aligning training and inference cache distributions within the teacher's 5-second horizon. Diffusion Forcing attempted to solve it by fundamentally redesigning the noise schedule so that the model always conditions on partially-noised context. All these approaches shared an assumption: if you fix the training-inference mismatch at the scale the teacher operates at, the autoregressive mechanism should extend gracefully.

Self-Forcing++ makes a fundamentally different diagnostic claim: the collapse is caused by a supervision gap, not an architectural limitation. The student model has never been trained to recover from its own accumulated errors because the teacher only provides supervision on clean, short clips. During training, the student always starts from a clean context (teacher-generated or self-generated clean frames within 5 seconds). During inference, the student encounters contexts that contain its own previously generated errors—degraded KV cache states that lie far from its training distribution. It collapses not because autoregressive generation must collapse, but because it was never taught what to do when errors are already present.

This reframing is intellectually significant because it redirects the research program. If the problem is architectural, the solution space involves new noise schedules (Diffusion Forcing), new attention mechanisms (attention sinks), or new caching strategies. If the problem is a supervision gap, the solution is astonishingly simple: expose the student to its own error-accumulated states during training, and use the teacher to show it how to recover. This is a conceptual shift from "build a better architecture" to "teach the model to correct itself"—it moves the burden from structural innovation to training methodology.

The evidence for this reframing is the paper's training budget scaling experiment (Section 4.4, Figure 6). At 1× training budget, the model collapses beyond 5 seconds, similar to Self-Forcing. At 4×, it maintains coherence but with limited motion. At 8×, backgrounds and semantics improve but temporal quality still degrades. At 20×, it produces high-fidelity 50-second videos. At 25×, it generates 255-second videos with negligible quality loss. This monotonic scaling with training compute alone—no architectural changes, no long-video data, no new noise schedules—is strong evidence that the capability was always latent in the architecture and was unlocked by sufficient corrective training. If the problem were fundamentally architectural, additional training would hit a ceiling; instead, we see continued improvement.

This insight connects to a broader principle that has emerged in language model training (where exposure bias was long considered an unavoidable limitation of autoregressive decoding, later mitigated by techniques like scheduled sampling) and in imitation learning (where DAGGER showed that training on the learner's own error states—rather than only on expert demonstrations—is essential for robust policy learning). Self-Forcing++ can be understood as applying this principle to the video generation domain, with the teacher serving as the expert that labels the student's error states.

Innovation 2: A Short-Horizon Teacher Can Supervise Long-Horizon Generation Without Ever Seeing Long Videos—If You Frame the Problem as Error Correction Rather Than Generation

This is the paper's most counterintuitive conceptual move and its deepest departure from prior work. The standard approach to extending generation length—both in video and in language—has been to seek supervision at the target horizon: train on longer videos, use a long-video teacher, or build architectural mechanisms (like Diffusion Forcing's variable noise) that somehow encode the long-horizon structure into the model's inductive biases. Each of these approaches assumes that to generate long videos well, you need some form of long-video supervision—either data or architectural priors.

Self-Forcing++ rejects this assumption entirely. The paper's key observation is that a teacher trained on 5-second clips possesses corrective knowledge that generalizes across temporal scales, even though it possesses no generative capability beyond 5 seconds. The teacher cannot produce a 100-second video, but it can look at any 5-second window within a 100-second student-generated video and recognize (via its score function) whether that window is realistic and how to denoise it toward realism. The student's role shifts from "generate a long video" to "generate a long video and then correct any segment the teacher flags as unrealistic"—and through iterative training, the generation and correction processes merge into a single capability.

This is not merely a technical trick; it is a fundamental insight about the nature of distributional knowledge in generative models. The teacher's score function $s_T$ encodes the gradient of the log-density of realistic video—it points toward higher-probability regions of the data manifold. This gradient is defined everywhere in the space of video-like signals, not just near the teacher's own generated outputs. A degraded 50-second student rollout is a point in this space, and the teacher's score function still provides a valid corrective direction from that point. The fact that the teacher cannot sample from the long-video distribution (because it can only generate 5 seconds at a time) does not mean it lacks knowledge about that distribution—it knows what realistic video looks like at any timescale, even if it cannot produce it natively.

The evidence that this knowledge transfer actually works is in the paper's quantitative results. Tables 1 and 2 show that Self-Forcing++, which receives no long-video supervision whatsoever, dramatically outperforms methods that do have architectural mechanisms for long-horizon generation. At 100 seconds, Self-Forcing++ achieves a visual stability of 84.22, compared to 39.21 for CausVid, 32.03 for Self-Forcing, 39.38 for MAGI-1, and 56.72 for SkyReels-V2 (Table 2). These methods all have some form of long-horizon mechanism—CausVid's overlapping frames, MAGI-1 and SkyReels-V2's diffusion forcing—yet they are substantially worse than a method that simply teaches the student to correct itself using a short-horizon supervisor. This is a strong result: it suggests that architectural mechanisms for long-horizon generation may be less important than corrective training, at least up to the 100-second scale tested.

This insight reframes the relationship between teacher capability and student capability in distillation more broadly. The conventional wisdom is that the teacher's generative horizon bounds the student's—you cannot distill a 50-second capability from a 5-second teacher. Self-Forcing++ shows that this is true for generative capability (the teacher cannot produce long videos to serve as targets) but false for evaluative capability (the teacher can evaluate and correct any window). By shifting from "distill the teacher's generation" to "distill the teacher's correction ability," the student's horizon becomes decoupled from the teacher's. This is a genuinely new paradigm for teacher-student transfer in generative modeling.

Innovation 3: The Dual Training-Inference Mismatch as a Diagnostic Framework That Explains Prior Failures and Guides Intervention

The paper introduces a specific diagnostic framework that characterizes the failure of prior autoregressive video generation methods in terms of two distinct mismatches between training and inference conditions: (1) temporal mismatch—training on short clips (5 seconds) but running inference at much longer horizons—and (2) supervision mismatch—the teacher provides dense supervision during training (every frame in the short clip) but the student encounters sparse, error-corrupted contexts during long-horizon inference because the teacher never supervises the student on its own degraded states.

This framework is intellectually valuable because it explains why each prior method fails in precisely the way it does, and it predicts what a successful method must address:

• CausVid addresses neither mismatch. It trains on short clips with fixed attention, infers with rolling attention, and recomputes overlapping frames—introducing new artifacts (over-exposure) on top of the unresolved mismatches. The framework predicts that CausVid will show both systematic visual artifacts (from the architectural mismatch) and quality degradation at long horizons (from the supervision mismatch). The evidence confirms both: Figure 4 shows persistent over-exposure, and Tables 1-2 show visual stability dropping to 40.47 at 50 seconds and 39.21 at 100 seconds.

• Self-Forcing addresses the temporal mismatch partially (by incorporating the KV cache during training) but leaves the supervision mismatch entirely unresolved. The student is trained on clean, short self-generated rollouts where every frame receives teacher supervision. At inference, frames far from the training horizon are generated from error-accumulated cache states the student has never practiced recovering from. The framework predicts that Self-Forcing will maintain quality within 5 seconds but collapse rapidly beyond—exactly the behavior observed (visual stability 40.12 at 50 seconds despite being competitive at 5 seconds; Table 1).

• Diffusion Forcing methods (SkyReels-V2, MAGI-1) partially address the temporal mismatch through variable noise scheduling (frames at different noise levels can serve as "soft" long-term memory) but introduce training instability due to combinatorial noise schedule complexity. The framework predicts that these methods will be competitive at medium horizons but still degrade—the evidence in Tables 1-2 confirms this pattern, with visual stability at 55.47 (SkyReels, 75s) and 43.28 (MAGI-1, 75s), better than CausVid/Self-Forcing but substantially worse than Self-Forcing++.

• Self-Forcing++ is designed to address both mismatches simultaneously. The rolling KV cache during training eliminates the temporal mismatch. The extended DMD with backward noise initialization eliminates the supervision mismatch by explicitly training the student on its own error-accumulated states at all temporal positions. The framework predicts that this method should show minimal degradation as horizon increases—and the evidence confirms this, with visual stability remaining high at 84.22 even at 100 seconds, and training budget scaling showing continued improvement to 255 seconds (Figure 6).

The diagnostic power of this framework extends beyond this paper. Any future method for long-horizon autoregressive generation can be analyzed by asking: does it address the temporal mismatch? Does it address the supervision mismatch? If the answer to either is no, the framework predicts a specific failure mode. This transforms the problem from "long-horizon generation is hard" (a vague observation) to "long-horizon generation fails because of two specific, addressable mismatches" (an actionable diagnosis). This is a significant conceptual contribution—it gives the field a vocabulary and a causal model for reasoning about autoregressive generation quality that was previously absent.

Innovation 4: VBench Systematically Rewards the Failure Modes It Should Penalize—A Benchmark Critique With Broader Implications

The paper's critique of VBench for long-video evaluation (Section 3.4, Figure 3) is not merely a practical observation about a specific benchmark; it identifies a fundamental evaluation failure that likely affects the entire field's assessment of long-video generation quality. The finding is this: the evaluation models underlying VBench's image quality and aesthetic quality metrics were trained on clean, short videos and systematically misinterpret over-exposure and error accumulation as stylistic features rather than defects. Over-exposed frames receive higher image quality scores (67.29 vs. 59.05 for clean frames), and degraded frames receive higher aesthetic quality scores (50.88 vs. 43.03 for clean frames).

This is a severe problem because VBench is the most widely used benchmark for video generation quality. If published results on long-video generation have relied on VBench's image quality and aesthetic quality metrics—and the paper argues that many have—then the field's understanding of which methods produce good long videos may be systematically distorted. Methods that produce over-exposed outputs (CausVid) or degraded outputs (Self-Forcing) will receive inflated scores, making them appear more competitive than they actually are. The paper's own results illustrate this: in Table 1, framewise quality scores for 50-second videos show CausVid at 61.56 and Self-Forcing at 61.06—competitive with Self-Forcing++ at 60.82—yet the visual stability metric shows Self-Forcing++ at 90.94 versus 40.47 and 40.12 respectively. VBench's framewise quality metric is effectively useless for discriminating long-video quality because it cannot tell the difference between genuine quality and degradation that happens to trigger high activations in outdated evaluation models.

The proposed solution—using Gemini-2.5-Pro as a judge with explicit rubrics for failure modes—demonstrates a broader principle: for emerging capabilities that introduce new failure modes, evaluation protocols must be co-designed with the capability, not naively extended from shorter-horizon benchmarks. The paper validates this approach with human correlation studies (94.2-100% Spearman correlation), establishing that MLLM-based evaluation with detailed rubrics can reliably detect failure modes that automated metrics miss.

This innovation has implications beyond video generation. As generative models extend their capabilities to longer sequences, higher resolutions, and more complex outputs, the evaluation infrastructure must evolve to detect failure modes that don't exist in the short, clean, curated samples on which existing metrics were calibrated. The paper's critique of VBench is a concrete example of a more general phenomenon: evaluation debt—the accumulation of outdated metrics that systematically misreport performance on new capability frontiers because they were designed for a simpler problem setting. The Visual Stability protocol offers a template for addressing this debt: use capable MLLMs as judges, define failure modes explicitly, and validate against human judgments.

Innovation 5: Training Budget Scaling as a Viable Path to Long-Video Generation Without Architectural Innovation

The paper's training budget scaling experiment (Section 4.4, Figure 6) reveals a finding that, if it generalizes to other models and domains, has significant implications for how the field approaches long-horizon generation. The observed pattern is: monotonically increasing training compute, with no change to architecture, data, or teacher, produces qualitatively distinct improvements in video generation capability. At 1× budget, the model generates coherent 5-second videos but collapses beyond. At 4×, it maintains subject consistency over longer horizons but with limited motion. At 8×, backgrounds and semantics improve. At 20×, it produces stable, high-fidelity 50-second videos. At 25×, it generates 255-second videos with "negligible quality loss."

This is not an obvious or expected result. A plausible alternative hypothesis—and one that many in the field might have held—is that autoregressive video generation has a hard quality ceiling determined by the teacher's capability and the base model's capacity, and that additional training beyond some threshold would produce diminishing returns. The scaling experiment falsifies this hypothesis: the ceiling, if it exists, is far higher than the teacher's 5-second horizon would suggest, and the path to it is simply more corrective training.

This finding is significant because it suggests that long-video generation—at least up to the ~4-minute scale tested—may not require fundamentally new architectures, new data sources, or new training paradigms. It may be achievable by scaling the training compute of existing approaches, provided that the training procedure (like extended DMD) correctly exposes the model to its own error states. If this principle generalizes, it would shift resource allocation in the field away from architectural innovation (new attention mechanisms, new noise schedules, new model designs) and toward training infrastructure (faster autoregressive rollouts, parallelized self-supervision loops, larger compute budgets for corrective training).

The paper is appropriately cautious about this interpretation—it presents the scaling result as an empirical observation, not as a scaling law—but the analogy to language model scaling laws (where capabilities emerged from scaling compute on existing architectures) is hard to miss. The paper's observation that "scaling the training budget is a viable path toward high-quality, long-duration video synthesis, circumventing the reliance on large-scale real video datasets, which are notoriously difficult to acquire" positions this finding as both a practical insight (you don't need long-video data) and a methodological one (invest compute in corrective training, not in architectural complexity).

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary evaluation uses two prompt sets. For short-horizon (5-second) evaluation, the paper follows the standard VBench protocol (Huang et al., 2024): 946 prompts across 16 quality dimensions. For long-horizon evaluation (50/75/100 seconds), the paper uses 128 prompts from MovieGen (Polyak et al., 2024), the same set used in CausVid. Training data (for the initial ODE distillation and Self-Forcing++ training, though not for evaluation) uses a filtered and LLM-extended version of VidProM (Wang & Yang, 2024), as in Self-Forcing. The paper does not require real video data for its main training procedure—only text prompts and the frozen teacher model—since backward noise initialization enables fully self-supervised training on student-generated rollouts.

• Base model(s). All autoregressive methods use Wan2.1-T2V-1.3B (Team Wan et al., 2025) as the base architecture, distilled into a few-step generator and converted to autoregressive operation. The bidirectional teacher is the same Wan2.1-T2V-1.3B model in its original (undistilled) form, used frozen for score computation in DMD training. The choice of Wan2.1-T2V-1.3B is inherited from CausVid and Self-Forcing, enabling direct comparison with those baselines at identical model scale. Two additional bidirectional models—LTX-Video (1.9B parameters; HaCohen et al., 2024) and Wan2.1 (1.3B, the undistilled version; Team Wan et al., 2025)—are included in Table 1 for reference but cannot generate videos at the long horizons tested.

• Metrics. The paper uses multiple metric categories. VBench Long metrics (aggregated from the standard VBench protocol) include: text alignment (semantic consistency with the prompt), overall consistency, CLIP score, temporal quality (which the paper argues is inflated by motion stagnation), subject consistency, background consistency, motion smoothness, dynamic degree (a measure of motion magnitude—higher means more movement), framewise quality (image-level fidelity), aesthetic quality, and imaging quality. The proposed Visual Stability metric (Section 3.4) uses Gemini-2.5-Pro as a judge on a 0-5 exposure quality rubric (mapped to 0-100) and is the paper's preferred metric for long-horizon quality, validated against human annotations with Spearman correlation of 94.2-100%. For temporal repetition, the paper adopts the NoRepeat Score from RIFLEx (Zhao et al., 2025), which measures the tendency of videos to cycle with fixed, recurring patterns.

• Baselines. The paper evaluates against six autoregressive methods and two bidirectional references:

  • NOVA (0.6B parameters; Deng et al., 2025): an autoregressive model that formulates video generation as non-quantized frame-by-frame and spatial set-by-set prediction.
  • Pyramid Flow (2B parameters; Jin et al., 2025): a flow-matching model with hierarchical multi-stage pyramids, evaluated at 5 seconds only.
  • MAGI-1 (4.5B parameters; Teng et al., 2025): a Diffusion Forcing-based model that progressively denoises per-chunk noise autoregressively, distilled to 16 steps for the paper's experiments.
  • SkyReels-V2 (1.3B parameters; Chen et al., 2025): a Diffusion Forcing-based model supporting potentially infinite rollouts.
  • CausVid (1.3B parameters; Yin et al., 2025): the direct predecessor that uses block-causal attention, KV caching, and overlapping frame recomputation for autoregressive generation.
  • Self-Forcing (1.3B parameters; Huang et al., 2025): the immediate baseline that incorporates KV cache during training and uses DMD loss on self-generated rollouts to mitigate CausVid's over-exposure.
  • LTX-Video (1.9B parameters; HaCohen et al., 2024): a bidirectional real-time latent diffusion model, included for 5-second reference only.
  • Wan2.1 (1.3B parameters, undistilled bidirectional; Team Wan et al., 2025): the teacher model itself, included for 5-second reference only.

• Generation budget / compute accounting. The paper does not measure inference compute in FLOPs for the main comparisons. Instead, models are evaluated at fixed video durations (5s, 50s, 75s, 100s), and quality metrics are compared at each duration. Throughput is reported in FPS (frames per second) in Table 1 for architectural context: the 1.3B autoregressive methods (CausVid, Self-Forcing, Ours) achieve 17.0 FPS, substantially faster than the bidirectional Wan2.1 at 0.78 FPS and competitive with the smaller NOVA (0.6B at 0.88 FPS). For the training budget scaling experiment (Section 4.4), "training budget" refers to total training compute, measured in multiples of the compute required to achieve coherent 5-second generation (1× baseline). Budgets of 4×, 8×, 20×, and 25× are tested. The paper does not provide wall-clock training times or GPU-hour estimates for these budget multiples.

• Cross-validation / statistical protocol. The paper does not report cross-validation for the main evaluation results. The 128 MovieGen prompts are used consistently across all long-horizon evaluations (Tables 1, 2, 4). For the Visual Stability metric's human validation study, 20 randomly sampled MovieGen videos were independently annotated by two authors and averaged, with Spearman's rank correlation computed against Gemini-2.5-Pro scores. The paper reports 100% correlation for the top three methods and 94.2% across all six baselines for 50-second videos, with "similar results" for 75-second and 100-second videos. No confidence intervals, error bars, or statistical significance tests are reported for any of the main quantitative results.

Main Quantitative Results

The paper's results are organized along two primary axes: (1) short-horizon (5-second) quality, establishing that Self-Forcing++ does not regress on the task its baselines were designed for, and (2) long-horizon (50s, 75s, 100s) quality, where the method's advantages emerge decisively. A third axis—training budget scaling (Section 4.4)—examines how generation capability evolves with increased compute.

Short-Horizon (5-Second) Quality: Parity with Self-Forcing, Superiority over Other Autoregressive Methods

The 5-second results in Table 1 establish that Self-Forcing++ is competitive with the best prior autoregressive methods on the task they were explicitly designed for, despite not being specifically optimized for short clips. The headline numbers:

Self-Forcing++ achieves a total score of 83.11 and a semantic score of 80.37 on 5-second videos, compared to Self-Forcing's 83.00 and 80.14 respectively—a marginal improvement that is within the range of sampling variation but confirms no regression. Both methods substantially outperform the remaining autoregressive baselines: CausVid (82.46 total, 77.84 semantic), SkyReels-V2 (82.67 total, 74.53 semantic), Pyramid Flow (81.72 total, 69.62 semantic), NOVA (80.12 total, 79.05 semantic), and MAGI-1 (79.18 total, 67.74 semantic).

The bidirectional references (Wan2.1 at 84.67 total, LTX-Video at 80.00) provide context for the upper bound of short-horizon quality—the distilled autoregressive methods have not fully closed the gap to their teacher, but the gap is modest (1.56 points between Self-Forcing++ and Wan2.1 on total score).

The quality score column shows a notable pattern: Self-Forcing++ achieves 83.79 versus Self-Forcing's 83.71 (essentially identical), while CausVid scores 83.61. This confirms that the over-exposure problem in CausVid (which Self-Forcing fixed) is a distinct issue from the error accumulation problem (which Self-Forcing++ addresses)—over-exposure manifests even at 5 seconds, while error accumulation only becomes visible at longer horizons.

The text alignment scores reveal a consistent advantage for the Self-Forcing lineage over other autoregressive methods: Self-Forcing++ (80.37), Self-Forcing (80.14), and CausVid (77.84) all exceed NOVA (79.05), SkyReels-V2 (74.53), MAGI-1 (67.74), and Pyramid Flow (69.62). This suggests that the DMD-based distillation and KV-cache training used in the CausVid/Self-Forcing/Self-Forcing++ lineage provides better semantic grounding than the Diffusion Forcing or flow-matching alternatives at this model scale.

Long-Horizon (50-Second) Quality: Dramatic Separation from All Baselines

At 50 seconds (Table 1), the performance landscape bifurcates sharply:

Visual Stability: Self-Forcing++ achieves 90.94, compared to 40.47 (CausVid), 40.12 (Self-Forcing), 45.94 (NOVA), 51.25 (MAGI-1), and 60.41 (SkyReels-V2). The gap between Self-Forcing++ and the next-best autoregressive method (SkyReels-V2) is 30.53 points—a factor of 1.5×.

This metric is the paper's primary quality indicator for long videos, and the magnitude of separation leaves little ambiguity. The interpretation is that all baselines exhibit systematic visual degradation (over-exposure, darkening, or noise) that Gemini-2.5-Pro reliably detects, while Self-Forcing++ maintains stable exposure and visual quality throughout the 50-second duration.

Dynamic Degree: Self-Forcing++ achieves 55.36, compared to 37.35 (CausVid), 34.35 (Self-Forcing), 31.96 (NOVA), 28.49 (MAGI-1), and 39.15 (SkyReels-V2). This represents a 41.5% improvement over CausVid and a 61.2% improvement over Self-Forcing.

Dynamic degree measures motion magnitude—higher values indicate more movement. The baselines' low dynamic degree scores confirm the paper's qualitative claim that they "collapse into nearly static sequences" at long horizons. Self-Forcing++ not only maintains visual quality but sustains motion dynamics that baselines lose. The combination of high visual stability (frames look good) and high dynamic degree (frames show meaningful motion) is what distinguishes the method—it is possible to achieve high visual stability by generating static scenes (which would also have low dynamic degree), but Self-Forcing++ achieves both simultaneously.

Text Alignment: Self-Forcing++ achieves 26.37 at 50 seconds, compared to 25.25 (CausVid), 24.77 (Self-Forcing), 24.58 (NOVA), 26.04 (MAGI-1), and 23.73 (SkyReels-V2). The improvement over Self-Forcing is 6.5%.

Text alignment degrades for all methods as horizon extends (compare 5-second scores in the 74-80 range to 50-second scores in the 22-26 range), indicating that maintaining prompt relevance over long durations is challenging regardless of method. Self-Forcing++'s advantage is modest but consistent—it maintains the best text alignment among autoregressive methods at all tested horizons.

Temporal Quality: Self-Forcing++ achieves 91.03, exceeding all baselines: 89.34 (CausVid), 88.17 (Self-Forcing), 86.53 (NOVA), 88.34 (MAGI-1), and 88.78 (SkyReels-V2).

The paper notes a crucial interpretive caveat: baseline temporal quality scores are inflated by motion stagnation. VBench's temporal quality metric evaluates short-term frame-to-frame consistency—if a video becomes static, frame-to-frame differences are minimal, and the metric scores this as "temporally consistent," even though the video has effectively died. Self-Forcing++ achieves higher temporal quality despite having more motion (higher dynamic degree), which is genuinely impressive—it means the model maintains smooth transitions even while sustaining meaningful movement.

Framewise Quality: The paper explicitly warns that this metric is "unreliable for long videos" and includes it "for reference." The values are: CausVid 61.56, Self-Forcing 61.06, Self-Forcing++ 60.82, SkyReels-V2 54.13, MAGI-1 54.20, NOVA 34.45. The similarity between Self-Forcing++ and its degraded baselines on this metric (despite the 50+ point gap in visual stability) validates the paper's critique that VBench's framewise quality models cannot distinguish genuine quality from degradation. This is itself an important negative result: if one evaluated these methods using only VBench's standard framewise quality metric, one would conclude that Self-Forcing++, CausVid, and Self-Forcing produce comparable long videos—a conclusion the visual stability metric and qualitative examples (Figure 4) emphatically contradict.

Long-Horizon (75-Second and 100-Second) Quality: The Gap Widens Further

Tables 2 and 4 extend the evaluation to 75 and 100 seconds. The key findings:

At 75 seconds (Table 2): Self-Forcing++ achieves Visual Stability 86.10 and Dynamic Degree 55.62. The gap to the next-best method (SkyReels-V2 at Visual Stability 55.47, Dynamic Degree 39.89) widens further—from 30.53 points at 50 seconds to 30.63 points for visual stability, and from 16.21 to 15.73 points for dynamic degree. Self-Forcing degrades to Visual Stability 35.00 and Dynamic Degree 29.15, confirming that its error accumulation worsens with horizon length. CausVid shows a similar pattern: Visual Stability drops from 40.47 (50s) to 39.84 (75s).

At 100 seconds (Table 2): Self-Forcing++ maintains Visual Stability 84.22 and Dynamic Degree 54.12—remarkably stable compared to the 75-second values (86.10 and 55.62, respectively; the slight decline is within sampling variation). In contrast, Self-Forcing collapses to Visual Stability 32.03 and Dynamic Degree 26.41—a 62.0% and 51.2% degradation relative to Self-Forcing++ respectively. CausVid reaches Visual Stability 39.21, MAGI-1 reaches 39.38, and SkyReels-V2 holds at 56.72—none within 25 points of Self-Forcing++.

The persistence of Self-Forcing++'s quality from 50s to 100s (visual stability: 90.94 → 86.10 → 84.22; dynamic degree: 55.36 → 55.62 → 54.12) is evidence that the method has fundamentally overcome the error accumulation that causes baselines to degrade with increasing horizon. The slight downward trend in visual stability (loss of ~6.7 points from 50s to 100s) suggests that degradation is not entirely eliminated but is drastically slowed—extrapolating linearly, it would take hundreds of seconds before visual stability approached baseline levels.

Text alignment at 100 seconds (Table 2): Self-Forcing++ achieves 26.04, compared to Self-Forcing's 22.00 (an 18.4% relative improvement). The absolute text alignment scores are low for all methods—no model maintains strong prompt relevance at 100 seconds—but Self-Forcing++ degrades more slowly than baselines. At 100 seconds, the ranking is Self-Forcing++ (26.04) > MAGI-1 (23.75) > CausVid (24.41) > NOVA (22.89) > SkyReels-V2 (22.05) > Self-Forcing (22.00).

Temporal Quality at 100 seconds (Table 2): Self-Forcing++ achieves 90.87, compared to 89.06 (CausVid), 87.39 (Self-Forcing), and 88.80 (SkyReels-V2). As with the 50-second results, these scores partially reflect the motion stagnation of baselines. Self-Forcing++'s ability to maintain high temporal quality while sustaining high dynamic degree (54.12 vs. baselines' 22-38 range) is the more informative comparison.

Full Dimensional Breakdown (Table 4)

The full VBench Long results in Table 4 provide granular insight into which quality dimensions degrade for baselines and which Self-Forcing++ preserves. At 100 seconds:

• Subject consistency: Self-Forcing++ 97.09, vs. 98.41 (CausVid), 97.39 (Self-Forcing), 98.35 (MAGI-1), 96.05 (SkyReels-V2). All methods score relatively high on this dimension, suggesting that object identity is maintained even in degraded videos—the failure mode is visual quality, not semantic coherence.

• Background consistency: Self-Forcing++ 95.53, vs. 97.46 (CausVid), 96.76 (Self-Forcing), 97.99 (MAGI-1), 96.52 (SkyReels-V2). Self-Forcing++ scores slightly lower than baselines on background consistency, which may reflect the higher dynamic degree—more motion inherently creates more background variation, which this metric may penalize.

• Motion smoothness: All methods score in the 98-99 range at 100 seconds (Self-Forcing++ 98.35, CausVid 98.54, Self-Forcing 98.52, MAGI-1 99.20, SkyReels-V2 98.86). These near-ceiling scores are consistent with the temporal quality findings—motion smoothness metrics are saturated and non-discriminative for long videos. Static or slow-moving videos score perfectly on smoothness because there is minimal frame-to-frame change to be non-smooth about.

• Aesthetic quality: Self-Forcing++ 53.00, vs. 57.22 (CausVid), 51.16 (Self-Forcing), 47.25 (MAGI-1), 46.33 (SkyReels-V2). The paper's critique of VBench's aesthetic quality model is validated here: CausVid's over-exposed frames receive the highest aesthetic scores (57.22), while Self-Forcing++'s properly exposed frames score lower (53.00). This is the "benchmark rewards failure modes" problem the paper identifies—CausVid's "brighter" over-exposed frames are interpreted as more aesthetically pleasing by a model that doesn't recognize over-exposure as a defect.

• Imaging quality: Self-Forcing++ 68.31, vs. 64.79 (CausVid), 65.35 (Self-Forcing), 54.55 (MAGI-1), 54.62 (SkyReels-V2). This is one of the few VBench dimensions where Self-Forcing++ leads, suggesting that its properly exposed frames do register as higher technical quality on at least this metric, even if the aesthetic quality model prefers over-exposure.

Temporal Repetition (Table 5)

The NoRepeat Score from RIFLEx provides an orthogonal quality dimension:

Self-Forcing++ achieves 98.44, versus 100.0 (Self-Forcing), 92.97 (CausVid), 95.31 (SkyReels-V2), 73.44 (MAGI-1), and 67.19 (NOVA). The RIFLEx reference score (best published result from that method, using a bidirectional model) is 89.0.

The interpretation: autoregressive methods that rely exclusively on KV caching (Self-Forcing, Self-Forcing++) resist temporal repetition almost perfectly because they generate each new frame conditioned on the previous ones without any mechanism that would induce cycling. Methods that use overlapping frame recomputation (CausVid, at 92.97) or variable noise schedules (SkyReels-V2 at 95.31, MAGI-1 at 73.44) are more susceptible to repetitive patterns, likely because their generation procedures create subtle periodicities in the denoising process.

Self-Forcing scores a perfect 100.0, slightly higher than Self-Forcing++'s 98.44. This 1.56-point difference is small and may reflect the trade-off between motion stability (Self-Forcing++ sustains more motion, creating more opportunities for near-repetition that the metric detects) and strict repetition avoidance (Self-Forcing tends toward stagnation, which is trivially non-repetitive because it's not repeating—it's stopped).

Training Budget Scaling (Section 4.4, Figure 6)

This experiment examines how video generation capability evolves as training compute increases, measured in multiples of the budget needed to achieve coherent 5-second generation:

1× budget (the baseline): Extending generation beyond 5 seconds "leads to significant temporal flickering and error accumulation, a failure mode similar to that of Self-Forcing." This confirms that the initial Self-Forcing++ training (at the same scale as Self-Forcing's training) does not yet solve the long-horizon problem—it establishes the corrective training framework but needs more compute to realize its benefits.

4× budget: The model "maintains semantic coherence over longer horizons, successfully rendering a consistent subject like the specified elephant." This is the first qualitative transition—from collapse to coherence. The paper does not provide quantitative metrics at this budget level, so the claim is supported by the qualitative example in Figure 6.

8× budget: The model "begins to generate detailed backgrounds and more semantically accurate subjects, although motion dynamics remain limited and temporal quality degradation persists." This is a partial improvement—visual fidelity improves but motion still suffers, suggesting that visual quality and motion quality may require different amounts of corrective training to stabilize.

20× budget: A "substantial improvement, producing high-fidelity videos that remain stable for over 50 seconds." This is the threshold at which Self-Forcing++ clearly exceeds all baselines at long horizons. The quantitative results in Tables 1-2 (50s, 75s, 100s) correspond to a model trained at approximately this budget level.

25× budget: The model "successfully generates a 255-second video with negligible quality loss." This is the paper's headline result—4 minutes and 15 seconds, 99.9% of the base model's positional embedding capacity, 50× longer than the Self-Forcing baseline—and it is achieved purely by scaling training compute, with no architectural changes, no long-video data, and no modification to the teacher.

The scaling pattern in Figure 6 shows a progression from incoherent ODE initialization (top row, barely recognizable content) → 1× (5-second coherence) → 4× (subject consistency) → 8× (background detail) → 20× (50-second stability) → 25× (255-second, high fidelity). The qualitative jump from 20× to 25× is visually striking: the 255-second video maintains consistent rendering of the elephant, savannah background, dust, and sunset lighting throughout the entire duration, with no visible degradation across the shown frames.

Ablation Studies and Robustness Checks

Attention window length (Table 3): The paper tests whether simply reducing the attention window during training (forcing the model to slide attention multiple times within the 5-second horizon) can approximate the benefits of extended DMD training. Using a 21-frame default window (matching the 5-second teacher horizon), the visual stability on 50-second videos is 40.12 (the Self-Forcing baseline). Reducing the window to 15 frames improves visual stability to 44.69, to 12 frames produces 42.19, and to 9 frames produces 52.50. The trend is non-monotonic (12 frames performs worse than 15), but the best reduced-window result (52.50 at 9 frames) still falls far short of Self-Forcing++'s 90.94. The paper notes that smaller windows come at the cost of "increased inconsistency, since the model now relies on much less context compared to the original 21-frame history." This ablation establishes that the extended DMD procedure provides benefits beyond what simpler attention-window manipulation can achieve. The qualitative results are visualized in Appendix Figure 7, showing that reduced-window models still exhibit degradation (darkening, loss of detail) at 50 seconds.

GRPO with optical-flow reward (Figure 5): The paper ablates the effect of the optional GRPO post-training step by comparing optical flow magnitude traces with and without GRPO. Without GRPO, the optical flow trace shows sharp spikes (indicating abrupt scene transitions), with a mean optical flow of 6.89 and variance (computed over 8-frame windows) of 24.52. With GRPO, the spikes are suppressed: mean optical flow drops to 2.00 and variance drops to 20.82. The paper explicitly notes that "our method can already generate consistent high quality long videos such as videos up to 4 minute 15 seconds before GRPO"—the GRPO step provides incremental improvement in temporal smoothness rather than being required for long-horizon capability. This is an important negative result in the positive sense: it shows that the extended DMD training alone is sufficient for the long-horizon scaling, and GRPO is an optional refinement for motion quality.

Noisy KV cache injection (Appendix Figure 7): As an additional ablation on error accumulation mitigation strategies, the paper tests manually injecting Gaussian noise into the KV cache (keys and queries) to simulate the effect of accumulated errors. This "yields a slight improvement in both image quality and visual stability compared to the original Self-Forcing" but "nonetheless fails to prevent substantial degradation in long-horizon video generation." The qualitative visualization in Figure 7 confirms this: the noisy-KV approach still produces degraded frames at 50 seconds. This ablation supports the paper's claim that the specific mechanism of extended DMD—training on actual self-generated error states with teacher corrective supervision—is necessary; simpler noise-injection schemes that approximate error accumulation do not suffice.

Diffusion Forcing comparison (Tables 1, 2, Appendix Figure 7): While not structured as a formal ablation, the paper provides evidence against the necessity of variable-noise-context approaches (Diffusion Forcing) for long-horizon generation. The authors note in Section 8.6:

"our work [shows in] tables 1, 2 and 4 that a context with variable noises is not absolutely required to achieve long horizon generation with little quality degradation."

SkyReels-V2 and MAGI-1 both use Diffusion Forcing (variable noise levels across frames), yet they substantially underperform Self-Forcing++ on visual stability at all long horizons. This is observational rather than controlled (the methods differ in more than just the noise schedule), but it challenges the premise that Diffusion Forcing's long-term memory advantages are necessary for extended generation.

EMA vs. no-EMA: The paper mentions (Implementation details, Section 8.1) that an Exponential Moving Average (EMA) of the student generator weights, starting at epoch 200, is used in the main results. The authors inspected the version without EMA and found it "can also generate long high quality videos but the EMA version performs better." No quantitative comparison is provided, so the magnitude of EMA's contribution is unknown, but the qualitative claim suggests EMA is helpful but not critical.

Critical Assessment

Claim 1: Self-Forcing++ extends high-quality video generation by 20× beyond the teacher's 5-second capability, producing 100-second videos that dramatically outperform baselines.

What the experiments demonstrate: The quantitative evidence strongly supports this claim within the tested regime. At 100 seconds (Table 2), Self-Forcing++ achieves Visual Stability 84.22 and Dynamic Degree 54.12, while the best baseline (SkyReels-V2) achieves 56.72 and 38.75 respectively. The 27.5-point gap in visual stability and 15.4-point gap in dynamic degree at 100 seconds are substantial and consistent with the qualitative examples in Figure 4, where Self-Forcing++ maintains visual quality and motion while baselines exhibit over-exposure (CausVid), darkening (Self-Forcing), or noise collapse (MAGI-1).

What the experiments do not demonstrate: The evaluation uses 128 MovieGen prompts—a fixed set that is not characterized in terms of difficulty, content diversity, or motion complexity. It is possible that these prompts happen to favor the types of scenes (relatively slow, continuous motion) where autoregressive methods excel, and that the 20× claim would not hold for prompts requiring rapid scene changes, complex object interactions, or fine-grained temporal synchronization (e.g., sports footage, dialogue scenes, precise physical simulations). The paper does not stratify results by prompt type or report per-prompt variance, so we cannot assess whether the improvement is uniform or concentrated on a subset of easier prompts.

A genuine weakness: The 128-prompt MovieGen set is the same set used by CausVid—it is not independently constructed or validated for long-video evaluation. The paper does not provide a rationale for why 128 prompts is sufficient, nor does it report confidence intervals that would indicate whether the observed gaps are statistically reliable given the sample size. For a paper that introduces a new evaluation protocol (Visual Stability), the continued reliance on a small, inherited prompt set for the main results is a limitation.

Claim 2: Visual stability is maintained as horizon scales from 50s to 100s, unlike baselines which degrade progressively.

What the experiments demonstrate: The visual stability scores—90.94 (50s), 86.10 (75s), 84.22 (100s)—show a modest decline of 6.7 points over a 2× horizon increase, compared to baseline declines that are larger in absolute terms (SkyReels-V2: 60.41 → 55.47 → 56.72, a volatile pattern with net decline; CausVid: 40.47 → 39.84 → 39.21, a slow decline from a much lower baseline). Dynamic degree follows a similar pattern: 55.36 → 55.62 → 54.12, essentially flat. This supports the claim that Self-Forcing++ has fundamentally mitigated error accumulation.

A genuine weakness: The paper evaluates at only three long-horizon points (50s, 75s, 100s). This is a sparse sampling of the temporal axis—it does not reveal whether quality degrades smoothly or exhibits threshold effects (e.g., stable until 80 seconds, then rapid collapse). The training budget scaling experiment (Figure 6) provides qualitative evidence at 255 seconds, but no quantitative metrics are reported at that horizon, making it impossible to characterize the quality trajectory between 100 and 255 seconds. A more thorough analysis would plot visual stability and dynamic degree as continuous functions of video length, revealing the shape of the degradation curve (if any) rather than just three point estimates.

Claim 3: Training budget scaling produces monotonic improvements in generation length and quality, suggesting a "scaling law" for corrective autoregressive training.

What the experiments demonstrate: Figure 6 is a compelling qualitative demonstration that 25× training budget produces a 255-second video where 1× collapses at 5 seconds. The progression is visually clear and monotonic: more compute → longer coherent generation. This is an existence proof that compute scaling works for this method.

What the experiments do not demonstrate: The paper does not provide quantitative metrics at different budget levels—no visual stability scores, no dynamic degree measurements, no text alignment data. The claim of "monotonic improvement" is supported only by the visual examples in Figure 6 and the qualitative descriptions in the text. This is a significant gap: for a paper that argues its primary contribution is enabling long-horizon generation through corrective training, the scaling properties of that corrective training are described qualitatively rather than quantified. We do not know, for example, whether the improvement from 1× to 4× is larger or smaller than from 20× to 25×—i.e., whether there are diminishing returns and, if so, at what budget they set in.

A genuine weakness: The paper does not report what "1× training budget" corresponds to in concrete terms (GPU-hours, number of training iterations, wall-clock time). This makes the scaling claim non-reproducible—a reader cannot estimate what resources would be required to replicate the 25× result on their own hardware or with their own base model. The absence of quantitative scaling curves (visual stability vs. training budget, dynamic degree vs. training budget) is the single largest gap in the experimental evaluation.

Claim 4: The proposed Visual Stability metric correctly identifies degradation that VBench's standard metrics miss or reverse.

What the experiments demonstrate: Figure 3 provides a clear demonstration: degraded and over-exposed frames receive higher VBench image quality scores (64.01, 67.29) than clean frames (59.05); degraded and over-exposed frames receive higher aesthetic quality scores (50.88, 53.64) than clean frames (43.03). The human validation study (94.2-100% Spearman correlation for Visual Stability vs. human judgments) provides evidence that the proposed metric aligns with human perception. The Gemini-2.5-Pro evaluations (Figures 8, 9) provide interpretable, detailed reasoning that correctly identifies exposure failures and their temporal persistence.

What the experiments do not demonstrate: The human validation study uses only 20 videos, annotated by two of the paper's own authors (not independent raters). This is a very small sample for establishing metric reliability, and the use of authors as annotators introduces potential confirmation bias—they designed the metric and have expectations about which methods should score well. Independent raters, blinded to method identity, would provide stronger validation. Additionally, the paper reports only Spearman correlation (rank-order agreement), not absolute agreement metrics (e.g., intraclass correlation) that would indicate whether the metric and humans assign the same absolute scores, not just the same rankings. A metric could have perfect rank correlation while systematically over- or under-estimating absolute quality.

A genuine weakness: The paper introduces Visual Stability as a replacement for VBench's unreliable metrics but continues to report all VBench metrics alongside it (Tables 1, 2, 4). This creates an interpretive tension: the paper argues that VBench framewise quality and aesthetic quality are misleading for long videos, yet includes them in the main results tables without always flagging their unreliability in the table captions themselves. A cleaner approach would be to relegate VBench's unreliable dimensions to an appendix and present only the validated metrics in the main text.

Claim 5: Self-Forcing++ outperforms all autoregressive baselines on long-horizon generation.

What the experiments demonstrate: This is the most straightforward claim and is strongly supported. At 50s, 75s, and 100s, Self-Forcing++ achieves the highest scores on Visual Stability, Dynamic Degree, Text Alignment, and Temporal Quality among all autoregressive methods tested (Tables 1, 2). The margins are large enough (25-30 points on Visual Stability at 100s) that even with unquantified variance, the ordering is unlikely to be statistical noise.

A missing baseline: The paper does not compare against LongLive (Yang et al., 2025), which it acknowledges in Section 7 (Discussion) as a concurrent work that "also incorporate[s] DMD into long self-rolled sequences in a windowed fashion with clean context." LongLive is described as achieving "high-quality videos up to several minutes long" and represents the most directly comparable alternative approach. The paper argues its method is "simpler" (no attention sink frames), but without a quantitative comparison, the reader cannot assess whether this simplicity comes at a quality cost or whether Self-Forcing++ genuinely outperforms its closest competitor. Given that LongLive was available at the time of writing (the paper discusses it in detail), its absence from the experimental comparison is a notable omission.

Another missing baseline: The paper does not evaluate a version of Self-Forcing that simply generates longer sequences at inference time with a larger KV cache, without any extended DMD training. This would isolate the contribution of the backward noise initialization and extended DMD from the contribution of simply running the existing Self-Forcing model for more steps. The existing Self-Forcing baseline uses the model's standard inference procedure, but it's unclear whether the collapse at long horizons is due to the inference procedure or the model's lack of long-horizon training. A "Self-Forcing with larger cache" ablation would be informative.

Claim 6: The method generates videos up to 4 minutes and 15 seconds, 99.9% of positional embedding capacity.

What the experiments demonstrate: The paper states that the 25× budget model generates a 255-second video (Figure 6, rightmost column). The calculation of "99.9% of maximally supported length" is explained in the footnote: the base model supports 1024 latent frames, and with a trunk size of 3, the maximum length is 1023 frames, and 255 seconds at the model's frame rate corresponds to this. The visual example in Figure 6 shows the 255-second video with frames sampled throughout its duration, and they appear visually consistent.

What the experiments do not demonstrate: Only one example is shown for the 255-second generation (one prompt: "A massive elephant walks slowly across a sunlit savannah..."). This is a single sample, and it is a prompt that describes slow, continuous motion—exactly the type of content where autoregressive methods can maintain coherence by simply continuing the existing motion trajectory. We have no evidence that the 255-second capability generalizes to other prompts, motion types, or scene complexities. The paper's demo page may contain additional examples, but the paper itself provides only this single qualitative demonstration for its headline length claim. A quantitative evaluation at this extreme horizon (even on a small prompt set) would substantially strengthen the claim.

Overall Experimental Design Assessment

Strengths:

• The evaluation spans multiple horizons (5s, 50s, 75s, 100s), enabling comparison of how methods' relative performance changes with duration.
• The introduction of Visual Stability and its validation against human judgments, while limited in scale, addresses a genuine methodological problem in the field.
• The training budget scaling experiment provides a qualitatively compelling demonstration of the method's potential, even if it lacks quantitative rigor.
• The ablation on attention window length cleanly isolates the contribution of extended DMD from simpler training modifications.
• Full dimensional breakdowns (Table 4) provide transparency into which VBench dimensions are affected and which are saturated.

Weaknesses:

• Small prompt set (128) for long-horizon evaluation with no confidence intervals or statistical testing. The reliability of the reported score differences cannot be assessed.
• Single base model (Wan2.1-T2V-1.3B) tested. The paper argues this model is "representative" but provides no evidence that the method transfers to other architectures (e.g., Hunyuan Video, CogVideoX) or other teacher-student scales.
• The 255-second claim rests on a single qualitative example with no quantitative evaluation at that horizon. The paper's most attention-grabbing result is its least rigorously evaluated.
• Training budget scaling is described qualitatively rather than quantified. The absence of performance-vs-budget curves limits the practical utility of the scaling observation.
• Missing baselines: LongLive (the most directly comparable concurrent method) and a "Self-Forcing with larger cache" ablation.
• Human validation for Visual Stability uses only 20 videos and author-annotators, providing weak evidence for a metric the paper advocates as a community standard.
• No evaluation on video diversity within the long horizon—are generated 100-second videos meaningfully different from each other, or does the corrective training encourage convergence to a limited set of "safe" motion patterns? The paper does not report diversity metrics (e.g., FVD, IS, or per-prompt variance).

What would strengthen the paper:

1. Scaling curves with quantitative metrics: Plot visual stability and dynamic degree as functions of training budget (1×, 2×, 4×, 8×, 16×, 25×) to characterize the scaling law. Plot visual stability as a function of video duration (0-100 seconds in 10-second increments) to characterize the degradation curve.

2. LongLive comparison: Include LongLive in Tables 1-2 to position Self-Forcing++ relative to its closest concurrent approach.

3. Multi-prompt evaluation at 255 seconds: Even 10-20 diverse prompts at the 255-second horizon, with Visual Stability and Dynamic Degree scores, would transform the headline claim from an anecdote to a result.

4. Independent human evaluation: A larger-scale human study with non-author raters, blinded to method identity, evaluating both exposure quality and overall preference, would strengthen the Visual Stability validation.

5. Confidence intervals on main results: Report standard errors or bootstrapped confidence intervals on the Visual Stability and Dynamic Degree scores to indicate whether the reported gaps are statistically distinguishable given the 128-prompt sample.

6. Base model transfer: Demonstrate that extended DMD training works with a different teacher-student pair (e.g., using Hunyuan Video as the teacher) to show the method is not specific to Wan2.1's architecture or training data.

6. Limitations and Trade-offs

The 255-Second Result Rests on a Single Qualitative Example

The assumption or constraint. The paper's most attention-grabbing claim—generating videos up to 4 minutes and 15 seconds (255 seconds), a 50× improvement over baseline—is supported by a single qualitative example in Figure 6. The paper shows frames from one video generated for one prompt ("A massive elephant walks slowly across a sunlit savannah, dust rising around its feet, the warm glow of sunset...") and states that this 25× training budget model "successfully generates a 255-second video with negligible quality loss" (Section 4.4). No quantitative metrics—Visual Stability, Dynamic Degree, Text Alignment, or any VBench dimension—are reported at the 255-second horizon. No other prompts are evaluated at this extreme duration.

The consequence. The paper's headline result is anecdotal rather than empirical. A single demonstration on a single prompt cannot establish that the 255-second capability generalizes. The chosen prompt ("elephant walks slowly across a sunlit savannah") is structurally favorable to autoregressive methods: it describes slow, continuous motion in an open outdoor scene with predictable temporal dynamics. We cannot know whether Self-Forcing++ would maintain quality at 255 seconds for prompts requiring rapid action, complex object interactions, scene cuts, camera movement, fine-grained synchronization (e.g., "a chef flips a pancake, catches it, and plates it with strawberries"), or indoor scenes with detailed background elements that degrade differently than open landscapes. The 50× improvement claim—while technically true for the demonstrated example—cannot be interpreted as a general capability claim without multi-prompt evaluation.

What evidence exists in the paper. Figure 6 (rightmost column) and the surrounding text in Section 4.4 provide the only evidence. The paper's demo page (https://self-forcing-plus-plus.github.io/) may contain additional 255-second examples, but these are not analyzed in the paper itself. The quantitative evaluation in Tables 1, 2, and 4 extends only to 100 seconds. The gap between 100 seconds (where the paper provides rigorous multi-prompt evaluation) and 255 seconds (where it provides one qualitative example) is 155 seconds—longer than the entire evaluated range. The 25× training budget model is not quantitatively benchmarked at any horizon.

The paper's own description of the training budget scaling progression provides indirect evidence of a potential ceiling: at 8× budget, "motion dynamics remain limited and temporal quality degradation persists" (Section 4.4). The jump from 8× (degraded motion) to 25× ("negligible quality loss") covers a 3.125× increase in training compute, but we do not know whether this improvement is specific to the elephant prompt's motion pattern or whether it generalizes. The 20× budget is described as producing "high-fidelity videos that remain stable for over 50 seconds"—this suggests that even at 20×, the model may not achieve the 255-second stability claimed at 25×, but no quantitative comparison between 20× and 25× is provided.

Mitigation status. The paper does not address this limitation directly. It does not frame the 255-second result as a preliminary demonstration requiring further validation. The abstract and introduction present the 4-minute-15-second generation as an established capability: "our method is capable of generating videos up to 4 minutes and 15 seconds, utilizing 99.9% of the base model's positional embedding capacity and representing a 50× improvement over the baseline" (Section 1). The title includes "Towards Minute-Scale High-Quality Video Generation," which appropriately hedges with "Towards," but the body of the paper does not maintain this caution. The limitation is implicitly acknowledged in the fact that the paper only quantitatively evaluates up to 100 seconds, but this discrepancy is never discussed. A multi-prompt evaluation at 255 seconds—even on 10-20 diverse prompts with Visual Stability scoring—would substantially strengthen the claim. Until such evaluation exists, the 255-second result should be understood as an existence proof (the method can reach this length for at least one prompt) rather than a demonstrated capability (the method reliably reaches this length across prompts).

---

Training Budget Scaling Is Described Qualitatively Rather Than Quantified

The assumption or constraint. The training budget scaling experiment (Section 4.4, Figure 6) reports improvements in generation capability as a function of training compute but provides no quantitative metrics at any budget level beyond the baseline comparison. The paper defines budget in abstract "multiples" (1×, 4×, 8×, 20×, 25×) where 1× is defined as "the training required to produce a coherent 5-second video" (Section 4.4) but does not report what this corresponds to in concrete terms: GPU-hours, number of training iterations, wall-clock time, total FLOPs, or number of self-rollouts generated. The paper describes the progression in qualitative terms: 1× shows "significant temporal flickering," 4× "maintains semantic coherence," 8× generates "detailed backgrounds and more semantically accurate subjects, although motion dynamics remain limited," 20× produces "high-fidelity videos that remain stable for over 50 seconds," and 25× generates a 255-second video "with negligible quality loss" (Section 4.4).

The consequence. The scaling observation—arguably the paper's most important finding because it suggests a path to arbitrarily long video generation without architectural innovation—cannot be reproduced, cost-estimated, or extrapolated by other researchers. A practitioner reading this paper cannot determine whether the 25× budget represents weeks of training on a single GPU or months on a large cluster. They cannot assess whether the scaling trend shows diminishing returns (e.g., the jump from 20× to 25× providing less improvement than the jump from 4× to 8×) or whether the relationship between budget and horizon length is approximately linear, sub-linear, or super-linear.

More fundamentally, the absence of quantitative scaling curves prevents the paper from establishing what kind of scaling law (if any) governs corrective autoregressive training. The paper draws an implicit analogy to language model scaling laws (where performance improves predictably with compute), but a scaling law requires quantitative characterization: performance as a function of compute, with measurements at multiple points that reveal the functional form. The paper provides five qualitative descriptions at five budget levels for a single prompt—insufficient to establish any functional relationship, let alone to predict what budget would be required for, say, 10-minute generation.

What evidence exists in the paper. Figure 6 provides visual evidence at five budget levels for one prompt, plus the ODE initialization baseline. The qualitative descriptions in the text (Section 4.4) summarize what is visible in these frames. The paper states that "at a 25× budget, the model successfully generates a 255-second video with negligible quality loss" and concludes that "scaling the training budget is a viable path toward high-quality, long-duration video synthesis, circumventing the reliance on large-scale real video datasets" (Section 4.4). No metrics—Visual Stability, Dynamic Degree, Text Alignment, FVD, or any other quantitative measure—are reported at any budget level for any prompt.

The main quantitative evaluation (Tables 1, 2, 4) corresponds to a model trained at approximately 20× budget (this is inferred from the text: "20× budget... produces high-fidelity videos that remain stable for over 50 seconds" and the main results evaluate at 50-100 seconds), but the paper does not explicitly state which budget level the main evaluation uses. This ambiguity means the reader cannot map the quantitative results to the scaling experiment: does the Visual Stability of 84.22 at 100 seconds (Table 2) correspond to the 20× or 25× model? If it is the 20× model, how much does the 25× model improve these metrics? The paper does not say.

The paper also does not report training time. The limitation section (Section 6) notes that the method has "slower training speed compared to teacher-forcing" and identifies this as a target for future work ("we will explore parallelizing the training process"), but provides no measurements of how much slower—no wall-clock time, no iteration count, no comparison to the training time of CausVid or Self-Forcing at comparable quality levels. A practitioner cannot weigh the cost of Self-Forcing++ training against the benefit of longer generation without this information.

Mitigation status. The paper identifies the high training cost as a limitation in Section 6: "Key drawbacks include slower training speed compared to teacher-forcing." The proposed future work includes "parallelizing the training process" to address this. However, the paper does not present the scaling experiment as a limitation—it presents it as a positive finding ("Training Budget Scaling") without quantifying the costs. The qualitative descriptions of the scaling behavior are treated as sufficient evidence for the claim that "scaling the training budget is a viable path." This is a missed opportunity: a paper that introduces a new training methodology with scaling properties should characterize those properties quantitatively, not anecdotally. The field learned this lesson from the language model scaling laws literature (Hoffmann et al., 2022; Kaplan et al., 2020), where precise measurement of compute-performance relationships enabled informed resource allocation decisions. Self-Forcing++ provides suggestive qualitative evidence of a scaling phenomenon but none of the quantitative characterization that would make it actionable.

---

The Method Is Validated on a Single Base Model Architecture; Transfer to Other Teacher-Student Pairs Is Unaddressed

The assumption or constraint. All experiments use Wan2.1-T2V-1.3B (Team Wan et al., 2025) as both the bidirectional teacher (frozen) and the base architecture for the autoregressive student (distilled). The paper does not test Self-Forcing++ with any other teacher model (e.g., Hunyuan Video, CogVideoX, Sora-distilled variants), any other base architecture, or any other model scale. The 1.3B parameter size is fixed throughout. The choice is inherited from CausVid and Self-Forcing, both of which also used Wan2.1-T2V-1.3B.

The consequence. The paper's claims about the effectiveness of extended DMD and backward noise initialization cannot be assumed to transfer to other model families. Three specific transfer risks exist:

1. Teacher quality dependence. The extended DMD procedure relies on the teacher's score function providing accurate corrective gradients for error-accumulated student states. If the teacher is weaker (less capable of evaluating whether a degraded 5-second window is realistic), the corrective signal will be noisier, and the student may not learn to recover from errors effectively. Conversely, if the teacher is stronger (e.g., a larger Wan model or a more capable architecture), the corrective signal may be more precise, potentially improving the student's recovery ability or enabling longer horizons. The paper provides no evidence about how teacher quality affects the scaling behavior.

2. Architecture-specific error accumulation patterns. Different model architectures may exhibit different error accumulation modes. Wan2.1-T2V-1.3B may accumulate errors in a way that is particularly amenable to windowed DMD correction (e.g., errors may be local in time, affecting primarily the frames immediately following the error source, making 5-second windows sufficient for correction). A different architecture might accumulate errors that are more globally distributed in time (e.g., an error at frame 10 manifests primarily at frame 50), requiring longer correction windows or different corrective strategies. The paper's finding that 21-frame (5-second) windows suffice for Wan2.1-T2V-1.3B does not guarantee that the same window size would work for other architectures.

3. Model scale effects. The paper operates at 1.3B parameters. It is unknown whether the corrective training benefits scale with model size—do larger models require proportionally more corrective training, less (because they have stronger priors), or the same? Do larger models hit different error accumulation ceilings? The paper's training budget scaling experiment (Section 4.4) characterizes scaling along the training compute axis but not along the model size axis. A practitioner considering whether to apply Self-Forcing++ to a larger model (e.g., a 5B or 14B parameter DiT) has no evidence about what to expect.

What evidence exists in the paper. The paper provides no cross-architecture evaluation. All tables (1, 2, 4), all figures (3, 4, 5, 6), and all ablation results (Table 3, Appendix Figures 7, 8, 9) use the same Wan2.1-T2V-1.3B base. The Discussion (Section 7) mentions that concurrent works (Rolling Forcing, LongLive) are "able to generate high-quality videos up to several minutes long" but does not state what base models those methods use, so even an indirect cross-architecture comparison is unavailable. The paper does mention in Section 8.6 that "StreamDiT has opted to distill the model first to limit the number of combinations which reduces the training instability" in the context of Diffusion Forcing methods, but this is about a different method, not about Self-Forcing++ on a different architecture.

The limitations section (Section 6) does not mention the single-architecture constraint. The future work discussion includes "investigate techniques for controlling the fidelity of latent vectors" and "incorporate long-term memory mechanisms," but not testing on other architectures.

Mitigation status. Not addressed. The paper implicitly treats the Wan2.1-T2V-1.3B architecture as representative ("we adopt the same base model... as Causvid and Self-Forcing," Section 8.1), but representativeness is assumed, not demonstrated. The paper could have strengthened its claims by testing on at least one additional teacher-student pair—even a smaller or larger variant of the Wan family would provide some evidence of transfer. The absence of any cross-architecture evaluation means the method's generality is an open question, not an established property.

---

The Method Does Not Address Long-Term Memory; Objects That Leave the Frame Will Not Be Faithfully Reconstructed Upon Return

The assumption or constraint. The autoregressive generation process uses a rolling KV cache with a fixed window of 21 latent frames (approximately 5 seconds). This means the model can only attend to the most recent 5 seconds of video history when generating each new frame. Any visual information that left the frame more than 5 seconds ago—an object that moved off-screen, a background element that was occluded, a character that exited the scene—is simply not present in the cache and cannot directly influence generation. The model must rely on its implicit memory (encoded in its parameters from training) to maintain consistency with the earlier video content, but it has no architectural mechanism for doing so.

The consequence. Extended videos that involve occlusion and re-appearance—a person walking behind a building and emerging on the other side, a car passing behind a truck, a bird flying behind a tree—are likely to exhibit content inconsistency. When the occluded object reappears, the model will regenerate it from scratch based on the current context (the last 5 seconds of visible frames) and the text prompt, which may specify the object's existence but not its precise appearance, position, or motion trajectory from earlier in the video. The regenerated object may differ in color, size, shape, or motion characteristics from its pre-occlusion appearance. The paper does not evaluate on prompts that require this type of long-term temporal reasoning.

This limitation is not specific to Self-Forcing++—it is inherent to any autoregressive generation method with a finite context window. However, it becomes increasingly consequential as video length increases: in a 5-second video, an object is unlikely to be occluded for longer than the context window; in a 255-second video, extended occlusions are likely in any scene with complex spatial layout. The paper's headline 255-second result (Figure 6) shows an elephant walking across a savannah—a scene where the subject remains continuously visible and the background is mostly open sky and distant landscape, with no occluding elements. This is the easiest case for the limited context window. Scenes with foreground elements, indoor environments with walls and furniture, or multi-character interactions would stress the memory limitation much more severely.

What evidence exists in the paper. The paper explicitly acknowledges this limitation in Section 6: "a lack of long-term memory, which can cause content divergence in regions occluded for extended periods." The future work section (Section 6) mentions plans to "incorporate long-term memory mechanisms into our autoregressive framework, which we believe is crucial for achieving true long-range temporal coherence," citing prior work on memory mechanisms (Li et al., 2025; Liu et al., 2025). The limitation is also briefly discussed in Section 8.6: "Diffusion forcing works by keeping a large number of frames in the current stage and apply different noise level for different frames. Thus, it naturally comes with better long term memory."

However, the paper provides no experimental characterization of the memory limitation—no evaluation on prompts requiring occlusion-based reasoning, no measurement of content consistency after extended occlusions, and no ablation showing how quality degrades as the interval between an object's disappearance and reappearance increases. The quantitative evaluation (Tables 1, 2, 4) uses the MovieGen prompt set, which is not stratified by memory requirements. The paper does not report what fraction of the 128 evaluation prompts involve occlusion, object re-appearance, or other long-term dependencies.

The ablation on attention window length (Section 4.3.1, Table 3) provides indirect evidence about the role of context length, but it tests shorter windows (9, 12, 15 frames) rather than longer ones that might improve memory. The finding that 9-frame windows achieve Visual Stability of 52.50 versus 40.12 for 21-frame windows (but with "increased inconsistency") suggests that reducing context degrades quality, but this is about short-term consistency (within the 21-frame window), not about long-term memory beyond the window.

Mitigation status. The paper acknowledges the limitation and proposes future work on long-term memory mechanisms, citing Hunyuan-GameCraft (Li et al., 2025) and WorldWeaver (Liu et al., 2025) as relevant approaches. However, no mitigation is implemented or evaluated in the current paper. The method as presented has a hard architectural bound: any information that leaves the 21-frame (5-second) context window is permanently lost to the generation process. The proposed future work—incorporating memory mechanisms—would require architectural changes beyond the current framework. The paper does not speculate on whether the extended DMD training procedure would be compatible with such mechanisms or whether combining corrective training with long-term memory would produce synergistic or interfering effects.

A practitioner deploying Self-Forcing++ for long-video generation should consider this limitation carefully. For content where visual elements remain continuously visible or where precise consistency after occlusions is not critical (landscape fly-throughs, abstract visualizations, slow pans across static scenes), the limited context window may be acceptable. For narrative content, multi-character scenes, or any scenario involving object permanence, the method as currently described is likely to produce noticeable inconsistencies.

---

Training Data Efficiency Is Not Characterized; the Absence of Real Video Data Has Unknown Quality Consequences

The assumption or constraint. Self-Forcing++ does not use real video data during its main training phase. The paper states: "In the training phase, since we utilize backward noise initialization, we don't need real data for training" (Section 8.1). The only external inputs are text prompts (from the filtered and LLM-extended VidProM dataset) and the frozen teacher model. The student generates its own training data through self-rollout, and the teacher provides supervision through its score function. This is presented as a practical advantage: "circumventing the reliance on large-scale real video datasets, which are notoriously difficult to acquire" (Section 4.4).

The consequence. The paper provides no evidence about what is lost by training exclusively on self-generated data. The student's training distribution is determined entirely by (1) the teacher's score function, (2) the text prompts, and (3) the student's own generation biases. If the teacher's score function has blind spots—regions of video space where it cannot reliably distinguish realistic from unrealistic content—those blind spots will propagate into the student through the DMD loss. If the text prompt distribution does not cover certain visual concepts, motion patterns, or scene types, the student will never practice generating or correcting those. If the student's self-generated rollouts systematically avoid certain regions of video space (because those regions are difficult to reach through autoregressive sampling from the initialization), the corrective training will never address errors in those regions.

This creates a potential quality ceiling that differs from methods trained on real videos. A model trained on real long videos would, in principle, learn the distribution of realistic content at the target horizon directly. Self-Forcing++ learns to correct its own errors, which is a fundamentally different objective: it learns to make self-generated content look realistic to the teacher, not to match the distribution of real long videos. If self-generated content has systematic biases (e.g., motion patterns that are easier for the autoregressive mechanism to maintain, color palettes that are less likely to trigger over-exposure), the corrective training will reinforce those biases rather than correcting them toward real-video statistics.

What evidence exists in the paper. The paper provides no comparison between Self-Forcing++ and a hypothetical model trained with real long-video data. This is understandable—such data is difficult to acquire—but it means the paper cannot characterize the quality gap between its fully self-supervised approach and the upper bound that real data would provide. The paper also does not analyze the diversity of its generated videos: no FVD (Fréchet Video Distance) scores, no Inception Score equivalents, no per-prompt variance measurements. The reader cannot assess whether Self-Forcing++ produces diverse long videos or whether it converges to a limited set of "safe" generation patterns that the teacher consistently approves.

The paper reports strong quantitative results compared to baselines (Tables 1, 2, 4), but all baselines are also trained without real long-video data (they either use short-video data, as CausVid and Self-Forcing do, or use architectural mechanisms like Diffusion Forcing that do not require long-video training data). The comparison therefore establishes that Self-Forcing++ is the best among methods that do not use real long-video data, which is a different claim from "Self-Forcing++ achieves quality comparable to what real long-video training would enable." The paper does not address this distinction.

The paper also does not analyze the computational cost of generating self-training data. Each training iteration requires autoregressively rolling out the student for $N$ frames (where $N$, e.g., 100 seconds' worth of frames, far exceeds the teacher's horizon). This is computationally expensive—the student must generate long videos sequentially, which cannot be parallelized across frames—and this cost grows with the training horizon $N$. The paper's limitation section mentions "slower training speed compared to teacher-forcing" (Section 6) but does not quantify the cost of self-rollout generation relative to, say, loading a batch of real videos from disk. A practitioner needs to know whether the self-supervised approach's data efficiency advantages (no data collection/curation cost) outweigh its computational disadvantages (generating training data on-the-fly). Without this comparison, the practical case for the self-supervised approach is incomplete.

Mitigation status. The paper does not address the quality consequences of training without real data. It presents the absence of real data as a benefit, not a limitation. The future work section (Section 6) does not mention investigating whether incorporating real video data—even short clips—would improve quality or accelerate training. The paper also does not discuss whether the self-supervised training could be combined with real data (e.g., mixing self-generated rollouts with real short videos for DMD training) to get the benefits of both. This is a methodological gap: the paper asserts that self-supervised corrective training is sufficient for long-horizon generation, but it does not test whether it is optimal or characterize what the self-supervision ceiling is relative to real-data training.

---

The Evaluation Protocol Has Unresolved Tensions: VBench Is Criticized But Still Reported, and Sample Sizes Are Small Without Confidence Intervals

The assumption or constraint. The paper makes a forceful methodological argument that VBench's image quality and aesthetic quality metrics are unreliable for long-video evaluation—they systematically prefer degraded and over-exposed frames (Section 3.4, Figure 3). The paper introduces Visual Stability as a validated alternative, supported by human correlation studies. However, the paper continues to report all VBench metrics alongside Visual Stability in the main results tables (Tables 1, 2, 4), including the framewise quality and aesthetic quality scores that the paper itself argues are misleading. The paper flags framewise quality as "unreliable for long videos" in a footnote (Table 1) and discusses the issue in Section 3.4, but it does not consistently mark which VBench metrics readers should trust and which they should discount when interpreting the tables.

The consequence. A reader encountering Tables 1 and 2 without carefully reading Section 3.4 might interpret the VBench scores at face value—for example, noting that CausVid achieves higher aesthetic quality (57.22) than Self-Forcing++ (53.00) at 100 seconds and concluding that CausVid produces more aesthetically pleasing long videos. This conclusion would be wrong (the aesthetic quality model is mistaking over-exposure for aesthetic appeal), but the table does not prevent it. The paper's decision to include unreliable metrics in the main results creates a potential for misinterpretation that undermines its own methodological critique. If VBench's image quality and aesthetic quality metrics are truly unreliable for long videos, they should be relegated to an appendix with a clear warning, not given equal visual weight with validated metrics in the main comparison tables.

Additionally, the paper does not provide confidence intervals, standard errors, or any measure of statistical reliability for its quantitative results. The Visual Stability scores in Table 2—90.94 (50s), 86.10 (75s), 84.22 (100s)—are reported as point estimates from 128 prompts, with no indication of variance across prompts. The difference between Self-Forcing++ (84.22) and SkyReels-V2 (56.72) at 100 seconds is 27.5 points, which is large enough that statistical significance is plausible even with moderate variance, but the reader cannot verify this. The decline in Self-Forcing++'s visual stability from 50s to 100s (90.94 → 84.22, a loss of 6.72 points) could represent genuine degradation or could be within the range of sampling variation—the paper provides no way to distinguish. The Dynamic Degree scores follow a similar pattern: 55.36 → 55.62 → 54.12 across 50s, 75s, and 100s. The non-monotonicity (slight increase at 75s) hints at sampling variance, but without error estimates, the reader cannot assess whether this fluctuation is meaningful.

What evidence exists in the paper. The human validation study for Visual Stability (Section 8.5) uses 20 randomly sampled MovieGen videos, annotated by two authors, with Spearman correlation of 94.2-100% against Gemini-2.5-Pro. This is the only quantitative reliability evidence in the paper. The sample size (20 videos) is very small for validating a metric proposed as a community standard. The use of paper authors as annotators (rather than independent raters blinded to method identity) introduces potential confirmation bias: the authors designed the metric, know which methods are expected to perform well, and may unconsciously align their judgments with the metric's expected behavior. Independent raters—ideally from a crowdsourcing platform, with inter-rater reliability statistics—would provide stronger evidence.

The paper does not discuss why it chose 128 MovieGen prompts for the main evaluation. Is 128 sufficient to achieve stable estimates of Visual Stability and Dynamic Degree? How was this prompt set constructed, and what types of content does it cover? The paper inherits this prompt set from CausVid but does not characterize its content distribution—does it include diverse motion types (fast, slow, static)? Diverse scene types (indoor, outdoor, abstract)? Diverse object counts (single subject, multiple interacting subjects)? Without this characterization, the reader cannot assess whether the evaluation covers the range of scenarios that a deployed long-video generation system would encounter.

Mitigation status. The paper partially addresses the VBench problem by introducing Visual Stability and validating it against human judgments, but it does not fully resolve the tension in its own presentation. The paper could have: (1) moved unreliable VBench metrics to an appendix, (2) added a clear visual indicator (e.g., gray text, asterisks with warnings) in the main tables to distinguish validated from unreliable metrics, or (3) restructured the tables to prioritize Visual Stability and the reliable VBench dimensions while de-emphasizing the unreliable ones. The current presentation treats all metrics symmetrically in the table layout, relying on text discussions elsewhere to disclaim some of them—a disconnection between the paper's methodological argument and its presentation choices.

The paper does not acknowledge the small sample size or the absence of confidence intervals as limitations. The human validation study is presented as establishing Visual Stability's reliability without discussion of its limitations (small sample, author annotators, rank correlation only). The future work section does not mention plans for larger-scale validation or independent annotation studies. The paper's methodological contribution—critiquing VBench and proposing an alternative—would be stronger if it modeled the evaluation rigor it advocates.

7. Implications and Future Directions

How This Work Changes the Landscape

Self-Forcing++ repositions the long-video generation problem from an architectural challenge to a training methodology challenge. The dominant narrative in the field—implicit in the design of CausVid's overlapping frame recomputation, Diffusion Forcing's variable noise schedules, and Self-Forcing's cache-aligned training—was that autoregressive video generation hits a fundamental quality ceiling imposed by error accumulation, and that overcoming it requires either architectural mechanisms that provide long-term memory (attention sinks, variable noise contexts) or access to long-video training data. Self-Forcing++ falsifies this narrative at the scale tested: it achieves 84.22 Visual Stability and 54.12 Dynamic Degree at 100 seconds using the same base architecture as Self-Forcing (Wan2.1-T2V-1.3B), the same distillation framework (DMD), and the same teacher (a 5-second bidirectional model), with no architectural changes beyond the rolling KV cache during training and no long-video data whatsoever. The only new ingredient is corrective supervision on self-generated error states—a training procedure change, not a model design change.

This is a conceptual reframing of the field's bottleneck. Before this work, a researcher deciding where to invest effort for long-video generation would reasonably consider: (a) designing new attention mechanisms that preserve long-term context, (b) developing noise schedules that maintain partial future information (Diffusion Forcing), (c) acquiring or generating long-video training datasets, or (d) engineering inference-time tricks like overlapping frames or positional encoding adjustments (RIFLEx). After this work, the investment calculus shifts: the paper's evidence suggests that corrective training on self-generated rollouts—teaching the model to recover from its own errors by leveraging a short-horizon teacher's evaluative knowledge—may be sufficient to unlock horizons far beyond the teacher's generative capability, potentially rendering some architectural innovations unnecessary for the generation lengths tested. This doesn't invalidate architectural research (long-term memory remains unsolved; see below), but it redirects priority: a team with fixed resources might now invest more heavily in training infrastructure and corrective supervision pipelines, and less in novel attention mechanisms, because the training methodology alone has been shown to deliver a 50× horizon extension where architectural approaches delivered 2-4×.

The paper also resolves a specific tension in the prior literature. Self-Forcing was the state-of-the-art for short-horizon autoregressive video quality—it fixed CausVid's over-exposure and achieved competitive 5-second scores (Table 1: 83.00 total score, comparable to the bidirectional Wan2.1 at 84.67). But it failed catastrophically beyond ~10 seconds (Visual Stability 40.12 at 50 seconds, dynamic degree collapsing to 26.41 at 100 seconds). The field might have interpreted this as evidence that the KV-cache-based autoregressive approach has an inherent horizon limit—that error accumulation is an unavoidable consequence of the sequential generation paradigm, and that fundamentally different architectures (like Diffusion Forcing's partially-noised context) are necessary for long videos. Self-Forcing++ shows this interpretation was wrong: the same autoregressive architecture, with the same KV cache mechanism, can generate 100-second videos at high quality when trained with corrective supervision on error states. The failure was not architectural but pedagogical—the model was never taught to recover from errors because its training never exposed it to error-accumulated states. This is a specific, falsifiable claim that reinterprets prior negative results and provides a clean target for future work.

The paper's evaluation innovation—identifying that VBench systematically rewards the failure modes it should penalize (Figure 3: over-exposed frames score higher on image quality, degraded frames score higher on aesthetic quality)—has implications beyond this paper. It demonstrates a general evaluation debt problem: benchmarks designed for short, clean videos become systematically misleading when applied to longer videos that exhibit novel failure modes (over-exposure, error-accumulated darkening, motion stagnation). This finding should prompt the field to re-examine published long-video results that relied on VBench's image and aesthetic quality metrics. If the metrics are as unreliable as Figure 3 suggests, the reported performance of prior long-video methods may be substantially overestimated. The Visual Stability protocol—using a capable MLLM with explicit failure-mode rubrics, validated against human judgments—offers a template for addressing this debt, though the current validation (20 videos, author annotators) is preliminary.

More broadly, Self-Forcing++ introduces a self-supervised corrective training paradigm for autoregressive generation that may transfer to other sequential generation domains. The core insight—that a teacher trained on short sequences can evaluate and correct any window within a longer student-generated sequence, even though it cannot generate long sequences itself—is not video-specific. It applies to any domain where: (1) autoregressive generation suffers from error accumulation at long horizons, (2) a teacher model exists that was trained on short sequences but has learned a rich evaluative function, and (3) backward noise initialization (or an equivalent mechanism) can create temporally consistent training states from self-generated rollouts. Candidate domains include long-form text generation (where language models degrade on extended outputs), audio/music generation (where temporal coherence over minutes is challenging), and trajectory planning in robotics (where compounding errors cause plans to diverge from feasible paths). The paper does not test these transfers, but the framework is general enough to invite them.

Follow-Up Research This Work Enables

Quantitative scaling laws for corrective autoregressive training. The paper's training budget scaling experiment (Section 4.4, Figure 6) provides qualitative evidence that more corrective training produces longer, higher-quality videos, but it provides no quantitative characterization: no metrics at each budget level, no functional form relating compute to horizon length, and no indication of whether returns diminish, remain constant, or accelerate. A direct follow-up would measure Visual Stability, Dynamic Degree, and FVD as functions of training budget (1×, 2×, 4×, 8×, 16×, 25×) on a standardized prompt set, producing scaling curves analogous to the pretraining scaling laws (Hoffmann et al., 2022). Key questions: Does the relationship follow a power law (performance ∝ compute^α)? Does the scaling exponent α vary with video length—i.e., does it take disproportionately more compute to extend from 100s to 200s than from 5s to 50s? Is there a horizon ceiling where additional corrective training provides zero benefit regardless of budget, and if so, does that ceiling correspond to the base model's positional embedding capacity (1024 latent frames for Wan2.1-T2V-1.3B) or to some other architectural constraint? The paper's current evidence (one prompt at 25× budget producing 255 seconds) is insufficient to answer any of these. A rigorous scaling study would require training runs at multiple budget levels, evaluation at multiple horizons per budget, and enough prompts (50-100) for statistical reliability. This is expensive but essential: without it, the paper's central claim—that scaling training compute is a viable path to long-video generation—remains an anecdote rather than a law.

Stress-testing the corrective training paradigm with adversarial prompts. The paper's evaluation uses 128 MovieGen prompts, which are not characterized by content type, motion complexity, or memory requirements. The 255-second demonstration uses a prompt ("elephant walks slowly across a sunlit savannah") that is structurally favorable to autoregressive methods: slow, continuous motion, open background, no occlusions, no scene changes. A rigorous follow-up would construct a horizon stress-test benchmark—a set of prompts designed to probe specific failure modes at extended lengths. Categories might include: (a) occlusion-heavy scenes (objects repeatedly passing behind foreground elements), testing whether the rolling KV cache's limited context causes content inconsistency upon reappearance; (b) rapid motion scenes (sports, dance, action sequences), testing whether the corrective training can maintain temporal coherence when motion magnitude is high; (c) multi-character interaction scenes (dialogue, physical interaction), testing whether the model maintains consistent character identity and spatial relationships; (d) scene-transition prompts (explicit cuts or narrative shifts), testing whether the model can handle discontinuities without collapsing; and (e) fine-grained temporal synchronization (musical performance, precise physical interactions), testing whether the few-step generation and rolling window create timing inaccuracies. Evaluating Self-Forcing++ on such a benchmark would reveal which failure modes the corrective training truly solves (error accumulation, over-exposure) and which it does not (long-term memory, rapid motion coherence, precise timing). Negative results on specific categories would be as informative as positive ones, guiding where architectural innovation remains necessary.

Cross-architecture transfer of extended DMD. The paper validates Self-Forcing++ exclusively on Wan2.1-T2V-1.3B. A natural follow-up would test whether the extended DMD procedure transfers to other teacher-student pairs. Strong candidates include: (a) Hunyuan Video (Kong et al., 2024) as teacher, to test whether a different DiT architecture with different training data produces a similarly effective corrective signal; (b) CogVideoX (Yang et al., 2024) as teacher, to test transfer to an expert-transformer architecture; and (c) a larger Wan variant (e.g., 5B or 14B parameters) as teacher with the 1.3B as student, to test whether a stronger teacher provides better corrective gradients that accelerate training or enable longer horizons. Each transfer would measure: does the student converge to similar long-horizon quality? At the same training budget? With the same hyperparameters (window size, denoising steps), or do these need re-tuning per architecture? If extended DMD transfers robustly across architectures, it strengthens the claim that corrective self-supervision is a general principle. If it fails on certain architectures, the failure mode would be informative—perhaps some teacher score functions are less reliable on out-of-distribution student-generated states, or some architectures accumulate errors in ways that sliding-window DMD cannot correct.

Combining corrective training with explicit long-term memory mechanisms. The paper acknowledges (Section 6) that the rolling 21-frame KV cache imposes a hard ceiling on long-term memory: information that leaves the cache is permanently lost. This is not addressed by the corrective training—extended DMD teaches the student to recover from short-term error accumulation but does not provide any mechanism for recalling visual information from more than 5 seconds ago. A direct follow-up would integrate a memory module into the Self-Forcing++ architecture and measure whether it improves consistency on occlusion-heavy prompts. Candidate memory mechanisms include: (a) a separate long-term KV cache that stores compressed representations of distant frames (following the attention-sink approach of LongLive or StreamingLLM); (b) a retrieval-augmented generation approach where the model explicitly queries a stored history of frame embeddings when generating content that may require recall; and (c) a hierarchical architecture where a coarse temporal summary is maintained alongside the fine-grained rolling cache. The key experiment would compare Self-Forcing++ with and without the memory module on prompts where an object leaves the frame for varying durations (5s, 10s, 20s, 50s) and then reappears, measuring whether the reappearing object's appearance, position, and motion are consistent with its pre-occlusion state. A positive result would extend the method's applicability to narrative and multi-character content; a negative result would establish that corrective training and long-term memory address orthogonal failure modes, and that both are necessary for general long-video generation.

Self-Forcing++ as a data engine for long-video dataset creation. The paper demonstrates that Self-Forcing++ can generate minutes-long videos with sustained visual quality and motion—a capability that, if reliable, could be used to generate training data for the next generation of video models. A concrete follow-up would: (a) use a 25× budget Self-Forcing++ model to generate 1,000-10,000 long videos (1-4 minutes each) from diverse prompts; (b) use these as training data for a bidirectional DiT (e.g., fine-tuning Wan2.1 on the generated long videos); and (c) measure whether the bidirectional model trained on synthetic long videos can generate higher-quality long videos than either the original bidirectional model (which was limited to 5 seconds) or the Self-Forcing++ model that generated the training data. This would test a self-improvement loop: autoregressive model generates long data → bidirectional model trains on it → bidirectional model becomes a better teacher for the next round of autoregressive distillation. The paper's training budget scaling result (more corrective training → longer, better videos) suggests that such a loop could iteratively extend the achievable horizon. The key question is whether synthetic long videos contain artifacts (subtle temporal inconsistencies, limited motion diversity) that, when used as training data, cause the bidirectional model to learn those artifacts rather than transcend them. This experiment would also address a gap in the current paper: the absence of any comparison to models trained on real long videos (which are scarce) by creating a synthetic long-video training set and measuring its effectiveness.

Practical Applications and Downstream Use Cases

Cost-efficient long-form content creation for pre-visualization and storyboarding. Film and animation productions routinely create rough pre-visualizations (animatics, layout reels) to plan camera movements, scene compositions, and timing before committing to expensive production rendering or live-action shooting. Currently, these are created manually by artists or with limited procedural tools. A Self-Forcing++ model generating 1-4 minute videos at 17 FPS (Table 1 throughput) from text descriptions could produce rough scene visualizations in minutes rather than days, at a fraction of the cost of manual creation. The model's sustained dynamic degree (54.12 at 100 seconds, Table 2) means these visualizations would maintain meaningful motion rather than becoming static, providing useful timing and composition information. The limitation is that current quality (Visual Stability 84.22, aesthetic quality 53.00 at 100 seconds, as scored by Gemini and VBench) is suitable for rough pre-visualization but not for final output—the content would need refinement by human artists. However, for the specific use case of internal planning and communication (director to cinematographer, production designer to VFX team), approximate visual quality with sustained motion dynamics is far more useful than high-quality 5-second clips that cannot convey scene flow.

Automated video loop and ambient content generation. Many digital display contexts require long-duration ambient video with no obvious loop point: digital signage, virtual backgrounds, gallery installations, meditation/relaxation apps, and streaming "mood" content. Current approaches either use short loops (which become visibly repetitive within seconds) or procedurally generated content (which often lacks visual richness). Self-Forcing++'s high NoRepeat score (98.44, Table 5) and sustained dynamic degree across minutes-long durations make it uniquely suited for this application: it can generate videos that maintain visual interest and motion for minutes without looping, from simple text prompts describing the desired ambiance. The 17 FPS throughput (Table 1) means a 4-minute ambient video could be generated in approximately 14 seconds of compute time, enabling on-demand generation rather than pre-rendered libraries. The key practical advantage over existing approaches is the combination of duration (no visible looping for 4+ minutes), motion continuity (dynamic degree doesn't collapse), and on-demand customizability (any text prompt). The limitation is content control—the model generates what it generates, and while the text prompt provides high-level guidance, precise control over composition, color palette, or motion patterns is not supported.

Training data augmentation for robotic manipulation and autonomous driving simulators. Simulation-based training for robotics and autonomous vehicles requires diverse, temporally coherent video of environments under varied conditions. Real-world data collection is expensive and constrained (safety, weather, geography), while procedural simulation often lacks visual realism. A Self-Forcing++ model could generate minutes-long first-person or drone-perspective videos of driving scenarios, indoor navigation, or object manipulation, conditioned on text descriptions of the environment, lighting, and objects present. The sustained visual stability (84.22 at 100 seconds) and dynamic degree (54.12) mean these synthetic videos would maintain scene consistency and motion over durations useful for training perception systems—unlike 5-second clips that capture only isolated moments. The key advantage over existing sim-to-real approaches is that the generated content is produced by a model trained on real video distributions (via the teacher's score function), so the visual statistics may be more realistic than procedural rendering, while the text-prompt interface enables systematic variation of conditions (weather, traffic density, object types). The limitation is that the model has no grounding in physics or 3D geometry—generated motion is plausible but not physically accurate, which limits utility for dynamics-sensitive training (e.g., precise collision prediction).

When to Prefer This Method

The paper does not explicitly frame a decision rule for choosing Self-Forcing++ over named alternatives; it presents Self-Forcing++ as strictly superior to CausVid, Self-Forcing, and Diffusion Forcing methods (SkyReels-V2, MAGI-1) on the metrics it evaluates, with no identified regime where a competing autoregressive method would be preferable. The only caveat appears in the limitations: the method has slower training speed than teacher-forcing (Section 6), and the paper does not compare against LongLive (Section 7), which may achieve comparable quality with a different mechanism (attention sink frames). In the absence of a clearly articulated tradeoff from the paper itself—beyond the generic acknowledgment that "Key drawbacks include slower training speed compared to teacher-forcing"—a formulaic decision matrix would impose a comparison the paper does not develop. The paper's position is that Self-Forcing++ is the new state-of-the-art for autoregressive long-video generation and that its approach (corrective training on self-generated error states) is sufficient to outperform alternatives, with the only open question being whether the scaling properties demonstrated for Wan2.1-T2V-1.3B transfer to other architectures, which the paper does not test.