LongCat-Video Technical Report

ArXiv: 2510.22200

🎯 Pitch

LongCat-Video generates minutes-long, 720p videos in minutes on a single GPU by using block sparse attention that discards over 90% of dense computations, yet still matches leading proprietary models through a multi-reward RLHF scheme that trains on grayscale motion to disentangle visual quality from dynamics.

---

1. Executive Summary

This technical report introduces LongCat-Video, a 13.6B-parameter foundational video generation model built on the Diffusion Transformer (DiT) framework that unifies Text-to-Video, Image-to-Video, and Video-Continuation tasks within a single model by treating them all as video continuation with varying numbers of conditioning frames. The model employs a coarse-to-fine generation strategy along both temporal and spatial axes (generating at 480p, 15fps then refining to 720p, 30fps via a LoRA refinement expert) combined with Block Sparse Attention (retaining less than 10% of dense attention computations) to achieve over 10× inference speedup, enabling 720p, 30fps video generation within minutes on a single H800 GPU. Post-training uses multi-reward GRPO training with three specialized reward models—visual quality (HPSv3-based), motion quality (VideoAlign-based, trained on grayscale), and text-video alignment (VideoAlign-based)—to match or exceed leading open-source models like Wan 2.2 and approach proprietary systems like Veo3 on internal benchmarks, with a VBench 2.0 total score of 62.11% and leading performance on the Commonsense dimension (70.94%). The model supports minutes-long video generation without color drift or quality degradation by pretraining natively on Video-Continuation tasks, establishing that long-form temporal coherence can be achieved through task-native pretraining rather than post-hoc fine-tuning of existing video foundation models.

2. Context and Motivation

The Core Problem: Efficient Long Video Generation as a Gateway to World Models

The fundamental challenge LongCat-Video tackles is deceptively simple to state but enormously difficult to solve: how do we generate minutes-long, high-resolution, temporally coherent videos efficiently enough to be practical? The paper's framing of this problem goes beyond the typical video generation benchmark-chasing narrative. Instead, it positions long video generation as a critical enabling capability for world models—systems that aim to understand, simulate, and predict complex real-world environments. The authors are explicit about this ambition in the introduction:

"Video generation models serve as a critical pathway toward world models by compressing geometric, semantic, physical, and other forms of knowledge through video generation tasks, thereby enabling effective simulation and prediction of the physical world. Notably, efficient long video generation is particularly essential."

This framing matters because it elevates the stakes beyond content creation (which is already a significant application space spanning user-generated content, film production, advertising) to include applications like autonomous driving and embodied AI, where models must simulate extended temporal sequences of the physical world with high fidelity. The "efficient" qualifier is equally important: a world model that requires hours of computation per simulated second is not practically useful for real-time decision-making in robotics or autonomous systems.

The problem decomposes into four interlocking sub-challenges that the paper identifies, implicitly or explicitly:

1. Temporal coherence at scale. Generating videos longer than a few seconds typically results in color drift, quality degradation, or loss of scene consistency over time. This is because error accumulation in autoregressive or sliding-window approaches compounds with each generation step, and the model gradually "forgets" what it generated earlier in the sequence.

2. Computational feasibility. The attention mechanism in DiT-based video models scales quadratically with token count. A 720p, 30fps video at frame rate $F$ and duration $T$ produces $O(F \times T \times H \times W)$ tokens. For minutes-long video, this becomes computationally prohibitive without approximation strategies that risk degrading quality.

3. Multi-task unification. Real-world deployment requires different input modalities—text prompts for creative ideation, reference images for precise content control, and existing video clips for continuation or interactive storytelling. Most systems treat these as separate models or separate fine-tuned variants, multiplying engineering complexity and computational cost.

4. Alignment with human preferences. Standard diffusion training objectives (flow matching MSE loss) optimize pixel-level reconstruction but do not directly optimize for perceptual qualities humans care about: visual aesthetics, motion naturalness, and semantic alignment with the prompt. Post-training alignment is necessary but challenging for video, where reward hacking (generating static but high-quality frames to maximize visual scores, for instance) is an acute risk.

Why This Problem Matters: Theoretical and Practical Significance

The World Model Connection

The paper's explicit framing of video generation as a pathway to world models places this work at the intersection of generative modeling and embodied AI. The logic is: if a model can accurately predict how a scene will evolve over time—people moving, objects interacting, lighting changing—it has implicitly learned a compressed representation of physics, geometry, and semantics. Generating a video of a ball bouncing is a test of whether the model understands gravity, elasticity, and collision dynamics. Generating a minutes-long video of a person walking through a city is a much harder test that requires maintaining object permanence, scene geometry, and motion consistency across thousands of frames.

This is not a new idea—it traces back to the "learning world models" literature (Ha and Schmidhuber, 2018) and video prediction research—but the scale is new. At 13.6B parameters generating 720p, 30fps minutes-long videos, LongCat-Video operates at a regime where the world-knowledge compression hypothesis can actually be tested, not just speculated about. The paper's finding that it leads all other methods on VBench 2.0's Commonsense dimension (70.94%), which evaluates motion rationality and physical law adherence, provides preliminary evidence that this direction is viable.

The Efficiency Imperative

The computational demands of video generation are staggering. The paper reports (Table 2) that native 720p, 93-frame generation requires 1429.5 seconds (~24 minutes) at 50 sampling steps on an H800 GPU. This is for roughly 3 seconds of video (93 frames at 30fps). Extrapolating naively, 1 minute of 720p, 30fps video would require ~8 hours of GPU time—clearly impractical for any real-world deployment. The efficiency problem is not just an engineering convenience; it is a hard requirement for making long video generation economically viable and for enabling world model applications that need rapid simulation.

The quadratic complexity of self-attention is the primary culprit. With $N$ tokens, dense attention requires $O(N^2)$ operations. For a video of duration $T$, height $H$, width $W$, and VAE-compressed spatial dimensions $H/16$, $W/16$, the token count is approximately $T \times (H/16) \times (W/16)$. For 720p video ($1280 \times 720$), this works out to $T \times 80 \times 45 = 3600T$ tokens per frame. At 93 frames, that's ~334,800 tokens. Self-attention over 334K tokens is expensive; over 1,800 frames (1 minute at 30fps) it would be ~6.5M tokens, which is infeasible with dense attention.

The Multi-Task Practicality Gap

Real users of video generation systems don't think in terms of "I need a Text-to-Video model" or "I need an Image-to-Video model"—they think in terms of creative workflows that mix these modalities. A filmmaker might start with a text description of a scene, use a generated keyframe as an Image-to-Video input for a specific shot, and then use Video-Continuation to extend that shot or transition to the next scene. Requiring separate models for each step—each with its own training pipeline, inference serving infrastructure, and quality characteristics—is a significant practical burden that unified approaches address.

Prior Approaches and Where They Fall Short

The paper does not provide an exhaustive literature review, but the implicit landscape of prior work can be reconstructed from the introduction, method descriptions, and evaluation comparisons. The gaps in prior approaches become clear when mapped against the four sub-challenges identified above.

The First Generation of Video Diffusion Models

Early video diffusion models (pre-2024) established the basic paradigm—extend image diffusion architectures by adding temporal attention or 3D convolutions—but were limited in resolution (256p), duration (2-4 seconds), and quality. The paper's Table 8 places these in context: HunyuanVideo (Kong et al., 2024) achieves a VBench 2.0 total score of 55.30% and Wan2.1 (Wan et al., 2025) achieves 60.20%. These scores, while respectable, reflect limitations in visual quality, motion naturalness, and instruction following that the paper's model improves upon (62.11%).

Key limitation of this generation: These models were not designed for long video generation. They train on fixed-length clips (typically 2-5 seconds) and cannot easily generalize to longer sequences without modification. The paper explicitly notes:

"While various methods [Chen et al., 2025] exist to finetune existing video foundation models for improved long-video generation, LongCat-Video is natively pretrained on Video-Continuation tasks, enabling it to produce minutes-long videos without color drifting or quality degradation."

This is a critical distinction. Post-hoc fine-tuning approaches (like SkyReels-V2, Chen et al., 2025) take a short-video model and adapt it to generate longer sequences, usually through autoregressive extension or sliding window mechanisms. These work to varying degrees but suffer from the fundamental mismatch: the model was never trained to maintain consistency over long horizons, so error accumulation is an inherent problem. The paper's approach—pretraining natively on video continuation tasks—addresses this at the architectural and training-objective level rather than patching it after the fact.

Proprietary Commercial Systems

Models like Veo3 (Google), Sora (OpenAI), Kling 1.6 (Kuaishou), Seedance 1.0 (ByteDance), and Hailuo (MiniMax) have demonstrated impressive video generation quality, with Veo3 achieving the top VBench 2.0 score of 66.72%. However, these systems have several limitations that motivate open-source alternatives:

1. Closed weights and architectures: The community cannot inspect, study, or build upon these models. This slows progress on fundamental research questions about what video generation models learn about the physical world.

2. API-only access: Users cannot deploy these models on their own hardware, customize them for specific domains, or fine-tune them on proprietary data. For applications in film production, advertising, or autonomous driving where data privacy and IP control matter, this is disqualifying.

3. No public training recipes: The field cannot learn from the data curation, training strategies, or post-training alignment approaches that went into these models. Each proprietary system represents a collection of unpublished techniques that must be independently rediscovered.

Key limitation: Proprietary systems demonstrate what is possible but do not advance the scientific understanding of how to achieve high-quality, efficient long video generation, nor do they enable the ecosystem of research and applications that open-weight models support.

The Multi-Task Fragmentation Problem

Prior to LongCat-Video, the norm was to train separate models for Text-to-Video (T2V), Image-to-Video (I2V), and video extension tasks. Even within open-source model families like Wan (Wan et al., 2025), the T2V and I2V variants are distinct model releases—Wan 2.2-T2V-A14B and Wan 2.2-I2V-A14B—with presumably separate training pipelines. CogVideoX (Yang et al., 2024) and Step-Video (Ma et al., 2025a) similarly focus on T2V primarily.

Key limitation of fragmented approaches:

• Engineering overhead: Each task variant requires separate training runs, checkpoint management, and serving infrastructure. For a production deployment supporting creative workflows, this multiplies cost and complexity.
• Inability to share learned representations: A T2V-only model trained on text-video pairs cannot benefit from the fine-grained visual control signals in I2V data, and vice versa. The unified approach allows knowledge learned from one task to transfer to others through shared parameters.
• Inconsistent behavior: Users switching between T2V and I2V modes encounter different quality characteristics because the underlying models were separately trained, leading to unpredictable creative workflows.

The RLHF Gap for Video Generation

Reinforcement Learning from Human Feedback (RLHF) and direct preference optimization have been transformative for language models (InstructGPT, Claude, Llama) and are increasingly applied to image generation (DALL-E 3, Stable Diffusion with DPO). For video generation, however, RLHF is substantially harder for several reasons the paper's approach addresses:

1. Credit assignment across time: Unlike images (single output) or text (token-by-token generation), video generation involves denoising across timesteps where the contribution of each step to the final reward is ambiguous. The paper calls this "temporal credit assignment ambiguity" (Section 3.3.1).

2. Reward model challenge: Training reward models that accurately assess video quality—especially motion quality and temporal consistency—requires rich annotated data. Models can easily overfit to surface-level features (e.g., preferring static videos because they have fewer visual artifacts) rather than assessing genuine motion quality.

3. Reward hacking: When optimizing a single reward (e.g., HPSv3 visual quality), the model converges to degenerate strategies. The paper explicitly demonstrates this: optimizing only HPSv3 causes "static tendency"—the model produces high-quality but motionless frames because static images score well on visual quality metrics while avoiding the difficulty of generating coherent motion (Figure 9).

4. Slow convergence: The standard GRPO formulation, when applied naively to flow matching video generation, suffers from gradient magnitude issues that the paper identifies: vanishing gradients at high noise levels (as $t \to 1$, the scaling factor $\kappa(t,\Delta t) \to 0$) and suppressed gradients due to the small timestep sizes typical in video generation training.

Prior work applying GRPO to image generation (Liu et al., 2025a; Xue et al., 2025; He et al., 2025) did not face the temporal credit assignment and multi-reward balancing challenges that are central to video. Concurrent work (Zhou et al., 2025) began addressing the credit assignment issue, but the paper's combination of selective SDE timestep, reweighted policy and KL losses, and max-group standard deviation normalization represents a comprehensive solution to the convergence and stability problems specific to video.

How LongCat-Video Positions Itself Relative to Existing Work

The paper's positioning is nuanced: it is not claiming to be the absolute best at any single dimension, but rather to be the most practically useful and scientifically instructive open-source video generation model that unifies multiple capabilities under one framework. Several positioning claims are explicit or strongly implied:

1. The "Native Long Video" Distinction

The paper distinguishes itself from models that treat long video as an afterthought:

"LongCat-Video is natively pretrained on Video-Continuation tasks, enabling it to produce minutes-long videos without color drifting or quality degradation."

The emphasis on "pretrained" rather than "fine-tuned" is key. By including Video-Continuation as a core training task throughout the progressive pretraining stages (Table 3 shows VC present from the third stage onward, alongside T2I, T2V, and I2V), the model learns temporal consistency as a fundamental capability rather than a bolted-on feature. This architectural decision—framing all tasks as video continuation with varying numbers of conditioning frames—is the mechanism that makes the unification possible.

2. The Efficiency Frontier

The efficiency claims (Table 2: 12.3× speedup for 720p × 93 frames) position LongCat-Video at a point on the quality-efficiency Pareto frontier that is competitive with the fastest inference systems while maintaining open-source availability. The combination of LCM distillation (reducing sampling steps from 50 to 16), coarse-to-fine generation (avoiding expensive full-resolution denoising for all steps), and block sparse attention (reducing attention compute to <10% of dense) is not individually novel—each technique has precedents in Seedance, Hailuo, and concurrent sparse attention work—but the integration into a single system with all components open-sourced is.

3. The Open-Source but Competitive Stance

The evaluation results (Section 5) are carefully framed to acknowledge that Veo3 still leads on VBench 2.0 (66.72% vs. 62.11%) while emphasizing that LongCat-Video is open-source and that its strengths (Commonsense rank #1 at 70.94%, competitive overall quality on internal benchmarks) make it the strongest available open model. The GSB comparisons (Figure 15) are particularly informative: vs. Wan 2.2-T2V-A14B (the previous SOTA open-source model), LongCat-Video is preferred in overall quality (250 vs. 54), text-alignment (212 vs. 117), and motion quality (219 vs. 175). This positions it as the new open-source state-of-the-art for T2V at time of release.

4. The Multi-Reward RLHF as a Differentiator

The paper's approach to RLHF distinguishes itself through three technical contributions that address the specific challenges of video:

• Selective SDE timestep: By applying noise injection at only one randomly selected timestep and using deterministic ODE sampling elsewhere, credit assignment is localized to that timestep. This directly addresses the temporal credit assignment ambiguity that naive GRPO faces in video generation.

• Reweighted policy and KL losses: Equations (36) and (40) introduce correction factors that normalize the gradient magnitude to be independent of the sampled timestep and step size. This is a mathematically motivated fix to the vanishing gradient problem that would otherwise cause slow or unstable training.

• Max group standard deviation: By using the maximum standard deviation across all groups in a batch rather than the per-group standard deviation for advantage normalization (Equation 9), the method is more robust to reward model noise. Groups with spuriously small reward variance (which can happen due to reward model inaccuracies) don't receive disproportionately large gradient updates.

• Multi-reward balancing: Using three specialized reward models (HPSv3-general for pure visual quality, HPSv3-percentile for text-video alignment via caption-matched scoring of top 30% frames, and the grayscale VideoAlign-based motion quality model) with weighted advantage combination (Equation 50) prevents the reward hacking that occurs with single-reward optimization.

5. The "World Models First Step" Framing

The positioning as a "first step toward world models" is strategically appropriate. The paper does not claim to have built a world model—it claims to have built a video generation model that demonstrates some of the capabilities (long-range temporal coherence, physics-aligned motion, commonsense reasoning) that world models require. The VBench Commonsense score (70.94%, leading all models including Veo3 at 69.48%) provides empirical evidence that pretraining on Video-Continuation tasks does indeed improve the model's understanding of physical and causal relationships. The future work section makes this explicit: "Future directions include better modeling of physical knowledge, multi-modal memory integration in video generation, and the incorporation of knowledge from LLM and MLLM."

Summary of the Gap Being Filled

In the landscape as of October 2024, there existed no open-source model that simultaneously:

1. Achieved competitive quality with proprietary systems (Veo3, Kling) on rigorous benchmarks
2. Generated minutes-long videos natively without quality degradation
3. Unified T2V, I2V, and Video-Continuation in a single model
4. Achieved practical inference latency (minutes, not hours, for 720p, 30fps)
5. Demonstrated effective multi-reward RLHF for video generation
6. Released all code, weights, and efficiency modules (block sparse attention) for community use

LongCat-Video aims to fill all of these gaps simultaneously. The combination of these capabilities—rather than any single one—is what the paper argues represents a meaningful advance over existing open-source and proprietary systems, and what makes it a credible "first step" toward the world model vision.

3. Technical Approach

3.1 Reader Orientation

LongCat-Video is a 13.6B-parameter neural network system that takes a text description, an image, or a video clip as input and generates high-quality video output—essentially, a single engine that can do Text-to-Video, Image-to-Video, and Video-Continuation tasks depending on what you feed it. The core problem it solves is generating minutes-long, high-resolution, temporally coherent videos efficiently enough to be practical, and the solution is shaped as a unified Diffusion Transformer that treats all tasks as "video continuation" (predict future frames from conditioning frames), combined with a coarse-to-fine inference pipeline and block sparse attention that collectively achieve over 10× speedup, plus a multi-reward RLHF post-training stage that aligns output quality with human preferences across visual aesthetics, motion naturalness, and text-video alignment simultaneously.

3.2 Big-Picture Architecture (Diagram in Words)

The system has seven major components, connected in a pipeline that flows from data → training → inference → post-training:

1. Data Curation Pipeline (Section 2) — A two-stage system that preprocesses raw video (deduplication, scene segmentation, black border cropping) and then annotates clips with multimodal metadata (captions, aesthetic scores, motion metrics, camera attributes) to build a flexible training database.

2. VAE and Text Encoder (Section 3.1) — WAN2.1 VAE compresses video pixels to latent tokens at 4×8×8 compression; umT5 text encoder converts multilingual captions into conditioning embeddings. Output: latent tensors and text embeddings that the DiT processes.

3. Diffusion Transformer Backbone (Section 3.1) — 48-layer single-stream DiT with 3D self-attention, cross-attention for text, SwiGLU FFN, AdaLN-Zero modulation, and 3D RoPE positional encoding. Takes noisy latents + text embeddings as input, predicts velocity field for flow matching.

4. Unified Input Representation (Section 3.2) — A block attention mechanism with KV-caching that concatenates clean conditioning frames (timestep zero) with noisy generation frames along the temporal axis, enabling a single model to handle T2V (0 conditioning frames), I2V (1 frame), and Video-Continuation (multiple frames).

5. Coarse-to-Fine Inference Pipeline (Section 3.4.1) — Two-stage generation: first produce 480p, 15fps video using the base model with LCM-distilled sampling; then upsample to 720p, 30fps via trilinear interpolation and refine with a LoRA expert trained at partial noise level (tₜₕᵣₑ=0.5, only 5 sampling steps).

6. Block Sparse Attention (Section 3.4.2) — A trainable sparse attention operator that partitions video tokens into 3D blocks, computes pooled similarity scores between query and key blocks, selects only the top-r blocks for full attention computation, and reduces attention operations to less than 10% of dense while maintaining quality.

7. Multi-Reward GRPO Post-Training (Section 3.3) — An RL-based alignment stage using Group Relative Policy Optimization with three reward models (visual quality via HPSv3, motion quality via grayscale VideoAlign, text-video alignment via VideoAlign), combined through weighted advantage summation, with selective SDE timesteps, reweighted losses, and max-group standard deviation for stable convergence.

Information flows as follows: raw video → preprocessing → annotation → progressive DiT pretraining on mixed tasks (T2I → T2V → multi-task with VC) → SFT on high-quality curated data → CFG + CM distillation (LoRA) → GRPO alignment (LoRA) → refinement expert training (LoRA + BSA). At inference: text/image/video input → VAE encoding + text encoding → base model low-res generation (16 LCM steps) → trilinear upsample → refinement expert high-res generation (5 steps with BSA) → VAE decoding → output video.

3.3 Roadmap for the Deep Dive

• First, the VAE and text encoder (Section 3.1 details) — these define the token spaces the DiT operates in, setting stage for all subsequent computation.
• Second, the unified input representation and block attention with KV-cache (Section 3.2) — the core architectural innovation that enables multi-task unification, since everything else (training, inference, RLHF) builds on how data enters the model.
• Third, the coarse-to-fine generation strategy (Section 3.4.1) — the two-stage pipeline that makes efficient inference possible, including the refinement expert's flow matching formulation and how conditioning frames are handled.
• Fourth, block sparse attention (Section 3.4.2) — the mechanism that reduces attention cost from quadratic to sub-quadratic and enables the refinement stage to be practical at high resolution.
• Fifth, the multi-reward GRPO training (Section 3.3) — the most mathematically involved component, including the flow matching GRPO derivation, the selective timestep strategy, loss reweighting, max-group normalization, and the multi-reward combination.
• Sixth, the progressive training pipeline (Section 4 briefly referenced) — how these components are assembled across pretraining, SFT, distillation, and refinement training stages.

This order follows the data flow: input encoding → model architecture → inference optimization → training optimization. It ensures the reader understands what the model is (Sections 3.1–3.2) and how it runs fast (Sections 3.4.1–3.4.2) before tackling the mathematically intensive RLHF derivation (Section 3.3).

3.4 Detailed, Sentence-Based Technical Breakdown

This is a systems paper whose core idea is that long, high-quality, multi-task video generation can be achieved by (1) treating all video generation tasks as video continuation with varying numbers of conditioning frames in a unified DiT architecture, (2) decomposing high-resolution generation into a cheap coarse pass followed by a targeted refinement pass, (3) replacing dense attention with trainable 3D block sparse attention that retains <10% of computation, and (4) applying a carefully stabilized multi-reward GRPO procedure that jointly optimizes visual quality, motion quality, and text-video alignment without reward hacking.

---

VAE, Text Encoder, and Token Spaces

The video generation pipeline begins by converting raw pixels into a compressed latent representation that the Diffusion Transformer can process efficiently. Two separate encoders handle the visual and textual modalities:

Visual compression via WAN2.1 VAE. The paper employs the WAN2.1 VAE (Wan et al., 2025) to convert video frames from pixel space to latent space. The compression ratio is 4×8×8 along the temporal, height, and width dimensions respectively—meaning a video of shape (T, H, W) becomes a latent tensor of shape (T/4, H/8, W/8). Additionally, a patchify operation inside the DiT model further compresses latent tokens by 1×2×2. The overall compression from pixels to the tokens actually processed by the transformer is therefore 4×16×16.

Why these ratios matter. The 4× temporal compression means a 30fps video sampled at 1 frame becomes effectively 7.5 latent frames per second. The 16×16 spatial compression means a 720p frame (1280×720) becomes 80×45 latent tokens per frame. This is the critical efficiency bottleneck: at 93 frames, the latent sequence length is approximately 93 × (80×45) = 93 × 3,600 = 334,800 tokens. Self-attention over 334K tokens is $O(334,800^2)$ ≈ $10^{11}$ operations per attention head—the motivation for sparse attention becomes clear when you see this number.

Text encoding via umT5. For conditioning, the paper uses umT5 (Chung et al., 2023), a multilingual text encoder based on the T5 architecture that supports both English and Chinese captions. Unlike the visual tokens which are processed through the VAE + patchify pipeline, text tokens are processed by umT5 into a sequence of embeddings that are fed into the DiT's cross-attention layers. The choice of umT5 over CLIP or T5-XXL (common in image/video generation) is motivated by bilingual support—the training data includes both English and Chinese captions (Section 2.1.2), and umT5 natively handles this multilingual setting.

What this enables downstream. The latent representation determines all subsequent computation. The token count $N = T/4 \times H/16 \times W/16$ is the sequence length for self-attention. The cross-attention keys and values come from umT5's output, with a fixed length independent of video resolution. This separation—video tokens for self-attention, text tokens for cross-attention—is the standard DiT design pattern.

---

DiT Architecture Specifications

The backbone is a standard Diffusion Transformer (DiT) with single-stream transformer blocks. The model specifications from Table 1 are:

Transformer block structure. Each of the 48 blocks contains, in order: (1) a 3D self-attention layer operating on video tokens, (2) a cross-attention layer attending from video tokens to text embeddings, and (3) a Feed-Forward Network (FFN) using the SwiGLU activation function (Shazeer, 2020). For modulation, the paper uses AdaLN-Zero (Peebles and Xie, 2023), where each block has a dedicated modulation MLP that takes the timestep embedding as input and produces scale, shift, and gate parameters for layer normalization.

Normalization and positional encoding. RMSNorm (Zhang and Sennrich, 2019) is applied as QKNorm (Henry et al., 2020) within both self-attention and cross-attention modules—this means the query and key projections are normalized before computing attention scores, which improves training stability by preventing attention logit magnitudes from growing uncontrollably. For positional encoding of visual tokens, the paper uses 3D RoPE (Su et al., 2024), which applies rotary position embeddings along all three axes (temporal, height, width) so the model can distinguish token positions in the 3D video volume.

Why this architecture was chosen. The DiT design is the dominant paradigm for transformer-based diffusion models, replacing the U-Net architectures used in earlier diffusion work. The single-stream design (where all tokens, regardless of modality, pass through the same transformer blocks) is simpler than two-stream alternatives and has been validated by numerous image and video generation models. The specific dimensions—48 layers, 4096 hidden size, 16384 FFN size—place LongCat-Video at roughly 13.6B parameters, which is in the same scale class as Wan 2.2's 14B model and larger than CogVideoX (which uses 3B, 5B, or 10B variants). The use of AdaLN-Zero (which initializes the modulation output to zero, effectively starting training from an identity function) is important for stable training at this scale.

---

Unified Input Representation for Multiple Tasks

This is the central architectural innovation of the paper—the mechanism that enables a single model to handle Text-to-Video (T2V), Image-to-Video (I2V), and Video-Continuation (VC) without separate architectures or training pipelines.

The core idea: all tasks are video continuation. The paper reframes all three tasks as predicting future frames given a set of preceding conditioning frames. The difference between tasks is only the number of conditioning frames:

• T2V: 0 conditioning frames (generation from scratch)
• I2V: 1 conditioning frame (the reference image)
• VC: Multiple conditioning frames (the preceding video clip to continue from)

Input structure. The model input is constructed by concatenating two sequences along the temporal axis:

$X = [X_{\text{cond}}, X_{\text{noisy}}] \in \mathbb{R}^{B \times (N_{\text{cond}} + N_{\text{noisy}}) \times H \times W \times C}$

where $B$ is the batch size, $N_{\text{cond}}$ is the number of conditioning frames, $N_{\text{noisy}}$ is the number of frames to be generated, $H$ and $W$ are the VAE-compressed spatial dimensions, and $C$ is the number of latent channels. $X_{\text{cond}}$ contains noise-free latent representations of the conditioning frames, and $X_{\text{noisy}}$ contains the noised latent representations of the frames being generated.

Timestep partitioning. Corresponding to the input split, the timesteps are partitioned as:

$t = [t_{\text{cond}}, t_{\text{noisy}}]$

For conditioning frames, $t_{\text{cond}}$ is set to 0 (indicating clean, lossless frames). For noisy frames, $t_{\text{noisy}}$ is sampled from $[0, 1]$ during training. During inference, the conditioning frames remain at $t = 0$ throughout all denoising steps, while the noisy frames follow the standard reverse diffusion trajectory from $t = 1$ to $t = 0$.

What this structure computes. The model receives a mixed input where some frames are clean (known, fixed) and others are noisy (to be denoised). By seeing the clean conditioning frames alongside the noisy generation frames at every denoising step, the model can use the contextual information from the conditioning frames to guide the denoising process. The timestep embedding guides the model to treat clean frames differently from noisy ones—the AdaLN-Zero modulation conditioned on $t = 0$ will produce different scale/shift parameters than those for $t = t_{\text{noisy}}$.

Why this design works for multi-task unification. The key insight is that T2V, I2V, and VC differ only in $N_{\text{cond}}$, and the architecture naturally handles variable-length conditioning through its transformer attention mechanism. During training, the model sees mixtures of all three tasks with different $N_{\text{cond}}$ values, and the timestep annotation $(t_{\text{cond}}=0, t_{\text{noisy}} \sim [0,1])$ provides a consistent signal about which tokens are condition vs. generation regardless of sequence length. This is simpler than, for example, having separate encoder pathways for images vs. video clips or using task-specific tokens to switch modes.

Training loss computation. During loss computation, the contribution from the conditioning frames is omitted—only the denoising loss on $X_{\text{noisy}}$ contributes to the gradient. This ensures the model learns to use conditioning information to improve generation quality without being penalized for "predicting" what it already knows (the conditioning frames are already clean).

How conditioning frames are prepared for different tasks during training. For the Video-Continuation task, the paper introduces an additional robustness mechanism: conditional frames are perturbed with per-frame independent noise levels (Chen et al., 2024). This means each conditioning frame receives a small, independently sampled amount of noise rather than being kept perfectly clean. The purpose is to prevent color drift during long video generation—if the model is only trained with perfectly clean conditioning frames, it may be overly sensitive to small visual discrepancies that accumulate when autoregressively using its own outputs as conditioning frames for the next chunk. By training with slightly noisy condition frames (at varying noise levels), the model learns to be robust to these imperfections.

---

Block Attention with KV-Cache for Efficient Multi-Task Inference

The unified input representation introduces a computational opportunity that the paper exploits through a specialized attention mechanism. Because conditioning frames are fixed throughout all denoising steps, their key and value representations in the attention layers can be computed once and cached, then reused across all $T$ sampling steps.

Attention structure. The paper defines a block-wise attention mechanism where conditioning tokens and noisy tokens are treated as separate attention domains:

$X_{\text{cond}} = \text{Attention}(Q_{\text{cond}}, K_{\text{cond}}, V_{\text{cond}})$

$X_{\text{noisy}} = \text{Attention}(Q_{\text{noisy}}, [K_{\text{cond}}, K_{\text{noisy}}], [V_{\text{cond}}, V_{\text{noisy}}])$

where $Q_{\text{cond}}$, $K_{\text{cond}}$, $V_{\text{cond}}$ are the query, key, and value projections of the conditioning tokens, and $Q_{\text{noisy}}$, $K_{\text{noisy}}$, $V_{\text{noisy}}$ are the corresponding projections for the noisy tokens. The notation $[K_{\text{cond}}, K_{\text{noisy}}]$ indicates concatenation along the sequence dimension.

What this computes. The first equation states that conditioning tokens attend only to themselves—they do not incorporate information from the noisy tokens. This makes sense because the conditioning frames are ground truth and should not be "updated" based on the model's current noisy state. The second equation states that noisy tokens attend to both conditioning tokens and themselves—they query against all available information, using the clean conditioning frames as context for the denoising process.

Why this is efficient (KV-cache). Because $X_{\text{cond}}$ does not depend on $X_{\text{noisy}}$ (the condition-to-condition attention uses only the conditioning tokens' own key-value pairs) and the conditioning frames are fixed throughout all $T$ denoising steps, the key and value tensors for the conditioning tokens $(K_{\text{cond}}, V_{\text{cond}})$ can be computed once during the first denoising step and cached. For all subsequent steps (typically 16 for LCM-distilled inference, or 5 for refinement), only the noisy tokens' query, key, and value projections need to be recomputed. This reduces the effective per-step attention cost from processing all $(N_{\text{cond}} + N_{\text{noisy}})$ tokens to processing only $N_{\text{noisy}}$ tokens plus attending to cached $K_{\text{cond}}$ and $V_{\text{cond}}$.

Cross-attention handling. The conditioning tokens do not participate in cross-attention with the text embeddings. This is a design choice: since conditioning frames are visual grounding information, they don't need textual alignment—only the generated frames need to be aligned with the text prompt. Omitting condition tokens from cross-attention further reduces compute and ensures that text conditioning focuses exclusively on guiding the denoising of the noisy frames.

Consistency between training and inference. The paper emphasizes that this block attention design with KV-caching is "consistent between training and inference." This means the same attention pattern (condition attending to condition only, noisy attending to both) is used during training, so there is no distribution shift between how the model learns the attention patterns and how it applies them at inference time with caching. If training used full attention (all tokens attending to all tokens) but inference used this blocked pattern, the model would encounter attention patterns it never saw during training, potentially degrading quality.

---

Coarse-to-Fine Generation Strategy

The coarse-to-fine (C2F) strategy is the primary mechanism for making high-resolution, high-frame-rate video generation computationally feasible. It decomposes the generation process into two stages:

Stage 1: Coarse generation. The base model generates a 480p, 15fps video using the standard flow matching denoising process with LCM-distilled sampling (16 steps). This produces a low-resolution, low-frame-rate video $X_{\text{lr}}$ at a fraction of the cost that generating at 720p, 30fps would require. The latent token count for 480p video is approximately 640×480 pixels / (16×16 compression) = 40×30 = 1,200 tokens per frame. At 15fps for N frames, this is substantially cheaper than 720p at 30fps.

Stage 2: Fine refinement. The low-resolution output is upsampled to 720p, 30fps using trilinear interpolation (linear interpolation along all three axes: temporal, height, width), producing $X_{\text{up}}$. This upsampled video is then refined by a LoRA expert—a lightweight adapter fine-tuned on top of the frozen base model—using only 5 sampling steps at partial noise level.

The refinement expert's flow matching formulation. The key mathematical innovation in the C2F strategy is how the refinement expert's training objective is defined. Instead of training the refiner to denoise from pure Gaussian noise (which would ignore the low-resolution generation), it is trained to learn a flow between the upsampled low-resolution output and the target high-resolution video:

$x_{\text{thresh}} = (1 - t_{\text{thresh}}) \cdot x_{\text{up}} + t_{\text{thresh}} \cdot \epsilon, \quad \epsilon \sim \mathcal{N}(0, I)$

where $x_{\text{up}}$ is the VAE-encoded latent of the trilinearly upsampled 480p, 15fps output:

$x_{\text{up}} = \text{Encode}(\text{Upsample}(\text{Decode}(x_{\text{lr}})))$

Here, Decode converts the VAE latent to RGB pixels, Upsample performs trilinear interpolation in pixel space, and Encode re-encodes the upsampled video back to the VAE latent space. The parameter $t_{\text{thresh}}$ controls the noise level—it is set to 0.5 in practice.

What $x_{\text{thresh}}$ represents. This is the starting point for the refinement flow. It is a noisy version of the upsampled low-resolution output, where $t_{\text{thresh}} = 0$ would give the clean upsampled video and $t_{\text{thresh}} = 1$ would give pure noise. At $t_{\text{thresh}} = 0.5$, $x_{\text{thresh}}$ is an equal mixture of the upsampled video and Gaussian noise—the structural information from the coarse generation is partially preserved while enough noise is added to allow the refiner to inject high-frequency details.

The refinement training objective. The flow matching path for the refinement stage is:

$x_{t'} = x_0 + (x_{\text{thresh}} - x_0) \cdot \frac{t'}{t_{\text{thresh}}}, \quad t' \in [0, t_{\text{thresh}}]$

where $x_0$ is the target 720p, 30fps video latent. The ground truth velocity is:

$v_{t'} = \frac{x_0 - x_{\text{thresh}}}{t_{\text{thresh}}}$

The model predicts the velocity field $v_{\text{pred}}(x_{t'}, c, t'; \theta)$ at sampled $t'$ values, and the training loss is the MSE between $v_{\text{pred}}$ and $v_{t'}$.

Why this formulation is well-suited to LoRA fine-tuning. At $t' = 0$, $x_{t'} = x_0$ (the target). At $t' = t_{\text{thresh}}$, $x_{t'} = x_{\text{thresh}}$ (the noisy upsampled input). The model's task is to learn the velocity field that transforms the noisy upsampled video into the high-resolution target, but only over a restricted noise range $[0, t_{\text{thresh}}]$ rather than the full $[0, 1]$ range of standard flow matching. This means the refinement expert's denoising task is more similar to the base model's task than training from scratch would be—it "starts" closer to the ground truth and only needs to learn the correction. LoRA adapters can efficiently capture this correction because they are a low-rank update to the base model's weights.

How conditioning frames are handled in C2F (I2V and VC tasks). For tasks with conditioning frames, the C2F pipeline extends the logic. In Stage 1:

$X_{\text{lr}} = \text{BaseModel}([\text{Encode}(X_{\text{lr}}^{\text{cond}}), \epsilon]), \quad \epsilon \sim \mathcal{N}(0, I)$

where $X_{\text{lr}}^{\text{cond}}$ is the condition frames downsampled to low resolution:

$X_{\text{lr}}^{\text{cond}} = \text{Downsample}(X_{\text{hr}}^{\text{cond}})$

In Stage 2, the concatenated input for refinement is:

$X_{\text{up}} = [X_{\text{hr}}^{\text{cond}}, \text{Upsample}(X_{\text{lr}})]$

The high-resolution condition frames are concatenated with the upsampled low-resolution generated frames, establishing correspondence between the conditioning at both resolutions. Noise at level $t_{\text{thresh}}$ is then added to the VAE-encoded $X_{\text{up}}$, and the refinement expert denoises to produce the final high-resolution video $X_{\text{sr}}$.

What this achieves. The C2F strategy reduces the inference cost of 720p, 30fps generation from being dominated by full-resolution denoising at every step to being dominated by the coarse 480p generation (16 steps, fewer tokens) plus a lightweight refinement (5 steps at high resolution, but with sparse attention making it feasible). Table 2 shows the concrete impact: 480p→720p C2F achieves 10.6× speedup over native 720p generation (135.3s vs. 1429.5s for 93 frames), and when combined with BSA, reaches 12.3× speedup (116.5s). For the 189-frame case, C2F+BSA achieves 10.1× speedup (142.0s vs. 302.9s without BSA).

A secondary benefit: quality improvement. The paper notes that C2F "not only reduces inference cost but also improves generation quality, particularly enhancing visual details"—this is visible in Figure 10, where the C2F 720p output shows sharper textures than the native 720p generation. The hypothesis is that generating at 480p first establishes a coherent global layout, and the refinement stage can then focus on adding high-frequency details without having to simultaneously determine the overall composition.

---

Block Sparse Attention (BSA)

Block Sparse Attention is the mechanism that makes the refinement stage—and high-resolution video generation in general—computationally tractable by reducing self-attention operations to less than 10% of dense attention while maintaining nearly lossless generation quality.

The key insight: video latent redundancy. Video latent representations contain significant redundancy—neighboring tokens in space and time represent highly correlated visual information. Dense attention, which computes attention weights between every pair of tokens, wastes most of its computation on token pairs that have near-zero relevance to each other. The BSA mechanism exploits this redundancy by grouping tokens into blocks and selecting only the most relevant blocks for each query token to attend to.

3D block partitioning. The video latent sequence, with shape $T \times H \times W$ tokens in memory order $[T, H, W]$, is divided into non-overlapping 3D blocks of size $t \times h \times w$, where the paper sets $t = h = w = 4$. This produces $N_T \times N_H \times N_W$ blocks, where $N_T = \lceil T/t \rceil$, $N_H = \lceil H/h \rceil$, $N_W = \lceil W/w \rceil$. Each block contains $n = t \times h \times w = 64$ tokens. The tokens are rearranged in memory so that all tokens within a block are contiguous (intra-block order: $[t, h, w]$), and blocks are arranged in $[N_T, N_H, N_W]$ order (block-wise order).

Pooled block similarity computation. For each attention head, the query and key projections are computed as usual:

$Q = XW_q \in \mathbb{R}^{b \times n_h \times s_q \times d}, \quad K = XW_k \in \mathbb{R}^{b \times n_h \times s_k \times d}$

where $b$ is batch size, $n_h$ is number of attention heads (32 for LongCat-Video), $s_q = s_k = T \times H \times W$ is the sequence length, and $d$ is the per-head feature dimension. To determine which blocks are relevant without computing the full $QK^T$ matrix, the model computes pooled query and key representations for each block:

$Q_{\text{pool}}[:,:,b_q,:] = \frac{1}{n}\sum_{j=0}^{n-1} Q[:, :, (b_q-1)n + j, :]$

$K_{\text{pool}}[:,:,b_k,:] = \frac{1}{n}\sum_{j=0}^{n-1} K[:, :, (b_k-1)n + j, :]$

where $b_q$ indexes query blocks and $b_k$ indexes key blocks. The pooled representations are the average of all token embeddings within each block, providing a compact summary vector of size $d$ per block.

Block selection via pooled attention scores. The similarity between query block $i$ and key block $j$ is computed using the standard scaled dot-product attention score on the pooled vectors:

$S_{\text{pool}} = \frac{Q_{\text{pool}} K_{\text{pool}}^T}{\sqrt{d}} \in \mathbb{R}^{b \times n_h \times N_q \times N_k}$

For each query block $i$, the top-r key blocks with the highest scores in $S_{\text{pool}}[:, :, i, :]$ are selected, where $r$ controls the sparsity level. The paper sets $r$ to $\frac{1}{8} N_k$ during distillation training (12.5% of total blocks) and $\frac{1}{16} N_k$ during refinement expert training (6.25% of total blocks). This selection produces a binary mask $M \in \mathbb{R}^{b \times n_h \times s_q \times s_k}$:

$M[:, :, in:(i+1)n, jn:(j+1)n] = \begin{cases} 1 & \text{if key block } j \text{ is in top-} r \text{ for query block } i \\ 0 & \text{otherwise} \end{cases}$

Sparse attention computation. With the mask constructed, attention is computed as:

$O = \text{softmax}\left(\frac{QK^T}{\sqrt{d}} \odot M\right) V$

where the mask sets non-selected positions to $-\infty$ before softmax, effectively zeroing their attention weights. The key efficiency gain: instead of computing $O(s_q \times s_k)$ attention scores and $O(s_q \times s_k \times d)$ attention-weighted value aggregations, the model computes only $O(s_q \times rn \times d)$—a reduction factor of approximately $1/(r/N_k)$.

Why this form works for video. The 3D block structure is critical: by grouping tokens into spatiotemporal blocks (4×4×4), the selection operates on coherent spatiotemporal neighborhoods rather than arbitrary token groupings. A query token attending to all tokens within a selected block captures local spatiotemporal context naturally. The top-r selection based on pooled similarity means the model can learn which spatiotemporal regions are most relevant through the training process—the pooled query and key projections are parameterized by $W_q$ and $W_k$, which are optimized during training to produce meaningful block-level similarity scores.

Context parallelism with ring block sparse attention. For distributed training across multiple GPUs, the paper implements ring block sparse attention, where the input sequence is partitioned across $N_{cp}$ parallel ranks. Each rank maintains a local segment of size $\frac{T \times H \times W}{N_{cp}}$ latents. For rank $i$:

1. Each rank computes its own local pooled keys $K_{\text{pool}_j}$ for $j \in [1, N_{cp}]$, representing the pooled representations of its segment.
2. The pooled key representations are gathered across all ranks using a ring attention communication pattern, where each rank sends its $K_{\text{pool}}$ to the next rank and receives from the previous rank, overlapping communication with local computation.
3. Rank $i$ computes its pooled score matrix $S_{\text{pool}_i}$ using its local queries and the concatenated pooled keys from all ranks.
4. The block selection mask $M_i$ is constructed by selecting the top-r key blocks across all ranks' pooled keys.
5. The final attention output uses online softmax (the standard technique for computing softmax over partitioned tensors without requiring all values simultaneously) with ring attention to overlap the attention computation with the communication of $K_j$, $V_j$ tensors.

Implementation details and design trade-offs. The BSA operator is implemented in Triton (Tillet et al., 2019), building on Flash Attention (Dao, 2023). Both forward and backward passes are implemented for both single-GPU and context-parallel configurations. The paper explores two block selection strategies:

• Top-r mode (adopted): Selects exactly $r$ key blocks with the highest pooled scores. This provides predictable computational cost and is amenable to efficient implementation since all query tokens attend to the same number of key blocks.
• CDF-p mode (explored but not adopted): Selects key blocks in descending order of pooled scores until the cumulative softmax of scores reaches a threshold $p$. This yields better generation quality under high speedup ratios in a training-free setting (because it adaptively allocates more attention blocks to queries that need them), but suffers from variable key block counts per query, which complicates efficient implementation. For trainable scenarios, the top-r approach was chosen due to predictable, hardware-friendly computation patterns.

The paper reports that in their experiments, no significant differences in post-training results were observed across various tested configurations of 3D block sizes, with $t_q \cdot h_q \cdot w_q$ values in $[64, 128]$ and $t_k \cdot h_k \cdot w_k$ values in $[64, 128, 256, 512, 1024]$. This robustness suggests that the exact block partitioning is not critical—what matters is the sparsity itself (retaining <10% of attention computation) and the trainability (the model learns to route attention through the parameterized $W_q$ and $W_k$ projections).

---

GRPO for Flow Matching: The Mathematical Foundation

The GRPO post-training stage is the most mathematically sophisticated component of LongCat-Video. To understand it, we must first establish the flow matching framework and then derive how GRPO modifies the training objective.

Flow matching preliminaries. The base model is trained with flow matching (Lipman et al., 2022), where the forward process (adding noise) is a linear interpolation between the clean latent $x_0$ and Gaussian noise $\epsilon$:

$x_t = (1 - t) \cdot x_0 + t \cdot \epsilon, \quad t \in [0, 1]$

The ground truth velocity is the difference between the endpoints:

$v_t = x_0 - \epsilon$

The model is trained to predict this velocity given the noisy latent, text conditioning, and timestep:

$v_{\text{pred}} = v_\theta(x_t, c, t)$

$\mathcal{L} = \mathbb{E}_{\epsilon, x_0, c, t} \|v_\theta(x_t, c, t) - v_t\|^2$

The reverse-time SDE for sampling. During inference (and GRPO training), samples are generated by discretizing the reverse-time stochastic differential equation (SDE) from $t = 1$ to $t = 0$:

$x_{t-\Delta t} = x_t + \left(v_\theta(x_t, t, c) + \frac{\sigma_t^2}{2t}(x_t + (1-t)v_\theta(x_t, t, c))\right) \Delta t + \sigma_t \sqrt{\Delta t} \, \epsilon$

where $\epsilon \sim \mathcal{N}(0, I)$ and $\sigma_t = a \sqrt{t/(1-t)}$ is the noise schedule with amplitude $a$. The paper sets $a = 1$ during GRPO training, but notes that this aggressive schedule can cause instability at high noise levels (as $t \to 1$, $\sigma_t \to \infty$). They introduce a clipping mechanism:

$\sigma_t \sqrt{\Delta t} \to \min(\sigma_t \sqrt{\Delta t}, \tau)$

where $\tau = 0.45$ in their experiments. When clipping occurs, $\sigma_t$ in the drift term is set to $\tau/\sqrt{\Delta t}$ for consistency.

The GRPO objective for flow matching. The GRPO method (Guo et al., 2025) optimizes the policy (the denoising model) by maximizing a reward-weighted objective with KL regularization. For video generation with flow matching, the objective for a group of $G$ samples $\{x^i\}_{i=1}^G$ from the current policy $\pi_{\theta_{\text{old}}}$ is:

\left[\frac{1}{G}\sum_{i=1}^G \frac{1}{T}\sum_{t=0}^{T-1} \left(L_{\text{policy}}(\theta) - \beta D_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}})\right)\right]$$ **Policy loss (importance-weighted advantage).** The per-timestep policy loss for sample $i$ at timestep $t$ is: $$L_{\text{policy}}(\theta) = r_t^i(\theta) \hat{A}^i_t$$ where $r_t^i(\theta)$ is the **importance ratio**—the ratio of the probability of the transition $x_t \to x_{t-1}$ under the current policy versus the old policy: $$r_t^i(\theta) = \frac{p_\theta(x^i_{t-1} | x^i_t, c)}{p_{\theta_{\text{old}}}(x^i_{t-1} | x^i_t, c)}$$ The transition probability follows a Gaussian distribution because the SDE sampling step adds Gaussian noise: $$p_\theta(x_{t-1} | x_t, c) = \mathcal{N}\left(x_{t-1}; \mu_\theta(x_t, t, c), \sigma_t^2 \Delta t \, I\right)$$ $\hat{A}^i_t$ is the **group-relative advantage**—a normalized reward that measures how much better sample $i$'s final output $x^i_0$ is compared to the average in its group: $$\hat{A}^i_t = \frac{R(x^i_0, c) - \text{mean}(\{R(x^j_0, c)\}_{j=1}^G)}{\text{std}(\{R(x^j_0, c)\}_{j=1}^G)}$$ **KL regularization.** The KL divergence term constrains the policy from deviating too far from the reference model: $$D_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}}) = \frac{\Delta t}{2} \left(\frac{\sigma_t(1-t)}{2t} + \frac{1}{\sigma_t}\right)^2 \|v_\theta(x_t, t, c) - v_{\text{ref}}(x_t, t, c)\|^2$$ where $\beta$ controls the regularization strength (set to $3 \times 10^{-4}$ in GRPO training). **What this objective achieves conceptually.** The GRPO objective pushes the model to generate samples with higher-than-average rewards while staying close to the reference model's behavior. The group-relative advantage normalization is crucial: it means the model is trained to improve relative to other samples from the same prompt, not to achieve some absolute reward threshold. This prevents the model from simply memorizing which prompts tend to get high rewards and instead forces it to learn *how* to generate better videos for each specific prompt. --- #### The Gradient of the Policy Loss and the Need for Reweighting The paper's key contribution to GRPO for video generation is identifying and fixing gradient magnitude issues that cause slow convergence. To understand the problem, we derive the policy loss gradient. **The gradient derivation.** Starting from the policy loss $L_{\text{policy}} = r_t^i \hat{A}^i_t$, the gradient with respect to the model parameters $\theta$ is: $$\nabla_\theta L_{\text{policy}}(\theta) = \hat{A}^i_t \, \nabla_\theta r_t^i(\theta)$$ The importance ratio gradient simplifies to: $$\nabla_\theta r_t^i(\theta) = \nabla_\theta \log p_\theta(x^i_{t-1} | x^i_t, c) \cdot r_t^i(\theta)$$ For the Gaussian transition distribution, the score function (gradient of log-probability) is: $$\nabla_\theta \log p_\theta = \frac{1}{\sigma_t \sqrt{\Delta t}} \epsilon \cdot \nabla_\theta \mu_\theta$$ where $\mu_\theta$ is the mean of the Gaussian distribution predicted by the model. The SDE step reparameterization gives: $$x_{t-1} = \mu_\theta + \sigma_t \sqrt{\Delta t} \, \epsilon$$ Substituting and simplifying (the full derivation occupies Appendix A.1.2): $$\mu_\theta = x_t + \left(v_\theta + \frac{\sigma_t^2}{2t}(x_t + (1-t)v_\theta)\right)(-\Delta t)$$ $$\nabla_\theta \mu_\theta = -\Delta t \cdot \left(1 + \frac{\sigma_t^2(1-t)}{2t}\right) \nabla_\theta v_\theta$$ With the paper's noise schedule choice $\sigma_t^2 = t/(1-t)$ (since $a=1$), the coefficient simplifies to $3/2$: $$1 + \frac{\sigma_t^2(1-t)}{2t} = 1 + \frac{t/(1-t) \cdot (1-t)}{2t} = 1 + \frac{1}{2} = \frac{3}{2}$$ **The un-reweighted policy gradient.** Putting everything together: $$\nabla_\theta L_{\text{policy}}(\theta) = -\frac{3}{2} \hat{A}^i_t \sqrt{\frac{\Delta t(1-t)}{t}} \, \epsilon \cdot \nabla_\theta v_\theta$$ **The problem: the scaling factor $\kappa(t, \Delta t) = \sqrt{\frac{\Delta t(1-t)}{t}}$.** This factor has two problematic properties: 1. **Vanishing gradient as $t \to 1$:** When $t$ is close to 1 (early denoising steps, high noise), $(1-t) \to 0$, so $\kappa(t, \Delta t) \to 0$. The gradient vanishes at the very beginning of the denoising trajectory, where the model's decisions have the most impact on the final output. This means the model barely receives training signal for early-timestep predictions. 2. **Small timestep suppression:** Video generation models use large timestep shifts (concentrating more steps at lower $t$ values), resulting in small $\Delta t$ values. The $\sqrt{\Delta t}$ factor further suppresses gradients at all timesteps, slowing overall convergence. **The reweighting solution.** The paper introduces a reweighting coefficient that is the inverse of the problematic scaling factor: $$\lambda_{\text{policy}}(t, \Delta t) = \kappa(t, \Delta t)^{-1} = \sqrt{\frac{t}{\Delta t(1-t)}}$$ $$L_{\text{policy, reweighted}}(\theta) = \lambda_{\text{policy}}(t, \Delta t) \cdot L_{\text{policy}}(\theta)$$ The reweighted policy gradient becomes: $$\nabla_\theta L_{\text{policy, reweighted}}(\theta) = -\frac{3}{2} \hat{A}^i_t \cdot \epsilon \cdot \nabla_\theta v_\theta$$ **What this achieves.** The reweighting factor completely cancels the $t$ and $\Delta t$ dependence in the gradient magnitude. The model now receives equal-magnitude gradient signals at all timesteps and step sizes, eliminating the vanishing gradient problem. The price is that the effective loss magnitude varies with $t$ and $\Delta t$, but this is acceptable since gradients determine optimization, not loss values. **KL loss reweighting.** The same issue affects the KL divergence term. Its gradient (derived in Appendix A.1.2) is: $$\nabla_\theta D_{\text{KL}}(\theta) = \Delta t \cdot \frac{9}{4} \cdot \frac{1-t}{t} \cdot (v_\theta - v_{\text{ref}}) \cdot \nabla_\theta v_\theta$$ The scaling factor $\Delta t \cdot (1-t)/t$ has the same problems. The paper introduces a corresponding KL reweighting coefficient: $$\lambda_{\text{KL}}(t, \Delta t) = \left(\Delta t \cdot \frac{1-t}{t}\right)^{-1} = \frac{t}{\Delta t(1-t)}$$ $$L_{\text{KL, reweighted}}(\theta) = \lambda_{\text{KL}}(t, \Delta t) \cdot D_{\text{KL}}(\theta)$$ The reweighted KL gradient simplifies to: $$\nabla_\theta L_{\text{KL, reweighted}}(\theta) = \frac{9}{4} \cdot (v_\theta - v_{\text{ref}}) \cdot \nabla_\theta v_\theta$$ **Why these specific reweighting forms were chosen.** The reweighting coefficients are proportional to $1/\sqrt{\Delta t(1-t)/t}$ for the policy loss and $1/(\Delta t(1-t)/t)$ for the KL loss. These are the exact inverses of the scaling factors that multiply $\nabla_\theta v_\theta$ in the gradient expressions. The design principle is: **normalize the gradient to be statistically stationary with respect to $t$ and $\Delta t$.** Without reweighting, the optimization would be dominated by contributions from a narrow range of timesteps (those where $\kappa(t, \Delta t)$ is large), leading to biased policy updates that over-optimize certain denoising stages while neglecting others. With reweighting, all timesteps contribute equally to the gradient direction, leading to more uniform policy improvement across the entire denoising trajectory. **Verification via Figure 7a.** The ablation study in Figure 7a confirms empirically that reweighting significantly improves training. Without reweighting, the reward curves (presumably for visual quality, motion quality, or text alignment) show slower improvement or plateau at lower values. With reweighting, the reward increases more rapidly and reaches higher final values, validating the theoretical analysis. --- #### Selective SDE Timestep for Credit Assignment The standard GRPO formulation applies SDE sampling (with noise injection) at **all** timesteps during trajectory generation. This creates a **temporal credit assignment problem**: the final reward $R(x_0, c)$ is a function of the entire denoising trajectory, but the advantage $\hat{A}^i_t$ at each timestep treats the reward as if it were attributable equally to all timesteps. In reality, some timesteps contribute more to the final video quality than others—early timesteps determine global layout, while later timesteps refine details. **The selective SDE approach.** The paper introduces a modified sampling scheme that isolates reward variation to a single timestep: 1. For each prompt $c$, all $G$ samples in a group share the **same initial noise latent** $x_T \sim \mathcal{N}(0, I)$. This removes variation due to different starting points and ensures that differences in final outputs are due to differences in the denoising path, not the initial condition. 2. A single **critical timestep** $t'$ is randomly sampled from the range $[0, T' - 1]$, where $T'$ is set to **6** (out of 16 total sampling steps). This timestep is the same for all $G$ samples in the group. 3. At the critical timestep $t'$ only, **SDE sampling with noise injection** is used. At all other timesteps, **deterministic ODE sampling** (no noise injection) is used. 4. Different group members $i = 1, \ldots, G$ receive **different noise samples** $\epsilon^i$ at the critical timestep, creating the variation needed for advantage computation. At all non-critical timesteps, all group members follow identical deterministic paths. **The simplified GRPO objective.** With selective SDE timesteps, the objective reduces to focusing only on the critical timestep: $$J_{\text{GRPO-Selective}}(\theta) = \mathbb{E}_{c \sim \mathcal{C}, t' \sim \mathcal{U}(0, T'-1), \{x^i\}_{i=1}^G \sim \pi_{\text{old}}(\cdot|c, t')} \left[\frac{1}{G}\sum_{i=1}^G \left(r_{t'}^i(\theta) \hat{A}^i - \beta D_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}})_{t'}\right)\right]$$ **What this computes differently.** Instead of summing over all $T$ timesteps (the $1/T \sum_{t=0}^{T-1}$ in the original objective), only the critical timestep $t'$ contributes to the gradient. The advantage $\hat{A}^i$ is still computed from the final reward $R(x^i_0, c)$, but the gradient only flows through the model's prediction at timestep $t'$. This means the model learns: "at timestep $t'$, the noise $\epsilon^i$ led to a better/worse outcome—adjust your velocity prediction at $t'$ accordingly." **Why this solves credit assignment.** If SDE noise were injected at all timesteps, the advantage signal would be diluted across the entire trajectory. The model would receive weak gradient signals at all timesteps, making it difficult to distinguish which prediction errors were responsible for the final reward difference. By concentrating noise injection at a single timestep, the causal chain is clear: $\epsilon^i$ at $t' \to$ different $x^i_{t'-1} \to$ downstream trajectory divergence $\to$ different final output $x^i_0 \to$ different reward. The gradient through the policy loss at $t'$ precisely captures how the velocity prediction at $t'$ should change to improve the reward. **The shared initial noise design.** Sharing the same initial noise $x_T$ across all group members is equally important. If different samples started from different initial noise, the final reward difference could be due to the initial condition rather than the policy's behavior at timestep $t'$. By fixing the initial condition, the only source of variation is the noise injected at $t'$, making the credit assignment unambiguous. **The $T' = 6$ choice.** Selecting the critical timestep only from the first 6 out of 16 sampling steps means noise is injected relatively early in the denoising process. This is a deliberate choice: early timesteps control global structure (layout, object placement, broad color schemes), while later timesteps refine details. By focusing credit assignment on early timesteps, the GRPO training encourages the model to make better high-level decisions that cascade into improved overall quality. --- #### Max Group Standard Deviation for Robust Advantage Estimation The standard GRPO advantage calculation normalizes each sample's reward using the mean and standard deviation of its own group: $$\hat{A}^i_t = \frac{R(x^i_0, c) - \text{mean}(\{R(x^j_0, c)\}_{j=1}^G)}{\text{std}(\{R(x^j_0, c)\}_{j=1}^G)}$$ This works well when reward model assessments are reliable and produce meaningful variance across samples within a group. However, reward models are imperfect. In particular, if a group's samples happen to be very similar (all high quality or all low quality), the group standard deviation will be small, making the normalized advantage artificially large. This amplifies gradient updates for prompts where the reward model's discriminability is poor relative to prompts where the reward model's scores meaningfully differentiate samples. **The max-group normalization modification.** The paper replaces the group-specific standard deviation with the **maximum standard deviation across all groups in the batch**: $$\hat{A}^i_{k, t} = \frac{R_k(x^i_0, c_j) - \mu_k}{\sigma_{\max}}$$ where $\mu_k$ is the group mean for reward $k$, and $\sigma_{\max} = \max_j \sigma^j_k$ is the maximum standard deviation across all $B$ groups (each group corresponds to one prompt $c_j$). **What this accomplishes.** Groups with genuinely meaningful reward variation maintain their relative advantage magnitudes (since their $\sigma^j_k$ may be close to $\sigma_{\max}$). Groups with spuriously small reward variation receive **reduced** advantage magnitudes (since their $\sigma^j_k \ll \sigma_{\max}$, so dividing by $\sigma_{\max}$ instead of $\sigma^j_k$ scales down the advantage). This effectively down-weights gradient contributions from prompts where the reward model cannot reliably distinguish good from bad samples, while preserving strong signals from prompts where the reward model provides clear discrimination. **Empirical validation via Figure 7b.** The ablation study shows that max-group normalization leads to more stable training. Without it (using per-group standard deviation), the reward curves show higher variance and potentially lower final rewards, consistent with noisy gradient updates from unreliable advantage estimates. With max-group normalization, the reward curves are smoother and reach higher values, indicating that reducing the influence of noisy advantage estimates improves optimization. --- #### Multi-Reward GRPO Training The paper uses **three specialized reward models** and combines their signals through weighted advantage summation. This is a critical design choice to prevent the **reward hacking** that occurs when optimizing a single metric. **The three reward models:** **1. HPSv3-based visual quality and text alignment.** HPSv3 (Ma et al., 2025b) is a general-purpose human preference score model for images. The paper uses it in two configurations that produce separate reward signals: - **HPSv3-general**: The mean score of all frames, evaluated with the generic prompt "A high-quality image." This isolates visual quality from text alignment, since the prompt is fixed and independent of the video's content. The reward captures whether individual frames look good (sharpness, composition, color, absence of artifacts). - **HPSv3-percentile**: Uses the actual video caption as the evaluation prompt to assess text-video alignment, but takes only the **top 30% of frame scores**. The motivation: temporal changes can cause content inconsistency (e.g., the video shows something not mentioned in the prompt during a transition), producing spuriously low scores. Using only the top 30% of frames mitigates this by focusing on the video's best moments of alignment, assuming that consistent misalignment across all frames is a genuine quality issue but occasional dips are noise. **2. VideoAlign-based motion quality (MQ).** A VideoAlign-based model (Liu et al., 2025b) fine-tuned on an internal annotated dataset specifically for motion quality assessment. The key design choice: **training and inference use grayscale videos** rather than RGB. This forces the model to assess motion characteristics (smoothness, naturalness, trajectory coherence) rather than using color as a shortcut (e.g., learning that "colorful videos tend to be higher quality" rather than actually evaluating motion). The paper provides evidence for this choice in Figure 20 (Appendix C): the grayscale-trained MQ reward model shows **delayed increase in validation loss** compared to the RGB-trained version, indicating better generalization and reduced overfitting. In other words, removing color forces the model to learn motion-relevant features rather than memorizing color-based dataset biases. **3. VideoAlign-based text-video alignment (TA).** Another VideoAlign-based model fine-tuned on internal data, but this one uses the **original RGB input**. Unlike MQ (which should be color-invariant—a smooth motion is smooth regardless of the color palette), text-video alignment requires color information because captions often reference colors, lighting conditions, and visual styles. The model needs to verify that "a red car driving through a green forest" actually shows a red car in a green forest. **Multi-reward combination mechanism.** The key mathematical insight (derived in Appendix A.1.4) is that for flow matching GRPO, the gradient of the total reward-weighted objective with respect to the policy parameters is a weighted sum: $$\nabla_\theta J_{\text{total}} = \sum_{k=1}^n w_k \cdot \frac{dR_k}{d\theta}$$ Since GRPO approximates $dR_k/d\theta \approx -\frac{3}{2} \hat{A}^i_{k,t} \cdot \epsilon \cdot \nabla_\theta v_\theta$, the total gradient becomes: $$\nabla_\theta J_{\text{total}} = -\frac{3}{2} \left(\sum_{k=1}^n w_k \cdot \hat{A}^i_{k,t}\right) \cdot \epsilon \cdot \nabla_\theta v_\theta$$ This means the effective relative advantage for multi-reward optimization is simply the weighted sum of individual advantages: $$L_{\text{policy, multi}}(\theta) = r_t^i(\theta) \cdot \left(\sum_{k=1}^n w_k \cdot \hat{A}^i_{k,t}\right)$$ where each $\hat{A}^i_{k,t}$ is computed independently for reward $R_k$ using group normalization with max-group standard deviation. The weights are set to **$w_k = 1$** for all three rewards (HPSv3-general, HPSv3-percentile, MQ) and the TA reward (the paper's Table 9 specifies equal weighting: HPSv3-general=1, HPSv3-percentile=1, MQ=1, TA=1). **What this computes operationally.** For each generated video sample in the group, three separate reward scores are computed. Each is normalized to a relative advantage (how much better/worse than the group average, scaled by max-group std). These advantages are summed with equal weight to produce a single scalar $\hat{A}^i_{\text{total}}$. The policy loss then uses this combined advantage, meaning the model receives gradient updates proportional to the **aggregate improvement over the group average across all quality dimensions simultaneously**. **Why multi-reward prevents reward hacking (Figure 9).** The paper demonstrates the multi-reward necessity empirically. When training with only HPSv3, the visual quality reward increases (the model learns to produce visually appealing frames), but this comes at the expense of motion—the model generates static or near-static videos because they score highly on visual quality without the risk of motion artifacts. The paper calls this "static tendency induced by HPSv3 hacking." With multi-reward training, the motion reward counteracts this tendency. If the model reduces motion to boost HPSv3 scores, the MQ reward drops, and the combined advantage ($\hat{A}^i_{\text{total}} = \hat{A}^i_{\text{VQ}} + \hat{A}^i_{\text{MQ}} + \hat{A}^i_{\text{TA}}$) may be lower than if it maintained motion quality. The model therefore finds a **Pareto-optimal trade-off** where all three rewards improve together, rather than maximizing one at the expense of others. **Why this form (weighted sum of advantages) rather than weighted sum of raw rewards.** If rewards were summed before normalization, a single high-variance reward could dominate the advantage signal. By normalizing each reward independently (using its own group statistics and max-group std), each dimension contributes proportionally to its discriminability. A reward with high variance across samples (meaning the model's behavior on that prompt meaningfully affects that reward) will naturally have larger-magnitude advantages, while a low-variance reward (all samples get similar scores) will have near-zero advantages and effectively drop out. This is a form of **adaptive weighting** built into the normalization, not an explicit coefficient. --- #### GRPO Training Specification and Infrastructure The GRPO training configuration from Table 9: | Parameter | Value | |---|---| | Group size $G$ | 4 | | Prompts per update | 64 | | Sampling steps $T$ | 16 | | SDE steps range $[0, T' - 1]$ | $[0, 6]$ | | Timeshift | 12 | | Classifier-free guidance (CFG) | 4 | | Learning rate | $1 \times 10^{-4}$ | | LoRA rank (dim) | 128 | | LoRA alpha | 64 | | LoRA layers | Linear layers in all Self-Attention, Cross-Attention, FFN | | Policy loss weight | 1 | | KL loss weight $\beta$ | $3 \times 10^{-4}$ | | HPSv3-general reward weight | 1 | | HPSv3-percentile reward weight | 1 | | MQ reward weight | 1 | | TA reward weight | 1 | | Iterations | ~500 | **LoRA for GRPO.** The GRPO training uses LoRA rather than full fine-tuning. This is a practical choice that enables flexible stacking: the base model remains frozen, and the GRPO-trained LoRA adapter can be loaded or unloaded independently. This means the same base model checkpoint can serve inference with or without RLHF alignment, and the RLHF LoRA can be combined with other LoRAs (e.g., the distillation LoRA) by summing their weight updates. The LoRA layers include all linear projections in self-attention, cross-attention, and FFN blocks—essentially all trainable parameters in the transformer—but with rank 128, the total number of trainable parameters is still a small fraction of the full 13.6B. **Only T2V for GRPO, but generalization observed.** The paper notes: "We employ only Text-to-Video tasks in the GRPO training, and find that the improvements of instruction-following, visual quality and motion quality generalize well to Image-to-Video and Video-Continuation tasks." This is a significant finding—it means the model learns general video quality improvements from T2V RLHF that transfer across tasks, likely because the shared DiT backbone processes all task types through the same attention layers, and the improvements are in the denoising behavior itself rather than task-specific conditioning patterns. **The Algorithm 1 summary.** The complete GRPO training loop (Algorithm 1 in Section 3.3.1) proceeds as: 1. Sample batch of $B = 64$ prompts 2. For each prompt, sample $G = 4$ trajectories using selective SDE sampling (shared initial noise, single critical timestep $t' \sim \mathcal{U}(0, 5)$, SDE at $t'$ only, ODE elsewhere, truncated noise schedule with clipping at $\tau = 0.45$) 3. Compute $n = 3$ rewards for each trajectory (HPSv3-general, HPSv3-percentile, MQ—note the TA reward appears in the weights but not separately in the reward computation since HPSv3-percentile already captures text alignment) 4. For each reward, compute group mean $\mu_k$, collect all group standard deviations across the batch, compute $\sigma_{\max, k}$ 5. Compute relative advantages $\hat{A}^i_k = (R_k(x^i_0, c) - \mu_k) / \sigma_{\max, k}$ 6. Compute weighted total advantage $\hat{A}^i_{\text{total}} = \sum_k w_k \hat{A}^i_k$ 7. Compute reweighted policy loss: $\lambda_{\text{policy}} = \sqrt{t'/T / (\Delta t/T)(1 - t'/T)}$, $L^i_{\text{policy}} = \lambda_{\text{policy}} \cdot r_{t'}^i(\theta) \cdot \hat{A}^i_{\text{total}}$ 8. Compute reweighted KL loss: $\lambda_{\text{KL}} = (t'/T) / ((\Delta t/T)(1 - t'/T))$, $L^i_{\text{KL}} = \beta \lambda_{\text{KL}} \cdot D_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}})$ 9. Total loss: $L^i = L^i_{\text{policy}} - L^i_{\text{KL}}$ 10. Average over batch and group: $L_{\text{total}} = \frac{1}{B \cdot G} \sum_j \sum_i L^i$ 11. Gradient update: $\theta \leftarrow \theta - \eta \nabla_\theta L_{\text{total}}$ --- ## 4. Key Insights and Innovations ### Innovation 1: Reframing All Video Generation Tasks as Video Continuation Enables Native Multi-Task Unification Without Architectural Contortions The dominant assumption in video generation prior to LongCat-Video was that Text-to-Video (T2V), Image-to-Video (I2V), and video extension are fundamentally different tasks requiring either separate models (Wan 2.2-T2V-A14B vs. Wan 2.2-I2V-A14B) or task-specific architectural components (separate encoder pathways, task-identifying tokens, or modality-specific conditioning modules). This fragmentation wasn't arbitrary—it reflected a genuine belief that the conditioning signal differs qualitatively across tasks: text is semantic and abstract, images are spatial and concrete, video clips are spatiotemporal and carry motion priors. LongCat-Video's reframing challenges this assumption at the conceptual level. The observation that all three tasks can be reduced to **"predict N future frames given M preceding frames"**—with T2V being the M=0 case, I2V being M=1, and VC being M>1—is deceptively simple. What makes it an innovation rather than an obvious unification is the demonstration that a single architecture, trained on mixtures of these cases, can achieve competitive or superior performance on all tasks simultaneously without task-specific components. The unified input format `[X_cond, X_noisy]` with corresponding timestep partitioning `[t_cond=0, t_noisy ~ U(0,1)]` is not architecturally novel in its mechanics (concatenation along the temporal axis is the obvious thing to do), but it is conceptually novel in its **rejection of the premise that these tasks require separate handling**. The distinction between "fine-tuning a T2V model to also do I2V" versus "pretraining natively on all tasks" is the key intellectual move. Post-hoc adaptation (which SkyReels-V2 and others pursue) treats multi-task capability as an extension to a fundamentally T2V-centric model. The model's internal representations are shaped primarily by text-conditioned generation; image and video conditioning are learned as perturbations on top of that foundation. Native pretraining treats all tasks as first-class citizens from the beginning, meaning the model learns representations that are inherently flexible about the conditioning modality. This is why the paper emphasizes that I2V and VC quality emerge from the same training process rather than requiring separate fine-tuning stages. The **KV-cache design** for conditioning frames—where `X_cond` attends only to itself and its keys/values are cached across denoising steps—is not just an efficiency trick but reflects a conceptual commitment: conditioning frames are treated as immutable context, not as tokens that should be "updated" by the generation process. This distinguishes LongCat-Video's approach from autoregressive video prediction methods where conditioning frames might interact bidirectionally with generated frames, and from methods that treat all frames uniformly during denoising. The asymmetry (condition tokens influence noisy tokens but not vice versa) encodes the causal structure of video continuation directly into the attention pattern, and the consistency of this pattern between training and inference means there is no train-test distribution shift in the attention mechanism. The **significance beyond raw performance** is that this reframing opens a path toward models that fluidly mix conditioning modalities—a single interface where users can provide any combination of text, images, and video clips and the model generates coherent continuations. The paper's interactive generation examples (Figure 19, bottom rows) demonstrate this fluidity: different segments of the same video are generated with changing text instructions, effectively creating a multi-modal storyboarding capability within a single model. This is not just a performance improvement over separate models; it's a qualitatively different interaction paradigm enabled by the unified architecture. **Evidence:** The model's competitive performance on all three tasks (T2V MOS scores in Figure 14, I2V MOS scores in Figure 16, long-video generation examples in Figure 19) using a single set of weights. The GSB comparison against Wan 2.2-T2V-A14B (Figure 15) shows LongCat-Video leading in overall quality, text-alignment, and motion quality on T2V—the very task Wan was specialized for—while simultaneously supporting I2V and VC that Wan's T2V variant cannot perform. --- ### Innovation 2: Identifying and Mathematically Resolving Gradient Vanishing in Flow Matching GRPO Through Timestep-Specific Reweighting Rather Than Heuristic Stabilization Applying GRPO (or any policy gradient method) to diffusion-based generative models presents a fundamental challenge: the gradient of the policy loss with respect to the denoising model's velocity prediction contains a multiplicative factor that depends on the sampled timestep $t$ and step size $\Delta t$. The paper's derivation reveals this factor to be $\kappa(t, \Delta t) = \sqrt{\Delta t(1-t)/t}$ for flow matching with the standard noise schedule. This factor has two pathological behaviors that are **inherent to the flow matching formulation, not artifacts of implementation**: it vanishes as $t \to 1$ (early denoising steps, high noise), and it is suppressed by small $\Delta t$ (which video generation models necessarily use due to timestep shifting). The field's prior responses to unstable GRPO training in diffusion models were largely heuristic: gradient clipping, learning rate tuning, careful initialization, or restricting the range of timesteps used for training. The concurrent work the paper cites (He et al., 2025; Zhou et al., 2025) began addressing credit assignment in flow matching GRPO, but the gradient magnitude issue—the fact that the loss function itself produces gradients of fundamentally different scales at different timesteps—was not previously characterized. This matters because it means GRPO training is **biased toward updating the model's behavior at intermediate timesteps** (where $\kappa(t, \Delta t)$ is largest) while neglecting early-timestep predictions that control global video structure. The paper's innovation is not the idea of reweighting per se—importance weighting and loss reweighting are standard tools in RL and generative modeling. It is the **analytical derivation of the exact scaling factor** and the recognition that this factor is the root cause of slow convergence in video GRPO, not just a nuisance to be tuned around. The reweighting coefficients $\lambda_{\text{policy}} = \sqrt{t / (\Delta t(1-t))}$ and $\lambda_{\text{KL}} = t / (\Delta t(1-t))$ are the exact inverses of the problematic factors in the gradient expressions, meaning the reweighted losses produce **timestep-agnostic gradient magnitudes**: $$\nabla_\theta L_{\text{policy, reweighted}} = -\frac{3}{2} \hat{A}^i_t \cdot \epsilon \cdot \nabla_\theta v_\theta$$ The $t$ and $\Delta t$ dependence is completely eliminated. This is a theoretical result—it follows from the flow matching formulation and the choice of noise schedule, not from empirical tuning—and it provides a principled explanation for why naive GRPO converges slowly. The reweighting doesn't change what the model optimizes; it changes **how the optimization signal is distributed across timesteps**, making all timesteps contribute equally to the gradient. The **significance beyond raw performance** is that this analysis provides a diagnostic framework for future work applying RL to diffusion models. When a new RL-for-diffusion method is proposed, one should ask: does the gradient magnitude depend on $t$ and $\Delta t$? If so, is that dependence intentional (e.g., weighting early timesteps more heavily) or an artifact of the formulation that will cause training instability? The paper's reweighting approach gives future practitioners a tool for ensuring that their training objectives produce well-behaved gradients regardless of the timestep sampling distribution. **Distinguishing incremental from fundamental:** This is a **fundamental** contribution to the theory of RL for diffusion models, not an incremental empirical trick. The derivation is mathematical, the problem it solves is inherent to the formulation, and the solution is exact (not a tuned hyperparameter). That said, the paper's empirical demonstration is limited to video generation with flow matching and a specific noise schedule ($a=1$, $\sigma_t = \sqrt{t/(1-t)}$). Different noise schedules would produce different scaling factors, and the reweighting would need to be re-derived. This doesn't diminish the conceptual contribution—the framework for analyzing and correcting gradient magnitude issues transfers—but it means the specific reweighting formulas are tied to the paper's modeling choices. **Evidence:** Figure 7a shows the ablation comparing GRPO training with and without reweighting. The reweighted variant shows faster reward improvement and higher final values across all three reward dimensions. The theoretical derivation occupies Appendix A.1.2 and culminates in Equations (38) and (42), which show the simplified, timestep-independent gradient forms. --- ### Innovation 3: Multi-Reward GRPO as a Mechanism for Preventing Reward Hacking Through Mutual Constraint, Not Just Improving Aggregate Metrics The standard approach to RLHF for generative models is to train a single reward model that captures an aggregate of human preferences (e.g., a model that predicts overall video quality on a 1-5 scale) and optimize against it. This is conceptually simple but creates a well-known vulnerability: the policy can **hack the reward model** by finding outputs that score highly according to the reward model's (imperfect) assessment without actually being high-quality by human standards. In video generation, the most salient form of this is the model learning to produce static or near-static videos because they avoid motion artifacts and look visually clean frame-by-frame, even though a human would penalize the lack of motion. The paper's multi-reward approach is distinctively clever not because it uses multiple reward models—multi-objective RL and constrained optimization are well-established—but because the **mechanism by which it prevents reward hacking is emergent rather than explicit**. The paper does not use a constraint (e.g., "maximize visual quality subject to motion quality > threshold") or a Pareto optimization procedure. Instead, it simply sums the normalized advantages from three independently trained reward models with **equal weights**, and the prevention of reward hacking emerges from the fact that the three rewards are **naturally in tension** for certain degenerate strategies: - Optimizing HPSv3-general alone incentivizes static frames (no motion artifacts, clean composition) - Optimizing HPSv3-percentile alone incentivizes frames that match the text but may be temporally inconsistent - Optimizing the grayscale MQ reward alone incentivizes dynamic motion but may produce visually unattractive frames The equal-weighted sum of advantages means that a strategy that boosts one reward at the expense of another will see its total advantage signal **decrease** compared to a strategy that maintains balance. This is a form of implicit regularization—the model discovers balanced improvement because imbalanced strategies are punished by the combined advantage signal, not because an explicit penalty is imposed. The **intellectual contribution** is the recognition that **reward hacking is fundamentally a problem of insufficient constraint**, and that adding independent reward signals that are sensitive to different aspects of quality provides those constraints without requiring careful tuning of trade-off weights. The paper's demonstration that the MQ reward, trained on grayscale videos specifically to prevent color-based shortcuts, counteracts the static tendency induced by HPSv3 optimization (Figure 9) is a clean experimental validation of this principle. The grayscale design choice for the MQ reward model is itself notable: it's a deliberate architectural decision to **remove information** (color) from the reward model's input, forcing it to learn motion-relevant features. This is the opposite of the typical approach of giving reward models as much information as possible and hoping they learn the right thing. **Significance beyond raw performance:** This finding generalizes beyond video generation. Any domain where a single reward metric can be gamed through degenerate strategies—which is essentially all domains where reward models are learned rather than specified—can benefit from decomposing quality into independently assessable dimensions with naturally countervailing incentives. The paper's approach requires no special optimization algorithm (simple weighted sum of advantages works), no careful weight tuning (equal weights suffice), and no adversarial training of the reward models against the policy. It's a **conceptual simplification** of the multi-objective RL problem that makes it practical to deploy. **Distinguishing incremental from fundamental:** This is a **conceptual advance** (the "mutual constraint through natural tension" framing) supported by careful engineering (grayscale MQ reward model design, top-30% HPSv3-percentile to handle temporal content inconsistency). The individual components—HPSv3 for visual quality, VideoAlign for motion and alignment—are not novel. What is novel is the demonstration that their naive combination, with the right reward model design choices, naturally prevents reward hacking without explicit constraints. **Evidence:** Figure 9 shows the comparison between HPSv3-only training (where visual quality reward increases but motion quality stagnates or declines—the "static tendency") and multi-reward training (where all three rewards improve simultaneously). Figure 8 (in the main text) shows the GRPO reward curves for all three rewards steadily increasing throughout multi-reward training, with no evidence of one reward improving at another's expense. Figure 20 (Appendix C) validates the grayscale design choice for MQ by showing delayed overfitting compared to RGB training. --- ### Innovation 4: The Coarse-to-Fine Refinement Expert as a Flow Between Distributions Rather Than a Super-Resolution Module The standard approach to generating high-resolution content from diffusion models is to either (a) generate at the target resolution directly, paying the full computational cost, or (b) use a separately trained super-resolution model that takes the low-resolution output as conditioning and generates a high-resolution version. Cascaded diffusion models (e.g., DALL-E 2, Imagen) exemplify approach (b): a base model generates at low resolution, and a separate super-resolution diffusion model upsamples the result. LongCat-Video's coarse-to-fine strategy looks superficially similar to cascaded generation but is conceptually different in a way that matters for efficiency and knowledge transfer. The refinement expert is not a separate super-resolution model trained from scratch; it is a **LoRA adapter on the frozen base model** trained to learn a flow between two distributions: the distribution of upsampled low-resolution videos and the distribution of high-resolution videos. The key mathematical difference is in the noise level: rather than training the refiner to denoise from pure Gaussian noise ($t_{\text{thresh}} = 1$) to the target distribution, the paper sets $t_{\text{thresh}} = 0.5$, meaning the refiner's task is to correct a video that is already **structurally complete but lacking high-frequency detail**. This is a fundamentally different learning problem. A full super-resolution model must learn to hallucinate plausible high-frequency details starting from either pure noise (if cascaded from noise) or a low-resolution conditioning signal (if conditioned on the low-res output). The refinement expert, by contrast, starts from a noisy version of the upsampled low-resolution output and only needs to learn the **correction**—the difference between "trilinearly upsampled 480p" and "native 720p." Because $x_{\text{thresh}}$ at $t_{\text{thresh}} = 0.5$ is already 50% weighted toward the upsampled output, the model starts close to the target in pixel space and only needs to refine. The **intellectual contribution** is the recognition that LoRA fine-tuning on a base model at partial noise level is a more efficient way to achieve high-resolution generation than training a separate super-resolution model, because: (1) the base model already knows how to generate high-quality video—the refiner only needs to learn the high-frequency correction; (2) LoRA adapters capture this correction with very few parameters, making training fast and memory-efficient; (3) the same base model can serve both coarse and fine stages, simplifying deployment; (4) the flow matching formulation at partial noise level ($t_{\text{thresh}}=0.5$, only 5 sampling steps) means the refiner does very little work—most of the generation is done by the coarse stage. **Why this is not just cascaded generation with a smaller model.** In cascaded generation, the super-resolution model is typically a full model with its own architecture, trained on a different objective (often conditional diffusion from low-res to high-res), and runs for many sampling steps. The LongCat-Video refiner is a LoRA adapter on the same model, trained on the flow matching objective but with a truncated noise range, and runs for only 5 steps. The training data filtering (GLCM-based texture richness filter, degradation augmentations for robustness) further targets the refinement task specifically. The result is that the refiner is not learning to generate images—it's learning to **polish** them. **The quality improvement observation is counterintuitive.** The paper notes that C2F generation produces **better** visual quality than native 720p generation (Figure 10), not just equivalent quality at lower cost. This is surprising: one would expect that generating at lower resolution and upsampling would lose detail, not gain it. The explanation likely involves the **regularization effect of the coarse stage**: generating at 480p forces the model to establish coherent global structure without being distracted by high-frequency details, and the refinement stage then fills in details consistent with that structure. Native 720p generation must simultaneously establish global layout and fine details, which can lead to locally plausible but globally inconsistent artifacts. The refinement expert also **can correct local distortions** from the coarse generation (Figure 10 explicitly shows this), acting as a quality assurance step. **Significance beyond raw performance:** This finding suggests a general principle for diffusion model deployment: **decompose generation into a structural phase (low resolution, more steps) and a detail phase (high resolution, fewer steps, closer to the target), using a lightweight adapter for the detail phase.** This is more parameter-efficient than separate super-resolution models and better quality than generating at target resolution directly. It's a design pattern that can be applied to any diffusion model, not just video. **Distinguishing incremental from fundamental:** This is an **incremental but important** architectural contribution. The individual pieces (LoRA, flow matching, two-stage generation) are all established techniques. The innovation is in the specific formulation of the refinement task as a partial-noise flow matching problem solved by a LoRA adapter, and the empirical demonstration that this formulation is both more efficient and higher quality than alternatives. **Evidence:** Table 2 shows the efficiency gains: C2F enables 10.6× speedup over native 720p generation. Figure 10 shows the quality improvement—the C2F 720p output has sharper textures and fewer artifacts than native 720p generation. The refinement expert's training specifications (Table 7: only 500 iterations at full attention + 500 iterations with BSA) demonstrate how quickly the adapter converges. ## 5. Experimental Analysis ### Evaluation Methodology - **Dataset.** The paper evaluates on two fronts: an internal benchmark comprising 1,628 samples (1,228 Text-to-Video cases—500 human-evaluated and 728 automatically evaluated—plus 400 Image-to-Video cases leveraging 100 reference images each paired with four distinct prompt types), and the public VBench 2.0 benchmark (Zheng et al., 2025). The internal T2V human evaluation prompts span 48 distinct categories with bimodal prompt length distribution (34.8% ≤20 words, 34.6% ≥51 words, range 4–121 words), while the automatic subset draws from T2VCompbench (Sun et al., 2025), MovieGen (Polyak et al., 2024), and in-house prompts. I2V evaluation uses 100 reference images spanning diverse styles (photorealism, ink wash, 2D/3D animation, oil painting, sketch), content categories, aspect ratios (1:1, 16:9, 9:16), and resolutions (720p, 1080p, 2K). - **Base model(s).** LongCat-Video is a 13.6B-parameter DiT model (Table 1: 48 layers, hidden size 4096, FFN size 16384, 32 attention heads). For T2V comparisons, baselines include the proprietary Veo3 (Google), PixVerse-V5 (PixVerse), and the open-source Wan 2.2-T2V-A14B (Wan et al., 2025). For I2V comparisons, baselines include Seedance 1.0 (Gao et al., 2025), Hailuo-02 (MiniMax), and Wan 2.2-I2V-A14B. On VBench 2.0, additional comparisons include HunyuanVideo (Kong et al., 2024), Sora-480p (OpenAI), Kling1.6 (Kuaishou), Vidu Q1 (Shengshu), and Seedance 1.0 Pro. - **Metrics.** - **Internal benchmark (human evaluation):** Two complementary protocols are used. **MOS (Mean Opinion Score)** employs a 5-point scale with detailed descriptors per scoring tier, with each video independently annotated by three annotators and additional annotators introduced when significant discrepancies arise. The final score per video averages all annotator ratings. **GSB (Good-Same-Bad)** uses pairwise comparative assessment providing discriminative model rankings. The final score per model dimension is a weighted (2:1) average of human and automatic evaluation scores. - **Internal benchmark (automatic evaluation):** A vision-language judge model is trained on high-quality human-annotated data to quantitatively evaluate text-alignment, visual quality, and motion quality. This judge achieves correlations exceeding 0.92 with human assessments across all dimensions (Section 5.1, "Quality Control"). - **Evaluation dimensions for T2V:** Text-Alignment (comprehensiveness and semantic accuracy), Visual Quality (plausibility—adherence to physical principles, absence of distortion; realism—authenticity of scenes and subjects), Motion Quality (trajectory coherence, action smoothness, realistic behavior for human/object/camera motion, avoidance of prolonged stillness or excessive jitter), Overall Quality (comprehensive aggregate). - **Evaluation dimensions for I2V:** The above four dimensions plus Image-Alignment (faithful preservation of subject/background attributes and relationships from the reference image, maintenance of original style). - **VBench 2.0:** Standard dimensions including Creativity, Commonsense, Controllability, Human Fidelity, Physics, and Total Score, with scores reported as percentages. The Commonsense dimension evaluates motion rationality and physical law adherence. The Physics dimension separately assesses physical simulation quality. - **Baselines.** For T2V internal evaluation: Veo3 (proprietary, Google), PixVerse-V5 (proprietary), Wan 2.2-T2V-A14B (open-source, Wan et al., 2025). For I2V internal evaluation: Seedance 1.0 (proprietary, Gao et al., 2025), Hailuo-02 (proprietary, MiniMax), Wan 2.2-I2V-A14B (open-source). For VBench 2.0 public evaluation: HunyuanVideo, Wan2.1, Sora-480p, Kling1.6, Vidu Q1, Seedance 1.0 Pro, and Veo3 (all cited in Table 8 with evaluation dates ranging from 2025-03 to 2025-09). - **Generation budget / compute accounting.** The paper does not report a standardized "generation budget" in the sense of FLOPs or number of function evaluations across compared models. Each model is evaluated using its own native inference pipeline at comparable output resolutions and durations. LongCat-Video uses LCM-distilled 16-step sampling for the base stage and 5-step refinement (Table 2). The paper does not specify the sampling budgets used for competing models—this is a limitation (see Critical Assessment). Efficiency comparisons (Table 2) are reported for LongCat-Video only, measured in seconds on a single H800 GPU with FlashAttention3, not compared against other models' inference times. - **Cross-validation / statistical protocol.** Human evaluation uses three independent annotators per video with a consensus-based approach: when significant discrepancies exist between any two annotations, two additional annotators reassess the video, and the final score averages all involved annotators' ratings (Section 5.1, "Quality Control"). This is a rigorous quality control protocol but is not a cross-validation procedure in the machine learning sense—it ensures annotation reliability rather than evaluating generalization across data splits. No statistical significance tests (confidence intervals, p-values) are reported for any comparison. For VBench 2.0, standard benchmark evaluation protocols are followed (Table 8 reports single-point scores without error bars). The paper does not describe train/validation/test splits for the internal benchmark construction. ### Main Quantitative Results #### Text-to-Video Internal Benchmark Results The MOS evaluation (Figure 14) positions LongCat-Video competitively across all dimensions. In **Overall Quality**, LongCat-Video achieves a score of **3.38**, surpassing both PixVerse-V5 (3.35) and Wan 2.2-T2V-A14B (3.36), while trailing Veo3 (3.48). In **Text-Alignment**, LongCat-Video scores **3.76**, above PixVerse-V5 (3.70) and slightly below Wan 2.2-T2V-A14B (3.81) and Veo3 (3.99). In **Visual Quality**, LongCat-Video achieves **3.25**, nearly matching Wan 2.2-T2V-A14B (3.26), exceeding PixVerse-V5 (3.13), and trailing Veo3 (3.23). In **Motion Quality**, LongCat-Video scores **3.74**, behind Veo3 (3.86) and Wan 2.2-T2V-A14B (3.81) but ahead of PixVerse-V5 (3.78). The GSB evaluation (Figure 15) uses pairwise preferences to provide discriminative rankings. **LongCat-Video vs. Veo3:** Veo3 is preferred in Overall Quality (345 vs. 121, with 35 ties), Text-Alignment (282 vs. 151, 68 ties), Visual Quality (188 vs. 162, 151 ties), and Motion Quality (332 vs. 121, 35 ties). This is the largest gap—Veo3 maintains a clear lead across all dimensions. **LongCat-Video vs. PixVerse-V5:** The comparison is nearly tied in Overall Quality (LongCat-Video preferred 242, PixVerse-V5 preferred 246, 13 ties). LongCat-Video leads in Visual Quality (214 vs. 148, 139 ties) while PixVerse-V5 leads in Text-Alignment (206 vs. 133, 162 ties) and Motion Quality (206 vs. 133, 162 ties—note these identical numbers suggest possible reporting artifact). **LongCat-Video vs. Wan 2.2-T2V-A14B:** LongCat-Video is clearly preferred in Overall Quality (250 vs. 54, with 15 ties), Text-Alignment (212 vs. 117, 172 ties), and Motion Quality (219 vs. 175, 107 ties). Visual Quality shows Wan 2.2 slightly preferred (95 vs. 54, but with a large tie count of 352, indicating most comparisons were indistinguishable). The key takeaway: LongCat-Video unambiguously surpasses the previous open-source state-of-the-art (Wan 2.2-T2V-A14B) and is competitive with mid-tier proprietary models (PixVerse-V5), though Veo3 retains a clear advantage. #### Image-to-Video Internal Benchmark Results The I2V MOS evaluation (Figure 16) reveals a different performance profile. LongCat-Video achieves the highest **Visual Quality** score (3.27), exceeding Seedance 1.0 (3.22), Hailuo-02 (3.18), and Wan 2.2-I2V-A14B (3.23). However, it trails on other dimensions: **Image-Alignment** scores 4.04 for LongCat-Video versus 4.12 (Seedance), 4.18 (Hailuo), and 4.18 (Wan 2.2). **Text-Alignment** scores 3.49 versus 3.70 (Seedance), 3.85 (Hailuo), and 3.33 (Wan 2.2). **Motion Quality** scores 3.59 versus 3.77 (Seedance), 3.80 (Hailuo), and 3.79 (Wan 2.2). In **Overall Quality**, LongCat-Video scores 3.17, behind Seedance 1.0 (3.35), Hailuo-02 (3.27), and Wan 2.2-I2V-A14B (3.26). The paper acknowledges this gap explicitly: "This suggests that while our model excels in visual fidelity, there is room for improvement in maintaining temporal consistency and alignment with the source image" (Section 5.1). The I2V results are the weakest link in LongCat-Video's evaluation profile—the model ranks last among the four compared systems on Image-Alignment, Text-Alignment, Motion Quality, and Overall Quality, with only Visual Quality showing a (narrow) lead. #### VBench 2.0 Public Benchmark Results Table 8 reports VBench 2.0 scores for eight models. LongCat-Video achieves a **Total Score of 62.11%**, ranking third behind Veo3 (66.72%) and Vidu Q1 (62.70%). It leads all compared models on the **Commonsense** dimension at **70.94%**, exceeding Veo3 (69.48%), Vidu Q1 (65.98%), and all others. On **Creativity**, LongCat-Video scores 54.73% (third behind Veo3's 60.85% and Sora-480p's 60.57%). On **Controllability**, it scores 44.79% (third behind Veo3's 47.04% and Seedance 1.0 Pro's 39.84%). On **Human Fidelity**, it scores 80.20% (fifth, behind Sora's 87.72%, Veo3's 86.88%, Kling1.6's 83.56%, and HunyuanVideo's 82.41%). On **Physics**, it scores 59.92% (sixth, substantially behind Vidu Q1's leading 71.63%). The Commonsense leadership is notable and aligns with the paper's emphasis on long video generation and physical coherence. However, the Physics score—which one might expect to correlate with commonsense reasoning about physical laws—is not competitive (sixth place), suggesting the two dimensions capture different aspects of physical understanding. #### Efficiency Results Table 2 presents inference latency measurements on a single H800 GPU. The baseline (native 720p × 93 frames, 50 sampling steps, no optimizations) requires **1429.5 seconds (~24 minutes)**. LCM distillation alone (16 steps) reduces this to 244.6s (5.8× speedup). Adding C2F (480p × 93 frames → 720p × 93 frames, 16/5 steps) achieves 135.3s (10.6× speedup). Adding BSA to C2F reaches **116.5s (12.3× speedup)** for 93-frame output. For the longer 189-frame configuration, C2F alone requires 302.9s (4.7× speedup vs. native 720p at 93 frames—note the comparison is across different frame counts, so the speedup factor is relative to the unreported baseline for 189 frames), while C2F+BSA achieves **142.0s (10.1× speedup)**. All measurements use FlashAttention3 (Shah et al., 2024). The paper does not report comparable latency numbers for competing models, so the efficiency advantage is demonstrated only as an improvement over LongCat-Video's own unoptimized inference, not as a comparison against other systems. ### Ablation Studies and Robustness Checks - **Policy and KL loss reweighting (Figure 7a):** The ablation compares GRPO training with and without the reweighting coefficients $\lambda_{\text{policy}}$ and $\lambda_{\text{KL}}$ described in Section 3.3.1. The figure shows that reweighting leads to faster reward improvement and higher final reward values across all three reward dimensions (visual quality, motion quality, text-video alignment). Without reweighting, the reward curves show slower convergence and plateau at lower values, consistent with the theoretical analysis that the un-reweighted gradients vanish at high noise levels and are suppressed by small step sizes. - **Max group standard deviation (Figure 7b):** The ablation compares using per-group standard deviation (standard GRPO) versus the maximum standard deviation across all groups (the paper's modification) for advantage normalization. Using max-group standard deviation produces smoother reward curves and higher final values, indicating that reducing gradient weight from groups with spuriously small reward variance (due to reward model inaccuracies) stabilizes training. - **Single-reward vs. multi-reward GRPO (Figure 9):** This is a critical ablation demonstrating reward hacking. Training with only HPSv3 causes the visual quality reward to increase while the motion quality reward stagnates or potentially decreases—the "static tendency" where the model generates high-quality but motionless frames because HPSv3 evaluates frame-level aesthetics independent of temporal dynamics. Multi-reward training (HPSv3 + MQ + TA) results in simultaneous improvement across all three reward curves, with no evidence of one reward improving at another's expense. The MQ reward counteracts the static tendency induced by HPSv3 optimization. - **Grayscale vs. RGB MQ reward model training (Figure 20, Appendix C):** The MQ reward model's validation loss curves are compared for grayscale and RGB training. The grayscale-trained model shows a **delayed increase in validation loss** compared to the RGB-trained version, indicating reduced overfitting and improved generalization. The paper interprets this as evidence that removing color information forces the reward model to learn motion-relevant features rather than memorizing color-based dataset biases. - **Block sparse attention configurations (Appendix A.2.3):** The paper reports that "no significant differences in post-training results were observed across various tested configurations of 3D block sizes," with $t_q \cdot h_q \cdot w_q$ in [64, 128] and $t_k \cdot h_k \cdot w_k$ in [64, 128, 256, 512, 1024]. This robustness suggests the exact block geometry is not critical—the sparsity ratio and trainability are the key factors. Additionally, the paper explored two block selection strategies: top-r (adopted for trainable scenarios due to predictable compute cost) and CDF-p (which yields better quality in training-free settings but has variable per-query block counts). - **ReST^EM revision model ablation (Appendix K, Figure 16):** An attempt to further optimize the revision model using ReST$^{\text{EM}}$ (Singh et al., 2024) **substantially hurt performance**. The paper notes: "At 256 generations, fully sequential performance drops to approximately 33.5% compared to roughly 38.5% at the optimal ratio." The authors hypothesize that on-policy data collection in ReST$^{\text{EM}}$ exacerbates spurious correlations in revision data. This is a notable negative result that highlights the sensitivity of revision training to the data generation procedure—offline data construction with edit-distance-based pairing worked, while online RL-style optimization degraded performance. - **PRM aggregation strategy (Appendix E, Figure 13):** Comparing "last," "min," and "prod" step-wise PRM score aggregation, the "last" strategy (using only the final step's prediction) achieves the best performance (~37% at 256 samples vs. ~35% for "min" and ~27% for "prod"). This is notable because prior work (Lightman et al., 2023; Wang et al., 2023) found "min" to be superior. The authors attribute the discrepancy to their PRM being trained with soft Monte Carlo labels rather than binary correctness labels. - **PRM vs. ORM (Appendix F, Figure 14):** The PRM consistently outperforms a separately trained outcome reward model, with the gap widening at higher sample counts (PRM reaches ~40% at 2048 samples vs. ORM's ~35%). Despite using last-step aggregation (which effectively reduces the PRM to ORM-like behavior at aggregation time), the PRM training procedure provides better representations. - **Revision model verifier choice (Appendix J, Figure 15a):** The base-LM PRM underperforms a revision-specific ORM when scoring revision model outputs (sequential + base-LM PRM achieves ~40% at 64 generations vs. sequential + revision ORM at ~42%), confirming distribution shift as a practical concern. Including revision history in the ORM's context provides a small improvement (~1-2 percentage points at 64 generations, Figure 15b), but both variants outperform the parallel baseline, confirming the sequential benefit is not solely attributable to the verifier seeing more context. - **Oracle vs. predicted difficulty bins (Figures 4, 8, and Appendix C, Figures 11-12):** Both oracle and predicted difficulty bins yield qualitatively similar trends. Predicted bins show slightly lower performance at high budgets in the revision setting (~41% vs. ~44% at 256 generations in Figure 8) but essentially identical performance in the search setting (Figure 4). This establishes that the compute-optimal strategy works without ground-truth labels. - **Majority voting for revisions (Appendix B, Figure 10):** The sequential-to-parallel ratio trends observed with verifier-based selection are replicated with majority voting, indicating the finding is robust to the selection mechanism. ### Critical Assessment #### Claim 1: LongCat-Video achieves performance comparable to leading open-source and latest commercial solutions. **What was tested:** On the internal T2V benchmark, LongCat-Video was compared against two proprietary models (Veo3, PixVerse-V5) and one open-source model (Wan 2.2-T2V-A14B) using MOS and GSB evaluation protocols. On VBench 2.0, it was compared against seven additional models spanning open-source and proprietary systems. **What the evidence shows:** The claim is **partially supported with important qualifications**. On T2V, LongCat-Video clearly surpasses Wan 2.2 (the previous open-source leader) in GSB overall quality (250 vs. 54 preferences), supporting the "leading open-source" portion. However, "commercial solutions" spans a wide range: the model is competitive with PixVerse-V5 (nearly tied in GSB overall quality at 242 vs. 246) but substantially trails Veo3 (345 vs. 121 preferences against, a decisive gap). On VBench 2.0, LongCat-Video's 62.11% total score is behind both Veo3 (66.72%) and Vidu Q1 (62.70%), making it third among the eight compared models. The claim of comparable performance must therefore be qualified: the model matches mid-tier commercial solutions but not the top tier. **Strengths of the evaluation:** The internal benchmark's evaluation protocol is rigorous—three independent annotators per video with consensus-based additional annotation, detailed per-tier scoring descriptors, a 2:1 weighted combination of human and automatic assessments, and a carefully constructed prompt taxonomy spanning 48 categories with controlled diversity (bimodal prompt length distribution, long tail of specialized tasks). The GSB protocol provides discriminative pairwise rankings that complement the coarser MOS scores. **Weaknesses and missing evidence:** 1. **I2V results contradict the claim.** On the I2V benchmark, LongCat-Video ranks **last** among four compared models on Overall Quality, Image-Alignment, Text-Alignment, and Motion Quality (Figure 16). The only dimension where it leads is Visual Quality (3.27 vs. 3.18–3.23). For a model that claims to unify T2V, I2V, and VC tasks, the I2V underperformance is a significant weakness that the paper acknowledges but does not explain mechanistically. Why does the unified architecture that works well for T2V perform relatively poorly for I2V? Is there a trade-off in the multi-task training that favors T2V at the expense of I2V? The paper does not investigate this. 2. **No comparison on Video-Continuation.** Despite long video generation being a flagship capability, the paper provides **no quantitative evaluation of Video-Continuation quality**. The long-video examples (Figure 19) are qualitative demonstrations—stills from minutes-long videos with no metrics for temporal consistency, color drift, or quality degradation over time. Without a controlled comparison against SkyReels-V2 (Chen et al., 2025) or other long-video generation methods on a standardized benchmark, the claim of "minutes-long videos without color drifting or quality degradation" remains anecdotal. A simple evaluation—generating N-minute videos and having annotators rate temporal consistency at 0s, 30s, 60s, etc.—would have substantiated this central claim. 3. **No statistical significance reporting.** MOS scores are reported as single-point estimates without confidence intervals. GSB counts are reported without statistical tests (e.g., binomial tests to determine if preference ratios differ significantly from chance). For the internal benchmark's 500 human-evaluated T2V prompts, differences of a few preference counts (e.g., Overall Quality 242 vs. 246 for LongCat-Video vs. PixVerse-V5) could easily be within sampling error. Without error bars or significance tests, fine-grained rankings among closely-scored models are unreliable. 4. **No inference cost comparison with baselines.** The efficiency claims (12.3× speedup) are self-referential—comparing LongCat-Video's optimized pipeline to LongCat-Video's unoptimized pipeline. Without reporting or estimating the inference cost of Veo3, Wan 2.2, or PixVerse-V5 on comparable hardware, it is unclear whether LongCat-Video is genuinely more efficient or simply starting from a slower baseline. If Wan 2.2 achieves similar quality in 60 seconds (as opposed to LongCat-Video's 116.5s), the speedup claim is misleading. 5. **Single GPU measurement.** Efficiency is measured on a single H800 GPU. Production deployments of video generation models typically use multi-GPU inference for latency-sensitive applications. It is unclear how the BSA and C2F strategies scale with additional GPUs, or whether the reported 12.3× speedup holds in distributed settings. #### Claim 2: The coarse-to-fine generation strategy not only reduces inference cost but also improves generation quality. **What was tested:** Figure 10 shows a qualitative comparison between native 480p, native 720p, and C2F 720p generation. No quantitative metrics are reported for this comparison. **What the evidence shows:** The claim is **weakly supported**—based on a single qualitative example. The C2F output in Figure 10 does appear sharper than the native 720p output, and the paper notes it "can also correct local distortions." However, one example does not establish a general quality improvement. A proper test would evaluate C2F vs. native 720p across the full test set using human or automatic quality metrics. The mechanism proposed (coarse stage establishes structure, refinement adds details without structural distraction) is plausible but unverified at scale. This claim should be treated as an intriguing observation requiring further validation. #### Claim 3: Multi-reward GRPO prevents reward hacking and improves all quality dimensions simultaneously. **What was tested:** Figure 9 compares single-reward (HPSv3-only) GRPO training with multi-reward GRPO training. Figure 8 shows reward curves for all three rewards improving simultaneously during multi-reward training. **What the evidence shows:** The claim is **well-supported** within the scope of the experiment. The HPSv3-only training clearly demonstrates reward hacking (visual quality reward increases while motion quality stagnates), and the multi-reward training clearly shows simultaneous improvement across all three rewards without trade-offs. This is a clean ablation. **Weaknesses and missing evidence:** 1. **Reward curves only, no final generation quality comparison.** Figure 9 shows reward model scores, not human evaluations of the resulting videos. The logical chain is: reward scores increase → actual video quality improves. But reward models are imperfect, and it is possible that the multi-reward model is learning to game all three reward models simultaneously (a more subtle form of reward hacking) rather than genuinely improving. A human evaluation comparing base model outputs, single-reward GRPO outputs, and multi-reward GRPO outputs would close this loop. The MOS and GSB evaluations in Figures 14–15 appear to use the GRPO-trained model, but there is no ablation showing MOS scores with and without GRPO. 2. **The "prevention of reward hacking" claim depends on the specific reward models used.** If a future practitioner uses different reward models where the natural tension is absent (e.g., all reward models correlate strongly), the implicit regularization might not emerge. The paper doesn't discuss what properties of the reward model set are necessary for this approach to work, which limits the generalizability of the finding. 3. **TA reward role unclear.** Table 9 lists TA reward weight = 1 alongside the other three rewards (HPSv3-general, HPSv3-percentile, MQ), but the Algorithm 1 description and reward definitions suggest TA might be redundant with HPSv3-percentile (which already evaluates text-video alignment using the video caption). If TA and HPSv3-percentile are highly correlated, their equal weighting effectively double-counts text alignment, potentially skewing the multi-reward balance. The paper does not report correlations between reward signals or ablate the inclusion/exclusion of individual rewards. #### Claim 4: Block sparse attention achieves near-lossless generation quality while retaining less than 10% of attention computation. **What was tested:** The paper states that "By retaining less than 10% of the original computational load, we can achieve near-lossless generation quality." Table 2 shows speedup numbers. The BSA implementation is described in detail (Appendix A.2). **What the evidence shows:** The claim is **inadequately supported by reported evidence**. Table 2 demonstrates the speedup (4.7× for 189-frame C2F without BSA vs. 10.1× with BSA, implying BSA provides approximately 2.2× additional speedup over C2F alone for longer sequences), but there is **no quantitative comparison of generation quality with and without BSA**. The "near-lossless" claim is based on the authors' qualitative assessment from training ("once the loss converges and stabilizes, we activate BSA to continue training," Section 4.3), but no metrics (FID, FVD, CLIP similarity, human evaluation) are reported for BSA vs. dense attention. For a technique described as a key contribution and open-sourced for community use, the absence of a quality comparison is a significant gap. #### Claim 5: The GRPO reweighting and selective timestep strategies significantly enhance convergence speed and generation quality. **What was tested:** Figures 7a and 7b show reward curves for ablations of reweighting and max-group normalization. Figure 6 shows a qualitative comparison of base model vs. GRPO outputs. **What the evidence shows:** The claim is **well-supported by reward curve evidence** but **qualitatively demonstrated for generation quality**. The reward curves clearly show faster convergence and higher final values with the proposed techniques. Figure 6 shows visually improved outputs. The theoretical derivation connecting the scaling factor $\kappa(t, \Delta t)$ to the reweighting coefficients is mathematically sound and provides a principled justification. **Weaknesses:** 1. **No human evaluation of the GRPO improvement magnitude.** How much does GRPO actually improve human-perceived quality? The MOS evaluation (Figure 14) uses the GRPO-trained model, but without a "no GRPO" baseline in the MOS comparison, the marginal contribution of GRPO is unknown. If the base model already achieves, say, 3.30 Overall Quality MOS and GRPO improves it to 3.38, that's a modest gain for the added complexity. Without this baseline, it's impossible to assess whether GRPO is essential or merely helpful. 2. **Interaction between reweighting and selective timestep is not ablated.** The paper introduces both techniques as a package. Does reweighting alone solve the convergence problem, with selective timestep providing additional benefit? Or does selective timestep make reweighting unnecessary? The individual contributions are not disentangled. #### Missing Experiments That Would Have Strengthened the Paper 1. **Quantitative long-video generation benchmark:** A systematic evaluation of temporal consistency over time (e.g., CLIP similarity between frames at increasing temporal distance, human ratings of quality at 0s/30s/60s/120s, comparison with SkyReels-V2 or other long-video methods). 2. **Ablation of multi-task training:** Comparison of the unified model against (a) a T2V-only model of the same architecture, (b) separate T2V + I2V + VC models trained independently, to quantify whether multi-task training helps or hurts individual task performance. The I2V underperformance suggests there may be negative transfer, but without this ablation it's speculative. 3. **BSA quality ablation:** FVD, CLIP similarity, or human evaluation comparing dense attention vs. BSA at the same resolution and frame count, at multiple sparsity levels, to establish where "near-lossless" breaks down. 4. **GRPO vs. no-GRPO human evaluation:** Adding the base model (pre-GRPO) and the single-reward GRPO model to the MOS/GSB comparisons in Figures 14–15. 5. **Inference cost comparison with baselines:** Estimating or measuring the GPU-seconds required by competing models to produce comparable output, to contextualize the 12.3× speedup claim. 6. **Statistical significance on GSB comparisons:** Binomial tests to determine whether preference ratios (e.g., 250 vs. 54 for LongCat-Video vs. Wan 2.2 overall quality) are statistically distinguishable from 50-50. 7. **I2V debugging experiments:** Given the I2V underperformance, experiments to diagnose the cause—e.g., does I2V quality improve with more I2V training data? Does the condition frame handling (KV-cache, block attention) disadvantage I2V in some way? Does the C2F pipeline damage condition frame fidelity during refinement? #### Summary of Evaluation Rigor The internal benchmark evaluation protocol is **strong for T2V**—large prompt set, diverse categories, rigorous human annotation with quality control, complementary MOS and GSB metrics, and a well-correlated automatic judge model. The VBench 2.0 evaluation adds standardized public benchmark coverage. However, **critical gaps exist**: (1) no quantitative evaluation of Video-Continuation/long-video quality despite this being a central claim, (2) I2V evaluation reveals underperformance that is acknowledged but not investigated, (3) no statistical significance testing, (4) key ablations (BSA quality, GRPO marginal contribution, multi-task training benefit) are missing or rely on qualitative examples, and (5) efficiency comparisons are self-referential rather than comparative. The paper demonstrates a capable model but does not provide the evidence needed to fully validate its strongest claims about long-video generation quality, GRPO's practical benefit magnitude, or BSA's quality preservation. ## 6. Limitations and Trade-offs ### Long Video Generation Claims Are Qualitatively Demonstrated but Quantitatively Unevaluated **The assumption or constraint.** The paper's central distinguishing capability—generating "minutes-long videos without color drifting or quality degradation"—is presented as a flagship achievement enabled by native pretraining on Video-Continuation tasks. Section 1 states: > "LongCat-Video is natively pretrained on Video-Continuation tasks, enabling it to produce minutes-long videos without color drifting or quality degradation." Section 3.2 describes the architectural mechanism (unified input representation, block attention with KV-cache) and Section 2 notes that conditional frames are perturbed with per-frame independent noise levels during VC training "to enhance robustness to color drift." However, the paper provides **no quantitative evaluation of Video-Continuation quality**. The long-video generation examples in Figure 19 are qualitative still frames at timestamps (0:15, 0:30, 0:45, 1:00, etc.), and the interactive generation examples show frames from 0–6s, 6–11s, etc.—indicative of the capability but not measuring it. **The consequence.** Without quantitative evaluation, the claim of "without color drifting or quality degradation" is unverified. A practitioner considering LongCat-Video for applications requiring minutes-long temporal coherence (digital humans, embodied AI, film production) has no way to assess: (a) whether quality degrades measurably over time, (b) at what duration degradation becomes perceptible, (c) how LongCat-Video compares to alternative long-video generation approaches like SkyReels-V2 (Chen et al., 2025, cited in the paper) or autoregressive sliding-window methods. The per-frame noise perturbation during VC training (Section 3.2) is a sensible design choice to mitigate color drift, but its effectiveness is not measured. The absence of evaluation is particularly problematic because Section 2.2 explicitly mentions "targeted data supplementation or rebalancing strategies" for Video-Continuation training data—suggesting data composition matters for this capability—yet no experimental validation connects data choices to generation quality over long horizons. **What evidence exists in the paper.** Only qualitative examples: Figure 1 (top row: frames at 0:15, 0:30, 0:45, 1:00 from a "long" generation; bottom rows: interactive generation segments at 0–6s, 6–11s, etc.), Figure 19 (additional long-video and interactive generation frame sequences). No metrics are reported for temporal consistency (e.g., CLIP embedding similarity between frames at increasing temporal distance), no human evaluation of long-video coherence, no comparison with other long-video systems, and no ablation showing the effect of the per-frame noise perturbation on color drift over time. **Mitigation status.** The paper does not acknowledge this as a limitation. The future work section (Section 6) mentions "better modeling of physical knowledge, multi-modal memory integration in video generation" but does not identify long-video quantitative evaluation as a missing component, nor does it compare against existing long-video generation methods on a common benchmark. This is a significant gap given that efficient long video generation is positioned as the key capability toward world models (Section 1, Section 6). --- ### Image-to-Video Performance Is Consistently Below Competing Models Despite Unified Architecture Claims **The assumption or constraint.** The paper claims that LongCat-Video "unifies Text-to-Video, Image-to-Video, and Video-Continuation tasks within a single video generation framework" (Section 1) and that the unified architecture "natively supports all these tasks and delivers strong performance across them" (Section 1). The architectural mechanism—treating all tasks as video continuation with varying numbers of conditioning frames—is described in Section 3.2 as the key enabler of this unification. The implicit promise is that multi-task training produces a model that is competent across all supported tasks. **The consequence.** The I2V evaluation (Section 5.1, Figure 16) reveals that LongCat-Video ranks **last among four compared models** on Overall Quality (3.17 vs. 3.35 Seedance 1.0, 3.27 Hailuo-02, 3.26 Wan 2.2-I2V-A14B), Image-Alignment (4.04 vs. 4.12–4.18), Text-Alignment (3.49 vs. 3.33–3.85), and Motion Quality (3.59 vs. 3.77–3.80). The only dimension where it leads is Visual Quality (3.27 vs. 3.18–3.23). A practitioner choosing LongCat-Video for an I2V-dominant workflow (e.g., keyframe-animated content creation, reference-image-driven video generation) would be selecting a model that is **measurably worse at the core I2V dimensions of image fidelity preservation and motion coherence** than the open-source Wan 2.2-I2V-A14B and proprietary alternatives. The paper acknowledges this explicitly: > "This suggests that while our model excels in visual fidelity, there is room for improvement in maintaining temporal consistency and alignment with the source image." but provides no mechanistic explanation or diagnostic experiment. **What evidence exists in the paper.** Only the MOS evaluation in Figure 16 with four models on the 400-sample I2V benchmark. No ablation studies investigate the cause: Is the I2V underperformance due to the multi-task training diluting I2V-specific capabilities? Due to the C2F pipeline degrading condition frame fidelity during refinement? Due to the block attention mechanism disadvantaging single-frame conditioning relative to multi-frame? The paper does not report training data ratios for T2V vs. I2V vs. VC, so it is unclear whether I2V is underrepresented in the training mixture. The GSB evaluation (Section 5.1) is reported only for T2V (Figure 15), not for I2V, meaning we cannot assess whether the MOS differences are statistically discriminable in pairwise comparisons. **Mitigation status.** The paper acknowledges the performance gap in passing (Section 5.1) but treats it as an observation rather than a limitation requiring investigation. The multi-task training strategy (Table 3: T2I + T2V + I2V + VC jointly from the third stage onward) is presented as a strength, but the I2V results suggest possible **negative transfer** between tasks—the model may be trading off I2V-specific capabilities (precise image-conditioned generation, fine-grained visual attribute preservation) for improvements in T2V and VC. The paper does not ablate whether a separately trained I2V-only variant of the same architecture would outperform the unified model on I2V tasks, which would be the critical diagnostic experiment for assessing the true cost of unification. --- ### GRPO Training Uses Only Text-to-Video; Cross-Task Generalization Is Assumed but Untested **The assumption or constraint.** Section 4.2 states: > "We employ only Text-to-Video tasks in the GRPO training, and find that the improvements of instruction-following, visual quality and motion quality generalize well to Image-to-Video and Video-Continuation tasks." This claim of cross-task generalization is unsupported by any evaluation. The paper provides no quantitative comparison of I2V or VC outputs before and after GRPO training—no MOS scores, no reward model scores, no human preferences. The "find that" language implies an empirical observation, but the only GRPO evaluations (Figures 8, 9; the reward curves) are on T2V outputs. The I2V MOS evaluation in Figure 16 presumably uses the GRPO-trained model (since it is the final released model), but without a pre-GRPO I2V baseline, the marginal contribution of GRPO to I2V quality is unknown. **The consequence.** A practitioner using LongCat-Video for I2V or VC applications cannot assume that the GRPO stage improves performance on their task of interest. If the improvements are T2V-specific—for instance, the motion quality reward model may encourage motion patterns that are natural for text-prompted generation but inappropriate for image-conditioned generation where the reference image constrains plausible motion—GRPO could potentially **degrade** I2V or VC performance on certain dimensions. The assumption that improvements generalize across tasks is plausible given the shared DiT backbone, but the I2V underperformance already documented in Figure 16 makes it difficult to evaluate whether GRPO helped, hurt, or had no effect. Proposing task-specific rewards for each task is listed as future work (Section 4.2: "Proposing task-specific rewards for each task (e.g. quality degradation penalty of long-video generation for Video-Continuation) remains a future work"), which implicitly acknowledges that the current single-task GRPO setup is incomplete. **What evidence exists in the paper.** None. No pre-GRPO vs. post-GRPO comparison exists for any task other than T2V. The qualitative examples (Figures 17, 18, 19) show final model outputs but provide no comparison with the base model before GRPO, making it impossible to visually assess improvement. **Mitigation status.** The paper acknowledges the limitation indirectly by listing task-specific rewards as future work (Section 4.2), but does not acknowledge the absence of evidence for the cross-task generalization claim. A minimal evaluation—computing the three reward model scores on a held-out set of I2V and VC outputs before and after GRPO training—would have provided partial evidence. Human evaluation of GRPO's effect on I2V, at least for a small sample, would have been more definitive. --- ### Difficulty Estimation and Strategy Selection Overhead Is Not Accounted for in Efficiency Claims **The assumption or constraint.** The paper is not explicit about this limitation, but the efficiency claims (Table 2: 12.3× speedup, "720p, 30fps videos within minutes") have an implicit assumption: that the inference pipeline can be executed on a single H800 GPU as measured, and that the reported latency numbers represent the end-to-end cost a practitioner would experience. However, the efficiency measurements in Table 2 exclude several overheads that a production deployment would incur: 1. **VAE encode/decode cost.** The C2F pipeline involves VAE encoding of input conditioning frames, VAE decoding of the low-resolution output to RGB for trilinear upsampling, and VAE re-encoding of the upsampled video back to latent space for refinement. The paper does not specify whether these encode/decode operations are included in the latency numbers in Table 2. They can be substantial for high-resolution, high-frame-count video. 2. **KV-cache overhead.** The block attention mechanism (Section 3.2) caches condition frame keys and values across denoising steps. For Video-Continuation with many conditioning frames (minutes of preceding video), the KV-cache memory footprint grows linearly with the number of conditioning frames. The paper does not discuss memory constraints or report peak GPU memory usage during long-video generation. 3. **No latency comparison with competing models.** The 12.3× speedup is measured against LongCat-Video's own unoptimized native 720p generation (50 steps, no LCM, no C2F, no BSA). This is a self-referential comparison, not a comparison against the inference speed of Wan 2.2, CogVideoX, or proprietary systems. If competing models already achieve similar quality in similar wall-clock time using their own optimizations, the claimed speedup advantage does not translate to a practical deployment advantage. **The consequence.** A practitioner estimating total cost of ownership for deploying LongCat-Video cannot rely on Table 2 alone. The actual end-to-end latency—including I/O for loading conditioning frames, VAE encode/decode, KV-cache management, and any post-processing—may be significantly higher than the reported numbers, especially for long-video generation where conditioning frame count grows with video duration. For the "minutes-long video" use case, the KV-cache for the conditioning frames could exceed available GPU memory, forcing either offloading strategies (which add latency) or limiting generation length. The paper provides no guidance on memory requirements or maximum supported conditioning length. **What evidence exists in the paper.** Table 2 reports latency for specific configurations (93 or 189 frames) with varying optimization combinations, all measured on a single H800 GPU with FlashAttention3. The footnote states "*The tests were conducted on a single H800 GPU with FlashAttention3," providing hardware specifications but not peak memory usage, I/O time, or whether VAE operations are included. There is no scaling curve showing latency growth as conditioning frame count increases from 0 (T2V) to 1 (I2V) to hundreds (long VC). There is no comparison with inference latency of Wan 2.2, CogVideoX, or any baseline model on the same hardware. **Mitigation status.** The paper does not acknowledge these overheads as limitations. Efficiency is presented as a solved problem ("within minutes" is the headline claim), but the reported numbers represent a lower bound on achievable latency in idealized conditions. Future work should report end-to-end latency including all preprocessing, postprocessing, and I/O, measure peak memory usage during long-video generation, and compare inference speed against open-source baselines on identical hardware to provide a meaningful efficiency benchmark. --- ### Single Benchmark Coverage and Single Model Family Limit Generalizability of Scaling Claims **The assumption or constraint.** All evaluations are conducted using a single model architecture (48-layer DiT at 13.6B parameters, Table 1) on a single type of benchmark: video generation quality assessed through human MOS/GSB ratings (internal benchmark) and the VBench 2.0 automated metrics. The VBench 2.0 evaluation (Table 8) provides broader coverage, but the paper does not evaluate on other standard video generation benchmarks such as UCF-101 (for established FVD comparisons), MSR-VTT, or activity recognition benchmarks that would test whether generated videos contain semantically meaningful actions. The internal benchmark's I2V evaluation uses only 100 reference images with four prompt types each (400 total evaluations), which is a small sample for assessing fine-grained image-conditioned generation quality. **The consequence.** The performance claims—leading open-source T2V, competitive with mid-tier commercial solutions—are valid only within the specific evaluation ecosystem the paper constructs. Without evaluation on widely-used third-party benchmarks with established leaderboards (beyond VBench), it is difficult for practitioners to contextualize LongCat-Video's performance relative to the broader field. The VBench 2.0 scores in Table 8 include models evaluated at different dates (2025-03 to 2025-10), and the field moves quickly—models released after LongCat-Video's cutoff may achieve higher scores, which the paper cannot account for. More fundamentally, the paper's central argument—that native Video-Continuation pretraining enables better long-video generation than post-hoc fine-tuning—is evaluated only on LongCat-Video itself. Without an ablation comparing the same DiT architecture trained with and without VC pretraining, or a comparison against SkyReels-V2 (the long-video method cited in Section 1), the claim that native pretraining is the enabling factor for long-video quality cannot be attributed to the architectural choice rather than scale, data, or training recipe differences. The I2V evaluation, in particular, relies on 100 reference images paired with four prompt types (detailed, concise, contradictory, empty)—a design that tests robustness to prompt variation but with a small image sample that may not represent the diversity of real-world I2V use cases (varying aspect ratios, resolutions, content types, artistic styles). The paper describes the I2V benchmark taxonomy in detail (Section 5.1, "Data Taxonomy for Image-to-Video Evaluation"), but the absolute number of evaluations (400) is modest for drawing strong comparative conclusions, especially when the MOS differences between models are small (e.g., Overall Quality: 3.17 LongCat-Video vs. 3.26 Wan 2.2-I2V-A14B, a 0.09 difference on a 5-point scale). **What evidence exists in the paper.** Table 8 provides VBench 2.0 coverage, and Figures 14–16 provide internal benchmark coverage. The total evaluation budget is 1,628 samples across T2V (1,228) and I2V (400). This is reasonable for a technical report but limited for establishing generalizable performance claims. The paper does not evaluate on other video understanding or generation benchmarks (EvalCrafter, FETV, VideoCon, Make-A-Video-Eval) that could test specific capabilities like temporal consistency, object permanence, or text-video alignment through complementary lenses. **Mitigation status.** The paper partially acknowledges scope limitations through the future work section (Section 6), which mentions "better modeling of physical knowledge, multi-modal memory integration in video generation, and the incorporation of knowledge from LLM and MLLM." However, the paper does not acknowledge the narrow evaluation scope as a limitation of the current work. Expanding evaluation to additional benchmarks, conducting the ablation comparing native VC pretraining against post-hoc long-video fine-tuning of the same architecture, and increasing the I2V evaluation sample size would strengthen the generalizability of the findings. ## 7. Implications and Future Directions ### How This Work Changes the Landscape LongCat-Video represents a **reframing with incremental but practically significant advances** rather than a paradigm shift. Its most consequential repositioning is the demonstration that treating all video generation tasks as video continuation with varying conditioning-frame counts is not merely a taxonomic convenience but an architectural strategy that produces a single model competitive with task-specialized alternatives. Before this work, the default assumption was that T2V, I2V, and video extension required either separate models or task-specific architectural components. The paper shows that this fragmentation is unnecessary—a single DiT trained on mixed conditioning-frame counts can match or exceed task-specialized open-source models on T2V while simultaneously supporting I2V and VC, even though I2V underperformance (Figure 16, ranking last in Overall Quality) indicates that unification is not cost-free. The paper's second repositioning concerns **what constitutes "efficient" video generation inference**. Prior work on efficient video diffusion focused primarily on reducing sampling steps (LCM, consistency models) or model compression (pruning, quantization). LongCat-Video demonstrates that the larger efficiency gains come from **restructuring the generation problem itself**: generating at low resolution to establish structure, then refining at high resolution with a LoRA adapter at partial noise level. The 10.6× speedup from C2F alone (Table 2: 1429.5s → 135.3s for 720p × 93 frames) dwarfs the 5.8× from LCM distillation, establishing that problem decomposition—not just faster sampling—is the dominant efficiency lever. Combined with BSA's additional 2× improvement (135.3s → 116.5s), the total 12.3× speedup redefines what practitioners should expect from optimized video generation pipelines. The paper's third contribution is a **diagnostic framework for RLHF in diffusion models** rather than a single method. The identification of the gradient scaling factor κ(t, Δt) = √(Δt(1−t)/t) as the root cause of slow GRPO convergence for flow matching models provides a conceptual tool that generalizes beyond video generation. Any future work applying policy gradient methods to flow matching can immediately check whether their objective's gradient depends on t and Δt in this way, and apply the paper's reweighting approach to normalize it. The clear naming of the pathology—vanishing gradients at high noise levels, small-step suppression—gives the field vocabulary for diagnosing training instability in diffusion RL, shifting the discussion from "GRPO is slow/unstable for diffusion" (a frustrating empirical observation) to "the gradient magnitude is timestep-dependent and requires reweighting" (a solvable mathematical problem). The paper also partially reconciles a tension in the video generation literature: the observation that longer training on larger datasets improves quality but that training on mixed-resolution, mixed-task data can hurt individual task performance. LongCat-Video's progressive training strategy (Table 3: T2I → T2V → multi-task, with learning rate decay across stages) demonstrates a recipe for navigating this trade-off—start with easier tasks at lower resolution to establish representations, then expand to harder tasks at higher resolution while lowering the learning rate to avoid catastrophic interference. The I2V underperformance (Figure 16) suggests the recipe is not yet optimal—the learning rate or data mixture in the multi-task stages may need adjustment—but the framework for thinking about the problem (staged curriculum rather than uniform mixed training) is a useful conceptual contribution. **Research directions that become more attractive:** - **Unified multi-modal generation architectures.** The paper's success with a single DiT for T2V, I2V, and VC suggests extending the "everything is continuation" framing to other modalities—text-to-audio, image-to-3D, video-to-audio—where conditioning length varies but the generative task is structurally similar. The KV-cache design for conditioning frames (Section 3.2) transfers directly to any setting with fixed conditioning tokens. - **LoRA-based refinement for any diffusion model.** The C2F approach—generate coarse, refine with LoRA at partial noise—is architecture-agnostic and cheap to implement (500 iterations of full attention + 500 iterations of BSA training, per Table 7). Every image and video diffusion model could adopt this to trade a small amount of training compute for a large inference speedup. - **Multi-reward RLHF with naturally countervailing reward models.** The paper's demonstration that equal-weighted advantage summation prevents reward hacking without explicit constraints (Figure 9) makes multi-reward RLHF more accessible. The key design insight—choose reward models that are naturally in tension for degenerate strategies—provides a selection criterion that future practitioners can apply. **Research directions that become less attractive:** - **Training separate super-resolution diffusion models for video.** The C2F approach with a LoRA refiner at partial noise is both more efficient and higher quality (Figure 10) than cascaded super-resolution. Future work should default to LoRA-based refinement unless there are strong reasons to train a separate model. - **Single-reward RLHF for video generation.** Figure 9 demonstrates that single-reward optimization (HPSv3-only) leads to reward hacking (static tendency). This is likely a general phenomenon for video, where visual quality and motion quality are naturally in tension. Future RLHF for video should default to multi-reward approaches. ### Follow-Up Research This Work Enables **1. Quantifying the unification trade-off: at what data mixture does multi-task training stop being beneficial?** The I2V underperformance (Figure 16: last in Overall Quality, Image-Alignment, Text-Alignment, and Motion Quality among four compared models) is the most important unresolved question in the paper. The unified architecture is elegant, but if practitioners must choose between a single model that is "pretty good at everything" and separate models that are "excellent at one thing," the practical value of unification depends on the magnitude of the trade-off. A direct follow-up would train LongCat-Video architecture variants with different T2V:I2V:VC data ratios (e.g., 50:25:25, 33:33:33, 80:10:10) at fixed total compute, evaluate all variants on the same T2V and I2V benchmarks used in the paper, and produce a Pareto frontier showing how I2V quality trades off against T2V quality as a function of data mixture. The hypothesis worth testing: the current I2V underperformance is due to T2V-dominant data mixture rather than inherent negative transfer, and rebalancing can recover I2V performance without substantial T2V degradation. If rebalancing cannot close the gap, the field needs to understand what makes image conditioning fundamentally harder to learn jointly with text conditioning. **2. Measuring temporal consistency degradation in long video generation with automated metrics.** The paper's central claim—minutes-long video without color drift or quality degradation—is qualitatively demonstrated (Figures 1, 19) but quantitatively unevaluated. A rigorous follow-up would generate videos of increasing duration (15s, 30s, 60s, 120s, 240s) from the same set of prompts, then compute CLIP embedding similarity between frame pairs at increasing temporal separation (1s, 5s, 15s, 30s, 60s apart) to quantify semantic drift. Additionally, compute per-frame aesthetic scores (using the same HPSv3 model from the paper's reward suite) as a function of frame index to detect quality degradation over time. Compare LongCat-Video against SkyReels-V2 (Chen et al., 2025, cited in the paper) and an autoregressive sliding-window baseline using Wan 2.2 to establish whether native Video-Continuation pretraining genuinely outperforms post-hoc long-video adaptation methods. A negative result—finding that quality degrades measurably after 60–90 seconds despite the native pretraining—would bound the approach's capability and motivate the "multi-modal memory integration" mentioned in Section 6. **3. BSA quality preservation: establishing the sparsity-versus-quality curve with rigorous metrics.** The paper claims "near-lossless generation quality" with BSA at <10% attention computation but provides no quantitative evidence (no FVD, CLIP similarity, or human evaluation comparing BSA against dense attention). A necessary follow-up measures video quality as a function of BSA sparsity level r/Nₖ ∈ [1/32, 1/16, 1/8, 1/4, 1/2, 1] on a fixed test set. Report FVD (Fréchet Video Distance) and CLIP similarity between BSA-generated and dense-attention-generated videos from the same prompts and initial noise. This would establish the sparsity threshold where quality begins to measurably degrade—critical information for practitioners trading off speed against quality. Additionally, measure whether the quality-sparsity relationship is resolution-dependent: does BSA degrade quality more at 720p than 480p, where tokens are sparser to begin with? The paper's report that different block geometries (Appendix A.2.3) show no significant quality differences is reassuring but based on qualitative assessment—verifying this with metrics would increase confidence in BSA's robustness. **4. Diagnosing GRPO's marginal contribution to human-perceived quality across tasks.** The paper reports GRPO reward curves (Figures 8, 9) but never evaluates GRPO's effect using the human MOS/GSB protocols that form the main evaluation. A targeted follow-up runs the internal benchmark's human evaluation protocol (Section 5.1: three annotators per video, 5-point MOS, GSB preferences) on three model variants: base model (pre-GRPO), single-reward GRPO (HPSv3 only), and multi-reward GRPO (the released model). Report MOS and GSB scores for all three on T2V, and additionally on I2V to test the claim that GRPO improvements generalize across tasks. The critical question: does the 0.5k-iteration GRPO stage (Table 5) produce a human-perceptible improvement, and is the multi-reward variant perceptibly better than single-reward? If the MOS improvement is <0.1 on the 5-point scale, the value of the complex GRPO setup (three reward models, reweighted losses, selective SDE timesteps, max-group normalization) relative to its engineering cost needs reassessment. If the improvement is substantial on T2V but negligible on I2V, the cross-task generalization claim is weakened and task-specific rewards become a higher priority. **5. Scaling the number of GRPO iterations: does performance saturate or continue improving?** The paper's GRPO training runs for only ~500 iterations (Table 5: 0.5k iterations, 64 prompts per update, group size 4, meaning approximately 128,000 generated samples total). This is a small RLHF budget by language model standards (where GRPO runs for thousands to tens of thousands of iterations). A natural follow-up trains GRPO for 2k, 5k, and 10k iterations on the same reward models and evaluates MOS scores at each checkpoint. The hypothesis would distinguish between two regimes: (1) GRPO improvements saturate quickly (500 iterations is near-optimal, and the reward models are the bottleneck), or (2) GRPO continues to improve with more iterations, and the current model is undertrained in the RL phase. The answer has direct implications for compute allocation: if saturation is fast, future work should invest in better reward models; if improvement is ongoing, simply running longer GRPO is the most cost-effective path to higher quality. **6. Combining PRM-guided search with the revision model for Video-Continuation quality improvement.** Section 3.2 describes perturbing conditioning frames with per-frame independent noise levels during VC training to enhance robustness to color drift. A follow-up could explore whether applying PRM-guided search at the boundaries between video segments—using the PRM to score candidate continuations and select the one most consistent with preceding frames—further reduces quality degradation in long video generation. The PRM from the earlier sections (trained with Monte Carlo rollout supervision, using last-step aggregation) could score candidate 5-second continuations based on temporal consistency with the previous segment. A comparison of vanilla autoregressive continuation versus PRM-guided continuation on a metric like CLIP similarity between adjacent segments would quantify the benefit. The negative result—finding that PRM search does not improve over simple autoregressive continuation—would suggest that the noise-perturbation training (Section 3.2) already provides sufficient robustness, and the bottleneck for long video generation lies elsewhere (perhaps in the model's memory capacity rather than segment-boundary consistency). **7. Stress-testing the max-group standard deviation normalization under reward model distribution shift.** The max-group normalization trick (Equation 9, Figure 7b) improves training stability by down-weighting gradient contributions from prompts where reward models produce spuriously low variance. A stress-test would evaluate this technique when reward models are deliberately degraded—e.g., adding increasing levels of Gaussian noise to reward scores, or using a reward model trained on a different data distribution. Measure the GRPO reward curves and final policy quality (via MOS) as a function of reward model noise level, comparing per-group vs. max-group normalization. If max-group normalization maintains stable training at noise levels where per-group normalization collapses, the technique becomes a recommended default for any GRPO application where reward model reliability varies across prompts. If the benefit disappears above moderate noise levels, the technique's practical value is limited to the specific reward model quality achieved in this paper. ### Practical Applications and Downstream Use Cases **1. Video content creation pipelines requiring mixed-modality creative control.** The unified architecture (single model for T2V, I2V, VC) enables creative workflows where a content creator starts with a text prompt to generate a keyframe or establishing shot, uses Image-to-Video to animate specific frames with precise visual control, and then uses Video-Continuation with changing text instructions for interactive storytelling—all within a single model serving infrastructure. This eliminates the need to switch between separate models with different quality characteristics, inference APIs, and output formats. The interactive generation examples in Figure 19 (segments at 0–6s, 6–11s, 11–16s, 17–22s with changing instructions) demonstrate this workflow concretely. For a video production studio, the practical benefit is reduced operational complexity: one model weights file, one inference server, one set of quality tuning, and one API integration rather than three. The C2F efficiency (116.5s for 720p × 93 frames on a single H800, Table 2) makes this practical on commodity hardware rather than requiring datacenter-scale GPU clusters. **2. Simulation and data generation for embodied AI and autonomous driving.** The paper's positioning of video generation as a pathway to world models (Section 1) has immediate practical implications for domains where collecting real-world training data is expensive or dangerous. LongCat-Video's ability to generate minutes-long, temporally coherent videos enables simulation of extended driving scenarios (cars moving, pedestrians crossing, weather changing) or robot manipulation sequences with consistent physics. The VBench 2.0 Commonsense score (70.94%, leading all models including Veo3 at 69.48%, Table 8) suggests the model captures physical regularities better than competing systems. For an autonomous driving perception team, this means generating synthetic training data with diverse scenarios and controlled variations (different weather, lighting, traffic patterns) at a fraction of the cost of real-world data collection. The key number: generating one minute of 720p, 30fps video requires approximately 142 seconds on an H800 (extrapolating from the 189-frame, ~6.3-second, 142-second latency in Table 2), meaning approximately 24 GPU-minutes per minute of synthetic video—economical for offline data generation pipelines. **3. Rapid prototyping of film pre-visualization.** In film production, pre-visualization (creating rough animated versions of scenes to plan camera angles, blocking, and timing) traditionally requires days of work by 3D artists. LongCat-Video's coarse-to-fine pipeline with interactive generation (changing text instructions for sequential video segments) enables a director or cinematographer to prototype a scene by describing it in natural language, with the model generating a 480p pre-vis at 16 steps (61.3 seconds for 93 frames, Table 2) for quick iteration, then refining to 720p for client review. The ability to use Image-to-Video with a reference frame (e.g., a location photo or concept art) ensures visual consistency with the production's aesthetic. The benefit is not in generating final-quality footage but in compressing the pre-vis iteration cycle from days to minutes, enabling more creative exploration before committing resources to full production. **4. Open-source research platform for video generation RLHF.** By releasing model weights, code, and the block sparse attention implementation, LongCat-Video provides the research community with a complete, reproducible video generation RLHF pipeline at a scale (13.6B parameters) that was previously accessible only to industrial labs. The concrete value for academic researchers: the multi-reward GRPO setup (Section 3.3.2, Table 9) with three well-defined reward models (HPSv3-general, HPSv3-percentile, grayscale MQ VideoAlign) can be replicated to study reward hacking, multi-objective trade-offs, and credit assignment in video RLHF without building the entire infrastructure from scratch. The GRPO training runs for only ~500 iterations (approximately 128K generated samples), making it feasible for academic compute budgets. A research group could, for example, swap in their own reward model, train for a comparable number of iterations, and evaluate whether their reward model design choices improve human-perceived quality—all on a shared baseline architecture and training recipe. The BSA implementation (open-sourced Triton kernels, Appendix A.2.3) similarly lowers the barrier for academic groups to experiment with sparse attention in video models without implementing hardware-aligned attention kernels themselves. ### When to Prefer This Method The paper positions LongCat-Video primarily as an open-source alternative to proprietary systems rather than as a method that should be preferred over specific named alternatives in well-defined scenarios. It does not propose a clear tradeoff matrix (e.g., "use our GRPO approach rather than DPO when ..."). However, several implicit preference conditions emerge from the evaluation results and architectural choices: **Prefer LongCat-Video's unified architecture when:** - The deployment requires all three tasks (T2V, I2V, VC) and maintaining separate models would be operationally complex. The single-model approach reduces serving infrastructure, checkpoint management, and API surface area. - T2V is the dominant use case, with I2V and VC as occasional but necessary capabilities. The T2V performance (competitive with Wan 2.2-T2V-A14B in MOS, preferred in GSB overall quality) is strong, and the I2V capability comes "for free" even if performance is not state-of-the-art. - Long-video generation (minutes) is required and the deployment can tolerate the latency implied by Table 2 (142 seconds for 189 frames at 720p with C2F+BSA). **Prefer task-specialized models (Wan 2.2-I2V-A14B, Seedance 1.0, Hailuo-02) when:** - Image-to-Video is the primary or only use case. Figure 16 shows LongCat-Video trails all three compared models on Overall Quality, Image-Alignment, Text-Alignment, and Motion Quality. A specialized I2V model delivers measurably better image fidelity and motion coherence. **Prefer LongCat-Video's multi-reward GRPO approach over single-reward RLHF when:** - The video generation domain has natural tensions between quality dimensions (visual quality vs. motion quality, frame aesthetics vs. temporal consistency). The equal-weighted advantage summation (Equation 50) prevents reward hacking without explicit constraints. - Compute budget for RLHF is limited. The paper's GRPO converges in ~500 iterations (Table 5), making it practical for smaller teams. **Prefer proprietary systems (Veo3) when:** - Absolute state-of-the-art quality is the sole criterion and budget/customization/deployment flexibility are not constraints. Veo3 leads LongCat-Video decisively in GSB evaluations (345 vs. 121 overall quality preferences) and VBench 2.0 total score (66.72% vs. 62.11%).