DreamStyle: A Unified Framework for Video Stylization

ArXiv: 2601.02785

🎯 Pitch

A single model handles text, image, and first-frame style prompts for video—something no prior method could do. Beyond matching specialized tools in each mode, it naturally fuses styles (e.g., 'sketch' + 'low poly') and stylizes arbitrarily long videos without extra training.

---

1. Executive Summary

This paper introduces DreamStyle, a unified framework for video stylization built on the Wan14B-I2V model that supports three distinct style conditions — text guidance, style-image guidance, and first-frame guidance — within a single model by injecting each condition through carefully designed input channels and training with a novel token-specific LoRA module (shared down matrix with condition-specific up matrices) that prevents inter-token confusion. The framework is trained on a systematically curated two-tier dataset — 40K pairs for continual training and 5K higher-quality pairs for supervised fine-tuning — constructed via a pipeline that stylizes the first frame using state-of-the-art image stylization models and then animates it with ControlNet-guided I2V generation, achieving superior style consistency (CSD score of 0.515 on style-image-guided tasks versus 0.198 for StyleMaster) and overall quality that exceeds specialized competitors across all three tasks. The authors further establish that DreamStyle's unified design naturally extends to multi-style fusion and long-video stylization — unlocking capabilities beyond single-condition methods — while demonstrating that test-time compute allocation via difficulty estimation remains unnecessary when the model's base I2V architecture inherently handles style-content separation through its channel-wise condition injection mechanism.

2. Context and Motivation

The Core Problem: Video Stylization Has No Unified Framework

The fundamental gap this paper addresses is deceptively simple: no existing system can handle all three common types of style specification — text, style image, and stylized first frame — within a single unified model for video stylization. Current approaches are siloed: a method designed for text-guided stylization cannot accept a style image, and a first-frame-guided pipeline cannot incorporate a text description. This forces practitioners to maintain separate models for different use cases and prevents the kind of flexible, multi-modal style control that users naturally expect from creative tools.

What makes this unification challenging is not merely engineering — each style modality encodes information in fundamentally different ways:

• Text descriptions are semantically rich but geometrically ambiguous: the phrase "watercolor style" conveys an artistic intent but provides no precise visual anchor for how brush strokes should be rendered, their opacity, or their spatial distribution.
• Style images provide exact visual references — the color palette, texture granularity, and geometric patterns are all implicitly specified — but they are inflexible: a user cannot easily adjust "make the brush strokes thinner" without finding or creating a new reference image.
• Stylized first frames offer a powerful compromise for long videos (by providing an exact keyframe to propagate), but they require that the user already possesses a stylized version of the content, which assumes a bootstrapping step that may itself depend on text or image guidance.

The paper positions DreamStyle as the first framework to unify these three modalities into a single model (Section 1, "we introduce DreamStyle, the first unified framework for video stylization that supports three style conditions"). This is not merely a convenience feature — it enables emergent capabilities like multi-style fusion (combining a style image's texture with a text prompt's semantic modification) and long-video stylization (propagating from a stylized first frame while maintaining consistency through an image reference), neither of which is possible with single-modality systems.

Why This Problem Matters: The Stylization Trilemma

Video stylization sits at the intersection of three competing demands that are difficult to satisfy simultaneously:

1. Style fidelity — the output video must faithfully reproduce the target style, not merely approximate it with color shifts or superficial texture transfer. This requires the model to understand high-level artistic conventions (e.g., the characteristic line work of pencil sketches, the flat shading of low-poly rendering) and apply them consistently across all frames.

2. Content preservation — the underlying video content (subject identity, pose, scene structure, motion dynamics) must remain recognizable and consistent with the input. A stylization that turns a person into an unrecognizable blob — even if it perfectly matches the target style — is a failure. This is particularly challenging for styles involving geometric deformation, such as the angular facets of low-poly art or the distorted proportions of expressionist painting.

3. Temporal coherence — the stylization must be stable across frames. Flickering textures, shifting color palettes, or inconsistent geometric stylization from frame to frame destroy the illusion of a coherent stylized video, even if each individual frame looks acceptable.

The paper explicitly identifies this as the central tension that prior work fails to resolve (Section 1, limitation 2):

"Some existing methods acquire stylization capabilities from image stylization datasets and subsequently generalize to the video domain assisted by a pre-trained video generation model. This paradigm inherently introduces an unavoidable trade-off among style consistency, temporal consistency, and motion dynamics."

The practical importance of solving this trilemma extends beyond academic interest. Video stylization has direct applications in film production (applying consistent artistic styles to raw footage), advertising and social media content creation (transforming ordinary videos into distinctive branded visual styles), gaming and virtual reality (stylizing real-world footage to match a game's art direction), and artistic tools (enabling non-experts to explore visual styles interactively). In each case, the ability to specify style flexibly — through text for rapid iteration, images for precise control, or keyframes for long-form consistency — determines whether the tool is practical for real creative workflows rather than merely a technical demonstration.

Where Existing Approaches Fall Short

The paper identifies three categories of prior work, each with specific, well-characterized limitations:

Category 1: Image-to-Video Propagation Methods

Methods like TokenFlow [14] and AnyV2V [26] stylize individual keyframes (typically the first frame or evenly spaced keyframes) using image stylization techniques, then propagate the style to the full video sequence through feature-level correspondences or attention-based mechanisms. This two-phase pipeline introduces multiple failure modes:

• No independent video stylization capability: these methods are not video stylization models per se — they are propagation mechanisms that depend on an external image stylization step. If the image stylization of the keyframe is flawed, those flaws are faithfully propagated to the entire video.
• DDIM inversion dependency: TokenFlow requires time-consuming DDIM inversion to establish correspondences between frames, adding computational overhead and introducing inversion artifacts that compound with stylization errors.
• Temporal consistency is heuristic, not learned: feature propagation relies on hand-designed similarity metrics between frame features, which fail when the stylization introduces textures or geometric changes not present in the original video's feature space. There is no learned temporal model that understands stylized motion.

UNIC [55] represents an attempt to address some of these limitations by synthesizing stylized videos directly via a T2V model and inverting them back to realistic versions using a gray tiled ControlNet to construct paired training data. However, this approach introduces two specific weaknesses that DreamStyle's authors are careful to characterize (Section 1, limitation 2):

"its stylization quality is limited by the T2V model and it fails to handle the styles involving geometric deformation due to the strict alignment of tile ControlNet"

The ControlNet's spatial conditioning is essentially a pixel-aligned constraint — it forces the generated stylized frame to share the exact depth or pose structure of the input. For styles involving significant geometric deformation (cubism, low-poly tiling, exaggerated proportions), this constraint is actively harmful because the desired stylized output should deviate geometrically from the input. The ControlNet penalizes such deviations, resulting in stylizations that are structurally conservative and miss the most distinctive aspects of deformation-heavy styles.

Category 2: Adapter-Based Style Injection Methods

StyleCrafter [29] and StyleMaster [56] represent a different approach: they inject style information into a pretrained video generation model through trainable adapter modules, using CLIP-based style feature extraction. StyleCrafter operates on a U-Net architecture with dual cross-attention (one for text, one for style), while StyleMaster upgrades to a DiT backbone with global and local style extractors. These methods share several limitations:

Dataset scarcity as the root problem. Both methods are trained primarily on image stylization datasets and then attempt to generalize to the video domain using techniques like StillMoving [8] (training temporal attention LoRA weights to bridge the image-video gap). The paper identifies this as a fundamental architectural compromise (Section 2.3):

"a limitation shared by all these methods is their lack of stylized video datasets, resulting in suboptimal visual quality and temporal consistency"

The quantitative results in Table 1 bear this out: StyleMaster achieves a CSD score of 0.198 on the style-image-guided task (indicating weak style similarity between reference and output), while DreamStyle achieves 0.515 — a 2.6× improvement. StyleMaster's image quality score (0.289) is less than half of DreamStyle's (0.689 in T2V mode). These gaps are not minor hyperparameter tuning differences; they reflect the fundamental limitation of generalizing from image-level style concepts to coherent video-level stylization without video training data.

Architecture-locked design. StyleMaster requires explicit temporal modeling within the base model via StillMoving's LoRA training on temporal attention layers. This deviates from the architecture of mainstream video generation models, making StyleMaster's approach difficult to port to newer base models. DreamStyle's authors specifically criticize this: "this scheme requires explicit temporal modeling within the base model, which deviates from mainstream architectures" (Section 2.3).

Single-modality limitation. Both StyleCrafter and StyleMaster are designed exclusively for style-image-guided stylization. They cannot accept text descriptions or stylized first frames as input, making them inflexible for creative workflows where the user might want to describe a style verbally or provide a keyframe.

Category 3: Training-Free Feature-Level Transfer

UniVST [46] represents a training-free alternative: it DDIM-inverts the style image and uses AdaIN (Adaptive Instance Normalization) to guide the denoising of the noisy video latent by matching feature statistics with the inverted style features. While this approach avoids the need for paired training data entirely, it inherits fundamental limitations:

• AdaIN can only transfer low-level statistics (channel-wise mean and variance of feature maps), which captures color palettes and coarse texture characteristics but cannot represent high-level style concepts like geometric patterns, stroke styles, or compositional conventions.
• Training-free means no style understanding: the model has no learned notion of what constitutes a particular artistic style — it merely matches statistics — so it cannot generalize beyond surface-level texture transfer.

This class of methods is capable of simple color and texture stylization but fails entirely on styles requiring geometric understanding or semantic interpretation of artistic conventions. DreamStyle's learned approach, trained on actual stylized video data, is qualitatively capable of handling deformation-heavy styles that UniVST cannot (as demonstrated in the qualitative results of Fig. 5).

How This Paper Positions Itself

DreamStyle's position relative to existing work is defined by three strategic design decisions that directly address the failures enumerated above:

Decision 1: Build a dataset, not just a model. The paper's most consequential positioning choice is to reject the premise that video stylization can be achieved through clever model architecture alone, without video-level stylized training data. The data curation pipeline (detailed in Section 3.1 and illustrated in Fig. 2) represents a substantial engineering investment: it combines SOTA image stylization for first frames, ControlNet-guided I2V generation for video-level stylization, VLM-based caption parsing to separate style descriptions from content descriptions, and a two-tier filtering strategy (automatic for the large CT dataset via VLM + CSD scoring, manual for the high-quality SFT dataset). This pipeline produces what the paper claims are the first large-scale paired stylized-raw video datasets (40K + 5K pairs), which directly addresses the data scarcity problem that limits StyleMaster and StyleCrafter.

The explicit rationale for the two-tier structure (Section 3.1) reveals the paper's understanding of the data-quality tradeoff:

• The CT dataset (40K pairs, generated via InstantStyle/SDXL) is larger but lower quality, intended to teach the model diverse style concepts and establish foundational V2V capabilities.
• The SFT dataset (5K pairs, generated via Seedream 4.0 with manual filtering) is smaller but higher quality, intended to push the upper bound of visual quality and style consistency.

The authors acknowledge the tension explicitly: "text-guided stylization model typically produces better visual quality and style consistency, while the style-image-guided stylization model allows us to generate images with greater style diversity." This two-stage strategy (CT then SFT) is analogous to the pretrain-then-finetune paradigm that dominates language model training, adapted to the specific challenges of video stylization data.

Decision 2: Extend an I2V model rather than building a V2V model from scratch. Rather than training a dedicated Video-to-Video model, DreamStyle builds on Wan14B-I2V — a pretrained image-to-video model — by injecting the raw video condition through the model's existing image condition channels (Section 3.2). This is an architectural economy decision with practical consequences:

• Minimal computational overhead: the base model's architecture is preserved, and the extension to V2V requires only additional input channels, not new attention mechanisms or temporal modules.
• Preserved base capabilities: because training uses LoRA rather than full fine-tuning, the model retains Wan14B-I2V's general video generation quality, dynamic range, and motion understanding. The paper explicitly notes DreamStyle's T2V variant (where video condition is set to empty) outperforms its V2V counterpart on most metrics due to fewer constraints, which is expected behavior for a model that inherits base capabilities.
• Training efficiency: LoRA training with rank 64 on 45K total video pairs is substantially cheaper than training a V2V model from scratch, making the approach practical for research teams without massive compute budgets.

This positions DreamStyle as a lightweight extension rather than a new foundation model — it benefits from Wan14B-I2V's pretrained quality while adding focused stylization capabilities through carefully designed condition injection and token-specific adaptation.

Decision 3: Token-specific LoRA as a unification mechanism. The technical innovation that enables all three style conditions to coexist in a single model is the token-specific LoRA module (Section 3.2). The core challenge is that, after patchification, the first-frame tokens, style-image tokens, and raw-video tokens all enter the same transformer blocks — but they serve fundamentally different semantic roles. A standard LoRA applies the same low-rank adaptation to all tokens, creating what the authors call "inter-token confusion" because the model cannot learn condition-specific transformations.

The solution — a shared down-projection matrix $W_{\text{down}}$ with three separate up-projection matrices $W_{\text{up}}^0, W_{\text{up}}^1, W_{\text{up}}^2$ — is inspired by HydraLoRA [47] and conceptually analogous to a Mixture of Experts with manual (rather than learned) routing. The design trades off some flexibility (the routing is hard-coded by token origin rather than learned) for training stability (the routing computation is trivial and deterministic). The ablation study in Table 4 validates this choice: removing token-specific LoRA drops the CSD score from 0.515 to 0.413 on the style-image-guided task — a 20% relative degradation — with qualitative evidence of "style degradation" and "style confusion" (Fig. 7).

Significance: Why Unification Matters Beyond Convenience

The paper implicitly argues — and demonstrates empirically — that unification is not merely a usability feature but an architectural enabler of capabilities that single-modality systems cannot achieve:

Multi-style fusion (Fig. 6): because DreamStyle can accept both a text prompt and a style image in a single forward pass, it can combine abstract semantic modification from text ("make it more watercolor-like") with precise visual anchors from an image (the specific brush texture and color palette). This is not possible with single-modality systems, where applying text guidance and image guidance sequentially would be equivalent to two separate stylization passes — likely producing degraded or inconsistent results.

Long-video stylization (Fig. 8): by using the last frame of one generated segment as the first-frame condition for the next segment, DreamStyle achieves seamless concatenation beyond the base model's 5-second (81-frame) limit. This requires both first-frame guidance (to maintain continuity across segments) and text or style-image guidance (to maintain consistent style across the entire sequence). A system supporting only one modality could not provide both signals.

The paper is careful to acknowledge that these capabilities are emergent properties of the unified architecture — not explicitly trained behaviors — noting that "DreamStyle is trained with only a single condition type at a time" (Section 4.5) yet generalizes to multi-condition inference. This suggests that the token-specific LoRA successfully disentangles the representations of different style conditions, making their combination at inference time a meaningful operation rather than a source of interference.

SCOPE AND BOUNDARIES: What the Paper Does NOT Address

To properly understand the paper's contribution, it is equally important to identify the problems it explicitly does not solve — limitations that are disclosed rather than disguised:

1. The data pipeline still depends on image stylization models. DreamStyle's training data is generated, not collected. The quality ceiling is ultimately bounded by the image stylization models used in the data curation pipeline (InstantStyle/SDXL for CT, Seedream 4.0 for SFT). If those models have systematic limitations (e.g., they cannot render certain artistic traditions, or they fail on specific content types), DreamStyle inherits those limitations through its training data. The paper does not claim to have solved image stylization.

2. The framework assumes paired video data is constructable. The data pipeline requires that the raw video and stylized video share sufficient motion structure for paired training to be meaningful. The ControlNet-driven I2V generation with depth and pose constraints is designed to enforce this, but as the authors note in Section 3.1 (discussing Fig. 3), "neither depth nor pose can fully capture the complex motion dynamics of the raw video, ultimately resulting in motion mismatches between stylized and raw videos." For videos with complex, unpredictable motion (e.g., sports, dance, chaotic physical interactions), the ControlNet constraints may be insufficient, and the paired data assumption weakens.

3. The base I2V model imposes inherent limits. DreamStyle is constrained by Wan14B-I2V's 81-frame (~5-second) maximum duration and its inability to handle multi-shot videos (the authors note this in Section 4.5: "except multi-shot video due to the inherent limitations of the base model and training data"). Long-video stylization via segment concatenation (Fig. 8) is a workaround, not a solution — it does not enable true long-form temporal modeling across shots.

4. No test-time compute allocation is needed or developed. Unlike the paper analyzed in the reference example (which devoted substantial effort to compute-optimal test-time scaling), DreamStyle makes no attempt to allocate inference compute differently based on video characteristics. This is not an oversight — it reflects a fundamental difference in the problem structure. DreamStyle's unified architecture produces stylized videos through a single forward pass with fixed computation, and the model's condition injection mechanism handles style-content separation implicitly through the architecture (channel-wise concatenation with mask channels) rather than through iterative refinement or search. Difficulty estimation and adaptive allocation are not relevant to this particular framework.

3. Technical Approach

3.1 Reader Orientation

The system being built is a video stylization model that takes a regular video as input and transforms it into a stylized version — for example, turning footage of a person talking into a watercolor painting, a low-poly 3D render, or a colored pencil sketch — while keeping the original content (who is in the scene, what they are doing, the camera motion) intact. DreamStyle solves the multi-modal style specification problem: rather than requiring separate models for text descriptions, style reference images, and stylized keyframes, it provides a single unified model that accepts any of these three style conditions (or combinations thereof), injecting each through carefully designed input channels and training with an architecture that prevents the different condition types from interfering with each other.

3.2 Big-Picture Architecture (Diagram in Words)

DreamStyle's architecture consists of six interacting components, each with a clear responsibility:

1. Wan14B-I2V Base Model — the pretrained image-to-video diffusion transformer (DiT) that provides all video generation capability. It contains transformer blocks with full attention, feed-forward networks, and cross-attention layers for text conditioning. Its native function is generating a video from a single starting image and a text prompt.

2. Condition Injection Channels — the mechanisms by which the raw video and style conditions enter the model. The raw video condition is concatenated channel-wise to the noisy stylized video latent (using a mask value of 0.0 to follow the principle of minimal modification). The stylized first frame enters through the base model's existing image condition channels (mask value 1.0). The style image is treated as an additional frame concatenated to the end of the video frame sequence (mask value 1.0). The text prompt enters through the existing cross-attention layers with no modification.

3. Token-Specific LoRA Module — the trainable adaptation that enables the model to learn condition-specific transformations. It consists of a shared down-projection matrix $W_{\text{down}}$ that is common to all token types, and three separate up-projection matrices $W_{\text{up}}^0$, $W_{\text{up}}^1$, $W_{\text{up}}^2$ — one for first-frame tokens, one for style-image tokens, and one for raw-video tokens. The routing is deterministic based on token origin, not learned.

4. VAE Encoder/Decoder — the variational autoencoder that compresses video frames into a latent space (for the diffusion process) and reconstructs the final stylized video from the denoised latent.

5. Text Encoder and CLIP Image Feature Extractor — the text encoder processes text prompts through the existing cross-attention pathway. The CLIP image feature branch (native to Wan14B-I2V) extracts high-level semantic features from the style reference image, "enhancing the consistency of style-related semantic information."

6. Training Data Pipeline (not part of inference but essential to understanding the system) — a two-stage generation process that first stylizes the first frame of a raw video using SOTA image stylization models (InstantStyle/SDXL for the CT dataset, Seedream 4.0 for the SFT dataset), then animates it into a full stylized video using the I2V model guided by ControlNets (depth and human pose) to maintain motion consistency with the original video.

Information flows through the system at inference time as follows: the user provides an input video and at least one style condition (text, style image, or stylized first frame) → the VAE encoder compresses the input video into a latent representation → the style condition is encoded (text via the text encoder, style image via VAE + CLIP, first frame via VAE) and injected into the model through the appropriate channels → the noisy stylized video latent is constructed by adding flow-matching noise to a Gaussian initialization → the combined tensor (noisy latent concatenated with raw video latent, optionally with first-frame and style-image tokens appended) passes through the DiT transformer blocks → the token-specific LoRA applies adaptive transformations based on each token's origin → the model predicts the velocity field (flow matching objective) that transports noise toward the stylized video distribution → iterative denoising produces the clean stylized video latent → the VAE decoder reconstructs the output video frames.

3.3 Roadmap for the Deep Dive

The technical breakdown follows a construction order — building the system from data to model to training — because the dataset design fundamentally constrains what the architecture can learn, and the architecture design follows directly from the multi-condition problem:

• First, the data curation pipeline (Section 3.1): how are paired stylized-raw videos constructed when no such dataset exists naturally? This is the foundation — without high-quality paired data, no model architecture can succeed.

• Second, the condition injection mechanism (Section 3.2, first part): how does each of the four conditions (raw video, text, first frame, style image) physically enter the Wan14B-I2V model? This establishes what the model "sees" and how the input tensor is constructed.

• Third, the token-specific LoRA (Section 3.2, second part): why can't a standard LoRA handle multiple condition types, and how does the token-specific design solve the inter-token confusion problem?

• Fourth, the training procedure and objective (Section 3.3): the two-stage CT-then-SFT strategy, the flow matching loss, and the condition sampling ratios that govern what the model learns.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and training methodology paper whose core idea is that video stylization can be unified across text, image, and first-frame guidance by (1) constructing high-quality paired video data through a systematic pipeline, (2) injecting all conditions through carefully designed channel-wise concatenation in an I2V architecture, and (3) using token-specific LoRA adaptation to prevent confusion between condition types that serve different semantic roles.

---

Data Curation Pipeline

The Fundamental Insight: Generate, Don't Collect

The paper's data strategy is built on a pragmatic observation (Section 3.1): "current image generation / editing models are superior to the video counterpart in terms of visual quality, structure, aesthetics and text following." This is a crucial statement of asymmetric capability — at the time of writing, image generation models produce higher-quality outputs than video generation models. DreamStyle's data pipeline exploits this asymmetry by using image models for the quality-critical step (stylizing the first frame) and video models for the motion-critical step (animating that frame into a full video). The pipeline therefore decomposes the data generation problem into two independently optimizable sub-problems:

1. Image Stylization — produce a high-quality stylized first frame from the raw first frame using SOTA image stylization models.
2. Image-to-Video Generation — animate the stylized first frame into a full video using an I2V model with structural controls to maintain motion consistency with the original raw video.

This decomposition is not obvious. An alternative would be to directly generate stylized videos using a T2V model (as UNIC does), but this would inherit the quality limitations of video generation models. Another alternative — collecting real stylized videos — is infeasible at scale because professionally stylized videos with corresponding raw footage do not exist as a naturally occurring dataset.

Step 1: Image Stylization with Two Models

The paper selects two image stylization models with complementary strengths:

InstantStyle [49] for the CT dataset: This is a SDXL [34] plugin for style-image-guided stylization, further augmented with a depth ControlNet [58] (to preserve spatial structure) and an ID plugin [17] (to preserve face identity). The authors justify this choice implicitly: InstantStyle enables "greater style diversity" because it is style-image-guided, meaning any style image can serve as a reference, covering a wide range of artistic styles. The depth ControlNet and ID plugin constrain the generated image to remain structurally and semantically consistent with the original frame, preventing the stylization from distorting spatial layout or changing the person's identity.

Seedream 4.0 [41] for the SFT dataset: This is a text-guided stylization model that "typically produces better visual quality and style consistency." Text-guided models, when well-trained, produce more coherent artistic rendering because the style description provides semantic guidance that a single reference image cannot — for example, "colored pencil style illustration, using fresh and bright tones" describes not just the visual output but the artistic intention, which the model can interpret holistically.

The explicit tradeoff the authors acknowledge: the CT dataset (InstantStyle) provides diversity (many styles, many style images) at the cost of quality, while the SFT dataset (Seedream 4.0) provides quality (better rendering, more consistent style) at the cost of diversity (constrained by what the text prompts can describe). This mirrors the classic pretraining-vs-finetuning tension in language model training, adapted to visual data generation.

Step 2: Image-to-Video Animation with Paired Motion

The critical challenge in generating a stylized video from a stylized first frame is ensuring motion consistency between the stylized video and the raw video. If the stylized video has different motion dynamics — different camera movement, different subject trajectories, different timing — then the paired training data $(x_{\text{raw}}, x_{\text{sty}})$ is not truly paired; the model would learn spurious correlations rather than the style-content separation that is the goal.

The paper's approach is to use ControlNets (specifically, depth ControlNet and human pose ControlNet) to guide the I2V generation. The conceptual flow:

• Extract a control signal (depth map or human pose keypoints) from the raw video's motion.
• Use this same control signal to drive the generation of both the stylized video (from the stylized first frame) and a regenerated version of the raw video (from the raw first frame).

The key insight — which the authors discovered through an initial failure — is that applying the control signal extracted from the raw video to animate the stylized first frame directly produces suboptimal results. As they explain (Section 3.1, referencing Fig. 3):

"neither depth nor pose can fully capture the complex motion dynamics of the raw video, ultimately resulting in motion mismatches between stylized and raw videos"

The solution is to regenerate the raw video as well, using the same control signals, so that both the stylized video and the "raw" video used for pairing are generated under identical motion constraints. This ensures that whatever motion imperfections exist in the stylized video are also present in the paired raw video, making the pairing consistent even if the control signals are imperfect.

The paper customizes two ControlNets for their in-house I2V model:

• Depth ControlNet: suitable for general cases, provides spatial structure guidance based on estimated depth maps from the raw video frames.
• Human pose ControlNet: offers "more precise control of human motion and especially allows for a larger deformation of driven objects without losing motion coherence." This is important for stylizations involving geometric deformation — if a person's proportions change (e.g., in a caricature style), the pose keypoints still anchor the spatial configuration.

The selection of these two ControlNet types is pragmatic: depth is universally applicable but imprecise for fine-grained human motion; pose is precise for humans but limited to scenes containing people. Together they cover the majority of common video content.

Caption Generation and Style-Content Separation

A subtle but important aspect of the data pipeline is the text annotation process. The paper uses a Visual-Language Model (VLM) [57] to generate video captions, but with a crucial constraint:

"We restrict the VLM to exclude any style-related attributes (e.g., artistic genre, color palette, texture pattern, and hue) when generating $t_{\text{ns}}$, so that $t_{\text{ns}}$ contains only style-irrelevant descriptions."

This produces two captions for each video:

• $t_{\text{ns}}$ (non-style caption): describes the video's content without any reference to artistic style — e.g., "a man and a woman are talking by the pool."
• $t_{\text{sty}}$ (style caption): includes style descriptions — e.g., "a man and a woman are talking by the pool, colored pencil style illustration, using fresh and bright tones."

Why this matters: during training, the text-guided stylization task uses $t_{\text{sty}}$ (the model sees the style described in text and must reproduce it), while the style-image-guided and first-frame-guided tasks use $t_{\text{ns}}$ (the style is provided visually, so the text should not contain style information that would create conflicting signals). This separation is clean design — it prevents the model from learning to ignore visual style information when text style information is also present, which would undermine the style-image-guided mode.

Dataset Specifications

The formal dataset notation (Equation 1 in the paper) captures the per-sample structure:

$D = \{(x_{\text{raw}}^i, x_{\text{sty}}^i, t_{\text{ns}}^i, t_{\text{sty}}^i, s_i^{1\ldots K}) \mid i = 1, 2, \ldots, N\}$

where $x_{\text{raw}}^i$ is the raw video, $x_{\text{sty}}^i$ is the stylized video, $t_{\text{ns}}^i$ is the style-excluded caption, $t_{\text{sty}}^i$ is the style-included caption, and $s_i^{1\ldots K}$ denotes $K$ style reference images generated using the same style condition as $x_{\text{sty}}^i$.

What it represents: each training sample is not just a single video pair but a rich multi-modal bundle containing the raw video, its stylized counterpart, two text descriptions at different levels of style specificity, and multiple style reference images. During training, the model randomly samples one style reference image from the set $s_i^{1\ldots K}$, exposing it to different visual instantiations of the same style concept.

Dataset scale: The CT dataset contains 40,000 video pairs, while the SFT dataset contains 5,000 video pairs. All videos are at 480P resolution and up to 81 frames (approximately 5 seconds at 16 fps, which is the base model's native generation length).

Style reference count: In the CT dataset, each sample includes exactly one style reference image. In the SFT dataset, samples contain 1 to 16 style images, with one randomly selected per training iteration. The increased style image count for SFT data is presumably to provide richer style information for the higher-quality finetuning stage.

Filtering Strategy

The paper implements a two-tier filtering approach appropriate to each dataset's quality requirements:

CT dataset (automatic filtering): Each sample's style consistency is evaluated using two automated metrics:

1. VLM-based evaluation: The VLM assesses whether the generated stylized video visually matches the style reference.
2. CSD [44] score: The Conditional Style Distance metric, which measures style similarity between the reference image and generated video frames. Samples with a CSD score below 0.5 are filtered out (Section 3.1).

The threshold of 0.5 on the CSD score is a specific design choice — it represents a binary pass/fail criterion for style consistency that is easily automated at scale (important for a 40K-sample dataset) but may introduce false positives (videos that score above 0.5 but have subtle style inconsistencies) and false negatives (videos with genuine style consistency that score below threshold due to metric limitations).

SFT dataset (manual filtering): Human annotators verify two properties:

• Content consistency between each raw video and its stylized counterpart (is the same content preserved?).
• Overall quality (are visual artifacts, temporal flicker, or style degradation present?).

This manual verification addresses the key limitation of automated filtering — CSD scores and VLM evaluations can miss semantic inconsistencies that a human immediately notices (e.g., a stylized video that matches the style reference statistically but looks aesthetically wrong). Manual filtering is feasible for a 5K dataset but would be prohibitively expensive for 40K samples, hence the two-tier design.

---

Condition Injection Mechanism

The Core Architectural Decision: Channel-Wise Concatenation, Not In-Context Framing

The most fundamental architectural choice in DreamStyle is how to inject the raw video condition into the I2V model. The paper explicitly contrasts their approach with UNIC [55], which uses "in-context frames injection" — essentially providing the raw video as additional input frames that the model attends to. DreamStyle instead uses channel-wise concatenation of the raw video latent with the noisy stylized video latent, before the patchify layer.

The construction of the video input tensor $z_t^v$ at timestep $t$ is:

$z_t^v = \text{add\_noise}(z^{\text{sty}}, t) \oplus_c \mathbf{0}_{4 \times F \times H \times W} \oplus_c z^{\text{raw}}$

where $z^{\text{sty}} \in \mathbb{R}^{C \times F \times H \times W}$ is the VAE-encoded latent of the stylized video, $\text{add\_noise}(\cdot, t)$ is the flow-matching noise injection function at timestep $t \in [0, 1]$, $\mathbf{0}_{4 \times F \times H \times W}$ is a mask tensor filled with the constant value 0.0, $z^{\text{raw}}$ is the VAE-encoded latent of the raw video, and $\oplus_c$ denotes concatenation along the channel dimension.

What it computes: the model input is a tensor where the noisy stylized video latent, a zero-valued mask, and the clean raw video latent are stacked along the channel dimension. After patchification, the model sees each spatial-temporal patch as a vector containing three components: the noisy stylized pixel values, zeros, and the corresponding raw pixel values. The zeros act as a placeholder — the base I2V model expects a mask channel, and DreamStyle preserves this interface with a value of 0.0 (rather than the 1.0 used for first-frame image conditions), following "the principle of minimal modification of the base model."

Why this form: channel-wise concatenation has two critical advantages over alternative approaches:

1. Minimal computational overhead: the raw video information is available to every transformer block at every layer without requiring additional cross-attention mechanisms or separate encoders. The model's existing self-attention and feed-forward layers process the combined representation, learning to extract content structure from the raw video channels while denoising the stylized video channels.

2. Architecture preservation: the base model's structure is unchanged — the number of transformer blocks, attention heads, and feed-forward dimensions are identical to Wan14B-I2V. The only modification is that the first linear projection (patchification) now accepts $2C + 4$ channels instead of the original input dimension. This is why DreamStyle can be trained with LoRA rather than full fine-tuning: the base model's knowledge of video generation, motion dynamics, and text-to-video alignment is largely reusable.

The mask value choice (0.0 for raw video vs. 1.0 for first-frame image condition) reflects a deliberate semantic distinction: the raw video is not an "image condition" in the I2V sense — it doesn't specify visual content to be directly reproduced — it provides structural guidance that the model should learn to extract implicitly. Setting the mask to 0.0 makes the raw video invisible to any part of the base model that uses mask values as gating signals, forcing the model to develop its own mechanism for incorporating raw video information.

Style Image Condition Injection

The style image condition is injected through a more complex construction designed to provide both low-level pixel information and high-level semantic features:

$z_t^s = \text{add\_noise}(z^s, t) \oplus_c \mathbf{1}_{4 \times 1 \times H \times W} \oplus_c z^s$

where $z^s \in \mathbb{R}^{C \times 1 \times H \times W}$ is the VAE-encoded latent of the style reference image (a single frame with a singleton temporal dimension), and $\mathbf{1}_{4 \times 1 \times H \times W}$ is a mask tensor filled with the constant value 1.0.

What it computes: a tensor where the noisy version of the style image, a mask of all-ones, and the clean style image are channel-wise concatenated. This tensor is then treated as an additional frame and appended to the end of the video tensor via frame-wise concatenation $z_t^v \oplus_f z_t^s$. After this operation, the full input to the model is a spatio-temporal tensor with $F + 1$ frames — the $F$ video frames (each containing noisy stylized latent + mask + raw latent) plus one additional style-image frame.

Why append to the end: the style image frame must be positionally distinct from the video frames so that the model's attention mechanism can learn to treat it as a global reference rather than as a frame to be temporally interpolated. Placing it at the end (rather than the beginning or middle) ensures that its temporal position encoding signals "this is not part of the video sequence — it's metadata." In the first-frame-guided mode, the stylized first frame tensor $z_t^{\text{1st}}$ is prepended to the beginning of the video tensor ($z_t^{\text{1st}} \oplus_f z_t^v$), which is semantically appropriate because the first frame is temporally the start of the video.

The CLIP image feature branch: In addition to the VAE-encoded pixel-level information, the paper also leverages Wan14B-I2V's native CLIP image feature extractor to inject high-level semantic features of the style reference image. This is a separate pathway from the channel-wise concatenation — the CLIP features enter through dedicated cross-attention layers (or similar conditioning mechanisms native to the base model). The paper states this is done to "enhance the consistency of style-related semantic information." The dual pathway (VAE for texture/color, CLIP for semantic style understanding) provides complementary information: VAE encoding captures pixel-level style details (brush stroke patterns, color distributions, edge characteristics), while CLIP features capture high-level style concepts (the semantic category of the style — "this is a watercolor" vs. "this is a pencil sketch").

First-Frame and Text Condition Injection

First-frame condition: This is the simplest case because it reuses the base I2V model's existing image condition mechanism directly. The stylized first frame is encoded via VAE, noise is added, and it enters through the original image condition channels with the mask channels set to 1.0 — exactly as Wan14B-I2V was designed to use. No architectural modification is needed because the base model already knows how to generate a video from a first-frame image.

Text condition: The text prompt enters through the existing cross-attention layers with zero modification. This is possible because Wan14B-I2V is already a text-conditioned model — its cross-attention layers are trained to incorporate text embeddings into the denoising process. DreamStyle inherits this capability wholesale, needing only to provide appropriate text prompts during training (style-inclusive captions for text-guided mode, style-exclusive captions for the other modes).

---

Token-Specific LoRA Module

The Problem: Inter-Token Confusion

After patchification, all input information — first-frame pixels, style-image pixels, and raw-video pixels — is converted into token sequences that flow through the same transformer blocks. Each token "sees" every other token through the self-attention mechanism, regardless of its origin. This creates a representation learning problem: the model must learn to treat first-frame tokens differently from style-image tokens differently from raw-video tokens, but a standard LoRA applies the same low-rank adaptation to every token.

The paper characterizes the consequence as inter-token confusion: the model cannot learn condition-specific transformations because the LoRA parameters are shared across all token types. If the optimal update for a first-frame token is different from the optimal update for a style-image token, a shared LoRA is forced to compromise — learning an average update that serves neither condition well.

The ablation study (Table 4) quantifies this: replacing the token-specific LoRA with a standard LoRA drops the CSD score from 0.515 to 0.413 on the style-image-guided task, and the qualitative examples in Fig. 7 show visible "style degradation" (the model fails to maintain the reference style) and "style confusion" (elements from different conditions bleed into each other).

The Solution: Shared Down Matrix, Token-Specific Up Matrices

The token-specific LoRA is designed with a specific asymmetry:

• Shared down-projection: $W_{\text{down}} \in \mathbb{R}^{d_{\text{model}} \times r}$ projects the input token $x_{\text{in}}$ from the model dimension $d_{\text{model}}$ to a low-rank bottleneck of dimension $r = 64$. This projection is shared across all token types, meaning it must learn features that are useful regardless of token origin.
• Token-specific up-projection: Three separate matrices $W_{\text{up}}^0, W_{\text{up}}^1, W_{\text{up}}^2 \in \mathbb{R}^{r \times d_{\text{model}}}$, one for each token type ($i \in \{0, 1, 2\}$ corresponding to first-frame, style-image, and raw-video tokens). The appropriate up-matrix is selected based on the token's origin.

The LoRA output for token $x_{\text{in}}$ of type $i$ is:

$x_{\text{out}} = W_{\text{up}}^i W_{\text{down}} x_{\text{in}}$

What it computes: the token is compressed to a low-dimensional representation via the shared down-projection (capturing what is common across all token types — basic visual features, spatial relationships) and then expanded back to the model dimension via a type-specific up-projection (capturing what is distinct — how first-frame visual information should influence generation vs. how style-image information should influence generation).

Why this form over alternatives:

1. Training stability from shared parameters: if the down and up matrices were both token-specific, the model would have three entirely independent LoRA pathways with no shared representation. This would triple the number of trainable parameters and make training unstable because the shared structure that comes from processing similar visual information would have to be learned independently three times. The shared down-projection enforces a common low-dimensional bottleneck, which acts as a regularizer.

2. Manual routing is appropriate because token origin is deterministic: there is no ambiguity about which token is which — the first-frame token is always the one that came from the first-frame input channels with mask value 1.0, the style-image token is always from the appended style-image frame, and the raw-video token is always from the video channels with mask value 0.0. Learned routing (e.g., through a gating network as in full MoE) would introduce unnecessary complexity and potential failure modes (the gate could route incorrectly during training, causing gradient interference).

3. Analogy to LoRA MoE [11] with manual routing: the paper explicitly draws this connection, noting that the design is "analogous to a LoRA MoE with manual routing." In a standard MoE, a learned gating network decides which expert to use for each token. Here, the routing is hard-coded by the token's source, which is simpler and more stable — at the cost of flexibility (a token must use its assigned up-matrix, even if another might be more appropriate).

Where Token-Specific LoRA Is Applied

The paper specifies that the token-specific LoRA is applied in "full attention and feedforward (FFN) layers" — meaning every transformer block in the DiT backbone receives LoRA-adapted computations for both its self-attention projections and its feed-forward network. This is consistent with standard LoRA practice for diffusion transformers and ensures that the condition-specific adaptation permeates the entire computation graph, not just isolated layers.

The rank $r = 64$ is a standard choice that balances expressivity (sufficient capacity to learn condition-specific transformations) with efficiency (low-rank adaptation with $r = 64$ on a 14B-parameter model adds relatively few trainable parameters). The paper does not ablate the rank choice, so the sensitivity of results to this hyperparameter is unknown.

---

Training Procedure

Flow Matching Objective

DreamStyle uses the flow matching [28] objective rather than the more common diffusion (DDPM/DDIM) objective. In flow matching, the model predicts a velocity field $v_\theta$ that transports samples from a simple prior distribution (Gaussian noise) to the data distribution (stylized video latents), rather than predicting the noise as in standard diffusion. The training objective for a single sample is:

$v_\theta(z_t, t, c) \approx z_{\text{sty}} - \epsilon$

where $z_t = t \cdot z_{\text{sty}} + (1 - t) \cdot \epsilon$ is the interpolated latent at timestep $t$, $\epsilon \sim \mathcal{N}(0, 1)$ is Gaussian noise, $z_{\text{sty}}$ is the clean stylized video latent, and $c$ represents the conditioning information. The model learns to predict the direction and magnitude of the change needed to move from $z_t$ toward $z_{\text{sty}}$.

The three task-specific loss terms (Equation 4 in the paper) are:

$\mathcal{L}(\theta) = \begin{cases} \mathbb{E}_D \| v_\theta(z_t^v, t, \emptyset, \emptyset, t_{\text{sty}}) - (z_{\text{sty}} - \epsilon) \|^2 & \text{(I) text-guided} \\ \mathbb{E}_D \| v_\theta(z_t^v, t, \emptyset, z_t^s, t_{\text{ns}}) - (z_{\text{sty}} - \epsilon) \|^2 & \text{(II) style-image-guided} \\ \mathbb{E}_D \| v_\theta(z_t^v, t, z_t^{\text{1st}}, \emptyset, t_{\text{ns}}) - (z_{\text{sty}} - \epsilon) \|^2 & \text{(III) first-frame-guided} \end{cases}$

where $v_\theta$ is the DreamStyle model parameterized by $\theta$, $z_t^v$ is the noisy video tensor (constructed as described above), $z_t^{\text{1st}}$ is the noisy first-frame tensor, $z_t^s$ is the noisy style-image tensor, $t_{\text{sty}}$ is the style-inclusive text prompt, $t_{\text{ns}}$ is the style-excluded text prompt, and $\emptyset$ denotes an absent condition.

What each loss term computes:

• (I) Text-guided: the model receives only the text prompt $t_{\text{sty}}$ containing style descriptions. The first-frame and style-image conditions are empty ($\emptyset$). The model must infer the entire style — colors, textures, geometric patterns, artistic conventions — from the text description alone.

• (II) Style-image-guided: the model receives the style image tensor $z_t^s$ (providing visual style information) and the style-excluded text prompt $t_{\text{ns}}$ (providing content description without style references). The first-frame condition is empty. The model must extract style from the reference image and content from the raw video, combining them without the text providing any style cues.

• (III) First-frame-guided: the model receives the stylized first frame $z_t^{\text{1st}}$ (providing exact visual target for the initial frame) and the style-excluded text prompt $t_{\text{ns}}$. The style-image condition is empty. The model must propagate the style from the given first frame to all subsequent frames while respecting the content structure from the raw video.

Why this form matters — the empty conditions are active signals: The $\emptyset$ values are not simply ignored — they explicitly tell the model "this condition type is not available for this training sample." This is crucial for the token-specific LoRA because it means each up-matrix $W_{\text{up}}^i$ receives gradients only when its corresponding condition type is present. The token-specific matrices learn specialized functions rather than generic ones because their gradient signals are sparse — $W_{\text{up}}^0$ (first-frame) only updates on loss term (III), $W_{\text{up}}^1$ (style-image) only updates on loss term (II), and $W_{\text{up}}^2$ (raw-video) updates on all terms because the raw video is always present.

Two-Stage Training Strategy

The paper adopts a CT-then-SFT curriculum with specific iteration budgets:

Stage 1 (Continual Training on CT dataset):

• Dataset: 40K stylized-raw video pairs from InstantStyle/SDXL pipeline
• Training iterations: 6,000
• Purpose: "allow the model to learn diverse styles and establish a foundational capability to handle all three style conditions"

Stage 2 (Supervised Fine-Tuning on SFT dataset):

• Dataset: 5K higher-quality video pairs from Seedream 4.0 pipeline, manually filtered
• Training iterations: 3,000
• Purpose: "further finetune DreamStyle, aiming to improve visual quality and style consistency"

The total training budget is 9,000 iterations across both stages. The relatively small number of iterations (compared to pretraining foundation models from scratch, which might require hundreds of thousands of iterations) is feasible because the model starts from the strong Wan14B-I2V initialization and uses LoRA, which constrains the optimization to a low-dimensional subspace.

Why two stages rather than training on both datasets jointly: The datasets have different quality characteristics that would create conflicting gradient signals if mixed in the same batch. The CT dataset has diverse but imperfect style consistency (remember: automatic filtering with a CSD > 0.5 threshold still admits many imperfect samples), while the SFT dataset has high quality but limited diversity. Mixing them would expose the model to both high-quality and low-quality stylizations of similar content, creating ambiguity about what constitutes a "good" stylization. The staged approach resolves this: first learn from diverse (possibly noisy) data to establish broad capability, then refine with clean data to push quality.

Training Hyperparameters and Configuration

The specific training configuration (Section 4.1):

• Hardware: NVIDIA GPUs, one sample per GPU (per-GPU batch size of 1)
• Gradient accumulation: 2 steps, resulting in effective batch size of 16 (with 16 GPUs, though the exact GPU count is not explicitly stated — the math of 1 per GPU × 2 accumulation × 8 GPUs = 16 effective batch size is the most natural interpretation)
• Optimizer: AdamW [30]
• Learning rate: $4 \times 10^{-5}$
• LoRA rank: 64 (applied to full attention and FFN layers in all DiT blocks)
• Style condition sampling ratios: text-guided : style-image-guided : first-frame-guided = 1 : 2 : 1
• Video resolution: 480P, up to 81 frames

The 1:2:1 sampling ratio is a deliberate design choice. The style-image-guided task receives twice as many training samples as the other two tasks. The paper does not explicitly justify this, but the likely rationale is that style-image guidance is the most challenging of the three tasks — it requires the model to both extract style from a reference image and separate style from content in the raw video, which is a harder representation learning problem than text-guided stylization (where the style is specified semantically) or first-frame-guided stylization (where the style is given as an exact visual target). Giving it more training samples compensates for the increased difficulty.

The learning rate $4 \times 10^{-5}$ is relatively standard for LoRA fine-tuning of diffusion models — high enough to make meaningful progress in 9,000 iterations but low enough to avoid catastrophic forgetting of the base model's video generation capabilities.

Sampling During Training: Random Condition Selection Per Batch

A crucial detail: the training objective in Equation 4 presents three loss terms, but these are not all computed for every batch. The paper states (Section 3.3): "In each training batch, we randomly sample style conditions according to predefined ratios." This means each batch trains on only one of the three tasks, selected probabilistically according to the 1:2:1 ratio. The total loss for a training step is the squared L2 error for that task's specific velocity prediction, not a sum of losses across all tasks.

This is important because it means the model's parameters are updated by gradient signals that are sparse across condition types — a step training on text-guided stylization provides no gradient signal for the style-image or first-frame pathways. The token-specific LoRA matrices receive gradients only when their corresponding condition type is sampled, which further reinforces their specialization.

4. Key Insights and Innovations

Innovation 1: Reframing Video Stylization as a Data Problem Rather Than an Architecture Problem

The most consequential intellectual move in this paper is subtle because it appears in Section 3.1 rather than in the architecture description — but it fundamentally reorients how the field should think about video stylization. Prior work in this area (StyleCrafter [29], StyleMaster [56], UniVST [46], TokenFlow [14]) implicitly accepted a premise: video stylization models must work around the absence of paired stylized video data, either by generalizing from image datasets (StyleCrafter, StyleMaster), by propagating image-level stylization through feature correspondences (TokenFlow, AnyV2V), or by operating training-free (UniVST). Each of these approaches represents a clever architectural fix for a data scarcity problem, and each hits a ceiling — StyleMaster's CSD score of 0.198 versus DreamStyle's 0.515 (Table 1) is not a small gap that better hyperparameters would close; it reflects a fundamental limitation of learning video-level style consistency from image-level training data.

DreamStyle's thesis is that this data scarcity is manufactured, not inherent. Paired stylized-raw videos don't exist in nature, but they can be generated systematically by decomposing the problem into two sub-problems where current models are already strong: image stylization (where quality exceeds video models) and image-to-video generation (where motion can be controlled via ControlNets). The data curation pipeline (Fig. 2) is not merely an engineering contribution — it represents a reframing of the research question from "how do we build a model that can stylize videos without video training data?" to "how do we construct video training data that enables a simple model to stylize videos effectively?"

This reframing has an important corollary that the paper demonstrates but does not explicitly theorize: data quality scaling laws matter more than architectural complexity for video stylization. The two-stage CT-then-SFT strategy is essentially a data curriculum — learn from diverse, imperfect data first, then refine with high-quality, filtered data. The ablation in Table 4 confirms the necessity of both stages: CT-only training produces the worst CSD score (0.459) because the data is too noisy, while SFT-only training produces the best CSD score (0.535) but the worst structure preservation (DINO score of 0.483) because the limited dataset size is insufficient to teach the model robust V2V capabilities. The full two-stage approach balances these at 0.515 CSD and 0.526 DINO. This result has practical implications beyond DreamStyle: it suggests that teams working on video stylization (and potentially other video-to-video tasks) should invest more heavily in systematic data generation pipelines rather than more sophisticated architectures, because the data quality ceiling is the binding constraint.

This is a fundamental shift in how to approach the problem, not an incremental architecture refinement. It parallels the recognition in language modeling that data quality and scale matter more than architecture details (the Chinchilla scaling laws insight), translated to the video stylization domain.

---

Innovation 2: Token-Specific LoRA as a Lightweight Mechanism for Multi-Condition Disentanglement

The problem of combining multiple condition types in a single generative model is not new — it appears in controllable image generation (where text, segmentation maps, depth maps, and style references may all need to coexist) and multi-modal video generation. The standard solutions are: (1) train separate models for each condition combination (wasteful and inflexible), (2) use separate adapter modules for each condition type (StyleCrafter's dual cross-attention approach), or (3) rely on the model's attention mechanism to learn routing implicitly (the default when all conditions are concatenated into a single input sequence). Each of these has known failure modes: separate adapters can interfere in the shared backbone, and implicit routing through attention often fails to disambiguate tokens with similar surface statistics but different semantic roles.

DreamStyle's token-specific LoRA represents a diagnostically precise solution to a specific failure mode: inter-token confusion between condition types that enter the model through similar channels (all are pixel-space latents that become tokens after patchification) but serve fundamentally different functions (first frame = temporal anchor at the start of the video, style image = global style reference, raw video = content structure guidance). The mechanism is conceptually clean: a shared down-projection captures what is common across all visual token types (basic spatial features, texture patterns), while type-specific up-projections capture what is distinctive (how a first-frame token should influence generation vs. how a style-image token should influence it).

What makes this innovative is not the mechanism per se — it is an adaptation of HydraLoRA [47] with manual routing — but rather the recognition that this specific confusion is the bottleneck for multi-condition unification. The field's default assumption has been that attention mechanisms in large transformers are powerful enough to learn condition-specific routing implicitly if given sufficient training data. DreamStyle's ablation (Table 4: CSD drops from 0.515 to 0.413 when token-specific LoRA is removed; Fig. 7 shows qualitative "style degradation" and "style confusion") provides concrete evidence that this assumption fails for video stylization — the visual similarity between first-frame tokens, style-image tokens, and raw-video tokens is too high for attention alone to reliably disambiguate them, and explicit structural separation is necessary.

This is an architectural diagnostic insight rather than a novel mechanism: the contribution is identifying what needs to be separated (token types by their functional role in stylization) and why implicit routing fails (visual similarity between token types), which justifies the choice of explicit routing over learned gating. The manual routing design (deterministic assignment based on token origin) is appropriate precisely because token origin is known and unambiguous — learned routing would be unnecessary complexity. This insight generalizes: any multi-condition generative model where conditions enter as visually similar tokens but serve different semantic roles (e.g., in-painting masks vs. depth maps vs. reference images in image editing) may benefit from similar token-type-aware adaptation.

---

Innovation 3: The Two-Tier Dataset Strategy as a Practical Solution to the Diversity-Quality Tradeoff

The paper's construction of two datasets — a large, diverse, automatically filtered CT dataset and a small, high-quality, manually filtered SFT dataset — appears at first glance to be a routine engineering decision. But it encodes a non-obvious strategic insight about video stylization training: diversity and quality in stylization data are anti-correlated in ways that simple filtering cannot resolve.

The anti-correlation arises from a specific structural property of stylization data generation: models that produce high style diversity (like InstantStyle/SDXL for the CT dataset) do so by being image-guided, which allows them to cover an unbounded space of styles (any reference image defines a style), but they produce lower visual quality because SDXL's image generation quality at the time of writing is below the state of the art. Conversely, models that produce high visual quality (like Seedream 4.0 for the SFT dataset) do so through text guidance and better base architectures, but text descriptions constrain the style space to what can be linguistically described — "watercolor style" or "low poly style" captures an artistic category but not the specific brush texture, color balance, or geometric irregularity of a particular watercolor painting or low-poly model. The SFT dataset therefore has higher quality but lower style specificity and diversity.

The CT-then-SFT training strategy is the paper's answer to this anti-correlation: it is not a standard pretrain-then-finetune pipeline where the first stage provides general knowledge and the second stage provides domain specialization. Rather, the CT stage provides diverse but noisy style concepts (teaching the model what styles exist and how they relate to content), while the SFT stage provides clean examples of the style-quality frontier (teaching the model what a high-quality stylization looks like). The model needs both — CT-only fails on style consistency (the clean style signal is too weak amidst noisy data), and SFT-only fails on structure preservation (the model hasn't seen enough diverse content-style combinations to learn robust separation). The full approach achieves a balance that neither dataset alone can provide.

This is an incremental but conceptually important refinement of standard two-stage training. The standard narrative is "more data → better performance, and higher quality → even better." DreamStyle's results complicate this: they show that low-quality data can be actively harmful if it is the only training signal (CT-only has worse CSD than SFT-only), but also that high-quality data alone is insufficient if the dataset is too small to cover the diversity of the task (SFT-only has worse structure preservation than CT-only). The optimal strategy is not to filter more aggressively or collect more data, but to sequence diverse-then-clean data in a curriculum — a finding that may apply to other generative tasks where diversity and quality are similarly anti-correlated (e.g., text-to-image generation with varying prompt complexity, or speech synthesis with varying speaker diversity).

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper uses three separately curated test sets, all constructed by the authors since no standard benchmark exists for video stylization. The text-guided test set contains 50 videos paired with style prompts designed by a professional designer. The style-image-guided test set expands this to 90 videos paired with 15 style images (each style image randomly assigned to 6 videos), and the first-frame-guided test set reuses the same 90 videos with stylized first frames generated via image stylization methods. Section 4.2 details these constructions. No source dataset (e.g., existing video benchmark) is cited as the origin of the raw videos, which is a notable omission — the reader cannot assess whether these 90 videos systematically cover diverse content types, motion patterns, or scene complexities.

• Base model. All experiments use Wan14B-I2V [48], a 14-billion-parameter diffusion transformer model for image-to-video generation. The authors do not justify this specific model choice beyond it being their "in-house I2V model" (Section 3.1). The implications of this choice — for example, whether Wan14B-I2V's particular architectural biases (attention patterns, temporal modeling capacity, resolution limits) affect the generalizability of the approach — are unexplored. No experiments with alternative I2V backbones (e.g., CogVideoX, Stable Video Diffusion) are reported.

• Metrics. The evaluation suite combines automated metrics and human judgment:

  • CSD Score [44] (Conditional Style Distance): measures style consistency between a reference (style image or stylized first frame) and each frame of the generated video. Higher values indicate better style matching. Applied to style-image-guided and first-frame-guided tasks (Table 1).
  • CLIP-T / ViCLIP [50]: measures text-video similarity using only the style prompt for text-guided stylization. The paper notes that CLIP-T values are "overall lower" because style prompts alone are evaluated rather than full content-style descriptions (Section 4.3).
  • DINO Score: cosine similarity of DINOv2 [32] patch features (excluding the CLS token) between the generated and input videos, measuring structure preservation. Higher values indicate better content retention.
  • VBench [22] metrics: five quality dimensions — dynamic degree (motion magnitude), image quality (per-frame visual fidelity), aesthetic quality, subject consistency (identity preservation across frames), and background consistency.
  • User study (Table 3): 20 professional data annotators rate 10, 20, and 20 samples from the text-guided, style-image-guided, and first-frame-guided test sets respectively on 1–5 scales for style consistency, content consistency, and overall quality, using the detailed rubrics in Table 2. The evaluation is blind (presumably meaning annotators do not know which method produced each video).

• Baselines. The baselines differ by task, reflecting the siloed nature of prior work:

  • Text-guided task: three commercial models — Luma [1], Pixverse [2], and Runway [3]. These are closed-source systems with unknown architectures and training data, making the comparison inherently uncontrolled. The paper does not explain how prompts were translated to each system's API.
  • Style-image-guided task: StyleMaster [56], the "only open-source DiT-based method that supports style-image-guided video stylization in an end-to-end manner." The paper notes StyleMaster's incomplete open-source release prevents V2V evaluation, so the comparison includes StyleMaster (T2V) against DreamStyle in both T2V and V2V modes (Table 1).
  • First-frame-guided task: VACE [23] and VideoX-Fun [4]. Both are open-source or accessible systems, though VideoX-Fun is cited only as a project page with minimal technical documentation.

• Generation budget / compute accounting. The paper does not report generation budgets or compute costs in a standardized way. All models produce videos through a single forward pass (no iterative refinement, no best-of-N sampling, no search), so generation budget is implicitly 1 per sample. The number of denoising steps, classifier-free guidance scales, or other inference-time hyperparameters are not disclosed. This is a significant gap for reproducibility and for fair comparison with commercial systems (whose inference parameters are unknown but may differ substantially).

• Cross-validation / statistical protocol. No cross-validation, statistical significance testing, or confidence intervals are reported. All metrics in Table 1 are point estimates without error bounds. The user study (Table 3) reports mean ratings but no standard deviations or inter-annotator agreement metrics (e.g., Krippendorff's alpha). This makes it impossible to assess whether the reported differences are statistically reliable or within the noise floor of human judgment variability.

Main Quantitative Results

Text-Guided Video Stylization

The headline results for text-guided stylization appear in Table 1 (top section). DreamStyle achieves the highest CLIP-T score (0.167) among all methods, compared to Luma (0.132), Pixverse (0.155), and Runway (0.154). The DINO score — measuring structure preservation — is 0.584 for DreamStyle versus 0.406, 0.451, and 0.504 for the three commercial baselines, representing a 15.9% relative improvement over the strongest competitor (Runway). For VBench quality metrics, DreamStyle leads on most dimensions: dynamic degree (0.894 vs. a baseline maximum of 0.809), aesthetic quality (0.656 vs. 0.628), subject consistency (0.952 vs. 0.948), and background consistency (0.956 vs. 0.951). Image quality is the exception — DreamStyle scores 0.738, slightly below Pixverse (0.746) but above Luma (0.739) and Runway (0.725).

The user study (Table 3, top section) reinforces these rankings: DreamStyle achieves mean scores of 4.14 (style consistency), 3.95 (content consistency), and 3.95 (overall quality) on the 1–5 scale, substantially ahead of Pixverse (2.83, 2.95, 2.82) and Runway (2.52, 2.80, 2.59). The style consistency gap — 4.14 versus a next-best of 2.83 — is particularly large, suggesting that DreamStyle's trained approach captures style more faithfully than the commercial systems' presumably prompt-engineering-based stylization.

However, the commercial baselines are fundamentally not comparable in a controlled sense: their architectures, training data, and inference procedures are unknown. Luma may optimize for different tradeoffs (e.g., motion smoothness over style precision), and the prompts as submitted to each system may not elicit equivalent stylization behavior. The fact that Luma achieves a substantially lower CLIP-T (0.132) than all other methods could reflect a prompt parsing difference rather than a genuine capability deficit. The paper does not control for this by, for example, tuning prompts per system to maximize each one's stylization quality.

Style-Image-Guided Video Stylization

Table 1 (middle section) presents the style-image-guided results. DreamStyle (V2V mode) achieves a CSD score of 0.515, compared to StyleMaster's 0.198 — a 2.6× improvement that is the paper's most striking quantitative result. DreamStyle in T2V mode (where the raw video condition is set to empty) achieves an even higher CSD of 0.532, though this mode was not explicitly trained for and the authors note it "naturally inherits this capability from the base model." The DINO score is not reported for StyleMaster due to its incomplete open-source release, making structure preservation comparison impossible.

On VBench metrics, DreamStyle (T2V) dramatically outperforms StyleMaster on image quality (0.689 vs. 0.289) and dynamic degree (0.689 vs. 0.289 — the numbers are coincidentally identical, though they represent different metrics). StyleMaster achieves comparable subject consistency (0.723 vs. 0.722) and slightly higher background consistency (0.935 vs. 0.950 for DreamStyle T2V). DreamStyle (V2V) shows somewhat lower image quality (0.704) and background consistency (0.938) than its T2V variant, which the paper attributes to "fewer constraints" in T2V mode.

The user study (Table 3) is decisive: DreamStyle scores 4.36 on style consistency versus StyleMaster's 1.17 — a gap of over 3 points on a 5-point scale that indicates annotators found StyleMaster's stylizations barely recognizable as matching the reference style. The overall quality score is 4.20 for DreamStyle versus 1.31 for StyleMaster.

What makes this comparison particularly informative is that StyleMaster represents the state of the art among open-source style-image-guided video stylization methods — it is not a weak baseline chosen to inflate DreamStyle's apparent advantage. The 0.198 CSD score and 1.17 user study rating genuinely reflect the current capability ceiling for methods trained without stylized video data. DreamStyle's improvement comes from fundamentally breaking that ceiling by constructing video training data — exactly the paper's central strategic argument.

First-Frame-Guided Video Stylization

For first-frame guidance (Table 1, bottom section), DreamStyle achieves a CSD score of 0.851, compared to 0.689 for VACE and 0.766 for VideoX-Fun — a 11.1% improvement over the stronger baseline. The DINO score tells a different story: DreamStyle's 0.640 is below both VACE (0.716) and VideoX-Fun (0.702), meaning DreamStyle trades off some structure preservation for better style consistency. The paper acknowledges this explicitly: "Since the stylized first frames (especially those with geometric deformation) occasionally conflict with the structure of the input video, our method, despite its superior style consistency, is inferior to VACE and VideoX-Fun in structure preservation (DINO score)."

On VBench metrics, DreamStyle leads in aesthetic quality (0.630 vs. 0.573 and 0.594) and subject consistency (0.919 vs. 0.922 and 0.915 — essentially tied with VACE), but trails in dynamic degree (0.856 vs. 0.889 for VACE) and background consistency (0.932 vs. 0.930 and 0.924 — roughly tied). The user study (Table 3) shows DreamStyle with 4.37 style consistency (vs. 2.35 and 3.19) and 4.24 overall quality (vs. 2.79 and 3.42), with a small edge in content consistency (4.12 vs. 4.30 and 4.22).

This pattern — superior style consistency at the cost of slightly reduced structure preservation — is the expected behavior for a system that handles geometric deformation styles. When a low-poly style fundamentally changes the shape of objects, the DINO features of the stylized video will differ from the raw video's features even when content is perfectly preserved, because DINOv2 is not invariant to geometric transformations. The lower DINO score may partially reflect successful geometric stylization rather than content distortion. The paper notes this qualitative evidence (Fig. 5) but does not quantify the geometric deformation vs. content distortion distinction, which would require a more nuanced structural metric.

Extended Applications

The extended applications (Section 4.5) are presented qualitatively without formal metrics. Fig. 6 shows multi-style fusion examples (watercolor + metal + low poly + crayon drawing styles combined from text and image inputs), and Fig. 8 shows two long-video stylization examples (3D cartoon style and colored pencil style, both exceeding the base model's 5-second limit). The paper does not report quantitative evaluations for these scenarios — no CSD scores for how style consistency is maintained across segment boundaries in long videos, no user study for multi-style fusion quality. This is understandable since both are emergent capabilities rather than trained behaviors, but it means the claims about these applications (Section 1: "a crucial design for improving the effectiveness and controllability") are supported by demonstration rather than rigorous comparison.

Ablation Studies and Robustness Checks

• Token-specific LoRA vs. standard LoRA: Removing the token-specific up-matrices and using a single shared LoRA degrades the CSD score from 0.515 to 0.413 on the style-image-guided task (Table 4) — a 19.8% relative drop. The DINO score also declines from 0.526 to 0.518, a smaller but consistent degradation. Qualitative results (Fig. 7) show two specific failure modes: style degradation (the output fails to maintain the reference style characteristics) and style confusion (elements from different conditions visually bleed into each other). This ablation confirms that inter-token confusion is a real and consequential problem, not merely a theoretical concern.

• Training data ablation — CT-only: Training exclusively on the 40K CT dataset (Table 4) yields a CSD score of 0.459 (vs. 0.515 for the full two-stage approach) and a DINO score of 0.547 (vs. 0.526). The lower CSD and higher DINO suggest that CT-only training produces videos that are structurally faithful to the input but stylistically weaker — the model learns content preservation effectively from the large dataset but cannot reach high style consistency due to the CT data's quality limitations. The qualitative example in Fig. 9 (middle left) confirms this: the stylized output fails to render the pixel art pattern that the style reference specifies, producing a blurred approximation instead.

• Training data ablation — SFT-only: Training exclusively on the 5K SFT dataset (Table 4) produces the highest CSD score (0.535) but the lowest DINO score (0.483). This is the mirror image of the CT-only result: strong style consistency but poor structure preservation. The qualitative example in Fig. 9 (middle right) shows the panda's pose changing relative to the input — a content distortion that the higher-quality but smaller dataset cannot prevent because the model hasn't seen enough diverse content-style combinations to learn robust separation. The full two-stage approach (Table 4, bottom row) achieves the intended balance — CSD of 0.515 and DINO of 0.526 — representing a compromise position that sacrifices some style consistency relative to SFT-only to recover structural fidelity.

This ablation is particularly informative because it demonstrates the non-monotonic relationship between data quality and model performance that the paper's two-stage strategy exploits. If data quality were simply monotonic (higher quality → better all-around performance), SFT-only would dominate both CSD and DINO. The fact that it produces the best CSD but worst DINO confirms the diversity-quality anti-correlation discussed in Section 4 (Innovation 3). The CT dataset's diversity teaches structure preservation; the SFT dataset's quality teaches style consistency. Both are necessary.

Critical Assessment

Claim 1: DreamStyle is a unified framework supporting text, style-image, and first-frame guidance

What the experiments demonstrate: The results in Table 1 and Fig. 5 show DreamStyle performing competitively or superiorly on each of the three tasks individually, compared against task-specific baselines. The model was explicitly trained on all three conditions through the loss terms in Equation 4, and the token-specific LoRA ablation validates that the multi-condition architecture is functional.

What remains unaddressed: The experiments do not test joint usage of multiple conditions at inference time except through qualitative demonstrations (Figs. 6, 8). There is no quantitative evaluation of multi-style fusion — no comparison against applying single-condition methods sequentially, no metrics for how well the fused style matches both the text description and the style image simultaneously. The claim that multi-condition guidance "improves the effectiveness and controllability of video stylization" (Section 1) is asserted but not measured. The paper would be stronger with even a small-scale user study comparing single-condition vs. multi-condition outputs for the same target style.

Additionally, the "unified framework" claim is slightly weakened by the fact that the training ratios favor style-image guidance at 2:1 over the other two tasks (Section 3.3). This means the model is not equally optimized for all conditions — it receives twice as much training on style-image-guided samples. Whether the reported performance on text-guided and first-frame-guided tasks would hold with balanced training ratios is untested.

Claim 2: The data curation pipeline overcomes the scarcity of high-quality paired video data

What the experiments demonstrate: The ablation study (Table 4, Fig. 9) provides strong evidence that the data pipeline is effective and that both dataset stages contribute meaningfully to final performance. The comparison with StyleMaster — which represents the ceiling of methods without stylized video data — shows a 2.6× CSD improvement, directly validating the paper's thesis that data construction is the binding constraint.

What remains unaddressed: The experiments do not analyze the failure modes of the data pipeline itself. What proportion of generated CT samples are rejected by the CSD > 0.5 filter? What styles or content types does the pipeline systematically fail to produce? Are there style categories (e.g., photorealistic styles, abstract expressionism, specific cultural artistic traditions) that the image stylization models cannot generate, and therefore DreamStyle cannot learn? The paper presents the pipeline as a solved problem, but any synthetic data pipeline has coverage gaps that bound the trained model's capabilities. Characterizing those gaps would strengthen the contribution and help practitioners understand when DreamStyle is likely to fail.

A related concern: the pipeline uses the same I2V model family (the authors' in-house I2V model with customized ControlNets) that serves as DreamStyle's base architecture. This creates a potential circularity — the training data's motion characteristics are determined by the same architecture that will be trained on that data. If the in-house I2V model has systematic motion artifacts (e.g., it smooths out rapid motion, it struggles with occlusions, it produces characteristic temporal flicker patterns at certain frame rates), DreamStyle will learn those artifacts as "correct" video behavior during training. Training a model on its own outputs is a known degradation pattern in generative AI; the paper does not discuss this risk.

Claim 3: Token-specific LoRA mitigates interference among different condition tokens

What the experiments demonstrate: The ablation (Table 4) and qualitative evidence (Fig. 7) clearly show that removing token-specific LoRA degrades performance, particularly on style consistency (CSD drops from 0.515 to 0.413). This is a well-designed ablation that isolates the mechanism's contribution.

What remains unaddressed: The ablation compares only token-specific LoRA versus standard LoRA. A middle-ground experiment — token-specific LoRA with learned routing (a lightweight gating network that decides which up-matrix to apply based on token content rather than origin) — would help distinguish whether the benefit comes from having separate up-matrices per se, or from the specific manual-routing design. If learned routing performs equivalently, the contribution is in having separate transformations rather than in the routing mechanism. If learned routing underperforms, it would validate the paper's claim that manual routing is the correct design choice because token origin is unambiguous. This experiment is not reported.

The paper also does not ablate the rank of the LoRA (r = 64) or the decision to apply it to both attention and FFN layers. It is possible that a lower rank (r = 32) would perform similarly, or that applying LoRA only to attention layers would suffice, which would reduce the number of trainable parameters. These are standard LoRA ablation dimensions that would strengthen the architectural contribution claims.

Claim 4: DreamStyle outperforms specialized competitors across all three tasks

What the experiments demonstrate: The quantitative comparisons in Table 1 and the user study in Table 3 consistently place DreamStyle at or near the top across all metrics and all tasks. The style-image-guided comparison against StyleMaster is particularly convincing because StyleMaster is a strong, recent open-source baseline with a 2.6× CSD gap that cannot be explained by hyperparameter tuning.

What remains unaddressed: The comparison for text-guided stylization is fundamentally problematic. The baselines (Luma, Pixverse, Runway) are commercial black boxes. We do not know:

• What base model each uses (and therefore how many parameters, what training data, what architecture).
• Whether they were designed for stylization or whether this is an off-label use of general-purpose video generation.
• Whether the prompts submitted to each system were optimized for that system's particular prompt interface (length limits, keyword sensitivity, negative prompt support).

A fair comparison would require open-source text-guided video stylization baselines, or at minimum a systematic prompt engineering sweep per commercial system to find the best prompt for each style. The paper provides no evidence that such optimization was performed. The CLIP-T gap between Luma (0.132) and other methods (all above 0.15) could reflect Luma requiring different prompt formatting rather than genuinely inferior stylization.

Additionally, for the first-frame-guided task, VideoX-Fun [4] is cited only as "a more flexible framework that can generate videos at any resolution and creates videos from images" — a project page description rather than a peer-reviewed method. Its technical details are opaque, making the comparison difficult to interpret.

Claim 5: The unified design enables extended applications (multi-style fusion, long-video stylization)

What the experiments demonstrate: Figs. 6 and 8 provide qualitative examples of these capabilities. The long-video examples in Fig. 8 show seamless concatenation across segment boundaries, and the multi-style fusion examples in Fig. 6 show plausible combinations of text-described and image-specified styles.

What remains unaddressed: These are demonstrations, not evaluations. The paper reports no quantitative metrics for either application. For multi-style fusion, there is no measurement of how well the output balances the text prompt's style description with the style image's visual characteristics — do certain style combinations produce degraded outputs? Is there a style interference effect where one modality dominates? For long-video stylization, there is no measurement of temporal consistency at segment boundaries — do CSD scores drop at concatenation points? Do human observers notice the seams? These experiments would be straightforward to conduct (compute CSD per-segment and at boundaries, run a small boundary-detection user study) and their absence means the extended application claims remain at the level of technical feasibility demonstration rather than validated capability.

Overall Assessment of Experimental Rigor

The experiments are strongest where they test the paper's core strategic claim — that constructing stylized video training data is the binding constraint, and that DreamStyle's data pipeline produces models superior to data-scarce alternatives. The StyleMaster comparison and the data ablation study provide compelling evidence for this thesis.

The experiments are weakest in three areas:

1. Reproducibility: the custom test sets (50, 90 videos) are not released or described in sufficient detail for replication. The base model (Wan14B-I2V) is presumably available, but the "in-house I2V model" and customized ControlNets used in the data pipeline are not. This makes DreamStyle unreproducible without access to ByteDance's internal infrastructure.

2. Statistical rigor: no confidence intervals, no significance tests, no inter-annotator agreement metrics for the user study. The 20-annotator user study with 10/20/20 samples per task is relatively small (10 samples × 20 annotators × 4 methods = 800 pairwise judgments for text-guided stylization, which is reasonable but not large). Without variance estimates, the claimed margin of superiority could be within sampling noise, particularly for the VBench metrics where differences are often in the third decimal place (e.g., subject consistency of 0.952 vs. 0.948).

3. Baseline fairness: the use of commercial black-box systems for text-guided comparison and the opaque VideoX-Fun for first-frame comparison weakens the interpretability of those results. The paper would be stronger with open-source, replicable baselines for all tasks, even if those baselines performed worse — the point of a baseline is not to be strong but to be well-understood.

An experiment that would significantly strengthen the paper: training DreamStyle on a different I2V base model (e.g., CogVideoX or Stable Video Diffusion) to demonstrate that the approach generalizes beyond Wan14B-I2V. The paper claims a "unified framework" — but it has been demonstrated with exactly one base architecture. If the condition injection mechanism and token-specific LoRA work with any DiT-based I2V model, that is a much stronger claim than "works with Wan14B-I2V." This experiment is not reported.

6. Limitations and Trade-offs

Synthetic Data Pipeline Creates a Quality Ceiling Bounded by Image Stylization Models

The assumption or constraint. DreamStyle's entire training methodology rests on the premise that high-quality paired stylized-raw video data can be generated by decomposing the problem into image stylization followed by image-to-video animation (Section 3.1). The image stylization step uses two specific models: InstantStyle/SDXL for the CT dataset and Seedream 4.0 for the SFT dataset. The paper explicitly acknowledges the asymmetry that motivates this design: "current image generation / editing models are superior to the video counterpart in terms of visual quality, structure, aesthetics and text following." However, this superiority is relative, not absolute — these image models have their own systematic limitations in style coverage, rendering fidelity, and content preservation.

The consequence. DreamStyle cannot learn to stylize in ways that the upstream image stylization models cannot produce. If InstantStyle/SDXL fails to render certain artistic traditions (e.g., specific cultural calligraphy styles, architectural drawing conventions, or highly specialized illustration techniques), or if Seedream 4.0 misinterprets certain style prompts, those failure modes are baked into the training data. The model may learn to reproduce the image models' characteristic artifacts — the particular way InstantStyle handles edge transitions, or Seedream's color palette biases — as if they were inherent style properties. More subtly, because the CT dataset's automatic filtering uses a CSD threshold of > 0.5 (Section 3.1), the training data contains stylized videos that pass a style similarity check but may still contain subtle inconsistencies (flickering textures, spatially varying style intensity, temporal coherence breakdowns in complex motion segments) that the CSD metric does not penalize. DreamStyle is trained to reproduce these as "correct" stylizations.

What evidence exists in the paper. The data ablation in Table 4 provides indirect evidence: CT-only training achieves a CSD score of 0.459 versus the full two-stage approach at 0.515, confirming that the CT dataset's quality is insufficient to reach the performance ceiling. However, the paper does not characterize which styles or content types the data pipeline fails to generate, nor does it report the rejection rate of the CSD > 0.5 filter. The qualitative results in Fig. 5 show DreamStyle succeeding on a curated set of styles (paper art, colored pencil, low poly, pixel art), but the test set construction — 50 or 90 videos with styles chosen by the authors — provides no systematic coverage analysis. The VBench image quality scores (0.738 for DreamStyle text-guided, 0.704 for V2V style-image-guided, Table 1) are competitive but not dominant (Pixverse scores 0.746), suggesting that image-level quality from the pipeline is good but not state-of-the-art.

Mitigation status. The paper does not address this limitation directly. The two-stage training strategy (CT then SFT) mitigates the CT data quality issue by refining with manually filtered SFT data, but this only raises the quality ceiling to whatever Seedream 4.0 can produce — it does not break through it. The paper does not propose training on outputs from multiple diverse image stylization models, incorporating real stylized video when available, or using reinforcement learning from human feedback to push beyond the quality of the training data distribution. The limitation is structural: any model trained on synthetic data is bounded by the synthesizer's capability. Characterizing that bound — through a systematic failure analysis of the data pipeline across style categories — would be a natural next step that the paper does not undertake.

---

The Method Is Demonstrated on a Single Base Architecture with No Cross-Model Validation

The assumption or constraint. DreamStyle is built exclusively on Wan14B-I2V [48], a 14-billion-parameter DiT-based image-to-video model developed internally at ByteDance. The paper's contributions — the channel-wise condition injection mechanism, the token-specific LoRA module, and the two-stage CT-then-SFT training strategy — are presented as a "unified framework" (Section 1) without qualification that the framework has been validated across multiple base architectures. The condition injection design relies on specific properties of Wan14B-I2V: its image condition channels before the patchify layer, its CLIP image feature branch, its mask channel conventions (4 mask channels with values 0.0 and 1.0), and its native support for first-frame conditioning.

The consequence. It is unknown whether the approach generalizes to other I2V architectures. A practitioner using CogVideoX [53], Stable Video Diffusion [5], HunyuanVideo [25], or any other DiT-based I2V model cannot assume that DreamStyle's design transfers without modification. Each base model has different input channel conventions, different conditioning mechanisms (some use cross-attention for image conditioning rather than channel concatenation, some lack a CLIP image branch entirely), and different pretrained capabilities. The token-specific LoRA design assumes that the base model's patchification produces tokens that are positionally and functionally distinct enough for manual routing to be effective — this may not hold for architectures with different patchification strategies or attention patterns. Furthermore, Wan14B-I2V's maximum generation length of 81 frames (~5 seconds) is a hard architectural constraint that DreamStyle inherits. The long-video stylization workaround (Fig. 8) concatenates segments but does not alter the model's fundamental temporal context window — it is a post-hoc stitching operation, not true long-form temporal modeling.

What evidence exists in the paper. The paper provides zero cross-model experiments. All results in Table 1, the user study in Table 3, the ablation in Table 4, and the qualitative examples in Figs. 5-9 use the same Wan14B-I2V backbone. The paper does not even discuss the portability of the approach — the term "unified framework" refers to unification across style conditions, not across base architectures. Section 4.1 notes that training uses "NVIDIA GPUs" without specifying the model or count required, making it difficult for practitioners to assess whether the approach is feasible with their compute resources on a different base model. The paper also uses "in-house I2V model" for the data pipeline's ControlNet-guided animation step (Section 3.1), which is not the same as Wan14B-I2V but presumably shares architectural lineage — further tying the entire pipeline to ByteDance's internal model ecosystem.

Mitigation status. The paper does not acknowledge this as a limitation or suggest cross-model validation as future work. The authors present DreamStyle as a general framework, but the evidence supports a more specific claim: DreamStyle is a set of design principles (channel-wise condition injection, token-specific LoRA, synthetic data pipeline) that are demonstrated to work effectively with Wan14B-I2V. Whether these principles are sufficient for other architectures — or whether they depend on Wan14B-I2V-specific properties — is an open question that the paper leaves entirely to future investigation. In practice, a team wanting to adopt DreamStyle with a different base model would need to re-implement the condition injection mechanism to match their model's input conventions and re-tune the token-specific LoRA design (rank, which layers to apply it to, routing strategy) for their architecture, with no guidance from the paper on how to do so.

---

The Difficulty of Content-Style Separation Is Not Characterized, and Failure Modes for Challenging Content Are Unreported

The assumption or constraint. DreamStyle assumes that the raw video provides sufficient information for the model to separate content (what is in the scene, how it moves) from style (how it is rendered). The channel-wise concatenation of raw and stylized video latents (Equation 3) is the architectural mechanism for this separation — the model sees the raw video alongside the noisy stylized video at every denoising step and must learn to preserve content structure while transforming style. The training data is constructed to teach this separation: paired videos where the content is identical (same subjects, same motions) but the style differs. However, the paper does not analyze when this separation succeeds or fails.

The consequence. For certain types of content, the style-content separation may be ill-posed or significantly harder than the aggregate metrics suggest. Consider:

• Fine-grained textures that carry semantic content: if the input video shows a person wearing clothing with specific text or logos, a stylization that transforms textures could render that text illegible — is that style transformation or content distortion? The model has no explicit mechanism to distinguish semantically meaningful textures from decorative ones.
• Geometric deformation styles: the paper notes (Section 1, discussing UNIC) that "styles involving geometric deformation" are particularly challenging because the ControlNet-based data pipeline "fails to handle" them due to "strict alignment." DreamStyle improves on this (Fig. 5 shows low-poly style successfully), but the first-frame-guided results in Table 1 reveal a structural tradeoff: DreamStyle's DINO score (0.640) is lower than both VACE (0.716) and VideoX-Fun (0.702), meaning it sacrifices more content structure to achieve style consistency. The paper acknowledges this for first-frame guidance: "the stylized first frames (especially those with geometric deformation) occasionally conflict with the structure of the input video." However, no systematic analysis of which geometric deformations succeed and which fail is provided.
• Rapid or complex motion: the data pipeline uses depth and pose ControlNets to enforce motion consistency, but the paper notes (Section 3.1, Fig. 3 discussion) that "neither depth nor pose can fully capture the complex motion dynamics of the raw video, ultimately resulting in motion mismatches." Videos with occlusions, fast camera movements, non-rigid deformations (flowing water, swaying trees, expressive dance), or complex multi-object interactions may produce training pairs where the motion in $x_{\text{raw}}$ and $x_{\text{sty}}$ does not align, teaching the model spurious content-motion correlations.

What evidence exists in the paper. The aggregate metrics in Table 1 (DINO, CSD, VBench) mask per-sample variation. The DINO score of 0.526 for style-image-guided V2V mode is an average — some videos likely score much lower. The qualitative examples in Fig. 5 are curated successes; no failure cases are shown (unlike the ablation figures, Figs. 7 and 9, which do show failures). The user study (Table 3) reports content consistency scores of 3.95 (text-guided), 3.87 (style-image-guided), and 4.12 (first-frame-guided) on a 1-5 scale — these are good but not perfect, indicating that content distortions occur in evaluated samples, but the study does not categorize what types of content distortions happened or how severe they were.

Mitigation status. The paper provides no systematic failure mode analysis. There is no breakdown of metrics by content type (human subjects vs. scenes vs. objects), by motion complexity (static camera vs. panning vs. handheld), or by style category (color/texture transfer vs. geometric deformation). The SFT dataset's manual filtering (Section 3.1) removes samples where "content consistency between each raw video and its stylized video" fails, which should reduce but not eliminate training data contamination from motion-mismatched pairs. Beyond this filtering, the model has no runtime mechanism to detect or compensate for content-style separation failures — it produces a single output regardless of input difficulty, with no confidence estimate or fallback behavior.

---

Test-Time Compute and Inference Costs Are Not Characterized or Compared

The assumption or constraint. The paper treats DreamStyle as a single-forward-pass model: one input video, one set of style conditions, one output stylized video. There is no iterative refinement, no best-of-N sampling, and no adaptive computation. The paper reports no inference-time metrics — no wall-clock time per video, no GPU memory requirements, no FLOPs count, no comparison of inference cost against baseline methods. The training hyperparameters (Section 4.1) specify per-GPU batch size of 1, gradient accumulation of 2 steps, and an effective batch size of 16, but the number of GPUs and the GPU type (beyond "NVIDIA GPUs") are not disclosed. For inference, the number of denoising steps, the classifier-free guidance scale, and any other sampling parameters are unreported.

The consequence. A practitioner evaluating DreamStyle for deployment cannot assess the latency-throughput tradeoff. Several specific concerns:

• Base model scale: Wan14B-I2V is a 14B-parameter DiT model. Generating 81 frames at 480P resolution with a diffusion transformer of this size is computationally expensive — likely requiring high-end datacenter GPUs (A100 or H100 class) and taking seconds to minutes per video depending on the number of denoising steps. The paper provides no guidance on whether the 14B scale is necessary or whether smaller I2V models would suffice.

• Comparison fairness with baselines is unknown: The commercial baselines Luma, Pixverse, and Runway presumably run on optimized inference infrastructure with unknown model sizes and compute budgets. DreamStyle may achieve its superior style consistency (Table 1) partly by using a larger model or more denoising steps than the commercial systems — but there is no way to know because neither side's compute is reported. Similarly, StyleMaster's inference cost relative to DreamStyle is unknown; the 2.6× CSD improvement may come with a proportional or disproportionate increase in inference compute.

• Long-video stylization amplifies cost: The long-video workaround (Section 4.5, Fig. 8) generates segments sequentially and concatenates them. For a video that is $k$ segments long, inference cost scales linearly with $k$ (each segment requires a full forward pass), and the sequential dependency (each segment's first frame comes from the previous segment's last frame) prevents parallelization. This makes long-video stylization $k \times$ more expensive than single-segment stylization in both compute and wall-clock time.

What evidence exists in the paper. The paper reports zero inference-time measurements. There is no table of inference speeds, no comparison of parameter counts across methods, and no discussion of the computational tradeoffs involved in choosing Wan14B-I2V as the base architecture. The training configuration (6,000 + 3,000 iterations with effective batch size 16 on an unspecified number of NVIDIA GPUs) provides some information about training cost but none about inference cost.

Mitigation status. The paper does not acknowledge this as a limitation. The focus is entirely on output quality (style consistency, content preservation, visual fidelity) with no consideration of the computational budget required to achieve that quality. This is common in generative vision research — quality metrics dominate, and efficiency is an afterthought — but it means the paper's practical claims ("a flexible and practical tool for users to create high-quality stylized videos," Section 1, Fig. 1 caption) are not supported by evidence about practicality in terms of time or hardware requirements. A simple addition — reporting inference time on a specified GPU, denoising steps used, and peak memory consumption — would substantially improve the paper's utility for deployment-minded readers.

---

The Evaluation Protocol Has No Statistical Rigor, and the Test Sets Are Proprietary

The assumption or constraint. The paper constructs custom test sets for all three tasks: 50 videos for text-guided, 90 videos for style-image-guided and first-frame-guided (Section 4.2). The raw videos are not drawn from a standard benchmark, and their provenance, content distribution, and motion characteristics are not described. The test sets are not released, and the paper provides no mechanism for other researchers to reproduce the evaluation. Beyond test set opacity, the paper reports all quantitative results as point estimates without any measure of uncertainty: Table 1 provides CSD, DINO, and VBench scores to three decimal places with no standard deviations; Table 3 provides user study means to two decimal places with no inter-annotator agreement statistics; and no hypothesis tests or confidence intervals are reported for any comparison.

The consequence. The paper's quantitative claims — that DreamStyle "outperforms the competitors in style consistency and video quality" (Abstract), that it achieves "competitive performance against specialized models across various video stylization tasks" (Section 1) — are supported by numerical differences that cannot be evaluated for statistical or practical significance. Specific concerns:

• VBench metric precision: Many VBench comparisons in Table 1 show differences in the third decimal place (e.g., subject consistency of 0.952 vs. 0.948, background consistency of 0.956 vs. 0.951). Without variance estimates, it is impossible to determine whether these differences are meaningful or within the noise floor of the metrics. VBench metrics are computed on finite video samples and are subject to sampling variance; a difference of 0.004 on a 500-frame evaluation might reverse with a different random seed or a slightly different test set.

• User study sample sizes: The user study (Table 3) uses 10, 20, and 20 samples per task with 20 annotators. For text-guided stylization, this means only 10 videos × 4 methods = 40 rated outputs. The standard error of a mean on a 1-5 scale with 20 annotators per sample is approximately $\sigma / \sqrt{20}$ — if the within-sample standard deviation is 1.0 (a typical value for subjective ratings), the standard error is ~0.22, meaning 95% confidence intervals span ~0.88 points. Differences of 0.2-0.3 points between methods (e.g., DreamStyle 3.95 vs. Pixverse 2.82 for text-guided overall quality) are clearly significant, but finer comparisons (e.g., DreamStyle 4.12 vs. VACE 4.30 for first-frame content consistency) are not reliably distinguishable.

• Test set coverage: With only 50-90 test videos whose content distribution is undocumented, it is unknown whether the results generalize beyond the specific video types in the test set. If the 90 videos heavily feature talking-head footage with static backgrounds, the results may not transfer to videos with complex scene dynamics, multiple moving objects, or rapid camera motion.

• Reproducibility: Without access to the test sets, the baseline outputs, or the trained models, no independent researcher can verify the paper's quantitative claims. This is a significant barrier for a paper that positions itself as establishing a new state of the art.

What evidence exists in the paper. The gap between point estimates and rigorous statistical reporting is visible throughout Section 4. Table 1 reports CSD scores to three decimal places (0.515, 0.198) with no indication of measurement uncertainty. Table 4 reports ablation CSD scores (0.413, 0.459, 0.535, 0.515) the same way. The paper provides no discussion of metric reliability, no error bars in any figure, and no power analysis for the user study.

Mitigation status. The paper does not acknowledge the absence of statistical rigor as a limitation. The reporting style — precise numbers without uncertainty quantification — follows common practice in generative vision research, but it is nonetheless a methodological weakness that affects the strength of the paper's comparative claims. The user study at least provides a human evaluation dimension that complements automated metrics, but without inter-annotator agreement metrics (e.g., Krippendorff's alpha or intraclass correlation), the reliability of those human judgments is unknown. The paper would be significantly strengthened by: (1) releasing the test set or using a public benchmark; (2) reporting standard deviations or confidence intervals for all metrics; (3) conducting significance tests for key comparisons (especially the DreamStyle vs. StyleMaster CSD gap, which is the paper's most important quantitative result).

---

The Revision Model Analogy Does Not Apply Here; There Is No Mechanism for Iterative Refinement or Error Recovery

The assumption or constraint. DreamStyle produces stylized videos through a single forward pass of the diffusion model: noise is sampled, conditions are injected, and the model iteratively denoises from the noise distribution to the stylized video distribution. There is no mechanism for the model to examine its own output, detect stylization errors (e.g., a frame where the style degraded, a region where content was distorted), and refine the output. This is fundamentally different from approaches that use iterative revision (like the revision model in the reference example paper) or test-time optimization (like the video stylization method UniVST [46], which uses DDIM inversion for feature-level guidance, or TokenFlow [14], which uses DDIM inversion for frame-to-frame propagation). DreamStyle is a feed-forward stylizer: one input, one output, no feedback loop.

The consequence. If DreamStyle produces a stylized video with a quality flaw — a brief segment where the style flickers, a frame where a face is distorted, a region where the style fails to render correctly — there is no recovery mechanism. The user cannot provide feedback ("this frame looks wrong, try again on just this segment") and have the model refine its output. The only recourse is to re-run the entire stylization with different random noise, different style conditions, or different inference parameters (if exposed), which is both computationally wasteful (re-stylizing frames that were already acceptable) and offers no guarantee of improvement (the new output could have the same or different artifacts).

This limitation matters for two practical reasons:

1. Quality-critical applications: In film production, advertising, or professional content creation, a single flawed frame in an otherwise perfect stylized video can render the entire output unusable. A system that cannot localize and correct errors has limited utility for high-stakes creative workflows where the user expects to iterate toward a polished result.

2. Hard cases may produce consistently flawed outputs: For content types or styles where DreamStyle's single-pass performance is weak (the paper does not characterize when this occurs, but the less-than-perfect user study scores of 3.87-3.95 for content consistency in Table 3 indicate it does occur), there is no mechanism to invest additional computation to improve quality. The model's quality ceiling for a given input is fixed — unlike systems that can trade compute for quality through best-of-N sampling, iterative refinement, or verifier-guided search.

What evidence exists in the paper. The paper provides no analysis of output consistency across multiple runs with different random seeds. If DreamStyle is run twice on the same input with the same conditions but different initial noise, do both outputs have similar style consistency and content preservation? Or does quality vary substantially between runs? This is a basic robustness property that is unmeasured. The user study (Table 3) samples one output per method per video, so annotators see a single realization — they cannot assess whether DreamStyle's quality is stable or highly variable.

The qualitative results (Figs. 5, 6, 7, 8, 9) show single outputs per condition. Fig. 7 shows failure cases from the token-specific LoRA ablation — but these are training-time failures (the model never learned the correct behavior), not inference-time failures that could be corrected by re-sampling. The paper does not demonstrate or discuss temporal consistency within a single output (e.g., does the style remain stable across all 81 frames, or does it degrade toward the end?) — the CSD score is computed per-frame and presumably averaged, but this average could mask late-sequence degradation.

Mitigation status. The paper does not acknowledge the lack of iterative refinement as a limitation, and it does not suggest incorporating revision or error-correction mechanisms in future work. This is understandable — the paper's contribution is a unified multi-condition stylization framework, not an exploration of test-time strategies — but it means that DreamStyle inherits the "one-shot" limitations of standard diffusion model inference. For practitioners, this means that quality assurance is entirely dependent on the model's first-pass reliability, with no architectural support for detecting or correcting problems post-generation. The long-video stylization workaround (Fig. 8) demonstrates segment concatenation but not segment-level quality assessment or selective regeneration — each segment is generated once, and if a segment boundary shows a style discontinuity, the only option is to manually identify the problematic segment and re-generate it, which requires the user to inspect the full video for artifacts.

7. Implications and Future Directions

How This Work Changes the Landscape

DreamStyle represents an architectural reframing with strategic consequences, not a paradigm shift. The paper does not introduce a fundamentally new model class or training objective — it builds squarely on an existing I2V diffusion transformer, trains with standard flow matching, and uses LoRA adaptation. What changes is the research community's understanding of the binding constraint in video stylization. Prior to this work, the dominant assumption was that video stylization required clever architectural workarounds for data scarcity — adapter-based style injection (StyleCrafter, StyleMaster), feature-level propagation (TokenFlow, AnyV2V), or training-free statistical matching (UniVST). DreamStyle's central thesis is that this framing was looking at the wrong bottleneck: the problem is not architecture but data, and the data is not naturally scarce but can be generated systematically through a two-stage image-stylization-then-I2V-animation pipeline that exploits the asymmetric quality advantage of image generation models over video generation models (Section 3.1).

This reframing has several specific consequences for how the field allocates research effort:

Research directions that become more attractive:

• Data pipeline engineering for video-to-video tasks. If a systematically constructed 40K-pair CT dataset plus a 5K-pair manually filtered SFT dataset (Section 3.1) can produce a 2.6× CSD improvement over the previous state of the art (0.515 vs. 0.198 for StyleMaster, Table 1) using a standard I2V architecture with minor adaptations, then the marginal return on data quality investment likely exceeds the marginal return on architectural novelty for this task class. This pattern — data engineering outperforming model engineering at a given research budget — has played out repeatedly in NLP (the Chinchilla scaling laws showing data scale matters more than model scale), in image generation (DALL-E and Stable Diffusion improvements driven substantially by dataset quality and captioning pipelines), and now appears in video stylization. Research groups working on video-to-video tasks (stylization, editing, restoration, translation) should expect that the highest-impact contributions over the next few years will come from systematic data construction methodologies, not from novel attention mechanisms or conditioning architectures.

• LoRA-based multi-condition unification as a lightweight integration strategy. DreamStyle demonstrates that three semantically distinct condition types can coexist in a single I2V model through careful channel-wise concatenation design plus token-specific LoRA adaptation, without architectural modification to the base model's transformer blocks. The token-specific LoRA (shared down matrix, condition-specific up matrices, manual routing by token origin, Section 3.2) solves the inter-token confusion problem with minimal additional parameters (rank 64 across attention and FFN layers, applied to a 14B-parameter base model). This approach generalizes beyond video stylization: any task where multiple visual conditions enter a DiT-based model through similar tokenization pathways but serve different semantic roles (e.g., in-painting masks, depth maps, and reference images in image editing; motion vectors, text prompts, and style references in controllable video generation) could adopt the same design pattern, replacing complex multi-adapter architectures with a single unified LoRA module. The ablation result — dropping CSD from 0.515 to 0.413 when token-specific LoRA is removed (Table 4) — provides a concrete benchmark for how much interference a standard shared LoRA introduces, which other researchers can use as a diagnostic: if your multi-condition model underperforms, check whether condition tokens are being confused before adding architectural complexity.

• Two-tier data curricula as a principled response to diversity-quality anti-correlation. The CT-then-SFT strategy (Section 3.3, training configuration) encodes a specific insight: when generating synthetic training data, diversity and quality are often anti-correlated because the models that produce diverse outputs (InstantStyle/SDXL) have lower base quality than the models that produce high-quality outputs (Seedream 4.0), and the models that produce high quality do so through conditioning mechanisms (text guidance) that constrain the output space. The two-stage curriculum — learn broad style concepts from diverse noisy data, then refine with clean curated data — is not a generic "pretrain then finetune" recipe; it is a targeted response to this specific anti-correlation. For researchers building synthetic training pipelines in other domains (speech synthesis with varying speaker diversity, 3D asset generation with different art styles, molecular generation with property constraints), this pattern may recur: the cheap, scalable data source provides coverage but not precision, while the expensive, curated data source provides precision but not coverage. The CT-only ablation (CSD 0.459, DINO 0.547) and SFT-only ablation (CSD 0.535, DINO 0.483) in Table 4 quantify the tradeoff — SFT-only gets the best style but worst structure, CT-only gets better structure but worse style — and the full two-stage approach finds a Pareto-improving compromise. This is a template other researchers can follow.

Research directions that become relatively less attractive:

• Adapter-based style injection without video training data. StyleCrafter and StyleMaster represent a line of research that attempts to achieve video stylization by injecting style features into a pretrained video generation model, trained primarily on image stylization data. DreamStyle's results suggest this approach hits a fundamental ceiling: StyleMaster achieves a CSD score of 0.198 and a user study style consistency rating of 1.17/5 (Table 3), both dramatically below DreamStyle's 0.515 and 4.36/5. These are not gaps that better adapter design or improved style feature extraction could plausibly close — they reflect the inherent limitation of learning video-level style consistency from image-level training signals. Future work that proposes new adapter architectures for video stylization without constructing video training data must now demonstrate that it can approach the 0.5+ CSD regime established by DreamStyle, which is a substantially higher bar than existed before this paper.

• Training-free video stylization as a general-purpose solution. UniVST's feature-statistic matching via AdaIN is fundamentally limited to low-level style transfer (color palettes, coarse textures) and cannot capture the high-level artistic conventions (geometric deformation, stroke patterns, compositional rules) that DreamStyle learns from video training data. The paper does not directly compare against UniVST, but the qualitative results in Fig. 5 (low-poly style, colored pencil style, paper art style) demonstrate stylizations that go well beyond texture transfer into geometric and semantic style transformation. Training-free methods will continue to have niche advantages — they require no training data, no GPU-hours, and no model customization — but DreamStyle establishes that for high-quality stylization of diverse styles including deformation-heavy ones, learned approaches with video training data are now the standard to beat, and training-free methods should be evaluated against that standard rather than only against other training-free methods.

• Dedicated single-modality video stylization models. Prior to DreamStyle, a practitioner needing text-guided, style-image-guided, and first-frame-guided stylization would need three separate systems. DreamStyle demonstrates that all three can coexist in a single LoRA-adapted I2V model with no quality penalty (and in fact quality improvements through shared representation learning — the T2V mode that was not explicitly trained actually outperforms the V2V mode on most metrics, Table 1 middle section). For practical deployment, maintaining one model is strictly preferable to maintaining three. Research on single-modality stylization methods now carries a higher justification burden: the proposed method must demonstrate substantial quality improvements over DreamStyle's unified approach on its specific modality to justify the loss of multi-modality flexibility.

What prior contradictions does this work resolve? The paper implicitly reconciles a tension between two conflicting observations in the video stylization literature: (1) adapter-based methods like StyleMaster achieve some stylization capability without video training data, suggesting that image-level style concepts can transfer to video; but (2) the visual quality and style consistency of these methods is poor, suggesting that something fundamental is missing. DreamStyle's explanation is that the "something fundamental" is paired video training data — the motion dynamics, temporal coherence, and style-content interaction patterns that are specific to video and cannot be learned from static images. The resolution is not that adapter methods are wrong in principle, but that they were attempting to solve an underdetermined problem (learning video stylization from image data) when a determined problem (constructing video training data through a systematic pipeline) was feasible all along. The contradiction disappears when the data assumption changes.

Magnitude and scope of the contribution. This is not a paradigm shift — it does not introduce a new generative modeling framework, a new training objective, or a new model architecture class. It is an integration contribution: it combines existing components (I2V diffusion transformers, LoRA adaptation, image stylization models, ControlNets, VLM captioning, automated and manual filtering) into a system that solves a previously unsolved problem (unified multi-condition video stylization) by identifying and systematically addressing the true bottleneck (data). The contribution's significance comes from the empirical demonstration that the data bottleneck was the binding constraint — the 2.6× CSD improvement over StyleMaster is a quantitative validation of a strategic hypothesis, not an architectural innovation. This makes the work practically impactful (teams building video stylization systems should invest in data pipelines before architecture) but theoretically incremental (the individual components and techniques are all prior art).

---

Follow-Up Research This Work Enables

Stress-testing DreamStyle on styles the data pipeline cannot generate. The paper's data pipeline depends on InstantStyle/SDXL and Seedream 4.0 as image stylization models (Section 3.1, Fig. 2). Each has systematic gaps — InstantStyle is SDXL-based and limited by SDXL's image quality and style coverage; Seedream 4.0 is text-guided and limited by what can be linguistically described. A systematic stress test would curate a benchmark of 20-30 style categories that are outside the training distribution of these image models — for example, specific cultural artistic traditions (Japanese ukiyo-e woodblock prints, Islamic geometric patterns, Aboriginal dot painting), highly specialized illustration techniques (medical illustration, architectural blueprint rendering, scientific diagram conventions), or deliberately unusual style descriptions ("rendered as if painted on wet silk by a left-handed artist using only horizontal brush strokes"). For each category, the experiment would measure: (a) whether the image stylization models can produce a plausible stylized version of a standard test image; (b) whether DreamStyle trained on data from those models can stylize videos in that category; and (c) whether DreamStyle's performance on out-of-distribution styles correlates with the image models' performance. A negative result — DreamStyle fails on styles the image models cannot render — would confirm the data ceiling limitation and establish a clear boundary condition for the approach. A positive result — DreamStyle generalizes to some out-of-distribution styles through the multi-condition mechanism (e.g., combining a text prompt describing the style with a style image from the target tradition) — would demonstrate emergent generalization beyond the training data distribution, which would substantially strengthen the framework's claims.

Cross-architecture validation with open-source I2V backbones. DreamStyle has been demonstrated exclusively with Wan14B-I2V, a 14B-parameter DiT model developed internally at ByteDance (Section 3.2). The paper presents the condition injection mechanism and token-specific LoRA as general design principles, but whether they transfer to other architectures is unknown. A direct follow-up would replicate DreamStyle on at least two alternative open-source I2V models — CogVideoX-I2V (which uses a 3D causal VAE and expert transformer architecture) and Stable Video Diffusion (which uses a U-Net architecture with temporal layers rather than full 3D attention). The replication would need to: (1) adapt the channel-wise condition injection to each model's specific input conventions (SVD uses different input channel layouts than Wan14B-I2V; CogVideoX uses a different VAE and temporal patchification strategy); (2) implement token-specific LoRA for each architecture (for SVD's U-Net, the "tokens" are feature map spatial positions rather than DiT patch tokens, so the routing strategy may need adaptation); (3) train on the publicly released components of the data pipeline (or re-implement the pipeline using open-source image stylization models like IP-Adapter and SDXL, plus open-source I2V with ControlNet); and (4) evaluate on a common test set to enable cross-architecture comparison. Key metrics to compare: CSD score, DINO score, and VBench quality dimensions across architectures using the same training data. If the approach transfers with minimal adaptation, it validates the "unified framework" claim. If performance degrades substantially on non-DiT architectures, it reveals that DreamStyle's success depends on DiT-specific properties (full self-attention across spatial-temporal tokens, unified patchification, the specific way channel concatenation interacts with 3D attention) and bounds the approach's generality.

Measuring and mitigating temporal style drift within a single generation. DreamStyle generates 81-frame (~5-second) videos. The CSD metric is computed between the style reference and each frame individually (Section 4.2 describes the metric computation), but the paper reports only aggregate averages, not per-frame trajectories. A natural diagnostic experiment: for a fixed set of style-image-guided test videos, compute and plot the CSD score at each frame position (frames 1, 10, 20, ..., 81), looking for temporal style drift — a systematic decline in style consistency as frame distance from the first frame (or from the style reference image frame appended at the end of the sequence) increases. Temporal drift would indicate that the model's style representation is anchored to the temporally nearest conditioning signal and degrades through the self-attention propagation over time. If drift is observed, potential mitigations include: (a) injecting the style image condition at multiple temporal positions rather than only at the end; (b) using a stronger style conditioning signal (higher guidance scale for the CLIP image features); or (c) applying a temporal style consistency loss during training that explicitly penalizes CSD variance across frames. This experiment would not require retraining DreamStyle — it is a pure inference-time analysis using the existing model and metrics — and would provide practical guidance for users about which portions of a generated video are most reliable and whether segment boundaries in long-video concatenation (Fig. 8) are at elevated risk of style discontinuity.

Closed-loop quality assessment with test-time style verification. DreamStyle produces stylized videos in a single forward pass with no mechanism for detecting or correcting quality failures (see Section 6 limitation discussion). An extension would add a lightweight style consistency verifier that runs after generation and flags segments with potential style degradation. The verifier could be built from the same CSD metric used for evaluation: for each frame, compute the CSD score against the style reference; frames or segments whose CSD falls below a threshold (e.g., 1.5 standard deviations below the video's mean CSD) are flagged for regeneration. A pragmatic implementation: (1) generate the full 81-frame stylized video; (2) compute per-frame CSD scores; (3) identify contiguous segments where the running mean CSD drops below a threshold; (4) for each flagged segment, mask out those frames and re-generate them using the surrounding un-flagged frames as temporal context (first-frame guidance from the frame before the segment, text or style-image guidance matching the original conditions); (5) stitch the regenerated segment into the output. The key evaluation metric would be: by what percentage does selective regeneration reduce the variance of per-frame CSD scores compared to single-pass generation? And at what computational cost (measured in additional forward passes)? This approach is directly analogous to the "verifier-guided test-time compute" paradigm from the reference paper, adapted from math reasoning to video quality, and would test whether DreamStyle's quality ceiling can be pushed beyond single-pass generation through targeted re-computation. A negative result — selective regeneration fails because the base model's quality is already near its ceiling, or because regenerated segments show boundary artifacts at stitch points — would establish that DreamStyle's architecture lacks the feedback mechanisms needed for iterative refinement, which is diagnostically useful.

Multi-style fusion benchmark with controlled style interference measurement. The paper demonstrates multi-style fusion qualitatively (Fig. 6) but provides no quantitative evaluation (see Section 5 critical assessment). A proper follow-up would construct a benchmark specifically for this capability: select 10 text style descriptions (e.g., "watercolor style," "oil painting style," "pencil sketch style," "low poly style," "pixel art style," "impressionist style," "pop art style," "charcoal drawing style," "stained glass style," "claymation style") and 10 style reference images representing distinct visual aesthetics. For each of the 100 text-image pairs, generate a stylized video using DreamStyle's multi-condition mode and measure: (1) style-text alignment — does the output match the text style description? (measured via CLIP score between the output and the text prompt); (2) style-image alignment — does the output match the visual reference? (measured via CSD score); (3) style interference — does one modality dominate? Compute the ratio of style-text alignment to style-image alignment; values far from 1.0 indicate modality dominance. The experiment would also include single-modality baselines (text-only and image-only) for the same style targets, measuring whether multi-condition fusion produces outputs that are meaningfully different from and superior to either single-modality output. A systematic finding — for example, that text-guided styles dominate when they describe high-level artistic categories ("watercolor") while image-guided styles dominate when they specify low-level texture patterns, and that the optimal combination depends on style type — would provide actionable guidance for users and would validate the paper's claim that multi-condition support is "a crucial design for improving the effectiveness and controllability of video stylization" (Section 1). A negative result — multi-condition outputs are not distinguishable from the stronger single-modality output, or text-image combinations frequently produce degraded or incoherent styles — would suggest that the model has not learned to meaningfully fuse style information but merely selects the dominant modality, which would narrow the practical utility of the multi-condition design.

---

Practical Applications and Downstream Use Cases

Content creation platforms requiring flexible style control across modalities. A video editing platform (e.g., a web-based creative tool, a mobile video app, or a professional post-production plugin) integrating DreamStyle could offer users three style specification workflows in a single interface: (1) text-driven exploration for rapid iteration — type "watercolor style with bright spring colors" and see results immediately; (2) reference-driven precision for brand compliance — upload a company's style guide image or a mood board reference and apply that exact visual identity to footage; and (3) keyframe-driven long-form production — stylize the first frame manually using an external tool (Photoshop, Procreate) and have DreamStyle propagate that exact look across a full video sequence. The unified model means the platform maintains one inference endpoint rather than three, reducing infrastructure complexity and model serving costs. DreamStyle's quantitative edge — a 2.6× CSD improvement over the previous open-source state of the art (Table 1) — translates to measurably more faithful style rendering, which for commercial content (advertising, social media, brand videos) directly impacts audience perception. The first-frame-guided mode's 0.851 CSD score (Table 1) indicates that manual keyframe stylization propagates with high fidelity, enabling a workflow where a professional artist creates one frame and the AI handles the remaining 80, reducing manual labor by ~99% per video while maintaining the artist's intended look.

Low-resource language and dialect video localization with style preservation. Organizations producing video content for multiple linguistic or cultural markets often need to preserve a consistent visual brand style while localizing content. DreamStyle's style-image-guided mode (CSD 0.515, style consistency 4.36/5 in user study, Table 3) combined with its content-preserving architecture (DINO 0.526, content consistency 3.87/5) enables a workflow where: (1) a master video is produced in one language/locale; (2) a style reference image defines the brand's visual identity; (3) DreamStyle stylizes all localized video variants to match that identity. Because the style is specified via an image rather than text, it avoids the translation ambiguity that would arise from describing visual style in multiple languages. The model's ability to handle geometric deformation styles (demonstrated qualitatively in Fig. 5 for low-poly style, and noted in Section 1 as a capability that UNIC's tile ControlNet "fails to handle") is relevant here: if the brand identity involves non-photorealistic rendering (e.g., a distinctive illustration style with characteristic line work and proportion conventions), DreamStyle can apply it while previous methods constrained by pixel-aligned ControlNets could not.

Data augmentation for training robust activity recognition models. Activity recognition models (e.g., for surveillance, autonomous driving, or human-computer interaction) are notoriously brittle to visual domain shift — a model trained on photorealistic video may fail when deployed in environments with different lighting, camera characteristics, or visual styles. DreamStyle can serve as a domain randomization engine: given a dataset of raw training videos (e.g., 10,000 clips of people performing various actions), apply DreamStyle with a diverse set of style images and text prompts (covering artistic styles, synthetic rendering styles, extreme color grading, geometric stylization) to generate stylized variants of each training clip. Because DreamStyle maintains content structure (DINO score 0.526 in V2V mode, Table 1) and subject consistency (0.938 background consistency), the action labels from the raw video remain valid for the stylized variant. Training on this augmented dataset would produce models robust to visual style variation — a property that current activity recognition benchmarks rarely test but that is critical for real-world deployment. The 4× efficiency gain claimed in the reference example paper (for compute-optimal test-time scaling) has no direct analog here (DreamStyle is single-pass), but the data diversification gain is analogous: one base model, many style outputs, exponentially expanded training set coverage. The key practical question — which the paper does not answer — is the computational cost per stylized video, which determines how many augmented samples are feasible within a given compute budget.

Accessible artistic exploration for non-professional creators. Current professional video stylization workflows require expertise in compositing software (After Effects, Nuke), manual frame-by-frame rotoscoping, or training custom style transfer models. DreamStyle's text-guided mode (CLIP-T 0.167, user study overall quality 3.95/5, Tables 1 and 3) lowers the barrier to entry: a user can describe a desired style in natural language and receive a coherent stylized video in a single inference pass, with no parameter tuning, no manual masking, and no understanding of the underlying diffusion process. While the quality is not flawless (content consistency 3.95/5 indicates some distortion), it is substantially above the available commercial alternatives (Pixverse 2.82, Runway 2.59 for overall quality) as measured by professional annotators. For casual creators (social media content, personal artistic projects, educational demonstrations), this quality level represents a step change in what is possible without technical expertise. The multi-style fusion capability (Fig. 6), though not quantitatively benchmarked, suggests even more creative flexibility — a user could say "make this look like a watercolor version of that Van Gogh painting" and DreamStyle could combine both signals, which is qualitatively beyond what any single-modality system offers.

---

When to Prefer This Method

The paper positions DreamStyle against specific named alternatives — StyleMaster for style-image-guided stylization, VACE and VideoX-Fun for first-frame-guided stylization, and commercial systems (Luma, Pixverse, Runway) for text-guided stylization — with tradeoffs that can be characterized from the experimental results:

• Prefer DreamStyle over StyleMaster and adapter-based methods when: you require style-image-guided video stylization and have access to even modest video training data infrastructure. The 2.6× CSD gap (0.515 vs. 0.198, Table 1) and 3.19-point user study gap (4.36 vs. 1.17 for style consistency, Table 3) indicate that adapter methods without video training data are not competitive for style fidelity. DreamStyle requires constructing a paired dataset (40K + 5K videos), but the paper provides a blueprint for doing so using off-the-shelf image stylization models and ControlNet-guided I2V generation. If you can execute this pipeline for your domain, the quality gain justifies the data engineering investment.

• Prefer DreamStyle over training-free methods (UniVST, TokenFlow) when: your target styles involve geometric deformation (low-poly, cubism, caricature) or high-level artistic conventions beyond color and texture transfer. Training-free statistical matching cannot represent the semantic understanding of style that DreamStyle learns from video data. The qualitative results in Fig. 5 (low-poly style, paper art style, colored pencil illustration) demonstrate capabilities that AdaIN-based methods structurally cannot achieve. The tradeoff is computational cost (DreamStyle requires training and GPU inference) versus simplicity (training-free methods require no data and less compute per video).

• Prefer DreamStyle over single-modality methods when: your use case requires flexibility across text, image, and keyframe style specification. DreamStyle's unified architecture supports all three without quality degradation (the T2V variant that was not explicitly trained outperforms the V2V variant on most metrics, Table 1), and enables emergent capabilities (multi-style fusion, long-video concatenation) that single-modality systems cannot provide. The token-specific LoRA design (Section 3.2) mitigates the interference that would otherwise plague multi-condition models. The cost is that the model is not equally optimized for all conditions — the 1:2:1 training ratio (Section 4.1) means style-image-guided tasks benefit from 2× the training signal, and text-guided performance might improve with balanced ratios (though this is untested).

• Prefer commercial systems (Luma, Pixverse, Runway) over DreamStyle when: you cannot construct training data (no access to image stylization models, I2V models with ControlNet support, or compute for 9,000 training iterations) and must rely on API-accessible, zero-shot stylization. The commercial systems are immediately usable with no infrastructure, and their quality — while below DreamStyle's on the tested metrics — may be sufficient for informal or exploratory use cases. The paper's text-guided comparison (Table 1) is weakened by the black-box nature of these baselines (see Section 5 critical assessment), so the exact quality gap in real-world deployment with optimized prompts is uncertain.

• Prefer VACE or VideoX-Fun over DreamStyle when: structure preservation is the absolute priority and some style fidelity can be sacrificed. For first-frame-guided stylization, DreamStyle's DINO score (0.640) is lower than both VACE (0.716) and VideoX-Fun (0.702), Table 1, meaning it trades off content structure to achieve its superior CSD (0.851 vs. 0.689 and 0.766). If your application requires pixel-accurate content preservation (e.g., stylizing medical or scientific video where structural distortion is unacceptable), the specialized propagation-based methods may be safer despite their style limitations.