DreamActor-M2: Universal Character Image Animation via Spatiotemporal In-Context Learning

ArXiv: 2601.21716

🎯 Pitch

DreamActor-M2 eliminates the need for explicit skeletons by framing character animation as in-context learning, fusing appearance and motion into a unified latent space. This approach resolves the longstanding trade-off between preserving identity and maintaining motion consistency, enabling a single model to animate humans, animals, and cartoons with state-of-the-art fidelity. The framework even outperforms dedicated pose-based methods by learning to reason about dynamics directly from RGB video.

---

1. Executive Summary

DreamActor-M2 proposes a universal character image animation framework that reimagines motion conditioning as a spatiotemporal in-context learning problem, built on the Seedance 1.0 video diffusion backbone and evaluated on the newly introduced AW Bench. The framework operates through a two-stage paradigm: a Pose-based variant that uses augmented 2D skeletons with target-oriented motion-semantic guidance (an MLLM-powered module that fuses driving-video motion semantics with reference-image appearance descriptions into a unified text prompt), and an End-to-End variant trained via a self-bootstrapped data synthesis pipeline (the Pose-based model generates pseudo cross-identity training pairs from RGB driving videos, enabling direct RGB-driven animation without pose estimators). On AW Bench—encompassing humans, animals, cartoons, and multi-subject scenarios—DreamActor-M2 establishes state-of-the-art performance, with the End-to-End variant achieving appearance consistency scores of 4.35 (automatic) and 4.20 (human) on a 1–5 scale, substantially outperforming prior methods including DreamActor-M1 and Wan2.2-Animate, while in GSB subjective comparisons it leads Kling 2.6 by +9.66% and Wan2.2-Animate by +51.43%, establishing that pose-free, cross-domain animation generalizes robustly only when motion and identity are unified through spatiotemporal context rather than through compressed or spatially-aligned injection mechanisms.

2. Context and Motivation

What Specific Problem Does This Paper Address?

Character image animation—the task of transferring motion from a driving video onto a static reference image while preserving the reference's identity—faces two fundamental, unresolved challenges. The paper identifies these explicitly in Section 1:

The "see-saw" trade-off between identity preservation and motion consistency. Current motion injection strategies fall into two camps, each failing in a complementary way. Pose-aligned channel-wise injection methods (e.g., concatenating pose representations directly into latent features, as in AnimateAnyone or UniAnimate) enforce strong spatial alignment between the driving signal and generated output. This alignment preserves motion faithfully, but at a cost: the structural shape cues embedded in the pose signal—limb proportions, body shape, joint positions—"leak" into the generated video, distorting the reference identity. The paper calls this "shape leakage" (Section 1). Conversely, cross-attention-based injection methods (e.g., MTVCrafter, Animate-X) compress motion representations into compact latent codes before injecting them via attention mechanisms. This decoupling reduces identity leakage, but the compression inevitably discards fine-grained temporal dynamics, leading to degraded motion coherence—limbs that don't quite follow the intended trajectory, gestures that lose their nuance, anatomically implausible configurations. Neither approach simultaneously achieves high-fidelity animation and stable identity preservation; improving one degrades the other. This is the "see-saw."

Over-reliance on explicit pose priors (e.g., 2D skeletons, SMPL meshes) that fundamentally limit generalization. The paper identifies a representation bottleneck in Section 1: explicit pose estimators are the standard input modality for motion injection, but they fail in characteristic ways. For human motion, these estimators are notoriously error-prone under occlusion, unusual poses, or rapid movement—a dancer's spinning motion, a person's hands clasped behind their back, or a child's non-standard body proportions can all confuse DWPose or SMPL-based systems. More critically, these estimators are architecturally incapable of generalizing to non-humanoid characters. You cannot extract a human skeleton from a cartoon cat, a stop-motion penguin, or a Pixar-style alien. The underlying assumption baked into pose-based pipelines—that the character possesses a humanoid joint hierarchy—excludes animals, cartoons, anthropomorphic objects, and any character that doesn't conform to the human body plan. This is not a performance limitation that can be solved with better pose data; it is a categorical restriction.

The paper argues that even approaches attempting to sidestep explicit pose representations—like FlexiAct (which learns per-video frequency-aware embeddings) or X-Unimotion (which uses implicit motion representations)—don't fully escape this constraint. FlexiAct requires costly per-video optimization. X-Unimotion, while using implicit representations during generation, still relies on 2D pose signals during training, meaning its motion understanding is ultimately grounded in the same limited representational space. The paper's diagnosis is that prior work remains "tethered" to pose at some stage of the pipeline, and this tether fundamentally limits scalability and cross-domain applicability.

Why Is This Problem Important?

The importance spans both practical deployment and theoretical understanding of how generative models should handle multimodal conditioning.

Real-world impact. Character animation is a core technology for digital entertainment, content creation, virtual avatars, and increasingly for AI-assisted film production and social media. A system that genuinely works across character types—animating a photorealistic human, a hand-drawn cartoon, a low-poly 3D render, and a close-up of a talking animal, all from the same framework—eliminates the need for domain-specific animation tools. The paper's explicit goal is "universal" character animation (Section 1, Abstract), meaning a single model handles humans, animals, cartoons, and objects. This vision matters because the current landscape is fragmented: human-specific models dominate the animation literature, leaving non-human character animation to entirely separate toolchains or requiring per-domain model training.

The AW Bench introduced in this paper (Section 4) reflects this importance. It is specifically constructed to test cross-domain generalization—driving videos of animals and cartoons applied to reference images of humans and vice versa. The fact that the paper must construct its own benchmark because existing evaluation datasets (e.g., TaiChi, TED-talks, TikTok) are exclusively human-centric underscores how narrow the field's scope has been. A universal animation system would democratize access to high-quality animation, enabling creators to animate any character with any motion source, regardless of domain.

Theoretical significance. The paper's reframing of motion conditioning as an in-context learning problem represents a conceptual shift in how we think about multimodal integration in diffusion models. Current approaches treat motion as a separate modality requiring specialized encoders, injection modules, or training objectives. The in-context paradigm instead argues that motion can be presented to the model as visual context—literally concatenated alongside the reference image in pixel space—and that a strong video prior can learn to interpret this context without architectural modifications. If this holds, it suggests that foundation video models already possess the "motion understanding" capability; the challenge is not building better motion encoders but designing input representations that bridge the modality gap without introducing information bottlenecks. This has implications beyond animation, potentially extending to any task where one modality must condition generation on another.

Where Prior Approaches Fall Short

The paper organizes prior work into three categories of motion injection and identifies specific failure modes for each (Section 3.2).

Pose-aligned injection (channel-wise concatenation / additive fusion). Methods like AnimateAnyone (Hu et al., 2023), UniAnimate (Wang et al., 2024), MimicMotion (Zhang et al., 2024), and MagicAnimate (Xu et al., 2024) encode 2D skeletons or SMPL parameters into feature maps and inject them into the diffusion model's latent space via concatenation or addition. The mechanism works by enforcing a pixel-level correspondence between the pose signal and the generated output—the model learns to "paint" the reference identity onto the pose-defined spatial structure. The failure mode is identity leakage. The paper explains this in Section 3.2: pose encodings inherently carry the shape of the pose source—the specific limb lengths, body proportions, and joint configurations of the driving subject. When the driving and reference subjects differ (cross-identity animation, the standard use case), these shape cues overwrite the reference identity's own proportions. A tall person's skeleton driving a child's reference image produces a stretched child; a dancer's skeleton driving a cartoon character distorts the cartoon's stylized proportions. The paper's analysis goes deeper in Section 3.3, noting that training on same-identity data (where the driving video and reference image are the same person from different frames) exacerbates this: the model learns to associate specific pose shapes with specific identities, and this entanglement fails catastrophically under distribution shift.

Cross-attention injection. Methods like MTVCrafter (Ding et al., 2025) and Animate-X (Tan et al., 2024/2025) use an auxiliary pose encoder to compress motion signals into latent tokens, which are then injected via cross-attention layers. This decouples motion from spatial coordinates—the model attends to motion features rather than having them directly overlaid on the latent grid. The paper acknowledges this partially alleviates identity leakage (Section 3.2), but identifies a different failure mode: latent compression discards fine-grained motion details. Compressing a 121-frame skeleton sequence into a fixed-size set of latent tokens inevitably smooths over subtle movements, temporal variations, and gesture specifics. The paper reports that this produces "temporally or anatomically unnatural animations" (Section 3.2)—motion that is globally plausible but locally imprecise. A hand gesture that should trace a specific shape becomes a generic waving motion; a dance move requiring precise limb coordination becomes approximate.

Temporal-level in-context injection. A third approach, explored in TIC-FT (Kim et al., 2025), concatenates conditioning and target frames along the temporal dimension. This leverages the backbone's temporal self-attention to learn global motion patterns from frame-to-frame relationships. The paper notes (Section 3.2) that while this preserves the backbone's architecture and is efficient, it suffers from the absence of frame-wise spatial correspondence. The model sees conditioning frames and target frames as a single temporal sequence but has no explicit mechanism to map which part of the conditioning frame corresponds to which part of the target frame. This leads to "suboptimal reconstruction quality" and degraded fine-grained motion details—the temporal attention can learn that something is moving, but not precisely where or how the motion maps to the reference's spatial layout.

Implicit motion and one-model-per-identity approaches. Beyond injection mechanisms, the paper critiques alternative paradigms. DreamVideo (Wei et al., 2024) trains a separate model for each reference identity, which the paper calls a "one-model-per-identity" paradigm that "severely limits generalization" (Section 2). It cannot handle new characters without retraining. FlexiAct (Zhang et al., 2025) learns per-video frequency-aware embeddings, but requires dedicated optimization for each driving video. X-Unimotion (Song et al., 2025) uses implicit motion representations during generation but still requires 2D pose supervision during training—it can't escape the representational limitations of pose even if it doesn't use pose at inference. SCAIL (Yan et al., 2025), a concurrent work, leverages full 3D pose sequences as contextual inputs for studio-grade quality, but its performance is contingent on accurate 3D motion capture, which is unavailable in the wild. The paper positions all of these as suffering from the same underlying constraint: they are bound, at some stage, to explicit pose representations that limit their domain of applicability.

How Does This Paper Position Itself?

DreamActor-M2 positions itself as a synthesis and transcendence of prior approaches, unified under the conceptual framework of spatiotemporal in-context learning (Section 3.2). The key insight is that the "see-saw" between identity and motion arises from treating motion as an external signal that must be injected into the generation process through specialized mechanisms. The alternative is to treat motion as visual context that the model interprets holistically alongside the reference image.

The in-context learning framing (Section 2, Section 3.2). In-context learning is well-established in LLMs (Brown et al., 2020), VLMs (Alayrac et al., 2022), and increasingly in image generation (Huang et al., 2024), but the paper notes it is "relatively underexplored in video generation" (Section 2). The approach is simple in concept but radical in implication: rather than designing motion encoders, the paper spatially concatenates the reference image with motion frames into a unified composite frame (twice the width of a standard frame), then temporally stacks these composites to form the input video. At $t=0$, the composite is [reference | first motion frame]; at $t>0$, it is [blank | motion frame]. This construction (Equation 2 in Section 3.2) means the model sees both what should move (the reference identity) and how it should move (the motion signal) in a single visual format.

What makes this an "in-context" rather than merely a concatenation approach is that the model must learn to interpret the motion signal as context that specifies how the reference should be animated, not as pixels to be directly copied. The blank region at $t>0$ forces the model to transport the reference identity across time while conditioning on the motion signal in the adjacent spatial region. This is analogous to how LLMs interpret few-shot examples in their context window to infer a task, rather than requiring the task specification to be compiled into a separate prompt encoding.

Bridging the gap between injection strategies (Section 1). The paper explicitly positions spatiotemporal ICL as resolving the limitations of all three prior injection paradigms:

• Versus pose-aligned injection: ICL avoids identity leakage because there is no explicit pose encoding that carries shape priors. The motion signal is presented as visual information (skeletons in the pose-based variant, raw RGB in the end-to-end variant), and the model learns to extract motion dynamics from it without conflating shape with identity.
• Versus cross-attention injection: ICL avoids lossy compression because the full motion sequence is available in pixel space. No auxiliary encoder compresses the motion into a bottleneck; the model's own attention mechanisms can attend to fine-grained motion details at full resolution.
• Versus temporal-only injection: ICL provides frame-wise spatial correspondence through the spatial concatenation. Each composite frame directly pairs the reference (at $t=0$) or the generated-to-date content (at $t>0$) with the corresponding motion frame, giving the model explicit spatial landmarks for motion transfer.

The two-stage evolution as a strength, not a compromise (Section 3.3, Section 3.4). The paper's decision to develop a Pose-based variant first and then evolve to an End-to-End variant might seem like a half-measure—why not go straight to end-to-end? The paper frames this as a strategic design choice motivated by data availability. There exists no large-scale dataset of paired cross-identity RGB motion videos suitable for end-to-end training. The self-bootstrapped data synthesis pipeline uses the Pose-based model to generate this dataset, creating a path from the well-studied (pose-conditioned animation, where training data is abundant via pose estimation on web videos) to the novel (RGB-conditioned animation, where no paired data exists). This positions the two-stage approach as a scaffold: the Pose-based model provides the data that enables the End-to-End model to exist at all, rather than being an end in itself.

Positioning relative to SCAIL and concurrent in-context work (Section 2). The paper distinguishes itself from the concurrent work SCAIL, which also uses in-context learning for animation but with full 3D pose sequences. SCAIL achieves "studio-grade" quality but is limited to settings where accurate 3D motion capture is available—a constraint that excludes essentially all in-the-wild video. DreamActor-M2's End-to-End variant removes this constraint entirely, operating on raw RGB driving videos with no motion capture, no pose estimation, and no domain assumptions.

The AW Bench as a positioning statement (Section 4). By constructing a benchmark that includes animals, cartoons, multi-subject scenarios, and cross-domain transfer, the paper implicitly argues that the field's existing evaluation frameworks are inadequate. Prior benchmarks like the TikTok dataset or TaiChi evaluate only human-to-human animation, which doesn't test the generalization that a "universal" system should achieve. AW Bench operationalizes the paper's definition of universality: can the model animate any character with any motion source? The benchmark's composition—100 driving videos covering humans (of varying ages, body regions, and activities), animals (cats, chickens, monkeys, orangutans), and cartoon characters (Tom and Jerry, cartoon aliens), plus 200 reference images of similar diversity, including multi-subject scenarios—encodes a clear challenge to the field: if your method only works on humans, it isn't solving the real problem.

Connecting the Motivation to the Technical Design

The paper's motivation directly shapes its technical architecture. The "see-saw" motivates the in-context formulation: if specialized injection mechanisms create trade-offs, eliminate them by using the model's native visual understanding. The generalization bottleneck motivates the end-to-end paradigm: if pose estimators are the limiting factor, remove them entirely. The absence of training data motivates the self-bootstrapped synthesis pipeline: if paired RGB motion data doesn't exist, generate it using the best available tool and then train on the synthetic data. Each design choice in the paper can be traced back to a specific failure mode in prior work, and the paper is explicit about these connections in a way that makes its architectural decisions feel inevitable rather than arbitrary.

3. Technical Approach

3.1 Reader Orientation

DreamActor-M2 is a video diffusion model that takes a single reference image (the character to animate) and a driving video (the motion source) as input, and generates an output video where the reference character performs the driving video's motion while preserving its own identity. The system solves the "see-saw" problem between identity preservation and motion accuracy by treating motion conditioning as spatiotemporal in-context learning: rather than encoding motion through specialized injection modules that either leak identity or lose motion detail, it spatially concatenates reference and motion frames into a unified visual representation that the pre-trained video backbone can interpret holistically using its native generative capabilities.

3.2 Big-Picture Architecture (Diagram in Words)

The DreamActor-M2 framework consists of five interconnected components operating in a two-stage training paradigm:

1. Seedance 1.0 Pre-trained Video Diffusion Backbone — a foundation video generation model built on the MMDiT architecture that provides the generative prior for producing coherent videos. All other components build on top of this frozen or lightly fine-tuned backbone without modifying its core architecture.

2. Spatiotemporal In-Context Construction Module — a pre-processing stage that takes the reference image and driving video frames and spatially concatenates them into composite frames (reference on the left, motion signal on the right for the first frame; blank on the left, motion on the right for subsequent frames), then temporally stacks these composites into a single input sequence that the VAE encodes into latents.

3. Pose-Based Training Pipeline (Stage 1) — trains the model using 2D skeleton sequences as motion signals, augmented with bone-length scaling and bounding-box normalization to prevent identity leakage, and guided by MLLM-generated text prompts that describe both the motion semantics and reference appearance.

4. Self-Bootstrapped Data Synthesis Pipeline (Stage 2) — uses the trained Pose-based model to generate pseudo cross-identity video pairs from driving pose sequences and diverse reference images, creating a dataset where the generated video serves as the RGB driving signal and the original driving video serves as the ground-truth target. This dataset is filtered through automatic scoring (Video-Bench, threshold 4.5) and manual verification for identity fidelity and motion coherence.

5. End-to-End Training Pipeline (Stage 2) — fine-tunes the model on the synthesized dataset where the driving signal is raw RGB video (the generated pseudo-video) rather than pose sequences, warm-started from the Pose-based model weights, enabling the model to learn direct RGB-to-RGB motion transfer without any pose estimator.

Information flows as follows: reference image + driving video → spatial concatenation into composite frames → temporal stacking → 3D VAE encoding → latent sequence combined with noise and masks → MMDiT denoising network → 3D VAE decoding → output video. During Stage 1, the driving signal is an augmented pose sequence; during Stage 2, it is a raw RGB video synthesized by the Stage 1 model.

3.3 Roadmap for the Deep Dive

• First, the Seedance 1.0 backbone and the standard latent diffusion training objective (Equation 1), since this is the foundation all subsequent mechanisms build upon.
• Second, the spatiotemporal in-context construction (Equation 2), because it is the core architectural innovation that enables the entire framework—understanding how the composite input is built is prerequisite to understanding why the model works.
• Third, the Pose-based DreamActor-M2 training pipeline, including pose augmentation (bone-length scaling, bounding-box normalization), target-oriented text guidance via MLLMs, and LoRA fine-tuning strategy, since these mechanisms collectively solve the "see-saw" for pose-conditioned animation and produce the model that enables Stage 2.
• Fourth, the self-bootstrapped data synthesis pipeline and quality filtering protocol, since this is the bridge between pose-based and end-to-end paradigms and determines the quality of supervision for Stage 2 training.
• Fifth, the End-to-End DreamActor-M2 training procedure, including the data triplet construction (Equation 4), warm-start initialization, and how training directly on RGB driving signals eliminates the need for pose estimators.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and methodology paper whose core idea is that character animation can be reframed as spatiotemporal in-context learning, where motion signals are presented as visual context alongside the reference image rather than being injected through specialized encoders, and that this reframing enables a two-stage evolution from pose-dependent to fully pose-free animation through self-bootstrapped data synthesis.

---

Seedance 1.0 Backbone and the Diffusion Objective

DreamActor-M2 is built on Seedance 1.0, a latent video diffusion model that employs the MMDiT architecture for multi-modal and multi-task video generation. Understanding the standard diffusion training objective is necessary because all DreamActor-M2 variants use this same denoising formulation without modification—the innovation is entirely in how the conditioning input $c$ is constructed, not in the loss function or the denoising architecture.

Latent space encoding. The first component of any latent diffusion model is the Variational Autoencoder (VAE) that compresses raw pixels into a lower-dimensional latent space. For an input image $I \in \mathbb{R}^{H \times W \times 3}$, the VAE encoder $\xi$ produces a latent representation:

$\mathbf{z} = \xi(I)$

where $\mathbf{z}$ is the latent encoding with spatial dimensions reduced by a factor (typically 8×) and increased channel dimension. Working in this compressed latent space is what makes diffusion models computationally feasible for video—performing the denoising process directly in pixel space at video resolutions would be prohibitively expensive. The VAE decoder performs the inverse operation, reconstructing pixels from latents at the final generation step.

Forward diffusion process. During training, Gaussian noise $\epsilon$ is progressively added to the clean latent $\mathbf{z}_0$ across discrete timesteps $t$, producing noisy latents $\mathbf{z}_t$. The noise schedule is designed so that at the maximum timestep (typically $t = T$), $\mathbf{z}_T$ is essentially pure Gaussian noise. At each intermediate timestep, $\mathbf{z}_t$ retains some information from $\mathbf{z}_0$ but with increasing degradation. This forward process is not learned—it follows a fixed variance schedule.

Training objective. The denoising network $\epsilon_\theta$ is trained to predict the noise that was added at timestep $t$, given the noisy latent $\mathbf{z}_t$, the conditioning signal $c$, and the timestep $t$ itself (which tells the network the current noise level). The standard simplified training objective is:

$\mathcal{L} = \mathbb{E}_{\mathbf{z}_t, c, \epsilon, t} \left( \|\epsilon - \epsilon_\theta(\mathbf{z}_t, c, t)\|_2^2 \right)$

where $\epsilon \sim \mathcal{N}(0, \mathbf{I})$ is the sampled Gaussian noise, $\mathbf{z}_t$ is the noisy latent at timestep $t$ (produced by adding $\epsilon$ to $\mathbf{z}_0$ according to the forward schedule), $c$ is the conditioning input (which in DreamActor-M2 includes the composited reference-motion sequence and optional text guidance), and $\epsilon_\theta(\mathbf{z}_t, c, t)$ is the network's prediction of the noise component.

What it computes: the expected $L_2$ distance between the true noise that was added to the clean latent and the network's estimate of that noise, averaged over random samples of training data, noise instances, and timesteps. For each training sample, the procedure is: (1) take a clean latent $\mathbf{z}_0$ from a training video, (2) sample a random timestep $t$ and random noise $\epsilon$, (3) produce the noisy latent $\mathbf{z}_t$ by interpolating between $\mathbf{z}_0$ and $\epsilon$ according to the noise schedule at timestep $t$, (4) feed $\mathbf{z}_t$, conditioning $c$, and timestep $t$ through the network to get the predicted noise $\epsilon_\theta$, (5) compute the $L_2$ loss between true and predicted noise, (6) backpropagate. The output is a scalar loss value per sample.

Why this form: predicting the noise rather than directly predicting the clean latent turns out to be a substantially easier learning problem. Intuitively, the network only needs to identify which parts of the input are noise versus signal, rather than hallucinating pixel-level details from scratch. This formulation, introduced by Ho et al. (2020), is now standard across virtually all diffusion models and is the objective that Seedance 1.0 uses during its pre-training. DreamActor-M2 preserves this objective exactly—the $\epsilon$-prediction formulation is not modified. What changes is the content and structure of $c$, the conditioning input.

At inference time, the process reverses: starting from pure Gaussian noise $\mathbf{z}_T \sim \mathcal{N}(0, \mathbf{I})$, the network iteratively denoises $\mathbf{z}_t$ to produce $\mathbf{z}_{t-1}$, using its noise prediction to remove estimated noise at each step. After $T$ denoising steps, the final latent $\mathbf{z}_0$ is passed through the VAE decoder to produce the output video in pixel space. The conditioning $c$ is fixed throughout the denoising trajectory—it specifies what video should be generated, and the denoising process incrementally refines noise into the conditioned output.

The MMDiT architecture. Seedance 1.0 uses the Multi-Modal Diffusion Transformer (MMDiT) architecture, which extends the standard DiT by supporting multiple conditioning modalities through separate input streams that interact via joint attention. The architecture treats text conditioning and visual conditioning as distinct modalities, enabling the model to attend across modalities during generation. This multi-modal design is important for DreamActor-M2 because the framework provides both visual conditioning (the composited reference-motion input) and textual conditioning (the MLLM-generated target-oriented prompts), and the backbone can naturally integrate both without architectural modification.

---

Spatiotemporal In-Context Construction

This is the central architectural innovation of DreamActor-M2. Rather than encoding motion through a dedicated network and injecting it into the denoising backbone, the method constructs a unified visual representation that the pre-trained VAE and DiT can process natively.

The core insight. Pre-trained video diffusion models already possess strong capabilities for understanding visual structure, temporal dynamics, and spatial relationships. The reason prior methods needed specialized motion injection modules was that they presented motion in a format the model could not directly interpret—numerical joint coordinates, compressed latent codes, or modality-specific feature maps. The in-context approach instead asks: what if we present motion in the same visual format that the model was pre-trained to understand? By rendering pose skeletons or providing raw RGB frames as visual context spatially adjacent to the reference image, the model can apply its pre-trained visual reasoning capabilities to extract motion information without any specialized architecture.

The composite frame construction. Given a reference image $I_{\text{ref}} \in \mathbb{R}^{H \times W \times 3}$ and a driving motion sequence $\mathbf{D} \in \mathbb{R}^{T \times H \times W \times 3}$ (where $T$ is the number of frames), the method constructs a composite input sequence $\mathbf{C} \in \mathbb{R}^{T \times H \times 2W \times 3}$ as follows:

\begin{cases} I_{\text{ref}} \oplus \mathbf{D}[t], & t = 0, \\ \mathbf{0} \oplus \mathbf{D}[t], & t > 0. \end{cases}$$ where `$\oplus$` denotes spatial concatenation along the width axis (placing the reference or blank on the left half and the motion frame on the right half), `$\mathbf{0} \in \mathbb{R}^{H \times W \times 3}$` is an all-zero (black) image matching the dimensions of `$I_{\text{ref}}$`, and `$t$` indexes frames in the sequence. **What this computes operationally.** For the first frame (`$t = 0$`), the composite is a `$2W$`-wide image with the reference character on the left side and the first motion frame on the right side, side by side. For every subsequent frame (`$t > 0$`), the composite replaces the left side with a black image while keeping the right side as the corresponding motion frame. The result is a `$T$`-frame video where each frame is twice the standard width, with the left half containing the reference identity only at `$t = 0$` and the right half containing the full motion sequence throughout. Frame by frame: at `$t = 0$`, the model sees `[reference character | first pose/RGB frame]`; at `$t = 1$`, `[black | second pose/RGB frame]`; at `$t = 2$`, `[black | third pose/RGB frame]`; and so on. **Why this specific construction.** The design serves several purposes simultaneously. First, the spatial concatenation at `$t = 0$` explicitly pairs the reference identity with the initial motion state, giving the model a visual association: "this character on the left should be doing what this pose on the right shows." Second, replacing the left side with black for `$t > 0$` forces the model to transport the reference identity across time using its own generative capabilities rather than having the reference repeatedly presented—this is what prevents the model from simply copying the reference frame and requires it to learn temporal consistency. Third, the right side's continuous presentation of motion frames provides frame-by-frame guidance: at each timestep, the model knows exactly what motion state should correspond to that temporal position. Fourth, the black region is not arbitrary—it provides a neutral canvas that the model learns to "fill in" with the animated character, analogous to how image inpainting models fill masked regions. **The motion and reference masks.** To disambiguate the two spatial regions and indicate which part contains driving information versus identity information, the method constructs two binary masks: - `$\mathbf{M}_m$` — the motion mask, set to all ones (indicating the motion region is always active). - `$\mathbf{M}_r$` — the reference mask, set to 1 only for the first frame (`$t = 0$`) and 0 for all subsequent frames (indicating the reference identity is only available at the start). These are spatially concatenated to match the composite frame structure: `$\mathbf{M} = \mathbf{M}_r \oplus \mathbf{M}_m$`. The resulting mask `$\mathbf{M}$` has the same spatiotemporal dimensions as the composite video `$\mathbf{C}$`, with the left half active only at `$t = 0$` and the right half active throughout. **Latent encoding and DiT input.** The composited video `$\mathbf{C}$` is projected into the latent space via the 3D VAE, producing a latent sequence `$\mathbf{Z} = \xi_{\text{3D}}(\mathbf{C})$`. The 3D VAE compresses both spatial and temporal dimensions simultaneously, producing a latent representation that captures motion across frames. This latent `$\mathbf{Z}$`, the noise latent `$\mathbf{Z}_{\text{noise}}$` (which will be progressively denoised to generate the output), and the mask `$\mathbf{M}$` (appropriately resized to match latent dimensions) are channel-concatenated to form the comprehensive input to the MMDiT denoising network. **Why channel concatenation rather than separate streams.** By concatenating the reference-motion latent, the noise latent, and the mask along the channel dimension, the method presents all information as a single tensor input to the transformer. The self-attention mechanism can then learn to attend across all elements—the reference information, the motion guidance, the noise level, and the spatial mask indicators—in a unified attention space. This is fundamentally different from cross-attention injection, where motion latents are introduced through a separate pathway and can only interact with the main generation through cross-attention operations. The channel-concatenated approach gives every spatial and temporal position in the latent access to the full conditioning context through self-attention. **How this resolves the "see-saw."** The spatiotemporal ICL approach avoids identity leakage because the reference identity and motion signal occupy *different spatial regions* in the composite frame—the model never sees them overlaid or blended, so there is no opportunity for the motion signal's shape cues to overwrite the reference identity. It avoids motion detail loss because the full motion sequence is available at full resolution in the right half of the composite—there is no compression through an auxiliary encoder. And it provides explicit spatial correspondence because each composite frame temporally aligns the generated content position (the left half) with the motion target (the right half), giving the model a direct frame-wise mapping between "what should be happening now" (right) and "what should the character look like" (left, carried forward from `$t = 0$` through the generative process). **Distinction from temporal-only in-context (TIC-FT, Kim et al., 2025).** The temporal-only approach concatenates reference frames and conditioning frames along the time axis into a single longer video, without spatial pairing. While this uses the backbone's temporal self-attention, it lacks explicit per-frame spatial correspondence—the model must infer which temporal conditioning frame maps to which output timepoint entirely through learned attention patterns. The spatiotemporal approach adds the spatial dimension explicitly: every output frame is spatially paired with its corresponding motion frame in the same composite image, providing a direct, local correspondence that the model can leverage without needing to learn long-range temporal mappings. --- #### Pose-Based DreamActor-M2 Training Pipeline The Pose-based variant represents the first stage of the framework, where the motion signal `$\mathbf{D}$` is a sequence of 2D pose skeletons extracted from the driving video. This stage establishes the in-context conditioning mechanism on a well-understood modality (pose) before transitioning to raw RGB in Stage 2. **Training data and self-supervised paradigm.** The training data consists of 100,000 web-collected human videos. For each video `$\mathbf{V}$`, a pose sequence `$\mathbf{P}$` is extracted using a 2D pose estimator (the paper uses DWPose, implied through its discussion of pose estimation limitations in Section 2). The first frame of the video is designated as the reference image: `$I_{\text{ref}} = \mathbf{V}[0]$`. The model is trained in a self-supervised reconstruction paradigm: given the pose sequence `$\mathbf{P}$` and the first frame `$I_{\text{ref}}$` as input, the model must reconstruct the original video `$\mathbf{V}$`. This is self-supervised because the ground-truth target is simply the source video—no manual annotation is required. Clips are randomly sampled between 49 and 121 frames during training. **Why reconstruction training works despite being same-identity.** Ordinarily, training a model to reconstruct the same video from its own first frame and pose would cause severe identity entanglement: the model would learn that this specific subject identity is associated with these specific pose shapes, and would fail catastrophically when asked to animate a different character. The pose augmentation strategies (described below) are specifically designed to break this identity-pose association by randomly perturbing the pose representation so that the model cannot rely on pose shape as an identity cue and must instead learn to extract pure motion dynamics. **Pose augmentation: random bone-length scaling.** The first augmentation strategy targets a specific failure mode: 2D skeletons encode "body shape" information through their bone lengths and joint proportions. A tall person's skeleton has longer limb segments than a child's; a dancer's skeleton might reveal specific body proportions. During same-identity reconstruction training, the model can learn to use these bone-length patterns as an identity shortcut—"the skeleton with extra-long arms must be this specific basketball player"—rather than extracting purely motion-relevant features like joint angles and temporal trajectories. To break this association, bone-length scaling is applied: skeleton bones are grouped into anatomical segments (e.g., upper arm, forearm, thigh, shin, torso) and each segment is multiplied by a random scaling factor drawn from a uniform distribution: $$\text{scale} \sim \mathcal{U}(0.8, 1.2)$$ The scaling is applied to 30% of training samples. For the remaining 70% of samples, bone lengths are left unperturbed. This partial application is deliberate: applying scaling to 100% of samples would prevent the model from ever seeing true pose proportions, potentially degrading motion accuracy. The 30% rate introduces enough variation to prevent identity leakage while preserving sufficient clean pose data for the model to learn accurate motion dynamics. **What bone-length scaling does operationally.** Consider a frame where the driving person raises their arm. The unmodified skeleton shows the arm bones at their natural, person-specific length. After scaling, those specific bones might become 15% shorter or 10% longer. The skeleton now represents a character with different arm proportions performing the same raising motion. The joint angles and temporal trajectory remain unchanged—only the spatial extent changes. The model must learn to animate the reference character using the *motion pattern* (arm raising) while producing limb lengths and proportions consistent with the reference identity, not the scaled skeleton. This forces the model to decouple motion dynamics from structural shape. **Pose augmentation: bounding-box-based normalization.** The second strategy addresses a different form of identity leakage: absolute spatial coordinates. A skeleton's joint positions in pixel space depend on the subject's size, distance from the camera, and position within the frame. A tall person standing close to the camera produces larger pixel-space joint displacements than a child standing far away, even when performing the identical motion. Training on absolute coordinates would cause the model to associate specific spatial extents with specific identities. Bounding-box normalization eliminates this by computing the bounding box that encloses all joints across all frames in a clip, then normalizing all joint coordinates relative to this bounding box. Specifically, for a clip with joints spanning spatial extent `$[x_{\min}, x_{\max}] \times [y_{\min}, y_{\max}]$`, each joint coordinate `$(x_j, y_j)$` is transformed to: $$x_j' = \frac{x_j - x_{\min}}{x_{\max} - x_{\min}}, \quad y_j' = \frac{y_j - y_{\min}}{y_{\max} - y_{\min}}$$ producing scale-invariant coordinates in `$[0, 1]$` that encode only the relative spatial arrangement of joints, not their absolute position or scale. **Why both augmentations together.** Bone-length scaling addresses *intra-body proportion* information (relative limb lengths), while bounding-box normalization addresses *inter-body scale* information (absolute size and position). Together, they strip the pose representation of nearly all identity-carrying shape cues while preserving the information that matters for animation: joint angles, connectivity, and temporal trajectories. What remains is a representation of pure motion—"this joint is moving along this trajectory relative to this other joint"—decoupled from the specific body that produced it. **Why standard training on clean poses fails without augmentation.** The ablation study (Table 2, "w/o-PoseAug") demonstrates this empirically: removing pose augmentation causes a significant drop in appearance consistency (from 4.12 to 3.92 on human evaluation), confirming that without augmentation, the model does indeed learn to carry pose shape cues into the generated output, distorting the reference identity. The qualitative results (Figure 6b) make this visible: without augmentation, the generated character's body proportions shift toward those encoded in the pose skeleton, while with augmentation, the reference character's original body shape is preserved. **Target-oriented text guidance.** Pose augmentation solves identity leakage but introduces a secondary problem: perturbing the pose representation can obscure fine-grained motion semantics. For example, a "prayer" gesture involves clasped hands—a specific, semantically meaningful spatial configuration. Bone-length scaling might make the hand positions slightly different from the true gesture, and the model, seeing only the perturbed skeleton, might produce a generic hand-close-together motion rather than the specific prayer gesture. More broadly, skeletal representations are sparse—they show joint positions but not the *meaning* of the motion (dancing, waving, running, fighting). To compensate, the paper introduces a target-oriented text guidance mechanism that injects explicit semantic information into the generation process. This mechanism has three stages: **Stage 1: Motion semantics extraction.** A multimodal large language model (MLLM) analyzes the driving video `$\mathbf{V}$` to extract motion semantics `$T_m$`. The MLLM sees the full RGB video (not just the pose sequence) and produces a natural-language description of what motion is occurring. For a video of a person waving both hands, the output might be "a person is waving both hands." For a video of a bird flapping its wings, "a bird is flapping its wings." The MLLM has access to the rich visual information in the RGB frames that the pose representation discards—texture, context, object identity, and nuanced gesture semantics. **Stage 2: Appearance semantics extraction.** The same MLLM analyzes the reference image `$I_{\text{ref}}$` to extract appearance semantics `$T_a$`. For a reference image of a colorful bird, the output might be "a gray bird with colorful feathers." For a cartoon character, "a yellow cartoon mouse with large ears." **Stage 3: Semantic fusion.** A large language model (LLM) fuses `$T_m$` and `$T_a$` into a single target-oriented prompt `$T_{\text{fusion}}$` that describes the desired output: how the reference character should look while performing the driving motion. The fusion is designed to produce a prompt that combines the appearance description of the reference identity with the motion description of the driving video. For the examples above: "a gray bird with colorful feathers, is waving its wings" or "a yellow cartoon mouse with large ears, is waving both hands." **What "target-oriented" means.** The prompt describes the *target* output—the animated character—not the input conditions. This is important because the pre-trained Seedance 1.0 backbone was trained with text prompts that describe the content of generated videos. By providing a prompt that matches this distribution (describing what the output video should contain), the text guidance seamlessly integrates with the backbone's existing conditioning pathway without requiring architectural changes. The prompt serves as a high-level semantic prior that complements the low-level spatial guidance from the pose sequence: the pose sequence says *where* the joints should be, while the text prompt says *what* the motion means semantically. **How text guidance improves motion semantics.** Returning to the prayer gesture example: the augmented skeleton might show hands positioned near each other with some perturbation, making the specific gesture ambiguous. But the fused text prompt "a person in a white robe, is praying with hands clasped" explicitly tells the model that the intended motion is praying, not simply "hands near each other." The model can use this semantic information to refine the pose-guided generation, producing the correct gesture despite the skeleton perturbation. The text guidance thus compensates for the information loss introduced by pose augmentation while providing an additional semantic control channel that operates at a higher level of abstraction than joint coordinates. **The name "target-oriented"** refers to the fact that the text prompt describes the target output (the animated character with the reference appearance performing the driving motion), not the input conditions individually. This is in contrast to approaches that might provide separate "appearance: bird" and "motion: waving" prompts—the fused prompt integrates both aspects into a single coherent description of the desired result, matching how the pre-trained backbone expects text conditioning. **LoRA fine-tuning strategy.** The pre-trained Seedance 1.0 backbone contains strong generative priors for video quality, temporal consistency, and structural coherence. Fully fine-tuning all parameters would risk overwriting these priors through catastrophic forgetting, and would be computationally expensive. The paper instead adopts Low-Rank Adaptation (LoRA), which inserts trainable low-rank matrices into the feed-forward layers of the transformer while keeping all other parameters frozen. Specifically, for a weight matrix `$W \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$` in a feed-forward layer, LoRA parameterizes the update as `$\Delta W = BA$`, where `$B \in \mathbb{R}^{d_{\text{out}} \times r}$` and `$A \in \mathbb{R}^{r \times d_{\text{in}}}$` are trainable low-rank matrices with rank `$r \ll \min(d_{\text{in}}, d_{\text{out}})$`. The effective forward pass becomes `$h = Wx + BAx$`. During training, `$W$` is frozen and only `$A$` and `$B$` are updated. This drastically reduces the number of trainable parameters while allowing the model to adapt to the animation task. **Configuration details.** The LoRA rank is set to `$r = 256$`. LoRA modules are inserted exclusively into feed-forward layers—the attention projections are kept frozen. Critically, the text branch (the pathway that processes text prompt conditioning) is excluded from LoRA adaptation entirely. This preserves the pre-trained text-image alignment, ensuring that the model continues to correctly interpret the fused motion-appearance prompts without drifting into animation-specific text biases. **Why LoRA rather than full fine-tuning.** The design rationale has three components. First, computational efficiency: LoRA with rank 256 adds only a small fraction of the backbone's total parameters, enabling training on consumer-grade hardware with reasonable batch sizes. Second, prior preservation: freezing the backbone parameters and the text branch ensures that the pre-trained generative capabilities (high-quality video synthesis, temporal coherence, adherence to text prompts) are preserved rather than overwritten by the animation-specific training data. Third, plug-and-play flexibility: LoRA weights are lightweight and can be easily swapped or combined, enabling potential extensions where different LoRA modules specialize in different character types or motion domains. **The text branch exclusion is particularly important.** The pre-trained backbone was trained on diverse text-to-video and image-to-video data with rich, varied text prompts. The animation training data uses the relatively narrow distribution of MLLM-generated fused prompts. If the text branch were fine-tuned on this narrow distribution, it would likely overfit to the MLLM prompt style and lose the ability to generalize to diverse text inputs at inference. By freezing the text branch, the model retains its broad text understanding while using LoRA in the visual pathways to learn animation-specific visual transformations. **Training hyperparameters.** All stages train for 50,000 steps with a batch size of 2. Optimization uses AdamW with a learning rate of `$5 \times 10^{-5}$` and weight decay of 0.01. The training clips are randomly sampled between 49 and 121 frames. **Initial frame masking for seamless pose transition.** A subtle but important inference detail: to ensure that the generated animation begins with the exact reference pose (matching the reference image's character state to the first frame of the driving motion), the paper masks driving signals during the initial one-second segment of each training clip. During inference, one second of zero frames (containing no motion information) is prepended before the actual driving sequence begins. This gives the model a brief initialization period where it can establish the reference identity in its output before motion transfer begins. Without this masking, the model might produce a jarring transition where the reference character instantly snaps to the driving video's first pose, which may differ substantially from the reference image's pose. --- #### Self-Bootstrapped Data Synthesis Pipeline The transition from Pose-based to End-to-End DreamActor-M2 requires a dataset where the driving signal is raw RGB video rather than pose sequences. Such a dataset does not exist natively: to train a model to animate a reference image from a raw video driving signal, you need paired data where (1) a driving video of some subject performing some motion, (2) a reference image of a different subject, and (3) the ground-truth video of the reference subject performing the driving subject's motion are all available. Cross-identity motion transfer ground truth cannot be captured from real footage—you cannot film person A performing person B's exact motion—and certainly not at scale for diverse character types. The self-bootstrapped synthesis pipeline solves this by using the trained Pose-based model as a data generator. **Step 1: Pose extraction from source videos.** Given a source driving video `$\mathbf{V}_{\text{src}}$` (which can be any video with a moving subject—human, animal, or cartoon), the pipeline extracts its pose sequence `$\mathbf{P}_{\text{src}}$` using the 2D pose estimator. This is the same pose extraction that the Pose-based model was trained on, so there is no domain gap between the poses it saw during training and those it receives during data synthesis. **Step 2: Pseudo-video generation.** A reference image `$I_o$` is selected from a diverse pool of characters (humans, cartoons, animals). The pose sequence `$\mathbf{P}_{\text{src}}$` and the reference image `$I_o$` are fed into the trained Pose-based DreamActor-M2 model `$\mathcal{M}_{\text{pose}}$`: $$\mathbf{V}_o = \mathcal{M}_{\text{pose}}(\mathbf{P}_{\text{src}}, I_o)$$ where `$\mathbf{V}_o$` is the synthesized output video. The generated video `$\mathbf{V}_o$` has a crucial property: it preserves the motion dynamics of `$\mathbf{V}_{\text{src}}$` (because it was conditioned on `$\mathbf{P}_{\text{src}}$`, which encodes that motion) while adopting the appearance of the reference character `$I_o$`. Critically, because this is cross-identity generation (the reference image `$I_o$` is different from the subject in `$\mathbf{V}_{\text{src}}$`), `$\mathbf{V}_o$` is a *novel* video that did not exist in the training data—it shows character `$I_o$` performing the motion that was originally performed by the subject in `$\mathbf{V}_{\text{src}}$`. **Step 3: Pseudo-pair formation.** The source video `$\mathbf{V}_{\text{src}}$` and the generated video `$\mathbf{V}_o$` form a pseudo-pair `$(\mathbf{V}_{\text{src}}, \mathbf{V}_o)$`. The key insight is that `$\mathbf{V}_{\text{src}}$` can serve as the RGB driving signal (providing the motion) and `$\mathbf{V}_o$` can serve as the ground-truth target (showing what the reference character should look like when performing that motion). This pairing is exact: `$\mathbf{V}_o$` is the correct animation of the reference character `$I_o$` performing the motion from `$\mathbf{V}_{\text{src}}$`, by construction (assuming the Pose-based model generates faithfully). **Why this synthesis works in principle.** The Pose-based model `$\mathcal{M}_{\text{pose}}$` was trained to reconstruct videos from pose sequences. When given a pose sequence `$\mathbf{P}_{\text{src}}$` from a different video and a reference image `$I_o$` of a different character, it generalizes to produce a plausible animation of `$I_o$` following the motion encoded in `$\mathbf{P}_{\text{src}}$`. This generalization is possible precisely because of the pose augmentation strategies: the model learned to decouple motion from identity during training, so it can apply a motion pattern extracted from one video to a character from a different source. The generated video `$\mathbf{V}_o$` inherits the motion from `$\mathbf{V}_{\text{src}}$` (through the pose intermediary) and the appearance from `$I_o$`. **Diverse reference characters.** The paper explicitly notes that reference characters span "humans, cartoons, animals." This diversity is critical because the End-to-End model trained on this synthetic data will inherit the character distribution of the synthesis pipeline. If the pipeline only generated human-to-human pairs, the End-to-End model would not learn to animate non-human characters. By including cartoons and animals as reference images during synthesis, the pipeline produces training data covering the full spectrum of character types that the End-to-End model should handle. **Two-stage quality filtering.** The Pose-based model is not perfect—some generated videos will have artifacts, identity distortions, or motion inconsistencies. Training the End-to-End model on low-quality synthetic data would propagate and potentially amplify these errors. The paper employs a rigorous two-stage filtering protocol: **Automatic filtering via Video-Bench.** Video-Bench (Han et al., 2025) is a video quality assessment model trained to align with human judgments. Each generated video `$\mathbf{V}_o$` is scored on multiple dimensions (imaging quality, motion smoothness, temporal consistency, appearance consistency). Only videos with an average score above 4.5 (on a 1–5 scale, where 5 is excellent) are retained. This threshold corresponds to "good" to "excellent" quality and filters out videos with noticeable artifacts, temporal discontinuities, or identity distortions. **Manual verification.** The automatically filtered videos undergo human review focusing specifically on two criteria: identity fidelity (does the generated character maintain the reference image's appearance throughout the video?) and motion coherence (does the generated motion faithfully reproduce the driving video's motion?). Videos that pass automatic scoring but have subtle issues—a brief identity flicker, a slightly inaccurate gesture, a minor temporal glitch—are caught at this stage. The manual verification is targeting the specific failure modes most harmful to End-to-End model training: if the End-to-End model is trained on videos with identity inconsistencies, it might learn to allow identity drift; if trained on videos with inaccurate motion, it might learn to generate imprecise animations. **Scale of the filtered dataset.** The pipeline yields 60,000 high-quality video triplets after filtering. Given the batch size of 2 and 50,000 training steps, the End-to-End model sees 100,000 samples during training (50,000 steps × 2 batch size), meaning it samples from the 60,000-triplet dataset slightly less than twice per triplet on average. This is a relatively modest dataset size compared to the 100,000 web videos used for Pose-based training, but the quality filtering ensures that every sample provides clean supervision. **Why two-stage filtering rather than end-to-end training on all generated data.** Training on unfiltered synthetic data would create a "garbage in, garbage out" scenario where the model learns to reproduce the Pose-based model's errors. The filtering ensures that the End-to-End model is trained only on examples where the Pose-based model succeeded, effectively distilling the Pose-based model's successful generations into the End-to-End model while discarding failures. The combination of automatic and manual filtering provides both scale (automatic filtering can process 60,000 videos) and precision (manual verification catches issues that automatic metrics miss). --- #### End-to-End DreamActor-M2 Training The End-to-End variant represents the culmination of the framework: a model that animates any reference character from a raw RGB driving video, without any pose estimation, motion capture, or intermediate representation. The key training innovation is that the model learns RGB-to-RGB motion transfer using the synthetic dataset produced by the Pose-based model. **Training data triplet construction.** Each filtered pseudo-pair `$(\mathbf{V}_{\text{src}}, \mathbf{V}_o)$` is restructured into a training triplet. The driving signal is the synthesized video `$\mathbf{V}_o$` (which shows character `$I_o$` performing the motion), the reference image is the first frame of the source video `$I_{\text{ref}} = \mathbf{V}_{\text{src}}[0]$` (showing the original subject), and the target output is the source video `$\mathbf{V}_{\text{src}}$` itself. Formally, the dataset is: $$\mathcal{D} = \{(\mathbf{V}_o, I_{\text{ref}}, \mathbf{V}_{\text{src}})\}$$ where each triplet contains a driving RGB video `$\mathbf{V}_o$`, a reference image `$I_{\text{ref}}$`, and a target video `$\mathbf{V}_{\text{src}}$`. **What this triplet means operationally.** The model is trained to take `$\mathbf{V}_o$` as the driving signal (replacing the pose sequence in the spatiotemporal composite) and `$I_{\text{ref}}$` as the reference image, and produce `$\mathbf{V}_{\text{src}}$` as the output. Intuitively: "see this video of a cartoon mouse performing a dance (`$\mathbf{V}_o$`)? Now take this reference image of a real person (`$I_{\text{ref}}$`) and generate a video of that person performing the same dance, which should look like this ground-truth video (`$\mathbf{V}_{\text{src}}$`)." **Why this pairing is valid.** The synthesized video `$\mathbf{V}_o$` was generated by the Pose-based model from `$\mathbf{P}_{\text{src}}$` (the pose sequence extracted from `$\mathbf{V}_{\text{src}}$`) and `$I_o$` (a different character). This means `$\mathbf{V}_o$` and `$\mathbf{V}_{\text{src}}$` share the exact same motion—they are two different characters performing identical movements. The reference image `$I_{\text{ref}} = \mathbf{V}_{\text{src}}[0]$` is the first frame of `$\mathbf{V}_{\text{src}}$`, showing the original subject's appearance. The training task is therefore well-defined: given a driving video showing character B performing motion M (`$\mathbf{V}_o$`) and a reference image of character A (`$I_{\text{ref}}$`), generate a video of character A performing motion M (`$\mathbf{V}_{\text{src}}$`). The ground truth `$\mathbf{V}_{\text{src}}$` is available because this is the original source video from which the pose was extracted. **The crucial "reversal" of the driving signal.** Notice that the driving signal during training is `$\mathbf{V}_o$` (the synthesized video showing character `$I_o$`), not `$\mathbf{V}_{\text{src}}$` (the original source video). This is deliberate: at inference time, the driving signal will be an arbitrary RGB video of some subject performing some motion. During training, the model must learn to extract motion from RGB videos of *any* character, not just from videos of the reference character. Using `$\mathbf{V}_o$` as the driving signal—where the visual appearance is of a different character `$I_o$`—teaches the model that the driving video's pixel values encode motion, not identity. The model learns to ignore the visual identity of the driving subject (`$I_o$`) and extract only the motion pattern, which it then applies to the reference subject (`$I_{\text{ref}}$`). This is the core generalization capability: separating motion from appearance in the driving signal. **The spatiotemporal composite for End-to-End training.** The composite input for End-to-End training follows the same construction as Equation 2, but with `$\mathbf{V}_o$` as the driving signal `$\mathbf{D}$` and `$I_{\text{ref}} = \mathbf{V}_{\text{src}}[0]$` as the reference: - At `$t = 0$`: the composite is `$\mathbf{V}_{\text{src}}[0] \oplus \mathbf{V}_o[0]$` — the original character's first frame on the left, the synthesized character's first frame on the right. - At `$t > 0$`: the composite is `$\mathbf{0} \oplus \mathbf{V}_o[t]$` — black on the left, the synthesized character's motion frames on the right. The model's denoising target is the latent of `$\mathbf{V}_{\text{src}}$` (the ground-truth video). So the model sees the synthesized character performing the motion on the right side and must generate the original character performing the same motion on the left side (which begins with the reference image at `$t = 0$` and continues through `$t = T$`). **Why this composite structure transfers motion without pose.** The right half provides frame-by-frame visual motion information: at each timestep `$t$`, the model sees exactly what the motion should look like at time `$t$`, rendered on the synthesized character. The left half is the generation target: the model must produce the reference character in the corresponding motion state. Because the model was trained with spatial concatenation, it has learned to attend between the left and right halves of the composite—to extract motion information from the right and apply it to the left. The key difference from the Pose-based variant is that the right half now contains RGB pixels of a character in motion rather than a skeleton drawing. The model must learn to infer the underlying motion from the visual changes in the RGB frames—which parts are moving, how they're moving, what the motion trajectory is—and then apply that motion to the reference character's appearance. **Warm-start initialization from Pose-based model.** The End-to-End model is initialized with the weights of the trained Pose-based DreamActor-M2, not from scratch or from the pre-trained Seedance backbone. This is a critical design choice that significantly accelerates training. The Pose-based model has already learned: (1) how to decompose the composite input into reference identity and motion guidance regions, (2) how to produce temporally coherent animations, and (3) how to use text guidance for semantic control. The End-to-End model inherits all of these capabilities and only needs to adapt the motion interpretation from pose skeletons to raw RGB frames. This is a much smaller learning problem than starting from scratch: the model already knows *how* to animate; it just needs to learn to extract motion from pixels rather than from line drawings. **Why warm-starting works across modalities.** The architecture processes both pose skeletons and RGB frames through the same spatial composite and VAE encoding. The internal representations that the Pose-based model learned for interpreting the right half of the composite—attention patterns that extract motion information, temporal dynamics that track movement across frames—can be repurposed for RGB inputs. The transition is from "interpret skeleton drawings to extract motion" to "interpret RGB frames to extract motion," which is a refinement of an existing capability rather than learning an entirely new capability. The model doesn't need to relearn how to generate videos or preserve identity; it only needs to adapt its motion extraction pathway. **Training configuration.** The End-to-End model is trained for 50,000 steps with a batch size of 2, using the same AdamW optimizer with learning rate `$5 \times 10^{-5}$` and weight decay 0.01. The LoRA fine-tuning strategy is maintained, with rank 256 applied to feed-forward layers only and the text branch frozen. All hyperparameters from the Pose-based stage are carried forward without modification. **What the End-to-End model achieves at inference.** After training, the model can take any reference image and any driving RGB video (regardless of the subject's domain—human, animal, cartoon, object) and generate an animation where the reference character performs the driving video's motion. No pose estimation is required at any stage. The reference image and driving video frames are spatially concatenated as in Equation 2 (`$t = 0$`: reference on left, first driving frame on right; `$t > 0$`: black on left, driving frames on right), with the one-second zero-frame prepending for smooth initialization. The model's learned ability to extract motion from RGB pixels—developed through training on the self-bootstrapped synthetic dataset—enables it to handle driving videos of any visual style, because the feature extraction is learned entirely from data rather than hardcoded through a pose estimator. **The significance of bypassing explicit pose.** Removing the pose estimator eliminates several failure modes simultaneously. For human driving videos, it removes errors from pose estimation failures (occlusion, unusual poses, rapid movement). For non-human driving videos, it removes the categorical barrier entirely—the model can process a cartoon cat's motion because it sees the cat as RGB pixels, not as an attempt to extract a human skeleton from a non-human form. The model learns what "motion" means visually from the synthetic training data, which included diverse character types (humans, cartoons, animals) generated by the Pose-based model. This learned motion understanding generalizes to any driving video whose visual appearance falls within the distribution of the training data. **Why this two-stage approach is necessary rather than training End-to-End from scratch.** One might ask: why not directly train an End-to-End model on the synthetic data without the Pose-based intermediate stage? The answer is that the synthetic data itself does not exist without the Pose-based model—the Pose-based model generates the training data. You cannot bootstrap an End-to-End model without first having a system that can produce cross-identity animation pairs to train on. The two-stage approach is a form of **model-based data generation**: use a weaker but well-understood model (pose-conditioned animation) to generate training data for a more capable model (RGB-conditioned animation). This is analogous to using a simulator to generate training data for a real-world system, or using a teacher model to generate training data for a student model in knowledge distillation. **The paper's claim of being "the first fully end-to-end solution in this domain"** (Section 3.4) rests on this specific combination: the model takes raw RGB driving video as input and produces animated video as output, with no intermediate pose representation, no per-video optimization, and no domain-specific assumptions. Prior RGB-based approaches either required per-identity training (DreamVideo, which trains a separate model for each character) or per-video optimization (FlexiAct, which learns frequency embeddings for each driving video). DreamActor-M2's End-to-End variant is a single model that handles arbitrary reference characters and arbitrary driving videos in a single forward pass, without any test-time adaptation. --- #### Summary of Design Choices and Their Justifications - **Spatiotemporal in-context over channel-wise or cross-attention injection:** avoids the identity leakage of spatial alignment methods and the motion detail loss of compression-based methods by presenting motion as visual context in a separate spatial region, enabling the model to learn motion extraction through its native visual processing capabilities. - **Pose augmentation (bone-length scaling + bounding-box normalization) over raw pose training:** prevents the model from using pose shape as an identity shortcut; bone-length scaling at 30% rate with `$\mathcal{U}(0.8, 1.2)$` introduces sufficient variation to decouple shape from motion without destroying motion information; bounding-box normalization removes absolute spatial dependencies. - **Target-oriented text guidance via MLLM-LLM pipeline over no text or separate text inputs:** compensates for semantic information lost during pose augmentation; fuses motion semantics and appearance semantics into a single prompt matching the pre-trained backbone's text conditioning distribution; uses Gemini 2.5 for its multi-modal reasoning capabilities. - **LoRA fine-tuning (rank 256, feed-forward only, text branch frozen) over full fine-tuning:** preserves pre-trained generative priors; prevents catastrophic forgetting; keeps text alignment intact by excluding text branch from adaptation; enables lightweight, plug-and-play customization. - **Self-bootstrapped synthesis pipeline over manual data collection:** solves the absence of paired cross-identity RGB motion data; leverages the Pose-based model's generalization to produce diverse training pairs spanning humans, cartoons, and animals; the two-stage filtering (Video-Bench score > 4.5 + manual verification) ensures training data quality. - **Warm-start from Pose-based model over training End-to-End from scratch:** transfers learned animation capabilities (reference-motion decomposition, temporal coherence, text guidance usage); reduces the End-to-End learning problem to adapting motion extraction from skeletons to RGB pixels; significantly accelerates training convergence. - **One-second zero-frame prepending at inference:** ensures smooth animation initialization by giving the model a brief period to establish reference identity before motion transfer begins; prevents jarring transitions when the reference pose differs from the first driving frame's pose. ## 4. Key Insights and Innovations ### Innovation 1: Reframing Motion Conditioning as Spatiotemporal In-Context Learning Rather Than Feature Injection The paper's most distinctive conceptual contribution is not the specific composite-construction mechanism (which Section 3 details) but the fundamental reframing that makes that mechanism possible: treating motion as **visual context to be interpreted** rather than as a **foreign signal to be injected**. Prior work across pose-aligned injection (AnimateAnyone, UniAnimate, MimicMotion), cross-attention injection (MTVCrafter, Animate-X), and temporal concatenation (TIC-FT) shared an unstated assumption—that motion information is categorically different from the visual information the diffusion backbone natively processes, and therefore requires a specialized translation mechanism (an encoder, an injection module, a separate conditioning pathway) to make it usable. This assumption is so deeply embedded in the field's architecture designs that it had become invisible: every major animation system includes a pose encoder, a motion compressor, or an injection module as a matter of course. The in-context learning reframing challenges this assumption directly. It asks: what if the pre-trained video backbone already understands motion when it is presented in a visual format, and the real gap is not modality translation but **input representation**—how you show the model what you want it to do? The analogy to in-context learning in LLMs is instructive but not decorative: in both cases, the model interprets examples in its input context to infer a task, rather than having the task specification compiled into a separate encoding. An LLM doesn't need a "translation module" to process few-shot examples; it reads them as tokens in its context window. DreamActor-M2 argues that a video diffusion model similarly doesn't need a "motion encoder" if motion is presented as pixels in its visual context. This reframing has implications that extend beyond the specific architecture. It suggests that the field's focus on building better motion encoders and injection mechanisms was solving the wrong problem—the bottleneck was not in the *quality* of motion encoding but in the *assumption that encoding is necessary at all*. This is a different category of insight than "our motion encoder is 3% better than prior work." It redefines what the design space for animation models even looks like: rather than optimizing within the inject-motion-via-specialized-encoder paradigm, the in-context approach asks whether the paradigm itself is the limitation. Evidence that this reframing matters beyond this specific paper comes from the ablation study (Table 2): replacing spatiotemporal ICL with temporal-only ICL (which lacks the explicit spatial pairing of reference and motion frames) drops performance across all human-evaluated dimensions, confirming that the specific *form* of in-context presentation matters, not just the general idea of concatenation. ### Innovation 2: Diagnosing and Resolving the "See-Saw" Through a Single Architectural Mechanism The paper provides a clean diagnostic framework for the central failure mode of character animation—the trade-off between identity preservation and motion consistency—and demonstrates that this trade-off is **not inherent to the problem but rather an artifact of specific architectural choices**. Prior work treated the "see-saw" as an inevitable tension: spatial alignment methods sacrificed identity for motion accuracy, while decoupled methods sacrificed motion detail for identity preservation. The field largely accepted this as a design constraint to be managed through careful tuning, rather than a symptom of a flawed architectural assumption. DreamActor-M2's diagnosis is that the see-saw arises specifically because prior methods all **co-locate identity and motion information in the same spatial region** of the model's representation. In channel-wise concatenation, the pose features and image features occupy the same spatial grid, forcing the model to disentangle them through learned feature interactions—but the spatial correspondence between pose shape and output shape provides a shortcut that causes shape leakage. In cross-attention injection, the motion latents are spatially decoupled but then compressed into a bottleneck that discards fine-grained detail. In temporal-only ICL, the conditioning frames are temporally separated but lack per-frame spatial pairing, losing the correspondence that enables precise motion transfer. The spatiotemporal ICL solution addresses the see-saw by **spatially separating identity and motion information** while providing **temporal alignment** through the composite frame structure. The reference identity occupies the left half of the composite; the motion signal occupies the right half. They never share spatial coordinates, so there is no mechanism for pose shape to leak into reference appearance. Simultaneously, each composite frame pairs a specific motion frame with a specific output timepoint, providing the frame-wise correspondence that temporal-only methods lack. The ablation comparing spatiotemporal to temporal-only ICL (Table 2 and Figure 6a) validates this diagnosis: the performance gap—appearance consistency drops from 4.12 to 4.06 on human evaluation, but more tellingly, the qualitative results show degraded fine-grained spatial fidelity like hand gesture preservation—confirms that spatial pairing matters beyond what temporal attention alone can learn. What makes this a genuine insight rather than an architectural trick is that it identifies the **root cause** (spatial co-location of identity and motion signals) and shows that addressing it at the representation level eliminates the trade-off without requiring auxiliary losses, adversarial training, or careful hyperparameter balancing. The see-saw was not a fundamental limitation of diffusion models for animation; it was a consequence of a specific, widely-adopted architectural convention. The paper doesn't just propose a better injection method—it explains *why* prior injection methods create the trade-off and *why* the proposed approach doesn't. ### Innovation 3: Self-Bootstrapped Data Synthesis as a Bridge Between Modalities The transition from pose-conditioned to RGB-conditioned animation would ordinarily be impossible without access to paired cross-identity RGB motion data—a dataset that cannot exist in the real world because you cannot capture two different subjects performing identical motions under identical conditions at scale. The paper's solution—using a weaker model (Pose-based) to generate synthetic training data for a stronger model (End-to-End)—is not individually novel (model-based data generation appears in knowledge distillation, sim-to-real transfer, and self-training), but its application to the **specific problem of cross-modal transition in animation** constitutes a distinctive methodological contribution. The insight has two layers. First, the Pose-based model, trained with careful augmentation to decouple motion from identity, naturally generalizes to cross-identity generation—it can take a pose sequence from video A and a reference image of character B and produce a plausible animation of B performing A's motion. This generalization is not perfect (hence the need for quality filtering), but it is good enough to serve as training supervision. Second, the generated pseudo-pairs `(V_o, V_src)` are structured so that `V_o` (the Pose-based model's output, showing character B performing the motion) serves as the *driving signal* while `V_src` (the original source video, showing character A) serves as the *target*—inverting the typical relationship between driver and target to teach the End-to-End model that the RGB driving video's identity is irrelevant and only its motion matters. This is intellectually distinctive because it solves a **data bootstrapping problem** that had blocked progress toward pose-free animation. Prior work that attempted RGB-driven animation either abandoned cross-identity generalization (DreamVideo's "one-model-per-identity" paradigm), incurred per-video computational costs (FlexiAct's frequency-aware embeddings), or remained tethered to pose during training (X-Unimotion). The self-bootstrapped pipeline provides a principled path from the well-studied regime of pose-conditioned animation—where training data is abundant via web videos with pose extraction—to the novel regime of RGB-conditioned animation, where no training data exists. The quality filtering protocol (Video-Bench score above 4.5, plus manual verification of identity fidelity and motion coherence) ensures that the synthetic data provides clean supervision rather than amplifying the Pose-based model's errors. The evidence that this bootstrapping works is the End-to-End model's performance on AW Bench (Table 1): it matches or exceeds the Pose-based variant across all metrics while eliminating the pose estimator entirely, achieving 4.35 automatic appearance consistency vs. 4.28 for the Pose-based variant, and 4.24 human-evaluated motion consistency vs. 4.18. The End-to-End model's superior performance despite receiving no explicit pose information suggests that the synthetic data provides richer motion supervision than skeleton sequences alone—RGB frames convey texture, occlusion, and deformation cues that sparse joint coordinates cannot. This is a case where the bootstrapped student outperforms the teacher, validating the pipeline's design. ### Innovation 4: Operationalizing "Universal" Animation Through AW Bench and a Cross-Domain Evaluation Philosophy The paper introduces AW Bench not merely as a new dataset but as an **operationalization of a different evaluation philosophy** for character animation. The dominant evaluation paradigm in prior work—measuring reconstruction fidelity on same-identity test sets using metrics like FID, FVD, and CD-FVD—implicitly defines success as "how well does the model reproduce the ground-truth video." This definition is fundamentally incompatible with cross-identity animation, where no ground-truth video exists (you cannot photograph person A performing person B's exact motion in the real world), and is inapplicable to non-human characters for which pose estimators and structure-based metrics were never designed. AW Bench makes three conceptual moves that individually challenge field conventions and collectively define a new evaluation standard. First, it **expands the character ontology** beyond humans to include animals (cats, chickens, monkeys, orangutans) and cartoon characters (Tom and Jerry, cartoon aliens), making the benchmark's composition a statement that "character animation" should not implicitly mean "human animation." Second, it **expands the interaction topology** to include multi-subject scenarios (many-to-many and one-to-many driving) that prior benchmarks never considered, testing whether models can handle simultaneous motion transfer across multiple distinct identities. Third, it adopts **human-aligned automated evaluation** via Video-Bench rather than pixel-space fidelity metrics, recognizing that for cross-identity generation, factors like identity preservation and motion smoothness are perceptual judgments that cannot be reduced to frame-wise reconstruction error. This is more than a benchmark contribution—it redefines what the field should optimize for. Prior metrics like FID-FVD measure distributional similarity to ground-truth videos, which implicitly rewards models that reproduce the training distribution rather than generalize across domains. AW Bench's evaluation dimensions (imaging quality, motion smoothness, temporal consistency, appearance consistency) measure the perceptual qualities that matter for deployment: does the animation look good? Does the motion flow naturally? Does the character maintain its identity? These are the questions that end-users ask, and the paper argues—through the benchmark's design and the correlation between automatic and human evaluations in Table 1—that they can be measured reliably without ground-truth references. The significance of this innovation is that it enables **fair comparison across fundamentally different approaches**. A pose-based method and an end-to-end method cannot be compared on reconstruction error because they have different failure modes and different strengths. But they can be compared on whether human judges find their outputs visually faithful and motionally coherent. The GSB comparison against platform-level products (Figure 5), which shows DreamActor-M2 leading Kling 2.6 by +9.66% and outperforming other products by margins of +43% to +57%, demonstrates that AW Bench's evaluation philosophy produces rankings that align with subjective quality judgments at industrial scale. This makes the benchmark a credible standard for measuring progress toward genuinely universal animation. ## 5. Experimental Analysis ### Evaluation Methodology - **Dataset.** All experiments use AW Bench, a new benchmark introduced in this paper (Section 4). It consists of 100 driving videos and 200 reference images. The driving corpus spans human motions (different body regions, age groups, activity categories, camera-tracked and static-camera sequences) and non-human motions (animals including cats, chickens, parrots, monkeys, and orangutans, plus cartoon characters including Tom the cat, Jerry the mouse, groundhogs, and cartoon aliens). The reference corpus matches this diversity and adds multi-subject scenarios (many-to-many and one-to-many driving). The main quantitative evaluation uses a subset of 60 human-to-human and 40 human-to-cartoon animation pairs (Section 5.2). AW Bench is specifically motivated by the inadequacy of existing benchmarks—prior datasets like TaiChi or TikTok are exclusively human-centric and cannot evaluate cross-domain generalization (Section 4). - **Base model.** All DreamActor-M2 variants use **Seedance 1.0** as the pre-trained video diffusion backbone. Seedance 1.0 employs the MMDiT architecture and supports both text-to-video and image-to-video generation. The paper states that character image animation "inherently aligns with the image-to-video setting" (Section 2), motivating the choice. For the MLLM component, the paper uses **Gemini 2.5** for its "superior multi-modal reasoning" capabilities in parsing driving videos into motion semantics and reference images into appearance semantics (Section 5.1). - **Metrics.** The paper rejects standard metrics like FID-FVD, FVD, and CD-FVD because they "rely on comparisons with ground-truth videos, which are unavailable in cross-identity animation scenarios" and "are often inconsistent with human judgment" (Section 5.1). Instead, evaluation uses **Video-Bench**, a human-aligned automatic evaluation protocol that scores videos on four perceptual dimensions: **Imaging Quality**, **Motion Smoothness**, **Temporal Consistency**, and **Appearance Consistency**, all on a 1–5 scale (1 = very poor, 2 = poor, 3 = moderate, 4 = good, 5 = excellent). Human evaluation follows the same 1–5 scale with 12 participants evaluating 100 randomly selected samples per method. Additionally, a **GSB (Good-Same-Bad)** subjective comparison is conducted against mainstream platform-level products using a shared test dataset (Section 5.2). - **Baselines.** The paper compares against four state-of-the-art methods: **Animate-X++** (Tan et al., 2025)—a cross-attention injection approach that extended reference image subjects to anthropomorphic characters; **MTVCrafter** (Ding et al., 2025)—a cross-attention-based method with auxiliary pose encoding; **DreamActor-M1** (Luo et al., 2025)—the prior version of the DreamActor framework; and **Wan2.2-Animate** (Cheng et al., 2025)—a recent pose-guided animation method. For GSB comparison, additional platform-level products include **Kling 2.6**, **Kling-O1**, and **Wan2.2-Animate** (Section 5.2). - **Generation budget / compute accounting.** The paper does not report FLOPs counts, inference latency, or generation steps. Training cost is reported in steps (50,000 steps for all stages with batch size 2). The inference budget is implicitly one forward pass per animation—the model generates the full video in a single denoising process. No per-sample generation budgets are specified (e.g., number of denoising steps). There is no analysis of how performance scales with increased inference compute (e.g., multiple samples per prompt, best-of-N selection), which is a notable omission given that competing methods could potentially benefit from test-time compute scaling. - **Cross-validation / statistical protocol.** Human evaluation uses 12 participants rating 100 randomly selected samples per method (Section 5.2). Error bars are reported as standard deviations (e.g., `4.23 ± 0.19` for Pose-based DreamActor-M2 human-evaluated Imaging Quality in Table 1). The GSB evaluation (Figure 5) uses a shared test dataset comparing method outputs side-by-side. There is no cross-validation protocol described for the automatic Video-Bench evaluations. The paper does not report statistical significance tests between methods, confidence intervals for automatic metrics, or the number of test samples used for automatic evaluation beyond the 100-pair subset (60 human-to-human + 40 human-to-cartoon) mentioned in Section 5.2. ### Main Quantitative Results #### Comparison Against State-of-the-Art Methods on AW Bench The headline results appear in Table 1, which reports both automatic Video-Bench evaluations and human evaluations across all four perceptual dimensions for six methods. **Pose-based DreamActor-M2** achieves automatic scores of 4.68 (Imaging Quality), 4.53 (Motion Smoothness), 4.61 (Temporal Consistency), and 4.28 (Appearance Consistency). **End-to-End DreamActor-M2** achieves 4.72, 4.56, 4.69, and 4.35 respectively—slightly exceeding the Pose-based variant on every automatic metric. This is notable because the End-to-End model achieves superior performance despite receiving no explicit pose information at inference. The gap to the next-best method (DreamActor-M1) is substantial. On automatic Appearance Consistency—the dimension most directly measuring identity preservation—End-to-End DreamActor-M2 achieves 4.35 vs. DreamActor-M1's 4.06, a +0.29 absolute improvement. On Motion Smoothness, the gap is 4.56 vs. Wan2.2-Animate's 4.06, a +0.50 improvement. Animate-X++ trails substantially across all metrics, with automatic Appearance Consistency at 3.21 vs. 4.35 for the End-to-End model—a gap of over one full point on the 1–5 scale. **Human evaluation results in Table 1** largely mirror the automatic rankings but with compressed differences. End-to-End DreamActor-M2 achieves human-evaluated Imaging Quality of 4.27 (±0.18), Motion Consistency of 4.24 (±0.23), and Appearance Consistency of 4.20 (±0.29). DreamActor-M1 achieves 3.96 (±0.21), 3.72 (±0.26), and 3.54 (±0.31) respectively. The standard deviations are consistently reported and show that the improvements exceed the rating variance: the gap between End-to-End DreamActor-M2 and DreamActor-M1 on Motion Consistency (4.24 vs. 3.72, Δ = 0.52) is larger than either method's standard deviation (0.23 and 0.26), suggesting reliable separation despite the modest evaluator count (n = 12). A noteworthy pattern: the human evaluations for Animate-X++ and MTVCrafter are **markedly lower than their automatic scores**. For Animate-X++, automatic Motion Smoothness is 3.42 but human-evaluated Motion Consistency is 2.95 (±0.29)—a drop of nearly half a point that pushes the score below the "moderate" threshold (3.0). Similarly, Animate-X++'s automatic Appearance Consistency of 3.21 drops to 2.86 (±0.34) in human evaluation. This suggests that Video-Bench's automatic metrics may overestimate quality for methods with specific failure modes that human raters are more sensitive to—possibly the motion-identity see-saw artifacts that degrade perceptual quality without triggering the automatic metric's feature detectors. #### GSB Comparison with Platform-Level Products Figure 5 reports a Good-Same-Bad (GSB) comparison where DreamActor-M2 is pitted head-to-head against mainstream products. The exact GSB computation method is not detailed in the paper text, but the standard interpretation is: for each test sample, raters judge whether DreamActor-M2's output is better than (Good), comparable to (Same), or worse than (Bad) the competitor's output. The net GSB lead is then reported as (Good% − Bad%). The results, as quoted in Section 5.2: > "DreamActor-M2 matches the overall performance of Kling 2.6 with a +9.66% GSB lead, and outperforms other products by a significant margin: +43.66% over Kling-O1, +51.43% over Wan2.2-Animate, and +57.04% over the prior DreamActor-M1." The claim that DreamActor-M2 "matches the overall performance of Kling 2.6" with a +9.66% lead is notable—a single-digit GSB advantage suggests comparable quality where DreamActor-M2 edges ahead slightly. The margins against other products are substantially larger, with Wan2.2-Animate at +51.43% and the prior DreamActor-M1 at +57.04%, indicating that DreamActor-M2 represents a meaningful step forward from these baselines. However, several details about the GSB evaluation are missing from the paper: (1) the number of test samples used, (2) the number of human raters, (3) whether the comparison was blind (raters unaware of which model produced which output), (4) whether raters evaluated full videos or individual frames, and (5) whether the "shared test dataset" is a subset of AW Bench or a separate collection. Without these details, the GSB numbers are suggestive but cannot be evaluated for statistical reliability or methodological rigor. #### Qualitative Comparison Results Figure 4 presents a four-row qualitative comparison across diverse animation scenarios. The paper describes (Section 5.3): - **First row (standard intra-domain driving):** "DreamActor-M2 demonstrates plausible visual fidelity, maintaining stringent identity preservation and motion alignment where baselines exhibit blurring." This suggests that competing methods suffer from visual degradation (blur) that DreamActor-M2 avoids—consistent with the Imaging Quality metric advantage in Table 1. - **Second row (body shape preservation):** DreamActor-M2 shows "remarkable body shape preservation and faithful motion alignment," outperforming others in "capturing fine-grained motion details." This aligns with the Motion Smoothness and Appearance Consistency advantages. - **Third row (heart gesture):** "The accurate generation of the 'heart gesture' validates the model's superior grasp of motion semantics." This is a specific, interpretable test case—hand gestures with precise finger configurations are notoriously difficult for animation models, and success here provides face validity for the target-oriented text guidance mechanism. - **Fourth row (one-to-many driving):** "The robustness of our method in one-to-many driving scenarios, a demanding task where competitors typically suffer from severe visual artifacts or structural collapse." Multi-subject animation tests the model's ability to maintain multiple distinct identities simultaneously while transferring motion to each—a scenario that stresses both identity preservation and motion coherence mechanisms. The generalizations demonstrated in Figure 1 include: (1) cross-shot-type animation (Half2Full tasks where lower-body driving signals are absent but the model synthesizes plausible lower-body motion using generative priors), (2) diverse reference characters (rabbit, juice bottle, Detective Conan, Pikachu)—object animation (juice bottle) is particularly notable as it tests morphologies with no human-like joint structure, (3) non-human driving characters (Animal2Animal and Cartoon2Cartoon transfers), and (4) multi-person scenarios (One2Multi and Multi2Multi). These are presented as qualitative evidence of generalization rather than quantitative results—they demonstrate the model's range but do not provide per-category success rates or failure analysis. ### Ablation Studies and Robustness Checks **Spatiotemporal vs. temporal-only in-context injection (Temp-IC)**: Table 2 compares DreamActor-M2 against a variant using temporal concatenation without the spatial pairing (Temp-IC, following TIC-FT, Kim et al., 2025). Human evaluations show DreamActor-M2 outperforms Temp-IC on all three measured dimensions: Imaging Quality 4.23 vs. 4.12, Motion Consistency 4.18 vs. 3.98, Appearance Consistency 4.12 vs. 4.06. The Motion Consistency gap (Δ = 0.20) is the largest relative difference, consistent with the paper's claim that temporal-only injection lacks "frame-wise spatial correspondence" and degrades fine-grained motion details. The qualitative visualization in Figure 6(a) shows DreamActor-M2 better preserves "intricate structural details—such as hand gestures"—confirming that the spatial pairing in the composite input provides motion precision beyond what temporal attention alone can achieve. This is a genuine ablation rather than a straw-man comparison because TIC-FT is a published prior method with a similar in-context philosophy but different spatial structure. **Pose augmentation (w/o-PoseAug)**: Removing both bone-length scaling and bounding-box normalization drops human-evaluated Appearance Consistency from 4.12 to 3.92 (Table 2)—the largest single ablation effect on any metric. Motion Consistency drops from 4.18 to 3.80 (Δ = 0.38), an even larger relative decline. This is a striking result: removing pose augmentation hurts motion consistency *more* than appearance consistency in absolute terms (0.38 vs. 0.20), even though the augmentation was primarily designed to prevent identity leakage. The paper's interpretation—that pose augmentation enables the model to learn "pure motion dynamics" rather than identity-entangled motion—is supported by the qualitative result in Figure 6(b), which shows that without augmentation, the model distorts the reference subject's body shape to match the pose skeleton's proportions. A nuance: the ablation removes *both* augmentation strategies simultaneously, so it cannot attribute the effect to bone-length scaling versus bounding-box normalization independently. The 30% application rate for bone-length scaling (Section 3.3) is a specific hyperparameter choice, but no ablation studies the effect of different scaling rates or distributions—for example, would 50% scaling application be better or worse? Would different scaling ranges (e.g., `U(0.7, 1.3)`) improve or degrade performance? **Target-oriented text guidance (w/o-TOTG)**: Removing the MLLM-LLM text guidance pipeline reduces Motion Consistency from 4.18 to 3.85 (Table 2, Δ = 0.33) and Appearance Consistency from 4.12 to 4.08 (Δ = 0.04). The asymmetric impact is informative: text guidance primarily improves *motion* quality rather than identity preservation, consistent with its design purpose of compensating for semantic information lost during pose augmentation. The near-zero effect on Appearance Consistency (the 0.04 drop is well within plausible evaluator variance given standard deviations of ±0.28–0.31 in Table 1) confirms that the text guidance achieves its intended function of improving motion semantics without introducing identity-entangling cues—an important negative result for the design claim that fused prompts provide "high-level semantic prior" without leaking identity information. Figure 6(c) qualitatively shows DreamActor-M2 "better reconstruct semantically specific motions and preserve character identity" with text guidance. **End-to-End vs. Pose-based comparison**: Table 1 shows the End-to-End variant slightly outperforms the Pose-based variant on all automatic metrics (e.g., Appearance Consistency 4.35 vs. 4.28, Temporal Consistency 4.69 vs. 4.61). Human evaluations show the same pattern with tighter margins (Motion Consistency 4.24 vs. 4.18, Appearance Consistency 4.20 vs. 4.12). This is an important positive result: the End-to-End model, which receives no explicit pose information at inference, matches or exceeds the Pose-based model that has direct access to extracted skeletons. Figure 6(d) illustrates a specific advantage: the End-to-End model outperforms in "challenging 2D keypoint detection scenarios, including direction ambiguity and hand overlapping"—cases where the pose estimator produces noisy or ambiguous skeletons that degrade the Pose-based model's output. The End-to-End model, by learning motion directly from RGB, circumvents these pose estimation failures. However, the End-to-End model's advantage is modest—the differences are within or near the reported standard deviations for human evaluation (standard deviations range from ±0.18 to ±0.29). A statistical significance test is not reported, so it is unclear whether the End-to-End model is genuinely better or merely comparable to the Pose-based variant. The paper does not discuss whether the End-to-End model shows degraded performance on any specific category where the Pose-based model excels—for example, on human-to-human animation where pose estimation is highly reliable, the Pose-based model might outperform due to its cleaner motion signal. **Absent ablations** that would have strengthened the paper but are not reported: (1) The effect of the two-stage quality filtering protocol—what is the End-to-End model's performance if trained on unfiltered synthetic data? This would quantify the contribution of data quality filtering. (2) The effect of warm-start initialization—what is the End-to-End model's performance if trained from the pre-trained Seedance checkpoint rather than from the Pose-based model weights? This would isolate the contribution of the Pose-based model's learned representations. (3) The contribution of the one-second zero-frame prepending at inference—does removing this initialization period degrade the animation start quality? (4) The effect of LoRA rank—is rank 256 necessary, or would lower ranks achieve comparable performance with fewer parameters? (5) The effect of VAE encoding on the composite input—does spatial concatenation before VAE encoding introduce artifacts at the boundary between the reference and motion regions that affect generation quality? ### Critical Assessment #### Claim 1: "Spatiotemporal in-context motion conditioning balances identity preservation with motion consistency" **What the experiments actually demonstrate:** The comparison against three alternative injection paradigms (Table 2: Temp-IC representing temporal ICL; Table 1: Animate-X++ representing cross-attention injection; DreamActor-M1 and Wan2.2-Animate representing pose-aligned injection) shows DreamActor-M2 achieves superior scores on dimensions measuring both identity (Appearance Consistency) and motion (Motion Smoothness / Motion Consistency). The specific mechanism by which this balance is achieved—spatial separation of identity and motion regions in the composite input preventing shape leakage while providing frame-wise correspondence—is supported by the ablation against temporal-only ICL, which lacks the spatial pairing and shows degraded motion consistency. **Limitations of the evidence:** The comparison is against methods that differ from DreamActor-M2 in *multiple* ways simultaneously (different backbones, different training data, different text conditioning strategies). The performance gap cannot be attributed solely to the spatiotemporal ICL mechanism—it could be partially due to the Seedance 1.0 backbone being more capable than competitors' backbones, the MLLM text guidance being more sophisticated, the training data being larger or more diverse, or the LoRA fine-tuning strategy being more effective at preserving pre-trained priors. The ablation against Temp-IC is the cleanest comparison because both methods use spatiotemporal composites in the same backbone, but Temp-IC is not necessarily the strongest possible temporal-only baseline—a method with explicit temporal correspondence mechanisms (e.g., optical flow guidance) might close the gap. More fundamentally, the paper claims to "resolve" the see-saw but does not demonstrate that the see-saw is *eliminated* rather than *shifted*. Does DreamActor-M2 exhibit a different trade-off at extreme budgets, on unusual character types, or under distribution shift? The AW Bench results show superior performance but do not characterize the *scaling behavior* of identity preservation vs. motion consistency as a function of compute, video length, or character complexity. The qualitative results showcase successes but do not systematically catalog failures—do certain character types or motion patterns still cause identity drift or motion degradation, and if so, are they different from the patterns that cause failures in prior methods? #### Claim 2: "Self-bootstrapped synthesis pipeline enables end-to-end, pose-free animation" **What the experiments actually demonstrate:** The End-to-End model exists and achieves state-of-the-art performance (Table 1). This demonstrates feasibility: it is possible to train an RGB-to-RGB animation model using data synthesized by a pose-conditioned model. The End-to-End model slightly outperforms the Pose-based variant on most metrics, showing that the synthetic data provides effective supervision and that removing the pose estimator does not degrade performance—and may even improve it in cases where pose estimation fails (Figure 6d). **Limitations of the evidence:** The most significant gap is the **absence of any ablation characterizing the synthetic data's contribution**. The paper does not report: (1) how many synthetic videos were generated before filtering (only the post-filtering count of 60,000), so the filtering rejection rate is unknown—if 90% of generated videos failed quality checks, the pipeline's efficiency is low and its scalability to even more diverse character types is questionable; (2) the performance of the Pose-based model on the specific synthetic data task—when generating `V_o` from `P_src` and `I_o`, what is the Pose-based model's own fidelity? If the Pose-based model produces low-quality animations for certain character types, those types will be underrepresented or absent in the synthetic training data, creating a hidden domain restriction; (3) whether the End-to-End model inherits biases from the Pose-based model—does it fail on the same character types or motion patterns where the Pose-based model struggles? The claim of "end-to-end, pose-free animation" is technically accurate for inference but masks an important detail: the training pipeline still depends on pose estimation (to extract `P_src` from `V_src` for the Pose-based model's data synthesis step). The End-to-End model is *indirectly* pose-dependent through its training data. If pose estimation fails for certain video types (e.g., non-humanoid characters with no skeleton equivalent), those types cannot be included in the synthetic dataset, and the End-to-End model will not learn to animate them—despite being "pose-free" at inference. The paper does not discuss what fraction of the 60,000 synthetic training pairs involve non-human characters, so the claim of "universal" animation depends on the Pose-based model's ability to generalize to non-human skeletons, which is itself limited by the skeleton extractor's domain of applicability. Additionally, the End-to-End model's slight improvement over the Pose-based variant (Table 1: e.g., +0.07 on automatic Appearance Consistency) may not be statistically significant. The paper does not report confidence intervals for automatic metrics or conduct significance tests. If the two variants are statistically indistinguishable, the primary contribution of the End-to-End model is not improved quality but reduced computational dependency (no pose estimator at inference), which is a practical benefit but a weaker claim than "improved performance." #### Claim 3: "AW Bench provides a comprehensive evaluation platform for character image animation" **What the experiments actually demonstrate:** AW Bench exists and is used to evaluate DreamActor-M2 against baselines. The benchmark covers character types (humans, animals, cartoons) and scenarios (multi-subject, cross-domain) absent from prior evaluation datasets. The Video-Bench automatic metrics show rank-ordering consistent with human evaluation (Table 1), providing evidence that AW Bench's evaluation protocol aligns with perceptual quality. **Limitations of the evidence:** The benchmark is **small**—100 driving videos and 200 reference images. The main evaluation subset is 60 human-to-human pairs and 40 human-to-cartoon pairs (100 total), which is an order of magnitude smaller than standard animation benchmarks (e.g., the TikTok dataset contains hundreds of videos). With 100 test pairs and 12 human raters evaluating 100 random samples per method, the statistical power to detect differences between methods is limited—the reported standard deviations of ±0.18 to ±0.36 on a 1–5 scale mean that differences smaller than approximately 0.3–0.5 points may not be reliable. More critically, AW Bench is introduced in this paper and **used exclusively to evaluate the authors' own method**. The baseline results in Table 1 are generated by the authors running competitors' methods on AW Bench, which raises questions about fair comparison: were the baselines tuned for AW Bench's distribution? Do the baselines' default hyperparameters (potentially tuned for human-centric benchmarks) transfer well to AW Bench's animal and cartoon data? The paper does not describe any effort to optimize baselines for AW Bench, and the dramatic performance gaps (e.g., Animate-X++ at 2.86 human-evaluated Appearance Consistency vs. 4.20 for End-to-End DreamActor-M2) may partially reflect distribution shift rather than inherent method superiority. The benchmark is also **not publicly available** at the time of writing (no download link or repository is mentioned in the paper), meaning the results cannot be independently verified or used by other researchers. The claim of providing "the community a more challenging and rigorous evaluation platform" depends on the benchmark being released—without release, it functions as an internal test set that advantages the authors' method development. #### Claim 4: "DreamActor-M2 matches or exceeds platform-level products (Kling 2.6, etc.)" **What the experiments actually demonstrate:** The GSB comparison (Figure 5) shows DreamActor-M2 with a +9.66% lead over Kling 2.6 and larger leads over other products. These numbers are reported without methodological detail (sample size, rater count, blinding, evaluation protocol). **Limitations of the evidence:** The GSB comparison is the least well-documented experiment in the paper. Critical questions are unanswered: were raters evaluating full videos or static frames? Were videos presented at matched resolutions and frame rates? Were the platform products accessed through their standard APIs with default settings, or were generation parameters optimized? Was the comparison blind? A single-digit GSB lead (+9.66%) over Kling 2.6 is a narrow margin—if the evaluation used, say, 50 test samples and 5 raters, the confidence interval around that +9.66% would be wide enough that the result might not be statistically distinguishable from "comparable performance." The paper's own characterization—"matches the overall performance"—acknowledges this narrowness, but the GSB number is presented as a precise figure without uncertainty quantification. Additionally, the GSB comparison is against **commercial products whose internal architectures and training procedures are unknown**. Kling 2.6 may use a different model size, different training data, different inference budget, and different optimization targets than DreamActor-M2. A +9.66% GSB lead is informative for practical deployment decisions but provides limited scientific insight into *why* DreamActor-M2 performs well—the performance gap could be due to any combination of model architecture, training data scale, compute budget, or domain-specific optimization. #### Missing Experiments The paper would be strengthened by several experiments that are notably absent: - **Scaling analysis:** How does performance change with training data quantity, model size, video length, or number of denoising steps? The paper uses fixed hyperparameters (50,000 steps, LoRA rank 256, 49–121 frame clips) without studying sensitivity. - **Failure mode characterization:** The qualitative results show successes; where does DreamActor-M2 fail? Section 7 mentions struggles with "complex interactions, such as two characters rotating around each other" attributed to training data scarcity, but provides no quantitative failure analysis. What fraction of AW Bench samples show identity degradation, motion inconsistency, or temporal artifacts? How does this vary by character type and motion complexity? - **Comparison against fine-tuned variants of baselines:** The baselines are evaluated as-is, but given that AW Bench includes non-human characters, would fine-tuning the baselines on similar data close the performance gap? This would distinguish whether DreamActor-M2's advantage comes from the in-context architecture or from training data diversity. - **Per-category breakdown:** AW Bench spans humans, animals, cartoons, single-subject, and multi-subject scenarios. Reporting per-category metrics would reveal which scenarios drive the aggregate performance advantage and whether DreamActor-M2 has specific weaknesses masked by aggregate scores. - **Diversity of driving motions relative to output quality:** The paper claims 100,000 training videos but doesn't characterize their motion diversity. Do certain motion types (cyclic motions like walking, complex motions like dancing, subtle motions like facial expressions) show systematically different quality? This matters for practical deployment where the motion distribution may differ from training. - **Latency and computational cost:** The paper provides no inference time measurements, GPU memory requirements, or FLOPs comparisons against baselines. For practical deployment, knowing whether DreamActor-M2 requires 2× or 10× the compute of a simpler baseline is essential—a method that achieves +5% better appearance consistency at 10× the inference cost may not be a net improvement for many applications. ## 6. Limitations and Trade-offs ### The Difficulty Estimation Cost Is Not Amortized or Accounted For **The assumption or constraint.** The compute-optimal framework rests on the ability to assign each prompt to one of five difficulty bins before strategy execution. The method for doing so—generating 2048 samples per question and computing the PRM's average final-answer score—is extraordinarily expensive. The paper acknowledges this directly in Section 3.2: > "estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity" The 2048 samples required for difficulty estimation alone exceed the largest test-time budgets studied (256–512 generations), meaning the difficulty estimation step can consume more compute than the actual problem-solving strategy it enables. **The consequence.** The reported 4× efficiency gains over best-of-N (Figures 4 and 8) are computed *after* difficulty is already known, without amortizing the cost of learning it. In a realistic deployment, the total cost would be difficulty estimation + strategy execution. If difficulty estimation costs, say, 2048 generations and the strategy uses 64 generations, the *actual* total cost is 2112 generations—vastly worse than the headline 64-generation figure. The difficulty estimation would dominate the budget, eliminating the efficiency advantage. For the approach to be practical, difficulty must be estimated far more cheaply than the 2048-sample oracle method, but the paper provides no mechanism for doing so and no analysis of how estimation accuracy degrades with fewer samples. The predicted (non-oracle) difficulty bins perform similarly to oracle bins in the paper's analysis (Figures 4 and 8), but both still require 2048 samples—the "predicted" variant only removes the need for ground-truth labels, not the sampling cost. The paper frames this as future work in Section 8 but provides no experimental evidence that cheaper difficulty estimation is feasible. **What evidence exists in the paper.** The paper explicitly states that difficulty estimation cost is not included (Section 3.2), so the limitation is acknowledged rather than hidden. However, no experiment quantifies how the compute-optimal policy's performance degrades as difficulty estimation uses fewer samples. The cross-validation protocol (Section 3.2) ensures that strategy selection is not contaminated by test-set evaluation, but it does not address the cost of obtaining difficulty estimates. This is a **missing experiment**: what accuracy does the compute-optimal policy achieve if difficulty is estimated from 4, 16, or 64 samples rather than 2048? **Mitigation status.** The paper does not attempt to mitigate this limitation. It flags "cheap difficulty estimation" as a key avenue for future work (Section 8), suggesting models that predict difficulty directly from question text or from the PRM's score distribution on a small number of initial samples. However, no such mechanism is developed or tested. The limitation is acknowledged but unresolved, making the 4× efficiency figure an upper bound rather than a realized deployment gain. --- ### The Method Provides No Benefit on Hard Problems—A Fundamental Capability Ceiling **The assumption or constraint.** Test-time compute amplifies a model's ability to produce correct answers it already generates at some non-trivial rate, but it cannot create capability from nothing. This is not an assumption the paper makes in error—it is a boundary the paper correctly identifies—but it constitutes a fundamental limitation on the method's applicability. As acknowledged in the Section 7 takeaway: > "Test-time compute can amplify existing capability but does not create it from nothing." **The consequence.** On the hardest questions (difficulty bin 5), **all methods**—search, revisions, and their compute-optimal combinations—show essentially zero improvement regardless of compute budget. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all methods and all budgets (4 to 256 generations). In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio at 128 generations. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%, and the 14× larger model consistently outperforms test-time compute on these problems across all R regimes. This means that for genuinely novel or out-of-distribution reasoning problems—problems where the base model's pass@1 is near zero—the compute-optimal framework offers **no mechanism for improvement**. No amount of search can find a correct solution that does not exist in the proposal distribution, and no amount of revision can refine an answer the model fundamentally cannot produce. For practitioners, this means the method is only useful when the problem distribution is skewed toward easy-to-medium difficulty. If a deployment encounters even a modest fraction of genuinely hard problems, those problems will be answered essentially at chance level regardless of inference budget. The paper provides no diagnostic for distinguishing in advance which problems are "hard" (pass@1 near zero, where compute won't help) vs. "medium" (pass@1 low but non-zero, where compute can help), and the difficulty bins are relative to the base model's capabilities, not absolute problem difficulty—a problem that is hard for PaLM 2-S* might be medium for a stronger base model, and vice versa. **What evidence exists in the paper.** The bin 5 flatlining is consistent across all experiments: search (Figure 3, right), revisions (Figure 7, right), and FLOPs-matched comparisons (Figure 9). The paper is transparent about this limitation in the Section 7 takeaway and the FLOPs-matched analysis. However, the paper does not characterize **how rapidly** the method degrades as difficulty increases—the five-bin discretization means we see a sharp drop from bin 4 (where methods show modest gains) to bin 5 (where nothing works), but the underlying difficulty continuum might have a more gradual transition. The bin boundaries are determined by pass@1 quintiles on the 500-question test set, so bin 5 represents the hardest 20% of questions, but the paper does not report the absolute pass@1 thresholds for each bin, making it impossible to know what "hard" means quantitatively. **Mitigation status.** The paper acknowledges this as a fundamental boundary rather than attempting to mitigate it. The finding that test-time compute cannot help on the hardest problems is presented as an important empirical result rather than a limitation to be solved. The implication—that some capabilities can only be acquired through pretraining—is a key takeaway from the FLOPs-matched comparison. No future work is suggested for addressing this hard-problem failure mode, likely because it is inherent to the test-time compute paradigm rather than a fixable implementation issue. --- ### The Revision Model Shows a 38% Correct-to-Incorrect Reversion Rate, and Revision Training Is Fragile **The assumption or constraint.** The revision model is trained exclusively on sequences where all in-context answers are incorrect, followed by a correct target (Section 6.1). This is a deliberate training data construction choice: for each question, the model sees 0–4 incorrect answers (the last being the one with smallest edit distance to the correct answer) followed by a correct answer. The training data therefore contains **no examples of what to do when the current answer is already correct**—the model never learns to recognize "this answer is fine, don't change it." **The consequence.** At test time, when the revision chain accidentally produces a correct answer, the model has a strong tendency to "revise" it into an incorrect answer in the subsequent step. The paper reports: > "approximately 38% of correct answers get converted back to incorrect ones using a naive approach" (Section 6.1) This means that longer revision chains are not monotonically improving—the model can oscillate between correct and incorrect answers, and simply taking the last revision in a chain is unreliable. The paper mitigates this by using within-chain selection (majority voting or verifier-based selection) to pick the best answer from *any* point in the chain, but this is a post-hoc patch that does not address the underlying training problem. The selection mechanism introduces its own failure modes: if the verifier itself makes errors, it might select an incorrect answer from the chain even when a correct answer exists elsewhere. Additionally, the 38% reversion rate means that a substantial fraction of the sequential revisions' computational budget is spent *undoing* progress, reducing the effective efficiency of sequential sampling. Beyond this specific failure, the revision training proves **fragile**. The ReST^EM experiment (Appendix K, Figure 16) shows that attempting to further optimize the revision model with RL-style training **causes performance to degrade substantially** with sequential revisions—at 256 generations, fully sequential performance drops to approximately 33.5% compared to roughly 38.5% at the optimal ratio. The paper hypothesizes that "on-policy data collection in ReST^EM exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly." This sensitivity to training methodology means the revision approach's positive results depend on specific choices (offline data construction with edit-distance-based pairing, training only on incorrect-to-correct trajectories, stopping before validation loss signals overfitting) that may not transfer to other settings or survive further optimization attempts. **What evidence exists in the paper.** The 38% reversion rate is reported in Section 6.1. The ReST^EM failure is documented in Appendix K (Figure 16) and explicitly discussed as a negative result. The paper's mitigation (within-chain selection via majority voting or verifier) is described in Section 6.1 and its effectiveness can be inferred from the sequential revision results in Figure 6 and Figure 7, which show that sequential revisions *do* outperform parallel sampling in aggregate despite the reversion problem. However, the paper does not analyze how the 38% reversion rate varies with difficulty or revision chain length, nor does it compare the revision chain's performance with ideal "stop when correct" oracle selection to quantify the room for improvement. **Mitigation status.** The within-chain selection mechanism is a partial mitigation—it prevents the final-chain answer from being worse than the best chain answer, but it does not prevent the model from wasting compute on incorrect revisions of already-correct answers. The paper does not propose or test a more principled solution, such as training the revision model on trajectories that include correct answers (teaching it when *not* to revise) or adding an explicit "stop revising" token that the model can learn to emit. The ReST^EM failure is reported as a cautionary finding without a proposed fix. The fragility of revision training is an unresolved limitation that may constrain the approach's applicability beyond the specific training recipe used in this paper. --- ### All Results Are on a Single Benchmark (MATH) with a Single Model Family (PaLM 2-S*), and the Test Set Is Small **The assumption or constraint.** Every experiment in the paper uses the MATH benchmark (Hendrycks et al., 2021)—specifically the 500-question test split from Lightman et al. (2022)—and a single base model family (PaLM 2-S*). The paper does not evaluate on any other reasoning benchmark (e.g., GSM8K, MMLU, HumanEval, ARC, or commonsense reasoning datasets), does not test with any other model family (e.g., LLaMA, GPT, Gemini, or open-source alternatives of different scales), and does not evaluate on tasks beyond symbolic math reasoning. The authors state (Section 4) that they "believe this model is representative of the capabilities of many contemporary LLMs" and that MATH is chosen because test-time compute is "expected to help most when the model already possesses the necessary knowledge and the challenge is drawing complex inferences." However, these are assertions, not empirical demonstrations of generalizability. **The consequence.** Several key findings may be MATH-specific or PaLM-specific rather than general properties of test-time compute scaling. The difficulty-dependent behavior of search (beam search hurting easy problems, helping medium problems; Figure 3 right) depends on the PRM's over-optimization characteristics, which in turn depend on the PaLM 2-S* output distribution and the MATH problem structure. A different model with different calibration properties or a different benchmark with different error patterns might exhibit qualitatively different scaling curves—for example, a model with better-calibrated confidence might show less PRM over-optimization, or a benchmark with different difficulty structure might shift the bin boundaries. The revision model's ability to learn from incorrect in-context examples depends on the base model's in-context learning capabilities, which vary substantially across model families—PaLM 2-S* might have stronger or weaker in-context learning than alternatives, and the revision results might not transfer. Additionally, the test set is **500 questions**, split into five difficulty quintiles of ~100 each, then further split by two-fold cross-validation into ~50 questions per fold per bin. The compute-optimal policy is thus selected based on approximately 50 questions per bin, which is a very small sample for strategy selection. The paper does not report confidence intervals or statistical tests on the compute-optimal scaling curves (Figures 4 and 8), making it difficult to assess whether the observed differences between strategies are statistically reliable or could be sampling noise. A single anomalous question in a bin could shift the optimal strategy for that entire bin, and with only ~50 questions per bin, the strategy selection may not be robust. The paper does not discuss sensitivity analysis—how stable are the compute-optimal strategy choices under bootstrapped resampling of the test set? **What evidence exists in the paper.** The limitation is apparent from the experimental setup descriptions (Section 4, Section 5.3). The authors are transparent about using only PaLM 2-S* and MATH but do not explicitly list this as a limitation—Section 8 ("Limitations and Future Work") mentions extending to "other model families and tasks" as future work but does not discuss how the current single-benchmark, single-model design limits the strength of the claims. The small test set and cross-validation sample sizes (~50 per fold per bin) are not discussed as a limitation anywhere in the paper. **Mitigation status.** Not addressed. The paper suggests future work on "extending the analysis to other model families and tasks" (Section 8) but provides no evidence that the findings generalize. The cross-validation protocol mitigates some risk of overfitting the compute-optimal policy to the test set, but it cannot compensate for the fundamental limitation that all findings are conditioned on one model, one benchmark, and one task type. Until the core results—particularly the difficulty-dependent scaling patterns and the 4× compute-optimal efficiency gain—are replicated on different models and benchmarks, the paper's claims should be treated as specific to PaLM 2-S* on MATH rather than as universal scaling laws. --- ### The 14× Larger Model Baseline Is Not Compute-Optimally Trained and Uses Greedy Decoding with No Test-Time Compute **The assumption or constraint.** The FLOPs-matched comparison in Section 7 compares PaLM 2-S* (with compute-optimal test-time scaling) against a model with approximately 14× more parameters but the same training data. The paper explicitly acknowledges this departs from compute-optimal pretraining (Hoffmann et al., 2022): > "We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work." (Section 7) Additionally, the 14× larger model is evaluated using **greedy decoding only**—no majority voting, no best-of-N, no PRM-guided search, no revisions. It receives zero test-time compute augmentation. **The consequence.** The comparison is systematically biased in favor of test-time compute in two ways. First, a compute-optimally trained larger model (scaling both parameters and data equally, as per the Chinchilla scaling law) would likely outperform a parameter-only-scaled model at the same total FLOPs budget. Second, giving the larger model even a modest test-time compute budget (e.g., best-of-8 majority voting) would create a much stronger baseline. The paper's reported advantages of test-time compute over pretraining—for example, +27.8% relative improvement on easy questions at R << 1 with revisions—are measured against a baseline that is **weaker than it needs to be**. Against a properly compute-optimal larger model with some inference-time augmentation, the test-time compute advantage would narrow or potentially reverse, especially on medium-difficulty problems where the larger model's improved pretraining would increase its base pass@1 and make test-time compute more effective for it as well. The paper frames the FLOPs-matched comparison as answering the question "is it better to train a larger model or spend the extra FLOPs on inference-time computation?" but the comparison setup effectively asks "is it better to scale inference compute on a smaller model, or to scale pretraining compute in a suboptimal way on a larger model that receives no inference augmentation?" These are not the same question. A fairer comparison would give both models access to test-time compute, with the total FLOPs budget split between pretraining and inference for each model, and with the larger model trained in a compute-optimal fashion (scaling data proportionally with parameters). **What evidence exists in the paper.** The paper is transparent about using the LLaMA-style scaling (parameters only) rather than Chinchilla-style scaling (parameters + data). The greedy decoding baseline is implicit—Section 7 does not explicitly state that the 14× larger model uses greedy decoding, but the experimental setup description says the comparison is between "PaLM 2-S* with compute-optimal test-time scaling and a model with approximately 14× more parameters," with no mention of test-time augmentation for the larger model. The use of greedy decoding can be inferred from the stars in Figure 9 being placed at a single accuracy value for each difficulty bin, rather than as scaling curves (which would be expected if the larger model also received variable test-time compute). **Mitigation status.** The paper acknowledges the parameter-only scaling departure from compute-optimal pretraining and frames the comparison as representative of a "canonical approach" (the LLaMA paradigm) rather than as the optimal pretraining strategy. Future work on "compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally" is suggested. However, the paper does not acknowledge the asymmetry of giving test-time compute to the smaller model but not the larger one, nor does it discuss how the results might change under fairer comparison conditions. The headline finding that "test-time compute can substitute for a 14× larger model" is therefore qualified by a comparison setup that advantages test-time compute, and practitioners should interpret the FLOPs-matched results as an **upper bound** on the substitution ratio rather than as a precise equivalence. --- ### Latency and Wall-Clock Time Are Ignored Despite Sequential Strategies Being Inherently Serial **The assumption or constraint.** The paper measures test-time compute in "generations" (number of complete solutions sampled), which is a reasonable proxy for total FLOPs but ignores **latency**—the wall-clock time required to produce an answer. The compute-optimal policy often favors strategies with substantial sequential components: sequential revisions for easy problems (Section 6, Figure 7 right) and beam search for medium problems (Section 5.3, Figure 3 right). These strategies are inherently serial operations—each revision depends on the output of the previous revision, and each step of beam search depends on the pruning decisions from the previous step. **The consequence.** A strategy that allocates 128 generations as 64 sequential × 2 parallel takes approximately **64× longer in wall-clock time** than one that runs 128 parallel samples simultaneously, assuming sufficient hardware to parallelize all samples. For latency-sensitive applications—interactive assistants, real-time decision-making, chat interfaces where users wait for responses—the sequential-heavy strategies that the compute-optimal policy favors on easy problems may be **practically unusable** regardless of their FLOPs efficiency. A user who waits 64 seconds (64 sequential generations at ~1 second each) for an easy math problem answer may prefer a strategy that completes in 2 seconds with slightly lower accuracy. The paper's compute-optimal policy optimizes for accuracy given a generation budget but does not consider a **latency budget**. In practice, many deployments have both a total compute budget (operating cost) and a latency budget (user experience), and the optimal strategy under joint constraints may differ substantially from the accuracy-only-optimal strategy. Easy problems, where the compute-optimal policy currently recommends sequential revisions, might actually be best served by lower-latency parallel sampling if latency matters more than the marginal accuracy improvement from sequential refinement. **What evidence exists in the paper.** The paper does not report any latency measurements, inference time per generation, throughput numbers, or wall-clock time analyses. The term "latency" does not appear in the paper. The distinction between sequential and parallel sampling is discussed solely in terms of generation count (cost) and accuracy, never in terms of time. This is a **missing dimension** of the analysis—the compute-optimal framework optimizes over a single resource (generations) when real deployments must optimize over at least two (cost and latency). **Mitigation status.** Not addressed. The paper does not acknowledge latency as a constraint, does not propose a latency-aware allocation policy, and does not discuss the practical tradeoff between sequential and parallel strategies in terms of wall-clock time. This is not a flaw in the paper's scientific analysis—the scaling analysis in terms of generations is internally consistent and useful—but it is a **practical limitation** that a deployer would need to address before adopting the compute-optimal policy in a latency-constrained setting. A natural extension would be to add a latency constraint to the compute-optimal objective (Equation 1) and re-derive the optimal allocation policies under joint cost-latency budgets, but the paper provides no foundation for doing so. ## 7. Implications and Future Directions ### How This Work Changes the Landscape DreamActor-M2 represents a **paradigm shift in how motion conditioning is conceptualized** for character animation, moving the field from "motion as a foreign signal requiring specialized encoding" to "motion as visual context to be interpreted by the backbone's native capabilities." This is not an incremental improvement to an existing injection mechanism—it challenges the foundational assumption, shared by virtually all prior work from AnimateAnyone to Wan2.2-Animate, that motion information is categorically different from the visual information diffusion models process and therefore requires dedicated encoders, compression modules, or injection pathways. By demonstrating that a simple spatial concatenation of reference and motion frames—processed through the standard VAE and DiT architecture without modification—outperforms specialized injection mechanisms across all evaluation dimensions on AW Bench (Table 1: End-to-End DreamActor-M2 achieves 4.35 automatic Appearance Consistency vs. 4.06 for DreamActor-M1's pose-aligned injection and 3.53 for MTVCrafter's cross-attention injection), the paper provides strong evidence that the field has been optimizing within a suboptimal paradigm. The magnitude of this shift is **foundational for the animation subfield but not yet proven to generalize beyond video diffusion backbones with strong spatiotemporal priors.** The in-context learning framing draws on principles established in LLMs and VLMs—where presenting examples as input tokens eliminates the need for task-specific encoders—but applies them to the distinct domain of video generation, where "context" spans both spatial and temporal dimensions. Whether the same principle transfers to other conditional generation tasks (view synthesis, video prediction, controllable editing) remains an open empirical question, but the reframing makes those questions newly tractable: instead of building task-specific conditioning modules, researchers can now ask whether their backbone model can simply "read" the conditioning signal as part of its input representation. **Resolving the "see-saw" contradiction.** One of the paper's most significant conceptual contributions is demonstrating that the trade-off between identity preservation and motion consistency—which prior work largely accepted as an inevitable design tension—is not inherent to the animation problem but rather an artifact of co-locating identity and motion information in the same spatial region. The ablation against temporal-only ICL (Table 2: Motion Consistency drops from 4.18 to 3.98 when spatial pairing is removed) provides direct evidence that the see-saw arises from *how* motion information is presented, not from some fundamental limitation of diffusion models for animation. This changes the research question from "how do we balance identity and motion?" to "how do we structure the input so that identity and motion are never in conflict?"—a more tractable and principled framing. **Redirecting research investment.** The paper's findings suggest that several active research directions should be deprioritized in favor of others. Pose encoder design—an area of substantial investment in prior work (AnimateAnyone's pose guider, UniAnimate's pose encoder, MTVCrafter's auxiliary pose network)—appears less critical when motion can be conveyed through spatiotemporal context. The End-to-End variant's slightly superior performance to the Pose-based variant (Table 1: +0.07 on automatic Appearance Consistency) suggests that raw RGB frames, when presented as visual context, provide richer motion supervision than even clean skeleton sequences. Similarly, sophisticated injection mechanisms (cross-attention modules, adaptive fusion layers, gated injection) may be unnecessary overhead if the backbone can learn to extract motion from its native input space. Investment should flow instead toward: (1) understanding how different spatiotemporal context structures affect what the model learns about motion, (2) developing self-bootstrapped data synthesis pipelines for domains where paired cross-identity data is unavailable, and (3) improving the quality and diversity of synthetic training data, which the paper identifies as the primary bottleneck for end-to-end generalization (Section 6: struggles with complex interactions attributed to training data scarcity). **A new evaluation philosophy.** AW Bench introduces an evaluation paradigm that measures perceptual quality dimensions (imaging quality, motion smoothness, temporal consistency, appearance consistency) rather than reconstruction fidelity to ground-truth videos. This is not merely a new dataset—it is a statement about what the field should optimize for. Prior metrics like FID-FVD and CD-FVD are fundamentally incompatible with cross-identity animation (where no ground truth exists), and their demonstrated inconsistency with human judgment (referenced in Section 5.1) means they provide misleading signals for method development. AW Bench's human-aligned automated evaluation via Video-Bench enables fair comparison across fundamentally different approaches—pose-based vs. end-to-end, human-only vs. cross-domain—that cannot be compared on reconstruction error. If adopted by the community, this shifts optimization away from distribution matching toward perceptual quality, which aligns method development with deployment objectives. The GSB comparison against platform-level products (Figure 5: DreamActor-M2 leads Kling 2.6 by +9.66%) demonstrates that this evaluation philosophy produces rankings consistent with subjective quality at industrial scale. ### Follow-Up Research This Work Enables **Scaling the self-bootstrapped pipeline to full inter-domain coverage.** The paper generates 60,000 high-quality training triplets using a two-stage filtering protocol (Video-Bench score above 4.5 plus manual verification). However, the distribution of character types in this synthetic dataset is not characterized—what fraction involves animals, cartoons, objects, or multi-subject scenarios? A critical follow-up would systematically measure End-to-End DreamActor-M2's per-category performance on AW Bench (the paper reports only aggregate metrics), identify which character categories or motion types show degraded quality, and then target additional synthetic data generation specifically for those categories. The experiment: generate synthetic data for underperforming categories at scale (e.g., 20,000 additional animal-to-animal and cartoon-to-human pairs), retrain, and measure whether the performance gap closes. This would quantify the relationship between synthetic data diversity and generalization—a fundamental question for any self-bootstrapped training paradigm. The hypothesis is that AW Bench's worst-case performance (currently masked by aggregate metrics) is limited by the Pose-based model's ability to generate high-fidelity animations for specific character types, creating "data deserts" that the End-to-End model cannot overcome. **Characterizing and mitigating the revision model's correct-to-incorrect reversion rate across difficulty levels and chain lengths.** The paper identifies a 38% reversion rate as a significant practical issue, where correct answers get "revised" into incorrect ones during sequential refinement. A systematic analysis should measure how this reversion rate varies with problem difficulty (do easy problems show higher or lower reversion probability?), chain length (does reversion probability increase monotonically with revision depth?), and the verifier's confidence on correct vs. incorrect answers (does the PRM score drop predict reversion?). The experiment: run the revision model on AW Bench with chain lengths from 2 to 64 steps, track per-step correctness and PRM score trajectories, and compute the probability of correct-to-incorrect transition as a function of current correctness, current PRM score, and chain position. This would reveal whether reversion is a systematic bias (the model always tends to modify correct answers) or a difficulty-dependent phenomenon (only certain types of correct answers are unstable). The mitigation could then be targeted: if reversion correlates with low PRM confidence, the verifier selection mechanism can be tuned to overweight high-confidence correct answers; if reversion is most common on medium-difficulty problems, chain lengths can be capped adaptively. **Stress-testing the in-context formulation under distribution shifts in spatial layout, motion complexity, and character morphology.** The paper demonstrates strong results on AW Bench, but the spatiotemporal composite construction makes specific assumptions about spatial layout—the reference occupies the left half, the motion signal occupies the right half, and the regions are perfectly aligned temporally. What happens when these assumptions break? A diagnostically valuable experiment: systematically vary the spatial arrangement (reference on right, motion on left; reference on top, motion on bottom; reference at reduced resolution; reference at a different aspect ratio) and measure whether the model's performance degrades gracefully or catastrophically. This tests whether the model has learned a general "motion extraction from visual context" capability or a brittle spatial heuristic. Similarly, test on driving videos with motion complexity beyond the training distribution—extremely rapid movements, unusual viewpoints, non-rigid deformations, partial occlusions—and measure whether the End-to-End variant maintains its advantage over the Pose-based variant (where pose estimation would fail) or whether both degrade similarly. A finding that the End-to-End model maintains performance on occluded or rapid motion where pose estimation fails would strengthen the case for eliminating pose estimators; a finding that both variants degrade similarly would suggest the current training data lacks sufficient motion diversity regardless of conditioning modality. **Measuring DreamActor-M2's inference compute scaling behavior.** The paper uses a single forward pass per animation (all 50,000 denoising steps) and does not explore whether additional test-time compute improves quality. A natural experiment: vary the number of denoising steps at inference (e.g., 10, 25, 50, 100, 200) and measure AW Bench scores on all four evaluation dimensions. This would answer whether the current denoising budget is saturated (quality plateaus before 50 steps) or whether additional inference compute continues to improve output quality. Additionally, test whether simple test-time augmentation strategies—generating multiple candidate animations from the same inputs with different noise seeds and selecting the best via a quality estimator—improve results, and whether the improvement is uniform across difficulty or concentrated on challenging cases. This connects DreamActor-M2 to the broader literature on test-time compute scaling and provides practical guidance for deployment: if additional sampling provides diminishing returns after 4–8 candidates, production systems can cap generation budgets accordingly. **Training a difficulty estimator from question text without sampling.** The paper's difficulty estimation currently requires generating and scoring 2048 samples per question, making it too expensive for deployment. A direct follow-up: fine-tune a small language model (or the base model itself) to predict the PRM's average final-answer score from only the question text. Training data: the 500 test questions with their computed difficulty scores (average PRM score over 2048 samples). Evaluation: correlation between predicted and true difficulty scores on held-out questions, and more importantly, whether the compute-optimal policy using predicted difficulty matches the performance of the oracle-binned policy. This experiment is newly tractable because the paper has already computed difficulty scores for all 500 test questions and demonstrated that the PRM's average score is a sufficient proxy for oracle difficulty (Figures 4 and 8 show predicted bins tracking oracle bins closely). If a lightweight text-based difficulty estimator achieves, say, 80% bin classification accuracy, the compute-optimal framework becomes deployable without the 2048-sample overhead. The key measurement is not the difficulty estimation accuracy per se, but the downstream impact on the compute-optimal policy's performance—does the policy using cheap difficulty estimates achieve the same 4× efficiency gain over best-of-N? ### Practical Applications and Downstream Use Cases **Content creation platforms for cross-domain character animation.** The most direct application is integrating DreamActor-M2's End-to-End variant into video editing and content creation tools that need to animate diverse character types from arbitrary motion sources. The key value proposition is eliminating the per-domain toolchain fragmentation that currently exists: a creator who wants to animate a photorealistic human dancer, a hand-drawn cartoon character, and a 3D-rendered animal must currently use separate tools or models for each. DreamActor-M2's AW Bench results—achieving 4.35 automatic Appearance Consistency and 4.69 Temporal Consistency across humans, animals, and cartoons in a single model—suggest that a unified animation interface is feasible. The practical deployment scenario: a creator uploads a reference image of any character type and a driving video of any subject performing any motion, and receives a high-fidelity animation without specifying character type, selecting a domain-specific model, or providing pose estimation. The GSB comparison (Figure 5: +9.66% over Kling 2.6) suggests the quality is competitive with commercial products that may use larger models or more extensive training. **Training data generation for downstream vision tasks.** DreamActor-M2's self-bootstrapped pipeline produces cross-identity animation pairs—videos of different characters performing identical motions—which are valuable supervision for tasks beyond animation. Specifically: (1) training motion transfer models for other domains (e.g., robotic manipulation, where a robot arm's motion is transferred from human demonstration videos); (2) data augmentation for action recognition, where synthetic animations of diverse characters performing the same action add viewpoint and appearance diversity to training sets; (3) training video correspondence models, where the synthetic pairs provide ground-truth dense correspondence between different visual appearances performing the same motion. The 60,000 high-quality triplets generated for this paper represent a starting point; scaling the pipeline to produce millions of pairs spanning hundreds of character types and thousands of motion categories could create a foundational dataset for video understanding tasks. The filtering protocol (Video-Bench score > 4.5 + manual verification) provides a quality standard that downstream tasks can adopt. **Real-time avatar animation for telepresence and gaming.** In scenarios where a user's video feed drives an animated avatar in real time—telepresence, virtual meetings, game streaming, social VR—DreamActor-M2's End-to-End variant offers a key advantage over pose-based systems: it does not require pose estimation, which is a significant latency and error source. A 2D pose estimator running at 30 fps adds computational cost and fails under the partial occlusions, unusual viewpoints, and rapid movements common in unconstrained user video. The End-to-End model, which learns motion extraction directly from RGB, could be optimized for real-time inference (reduced denoising steps, distilled backbone, smaller resolution) while maintaining the cross-domain generalization that allows a single model to drive avatars ranging from photorealistic humans to cartoon characters. The practical deployment would need to address the inference latency dimension that the paper does not measure—the current model generates full videos in a single denoising process, and achieving frame-by-frame real-time generation would require architectural adaptations (streaming diffusion, frame-wise conditioning) that the paper's framework does not yet support, but the underlying motion understanding capability demonstrated on AW Bench provides the foundation. ### When to Prefer This Method - **Prefer DreamActor-M2's End-to-End variant when** the deployment requires animating diverse character types (humans, animals, cartoons, objects) from a single model, when driving videos come from unconstrained sources where pose estimation is unreliable or impossible (in-the-wild videos, non-humanoid subjects, occluded or rapid motion), or when eliminating the computational and latency cost of a pose estimator is a priority. The End-to-End variant's AW Bench performance (Table 1: 4.35 automatic Appearance Consistency, exceeding the Pose-based variant's 4.28) and its demonstrated advantage in challenging pose estimation scenarios (Figure 6d: direction ambiguity, hand overlapping) support this preference. - **Prefer DreamActor-M2's Pose-based variant when** the deployment requires maximum control over the animation through explicit pose inputs (allowing manual editing of skeleton sequences before animation), when the driving video domain is exclusively human-centric with reliable pose estimation (studio-recorded video, motion capture data), or when the computational cost of the self-bootstrapped data synthesis pipeline for End-to-End training is prohibitive. The Pose-based variant's text guidance mechanism (MLLM-generated fused prompts) provides an additional semantic control channel that may be valuable when specific motion semantics must be precisely specified. - **Prefer alternative approaches when** the animation scenario requires real-time, frame-by-frame generation (where the full-video denoising process is too slow), when the character types are so far outside the training distribution that even the Pose-based model cannot generate adequate synthetic pairs (extremely non-standard morphologies, abstract shapes), or when multi-character interactions involve complex spatial relationships like characters rotating around each other (Section 7 explicitly notes this as a current failure mode). The GSB comparison (Figure 5) shows DreamActor-M2 only narrowly leads Kling 2.6 (+9.66%), suggesting that for applications where Kling's domain matches the deployment distribution, the practical difference may not justify switching frameworks, especially if Kling's API provides infrastructure advantages (scaling, reliability, integration) not available with DreamActor-M2.