SCAIL: Towards Studio-Grade Character Animation via In-Context Learning of 3D-Consistent Pose Representations

ArXiv: 2512.05905

🎯 Pitch

Rendering bones as depth-aware cylinders instead of flat 2D keypoints allows a diffusion transformer to animate a casual photo into a complex backflip or a multi-character dance without the character’s limbs morphing or swapping identities. By feeding the model the entire motion sequence at once rather than frame-by-frame, SCAIL reasons about global temporal context to maintain structural coherence through severe occlusions and cross-body-shape transfers where prior commercial and open-source tools collapse.

---

1. Executive Summary

This paper introduces SCAIL (Studio-grade Character Animation via In-context Learning), a DiT-based character animation framework that synthesizes videos from a reference image and a driving video. It targets two bottlenecks through named mechanisms: a 3D-consistent pose representation that rasterizes bones as spatial cylinders to encode depth and occlusion (versus the ambiguous 2D keypoints or identity-leaking SMPL meshes of prior methods), and full-context pose injection that concatenates the entire pose sequence with noisy video tokens to enable spatiotemporal reasoning (versus per-frame channel concatenation that provides only local motion cues). On the authors' proposed Studio-Bench—a benchmark of 250+ challenging single- and multi-character clips spanning complex motions, cross-identity transfers, and stylized characters—SCAIL's 14B model achieves 53.3% motion accuracy and 55.8% kinesiology consistency in user studies, substantially outperforming prior open-source methods (VACE, Wan-Animate) that fall below 35% on these metrics, while also surpassing a commercial 3D-foundation-model baseline (Viggle) in motion naturalness. The approach demonstrates that a 3D-consistent, identity-agnostic pose signal combined with full-sequence pose attention enables faithful complex-motion transfer across diverse character shapes and multi-person occlusion scenarios, establishing that test-time 3D camera optimization plus training-time bone-scaling augmentation can decouple motion from identity only when the pose representation preserves depth-ordering and the injection mechanism grants the model access to global temporal context.

2. Context and Motivation

The Core Problem: Studio-Grade Motion Transfer Under Complex Conditions

The fundamental question this paper tackles is: how do we transfer motion from a driving video onto a reference character image while preserving both motion fidelity and the character's structural identity, even when the motion is complex, occlusions occur, and the character differs significantly from the driving subject?

This is not merely an academic exercise. For film production, virtual production pipelines, and professional content creation, character animation remains bottlenecked by a multi-stage workflow that requires motion capture hardware, manual rigging, and expert labor. A video-based approach—feed in a reference image of the character and a driving video of someone performing the desired action, and receive a temporally coherent animation of that character performing that motion—promises to collapse this entire pipeline into a single inference step. However, existing methods consistently break down precisely in the scenarios that matter most for production: complex motions (turning, flipping, rolling), multi-person interactions (dancing, hugging, fighting), and cross-identity animation where the reference and driving subjects differ in body shape, proportions, or domain (e.g., a real person driving a stylized 3D character).

The paper frames this as a "studio-grade" requirement: the bar is not basic motion following (which existing methods largely achieve on simple actions), but rather structural fidelity, temporal consistency, and motion accuracy that holds up under the challenging, diverse conditions encountered in real production settings. The gap between what current methods can do and what production demands constitutes the paper's central problem.

Why This Problem Matters: Beyond Incremental Improvement

The significance of this problem operates at multiple levels:

Practical impact on production workflows. The conventional character animation pipeline—motion capture suit → skeletal retargeting → rigging → rendering—is expensive, hardware-dependent, and labor-intensive. A video-driven approach that works reliably under studio-grade conditions could dramatically reduce the cost and expertise barriers to high-quality character animation, making it accessible to smaller studios, independent creators, and real-time applications. The paper explicitly positions itself as targeting "film production" and "production-level readiness," signaling that the goal is not marginal improvement on easy cases but robustness sufficient for professional deployment.

A fundamentally unresolved technical challenge. The gap between what current methods achieve on simple motions and what they fail at under complex conditions reveals that the underlying technical approach—extract a skeletal pose signal from a driving video, inject it into a video generation model—is not itself flawed, but rather that the specific choices of pose representation and pose injection mechanism are insufficient. This elevates the problem from "make the model better" to "identify the root cause of failure and design representations and injection strategies that address it." The paper's framing suggests that current methods have hit a wall that cannot be overcome by simply scaling data or model size; the representations themselves must change.

A benchmark gap that obscures progress. The authors observe that existing evaluation benchmarks primarily test only simple actions, failing to capture the challenges present in production. This means the field lacks a reliable signal for whether methods are actually progressing toward practical utility or merely overfitting to easy cases. The construction of Studio-Bench is therefore not just an evaluation tool but a diagnosis: existing methods that appear competitive on standard benchmarks collapse when evaluated on challenging, production-relevant scenarios.

Where Previous Approaches Fall Short

The paper identifies specific technical failure modes in prior work, organized around two axes: the pose representation used, and the mechanism for injecting pose information into the generation model.

The Pose Representation Problem

Prior methods use either 2D keypoint-based skeletons or SMPL mesh-based representations, each with distinct and complementary failure modes:

2D keypoints (DWPose, ViTPose): Most existing character animation methods—AnimateAnyone, UniAnimate-DiT, MimicMotion, Wan-Animate, VACE—extract 2D skeletal keypoints from the driving video and use these as the motion conditioning signal. The paper identifies three distinct failure mechanisms:

1. Noisy predictions and missing occlusion information. 2D keypoint estimators produce inherently noisy predictions, and critically, they cannot encode depth or occlusion relationships. When a limb is occluded or when limbs cross (common in turning, rolling, or multi-person scenes), 2D keypoints provide an ambiguous or misleading signal: the model has no way to know which limb is in front of which, or whether a missing keypoint indicates occlusion versus estimation failure.

2. 2D-3D mismatch during adaptation. When the reference character differs in body shape from the driving subject (cross-identity animation), prior methods apply 2D skeleton scaling during training or heuristic retargeting at test time. However, scaling a 2D skeleton is fundamentally a deformation operation: it stretches or compresses bone lengths in pixel space, altering the apparent motion dynamics and introducing distortion that is inconsistent with 3D kinematics. The paper explicitly states this:

   "such adaptation inherently suffers from deformation due to inconsistency with 3D motion dynamics, further amplifying estimation noise and inaccuracies"

   This is a subtle but critical point: the adaptation step that prior methods use to handle body-shape discrepancies actually degrades the motion signal itself, creating a fundamental tradeoff between identity matching and motion fidelity.

3. Inability to disambiguate front/back. The paper gives a concrete example: during a turning motion, a 2D skeleton provides the same spatial pattern regardless of whether the person is facing front or back. From the model's perspective, there is no cue to distinguish these cases, leading to "awkward postures" where the model fails to understand which side of the body is visible.

SMPL mesh controls: Methods like Champ and VividPose use SMPL parametric body models to provide richer 3D information. However, the paper identifies a critical downside:

"which offer strong 3D human priors but cause severe identity leakage"

SMPL encodes a strong prior about human body shape, effectively superimposing the driving subject's body proportions onto the generated animation. When the reference character has a different body shape—particularly for stylized or non-humanoid characters (thin-limb anime characters, plush toys, figurines)—this prior interferes with identity preservation, causing the generated character to drift toward the driving subject's proportions. The phrase "identity leakage" captures this precisely: the mesh representation leaks information about the driver's identity into the generated output, compromising the reference character's appearance.

The deeper issue: neither representation is adequate for multi-person scenarios. Multi-person interactions introduce occlusion between characters, requiring the pose representation to encode inter-character spatial relationships. The paper notes that existing 2D methods fail to correctly distinguish limbs belonging to different characters during heavy occlusion, causing "body parts of different characters merge during motion." SMPL methods, being inherently single-person parametric models, offer limited natural extension to multi-character scenarios without custom architecture modifications.

The Pose Injection Problem

Beyond the representation itself, the paper identifies a critical limitation in how pose information is provided to the generation model. The dominant approach is channel-wise concatenation: the pose sequence is spatially aligned and concatenated with the noisy video latents along the channel dimension, then processed by the denoising network. This is used by Wan-Animate, RealisDance-DiT, and the authors' own initial implementation.

The paper identifies a specific and non-obvious failure mode of this approach:

"we find it tend[s] to generate unnatural human pose when the motion is complex despite conditioned on accurate and unambiguous pose"

Even when the pose representation itself is correct, per-frame channel concatenation fails to provide sufficient temporal context for the model to interpret the motion correctly. The paper gives the concrete example of a turning motion: the model "often fails to correctly distinguish between front and back views, generating awkward postures." This is surprising because the pose information is technically present in each frame—but the model cannot access the sequence-level context that disambiguates left/right, front/back, or the direction of motion.

The diagnosis is that channel concatenation provides primarily local motion cues—each frame's generation attends primarily to that frame's pose—rather than enabling the model to reason about global motion dependencies:

"Since motion is inherently sequential, many actions can only be interpreted correctly within a temporal context."

This is a fundamental insight: pose following is not a per-frame alignment problem but a spatiotemporal reasoning problem. A turning motion is defined by the relationship between consecutive frames—the direction of rotation, the progression of occlusion, the continuity of limb trajectories. A per-frame injection scheme fundamentally cannot capture these relationships, regardless of the quality of the pose representation itself.

The Data and Evaluation Gap

The paper identifies two additional structural weaknesses in prior work that compound the technical limitations:

1. Training data lacks complexity and diversity. Standard video datasets may be dominated by static or low-motion clips where characters are not the primary focus, or contain incomplete body views. The authors argue that existing methods have not been trained on data curated specifically for the demands of studio-grade animation, particularly regarding motion complexity, character diversity (including stylized/non-realistic characters), and multi-person scenarios. This is not a data volume problem but a data relevance problem: even large-scale training may be ineffective if the data distribution does not cover the challenging cases that production requires.

2. Evaluation benchmarks do not reflect production requirements. The paper explicitly criticizes prior evaluation:

   "Previous evaluations primarily focus on simple actions, which fail to capture the challenges present in film production, such as complex and dynamic motions, multi-person interactions, and cross-domain animations."

   This means the field has been optimizing toward the wrong target. Methods that appear competitive on standard benchmarks with simple motions (walking, basic gestures) may fail catastrophically on the challenging scenarios that matter for practical deployment. The gap between benchmark performance and real-world utility is particularly acute in this domain because the hardest cases are precisely the ones that production workflows need to handle.

How This Paper Positions Itself

SCAIL positions itself not as an incremental improvement to existing architectures, but as a rethinking of the two core representations that prior methods have taken for granted: the pose signal and the injection mechanism. The paper's framing suggests that these two choices are the root cause of the observed failure modes, and that addressing them with more principled alternatives can achieve a qualitative leap in capability rather than marginal gains from scaling.

On pose representation: 3D-consistent, identity-agnostic. The paper rejects both 2D keypoints (ambiguous, noisy, lose depth) and SMPL meshes (identity-leaking, human-specific) in favor of a cylindrical rasterization approach (rendered from 3D keypoints estimated by NLFPose) that:

• Encodes depth and occlusion explicitly through 3D spatial rendering and ray marching
• Remains identity-agnostic because it discards body shape priors and relies only on bone direction and connectivity
• Supports augmentation (bone scaling during training to simulate diverse body shapes while preserving motion semantics) and retargeting (optimizing a camera projection to align the 3D driving pose with the 2D reference frame) without distorting motion

The paper explicitly frames this as providing "a robust and flexible motion signal" that bridges the gap between the driving video's motion and the generated character's appearance, addressing what they identify as the primary bottleneck: "current skeletal pose representations ... fail to adequately capture 3D structure of driving motions, inter-character spatial and occlusion relationships, and temporal correlations of motion sequences."

On pose injection: full-context, sequence-attending. Rather than per-frame channel concatenation, the paper proposes concatenating the entire spatially-downsampled pose sequence with the noisy video tokens, enabling the DiT's self-attention to operate over the full spatiotemporal pose context. The paper explicitly names this as "in-context learning" (echoing the terminology from language models), positioning the pose sequence not as a per-frame control signal but as a context the model attends to holistically when generating each frame.

The "Pose-Shifted RoPE" mechanism further supports this: by shifting the positional encodings of pose tokens along the width dimension (separating them from reference image and noisy video tokens), the model can distinguish driving signals from content tokens without spatial alignment requirements, allowing the augmented/retargeted pose to be effective even when it no longer pixel-aligns with the video content.

On evaluation: a benchmark for the problem that matters. The paper positions Studio-Bench as filling the gap between standard benchmarks (easy motions) and production requirements (complex actions, multi-person, cross-domain). It is constructed to be adversarial in the sense that it tests precisely the scenarios where prior methods fail, providing a more honest signal of practical capability.

On data: curation over volume. Rather than simply training on more data, the paper develops a filtering pipeline that selects for motion complexity (via 3D keypoint-derived motion speed metrics), character presence (via YOLO detection), and body completeness (via 2D keypoint filtering), with a final manual quality inspection stage. This reflects the paper's thesis that data quality and relevance, not just quantity, determines performance on challenging cases—and that existing large-scale datasets are insufficiently curated for the studio-grade animation task.

3. Technical Approach

3.1 Reader Orientation (Approachable Technical Breakdown)

SCAIL is a video generation system built on a Diffusion Transformer (DiT) that takes a single reference image of a character and a driving video of someone (or multiple people) performing a motion, and produces a temporally coherent video of the reference character performing that exact motion while preserving the character's visual identity. It solves the problem that existing methods break down under complex motions (turning, flipping, multi-person interactions) because their 2D keypoint representations are ambiguous about depth and occlusion, and their per-frame pose injection provides only local motion cues rather than global temporal context—SCAIL addresses both root causes by replacing 2D keypoints with 3D-cylindrical skeletal renderings that encode depth ordering, and by concatenating the entire pose sequence with video tokens so the transformer can attend to the full motion context when generating each frame.

3.2 Big-Picture Architecture (Diagram in Words)

The SCAIL system has five major components, operating in two phases—training-time data preparation and inference-time generation:

1. Data Curation Pipeline — A multi-stage filtering system that ingests raw videos and produces high-quality video-pose pairs. It detects characters (YOLO), filters for body completeness (DWPose), extracts 3D keypoints (NLFPose) with a segmentation-based approach for multi-person videos (SAMURAI tracking + split), renders 3D skeletons as cylindrical segments via a rasterizer, filters out low-motion clips using a 3D keypoint-derived speed metric, and employs both VLM captioning and manual inspection for final quality control. Output: ~250K single-character and ~20K multi-character video-pose pairs, with a 4,000-clip high-dynamic finetuning subset.

2. 3D-Consistent Pose Representation — The core motion signal. 3D body keypoints are estimated from video, connected into a skeleton in 3D space according to anatomical topology, represented as cylindrical segments (bones rendered as spatial cylinders), and rasterized onto a 2D canvas via ray marching. During training, random bone-length scaling (from a pre-synthesized Figure Pool) is applied to make the representation identity-agnostic; during cross-driven inference, a camera projection matrix is optimized via $\mathcal{L}_{\text{proj}}$ to spatially align the 3D driving pose with the 2D reference frame. Output: a rendered pose video where each pixel encodes depth ordering and person identity (via distinct hues).

3. Full-Context Pose Injection Module — The mechanism for feeding the pose sequence into the DiT. Rather than per-frame channel concatenation, the entire spatially-downsampled (2×) pose video is patchified and concatenated token-wise with the noisy video latents, the embedded reference image tokens, and optional text tokens. A Pose-Shifted RoPE scheme shifts the positional encodings of pose tokens along the width dimension by a constant $W_{\text{max}}$, separating them from content tokens so the model can distinguish driving signals regardless of spatial alignment. Output: a unified token sequence that the DiT's self-attention operates over holistically.

4. Denoising DiT Backbone — A Wan2.1-based Image-to-Video (I2V) Diffusion Transformer operating in latent space. It takes the concatenated token sequence (reference image + noisy video latents + full pose context + optional text), applies multi-layer self-attention with the Pose-Shifted RoPE encodings, and iteratively denoises the video latents over $T$ timesteps using classifier-free guidance (CFG scale = 4). The reference image CLIP feature is injected via the original I2V pattern, and an I2V mask (all-one for reference/pose, all-zero for noisy video) distinguishes conditional tokens. Output: clean latent video frames decoded by the VAE into pixel-space animation.

5. 3D-Consistent Adaptation Subsystem — A two-stage mechanism that decouples motion from identity. During training, bone-scaling augmentation (rate 0.8) synthesizes diverse body proportions while preserving motion semantics; camera parameter perturbation adds robustness to the inference alignment step. During cross-driven inference, the camera projection matrix is optimized to minimize the $\mathcal{L}_{\text{proj}}$ between projected 3D keypoints and estimated 2D reference keypoints, retargeting the motion without distorting bone lengths or motion dynamics. Output: a retargeted 3D video sequence that anchors the motion to the reference character's spatial position.

Information flows: raw videos enter the data curation pipeline → filtered videos undergo 3D keypoint extraction and cylindrical rasterization → the rendered pose video is fed through the pose injection module alongside the reference image and (optionally) text → the token sequence enters the DiT backbone with Pose-Shifted RoPE → denoised latents are decoded into pixel-space animations.

3.3 Roadmap for the Deep Dive

• First, the data curation pipeline — how training data is constructed, because the paper's thesis is that data quality (motion complexity, character diversity, multi-person coverage) is as important as model architecture, and the curation choices directly enable the challenging scenarios in Studio-Bench.

• Second, the 3D-consistent pose representation — how skeletons are extracted, rendered, augmented, and retargeted, because this is the foundational motion signal that replaces the ambiguous 2D keypoints and identity-leaking SMPL meshes of prior work, and understanding it requires knowing what NLFPose produces, how cylinders are rasterized, and how scaling/camera optimization work.

• Third, the full-context pose injection and Pose-Shifted RoPE — how the DiT receives and attends to the pose sequence, because the paper's central architectural claim is that per-frame channel concatenation fails to provide global motion context, and the concatenation-plus-shifted-RoPE design is what enables spatiotemporal reasoning over full motion sequences.

• Fourth, the 3D-consistent adaptation subsystem — the augumentation and retargeting details, because these are what make the representation identity-agnostic and enable cross-identity animation without motion distortion, which is one of the paper's two key innovations (alongside full-context injection).

• Fifth, the training procedure and model variants — the concrete hyperparameters, schedule, and architecture choices for the 1.3B and 14B models, because replication and practical deployment require these specifics.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and methods paper whose core idea is that achieving studio-grade character animation requires simultaneously solving two representation problems: (1) the pose signal must encode 3D structure, depth ordering, and occlusion while remaining independent of any specific character's body shape, and (2) the injection mechanism must grant the model access to the full temporal context of the pose sequence so that motion semantics (front/back, rotation direction, limb ownership in multi-person scenes) can be disambiguated. The solution is a cylindrical 3D skeletal rendering pipeline with training-time bone-scaling augmentation and inference-time camera optimization, combined with a DiT architecture where the entire pose sequence is tokenized and concatenated with video latents, and positional encodings are shifted to separate pose context from content tokens.

---

Data Curation Pipeline

The paper constructs its training data through a multi-stage automated filtering pipeline followed by manual quality inspection, driven by the thesis that existing large-scale video datasets are insufficient for studio-grade animation because they contain too many static, low-motion, and non-character-focused clips.

Stage 1: Character detection and filtering. Each raw video clip is first processed by YOLO [33] to detect human presence. Clips where no character is detected, or where characters are too small relative to the frame (indicating the person is not the primary subject), are discarded. For multi-person videos, the number of bounding boxes from YOLO determines whether a clip is classified as single-character or multi-character; clips with too many characters are discarded to maintain clean training signals.

Stage 2: Body completeness filtering. Remaining clips undergo 2D keypoint extraction using DWPose [46]. Clips are retained only if the detected pose is visually complete—covering either the upper body or the full body—and clips with portrait-only framing or incomplete body views are filtered out. This ensures that the model trains on full-body motion signals, not partial occlusions that would provide ambiguous pose conditioning.

Stage 3: Multi-person segmentation and extraction. For multi-person videos, the paper identifies a critical failure mode in standard pose extraction pipelines: "the default extraction pipeline often fails to detect correct limbs under occlusion, especially in multi-human interactions." To address this, the authors employ SAMURAI [45], a zero-shot visual tracker built on the Segment Anything Model, to track and segment the mask of each individual character. The multi-person video is split into multiple single-human video splits, each containing one main character. This segmentation-and-extraction approach is crucial because it allows the subsequent 3D keypoint estimator (NLFPose) to operate on single-person inputs, which is substantially more reliable than direct multi-person pose estimation methods like PromptHMR [39], particularly "in wild cases involving complex interaction." After extracting individual 3D keypoints for each character, the skeletons are composed together in 3D space and rasterized jointly, preserving inter-person occlusion relationships.

Stage 4: 3D keypoint extraction and motion filtering. NLFPose [26] estimates 3D body keypoints $\mathcal{J} = \{J_{i,j}\}_{i=1, j=1}^{T, N}$ for each frame $i$ and joint $j$, where $T$ is the number of frames and $N$ is the number of joints. The paper computes a motion speed metric from these 3D keypoints to filter out static or low-motion clips:

$v = \frac{1}{T} \sum_{t=1}^{T-1} \sum_{j} \left\| \bar{J}_{t+1,j} - \bar{J}_{t,j} \right\|_2$

where $\bar{J}_t = J_t - J_t^{\text{root}}$ denotes the 3D human joint positions relative to the body center at frame $t$, and $j$ iterates over joints in the screen.

What it computes: the average Euclidean distance traveled by all screen-visible joints between consecutive frames, normalized by the number of frames. The root-relative normalization subtracts the body center position (typically the pelvis or hip midpoint), so that global body translation (walking across the frame) does not inflate the metric—only relative limb motion contributes.

Why this form: using 3D keypoints rather than 2D pixel displacements means the metric is invariant to camera distance and character scale, making it comparable across videos with different framing. The root-relative normalization ensures the metric captures pose change (actual motion of limbs relative to the body) rather than scene translation, which is the relevant signal for filtering: a video of someone walking at a constant pace would have low pose-change even if they move across the frame, while a video of someone dancing in place would have high pose-change. Clips with motion speed below a threshold are discarded, "compelling the model to learn from complex motion dependencies."

Stage 5: VLM captioning and quality verification. A large language model [7] (ChatGLM) analyzes video captions to complement the rule-based filtering, specifically checking for multi-person motion scenarios where bounding box logic alone might be insufficient. This serves as a semantic-level filter: captions describing complex interactions (dancing, fighting, hugging) can help identify clips that the geometric filters might have retained but where the interaction quality is insufficient.

Stage 6: Finetuning subset curation. From the filtered dataset (~250K high-quality motion-rich video-pose pairs, including ~20K multi-person pairs), the authors select the 12K samples with the highest motion speed from dance and general motion categories. A final manual clarity inspection reduces this to approximately 4,000 high-dynamic videos with minimal blur. This finetuning subset explicitly targets the most challenging motion scenarios and is used for the second training stage of the 14B model.

Data sources. The training dataset draws from three sources: (1) samples retrieved from the authors' internal base model training data and other downstream tasks, (2) a large collection of high-resolution dance videos from Bilibili and YouTube, and (3) additional sports videos including gymnastics and figure skating. To ensure character diversity, the authors "maintain a certain proportion of stylized content, including 3D and 2D animations ... as well as MMD and Live2D animations." This deliberate inclusion of non-photorealistic characters is important because it means the training distribution covers the cross-domain animation scenarios (real person driving a stylized character) that Studio-Bench evaluates.

Design rationale. The curation pipeline embodies a specific philosophy: filter for data relevance (motion complexity, body completeness, character focus, multi-person coverage) rather than just data volume. The paper argues that existing methods' failures on complex motions are partly attributable to training on data that does not represent the challenging cases production demands, so curation is as important as architecture.

---

3D-Consistent Pose Representation

The pose representation is the foundational motion signal that replaces both the 2D keypoints and SMPL meshes of prior work. The paper's central claim is that a representation must simultaneously satisfy three properties: encode depth and occlusion (which 2D keypoints cannot), remain identity-agnostic (which SMPL meshes, with their human body shape priors, cannot), and support augmentation and retargeting without distorting motion semantics (which 2D scaling cannot).

Step 1: 3D keypoint estimation. Starting from the driving video, NLFPose [26] estimates 3D body keypoints $\mathcal{J} = \{J_{i,j}\}_{i=1, j=1}^{T, N}$ for each frame $i$ and joint $j$. NLFPose is a neural-localizer-field-based estimator that predicts continuous 3D joint positions. The paper notes that NLFPose provides "accurate depth estimation," which is critical for inter-person occlusion handling. The estimated 3D joints are connected according to the standard human skeletal topology in 3D space, forming a 3D skeleton for each person in each frame. For multi-person videos, this extraction is performed separately on each SAMURAI-segmented single-person split, and the resulting skeletons are composed together in 3D space before rasterization.

Step 2: Cylindrical rasterization. The connected 3D skeleton is rendered as a set of spatial cylinders—each bone (connection between two joints) becomes a 3D cylinder of radius $r$ extending from joint $m$ to joint $n$ in the bone set $\mathcal{B}$. Formally, for $P$ people over $T$ frames:

$\mathcal{C} = \left\{ \text{Cylinder}(J_{i,p,m}, J_{i,p,n}, r) \mid 1 \leq i \leq T, 1 \leq p \leq P, (m,n) \in \mathcal{B} \right\}$

What it represents: a geometric description of the 3D skeleton as a collection of volumetric primitives (cylinders), each defined by two endpoint positions in 3D and a fixed radius. The set $\mathcal{C}$ is the complete 3D representation of all skeletons across all frames.

Why cylinders rather than lines: rendering bones as volumetric cylinders rather than thin lines means that when rasterized, closer (unoccluded) bones will properly occlude farther ones through the depth buffer, encoding spatial occlusion relationships directly in the rendered image. A line-based rendering would provide only 2D projection but would lose the depth ordering information that cylinders naturally encode through their 3D volume.

These cylinders are converted into spatial voxels and rasterized onto the 2D image plane using ray marching implemented via Taichi [16], a high-performance GPU computing library. The paper reports this rendering strategy "is highly optimized for modern GPUs, introducing negligible computational overhead." The output is a rendered frame sequence where each pixel encodes which bone is visible (if any) and at what depth, providing an explicit depth-ordering signal.

Why ray marching: ray marching processes each pixel independently by marching a ray through the 3D volume and computing intersections with the cylindrical voxels, naturally handling occlusion through depth comparisons. This is more expensive than simple line drawing but produces proper occlusion relationships—a cylinder in front correctly hides a cylinder behind it—which 2D projection of points (as in keypoint-based methods) fundamentally cannot achieve.

Person identification via hue. To enable the model to distinguish between characters in multi-person scenes, each person is assigned a distinct color scheme when rendering their skeleton. This provides an explicit identity cue: "This enables the model to directly learn how to distinguish characters from the representation. We observe that this design effectively alleviates identity switching when the positions of the people interchange." This is simpler and more flexible than hardcoded architectural designs (such as separate network branches for each person, as in DanceTogether [3]), because it encodes identity information in the representation itself, allowing the model to learn person-specific attention patterns through standard training rather than specialized modules.

Step 3: 3D-consistent adaptation and augmentation. The core innovation of the representation is not just the 3D rendering, but how it enables augmentation (training) and retargeting (inference) that preserve motion semantics while decoupling identity.

Training-time augmentation (bone scaling). The paper synthesizes a set of scale parameters $\mathcal{S}$, where each $S_i \in \mathcal{S}$ is a set of scaling factors for each bone. The scaling is applied from proximal to distal along each limb, taking either the clavicle or the pelvis as the central reference:

$\tilde{\mathcal{J}} = \text{Aug}(S, \mathcal{J})$

What this computes: the 3D keypoints $\mathcal{J}$ are transformed by scaling each limb's bone lengths according to the sampled scaling parameters $S$, producing an augmented 3D skeleton $\tilde{\mathcal{J}}$ that represents the same motion but with different body proportions. The scaling is applied in 3D space, not in pixel space, which means bone directions (angles) are preserved—only lengths change.

Why proximal-to-distal from center: scaling limbs outward from the body center (clavicle/pelvis) ensures that the relative orientation of bones—the joint angles that define the motion—remain unchanged. Only the lengths of the limb segments are scaled. This is fundamentally different from 2D skeleton scaling, which operates in pixel space and distorts apparent bone angles when the skeleton is not fronto-parallel to the camera.

The paper uses a high augmentation rate of 0.8 (80% of training samples undergo scaling), simulating "binding motion to characters with diverse body shapes." This is critical for identity-agnostic training: the model learns that the motion signal is carried by the temporal pattern of bone angles and trajectories, not by the specific bone lengths in the rendered image.

Inference-time retargeting (camera optimization). When the reference character differs from the driving subject (cross-driven animation), the 3D driving pose must be spatially aligned with the reference frame so the model knows where to position the motion. Unlike prior 2D methods that apply heuristic skeleton scaling (which distorts motion), the paper optimizes only the camera projection matrix:

$\mathcal{L}_{\text{proj}} = \frac{1}{N} \sum_{j=1}^{N} \left\| \Pi\left( \tilde{\text{Cam}}^{3 \times 3} \tilde{J}_{0,j} \right) - P_j^{\text{ref}} \right\|_2^2$

where $\Pi(\cdot)$ is the perspective projection, $\tilde{\text{Cam}}^{3 \times 3}$ is the camera projection matrix being optimized, $\tilde{J}_{0,j}$ is the 3D position of joint $j$ in the first driving frame (after any training-style augmentation), $P_j^{\text{ref}}$ is the estimated 2D position of joint $j$ in the reference image, and $N$ is the number of joints.

What it computes: the mean squared error between the 2D projections of the driving subject's 3D keypoints (using the current camera estimate) and the detected 2D keypoints of the reference character. The camera matrix $\tilde{\text{Cam}}^{3 \times 3}$ is a $3 \times 3$ matrix that parameterizes the perspective projection (intrinsic parameters such as focal length and principle point, plus extrinsic rotation/translation). By optimizing this matrix to minimize $\mathcal{L}_{\text{proj}}$, the system finds the camera viewpoint that makes the driving pose project onto the reference character's detected keypoints as closely as possible.

Why optimize the camera rather than scale the skeleton: changing the camera projection merely changes the viewpoint from which the 3D skeleton is rendered—it does not alter the 3D skeleton itself. The actual bone directions, motion trajectories, and relative joint positions remain unchanged. This is the key property that distinguishes 3D camera optimization from 2D skeleton scaling: "3D Retarget helps transfer the motion to the person without introducing position change. Compared to 2D Retarget, 3D Retarget keeps the original motion without introducing limb length distortion." The appendix (Figure A2) demonstrates this visually, showing that 2D retargeting produces unreasonable scaling factors that distort body proportions, while 3D retargeting preserves the original limb proportions.

The optimized camera matrix is then used to render the entire driving pose sequence, producing a retargeted pose video that is spatially aligned with the reference frame but preserves the full 3D motion semantics.

Training-time camera perturbation. Because the inference-time camera optimization may introduce approximation error (the projection alignment is imperfect), the paper simulates this error during training by applying the same camera alignment method using the reference image and the first frame of the driving sequence, and then "exerting modest disturbance on camera parameters $\text{Cam}^{3\times3}$ to enhance the model's robustness towards camera variance." This training-inference gap mitigation is important: if the model were only trained with perfectly aligned poses (where the training-time reference frame and driving video start from the same camera viewpoint), it would be sensitive to the misalignment that inevitably occurs during inference-time camera optimization. By introducing controlled perturbation during training, the model learns to be robust to small camera parameter errors.

Hand and face augmentation. The primary pose representation uses 3D body keypoints, but hands and faces require finer-grained control. The paper overlays 2D hand and face keypoints (extracted via DWPose [46]) onto the rendered 3D skeleton sequences. During augmentation or retargeting, "2D keypoints are shifted to match the reformed 3D skeleton after body rescaling and camera manipulation ... to minimize the unintended influence of the 2D facial and hand signals on the 3D pose representation." This means: after the 3D body skeleton undergoes scaling or camera optimization, the overlaid 2D hand/face keypoints are repositioned to stay consistent with the transformed body. Without this realignment, the hand/face keypoints would be spatially inconsistent with the moved body skeleton, potentially confusing the model.

Integration with the Figure Pool and Camera Pool (Figure 2). The paper's Figure 2 illustrates the full pipeline: a pre-computed Figure Pool contains diverse scale parameters $S_1, S_2, S_3, S_4, \ldots$ synthesized from bone scaling rules; a Camera Pool contains camera parameters $\text{Cam}_1, \text{Cam}_2, \ldots$ for perturbation. During training, the system randomly samples a figure scale set $S_i$ from the Figure Pool and a camera $\text{Cam}_k$ from the Camera Pool, applies the augmentation to the estimated 3D keypoints $\mathcal{J}$, renders the augmented skeleton, performs camera perturbation, overlays adjusted 2D hand/face keypoints, and produces the final training pose video. During inference for cross-driven animation, the retargeting optimization (using only the reference image and first driving frame) replaces the random camera sampling, producing a retargeted camera $\tilde{\text{Cam}}^{3\times3}$ that is used for the entire sequence.

Spatial pooling. The paper notes that "the rendered pose sequence is relatively sparse as most frame areas consist of non-informative black pixels." To reduce the token count for full-context concatenation, spatial pooling is applied to the pose video, reducing the resolution by 2× (resulting in 1/4 of the original token count). Empirically, "with 2× downsampling the pose following ability is nearly unaffected," likely because the cylindrical skeleton rendering is a sparse signal with large uniform regions—spatial pooling compresses mostly black space without losing discriminative bone-edge information.

---

Full-Context Pose Injection and Pose-Shifted RoPE

The second core innovation is the mechanism by which the pose sequence is injected into the DiT. The paper argues that the dominant approach—channel-wise concatenation—provides only local per-frame pose cues because each frame's generation attends primarily to that frame's pose encoding, failing to provide the temporal context needed to interpret motion correctly.

Channel concatenation baseline and its limitation. In channel concatenation (illustrated in Figure 4, bottom), the pose sequence is spatially aligned with the video frames, embedded (via a small convolutional or MLP encoder), and concatenated with the noisy video latents along the channel dimension. The denoising network processes this concatenated tensor, which means that for each spatial position in the output, the network can access the pose information at that same spatial position and same frame. The paper implements this following RealisDance-DiT [51] and finds it "shows decent pose-following capability" but fails on complex motions:

"A typical example is the turning motion, where the model often fails to correctly distinguish between front and back views, generating awkward postures."

The diagnosis is that per-frame channel concatenation provides only local temporal context: the network sees one frame of pose at a time, and while temporal self-attention (in the DiT's transformer blocks) does operate across frames, the pose information is spatially bound to each frame's latent features, making it harder for the model to aggregate pose patterns across time into a coherent motion interpretation. In a turning motion, the distinction between front and back is only apparent by comparing consecutive frames—the rotation direction, the progression of which limbs become occluded—and a per-frame injection scheme buries this information in the feature dimension of each frame's latent, requiring the transformer's temporal attention to reconstruct the motion semantics from scattered feature cues.

Full-context concatenation. The proposed alternative (Figure 4, top) treats the pose sequence as an explicit token sequence that is concatenated with the video tokens:

1. The rendered pose video (potentially spatially pooled by 2×) is patchified using a separate patch embedding layer from the video's VAE patchification. This produces a set of pose tokens that maintain spatial and temporal structure.

2. These pose tokens are directly concatenated with the noisy video tokens (from the VAE latent patchification), the reference image tokens (embedded with the same patchification strategy), and optional text tokens. The concatenation is along the sequence dimension, not the channel dimension.

3. The DiT's self-attention operates over this entire unified token sequence. When generating any frame, the attention mechanism can attend to pose tokens from the same frame (spatial correspondence) and pose tokens from any other frame in the sequence (temporal context). This means "the model to reason over the entire pose sequence"—a turning motion's rotation direction is directly available as a pattern across consecutive pose tokens, not buried in per-frame feature channels.

Why this enables spatiotemporal reasoning: in a standard transformer, self-attention computes pairwise interactions between all tokens in the sequence. By including pose tokens in the same sequence as video tokens, the model can learn attention patterns where video tokens at frame $t$ attend to pose tokens at frames $t-1$, $t$, $t+1$, and beyond, building a representation that integrates motion context across time. The paper gives the concrete example of the running scenario (Figure 9, case 2): when the pose estimator misidentifies left/right and extracts a forward-lunge pose during a running sequence, the model with full-context injection can still generate correct running posture because "conditions from the reference frame and estimations in other frames clearly reflect the semantic context of running along the river." The global pose context overrides the local pose error—a capability that per-frame injection fundamentally cannot provide because no single frame's pose contains the correction signal.

Cost considerations. Full-context concatenation increases the sequence length because the pose tokens are added to the existing video tokens. With spatial 2× downsampling, the pose tokens constitute approximately 1/4 of the original video token count. The paper reports inference-time costs at 512×896 resolution, 81 frames, 20 diffusion steps on an H100 GPU:

• Channel concatenation (14B): 286.11 seconds (0.283 FPS), 61.7 GB memory
• Full-context (14B): 380.78 seconds (0.213 FPS), 68.5 GB memory

The full-context approach is approximately 33% slower and uses 11% more memory, which the paper acknowledges as a "modest efficiency trade-off ... acceptable, particularly in studio-grade scenarios which prioritize stringent accuracy and stability."

Architectural simplicity. An important design choice: "full-context pose injection introduces no new parameters except the additional patchify layer to the original model, offering a more streamlined architecture compared to stacking additional DiT layers in residual context-tuning methods [17]." This means the approach does not require a separate encoder network, adapter modules, or cross-attention layers for the pose—it leverages the existing DiT self-attention, simply adding pose tokens to the input sequence. This is both parameter-efficient and conceptually clean: the model learns to attend to pose information through the same mechanism it uses for video tokens, with the positional encoding distinguishing the two types of tokens.

Pose-Shifted RoPE. The standard 3D Rotary Position Embedding (RoPE) in the Wan DiT assigns each token a positional encoding based on its (t, h, w) coordinates—the temporal frame index $t$ and the spatial height $h$ and width $w$ positions, all starting from zero. This works when the pose tokens are spatially aligned with the video tokens (as in channel concatenation), but in the full-context setting, the pose tokens are in a separate region of the token sequence after concatenation, and their spatial coordinates may not align with the video's spatial grid after augmentation and retargeting.

The Pose-Shifted RoPE applies a constant shift along the width dimension specifically to the pose tokens:

$\text{Pos}^{\text{pose}} = [t, h, W_{\text{max}} : W_{\text{max}} + \text{shift}W]$

where $t$ is the frame index, $h$ is the height position, and the width position is offset by $W_{\text{max}}$ (the width of the reference image, placing pose tokens after the reference image in the width dimension), with an additional $\text{shift}W$ constant.

What this encodes: pose tokens have positional indices that are shifted along the width dimension by a large constant offset, effectively placing them "to the right" of all reference image tokens. This means the positional encoding distinguishes three token types: reference image tokens occupy width positions 0 to $W_{\text{max}}-1$, noisy video tokens occupy width positions 0 to $W_{\text{latent}}-1$ (where $W_{\text{latent}}$ is the latent spatial width), and pose tokens occupy width positions $W_{\text{max}}$ to $W_{\text{max}} + \text{shift}W$. The attention mechanism can learn different attention patterns for these token groups based on their positional encodings.

Why a large shift: "Considering the modality gap between the noisy video tokens and the driving pose tokens, we observe that the model performs best particularly when the shift distance is a relatively large constant." A large shift creates a clear separation in the positional space, preventing the model from confusing pose tokens with video tokens that have similar spatial coordinates. Without this separation, the model might attend to pose tokens as if they were video tokens with unusual content, or vice versa. The shift effectively marks the pose tokens as belonging to a distinct "spatial location" that the model learns corresponds to the driving signal.

Mean-pooling of RoPE frequencies. To accommodate different downsampling ratios (2× spatial pooling on pose tokens), the paper performs mean-pooling on the 3D-RoPE frequencies according to the applied ratio. RoPE encodes position through sine/cosine functions with different frequencies; downsampling the pose resolution means the effective spatial positions are coarser, and the positional frequencies must be adjusted accordingly. Mean-pooling the frequency grid averages pairs of neighboring frequency values, which is the natural operation for reducing spatial resolution in the frequency domain.

Integration with the I2V architecture and mask. The SCAIL model builds on the Wan2.1 Image-to-Video (I2V) base, which has a specific mechanism for injecting the reference image: the reference image is placed at the beginning of the video sequence (frame $T=0$), and its CLIP feature is injected into the DiT through cross-attention or feature concatenation. The paper modifies this for the pose-conditioning setting:

• Sequence structure: The reference image is concatenated at the beginning of the sequence, and generation starts from $T=1$ (the first generated frame, not the reference frame). The noisy video latents occupy frames $T=1$ through $T=T_{\text{max}}$, and the pose tokens occupy a separate segment of the sequence.

• I2V mask: The original Wan-I2V model uses a mask to distinguish conditional tokens (reference image) from generated tokens (noisy video). The paper adapts this by applying "an all-one mask for the reference image and the driving sequence, and an all-zero mask for the noisy video sequence." This means the reference image and the entire pose sequence are treated as unconditional (not denoised, always fully visible), while the noisy video latents are masked as denoised tokens. This design helps the model "better distinguish the conditional tokens and the noisy video sequence."

• Reference CLIP feature injection: The reference image's CLIP embedding is injected using the original I2V pattern, providing appearance conditioning for the generated character.

Summary of injection mechanism operation (inference):

1. The reference image is patchified and embedded (CLIP feature extracted), producing reference tokens.
2. The driving pose video (after any retargeting) is spatially pooled (2×), patchified, and embedded, producing pose tokens with Pose-Shifted RoPE positional encodings.
3. The noisy video latents are patchified and embedded, producing video tokens.
4. The three token groups are concatenated into a single sequence: [reference tokens | pose tokens | video tokens].
5. The I2V mask is applied: mask = 1 for reference and pose tokens, mask = 0 for video tokens.
6. The DiT processes the full sequence through $N$ transformer blocks, each applying self-attention over the entire unified token set, followed by feed-forward layers.
7. After $T$ diffusion timesteps, the denoised video latent is decoded by the VAE into pixel-space frames.

---

3D-Consistent Adaptation Subsystem Details

The augmentation and retargeting procedures introduced in the pose representation section constitute a subsystem with specific parameters and design choices that deserve separate treatment because they are the mechanism by which identity-agnostic motion transfer is achieved.

Training augmentation parameters. The augmentation rate is set to 0.8—80% of training samples undergo bone scaling and camera perturbation. The remaining 20% are presented without augmentation, ensuring the model also learns the baseline case of identity-matched motion (self-driven animation). The paper notes: "As our augmentation strategy is designed to maximally preserve the original motion, we use a high augmentation rate of 0.8 to achieve the balance between the pose-following accuracy and motion transfer robustness." A higher rate forces the model to rely less on bone-length cues and more on bone-angle and temporal-trajectory cues, which are invariant under the scaling transformation.

Figure Pool construction. The scale parameters $\mathcal{S}$ are synthesized based on the anatomical rule of proximal-to-distal scaling from the clavicle or pelvis center. This is not random scaling—it respects the physical constraint that limbs scale outward from the body center. For each bone in the bone set $\mathcal{B}$, a scaling factor is drawn, producing a complete set of per-bone scale factors $S_i$. Multiple such sets are pre-computed and stored in the Figure Pool for efficient random sampling during training. The paper does not specify the range of scaling factors, but the assumption from the qualitative results (Figure A4) is that they cover substantial variation, simulating characters ranging from short-limbed plush toys to long-limbed stylized figures.

Camera Pool and perturbation. Similarly, a set of camera parameters is pre-computed, representing varied camera viewpoints (zoom in/out, move up/down/left/right, principle point shift). During training, the actual perturbation applies "modest disturbance" to the inferred camera parameters from the reference-to-first-frame alignment. This means: for each training sample, the system first computes the camera alignment between the reference frame and the first driving frame (using the same $\mathcal{L}_{\text{proj}}$ optimization), and then adds a small random perturbation. The magnitude of the perturbation controls the robustness-accuracy tradeoff—too large, and the model cannot learn spatial correspondence; too small, and the model is sensitive to inference-time alignment errors.

Inference retargeting optimization. During cross-driven inference, the $\mathcal{L}_{\text{proj}}$ optimization is performed to find $\tilde{\text{Cam}}^{3\times3}$. This optimization targets the first driving frame and the reference image: $\tilde{J}_{0,j}$ (the 3D keypoints of the first driving frame) and $P_j^{\text{ref}}$ (the estimated 2D keypoints of the reference image). The optimization is a numerical minimization of the squared projection error, producing a camera matrix that aligns the driving skeleton's projection with the reference character's keypoint positions. This optimized camera is then used for all frames of the driving sequence, projecting the entire 3D pose sequence through the same viewpoint.

Why this is identity-agnostic: the core insight is that optimizing only the camera projection leaves the 3D skeleton itself completely unchanged—all joint positions, bone directions, and motion trajectories remain as estimated from the driving video. The character's body proportions are determined by the rendered 3D skeleton, which carries the driver's proportions, but because the model was trained with extensive bone scaling (simulating diverse body shapes), it learns that the motion signal is the temporal pattern, not the specific bone lengths. In cross-driven inference, the reference image's CLIP feature provides the appearance target, and the 3D skeleton provides the motion trajectory—the model generates frames where the reference character's appearance is mapped onto the driving motion, with the bone scaling augmentation ensuring robustness to the mismatch between the driver's proportions and the reference character's appearance.

Hand and face overlay alignment. The 2D hand and face keypoints extracted by DWPose [46] are overlaid onto the rendered 3D body skeleton. These keypoints are in pixel coordinates relative to the original frame. When the 3D body skeleton is augmented (bone scaling) or retargeted (camera optimization), the pixel positions of body joints change. To maintain consistency, the overlay process shifts the 2D hand/face keypoints to match the reformed 3D skeleton's projections. The paper notes this is "to minimize the unintended influence of the 2D facial and hand signals on the 3D pose representation"—without this realignment, the hand/face keypoints would point to different locations than the body joints they belong to, creating a contradictory signal that could degrade pose following.

---

Training Procedure and Model Variants

Base architecture. SCAIL builds on the Wan2.1 video generation foundation model [36]. The Wan2.1 models are DiT-based latent video diffusion models pretrained on large-scale video data. SCAIL finetunes two sizes:

• 1.3B version: Finetuned from the Wan2.1-1.3B-Fun-Inp backbone. Training ran for 6,000 steps with batch size 96 and learning rate $1 \times 10^{-5}$, using 32 NVIDIA H100 GPUs for approximately two days. The optimizer is AdamW [21].

• 14B version: Finetuned from the Wan2.1-I2V-14B backbone (the Image-to-Video variant, which already includes reference image conditioning). Training occurs in two stages:

  • Pretraining stage: 8,000 steps, batch size 96, learning rate $1 \times 10^{-5}$, using 128 NVIDIA H100 GPUs with sequence parallelism enabled, for over four days.
  • Finetuning stage: After convergence, an additional 400 steps with the same batch size and a reduced learning rate of $4 \times 10^{-6}$, using the curated 4,000-clip high-dynamic subset (the manually inspected, highest-motion-speed samples).

Two-stage training rationale. The pretraining stage exposes the model to the full diversity of the filtered dataset (~250K clips), learning the general mapping from pose sequences to character animations across diverse motions, characters, and single/multi-person scenarios. The finetuning stage focuses on the most challenging cases—the highest-motion-speed clips, manually verified for clarity—to refine the model's ability to handle extreme motions without artifacts. This is a standard curriculum strategy: broad coverage followed by targeted refinement on hard cases.

Learning rate schedule. The paper does not specify a warmup or decay schedule, but notes that the pretraining stage uses $1 \times 10^{-5}$ for both model sizes, and the 14B finetuning reduces to $4 \times 10^{-6}$, which is approximately 40% of the pretraining rate—a modest reduction consistent with fine-tuning on a smaller, higher-quality subset.

Inference settings. At inference time, the classifier-free guidance (CFG) scale is set to 4, "offering a favorable balance between pose following and video fidelity." Higher CFG values would strengthen the conditioning signal (pose following) at the cost of video quality (over-saturation, artifacts), while lower values produce more diverse but less controlled output. The value of 4 is in the typical range for DiT-based video generation models.

Sequence parallelism. The 14B model training uses sequence parallelism, a distributed training technique where the token sequence is partitioned across GPUs, enabling training of long sequences (full-context concatenation increases sequence length) without exceeding per-GPU memory limits. This is enabled by the transformer architecture's independence across sequence positions in attention computation.

No new parameters beyond patchification. The full-context injection scheme adds only the pose patchify layer—a small convolutional or linear projection that converts the rendered pose frames into the same embedding dimension as the video latents. All other components (DiT transformer blocks, VAE encoder/decoder, CLIP reference embedding) are inherited from the Wan2.1 base model and finetuned. This is in contrast to adapter-based approaches (e.g., Pose Guider in AnimateAnyone [15], residual context-tuning modules in VACE [17]) that add trainable adapter networks or additional DiT layers, increasing parameter count and architectural complexity.

Training-inference gap mitigation. The training data construction deliberately simulates inference conditions: the reference image and driving video are "sampled from different parts of the video," meaning the model learns cross-frame appearance transfer during training. Combined with the camera perturbation (simulating retargeting error) and bone scaling (simulating body-shape mismatch), the training protocol aims to cover the distribution of inputs the model will encounter during cross-driven inference. The paper's ablation on augmentation (Appendix A2, Figure A3) quantifies this: removing 3D augmentation during training significantly degrades Physical Consistency and Identity Similarity metrics in cross-driven animation, confirming that the augmentation strategy is essential for generalization to unseen character shapes.

Data proportions and diversity. The dataset maintains deliberate diversity along multiple axes: motion types (dance, sports, gymnastics, figure skating, martial arts, acrobatics, everyday actions), character styles (photorealistic humans, 3D animations, 2D animations, MMD, Live2D), and scenarios (single-person, multi-person, environment interaction). This diversity is not just about volume—it ensures that the model encounters the distribution of challenges (complex motions, stylized characters, occlusions) that Studio-Bench evaluates. The paper's curation philosophy is that "data quality and relevance, not just quantity, determines performance on challenging cases."

4. Key Insights and Innovations

Innovation 1: Depth-Aware, Identity-Agnostic Pose Representation as a First-Class Design Choice

The paper's most fundamental conceptual contribution is reframing pose representation from an extraction detail to the central architectural decision governing motion transfer. Prior work treated pose representation as a commodity—2D keypoints from DWPose or ViTPose were the default, occasionally augmented with SMPL meshes. The field's attention was on the generation architecture (U-Net vs. DiT, adapter designs, ReferenceNet modules), implicitly assuming that if the model were powerful enough, it could overcome limitations in the pose signal. This paper demonstrates that assumption is false: the representation itself is the bottleneck, and no amount of architectural sophistication can compensate for an ambiguous, depth-less motion signal.

This is a diagnostic reframing more than an architectural novelty. The insight is not "3D is better than 2D"—that would be an obvious but shallow claim. The insight is that what constitutes a good pose representation depends on what the generation model needs to reason about, and for studio-grade animation, the model needs three specific properties that prior representations fail to provide: (1) depth ordering to disambiguate occlusions (front/back limbs, inter-character occlusion), (2) identity independence so that the motion signal doesn't contaminate the appearance signal, and (3) augmentation-compatibility so that training-time diversity (simulating different body shapes) and inference-time retargeting (aligning to different characters) don't distort the motion semantics.

The paper explicitly names the failure of each prior approach against these criteria: 2D keypoints fail (1) by design (no depth channel) and (3) because 2D skeleton scaling is a deformation, not a rigid transformation, introducing "2D-3D mismatch noise." SMPL meshes fail (2) because their strong parametric body prior constitutes "identity leakage"—the mesh encodes the driver's body shape, and when the reference character differs, the mesh prior fights against appearance preservation. Neither representation was designed to satisfy all three requirements simultaneously; prior work simply didn't frame the problem in these terms.

What makes this intellectual move distinctive is that it reveals a latent trilemma in pose representation design—and then shows that cylindrical 3D rasterization resolves the trilemma through a specific combination of properties: the rendering process explicitly encodes depth via ray marching (requirement 1), the representation discards body shape priors by using only bone connectivity and direction (requirement 2), and the 3D nature of the signal enables rigid-transformation-based augmentation and retargeting that preserves motion semantics (requirement 3). This is not just a better representation; it's a representation whose design was derived from analyzing why existing approaches fail, then constructing something that systematically addresses each failure mode.

The evidence that this matters beyond the specific cylinder implementation: the ablation comparing 2D and 3D pose under the same architecture (Table 2) shows PSNR dropping from 18.08 to 17.08 and FVD increasing from 228.62 to 295.36 when switching from 3D to 2D pose—substantial degradation that cannot be recovered by the stronger injection mechanism (Channel Concat-1.3B w/ 2D Pose has PSNR 17.12, FVD 296.23, showing that even with otherwise-identical architecture, the pose representation alone accounts for major performance differences). This confirms that representation quality is not a second-order effect; it fundamentally bounds what the generation model can achieve, regardless of how the pose is injected.

---

Innovation 2: Full-Context Pose Injection as In-Context Motion Reasoning

The paper's second conceptual contribution is recognizing that motion following is a spatiotemporal reasoning problem, not a per-frame alignment problem, and that the injection mechanism must provide the model with the temporal context needed to resolve ambiguities that are irreducible at the single-frame level. This challenges the dominant assumption in the field—reflected in the near-universal adoption of channel concatenation (AnimateAnyone, UniAnimate-DiT, Wan-Animate, RealisDance-DiT, the authors' own baseline)—that providing per-frame pose information is sufficient if the generation model has temporal self-attention layers.

The key distinction is between local pose cues and global motion semantics. Per-frame channel concatenation provides local cues: at each spatial position and each timestep, the model can access the pose at that position and time. Temporal self-attention in the DiT does allow cross-frame interactions, but the pose information is buried in per-frame feature channels, requiring the model to disentangle and re-integrate motion patterns from scattered feature representations. The paper's insight is that this is asking the model to reconstruct motion semantics from local evidence—it can do it to some degree (channel concatenation works on simple motions), but it's inefficient and fragile, breaking down when motions are complex or when local pose cues are ambiguous.

Full-context concatenation reframes the problem: instead of the model having to extract motion from per-frame signals, the entire pose sequence is presented as a unified context that the attention mechanism can query directly. This is explicitly named in-context learning, drawing a deliberate analogy to the language model literature where providing examples or context tokens enables reasoning that would be impossible from the prompt alone. The pose sequence becomes a context the model "reads" when generating each frame, analogous to how an LLM attends to earlier tokens in a conversation to maintain coherence.

What makes this more than an incremental architectural change (concatenation along sequence vs. channel dimension) is the diagnosis that motivated it: the paper identifies a specific, reproducible failure case—turning motions, where "the model often fails to correctly distinguish between front and back views"—that persists despite accurate per-frame pose signals. The turning motion is diagnostic precisely because the front/back ambiguity is only resolvable by comparing consecutive frames: the rotation direction, the progression of limb occlusion, and the continuity of joint trajectories define "turning," but no single frame contains this information. Per-frame injection provides all the local evidence but none of the relational structure; the model must infer the relation, and it often fails. Full-context injection provides the relation directly by making consecutive pose frames co-attendable as a single context.

The ablation supports this as a fundamental architectural choice, not a minor implementation detail: removing full-context injection (switching to channel concatenation) on the 1.3B model drops PSNR from 18.08 to 17.69 and increases FVD from 228.62 to 263.63 (Table 2). The qualitative evidence is even more telling: Figure 9 case (2) shows that when the pose estimator produces an incorrect local pose (a forward lunge from left/right confusion during running), the full-context model recovers correct running posture from the global motion context, while the channel-concat model produces the incorrect lunge. This demonstrates that full-context injection enables a capability that per-frame injection cannot replicate: global motion context can override local pose errors, because attention over the full sequence allows the model to weight temporal context (which is consistently "running") more heavily than local evidence (which is momentarily wrong). This is not a refinement—it's a qualitatively different operation.

---

Innovation 3: The Camera Optimization Approach to Cross-Identity Retargeting

The third conceptual contribution is replacing heuristic skeleton scaling with camera optimization as a projection-alignment operation that preserves motion semantics. Prior methods (Animate-X, UniAnimate-DiT) handle cross-identity animation by scaling the 2D skeleton to match the reference character's proportions. The paper argues this is fundamentally broken architecture: scaling a 2D skeleton changes the apparent bone angles because the transformation is applied in pixel space, not in 3D, and the skeleton's projection onto the image plane is not invariant under scaling. The result is that the motion signal provided to the model is distorted—the heuristic retargeting that is supposed to enable cross-identity transfer actually degrades the motion information.

This is a reframing of the retargeting problem from a skeleton transformation problem to a viewpoint selection problem. The key conceptual move is recognizing that the 3D skeleton already contains the correct motion, and the only thing that needs to change is the camera through which it is viewed. The camera projection matrix $\tilde{\text{Cam}}^{3 \times 3}$ is optimized to align the projected 3D keypoints with the reference character's 2D keypoints via $\mathcal{L}_{\text{proj}}$, but the 3D skeleton itself—all joint positions, bone directions, and motion trajectories—remains exactly as estimated from the driving video. Motion semantics are preserved because the transformation is a rigid projection, not a deformation.

The innovation's significance extends beyond the specific camera optimization implementation. It establishes a preservation principle for motion transfer: the motion signal should be transformed only through operations that preserve the kinematic structure of the motion (rigid transformations, projection changes), never through operations that deform the skeleton (scaling, stretching). This principle is not stated explicitly in the paper but is implicit in the design: bone scaling during training preserves bone directions (only lengths change), and camera optimization during inference preserves all 3D structure (only the viewpoint changes). Both operations are "motion-preserving" in a way that 2D skeleton scaling is not. This is a transferable insight: any future work on cross-identity animation should evaluate whether its retargeting operations are motion-preserving or motion-deforming, with the latter being fundamentally problematic.

The paper provides implicit evidence for this principle through what is not present in the results: the 3D augmentation ablation (Appendix A2, Figure A4) shows that even with a high augmentation rate of 0.8, self-driven animation accuracy is unaffected ("there was no significant difference in the metrics compared to when augmentation was not used"). This is surprising if you think of augmentation as a distortion—it should degrade performance. The fact that it doesn't confirms that the augmentation is motion-preserving: the model learns to map the same motion (same bone angles, same trajectories) through different body shapes (different bone lengths), without confusion, because the motion signal itself is invariant under bone-length scaling. A 2D scaling augmentation would not have this property and would likely degrade self-driven performance, as the apparent motion would change when skeletons are stretched in pixel space.

---

Innovation 4: The Curation-as-Architecture Thesis for Studio-Grade Data

The paper articulates, through its data pipeline design and evaluation benchmark construction, a thesis that is more methodological than technical: for studio-grade character animation, data curation decisions are architectural decisions, not preprocessing details. What you filter out—static clips, partial-body views, low-motion content, occlusion-heavy frames where keypoint estimation fails—determines what the model learns to be robust to, just as much as the model architecture determines what it can represent.

This is not the generic claim that "data quality matters." The specific argument is that failure modes on complex cases are determined by the gap between the training distribution and the deployment distribution, and that closing this gap requires deliberate curation to ensure the training distribution covers the challenging cases that production requires. Standard large-scale video datasets, by virtue of being drawn from the natural distribution of internet video, are dominated by static, low-motion, simple-action content. Training on such data produces models that work well on simple actions (the long tail of the training distribution) but fail on complex motions (which are rare in the training data and therefore poorly modeled). The model's apparent capability ceiling is set not by architecture or scale but by the data's coverage of the target distribution.

What makes this intellectually distinctive is that the paper operationalizes this thesis through a filtering criterion derived from the task definition: the motion speed metric $v = \frac{1}{T} \sum_{t=1}^{T-1} \sum_j \|\bar{J}_{t+1,j} - \bar{J}_{t,j}\|_2$ is not an arbitrary quality heuristic but a direct measure of whether the data contains the property the model needs to learn (complex motion dependencies). The 3D root-relative formulation ensures the metric measures pose change rather than scene translation, making it a task-relevant filter rather than a generic activity detector. This is curation driven by what the model needs to learn, not by what produces visually appealing training clips.

The evidence for this thesis is distributed across the paper rather than concentrated in a single ablation: the motion-rich filtering produces a training set that enables performance on Studio-Bench's complex motions, and the finetuning subset (4,000 highest-motion-speed, manually inspected clips) provides the final quality push. The paper doesn't ablate the curation pipeline directly (comparing models trained on filtered vs. unfiltered data), which is a limitation, but the Studio-Bench construction itself validates the thesis: by deliberately selecting challenging cases (complex motions, cross-identity, multi-person, stylized characters), the benchmark reveals that prior methods trained on less-curated data fail systematically on these scenarios, while SCAIL, trained on curated data that covers these distributions, succeeds. The benchmark is thus both an evaluation tool and a demonstration of the curation thesis: the performance gap between methods on Studio-Bench is partly attributable to the distribution gap between their training data and the benchmark's difficulty profile.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All experiments are evaluated on the authors' proposed Studio-Bench, a purpose-built benchmark constructed specifically to test studio-grade character animation capabilities. It consists of two parts: a Self-Driven subset with 130 clips for evaluating motion adherence under complex single- and multi-person actions (enabling direct quantitative comparison against ground-truth videos), and a Cross-Driven subset with 120 single-character pairs plus 10 multi-character pairs for evaluating generalization when the reference image and driving video differ in identity or domain. Evaluation samples are strictly excluded from the training data. The paper notes that prior benchmarks "primarily focus on simple actions, which fail to capture the challenges present in film production" (Section 3.4), motivating this construction.

• Base model(s). Two model scales of SCAIL are trained and evaluated, both building on the Wan2.1 video generation foundation model [36]: a 1.3B-parameter version finetuned from Wan2.1-1.3B-Fun-Inp, and a 14B-parameter version finetuned from Wan2.1-I2V-14B (the Image-to-Video variant with built-in reference image conditioning). The 1.3B model serves primarily for ablation studies, while the 14B model is used for the main quantitative comparisons against baselines. The authors argue Wan2.1 is "representative" as a DiT-based I2V backbone. For the user study comparison against the commercial product Viggle [35], the paper notes that Viggle "is widely believed to rely on a 3D foundation model rather than video diffusion and can be a strong baseline for frame-by-frame motion accuracy under complex scenarios" (Section 4.2).

• Metrics. The paper employs two distinct evaluation regimes corresponding to the two Studio-Bench subsets:

  • Self-Driven subset: Standard video quality metrics computed by comparing generated outputs against paired ground-truth videos: PSNR [14], SSIM [40], LPIPS [48] (perceptual similarity), and FVD [32] (video-level distributional distance). These are pixel- and feature-space fidelity measures that require ground-truth reference.
  • Cross-Driven subset: Four human-evaluated metrics collected via a "detailed blinded user study" (Section 4.2) where users vote for the best-performing model in pairwise comparisons: (1) Motion Accuracy (Mot-Acc): "how faithfully the generated motion follows the driving signal in a frame-by-frame manner"; (2) Kinesiology Consistency (Kin-Consis): evaluating "whether joint rotations and body movements remain anatomically feasible and temporally coherent, penalizing sudden twists or physically impossible poses"; (3) Physical Consistency (Phy-Consis): assessing "whether the generated motions comply with basic physical constraints such as gravity, support, and momentum conservation, penalizing unrealistic behaviors like hovering in midair"; (4) Identity Similarity (ID-Sim): measuring "the consistency of the subject's appearance with the reference image." For the best-model-selection evaluation (Table 1), each metric's score represents the percentage of evaluations where that model was selected as best. A separate Win/Tie/Lose user study (Figure 6) collects head-to-head preferences between SCAIL and each baseline, reporting win rate, tie rate, and loss rate.

• Baselines. The paper compares against four methods chosen to span the landscape of current approaches:

  • UniAnimate-DiT [38]: A DiT-based human image animation method that uses 2D keypoints as motion control. Selected as a representative of the DiT architecture family with conventional 2D pose representation. The paper notes this method is omitted from the Viggle comparison "due to obvious artifacts under studio-grade demandings" (Section 4.2).
  • VACE [17]: An "all-in-one video creation and editing" DiT-based model that supports character animation. Uses channel-concatenation-style pose injection. The authors implement retargeting logic from UniAnimate-DiT for VACE, since VACE does not natively support retargeting.
  • Wan-Animate [4]: A character animation method built specifically on the Wan foundation model family, using channel concatenation for pose injection. This serves as the closest architectural baseline since SCAIL also builds on Wan.
  • Viggle [35]: A closed-source commercial product that the paper characterizes as "widely believed to rely on a 3D foundation model rather than video diffusion." Included specifically as a strong baseline for motion accuracy under complex scenarios, since 3D-foundation-model approaches may have different failure modes than video diffusion methods. Evaluated only in the Win/Tie/Lose user study, not in Table 1.
  • Additionally, the ablation studies (Table 2) include MimicMotion [49] as a U-Net-based baseline, representing the pre-DiT generation of character animation methods. And a Channel Concat-1.3B internal baseline (the authors' own 1.3B model with full-context injection replaced by channel concatenation) is used for ablation.

  All compared methods in Table 1 are "built upon 14B Wan foundation models" (Table 1 caption), ensuring that model scale and base architecture are controlled.

• Generation budget / compute accounting. The paper does not use a FLOPs-based compute budget in the style of the example paper; instead, the primary comparison strategy is architecture-controlled: all baselines compared in Table 1 use the same 14B Wan backbone, so generation cost is approximately equal (modulo differences in injection mechanism). The paper reports inference costs for the two injection schemes in the appendix: at 512×896 resolution, 81 frames, 20 diffusion steps on an H100 GPU, channel-concat uses 286.11 seconds with 61.7 GB memory, while full-context uses 380.78 seconds with 68.5 GB memory—a 33% increase in time and 11% increase in memory. The authors characterize this as a "modest efficiency trade-off" acceptable for studio-grade scenarios. For ablation studies, all models (including the 1.3B variants) are trained with identical data, batch size, learning rate, and training steps to ensure fair comparison. Classifier-free guidance scale is fixed at 4 for all models during inference.

• Cross-validation / statistical protocol. The user study includes an explicit rigor measure: "the user study for selecting the best-performing model in Table 1 and the win/tie/lose evaluation in Figure 6 were conducted on different batches of participants, allowing the two sets of results to serve as cross-validations" (Section 4.2). This provides some independence between the two evaluation formats, though the paper does not report inter-annotator agreement, number of annotators, or confidence intervals. For the quantitative metrics on the Self-Driven subset, standard deterministic computation applies (PSNR, SSIM, LPIPS, FVD are computed directly from generated and ground-truth frame pairs). No k-fold cross-validation or statistical significance testing is reported.

Main Quantitative Results

Self-Driven Animation: Pixel-Space and Feature-Space Fidelity

Table 1 reports quantitative comparisons on the Self-Driven subset of Studio-Bench for the 14B models. SCAIL-14B achieves the best scores across all four standard metrics:

• PSNR: 19.22 (vs. 18.54 for Wan-Animate, 17.79 for UniAnimate-DiT, 16.73 for VACE). The improvement over the next-best method (Wan-Animate) is 0.68 dB.
• SSIM: 0.660 (vs. 0.648 for Wan-Animate, 0.637 for UniAnimate-DiT, 0.588 for VACE).
• LPIPS: 0.206 (lower is better; vs. 0.221 for Wan-Animate, 0.242 for UniAnimate-DiT, 0.263 for VACE).
• FVD: 176.16 (lower is better; vs. 187.61 for Wan-Animate, 264.71 for VACE, 362.27 for UniAnimate-DiT).

These results establish that SCAIL improves video reconstruction fidelity over all baselines. Notably, the gap is largest on FVD (the video-level distributional metric), where SCAIL outperforms VACE by 88.55 and UniAnimate-DiT by 186.11, suggesting that the benefits of full-context pose injection and 3D pose representation are most pronounced for temporal coherence rather than per-frame image quality alone. This is consistent with the paper's claim that the 3D pose representation and full-context injection improve spatiotemporal reasoning: frame-level metrics (PSNR, SSIM) show modest gains, while video-level metrics (FVD) show substantial gains, indicating that the primary improvement is in motion consistency across frames rather than static frame quality.

The relatively small gap between SCAIL and Wan-Animate on PSNR and SSIM (both building on the same Wan backbone) versus the larger gap on FVD reinforces the interpretation that the architectural changes (full-context injection, Pose-Shifted RoPE) specifically benefit temporal modeling rather than per-frame generation.

Cross-Driven Animation: User Study Best-Model Selection

Table 1 also reports the human evaluation metrics on the Cross-Driven subset (explicitly noted as based on the 120 single-character pairs). The results show:

• Motion Accuracy (Mot-Acc): SCAIL-14B achieves 53.3% (meaning SCAIL was selected as best in 53.3% of evaluations for motion fidelity), compared to 35.0% for Wan-Animate, 9.2% for VACE, and 2.5% for UniAnimate-DiT.
• Kinesiology Consistency (Kin-Consis): SCAIL-14B achieves 55.8%, compared to 28.3% for Wan-Animate, 14.2% for VACE, and 1.7% for UniAnimate-DiT.
• Physical Consistency (Phy-Consis): SCAIL-14B achieves 56.7%, compared to 24.2% for Wan-Animate, 18.3% for VACE, and 0.8% for UniAnimate-DiT.
• Identity Similarity (ID-Sim): SCAIL-14B achieves 45.8%, compared to 20.0% for Wan-Animate, 32.5% for VACE, and 1.7% for UniAnimate-DiT.

On all four metrics, SCAIL-14B is selected as the best model by a substantial margin over every baseline. The relative ordering of baselines varies by metric: Wan-Animate is the second-best on Mot-Acc (35.0%), Kin-Consis (28.3%), and Phy-Consis (24.2%), while VACE leads Wan-Animate on ID-Sim (32.5% vs. 20.0%). This suggests that Wan-Animate and VACE have complementary strengths—Wan-Animate better preserves motion structure while VACE better retains identity—but SCAIL outperforms both on all dimensions, indicating the approach improves both motion and appearance without the tradeoff.

The gap is most dramatic on Physical Consistency (56.7% vs. 24.2% for the next-best baseline), where SCAIL's 3D pose representation is expected to provide the greatest advantage: by encoding depth ordering and 3D structure, the model can generate motions that respect physical constraints (limbs don't interpenetrate, characters don't float) that 2D keypoint-based methods struggle with.

The Identity Similarity metric shows that VACE (32.5%) outperforms Wan-Animate (20.0%), which is noteworthy because both use similar 2D pose representations. This may reflect differences in how the two methods handle reference image conditioning (VACE's architecture choices for appearance preservation) rather than differences in pose representation. SCAIL's 45.8% on ID-Sim suggests the 3D augmentation strategy (bone scaling during training) effectively decouples motion from identity, though the margin over baselines is smaller here than on motion metrics, consistent with the paper's focus on motion transfer rather than appearance modeling.

Win/Tie/Lose User Study Against Baselines

Figure 6 reports a separate user study comparing SCAIL-14B against VACE, Wan-Animate, and Viggle in head-to-head evaluations across the same four metrics. The results:

• Versus VACE: SCAIL wins on 89.2% of Mot-Acc evaluations (tie 10.0%, lose 0.8%), 80.8% on Kin-Consis (tie 15.8%, lose 3.3%), 77.5% on Phy-Consis (tie 5.8%, lose 16.7%), and 48.3% on ID-Sim (tie 20.0%, lose 31.7%). The ID-Sim result is the only metric where SCAIL does not strongly dominate VACE, with a substantial loss rate indicating VACE is preferred for identity preservation in some cases.

• Versus Wan-Animate: SCAIL wins on 36.7% of Mot-Acc evaluations (tie 30.0%, lose 33.3%), 45.0% on Kin-Consis (tie 24.2%, lose 30.8%), 46.7% on Phy-Consis (tie 28.3%, lose 25.0%), and 49.2% on ID-Sim (tie 25.0%, lose 25.8%). The comparison against Wan-Animate is much tighter than against VACE, with win rates clustering around 37–49% and substantial tie rates (24–30%). This is consistent with Wan-Animate being the strongest open-source baseline (as seen in Table 1) and sharing the same Wan backbone.

• Versus Viggle: SCAIL wins on 21.7% of Mot-Acc evaluations (tie 54.2%, lose 24.2%), 34.2% on Kin-Consis (tie 34.2%, lose 31.7%), 85.0% on Phy-Consis (tie 9.2%, lose 5.8%), and 63.3% on ID-Sim (tie 18.3%, lose 18.3%). This is the most informative comparison, as Viggle represents a different technical approach (3D foundation model). On Mot-Acc, the result is essentially a tie (54.2% tie rate, with SCAIL's 21.7% win and 24.2% lose nearly balanced), indicating that Viggle's 3D-based approach provides competitive frame-by-frame motion accuracy. However, on Kin-Consis (34.2% win, 34.2% tie, 31.7% lose) and especially Phy-Consis (85.0% win), SCAIL substantially outperforms Viggle. This is a key finding: the paper explicitly frames Viggle as a strong baseline for motion accuracy, and SCAIL's advantage is not in raw motion following (where the two are comparable) but in generating natural motion—anatomically feasible poses, physically plausible dynamics, and consistent identity. The 85.0% win rate on Physical Consistency against a 3D-foundation-model baseline is the paper's strongest evidence that the combination of 3D pose representation and full-context injection produces more physically plausible animations than alternative 3D-based approaches.

The cross-validation note (different participant batches for Table 1 and Figure 6) adds credibility: the consistent finding that SCAIL outperforms VACE and Wan-Animate across both evaluation formats suggests the results are not an artifact of a specific participant pool or evaluation design.

Ablation Results: 1.3B Model Quantitative Comparisons

Table 2 reports the ablation study conducted on the SCAIL-1.3B model, evaluated on the Self-Driven subset using the same four quantitative metrics. The full SCAIL-1.3B model achieves PSNR 18.08, SSIM 0.639, LPIPS 0.249, and FVD 228.62. The ablations reveal:

• SCAIL-1.3B w/ 2D Pose: Replacing the 3D-consistent pose representation with 2D keypoints (from DWPose) while keeping full-context injection and Pose-Shifted RoPE drops PSNR to 17.08 (a decrease of 1.00), SSIM to 0.619 (down 0.020), and increases LPIPS to 0.284 (up 0.035) and FVD to 295.36 (up 66.74). This is the largest degradation of any single ablation, confirming that pose representation quality is the dominant factor for overall video fidelity, consistent with the paper's claim that the 3D pose representation is a primary bottleneck addressed by SCAIL.

• SCAIL-1.3B w/o P-RoPE: Removing Pose-Shifted RoPE (but keeping full-context injection and 3D pose) drops PSNR to 17.79 (down 0.29), SSIM to 0.637 (down 0.002), and increases LPIPS to 0.280 (up 0.031) and FVD to 269.35 (up 40.73). The effect on SSIM is minimal, suggesting that static image structure is largely preserved without Pose-Shifted RoPE, but the large increases in LPIPS (perceptual similarity) and FVD (temporal coherence) indicate that the shifted positional encoding is specifically important for motion quality and temporal consistency.

• Channel Concat-1.3B: Replacing full-context injection with channel concatenation (while keeping 3D pose and no P-RoPE) drops PSNR to 17.69 (down 0.39 from SCAIL-1.3B), SSIM to 0.626 (down 0.013), and increases LPIPS to 0.262 (up 0.013) and FVD to 263.63 (up 35.01). The degradation is smaller than the 2D pose ablation but larger than the P-RoPE ablation, establishing an ordering of component importance: 3D pose representation > full-context injection > Pose-Shifted RoPE. This hierarchy is consistent with the paper's framing: the pose representation is the foundational signal, the injection mechanism determines how effectively the model can use that signal, and the positional encoding refines the injection.

• Channel Concat-1.3B w/ 2D Pose: Combining both degradations (2D pose + channel concatenation) yields the worst performance among the DiT-based variants: PSNR 17.12, SSIM 0.624, LPIPS 0.282, FVD 296.23. This is approximately the same as the 2D pose ablation with full-context injection on FVD (295.36 vs. 296.23), suggesting that the degradation from 2D pose dominates and that full-context injection cannot compensate for poor pose quality—the injection mechanism and representation quality interact multiplicatively.

• MimicMotion [49]: As a U-Net-based, non-DiT baseline, MimicMotion achieves PSNR 17.01, SSIM 0.630, LPIPS 0.314, and FVD 334.24. The very high FVD (334.24 vs. 228.62 for SCAIL-1.3B) highlights the architectural advantage of DiT over U-Net for temporal modeling, consistent with broader trends in video generation. The LPIPS of 0.314 is notably worse than any DiT variant, suggesting U-Net architectures produce more perceptually distorted frames even when SSIM (0.630) remains competitive.

The relative ordering of ablation severity (SCAIL-1.3B > w/o P-RoPE > Channel Concat-1.3B > Channel Concat-1.3B w/ 2D Pose ≈ SCAIL-1.3B w/ 2D Pose > MimicMotion) holds across all four metrics, providing a consistent ranking.

Difficulty-Aware Analysis (Implicit in the Benchmark Design)

While the paper does not report a formal difficulty-conditioned analysis in the style of the example paper, the structure of Studio-Bench itself provides implicit difficulty stratification. The Self-Driven subset contains complex actions (dance, sports, martial arts, acrobatics) that stress motion accuracy; the Cross-Driven subset stresses generalization to unseen characters. The results in Table 1 and Figure 6 can be read as a difficulty gradient:

• Easy cases (simple motions, similar identity): All baselines perform reasonably, as evidenced by prior work's competitive performance on standard benchmarks. The paper does not explicitly test simple motions, but the ablation results (Table 2) showing that channel concatenation with 3D pose (PSNR 17.69, FVD 263.63) is not far below full SCAIL-1.3B (PSNR 18.08, FVD 228.62) suggest that the injection mechanism matters less when motions are simple—the 0.39 PSNR gap may be largely attributable to challenging cases where full-context reasoning is needed.

• Medium-hard cases (complex single-person motions): The Self-Driven subset's focus on complex actions (turning, flipping, acrobatics) is where full-context injection provides the most benefit, as shown by the FVD gap between Channel Concat-1.3B (263.63) and SCAIL-1.3B (228.62).

• Hard cases (multi-person, cross-domain): The Cross-Driven subset, particularly multi-character cases with heavy occlusion and stylized reference characters, is where the 3D pose representation's depth and occlusion encoding provides the greatest advantage, as demonstrated by the qualitative results (Figures 7, 8, A6, A7, A8) where baselines exhibit limb merging, body-shape drift, and structural collapse.

Ablation Studies and Robustness Checks

Pose representation (3D vs. 2D): Table 2 quantifies the degradation from switching to 2D keypoints: PSNR drops 1.00, FVD increases by 66.74. Qualitative evidence in Figure A1 shows specific failure modes: case (1) demonstrates distorted limbs when transferring motion to a significantly different reference image using 2D pose, and case (2) shows that 2D pose ambiguity causes incorrect front/back limb interpretations. This ablation is the most impactful single component change, confirming that the 3D representation is the primary performance driver. The implementation is a fair comparison: the 2D version uses "similar figure settings and same augmentation ratio of 0.8" with retargeting logic from UniAnimate-DiT applied during inference (Appendix A2.1), meaning the comparison isolates the representation while controlling for augmentation and retargeting strategy.

Full-context injection vs. channel concatenation: Table 2 shows Channel Concat-1.3B vs. SCAIL-1.3B: PSNR drops 0.39, FVD increases 35.01. Figure 9 provides the qualitative evidence for why: case (2) demonstrates that when the pose estimator produces an incorrect local pose (forward lunge during a running sequence), the full-context model recovers correct running posture from temporal context, while the channel-concat model reproduces the error. Case (1) shows that full-context injection with P-RoPE yields strongest disentanglement of identity from pose, preserving subject appearance while following motion. These qualitative examples operationalize the paper's theoretical claim that full-context injection enables "in-context motion reasoning" rather than per-frame alignment.

Pose-Shifted RoPE: Table 2 shows removal drops PSNR by 0.29 and increases FVD by 40.73. Figure 9 provides qualitative evidence: case (1) shows more accurate hand articulation and foot grounding with P-RoPE, and case (3) is referenced in the context of disentangling identity from pose. The paper's ablation note that P-RoPE yields "strongest disentanglement of character identity and pose guidance" is consistent with the design rationale: the shifted positional encoding prevents the model from confusing pose tokens with video tokens, improving appearance preservation (ID-Sim) while maintaining motion following.

3D augmentation (ablation in Appendix A2.2): For self-driven animation, the paper reports "no significant difference in the metrics compared to when augmentation was not used." This is a surprisingly positive finding: high-rate augmentation (0.8) does not degrade self-driven performance, meaning the model successfully learns that motion semantics are invariant under bone-length scaling. For cross-driven animation, a user study comparing SCAIL-1.3B with and without 3D augmentation (Figure A3) shows that augmentation "clearly enhances the metric of Physical Consistency and Identity Similarity": win rate for Phy-Consis is 41.7% (vs. 37.5% lose, 20.8% tie), and for ID-Sim is 39.2% (vs. 15.8% lose, 45.0% tie). The qualitative example in Figure A4 shows identity leakage without augmentation: a character with substantial body-shape differences from the driver exhibits proportion drift when augmentation is removed. This confirms the paper's claim that bone-scaling augmentation is essential for identity-agnostic motion transfer—without it, the model associates specific bone lengths with specific motions and fails when the reference character has different proportions.

3D retargeting vs. 2D retargeting (Appendix A2.2): Figure A2 visualizes the difference: 2D retargeting introduces unreasonable limb-length distortions because scaling operates in pixel space, while 3D retargeting preserves original limb proportions by only optimizing the camera projection. The paper notes that Studio-Bench "only include[s] cases where 2D Retarget works well for a fair comparison of the the model itself's performance against other baselines," but that "in wild scenarios, however, 3D information and camera parameters can help create highly robust retarget rules that are suitable for production-level use." This is an honest acknowledgment that the quantitative comparison on Studio-Bench is conservative (favorable to 2D baselines), and that the 3D retargeting advantage would be larger in truly unconstrained wild settings.

Architecture scaling (1.3B vs. 14B): While not presented as an ablation, the comparison between Table 1 (14B results) and Table 2 (1.3B results) provides implicit scaling evidence. The 14B model substantially outperforms the 1.3B on all metrics (quantitative self-driven metrics from Table 1 vs. Table 2, and the user study results are reported only for 14B), consistent with the expected benefits of scaling model capacity. The two-stage training for 14B (broad pretraining + targeted high-motion finetuning) versus single-stage training for 1.3B suggests that the 14B benefits from both increased capacity and the curriculum strategy, though the contributions of each are not disentangled.

Spatial pooling ratio: The paper notes that with 2× spatial downsampling, "the pose following ability is nearly unaffected" (Section 3.2). This is an efficiency-motivated finding rather than a full ablation, but it establishes that the full-context injection does not require full-resolution pose tokens, reducing the sequence length increase to approximately 1/4 of the video token count.

Data curation pipeline: The paper does not ablate the data curation pipeline directly—there is no comparison between a model trained on the curated dataset versus one trained on a generic, uncurated dataset of equal size. This is a notable missing ablation, as the paper's thesis is that curation is essential. The evidence is implicit: SCAIL's performance on complex Studio-Bench motions versus baselines' failures suggests that curation matters, but causal attribution to the curation pipeline cannot be isolated from the architectural contributions without a controlled experiment.

Critical Assessment

The experiments provide substantive evidence for the paper's central claims, though with important qualifications regarding scope, independence of contributions, and missing controlled comparisons. I assess each claim individually.

Claim 1: The 3D-consistent pose representation enables faithful complex-motion transfer that 2D keypoints and SMPL meshes cannot achieve.

What was tested: The 2D vs. 3D pose ablation on the 1.3B model (Table 2) directly demonstrates that switching from 3D to 2D pose degrades all quantitative metrics, with PSNR dropping 1.00 and FVD increasing by 66.74. The qualitative evidence (Figures A1, A2) shows specific failure modes of 2D pose (distorted limbs, front/back confusion) that 3D pose resolves.

What was not tested: There is no direct SMPL-based baseline implemented within the SCAIL architecture. The paper's claim that SMPL "causes severe identity leakage" is argued conceptually and supported by reference to prior work, but not experimentally validated under controlled conditions (same architecture, same training data, SMPL vs. cylindrical rendering). The comparison is only to existing SMPL-based methods (Champ) on different architectures, making it impossible to isolate representation quality from architectural differences. A controlled ablation training SCAIL with SMPL mesh rendering instead of cylindrical rasterization would have directly tested this claim.

Verdict: The advantage of 3D cylindrical rendering over 2D keypoints is well-supported. The advantage over SMPL meshes is theoretically motivated but not experimentally validated within the controlled SCAIL framework. The claim about SMPL is better characterized as a design argument than an experimentally verified finding of this paper.

Claim 2: Full-context pose injection enables spatiotemporal reasoning that per-frame channel concatenation cannot replicate.

What was tested: The channel-concat ablation on the 1.3B model (Table 2) shows full-context injection improves PSNR by 0.39 and reduces FVD by 35.01 compared to channel concatenation with the same 3D pose representation. Figure 9 provides the critical qualitative evidence: the model with full-context injection recovers correct motion from global context even when a local pose frame is incorrect (case 2), demonstrating "in-context motion reasoning."

What was not tested: The paper does not isolate whether the benefit comes from the full-sequence concatenation per se, or from the increased token count (more parameters attend to more tokens) or the explicit separation of pose tokens from video tokens. A design where the pose sequence is still concatenated but with a different attention mask (e.g., restricting pose attention to a local temporal window) could test whether global temporal access is specifically responsible for the improvement. The paper also doesn't test whether full-context injection helps uniformly across difficulty levels—the results are aggregated, making it impossible to determine whether the improvement is concentrated in complex motions (where temporal context is most needed) or distributed across all cases.

Verdict: The advantage of full-context injection over channel concatenation is convincingly demonstrated both quantitatively and qualitatively. The "in-context reasoning" interpretation (global context overrides local errors) is supported by the qualitative example but not systematically quantified—the frequency of local pose errors and the recovery rate with full-context injection would be needed to substantiate the strong claim that this represents a qualitatively different mode of operation. The ablation establishes that full-context injection improves performance; whether it does so through the specific mechanism of temporal-context-based correction (versus simply providing more information) is plausible but not proven.

Claim 3: The 3D-consistent adaptation strategy (bone scaling + camera optimization) decouples motion from identity, enabling robust cross-identity animation without motion distortion.

What was tested: The 3D augmentation ablation (Figure A3, A4) shows that removing bone-scaling augmentation degrades Physical Consistency and Identity Similarity in cross-driven animation, with the qualitative example (Figure A4) demonstrating identity leakage (character proportions drift toward driver). The 3D vs. 2D retargeting comparison (Figure A2) shows that 2D retargeting introduces limb-length distortion while 3D camera optimization preserves proportions.

What was not tested: The augmentation is ablated as a binary (on/off), but not at different rates. The paper uses a fixed 0.8 augmentation rate; whether 0.5 or 0.95 would work better (or whether the rate is robust) is unexplored. The camera optimization is not ablated against alternative alignment methods—the paper compares to 2D retargeting (heuristic skeleton scaling) but not to, e.g., direct affine alignment of 2D keypoints without scaling. Additionally, the augmentation ablation relies on a user study rather than automated metrics (since cross-driven evaluation lacks ground truth), and the win rates for Phy-Consis (41.7% win vs. 37.5% lose) and ID-Sim (39.2% win vs. 15.8% lose) are suggestive rather than statistically decisive—with 45.0% tie rate on ID-Sim, the reliable signal is weak.

Verdict: The experiments show that augmentation matters for cross-driven performance and that 3D retargeting avoids visible distortion that 2D retargeting introduces. However, the claim that the adaptation strategy fully "decouples motion from identity" is stronger than what the experiments support. The ID-Sim comparison against VACE (Table 1, 45.8% vs. 32.5%) shows SCAIL leads but not dominantly, and the Viggle Win/Tie/Lose on ID-Sim (63.3% win, 18.3% tie, 18.3% lose) suggests identity preservation is not perfect. The experiments demonstrate improved identity-motion decoupling, not complete decoupling.

Claim 4: SCAIL achieves state-of-the-art performance, advancing character animation toward production-level readiness.

What was tested: The quantitative comparisons (Table 1, Figure 6) show SCAIL-14B outperforms all open-source baselines across all metrics, and outperforms Viggle on naturalness metrics (Kin-Consis, Phy-Consis) while being competitive on motion accuracy. The qualitative results (Figures 7, 8, A6, A7, A8) show fewer artifacts, better structural preservation, and more plausible motion in challenging scenarios.

What was not tested: "Production-level readiness" is a high bar that implies robustness, reliability, and generalizability across the full range of production scenarios. The experiments are limited to Studio-Bench (250+ clips), which—while more challenging than prior benchmarks—is still a curated test set. The paper does not evaluate on truly in-the-wild data, user-submitted videos, or edge cases that would emerge in production use (unusual camera angles, extreme body shapes, characters with non-standard topology like wings or tails, very long sequences beyond 81 frames). The test set size (130 self-driven clips, 120 cross-driven pairs) is modest, and the absence of confidence intervals or statistical testing makes it impossible to assess whether the reported improvements are reliable or within the range of sampling variance. No human evaluation inter-annotator agreement metrics are reported.

Additionally, the paper only evaluates on its own proposed benchmark. While the authors argue prior benchmarks are insufficient, the absence of results on existing standard benchmarks (e.g., TikTok dataset, TED-talks, or the evaluation sets used by AnimateAnyone or UniAnimate-DiT) makes it impossible to compare SCAIL's performance to the broader literature except in the narrow context of Studio-Bench. This is a significant external validity limitation. The claim of "state-of-the-art" is relative to the specific baselines tested on Studio-Bench and cannot be generalized to all existing methods on all benchmarks.

A further limitation: the paper does not report failure cases systematically. The qualitative results showcase successes but do not characterize when or how SCAIL fails. The appendix mentions that "the accuracy of multi-person pose estimation is still not as precise as that for single-person scenarios" (Appendix A5.1) and that facial expression control is limited, but these limitations are not quantified. A production-readiness claim would be strengthened by a failure analysis: what percentage of generated clips contain noticeable artifacts, what motion types still cause problems, and what is the worst-case performance.

Verdict: SCAIL convincingly outperforms the compared baselines on the challenging cases in Studio-Bench, establishing it as the strongest method among those tested. However, "production-level readiness" is aspirational rather than demonstrated—the evaluation scope, test set size, and absence of failure analysis mean the paper shows capability on a rigorous benchmark, not production deployment robustness. The claim is better characterized as "substantial progress toward production-relevant performance on challenging benchmark scenarios."

Cross-cutting methodological concerns:

• Single base model family (Wan2.1): All results, including baselines, use the Wan backbone. The generalizability of the 3D pose representation and full-context injection to other DiT architectures (CogVideoX, Sora-style models) or U-Net architectures is untested. The underlying assumption—that Wan2.1 is "representative"—may not hold if specific properties of Wan's training or architecture interact with the proposed components.

• Missing data ablation: The most significant unaddressed confound is the data curation pipeline. The paper argues curation is essential, and SCAIL is trained on curated data while baselines may be trained on less carefully curated data. A model trained with the SCAIL architecture but on uncurated data (or a baseline trained on SCAIL's curated data) would disentangle data effects from architectural effects. Without this, the performance improvements attributable to the architecture versus the training data cannot be separated.

• No multi-person quantitative comparison: The multi-person results are solely qualitative (Figure 8). While the paper notes some baselines are incompatible with multi-character settings, the absence of quantitative metrics for multi-person animation means the claim about "inter-character spatial and occlusion relationships" is supported only by visual examples, not statistical evidence. The Win/Tie/Lose study (Figure 6) and Table 1 metrics are explicitly based on single-character pairs.

• Computational cost not factored into comparison: The full-context injection is 33% slower than channel concatenation at inference. While the paper argues this is acceptable for studio-grade scenarios, the performance comparisons in Table 1 do not normalize by compute cost. If a channel-concat baseline were allowed proportionally more diffusion steps to match SCAIL's inference time, the performance gap might narrow. The paper does not explore this tradeoff.

• User study methodological details are thin: No information on the number of participants, their expertise, inter-annotator agreement, or the number of evaluations per comparison is reported. The "cross-validation" across different participant batches is mentioned but not quantified (how many participants in each batch? Were demographics comparable?). These omissions make it difficult to assess the reliability of the human evaluation results.

Overall, the experiments provide strong evidence that the 3D pose representation and full-context injection improve performance on challenging character animation scenarios relative to the tested baselines. The ablation studies effectively isolate the contribution of individual components, with the 3D pose representation emerging as the single most impactful design choice. However, several central claims (identity-motion decoupling, in-context reasoning, production readiness) are supported with qualitative evidence that is suggestive but not systematically quantified, and important controlled comparisons (SMPL baseline, uncurated data baseline, multi-person metrics) are absent. The evaluation is rigorous within its defined scope but leaves open questions about generalizability, attribution of gains to data versus architecture, and statistical reliability.

6. Limitations and Trade-offs

The Cost of Difficulty Estimation Is Not Amortized

The paper does not implement a difficulty-estimation pipeline in the style of the example paper; however, SCAIL's pipeline contains an analogous unaccounted cost: the 3D pose extraction and retargeting inference overhead. The paper reports that full-context injection increases inference time by 33% (from 286.11 seconds to 380.78 seconds at 512×896 resolution, 81 frames, 20 steps on an H100 GPU; Appendix A1) and memory by 11% (61.7 GB to 68.5 GB). However, this accounting only covers the denoising step—it does not include the cost of the upstream pose extraction and retargeting pipeline:

• Multi-person segmentation (SAMURAI tracking + mask splitting): The paper explicitly notes that the default NLFPose extraction pipeline fails under occlusion and requires "a segmentation-and-extraction approach" using SAMURAI [45] to split multi-person videos into single-person clips before keypoint estimation. SAMURAI is a motion-aware memory-based tracker built on Segment Anything; its per-video cost is not characterized.
• Retargeting optimization ($\mathcal{L}_{\text{proj}}$): For cross-driven animation, the camera projection matrix is optimized per input pair via iterative minimization of the projection loss. The paper does not report the number of optimization steps, convergence time, or computational cost of this step.
• No difficulty-escalation mechanism: Unlike the example paper's difficulty-conditioned allocation, SCAIL applies the same full pipeline (3D extraction, retargeting if cross-driven, full-context denoising) to every input regardless of motion complexity. There is no mechanism to route simple motions through a cheaper pipeline (e.g., channel concatenation or 2D keypoints), meaning the full overhead is paid even when it provides minimal benefit.

Consequence: The headline inference cost of 380.78 seconds represents a lower bound. In cross-driven, multi-person scenarios (precisely the cases where SCAIL's advantages are most pronounced), the total pipeline cost—pose extraction, segmentation, retargeting optimization, and full-context denoising—could be substantially higher, and is not characterized. A practitioner evaluating SCAIL for production deployment cannot estimate end-to-end latency or throughput from the reported numbers, since the pose preprocessing cost is unmeasured. Furthermore, for simple motions where channel concatenation would suffice (as suggested by the smaller performance gap in the Channel Concat-1.3B ablation: PSNR 17.69 vs. 18.08, Table 2), the full pipeline is unnecessarily expensive.

Evidence: Appendix A1 reports only the denoising cost comparison (channel-concat: 286.11s, full-context: 380.78s). No end-to-end timing including pose extraction or retargeting is reported anywhere in the paper or appendix. The SAMURAI pipeline is described in Section 3.3 and the retargeting loss in Section 3.2, but neither is characterized in terms of runtime or memory.

Mitigation status: Not addressed. The paper does not mention this as a limitation and does not report end-to-end inference costs. The spatial pooling (2× downsampling of pose tokens) is presented as an efficiency optimization within the denoising step, but no comparable optimization is applied to the preprocessing stage.

---

Multi-Person Pose Estimation Remains a Fundamental Bottleneck

The paper acknowledges in the appendix:

"Although we have adopted a relatively effective multi-person pose extracting pipeline, the accuracy of multi-person pose estimation is still not as precise as that for single-person scenarios." (Appendix A5.1)

This is a structural limitation of the approach: SCAIL's 3D pose representation is only as good as the underlying 3D keypoint estimator (NLFPose), and the segmentation-based multi-person pipeline (SAMURAI → split → single-person NLFPose → compose) introduces its own failure modes. The paper reports that this custom pipeline "provides more accurate estimation than direct multi-person pose estimation methods like PromptHMR, especially in wild cases involving complex interaction" (Section 3.3), but "more accurate" is relative to an already-weak baseline, not an absolute guarantee of correctness.

Consequence: When pose estimation fails—misidentified limbs, swapped person identities, missing joints under heavy occlusion—SCAIL's full-context injection may be insufficient to compensate. Figure 9 case (2) shows an example where full-context injection recovers from a local pose error (left/right confusion during running), but this is a single-frame error in an otherwise correct sequence. Under persistent, multi-frame pose estimation failures—as the paper notes can occur in "heavy occlusions" where "they detect only the front character in certain frame" (Section 4.3)—the model may have no correct context to draw on. The worst-case behavior under severely degraded pose input is not characterized: does the model produce degraded but plausible output, or does it collapse entirely? The paper only shows recovery from isolated errors, not failure under systematic pose degradation.

Evidence: The appendix quote above is the only explicit acknowledgment. The qualitative results for multi-person scenarios (Figure 8) show successes but not failures; no systematic evaluation of pose estimation accuracy on the Studio-Bench dataset is reported, and no correlation between pose estimation error and generation quality is measured. The ablation study (Table 2) compares 3D vs. 2D pose representations but does not vary the quality of the 3D pose estimate itself (e.g., by adding noise or occlusions).

Mitigation status: The paper explicitly defers this to future work: "we still look forward to advances in the field of multi-person pose estimation to further improve the fidelity of motion replication" (Appendix A5.1). No model-level robustness mechanism (e.g., pose uncertainty conditioning, fallback strategies) is implemented; the model is trained on extracted poses and trusts them at inference time.

---

Evaluation Limited to a Single, Self-Constructed Benchmark on a Single Model Family

All quantitative results are on Studio-Bench, proposed and constructed by the authors. The paper explicitly excludes existing standard benchmarks:

"Previous evaluations primarily focus on simple actions, which fail to capture the challenges present in film production" (Section 3.4).

While Studio-Bench is more challenging, its exclusive use means:

• No comparison to the broader literature on standard benchmarks: Methods that report results on TikTok, TED-talks, or the evaluation protocols of AnimateAnyone or UniAnimate-DiT cannot be directly compared to SCAIL. The paper's claim of "state-of-the-art" (Section 5) is relative only to the four baselines tested (UniAnimate-DiT, VACE, Wan-Animate, Viggle), not to the full space of published methods.
• All compared methods use the Wan2.1 backbone: Table 1 notes that "all compared methods are built upon 14B Wan foundation models." This controls architecture but means the results cannot distinguish whether SCAIL's improvements generalize to other DiT families (CogVideoX, Sora-style) or U-Net architectures.
• Benchmark construction risk: Studio-Bench's motion distribution (Figure A5: dance, fitness, martial arts, acrobatics, basic motions, interaction) reflects the authors' definition of "studio-grade." A practitioner with a different motion distribution (e.g., conversational gestures, sign language, subtle facial animation) cannot infer performance from the reported results.
• Small test set: 130 self-driven clips and 120 cross-driven single-character pairs provide limited statistical power. The absence of confidence intervals or standard deviations on the quantitative metrics makes it impossible to assess whether the reported improvements (e.g., PSNR 19.22 vs. 18.54 for Wan-Animate) are statistically reliable.

Consequence: The performance characterization is simultaneously the paper's strongest contribution (a challenging, production-relevant benchmark) and its primary external validity limitation. A deployment decision cannot be made solely on Studio-Bench results; the method's performance on the target application's motion distribution is unknown without additional evaluation. The single-backbone constraint means the relative contribution of the Wan2.1 architecture versus SCAIL's specific innovations is confounded: if Wan2.1's pretraining is particularly well-suited to the curation pipeline, results may not transfer.

Evidence: The paper never reports results on any external benchmark. The Studio-Bench test sizes (130, 120) are stated in Section 3.4. No standard deviations, confidence intervals, or statistical tests are reported for any metric in Table 1, Table 2, or Figure 6. The user study methodological details (number of participants, inter-annotator agreement) are absent.

Mitigation status: Not addressed. The paper does not acknowledge the single-benchmark limitation or the absence of results on standard benchmarks. The construction of Studio-Bench is framed entirely as a positive contribution (filling an evaluation gap), without noting the tradeoff in comparability.

---

Data Curation Effect Is Not Separated from Architectural Innovation

The paper devotes substantial attention to its data curation pipeline (Section 3.3, Figure 5): YOLO-based character filtering, DWPose-based body completeness filtering, motion-speed-based low-motion removal, SAMURAI-based multi-person segmentation, VLM caption verification, and a final manual inspection yielding a 4,000-clip high-dynamic finetuning subset. The paper's thesis is explicit:

"data quality and relevance, not just quantity, determines performance on challenging cases" (Section 4 in the prior analysis).

However, no ablation isolates the effect of data curation from the effect of architecture. All baselines compared in Table 1 and Figure 6 are trained on their original (non-SCAIL-curated) data. The SCAIL models (both 1.3B and 14B) are trained on the curated dataset with deliberate inclusion of stylized characters, multi-person interactions, and high-motion clips.

Consequence: The performance gap between SCAIL and baselines cannot be attributed to the 3D pose representation and full-context injection alone. It is confounded with training data quality and distribution. A baseline (e.g., Wan-Animate or VACE) retrained on SCAIL's curated dataset might close some or much of the gap, particularly on metrics like Identity Similarity (where VACE, at 32.5%, outperforms Wan-Animate's 20.0% in Table 1, suggesting training data differences rather than architecture alone may drive identity preservation). The paper's central architectural claims—that 3D cylindrical rendering and full-context injection are necessary for studio-grade performance—would require showing that these components improve performance holding data constant.

Evidence: No data ablation is reported anywhere in the paper or appendix. The training data sources are described (internal base model data, Bilibili/YouTube dance videos, sports videos; Appendix A3), but no model trained on uncurated data with the SCAIL architecture is compared. The finetuning subset ablation (4,000 high-dynamic clips for the 14B second stage) is not evaluated as an independent variable—the paper does not report 14B performance with and without the finetuning stage.

Mitigation status: Not addressed. The paper treats the curation pipeline as an integral part of the SCAIL framework rather than a variable to be controlled. The claim that curation "is as important as architecture" (implied by the depth of description in Section 3.3) is asserted but not experimentally validated within the paper's own controlled comparisons.

---

Cross-Driven Animation Lacks Automated Metrics; User Study Reliability Is Underreported

The Cross-Driven subset of Studio-Bench (the 120 single-character pairs used for quantitative evaluation in Table 1) has no ground-truth videos—the reference image and driving video come from different sources by construction. Consequently, standard video quality metrics (PSNR, SSIM, LPIPS, FVD) cannot be computed for cross-driven evaluation. All cross-driven results in Table 1 rely entirely on human evaluation (Motion Accuracy, Kinesiology Consistency, Physical Consistency, Identity Similarity).

Consequence: The reported cross-driven metrics have unknown statistical reliability. No information is provided about:

• Number of participants or evaluations per comparison: The Win/Tie/Lose study (Figure 6) and the best-model-selection study (Table 1) used "different batches of participants" (Section 4.2), but participant counts are not reported.
• Inter-annotator agreement: No metric (e.g., Krippendorff's alpha, Fleiss' kappa, raw agreement rate) is reported. If annotators disagreed substantially, the percentages in Table 1 reflect averaging over noisy individual judgments rather than a reliable quality signal.
• Annotator expertise: Studio-grade character animation evaluation may require domain knowledge (identifying anatomically implausible poses, assessing physical consistency). Whether annotators were trained, had relevant expertise, or were drawn from a general population is not stated.
• Statistical significance: No confidence intervals, error bars, or hypothesis tests are reported for any human evaluation metric.

The "cross-validation" claim—that using different participant batches for Table 1 and Figure 6 provides cross-validation—is weak: if both batches were drawn from the same population with the same biases, the results would be correlated by design. Cross-validation in the statistical sense requires independence, which different participant batches provide, but without reporting agreement within each batch, the reliability of either batch individually cannot be assessed.

Evidence: Section 4.2 describes the metrics and notes the cross-validation design, but provides no quantitative details about the user study methodology. The raw percentages in Table 1 and Figure 6 are reported without error characterization.

Mitigation status: The paper acknowledges the need for user studies (given the absence of ground truth for cross-driven evaluation) and implements a cross-validation design, which is a reasonable attempt at rigor. However, the absence of standard user study reporting practices (participant count, agreement metrics, confidence intervals) means the cross-driven results cannot be evaluated for statistical reliability. This is a methodological weakness that undermines the strongest quantitative claims (e.g., 53.3% Mot-Acc, 55.8% Kin-Consis), particularly since the absolute differences between methods, while visually large, are untested for significance.

---

The ~14× Scaling and Compute-Tradeoff Regime Analysis Is Absent

The paper evaluates two model scales (1.3B and 14B, approximately an order of magnitude apart) but does not explore the tradeoff between model scale and test-time compute. The 14B model substantially outperforms the 1.3B on all metrics (compare Table 1 vs. Table 2: the 14B's FVD of 176.16 vs. the 1.3B's 228.62, though note these are evaluated on different dataset subsets and metrics). However, the paper provides no scaling analysis: at what point does additional model capacity saturate? Could a 7B model with more inference budget match the 14B? Does the full-context injection benefit scale with model size, or is it architecture-dependent?

This is a consequential omission for two reasons:

1. Practitioner deployment decisions: The 14B model requires 128 H100 GPUs for training and 68.5 GB for inference (Appendix A1). A practitioner with a 7B-capable deployment budget cannot determine from the paper whether the SCAIL approach would still provide benefits at that scale, or whether the 14B model's performance is necessary for studio-grade results.

2. The pretraining-vs-inference tradeoff (analogous to the example paper's Section 7) is entirely unexplored. Would a SCAIL-finetuned 14B model with full-context injection outperform a hypothetical SCAIL-finetuned 20B model with channel concatenation? Would spending additional inference compute (more diffusion steps, longer sequences) on a smaller SCAIL model match the larger model's performance? The paper's focus on architectural comparison at fixed scale leaves these deployment-critical questions unanswered.

Evidence: The paper reports two model sizes (Section 4.1, Tables 1 and 2) but does not ablate or analyze the effect of scale. No scaling curve (performance vs. parameters or performance vs. inference FLOPs) is presented. The inference cost comparison (Appendix A1) is reported only for the 14B model, preventing any efficiency-vs-quality analysis across model sizes.

Mitigation status: Not addressed. The paper does not frame this as a limitation or suggest future scaling analysis. The two model sizes are presented as implementation variants (1.3B for ablation, 14B for main results) rather than as data points in a scaling study. This is a gap in the practical deployment guidance the paper could provide, particularly given the paper's production-oriented framing.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the conversation around character animation from architecture-first engineering to representation-first diagnosis. Prior work treated pose signal extraction as a preprocessing detail—plug in DWPose or ViTPose, embed the 2D keypoints, and focus design effort on the denoising backbone (U-Net vs. DiT, adapter modules, ReferenceNet variants). The underlying assumption was that generation architecture was the primary performance bottleneck, and that a sufficiently powerful model could overcome limitations in its conditioning signal. SCAIL's core contribution is demonstrating that this assumption is false: the pose representation itself is the rate-limiting factor, and architectural sophistication cannot compensate for an ambiguous, depth-less, identity-contaminated motion signal.

This is a diagnostic reframing rather than a paradigm shift. The technical building blocks—3D keypoint estimation, cylindrical rasterization, DiT-based video diffusion, sequence-level concatenation—are individually well-established. The paper's intellectual move is in which problems it identifies as root causes and how it designs representations specifically to address those causes. Specifically, it establishes three design requirements for a motion representation that prior work had not articulated as a coherent set: (1) encode depth ordering and occlusion explicitly (so the model does not need to infer 3D structure from 2D evidence), (2) remain independent of any specific character's body shape (so the motion signal does not contaminate the appearance signal), and (3) support augmentation and retargeting through motion-preserving transformations only (rigid projections and bone-length scaling, never deformations). The cylindrical 3D rendering plus camera-optimization retargeting is a specific instantiation of these principles; the principles themselves are the more transferable contribution.

The work also resolves a latent tension in prior results: why some methods (using 2D keypoints) appeared competitive on standard benchmarks while failing dramatically on complex motions, and why SMPL-based methods offered better motion accuracy but suffered identity leakage. The answer, per this paper, is that neither representation satisfies all three requirements—2D keypoints fail depth encoding (requirement 1) and motion-preserving retargeting (requirement 3); SMPL meshes fail identity independence (requirement 2). The field's contradictory experience with these representations was an artifact of testing on different difficulty distributions. SCAIL's Studio-Bench is specifically designed to expose these failure modes by testing under distribution shifts (complex motions, cross-identity, multi-person, stylized characters) that standard benchmarks do not cover. This means the paper not only proposes a new method but also provides the diagnostic tool (Studio-Bench) to determine whether future methods genuinely address the root causes or merely overfit to easy cases.

Which research directions become more attractive:

• Pose representation design as a first-class research problem. Before this paper, pose representation was a commodity. After this paper, it is demonstrably the primary design choice governing motion transfer fidelity (the 1B-parameter ablation in Table 2 shows the 3D-to-2D switch causes the single largest degradation: PSNR −1.00, FVD +66.74). This redirects research attention from denoising architecture toward representation design—new ways of encoding depth, occlusion, and multi-person relationships that satisfy the three requirements while being cheaper to extract and render than cylindrical rasterization.

• Verifier and evaluation design for motion quality. The paper's user study metrics—Kinesiology Consistency, Physical Consistency, Identity Similarity—formalize dimensions of motion quality that pixel-space metrics (PSNR, SSIM) cannot capture. The 85.0% win rate on Physical Consistency against Viggle (a 3D-foundation-model baseline that is competitive on raw motion accuracy) demonstrates that motion naturalness and motion accuracy are distinct properties requiring separate evaluation. This suggests that the field should develop automated proxies for these human-judged dimensions, analogous to how LPIPS automated perceptual similarity, to enable faster iteration.

• Data curation as an experimental variable, not an engineering detail. The paper's explicit thesis that curation determines performance on challenging cases is asserted but not ablated, making it simultaneously the most provocative methodological claim and the most significant unaddressed confound. Future work that demonstrates (or refutes) this claim—by training baselines on SCAIL's curated data or SCAIL on baseline data—would either elevate data curation to a first-class research contribution or redirect credit from data to architecture.

Which directions become less attractive:

• Incremental improvements to 2D keypoint quality. The paper's ablation shows that switching from 3D to 2D pose under an otherwise identical architecture causes a 1.00 PSNR drop. This suggests 2D keypoint estimation quality improvements (better keypoint detectors, more keypoints, confidence-weighting) are band-aids on a fundamentally limited representation: no 2D representation can encode depth ordering, and 2D skeleton scaling will always be a deformation, not a motion-preserving transformation. Resources are better spent on improving 3D keypoint estimation and developing new 3D rendering strategies.

• Adapter-based and cross-attention-based pose injection. The paper's full-context concatenation approach adds no new parameters beyond a patchify layer, while prior adapter-based methods (Pose Guider, residual context-tuning) add trainable adapter networks. The ablation shows full-context injection outperforms channel concatenation, and the architectural simplicity is a secondary advantage—it suggests that providing pose as sequence-level context to the existing self-attention is both more effective and more efficient than designing specialized cross-attention or adapter pathways. Complex injection architectures that don't provide full temporal context become harder to justify.

• SMPL as the default 3D representation. The paper's critique of SMPL—"severe identity leakage" from strong body shape priors—is argued conceptually but not tested as a controlled ablation. Nevertheless, the cylindrical rasterization approach demonstrates that you can get 3D structure benefits (depth, occlusion) without the identity contamination that parametric body models introduce. This suggests the field should move away from SMPL as the default 3D representation and toward identity-agnostic alternatives, at least for cross-identity animation.

---

Follow-Up Research This Work Enables

Ablating data curation against architecture under controlled conditions. The paper's strongest untested claim is that data curation is as important as architecture for studio-grade performance. A direct follow-up would train two models: (1) SCAIL architecture on a generic, uncurated dataset of comparable size to the curated set, and (2) a baseline architecture (e.g., Wan-Animate or VACE) on SCAIL's curated dataset, both evaluated on Studio-Bench. If (1) substantially underperforms (2), the curation thesis is supported and data becomes a first-class research contribution; if (2) substantially outperforms (1), the architectural innovations are confirmed as the primary driver; if both perform similarly, the two factors are substitutable and the field should optimize whichever is cheaper. This experiment would also clarify the attribution of SCAIL's performance gains between data and architecture, which the current paper does not disentangle. The test would require releasing the curated training set or providing sufficient detail for exact reproduction.

Correlating pose estimation error with generation quality to establish a verifier reliability frontier. The paper shows qualitatively (Figure 9, case 2) that full-context injection can recover from isolated pose estimation errors, but does not characterize the relationship between pose error magnitude and generation degradation. A systematic study would measure: (1) the per-frame 3D keypoint estimation error of NLFPose on a densely-annotated subset of Studio-Bench (e.g., using multi-view capture or manual annotation as ground truth), (2) the correlation between that error and generation quality metrics (FVD, user-study scores) at different error magnitudes, and (3) whether full-context injection's recovery capability has a threshold beyond which errors become uncorrectable. This would operationalize the paper's implicit hypothesis that full-context injection provides robustness up to some error level, and would establish a "verifier reliability frontier" analogous to the example paper's finding that PRM over-optimization limits search scaling. It would also identify whether multi-person pose estimation quality is the current bottleneck (as the appendix suggests) and, if so, at what error rate multi-person generation breaks down.

Replacing cylindrical rasterization with alternative depth-encoding representations. The paper's cylindrical 3D rendering is one specific instantiation of its three design requirements. Are there cheaper or more informative alternatives? Candidates include: (a) depth-map rendering where each pixel encodes the z-distance of the nearest bone rather than binary occupancy, providing richer depth information at similar rendering cost; (b) volumetric density fields that encode per-voxel bone probability rather than hard geometry, potentially providing smoother gradients for the model; (c) multi-view rendered skeletons that provide explicit 3D information from two or three canonical viewpoints (front, side) without requiring the model to infer 3D from a single projection; (d) learned neural representations that compress the 3D skeleton into a latent code and decode to conditioning tokens, trading some interpretability for potential compactness and robustness to estimation noise. A strong follow-up would implement 2–3 of these alternatives within the same SCAIL architecture (replacing only the rasterization step), train on the same curated data, and evaluate on Studio-Bench to determine whether the cylinder representation is optimal or just the first working example of a broader class of depth-encoding representations. The key metric would be performance at equal rendering cost, since the paper emphasizes that the Taichi-based ray marching introduces "negligible computational overhead."

Testing whether full-context injection enables zero-shot temporal reasoning or merely better local alignment. The paper interprets the channel-concat ablation (Table 2) and qualitative Figure 9 as evidence that full-context injection enables the model to reason about global motion semantics. However, the mechanism could be simpler: maybe the model is just attending to nearby pose frames to resolve local ambiguities (a 3–5 frame window would suffice for the turning example) rather than reasoning about the full sequence. A diagnostic experiment would restrict the temporal attention window for pose tokens: train and evaluate variants where pose tokens at frame $t$ can only attend to pose tokens within $\pm k$ frames, for $k \in \{1, 3, 5, 10, \text{unlimited}\}$. If performance saturates at small $k$ (e.g., $k=5$ matches unlimited), the benefit is local temporal disambiguation, not global motion reasoning. If performance continues improving with larger $k$ and particularly on long-range motion dependencies (e.g., motions where the beginning of a sequence disambiguates the end, such as a dance routine where the first move determines the style of later moves), the paper's "in-context learning" framing is supported. This experiment would also have practical implications: if small $k$ suffices, the sequence length increase from full-context injection could be reduced by restricting attention rather than concatenating the full sequence, recovering some of the 33% inference-time overhead.

Extending to domains with different occlusion and deformation patterns. SCAIL is evaluated exclusively on human (and humanoid/anthropomorphic) characters. How well does the 3D cylindrical representation + full-context injection transfer to: (a) quadruped or multi-limb animation (horses, dogs, dragons), where the skeletal topology differs and limb occlusion patterns are more complex; (b) non-skeletal deformation (cloth, hair, soft-body characters), where the cylindrical bone model is a poor fit and alternative 3D representations (mesh, point cloud, neural fields) may be needed; (c) multi-object interactions with occlusion (a character holding and manipulating an object), where the representation must encode both the character's skeleton and the object's geometry? For (a), the existing pipeline could be tested with minimal modification by using animal pose estimators and appropriate skeletal topology; the key question is whether the advantages of 3D over 2D pose transfer. For (b), a fundamentally different representation would be needed, testing the limits of the paper's "representation design is the primary bottleneck" thesis—if the bottleneck shifts from representation to architecture when deformation is complex, that clarifies the scope of the paper's contribution.

Automated metrics for Kinesiology Consistency and Physical Consistency. The paper's user study metrics capture dimensions of motion quality that standard automated metrics do not: anatomical feasibility, temporal coherence of joint rotations, physical constraint compliance. A strong follow-up would develop automated proxies: (a) train a classifier on the user study judgments (SCAIL vs. baselines) to predict Kin-Consis and Phy-Consis scores from generated videos, using the paper's own user study data as labels; (b) use a pretrained 3D human pose estimation model to extract joint trajectories from generated videos and compute biomechanical validity metrics (joint angle ranges, velocity smoothness, ground contact detection); (c) correlate these automated metrics with the user study scores to validate them. If successful, these automated metrics would enable much faster iteration on animation methods (no need for user studies during development) and could be released as part of Studio-Bench to standardize evaluation. The paper's cross-validation design (different participant batches for Table 1 and Figure 6) provides a starting point, but the raw user study data would need to be released or recollected at scale.

---

Practical Applications and Downstream Use Cases

Character animation for independent and mid-size production studios. The explicit framing throughout the paper is "studio-grade" animation for "film production." The practical value proposition is compressing the motion-capture → rigging → rendering pipeline into a single video-driven step. For independent studios that cannot afford motion capture hardware, SCAIL enables a workflow where a reference character image (concept art, character design) is animated using a driving video of a human performer. The 53.3% motion accuracy and 55.8% kinesiology consistency scores (Table 1), while not perfect, represent substantial progress over open-source alternatives that fall below 35% on these metrics. The 85.0% win rate on Physical Consistency against the commercial Viggle product (Figure 6) suggests SCAIL produces fewer physically implausible artifacts than a competing commercial tool, which is critical for production where retakes are expensive. The 33% inference overhead over channel concatenation (380.78 vs. 286.11 seconds for 81 frames) is acceptable in an offline rendering context where quality matters more than latency. A studio could batch-process animations overnight, achieving near-production quality without motion capture hardware.

Data augmentation for training pose-conditioned models in style transfer and virtual try-on. The paper's data curation pipeline—particularly the 3D keypoint extraction, bone-scaling augmentation, and camera perturbation—produces diverse, high-quality video-pose pairs with controlled variation in body shape and viewpoint. This pipeline could be adapted to generate training data for related tasks: (a) virtual try-on where a garment must deform according to driving pose, benefiting from the same 3D-consistent pose signal; (b) style transfer for character animation where the reference image is a stylized rendering (anime, painting) and the driving video is real—SCAIL's explicit inclusion of "3D and 2D animations, MMD, Live2D" in training data (Appendix A3) means the model already handles such cross-domain transfers; (c) motion retargeting for non-human characters (cartoon animals, mascots) where bone-scaling augmentation simulates the extreme body-shape differences. The curation pipeline's motion-speed filtering (Equation 5: $v = \frac{1}{T} \sum_{t=1}^{T-1} \sum_j \|\bar{J}_{t+1,j} - \bar{J}_{t,j}\|_2$) provides a domain-agnostic way to filter for useful motion complexity regardless of the specific character type.

Previsualization for choreography and stunt planning. In film production, choreographers and stunt coordinators often need to visualize how a complex motion will look when performed by a character whose appearance differs from the performer. SCAIL's cross-driven animation capability (120 single-character and 10 multi-character test pairs) directly enables this: a stunt coordinator films a reference performance, and SCAIL generates an animation of the actual character performing that motion. The multi-person capability is particularly valuable here—a dance scene with two characters can be previz'd from two separate performer videos composed into the same frame. The qualitative results (Figure 8) showing SCAIL handling heavy occlusion and distinguishing character identities in multi-person dance scenarios suggest it is already functional in this regime, though the appendix's caveat that multi-person pose estimation "is still not as precise as that for single-person scenarios" means choreography with tight physical contact may still produce artifacts. The 33% inference overhead is negligible in a previsualization context where a few iterations are sufficient.

Synthetic data generation for training character-consistency models. An emerging line of research trains models to maintain character consistency across video frames—the "who is who" problem in multi-character tracking and generation. SCAIL's ability to generate temporally coherent videos with explicit character identity (via hue-coded skeletons) and preserved appearance (via reference image conditioning) makes it a potential tool for generating synthetic training data for such models. Unlike real video data, SCAIL-generated videos come with perfect ground truth: the 3D skeleton provides exact joint positions and person identities, and the reference image provides the target appearance. A dataset of SCAIL-generated videos with automatically labeled identities, occlusions, and joint positions could train character-consistency and person-reidentification models without expensive manual annotation. The key requirement is that generated videos are realistic enough to transfer to real video—the paper does not evaluate this directly (all evaluation is on Studio-Bench, not on downstream task transfer), but the "studio-grade" quality claims suggest this may be viable for controlled production settings.

---

When to Prefer This Method Over Alternatives

The paper does not articulate an explicit decision rule or systematic tradeoff against named alternatives outside of the experimental comparisons. The baselines (UniAnimate-DiT, VACE, Wan-Animate, Viggle) are compared quantitatively and qualitatively, but the paper does not specify a deployment heuristic of the form "prefer SCAIL when condition X holds, prefer baseline Y when condition Z holds." The closest the paper comes to such guidance is the observation that full-context injection's 33% inference overhead is "acceptable, particularly in studio-grade scenarios which prioritize stringent accuracy and stability" (Appendix A1), implying that for latency-sensitive applications, channel concatenation may be preferred, but this tradeoff is not systematically characterized. As such, a formal decision matrix would be a fabrication of generic boilerplate rather than a faithful representation of the paper's claims. The paper's contribution is better characterized as a universal improvement (on its evaluation benchmark) rather than a method with clearly delineated preference regions against specific alternatives.