Architecture2025~117 minarXiv:2505.04512

HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation

score 3Teng Hu, Zhentao Yu, Zhengguang Zhou +4Tencent Hunyuan↑ 36★ 1,209May 7, 2025cs.CVhttps://github.com/Tencent/HunyuanCustom

🎯 Pitch

Current video generators can’t reliably show your dog or your product—identity breaks down across frames. HunyuanCustom solves this by fusing LLaVA-based image understanding with a temporal identity injection trick, then layering on separate audio and video controls, creating the first system that keeps faces and objects recognizably intact even when you’re remixing them with music, new backgrounds, and multi‑subject prompts.

1. Executive Summary

HunyuanCustom introduces a multi-modal customized video generation framework built on HunyuanVideo that emphasizes subject-consistent generation across text, image, audio, and video conditioning modalities. The paper proposes a text-image fusion module based on LLaVA for enhanced multi-modal understanding (replacing description tokens in text prompts with image features) and an image ID enhancement module that leverages temporal concatenation to propagate identity features across video frames, along with modality-specific injection mechanisms — AudioNet (spatial cross-attention with temporally aligned audio features) for audio-driven generation and a video-driven injection module (patchify-based feature alignment and latent addition) for video-conditioned editing — to achieve decoupled multi-modal control while preserving subject identity. Evaluated against both commercial products (Vidu 2.0, Keling 1.6, Pika, Hailuo) and open-source methods (SkyReels-A2, VACE), HunyuanCustom achieves best-in-class ID consistency (Face-Sim) and subject similarity (DINO-Sim) across single-subject, multi-subject, audio-driven, and video-driven video customization tasks, establishing that strong identity preservation and multi-modal controllability can be jointly realized only when identity injection (LLaVA + temporal concatenation) is architecturally disentangled from audio and video condition pathways.

2. Context and Motivation

The Core Problem: Customized Video Generation Is Simultaneously Under-Controlled and Overly Narrow

At its heart, this paper addresses a fundamental tension in the rapidly maturing field of video generation. Modern video generation models — whether open-source frameworks like CogVideoX, Wan, and HunyuanVideo, or commercial systems like Sora and Keling — can produce remarkably high-quality, diverse videos from text prompts. But they share a critical deficiency: they offer users almost no precise control over what specific entities appear in the generated video. If you prompt "A golden retriever playing frisbee in a park," the model will generate some dog — but not your dog. If you need a video of your specific product being demonstrated by your specific spokesperson, existing text-to-video models provide no mechanism to constrain the output to those particular identities.

This capability — known in the literature as customized video generation or subject-consistent video generation — is fundamentally different from text-to-video generation. It requires the model not merely to understand a text description, but to extract, encode, and faithfully reproduce the specific visual identity of one or more input subjects throughout a generated video, while simultaneously adhering to text-described actions, scenes, and interactions. The tension is that these two requirements — identity preservation and text-driven content generation — are often in conflict. Aggressively injecting identity information can cause the model to copy-paste the reference image statically (the "copy-paste problem" the paper identifies in Keling's outputs), while weak identity injection causes the subject's appearance to drift or morph across frames.

This paper argues that the real problem goes deeper: existing customized video generation methods are not just flawed in implementation — they are architecturally limited in scope. They typically support only a single conditioning modality (an image of the subject) and a single task (reproducing that subject in a generated video). Real-world video production, however, is inherently multi-modal. A virtual human advertisement needs the spokesperson's identity (image), their speech (audio), and the product they're demonstrating (another image, possibly inserted into a specific scene context). A video editing workflow needs the source video (video), masks indicating what to replace (video), and reference images of what to insert (image). A singing avatar needs identity (image) and synchronized singing (audio), in a scene that may differ from the reference image (text). None of these workflows are supported by existing single-modality methods.

The paper's framing of this gap is worth examining carefully. They are not merely proposing incremental improvements to any one task — rather, they identify a structural hole in the field: the absence of a unified architecture that can handle arbitrary combinations of image, audio, video, and text conditioning while preserving subject identity across all of them. This is what they term multi-modal identity-centric video customization, and the key technical challenge is that the identity injection mechanism must be architecturally disentangled from the audio and video conditioning pathways so that each modality can be independently controlled without interference.

Why This Problem Matters: Applications That Were Previously Impossible

The paper motivates its work through a series of concrete applications (Section 1 and elaborated in Sections 5.3–5.5) that reveal the practical limitations of prior systems:

Virtual human advertising. Companies increasingly want to create personalized video advertisements where a specific human spokesperson interacts with a specific product. Prior systems could either animate a static face (coarse, headshot-level only, no scene variation) or generate videos of generic humans without identity constraints. Neither approach produces an advertisement-quality video of a specific person handling a specific product in a controlled scene. HunyuanCustom's multi-subject customization capability (Section 5.3, Figure 7) directly targets this gap by accepting both a human reference image and a product reference image, then generating a video where the identified person interacts naturally with the identified product.

Virtual try-on with audio. Online fashion retailers want to show a shopper how a specific garment would look on them, speaking naturally in a realistic scene. Prior audio-driven animation systems — the paper cites Loopy (Jiang et al., 2024) and SONIC (Ji et al., 2024) — take a single person image and audio, then generate a talking-head video where the person's posture, clothing, and background are frozen identically to the input image. This is animation, not generation: you cannot change the person's outfit, pose, or environment. HunyuanCustom's audio-driven multi-subject customization (Section 5.4, Figure 9) accepts a reference person, a reference garment, text describing the scene, and audio — and generates a video where the identified person wears the identified garment, speaks the audio, in the described scene.

Video subject replacement and insertion. Video editors routinely need to replace one object with another in existing footage — for example, swapping one vehicle model for another in a car commercial, or inserting a product into a previously shot scene. This is a highly constrained task: the background, camera motion, lighting, and occlusion patterns from the source video must be preserved exactly, while the target identity must integrate seamlessly. As the paper demonstrates in Section 5.5 (Figure 10), prior approaches either produce boundary artifacts by rigidly following input masks (VACE) or produce copy-paste effects where the new subject appears as a floating overlay rather than an integrated part of the scene (Keling).

These applications are not theoretical — they represent existing commercial workflows that currently require expensive manual video production. The paper's framing implies that automated customized video generation with identity preservation could dramatically reduce production costs and enable personalization at scale that is simply not feasible with current techniques.

Where Prior Approaches Fall Short

The paper's related work section (Section 2) provides a structured taxonomy of prior approaches, and the limitations it identifies are specific and technical enough that they deserve careful exposition.

Instance-Specific Customization: The Per-Identity Fine-Tuning Paradigm

The earliest customized generation approaches — Textual Inversion (Gal et al., 2022) and DreamBooth (Ruiz et al., 2023) — originated in the image domain. The core idea is: for each specific subject you want to generate, you collect several reference images of that subject and fine-tune the pretrained model (or learn a special text token embedding) such that the model associates the subject's identity with a unique text token (e.g., "a [V] dog" for your specific dog). Then, at inference, including that token in the prompt causes the model to generate images of that specific subject.

Extensions to video — Still-Moving (Chefer et al., 2024), CustomCrafter (Wu et al., 2025), CustomVideo (Wang et al., 2024b), DisenStudio (Chen et al., 2024) — adapted this paradigm in various ways. Still-Moving first fine-tunes to generate static frames, then fine-tunes again on those static frames to produce motion. CustomCrafter repeats the reference image as a static video and embeds it into the text space via Textual Inversion before fine-tuning the video model. DisenStudio associates each subject with a specific spatial attention region so that different subjects can receive different actions.

The fundamental limitation is right there in the description: every new identity requires a separate fine-tuning run. For a production system that needs to handle arbitrary new users, products, or scenes on demand, this is a non-starter. Fine-tuning a large video generation model is computationally expensive, takes hours of GPU time per identity, and requires multiple high-quality reference images of each subject. The paper is explicit: "these methods rely on instance-specific optimization, which poses a challenge in real-time or large-scale video customization" (Section 2.2). This is a critical distinction — the paper's target is end-to-end customization, where the model generalizes to arbitrary new identities at inference time with no per-identity optimization.

End-to-End Customization, But Only for Faces

A second wave of work moved toward end-to-end (zero-shot, training-free-at-inference) customization by training a separate conditioning network that injects identity information into the video generation model. The paper specifically cites three such works:

ID-Animator (He et al., 2024): Introduces a face adapter module and a facial identity loss during training. The face adapter extracts identity features from a reference face image and injects them into the video generation model's feature space, allowing the generation of videos containing that identity. But it only works for faces — it has no mechanism for preserving non-facial details (clothing, body shape, object appearance).
ConsisID (Yuan et al., 2024): Decomposes facial information into low-frequency (global structure, face shape) and high-frequency (texture details, skin tone) components extracted by separate encoders, enabling more comprehensive face identity capture. Still face-only.
MovieGen (Polyak et al., 2025): Injects facial identity into the text space and additionally uses facial images from different source videos to guide generation, which helps alleviate facial copying (where the model simply reproduces the exact reference image rather than generating the face in a new pose). Still face-only.

The paper's critique of these methods is pointed: they "specialize in generating videos for a single human ID" and "are unable to handle arbitrary objects" (Section 1). If you want to customize any non-face subject — an animal, a vehicle, a product, a building — these methods provide no mechanism to do so. This is a severe restriction given the commercial applications the paper targets.

Multi-Subject Customization in Existing End-to-End Methods

The most recent generation of work — ConceptMaster (Huang et al., 2025), Video Alchemist (Chen et al., 2025), Phantom (Liu et al., 2025), SkyReels-A2 (Fei et al., 2025), and VACE (Jiang et al., 2025) — extends end-to-end customization to handle multiple subjects by binding text prompt tokens to corresponding reference images. The idea is that you assign a description word in the text prompt to each reference image (e.g., the word "man" corresponds to a specific man's image, the word "ball" corresponds to a specific ball's image), and the model learns to route identity information for each subject to the region of the video where that subject appears.

While these methods represent progress toward the paper's goals, the authors identify three specific failure modes:

Identity consistency degradation in multi-subject scenarios: "due to the mutual influence between multiple IDs and the complexity of interactions among multiple subjects, there remains significant room for improvement in maintaining and interacting with multiple subject IDs" (end of Section 2.2). When multiple identity signals are injected simultaneously, they interfere — the man's identity information can bleed into the object's appearance and vice versa.
Modality limitations: All the above methods, including the most recent multi-subject ones, operate exclusively in the image + text → video paradigm. They provide no mechanism for audio conditioning or video conditioning. The paper identifies this as a significant gap in the literature: "these methods struggle with maintaining subject consistency and video quality, and their reliance on single-modality (image-driven) inputs restricts their broader applicability" (Section 1).
The VACE multi-task tension: VACE (Jiang et al., 2025) is the one prior work that does break out of the image-only paradigm, supporting multi-modal conditioning (image, video, text) for combined creation and editing tasks. But the paper identifies a critical design flaw: VACE's approach of training a single model on "excessive training tasks" causes it to fail at identity preservation — the very core task the paper prioritizes. As they state in Section 1: "VACE... introduces a multi-modal-conditioned video generation framework. However, its excessive training tasks compromise ID consistency." This reveals a non-obvious design tension: multi-modal conditioning and identity preservation are not naturally complementary; they can actively conflict if not carefully architecturally disentangled. The paper's core architectural contribution — the disentanglement of modality-specific injection pathways from the shared identity pathway — is directly motivated by this observed failure mode in VACE.

How This Paper Positions Itself Relative to Existing Work

The paper's positioning is explicit and multi-layered:

Foundation model choice. The work is built on HunyuanVideo (Kong et al., 2024), a 13B-parameter diffusion transformer video generation model. The authors cite HunyuanVideo's strong temporal modeling capabilities (via 3D-RoPE positional encodings, causal 3D VAE compression, and efficient spatio-temporal attention) as the foundation that makes their identity enhancement strategy (temporal concatenation) viable. They do not present their choice of base model as incidental — they argue that the base model's inherent temporal information transmission capability is what enables the image ID enhancement module to work as designed. A base model with weaker temporal modeling would likely see less benefit from temporal concatenation of identity latents.

Priority ordering. The paper explicitly prioritizes identity consistency above all else in its design philosophy. This is stated clearly: "in this work we prioritize subject-consistent generation and develop a multi-modal customized video generation model that robustly preserves subject consistency" (Section 1). This is not a neutral statement — it is an explicit rebuttal to VACE's approach of distributing model capacity across many tasks and consequently sacrificing identity quality. The paper's architectural decisions (disentangled modality pathways, separate injection mechanisms for audio and video that don't interfere with the identity pathway) flow directly from this priority ordering.

The disentanglement principle. Perhaps the most important intellectual contribution in the paper's positioning is the architectural principle of disentangled modality injection. Prior work (including VACE) treated all conditioning signals as roughly equivalent streams that can be merged through similar adapter mechanisms. HunyuanCustom argues — and demonstrates through ablation studies in Section 5.6 — that identity information, audio information, and video information must enter the model through architecturally distinct, non-interfering pathways:

Identity information travels through two complementary routes: (1) the LLaVA-based text-image interaction module, which extracts high-level semantic identity features (category, color, shape) and injects them via the text prompt template; and (2) the temporal concatenation of image latents with 3D-RoPE positioning, which provides fine-grained identity details (texture, precise appearance) broadcast across frames through the model's temporal attention.
Audio information travels through AudioNet, using spatial cross-attention on a per-frame basis, which aligns audio features temporally with video latents without touching the identity injection pathway.
Video conditioning travels through a separate VAE encoding + patchify alignment pipeline and is added directly to the noisy video latents, again without interfering with either identity or audio pathways.

This three-way architectural disentanglement is what the paper argues enables "decoupled control over image, audio, and video conditions" (Section 3.1). It means that you can independently specify who appears in the video (via image + text), what they say (via audio), and what background/scene they appear in (via a conditioning video, if desired) — each through its own dedicated mechanism. No prior work had demonstrated this level of compositional control for video generation.

Scope: what they do and don't claim. The paper positions its contribution as a unified multi-modal architecture rather than claiming superior performance on any single narrow task. The individual components — LLaVA for image-text interaction, temporal concatenation for identity, spatial cross-attention for audio, latent addition for video conditioning — are each individually based on known techniques (LLaVA is adapted from image understanding literature, temporal concatenation echoes Still-Moving's use of static frames, audio-driven animation is a well-studied subfield). The novelty, as the paper frames it, is in the architectural integration and mutual disentanglement that enables all of these modalities to coexist in a single model while all preserving identity consistency — something that, by their account and supported by their comparisons, no single prior system achieves.

The multi-modal identity-centric vision. The paper uses the term "identity-centric" deliberately throughout. It is not trying to build a general-purpose multi-modal video generator that can do everything at equal quality. It is building a system where subject identity is the anchor, and all other modalities (audio, video conditioning, text action/scene descriptions) operate relative to that anchor. This distinguishes their approach from VACE, which (the paper argues) treats all tasks as equally important and consequently degrades identity performance. HunyuanCustom's design is fundamentally hierarchical: identity first, then audio and video conditioning as orthogonal additions to the identity-preserved generation.

This framing sets up the paper's experimental evaluation: they compare primarily on metrics that measure identity preservation (Face-Sim for human faces, DINO-Sim for general subject similarity), and then show that their method remains competitive on broader quality metrics (CLIP-B-T for text alignment, temporal consistency, dynamic degree) despite its heavy prioritization of identity — demonstrating that the identity-first design does not come at the cost of other quality dimensions.

3. Technical Approach

3.1 Reader Orientation

HunyuanCustom is a multi-modal video generation system that takes a reference image of a subject (a person, animal, product, or any object), an optional text description of the desired action and scene, and optionally audio or a conditioning video, and outputs a temporally coherent video where that specific subject appears performing the described actions while maintaining its visual identity throughout all frames. The problem it solves is the architectural challenge of injecting faithful subject identity into a video generation model while simultaneously enabling independent control through audio and video conditioning — and the shape of the solution is a set of architecturally disentangled injection pathways, each handling one modality without interfering with the others, all built on a shared identity-preserving foundation that uses a large vision-language model (LLaVA) for semantic understanding and temporal concatenation of image latents for fine-grained detail propagation.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components, each with a distinct responsibility:

HunyuanVideo Base Model — the pretrained 13B-parameter diffusion transformer that performs the actual video generation from noisy latents. It provides the spatio-temporal modeling capability (via 3D-RoPE, causal 3D VAE, and efficient attention) that all conditioning mechanisms build upon. It is fully fine-tuned during training, not frozen.
LLaVA-based Text-Image Fusion Module — a vision-language model that takes both the text prompt and the reference image(s) as input, replaces subject description tokens with image feature embeddings, and produces a fused multi-modal representation that captures high-level semantic identity (category, color, shape) and its relationship to the textual description. This is how the model "understands" that the word "man" in the prompt corresponds to the specific visual identity provided.
Image ID Enhancement Module — a temporal concatenation mechanism that takes the reference image, encodes it into the same latent space as video frames using the 3D VAE, positions it at a negative time index before frame 0, and concatenates it along the temporal axis with the noisy video latents. The model's temporal self-attention then propagates fine-grained identity details (texture, precise appearance) forward through all generated frames.
AudioNet (Audio Injection Module) — a modality-specific conditioning pathway that extracts multi-level audio features from input audio, temporally aligns them with the compressed video latent sequence, and injects them via spatial cross-attention applied independently to each frame. This enables the subject to speak, sing, or respond to audio cues while the identity injection pathways remain untouched.
Video-Driven Injection Module — another modality-specific pathway that takes a conditioning video (for background, scene context, or subject replacement), encodes it through the 3D VAE, aligns its features with the noisy video latents through a patchify-based alignment network, and adds the aligned features directly to the latent representation. This enables video editing tasks (subject replacement, insertion) without the quadratic computational cost of temporal concatenation.

Information flows through the system in a carefully orchestrated sequence that respects the disentanglement principle. Here is what happens when the system receives inputs:

Step 1 (Identity encoding — text pathway): The text prompt and reference image(s) enter the LLaVA model. The text prompt is templated so that subject description tokens (e.g., "man," "product") are replaced with <image> tokens or have identity prompts appended (e.g., "The man looks like <image>"). LLaVA extracts 24×24 image hidden features per reference image and embeds them into the text sequence alongside a <SEP> separator token to prevent image features from overwhelming text comprehension. The output is a fused token sequence encoding both the semantic identity and the textual action/scene description.

Step 2 (Identity encoding — latent pathway): In parallel, the reference image is resized to match video frame dimensions and encoded into latent space using the pretrained causal 3D VAE, producing an image latent z_I of shape [wh × c] (width × height × channels). This latent is assigned a 3D-RoPE position embedding at time index -1 (before the first video frame) and with a spatial shift (i+w, j+h) to prevent the model from simply copying the reference image into the generated frames.

Step 3 (Noise initialization and temporal concatenation): The video latent z_t is initialized from Gaussian noise and, if this is a training step, interpolated with the clean video latent z_1 to produce the noisy latent at timestep t. The image latent z_I is concatenated with the video latent z_t along the temporal axis, producing a combined latent z = [z_I; z_t] of shape [(f+1)wh × c] — one image frame followed by f video frames. This combined latent enters the diffusion transformer's self-attention, where the temporal attention mechanism propagates identity information from the image frame forward through all generated frames.

Step 4 (Audio conditioning, if present): If audio is provided, AudioNet extracts audio features into a tensor of shape [f' × 4 × c] where f' is the number of audio frames (temporally compressed 4:1 relative to the video). These features are padded to align with the (f+1) video frames, then rearranged so that every 4 consecutive audio tokens form one frame's audio representation, yielding f_A of shape [b, (f+1), 16, c]. Spatial cross-attention is applied independently per frame: the audio features attend to the video latents within each spatial slice [wh, c], but not across frames. This audio-injected latent is added back to the main latent pathway: z_{t,A} = z_t' + λ_A × CrossAttn(f_A, z_t'), where λ_A is a learnable or tunable weight controlling audio influence.

Step 5 (Video conditioning, if present): If a conditioning video is provided, it is encoded by the same 3D VAE, then compressed and serialized by the pretrained video tokenizer. A four-layer fully connected network (the patchify-based feature alignment network) maps these conditioning features into the same space as the noisy video latents. The aligned features are added frame-by-frame to the video latent representation along the temporal dimension, producing the final conditioned latent without changing its shape or adding computational overhead beyond the initial encoding and alignment.

Step 6 (Diffusion denoising and decoding): The conditioned latent (now carrying identity, audio, and video conditioning, each through its own non-interfering pathway) passes through the diffusion transformer's denoising steps, which are trained to predict the velocity field v_t that brings the noisy latent toward the clean latent z_1. After denoising, the final latent is decoded by the 3D VAE decoder into pixel-space video frames.

The critical architectural invariant is that identity information (from LLaVA's text pathway and from the temporal concatenation of the reference image), audio information (from AudioNet's spatial cross-attention), and video information (from the alignment network's direct latent addition) never share the same injection mechanism. They are added at different points, through different operations, to different parts of the latent representation, ensuring that enabling audio conditioning cannot degrade identity preservation, and vice versa.

3.3 Roadmap for the Deep Dive

The following detailed breakdown proceeds in this order because each component builds on the previous one:

First, the Flow Matching training framework, since all the conditioning mechanisms are trained under this objective and it defines what the model learns to predict.
Second, the LLaVA-based text-image interaction module, because it establishes how semantic identity information enters the model and interacts with text — the foundational capability on which all other tasks depend.
Third, the image ID enhancement module (temporal concatenation with 3D-RoPE), which complements LLaVA's high-level semantic features with fine-grained texture and detail features — completing the identity injection story.
Fourth, the multi-subject extension, since it generalizes the single-subject identity mechanisms to handle multiple simultaneous identities with independent spatial positioning.
Fifth, the AudioNet module, because audio conditioning is added as an orthogonal pathway on top of the already-established identity foundation.
Sixth, the video-driven injection module, which similarly extends the identity foundation with a separate video conditioning pathway designed for computational efficiency.

3.4 Detailed, Sentence-Based Technical Breakdown

This is an architectural design and integration paper whose core idea is that subject-consistent video generation across multiple modalities requires identity injection (via LLaVA semantic features and temporal latent concatenation) to be architecturally disentangled from audio conditioning (via spatial cross-attention) and video conditioning (via direct latent addition), and that a unified model designed around this principle can achieve best-in-class identity preservation while supporting all conditioning modalities.

Flow Matching Training Framework

The entire HunyuanCustom model is trained under the Flow Matching framework (Lipman et al., 2022), which the base HunyuanVideo model also uses. Flow Matching is a generative modeling paradigm that learns a time-dependent vector field mapping samples from a simple prior distribution (Gaussian noise) to the target data distribution (video latents). Unlike standard diffusion models that learn to predict the noise added to a clean sample, Flow Matching models learn to predict the velocity — the instantaneous direction and magnitude of change — that transports a noisy sample toward a clean sample along a straight-line trajectory.

Training sample construction. The training process begins with a clean video latent z_1 obtained by encoding a training video through the pretrained causal 3D VAE. Noise z_0 is sampled from a standard Gaussian: z_0 ~ N(0, I). A timestep t is sampled from a logit-normal distribution (Esser et al., 2024) — this sampling strategy concentrates probability mass near t = 0 and t = 1, where the velocity is largest, and reduces mass in the middle, which improves training efficiency for high-resolution generation. The noisy latent at timestep t is constructed by linear interpolation (straight-line path) between the noise and the clean latent:

$z_t = (1 - t) \cdot z_0 + t \cdot z_1$

where $z_t$ is the noisy latent at time $t$ , $z_0$ is the initial Gaussian noise, and $z_1$ is the clean video latent.

What it computes: each training sample $z_t$ is a point along the straight line from noise to target. At $t = 0$ , $z_t = z_0$ (pure noise); at $t = 1$ , $z_t = z_1$ (clean target); at intermediate $t$ , $z_t$ is a partially corrupted version of the target video.

Why this form: the linear interpolation path means the true velocity $u_t = dz_t/dt = z_1 - z_0$ is constant — it is simply the vector difference between the clean latent and the noise, independent of $t$ . This simplifies the learning problem: rather than learning a time-varying target, the model needs to learn to predict this fixed vector from the appearance of the partially corrupted latent. This constant-velocity property makes the training signal consistent across timesteps.

Training objective. The model $v_\theta$ takes as input the noisy latent $z_t$ , the timestep $t$ , and all conditioning signals (text features from LLaVA, the concatenated image latent, and optionally audio features from AudioNet and video features from the alignment network), and predicts the velocity:

$\mathcal{L}_{\text{generation}} = \mathbb{E}_{t, z_0, z_1} \| v_t - u_t \|^2$

where $v_t = v_\theta(z_t, t, \text{conditioning})$ is the model's predicted velocity, $u_t = z_1 - z_0$ is the true velocity (constant across $t$ ), and the expectation is over the logit-normal-distributed $t$ , the Gaussian noise $z_0$ , and the training video distribution $z_1$ .

What it computes: the mean squared error between the predicted velocity vector and the true velocity vector (the constant difference between clean latent and noise). Lower loss means the model correctly identifies the direction and magnitude needed to transform the current noisy latent toward the clean target.

Why this form: the MSE loss for velocity prediction is a form of score matching — learning the gradient of the log-density along the straight-line path. Unlike noise prediction (which becomes ill-conditioned near $t = 0$ since the signal-to-noise ratio diverges), velocity prediction has a well-conditioned target (the constant vector $z_1 - z_0$ ) at all timesteps. This uniform conditioning is particularly important for video generation, where the model must handle both global structure (determined early in denoising, near $t = 0$ ) and fine details (determined late, near $t = 1$ ).

Full fine-tuning. The paper trains by fully fine-tuning all weights of both the pretrained HunyuanVideo base model and the LLaVA model, rather than using parameter-efficient fine-tuning (PEFT) methods like LoRA. The authors state this is "to endow our model with a more extensive representational capacity and enable it to capture and learn a broader range of complex patterns" (Section 3.4). This choice reflects a deliberate tradeoff: full fine-tuning is more expensive and requires more data, but it allows the model to repurpose its entire capacity for the identity-conditional generation task rather than being constrained to low-rank adaptations of the original weights. This is particularly important for the LLaVA model, which was originally trained for multi-modal understanding (e.g., visual question answering, image captioning), not for encoding identity features for video generation — the full fine-tuning allows it to adapt its representations to serve this new purpose.

Inference (denoising). At inference time, the process is reversed: start from pure noise $z_0 ~ N(0, I)$ , and iteratively apply the model's predicted velocity to transport the sample toward the data distribution. The conditioning signals (identity images, text, optional audio, optional video) are provided at every denoising step, and the model's velocity predictions incorporate all of these. The specific ODE solver and number of denoising steps are inherited from HunyuanVideo's inference pipeline, which the paper does not modify — all the changes are in what conditioning information enters the model, not in how the diffusion process itself is solved.

LLaVA-Based Text-Image Interaction

The LLaVA-based text-image interaction module is the semantic identity injection pathway — it is how the model learns to associate the specific reference image with the subject description words in the text prompt. This module is built on the observation that HunyuanVideo already uses LLaVA (Liu et al., 2023a) as its text encoder, which means LLaVA's internal text representations have been co-trained with the video generation model's denoising network. By extending LLaVA's input to include images alongside text, the paper leverages both this pre-existing text-video alignment and LLaVA's native multimodal understanding capabilities.

The core mechanism: templating and token replacement. The module operates by transforming the raw text prompt into a structured template that explicitly links image features to specific words. The paper explores two template strategies:

Template 1 — Image-embedded: The description word T_I in the text is directly replaced with an <image> token. For example, if the text prompt is "A man is playing guitar" and the reference image is of a specific man, the resulting template is: "A <image> is playing guitar." The <image> token later gets replaced by 24×24 LLaVA image hidden features, so the model receives the image features in the exact syntactic position where the subject description would normally appear.

Template 2 — Image-appended: Instead of inline replacement, an identity prompt is appended to the end of the original text: "The T_I looks like <image>." For the same example, this produces: "A man is playing guitar. The man looks like <image>." The original text is preserved intact, and the image is linked to the subject description through an explicit association clause at the end.

Between the text prompt and the identity prompt, a special <SEP> (separator) token is inserted. The paper explains this design choice: "Since the image feature tokens are significantly longer than the text feature tokens, to prevent the image features from overly influencing text comprehension, we insert a special token <SEP> between the text prompt and the image prompt. This helps the LLaVA model retain the information from the text prompt while establishing a connection between the text prompt and the image identity" (Section 3.4).

Image feature extraction. The reference image is processed by LLaVA's vision encoder, which produces a grid of image features that is flattened into a sequence of tokens. The paper specifies that each reference image is represented by 24×24 = 576 image hidden features. This resolution is the output of LLaVA's vision transformer before projection into the language model's embedding space. The 576 image tokens are inserted into the text token sequence at the position of the <image> placeholder, alongside the <SEP> token and the surrounding text tokens.

What LLaVA contributes. The paper's ablation study (Section 5.6, Figure 11) reveals that removing the LLaVA branch entirely causes the model to "fail to capture any significant details from the target image." This indicates that LLaVA is doing more than just passing through image information — it is extracting the semantic-level identity features that the model needs to understand what the subject is (its category, its general appearance, its distinguishing high-level attributes) so that the text-driven action descriptions can be correctly applied to it. Without LLaVA, the identity enhancement module (temporal concatenation) alone cannot compensate — it provides fine-grained pixel-level and texture-level information, but not the semantic binding between visual identity and linguistic description.

Why LLaVA rather than CLIP or a newly trained branch. The paper explicitly contrasts this approach with prior methods that "either lack a design for interactive understanding between image and text or rely on additional newly trained branch networks to achieve this interaction" (Section 3.4). Using LLaVA — a model already trained for multimodal interaction and already embedded in HunyuanVideo's text understanding pipeline — avoids the need to train a new cross-modal fusion network from scratch. The weights of the pretrained LLaVA already encode the ability to relate visual concepts to linguistic descriptions (e.g., understanding that a specific pattern of pixels corresponds to "a red jacket" and that "red jacket" can be the subject of "is walking"), which is precisely the capability needed for text-guided identity-conditional generation. Full fine-tuning then adapts these pretrained multimodal representations to the specific needs of the video generation task.

A subtle limitation the paper does not discuss: LLaVA's image features are 24×24 tokens, which the paper notes is "significantly longer than the text feature tokens." At 576 tokens per reference image, adding even a single subject significantly expands the context length that the diffusion transformer must attend to. The <SEP> token mitigates attention interference but does not reduce the computational cost. For multi-subject customization with m reference images, the text sequence contains m × 576 image tokens plus the text tokens, which scales linearly in the number of subjects. The paper does not analyze how this scaling affects inference cost or attention quality.

Image ID Enhancement Module (Temporal Concatenation)

If the LLaVA module provides the semantic understanding of identity ("this is a golden retriever wearing a red collar"), the image ID enhancement module provides the perceptual identity details — the specific fur pattern, the exact shade of red, the precise shape of the ears. These are the features that distinguish one golden retriever from another, and they are critical for the identity preservation metrics (Face-Sim, DINO-Sim) the paper prioritizes.

The core idea: treat the reference image as a frame and let temporal attention do the work. The HunyuanVideo base model uses 3D-RoPE (Rotary Position Embedding) across three axes: frame index (temporal), width, and height. This positional encoding scheme gives every pixel in the video latent a unique position vector that encodes where it sits in the video's spatio-temporal grid. The model's self-attention then uses these position encodings to learn spatio-temporal relationships — which pixels in which frames should attend to which other pixels.

The ID enhancement module exploits this mechanism by treating the reference image as if it were a frame at time index -1, positioned immediately before the first generated frame (time index 0). The image is encoded into the same latent space as the video frames using the same pretrained causal 3D VAE, producing:

$z_I \in \mathbb{R}^{wh \times c}$

where $w$ and $h$ are the width and height of the latent representation (after VAE spatial compression), and $c$ is the feature dimension (the number of channels in the VAE latent space).

This image latent is concatenated with the noisy video latent along the first sequence dimension (the temporal axis, since each frame occupies a contiguous block of $wh$ spatial tokens):

$z = \{z_I, z_t\} \in \mathbb{R}^{(f+1)wh \times c}$

where $f$ is the number of video frames (after VAE temporal compression), so the combined latent has $f+1$ total frames (the reference image plus $f$ generated frames), each contributing $wh$ tokens, for a total sequence length of $(f+1)wh$ tokens.

3D-RoPE positioning for the reference image. Each pixel in the video latent already receives a 3D-RoPE encoding based on its frame index $f$ , width position $i$ , and height position $j$ : RoPE(f, i, j). The reference image latent needs its own position encoding so that the temporal attention mechanism can distinguish it from the generated frames and learn how to propagate identity information forward. The paper assigns the reference image to frame index -1, placing it temporally before all generated frames. Additionally, inspired by Omnicontrol (Tan et al., 2024), the paper applies a spatial shift to the image latent's width and height coordinates:

$\text{RoPE}_{z_I}(f, i, j) = \text{RoPE}(-1, i + w, j + h)$

What this equation computes: each pixel in the reference image latent is treated as if it sits at time -1 (one frame before the start of the video), at a spatial position shifted by the full latent width and height — effectively placing the reference image in a completely different spatial region from the generated frames.

Why this form is essential — the copy-paste prevention mechanism. Without the spatial shift, the reference image latent at position (i,j) would have the same spatial RoPE as the generated frame pixel at (i,j). The temporal attention mechanism would then tend to directly copy the reference pixel to the corresponding generated frame pixel — producing the "copy-paste" artifact where the first frame is identical to the reference image and subsequent frames show the reference image pasted statically into each frame without motion or integration. By shifting the spatial coordinates by +w and +h, the reference image pixels are positioned in a completely disjoint region of the RoPE space from any pixel in the generated video, forcing the model to learn a more abstract, non-trivial mapping from reference appearance to generated appearance. The model cannot simply copy the pixel at (-1, i, j) to (0, i, j) because those two positions have different RoPE encodings and thus different attention patterns.

How temporal attention propagates identity. With the reference image latent concatenated at time -1 and all subsequent frames at times 0 through $f-1$ , the model's temporal self-attention can learn to attend from each generated frame back to the reference image frame. The key insight is that the HunyuanVideo base model already has strong temporal attention priors (it was pretrained on video data where frame-to-frame information flow is natural), so it can learn to propagate the fine-grained appearance information from the reference frame forward through time without explicit optical flow, tracking, or warping mechanisms. The ablation study (Section 5.6) confirms this: removing the identity enhancement module (leaving only LLaVA) captures "global identity information but misses detailed identity features," while the full model captures both.

Channel concatenation vs. temporal concatenation — a key design choice validated by ablation. The paper's ablation study (Section 5.6, Figure 11) compares temporal concatenation (their method) against an alternative: channel concatenation, where the image latent is concatenated with the video latent along the channel dimension rather than the temporal axis. The channel-concatenation variant "shows poor generation quality" and "suffers from a severe blurring effect in the initial frames." The paper attributes this to the fact that temporal concatenation leverages the video model's existing temporal modeling priors — the attention patterns that naturally propagate information across frames — whereas channel concatenation requires the model to learn an entirely new cross-channel feature extraction pathway that it was not pretrained for.

For multi-subject scenarios (described in the next sub-section), each of $m$ reference images receives its own temporal position and 3D-RoPE encoding. The $k$ -th reference image is assigned to frame position $-k$ with the same spatial shift:

$\text{RoPE}_{z_{I,k}}(f, i, j) = \text{RoPE}(-k, i + w, j + h)$

The combined latent becomes:

$z = \{z_{I,1}, z_{I,2}, \ldots, z_{I,m}, z_t\} \in \mathbb{R}^{(f+m)wh \times c}$

Each reference image occupies a unique negative time index $-1, -2, \ldots, -m$ , making them temporally distinct from each other and from all generated frames. The spatial shift (i+w, j+h) is the same for all reference images, so they share the same spatial RoPE region (distinct from the generated video) but are separated in the temporal dimension.

Multi-Subject Customization Extension

Multi-subject customization extends the single-subject framework to handle $m \geq 2$ reference images, each representing a different subject that should appear in the generated video interacting with the others. The paper treats multi-subject customization as a fine-tuning stage on top of the single-subject model, not a separate model trained from scratch.

Input specification. The inputs are a set of reference images $\{I_1, I_2, \ldots, I_m\}$ with corresponding text description tokens $\{T_{I,1}, T_{I,2}, \ldots, T_{I,m}\}$ that appear in the text prompt. Each image is coupled to its description token through the image-appended template: "the $T_{I,k}$ looks like <image>." These identity prompts are concatenated with separators, and each <image> token is replaced by the LLaVA-extracted 24×24 image features for the corresponding reference image.

Latent concatenation for multiple subjects. Each reference image $I_k$ is independently encoded by the 3D VAE into a latent $z_{I,k}$ . All $m$ image latents are concatenated with the video latent along the temporal axis, producing a combined latent of $f + m$ frames (the $m$ reference images plus $f$ video frames). Each reference latent $z_{I,k}$ is assigned to temporal position $-k$ and receives the RoPE encoding $\text{RoPE}(-k, i + w, j + h)$ as described above. This creates $m$ temporally distinct reference frames, each in the spatially shifted RoPE region, all accessible to the temporal attention of all generated frames.

Training strategy — fine-tuning from single-subject checkpoint. Rather than training the multi-subject model from scratch, the paper starts from the fully-trained single-subject customization model and continues fine-tuning on multi-subject data. This makes sense architecturally: the single-subject model has already learned how to use LLaVA for semantic identity binding and temporal concatenation for detail propagation; the multi-subject fine-tuning only needs to learn how to handle multiple simultaneous identities and their interactions.

Data construction for multi-subject training (Section 3.3, Multi-Subject Extraction). The paper filters the video dataset to find scenes where multiple subjects interact, specifically "videos from single-person datasets that involve interactions between humans and objects." Subject extraction follows a specific pipeline: Florence2 (Xiao et al., 2024) extracts bounding boxes for each subject mentioned in the video captions; GroundingSAM2 (Ravi et al., 2024; Liu et al., 2023b) performs segmentation on these bounding box regions; and clustering removes frames that do not contain all subjects. To mitigate the copy-paste problem (where the model learns to copy the reference image exactly rather than generate it in a new pose), the paper uses the first 5 seconds of each video for training and the subsequent 15 seconds for subject segmentation — ensuring that the training frames and the reference images come from different temporal segments of the same video, so the model must learn to generate the subject in new poses rather than simply reproducing the reference frame.

AudioNet: Identity-Disentangled Audio Conditioning

AudioNet is the audio injection pathway, designed to add audio conditioning (speech, singing, environmental sounds) to the identity-preserved video generation without interfering with the identity injection mechanisms. The key design principle is disentanglement: AudioNet uses a completely different operation (spatial cross-attention) from the identity pathways (LLaVA text fusion and temporal latent concatenation), applied to a different part of the representation, ensuring that audio and identity signals do not compete for model capacity.

Motivation — going beyond audio-driven animation. The paper explicitly positions audio-driven customization against prior audio-driven human animation methods (Loopy, SONIC) which work by taking "a human image and audio as input to animate the character in the image to speak the corresponding speech" but "the character's posture, attire, and setting remain consistent with the input image, limiting the ability to generate videos of the target character in different postures, attire, and settings" (Section 4.1). HunyuanCustom's audio capability is fundamentally different: it generates a video where the identity subject speaks the audio, but in any text-described scene, posture, and attire — not just the one in the reference image.

Audio feature extraction and temporal alignment. The input audio is processed by Whisper (Radford et al., 2023) to extract audio features. The raw audio feature tensor has shape $f' \times 4 \times c$ , where $f'$ is the number of audio frames (before temporal compression to match video), $4$ is the number of audio tokens per frame (a design choice made by the authors), and $c$ is the feature dimension.

The video latent, after VAE compression, has $f = \lfloor f'/4 \rfloor + 1$ frames — the floor division by 4 comes from the VAE's temporal compression ratio, and the $+1$ accounts for the uncompressed initial frame. With the identity image concatenated, the full latent (entering AudioNet) has $f+1$ frames. To align audio features temporally with this compressed video representation, two operations are performed:

Padding: Audio features are padded before the initial frame to match the $(f+1) \times 4$ temporal resolution (the identity image frame at position -1 also needs an aligned audio feature, even though it represents a static image — the padding ensures the temporal shapes match).
Aggregation: Every 4 consecutive audio frames are aggregated into a single frame by rearrangement:

$f_A = \text{Rearrange}(f_{A,0}): [b, (f+1) \times 4, 4, c] \rightarrow [b, (f+1), 16, c]$

What this equation computes: the input audio feature tensor of shape [batch, (f+1)×4, 4, channels] is reshaped into [batch, (f+1), 16, channels]. Each group of 4 consecutive audio frames (each contributing 4 tokens) is flattened into a single frame's audio representation of 16 tokens.

Why this rearrangement: the VAE compresses the video temporally by a factor of 4, so 4 original video frames map to 1 latent frame. The audio must undergo the same compression to be temporally aligned — each latent frame should receive audio features corresponding to the same time window it represents. The rearrangement from 4 × 4 = 16 tokens per 4-frame window to a flat 16 tokens per compressed frame ensures that the audio representation retains the same total information per time step while aligning temporally with the spatially-compressed video latents.

Spatial cross-attention for audio injection. With the temporally aligned audio features $f_A$ and the video latent $z_t$ , audio information is injected via cross-attention applied per frame, independently. The key equations are:

$z'_{t,A} = \text{Rearrange}(z_t): [b, (f+1)wh, c] \rightarrow [b, f+1, wh, c]$

$z''_{t,A} = z'_{t,A} + \lambda_A \times \text{CrossAttn}(f_A, z'_{t})$

$z_{t,A} = \text{Rearrange}(z''_{t,A}): [b, f+1, wh, c] \rightarrow [b, (f+1)wh, c]$

What these equations compute, step by step:

Step 1 — Decouple temporal and spatial dimensions: The video latent is rearranged from a flat sequence of $(f+1)wh$ tokens into a $(f+1) \times (wh)$ grid — each of the $f+1$ frames is separated into its own $wh$ spatial tokens. This decoupling is necessary because spatial cross-attention operates within each frame independently.

Step 2 — Apply per-frame cross-attention: For each frame, the $wh$ spatial tokens attend to the 16 audio tokens for that frame via standard cross-attention. The audio tokens serve as keys and values; the spatial tokens serve as queries. The result is a residual added back to the spatial tokens, scaled by $\lambda_A$ , a weight that controls the strength of audio influence.

Step 3 — Recombine: The $(f+1) \times (wh)$ grid is rearranged back into the flat $(f+1)wh$ token sequence, matching the original latent shape.

Why spatial cross-attention rather than temporal cross-attention or full spatio-temporal cross-attention. Spatial cross-attention means that when frame $k$ 's spatial tokens attend to audio features, they only attend to the audio features for frame $k$ — not to audio features for earlier or later frames. This per-frame isolation "prevents inter-frame interference between audio and video at different frames" (Section 4.1). If the model used full spatio-temporal cross-attention, a spatial token at frame 10 could attend to audio features from frame 5, creating unnatural correlations (e.g., the mouth shape at frame 10 influenced by audio that should only affect frame 5). Per-frame spatial cross-attention ensures strict temporal alignment: each frame's generated content is only influenced by the audio that temporally corresponds to that frame.

Why the disentanglement design matters here. AudioNet uses cross-attention with the video latents as queries and audio features as keys/values. The identity injection pathways (LLaVA text fusion and temporal latent concatenation) use entirely different mechanisms — the LLaVA features enter through the text token stream processed by the diffusion transformer's self-attention, and the image latent enters through temporal concatenation processed by the same self-attention. There is no shared mechanism between audio and identity injection. This means that training the model to use audio conditioning does not modify the weights that handle identity injection (except through the shared backbone of the diffusion transformer, which is fully fine-tuned), and the audio pathway can be enabled or disabled at inference time by setting $\lambda_A = 0$ without affecting identity preservation.

Video-Driven Injection Module (Video Conditioning for Editing)

The video-driven injection module enables video-conditioned generation for editing tasks — specifically, subject replacement (replacing an object in a source video with a target identity) and subject insertion (adding a target identity into a background video). Like AudioNet, this module is designed to be computationally efficient and architecturally disentangled from the identity injection pathways.

The computational challenge of video conditioning. A naive approach to conditioning on a source video would be to concatenate the source video latents with the generated video latents along the temporal axis, similar to how the identity image is concatenated. The paper explicitly rejects this approach, noting that it "doubles the sequence length and quadratic growth in attention computation" (Section 4.2) — if the generated video has $f$ frames and the source video has $f$ frames, concatenation produces $2f$ frames, and the self-attention cost scales as $O((2f \cdot wh)^2) = 4 \times$ the cost for a single $f$ -frame video. Another approach, used by VACE, injects video conditions via separate adapter modules that "double the computational cost and severely limit efficiency."

HunyuanCustom's solution: compress, align, then add — a three-stage pipeline that injects video conditioning without increasing the sequence length at all.

Stage 1: Video compression and tokenization. The conditioning video is encoded using the same pretrained causal 3D-VAE that encodes the reference image. The resulting latent representation is then further compressed and serialized using the pretrained video tokenizer from HunyuanVideo. This produces a compact feature representation of the conditioning video.

Stage 2: Feature alignment via patchify-based network. The conditioning video features come from a clean, noise-free video, whereas the video latents $z_t$ are noisy (they represent a partially denoised video at timestep $t$ during training, or progressively denoised latents during inference). Directly adding clean features to noisy latents would create a distribution mismatch. The alignment network — a four-layer fully connected network — maps the conditioning video features into the same distributional space as the noisy video latents. The paper frames this as a "patchify-based" alignment because it operates on the tokenized, serialized latent patches that the diffusion transformer processes.

Stage 3: Identity-disentangled addition. The aligned conditioning features are added directly to the video latents. The paper explores two addition strategies:

Concatenation-based (rejected): Concatenate conditioning features with video latents along the token dimension, then project back to the original dimension via a learned linear layer. The paper reports that this "struggles to preserve content information and suffers from substantial information loss" — the projection acts as a bottleneck that discards important details.
Addition-based (adopted): Add the aligned conditioning features to the video latents frame-by-frame along the temporal dimension, preserving the original feature dimensions. This is described as "more effective content injection" because it avoids the information bottleneck of the concatenation-and-project approach and because the prior feature alignment step ensures the conditioning features and video latents are well-matched for direct addition.

How the identity disentanglement works in video conditioning. The addition operation modifies the video latent $z_t$ directly, in a way that is separate from both the temporal concatenation of the reference image latent and the LLaVA text pathway. The video conditioning pathway only interacts with the video latent — it does not touch the reference image latent (which sits at time index -1 before all video frames) or the text features (which flow through the diffusion transformer's cross-attention). During inference, enabling video conditioning does not alter any part of the identity injection pipeline; it simply adds a conditioning signal to the video latents before they enter the denoising network.

Computational cost. Because the aligned features are added directly to the video latents without changing the sequence length, the video conditioning introduces zero additional computational overhead during the diffusion denoising steps beyond the initial encoding and alignment (which are one-time costs). The self-attention and cross-attention in the diffusion transformer operate on the same $(f+1)wh$ -token sequence regardless of whether video conditioning is enabled. This makes the approach "efficient and lightweight" (Section 4.2) compared to concatenation-based or adapter-based alternatives, and it is critical for the practical deployment of video editing at scale.

Data Construction Pipeline

The paper devotes significant detail (Section 3.3) to the data construction pipeline, reflecting their claim that "high-quality data plays a crucial role in tasks such as subject consistency, video editing, and audio-driven video generation." The pipeline has several stages that feed into the training of the modules described above.

Raw data sources. Data is sourced from diverse channels under GDPR compliance, spanning eight major categories: humans, animals, plants, landscapes, vehicles, objects, architecture, and anime. The paper also incorporates open-source datasets (explicitly mentioning OpenHumanvid) after rigorous curation and processing.

Scene detection and filtering. Videos are segmented into single-shot clips using PySceneDetect (Castellano, 2020), which detects hard cuts. Clips with excessive text are filtered using textbpn-plus-plus (Zhang et al., 2023b), and videos containing subtitles, watermarks, and logos are cropped. Videos are standardized to a short side of 512 or 720 pixels and limited to 5 seconds (129 frames at ~25.8 fps). A final quality filter uses the koala-36M model (Wang et al., 2024a) with a dataset-specific threshold of 0.06 to filter based on aesthetic quality, motion magnitude, and scene brightness. The paper notes that koala-36M's pretrained threshold was not suitable and required custom calibration.

Subject extraction — single human subjects. For human subjects, Qwen7B (Bai et al., 2023) labels all subjects in each frame and extracts their IDs. A clustering algorithm (Union-Find) computes the appearance frequency of each ID across frames; the most frequent ID is selected as the target subject. Videos are discarded if no ID appears in at least 50 frames. YOLO11X (Khanam and Hussain, 2024) provides human body bounding boxes, while InsightFace (Ren et al., 2023) provides face bounding boxes. A validation check discards YOLO11X bounding boxes if the face is less than 50% of the body bounding box area (indicating a likely detection error).

Subject extraction — non-human subjects. QwenVL (Bai et al., 2025b) extracts subject keywords from the video, which are passed to GroundingSAM2 (Ravi et al., 2024; Liu et al., 2023b; Ren et al., 2024a, b; Jiang et al., 2024b) to generate masks and bounding boxes. Bounding boxes smaller than 30% of the video dimensions are discarded. QwenVL classifies the main subject into one of the eight predefined categories, and balanced sampling is applied across categories.

Subject extraction — multi-subject. QwenVL filters single-person videos that contain human-object interactions. Florence2 extracts bounding boxes for all subjects mentioned in the video captions, and GroundingSAM2 performs segmentation on these regions. Frames not containing all subjects are removed by clustering. The first 5 seconds of video are used for training; the subsequent 15 seconds are used for subject segmentation (extracting the reference images from a different temporal segment than the training frames, which helps prevent the model from learning simple copy-paste).

Video resolution standardization. A union bounding box is computed from all subject bounding boxes. Cropping ensures at least 70% of the union bounding box area is preserved. Multiple aspect ratios are supported: 1:1, 3:4, and 9:16.

Video annotation. A structured annotation model developed by the HunYuan team labels each video with long descriptions, short descriptions, background styles, and camera movement descriptions. These structured annotations are used to enhance training captions.

Mask data augmentation (for video editing). For the video subject replacement task, directly using extracted subject masks for training causes overfitting: the model learns to exactly reproduce the mask shape rather than adapting to different-shaped replacement subjects. For example, replacing a doll without ears with one that has ears would fail because the mask shape does not account for the ears. The paper applies mask dilation and mask-to-bounding-box conversion to soften boundaries and mitigate this overfitting.

Audio data processing. For audio-driven training data, LatentSync (Li et al., 2024a) evaluates audio-video synchronization quality; clips with confidence below 3 are discarded, and audio-video offsets are adjusted to zero. HyperIQA scores are computed to filter out clips scoring below 40, ensuring high visual quality. Whisper (Radford et al., 2023) extracts the audio features used as input to AudioNet during training.

Summary of Design Choices and Their Justifications

Full fine-tuning of both HunyuanVideo and LLaVA, rather than parameter-efficient methods: gives the model full representational capacity to repurpose pretrained weights for the identity-conditional generation task, which requires substantial adaptation of the LLaVA vision-language representations.
LLaVA for text-image interaction rather than CLIP or a custom branch: leverages LLaVA's pretrained multimodal interaction capabilities and its pre-existing integration with HunyuanVideo's text encoding pipeline, avoiding the need to train a new cross-modal network.
Temporal concatenation with spatially shifted 3D-RoPE for identity enhancement rather than channel concatenation: exploits the video model's pretrained temporal attention priors; the spatial shift prevents simple copy-paste of reference pixels.
Image-appended and image-embedded templates with <SEP> token: prevents the 576 image feature tokens from overwhelming the shorter text token sequence while still allowing attended interaction between text and image representations.
Spatial cross-attention (per-frame) for audio rather than full spatio-temporal cross-attention: prevents inter-frame audio-video interference and ensures strict temporal alignment between spoken content and generated lip movements.
Latent addition for video conditioning rather than temporal concatenation or adapter modules: avoids quadratic attention cost growth and adapter parameter overhead; the four-layer alignment network handles the clean-noisy distribution mismatch.
First 5 seconds for training, subsequent 15 seconds for subject segmentation in multi-subject data: ensures training frames and reference images come from different temporal segments, forcing the model to generate subjects in new poses rather than copying reference frames.
Mask dilation and bounding box conversion for video editing training: prevents overfitting to exact mask boundaries and enables the model to gracefully handle replacement subjects with different shapes than the original subjects.

4. Key Insights and Innovations

The field's prior assumption — embodied most directly in VACE (Jiang et al., 2025), the only previous multi-modal conditioning framework for video — was that supporting multiple conditioning modalities (image, video, text) is primarily a training data and task coverage problem: gather sufficient data for each modality combination, define appropriate training objectives for each, and jointly optimize a single model. Under this assumption, the model's shared parameters learn to route different conditioning signals appropriately through the same architectural pathways.

HunyuanCustom diagnoses a different, more fundamental problem: conditioning modalities that share injection mechanisms will interfere with each other in ways that cannot be resolved by data scale or joint training alone. The paper's central architectural insight — which emerges most clearly from the contrast with VACE — is that identity preservation, audio conditioning, and video conditioning must enter the generation process through operations of fundamentally different types applied at different points in the architecture, because they serve qualitatively different functions that make incompatible demands on the feature representations they modify.

Concretely: identity information requires both high-level semantic binding (what category of subject, what general appearance) and low-level perceptual detail (exact texture, precise color). This dual requirement is met by the combination of LLaVA text fusion (semantic) and temporal latent concatenation (perceptual). Audio conditioning requires strict temporal alignment on a per-frame basis so that speech content at frame k influences only the generated content at frame k — a constraint that demands spatial cross-attention applied independently per frame, not full spatio-temporal attention. Video conditioning requires the generated content to conform to a reference scene structure (background, motion patterns, occlusion boundaries) while replacing the object of interest — a constraint best served by direct feature-space addition after alignment that avoids the quadratic attention cost of concatenation.

If these three conditioning pathways were merged through a single mechanism (e.g., a shared adapter module that processes all conditioning signals and injects them at a single point), the model would face irreducible tradeoffs: the representational format optimized for identity detail propagation (temporal concatenation with broad temporal attention across all frames) would conflict with the format needed for temporally precise audio alignment (per-frame isolation). The architectural response — three distinct injection operations (self-attention on concatenated latents, cross-attention on per-frame spatial slices, direct addition of aligned features) at three different points in the computation graph — is the paper's key intellectual move, transforming what appeared to be a data coverage problem into an architecture design problem.

Evidence: The paper does not ablate this disentanglement directly (it is the architectural premise, not a single module you can remove). But the evidence for its necessity comes from the VACE comparison: VACE, which the paper explicitly characterizes as using "excessive training tasks" that "compromise ID consistency" (Section 1), achieves dramatically worse identity preservation than HunyuanCustom (Figure 4, row 7: VACE "cannot maintain ID consistency"). The fact that VACE — which supports multi-modal conditioning — fails at the very task HunyuanCustom succeeds at (identity preservation) while HunyuanCustom adds audio and video conditioning on top of identity without degrading it (Tables 1, Figures 8-10) is the empirical demonstration that the disentanglement works.

Significance: This is a fundamental architectural insight rather than a metric gain. It reframes the multi-modal generation problem from "how much data and how many tasks" to "which operations for which modalities," providing a design principle that future multi-modal systems can adopt. The principle — that conditioning signals with different temporal and spatial characteristics require distinct injection mechanisms — generalizes beyond the specific modalities studied here to any future modality (depth maps, semantic layouts, motion vectors) that might be added to video generation systems.

Innovation 2: Separating Semantic and Perceptual Identity into Complementary Injection Pathways

Prior end-to-end customization methods (ID-Animator, ConsisID, MovieGen, ConceptMaster, Phantom, SkyReels-A2) typically inject identity through a single pathway: a learned adapter network that extracts features from the reference image and injects them at one or more layers of the video generation model. Whether the features come from a face recognition network (ArcFace), a general vision encoder (CLIP, DINO), or a custom-trained extractor, the implicit assumption is that a single feature representation can simultaneously capture both the semantic-level identity information (what kind of subject this is, its high-level appearance categories) and the perceptual-level identity information (the specific fine-grained details that distinguish this instance from all others of the same category).

HunyuanCustom breaks this assumption by deploying two complementary identity pathways that serve different representational functions and use different mechanisms:

LLaVA for semantic identity: The vision-language model extracts high-level features — category membership, general color and shape attributes, the relationship between the visual subject and the linguistic description — and injects them through the text token stream. The ablation (Section 5.6) reveals that removing LLaVA causes the model to "fail to capture any significant details from the target image," indicating that LLaVA's semantic features are necessary for the model to understand which subject to generate and how it relates to the text description.
Temporal concatenation for perceptual identity: The raw image latent, concatenated at time -1 with shifted 3D-RoPE position, propagates fine-grained appearance details (exact face structure, specific clothing texture, precise product labeling) through the model's temporal self-attention. The ablation shows that having LLaVA but removing temporal concatenation captures "global identity information but misses detailed identity features."

The intellectual move is recognizing that these two forms of identity information — semantic binding and perceptual detail — are not just different resolutions of the same feature hierarchy, but qualitatively different types of information that benefit from fundamentally different architectural treatments. Semantic binding benefits from cross-modal attention between visual and linguistic representations (LLaVA's native capability). Perceptual detail benefits from direct spatio-temporal feature propagation with minimal compression (temporal concatenation preserves the full VAE latent representation of the reference image without any learned bottleneck).

Comparison to prior work: ConsisID's frequency decomposition (separating low-frequency and high-frequency facial features) shares the intuition that different identity aspects need different treatment, but it operates entirely within the face feature extraction stage, not at the architectural level of how features enter the generation model. HunyuanCustom's separation is architectural — two different mechanisms, at different points in the computation graph, with different representational formats — not just two different feature extractors feeding into the same injection mechanism.

Evidence: The ablation in Figure 11 directly demonstrates the complementarity. Without LLaVA: no identity preservation at all. With LLaVA but without temporal concatenation: captures global identity (the right person category, roughly right appearance) but misses details. With both: captures both. The fact that neither pathway alone is sufficient and that they capture incommensurable aspects of identity is the empirical validation of this conceptual separation.

Significance: This is a diagnostic reframing rather than a new technique. It suggests that future work on identity-preserving generation should not ask "what's the best feature extractor for identity?" but rather "what are the distinct representational requirements of identity, and which architectural mechanisms best serve each?" The answer may be different for different subject types (faces vs. objects vs. scenes) and different generation tasks, but the diagnostic framework — separate the semantic binding problem from the perceptual detail propagation problem — is general.

Innovation 3: The Copy-Paste Problem as a RoPE Positional Encoding Phenomenon, and Its Architectural Solution

The "copy-paste problem" in customized generation — where the model simply reproduces the reference image statically in the first frame rather than generating the subject in novel poses and motions — has been observed in prior work (MovieGen notes it, Keling exhibits it in the paper's comparisons in Figure 4, row 2). The typical understanding is that this is an optimization problem: the model finds a shortcut solution (copy the reference) that minimizes training loss but fails to generalize to novel poses.

HunyuanCustom offers a more precise diagnostic: the copy-paste problem is specifically a positional encoding phenomenon. When the reference image latent is concatenated along the temporal axis and assigned the same spatial RoPE coordinates as the generated frames, the model's spatio-temporal attention can trivially map reference pixels to corresponding generated frame pixels — pixel (i,j) in the reference image attends to and is attended by pixel (i,j) in every generated frame, creating an identity mapping that requires almost no representational work. The model's pretrained temporal attention priors, which normally propagate information across frames with natural motion transformations, are hijacked into a static-copy regime because the positional encoding makes static copying the easiest learned behavior.

The solution — spatial shifting of the reference image's RoPE by (+w, +h) — transforms this from an optimization problem into an architectural one. By placing the reference image latent in a completely disjoint spatial region of the RoPE space (shifted by the full latent width and height), the model can no longer rely on position-identity shortcuts. A reference pixel at (-1, i, j) and a generated frame pixel at (0, i, j) have completely different RoPE encodings. To propagate identity information, the model must learn to attend across these disjoint spatial regions, which forces it to develop a more abstract identity transfer mechanism that can handle pose variations, viewpoint changes, and motion.

Why this matters beyond this specific solution: The insight that positional encoding design can induce or prevent specific failure modes in video generation is broadly applicable. Any condition that is concatenated along the temporal axis — not just reference images, but also conditioning videos, depth maps, semantic layouts — interacts with the model's spatio-temporal attention through its RoPE encoding. The choice of position assignment is not a minor hyperparameter; it fundamentally shapes what kinds of information transfer are easy vs. hard for the model to learn. The spatial shift trick is a specific instance of a more general principle: when concatenating conditioning latents, deliberately misalign their spatial RoPE from the generated frames' RoPE to force the model to learn nontrivial information transfer rather than position-identity shortcuts.

Evidence: The channel-concatenation ablation (Figure 11) shows the alternative: channel concatenation (which avoids the spatial RoPE issue entirely by putting the reference in a different channel dimension) produces "severe blurring in the initial frames" similar to Vidu's behavior. This suggests that both naive temporal concatenation (without spatial shift) and channel concatenation are suboptimal — the former causes copy-paste, the latter causes blur. The spatial-shifted temporal concatenation navigates between these failure modes.

Significance: This is a mechanistic insight — it explains a known failure mode at the level of architectural mechanism rather than training dynamics. It is precisely the kind of insight that enables principled architecture design: rather than treating copy-paste as a mysterious optimization problem to be mitigated with more data or regularization, the paper identifies the specific representational cause and addresses it architecturally.

Innovation 4: Audio-Driven Video Customization as a Generalization of Audio-Driven Animation, Enabled by Identity-Audio Disentanglement

Audio-driven human animation — making a face in a photo speak — is a well-studied problem with dedicated architectures (Loopy, SONIC). These methods take a single human image and an audio clip and output a talking-head video where the person's face animates to match the speech. Critically, the posture, clothing, background, and camera viewpoint are all frozen from the input image. This is not a limitation of implementation — it is inherent to the formulation: the input is a single image, so there is no source of information about what the person looks like from other angles, in other poses, or in other scenes.

HunyuanCustom redefines the audio-conditioned generation problem by making identity the anchor rather than making the reference image the anchor. The identity is extracted and preserved through the dual identity injection pathways, which means the model knows what the subject looks like in a representation that generalizes across poses, camera angles, and scenes. The audio conditioning (via AudioNet's spatial cross-attention) is added on top of this generalized identity representation, not on top of a single frozen image. The result is a qualitative expansion of capability: rather than animating the reference image, the system generates a video where the identified subject speaks the audio in any text-described scene, posture, and attire.

What makes this an innovation rather than an incremental improvement: This is not just "talking head but with a different background." The paper shows (Figure 8) examples where the subject appears in period-appropriate attire (Ming Dynasty setting) without explicit prompting, where the subject interacts with other humans and objects while speaking, and where the subject's facial expressions are "vivid and realistic" and distinct from the reference image expression. These are capabilities that are definitionally impossible for image-plus-audio animation systems, because the image provides no information about alternative clothing, new scenes, or expressive variations. The innovation is the reframing of audio-driven generation from animation of a specific image to generation of a specific identity — a conceptual shift enabled by the disentangled architecture that separates identity preservation (what distinguishes this person from all others) from image conditioning (what this person looks like in one specific view).

Comparison to prior work: Prior audio-driven animation methods need to solve a fundamentally different problem: given a single viewpoint, hallucinate realistic motion that matches the audio while maintaining identity. The hallucination of motion from a single image is itself a hard problem. HunyuanCustom bypasses this entirely because it is generating from the video model's learned motion priors, not hallucinating from a single image. The identity constraint ensures the motion is attributed to the correct subject, but the motion itself comes from the model's understanding of how humans move (learned during video pretraining), not from warping a single reference frame.

Evidence: Figure 8 demonstrates qualitatively that the model generates diverse scenes, postures, and interactions while preserving identity. The paper does not provide quantitative metrics for the audio-driven setting (a limitation they acknowledge implicitly by only reporting metrics for the image-driven task in Table 1), but the qualitative range of outputs — from close-up speaking to full-body interactions in historical settings — demonstrates capabilities beyond any single-image animation method.

Significance: This is a reframing of the problem class. Audio-driven generation moves from being an image animation problem to being an identity-conditioned video generation problem, with the same architectural primitives (identity injection, audio injection) as the rest of the system. This unification — where audio-driven generation is not a separate model or a separate training objective but simply another orthogonal conditioning pathway on the same identity anchor — is what makes the multi-modal vision coherent.

5. Experimental Analysis

Evaluation Methodology

Dataset. All experiments use internally collected and curated video data, described in detail in Section 3.3, spanning eight categories (humans, animals, plants, landscapes, vehicles, objects, architecture, and anime) and incorporating open-source datasets such as OpenHumanvid after rigorous processing. For quantitative evaluation of image-driven single-subject customization, the authors generate 100 videos with human identities and 100 videos with nonhuman identities, for a total of 200 evaluation samples per method, using prompts and reference images that are not otherwise specified as coming from a standardized public benchmark.
Base model(s). All work builds on HunyuanVideo (Kong et al., 2024), a 13B-parameter diffusion transformer video generation model that uses Flow Matching, 3D-RoPE positional encodings, and a causal 3D VAE. The LLaVA model (Liu et al., 2023a), which serves as the text encoder in HunyuanVideo, is also fully fine-tuned. The paper does not report experiments on other base model families or scales.
Metrics. The paper employs five metrics, each assessed on the generated video output:
- ID consistency (Face-Sim): ArcFace (Deng et al., 2019) extracts face embeddings from the reference image and each generated frame; the metric is the average cosine similarity over all frames.
- Subject similarity (DINO-Sim): YOLOv11 (Khanam and Hussain, 2024) segments the human subject in each frame; DINO-v2 (Oquab et al., 2023) features are extracted for the reference subject and the segmented regions; the metric is the cosine similarity between these feature representations.
- Text-video alignment (CLIP-B-T): CLIP-B (Radford et al., 2021) computes the alignment score between the input text prompt and the generated video frames.
- Temporal consistency (Temp-Consis): Following VBench (Huang et al., 2024a), CLIP-B similarity is computed between each frame and its adjacent frames, as well as between each frame and the first frame, to assess temporal coherence.
- Dynamic degree (DD): Following VBench (Huang et al., 2024a), this measures the magnitude of object motion across frames.
Baselines. The paper compares against six methods spanning commercial products and open-source models:
- Commercial: Vidu 2.0 (Vidu, 2025), Keling 1.6 (Keling, 2025), Pika (Pika, 2025), and Hailuo (Hailuo, 2025).
- Open-source: SkyReels-A2 (Fei et al., 2025) and VACE (Jiang et al., 2025).
For video-driven video customization (Section 5.5), the comparison set narrows to VACE and Keling. For audio-driven and multi-subject tasks, the comparison is primarily qualitative through figures rather than quantitative metrics, and baseline comparisons for these tasks rely on the same set of methods where applicable (Vidu, Keling, Pika, Hailuo, VACE, SkyReels-A2 for multi-subject; prior audio-driven animation methods Loopy and SONIC are cited in Section 4.1 but not quantitatively compared).
Generation budget / compute accounting. The paper does not measure compute in FLOPs, GPU-hours, or generation budget (number of denoising steps or samples). Comparisons across methods are made on the basis of the same task specification (same reference images, same text prompts) with each model's default inference settings. The paper does not report whether inference compute is matched across methods—for example, whether all methods use comparable numbers of denoising steps, comparable latent resolutions, or comparable total FLOPs per generated video. This is a notable omission for a paper making comparative performance claims.
Cross-validation / statistical protocol. No cross-validation, statistical significance testing, confidence intervals, or error bars are reported. The 200 generated videos per method (100 human, 100 nonhuman) for the quantitative evaluation in Table 1 represent a single evaluation run with no reported measure of variance. Ablation studies (Section 5.6) are presented as qualitative visual comparisons (Figure 11) without quantitative metrics.

Main Quantitative Results

Image-Driven Single-Subject Customization (Table 1, Figures 4 and 5)

The paper's headline quantitative result is that HunyuanCustom achieves the best ID consistency and subject similarity among all compared methods while maintaining competitive text-video alignment and temporal consistency.

Face-Sim (ID consistency): HunyuanCustom achieves the highest score at 0.426, outperforming the next best method, Vidu 2.0 (0.394), by approximately 8%. Hailuo achieves 0.391, Keling achieves 0.367, Pika achieves 0.252, SkyReels-A2 achieves 0.231, and VACE achieves 0.219. The gap between HunyuanCustom and the leading commercial product (Vidu) is moderate but consistent with the paper's claim of best-in-class identity preservation.

DINO-Sim (subject similarity): HunyuanCustom achieves 0.611, again the highest. SkyReels-A2 achieves the second-highest at 0.589, followed by Vidu at 0.550, Keling at 0.546, VACE at 0.544, Pika at 0.480, and Hailuo at 0.474. The gap between HunyuanCustom and the second-place method (SkyReels-A2) is approximately 3.7%. Notably, Hailuo—which performed reasonably on Face-Sim (0.391)—drops to the second-worst DINO-Sim score, consistent with the paper's qualitative observation that Hailuo "can only maintain ID consistency, but not full-body consistency" (Section 5.2, qualitative comparison paragraph). This divergence between face-level and full-subject metrics is precisely the limitation the paper's identity enhancement module is designed to address.

CLIP-B-T (text-video alignment): Hailuo achieves the highest score at 0.318, followed by HunyuanCustom at 0.315, Pika at 0.313, Keling at 0.308, Vidu at 0.298, VACE at 0.291, and SkyReels-A2 at 0.287. HunyuanCustom's score is essentially tied with Hailuo (difference of 0.003) and within 1% of the best score—the paper describes this as "comparable results in prompt following." The fact that Hailuo leads on text alignment while trailing on subject similarity supports the paper's claim that Hailuo follows text instructions well "with only ID consistency, sacrificing the consistency of non-human subjects" (Section 5.2, quantitative comparison paragraph).

Temp-Consis (temporal consistency): Vidu achieves the highest score at 0.975, followed by Keling at 0.974, Pika at 0.969, Hailuo at 0.963, HunyuanCustom at 0.960, SkyReels-A2 at 0.957, and VACE at 0.940. HunyuanCustom sits in the middle of the pack, within ~1.5% of the best score. The paper does not claim superiority on temporal consistency, only that results are "comparable."

Dynamic Degree (DD): Hailuo leads at 34.807, followed by Keling at 26.355, SkyReels-A2 at 19.767, Pika at 18.223, HunyuanCustom at 15.988, Vidu at 9.550, and VACE at 6.681. The paper notes that Vidu and VACE "perform poorly" on this metric, attributing it to "the small size of the model" (Section 5.2, quantitative comparison paragraph). HunyuanCustom's dynamic degree is moderate—neither the best nor the worst—which is reasonable given that aggressive motion can work against identity preservation (fast-moving subjects are harder to keep consistent).

Qualitative observations (Figures 4 and 5): The paper provides detailed qualitative comparisons. For human subjects (Figure 4): Vidu, SkyReels-A2, and HunyuanCustom "achieve relatively good results in prompt alignment and subject consistency," but HunyuanCustom's "video quality is better than Vidu and Skyreels, thanks to the good video generation performance of our base model" (Section 5.2). Keling exhibits "a copy-paste problem" in row 2 where the first frame is identical to the reference image. Pika "performs poorly in consistency and is prone to subtitle problems." Hailuo "can only maintain ID consistency, but not full-body consistency." VACE "cannot maintain ID consistency" (row 7). For object subjects (Figure 5): Keling shows subject motion that is "too fast and blurs" (row 2). HunyuanCustom is described as generating videos "with high identity consistency, while keeping a good generation quality and diversity."

Critical reading of Table 1: The paper's claim that HunyuanCustom "outperforms all existing methods in customized video generation" (Section 1) is supported specifically for the identity preservation metrics (Face-Sim and DINO-Sim). For the other three metrics (text alignment, temporal consistency, dynamic degree), HunyuanCustom does not lead and is often mid-pack. This is consistent with the paper's stated priority ordering (identity first), but it means the claim of overall superiority depends on how one weights identity metrics against quality and diversity metrics. An evaluation that weighted all five metrics equally would not necessarily rank HunyuanCustom first.

Multi-Subject Video Customization (Figure 6, Section 5.3)

Multi-subject customization is evaluated qualitatively, not quantitatively. Figure 6 presents a side-by-side comparison of HunyuanCustom against Vidu, Keling, Pika, Hailuo, VACE, and SkyReels-A2 on multi-subject generation. The paper reports:

Pika "can generate the specified subjects but exhibits instability in video frames, with instances of a man disappearing in one scenario and a woman failing to open a door as prompted" (Section 5.3).
Vidu and VACE "partially capture human identity but lose significant details of non-human objects, indicating a limitation in representing non-human subjects" (Section 5.3). This is particularly noteworthy given that VACE is the only other multi-modal conditioning framework—it supports multiple subjects in principle but fails to preserve non-human object details in practice.
SkyReels-A2 "experiences severe frame instability, with noticeable changes in chips and numerous artifacts" (Section 5.3).
HunyuanCustom "effectively captures both human and non-human subject identities, generates videos that adhere to the given prompts, and maintains high visual quality and stability" (Section 5.3).

The virtual human advertisement application (Figure 7) demonstrates HunyuanCustom taking a human image and a product image as input and generating advertisement videos where the identified person interacts naturally with the identified product while preserving the product's text details. This is presented as an application that "previous methods cannot achieve" (Section 5.3).

Critical assessment: The complete absence of quantitative metrics for multi-subject customization is a significant limitation. The paper does not report Face-Sim, DINO-Sim, or any other metric for the multi-subject setting, making it impossible to quantify how much multi-subject performance degrades relative to single-subject performance, or to verify the paper's claim that HunyuanCustom "effectively captures both human and non-human subject identities" beyond a handful of cherry-picked examples. The qualitative results in Figure 6 show six examples—an extremely small sample for evaluating a capability the paper positions as central to its contributions.

Audio-Driven Video Customization (Figures 8 and 9, Section 5.4)

Audio-driven customization is evaluated purely qualitatively through application demonstrations. Figure 8 shows audio-driven single-subject customization where HunyuanCustom generates videos of the identified subject speaking the provided audio in text-described scenes and postures. The paper highlights several qualitative capabilities:

The subject interacts with other subjects or humans while speaking (rows 3-6 of Figure 8).
The subject appears in period-appropriate attire when the scene is set in the Ming Dynasty without explicit clothing prompts (row 7)—evidence of "robust world modeling and generalization capabilities" (Section 5.4).
A woman exhibits "vivid and realistic expressions distinct from the input image" (row 8)—evidence that the model generates novel expressions rather than copying the reference expression, which prior audio-driven animation methods cannot do since they are constrained to animate the single input image.

Figure 9 demonstrates audio-driven multi-subject customization in a virtual try-on scenario: the subject wears specified attire, speaks the given audio, and appears in a scene that differs from any reference image, all while preserving both the human identity and the garment identity.

Critical assessment: This evaluation is entirely qualitative. There are no metrics for audio-visual synchronization (lip-sync accuracy), no comparison against dedicated audio-driven animation methods (Loopy, SONIC) on their home territory (talking-head generation with identity from a single image), and no quantitative assessment of how well identity is preserved when audio conditioning is added versus when it is not. The claim that HunyuanCustom enables audio-driven customization with "high diversity, while keeping the identity well" (Section 5.4) is visually plausible from the provided examples but is not quantitatively substantiated. This is a significant gap given that the paper positions audio-driven customization as one of its three multi-modal extensions (alongside video-driven) and as a qualitative advance over prior audio-driven animation.

Video-Driven Video Customization (Figure 10, Section 5.5)

Video-driven customization (specifically, video subject replacement) is evaluated qualitatively against VACE and Keling in Figure 10. The task is: given a source video, object masks indicating regions to be replaced, and a target subject image, replace the masked region with the target subject while preserving the background, motion, and scene integration.

The paper reports three distinct failure modes across the compared methods:

VACE "suffers from boundary artifacts due to strict adherence to the input masks, resulting in unnatural subject shapes and disrupted motion continuity" (Section 5.5).
Keling "exhibits a copy-paste effect, where subjects are directly overlaid onto the video, leading to poor integration with the background" (Section 5.5).
HunyuanCustom "effectively avoids boundary artifacts, achieves seamless integration with the video background, and maintains strong identity preservation" (Section 5.5).

Critical assessment: The evaluation is qualitative only. There are no metrics for background preservation (how well the non-masked regions of the source video are reproduced), temporal coherence of the replaced subject, or integration quality (how naturally the replaced subject blends with the source video's lighting and occlusion patterns). The comparison set is limited to two baselines (VACE and Keling), which is reasonable since most other compared methods (Pika, Hailuo, SkyReels-A2, Vidu) may not support video-conditioned replacement but means that the evaluation cannot quantify HunyuanCustom's advantage over a broader field. The number of examples shown is small (Figure 10 shows a limited set of qualitative results), and there is no information about how many test cases were run or what the failure rate is.

Ablation Studies and Robustness Checks

The paper presents one ablation study (Section 5.6, Figure 11) comparing four model variants:

Full HunyuanCustom (LLaVA text-image fusion + temporal concatenation identity enhancement)
Without LLaVA (identity enhancement only, no LLaVA-based text-image interaction)
Without identity enhancement (LLaVA only, no temporal concatenation)
Channel concatenation instead of temporal concatenation (LLaVA + channel-concatenated image latents)

LLaVA text-image fusion module (removing LLaVA entirely): The model without LLaVA "exhibits poor identity preservation" and "fails to capture any significant details from the target image" (Section 5.6). This ablation result (visible in Figure 11) demonstrates that LLaVA "not only conveys prompt information but also extracts key identity features." This is a strong result: LLaVA is not merely a convenience (using an existing text encoder) but is essential for the identity preservation task, suggesting that semantic-level identity binding through a vision-language model is a necessary component that temporal concatenation alone cannot supply.

Identity enhancement module (temporal concatenation): The model with LLaVA but without identity enhancement "captures global identity information but misses detailed identity features" (Section 5.6). This result (Figure 11) confirms the paper's two-pathway design: LLaVA handles high-level semantic identity (category, general appearance), while temporal concatenation handles fine-grained perceptual details (exact texture, precise features). Neither pathway alone is sufficient; both are necessary.

Channel concatenation vs. temporal concatenation: The channel-concatenation variant "shows poor generation quality" and "suffers from a severe blurring effect in the initial frames, similar to results from Vidu" (Section 5.6). Despite capturing identity well, the blur artifact makes it practically unusable. The paper interprets this as evidence that temporal concatenation "aids in effectively capturing target information through strong temporal modeling priors and minimizes the impact on generation quality." This is a non-obvious finding: both concatenation strategies provide the model with the same reference image information; the difference in output quality stems entirely from how that information is structured for the model's attention mechanism. Temporal concatenation with 3D-RoPE is superior because the pretrained temporal attention priors in HunyuanVideo naturally propagate information across frames—a capability that channel concatenation cannot leverage.

What is missing from the ablation study:

No quantitative ablation metrics. Figure 11 is a qualitative figure; the paper does not report Face-Sim, DINO-Sim, or any other metric for the ablated variants. This makes it impossible to quantify the contribution of each component—we cannot say that LLaVA improves Face-Sim by X points or that temporal concatenation improves DINO-Sim by Y points. The ablation confirms that both components matter but not how much they matter relative to each other or relative to the full model.
No ablation of the spatial shift in RoPE. The paper argues that spatially shifting the reference image's RoPE by (+w, +h) prevents the copy-paste problem, but this claim is never ablated. A comparison of temporal concatenation with and without the spatial shift would directly test the paper's mechanistic explanation for why temporal concatenation works. Without this ablation, the claim about RoPE spatial shift preventing copy-paste remains an interpretation rather than an empirically verified mechanism.
No ablation of the <SEP> token in the LLaVA template. The paper argues that the <SEP> token prevents image features from overwhelming text comprehension, but this is not ablated.
No ablation of the template strategy (image-embedded vs. image-appended). The paper describes two template strategies but does not report which one is used in the final model or whether the choice matters.
No ablation of AudioNet components. The spatial cross-attention mechanism, the temporal alignment strategy, the λ_A weight, and the choice of using Whisper features are not ablated. The claim that AudioNet is "identity-disentangled" is an architectural claim, not an ablated one—the paper does not show that an alternative audio injection mechanism (e.g., concatenation-based or full spatio-temporal cross-attention) would interfere with identity preservation.
No ablation of video conditioning strategies beyond the concatenation-vs-addition comparison. The paper compares concatenation-based and addition-based video conditioning but does not ablate the four-layer alignment network, the choice of VAE for encoding the conditioning video, or the effect of adding video conditioning on identity preservation metrics.
No ablation of data quality or quantity. The paper emphasizes that "high-quality data plays a crucial role" (Section 3.3) and describes a detailed data processing pipeline, but there is no ablation showing how model performance varies with data quality (e.g., with and without the koala-36M filtering, with different scene detection thresholds) or data quantity (scaling the training set size).

Critical Assessment

This section evaluates whether the paper's experiments genuinely support its central claims, identifies specific evidential gaps, and surfaces weaknesses that qualify the reported results.

The paper's primary empirical claim is that HunyuanCustom "significantly outperforms state-of-the-art open- and closed-source methods in terms of ID consistency, realism, and text-video alignment" (Abstract).

This claim is partially supported for human-centric single-subject customization, where quantitative metrics (Table 1) show HunyuanCustom leading on Face-Sim (0.426) and DINO-Sim (0.611), and being competitive on CLIP-B-T (0.315, within 1% of the best score). However, several qualifications are necessary:

The metrics evaluate different things than the words suggest. "Realism" is not directly measured—the paper does not report FVD (Fréchet Video Distance), IS (Inception Score), user studies, or any perceptual realism metric. The metrics that are reported (Face-Sim, DINO-Sim) measure identity preservation, not realism. CLIP-B-T measures text alignment, not realism. Temporal consistency and dynamic degree measure temporal coherence and motion magnitude, which relate to but do not directly capture realism. A method with high Face-Sim and DINO-Sim could still produce visibly unnatural videos (wrong lighting, unnatural motion, anatomical implausibilities)—the metrics would not capture this. The paper's qualitative claims about "video quality" (e.g., "our video quality is better than Vidu and Skyreels") are not operationalized in any metric.

The evaluation sample size is small and unspecified in critical ways. The 200 videos (100 human, 100 nonhuman) for the quantitative evaluation represent a single run with no variance estimates. The test prompts and reference images are not specified or made available, so the evaluation is not reproducible. The paper does not describe how the 200 test cases were selected—whether they were randomly sampled, adversarially selected, or curated—which matters for assessing whether the results generalize. For a paper claiming to outperform both commercial products (backed by large engineering teams with proprietary training data) and open-source methods, 200 examples is a thin empirical basis.

The comparison is not compute-matched. Different methods may use vastly different amounts of inference compute. HunyuanVideo is a 13B-parameter model; Vidu, Keling, Pika, and Hailuo are commercial products with undisclosed model sizes and inference costs. VACE is built on Wan (Wang et al., 2025), a different base model with different compute requirements. The paper does not report or control for inference FLOPs, denoising steps, or generation time. If HunyuanCustom uses more inference compute than the compared methods, its superior identity preservation might reflect a compute advantage rather than an architectural one. This is particularly relevant because the paper's architectural contributions (temporal concatenation adding one frame to the sequence length, LLaVA adding 576 image tokens to the text sequence) increase the per-step computational cost relative to the base HunyuanVideo model—a cost that is not quantified or compared against baselines.

The baseline evaluation may not use each method optimally. The paper does not describe how prompts were constructed for each baseline method or whether each method's recommended best practices were followed. Commercial products like Vidu and Keling may support different prompt formats, negative prompts, or tuning parameters that the paper's evaluation did not exploit. This is a standard challenge in comparing against closed-source methods, but it means the reported baseline numbers may understate what those methods can achieve with optimal use.

The paper claims that HunyuanCustom "enables audio- and video-conditioned generation" through modality-specific injection mechanisms that are "effectively disentangled with the image-level identity condition module" (Section 3.1), achieving "robustness across downstream tasks" (Abstract).

This claim is only weakly supported by the reported experiments.

Audio-driven customization has no quantitative evaluation whatsoever. The paper presents qualitative examples in Figures 8 and 9, describing them in enthusiastic terms ("vividly," "robust," "significantly enhance"), but provides no metrics. There is no comparison against dedicated audio-driven animation methods (Loopy, SONIC) on standard metrics like lip-sync accuracy (LSE-D, LSE-C), no ablation showing that the AudioNet pathway does not degrade identity preservation relative to the audio-free baseline, and no quantification of how well the model handles diverse audio conditions (different speakers, languages, background noise, music vs. speech). The claim that HunyuanCustom "extends conventional audio-driven human animation by allowing the specified identity to perform freely in arbitrary scenes and actions" (Section 3.2) is an exciting capability, but without quantitative evaluation, the reader cannot assess how often this capability succeeds, how often identity degrades, or how the audio-visual synchronization quality compares to dedicated methods.

Video-driven customization shares these limitations. The comparison in Figure 10 against VACE and Keling is qualitative only and shows a small number of examples. The paper claims HunyuanCustom "effectively avoids boundary artifacts, achieves seamless integration with the video background, and maintains strong identity preservation" (Section 5.5), but provides no metrics for background preservation (e.g., PSNR or LPIPS between the source and generated video in non-masked regions), no metrics for integration quality (e.g., a user study or learned perceptual metric), and no quantification of how often the replacement succeeds versus produces artifacts.

The disentanglement claim is architectural, not empirical. The paper argues that audio and video conditioning pathways are "effectively disentangled" from identity pathways because they use different operations (spatial cross-attention vs. temporal concatenation vs. latent addition). But the paper does not empirically demonstrate that this disentanglement works as intended—for example, by showing that identity metrics remain unchanged when audio or video conditioning is enabled versus disabled. Without this evidence, the architectural design is a plausible hypothesis about why the model might work well, but not a demonstrated fact.

The paper claims that HunyuanCustom's virtual human advertisement application (Figure 7) demonstrates capabilities that "previous methods cannot achieve" (Section 5.3).

This is a limited form of evidence. Figure 7 shows generated examples, but the paper does not report:

How many advertisement videos were generated and how many were usable (success rate).
Whether the product text and details are accurately preserved (a critical requirement for real advertising—a product with garbled text is unusable regardless of identity quality).
Whether the human-product interaction is natural across a range of product types and interaction types, or whether the examples in Figure 7 were cherry-picked from many attempts.

The examples are visually impressive, but the claim that previous methods "cannot achieve" this is strong. Prior methods like ConceptMaster and SkyReels-A2 support multi-subject generation; the paper's own Figure 6 shows that SkyReels-A2 and Pika can generate multi-subject videos, albeit with quality issues. The claim that HunyuanCustom uniquely enables this application is better supported by the qualitative comparison in Figure 6 showing that competing methods lose non-human subject details, but the evidence is still qualitative and sample-limited.

The paper's ablation study (Section 5.6) supports the claim that both LLaVA and temporal concatenation are necessary, but is incomplete in critical ways.

The ablation demonstrates that removing either LLaVA or temporal concatenation degrades performance qualitatively (Figure 11). This supports the paper's two-pathway design. However, the ablation has significant gaps:

No quantitative ablation metrics: Without Face-Sim, DINO-Sim, or other metrics for the ablated variants, the ablation cannot distinguish between a component that provides a small improvement (e.g., +0.01 Face-Sim) and one that is essential (e.g., +0.20 Face-Sim). The qualitative differences in Figure 11 are visible, but quantified differences would allow the reader to weigh the relative importance of each component and assess whether the added complexity of the two-pathway design is justified by the magnitude of improvement.

No ablation on multi-modal interaction: The ablation only covers the image-driven single-subject setting. It does not ablate AudioNet or the video injection module, and it does not examine how the components interact when multiple modalities are combined. For example: does enabling audio conditioning change identity preservation? Does combining audio and video conditioning cause interference? The paper's central architectural claim (disentanglement) predicts that these combinations should not cause interference, but this prediction is never tested.

No ablation of training data or training procedure: The paper describes a detailed data processing pipeline (Section 3.3) and argues that "high-quality data plays a crucial role" and that "the incorporation of high-standard data is crucial for achieving substantial improvements in model performance," but provides no ablation varying data quality or quantity. The reader cannot assess whether the performance gains come from the architectural innovations or from the carefully curated training data.

The ReST $^{EM}$ ablation from the reference example paper is not present here. The paper describes using full fine-tuning rather than parameter-efficient methods and argues this "unlocks full potential," but does not ablate this choice (e.g., comparing full fine-tuning against LoRA fine-tuning of the same architecture). This makes it impossible to assess whether the performance gains are attributable to the architectural design or simply to the greater capacity afforded by full fine-tuning.

Summary of evidential strengths and weaknesses:

Strengths:

The quantitative comparison on single-subject customization (Table 1) is the strongest empirical contribution, showing HunyuanCustom leading on identity metrics (Face-Sim, DINO-Sim) while maintaining competitive text alignment and temporal consistency against a broad set of commercial and open-source baselines.
The qualitative comparisons (Figures 4-6, 8-10) are comprehensive in scope, covering single-subject, multi-subject, audio-driven, and video-driven settings.
The ablation study (Figure 11), despite its lack of metrics, clearly demonstrates that both LLaVA and temporal concatenation are individually necessary—removing either causes qualitatively obvious degradation.

Weaknesses:

The evaluation has no statistical rigor (no confidence intervals, no significance tests, no cross-validation, single evaluation run).
The test set is small (200 videos) and its construction is not described in reproducible detail.
There is no compute-matching across methods, making it possible that HunyuanCustom's advantages reflect greater inference compute rather than better architecture.
Audio-driven and video-driven customization—two of the paper's three multi-modal extensions—have zero quantitative evaluation. The paper's claims about these capabilities rest entirely on qualitative examples of unspecified selection criteria.
The disentanglement principle—the paper's central architectural claim—is not empirically tested. No experiment demonstrates that enabling audio or video conditioning leaves identity metrics unchanged, or that the separate injection pathways genuinely prevent interference.
The ablation study has no quantitative metrics, no ablation of key design choices (RoPE spatial shift, <SEP> token, template strategy, full fine-tuning vs. PEFT), and no ablation of data quality or quantity.
The metrics themselves (Face-Sim, DINO-Sim, CLIP-B-T) do not directly measure "realism," "video quality," or "diversity"—terms the paper uses in its qualitative descriptions without operationalization.
No user study is reported, which would be particularly valuable for assessing subjective qualities like "naturalness of interaction," "realism," and "seamless background integration" that the paper claims but cannot measure with automated metrics.

The paper demonstrates that HunyuanCustom achieves strong identity preservation in image-driven single-subject video customization, and provides suggestive qualitative evidence that this capability extends to multi-subject, audio-driven, and video-driven settings. The quantitative evidence for the multi-modal extensions and the empirical validation of the disentanglement principle are, however, substantially incomplete. The paper's architectural contributions are well-motivated and the qualitative results are visually impressive, but the experimental evaluation does not fully rise to the level of rigor needed to substantiate all of its claims.

6. Limitations and Trade-offs

The assumption or constraint. The paper positions audio-driven and video-driven customization as two of its three primary modality extensions (Sections 3.2, 4.1, 4.2), alongside the core image-driven single-subject capability. The Abstract claims that HunyuanCustom enables these tasks and that the authors "validate its robustness across downstream tasks, including audio and video-driven customized video generation." Yet the evaluation of both modalities is entirely qualitative — Figures 8, 9, and 10 present example outputs with descriptive commentary, but no metric (Face-Sim, DINO-Sim, lip-sync accuracy, background preservation error, integration quality, or any other) is reported for either modality. The paper does not state this as an explicit limitation, but the absence of quantitative evaluation is a de facto constraint: the reader cannot assess how well these capabilities work beyond the handful of curated examples shown.

The consequence. Without quantitative evaluation, the paper's core architectural claim — that the disentangled modality injection pathways (AudioNet for audio, patchify-based alignment for video) enable robust multi-modal customization without degrading identity preservation — remains architecturally motivated but empirically unverified. A practitioner considering deploying HunyuanCustom for audio-driven virtual try-on (Section 5.4, Figure 9) or video subject replacement (Section 5.5, Figure 10) has no basis for estimating the success rate, the failure modes, or the conditions under which performance degrades. Several specific failure modes cannot be assessed:

Audio-identity interference: Does enabling AudioNet change Face-Sim or DINO-Sim scores relative to audio-free generation? If identity preservation degrades when audio conditioning is active, the disentanglement design may not work as intended.
Lip-sync quality: The paper shows examples of subjects speaking (Figure 8), but without metrics like LSE-D or LSE-C (standard in the audio-driven animation literature that the paper cites — Loopy, SONIC, LatentSync), the reader cannot assess whether HunyuanCustom's audio-visual synchronization approaches the quality of dedicated methods.
Video editing fidelity: For the video subject replacement task (Figure 10), the paper claims "seamless integration with the video background," but provides no measure of background preservation (e.g., PSNR, LPIPS, or SSIM between source and output in non-masked regions) and no measure of subject integration quality (e.g., a user study or learned perceptual metric).

What evidence exists in the paper. The paper provides qualitative figures only:

Figure 8: Eight rows of audio-driven single-subject generation examples, annotated with descriptive observations (e.g., "vivid and realistic expressions," "period-appropriate attire," "effective interaction with other subjects").
Figure 9: Four rows of audio-driven multi-subject virtual try-on examples, with no quantitative annotation.
Figure 10: A side-by-side qualitative comparison of video subject replacement against VACE and Keling, showing a limited number of examples.

The paper does not report how many test cases were evaluated for these modalities, how examples were selected for the figures, or what the failure rate is. The quantitative metrics in Table 1 cover only image-driven single-subject customization — they do not extend to audio or video conditioning.

Mitigation status. The paper does not acknowledge this as a limitation. There is no discussion of why quantitative evaluation was not performed for the multi-modal extensions, no commitment to future quantitative benchmarking, and no stated plan to release evaluation protocols or test sets that would enable reproducible assessment of audio and video conditioning performance. The paper's claims about these modalities ("robust world modeling," "seamless integration," "significantly enhance") rest entirely on the qualitative evidence in the figures, which is insufficient to support deployment decisions.

Single Base Model, Single Dataset Family: No Evidence of Cross-Architecture Generalization

The assumption or constraint. All experiments use HunyuanVideo (Kong et al., 2024) as the base generation model — a 13B-parameter diffusion transformer with 3D-RoPE, causal 3D VAE, and Flow Matching training. The paper does not evaluate any of its architectural contributions on other video generation model families (e.g., Wan, CogVideoX, Allegro, Sora-like architectures) or other model scales. The paper states in Section 4 that "our video quality is better than Vidu and Skyreels, thanks to the good video generation performance of our base model, i.e., Hunyuanvideo-13B," which explicitly ties output quality to the base model rather than the customization architecture. The training data is internally collected and curated (Section 3.3), spanning eight categories but not publicly released or benchmarked against a common test set.

The consequence. Several of the paper's key architectural claims may be specific to the HunyuanVideo base model and would not transfer to other architectures:

The temporal concatenation + 3D-RoPE identity enhancement strategy relies on the base model's "strong temporal modeling priors" and "efficient information transmission capability in the temporal dimension" (Section 3.4). A base model with weaker temporal attention (e.g., one that uses 2D spatial attention with separate temporal convolution blocks rather than full 3D spatio-temporal attention) might not propagate identity information effectively from a single concatenated reference frame. The paper's ablation showing that channel concatenation produces "severe blurring" (Figure 11) is HunyuanVideo-specific — a different base model architecture might handle channel concatenation differently.
The LLaVA text-image fusion module leverages the fact that HunyuanVideo already uses LLaVA as its text encoder. A base model using a different text encoder (e.g., CLIP, T5) would require re-engineering this component, and the paper provides no evidence that the two-pathway identity injection (semantic via language model + perceptual via temporal concatenation) is beneficial outside the LLaVA context.
The Flow Matching training objective with logit-normal timestep sampling is specific to HunyuanVideo. The paper does not demonstrate that the architectural contributions would work under standard diffusion (noise prediction with linear schedule) or under other generative frameworks.

A practitioner using a different base video generation model (or planning to build on a future release) cannot determine from this paper whether the architectural principles — disentangled modality injection, dual-pathway identity preservation, AudioNet's spatial cross-attention — are generalizable or HunyuanVideo-specific.

What evidence exists in the paper. The paper explicitly attributes some advantages to the base model. In the qualitative comparison (Section 5.2), the authors state that HunyuanCustom's video quality advantage over Vidu and SkyReels-A2 is "thanks to the good video generation performance of our base model, i.e., Hunyuanvideo-13B." This is an honest attribution but it means that a practitioner using a weaker base model should not expect to achieve the same quality, even with the same customization architecture. The paper provides no ablation where the customization architecture is ported to a different base model, and no experiment varying base model scale to assess whether the identity preservation gains scale with model size.

Mitigation status. The paper does not claim cross-architecture generalization as a contribution — it presents HunyuanCustom as a specific system built on HunyuanVideo. However, the paper also frames its architectural innovations (disentanglement principle, dual-pathway identity injection, AudioNet's spatial cross-attention) as general design insights, not as HunyuanVideo-specific implementation details. Without cross-architecture validation, the generalizability of these insights remains an open question. The authors do not discuss this limitation or suggest multi-architecture evaluation as future work.

Difficulty Estimation and Allocation Strategy Are Entirely Absent — the Model Applies Uniform Computation to All Inputs

The assumption or constraint. The paper presents HunyuanCustom as a fixed-architecture, fixed-computation inference pipeline: every input — regardless of the number of subjects, the complexity of the text prompt, the presence of audio or video conditioning, or the difficulty of preserving the specific identity — receives the same architectural treatment with the same computational budget. There is no mechanism for adaptive compute allocation: no difficulty estimation, no early exiting, no variable denoising steps based on prompt complexity, and no fallback or escalation strategy for cases where the model's initial output is unsatisfactory.

This contrasts sharply with the reference paper's central innovation — compute-optimal test-time scaling — which demonstrated 4× efficiency improvements by adaptively allocating inference compute based on estimated problem difficulty. HunyuanCustom makes no attempt at adaptive allocation, which is a missed opportunity given the clear difficulty variation across the paper's own qualitative results: some subjects and scenes are harder to preserve than others (multi-subject interactions, fast motion, complex backgrounds), and some conditioning modalities (audio + video simultaneously) likely create harder generation problems than simpler single-modality cases.

The consequence. A practitioner deploying HunyuanCustom faces an all-or-nothing cost structure: every generation request invokes the full 13B-parameter model with LLaVA features (576 image tokens per subject), temporal concatenation (adding m frames for m subjects to the sequence length), and possibly AudioNet cross-attention and video conditioning addition, all running through the full denoising schedule. The computational cost per generated video is:

Substantially higher than the base HunyuanVideo model due to the additional tokens (LLaVA's 576 image tokens expand the text sequence; temporal concatenation adds m frames to the latent sequence, increasing self-attention cost by approximately (f+m)²/f² relative to the base model).
Not quantified anywhere in the paper. The paper does not report inference time, GPU memory consumption, FLOPs per generated video, or any comparison of computational cost against the baseline methods (Vidu, Keling, Pika, etc.). As noted in Section 5, the evaluation is not compute-matched across methods — HunyuanCustom may achieve its identity preservation advantage partly by spending more inference compute per video than the compared methods.

Furthermore, because there is no difficulty estimation, the model cannot warn the user when a request is likely to fail. The paper's qualitative results show variation in output quality (some examples in the figures are clearly more successful than others), but the system provides no mechanism to detect low-confidence generations or to allocate additional compute (e.g., more denoising steps, multiple samples with best-of-N selection) to challenging cases.

What evidence exists in the paper. The paper provides no data on computational cost, no measurement of inference time or memory, and no analysis of how cost scales with the number of subjects or conditioning modalities. The reader cannot determine whether the 8% Face-Sim improvement over Vidu (0.426 vs. 0.394) comes at a 2×, 5×, or 10× inference cost premium. The paper's silence on this topic is a significant gap for any practitioner evaluating deployment feasibility.

Mitigation status. The paper does not acknowledge this as a limitation, does not report computational cost, and does not propose adaptive compute allocation as future work. This is a structural limitation of the paper's evaluation paradigm: it treats video customization as a single uniform task rather than a distribution of tasks with varying difficulty, and consequently misses the opportunity to explore efficiency-vs-quality tradeoffs that are central to practical deployment.

The Copy-Paste and Artifact Prevention Mechanisms Are Incompletely Ablated and Their Failure Conditions Are Unknown

The assumption or constraint. The paper introduces two specific architectural mechanisms to prevent known failure modes in customized generation:

The spatial shift of the reference image's 3D-RoPE by (+w, +h) (Section 3.4, Equation 1) is motivated as a copy-paste prevention measure: "to prevent the model from simply copying and pasting the target image into the generated frames, we introduce a spatial shift for the image latents." The paper argues that without this shift, the model's temporal attention can trivially map reference pixels to corresponding generated frame pixels via position-identity shortcuts.
The mask data augmentation strategies (Section 3.3, "Mask Data Augmentation") — specifically mask dilation and conversion of masks to bounding boxes — are introduced to prevent overfitting in video subject replacement: "directly using the extracted subject masks for training can lead to overfitting when replacing objects of different types or shapes."

Both mechanisms are presented as solutions to specific failure modes, but neither is ablated. The paper provides no experiment showing that the copy-paste problem occurs without the RoPE spatial shift and is resolved with it, and no experiment comparing video replacement quality with and without mask augmentation.

The consequence. The reader cannot determine:

Whether the RoPE spatial shift actually prevents copy-paste. The paper's qualitative comparison shows that Keling exhibits copy-paste (Figure 4, row 2) and that HunyuanCustom does not, but this is a cross-method comparison, not an ablation of the spatial shift within HunyuanCustom. The difference could be due to other architectural factors (LLaVA identity binding, training data construction, the base model's generation priors) rather than the RoPE shift specifically. Without a controlled ablation (HunyuanCustom with and without the spatial shift), the paper's mechanistic explanation for why copy-paste is avoided is an untested hypothesis.
When the RoPE spatial shift might fail. If the shift (+w, +h) is sufficient to prevent copy-paste at the current latent resolution, would it remain sufficient at higher resolutions (where the relative magnitude of the shift decreases)? Would it prevent copy-paste for very large subjects that occupy most of the frame (where w and h are dominated by the subject)? The paper provides no analysis of these edge cases.
Whether the mask augmentation strategies are necessary or sufficient for artifact-free video replacement. The paper's comparison in Figure 10 shows that HunyuanCustom avoids the boundary artifacts that plague VACE and the copy-paste overlay effect of Keling, but these comparisons involve entirely different models — the differences cannot be attributed to mask augmentation specifically. The paper does not report how often video replacement fails with HunyuanCustom, what the failure modes are, or whether mask augmentation reduces the failure rate.

What evidence exists in the paper. The paper provides qualitative evidence that HunyuanCustom avoids copy-paste (Figures 4, 5) and boundary artifacts (Figure 10), but this evidence compares HunyuanCustom against other methods, not HunyuanCustom with and without the proposed mechanisms. The ablation study (Section 5.6, Figure 11) compares four variants related to the identity injection module but does not include a RoPE spatial shift ablation or a mask augmentation ablation.

Mitigation status. The paper does not acknowledge the absence of these ablations as a limitation. The copy-paste prevention and artifact prevention mechanisms remain architecturally motivated but empirically unvalidated at the component level. A practitioner implementing these mechanisms in a different system cannot assess their individual contributions or their robustness to variations in latent resolution, subject size, or replacement geometry.

The Evaluation Suite Cannot Measure "Realism," "Naturalness," or "Diversity" — Core Claims Made in Qualitative Descriptions

The assumption or constraint. The paper's quantitative evaluation (Table 1) uses five automated metrics: Face-Sim (ArcFace cosine similarity), DINO-Sim (DINO-v2 feature similarity on segmented subjects), CLIP-B-T (CLIP score for text alignment), Temporal Consistency (frame-to-frame CLIP similarity), and Dynamic Degree (motion magnitude from VBench). These metrics are drawn from the standard VBench benchmark suite and prior work on identity-preserving generation.

However, the paper makes claims about output qualities that these metrics cannot measure:

"Realism": The Abstract claims superior performance in "realism," and the qualitative sections describe outputs as achieving "high visual quality and stability" (Section 5.3), "vivid and realistic expressions" (Section 5.4), and "seamless integration" (Section 5.5). None of the five metrics measures perceptual realism. Face-Sim measures identity matching, not whether the face looks photorealistic. DINO-Sim measures feature similarity, not whether the subject looks like a plausible instance of its category. CLIP-B-T measures semantic alignment, not frame-level visual fidelity.
"Naturalness of interaction": The paper claims that in virtual human advertisements, "the interaction between the human and the product appears natural" (Section 5.3). No metric evaluates interaction quality. The metrics cannot distinguish between a video where the human convincingly holds and looks at the product and one where the human and product coexist in the frame but appear disconnected.
"Diversity": The paper claims that audio-driven customization generates videos "across various scenes and postures specified by text prompts with high diversity" (Section 5.4). The Dynamic Degree metric measures motion magnitude but not the diversity of scenes, postures, or generated content across multiple runs with the same input. There is no metric for the variety of outputs generated from the same conditioning, which is important for creative applications where the user wants multiple distinct options.

The consequence. A practitioner reading the paper's claims about "realism" and "naturalness" cannot verify these claims from the reported metrics and cannot benchmark HunyuanCustom against other methods on these dimensions. There are several specific risks:

High Face-Sim + low realism is possible: A model could achieve excellent identity matching while producing videos that are visibly artificial (plastic skin textures, unnatural lighting, anatomical errors) — the metrics would report high Face-Sim and DINO-Sim but the videos would be practically unusable. The paper provides no evidence that HunyuanCustom avoids this failure mode.
The "natural interaction" claim is unverifiable without human evaluation. Interaction quality is inherently subjective and multi-faceted (spatial plausibility of contact, temporal coordination of action, attention/gaze consistency between subjects). Automated metrics cannot assess these qualities — a user study or human evaluation benchmark would be needed, and the paper provides neither.
**"High diversity" can mean the model simply generates different random variations of the same basic output, or it can mean the model genuinely explores different scene interpretations, poses, and compositions. The metrics cannot distinguish between these — two videos with very different scenes and actions could have similar Dynamic Degree scores if the motion magnitudes are comparable.

What evidence exists in the paper. The paper relies on qualitative figures (4-11) and author commentary to support claims about realism, naturalness, and diversity. The quantitative metrics in Table 1 evaluate identity preservation, text alignment, temporal coherence, and motion magnitude — all legitimate and useful metrics, but collectively insufficient to support the broader qualitative claims. The paper does not report FVD (Fréchet Video Distance), IS (Inception Score), KID (Kernel Inception Distance), a user study, or any other perceptual quality or diversity metric.

Mitigation status. The paper does not acknowledge the gap between its qualitative claims and its quantitative metrics. No user study is proposed as future work. The Abstract's claim about "realism" is presented as if it follows from the reported metrics, but it does not — Face-Sim, DINO-Sim, CLIP-B-T, temporal consistency, and dynamic degree collectively do not measure realism. This is a mismatch between the paper's rhetorical claims and its empirical evidence that practitioners should note when evaluating the method for applications where perceptual realism is a hard requirement (e.g., advertising, virtual try-on, professional video production).

No Latency or Throughput Analysis Despite Sequential Dependencies in the Architecture

The assumption or constraint. The paper's architectural design introduces several sequential computational dependencies that have implications for latency (wall-clock time to generate one video) and throughput (videos generated per unit time on fixed hardware), but the paper provides no analysis of either. The relevant architectural features include:

Temporal concatenation adds m frames to the latent sequence (one per reference subject). Since the diffusion transformer's self-attention cost scales quadratically in the sequence length, adding m frames increases the per-denoising-step cost by a factor of approximately ((f+m)/f)². For f = 16 frames (typical after VAE temporal compression) and m = 2 subjects, this is roughly a 27% increase in self-attention cost per step. For m = 4 subjects, the increase is approximately 56%.
LLaVA adds 576 image tokens per reference subject to the text token sequence. This increases the cross-attention cost in the diffusion transformer (where generated tokens attend to text+image tokens) by a factor proportional to the lengthened text sequence. For m = 2 subjects, this doubles the image token count from 576 to 1152, plus the separator tokens.
AudioNet's spatial cross-attention adds an additional attention operation per frame per denoising step, increasing per-step cost by an amount proportional to (f+1) × wh × 16 (spatial tokens per frame attending to 16 audio tokens).
Multi-subject generation is inherently sequential in the number of subjects at the representation level: LLaVA must encode all subjects' images before the diffusion process begins, and the temporally concatenated latents must be assembled before denoising. There is no parallelism across subjects.

The consequence. A practitioner cannot estimate:

Latency for interactive applications: If a virtual try-on application requires generating a video within a few seconds of user input, does HunyuanCustom meet this bar? The latency depends on the denoising schedule (inherited from HunyuanVideo and not specified), the cost per step (increased by the factors above relative to the base model), and the hardware (not specified). Without latency data, the feasibility of interactive deployment is unknown.
Throughput for batch processing: If a video production pipeline needs to generate thousands of customized videos overnight, how many GPU-hours are required? The paper provides no throughput numbers, no GPU specifications, and no scaling analysis with batch size.
Cost scaling with complexity: How much more expensive is a multi-subject audio-driven video compared to a single-subject text-only video? The architectural analysis above suggests non-trivial multiplicative factors, but without empirical measurement, the practitioner cannot budget for these differences or decide whether the added capability justifies the added cost.

What evidence exists in the paper. The paper provides no latency, throughput, GPU memory, or FLOPs data. The base HunyuanVideo model is described as a 13B-parameter diffusion transformer, and the paper states that all weights (including LLaVA) are fully fine-tuned, but the inference hardware, batch size, number of denoising steps, and generation time are not reported. The paper does not compare the computational cost of generating a video with HunyuanCustom against generating a video with the compared methods (Vidu, Keling, etc.), which is a significant omission given that the methods may use different model sizes, different numbers of denoising steps, and different levels of inference optimization.

Mitigation status. The paper does not acknowledge this as a limitation and does not report any computational cost metrics. This is a particularly consequential omission because the paper's architectural innovations — while motivated by quality concerns — have clear computational costs that should be part of any deployment evaluation. A practitioner who deploys HunyuanCustom for multi-subject audio-driven generation based on the paper's quality claims may discover deployment-time costs that were not disclosed in the paper, which is a practical risk the paper should have addressed.

7. Implications and Future Directions

How This Work Changes the Landscape

HunyuanCustom makes a case for architectural disentanglement as a first-class design principle in multi-modal video generation — a shift from the field's implicit assumption that conditioning modalities can share injection mechanisms if the model is trained on enough tasks (VACE's approach) toward the view that modalities with different temporal and spatial characteristics require fundamentally different architectural pathways that must not interfere. This is not a paradigm shift in the Kuhnian sense — the individual techniques (LLaVA for vision-language fusion, temporal concatenation for identity, cross-attention for audio, direct latent addition for video conditioning) are each drawn from existing literatures. The contribution is methodological: it demonstrates that the integration architecture — how these mechanisms coexist without interference — is the critical design decision, not the individual mechanisms themselves. This reframes multi-modal video generation from a training-data-sufficiency problem (can we collect enough multi-modal paired data?) to an architecture-design problem (can we design pathways that don't conflict?), which is a more tractable reframing with clearer scientific hypotheses.

The most significant conceptual contribution is the diagnosis that identity preservation and multi-modal conditioning are naturally in tension, not naturally complementary. Prior work assumed that adding conditioning modalities was purely additive — more information helps the model. HunyuanCustom's comparison with VACE provides evidence for the opposite: VACE supports more modalities but sacrifices identity consistency (Face-Sim of 0.219 vs. HunyuanCustom's 0.426 in Table 1), suggesting that when conditioning signals share architectural pathways, they compete for representational capacity and the model makes tradeoffs. HunyuanCustom's resolution — give each modality its own operation type (self-attention for identity, cross-attention for audio, direct addition for video) at a different point in the computation graph — provides a design template for future multi-modal systems: before adding a new conditioning modality, ask not just "can we collect training data?" but "does this modality's temporal/spatial characteristic conflict with existing conditioning pathways, and what operation type avoids that conflict?"

The paper also subtly reframes audio-driven generation from an animation problem to a video generation problem. Prior audio-driven methods (Loopy, SONIC) start from a single image and hallucinate motion — a fundamentally underconstrained problem. HunyuanCustom starts from a learned identity representation and generates motion from the video model's pretrained priors — a fundamentally different problem structure. If this reframing holds under rigorous quantitative evaluation (which the paper does not yet provide), it would redirect audio-driven generation research away from better image-warping and toward better identity representations in video models.

What the paper does not change: It does not provide a scaling law, a compute-optimal allocation strategy, or a theoretical framework. The contributions are architectural and empirical, not theoretical. The absence of difficulty estimation, adaptive compute allocation, and computational cost analysis means the paper does not advance the efficiency frontier of video generation — it trades additional compute (more tokens, more attention operations, full fine-tuning) for better identity preservation, without quantifying the tradeoff. This is a legitimate engineering contribution but not a conceptual breakthrough in understanding how generation quality scales with compute.

Follow-Up Research This Work Enables

Rigorous quantitative evaluation of the disentanglement hypothesis. The paper's central architectural claim — that separate injection pathways prevent inter-modality interference — is untested. A straightforward follow-up would measure Face-Sim and DINO-Sim for image-only generation, image+audio generation, image+video generation, and image+audio+video generation, all using the same test prompts and reference images, and test whether identity metrics degrade as modalities are added. A null result (identity degrades with added modalities despite disentangled pathways) would indicate that the interference is in the shared backbone, not the injection pathways, requiring a different architectural strategy (e.g., modality-specific adapters rather than full fine-tuning). A positive result (identity is preserved across all modality combinations) would validate the disentanglement principle and raise the bar for future multi-modal architectures to demonstrate similar non-interference. The evaluation should use the same metrics as Table 1 (Face-Sim, DINO-Sim, CLIP-B-T, temporal consistency, dynamic degree) on the same 200-video test set, plus lip-sync metrics (LSE-D, LSE-C) for audio conditions and background preservation metrics (PSNR, LPIPS in non-masked regions) for video conditions.

Reproduction on an alternative base model to test architectural generalizability. All experiments use HunyuanVideo-13B. The paper's architectural claims (temporal concatenation with shifted RoPE, dual-pathway identity, disentangled modality injection) would be substantially strengthened or qualified by porting the same architecture to a different video generation model — for instance, Wan (Wang et al., 2025), which VACE uses, or CogVideoX (Yang et al., 2024). A successful reproduction (competitive identity metrics on the same or a comparable test set) would demonstrate that the design principles are model-agnostic. A failure (identity metrics degrade relative to the base model's native performance, or the disentanglement breaks down) would reveal which aspects of HunyuanVideo's architecture (3D-RoPE, causal VAE, Flow Matching, specific attention patterns) the customization strategy depends on. This experiment would directly address the paper's unstated assumption that its contributions generalize beyond HunyuanVideo.

Difficulty-conditioned allocation of modalities and compute for video customization. The paper applies the same architecture and the same (unstated) inference compute to every input. A natural extension — directly inspired by the reference paper's compute-optimal scaling strategy — would develop a difficulty estimator for video customization that predicts, from the reference image and text prompt alone, how challenging identity preservation will be (e.g., based on the distinctiveness of facial features, the complexity of the described motion, or the number of subjects). Difficult cases could receive more denoising steps, multiple samples with best-of-N selection using identity metrics as the scoring function, or escalation to a larger base model. Easy cases (single subjects with simple motions) could receive fewer denoising steps or reduced token budgets (shorter LLaVA feature sequences, fewer concatenated reference frames). The key metric would be the tradeoff curve between identity preservation (Face-Sim, DINO-Sim) and inference cost (GPU-seconds or FLOPs per video), analogous to the efficiency curves in the reference paper's Figures 4 and 8.

User study evaluating subjective qualities that automated metrics cannot capture. The paper claims superiority in "realism," "naturalness of interaction," and "diversity" — none of which are measured by the automated metrics in Table 1. A rigorous follow-up would conduct a paired comparison user study: for each of 50-100 test cases (covering single-subject, multi-subject, audio-driven, and video-driven settings), present raters with videos from HunyuanCustom and from the best competing method (Vidu for single-subject, the best qualitative performer from Figure 6 for multi-subject) in randomized side-by-side order, and ask forced-choice questions about: (1) which video looks more realistic, (2) which video better preserves the identity of the reference subject(s), (3) in multi-subject cases, which video shows more natural interaction between subjects, and (4) in audio-driven cases, which video has better lip-sync. The study should also include a "both equally good" and "both equally bad" option to avoid forced distinctions when quality is comparable. Results reported as win/loss/tie rates with confidence intervals would provide the subjective evidence the paper currently lacks, and would either validate or qualify the paper's qualitative claims.

Adversarial stress-testing of identity preservation under distribution shift. The paper's evaluation uses reference images and prompts that are presumably drawn from the same distribution as the training data (eight-category balanced dataset). A stress-test would evaluate HunyuanCustom on reference images with characteristics that are underrepresented or absent from the training distribution: extreme facial expressions, heavy occlusions (sunglasses, masks, hands covering face), unusual lighting (harsh shadows, colored lighting, low-light), non-photorealistic styles (paintings, sketches, cartoons), and subjects from demographic groups or clothing styles unlikely to be well-represented in the training data. The key metric would be the degradation in Face-Sim and DINO-Sim relative to in-distribution reference images. High degradation would reveal that the identity injection mechanisms rely on features (facial landmarks, texture patterns) that are fragile to distribution shift, motivating research on robust identity representations. Low degradation would demonstrate that the dual-pathway design (semantic + perceptual) provides genuine generalization beyond surface-level appearance matching.

Combine identity injection with test-time optimization for challenging cases. The paper's architecture is fully feed-forward at inference time — no per-instance optimization. For cases where the feed-forward identity preservation fails (e.g., highly challenging multi-subject interactions with occlusion), a hybrid approach could use HunyuanCustom's initial output as a starting point and then apply a small number of test-time fine-tuning steps (similar to DreamBooth's per-instance optimization at inference, but starting from HunyuanCustom's already-strong identity baseline rather than from scratch). The hypothesis is that feed-forward customization handles the "easy" identity matching (getting the general appearance right), and lightweight test-time optimization handles the residual identity error (fine-tuning the specific details that weren't captured). The key metric would be the tradeoff between identity improvement (Face-Sim gain) and additional inference cost (fine-tuning steps × per-step cost), with the goal of achieving near-perfect identity preservation (Face-Sim > 0.9, say) for the small fraction of cases where feed-forward customization alone is insufficient, without paying the full DreamBooth cost for every input.

Practical Applications and Downstream Use Cases

Personalized video advertising at scale, with quantifiable quality requirements. Current production of personalized video advertisements (e.g., showing a specific consumer using a specific product in a short video clip) requires manual video production — shooting, editing, compositing — at costs that make personalization economically infeasible except for very high-value customers. HunyuanCustom's multi-subject capability (Figure 7) enables automated generation of such videos from a consumer's photo and a product image. The 0.426 Face-Sim and 0.611 DINO-Sim reported in Table 1 provide a quantitative baseline for identity preservation that advertising platforms can evaluate against their quality requirements — for example, if a platform requires that the generated spokesperson be recognizable by acquaintances with >95% accuracy, the relationship between automated metrics (Face-Sim, DINO-Sim) and human recognition accuracy would need to be calibrated through a targeted user study, but the automated metric baselines are available from Table 1 as a starting point for that calibration. The key deployment consideration that the paper does not address is computational cost per generated advertisement, which will determine whether automated generation is cheaper than manual production at the target scale.

Virtual try-on with audio for e-commerce product pages. Current e-commerce virtual try-on is almost exclusively image-based — a static image of the shopper wearing the garment. HunyuanCustom's audio-driven multi-subject capability (Figure 9) enables a qualitatively different experience: a short video where the shopper's identity wears the garment, speaks (e.g., describing the product features, or providing a personalized greeting), and appears in a scene that differs from any reference photo. This moves virtual try-on from a static "how does it look?" to a dynamic "how does it look in motion, and how would I present it?" — a richer signal for purchase decisions. The deployment feasibility depends on latency: if generating a 5-second virtual try-on video takes minutes on a high-end GPU, the experience is batch-oriented (generate overnight, review in the morning). If it takes seconds, it could be interactive (generate on the product page while the shopper browses). The paper provides no latency data, so the deployment architecture (batch vs. interactive) is undetermined from the paper alone, but the capability itself is demonstrated.

Video subject replacement for post-production editing in film and advertising. The paper's video-driven customization capability (Figure 10) targets a specific pain point in professional video editing: replacing one object with another in existing footage while preserving background, lighting, camera motion, and occlusion patterns. The paper's demonstration that HunyuanCustom avoids both the boundary artifacts of mask-adherent approaches (VACE) and the copy-paste overlay effect of naive replacement (Keling) is qualitatively compelling. For professional post-production, the key metric is not just identity preservation of the replacement subject but also background fidelity — how well the non-masked regions of the source video are reproduced exactly. A post-production house evaluating HunyuanCustom would need to measure PSNR or LPIPS between source and output in non-masked regions (metrics the paper does not report) to determine whether the approach meets the exact-reproduction requirements of professional editing, where even minor background changes are unacceptable. If background fidelity is high, this capability could replace manual rotoscoping and compositing for a significant fraction of product placement and object replacement shots, with cost savings proportional to the fraction of shots that can be handled automatically.

When to Prefer This Method

The paper explicitly positions HunyuanCustom against several classes of alternatives — commercial video customization products (Vidu, Keling, Pika, Hailuo), open-source multi-subject methods (SkyReels-A2, VACE), and prior audio-driven animation methods (Loopy, SONIC) — and the experimental results in Table 1 and the qualitative comparisons in Figures 4-10 support specific preference criteria that the paper articulates directly or implicitly through its priority ordering.

Prefer HunyuanCustom when identity preservation is the primary requirement and you can accept moderate tradeoffs on text alignment, temporal consistency, and dynamic motion. The quantitative evidence in Table 1 shows HunyuanCustom leads on Face-Sim (0.426) and DINO-Sim (0.611) but is mid-pack on CLIP-B-T (0.315, third place), temporal consistency (0.960, fifth place), and dynamic degree (15.988, fifth place). If your application can tolerate slightly less camera motion or slightly looser text adherence in exchange for reliably preserving subject identity — as is the case for virtual human advertising (the identity of the spokesperson must be recognizable; moderate motion is acceptable) or product demonstration (the product's details must be preserved; dynamic camera work is secondary) — HunyuanCustom's identity-first design is well-matched.

Prefer HunyuanCustom when you need multiple conditioning modalities (image + audio + video) in a single model, rather than orchestrating separate specialized models. No other method evaluated in the paper supports all three non-text modalities simultaneously from a single model. VACE supports image, video, and text but not audio, and sacrifices identity consistency (Face-Sim 0.219 vs. HunyuanCustom's 0.426). Commercial products (Vidu, Keling, Pika, Hailuo) support image-driven customization but do not offer audio or video conditioning as described in the paper. Dedicated audio-driven animation methods (Loopy, SONIC) handle audio but not multi-subject scenes or video conditioning. If your workflow requires, for example, generating a video where a specific person speaks specific audio in a specific background scene — a combination that cuts across multiple specialized models — HunyuanCustom provides a unified interface. The caveat is that the audio and video conditioning capabilities are not quantitatively evaluated, so prefer HunyuanCustom for these modalities when qualitative demonstrations are sufficient to justify the integration cost, and plan for additional in-house evaluation before production deployment.

Prefer commercial products (Vidu, Keling) when dynamic motion, text alignment, or production-level polish in non-identity dimensions is critical and identity can be verified post-hoc. In Table 1, Hailuo leads on CLIP-B-T (0.318) and Dynamic Degree (34.807), and Vidu leads on Temporal Consistency (0.975). If your application requires highly dynamic, fast-moving videos where motion quality is the primary requirement and identity can be checked after generation (discarding videos where identity drifts), a commercial product with better motion metrics may produce more usable outputs overall, even if the per-video identity failure rate is higher. HunyuanCustom's lower dynamic degree (15.988) suggests it generates more conservative motion — appropriate for product demonstrations and virtual try-on, less appropriate for action scenes or sports content.

Prefer dedicated audio-driven animation methods (Loopy, SONIC) when the input is a single face image and the output is a talking-head video with fixed background and posture — the classic audio-driven portrait animation task. HunyuanCustom's audio-driven capability is designed for a different use case (scene-diverse, posture-variable generation with identity preservation), and the paper provides no evidence that it matches dedicated methods on their home territory (lip-sync accuracy for close-up talking heads with a single reference image). If your application fits the classic portrait animation paradigm, the specialized methods — which have been quantitatively benchmarked on lip-sync metrics — remain the safer choice until HunyuanCustom's audio-visual synchronization is quantitatively evaluated against them.

Dependency tree

Papers this one builds on, walking backwards through the same domain and shared topics.

BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs2025· architecture← in-domain predecessor
- The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits2024· pretraining← LLM-judged similar
- Multi-Token Attention2025· architecture← in-domain predecessor
- BitNet a4.8: 4-bit Activations for 1-bit LLMs2024· architecture← similar topic
LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation2026· architecture← similar topic
- Qwen-Image-2.0 Technical Report2026· architecture← in-domain predecessor
- FP4 Explore, BF16 Train: Diffusion Reinforcement Learning via Efficient Rollout Scaling2026· training-methods← topic thread
- Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation2025· architecture← similar topic
- MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generation2025· architecture← similar topic
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer2025· architecture← similar topic
- InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation2025· architecture← in-domain predecessor
- FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Space2026· architecture← similar topic
- Block Transformer: Global-to-Local Language Modeling for Fast Inference2024· architecture← similar topic
- GoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression2024· architecture← similar topic

What it influenced

Later papers in the same chain. Direction reversed.

TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times2025· inference-optimization→ LLM-judged similar
- FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Space2026· architecture→ LLM-judged similar
- DreamStyle: A Unified Framework for Video Stylization2026· diffusion→ LLM-judged similar
- ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation2026· diffusion→ LLM-judged similar
- Transition Matching Distillation for Fast Video Generation2026· inference-optimization→ LLM-judged similar
- Plenoptic Video Generation2026· vision→ LLM-judged similar
- FlowAct-R1: Towards Interactive Humanoid Video Generation2026· vision→ LLM-judged similar
- Kascade: A Practical Sparse Attention Method for Long-Context LLM Inference2025· inference-optimization→ in-domain predecessor
- Efficient Autoregressive Video Diffusion with Dummy Head2026· inference-optimization→ similar topic
- Accelerating Speculative Decoding with Block Diffusion Draft Trees2026· inference-optimization→ similar topic
MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encoder2025· architecture→ in-domain predecessor
- VibeVoice Technical Report2025· inference-optimization→ LLM-judged similar
- Log-Linear Attention2025· architecture→ in-domain predecessor
- Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering2025· architecture→ similar topic
- FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning2026· architecture→ similar topic
Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation2025· architecture→ similar topic
- NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model2025· reasoning→ LLM-judged similar
- Universal YOCO for Efficient Depth Scaling2026· architecture→ LLM-judged similar
- MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generation2025· architecture→ similar topic
- LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation2026· architecture→ similar topic
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer2025· architecture→ similar topic
- Fast-dLLM v2: Efficient Block-Diffusion LLM2025· llm-systems→ LLM-judged similar
- Less is More: Recursive Reasoning with Tiny Networks2025· architecture→ in-domain predecessor
- Block Transformer: Global-to-Local Language Modeling for Fast Inference2024· architecture→ similar topic
- GoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression2024· architecture→ similar topic