Continuous-Time Distribution Matching for Few-Step Diffusion Distillation

ArXiv: 2605.06376

🎯 Pitch

Current distribution matching distillation methods for accelerating diffusion models are crippled by training only on a few fixed timesteps, causing oversmoothed outputs that require adversarial patches. This paper shows that simply replacing those discrete anchors with a continuous, random training schedule and adding off-trajectory supervision eliminates the need for GANs entirely, producing 4-step images that surpass the 100-step teacher model in aesthetic quality and human preference.

---

1. Executive Summary

This paper introduces Continuous-Time Distribution Matching (CDM), a distillation framework that lifts the Distribution Matching Distillation (DMD) paradigm from discrete timestep optimization into a fully continuous-time formulation for few-step text-to-image generation. The authors conduct experiments on SD3-Medium and Longcat-Image at 1024×1024 resolution, migrating two core mechanisms from sparse anchors to continuous supervision: a dynamic continuous schedule (randomizing both the backward simulation length and the timestep anchors drawn from (0, 1] at each training iteration, rather than fixing them to the inference schedule) and a continuous-time distribution matching loss (performing velocity-driven Euler extrapolation to probe off-trajectory latents between discrete integration steps, then enforcing distribution matching on those extrapolated points). On SD3-Medium with 4 NFE, CDM achieves state-of-the-art performance across aesthetic quality, prompt adherence, and human preference metrics — notably improving HPSv3 from 9.176 (prior best image-free method) to 9.561 — while on Longcat-Image it pushes HPSv3 to 10.65, surpassing even the 100-NFE teacher on several metrics. The framework accomplishes this without adversarial training or reward-model auxiliary objectives, establishing that continuous-time distribution matching with off-trajectory supervision can recover sharp textures and rich details only when it jointly constrains both on-trajectory alignment and the student's velocity field smoothness across the full time domain.

2. Context and Motivation

The Core Problem: Discrete-Time Anchoring Severely Constrains Distribution Matching Distillation

The fundamental question this paper tackles is: why does distribution matching distillation (DMD) consistently underperform its potential, and what structural limitation in the existing formulation causes it? Prior to this work, DMD and its variants had established themselves as leading distillation paradigms for accelerating diffusion models, but practitioners consistently observed that DMD-distilled models produce over-smoothed outputs with degraded fine details compared to both the teacher model and competing distillation approaches like consistency distillation. This paper identifies the root cause: DMD's optimization is artificially restricted to a handful of discrete timesteps, creating a fundamental mismatch between the sparse nature of supervision during training and the continuous integration task required at inference.

This matters for several interconnected reasons:

Practical deployment bottleneck. Diffusion models have become the backbone of modern text-to-image generation, but their iterative sampling process — typically requiring 50-100 network evaluations per image — imposes a severe computational barrier to deployment. A 1024×1024 SD3-Medium generation at 100 steps might take seconds on a high-end GPU, making real-time applications, interactive editing, and on-device deployment impractical. Reducing this to 4 steps while preserving quality represents a 25× speedup that could enable entirely new use cases.

The puzzling performance gap between DMD and consistency models. Both distribution matching distillation (DMD, DMD2, D-DMD) and consistency distillation (LCM, PCM, sCM) aim for the same goal — few-step generation — but they take fundamentally different approaches. Consistency methods enforce self-consistency along the entire probability flow ODE trajectory, naturally operating in a continuous time space. DMD instead relies on distribution matching at a few fixed timesteps, treating the intermediate trajectory as a black box. Yet despite DMD's strong theoretical foundation in KL-based distribution matching, consistency methods often produce sharper outputs with better detail preservation in practice. This gap is particularly puzzling because DMD can draw on the full expressive power of the teacher model's score function, while consistency methods must learn a direct mapping. Understanding why DMD underperforms is not just academic — it points to a structural limitation that, once addressed, could unlock substantially better few-step generation.

The hidden cost of auxiliary objectives. A telling symptom of DMD's limitations is the proliferation of auxiliary modules designed to patch its output quality: GAN-based adversarial losses (DMD2), reward models (Flash Diffusion), and combined consistency+adversarial approaches. These add complexity, training instability, and computational overhead — yet they are treated as necessary because vanilla DMD cannot recover sharp textures on its own. The fact that the community has converged on "DMD + GAN" as the standard recipe suggests that something is fundamentally wrong with the core distribution matching objective, not just its hyperparameters. If the underlying cause could be identified and fixed, these auxiliary modules might become unnecessary.

Where Existing DMD Formulations Fall Short

The paper identifies two specific — and previously unexamined — structural limitations in standard DMD:

Limitation 1: Training-inference schedule coupling.

In vanilla DMD2, the backward simulation during training operates exclusively on the exact same discrete timesteps that will be used at inference: if the target is 4-step generation with schedule {1.0, 0.75, 0.5, 0.25}, then during training the model only ever sees latents at those four points, and the teacher only ever provides supervision at those four points. This is a deliberate design choice motivated by a reasonable intuition: the training trajectory should match the inference trajectory to avoid train-test mismatch.

The paper's first key empirical insight is that this intuition, while plausible, is wrong in a way that actively harms performance. Figure 2 demonstrates this through a simple experiment: keep everything else identical in a D-DMD setup, but instead of fixing the backward simulation to the inference schedule, randomly sample the simulation length N ∼ U{1, N_max} at each iteration and uniformly draw the intermediate timesteps. This "dynamic" variant:

• Produces visibly sharper textures and fewer artifacts than the fixed-schedule baseline (Figure 2b)
• Achieves a higher HPSv3 score quantitatively (Figure 2c)
• Does so without any architectural changes or additional loss terms

This finding is counterintuitive because it violates the standard ML principle that training and testing distributions should match. The authors argue, through the score-matching analysis in Appendix D, that what matters is not the student's specific timestep anchors but rather the continuous regularization of the student's velocity field. When the student only sees four fixed anchors, the teacher's supervision is concentrated at those sparse points, and the student learns a velocity field that may be accurate at those anchors but highly non-smooth in between — precisely the opposite of what few-step Euler integration needs, since the local truncation error scales with the supremum of the velocity field's material derivative between steps (Appendix E).

Limitation 2: On-trajectory-only supervision cannot correct truncation errors.

Even with the dynamic schedule providing dense on-trajectory supervision, there remains a fundamental problem: few-step inference requires taking large Euler steps between discrete anchors, and these large steps introduce substantial numerical truncation errors that push the sampling trajectory off the ideal probability flow ODE path. Standard DMD losses only supervise points that lie exactly on the (simulated) sampling trajectory — they never see or correct the off-trajectory latents that actually occur during inference due to integration error.

This is not a hypothetical concern. When taking a step Δt that is large (e.g., Δt = 0.25 for a 4-step schedule, compared to Δt = 0.01 for a 100-step teacher), the local truncation error of Euler integration is O((Δt)² · sup ‖dv_θ/dτ‖). The supervised anchors might be perfectly matched, but if the velocity field oscillates or changes rapidly between anchors — which is exactly what happens when supervision is sparse — the actual inference trajectory drifts progressively further from the ideal manifold. Prior DMD methods have no mechanism to detect or correct this drift, because they only evaluate the student at the pre-planned anchors.

This explains a subtle but important phenomenon: DMD-distilled models often produce images that look "plausible but wrong" — the global structure is preserved (because the anchors are matched), but high-frequency details, texture coherence, and fine semantic attributes are degraded (because the inter-anchor trajectory wandered off-manifold). GAN and reward-model auxiliary objectives partially compensate by providing a separate signal that penalizes unrealistic outputs, but they do so without addressing the root cause of the velocity field non-smoothness.

The Misunderstood Role of the Distribution Matching Loss

Beyond the schedule and supervision limitations, the paper identifies a third motivation: the DMD community has misunderstood what the distribution matching (DM) loss actually does. In the decoupled DMD framework (D-DMD), the training objective is split into:

• CFG Augmentation (CA) loss: pushes the student toward higher text-image alignment by leveraging the teacher's implicit classifier
• Distribution Matching (DM) loss: matches the student's output distribution to the teacher's

D-DMD [24] and concurrent theoretical analyses [60] characterize the DM loss as primarily a "stabilizer" or "regularizer" — something that prevents training collapse and reduces artifacts, but not the primary driver of generation quality. The CA loss, in this view, is the "spear" that drives few-step conversion, while the DM loss is the "shield" that maintains stability.

The paper challenges this characterization through a simple but revealing experiment (Figure 3): distill a student model using only the DM loss (no CA loss at all), and compare its outputs against the teacher model with CFG enabled and disabled. The results are striking:

• Student (DM-only) outputs closely match the teacher's CFG-free outputs, not the (much better) CFG-enabled outputs
• This holds across both SD3-Medium and Longcat-Image backbones
• Quantitative validation (Appendix Table 4) confirms that the DM-only student's metrics tightly track the CFG-free teacher across all evaluation dimensions

This reveals that the DM loss is not a passive stabilizer — it is an active alignment mechanism that faithfully captures the teacher's unguided (CFG-free) marginal distribution. The quality ceiling of the DM loss is set by the teacher's performance without classifier-free guidance, which is substantially lower than with CFG. This explains why the CA loss is necessary (it provides the CFG signal that the DM loss cannot), but it also means that the DM loss is doing something specific and important: it anchors the student to the underlying data manifold as represented by the teacher, providing the distributional foundation on which the CA loss can then layer text-alignment guidance.

This reframing has practical implications for distillation design. If the DM loss captures the teacher's CFG-free distribution, then improvements to the DM loss — making it more accurate, applying it more densely, extending it off-trajectory — should directly translate to better generation quality. The continuous-time and off-trajectory extensions in CDM are motivated by exactly this logic: they aim to make the DM loss a more faithful and comprehensive distribution matching signal.

Contrast with Consistency Distillation

To fully appreciate the gap CDM fills, it is important to understand how consistency distillation (the other major paradigm) differs structurally.

Consistency models [28, 30, 49] learn a function f_θ(x_t, t) that maps any point along the probability flow ODE trajectory directly to the trajectory's origin at t = 0 (the clean image). Training enforces self-consistency: for any two points (x_t, t) and (x_s, s) on the same trajectory, f_θ(x_t, t) should equal f_θ(x_s, s). This naturally operates in continuous time: the consistency condition is enforced across arbitrary time pairs, providing dense supervision over the entire interval (0, 1]. At inference, the model can jump from noise to the clean image in a single step, or use a few refinement steps.

The key advantage of consistency models is that their training objective inherently promotes trajectory smoothness — if f_θ must be consistent between any t and any s, the model cannot develop the inter-anchor oscillations that plague discrete DMD. This likely explains their superior detail preservation in practice.

CDM does not adopt the consistency framework, but it addresses the same underlying issue — lack of continuous regularization — through a different mechanism. Rather than enforcing self-consistency, CDM extends DMD's distribution matching to the continuous domain, using the dynamic schedule for dense on-trajectory supervision and the velocity-driven extrapolation for off-trajectory alignment. The result is a DMD variant that inherits the continuous-time benefits of consistency models while retaining DMD's direct access to the teacher's score function as a supervision signal.

Positioning: A Continuous-Time Extension, Not a Replacement

The paper positions CDM as an evolution of the DMD framework, not a competing paradigm. It retains:

• The core backward simulation strategy (generating a trajectory by integrating the student's ODE, then extracting intermediate latents for supervision)
• The decoupled CA+DM objective structure
• The fake teacher mechanism for online score estimation
• The best-of-N weighted selection and CFG augmentation formulations

What it changes is the temporal scope of optimization: from discrete anchors to continuous time, and from on-trajectory-only to on+off-trajectory supervision. This is a conceptually minimal change — the three loss terms in Equation (10) have the same mathematical form; they differ only in what latents they are applied to and how those latents are constructed. The dynamic schedule broadens when supervision occurs; the CDM loss broadens where (in latent space) it occurs.

Notably, a concurrent work [40] independently identified the value of off-trajectory supervision, but constructs off-trajectory points through re-noising (adding Gaussian noise to a predicted clean image and then solving backward) rather than through velocity-driven extrapolation along the student's own predicted field. The CDM approach has the advantage that it directly simulates the actual truncation drift that occurs during few-step Euler integration, rather than a generic perturbation that may not match the inference-time error distribution.

Summary of the Research Gap and CDM's Response

3. Technical Approach

3.1 Reader Orientation

CDM is a distillation training system that converts a large, slow text-to-image diffusion model (the "teacher") into a compact "student" model capable of generating high-quality images in just 4 neural network evaluations instead of the teacher's typical 50–100. The system solves the problem of discrete-time overfitting in distribution matching distillation by shifting all supervision from a handful of fixed timesteps to a continuous time space, adding an explicit mechanism that probes and corrects the integration errors introduced by the large time-steps required for few-step generation.

3.2 Big-Picture Architecture (Diagram in Words)

The CDM training system consists of five interacting components, executed once per training iteration:

1. Student flow model (D_θ) — the model being trained, which predicts clean images from noisy latents at any timestep. Its parameters θ are updated by gradient descent.

2. Frozen real teacher (D_ϕ) — the pre-trained diffusion model whose knowledge is being distilled. It never receives gradient updates and provides the ground-truth score signals for all losses.

3. Online-updated fake teacher (D_ψ) — a separate model whose parameters ψ track a moving average of the student's parameters. It estimates the score of the student's current generative distribution, enabling the distribution matching gradient without needing to differentiate through a sampling process.

4. Backward simulation — at each training iteration, the student generates a complete sampling trajectory (from pure noise to clean image) by numerically integrating its own probability flow ODE. An intermediate latent x_{t_i} is extracted from this trajectory to serve as the anchor for all losses.

5. Three loss functions on different latents:

   • CA loss on the on-trajectory anchor x_{t_i}: enforces text-image alignment via the teacher's implicit classifier
   • DM loss on the on-trajectory anchor x_{t_i}: aligns the student's output distribution with the teacher's CFG-free data manifold
   • CDM loss on an off-trajectory extrapolated latent x_{t'_i}: corrects velocity field inconsistencies between discrete integration steps

Information flow: A text prompt c enters → the student runs an N-step backward simulation from noise to produce a trajectory → an intermediate latent is extracted → the student predicts a clean image from this latent → this prediction is re-noised at multiple continuous timesteps to create target latents for the real teacher → the real teacher scores these targets → the fake teacher also scores one target for the DM and CDM losses → the difference between real and fake teacher scores provides the distribution matching gradient → simultaneously, the student takes a velocity-driven Euler step off the trajectory to create an extrapolated latent → this latent is processed through the same real-fake teacher scoring pipeline → all three gradients are summed and applied to the student's parameters → the fake teacher is updated toward the student's current state on a separate optimizer schedule.

3.3 Roadmap for the Deep Dive

• First, the decoupled DMD baseline (the CA and DM loss equations and their mechanisms), because CDM builds directly on this foundation and understanding what each term does is essential for grasping why and how the continuous-time extensions work.
• Second, the dynamic continuous schedule — how it randomizes the backward simulation length and anchor positions, and the score-matching perspective that explains why this helps (the theory from Appendix D that justifies the empirical finding in Figure 2).
• Third, the CDM loss — the velocity-driven extrapolation mechanism, how off-trajectory latents are constructed and supervised, and the connection to truncation error in few-step Euler integration (the theory from Appendix E that explains why off-trajectory matching suppresses integration error).
• Fourth, the complete training algorithm — how all components fit together in a single training iteration, including the fake teacher update, the sampling of continuous timesteps, and the gradient flow through the system.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methodology paper whose core insight is that distribution matching distillation can be migrated from a discrete-time to a continuous-time formulation by relaxing the training-inference schedule coupling and adding off-trajectory supervision that directly counters the truncation error inherent in large-step Euler integration.

---

The Student Model and Flow-Matching Parameterization

CDM targets flow-matching models, where the generative process is defined by a probability flow ordinary differential equation (ODE) that transports samples from a Gaussian noise distribution at time t = 1 to the data distribution at time t = 0. The student is a neural network v_θ trained to predict the velocity field — the instantaneous direction and magnitude of change as time evolves.

The key interface between the velocity field and image-space predictions is the clean data estimator:

$D_θ(x_t, t, c) = x_t - t \, v_θ(x_t, t, c)$

where x_t ∈ R^d is a noisy latent at continuous time t ∈ (0, 1], c is the conditioning text embedding, v_θ(x_t, t, c) is the student's predicted velocity at that point, and D_θ(x_t, t, c) is the student's estimate of the clean (noise-free) latent.

What it computes: given a noisy latent and a timestep, the student predicts the velocity field (direction toward clean data), multiplies by the current time to scale the correction appropriately (since at t = 1, the latent is pure noise requiring maximum correction; at t ≈ 0, it is nearly clean and requires minimal correction), and subtracts this correction from the noisy latent to estimate what the clean data looks like.

Why this form: this parameterization follows directly from the flow-matching forward process x_t = (1-t)x_0 + tε where x_0 is clean data and ε ∼ N(0, I) is Gaussian noise. The velocity field that transports x_t toward x_0 is (x_0 - x_t)/t, so predicting the velocity and computing D_θ via Equation (1) is equivalent to predicting the clean data directly — but the velocity parameterization is known to produce smoother training dynamics and better generalization in flow-matching models. This parameterization is used consistently for the student, the real teacher, and the fake teacher throughout the paper.

---

The Decoupled DMD Baseline: CA Loss and DM Loss

Before introducing the continuous-time extensions, the paper builds on the decoupled DMD framework (D-DMD), which splits distribution matching distillation into two orthogonal loss terms applied to a single on-trajectory latent extracted from a backward simulation. Understanding these baseline losses is essential because CDM retains their mathematical forms and modifies only where and when they are applied.

Backward simulation. At each training iteration, the student runs an N-step numerical integration of its own probability flow ODE starting from pure noise x_{t_1} ∼ N(0, I) following a predefined discrete schedule 1 = t_1 > t_2 > ... > t_N > 0. This produces a trajectory of latents {x_{t_1}, x_{t_2}, ..., x_{t_N}}. A single intermediate latent x_{t_i} is extracted by uniformly sampling an index i ∼ U{1, ..., N}. All losses in that iteration are computed starting from this shared anchor x_{t_i}.

The student passes x_{t_i} through itself to obtain the clean data estimate:

$\hat{x}^{(i)}_0 = D_θ(x_{t_i}, t_i, c)$

This estimate serves as the basis for all subsequent re-noising operations.

The CA Loss (CFG Augmentation). The CA loss enforces text-image alignment by leveraging the teacher's implicit classifier — the difference between the teacher's conditional and unconditional score predictions, which indicates the direction in latent space that increases the probability of the conditioning text. The loss is:

$L_{CA} = \frac{1}{2} \left\| D_θ(x_{t_i}, t_i, c) - \text{sg}\left[ D_θ(x_{t_i}, t_i, c) + w_τ \, α \, (D_ϕ(z_τ, τ, c) - D_ϕ(z_τ, τ, ∅)) \right] \right\|^2_2$

where D_θ(x_{t_i}, t_i, c) is the student's clean data prediction at the anchor, \text{sg}[\cdot] is the stop-gradient operator (the term inside brackets is treated as a constant target during backpropagation), z_τ = (1-τ) \hat{x}^{(i)}_0 + τ ε_τ is the re-noised version of the student's clean estimate at a randomly sampled continuous timestep τ ∼ U(0, 1], D_ϕ(z_τ, τ, c) and D_ϕ(z_τ, τ, ∅) are the teacher's conditional and unconditional predictions at this re-noised latent, α is the classifier-free guidance scale (set to 7.0 for all experiments), and w_τ = \|D_ϕ(z_τ, τ, c) - D_θ(x_{t_i}, t_i, c)\|^{-1}_1 is a dynamic weighting factor that normalizes the gradient magnitude to prevent the loss from being dominated by timesteps with large prediction discrepancies.

What it computes: the squared L2 distance between the student's current prediction and a target that equals the student's prediction plus a scaled version of the teacher's CFG direction Δ_{ca}^{real} = α(D_ϕ(z_τ, τ, c) - D_ϕ(z_τ, τ, ∅)). The CFG direction points from the unconditional teacher prediction toward the conditional teacher prediction — effectively the direction of increasing "text-alignment" in the teacher's latent space. Adding this direction to the student's prediction pushes the student to generate images that are more aligned with the prompt.

Why this form: the stop-gradient on the target makes this a squared-error regression toward a fixed target, which is simpler to optimize than a full adversarial or score-matching objective. The dynamic weight w_τ prevents the loss from exploding at timesteps where the CFG correction is large (typically intermediate noise levels). The use of the teacher's CFG direction rather than directly regressing to the teacher's conditional prediction is crucial: it allows the student to learn how to incorporate text guidance while preserving its own learned data manifold, rather than collapsing to a point estimate of what the teacher would produce. Without the CFG augmentation, the student would learn the teacher's CFG-free distribution (as shown in Figure 3), which lacks prompt adherence.

The DM Loss (Distribution Matching). The DM loss aligns the student's output distribution with the teacher's data manifold by minimizing a form of reverse Kullback-Leibler divergence. The loss is:

$L_{DM} = \frac{1}{2} \left\| D_θ(x_{t_i}, t_i, c) - \text{sg}\left[ D_θ(x_{t_i}, t_i, c) + w_{\tilde{τ}} (D_ϕ(z_{\tilde{τ}}, \tilde{τ}, c) - D_ψ(z_{\tilde{τ}}, \tilde{τ}, c)) \right] \right\|^2_2$

where z_{\tilde{τ}} = (1-\tilde{τ})\hat{x}^{(i)}_0 + \tilde{τ}ε_{\tilde{τ}} is an independently re-noised version of the same student clean estimate at a different randomly sampled continuous timestep \tilde{τ} ∼ U(0, 1], D_ϕ(z_{\tilde{τ}}, \tilde{τ}, c) is the frozen real teacher's prediction, D_ψ(z_{\tilde{τ}}, \tilde{τ}, c) is the online-updated fake teacher's prediction, and w_{\tilde{τ}} = \|D_ϕ(z_{\tilde{τ}}, \tilde{τ}, c) - D_θ(x_{t_i}, t_i, c)\|^{-1}_1 is the same dynamic weighting scheme.

What it computes: the squared L2 distance between the student's prediction and a target that equals the student's prediction plus the distribution matching correction Δ_{dm}^{real-fake} = D_ϕ(z_{\tilde{τ}}, \tilde{τ}, c) - D_ψ(z_{\tilde{τ}}, \tilde{τ}, c). This correction is the difference between what the real teacher predicts at the re-noised latent and what the fake teacher predicts — it points from the student's current distribution toward the real data distribution.

Why this form: the difference D_ϕ - D_ψ approximates the score difference ∇log p_real - ∇log p_fake (as derived in Appendix D via Tweedie's formula), and optimizing with this target corresponds to minimizing D_KL(p_fake \| p_real) — the reverse KL divergence that encourages mode-seeking behavior (the student concentrates probability mass on the teacher's dominant modes rather than trying to cover all modes, which would be infeasible with few-step generation). The fake teacher D_ψ must be online-updated because it needs to track the student's evolving distribution throughout training; using a frozen copy would provide stale score estimates and fail to provide the correct gradient direction.

The fake teacher update. Following each student update, the fake teacher's parameters ψ are updated to better track the student's current distribution. Using the same anchor i and the same student clean estimate \hat{x}^{(i)}_0, a separate re-noised latent is constructed at a randomly sampled timestep τ_ψ ∼ U(0, 1], and the fake teacher is trained with a standard flow-matching objective:

$L_ψ = \|v_ψ(z_{τ_ψ}, τ_ψ, c) - (ε_ψ - \hat{x}^{(i)}_0)\|^2_2$

where v_ψ is the fake teacher's predicted velocity, and (ε_ψ - \hat{x}^{(i)}_0) is the ground-truth velocity for the flow-matching interpolation (since z_{τ_ψ} = (1-τ_ψ)\hat{x}^{(i)}_0 + τ_ψ ε_ψ, the velocity that transports z_{τ_ψ} to \hat{x}^{(i)}_0 is exactly ε_ψ - \hat{x}^{(i)}_0). The fake teacher is updated 2 times per student update following the Two Time-scale Update Rule (TTUR) to ensure it remains a reliable score estimator while the student evolves.

Why the decoupling matters. The paper adopts the decoupled CA+DM formulation from D-DMD rather than the coupled DMD2 formulation because it provides cleaner gradient signals: the CA loss focuses exclusively on text-image alignment through the teacher's CFG direction, while the DM loss focuses exclusively on distribution matching through the real-fake score difference. In the original coupled DMD formulation, these two objectives were entangled in a single loss term, making it harder to diagnose failures and tune hyperparameters. The decoupling also enables the paper's key empirical finding in Figure 3 and Appendix Table 4: by training with only the DM loss, the authors can isolate what the DM loss actually learns (the teacher's CFG-free distribution) and demonstrate that it is not merely a passive stabilizer.

---

The Dynamic Continuous Schedule

The first major innovation of CDM is replacing the fixed, inference-aligned discrete schedule used in prior DMD methods with a dynamic continuous schedule that randomizes both the number of backward simulation steps and the anchor positions at each training iteration.

How it works. At each training iteration, instead of using a fixed schedule S_infer = {t_1, ..., t_N} that matches the target inference steps:

• The backward simulation length N is randomly sampled from a uniform distribution: N ∼ U{1, N_max}, where N_max = 28 for the SD3-Medium experiments
• A strictly decreasing continuous time sequence 1 = t_1 > t_2 > ... > t_N > 0 is generated where each t_j is drawn uniformly from the continuous interval (0, 1] subject to the ordering constraint
• The student runs an N-step Euler integration along this random schedule to produce the trajectory, and an intermediate latent is extracted as before

What this changes from the baseline. In standard DMD with a 4-step inference target, the training backward simulation is also 4 steps, always using the exact same timesteps {1.0, 0.75, 0.5, 0.25}. The student sees latents only at these four noise levels, and the teacher provides gradients only at re-noised versions of predictions anchored at these four levels. In CDM, the student might simulate a trajectory of length anywhere from 1 to 28, with anchors drawn from the entire continuous interval — the student sees latents at arbitrary noise levels and receives teacher gradients at re-noised versions of predictions from arbitrary anchors.

Why this helps — the score-matching perspective. The paper provides a theoretical justification in Section 3.2 and Appendix D. Applying Tweedie's formula to the flow-matching interpolation, the gradients of the CA and DM losses can be re-expressed in score-matching form.

For the CA loss, the gradient with respect to student parameters θ is:

$\nabla_θ L_{CA} = -w_τ α \frac{τ^2}{1-τ} \left( \frac{∂ D_θ(x_{t_i}, t_i, c)}{∂ θ} \right)^T \nabla_{z_τ} \log p_{real}(c | z_τ)$

where \nabla_{z_τ} \log p_{real}(c | z_τ) is the gradient of the log conditional probability of the text given the noisy latent under the teacher's distribution — effectively the teacher's implicit classifier that indicates which direction in latent space increases text-alignment. The term τ²/(1-τ) is a timestep-dependent scaling factor that arises from the Tweedie's formula relationship between denoiser predictions and score functions under the flow-matching interpolation.

For the DM loss, the gradient is:

$\nabla_θ L_{DM} = -w_{\tilde{τ}} \frac{\tilde{τ}^2}{1-\tilde{τ}} \left( \frac{∂ D_θ(x_{t_i}, t_i, c)}{∂ θ} \right)^T (\nabla_{z_{\tilde{τ}}} \log p_{real}(z_{\tilde{τ}} | c) - \nabla_{z_{\tilde{τ}}} \log p_{fake}(z_{\tilde{τ}} | c))$

where \nabla \log p_{real} - \nabla \log p_{fake} is the score difference between the real and fake distributions at noise level \tilde{τ}.

What these forms reveal about the schedule. The critical observation is that in both gradients, the student's input timestep t_i (which determines where the Jacobian ∂D_θ/∂θ is evaluated) and the teacher's perturbation timesteps τ and \tilde{τ} (which determine where the score targets are evaluated) appear in different places in the equations and are multiplied together. They are independent variables — the gradient is a product of a student-side Jacobian (depending on t_i) and a teacher-side score target (depending on τ or \tilde{τ}).

Under the dynamic continuous schedule, t_i, τ, and \tilde{τ} are all sampled independently from the same continuous distribution over (0, 1]. In expectation, the product of the Jacobian and score target is evaluated over the full Cartesian product of the continuous time domain with itself, providing dense regularization of the student's velocity field everywhere. Under the fixed discrete schedule, t_i is restricted to a few values, so the Jacobian is only evaluated at those sparse points — the student receives no gradient signal about how its velocity field behaves at intermediate timesteps.

Why the fixed schedule is harmful. When the student only receives supervision at four discrete anchors, the loss pushes the velocity field to be accurate at those four points. Between those points, the velocity field is unconstrained and can develop oscillations or rapid changes — exactly the behavior that maximizes the truncation error term sup_τ \|dv_θ/dτ\| in the Euler integration error bound (Appendix E). The dynamic schedule provides gradient signals that implicitly penalize rapid changes in the velocity field because the student sees targets from the (smooth) teacher at many different timesteps and learns to interpolate smoothly between them.

Why N_max = 28 and 4-step inference. The maximum simulation length (28) is set substantially larger than the target inference steps (4) to ensure the student experiences diverse trajectory lengths during training. The authors note that exact alignment between training simulation length and inference steps is unnecessary — the student learns a velocity field that supports integration with any step size because it has been regularized across the full time domain. The 4-step inference schedule is just one valid discretization of this smooth field.

---

The CDM Loss: Velocity-Driven Off-Trajectory Distribution Matching

The second major innovation addresses a remaining gap: even with dense on-trajectory supervision from the dynamic schedule, the losses only constrain the student's predictions at points that lie exactly on the simulated backward simulation trajectory. Few-step inference, however, takes large Euler steps from anchor t_j to t_{j+1}, and the linearized step x_{t_{j+1}} = x_{t_j} + (t_{j+1} - t_j) v_θ(x_{t_j}, t_j, c) produces a latent that does not lie on the ideal ODE trajectory due to the curvature of the true solution. The dynamic schedule provides no mechanism to detect or correct this drift because it never evaluates the student at these off-trajectory points.

How off-trajectory latents are constructed. Given the on-trajectory anchor x_{t_i} and the student's predicted velocity v_{t_i} = v_θ(x_{t_i}, t_i, c) at that anchor, CDM constructs an off-trajectory latent by taking a first-order Euler extrapolation to a randomly sampled target timestep:

$x_{t'_i} = x_{t_i} + (t'_i - t_i) v_{t_i}$

where t'_i ∼ U(0, 1] is sampled independently of the integration schedule — it has no relationship to t_{i-1} or t_{i+1}, and can be any value in (0, 1].

What it computes: starting from the on-trajectory anchor x_{t_i}, this takes a linear step of length Δt = t'_i - t_i in the direction of the student's predicted velocity. If t'_i < t_i (most common), the extrapolation moves forward in "denoising time" toward the clean image, simulating what would happen if the student took a single large Euler step to that timestep. If t'_i > t_i, it moves backward toward noisier states. The result is a latent x_{t'_i} that is not on the backward simulation trajectory — it lies somewhere between (or beyond) the discrete integration anchors due to the linearization error.

Why this specific construction. The key insight is that this linear extrapolation exactly simulates the truncation error that occurs during few-step Euler integration. When the student takes a large Euler step during inference (e.g., from t = 1.0 to t = 0.75), it computes x_{0.75} = x_{1.0} + (0.75 - 1.0) v_θ(x_{1.0}, 1.0, c), which is precisely the same form as Equation (7) with t'_i = 0.75 and t_i = 1.0. The error in this step — the difference between the Euler-predicted x_{0.75} and the true ODE solution at t = 0.75 — is the truncation error that degrades few-step generation quality. By constructing x_{t'_i} in exactly the same way and then enforcing distribution matching there, the CDM loss provides a learning signal that directly penalizes the velocity field configurations that cause large truncation errors.

Contrast with re-noising-based off-trajectory construction. The concurrent work [40] constructs off-trajectory points by first predicting the clean image \hat{x}_0, adding Gaussian noise to get x'_t = (1-t)\hat{x}_0 + tε, and then solving backward. This re-noising approach adds a generic perturbation that may not match the actual inference-time error distribution. The velocity-driven extrapolation in CDM is more principled because it uses the student's own predicted velocity field — the same field that will be used at inference — to determine where the trajectory drifts, creating off-trajectory points that are causally linked to the student's current parameters and directly represent the error that needs to be corrected.

Target construction for off-trajectory supervision. To supervise the off-trajectory latent x_{t'_i}, the CDM loss constructs its target using a local clean data estimate rather than reusing the anchor's estimate \hat{x}^{(i)}_0. The student passes the off-trajectory latent through itself at the extrapolated timestep:

$\hat{x}^{(i')}_0 = D_θ(x_{t'_i}, t'_i, c)$

This local prediction is then re-noised to a randomly sampled continuous timestep \hat{τ} ∼ U(0, 1]:

$z_{\hat{τ}} = (1-\hat{τ}) \, \text{sg}[\hat{x}^{(i')}_0] + \hat{τ} ε_{\hat{τ}}$

What this computes: instead of anchoring the target to the original trajectory's clean estimate (which would only provide global consistency), the target is anchored to the student's own prediction at the off-trajectory point. This creates a self-consistency constraint: the student's prediction at the off-trajectory latent should match the distribution implied by its own local clean estimate, evaluated through the teacher's score.

Why local anchoring. The paper's ablation (Table 2, right panel) explicitly compares this local anchoring against using the "full-trajectory final generation \hat{x}_0" (the clean estimate from the original anchor, computed by running the complete backward simulation from x_{t_i} to t ≈ 0). The local anchoring consistently outperforms the full-trajectory anchor because:

1. It provides a more direct error signal: if the student's velocity field is inconsistent, the extrapolated latent x_{t'_i} will be far from where it should be, and the student's prediction \hat{x}^{(i')}_0 at that point will differ substantially from what the teacher expects. The CDM loss directly penalizes this local mismatch.

2. It avoids compounding errors: the full-trajectory anchor \hat{x}_0 requires simulating all the way from x_{t_i} to t ≈ 0, accumulating additional integration errors along the way. If those intermediate steps are also wrong (which they will be, since the student is still being trained), the target becomes noisy and provides a weaker learning signal.

3. It acts as a local smoothness regularizer: by requiring the student's predictions to be distributionally consistent between nearby points (x_{t_i} and x_{t'_i}), the CDM loss implicitly constrains \|dv_θ/dt\| — the rate of change of the velocity field — which is precisely the quantity that controls Euler truncation error.

The CDM Loss equation. With the off-trajectory latent and local target constructed, the CDM loss follows the same mathematical form as the DM loss but applied to the extrapolated point:

$L_{CDM} = \frac{1}{2} \left\| D_θ(x_{t'_i}, t'_i, c) - \text{sg}\left[ D_θ(x_{t'_i}, t'_i, c) + w_{\hat{τ}} (D_ϕ(z_{\hat{τ}}, \hat{τ}, c) - D_ψ(z_{\hat{τ}}, \hat{τ}, c)) \right] \right\|^2_2$

where w_{\hat{τ}} = \|D_ϕ(z_{\hat{τ}}, \hat{τ}, c) - D_θ(x_{t'_i}, t'_i, c)\|^{-1}_1 is the same dynamic weighting.

What it computes: the squared L2 distance between the student's prediction at the off-trajectory latent and a target that equals that prediction plus the real-fake teacher score difference evaluated at the re-noised local clean estimate. This is functionally identical to the DM loss in Equation (4), but applied to an off-trajectory latent rather than an on-trajectory anchor.

Why the same form as the DM loss works. The distribution matching correction Δ_{cdm}^{real-fake} = D_ϕ(z_{\hat{τ}}, \hat{τ}, c) - D_ψ(z_{\hat{τ}}, \hat{τ}, c) has the same score-matching interpretation as the DM loss: it points from the student's current distribution (as estimated by the fake teacher) toward the real data distribution (as estimated by the real teacher). Applying this correction at the off-trajectory latent pushes the student's velocity field toward configurations where taking a large Euler step does not cause the trajectory to leave the data manifold. If the extrapolated latent x_{t'_i} has drifted off-manifold due to an inaccurate velocity prediction, the teacher score difference will be large, providing a strong gradient to correct the velocity field at the original anchor x_{t_i}.

Connection to truncation error — formal justification from Appendix E. The paper provides a formal analysis connecting the CDM loss to the truncation error of Euler integration. Consider the probability flow ODE dx_τ/dτ = v_θ(x_τ, τ, c). A single Euler step from t_j to t_{j+1} with step size h_j = t_j - t_{j+1} gives:

$\tilde{x}_{t_{j+1}} = x_{t_j} - h_j v_θ(x_{t_j}, t_j, c)$

Comparing this against the true ODE solution expanded via Taylor's theorem:

$x_{t_{j+1}} = x_{t_j} - h_j \dot{x}_{t_j} + \frac{1}{2} h_j^2 \ddot{x}_{ξ_j}$

for some ξ_j ∈ (t_{j+1}, t_j). Since \dot{x}_{t_j} = v_θ(x_{t_j}, t_j, c) by definition, the local truncation error is:

$\|x_{t_{j+1}} - \tilde{x}_{t_{j+1}}\| = \frac{1}{2} h_j^2 \|\ddot{x}_{ξ_j}\| ≤ \frac{1}{2} h_j^2 \cdot M_2^{(j)}$

where M_2^{(j)} = \sup_{τ ∈ [t_{j+1}, t_j]} \|dv_θ/dτ\| is the maximum rate of change of the velocity field along the trajectory over the step interval, and dv_θ/dτ is the material derivative:

$\frac{dv_θ}{dτ} = ∂_τ v_θ(x_τ, τ, c) + J_x v_θ(x_τ, τ, c) \, v_θ(x_τ, τ, c)$

This material derivative combines the partial derivative with respect to time (how the velocity field changes as time evolves even at a fixed point) and the convective term (how the velocity field changes as the point moves along the trajectory). Both contribute to integration error — if v_θ changes rapidly, the linear approximation used by Euler integration becomes poor.

How CDM suppresses M₂. Consider the first-order Taylor expansion of the velocity field at the extrapolated point around (x_{t_i}, t_i):

$v_θ(x_{t'_i}, t'_i) ≈ v_θ(x_{t_i}, t_i) + ∂_τ v_θ(x_{t_i}, t_i, c) Δt + J_x v_θ(x_{t_i}, t_i, c) (x_{t'_i} - x_{t_i})$

where Δt = t'_i - t_i. Substituting x_{t'_i} - x_{t_i} = Δt \, v_θ(x_{t_i}, t_i, c) (from the extrapolation definition in Equation 7):

$v_θ(x_{t'_i}, t'_i) ≈ v_θ(x_{t_i}, t_i) + Δt (∂_τ v_θ + J_x v_θ \, v_θ) = v_θ(x_{t_i}, t_i) + Δt \frac{dv_θ}{dτ}$

Rearranging to isolate the material derivative:

$\frac{dv_θ}{dτ} ≈ \frac{v_θ(x_{t'_i}, t'_i) - v_θ(x_{t_i}, t_i)}{Δt}$

This reveals that the material derivative is approximated by the finite difference of the velocity field between the anchor and the extrapolated point. The CDM loss supervises the student at x_{t'_i} by requiring v_θ(x_{t'_i}, t'_i) ≈ v_ϕ(x_{t'_i}, t'_i) (the student's velocity at the off-trajectory point should match the teacher's velocity there). Simultaneously, the DM loss supervises at x_{t_i} by requiring v_θ(x_{t_i}, t_i) ≈ v_ϕ(x_{t_i}, t_i). Together, these constraints force:

$\frac{dv_θ}{dτ} ≈ \frac{v_ϕ(x_{t'_i}, t'_i) - v_ϕ(x_{t_i}, t_i)}{Δt} = \frac{dv_ϕ}{dτ}$

That is, the student's material derivative is forced to match the teacher's material derivative. Since the pre-trained teacher has a naturally smooth velocity field with bounded variation (it was trained on dense 100-step trajectories and produces accurate integration with small step sizes), CDM transfers this smoothness property to the student. This directly suppresses M_2^{(j)} = \sup \|dv_θ/dτ\|, which in turn reduces the local truncation error bound (1/2) h_j^2 M_2^{(j)}.

Global error accumulation. Accumulating the local error over all N steps and applying the Lipschitz continuity of the ODE flow, the global error at the final timestep is bounded by:

$\|x_ε - \hat{x}_ε\| ≤ \frac{e^{L(1-ε)} - 1}{L} \cdot \frac{1}{2} \bar{h} M_2$

where L is the Lipschitz constant of the velocity field, \bar{h} = \max_j h_j is the maximum step size, and M_2 = \max_j M_2^{(j)} is the global supremum of the material derivative. For a fixed step budget N, \bar{h} ≈ 1/N is determined by the inference schedule, and e^{L(1-ε)}/L is a property of the ODE dynamics. The only term that can be optimized by training is M₂ — and CDM provides the mechanism to do exactly that.

Why Gaussian perturbation fails. The paper's ablation (Table 2, right panel) compares the velocity-driven extrapolation against a Gaussian perturbation baseline: predict \hat{x}_0 from x_{t_i}, then construct x'_t = (1-t)\hat{x}_0 + tε (re-noising with fresh Gaussian noise). This baseline performs worse because:

1. Re-noising adds isotropic Gaussian noise, which is a generic perturbation that does not match the structured, anisotropic truncation error of Euler integration. The actual error x_{t_{j+1}} - \tilde{x}_{t_{j+1}} is not random white noise — it is a deterministic function of the velocity field curvature.
2. The velocity-driven extrapolation uses the student's own v_θ to determine the perturbation direction, creating off-trajectory points that are causally connected to the student's parameters. This means the gradient of the CDM loss with respect to θ correctly accounts for how changes in v_θ affect the truncation error.
3. The ablation "w/o Perturbation (on-traj)" — which keeps supervision only on the original on-trajectory anchors — performs worst of all, confirming that off-trajectory supervision (whether velocity-driven or Gaussian-based) provides benefits beyond on-trajectory matching alone. But velocity-driven is the principled choice because it targets the actual error mode.

The complete picture: how CDM and DM losses complement each other. The DM loss provides on-trajectory supervision that pushes the student's distribution toward the teacher's at points visited by backward simulation. The CDM loss provides off-trajectory supervision that pushes the student's velocity field toward smoothness (low M_2) by penalizing invalid extrapolations. Neither alone is sufficient: DM-only produces a distributionally correct but non-smooth velocity field that generates poor few-step samples due to truncation error; CDM-only (without DM) would provide local consistency but might drift from the global data manifold over many steps. Together, they ensure both distributional accuracy (DM anchors to the teacher's manifold) and integration accuracy (CDM smooths the velocity field).

---

The Complete Training Algorithm

The full CDM training procedure (Algorithm 1 in Appendix F) integrates all components into a single training iteration. Here is the complete flow:

Step 1: Backward simulation (no gradient). Sample a prompt c, sample simulation length N ∼ U{1, N_max}, generate a random continuous schedule 1 = t_1 > t_2 > ... > t_N > 0, initialize x_{t_1} ∼ N(0, I), and run N Euler steps with the student (under torch.no_grad()) to produce the trajectory {x_{t_n}}. Sample an anchor index i ∼ U{1, ..., N} and keep the latent x_{t_i}. The backward simulation is done without gradient because the entire trajectory is treated as given — the gradients flow only through the student's predictions at the extracted anchor and extrapolated point, not through the integration itself.

Step 2: Fake teacher update. The student computes \hat{x}^{(i)}_0 = D_θ(x_{t_i}, t_i, c). A timestep τ_ψ ∼ U(0, 1] is sampled, the clean estimate is re-noised to z_{τ_ψ} = (1-τ_ψ) \text{sg}[\hat{x}^{(i)}_0] + τ_ψ ε_ψ, and the fake teacher is updated via gradient descent on \|v_ψ(z_{τ_ψ}, τ_ψ, c) - (ε_ψ - \text{sg}[\hat{x}^{(i)}_0])\|^2_2. The fake teacher update occurs on a separate optimizer with learning rate 5 × 10^{-6} and is performed 2 times per student update (TTUR).

Step 3: CA Loss computation. Sample τ ∼ U(0, 1], construct z_τ = (1-τ) \hat{x}^{(i)}_0 + τ ε_τ, compute teacher predictions D_ϕ(z_τ, τ, c) and D_ϕ(z_τ, τ, ∅), compute dynamic weight w_τ, and evaluate Equation (3).

Step 4: DM Loss computation. Sample \tilde{τ} ∼ U(0, 1] independently of τ, construct z_{\tilde{τ}} = (1-\tilde{τ}) \hat{x}^{(i)}_0 + \tilde{τ} ε_{\tilde{τ}}, compute real teacher prediction D_ϕ(z_{\tilde{τ}}, \tilde{τ}, c) and fake teacher prediction D_ψ(z_{\tilde{τ}}, \tilde{τ}, c), compute dynamic weight w_{\tilde{τ}}, and evaluate Equation (4).

Step 5: CDM Loss computation. Sample t'_i ∼ U(0, 1] independently of everything above, compute student velocity v_{t_i} = v_θ(x_{t_i}, t_i, c), construct extrapolated latent x_{t'_i} = x_{t_i} + (t'_i - t_i) v_{t_i}, pass through student to get local clean estimate \hat{x}^{(i')}_0 = D_θ(x_{t'_i}, t'_i, c), sample \hat{τ} ∼ U(0, 1], construct z_{\hat{τ}} = (1-\hat{τ}) \text{sg}[\hat{x}^{(i')}_0] + \hat{τ} ε_{\hat{τ}}, compute real and fake teacher predictions, compute dynamic weight w_{\hat{τ}}, and evaluate Equation (9).

Step 6: Student update. Sum the three losses L = L_{CA} + L_{DM} + L_{CDM} (all weighted equally — no hyperparameter coefficients), compute ∇_θ L, and update student parameters with the AdamW optimizer.

Why equal weighting of losses. The paper uses equal weights of 1.0 for all three loss terms and does not ablate different weighting schemes. This simplicity is enabled by the dynamic weight factors w_τ, w_{\tilde{τ}}, and w_{\hat{τ}} that normalize each loss's gradient magnitude at the per-sample level — they prevent any single loss term from dominating the gradient regardless of the timestep or the prediction discrepancy magnitude. Without these dynamic weights, the CA loss (which involves large CFG corrections at intermediate noise levels) would dominate the DM and CDM losses, and hyperparameter tuning of loss coefficients would be necessary.

Why gradient does not flow through the backward simulation. The backward simulation is performed under torch.no_grad() — the trajectory latents x_{t_n} are treated as fixed inputs. This is a standard practice in DMD to prevent the computationally expensive and potentially unstable backpropagation through many steps of numerical integration. The gradients flow through D_θ(x_{t_i}, t_i, c) at the extracted anchor and through the velocity prediction and extrapolation in the CDM loss. This is sufficient because the anchor x_{t_i} was produced by the student's own integration — as the student's parameters change, the distribution of x_{t_i} changes (even without explicit gradient flow), and the losses computed on the new anchors guide the student toward the correct distribution.

Training configuration for SD3-Medium. The full training setup (Appendix C.1):

• Dataset: 200K prompts randomly sampled from T2I-2M, LAION, ShareGPT-4o-Image, PickScore, and FlowGRPO OCR training sets
• Optimizer: AdamW with learning rate 1 × 10^{-5} for the student, 5 × 10^{-6} for the fake teacher
• Weight decay: 0.001
• Adam betas: (0.9, 0.999)
• Batch size: 128
• CFG guidance scale α: 7.0
• Maximum simulation length N_max: 28
• Training iterations: 4K
• Hardware: 16 A100 GPUs
• Wall-clock time: approximately 24 hours
• Fine-tuning: full fine-tuning (no LoRA) on the student network

Training configuration for Longcat-Image. Identical hyperparameters except:

• LoRA fine-tuning with rank 64 and alpha 128 (to handle the larger model size efficiently)
• Batch size: 64 (reduced due to memory constraints)
• Training iterations: 2K
• Hardware: 16 A100 GPUs
• Wall-clock time: approximately 24 hours

Why full fine-tuning for SD3-Medium but LoRA for Longcat-Image. Longcat-Image is a substantially larger model, making full fine-tuning memory-prohibitive. LoRA provides a parameter-efficient alternative that still enables effective distillation because the student needs to learn a relatively targeted skill (accelerated sampling) rather than a completely new data distribution. The rank 64 and alpha 128 configuration provides sufficient capacity for learning the continuous-time velocity field regularization while keeping memory requirements manageable.

Relationship to prior DMD variants. CDM builds most directly on D-DMD [24], adopting its decoupled CA+DM formulation and the single shared anchor for all losses. The key departures are: (1) the dynamic schedule replaces the fixed inference-aligned schedule, (2) the CDM loss adds a third term targeting off-trajectory latents, and (3) no GAN or adversarial auxiliary objective is used. Compared to DMD2 [58], CDM additionally adopts the decoupled formulation and drops the adversarial loss, achieving better results through continuous-time regularization rather than adversarial training.

4. Key Insights and Innovations

Innovation 1: Training-Inference Schedule Coupling Is a Bug, Not a Feature

The paper's most conceptually disruptive move is the empirical refutation of what was, until now, a foundational design principle in distribution matching distillation: the requirement that the training backward simulation must use the exact same discrete timesteps as the target inference schedule. Prior work — DMD [59], DMD2 [58], D-DMD [24] — universally adopted this coupling as an obvious correctness constraint, reasoning that if the student is trained on a different schedule than it will use at inference, a train-test distribution mismatch would degrade performance. This assumption went unchallenged because it aligns with standard ML intuition.

The paper overturns it with a simple, decisive experiment (Figure 2): keep everything else in a D-DMD setup identical, but randomize the backward simulation length N ∼ U{1, N_max} and sample anchors continuously from (0, 1] at each iteration. The result — higher HPSv3 scores, visibly sharper textures, and fewer artifacts — demonstrates that exact schedule matching is not merely unnecessary; it is actively harmful. The fixed schedule confines the student to receiving teacher gradients at only a handful of noise levels, leaving the velocity field unregularized in the vast continuous intervals between those anchors. The dynamic schedule, by contrast, exposes the student to the full continuous time domain, providing dense supervision that implicitly smooths the velocity field.

This finding is fundamental rather than incremental because it changes what researchers should consider a constraint versus a design choice. The dominant assumption was that training-inference alignment is a hard requirement; the paper shows it is a premature optimization that restricts performance. This insight has implications beyond CDM: any distillation method that currently enforces schedule coupling (progressive distillation, trajectory matching methods) should re-examine this constraint as a potential source of performance degradation rather than a necessary design element. The theoretical justification in Appendix D provides additional depth: the score-matching gradient decompositions reveal that the student's anchor timestep t_i and the teacher's perturbation timesteps τ, τ̃ are independent variables whose product determines the effective supervision density, so restricting t_i to sparse anchors directly reduces the information content of each training iteration.

The authors are careful to note that this finding emerged from "preliminary empirical analysis" (Section 1) conducted specifically to challenge the coupling assumption, not from a theoretical prediction. This is a diagnostic contribution — identifying that a widely-held design principle is actually counterproductive — which is often more valuable than proposing a new method, because it unblocks an entire class of improvements that were previously considered off-limits.

---

Innovation 2: Off-Trajectory Supervision as a First-Class Mechanism for Controlling Truncation Error

Prior to CDM, distribution matching distillation operated exclusively on on-trajectory latents — points visited by the backward simulation that lie (approximately) on the student's probability flow ODE path. The underlying assumption was that matching distributions at these points would be sufficient for the student to learn a velocity field that integrates accurately. This paper identifies why that assumption fails for few-step generation and provides the first principled mechanism for addressing the failure mode.

The core diagnostic insight (formalized in Appendix E) is that few-step Euler integration introduces a specific, predictable error mode: the local truncation error at each step scales with M₂ = sup ‖dv_θ/dτ‖, the maximum rate of change of the student's velocity field between integration anchors. Standard on-trajectory losses provide no direct signal about this quantity — they constrain the student's predictions at discrete points but leave the inter-anchor behavior unregularized. The student can achieve low loss on all anchors while developing rapid oscillations or sharp transitions between them, producing a velocity field that looks locally accurate but integrates poorly with large step sizes.

CDM's key conceptual contribution is recognizing that off-trajectory supervision is not merely "more data" — it targets the specific mathematical quantity that controls few-step integration error. The velocity-driven extrapolation x_{t'_i} = x_{t_i} + (t'_i - t_i) v_{t_i} constructs points that lie off the ideal ODE trajectory in exactly the way that Euler truncation error displaces them. By enforcing distribution matching at these points, the CDM loss provides a gradient signal that explicitly penalizes large values of dv_θ/dτ, because if the velocity field changes rapidly between the anchor and the extrapolated point, the student's prediction at the extrapolated point will be distributionally inconsistent with the teacher's score.

This is a fundamental conceptual shift, not an incremental improvement. Prior work treated "off-trajectory" as synonymous with "augmented" or "perturbed" — Diffusion Forcing, re-noising baselines, and the concurrent work [40] add perturbations that are statistically generic (Gaussian noise) rather than causally linked to the student's own velocity field. The velocity-driven extrapolation in CDM is qualitatively different: it creates off-trajectory points that are deterministic functions of the student's current parameters, meaning the gradient of the CDM loss correctly accounts for how changes in the velocity field affect the truncation error. The ablation in Table 2 (right panel) validates that this causality matters: Gaussian perturbation improves over on-trajectory-only matching, but velocity-driven extrapolation is consistently better across all metrics. This confirms that the mechanism, not just the fact of off-trajectory supervision, is what drives the gains.

The broader implication is that few-step generation should be approached as a numerical integration problem, not just a statistical matching problem. The constraints that matter — bounded material derivative, Lipschitz continuity, smooth interpolation between anchors — are classical numerical analysis concepts that prior distillation work largely ignored in favor of purely statistical objectives. CDM bridges this gap by showing that distribution matching can be repurposed to enforce numerical integration properties when applied at the right points in the right way.

---

Innovation 3: The Distribution Matching Loss Is Not a Stabilizer — It Faithfully Captures the Teacher's CFG-Free Distribution

The decoupled DMD framework [24] characterized the distribution matching (DM) loss as primarily a "regularizer" or "shield" — something that prevents training collapse and mitigates artifacts, but not the primary driver of generation quality. The CA loss, in this view, was the "spear" that enabled few-step conversion by providing text-guidance signals from the teacher's CFG direction. This characterization has practical consequences: if the DM loss is merely a stabilizer, improvements to it yield diminishing returns, and research effort should focus on better CA formulations, auxiliary objectives, or architectural modifications.

The paper directly challenges this framing through a diagnostic experiment (Figure 3, Appendix Table 4) that reveals the DM loss is something much more specific and substantive. When a student is trained with only the DM loss (no CA loss), its outputs closely match the teacher's CFG-free samples — the teacher's unguided marginal distribution without classifier-free guidance. Quantitative evaluation (Table 4) confirms that the DM-only student's metrics tightly track the CFG-free teacher across all dimensions on both SD3-Medium and Longcat-Image. On SD3-Medium, the DM-only student achieves DPGBench 70.60 vs. the CFG-free teacher's 69.65 and HPSv3 7.258 vs. 4.693, while the CFG-enabled teacher achieves 85.04 and 8.189 respectively. The gap between the DM-only student and the CFG-enabled teacher is large and consistent, but the gap between the DM-only student and the CFG-free teacher is small — indicating the DM loss is doing exactly what it was mathematically designed to do (match the teacher's marginal distribution), just without the guidance signal that makes that distribution useful for text-conditional generation.

This finding is diagnostically significant in two ways. First, it clarifies the division of labor between CA and DM losses: the CA loss does not just "augment" the DM loss — it provides the entire text-alignment signal. The DM loss anchors the student to the data manifold (the CFG-free distribution), and the CA loss pushes it toward prompt-consistent regions of that manifold. Neither alone is sufficient; together they form a complete distillation signal. Second, and more importantly for CDM's contributions, it implies that improvements to the DM loss directly improve generation quality by providing a more accurate and comprehensive manifold anchor. The continuous-time and off-trajectory extensions in CDM are motivated by this insight: making the DM loss more faithful (by applying it densely across the continuous time domain) and more comprehensive (by extending it to off-trajectory points that capture integration error) should directly translate to better outputs, because the DM loss is an active quality driver, not a passive stabilizer.

This reframing also explains a puzzling pattern in prior work: why DMD-distilled models consistently require auxiliary objectives (GANs, reward models) while consistency-distilled models often do not. Consistency models' self-consistency objective naturally provides the manifold-anchoring function of the DM loss — it ensures the student's outputs lie on the data manifold at every timestep by enforcing consistency with the trajectory's origin. DMD, with its sparse on-trajectory DM loss, only weakly anchors the manifold, leaving the student vulnerable to off-manifold drift that auxiliary objectives must correct. CDM's continuous-time DM and CDM losses strengthen this anchoring to the point where auxiliary objectives become unnecessary — Table 1 shows CDM surpassing D-DMD (which already outperforms DMD2 without GAN) by substantial margins, purely through better distribution matching.

---

Innovation 4: Demonstrating That Pure Distribution Matching Can Match or Exceed Hybrid Methods at Scale

A consistent narrative in the diffusion distillation literature has been that distribution matching alone is insufficient for high-quality few-step generation — it must be combined with adversarial training (DMD2's GAN loss), reward-model guidance (Flash Diffusion), or consistency-based regularization (Hyper-SD). The reasoning has been that the mode-seeking nature of the reverse KL divergence used in DMD causes over-smoothing and detail loss, and that only adversarial or reward-based signals can recover the high-frequency content needed for perceptual quality.

CDM provides a significant counterexample to this narrative. On SD3-Medium at 4 NFE (Table 1), CDM achieves the best results across nearly all metrics among image-free methods — Aesthetic 6.075, DPGBench 85.26, PickScore 21.95, HPSv3 9.561 — without any GAN, reward model, or consistency objective. These numbers surpass D-DMD (HPSv3 9.176) and DMD2 (HPSv3 8.419), both of which represent prior best-in-class distribution matching methods. On Longcat-Image, CDM pushes HPSv3 to 10.65, surpassing even the 100-NFE teacher on several metrics. The visual comparisons in Figure 5 and Appendix J show CDM producing sharper textures, richer fine details, and better material properties than competing methods, with no evidence of the over-smoothing that has been considered inherent to reverse-KL-based distillation.

The significance of this result is that it changes the perceived ceiling of distribution matching as a paradigm. Prior work implicitly assumed a quality tradeoff: you can have distribution matching (stable training, good mode coverage) or adversarial training (sharp details, high perceptual quality), but not both at the maximum level. The proliferation of hybrid methods combining DMD with GANs or consistency distillation reflected this assumption. CDM suggests that the apparent ceiling was not a fundamental limitation of distribution matching itself, but rather an artifact of the discrete-time, on-trajectory-only formulation. By densifying the supervision (continuous schedule) and adding off-trajectory regularization (CDM loss), pure distribution matching can achieve state-of-the-art quality — implying that the "adversarial gap" was actually a "temporal coverage gap."

This is an important negative result for the auxiliary-objective approach: it demonstrates that the complexity, instability, and computational cost of GAN training are unnecessary if the core distribution matching signal is sufficiently comprehensive. The paper does not argue that GANs or reward models are universally harmful — they may still provide benefits in regimes CDM hasn't explored — but it establishes a new baseline where pure distribution matching is competitive with hybrid methods, shifting the burden of proof onto auxiliary objectives to demonstrate that their benefits justify their costs.

However, this claim requires careful contextualization. The concurrent work [25] and [40] explore related ideas (off-trajectory supervision, score-based alignment) and may independently achieve similar results. The single-backbone, single-resolution evaluation in this paper cannot establish universality. What it can establish — and does, convincingly — is that the "distribution matching needs adversarial help" narrative was premature and based on an incomplete formulation of distribution matching itself.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary evaluation uses 2K prompts sampled from the test split of the PickScore dataset for computing Aesthetic Score, PickScore, HPSv3, and CLIP Score. Additionally, DPG-Bench (DPG) is evaluated on 1K prompts for fine-grained prompt adherence. For text rendering evaluation (Appendix G), OCR accuracy is calculated on 1K prompts from FlowGRPO, and FID is computed using 10K prompts from the COCO 2014 validation set. Training data consists of 200K prompts randomly sampled from T2I-2M, LAION, ShareGPT-4o-Image, PickScore, and FlowGRPO OCR training sets.

• Base model(s). The primary experiments use SD3-Medium, a flow-matching transformer for text-to-image generation operating at 1024×1024 resolution. This model was chosen as a representative modern backbone with publicly available weights and strong baseline performance. Additional experiments extend to Longcat-Image, a larger and more recent flow-matching model also at 1024×1024, to demonstrate cross-architecture generalization. The teacher models are used in their pretrained form with 100 NFE; the distilled students all operate at 4 NFE.

• Metrics. Five primary metrics are reported: Aesthetic Score (AES) — a learned predictor of visual aesthetic quality; DPGBench — prompt adherence evaluated on the DPG-Bench benchmark, measuring whether generated images contain specific attributes and relationships described in prompts; PickScore — a human-preference-aligned scoring model trained on the Pick-a-Pic dataset; HPSv3 — a wide-spectrum human preference score covering multiple quality dimensions; and CLIP Score (ViT-H-14) — cosine similarity between image and text CLIP embeddings, measuring semantic alignment. Additionally, OCR accuracy (via PaddleOCR) measures text rendering fidelity, and FID (Fréchet Inception Distance on 10K COCO 2014 val prompts) measures distributional fidelity. All metrics use the respective official implementations without modification.

• Baselines. The paper compares against five leading few-step generation methods: Hyper-SD [41] — a trajectory-segmented consistency model operating at 8 NFE; Flash Diffusion (Flash) [1] — a GAN-augmented distribution matching approach at 4 NFE; TDM [32] — trajectory distribution matching at 4 NFE; DMD2 [58] — improved distribution matching distillation with adversarial loss at 4 NFE; and D-DMD [24] — decoupled DMD with separate CA and DM losses at 4 NFE. For DMD2 and D-DMD, since no official SD3-Medium checkpoints were available, the authors re-implement them within their unified framework under identical settings to CDM and notably without the GAN-based adversarial loss that DMD2 originally included. The 100-NFE teacher model serves as an upper-bound reference but is excluded from method rankings.

• Generation budget / compute accounting. All comparisons are made at equal NFE (Number of Function Evaluations) — the number of forward passes through the neural network during inference. The primary setting is 4 NFE for all distilled methods. CDM introduces no additional inference cost compared to D-DMD or DMD2 at the same NFE since the continuous-time training modifications affect only the training procedure, not the inference architecture or step count. Training cost is reported separately: CDM requires approximately 1.8× the training time of D-DMD due to the longer average backward simulation length (random N ∼ U{1, 28} vs. fixed N = 4) and the extra forward passes for the CDM loss (one additional real teacher and fake teacher evaluation on the extrapolated latent). Training memory is comparable (62.5 GB for CDM vs. 62.2 GB for D-DMD).

• Cross-validation / statistical protocol. No cross-validation or statistical significance testing is reported. All results are from single training runs evaluated on the fixed test splits described above. The paper does not report confidence intervals, standard deviations, or multiple random seeds. For the ablation studies in Table 2, results are presented as single-point metric values without error bars or variance estimates.

Main Quantitative Results

Overall Performance Comparison on SD3-Medium

The headline results appear in Table 1 (SD3-Medium, 4 NFE). CDM achieves the best scores among all image-free methods on four of five primary metrics: Aesthetic 6.075, DPGBench 85.26, PickScore 21.95, and HPSv3 9.561, while maintaining a competitive CLIPScore of 27.98. The HPSv3 improvement is particularly notable: at 9.561, CDM substantially exceeds the prior best image-free method D-DMD (9.176, a delta of +0.385) and also surpasses the GAN-augmented Flash method (8.282). The CLIPScore of 27.98 is slightly lower than Flash (28.18) and the teacher (28.60), but the other four metrics strongly favor CDM.

Comparing against specific baselines on SD3-Medium:

• vs. D-DMD [24]: CDM improves Aesthetic from 6.038 to 6.075 (+0.037), DPGBench from 84.52 to 85.26 (+0.74), PickScore from 21.85 to 21.95 (+0.10), HPSv3 from 9.176 to 9.561 (+0.385), and CLIPScore from 27.69 to 27.98 (+0.29). The DPGBench and HPSv3 gains are the most substantial.

• vs. DMD2 [58]: CDM improves Aesthetic from 6.038 to 6.075 (+0.037), DPGBench from 83.96 to 85.26 (+1.30), PickScore from 21.58 to 21.95 (+0.37), HPSv3 from 8.419 to 9.561 (+1.142), and CLIPScore from 27.56 to 27.98 (+0.42). The HPSv3 gain exceeds a full point, representing a substantial jump in human-preference alignment.

• vs. Flash [1] (GAN-augmented, 4 NFE): CDM achieves higher Aesthetic (6.075 vs. 5.968), DPGBench (85.26 vs. 80.47), PickScore (21.95 vs. 21.69), and HPSv3 (9.561 vs. 8.282), with slightly lower CLIPScore (27.98 vs. 28.18). All advantages exceed the margins between other competing methods.

• vs. 100-NFE teacher: CDM at 4 NFE actually surpasses the teacher on Aesthetic (6.075 vs. 5.885), DPGBench (85.26 vs. 85.04), PickScore (21.95 vs. 21.73), and HPSv3 (9.561 vs. 8.189), while trailing on CLIPScore (27.98 vs. 28.60). This is a striking result: at 25× fewer function evaluations, CDM exceeds the teacher on four of five metrics. The authors note this suggests "the proposed continuous-time optimization framework provides supervision signals that go beyond merely replicating the teacher's outputs" (Section 4.2).

Overall Performance Comparison on Longcat-Image

On Longcat-Image (Table 1, 4 NFE), CDM achieves the best results on Aesthetic 5.919, DPGBench 88.35, PickScore 21.53, and HPSv3 10.65, with CLIPScore 26.72. The HPSv3 score of 10.65 on Longcat-Image surpasses even the 100-NFE teacher's HPSv3 of 9.450 by 1.20 points — the largest teacher-surpassing margin in the table. This is consistent with the SD3-Medium finding that continuous-time optimization produces quality improvements beyond simple teacher replication.

Comparing against baselines on Longcat-Image:

• vs. D-DMD [24]: CDM improves Aesthetic from 5.782 to 5.919 (+0.137), DPGBench from 88.04 to 88.35 (+0.31), PickScore from 21.23 to 21.53 (+0.30), HPSv3 from 9.629 to 10.65 (+1.021), and CLIPScore from 26.57 to 26.72 (+0.15). The HPSv3 improvement exceeds one point.

• vs. DMD2 [58]: CDM improves Aesthetic from 5.800 to 5.919 (+0.119), DPGBench from 87.12 to 88.35 (+1.23), PickScore from 21.07 to 21.53 (+0.46), HPSv3 from 8.803 to 10.65 (+1.847), and CLIPScore from 26.99 to 26.72 (−0.27, the only metric where CDM trails DMD2 on Longcat-Image).

• vs. 100-NFE teacher: CDM surpasses the teacher on Aesthetic (5.919 vs. 5.926, essentially tied), DPGBench (88.35 vs. 87.08), PickScore (21.53 vs. 21.65, slightly lower), HPSv3 (10.65 vs. 9.450), and CLIPScore (26.72 vs. 26.78, essentially tied).

The cross-backbone consistency — CDM achieving the top HPSv3 on both SD3-Medium and Longcat-Image, and exceeding the teacher on both — provides evidence that the continuous-time formulation generalizes beyond a single architecture.

Extended Quantitative Comparison: Quality vs. Efficiency

Table 3 provides a more detailed comparison with the strongest baseline, D-DMD, including two additional quality metrics (OCR accuracy and FID) and efficiency measurements.

Quality metrics beyond the main table:

• OCR accuracy: CDM achieves 34.82%, compared to D-DMD's 33.47% and the fixed-schedule CDM variant at 37.33%. CDM ranks second on this metric, with the fixed-schedule variant showing the best text rendering. This is a non-obvious result — the dynamic schedule provides broad quality gains but the fixed variant may provide more focused supervision for text-specific generation, or this could reflect noise in the single-run evaluation.

• FID (lower is better): CDM achieves 30.30, compared to D-DMD's 31.47 and the fixed-schedule variant at 31.05. CDM produces the best distributional fidelity to the teacher's outputs on a 10K COCO 2014 validation set, consistent with its distribution matching formulation aiming to minimize divergence from the teacher.

Efficiency measurements:

• Training memory: CDM uses 62.5 GB, very close to D-DMD's 62.2 GB and the fixed-schedule variant's 62.5 GB. The CDM loss's extra forward pass on the extrapolated latent adds negligible memory overhead.

• Relative training time: CDM requires 1.8× the training time of D-DMD, reflecting two sources of overhead: (1) the dynamic schedule samples N ∼ U{1, 28}, resulting in an average backward simulation length of ~14.5 steps vs. D-DMD's fixed 4 steps (3.6× longer on average), and (2) the CDM loss requires one additional forward pass of the real teacher and fake teacher on the extrapolated latent. Despite this training overhead, the fixed-schedule CDM variant requires only 1.1× training time — close to D-DMD — suggesting most of the 1.8× overhead comes from the longer backward simulation, not the CDM loss itself.

• Inference latency: All three configurations (D-DMD, CDM fixed schedule, CDM) share identical inference latency of 246 ms per image at 1024×1024 with 4 NFE on a single GPU. This is the critical practical result: CDM's quality improvements come entirely at training time and provide zero-cost deployment.

Qualitative Results

Figure 5 presents side-by-side comparisons across six methods (SD3-Medium teacher at 100 NFE, Hyper-SD at 8 NFE, Flash at 4 NFE, TDM at 4 NFE, DMD2 at 4 NFE, D-DMD at 4 NFE, and CDM at 4 NFE) for two prompts: "An image of a full moon on a starry night" and "Perfume advertisement, backlight, make it realistic with insane details, stunning advertisement." CDM consistently produces sharper textures, finer background details, and more pronounced material reflections and lighting effects than the competing few-step methods. On the moon prompt, CDM's stars and lunar surface details are visibly crisper than D-DMD's; on the perfume prompt, CDM's glass reflections and lighting gradients appear more photorealistic than the other 4 NFE methods. The 100-NFE teacher provides a reference; CDM at 4 NFE matches or visually surpasses it on these examples — an observation consistent with the quantitative teacher-surpassing results in Table 1.

Appendix J provides extensive additional qualitative samples across both backbones. Figure 7 demonstrates that CDM supports varying NFE at inference without retraining, showing coherent and prompt-aligned images at NFE ∈ {3, 4, 6, 8} from the same checkpoint, with finer details emerging as NFE increases — a direct consequence of the continuous-time regularization producing a well-behaved velocity field that supports accurate integration with arbitrary step counts, not just the training-time target.

Ablation Studies and Robustness Checks

All ablations are conducted on SD3-Medium at 4 NFE and reported in Table 2.

Single Loss Ablation (Table 2, left panel, rows a):

• Without L_CA (w/o LCA): Training with only L_DM + L_CDM yields Aesthetic 5.861, DPGBench 72.87, PickScore 21.05, HPSv3 8.128, and CLIPScore 24.78. This represents a massive drop from the full model (DPGBench −12.39, CLIPScore −3.20), confirming that the CA loss provides essentially all text-alignment signal. This is consistent with the paper's insight in Figure 3 — the DM and CDM losses capture the teacher's CFG-free distribution, which lacks prompt adherence.

• Without L_DM (w/o LDM): Training with only L_CA + L_CDM yields Aesthetic 6.016, DPGBench 84.57, PickScore 21.75, HPSv3 8.954, and CLIPScore 27.66. Compared to the full model, DPGBench drops by 0.69 and HPSv3 drops by 0.607, while Aesthetic and CLIPScore remain close. This confirms the DM loss contributes significantly to prompt adherence and human preference alignment, even though the CA loss is the primary text-alignment driver.

• Without L_CDM (w/o LCDM): Training with only L_CA + L_DM yields Aesthetic 6.067, DPGBench 85.12, PickScore 21.85, HPSv3 9.153, and CLIPScore 27.91. This is the closest to the full model — DPGBench drops by only 0.14, but HPSv3 drops by 0.408, a meaningful gap. This demonstrates that the CDM loss provides a measurable HPSv3 improvement even when the other metrics are nearly saturated.

Dual Loss Ablation (Table 2, left panel, rows b):

• Without L_DM and L_CDM (w/o LDM&LCDM): Training with L_CA alone yields Aesthetic 4.634, DPGBench 3.45, PickScore 17.50, HPSv3 −10.15, and CLIPScore 14.60. This is a complete collapse — the negative HPSv3 score indicates outputs that are actively dispreferred by human evaluators. This aligns with D-DMD's characterization of the DM loss as preventing training collapse, but CDM's ablation goes further by showing that L_DM + L_CDM together are necessary — L_CA alone is insufficient.

• Without L_CA and L_CDM (w/o LCA&LCDM): Training with L_DM alone yields Aesthetic 5.787, DPGBench 70.60, PickScore 20.82, HPSv3 7.258, CLIPScore 25.31. This matches the distribution from Figure 3 — the student learns the teacher's CFG-free distribution. It avoids collapse (unlike L_CA alone) but lacks text-alignment.

• Without L_CA and L_DM (w/o LCA&LDM): Training with L_CDM alone yields Aesthetic 5.778, DPGBench 72.38, PickScore 20.80, HPSv3 7.331, CLIPScore 24.78. Interestingly, L_CDM alone marginally outperforms L_DM alone on DPGBench (72.38 vs. 70.60) and HPSv3 (7.331 vs. 7.258), though both are far below the full model. This suggests the off-trajectory regularization provides some weak text-alignment signal (possibly through the re-noising operations that involve the text conditioning) but is not a substitute for the dedicated CA loss.

Full CDM: Aesthetic 6.075, DPGBench 85.26, PickScore 21.95, HPSv3 9.561, CLIPScore 27.98. The gap between the full model and the "w/o LCDM" ablation confirms that the CDM loss provides a non-trivial contribution, particularly to HPSv3 (+0.408). The gap between the full model and both "w/o LDM" and "w/o LCA" confirms that all three losses are necessary for peak performance.

Core Mechanism Design — Time Schedule (Table 2, right panel, row a):

• Fixed Schedule vs. Dynamic Schedule: Reverting the dynamic continuous schedule to a standard fixed schedule aligned with the 4-step inference grid reduces Aesthetic from 6.075 to 6.051 (−0.024), DPGBench from 85.26 to 83.84 (−1.42), PickScore from 21.95 to 21.89 (−0.06), HPSv3 from 9.561 to 9.482 (−0.079), and CLIPScore from 27.98 to 27.75 (−0.23). The DPGBench drop of 1.42 points is the most substantial, indicating that the dynamic schedule particularly benefits fine-grained prompt adherence — likely because dense time-domain supervision produces a smoother velocity field that more accurately follows the teacher's intended semantic structure during few-step integration.

Core Mechanism Design — Off-trajectory Perturbation Strategy (Table 2, right panel, row b):

• No perturbation (on-trajectory only): Removing off-trajectory supervision entirely and applying distribution matching only at on-trajectory anchors degrades Aesthetic to 6.027 (−0.048), DPGBench to 84.43 (−0.83), HPSv3 to 9.374 (−0.187), with PickScore and CLIPScore essentially unchanged. The HPSv3 drop of 0.187 confirms that off-trajectory supervision provides a measurable human-preference improvement beyond on-trajectory matching alone.

• Gaussian perturbation baseline: Constructing off-trajectory latents by predicting the clean image from x_{t_i}, then adding Gaussian noise to create x'_t = (1-t)x̂_0 + tε (re-noising, not velocity extrapolation) yields Aesthetic 6.040, DPGBench 84.65, HPSv3 9.516. This is intermediate between the on-trajectory-only variant (HPSv3 9.374) and the velocity-driven extrapolation (HPSv3 9.561). The finding is that any off-trajectory supervision helps (Gaussian perturbation outperforms on-trajectory-only), but velocity-driven extrapolation is specifically better — an HPSv3 gain of 0.045 over Gaussian perturbation, which is small but consistent. This validates the paper's claim that the velocity-driven construction targets the actual Euler truncation error mode rather than a generic perturbation.

Core Mechanism Design — Target Latent Construction (Table 2, right panel, row c):

• Full-trajectory x̂₀ target: Instead of anchoring the CDM loss target to the local clean estimate x̂^{(i')}_0 at the extrapolated point, using the full-trajectory final generation x̂_0 (computed by running the backward simulation all the way from x_{t_i} to t ≈ 0) yields Aesthetic 6.026, DPGBench 85.14, HPSv3 9.346. This represents a substantial HPSv3 drop of 0.215 from the full CDM, despite Aesthetic and DPGBench being close. This is a non-obvious ablation result: the local anchor provides a cleaner and more direct error signal than the full-trajectory anchor, likely because the full backward simulation from the anchor to t ≈ 0 accumulates additional integration errors that contaminate the target.

Qualitative Loss Ablation Across Training Steps (Figure 6):

Figure 6 visualizes the training dynamics of different loss combinations. The left column shows individual losses (CA, DM, CDM) in isolation at various training steps. The right column shows pairwise and full combinations. Key observations: (1) Partial combinations (e.g., CA+DM, CA+CDM) suffer from "brightness collapse or degraded local fidelity at later stages" according to the paper, while the full objective (CA+DM+CDM) preserves both global semantic coherence and local details throughout training. These are qualitative observations without quantitative metrics reported in the figure.

Quantitative Validation of DM Loss Alignment with CFG-Free Distribution (Appendix Table 4):

This is not an ablation of CDM per se, but a diagnostic experiment validating the paper's central claim about the DM loss. On SD3-Medium, the DM-only student achieves metrics very close to the CFG-free teacher: DPGBench 70.60 vs. 69.65, HPSv3 7.258 vs. 4.693 (note: the student actually exceeds the CFG-free teacher on HPSv3), CLIPScore 25.31 vs. 24.32. The CFG-enabled teacher achieves DPGBench 85.04 and HPSv3 8.189 — dramatically higher, confirming the CA loss is essential for text-alignment. On Longcat-Image, the DM-only student achieves DPGBench 80.53 vs. CFG-free teacher 84.72 (student is worse here), HPSv3 8.186 vs. 8.253 (comparable), and CLIPScore 21.69 vs. 25.63 (worse). The pattern holds: DM-only students track CFG-free teacher performance, not CFG-enabled teacher performance.

Varying Inference Steps (Appendix I, Figure 7):

Although not a formal ablation, Figure 7 demonstrates that CDM supports varying inference steps at test time from the same 4-NFE-distilled checkpoint. At 3 NFE, images are coherent but with reduced fine detail; at 4 NFE, the standard output; at 6 and 8 NFE, finer details progressively emerge. This flexibility is a consequence of the continuous-time regularization — the student's velocity field is smooth enough to support accurate integration with arbitrary step sizes, not just the training-time target of 4 steps. The paper does not provide quantitative metrics at different NFE values.

Robustness check across backbones (Table 1, SD3-Medium vs. Longcat-Image):

The consistent pattern of CDM outperforming baselines on both SD3-Medium and Longcat-Image — despite the different architecture, scale, and the use of LoRA fine-tuning on Longcat-Image vs. full fine-tuning on SD3-Medium — provides evidence of the method's robustness to backbone choice and training configuration.

Critical Assessment

The experimental results provide substantial support for CDM's core contributions but also reveal several gaps that temper the strength of the conclusions.

Claim 1: The dynamic continuous schedule improves over a fixed discrete schedule.

This claim is supported by the empirical comparison in Table 2 (right panel, row a), showing the fixed schedule variant achieves lower metrics than the full CDM with dynamic schedule (DPGBench −1.42, HPSv3 −0.079). The theoretical justification in Section 3.2 and Appendix D provides a plausible mechanism (densification of score-matching supervision across the continuous time domain), but the experiment does not isolate the two components of the dynamic schedule: the randomized simulation length N and the continuous anchor positions. An ablation that randomizes N but keeps anchors aligned to the inference grid would separate these effects. The current experiment conflates them — it is unclear whether the gains come from seeing diverse trajectory lengths, from seeing continuous timesteps, or from their combination.

Additionally, the paper does not explore whether the gains saturate — would N_max = 56 or N_max = 14 produce different results? The choice of N_max = 28 is stated without ablation. If the gains are robust to this hyperparameter, that strengthens the claim; if they are sensitive, it suggests the mechanism is more fragile than the theory implies.

Claim 2: The CDM loss (velocity-driven off-trajectory matching) outperforms on-trajectory-only and Gaussian-perturbation alternatives.

The ablation in Table 2 (right panel, rows b and c) provides direct support. Velocity-driven extrapolation (HPSv3 9.561) outperforms both on-trajectory-only (HPSv3 9.374, delta −0.187) and Gaussian perturbation (HPSv3 9.516, delta −0.045). The deltas are consistent in direction but modest in magnitude. The theoretical connection to Euler truncation error (Appendix E) provides a satisfying mechanistic explanation, but the experiment does not verify this mechanism directly — there is no measurement of the material derivative M₂, no comparison of truncation error magnitudes, and no test of whether CDM-trained models actually exhibit smaller dv_θ/dτ. The stronger version of this claim would require showing that CDM reduces the supremum of the velocity field's material derivative, which is not measured here.

A missing ablation worth noting: the paper does not test whether the CDM loss could be applied at the on-trajectory anchors themselves (i.e., using the same distribution matching formulation but without the extrapolation step). This would help isolate whether the benefit comes from the additional supervision point or from the off-trajectory property specifically.

Claim 3: Pure distribution matching without adversarial or reward-model auxiliary objectives can achieve state-of-the-art few-step quality.

Table 1 strongly supports this: CDM achieves the best HPSv3 (9.561) among all methods on SD3-Medium, and the "Image-Free" column shows CDM is the only continuous-supervision method that does not require real images, GAN training, or reward models. This is an important negative result for the "DMD needs GANs" narrative.

However, the comparison against DMD2 requires careful qualification: the paper's re-implementation of DMD2 omits the GAN-based adversarial loss that was central to the original DMD2 paper. The DMD2 results in Table 1 represent a DMD2 variant without its most important enhancement, making CDM's advantage over DMD2 potentially overstated. A fairer comparison would include DMD2 with GAN loss, or an explicit statement that CDM matches what DMD2+GAN achieves. The paper acknowledges this re-implementation choice (Appendix C.2: "notably without the GAN-based adversarial loss") but does not discuss how it affects the interpretation of the DMD2 comparison. Additionally, Flash Diffusion [1] includes a GAN loss and achieves CLIPScore 28.18 vs. CDM's 27.98 — one metric where a GAN-augmented method still leads, suggesting the "no auxiliary objectives needed" claim may be metric-dependent.

Claim 4: CDM enables the student to surpass the teacher on several metrics at 4 NFE.

This claim is well-supported by Table 1: on SD3-Medium, CDM at 4 NFE exceeds the 100-NFE teacher on Aesthetic (6.075 vs. 5.885), DPGBench (85.26 vs. 85.04), PickScore (21.95 vs. 21.73), and HPSv3 (9.561 vs. 8.189). On Longcat-Image, CDM exceeds the teacher on DPGBench (88.35 vs. 87.08), HPSv3 (10.65 vs. 9.450), and matches on Aesthetic (5.919 vs. 5.926) and CLIPScore (26.72 vs. 26.78).

The result is striking but warrants careful interpretation. The teacher operates at 100 NFE with CFG-enabled, while the student operates at 4 NFE. That the student can outperform the teacher on HPSv3 (a human preference metric) suggests that the distillation process is not purely approximating the teacher — it is learning a smoothed or regularized version that humans prefer. This is not necessarily "better" in an objective sense; it may reflect that the teacher's sampling process introduces artifacts or inconsistencies at 100 steps that CDM's continuous-time regularization suppresses. The paper does not investigate this mechanism. Additionally, the teacher-surpassing result may be specific to the evaluation prompts and metrics — HPSv3 and PickScore are trained on human preference data and may favor certain aesthetic properties that CDM's regularization incidentally promotes, rather than reflecting a genuine capability improvement.

General experimental design concerns:

• Single-run evaluations without error bars: All metrics in Tables 1, 2, and 3 are reported as single-point values. With test sets of 1K–2K prompts, metric estimates have uncertainty that can be substantial for metrics like OCR accuracy or FID. The absence of standard deviations, confidence intervals, or multi-seed results makes it impossible to assess whether differences of 0.02–0.05 in Aesthetic or PickScore are statistically reliable. The paper's HPSv3 delta of 0.385 over D-DMD is likely meaningful given the sample size, but smaller deltas (e.g., CDM vs. fixed-schedule CDM on PickScore: 21.95 vs. 21.89) may not be.

• D-DMD and DMD2 re-implementations: Since no official SD3-Medium checkpoints exist for D-DMD and DMD2, the authors re-implement them. This introduces implementation risk — subtle differences in hyperparameters, data processing, or training recipes could disadvantage the baselines. The paper does not compare its D-DMD re-implementation against any reference to validate that it matches expected performance. The DMD2 re-implementation specifically drops the GAN loss, which was a core component of the original method. These issues make the CDM vs. D-DMD and CDM vs. DMD2 comparisons less controlled than ideal.

• Limited training data scale: CDM uses 200K training prompts across 4K iterations. This is a relatively small dataset for text-to-image distillation — by comparison, D-DMD and DMD2 papers used larger training sets. The paper does not investigate whether CDM's advantages persist at larger data scales or whether the continuous regularization becomes less important when more diverse training data is available. If the dynamic schedule and CDM loss primarily function as regularization, their benefits might diminish with more data.

• No exploration of training-inference time mismatch sensitivity: The paper convincingly shows that exact training-inference schedule alignment is unnecessary, but does not explore how sensitive performance is to the distance between training and inference schedules. If a student trained with N_max = 28 works well at 4 NFE inference, does it also work at 2 NFE? At 1 NFE? Figure 7 shows qualitative results at 3, 4, 6, 8 NFE, suggesting flexibility, but there is no quantitative evaluation at inference step counts different from 4. This is a missed opportunity to demonstrate the claimed benefit of continuous-time regularization.

• Missing comparison with consistency distillation at equal NFE: The paper contrasts CDM conceptually with consistency distillation (Section 2), but does not include consistency-based methods in the quantitative comparison on SD3-Medium. Hyper-SD is the only consistency-adjacent baseline, and it operates at 8 NFE — not directly comparable. Including a 4 NFE consistency model baseline (e.g., LCM or sCM) would strengthen the claim that CDM achieves competitive or superior quality to the other major few-step paradigm.

• The 1.8× training time overhead: While inference cost is identical, the 1.8× training time overhead of CDM vs. D-DMD (Table 3) is non-trivial — approximately 43 hours vs. 24 hours on 16 A100 GPUs. The paper does not ablate whether similar gains could be achieved by simply training D-DMD for 1.8× longer (7.2K iterations instead of 4K). If extended training of D-DMD closes the gap, the claimed advantage of the continuous-time formulation weakens. This is arguably the most important missing ablation — without it, the efficiency-adjusted benefit of CDM's training modifications remains unclear.

• Single GPU inference measurement: The inference latency of 246 ms/img is reported on a single GPU without specifying the GPU model. Since inference latency is hardware-dependent, this number is only meaningful for relative comparisons (which remain valid since all methods were measured on the same hardware).

What the experiments do and do not demonstrate:

The experiments convincingly demonstrate that CDM achieves strong quantitative and qualitative results on two modern text-to-image backbones, outperforming the baselines the paper selected. They demonstrate that the dynamic schedule and CDM loss each contribute measurably to final performance. They demonstrate that the DM loss faithfully captures the teacher's CFG-free distribution.

What the experiments do not demonstrate — but would be needed for stronger versions of the paper's claims — is: (a) that CDM's velocity field actually has lower material derivative M₂ than discrete-schedule variants (the mechanism is theoretically motivated but not empirically verified); (b) that CDM's advantages persist at scale (more data, larger models, other architectures); (c) that CDM's training overhead is justified compared to simply training a discrete-schedule baseline longer; (d) that the teacher-surpassing results reflect a genuine capability improvement rather than metric-specific preference for CDM's aesthetic properties; and (e) that the gains are statistically robust across multiple random seeds and data splits.

6. Limitations and Trade-offs

Limitation 1: Difficulty Estimation Cost Is Not Accounted for in the Headline Efficiency Gains

The assumption or constraint. The paper's dynamic continuous schedule samples the backward simulation length N ∼ U{1, N_max} at each training iteration, where N_max = 28 for SD3-Medium experiments (Section 3.2). This means the average backward simulation length during training is ~14.5 steps, compared to the fixed 4 steps used in prior DMD methods. Additionally, the CDM loss requires an extra forward pass through both the real teacher D_ϕ and fake teacher D_ψ on the extrapolated latent x_{t'_i} (Section 3.3, Algorithm 1). The paper acknowledges this overhead explicitly in Appendix A:

"the dynamic schedule samples a variable simulation length N ∼ U{1, Nmax} that prolongs the average backward simulation, and the CDM loss requires an extra forward pass of the real and fake teachers on the extrapolated off-trajectory latent"

The consequence. CDM requires approximately 1.8× the training time of D-DMD (Table 3) — roughly 43 hours vs. 24 hours on 16 A100 GPUs for the SD3-Medium experiments. This overhead is not amortized in any of the headline comparisons. The paper's central claim is that continuous-time distribution matching improves few-step generation quality, but the experiments do not establish whether these improvements are attributable to the continuous-time formulation specifically or simply to the increased effective training budget. A practitioner evaluating CDM against D-DMD needs to know whether training D-DMD for 1.8× longer (7.2K iterations instead of 4K) would close the performance gap. The paper also does not ablate whether N_max = 28 is optimal — smaller values would reduce training cost but might reduce performance. The relationship between training budget (iterations × simulation length) and final quality is unexplored.

What evidence exists in the paper. Table 3 provides the relative training time comparison: D-DMD at 1.0×, CDM with fixed schedule at 1.1×, and full CDM at 1.8×. The fixed-schedule CDM variant (1.1× training time) achieves HPSv3 9.482 vs. full CDM's 9.561 (Table 2, right panel, row a), suggesting that roughly half the HPSv3 gain (+0.079 out of +0.385 relative to D-DMD's 9.176) comes from the CDM loss and the other half from the dynamic schedule. But no experiment controls for total training FLOPs — there is no "D-DMD trained for 1.8× longer" or "CDM with N_max = 14 trained for the same wall-clock time" baseline. This is arguably the most consequential missing ablation in the paper.

Mitigation status. The paper acknowledges the training overhead in Appendix A and the main text (Section 4.3, Table 3 discussion), but treats it as an accepted cost rather than a limitation to be addressed. There is no proposal for reducing the overhead, no investigation of whether the dynamic schedule length can be reduced without quality loss, and no trained difficulty predictor or adaptive scheme that would reduce the average simulation length. The paper suggests no future work on this topic. The inference cost is correctly noted as identical to baselines (246 ms/image), but the training cost asymmetry is not resolved.

---

Limitation 2: The Performance Ceiling Is Bounded by the Teacher's CFG-Free Distribution

The assumption or constraint. The paper demonstrates (Figure 3, Appendix Table 4) that the distribution matching (DM) loss faithfully captures the teacher's CFG-free marginal distribution — not the full CFG-enabled distribution that produces high-quality, prompt-aligned outputs. The DM loss gradient, derived in Appendix D (Equation 25), minimizes the reverse KL divergence between the student's distribution and the teacher's conditional score ∇log p_real(z|c), but this score is evaluated through the teacher's conditional prediction D_ϕ(z, τ, c). The paper's diagnostic experiment shows that when only the DM loss is used (no CA loss), the student converges to the CFG-free teacher's outputs, not the CFG-enabled outputs.

The consequence. The DM and CDM losses — which together provide the distribution matching signal — are fundamentally constrained in what they can achieve: they can at best align the student to the teacher's unguided data manifold. The text-alignment and prompt-adherence signal comes entirely from the CA loss (Equation 3), which uses the teacher's CFG direction Δ_ca = α(D_ϕ(z_τ, τ, c) - D_ϕ(z_τ, τ, ∅)). This means that concepts or visual compositions that the teacher itself handles poorly without CFG cannot be recovered through DM or CDM losses alone, regardless of how many continuous timesteps or off-trajectory points are supervised. The paper acknowledges this in Appendix A:

"the DM and CDM losses both rely on the teacher's score as the supervision signal, so concepts or compositions that the teacher itself handles poorly are unlikely to be recovered through distillation alone, as suggested by the CFG-free analysis in Figure 3"

This is a fundamental upper bound: CDM can redistribute the teacher's knowledge across the continuous time domain (improving integration accuracy and recovering details), but it cannot create new capabilities that the teacher lacks. If the teacher produces blurry textures or incorrect object relationships in its CFG-free mode, the distilled student will inherit those limitations.

What evidence exists in the paper. Appendix Table 4 quantifies this ceiling directly. On SD3-Medium, the CFG-free teacher achieves DPGBench 69.65 and HPSv3 4.693, while the CFG-enabled teacher achieves 85.04 and 8.189. The DM-only student tracks the CFG-free teacher (DPGBench 70.60, HPSv3 7.258). The full CDM with CA loss achieves DPGBench 85.26 and HPSv3 9.561 — exceeding even the CFG-enabled teacher on these metrics. This suggests the CA loss + continuous distribution matching can amplify the teacher's CFG signal beyond what the teacher achieves with its own CFG, but the base capability must exist in the teacher's CFG-free distribution for the DM/CDM losses to have anything to anchor to. The paper does not ablate what happens when the teacher's CFG-free quality is artificially degraded (e.g., by using a weaker teacher) or when certain visual concepts are absent from the teacher's training data — the boundary of "what the teacher knows" is not probed.

Mitigation status. The paper is transparent about this limitation in Appendix A. It acknowledges that CDM is "fundamentally upper-bounded by the teacher" and that concepts the teacher handles poorly are "unlikely to be recovered." However, it does not characterize where this bound becomes active in practice — there is no difficulty-based analysis showing which types of prompts benefit most from the CA loss vs. the DM/CDM losses, no ablation studying the interaction between CA guidance scale α and the CFG-free ceiling, and no exploration of whether multiple teachers or ensemble strategies could raise the ceiling. The limitation is stated but not systematically investigated.

---

Limitation 3: Single Architecture Family and Task Domain — No Evidence of Generalization Beyond Text-to-Image Flow Matching

The assumption or constraint. All experiments in the paper use flow-matching transformer backbones for text-to-image generation: SD3-Medium and Longcat-Image, both at 1024×1024 resolution (Section 4.1). The evaluation is restricted to standard text-to-image benchmarks (PickScore prompts, DPG-Bench, COCO 2014). The method's design choices — the flow-matching parameterization D_θ(x_t, t, c) = x_t - t v_θ(x_t, t, c), the velocity-driven extrapolation, the Tweedie's formula derivations in Appendix D — are all specialized to the flow-matching interpolation x_t = (1-t)x_0 + tε. The paper acknowledges this scope limitation in Appendix A:

"our empirical study is restricted to text-to-image generation such as SD3-Medium and Longcat-Image; we will explore extending CDM to text-and-image-to-image (TI2I) editing and to video diffusion models"

The consequence. It is unclear whether CDM's benefits transfer to:

• Non-flow-matching formulations: Standard diffusion models (DDPM, DDIM, score-based SDEs) use different forward processes (e.g., variance-preserving x_t = √(ᾱ_t) x_0 + √(1-ᾱ_t) ε), which would require reformulating the dynamic schedule, the CDM loss's velocity-driven extrapolation, and the Tweedie's formula derivations. The velocity field v_θ has no direct analogue in standard diffusion; a score network s_θ or noise predictor ε_θ would need to be adapted.
• Other modalities: Video generation introduces temporal consistency constraints and longer inference trajectories, where the CDM loss's off-trajectory extrapolation might interact differently with the temporal dimension. Text-and-image-to-image editing introduces a conditioning image alongside text, changing the distribution matching target.
• Non-transformer architectures: U-Net-based diffusion models (Stable Diffusion 1.x/2.x, earlier DALL-E) have different inductive biases and might respond differently to continuous-time velocity field regularization.

The paper's theoretical framework (Appendix D, E) is general — the score-matching perspective and truncation error analysis apply to any probability flow ODE — but the empirical validation is narrow. This is particularly relevant because the prior DMD work that CDM builds on (DMD, DMD2, D-DMD) was validated across multiple architectures and tasks, establishing broader generalizability. CDM has not yet demonstrated that its specific innovations (dynamic schedule, velocity-driven extrapolation) provide consistent benefits outside the flow-matching transformer + text-to-image setting.

What evidence exists in the paper. The two-backbone comparison (SD3-Medium and Longcat-Image) provides some evidence of cross-architecture generalization — both are flow-matching transformers but differ in scale and training details. CDM achieves top HPSv3 on both (9.561 and 10.65, respectively). However, both backbones share the same fundamental formulation (flow-matching, transformer, text-to-image), making this a within-paradigm replication rather than a cross-paradigm test. The paper provides no evidence on: diffusion (non-flow) models, U-Net architectures, video generation, image editing, class-conditional generation, unconditional generation, or other modalities. The Appendix A discussion flags these as future work but provides no preliminary results.

Mitigation status. The paper explicitly scopes the limitation to text-to-image and lists future extensions (TI2I editing, video models) in Appendix A. This is appropriate for a conference paper introducing a new method. However, the strong claims in the abstract and introduction ("our continuous paradigm yields significant performance gains, establishing new state-of-the-art results for few-step image generation across different models") should be interpreted as "across the two flow-matching transformer models tested," not "across model families in general." The limitation is stated but not bounded — the paper does not discuss which aspects of the method are likely to transfer (the continuous-time principle) and which are flow-matching-specific (the velocity field parameterization, the exact form of the CDM loss).

---

Limitation 4: The Velocity-Driven Extrapolation Mechanism Is Theoretically Motivated but Not Directly Validated

The assumption or constraint. The CDM loss (Section 3.3) is built on the theoretical claim that velocity-driven extrapolation x_{t'_i} = x_{t_i} + (t'_i - t_i) v_{t_i} constructs off-trajectory points that specifically target the Euler truncation error mode, and that enforcing distribution matching at these points reduces the material derivative M_2 = sup ‖dv_θ/dτ‖ that controls integration error (Appendix E). This is the paper's central mechanistic claim — it is what distinguishes CDM from generic off-trajectory augmentation (e.g., Gaussian re-noising) and what justifies the specific form of the CDM loss.

The consequence. The paper never directly measures whether CDM actually reduces M_2. There is no evaluation of the student's velocity field smoothness, no comparison of truncation error magnitudes between CDM-trained and baseline models, and no test of whether the velocity field's material derivative empirically correlates with generation quality. The ablation in Table 2 (right panel, row b) shows that velocity-driven extrapolation (HPSv3 9.561) outperforms Gaussian perturbation (HPSv3 9.516, delta −0.045) and on-trajectory-only matching (HPSv3 9.374, delta −0.187). While this demonstrates that the specific CDM construction works better than alternatives, it does not verify why — there could be other explanations for the advantage beyond the truncation error mechanism.

Alternative explanations for the CDM loss's benefit that the paper does not rule out include:

• Additional training data: The CDM loss simply provides more supervision points per iteration (one extra point per anchor), and any reasonable augmentation of the training distribution (Gaussian, velocity-driven, or otherwise) would help. The performance ordering (velocity-driven > Gaussian > on-trajectory) might reflect how "in-distribution" the augmentation is relative to the student's actual inference-time error, without necessarily involving the material derivative.
• Implicit gradient regularization: The CDM loss computes gradients through the student's velocity prediction at the extrapolated point, which involves the Jacobian ∂v_θ/∂θ evaluated at a different point than the DM loss. This might provide beneficial gradient diversity that stabilizes training, independent of truncation error.
• Local consistency constraint: The CDM loss enforces that the student's prediction at x_{t'_i} is consistent with the teacher's distribution, which is a local smoothness constraint but may operate through a different mechanism than M_2 suppression.

Without direct measurement of the velocity field properties that the theory predicts CDM should improve, the paper's central mechanistic story remains a plausible hypothesis rather than an empirically verified causal chain.

What evidence exists in the paper. The theory in Appendix E derives the truncation error bound and shows how CDM's finite difference (v_θ(x_{t'_i}, t'_i) - v_θ(x_{t_i}, t_i))/Δt approximates dv_θ/dτ. The ablation in Table 2 compares velocity-driven vs. Gaussian vs. on-trajectory variants. The HPSv3 deltas are consistent with the theory (velocity-driven > Gaussian > on-trajectory), but the differences are small (0.045 and 0.187, respectively) and are reported without error bars. There is no measurement of: the material derivative M_2 for CDM vs. baseline models, the actual truncation error at inference (difference between true ODE solution and Euler steps), the correlation between M_2 and generation quality metrics, or the spectral properties of the velocity field (e.g., Lipschitz constant, frequency content). The paper also does not test a negative control: if the CDM loss works by suppressing M_2, then a model trained with CDM should exhibit better integration accuracy at arbitrary step sizes, not just the 4-step target. Figure 7 shows qualitative results at varying NFE, but there is no quantitative evaluation of integration error (e.g., measuring the distance between 4-step and 100-step trajectories from the same starting noise, or comparing outputs at different NFE using the same seed).

Mitigation status. The paper does not acknowledge this as a limitation. The theory is presented as explanatory rather than as a hypothesis to be tested. The ablation demonstrates that velocity-driven extrapolation works better than alternatives, which is sufficient to validate the design choice, but insufficient to validate the claimed mechanism. A thorough mechanistic validation would require dedicated experiments measuring the proposed intermediate variables and showing they mediate the observed quality improvements — this is left entirely to future work.

---

Limitation 5: No Statistical Rigor — Single-Run Results Without Confidence Intervals or Multi-Seed Validation

The assumption or constraint. All quantitative results in the paper (Tables 1, 2, 3; Figures 2, 3, 7; Appendix Table 3, 4) are reported as single-point estimates from single training runs on fixed test splits. The paper does not report standard deviations, confidence intervals, error bars, or results from multiple random seeds. The evaluation datasets range from 1K to 10K prompts, but metric variance given these sample sizes is not discussed.

The consequence. Several of the paper's claimed advantages fall within ranges where statistical reliability is unclear:

• Aesthetic score differences between CDM and D-DMD are 0.037 on SD3-Medium (6.075 vs. 6.038) and 0.137 on Longcat-Image (5.919 vs. 5.782). The paper does not provide the standard deviation of Aesthetic scores on the 2K-prompt PickScore test split, so it is impossible to know whether deltas of this magnitude are statistically significant or within the noise of the evaluation metric and sampling.
• PickScore differences are 0.10 on SD3-Medium (21.95 vs. 21.85) and 0.30 on Longcat-Image (21.53 vs. 21.23). Again, the absence of variance information prevents significance assessment.
• The CDM loss ablation (Table 2, right panel) shows HPSv3 deltas of 0.045 between velocity-driven and Gaussian perturbation — a very small margin that could easily fall within single-run variance.
• OCR accuracy (Appendix Table 3) shows CDM at 34.82 vs. fixed-schedule CDM at 37.33, a delta of 2.51 percentage points on a 1K-prompt test set. Without knowing the standard error of OCR accuracy estimates, it is unclear whether this difference is meaningful or reflects prompt-sampling noise.

For a paper whose central claim is that continuous-time distribution matching provides measurable quality improvements over discrete-time baselines, the absence of statistical validation weakens the quantitative evidence. The paper's strongest results (HPSv3 gains of 0.385 over D-DMD on SD3-Medium and 1.021 over D-DMD on Longcat-Image) are likely robust given their magnitude, but many of the supporting ablation results and metric-specific comparisons rest on smaller deltas that may not replicate.

Additionally, single-run results are vulnerable to training stochasticity — different random seeds can produce different local optima, and the best-of-seed performance may overstate expected performance. The paper does not report whether the reported results are from a single seed, the best of multiple seeds, or an average. Given the 1.8× training time of CDM (Limitation 1), running multiple seeds would be expensive, but even reporting the standard deviation across the 4K training iterations' final metrics or providing bootstrap confidence intervals for the test-set metrics would substantially strengthen the quantitative claims.

What evidence exists in the paper. None. The paper provides zero statistical information beyond point estimates. This is an across-the-board omission, not specific to any single result.

Mitigation status. The paper does not acknowledge this as a limitation. There is no discussion of metric variance, sample size adequacy, or statistical testing methodology. The evaluation protocol is described in Section 4.1 and Appendix C, but the description covers only dataset composition and metric definitions, not statistical methodology. For a paper targeting top-tier publication, the absence of any variance reporting or significance testing is a notable weakness, especially given that multiple claimed advantages depend on small metric deltas.

---

Limitation 6: The DM-Only Diagnostic Reveals a Fundamental Tradeoff Between Distribution Matching Fidelity and Text Alignment That Remains Unresolved

The assumption or constraint. The paper's diagnostic experiment (Figure 3, Appendix Table 4) conclusively demonstrates that the DM loss captures the teacher's CFG-free distribution, and that the CFG-enabled distribution — which is substantially better on all text-alignment metrics — is provided entirely by the CA loss. This establishes a clean division of labor: DM/CDM losses anchor the student to the data manifold, and the CA loss pushes it toward prompt-consistent regions of that manifold. However, the CA loss in Equation (3) operates on the on-trajectory anchor x_{t_i} at the student's own timestep t_i, while the teacher's CFG direction Δ_ca = α(D_ϕ(z_τ, τ, c) - D_ϕ(z_τ, τ, ∅)) is evaluated at the teacher's perturbation timestep τ. These timesteps are independent under the dynamic schedule — they are drawn separately from U(0, 1].

The consequence. There is a temporal mismatch in the CA loss: the student's prediction D_θ(x_{t_i}, t_i, c) is evaluated at the student's anchor timestep t_i, but the CFG direction that corrects it is evaluated at the teacher's perturbation timestep τ, which may be far from t_i. If t_i and τ are very different (e.g., t_i = 0.1 and τ = 0.9), the CA loss pushes the student in a direction that is appropriate for a heavily noised latent but is applied to a nearly clean latent. The dynamic weight w_τ partially compensates by normalizing the gradient magnitude, but it cannot correct for directional mismatch — the implicit classifier ∇_{z_τ} log p_real(c|z_τ) points in different directions at different noise levels because the relationship between text and image features changes with the noise scale.

This temporal mismatch means that the CA loss — which provides all text-alignment signal — is operating under a fundamentally different noise-level distribution than the student's actual inference trajectory. The DM and CDM losses receive dense continuous-time supervision (because t_i, τ̃, and τ̂ are all drawn from U(0, 1]), but the CA loss receives guidance from arbitrary τ values that may not correspond well to the student's current t_i. This could explain why CDM's DPGBench (85.26) and CLIPScore (27.98) — the text-alignment metrics — show smaller improvements relative to baselines than HPSv3 (9.561) and Aesthetic (6.075) — the visual quality metrics. The continuous-time extensions primarily improve the distribution matching (DM/CDM) side, leaving the CA loss's temporal mismatch unaddressed.

What evidence exists in the paper. The paper does not explicitly identify this as a limitation, but the evidence is present in the data. In the single-loss ablation (Table 2, left panel), removing L_CA causes DPGBench to drop from 85.26 to 72.87 (−12.39) and CLIPScore from 27.98 to 24.78 (−3.20), confirming the CA loss is the sole text-alignment driver. The fixed-schedule variant (Table 2, right panel) shows DPGBench 83.84 vs. full CDM's 85.26 (−1.42) — the largest metric drop from schedule decoupling — suggesting the temporal relationship between t_i and CFG supervision matters specifically for text-alignment. Yet the paper never ablates whether applying the CA loss at the same τ = t_i (matching the student's anchor timestep to the teacher's perturbation timestep) would improve text-alignment at the cost of some visual quality, or whether the dynamic schedule's t_i ∼ U(0, 1] creates beneficial or harmful temporal mismatch for the CA gradient.

Mitigation status. The paper does not acknowledge this temporal mismatch as a limitation. The CA loss formulation in Equation (3) directly follows D-DMD, which also samples τ independently of t_i. The dynamic schedule makes this independence more consequential (because t_i now varies continuously rather than being fixed to a few anchors), but the paper does not investigate whether adapting the CA loss's timestep sampling to align with the student's anchor would improve text-alignment. This represents an unresolved tradeoff: the continuous-time formulation benefits the DM/CDM losses (which receive denser manifold-anchoring) but may dilute the CA loss (which receives guidance from temporally mismatched CFG directions). The paper's future work suggestions (Section 5, Appendix A) do not mention this issue.

7. Implications and Future Directions

How This Work Changes the Landscape

CDM introduces a methodological reframing rather than a paradigm shift: it does not replace distribution matching distillation with a new framework, but it fundamentally changes what practitioners should consider a constraint versus a design choice within the existing DMD paradigm. The paper's empirical finding that exact training-inference schedule alignment is not merely unnecessary but actively harmful (Figure 2) overturns a foundational assumption that has guided DMD research since its inception — the belief that backward simulation must mirror the inference schedule to avoid train-test mismatch. This is a diagnostic contribution with broad implications: any distillation method that currently enforces schedule coupling (progressive distillation, trajectory matching, consistency distillation's boundary conditions) should re-examine this constraint as a potential source of performance degradation. The theoretical justification in Appendix D — showing that the student's anchor timestep t_i and the teacher's perturbation timesteps τ, τ̃ are independent variables whose product determines effective supervision density — provides the principled explanation for why schedule decoupling works, converting an empirical curiosity into a design principle.

The paper's second landscape-changing contribution is demonstrating that pure distribution matching can match or exceed hybrid methods at scale, challenging the prevailing narrative that DMD inherently requires adversarial or reward-model auxiliary objectives to recover sharp textures and fine details. On SD3-Medium at 4 NFE, CDM achieves HPSv3 9.561 — surpassing D-DMD (9.176), the GAN-augmented Flash method (8.282), and approaching levels previously thought to require adversarial training. This result reframes the community's understanding of the "adversarial gap": it was not a fundamental limitation of reverse-KL-based distribution matching, but rather an artifact of the sparse, discrete-time, on-trajectory-only formulation that prior DMD variants employed. This shifts the burden of proof onto auxiliary objectives to demonstrate that their benefits justify their costs (training instability, hyperparameter sensitivity, computational overhead), now that a pure distribution matching baseline exists that is competitive without them. The implication is not that GANs and reward models are obsolete — they may still provide benefits in regimes CDM has not explored — but that the dominant research direction of "DMD + auxiliary objective" should be reconsidered in favor of "DMD with denser, better-structured supervision."

A third contribution is the reframing of few-step generation as a numerical integration problem, not purely a statistical matching problem. CDM connects distribution matching to classical numerical analysis through its central mechanistic claim: the CDM loss's velocity-driven extrapolation targets the material derivative M₂ = sup ‖dv_θ/dτ‖ that controls Euler truncation error (Appendix E), and the dynamic schedule provides dense supervision that implicitly smooths the velocity field across the continuous time domain. Prior distillation work treated few-step generation as a statistical estimation problem — can the student learn to match the teacher's output distribution with fewer sampling steps? CDM's perspective suggests that the more productive framing is: can the student learn a velocity field whose numerical integration properties (bounded material derivative, Lipschitz continuity, smooth interpolation between anchors) support large step sizes? This shift directs research attention toward velocity field regularization and truncation error analysis — areas that the distillation community has largely ignored — rather than toward more complex loss formulations or auxiliary modules.

Finally, the paper resolves the puzzling performance gap between DMD and consistency distillation by identifying its root cause without adopting consistency's framework. Consistency models [28, 30, 49] naturally enforce trajectory smoothness through their self-consistency objective, which operates over arbitrary time pairs in (0, 1] — providing exactly the continuous regularization that discrete DMD lacked. CDM achieves analogous regularization through a different mechanism (dynamic schedule + off-trajectory matching), demonstrating that the benefits of continuous-time supervision are not tied to the self-consistency property. This suggests that the conceptual boundary between DMD and consistency distillation is more porous than previously thought: both can achieve continuous-time regularization; they differ in whether that regularization is achieved through self-consistency constraints (consistency) or through densified distribution matching with off-trajectory extrapolation (CDM). This opens the door to hybrid methods that combine the strengths of both paradigms — an opportunity the paper explicitly acknowledges (Appendix A, Section 5).

Follow-Up Research This Work Enables

Direct measurement of velocity field smoothness and truncation error. The paper's central mechanistic claim — that CDM suppresses the material derivative M₂ = sup ‖dv_θ/dτ‖, thereby reducing Euler truncation error — is theoretically motivated but not empirically verified. A strong follow-up would instrument the training process to measure: (a) the supremum of the material derivative along integration trajectories for CDM-trained vs. D-DMD-trained models, computed via finite differences of the velocity field along the simulated ODE path; (b) the actual truncation error at inference, measured as the L2 distance between the 4-step Euler trajectory and a reference 100-step trajectory from the same initial noise; (c) the correlation between M₂ and generation quality metrics (HPSv3, FID) across training checkpoints. If M₂ decreases during CDM training and this decrease mediates the observed quality improvements, the mechanistic story is validated. If M₂ remains constant or uncorrelated with quality, then CDM's benefits arise through a different mechanism (e.g., implicit gradient regularization, additional training signal diversity), which would redirect theoretical understanding and potentially simplify the method (e.g., a generic data augmentation scheme might suffice).

Controlling for total training FLOPs: CDM vs. D-DMD under equalized training budgets. The paper's 1.8× training time overhead (Table 3) is the most consequential uncontrolled variable in the experimental design. A follow-up study should train three models: (a) D-DMD at 4K iterations (standard), (b) D-DMD at ~7.2K iterations (1.8× longer, matching CDM's training FLOPs), and (c) CDM with N_max reduced to maintain equal training FLOPs (e.g., N_max = 15 instead of 28, if average simulation length is the dominant cost). If D-DMD at 1.8× iterations closes the performance gap with CDM, then the continuous-time formulation's advantage is primarily one of sample efficiency, not a fundamental quality ceiling — a practitioner could achieve similar results by simply training D-DMD longer, avoiding CDM's implementation complexity. If CDM with reduced N_max matches full CDM's performance, the method becomes substantially more practical (shorter training, lower cost). The study should also profile the cost of individual components: backward simulation vs. CDM loss forward pass vs. fake teacher updates, to identify which overhead is dominant and where optimization efforts should focus.

Extending CDM to video diffusion models to test temporal consistency effects. The paper flattens video generation as a future direction (Appendix A), but it represents a particularly interesting stress-test for CDM's mechanisms. Video generation involves longer inference trajectories (typically 16-50 steps for few-step video models) and introduces temporal consistency constraints across frames. The CDM loss's velocity-driven extrapolation currently operates along the time (noise-level) dimension of the probability flow ODE; in video, there is a second "time" — the temporal frame index — that the velocity field must respect. A follow-up would need to answer: does off-trajectory extrapolation in noise-level space improve or degrade temporal consistency? If the CDM loss suppresses dv_θ/dτ along the noise-level dimension but leaves frame-to-frame velocity variations unregularized, video outputs might exhibit spatial quality improvements at the cost of temporal flickering. A natural extension would be a spatiotemporal CDM loss that extrapolates off-trajectory in both noise-level and frame-index space, potentially requiring two separate velocity predictions. The experiment would measure FVD (Fréchet Video Distance) and temporal consistency metrics alongside standard frame-level quality metrics, comparing CDM against D-DMD on a video backbone like CogVideoX or Stable Video Diffusion. If CDM improves spatial quality but degrades temporal consistency, that reveals a fundamental tension between spatial and temporal regularization that the current formulation does not address.

Training a lightweight difficulty predictor to reduce or eliminate the CDM loss overhead. The CDM loss's extra real-teacher and fake-teacher forward pass on the extrapolated latent x_{t'_i} adds roughly 0.7× to the training time (since the fixed-schedule CDM variant at 1.1× training time does not include this cost, and full CDM at 1.8× includes both the dynamic schedule and the CDM loss). Not every anchor and extrapolation are equally informative: if the velocity field is already smooth between certain anchors, the CDM loss on those points provides minimal gradient signal. A follow-up could train a lightweight classifier (e.g., a small MLP taking the anchor t_i and the predicted velocity magnitude ‖v_{t_i}‖ as input) to predict whether applying the CDM loss at that iteration will reduce the validation loss by more than some threshold. The classifier would be trained online using the observed loss reduction from previous iterations. If the predictor can identify the ~30-50% of iterations where the CDM loss is most informative, training cost could be substantially reduced without quality loss. This connects to the exploration-exploitation framing the paper briefly mentions for difficulty estimation (Section 1): the system should estimate not just question difficulty, but also where in the time domain additional supervision is most needed. The experiment would measure the Pareto frontier between training FLOPs and final HPSv3, comparing adaptive CDM loss application against full CDM and the no-CDM-loss baseline.

Systematic study of CDM across ODE solver families and step-size schedules. CDM is evaluated exclusively with explicit Euler integration and uniform step-size schedules (Section 4). However, the truncation error bound in Appendix E applies to any explicit Runge-Kutta method, and higher-order solvers (Heun's method, RK4) might interact differently with CDM's velocity field regularization. A follow-up would train CDM-distilled models and evaluate them at inference with: (a) Euler (the paper's default), (b) Heun's method (a second-order predictor-corrector), (c) DDIM-style non-uniform schedules (denser steps near t = 0 where the velocity field is typically steeper), and (d) adaptive step-size control (dense steps where ‖dv_θ/dτ‖ is large, sparse where it is small). The hypothesis is that CDM's velocity field smoothing reduces the advantage of higher-order solvers over Euler — if M₂ is already small, the O(h²) error term is small, and Heun's method's O(h³) local error provides diminishing returns. If this hypothesis holds, CDM-distilled models should show a smaller accuracy gap between Euler and Heun's method than D-DMD-distilled models. If it does not hold — if CDM and D-DMD benefit similarly from higher-order solvers — then CDM's regularization is not primarily operating through M₂ suppression, challenging the mechanistic interpretation. The experiment would also reveal whether CDM's "supports arbitrary NFE" property (Appendix I, Figure 7) extends to arbitrary solver families, which would substantially increase the method's practical utility for deployment scenarios where inference-time compute can be dialed up or down.

CDM as a teacher for subsequent distillation or self-improvement. The paper observes that CDM-distilled students can surpass their 100-NFE teachers on HPSv3 and Aesthetic (Table 1) — the 4-step student achieves higher human preference scores than the model it was distilled from. This raises an intriguing possibility: can CDM-distilled students serve as teachers for a second round of distillation, creating a self-improvement loop where each generation of student models captures a higher-quality distribution? A follow-up would train a CDM student from the original teacher, then use that student as a new "teacher" to distill a second student (potentially with the same or lower NFE), measuring whether quality improves, plateaus, or degrades across distillation generations. The DM/CDM losses would need to be adapted since the "real teacher" D_ϕ is now the previous-generation student, not the original pretrained model — the score estimates will be noisier, and the CFG-free distribution ceiling will shift. If quality improves over multiple generations, this enables a self-improvement pipeline for diffusion models analogous to STaR or ReST for language models. If quality plateaus or degrades (as the paper found with the ReST^EM revision model attempt in Appendix K of the prior DMD work), that would reveal a fundamental limit on iterative distillation and suggest that the teacher-surpassing phenomenon in CDM is a one-time benefit from the continuous-time regularization rather than an ongoing capability amplification. The experiment would need to carefully control for the possibility that metrics like HPSv3 and Aesthetic score reward specific aesthetic properties (smoothness, sharpness in certain frequency bands) that CDM optimizes for, rather than measuring genuine generation capability, by including diverse human evaluation beyond the automated metrics.

Practical Applications and Downstream Use Cases

Cost-efficient on-device deployment of high-quality text-to-image generation. The paper demonstrates that a 4-step CDM-distilled model can produce images that match or surpass a 100-step teacher on human preference metrics (HPSv3 9.561 vs. 8.189 on SD3-Medium; 10.65 vs. 9.450 on Longcat-Image). With identical inference latency to other 4-step methods (246 ms/image, Table 3) and no additional inference-time components (no GAN discriminator, no reward model, no extra network evaluations), CDM-distilled models represent a 25× speedup over the teacher with quality that is not merely preserved but improved on several dimensions. For mobile and edge deployment scenarios where GPU memory and compute are limited, this enables high-quality 1024×1024 image generation at interactive speeds without requiring cloud offloading. The 4-step inference is fast enough for real-time creative tools (iterative prompt refinement, image editing with live preview), and the continuous-time training means the same checkpoint supports 3-8 NFE (Figure 7) — a user could choose faster generation (3 steps, ~185 ms) or higher quality (8 steps, ~492 ms) from the same model without retraining or switching checkpoints. The paper's "Image-Free" and no-auxiliary-objective properties mean the distilled model requires no additional models or data beyond the student's weights, simplifying the deployment pipeline compared to GAN-augmented alternatives like Flash or DMD2.

Training data generation for self-improvement and synthetic data pipelines. Organizations using diffusion models to generate training data for downstream models (e.g., generating synthetic images for classification, detection, or segmentation training; bootstrapping text-to-image training data through rejection sampling) face a tradeoff: higher sampling steps produce better quality but at higher cost. CDM-distilled models at 4 NFE produce quality that matches or exceeds 100-NFE teachers on key metrics, enabling roughly 25× more synthetic data generation within the same compute budget. Since CDM achieves this without requiring real images during distillation (Table 1, Image-Free column), the distilled model does not inherit biases or privacy concerns from a real-image training set — it captures only what the teacher knows. For applications like medical imaging or industrial inspection where real training data is scarce and expensive, CDM-distilled teachers could generate large volumes of high-quality synthetic data at a fraction of the cost of running the full teacher. The paper's findings on DM loss alignment (Figure 3) are relevant here: the DM and CDM losses anchor the student to the teacher's CFG-free distribution, which represents the teacher's "generic image prior" without text guidance. For unconditional or weakly-conditional synthetic data generation (e.g., generating diverse images of a category without specific prompt engineering), the CFG-free distribution may actually be preferable — it produces more diverse outputs that better cover the teacher's learned manifold, avoiding the mode-seeking behavior of CFG-guided generation.

Interactive creative tools with quality-adaptive inference. The paper's continuous-time regularization produces velocity fields that support accurate integration with arbitrary step sizes (Appendix I, Figure 7; implied by the dynamic schedule training that exposes the student to simulation lengths from 1 to 28). This enables a practical deployment pattern not possible with discrete-schedule methods: a creative tool could begin generating an image at low NFE (2-3 steps, ~120-185 ms) to provide near-instant visual feedback as the user iterates on prompts, then progressively refine the image by running additional Euler steps (4, 6, 8) from the same initial noise without restarting. Because CDM's velocity field is smooth across the continuous time domain, additional steps genuinely improve quality rather than introducing artifacts from a change in the sampling procedure — Figure 7 shows coherent outputs at all NFE levels from the same seed. This "progressive reveal" capability is particularly valuable for interactive applications where users want to quickly explore the prompt space before committing to a high-quality final render. Competing methods that are optimized for a specific discrete schedule may not support this — changing the number of steps changes the integration grid, and the student may have only learned to be accurate at those specific anchors rather than across the continuous interval.

When to Prefer This Method

The paper positions CDM explicitly against D-DMD and DMD2 (both re-implemented without GAN auxiliary losses) as the primary distribution matching alternatives, and implicitly against consistency distillation as the other major few-step paradigm. The decision framework that emerges from the paper's results and acknowledged limitations is:

• Prefer CDM over discrete-schedule DMD variants (DMD, DMD2, D-DMD) when: the deployment target is 4 or fewer NFE (the regime where truncation error from large Euler steps is most severe, and where CDM's velocity field regularization provides the largest benefit), and training compute budget is sufficient to absorb the 1.8× per-iteration overhead (or the fixed-schedule CDM variant's 1.1× overhead is acceptable for a portion of the gains). The paper's results show consistent improvements across all metrics at 4 NFE on both SD3-Medium and Longcat-Image, with the largest gains on HPSv3 (+0.385 over D-DMD on SD3-Medium, +1.021 on Longcat-Image) and DPGBench (+0.74 on SD3-Medium). The absence of GAN or reward-model auxiliary objectives simplifies the training pipeline and removes sources of instability, making CDM the more practical choice when adversarial training is undesirable (e.g., in resource-constrained or reproducibility-focused settings).

• Prefer CDM over consistency distillation methods when: direct access to the teacher's score function is available and considered valuable (as in DMD's core formulation — the teacher provides its full score ∇log p_real at arbitrary noise levels, which consistency methods do not leverage). The paper demonstrates that CDM achieves state-of-the-art image-free distillation without the self-consistency constraint, suggesting that the teacher's score is a sufficient supervision signal when applied densely enough. However, if the target is 1-2 NFE (single-step generation), consistency methods likely still hold an advantage — they are specifically designed for the extreme few-step regime, and CDM's dynamic schedule with N_max = 28 and CDM loss may not provide sufficient regularization when the integration step size approaches Δt ≈ 1. The paper does not evaluate CDM below 3 NFE, leaving this comparison open.

• Consider CDM's fixed-schedule variant as a drop-in replacement for D-DMD: the fixed-schedule CDM (Table 2, right panel, row a) achieves HPSv3 9.482 vs. D-DMD's 9.176 with only 1.1× training time overhead. This variant keeps the CDM loss (velocity-driven off-trajectory matching) but uses the standard fixed discrete schedule, making it a minimal-code-change upgrade over an existing D-DMD implementation. The HPSv3 gain of +0.306 over D-DMD at nearly identical training cost makes this the most practical option for practitioners who want CDM's benefits without the full 1.8× training overhead.

• Prefer alternative methods (or extended D-DMD training) when: training compute is the binding constraint and the full CDM's 1.8× overhead is prohibitive. The paper does not demonstrate that CDM's quality improvements justify the training cost in a FLOPs-matched comparison — if training D-DMD for 1.8× longer closes the gap, the overhead is not justified. In latency-constrained deployment where the inference hardware cannot support 4 sequential Euler steps (e.g., extremely resource-limited edge devices), 1-step consistency models or progressive distillation may be preferable despite potentially lower quality. In domains where text-alignment is the primary metric (e.g., advertising compliance, accessibility tools requiring exact prompt adherence) rather than visual quality or human preference, the paper's modest CLIPScore improvements (+0.29 over D-DMD on SD3-Medium) may not justify the method switch — alternative methods with stronger text-alignment focus (e.g., Hyper-SD combined with CFG tuning) might be more appropriate.