LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectories

ArXiv: 2604.15311

🎯 Pitch

Direct-gradient fine-tuning of flow matching models collapses when trying to update early generation steps—the memory cost and gradient explosion from backpropagating through dozens of ODE steps make it impossible. LeapAlign solves this by carving the full trajectory into just two trainable leaps that skip intermediate steps, then applying a gradient discounting scheme that preserves cross-timestep dependencies instead of discarding them, enabling stable, memory-constant updates at any timestep and unlocking early-step optimization critical for image layout.

---

1. Executive Summary

This paper introduces LeapAlign, a post-training method for aligning flow matching models with human preferences by enabling direct reward gradient backpropagation to early generation steps without incurring prohibitive memory cost or gradient explosion. The method constructs a two-step leap trajectory—carved from a standard full-run ODE sampling trajectory by designing two consecutive one-step leaps that each skip multiple sampling steps—and backpropagates reward gradients only through this shortened path, keeping memory constant while allowing randomized selection of start and end timesteps to update any generation step. Fine-tuning the Flux model with LeapAlign consistently outperforms state-of-the-art GRPO-based methods (DanceGRPO, MixGRPO) and direct-gradient methods (ReFL, DRaFT-LV, DRTune) across six automatic evaluators for general preference alignment and yields a GenEval overall score of 0.7420 (compared to 0.7232 for MixGRPO and 0.7101 for DRTune), establishing that propagating gradients to early timesteps—which largely determine image layout—is critical for compositional alignment improvements, but only when the nested gradient term capturing cross-timestep dependencies is preserved through gradient discounting rather than discarded.

2. Context and Motivation

The Core Problem: Early Steps Cannot Receive Reward Gradients

The fundamental puzzle this paper tackles is architectural: flow matching models have a fully differentiable generation process—unlike large language models—yet existing fine-tuning methods cannot exploit this differentiability to update early generation steps. The generation trajectory in a flow matching model proceeds from a Gaussian noise sample ( $x_1$ ) through dozens of ODE sampling steps to produce a final image ( $x_0$ ). Because each step is a differentiable function of model parameters ( $\theta$ ), a reward signal computed at the final image can, in principle, be backpropagated through the entire chain to update parameters at any timestep via the chain rule. This is the central theoretical advantage of what the paper terms direct-gradient methods—they use the native differentiability of the sampler rather than resorting to policy gradient estimates with their attendant variance and stochasticity.

However, this theoretical advantage is almost entirely unrealized in practice. The paper identifies two concrete, intertwined obstacles that make backpropagation through long trajectories intractable:

1. Prohibitive memory cost: Each ODE sampling step produces activations that must be retained for backpropagation. With 25–50 steps (typical for high-quality generation), the memory required to store the full computation graph exceeds GPU capacity, especially for large models like Flux.

2. Gradient explosion: The chain rule multiplies Jacobians ( $\partial v_\theta(x_t) / \partial x_t$ ) across timesteps. Since each Jacobian can have eigenvalues greater than one, the product over many steps can produce gradients with enormous magnitude, destabilizing optimization. The paper explicitly notes this in Section 1: "backpropagating through long trajectories results in prohibitive memory costs and gradient explosion."

The practical consequence of these obstacles is that existing direct-gradient methods update only one or a few timesteps near the end of the trajectory—typically the final step or the last few steps before the clean image ( $x_0$ ). ReFL (Xu et al., 2023) randomly selects a single timestep near the end and uses a one-step leap prediction to estimate the final image. DRaFT-LV (Clark et al., 2023) updates only the last sampling step, reducing gradient variance by repeatedly noising the final image and aggregating reward gradients. Neither touches early timesteps.

This is a critical gap because early generation steps determine the global layout and composition of the final image. As the paper notes (Section 1, citing Hertz et al., 2022 and Liang et al., 2025): "early steps that largely determine image layout are not updated." In text-to-image generation, the first few denoising steps establish the spatial arrangement of objects, their relative positions, and the overall scene structure—the very properties that compositional alignment tasks like GenEval are designed to test. If a fine-tuning method cannot modify these early steps, it cannot meaningfully improve compositional text-image alignment, regardless of how much it refines local details at later steps.

Why This Problem Matters

The paper's framing makes clear that this is not merely an implementation detail—it has direct consequences for what kinds of improvements post-training can achieve. The GenEval benchmark (Ghosh et al., 2023) tests six categories of compositional generation: single-object generation, two-object generation, counting, colors, spatial position, and attribute binding. These tasks require the model to correctly arrange multiple objects according to spatial and attributional relationships specified in the prompt. The evidence in the paper (Table 2, Figure 3) shows a clear performance hierarchy that maps directly to the architectural question of early-step access:

• Methods that cannot update early steps (ReFL, DRaFT-LV) show only marginal improvements in GenEval overall score over the pretrained Flux model (0.7011 and 0.7024 vs. 0.6535). Their generated layouts "remain similar to those of the pretrained model" (Section 6.2, Figure 3).
• Methods that can update early steps (DRTune, LeapAlign) show substantially larger improvements (0.7101 and 0.7420). The paper explicitly connects this: "these results indicate the benefit of fine-tuning early steps."
• Our method, which updates early steps while preserving the nested gradient, achieves the strongest results, particularly on the most compositionally demanding categories: +9.96 points on two-object generation, +6.12 points on colors, +10.75 points on position, and +20.75 points on attribute binding over the pretrained model, and substantially ahead of DRTune in several categories (e.g., 96.46 vs. 93.69 on two-object, 80.59 vs. 76.86 on colors, 66.00 vs. 55.50 on attribute binding).

This is not just about winning benchmarks. Compositional generation is a fundamental capability for text-to-image systems deployed in real applications—users expect "a yellow bicycle and a red motorcycle" to produce images containing both objects with their correct colors in a coherent spatial arrangement. The paper argues that improving compositional alignment requires modifying early generation steps, and that current methods cannot do this effectively, creating a gap between what the differentiable architecture theoretically supports and what post-training methods actually achieve.

Beyond composition, the paper identifies broader practical stakes. GRPO-based methods (DanceGRPO, MixGRPO), which dominate recent flow matching post-training literature, are described as having "a considerable level of stochasticity and variance" (Section 1) because they treat the generation process as a discrete sequence of actions, applying policy gradients designed for LLMs. This approach discards the continuous, differentiable structure that makes flow matching models architecturally distinct from autoregressive LLMs. If direct-gradient methods could overcome the memory and stability barriers that currently limit them to late-step updates, they could offer "faster convergence and more stable training" (Section 1) than policy-gradient alternatives, making preference alignment more efficient and reliable.

Prior Approaches and Their Limitations

The paper situates itself against a multi-branch landscape of post-training methods for diffusion and flow matching models, each with demonstrated successes but also specific shortcomings that motivate the LeapAlign design.

Policy-Gradient Methods: Borrowed from LLMs, Limited by Variance

GRPO (Group Relative Policy Optimization), originally designed for LLM post-training (Shao et al., 2024), has been adapted to flow matching models by several recent works: DanceGRPO (Xue et al., 2025), Flow-GRPO (Liu et al., 2025), and MixGRPO (Li et al., 2025). These methods convert the deterministic ODE sampling process into an equivalent SDE formulation by adding noise at each step, then apply the GRPO loss across generation steps. The fundamental issue, as the paper frames it, is that these are policy gradient methods: they estimate the gradient of expected reward with respect to model parameters by sampling trajectories and weighting updates by reward. Policy gradients are inherently stochastic and high-variance because they must estimate an expectation from finite samples, and the variance scales with the number of timesteps and the reward variance. In contrast, direct-gradient methods compute the exact gradient of the reward with respect to parameters through the chain rule, providing a deterministic, lower-variance signal. The paper notes (Section 1): "Because the text generation process of LLMs is not differentiable, policy gradient forms the basis of these methods, which inevitably adds a considerable level of stochasticity and variance." The implication is that applying policy gradients to flow matching models is a mismatch—forcing a discrete optimization framework onto a continuous, differentiable process, sacrificing the efficiency and stability that direct gradient computation could provide.

Direct-Gradient Methods: Differentiable, but Constrained to Late Steps

Three direct-gradient methods form the paper's primary technical lineage, each introducing ideas that LeapAlign builds upon while addressing their core limitation:

ReFL (Reward Feedback Learning; Xu et al., 2023): ReFL exploits the one-step leap prediction property of diffusion models—from any intermediate latent ( $x_t$ ), the model can estimate the clean image ( $\hat{x}_0$ ) in a single step. ReFL randomly selects a single timestep ( $t_{\min}$ ) near the end of the trajectory, computes ( $\hat{x}_0$ ) from ( $x_{t_{\min}}$ ), evaluates the reward on ( $\hat{x}_0$ ), and backpropagates the reward gradient only through that one step. This keeps memory constant and avoids gradient explosion, but it has two critical limitations: (1) only a single late timestep is updated per iteration, and (2) the reward is evaluated on a noisy approximation ( $\hat{x}_0$ ) rather than the actual final image ( $x_0$ ), which may contain noise and artifacts that make the reward model's assessment less reliable. The paper positions the leap trajectory as a direct extension of this one-step leap idea—using two leaps instead of one to reach further back in the trajectory—and the use of the actual ( $x_0$ ) for reward evaluation as a key improvement (Section 4.4).

DRaFT-LV (Differentiable Reward Fine-Tuning with Low Variance; Clark et al., 2023): DRaFT-LV updates only the last sampling step of the generation trajectory. To reduce gradient variance, it repeatedly noises the final image using the forward process and aggregates reward gradients across these noisy variants—essentially averaging the gradient over multiple nearby states. This provides a lower-variance estimate than ReFL's single-sample approach, but it shares ReFL's fundamental limitation: it cannot reach early timesteps. The paper identifies DRaFT-LV as a late-step-only method (Table 1) and shows it underperforms methods with early-step access on GenEval (0.7024 vs. 0.7101 for DRTune, 0.7420 for LeapAlign in Table 2).

DRTune (Deep Reward Tuning; Wu et al., 2024): DRTune is the closest predecessor to LeapAlign and the only prior direct-gradient method that can update early generation steps. Its key insight is to stop the gradient at the model input—that is, during backpropagation, the gradient with respect to the input latent ( $x_t$ ) is detached (via stop_gradient), preventing the chain rule from propagating through multiple steps. This avoids both the memory cost (no need to store the full computation graph) and gradient explosion (no Jacobian multiplication across steps). DRTune can thus fine-tune multiple steps per rollout (the paper uses ( $K = 2$ )) at any position in the trajectory.

However, DRTune's solution introduces its own limitation: removing the gradient flow through the model input also removes the nested gradient term. The paper formalizes this in Equation 8 (Section 4.3). The gradient of the final image with respect to parameters, when backpropagated through two leap steps, decomposes into:

$$\frac{\partial x_0}{\partial \theta} = -j \frac{\partial v_\theta(x_j)}{\partial \theta} - (k-j)\frac{\partial v_\theta(x_k)}{\partial \theta} + j(k-j)\frac{\partial v_\theta(x_j)}{\partial x_j}\frac{\partial v_\theta(x_k)}{\partial \theta}$$

The first two terms are single-step gradients—each arises from the gradient of a single one-step leap prediction. The third term is the nested gradient—it captures how changes to the model at timestep ( $k$ ) affect the latent ( $x_j$ ), which in turn affects the velocity prediction at timestep ( $j$ ), propagating through to the final image. This term represents the cross-timestep interaction: the model at step ( $k$ ) and the model at step ( $j$ ) do not operate independently; the output of step ( $k$ ) determines the input to step ( $j$ ), so the gradient should account for how parameters at step ( $k$ ) influence what step ( $j$ ) sees. DRTune's stop-gradient on the model input sets ( $\partial v_\theta(x_j)/\partial x_j = 0$ ), which removes this nested gradient entirely. The paper argues this "discards substantial gradient flow and leads to incomplete optimization" (Section 1) and provides experimental evidence that even a discounted version of the nested gradient improves performance over complete removal (Figure 4a: ( $\alpha = 0.3$ ) outperforms ( $\alpha = 0$ )).

A subtle but important distinction the paper makes: DRTune's approach is to stop the gradient at the input to the velocity model—i.e., ( $v_\theta(\text{stop\_gradient}(x_t), t, c)$ ). This means the model still receives the correct latent as input during the forward pass (so the velocity prediction is accurate), but no gradient flows backward through ( $x_t$ ) to earlier steps. The trade-off is clean: zero memory growth and zero explosion risk, at the cost of zero cross-timestep gradient signal. LeapAlign's gradient discounting (Section 4.3) can be understood as a continuous interpolation between this extreme (( $\alpha = 0$ ), identical to DRTune's nested gradient removal) and the full nested gradient (( $\alpha = 1$ )), with an empirically chosen ( $\alpha = 0.3$ ) offering the best trade-off.

A Crucial Architectural Observation

The paper makes a sharp observation that contextualizes why the prior work developed this way: the differentiable generation process is what distinguishes flow matching models from LLMs, yet all existing post-training methods either (a) treat the process as discrete (policy gradients) or (b) backpropagate through only a tiny suffix of it (direct-gradient methods). This creates a strange situation where the defining architectural feature—differentiability—is almost entirely unused for the portion of the trajectory where it matters most (early steps). The paper's statement "Essentially, what makes flow matching models differ from LLMs is that the sampling process of the former is continuous and differentiable" (Section 1) is not just pedagogical—it frames the entire research problem: how can we design a method that fully exploits this differentiability without paying the costs that have prevented prior methods from doing so?

How This Paper Positions Itself

LeapAlign does not propose an entirely new paradigm. It operates within the direct-gradient framework established by ReFL, DRaFT-LV, and DRTune. Its contribution is to identify the specific mechanism—the leap trajectory—that resolves the tension between the desire to update early steps and the practical constraints of memory and gradient stability that prior methods could only address with compromises (late steps only, or early steps with gradient removal).

The paper's positioning is explicitly comparative and cumulative (Table 1 aligns all four direct-gradient methods across four axes: Early Steps, Nested Gradient, Leap Trajectory, Multi-Step). LeapAlign is the only method that satisfies all four desiderata. The leap trajectory is positioned as the enabler: by shortening the long trajectory into exactly two steps (via two one-step leaps with latent connectors), it makes the memory cost constant regardless of which timesteps are selected, and it bounds gradient explosion to a manageable two-step product that gradient discounting can then moderate. The randomized selection of ( $k$ ) and ( $j$ ) ensures that, over many iterations, all timesteps—including the earliest ones that determine layout—receive gradient updates.

The paper also positions its evaluation to directly test whether this architectural difference translates to measurable gains. The GenEval benchmark is specifically chosen because compositional alignment is the capability that early-step fine-tuning should most directly improve. The consistent improvement over DRTune across all GenEval categories, despite both methods being able to update early steps, is presented as evidence that the nested gradient term—the one DRTune removes and LeapAlign preserves—is not merely theoretically interesting but practically important for learning dependencies across generation steps.

Finally, the paper positions itself as a foundation for future work that combines direct-gradient methods with other advances. Section 5 notes that LeapAlign can accommodate any differentiable reward model, and the experiments confirm this across CLIP-based (HPSv2.1, PickScore) and VLM-based (HPSv3) rewards. The extension to non-differentiable rewards via differentiable value models is flagged as future work, suggesting the leap trajectory mechanism could generalize beyond the fully differentiable setting.

3. Technical Approach

3.1 Reader Orientation

LeapAlign is a fine-tuning system that teaches a flow matching image generation model to produce images that better align with human preferences and compositional text prompts by computing exact reward gradients and routing them backward through the model's generation process—but only along a cleverly shortened two-step path that avoids the memory explosion and gradient instability that would normally make this impossible for early generation steps. The system solves the problem of "how do we update the model at timesteps near the noise (where image layout is determined) when backpropagating through 25–50 sampling steps would exceed GPU memory and cause gradients to explode?" by carving a leap trajectory out of a full generation run: instead of backpropagating through all 25 steps, it selects two intermediate timesteps, makes two large one-step predictions that skip most of the trajectory, and backpropagates only through those two steps, keeping memory constant regardless of which timesteps are chosen.

3.2 Big-Picture Architecture

The system has five interacting components:

1. Pre-trained flow matching model (Flux) — the base image generator that takes a text prompt and Gaussian noise, runs an ODE solver for 25 steps, and produces a 720×720 image. All parameters of this model are fine-tuned.

2. Full generation trajectory — a complete forward pass through all 25 ODE steps storing every intermediate latent $x_t$ and velocity prediction $v_\theta(x_t)$. This trajectory is generated online during each training iteration and serves as the raw material from which the leap trajectory is carved.

3. Leap trajectory constructor — randomly selects two timesteps $k > j$ from the full trajectory, applies the one-step leap prediction formula (Equation 3) to jump from $x_k$ to $\hat{x}_{j|k}$ and from $x_j$ to $\hat{x}_{0|j}$, then uses latent connectors (Equations 6–7) to snap these predictions back to the actual latents $x_j$ and $x_0$ while preserving differentiability. The result is a two-step computational path $x_k \to (\hat{x}_{j|k} \dashrightarrow x_j) \to (\hat{x}_{0|j} \dashrightarrow x_0)$ through which gradients will flow.

4. Gradient discounting mechanism — a modification to the second leap step that scales the nested gradient term (the cross-timestep interaction $\partial v_\theta(x_j)/\partial x_j \cdot \partial v_\theta(x_k)/\partial\theta$) by a factor $\alpha \in [0,1]$ instead of removing it entirely. This preserves the full gradient structure while controlling magnitude.

5. Trajectory-similarity weighting — a per-sample loss weight computed from how closely the leap predictions $\hat{x}_{j|k}$ and $\hat{x}_{0|j}$ match the actual latents $x_j$ and $x_0$. Leap trajectories that deviate significantly from the true generation path receive lower weight.

Information flows as follows: a text prompt enters the system → the pre-trained Flux model runs a full 25-step ODE trajectory, recording all latents → two random timesteps $k$ and $j$ are selected → the leap trajectory is constructed using the one-step leap predictions at $k$ and $j$ with latent connectors → the reward model scores the actual final image $x_0$ → the reward gradient is backpropagated only through the two-step leap path, with the nested gradient scaled by $\alpha$ → parameters are updated → the process repeats, with different random $(k, j)$ pairs ensuring all timesteps eventually receive gradients.

3.3 Roadmap for the Deep Dive

• First, the one-step leap prediction formula (Equation 3): the mathematical property of rectified flow matching that makes leap trajectories possible—the ability to estimate any future latent from any current latent using only the velocity prediction at the current step. Without this, the entire method collapses.

• Second, the leap trajectory construction: how the two one-step leaps are composed, what the latent connectors do mechanistically, and why this two-step design bounds memory while allowing randomized timestep selection to reach any generation step.

• Third, the gradient structure through two leaps: the decomposition of $\partial x_0 / \partial \theta$ into single-step and nested gradient terms (Equation 8), what each term represents physically, and why the nested gradient matters for capturing cross-timestep dependencies.

• Fourth, gradient discounting: how modifying the input to the velocity model at timestep $j$ (Equation 9) scales the nested gradient by $\alpha$ (Equation 10), and why this continuous control is superior to DRTune's binary removal ($\alpha = 0$).

• Fifth, the fine-tuning objective and trajectory-similarity weighting: the hinge loss (Equation 11) for reward maximization, why the reward is evaluated on the actual image $x_0$ rather than the approximated $\hat{x}_{0|j}$, and how the similarity weight (Equation 12) up-weights faithful leap trajectories.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a method paper whose core idea is that backpropagating reward gradients through a shortened two-step trajectory constructed from a full ODE run enables stable gradient-based fine-tuning of any generation step—including early steps critical for layout—without the memory and numerical costs of full-trajectory backpropagation, and that preserving the nested gradient through discounting (rather than removing it) provides measurable improvements in compositional alignment.

One-Step Leap Prediction: The Mathematical Enabling Property

The entire LeapAlign method rests on a single property of rectified flow matching models: from any latent $x_k$ at timestep $k$, the model can directly estimate the latent $x_j$ at any other timestep $j$ using only the velocity prediction at $k$. This property is what makes "leaps" possible—instead of stepping through the ODE one small increment at a time, the model can jump directly across many timesteps in a single computation.

The underlying scheduler. Rectified flow matching (Liu et al., 2022) uses a linear interpolation between data and noise:

$\alpha_t = 1 - t, \quad \beta_t = t$

so that the forward noising process (Equation 1) becomes:

$x_t = (1 - t) x_0 + t x_1$

where $x_0 \sim p_{\text{data}}$ is a real image, $x_1 \sim \mathcal{N}(0, I)$ is Gaussian noise, and $t \in [0, 1]$ is the timestep parameter (with $t = 1$ corresponding to pure noise and $t = 0$ to the clean image). The velocity field is then $v = dx_t/dt = x_1 - x_0$, a constant vector pointing from the clean image to the noise.

The one-step leap prediction formula. The paper derives in Appendix F that, for a general scheduler $(\alpha_t, \beta_t)$, the latent at timestep $j$ can be estimated from the latent and velocity at timestep $k$ as:

$\hat{x}_{j|k} = \alpha_j \hat{x}_{0|k} + \beta_j \hat{x}_{1|k}$

where $\hat{x}_{0|k}$ and $\hat{x}_{1|k}$ are estimates of the clean image and the noise respectively, both computed from $x_k$ and $v(x_k, k)$. Under the rectified flow scheduler $\alpha_t = 1-t, \beta_t = t$, the time derivatives are $\dot{\alpha}_k = -1$ and $\dot{\beta}_k = 1$, and the formula simplifies to the remarkably simple Equation 3:

$\hat{x}_{j|k} = x_k - (k - j) \, v_\theta(x_k, k)$

where $k, j \in [0, 1]$ are timesteps, $x_k$ is the latent at timestep $k$, and $v_\theta(x_k, k)$ is the velocity predicted by the flow matching model at that latent. (The paper uses the shorthand $v_\theta(x_k)$ for the velocity after classifier-free guidance, omitting the text condition and timestep for notational simplicity.)

What this equation computes operationally: given the latent state at timestep $k$ and the model's velocity prediction at that state, subtract $(k - j)$ times the velocity vector from the latent to obtain an estimate of what the latent would be at timestep $j$. If $k > j$ (moving toward the clean image), this subtracts a multiple of the velocity, moving in the direction opposite to the flow; if $k < j$ (moving toward noise), it would add a multiple. The intuition is that in rectified flow, the trajectory is a straight line, so moving along it is just linear extrapolation.

Why this form matters for LeapAlign. The one-step leap prediction replaces $|k - j|/\Delta t$ individual ODE steps—where $\Delta t$ is the solver step size—with a single model evaluation. This is what makes leap trajectories have constant (two-step) computational cost regardless of which timesteps are selected. Without this property, reaching from an early timestep (e.g., $k = 0.8$) to a late timestep (e.g., $j = 0.2$) would require stepping through dozens of intermediate latents, each stored for backpropagation, reproducing the memory explosion problem. The one-step leap prediction collapses all those intermediate steps into a single function of $x_k$ and $\theta$, making direct gradient backpropagation from the final image to early timesteps tractable.

A critical subtlety: $\hat{x}_{j|k}$ is an approximation. Because the flow matching model is imperfectly trained, the velocity prediction $v_\theta(x_k, k)$ is not exactly the true velocity, and because the actual ODE trajectory may deviate slightly from a straight line in practice (the model is trained to make trajectories straight, but not perfectly), the leap prediction will not exactly equal the latent that would be reached by running the numerical solver. This approximation error is why the paper introduces latent connectors (Section 4.2) and trajectory-similarity weighting (Section 4.5)—to account for and mitigate the fact that leap predictions are not exact.

Leap Trajectory Construction: Carving a Two-Step Path from the Full Trajectory

The leap trajectory construction procedure takes a full 25-step ODE sampling run (produced online during each training iteration) and creates a two-step differentiable path that approximates the full trajectory while enabling gradient flow to arbitrary timesteps.

Step 1: Run the full trajectory. At each training iteration, the system samples a text prompt, draws Gaussian noise $x_1 \sim \mathcal{N}(0, I)$, and runs the complete 25-step ODE sampling process using the current model parameters $\theta$. The latents $x_T, x_{T-1}, \ldots, x_0$ are recorded (where $T = 25$, with $x_{25}$ corresponding to noise at $t = 1$ and $x_0$ to the clean image). The velocity predictions $v_\theta(x_t)$ at each step are also recorded. This full trajectory is generated without constructing the computation graph for backpropagation—gradients are disabled during this forward pass (Algorithm 1, line 14: disable_grad()), so no memory is consumed storing activations. Only the numerical values of the latents and velocities are retained.

Step 2: Randomly select two timesteps. Two timesteps $k$ and $j$ are randomly selected from the set $\{1, \ldots, T\}$ with the constraint $k > j$. The paper uses fully random selection within $[0, 1]$ (normalized to the generation step indices) and shows in Figure 4f that random selection slightly outperforms fixing the distance between $k$ and $j$. The randomness is essential: over many iterations, all pairs $(k, j)$ are eventually sampled, meaning every timestep—including the earliest ones ($k$ near $T$)—receives gradient updates. This is how LeapAlign achieves the "any generation step" property without needing to backpropagate through the entire trajectory in a single iteration.

Step 3: Construct the first leap. At timestep $k$, the model's velocity $v_\theta(x_k)$ is re-evaluated with gradients enabled (Algorithm 1, line 12: enable_grad()). The one-step leap prediction formula (Equation 4) estimates the latent at timestep $j$:

$\hat{x}_{j|k} = x_k - (k - j) \, v_\theta(x_k)$

where $x_k$ is the detached latent recorded from the full trajectory. The subtraction $(k - j) v_\theta(x_k)$ is a single differentiable operation—the gradient of $\hat{x}_{j|k}$ with respect to $\theta$ flows through $v_\theta(x_k)$ and thus through the model parameters at timestep $k$.

Step 4: Latent connector at timestep $j$. Because $\hat{x}_{j|k}$ is only an approximation of the true $x_j$, LeapAlign aligns the predicted latent to the actual latent from the full trajectory using a latent connector (Equation 6):

$x_j^{\text{(leap)}} = \hat{x}_{j|k} + \texttt{stop\_gradient}(x_j - \hat{x}_{j|k})$

What this does mechanistically: The difference $x_j - \hat{x}_{j|k}$ is the error in the one-step leap prediction—the vector from the approximated latent to the actual latent. The stop_gradient operation detaches this error from the computation graph, treating it as a constant. Adding this constant error to $\hat{x}_{j|k}$ makes $x_j^{\text{(leap)}}$ numerically equal to the actual $x_j$, but gradients flow only through $\hat{x}_{j|k}$ (and thus through $v_\theta(x_k)$ and back to $\theta$), not through the correction term. The result is that the forward pass uses the correct latent $x_j$, ensuring the next velocity prediction $v_\theta(x_j)$ operates on the right input, while the backward pass treats the leap prediction error as irreducible and does not penalize the model for imperfect approximation.

Step 5: Construct the second leap. With $x_j^{\text{(leap)}}$ as input (which equals $x_j$ numerically), the model computes $v_\theta(x_j)$ with gradients enabled. The second one-step leap prediction (Equation 5) estimates the clean image:

$\hat{x}_{0|j} = x_j - j \, v_\theta(x_j)$

where $j$ is the timestep value (normalized; for the 25-step discretization, the actual value would be $j/25$). This step is symmetric to the first leap: a single differentiable operation that jumps from $x_j$ directly to an estimate of $x_0$.

Step 6: Latent connector at the final image. The same latent connector pattern aligns $\hat{x}_{0|j}$ to the actual $x_0$ (Equation 7):

$x_0^{\text{(leap)}} = \hat{x}_{0|j} + \texttt{stop\_gradient}(x_0 - \hat{x}_{0|j})$

After this step, $x_0^{\text{(leap)}}$ numerically equals the actual final image $x_0$ from the full trajectory, but gradients flow through the two leap predictions and their velocity computations back to $\theta$.

Why two steps? The paper experiments with one, two, and three-step leap trajectories (Figure 4b). One step (a single leap from $x_k$ directly to $\hat{x}_{0|k}$) underperforms two steps—likely because a single large leap compounds the approximation error. Three steps increase memory usage (three model evaluations' activations must be stored instead of two) without providing better results than two steps. Two steps hit a sweet spot: they allow reaching far back into the trajectory (by selecting $k$ near the noise end) while keeping the per-leap distance manageable, and they capture a non-trivial cross-timestep interaction (the nested gradient between steps $k$ and $j$).

Why random selection of $k$ and $j$? If $k$ and $j$ were fixed (e.g., always the first and last steps), only those two timesteps would ever receive gradients, defeating the purpose of enabling updates to any generation step. Random selection over $[0, 1]$ ensures that over many iterations, the distribution of $(k, j)$ pairs covers the entire trajectory. The paper shows (Figure 4f) that random selection slightly outperforms fixing the distance $k - j$—the randomness may help by exposing the model to diverse gradient paths, preventing overfitting to any particular pair of timesteps.

What the leap trajectory enables: Because the computational path from $x_k$ to $x_0$ involves exactly two velocity model evaluations (at $k$ and at $j$), the memory required for backpropagation is constant irrespective of which timesteps are chosen. A leap trajectory with $k$ near the noise end and $j$ near the data end uses exactly the same memory as one with both $k$ and $j$ near the data end. The gradient magnitude is also bounded—it involves the product of at most two Jacobians (the nested gradient term $\partial v_\theta(x_j)/\partial x_j \cdot \partial v_\theta(x_k)/\partial \theta$), compared to the product of 25 Jacobians in full-trajectory backpropagation. This is what makes gradient explosion manageable (via discounting) rather than catastrophic.

The Gradient Structure Through Two Leaps: Single-Step vs. Nested Gradients

When the reward gradient is backpropagated through the leap trajectory, the total gradient with respect to model parameters decomposes into three terms with distinct physical meanings. Understanding this decomposition is central to the paper's argument for why gradient discounting is necessary and why complete nested gradient removal (DRTune's approach) is suboptimal.

The paper derives the gradient in Appendix G. Starting from the leap trajectory without gradient discounting, the forward pass is:

$\hat{x}_{j|k} = x_k - (k - j) v_\theta(x_k)$ $x_j^{\text{(leap)}} = \hat{x}_{j|k} + \texttt{stop\_gradient}(x_j - \hat{x}_{j|k})$ $\hat{x}_{0|j} = x_j^{\text{(leap)}} - j \, v_\theta(x_j^{\text{(leap)}})$ $x_0^{\text{(leap)}} = \hat{x}_{0|j} + \texttt{stop\_gradient}(x_0 - \hat{x}_{0|j})$

Since the latent connectors ensure $x_j^{\text{(leap)}} = x_j$ and $x_0^{\text{(leap)}} = x_0$ in the forward pass, and the stop_gradient operations zero out the gradient through the correction terms, the gradient of the final image with respect to parameters $\theta$ is (Equation 8):

$\frac{\partial x_0}{\partial \theta} = -j \frac{\partial v_\theta(x_j)}{\partial \theta} - (k - j) \frac{\partial v_\theta(x_k)}{\partial \theta} + j(k - j) \frac{\partial v_\theta(x_j)}{\partial x_j} \frac{\partial v_\theta(x_k)}{\partial \theta}$

where $j$ and $k$ are the timestep values (normalized to $[0, 1]$), $v_\theta(x_j)$ is the velocity at timestep $j$, $v_\theta(x_k)$ is the velocity at timestep $k$, $\partial v_\theta(x_j)/\partial \theta$ is the direct gradient of the velocity at $j$ with respect to parameters, $\partial v_\theta(x_k)/\partial \theta$ is the direct gradient at $k$, and $\partial v_\theta(x_j)/\partial x_j$ is the Jacobian of the velocity at $j$ with respect to its input latent.

What each term represents operationally:

• First term $-j \, \partial v_\theta(x_j)/\partial\theta$: the direct gradient from the second leap. Changing $\theta$ changes the velocity prediction at timestep $j$, which changes $\hat{x}_{0|j}$ by $-j$ times that change (from Equation 5: moving in the negative velocity direction scaled by $j$). This term captures how parameters affect the model's behavior at timestep $j$ independently of what happened at timestep $k$.

• Second term $-(k-j) \, \partial v_\theta(x_k)/\partial\theta$: the direct gradient from the first leap. Changing $\theta$ changes the velocity prediction at timestep $k$, which changes $\hat{x}_{j|k}$ by $-(k-j)$ times that change. Because $x_j^{\text{(leap)}}$ inherits this change (through the differentiable path of the latent connector), the second leap operates on a shifted input, and that shift propagates to $\hat{x}_{0|j}$ and thus to $x_0$. This term captures how parameters affect the model at timestep $k$, again independently.

• Third term $j(k-j) \, \partial v_\theta(x_j)/\partial x_j \cdot \partial v_\theta(x_k)/\partial\theta$: the nested gradient. This term captures the interaction: changing $\theta$ at timestep $k$ shifts the latent that timestep $j$ receives as input (through $\partial v_\theta(x_k)/\partial\theta$ affecting $\hat{x}_{j|k}$), and this shift in input changes the velocity prediction at timestep $j$ (through the Jacobian $\partial v_\theta(x_j)/\partial x_j$). The factor $j(k-j)$ comes from the chain rule: $-(k-j)$ from the first leap's sensitivity to velocity, times $-j$ from the second leap's sensitivity, times $-1$ from the chain rule through the Jacobian, yielding $+j(k-j)$.

Why the nested gradient matters physically. The velocity model at timestep $j$ sees the latent $x_j$. But $x_j$ is not arbitrary—it was reached from $x_k$ by the model's own predictions (or their ODE-stepped equivalents in the full trajectory). If the model at timestep $k$ produces a slightly different velocity, the resulting $x_j$ will be slightly different, and the model at timestep $j$ must then respond to that different input. The nested gradient captures this dependency: it tells the optimizer how changing parameters at early steps affects what later steps need to do, encouraging coordination across timesteps. Removing the nested gradient (as DRTune does by stopping the gradient at $x_j$) means the model at timestep $k$ is optimized as if the model at timestep $j$ will not adapt to whatever $x_j$ it produces. This is a form of incomplete optimization—the optimizer sees only the direct effect of early-step changes and misses the second-order effect through later steps.

Why the nested gradient causes problems. The Jacobian $\partial v_\theta(x_j)/\partial x_j$ can have eigenvalues with magnitude greater than one, especially in the FiT (Flow Transformer) backbone used by Flux, which contains self-attention and feed-forward layers. When multiplied by $j(k-j)$ which can be as large as $0.25$ (when $j = k/2$, maximizing the product), the nested gradient term can dominate the total gradient, producing very large updates that destabilize training. This is precisely the "gradient explosion" problem the paper identifies: it is not that the full trajectory accumulates 25 Jacobian products (which would be astronomically unstable), but that even a single Jacobian product across two steps can produce large gradients. Gradient discounting addresses this by scaling the problematic term without removing it.

A note on the derivation in Appendix G: The paper's derivation treats the rollout states $x_k$, $x_j$, and $x_0$ from the full trajectory as detached constants. This means the gradient computation does not need to know how $x_k$ was reached from $x_{k+1}$, etc.—the full trajectory is treated as a given, and only the leap trajectory contributes to gradients. This is why memory is constant: the full trajectory is generated without a computation graph, and only the two leap steps build differentiable operations.

Gradient Discounting: Scaling the Nested Gradient Instead of Removing It

DRTune addresses gradient explosion by stopping the gradient at the model input: $v_\theta(\texttt{stop\_gradient}(x_t), t, c)$. This sets $\partial v_\theta(x_j)/\partial x_j = 0$ in Equation 8, removing the nested gradient term entirely. The paper argues this "discards substantial gradient flow and leads to incomplete optimization" (Section 1). LeapAlign's alternative, gradient discounting, preserves the nested gradient but scales it down by a factor $\alpha \in [0, 1]$, giving continuous control over the trade-off between gradient completeness and stability.

The mechanism. Gradient discounting modifies only the second leap step. Instead of computing the velocity at $x_j$ directly, the input to the velocity model is blended between the differentiable $x_j$ and its stopped-gradient version (Equation 9):

$\hat{x}_{0|j} = x_j - j \, v_\theta\!\left(\alpha x_j + (1 - \alpha) \, \texttt{stop\_gradient}(x_j)\right)$

where $\alpha$ is the discounting factor and $x_j$ here refers to $x_j^{\text{(leap)}}$ from the latent connector.

What this does in the forward pass: The blended input $\alpha x_j + (1-\alpha) \, \texttt{stop\_gradient}(x_j)$ numerically equals $x_j$ (since $x_j + 0 = x_j$), so the velocity prediction $v_\theta$ receives the correct input and produces the correct output. The forward pass is identical regardless of $\alpha$. The magic is in the backward pass: when computing gradients, the stop_gradient on the second term means only the $\alpha x_j$ portion contributes to the gradient flowing into $x_j$. Specifically:

$\frac{\partial}{\partial x_j} (\alpha x_j + (1-\alpha) \, \texttt{stop\_gradient}(x_j)) = \alpha$

So the Jacobian $\partial v_\theta(x_j)/\partial x_j$ is effectively multiplied by $\alpha$ in the chain rule. The resulting gradient (Equation 10) becomes:

$\frac{\partial x_0}{\partial \theta} = -j \frac{\partial v_\theta(x_j)}{\partial \theta} - (k - j) \frac{\partial v_\theta(x_k)}{\partial \theta} + \alpha \, j(k - j) \frac{\partial v_\theta(x_j)}{\partial x_j} \frac{\partial v_\theta(x_k)}{\partial \theta}$

where the only change from Equation 8 is the $\alpha$ factor multiplying the nested gradient term. The single-step gradient terms are unaffected because they do not involve $\partial v_\theta(x_j)/\partial x_j$.

Why this form is powerful. The $\alpha$ parameter gives continuous control between two extremes:

• $\alpha = 0$: the nested gradient is completely removed. This is equivalent to DRTune's stop-gradient approach. The gradient consists only of the two single-step terms.
• $\alpha = 1$: the full nested gradient is used without any discounting. This provides the complete gradient signal but risks instability from large Jacobian products.

The paper finds empirically that $\alpha = 0.3$ works best (Figure 4a). This means the nested gradient is retained at 30% of its full magnitude—enough to provide useful cross-timestep coordination signal, but not so much that it destabilizes training. A critical result in the ablation: even $\alpha = 0$ (LeapAlign without nested gradient) achieves 0.4064 HPSv2.1, which already exceeds DRTune's 0.3882 (Table 2). This shows that the leap trajectory design itself—the two-step construction, the reward evaluation on $x_0$, the trajectory-similarity weighting—provides benefits beyond the nested gradient preservation. The full $\alpha = 0.3$ version pushes this to 0.4092.

Additional analysis in Appendix C (Figure 6): The paper isolates the nested gradient's contribution by removing the single-step gradient at timestep $k$ (Equation 14) and fine-tuning only through the nested gradient term. With $\alpha = 1$ (full nested gradient), the average gradient norm during training spikes to approximately 2.7 and performance drops (0.4156 HPSv2.1). With $\alpha = 0.3$, the gradient norm is reduced to about 1.8 and performance reaches 0.4250. With $\alpha = 0$ (nested gradient removed), the model cannot learn from that step at all, yielding 0.4208. This directly visualizes the trade-off: the nested gradient contains useful signal (0.4250 > 0.4208) but only when its magnitude is controlled.

Why scaling rather than clipping or normalizing? The paper does not discuss alternatives like gradient clipping (which caps the norm after computing the full gradient) or adaptive normalization. The discounting approach is architecturally cleaner: it modifies the computational graph itself rather than post-processing gradients, which means the optimizer sees a consistently scaled signal. It also reduces the gradient magnitude before backpropagation, which means the single-step gradient terms (which are stable) are not affected by the presence of a large nested gradient—they would be with post-hoc gradient clipping, which normalizes the entire gradient vector uniformly.

Fine-Tuning Objective: Hinge Loss on Actual Image Reward

The fine-tuning objective maximizes the reward of the generated image while preventing reward hacking—a phenomenon where the model learns to exploit quirks of the reward function to achieve high scores without genuinely improving image quality.

Why a hinge loss? Directly maximizing reward (e.g., $\mathcal{L} = -r(x_0)$) can cause the optimizer to push rewards arbitrarily high by producing images that exploit the reward model's blind spots—images that score well but are visually degraded or semantically nonsensical. This is the well-known reward hacking problem. The paper adopts the hinge-style loss from Xu et al. (2023) (Equation 11):

$\mathcal{L}_{\text{raw}} = \max(0, \lambda - r(x_0))$

where $r(\cdot)$ is the differentiable reward model (HPSv2.1, PickScore, or HPSv3) and $\lambda$ is a threshold hyperparameter that controls the saturation point of the loss.

What this computes: If the reward $r(x_0)$ exceeds the threshold $\lambda$, the loss is zero—the model is not penalized, but it also receives no gradient signal to further increase the reward. If the reward is below $\lambda$, the loss is $\lambda - r(x_0)$, which is positive and decreases as reward increases. The gradient of this loss with respect to the image is $-\nabla_{x_0} r(x_0)$ when $r(x_0) < \lambda$ and zero otherwise. So the model is pushed to increase reward only up to the threshold, after which optimization stops applying pressure.

Why this form: The hinge loss prevents unbounded reward maximization. Once the threshold is reached, there is no incentive to push reward higher, which prevents the model from drifting into reward-hacking regimes that might produce even higher (but meaningless) scores. The threshold $\lambda$ is set according to the reward model's scale: by default $\lambda = 0.55$ for HPSv2.1 (whose scores roughly range from 0.2 to 0.4 for the pretrained model), $\lambda = 0.4$ for PickScore, and $\lambda = 13.5$ for HPSv3. These values represent targets somewhat above the pretrained model's typical scores, providing room for improvement without being unreachable.

The paper includes an ablation on $\lambda$ (Table 5): $\lambda = 0.35$ yields inferior performance (0.3860 HPSv2.1) because the model is under-optimized—the loss saturates before meaningful improvements are achieved. $\lambda = 0.95$ also degrades performance (0.4023), particularly out-of-domain (HPSv3 drops from 15.77 to 15.73, PickScore from 23.71 to 23.51, ImageReward from 1.51 to 1.39). Excessively high thresholds push optimization too aggressively, encouraging reward overfitting at the expense of generalization. The chosen $\lambda = 0.55$ provides the best balance.

Why evaluate the reward on $x_0$ rather than $\hat{x}_{0|j}$? This is a deliberate design choice that the paper contrasts with ReFL, which evaluates reward on the one-step approximation $\hat{x}_{0|t_{\min}}$. The actual final image $x_0$ is the output of the full 25-step ODE solver; it is the image that would be shown to a user. The approximation $\hat{x}_{0|j}$, produced by a single leap from an intermediate latent, may contain noise, blur, or artifacts because the leap prediction is not exact—especially when $j$ is large (far from the clean image). Evaluating reward on a noisy approximation introduces two problems: (1) the reward signal may be inaccurate (the reward model sees a degraded image), and (2) the gradient backpropagated through the reward model is based on this degraded image rather than the actual output, potentially optimizing the wrong objective. LeapAlign computes the reward on $x_0$ (the actual image from the full trajectory), ensuring faithful reward evaluation, while using the leap trajectory only as the gradient propagation path—the reward itself is not backpropagated through the full trajectory, only through the two-step leap.

Trajectory-Similarity Weighting: Up-Weighting Faithful Leap Trajectories

A subtle issue with leap trajectories is that the one-step leap predictions $\hat{x}_{j|k}$ and $\hat{x}_{0|j}$ are approximations. The quality of these approximations varies across timestep pairs: when $k$ and $j$ are close together (small leap distance), the approximation tends to be good; when they are far apart, the approximation error grows. If a leap trajectory deviates significantly from the true generation path, the gradient propagated through it may be misleading—it reflects how the model's parameters affect an approximate trajectory rather than the true one. Trajectory-similarity weighting addresses this by giving more influence to leap trajectories that closely match the actual path.

Similarity measurement. The paper measures similarity as the average absolute difference between predicted and actual latents at the two connection points (Section 4.5):

$d_j = \texttt{mean}\!\left(|x_j - \hat{x}_{j|k}|\right), \quad d_0 = \texttt{mean}\!\left(|x_0 - \hat{x}_{0|j}|\right)$

where $| \cdot |$ denotes element-wise absolute value, and mean averages over all spatial locations and channels of the latent tensor. $d_j$ measures how far the first leap prediction missed the true $x_j$; $d_0$ measures how far the second leap prediction missed the true $x_0$. Both are non-negative scalars; smaller values indicate better approximation.

Clamping with a minimum distance $\tau$. To prevent the weighting from becoming unboundedly large when predictions happen to match almost perfectly (which would overemphasize noise-aligned trajectories), each distance is clamped with a minimum value $\tau = 0.1$:

$w_{\text{sim}} = \frac{1}{\max(d_j, \tau) + \max(d_0, \tau)}$

What this computes: If both distances are small (good approximations), the denominator is small and $w_{\text{sim}}$ is large—this leap trajectory receives high weight. If either distance is large (poor approximation), the denominator is large and $w_{\text{sim}}$ is small—the trajectory is down-weighted. The $\tau$ clamping caps the maximum possible weight at $1/(2\tau) = 5$, preventing any single trajectory from dominating. The weight is detached via stop_gradient (Equation 13):

$\mathcal{L} = \texttt{stop\_gradient}(w_{\text{sim}}) \cdot \mathcal{L}_{\text{raw}}$

so it scales the loss magnitude without contributing its own gradient terms. The weighting is applied per-sample in the batch.

Why this form: Inverse-distance weighting is a natural choice: trajectories that deviate more from the true path are less reliable and should contribute less to the parameter update. The sum in the denominator (rather than, say, the product) means that if either leap is poor, the weight drops—both connection points must be reasonably accurate for the trajectory to be trusted. The paper's ablation (Figure 4d) shows that considering similarity at only one point ($d_j$ or $d_0$ alone) improves over no weighting (0.4087 and 0.4067 vs. 0.4030 HPSv2.1), but considering both ($d_j$ and $d_0$) achieves the best result (0.4092), confirming that accuracy at both leaps matters.

A deeper interpretation: Trajectory-similarity weighting can be seen as a form of importance sampling for gradient estimates. The leap trajectory is an approximation of the true generation path; the error in this approximation introduces bias in the gradient. By down-weighting high-error trajectories and up-weighting low-error ones, the weighted average gradient is closer to the gradient that would be obtained from the true trajectory (if full backpropagation were feasible). The stop_gradient on $w_{\text{sim}}$ means this is a heuristic weighting rather than a proper importance weight (which would require density ratios), but it is simple and empirically effective.

Summary of Design Choices and Their Justifications

• Two-step leap trajectory over one-step or three-step: hits the sweet spot between reaching early timesteps (requires two leaps for realistic distance per leap) and memory efficiency (three steps use more memory without better results).
• Random selection of $k$ and $j$ over fixed pairs: ensures all timesteps receive gradient updates over the course of training, enabling the "any generation step" property crucial for compositional alignment.
• Gradient discounting with $\alpha = 0.3$ over DRTune's complete removal ($\alpha = 0$): preserves useful cross-timestep coordination signal while controlling gradient magnitude to prevent instability.
• Hinge loss over direct reward maximization: prevents reward hacking by saturating the loss once reward exceeds a threshold, avoiding unbounded optimization toward exploitatively high scores.
• Reward evaluation on $x_0$ over $\hat{x}_{0|j}$: provides faithful reward assessment of the actual generated image rather than a potentially noisy one-step approximation, ensuring the reward signal reflects real output quality.
• Trajectory-similarity weighting with both $d_j$ and $d_0$: up-weights leap trajectories that faithfully approximate the true generation path, reducing the influence of noisy gradient estimates from poor approximations.
• Latent connectors via stop_gradient: allow the forward pass to use correct latents (ensuring accurate velocity predictions) while restricting gradient flow to the leap path (controlling memory and gradient magnitude).

4. Key Insights and Innovations

Innovation 1: The Leap Trajectory as a Compute-Memory Abstraction That Decouples Gradient Path Length from Timestep Reach

The field's prior approach to direct-gradient fine-tuning of diffusion and flow matching models was governed by an implicit assumption: the length of the backpropagation path through the generation trajectory must equal the number of timesteps between the gradient source and the parameter being updated. To update a parameter at an early timestep (e.g., step 20 of 25), the gradient must flow backward through all intermediate timesteps, accumulating memory and Jacobian products proportional to the distance from the final image. This directly couples two properties that practitioners want to control independently: how far back in the trajectory you can reach (which determines whether layout-relevant early steps get updated) and how much memory and numerical risk you incur (which determines whether training is feasible at all). Prior direct-gradient methods accepted this coupling as a hard constraint and responded by restricting reach: ReFL and DRaFT-LV update only one late timestep, while DRTune stops the gradient at model inputs to break the chain but at the cost of discarding cross-timestep gradient information.

LeapAlign's core conceptual move is to sever this coupling entirely by constructing a two-step computational path whose endpoints are arbitrary timesteps from the full trajectory. The mechanism—carving two one-step leaps from a pre-run full trajectory using the rectified flow one-step prediction formula, then snapping predictions back to actual latents via stop_gradient connectors—is described in Section 3. What makes this intellectually distinctive is not the mechanism itself but the abstraction it embodies: the leap trajectory is a surrogate computational graph that approximates the gradient flow of the true trajectory while having constant depth (two model evaluations) regardless of which timesteps are selected. Memory cost becomes independent of reach. Gradient explosion risk is bounded to a two-Jacobian product rather than scaling with trajectory length. This converts a hard architectural impossibility ("you cannot backpropagate from $x_0$ to $x_{20}$ without storing 20 steps of activations") into an implementation choice ("you can reach $x_{20}$ with exactly two steps of activations, at the cost of approximating the gradient path").

This is a fundamental reframing rather than an incremental improvement. Prior work operated within the paradigm of "how do we make backpropagation through long trajectories stable?"—hence gradient checkpointing, stop-gradient, and low-variance re-noising. LeapAlign changes the question to "do we need to backpropagate through the long trajectory at all, or can we construct a shorter one that preserves the essential gradient structure?" The answer is the leap trajectory, and the experimental evidence in Table 2 and Figure 4e confirms that this abstraction works: LeapAlign with full-timestep-range selection $[0, 1]$ achieves a GenEval overall score of 0.7420 versus 0.7107 for the $[0, 1/2]$ variant restricted to later timesteps, demonstrating that reaching early steps matters and that the leap trajectory makes reaching them practical. The constant-memory property is visualized in Figure 4b: two-step leap trajectories use ~90–95% of the memory of one-step but substantially outperform them; three-step trajectories increase memory without providing additional benefit.

A useful comparison: this is analogous to how truncated backpropagation through time (BPTT) in RNN training replaced full-sequence backpropagation—not because truncated gradients are mathematically equivalent, but because they make optimization tractable while preserving sufficient signal. The leap trajectory is a flow-matching-specific form of truncated backpropagation where the truncation is variable (random $k, j$ pairs) rather than fixed, enabling coverage of the full trajectory over many iterations.

Innovation 2: The Nested Gradient as a Necessary Signal, Not Just a Numerical Nuisance

The dominant approach to handling gradient explosion in direct-gradient methods—exemplified by DRTune—is to remove the problematic term entirely by stopping the gradient at the velocity model's input. This treats the nested gradient $\partial v_\theta(x_j)/\partial x_j \cdot \partial v_\theta(x_k)/\partial \theta$ (Equation 8) as an undesirable artifact: it causes instability, so eliminate it. DRTune's design reflects this view: the stop-gradient on $x_t$ is a surgical removal of the cross-timestep Jacobian product, leaving only the single-step gradients $\partial v_\theta(x_t)/\partial \theta$ at each training timestep.

LeapAlign makes the counterargument that the nested gradient is not merely noise or excess magnitude—it carries structurally important information about how parameter changes at early timesteps affect what later timesteps receive as input. This is a diagnostic insight, not obvious from the gradient formula alone. The nested gradient term $j(k-j) \, \partial v_\theta(x_j)/\partial x_j \cdot \partial v_\theta(x_k)/\partial \theta$ captures the chain of dependence: changing $\theta$ shifts the velocity at $k$ → this shifts the predicted $\hat{x}_{j|k}$ → the velocity model at $j$ sees a different input → its output changes → the final image changes. This is the gradient signal for coordination across timesteps—it tells the optimizer that making the model at step $k$ produce a slightly different latent will require the model at step $j$ to adapt accordingly. Removing this term (as DRTune does, effectively setting $\partial v_\theta(x_j)/\partial x_j = 0$) means the model at $k$ is optimized as if changes to its output won't affect how later steps process that output.

The paper's key move is to recharacterize the nested gradient from a stability problem to a signal-to-noise trade-off. The innovation is not the gradient discounting mechanism itself (scaling the Jacobian by blending $\alpha x_j + (1-\alpha) \, \texttt{stop\_gradient}(x_j)$)—that's an implementation detail covered in Section 3. The innovation is the conceptual reframing that leads to discounting rather than removal: the nested gradient is useful but its magnitude needs control, not elimination. This moves the design question from "how do we remove the problematic term?" to "how much of the nested gradient signal can we retain while maintaining stable training?"

The evidence supporting this reframing is multi-layered. First, the direct comparison in Figure 4a: LeapAlign with $\alpha = 0$ (nested gradient completely removed, equivalent to DRTune's approach but within the leap trajectory framework) achieves 0.4064 HPSv2.1; $\alpha = 0.3$ achieves 0.4092; $\alpha = 1.0$ (no discounting) drops to 0.4048. The inverted-U shape confirms that the nested gradient is useful (0.4092 > 0.4064) but only when controlled—a pure signal interpretation would predict monotonic improvement with $\alpha$. Second, the GenEval results (Table 2) show LeapAlign substantially ahead of DRTune on the most compositionally demanding tasks (two-object: 96.46 vs. 93.69; attribute binding: 66.00 vs. 55.50; position: 30.25 vs. 27.50). These tasks specifically require coordinated generation across timesteps—early steps set object positions, later steps refine details and attributes—making the nested gradient's cross-timestep coordination signal particularly valuable. Third, Appendix C (Figure 6) isolates the nested gradient by training only through that term (removing the single-step gradient at $k$): with $\alpha = 0.3$, the model achieves 0.4250 HPSv2.1, exceeding the 0.4208 from $\alpha = 0$, while $\alpha = 1.0$ spikes gradient norms to ~2.7 and degrades to 0.4156. This demonstrates that the nested gradient alone—properly scaled—carries sufficient signal to drive meaningful improvement.

This is a moderate conceptual advance rather than a paradigm shift: the insight that Jacobian products carry useful dependencies is not novel in the broader optimization literature, but its specific application to the diffusion/flow matching fine-tuning setting—and the demonstration that a simple scaling mechanism suffices to extract value from it—was not previously established. The paper resolves an implicit tension in the prior work: DRTune showed that early-step updates are valuable but that full gradients are unstable; LeapAlign shows that the instability can be managed without sacrificing the cross-timestep signal, through a mechanism (gradient discounting) that is both simpler and more effective than the stop-gradient approach.

Innovation 3: Difficulty-Agnostic Timestep Selection as a Mechanism for Universal Step Coverage

This innovation is subtler than the first two but represents an important practical insight. Prior direct-gradient methods that update only late steps (ReFL, DRaFT-LV) do so by design—their architectures hard-code that only timesteps near $t \approx 0$ receive gradients. DRTune can update any step but requires specifying which steps to train ($K$ training timesteps selected from the trajectory). The implicit assumption in these designs is that the practitioner must choose which timesteps to update, either by restricting to a late range (ReFL) or by selecting a fixed schedule (DRTune).

LeapAlign abandons this assumption entirely through uniform random selection of $k$ and $j$ over $[0, 1]$. The timesteps to update are not chosen based on any prior about which portions of the trajectory are most important—they are sampled randomly at each iteration. Over the course of training, every timestep receives gradient updates with equal expected frequency, and every pair of timesteps with $k > j$ has non-zero probability of being selected. Figure 4f confirms that random selection slightly outperforms a fixed-distance variant (0.4092 vs. 0.4084 HPSv2.1), and Figure 4e shows that the full $[0, 1]$ range outperforms the restricted $[0, 1/2]$ range on GenEval (0.7420 vs. 0.7107).

The conceptual shift here is from designing which steps to update to designing a mechanism that covers all steps automatically. This is not merely a convenience—it reflects a hypothesis about what makes early-step fine-tuning effective. The paper's results suggest that the benefit comes not from any single timestep or pair of timesteps but from the model receiving gradient signals at all positions in the trajectory, learning to coordinate its behavior across the full generation process. A fixed schedule (e.g., always training $k$ at the noise end and $j$ at the data end) might over-optimize those specific steps while leaving intermediate steps unchanged, creating inconsistencies where early and late steps produce incompatible latents. Random selection ensures that optimization pressure is distributed, encouraging the model to produce consistent behavior throughout the trajectory.

This connects to a broader principle: when the optimal allocation of a resource (here, gradient updates across timesteps) is unknown and likely problem-dependent, uniform randomization with repeated sampling is a robust default strategy that avoids premature commitment to suboptimal allocations. The paper does not frame it this way explicitly, but the empirical result that random beats fixed supports this interpretation. It is an incremental but practically important insight—it simplifies training (no need to tune timestep schedules) while matching or exceeding the performance of more carefully designed selection strategies.

Innovation 4: Diagnosis of Compositional Alignment as an Early-Step Phenomenon, Validated by Architectural Access

Perhaps the paper's most scientifically valuable contribution is the empirical demonstration that compositional text-image alignment improvements require gradient access to early generation steps, and that methods varying only in their ability to reach those steps show a clean performance hierarchy on compositional benchmarks. This is not a method innovation but a diagnostic finding that clarifies why prior post-training methods struggled on tasks like GenEval and what architectural property is necessary to address them.

The evidence forms a striking gradient of GenEval performance that maps directly to early-step access (Table 2):

The pattern is clean: methods without early-step access (ReFL, DRaFT-LV) cluster around 0.70–0.70 GenEval overall; DRTune with early-step access but removed nested gradient reaches 0.71; LeapAlign with full early-step access and preserved nested gradient reaches 0.74. The gap between ReFL/DRaFT-LV and DRTune (~0.01 overall, but larger on two-object and attribute binding) isolates the value of early-step access. The gap between DRTune and LeapAlign (~0.03 overall, with substantial deltas on two-object +2.77, attribute binding +10.50) isolates the value of the nested gradient.

Figure 5 (Appendix A) reinforces this: LeapAlign's GenEval score rises faster and higher than DRTune's during training, which in turn rises faster than ReFL and DRaFT-LV. The qualitative results in Figure 3 provide visual corroboration: ReFL and DRaFT-LV produce layouts that "remain similar to those of the pretrained model," while LeapAlign "substantially modifies the global structure."

This is a significant diagnostic advance because it provides a causal explanation for a pattern observed across multiple post-training methods. Prior work might have attributed MixGRPO's GenEval advantage (0.7232) to its multi-reward training or SDE formulation, but LeapAlign's single-reward training (HPSv2.1 only) achieves 0.7420, suggesting that the architectural capability to update early steps—combined with preserving cross-timestep gradient signal—is a more fundamental driver of compositional improvements than reward model choice or optimization algorithm. The finding reframes the research question from "which RL or DPO variant works best for alignment?" to "does your method give gradients to early timesteps, and does it preserve their interaction with later timesteps?"

This insight also reconciles why policy-gradient methods like DanceGRPO (0.6775 GenEval) and MixGRPO (0.7232) show a performance range—policy gradients can update early steps (the SDE formulation applies GRPO loss across all timesteps), but the stochasticity and variance of policy gradient estimates make the signal noisier than direct gradients, resulting in less effective early-step optimization. The paper does not make this argument explicitly but the results are consistent with it: direct-gradient methods with early-step access (LeapAlign at 0.7420) outperform policy-gradient methods with early-step access (MixGRPO at 0.7232), which in turn outperform direct-gradient methods without early-step access (ReFL at 0.7011). The hierarchy is early-step access + low-variance gradients > early-step access + high-variance gradients > late-step-only + low-variance gradients.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper uses three prompt sets. For general preference alignment, the primary training set is 50,000 prompts sampled from the HPDv2 dataset (Wu et al., 2023), following prior works (DanceGRPO, MixGRPO). A secondary training set uses prompts from MJHQ-30k (Li et al., 2024) excluding a held-out test split. For compositional alignment, the training set is a 50,000-prompt dataset generated using the official GenEval scripts (Ghosh et al., 2023), spanning six task categories with ratio 7:5:3:1:1:0 for Position, Counting, Attribute Binding, Colors, Two Objects, and Single Object respectively. Test evaluation uses the 400-prompt HPDv2 test set (1,600 images total, 4 per prompt), a 500-prompt random split from MJHQ-30k, and the 553-prompt GenEval test set with 4 images per prompt (2,212 images total).

• Base model(s). All experiments fine-tune FLUX.1-dev (Black Forest Labs, 2024), a state-of-the-art open-source rectified flow matching model. The paper states that the model is chosen because it is "representative of contemporary capabilities" for high-quality text-to-image generation and provides a strong pretrained baseline. Additional experiments in Appendix B fine-tune Stable Diffusion 3.5 Medium (Esser et al., 2024) to verify generality across flow matching architectures. The FLOPs-matched comparison from the executive summary is not present in this paper—this is a method paper focused on post-training alignment quality, not on pretraining-vs-inference compute trade-offs.

• Metrics. For general preference alignment, six automatic evaluators assess the 1,600 generated images from the HPDv2 test set. HPSv2.1 (Wu et al., 2023) serves as both the primary training reward model and an in-domain metric. HPSv3 (Ma et al., 2025), PickScore (Kirstain et al., 2023), and ImageReward (Xu et al., 2023) serve as out-of-domain human preference metrics. UnifiedReward-Alignment and UnifiedReward-IQ (Wang et al., 2025) assess image-text alignment and overall image quality respectively. For compositional alignment, the GenEval benchmark uses rule-based evaluators to automatically determine correctness across six categories (Single Object, Two Object, Counting, Colors, Position, Attribute Binding) and reports an overall weighted score. For MJHQ-30k evaluation, HPSv3 is used as the metric given that HPSv3 is the reward model for those training runs.

• Baselines. The paper compares against two classes of methods. Policy-gradient methods: DanceGRPO (Xue et al., 2025) and MixGRPO (Li et al., 2025), using their official Flux checkpoints from Hugging Face trained on the same prompt sets for the same number of iterations. DanceGRPO uses HPSv2.1 as the reward model; MixGRPO jointly optimizes HPSv2.1, PickScore, and ImageReward. Direct-gradient methods: ReFL (Xu et al., 2023), adapted to Flux from the official implementation; DRaFT-LV (Clark et al., 2023) and DRTune (Wu et al., 2024), reproduced from pseudo-code in their papers due to absence of official implementations. All direct-gradient baselines use HPSv2.1 as the reward model for the main comparison. The pretrained FLUX.1-dev without fine-tuning is also reported.

• Generation budget / compute accounting. The paper measures training cost implicitly through the number of training iterations (300) and online rollout cost: each iteration generates one image at 720×720 resolution using 25 ODE sampling steps with CFG scale 3.5, then constructs the leap trajectory. There is no formal FLOPs accounting or compute-matched comparison—this is a quality-at-fixed-training-budget evaluation, not a compute-efficiency study. All methods are compared after the same number of training iterations (300) on the same prompt sets, with the same number of GPUs (16). Evaluation uses 50 sampling steps with the same resolution and CFG scale. The paper does not report wall-clock time or GPU-hours, and does not measure inference cost relative to baselines (though LeapAlign's training cost is comparable to DRTune's since both run full 25-step rollouts and update two steps per iteration).

• Cross-validation / statistical protocol. The paper does not employ cross-validation, statistical significance testing, or multiple training seeds. Results from a single training run are reported. For GenEval, the variance reduction strategy is generating 4 images per prompt (2,212 total), using deterministic rule-based evaluators to avoid scorer variance. For HPDv2 evaluation, 4 images per prompt are generated (1,600 total) to reduce sampling variance in reward scores. The paper does not report confidence intervals, standard deviations, or min/max across runs for any metric.

Main Quantitative Results

General Preference Alignment on HPDv2

Headline result. LeapAlign achieves the highest average scores across all six evaluators on the HPDv2 test set, including the in-domain metric HPSv2.1 (0.4092) and out-of-domain metrics HPSv3 (15.7678), PickScore (23.7137), UnifiedReward-Alignment (3.4984), UnifiedReward-IQ (3.7244), and ImageReward (1.5104). Table 2 presents the full comparison.

Comparison with policy-gradient methods. Against DanceGRPO—trained with the same HPSv2.1 reward, same prompt set, and same number of iterations—LeapAlign achieves 0.4092 vs. 0.3451 on HPSv2.1 (+18.6%), 15.7678 vs. 14.8336 on HPSv3 (+6.3%), and 23.7137 vs. 23.1186 on PickScore (+2.6%). The gaps are substantial and consistent across evaluators. Against MixGRPO—which jointly optimizes three reward models (HPSv2.1, PickScore, ImageReward) giving it an inherent multi-objective advantage—LeapAlign, trained with only HPSv2.1, achieves 0.4092 vs. 0.3692 on HPSv2.1 (+10.8%) and 23.7137 vs. 23.5184 on PickScore (+0.8%), while remaining competitive on ImageReward (1.5104 vs. 1.6155 for MixGRPO). This is notable because MixGRPO's training explicitly optimizes ImageReward while LeapAlign's does not, yet LeapAlign closes most of the gap on other metrics and exceeds it on HPSv2.1 and PickScore. UnifiedReward scores favor LeapAlign on both alignment (3.4984 vs. 3.4393) and image quality (3.7244 vs. 3.6241).

Comparison with direct-gradient methods. Against the strongest direct-gradient baseline, DRTune (0.3882 HPSv2.1), LeapAlign achieves 0.4092, a gain of +5.4%. The improvement is larger on HPSv3 (15.7678 vs. 15.5606, +1.3%), ImageReward (1.5104 vs. 1.3562, +11.4%), and PickScore (23.7137 vs. 23.5185, +0.8%). Against ReFL (0.3852) and DRaFT-LV (0.3859), LeapAlign's advantage is +6.2% and +6.0% on HPSv2.1 respectively. The hierarchy among direct-gradient methods is consistent with the paper's architectural claims: LeapAlign (early steps + nested gradient) > DRTune (early steps, no nested gradient) > ReFL/DRaFT-LV (late steps only).

Training dynamics. Figure 1a shows the average HPSv2.1 reward during training, computed from generated images $x_0$. LeapAlign's reward curve rises faster and reaches a higher final level than DRTune's. The paper does not plot reward curves for other baselines.

Compositional Alignment on GenEval

Headline result. LeapAlign achieves a GenEval overall score of 0.7420, outperforming all baselines: the pretrained Flux (0.6535), DanceGRPO (0.6775), MixGRPO (0.7232), ReFL (0.7011), DRaFT-LV (0.7024), and DRTune (0.7101). Table 2 presents per-category and overall scores.

Category-level analysis. The largest absolute gains over DRTune occur on Attribute Binding (66.00 vs. 55.50, +10.50 points), Colors (80.59 vs. 76.86, +3.73), Two Objects (96.46 vs. 93.69, +2.77), and Position (30.25 vs. 27.50, +2.75). Counting shows a slight regression (72.50 vs. 73.12, −0.62), and Single Object is at ceiling for all methods (99.38). The pretrained model's weakest categories—Position (19.50) and Attribute Binding (45.25)—show the largest relative improvements from LeapAlign (+10.75 and +20.75 respectively), consistent with the claim that early-step fine-tuning is critical for tasks requiring global layout and object-attribute relationships. Against MixGRPO—the best non-direct-gradient baseline—LeapAlign leads by +0.0188 overall, with the largest gaps on Attribute Binding (66.00 vs. 56.25, +9.75) and Colors (80.59 vs. 80.05, +0.54), while trailing on Counting (72.50 vs. 80.00) and Two Objects (96.46 vs. 93.69) but by smaller margins than its advantages.

Training dynamics for GenEval. Figure 5 (Appendix A) plots GenEval overall score improvement during fine-tuning for the four direct-gradient methods. LeapAlign's curve rises both faster (steeper slope in early iterations) and higher (final value) than DRTune, DRaFT-LV, and ReFL. DRTune, which can update early steps, rises faster than ReFL and DRaFT-LV (which cannot), consistent with the claim that early-step access matters. The gap between LeapAlign and DRTune widens over the course of training, reaching ~0.03 at 300 iterations, suggesting that the nested gradient preservation provides compounding benefits over time.

Qualitative evidence. Figure 3 shows side-by-side generations for six prompts from the GenEval benchmark. For prompts requiring compositional reasoning ("a photo of a yellow bicycle and a red motorcycle," "a photo of a bicycle above a parking meter," "a photo of a suitcase right of a boat," "a photo of green skis and a brown airplane," "a photo of two carrots," "a photo of a red giraffe"), LeapAlign produces images where the specified objects, colors, and spatial relationships are accurately rendered. In contrast, ReFL and DRaFT-LV often fail to place objects correctly or misattribute colors—consistent with their inability to update early layout-determining steps. DRTune shows improvements over ReFL/DRaFT-LV but still exhibits errors (e.g., incorrect object placement in the bicycle-motorcycle prompt). Figure 7 (Appendix H) provides additional qualitative comparisons across all methods.

Robustness Across Reward Models, Prompt Sets, and Model Architectures

Headline result. LeapAlign demonstrates consistent superiority over direct-gradient baselines when the reward model, training prompt set, and base model are varied. This section addresses the concern that the main results might be specific to the HPSv2.1 + HPDv2 + Flux configuration.

Different reward models and prompt sets (Table 3). When fine-tuning Flux with PickScore on the HPDv2 prompt set, LeapAlign achieves 25.7589 PickScore vs. 25.2373 for ReFL (+2.1%), 24.9596 for DRaFT-LV (+3.2%), and 25.1021 for DRTune (+2.6%). The HPSv2.1 scores for this run are also reported (0.4092 for LeapAlign), confirming that training with PickScore generalizes to HPSv2.1 evaluation. When fine-tuning Flux with HPSv3 on the MJHQ-30k prompt set, LeapAlign achieves 12.5855 HPSv3 vs. 11.7642 for ReFL (+7.0%), 11.2701 for DRaFT-LV (+11.7%), and 12.0023 for DRTune (+4.9%). The consistent pattern across both reward models and prompt sets supports the claim that LeapAlign's advantages are not tied to a specific reward function.

Different flow matching model (Table 4, Appendix B). When fine-tuning Stable Diffusion 3.5 Medium with HPSv2.1 on HPDv2, LeapAlign achieves 0.3915 HPSv2.1 vs. 0.3833 for ReFL (+2.1%), 0.3506 for DRaFT-LV (+11.7%), and 0.3828 for DRTune (+2.3%). Out-of-domain metrics also favor LeapAlign: HPSv3 15.5780 vs. 15.2541 for DRTune (+2.1%), PickScore 23.6180 vs. 23.4711 (+0.6%), UnifiedReward-Alignment 3.4896 vs. 3.4738 (+0.5%), UnifiedReward-IQ 3.7182 vs. 3.6667 (+1.4%), and ImageReward 1.4736 vs. 1.4320 (+2.9%). The gap between LeapAlign and DRTune is smaller on SD3.5-M than on Flux (0.3915 vs. 0.3828, +2.3% vs. 0.4092 vs. 0.3882, +5.4% on HPSv2.1), which the paper does not comment on but may reflect differences in model architecture or pretraining quality.

Ablation Studies and Robustness Checks

All ablations in this section use HPSv2.1 as the reward model, Flux as the base model, and the HPDv2 training/test splits, unless otherwise noted. Results are reported at 300 training iterations.

Gradient discounting factor α (Figure 4a): Setting α = 0.3 yields the best HPSv2.1 score (0.4092). Removing the nested gradient entirely (α = 0) achieves 0.4064—still above DRTune's 0.3882 from Table 2, indicating that the leap trajectory design contributes gains independent of nested gradient preservation. Using the full nested gradient without discounting (α = 1.0) degrades performance to 0.4048, confirming that uncontrolled Jacobian products destabilize optimization. The inverted-U shape supports the paper's central claim that the nested gradient is useful but requires magnitude control.

Number of leap steps (Figure 4b): Two-step leap trajectories provide the best trade-off. One step achieves 0.4034 HPSv2.1 (below two-step's 0.4092), while three steps achieve 0.4083 with higher memory usage (~5% increase in normalized memory, from ~0.95 to ~1.00). The paper interprets this as evidence that two steps effectively capture cross-timestep dependencies while keeping memory constant; three steps add computation without benefit. Notably, even the one-step variant (0.4034) outperforms DRTune (0.3882) and ReFL (0.3852) from Table 2, attributed to the combination of reward evaluation on $x_0$ (rather than one-step approximations) and trajectory-similarity weighting.

Reward model input (Figure 4c): Using the actual final image $x_0$ as reward input yields 0.4092 HPSv2.1. Using the one-step approximation $\hat{x}_{0|j}$ with trajectory-similarity weighting at both $x_j$ and $x_0$ drops to 0.4074. Using $\hat{x}_{0|j}$ with weighting only at $x_j$ drops further to 0.4038. This confirms that reward evaluation on the actual generated image provides more reliable supervision than evaluation on a single-step approximation, even when trajectory similarity is accounted for.

Trajectory-similarity weighting scheme (Figure 4d): The full method (measuring similarity at both $x_j$ and $x_0$, Equation 12) achieves 0.4092. Using similarity at $x_j$ only (i.e., weighting by $1/\max(d_j, \tau)$) achieves 0.4087. Using similarity at $x_0$ only achieves 0.4067. Removing weighting entirely (uniform weights) drops to 0.4030. The monotonic improvement with more similarity measurement points supports the claim that leap trajectory fidelity to the true generation path matters for gradient quality. The gap between no weighting (0.4030) and full weighting (0.4092) is +1.5%.

Training timestep range (Figure 4e): Constructing leap trajectories by randomly selecting $k, j$ from the full range $[0, 1]$ achieves a GenEval overall score of 0.7420. Restricting selection to $[0, 1/2]$ (later half of the trajectory, closer to the clean image) drops to 0.7107, a decrease of 0.0313. This directly supports the paper's claim that updating early generation steps (nearer to noise, larger $t$) is critical for compositional alignment tasks in GenEval. The gap is substantial—0.0313 overall GenEval score—making this the most impactful single ablation.

Selection strategy for $k$ and $j$ (Figure 4f): Random selection of $k$ and $j$ achieves 0.4092 HPSv2.1. Fixing the distance between $k$ and $j$ to 1/2 (i.e., always a fixed-size leap) achieves 0.4084, a slight decrease. Random selection is chosen for implementation simplicity and slightly better performance.

Loss threshold λ (Table 5): With λ = 0.55, LeapAlign achieves 0.4092 HPSv2.1 and strong out-of-domain scores (HPSv3 15.7678, PickScore 23.7137, ImageReward 1.5104). Lowering λ to 0.35 reduces HPSv2.1 to 0.3860 with degraded out-of-domain metrics (HPSv3 15.3635, PickScore 23.4735, ImageReward 1.3510)—under-optimization. Raising λ to 0.75 maintains HPSv2.1 at 0.4091 but slightly reduces out-of-domain scores (HPSv3 15.7274, PickScore 23.7061, ImageReward 1.4844). Raising λ further to 0.95 degrades performance across the board (HPSv2.1 0.4023, HPSv3 15.7254, ImageReward 1.3888)—over-optimization causing reward overfitting. The optimal λ = 0.55 balances optimization strength and generalization.

Nested gradient isolation experiment (Appendix C, Figure 6): When the first leap step is trained only through the nested gradient (the single-step gradient at timestep $k$ is removed, Equation 14), α = 0.3 achieves 0.4250 HPSv2.1 with an average gradient norm of ~1.8. α = 0 achieves 0.4208 (no nested gradient contribution). α = 1.0 drops to 0.4156 with gradient norms spiking to ~2.7. This is perhaps the paper's most diagnostically important experiment: it demonstrates that the nested gradient alone—when properly scaled—can drive meaningful optimization (0.4250 exceeds the full LeapAlign result of 0.4092, presumably because isolating the nested gradient avoids interference with single-step gradients, though the paper does not explore this interaction). The gradient norm visualization confirms the mechanism: α directly controls gradient magnitude.

Negative result: ReST-EM revision training. The paper does not contain negative results on alternative training strategies within LeapAlign. The method is presented in its final form with ablations exploring hyperparameter choices and component removals—all of which show monotonic or inverted-U patterns consistent with the paper's claims. There is no experiment where an expected improvement fails to materialize, no alternative design that underperforms for surprising reasons, and no dataset where the method's advantages disappear.

Critical Assessment

Does the evidence support the claim that LeapAlign outperforms prior methods?

The claim that LeapAlign "consistently outperforms state-of-the-art GRPO-based and direct-gradient methods across various metrics" (abstract) is well-supported by Table 2, with the following qualifications. First, the direct-gradient baselines (ReFL, DRaFT-LV, DRTune) were reproduced by the authors from pseudo-code due to the absence of official implementations. This introduces implementation risk—the reproduced versions may not match the performance of properly tuned originals, potentially inflating LeapAlign's apparent advantage. The paper provides training details for these baselines (Appendix E: DRTune with K=2 training timesteps, ReFL with early-stop from last 11 of 25 steps, DRaFT-LV with n=2 re-noising steps) that follow the original papers' recommendations, but the absence of official code means hyperparameters may not be optimal for Flux specifically. Second, MixGRPO jointly optimizes three reward models while LeapAlign uses only one—making the comparison asymmetric in LeapAlign's favor for HPSv2.1 (in-domain) but asymmetric in MixGRPO's favor for ImageReward. LeapAlign remains competitive on ImageReward (1.5104 vs. 1.6155) and wins on PickScore (23.7137 vs. 23.5184), suggesting the advantage is not purely from reward model mismatch. Third, the paper does not report results for Flow-GRPO (Liu et al., 2025), another contemporaneous GRPO-based method, though DanceGRPO and MixGRPO are the most established baselines.

Does the evidence support the claim that early-step access is critical for compositional alignment?

This claim is the best-supported in the paper. The GenEval results show a clean hierarchy: ReFL/DRaFT-LV (late steps only) at ~0.70 overall < DRTune (early steps, no nested gradient) at 0.7101 < LeapAlign (early steps, nested gradient) at 0.7420. The training timestep range ablation (Figure 4e) shows a causal effect: restricting to [0, 1/2] reduces GenEval from 0.7420 to 0.7107, isolating the contribution of early timesteps. The qualitative results (Figure 3) provide visual corroboration that methods without early-step access produce layouts similar to the pretrained model. The improvement on compositionally demanding categories (two-object +9.84 over pretrained, attribute binding +20.75) is substantially larger than on simpler categories (single-object at ceiling), consistent with early steps being more important for layout. However, there is an important caveat: the evidence shows that LeapAlign's specific combination of early-step access and nested gradient preservation improves GenEval, but it does not disentangle whether early-step access alone (without the leap trajectory's other innovations) would suffice. DRTune provides the closest data point (early steps, no nested gradient, GenEval 0.7101), and while it outperforms late-step-only methods, it trails LeapAlign by 0.0319, suggesting that early-step access is necessary but not sufficient—how those early steps are updated matters too.

Does the evidence support the claim that the nested gradient is useful and that gradient discounting extracts value from it?

The evidence for this claim is multi-layered and generally convincing, with one notable limitation. The α ablation (Figure 4a) shows that α = 0.3 > α = 0 > α = 1.0, supporting the utility of a properly scaled nested gradient. The Appendix C experiment (Figure 6) cleanly isolates the nested gradient's contribution: it alone can drive optimization (0.4250 with α = 0.3) and its removal hurts performance (0.4208 with α = 0). The gradient norm visualization confirms the mechanism—α directly controls magnitude.

However, the paper does not provide the most incisive ablation: what happens if you apply gradient discounting to DRTune? DRTune's stop-gradient approach sets α = 0 by construction. The paper's implicit argument is that DRTune at α = 0 is suboptimal because α = 0.3 is better, but this comparison is confounded by LeapAlign's other design choices (leap trajectory construction, reward on $x_0$, trajectory-similarity weighting). The isolated comparison is LeapAlign at α = 0 (0.4064 HPSv2.1, Figure 4a) vs. DRTune (0.3882, Table 2). The gap of 0.0182 cannot be attributed to the nested gradient (both remove it), so it must come from LeapAlign's other components. This means we cannot directly measure how much of DRTune's underperformance is from missing nested gradient vs. from other factors, and thus the paper's claim that gradient discounting specifically addresses DRTune's limitation is not directly isolated. A cleaner experiment would be to implement gradient discounting within DRTune's framework (keeping its training timestep selection and multi-step updates but adding the α-blended input) to measure the marginal contribution of nested gradient preservation while controlling for other differences.

Does the evidence support generalization across reward models, prompt sets, and architectures?

The paper provides evidence across three reward models (HPSv2.1, PickScore, HPSv3), two prompt sets (HPDv2, MJHQ-30k), and two base models (Flux, SD3.5-M). This is good breadth for a method paper. However, the evidence has limitations. For SD3.5-M (Table 4), only direct-gradient baselines are compared—no policy-gradient baselines (DanceGRPO, MixGRPO) are included, so we cannot assess whether LeapAlign's advantage over GRPO methods generalizes to other architectures. The MJHQ-30k experiments (Table 3, right columns) use the same training dataset for all methods but only report numbers for LeapAlign in the main comparison—the paper reports HPSv3 scores for all direct-gradient methods but does not include policy-gradient baselines or full table comparisons as with HPDv2. All experiments use a single resolution (720×720 for Flux, 512×512 for SD3.5-M), a single CFG scale (3.5), and a single number of inference steps (50 for evaluation). How LeapAlign's advantage scales with these parameters is unexplored.

Absent experiments and genuine weaknesses

Several experiments would have meaningfully strengthened the paper:

1. Multiple training seeds / error bars. All results are from single training runs. With 300 iterations on 16 GPUs, variance across runs could be non-trivial. Without error bars, we cannot assess whether LeapAlign's 0.4092 vs. DRTune's 0.3882 on HPSv2.1 (a +5.4% relative improvement) is statistically reliable or within run-to-run noise. Given that the paper's central claim is that LeapAlign "consistently outperforms" prior methods, this omission is significant.

2. Human evaluation. All metrics are automatic evaluators, which are themselves imperfect proxies for human preference. The paper uses six evaluators to mitigate evaluator-specific biases, but there is no direct human judgment study, leaving open the possibility of reward model overfitting (the methods are optimized against the same class of models used for evaluation). Figure 1a shows that training rewards for LeapAlign increase substantially—a human evaluation would help confirm that this translates to perceptible quality improvements rather than reward hacking.

3. Scaling behavior with training iterations and model size. The paper trains for a fixed 300 iterations. Does LeapAlign's advantage grow, shrink, or remain constant with more training? Does it hold for smaller models (where memory constraints are less binding and full-trajectory backpropagation might become feasible) or larger models (where the nested gradient might be more important)? These questions are unaddressed.

4. Comparison with full-trajectory backpropagation at reduced resolution or step count. The paper argues that full-trajectory backpropagation is infeasible due to memory and gradient explosion. But what if the resolution is reduced (e.g., 256×256) or the number of steps is reduced (e.g., 10 steps)? At what point does the leap trajectory approximation become unnecessary? The paper does not establish the boundary conditions where its method is actually required vs. where simpler approaches would suffice.

5. Ablation on the number of full-trajectory rollout steps. The paper uses 25-step rollouts for training. How does LeapAlign perform with fewer rollout steps (which would produce noisier images and potentially less reliable reward signals) or more steps (which might improve reward quality but increase training cost)? The interaction between rollout step count and leap trajectory fidelity is unexplored.

6. FLOPs or wall-clock comparison. While the paper explicitly does not position itself as a compute-efficiency method (unlike the reference paper in the example, which provides detailed FLOPs-matched comparisons), the absence of any timing or memory data makes it difficult to assess practical trade-offs. How much faster or slower is LeapAlign training compared to DRTune or ReFL? The leap trajectory construction adds the cost of running the full 25-step trajectory (shared with DRTune) plus two velocity evaluations with gradients enabled, plus the reward backward pass through two steps. DRTune also runs the full trajectory and evaluates velocity at its K=2 training timesteps. ReFL runs the full trajectory and evaluates one leap. DanceGRPO requires maintaining multiple trajectory samples per prompt for GRPO's group-relative advantage computation. Without timing data, the practical cost-benefit of LeapAlign is unclear.

7. Analysis of failure modes. The paper reports Counting as a category where LeapAlign slightly regresses vs. DRTune (72.50 vs. 73.12, Table 2). There is no analysis or hypothesis for why. Similarly, the paper does not show examples where LeapAlign performs worse than baselines—the qualitative results are all cherry-picked successes. Understanding when and why the method fails would be as informative as knowing when it succeeds.

6. Limitations and Trade-offs

The Leap Trajectory Approximates the True Gradient Path — and the Approximation Error is Never Quantified

The assumption or constraint. LeapAlign substitutes the true gradient path through the full generation trajectory with a surrogate path through a two-step leap trajectory. The one-step leap predictions $\hat{x}_{j|k} = x_k - (k-j)v_\theta(x_k)$ and $\hat{x}_{0|j} = x_j - j v_\theta(x_j)$ are approximations — they would equal the true latents only if the flow matching model perfectly learned the straight-line velocity field and if the numerical ODE solver produced exactly straight trajectories. In practice, the model is imperfect and the trajectory may deviate. The paper introduces latent connectors ($x_j^{\text{(leap)}} = \hat{x}_{j|k} + \texttt{stop\_gradient}(x_j - \hat{x}_{j|k})$) to align the predicted and actual latents in the forward pass, but the backward pass still propagates gradients through the approximate path — the gradient $\partial \hat{x}_{j|k} / \partial \theta$ reflects how parameters affect the leap prediction, not how they affect the true $x_j$ reached by the ODE solver. The difference $x_j - \hat{x}_{j|k}$ is the approximation error; it is detached from the computation graph, meaning the optimizer receives no gradient signal about this error.

The consequence. The gradient that LeapAlign provides to the optimizer is a biased estimate of the true gradient through the full trajectory. The bias is proportional to the leap prediction error, which the paper never measures or bounds. When the error is large — which likely occurs when $k - j$ is large (a long leap spanning many ODE steps) or when the velocity model is inaccurate at the selected timesteps — the gradient direction may be substantially misaligned with the direction that would actually improve the full generation process. The trajectory-similarity weighting (Section 4.5) down-weights trajectories with large errors, but this only reduces their influence on the update — it does not correct the bias within any single trajectory. A trajectory with error $d_j = 0.05$ and $d_0 = 0.05$ receives a weight of $1/(0.05 + 0.05) = 10$ (clamped to $1/(0.1 + 0.1) = 5$ maximum), but the gradient direction within that trajectory may still point in a suboptimal direction due to the approximation.

Critically, the paper does not establish when the approximation is good enough. The gap between $\hat{x}_{j|k}$ and $x_j$ depends on the model's training quality, the specific timesteps $(k, j)$, the prompt, and the random seed. Without quantifying this error or characterizing its distribution across trajectories, a practitioner cannot predict whether LeapAlign will provide reliable gradients for their specific model, prompt distribution, or timestep range. There is also a deeper theoretical concern: the method optimizes parameters using gradients from the leap trajectory, but the reward is evaluated on $x_0$ from the full trajectory. This creates a mismatch — the loss $\max(0, \lambda - r(x_0))$ depends on the full trajectory's output, but the gradient $\partial \mathcal{L} / \partial \theta$ is computed as if $x_0$ were produced by the leap trajectory. This is a form of surrogate objective optimization, where the true objective and the surrogate share a minimum only when the leap predictions are exact. When they are not, the surrogate gradient may point in a direction that improves the leap trajectory's reward (if it were actually used for generation) but not the full trajectory's reward.

What evidence exists in the paper. The paper does not directly measure the leap prediction error, report its distribution, or correlate it with downstream performance. The trajectory-similarity weighting ablation (Figure 4d) shows that up-weighting low-error trajectories improves performance (0.4092 with full weighting vs. 0.4030 without, a +1.5% relative gain), confirming that error matters — trajectories with larger deviations provide worse gradients. But the paper does not report what the actual distances $d_j$ and $d_0$ look like in practice (e.g., mean, variance, distribution shape), how they vary with $k - j$, or what fraction of trajectories have errors large enough to be harmful. Figure 4b shows that one-step leap trajectories (a single leap from $x_k$ to $\hat{x}_{0|k}$, which compounds the approximation error by making an even larger jump) underperform two-step trajectories (0.4034 vs. 0.4092), consistent with larger leaps having larger errors. But this is indirect — the one-step variant also has fewer parameters receiving gradients and no cross-timestep interaction, so the performance drop cannot be attributed solely to approximation error.

Mitigation status. The paper partially mitigates this through trajectory-similarity weighting (Section 4.5) and through the two-step design (which keeps per-leap distance smaller than a single leap from $k$ to $0$). The $\tau = 0.1$ clamp prevents zero-error trajectories from receiving infinite weight. However, there is no mechanism to reduce the approximation error itself (e.g., by iteratively refining the leap prediction or using a higher-order approximation), only to down-weight trajectories where it is large. The paper does not suggest future work on quantifying or reducing leap prediction error, instead focusing on extensions to video generation and non-differentiable rewards.

---

Training Cost and Memory Are Constant Per-Iteration — but the Full-Trajectory Rollout Cost Is Unaccounted for in Comparisons With Late-Step Methods

The assumption or constraint. LeapAlign compares against ReFL and DRaFT-LV — methods that update only one late timestep per iteration — and claims superior performance. However, LeapAlign and DRTune both incur the cost of running a full 25-step ODE trajectory at every training iteration to provide the raw material for leap trajectory construction. ReFL similarly runs a full trajectory (to reach $t_{\min}$ before making its one-step leap), but DRaFT-LV's cost model is different: it updates only the last step and does not require running the full trajectory with gradients stored. The paper provides no wall-clock time measurements, no GPU-hour comparisons, and no FLOPs accounting for any method. The comparison is purely a quality-at-fixed-iteration-count evaluation: all methods are trained for 300 iterations on 16 GPUs, and the resulting model quality is compared.

The consequence. The headline result — LeapAlign achieves 0.4092 HPSv2.1 vs. 0.3859 for DRaFT-LV — does not account for any difference in per-iteration cost. If DRaFT-LV iterations are significantly faster (because it only stores activations for one step rather than running a full trajectory, even without gradients, plus two gradient-enabled velocity evaluations and backpropagation through two leaps), then a fair comparison at equal wall-clock time or equal GPU-hours might give DRaFT-LV more iterations to close the gap. The paper's implicit claim that LeapAlign is more effective per unit of compute is not supported by any compute-normalized comparison.

The more subtle cost is the online rollout. At each iteration, LeapAlign generates a full 720×720 image at 25 ODE steps with CFG scale 3.5. This generation is done without a computation graph (gradients disabled), but it still consumes GPU cycles that are not part of the gradient computation. For ReFL, the full trajectory up to $t_{\min}$ is similarly generated without gradients, but $t_{\min}$ is near the end (last 11 of 25 steps), so only ~14–15 steps are run, versus LeapAlign's full 25. For DRaFT-LV, only the last step requires the actual image — the paper is unclear whether the full trajectory is run or whether a short-cut initialization is used. For policy-gradient methods like DanceGRPO, the cost structure is entirely different (multiple trajectory samples per prompt, SDE sampling, advantage computation), making direct GPU-hour comparison essential but absent.

What evidence exists in the paper. None. There are no timing measurements, no memory usage comparisons beyond the normalized bar chart in Figure 4b comparing 1/2/3-step leap trajectories within LeapAlign (which shows 3-step uses ~5% more memory than 2-step, and 1-step uses ~85% of 2-step's memory — but these are relative, unlabeled, and not compared to any baseline). The paper does not report training GPU-hours, images-per-second during training, or inference latency. Appendix E mentions training uses 16 GPUs but does not specify GPU type or interconnect. For a method paper where the central claim is making gradient propagation to early steps "practical," the absence of cost measurements is a significant gap for any practitioner deciding whether to adopt it over computationally lighter alternatives.

Mitigation status. Not addressed. The paper positions itself as a quality improvement over prior work, not as a cost-efficiency improvement (the abstract claims "consistently outperforms," not "more efficiently achieves"). The cost of full-trajectory rollouts is an inherent requirement of the method — without the full trajectory, there are no latents $x_k, x_j, x_0$ to carve the leap trajectory from — so it cannot be eliminated without fundamentally changing the approach. The paper does not flag this as a limitation or suggest cost-reduction strategies (e.g., shorter rollouts, cached trajectories, amortization across multiple leap pairs from the same rollout).

---

All Evaluations Use Automated Metrics — No Human Judgment Study Confirms Perceptual Improvements or Rules Out Reward Overfitting

The assumption or constraint. Every reported result — HPSv2.1, HPSv3, PickScore, ImageReward, UnifiedReward, and the GenEval rule-based evaluators — comes from an automatic scoring system. The training reward models (HPSv2.1, PickScore, HPSv3) themselves belong to the same family of CLIP-based or VLM-based preference models as the evaluation metrics. This creates a direct risk of reward overfitting: the model may learn to produce images that exploit the reward model's inductive biases — achieving high scores without genuine perceptual quality improvements that a human would notice or prefer. The paper acknowledges this risk in Section 4.4 when justifying the hinge loss ("Directly maximizing reward values often leads to reward hacking"), but uses the hinge loss to address score magnitude overfitting (preventing unboundedly high rewards) rather than representation overfitting (the model learning reward-model-specific artifacts).

The consequence. We cannot be confident that LeapAlign's improvements over baselines — e.g., 0.4092 vs. 0.3882 on HPSv2.1 (+5.4%) — correspond to improvements a human would perceive or prefer. The risk is asymmetric: LeapAlign's stronger reward optimization (higher and faster-increasing reward curves in Figure 1a) could indicate either genuine quality improvement or more aggressive exploitation of the reward model. The fact that LeapAlign also improves on out-of-domain metrics (HPSv3, PickScore, ImageReward, UnifiedReward) mitigates this concern somewhat — overfitting to HPSv2.1 would not necessarily transfer to PickScore or HPSv3, which have different architectures and training data. But all these evaluators share architectural DNA (CLIP backbones or ViT-based vision encoders fine-tuned on human preference data), and a model that learns to produce CLIP-friendly features may score well across multiple CLIP-derived metrics without human-noticeable improvements.

The GenEval results partially bypass this concern because the evaluators are rule-based (object detectors + color classifiers + spatial relation checkers) rather than learned preference models. LeapAlign's strong GenEval performance (0.7420 vs. 0.7101 for DRTune) suggests that the improvements in two-object, color, position, and attribute binding accuracy reflect genuine compositional capability improvements — rule-based evaluators are harder to "hack" without actually placing objects correctly. But GenEval is a narrow benchmark (6 categories, 553 prompts) and does not capture overall image quality, aesthetics, or fidelity — there could be a trade-off where improved composition comes at the cost of degraded visual quality that automatic metrics miss.

What evidence exists in the paper. Zero. The paper includes no user study, no human preference evaluation, and no qualitative assessment by human raters. The qualitative figures (Figures 3, 7, 8, 9) are paper-curated examples showing LeapAlign's improvements — they are illustrative, not systematically sampled or evaluated. Figure 3's examples compare methods but are presented without any human judgment data (e.g., "in a blind test, X% of raters preferred LeapAlign over DRTune"). The UnifiedReward-IQ metric (3.7244 for LeapAlign vs. 3.6679 for DRTune) is the closest proxy for overall image quality, but it is still an automated evaluator trained on human preference data with unknown calibration.

Mitigation status. Not addressed. The paper does not acknowledge the absence of human evaluation as a limitation. The use of six diverse automatic evaluators is the paper's implicit mitigation strategy — if LeapAlign wins across multiple independently developed evaluators, it is less likely to be purely reward-hacking a single metric. But this is a correlational argument, not a causal one. Human evaluation is the established standard in preference alignment research (it is the "H" in RLHF), and its complete absence in a paper claiming to improve "alignment with human preferences" (abstract) is a substantive gap.

---

Generalization Is Tested Only on Two Architectures, One Task Family, and One Resolution — the Boundaries of Applicability Are Unknown

The assumption or constraint. The paper evaluates LeapAlign on exactly two flow matching models — FLUX.1-dev (primary) and Stable Diffusion 3.5 Medium (Appendix B) — both fine-tuned on the exact same task: text-to-image generation from short prompts. All training and evaluation uses a single resolution (720×720 for Flux, 512×512 for SD3.5-M) and a single inference-time configuration (50 evaluation steps, CFG scale 3.5). The reward models, prompt distributions, and evaluation benchmarks all come from the human preference alignment literature in text-to-image generation. The paper does not test on: (a) other generation tasks (image editing, inpainting, super-resolution, style transfer), (b) other modalities (video, 3D, audio) despite mentioning video as future work, (c) other model families (Diffusion Transformers without rectified flow, pixel-space diffusion models, autoregressive image generators), or (d) different resolutions or aspect ratios. The paper's claim in Section 5 that LeapAlign can accommodate "any differentiable reward model" is supported (tested with HPSv2.1, PickScore, HPSv3), but the deeper claim that the method works for flow matching models generally is based on exactly two models from different families (Flux DiT, SD3.5-M MMDiT).

The consequence. A practitioner considering LeapAlign for a model, task, or resolution outside the tested configuration has no empirical basis to predict whether the method will work, what hyperparameters to use, or what failure modes to expect. The leap trajectory depends on the one-step leap prediction formula (Equation 3), which assumes a rectified flow scheduler ($\alpha_t = 1 - t, \beta_t = t$). While this covers current state-of-the-art models (Flux, SD3, SD3.5), models using other schedulers (DDPM cosine schedule, VP-SDE, EDM schedulers) would require re-deriving the leap prediction formula and may have different approximation error characteristics. The method also assumes the reward model is differentiable — this excludes many real-world reward signals (human binary feedback, non-differentiable classifiers, rule-based metrics that operate on rendered pixels rather than latent features, API-based evaluators). The paper acknowledges this explicitly in Section 5 ("Extending LeapAlign to non-differentiable rewards, perhaps via differentiable value models [4], is future work"), but it is a binding constraint for current deployment.

The SD3.5-M results (Table 4) show a narrower gap between LeapAlign and DRTune than on Flux (0.3915 vs. 0.3828 on HPSv2.1 for SD3.5-M, a +2.3% gap, vs. 0.4092 vs. 0.3882, a +5.4% gap for Flux). The paper does not comment on this, but it raises the possibility that LeapAlign's advantage is partially model-dependent — perhaps Flux's architecture or pretraining quality makes it benefit more from the nested gradient, or perhaps DRTune's simpler approach is more competitive on other architectures. Without testing on more models, we cannot determine whether the observed advantage is robust or specific to the Flux family.

The absence of video generation experiments is particularly notable given that the paper's conclusion promises this as immediate future work. Video generation involves longer trajectories (more sampling steps), higher memory pressure, and more complex spatiotemporal composition — all of which would stress-test LeapAlign's core claims about early-step access and constant memory. A video diffusion model requiring 50–100 steps would make the leap trajectory's two-step bound even more impactful (or expose that the approximation error becomes untenable over very long leaps).

What evidence exists in the paper. Table 4 (Appendix B) provides the only cross-architecture comparison, confirming that LeapAlign generalizes to SD3.5-M for the same task and reward model. Table 3 shows generalization across reward models (PickScore, HPSv3) and training sets (HPDv2, MJHQ-30k) but only on Flux. All experiments use the same inference configuration. The paper does not ablate resolution, CFG scale, or number of inference steps. It does not test on non-rectified-flow models, non-image tasks, or non-differentiable rewards. There are no experiments with different prompt lengths, prompt languages, or artistic style specifications.

Mitigation status. Partial. The paper provides two distinct model architectures and three reward models, which is more than typical for a method paper in this subfield. It explicitly acknowledges two limitations: non-differentiable rewards as future work (Section 5) and one-step/few-step models as out of scope (Section 5: "it is more important to design fine-tuning methods for multi-step models"). It does not acknowledge the lack of video, editing, or multimodal evaluation or the resolution/inference-configuration generalization gap. The SD3.5-M experiment is a genuine but incomplete attempt at demonstrating generality — cross-task and cross-modality tests would be needed to establish broad applicability.

---

The Nested Gradient's Contribution Is Not Cleanly Isolated from LeapAlign's Other Design Choices

The assumption or constraint. The paper's central narrative is that DRTune's key weakness is removing the nested gradient ($\partial v_\theta(x_j)/\partial x_j \cdot \partial v_\theta(x_k)/\partial \theta$), and that LeapAlign's gradient discounting preserves this signal, explaining LeapAlign's superior performance. However, LeapAlign differs from DRTune along multiple axes simultaneously: (1) it constructs two-step leap trajectories rather than using DRTune's $K$-step training with stop-gradients at each input, (2) it evaluates the reward on $x_0$ rather than on a one-step approximation, (3) it applies trajectory-similarity weighting, (4) it uses random timestep selection over $[0,1]$ rather than DRTune's fixed training timestep schedule, (5) it applies gradient discounting only at the second leap step rather than at every step. The comparison between LeapAlign and DRTune (Table 2: 0.4092 vs. 0.3882 on HPSv2.1; 0.7420 vs. 0.7101 on GenEval) thus conflates the nested gradient contribution with all other design differences. The paper does not provide the cleanest ablation: taking DRTune's exact algorithm and adding gradient discounting (the $\alpha$-blended input from Equation 9) to measure the marginal improvement from preserving the nested gradient while controlling for all other factors.

The consequence. We cannot determine how much of LeapAlign's advantage over DRTune is attributable to the nested gradient specifically, versus the leap trajectory construction, the $x_0$ reward evaluation, the random timestep selection, or the trajectory-similarity weighting. This matters because each of these components has different computational costs and complexity. If the nested gradient contributes only a small fraction of the improvement — and the leap trajectory and trajectory-similarity weighting contribute most of it — then a simpler method that adds trajectory-similarity weighting to DRTune might achieve most of LeapAlign's benefit with lower implementation complexity.

The available evidence provides partial answers but not a definitive decomposition. Comparing LeapAlign at $\alpha = 0$ (no nested gradient, 0.4064 HPSv2.1, Figure 4a) with DRTune (0.3882, Table 2) gives a gap of 0.0182 that cannot be attributed to the nested gradient (both remove it), so it must come from other design differences. This 0.0182 gap is actually larger than the 0.0028 gap between $\alpha = 0$ and $\alpha = 0.3$ within LeapAlign (0.4064 vs. 0.4092, Figure 4a). This suggests that LeapAlign's non-nested-gradient innovations (leap trajectory, $x_0$ reward evaluation, trajectory-similarity weighting) account for roughly 6.5× more of the improvement over DRTune than the nested gradient does (0.0182 ÷ 0.0028 ≈ 6.5). If this is correct, the paper's emphasis on the nested gradient as the key differentiator from DRTune is misleading — the leap trajectory design is actually the primary driver of gains, and the nested gradient is a smaller (though still positive) contributor.

What evidence exists in the paper. The $\alpha$ ablation (Figure 4a) isolates the nested gradient's effect within LeapAlign: going from $\alpha = 0$ to $\alpha = 0.3$ improves HPSv2.1 from 0.4064 to 0.4092 (+0.0028). The Appendix C experiment (Figure 6) further isolates the nested gradient by training only through that term, showing it alone can drive optimization to 0.4250. The missing experiment is the symmetric one: what is DRTune's performance with $\alpha = 0.3$ gradient discounting added? That experiment would isolate the nested gradient's contribution in DRTune's framework, allowing a clean measurement of how much of the LeapAlign-vs-DRTune gap is genuinely from nested gradient preservation vs. from the leap trajectory and other components. Without it, the paper's attributions remain suggestive but not causally verified.

Mitigation status. Not addressed. The paper frames the nested gradient argument primarily as a theoretical insight ("DRTune stops the gradient … discarding substantial gradient flow," Section 1) rather than a quantitative claim about fractional contribution. The $\alpha$ ablation and Appendix C provide supporting evidence that the nested gradient is useful on the margin, which is a valid finding. But the paper does not attempt to decompose the LeapAlign-vs-DRTune gap into component contributions, leaving open the possibility that a simpler combination of design choices could achieve comparable results without gradient discounting.

---

The Method Assumes a Differentiable Reward Model Trained on Images from the Base Model's Distribution — and Section 6.3 Shows This Distribution Shifts During Fine-Tuning

The assumption or constraint. LeapAlign uses a fixed, pre-trained reward model (HPSv2.1, PickScore, or HPSv3) throughout fine-tuning. As the Flow model's parameters change, the distribution of generated images $x_0$ shifts away from the distribution on which the reward model was trained. The reward model — typically a CLIP variant fine-tuned on human preference pairs from pre-trained text-to-image models — may not provide accurate or well-calibrated scores on images from the fine-tuned distribution. This is the standard distribution shift problem in reward-model-based fine-tuning: the reward model is a proxy for human preference that becomes less reliable as the policy (the Flow model) diverges from the distribution the reward model was trained on.

The paper's own evidence hints at this problem. Figure 1a shows that LeapAlign's training reward (computed by HPSv2.1 on generated images) increases substantially during fine-tuning — from the pretrained model's baseline (~0.31 according to Table 2, though the curve starts near 0.30 in Figure 1a) to a final value above 0.45. Meanwhile, the out-of-domain PickScore and HPSv3 improvements are more modest. If the reward model were perfectly aligned with true human preference, training reward and out-of-domain metrics would move together. The divergence pattern — training reward increases more than generalization metrics — is consistent with some degree of reward overfitting.

The consequence. There is no guarantee that continued training would continue to improve genuine image quality or human preference alignment. The hinge loss with threshold $\lambda$ provides a hard ceiling (once reward exceeds $\lambda$, loss is zero, and no further gradients flow), which partially mitigates this by preventing unbounded reward optimization. But the choice of $\lambda$ itself is critical: set too low, the model stops improving before reaching its potential; set too high, the model pushes into regions where the reward model is poorly calibrated, risking reward hacking. The paper tunes $\lambda$ on the evaluation metrics (Table 5: 0.55 is best), which means $\lambda$ is effectively chosen based on the same out-of-domain metrics used for final evaluation — a mild form of test-set leakage through hyperparameter selection, though the degree is limited since $\lambda$ is a single scalar.

More subtly, the trajectory-similarity weighting uses $x_0$ (the full-trajectory image) to compute $d_0 = \texttt{mean}(|x_0 - \hat{x}_{0|j}|)$ and evaluates the reward on $x_0$. But $x_0$ is constantly changing as the Flow model trains. The distances $d_j$ and $d_0$ that define trajectory similarity are computed using the current model, meaning the weighting adapts during training. This is not explicitly accounted for — the weighting is designed to account for leap prediction accuracy, but the leap prediction accuracy itself changes as parameters update. The paper provides no analysis of how $d_j$ and $d_0$ evolve over training, whether the weighting becomes more or less aggressive over time, or whether $\tau = 0.1$ remains appropriate throughout training.

What evidence exists in the paper. Figure 1a shows the training reward curve for LeapAlign and DRTune, confirming that in-domain reward increases substantially (+0.15 for LeapAlign over 300 iterations). Table 6.2 shows the gap between in-domain improvement (+0.1014 over pretrained on HPSv2.1, from 0.3078 to 0.4092) and out-of-domain improvement on ImageReward (+0.4649, from 1.0455 to 1.5104, a +44% relative gain, versus no baseline numbers reported as relative improvements). The paper does not report reward model confidence, calibration, or agreement with human judgments at any point during training. The $\lambda$ ablation (Table 5) shows that higher $\lambda$ values (0.75, 0.95) degrade out-of-domain metrics while maintaining or slightly reducing in-domain scores — classic reward overfitting. The fact that LeapAlign outperforms baselines on all out-of-domain metrics suggests that overfitting is not catastrophic at 300 iterations with $\lambda = 0.55$, but it does not guarantee that the improvements over baselines are genuine rather than partially driven by shared inductive biases across CLIP-derived evaluators.

Mitigation status. Partially addressed through the hinge loss (prevents unbounded reward maximization) and the use of multiple out-of-domain evaluators (provides cross-checks against single-metric overfitting). The paper does not discuss distribution shift in the reward model explicitly, does not suggest techniques to address it (e.g., periodically re-training the reward model on on-policy data, using an ensemble of reward models, applying a KL penalty to stay close to the base distribution), and does not characterize how reward calibration degrades over training. The GenEval rule-based evaluators are immune to reward model distribution shift and show strong LeapAlign improvements — this is the strongest evidence that the gains are not purely reward hacking, but GenEval is a narrow compositional benchmark, not a holistic quality assessment.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper makes a diagnostic and methodological intervention rather than a paradigm shift. Its primary contribution to the field's intellectual landscape is to reframe the direct-gradient fine-tuning problem from one of memory management to one of gradient path abstraction. Prior work — ReFL, DRaFT-LV, DRTune — all accepted an implicit constraint: the gradient path from the final image to the parameter being updated must trace the same timestep sequence as the generation trajectory. The design question was "how do we make backpropagation through this long path stable?" leading to solutions that either shortened the path (ReFL: one step; DRaFT-LV: last step only) or severed the chain rule at each junction (DRTune: stop-gradient at model inputs). LeapAlign argues that this constraint is self-imposed and unnecessary: you can construct a surrogate gradient path — the two-step leap trajectory — that approximates the gradient flow of the full trajectory without sharing its length, enabling backpropagation from the final image to any timestep with constant memory and bounded gradient magnitude.

This reframing shifts the conversation in two ways. First, it opens the design space for direct-gradient methods beyond the binary choice of "update late steps only" or "remove cross-timestep gradients." The leap trajectory shows that gradient propagation depth and timestep reach are decoupled design axes — you can reach arbitrarily early steps with arbitrarily shallow computation graphs, at the cost of approximation error. This is a generalizable insight: any generative model with a one-step prediction property (not just rectified flow, but potentially consistency models, flow matching with other schedulers, or even diffusion models with Tweedie's formula) could construct surrogate gradient paths of controllable depth, trading off gradient fidelity against memory and stability. Second, it elevates the nested gradient from nuisance to signal. DRTune treated the Jacobian product $\partial v_\theta(x_j)/\partial x_j \cdot \partial v_\theta(x_k)/\partial \theta$ as a stability hazard to be eliminated. LeapAlign shows that this term — which captures how changes at early steps propagate through later steps' inputs — contains useful optimization signal, and that controlling its magnitude (via discounting) is superior to discarding it. This recharacterizes a design decision that was previously seen as purely defensive (stability preservation) as a signal-to-noise trade-off that can be tuned.

The paper also provides a diagnostic resolution to a question that the prior direct-gradient literature could not fully answer: why do methods that update only late steps plateau on compositional benchmarks? The GenEval results establish a causal chain: compositional alignment (spatial relationships, attribute binding, multi-object layout) requires modifying early generation steps; methods that cannot reach those steps (ReFL, DRaFT-LV) cannot substantially improve composition; methods that reach early steps but discard cross-timestep dependencies (DRTune) improve but leave performance on the table; methods that reach early steps and preserve those dependencies (LeapAlign) achieve the strongest compositional gains. This hierarchy — GenEval overall scores of 0.6535 (pretrained) < 0.7011–0.7024 (late-step only) < 0.7101 (early steps, no nested gradient) < 0.7420 (early steps + nested gradient) — is clean and monotonic. It explains why prior post-training methods, even sophisticated ones like MixGRPO (0.7232), underperform on composition. MixGRPO uses policy gradients which can in principle update early steps, but the stochasticity and variance of REINFORCE-style estimates make the signal noisier than exact gradients, and the GRPO formulation's reliance on group-relative advantages may dilute per-timestep credit assignment. LeapAlign's advantage over MixGRPO is strongest on the most compositionally demanding categories: attribute binding (66.00 vs. 56.25, +9.75) and position (30.25 vs. 24.25, +6.00). These are precisely the tasks where precise coordination between early layout-determining steps and later attribute-rendering steps matters most, and where the nested gradient's cross-timestep signal would be most valuable.

Research directions that become more attractive as a result of this work:

• Surrogate gradient path design in generative models. The leap trajectory demonstrates the viability of training on computational graphs that are structurally different from the inference graph. This opens the door to investigating other surrogate constructions: higher-order leap predictions (e.g., using two Taylor expansion terms instead of one to reduce approximation error), three-step trajectories with different connection topologies, or adaptive leap step sizes that concentrate gradient capacity where the approximation error is smallest. The paper's trajectory-similarity weighting can be seen as a first step toward importance-sampling these surrogate paths — more principled weighting using density ratios or control variates could further reduce bias.

• Verifier robustness and reward model distribution shift in image generation. The paper shows that in-domain reward increases substantially during fine-tuning (Figure 1a), that $\lambda$ must be carefully tuned to balance optimization strength and generalization (Table 5), and that out-of-domain gains are more modest than in-domain gains. This is the same verifier over-optimization pattern that has been extensively studied in the LLM RLHF literature and in the test-time compute scaling paper discussed in the executive summary. LeapAlign's framework makes this phenomenon newly tractable for image generation because the gradient path is short and controllable — one could systematically study how reward model calibration degrades as the leap trajectory's endpoint distribution diverges from the pretraining distribution, and design mitigation strategies (KL penalties, on-policy reward model updates, ensemble verification) that integrate cleanly with the leap trajectory gradient path.

Research directions that become less attractive:

• Sophisticated search or RL algorithms for flow matching post-training that do not exploit differentiability. The paper shows that a relatively simple direct-gradient method with the right gradient path design outperforms DanceGRPO and MixGRPO — both recently developed and carefully tuned GRPO variants — on both general preference and compositional alignment. If the gap persists in further comparisons (with human evaluation, on more models, at scale), it would suggest that the primary bottleneck in flow matching post-training is not the optimization algorithm but whether the gradient signal reaches early steps with cross-timestep dependencies intact. Research effort might then shift away from developing new policy-gradient variants for flow matching and toward improving surrogate gradient path fidelity, reward model robustness under distribution shift, and cheap difficulty/error estimation for adaptive leap construction.

• Full-trajectory backpropagation as a research target. The paper implicitly argues that the memory and stability costs of full-trajectory backpropagation are unaffordable and will remain so as models scale, and that surrogate paths like the leap trajectory are the practical path forward. If the leap trajectory's 4× efficiency over best-of-N baseline (a different paper) or its consistent 5–20% improvement over DRTune holds at larger scales, it reduces the motivation to pursue gradient checkpointing, reversible architectures, or mixed-precision techniques to make full backpropagation feasible — the surrogate approach is simpler and already covers the important use case (early-step updates for layout).

Follow-Up Research This Work Enables

Quantifying and reducing leap prediction error to characterize where LeapAlign's gradient estimates are reliable. The paper never measures the distribution of approximation errors $d_j = \texttt{mean}(|x_j - \hat{x}_{j|k}|)$ and $d_0 = \texttt{mean}(|x_0 - \hat{x}_{0|j}|)$ that form the basis for trajectory-similarity weighting. A detailed diagnostic study would measure these distances across the full range of $(k, j)$ pairs, for different models (Flux vs. SD3.5), different training stages (pretrained vs. mid-fine-tuning vs. converged), and different prompt types (simple vs. compositional). The key question: is there a regime — defined by $k - j$ distance, model training quality, or prompt complexity — where the leap prediction error becomes large enough that the gradient direction is meaningfully biased, and does trajectory-similarity weighting adequately compensate? The study would correlate per-trajectory error with per-trajectory gradient cosine similarity to the (expensive) full-trajectory gradient, establishing the boundary conditions for LeapAlign's reliability. If the approximation error concentrates in specific timestep regions (e.g., very early steps where the latent is nearly pure noise), adaptive leap step sizes or multi-step trajectories in those regions could improve gradient fidelity at marginal cost.

Combining LeapAlign with on-policy reward model fine-tuning to combat distribution shift. Table 5 shows that higher $\lambda$ values degrade out-of-domain metrics — a signature of reward overfitting as the Flow model's output distribution drifts. A natural extension is to periodically fine-tune the reward model on images generated by the current Flow model, using human preference labels or an ensemble of existing reward models as a proxy supervision signal. LeapAlign's two-step gradient path makes this computationally viable: the reward model itself could be fine-tuned using the same leap trajectory construction (the reward model's gradient with respect to its own parameters can be backpropagated through the two-step path, requiring only two stored activations). This would create a co-training loop where the Flow model and reward model are both updated from the same trajectories, with the reward model adapting to the shifting image distribution. A concrete experiment: train LeapAlign for 300 iterations with HPSv2.1, then fine-tune HPSv2.1 on 10,000 images from the partially trained model (with preference labels from an ensemble of HPSv3, PickScore, and ImageReward as a proxy for human judgment), then resume LeapAlign training with the updated reward model, and measure whether the out-of-domain gap narrows compared to continued training with the frozen reward model.

Stress-testing the leap trajectory on video generation to determine whether the constant-memory property becomes decisive at long trajectory lengths. The paper concludes by promising video generation as immediate future work, but this is more than a casual extension — it is the regime where LeapAlign's core architectural claim (constant memory regardless of timestep reach) should provide the largest relative advantage. Video diffusion models typically use 50–100 sampling steps to generate 16–64 frames, producing trajectories 10–100× longer than the 25-step image trajectories studied here. Full-trajectory backpropagation is completely infeasible at this scale, making late-step-only methods severely limited (they can only refine the final few frames' details) and making DRTune's step-independent gradient (no cross-timestep signal) potentially more damaging because temporal consistency errors compound over many frames. The concrete experiment: fine-tune a video flow matching model (e.g., a Flux-derived video DiT) on a compositional video prompt benchmark (VBench, EvalCrafter) with LeapAlign, comparing against ReFL (late frames only), DRTune, and a GRPO baseline (DanceGRPO extended to video). The key measurement is whether LeapAlign's advantage over DRTune grows with trajectory length — if the nested gradient's cross-timestep signal becomes more important when the trajectory is longer (because errors in early frames propagate through more subsequent frames), the relative gain should increase with the number of video frames, providing clean evidence for the nested gradient's causal role.

Replacing automatic evaluators with human preference judgments to establish whether LeapAlign's metric gains translate to perceptual improvements. The paper evaluates entirely with automatic metrics, which share architectural biases with the reward models used for training. A rigorous human evaluation would settle whether LeapAlign's 0.4092 vs. 0.3882 on HPSv2.1 (a 5.4% relative increase) and its 0.7420 vs. 0.7101 on GenEval (a 4.5% increase) correspond to images that humans actually prefer. The experiment should use the standard two-alternative forced choice (2AFC) protocol: for 200–500 prompts sampled from HPDv2 and GenEval, present raters with side-by-side images from LeapAlign and DRTune (or MixGRPO, or pretrained Flux) and ask which better matches the prompt and which has higher overall quality. The study should sample prompts stratified by GenEval category to test whether LeapAlign's largest metric gains (attribute binding +20.75, position +10.75) correspond to the largest human preference gaps. A null result — humans do not prefer LeapAlign over baselines despite the metric gap — would indicate that the metrics are sufficiently correlated with each other and with the training reward that they collectively measure reward model overfitting rather than genuine improvement. A positive result — humans prefer LeapAlign, and the preference margin correlates with per-category metric gains — would validate the paper's central claim and strongly motivate further work on surrogate gradient path methods.

Investigating whether the one-step leap prediction formula generalizes to non-rectified-flow schedulers, and whether alternative surrogate paths are needed. LeapAlign critically depends on Equation 3: $\hat{x}_{j|k} = x_k - (k - j) v_\theta(x_k)$, which assumes a rectified flow scheduler $\alpha_t = 1 - t, \beta_t = t$. Many diffusion and flow matching models use other schedulers — DDPM cosine, EDM $\sigma(t)$ parameterization, VP-SDE — where the one-step prediction formula involves time-dependent coefficients (as derived in Appendix F, Equation 20). The leap trajectory construction should be applicable to these schedulers by using the general formula (Equation 20) instead of the simplified rectified flow version. A systematic study across schedulers would: (a) derive the leap prediction formula for each scheduler family, (b) measure the approximation error as a function of scheduler and leap distance, (c) train LeapAlign on models with different schedulers (e.g., SDXL with its DDPM scheduler vs. SD3 with rectified flow vs. a hypothetical Flux variant with an EDM scheduler), and (d) determine whether the rectified flow's straight-line trajectories provide a meaningful advantage for leap prediction accuracy. If non-rectified-flow schedulers produce significantly larger approximation errors, that would circumscribe LeapAlign's applicability to the rectified flow family (currently Flux, SD3, SD3.5) and motivate research into scheduler-adaptive leap prediction or higher-order approximations. If the errors are comparable, LeapAlign generalizes broadly to the diffusion model family.

Ablating LeapAlign's individual components within the DRTune framework to decompose the performance gap. The paper shows LeapAlign outperforms DRTune (0.4092 vs. 0.3882 HPSv2.1, 0.7420 vs. 0.7101 GenEval) but does not isolate which of LeapAlign's innovations — leap trajectory construction, $x_0$ reward evaluation, trajectory-similarity weighting, random timestep selection, gradient discounting — accounts for what fraction of the gap. A component-level ablation within DRTune's codebase would answer this. Concretely, start with DRTune (K=2 training timesteps, stop-gradient at each input) and incrementally add: (1) trajectory-similarity weighting on DRTune's training steps, (2) reward evaluation on $x_0$ instead of a one-step approximation, (3) randomized instead of fixed timestep selection, (4) gradient discounting (replacing stop-gradient with the $\alpha$-blended input at the second training step), (5) the full leap trajectory construction. This decomposition would identify which components provide the largest marginal gains and whether the leap trajectory is necessary or whether a less invasive modification to DRTune (e.g., adding trajectory-similarity weighting and reward-on-$x_0$) captures most of the benefit. The results would guide practitioners on what to implement first and would clarify whether the paper's emphasis on the nested gradient is empirically justified or whether the leap trajectory's other properties (better reward signal, adaptive weighting) are the primary drivers.

Practical Applications and Downstream Use Cases

Fine-tuning open-source Flow models for compositional text-to-image applications. The most immediate use case is for practitioners deploying Flux or SD3.5 in applications where compositional accuracy matters — e-commerce product visualization ("a red sofa to the left of a blue armchair"), architectural rendering ("a glass building with a park in front"), or instructional illustration ("a diagram showing steps 1, 2, and 3 from left to right"). LeapAlign's GenEval results translate directly: on the two-object task, LeapAlign achieves 96.46% accuracy vs. 86.62% for pretrained Flux, meaning roughly 1 in 10 prompts that previously failed (two objects not both present or incorrectly arranged) now succeed. On attribute binding — assigning the correct color or attribute to each object — the improvement is from 45.25% to 66.00%, roughly halving the error rate. For a production system generating thousands of images daily where compositional errors require human filtering or regeneration, these improvements have direct throughput and cost implications. The training recipe is fully specified (HPSv2.1 reward, 300 iterations on 16 GPUs, $\lambda = 0.55$, $\alpha = 0.3$, $\tau = 0.1$``) and uses publicly available models and datasets (Flux weights, HPDv2 prompts), making it immediately reproducible.

Cost-efficient preference alignment for smaller Flow models on consumer hardware. The paper's SD3.5-M experiment (Table 4) shows LeapAlign generalizes to a medium-sized model at lower resolution (512×512, 200 iterations). The constant-memory property of the leap trajectory means that LeapAlign's GPU memory requirement is independent of how far back in the trajectory it reaches — unlike full-trajectory backpropagation, which would scale with the number of steps. This makes LeapAlign feasible on GPUs with limited VRAM (e.g., consumer 24GB cards) where storing 50-step activation chains is impossible. A practitioner with a single 24GB GPU could fine-tune SD3.5-M on a custom prompt dataset (e.g., product photos, artistic styles, specific object types) using a CLIP-based reward model fine-tuned on their domain, achieving compositional and preference improvements without access to datacenter-scale hardware. The 200-iteration training run at 512×512 with batch size adjusted for single-GPU would complete in hours rather than days, making rapid iteration practical.

Improving compositional accuracy in self-improvement and synthetic data pipelines. A common workflow in production image generation is to use a model to generate many candidates, filter them with a quality estimator (reward model, aesthetic scorer, or object detector), and fine-tune on the high-scoring subset. LeapAlign can be integrated directly into this loop: instead of standard fine-tuning on selected images, use LeapAlign's leap trajectory construction with the quality estimator as the reward model, so the model receives gradient signals about which generation steps to modify to improve quality, not just which final images are preferred. The GenEval results are particularly relevant here — many synthetic data pipelines fail on compositional prompts (e.g., generating training data for an object detection model requires accurate multi-object layouts), and LeapAlign's strong two-object (96.46%) and attribute binding (66.00%) performance directly addresses this failure mode. The trajectory-similarity weighting mechanism would naturally adapt: prompts where the model consistently produces compositional errors would have larger $d_0$ distances (the clean image $x_0$ doesn't match the one-step approximation well, indicating the model is uncertain or producing inconsistent trajectories), causing LeapAlign to down-weight noisy gradients on those prompts and focus learning on trajectories where the gradient signal is more reliable.

When to Prefer This Method

The paper explicitly positions LeapAlign against both GRPO-based methods (DanceGRPO, MixGRPO) and prior direct-gradient methods (ReFL, DRaFT-LV, DRTune) and provides the evidence base for clear trade-off conditions:

• Prefer LeapAlign over DRTune (and other direct-gradient methods) when the training objective requires updating early generation steps and cross-timestep dependencies matter. The GenEval results in Table 2 establish this condition concretely: for compositional tasks (two-object, colors, position, attribute binding), LeapAlign substantially outperforms DRTune (attribute binding 66.00 vs. 55.50, position 30.25 vs. 27.50). For tasks where late-step-only updates suffice — which may include simple aesthetic quality improvements or style transfer — the gap narrows and a simpler method like ReFL or DRTune may be adequate. The training timestep range ablation (Figure 4e) provides a direct diagnostic: if restricting LeapAlign's timestep range to $[0, 1/2]$ (later steps only) causes a large GenEval drop relative to the full $[0, 1]$ range, the application likely benefits from early-step updates and LeapAlign is preferred.

• Prefer LeapAlign over GRPO-based methods when training with a single, differentiable reward model and stable, low-variance optimization is desired. LeapAlign uses exact gradients through the leap trajectory, avoiding the sampling noise and advantage estimation variance inherent to policy gradient methods. The reward curves (Figure 1a) and GenEval training dynamics (Figure 5) show LeapAlign's reward and accuracy increase faster and more smoothly than DRTune's; policy-gradient methods typically require more iterations or larger batch sizes to reduce variance. For applications where compute budget for training is limited (e.g., iterating on a custom reward model with many experiments), LeapAlign's gradient efficiency is an advantage — it achieves higher final quality (0.4092 HPSv2.1) than MixGRPO (0.3692) and DanceGRPO (0.3451) in the same number of iterations with the same prompt set. However, if the reward model is non-differentiable (e.g., human binary feedback, API-based evaluators, non-differentiable object detectors), GRPO methods or DPO variants remain the only option — LeapAlign's direct-gradient approach is inapplicable by design.

• Prefer LeapAlign when training on consumer or single-GPU hardware where full-trajectory backpropagation is infeasible but early-step updates are desired. The constant-memory property of the two-step leap trajectory means that GPU memory usage is bounded regardless of how many generation steps the model uses during inference. This is not directly compared to baselines in the paper (Figure 4b only compares 1/2/3-step variants within LeapAlign), but it follows from the architecture: LeapAlign stores activations for exactly two velocity model evaluations plus the reward backward pass, independent of whether the full trajectory has 25, 50, or 100 steps. DRTune would similarly have constant memory (it also uses a fixed number of training steps), but DRTune removes the nested gradient, sacrificing the cross-timestep signal that LeapAlign preserves. For applications on memory-constrained hardware where compositional accuracy matters — fine-tuning an open-source model for a specific product photography domain, for instance — LeapAlign provides early-step access that ReFL and DRaFT-LV cannot, while preserving gradient information that DRTune discards.