Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

ArXiv: 2209.03003

🎯 Pitch

You can generate high-quality images in a single neural network evaluation—no adversarial training, no diffusion noise—by simply learning to follow straight-line paths between noise and data. This paper shows that recursively 'rectifying' such flows straightens them enough that one Euler step suffices, achieving state-of-the-art one-step generation on CIFAR-10 without a GAN.

---

1. Executive Summary

This paper introduces rectified flow, a simple ODE-based approach that learns to transport between two empirical distributions by fitting velocity fields to follow straight-line paths connecting source and target samples as directly as possible, providing a unified solution for both generative modeling (Gaussian noise → images) and domain transfer (human faces → cat faces). The core mechanism is solving a nonlinear least squares regression that predicts the direction vector (X₁ − X₀) from interpolated points Xₜ = tX₁ + (1−t)X₀ — termed rectification — which rewires crossing interpolation paths into non-crossing flow trajectories with provably non-increasing convex transport costs, while a recursive reflow procedure straightens the resulting paths so they can be simulated accurately with coarse time discretization. Training flow models (DDPM++ U-Nets) on CIFAR-10 achieves an FID of 4.85 and recall of 0.51 with a single Euler step after 2-reflow plus distillation — outperforming all prior one-step ODE and U-Net-based GAN methods — while requiring only 2–5 Euler steps for high-quality generation on 256×256 resolution datasets (LSUN, CelebA-HQ, AFHQ cat). The method yields these gains through a purely ODE-based training objective that avoids adversarial instability and the subtle hyperparameter choices of diffusion models, establishing that straight-path transport admits both theoretical guarantees and state-of-the-art practical performance without requiring diffusion noise or SDE machinery.

2. Context and Motivation

The Fundamental Tension in Unsupervised Learning: We Lack Paired Data

The central problem this paper tackles is deceptively simple but profoundly constraining across virtually all forms of unsupervised learning: we rarely observe paired input/output examples that would allow us to train transport maps using standard supervised regression. This gap manifests in two canonical problems that the paper unifies under a single framework:

Generative modeling: We want to map samples from a simple noise distribution $\pi_0$ (e.g., a standard Gaussian) to samples from a complex data distribution $\pi_1$ (e.g., natural images). If we had paired examples $(z, x)$ where $z \sim \pi_0$ and $x$ is the "correct" image corresponding to that noise vector, we could simply train a neural network to predict $x$ from $z$ via supervised regression. But such pairing does not exist — by definition, we only have unpaired samples from each distribution separately.

Domain transfer: We want to map images from one domain $\pi_0$ (e.g., human faces) to another domain $\pi_1$ (e.g., cat faces) while preserving identity-relevant features. Again, we lack paired examples — we have collections of human faces and cat faces, but no ground-truth mapping telling us which cat face "corresponds to" which human face.

The paper frames both problems as instances of the transport mapping problem (Section 1): given empirical observations from two distributions $\pi_0$ and $\pi_1$ on $\mathbb{R}^d$, find a transport map $T: \mathbb{R}^d \to \mathbb{R}^d$ such that $Z_1 = T(Z_0) \sim \pi_1$ when $Z_0 \sim \pi_0$. The challenge is that the coupling $(Z_0, Z_1)$ — the pairing specifying which source point maps to which target point — is fundamentally underdetermined by the marginal distributions alone. Every generative modeling and domain transfer method must somehow resolve this ambiguity, and the choice of how to do so defines the method's properties.

Why This Matters: The Inference Cost Bottleneck in Modern Generative Models

The importance of this problem has been dramatically amplified by the recent dominance of continuous-time generative models, particularly score-based diffusion models [71–73] and denoising diffusion probabilistic models (DDPM) [23]. These methods have achieved a watershed moment in generative modeling: they surpass GANs on image generation in both quality and diversity, without suffering from the training instability and mode collapse that plagued adversarial approaches [12, 53, 61, 64]. The training procedure is elegantly simple — essentially a denoising objective that can be optimized stably at scale — which has enabled their successful deployment in massive text-to-image models with astonishing results.

However, this success comes with a critical practical drawback that the paper identifies as a key motivation:

"a key drawback of continuous-times models is the high computational cost in inference time: drawing a single point (e.g., image) requires to solve the ODE/SDE with a numerical solver that needs to repeatedly call the expensive neural drift function."

Concretely, standard diffusion models require hundreds to thousands of neural network evaluations to generate a single image. DDPM [23] uses 1,000 steps; the SDE-based methods in [73] use 2,000 steps. Even when converted to ODEs via the probability flow ODE [73] or DDIM [70] techniques, these models still require dozens to hundreds of steps (Table 1 shows RK45 solvers using 110–176 adaptive steps for CIFAR-10). This makes deployment expensive and latency-sensitive applications impractical.

The paper therefore identifies a clear engineering imperative: bridge the gap between one-step models (like GANs) and continuous-time models by making ODE-based generative models that can be simulated accurately with very few time steps — ideally one. This is not merely an incremental speedup; it represents a qualitative shift in the deployability of high-quality generative models.

Prior Approaches and Their Shortcomings

The paper positions its contribution against four major classes of existing methods, each with distinct limitations:

1. Traditional One-Step Models: GANs, VAEs, and Normalizing Flows

GANs [19, 4, 43] have historically been the dominant approach for high-quality image generation, producing results in a single forward pass. However, the paper identifies well-known failure modes:

"GANs are known to suffer from numerically instability and mode collapse issues, and require substantial engineering efforts and human tuning, which often do not transfer well across different model architecture and datasets."

The training procedure relies on a minimax game between generator and discriminator, which is fundamentally less stable than optimization-based training and requires careful balancing. Mode collapse — where the generator produces only a subset of the data distribution — remains a persistent problem that limits diversity.

VAEs [32] and normalizing flows [62, 13, 14] avoid adversarial training by using maximum likelihood estimation (MLE). However, MLE introduces its own constraints:

"MLE tends to be intractable for complex models, and hence requires approximate variational or Monte Carlo inference techniques such as those used in variational auto-encoders (VAE), or special model structures such as normalizing flow and auto-regressive models, to yield tractable likelihood, causing difficult trade-offs between expressive power and computational cost."

VAEs typically restrict themselves to simple conditional Gaussian distributions to make the variational approximation tractable, limiting expressiveness. Normalizing flows require specially designed invertible architectures with expensive Jacobian determinant calculations, constraining model design. Both approaches sacrifice architectural flexibility for tractable likelihood.

2. Continuous-Time SDE Models (Diffusion): Training vs. Inference Asymmetry

Score-based SDE models [71–73] and DDPM [23] solved the training stability problem by using a simple denoising objective that avoids both adversarial training and likelihood intractability. The training procedure is elegant: gradually corrupt data with noise following a prescribed forward diffusion process, then learn to reverse this process by predicting the noise (or score function) at each noise level.

However, the paper identifies two specific shortcomings beyond the inference cost issue:

Hyperparameter complexity: Despite the conceptual simplicity of the training objective, these methods embed substantial complexity in their design space:

"the existing denoising diffusion techniques require substantial hyper-parameter search in an involved design space and are still poorly understood both empirically and theoretically [29]."

The choice of noise schedule ($\sigma_t$), drift coefficients ($\eta_t$), and the relationship between $\alpha_t$ and $\beta_t$ that govern how data and noise are mixed over time — all of these are consequences of the SDE derivation from Ornstein-Uhlenbeck processes, not principled choices for the ODE that ultimately gets used at inference time. The paper's Figure 5 demonstrates empirically that these choices lead to curved trajectories and non-uniform progression speed, both of which harm performance when using coarse time discretization.

Unnecessary SDE machinery for ODE inference: A key insight driving this work is that if the goal is ultimately to use an ODE for fast inference (as in probability flow ODEs and DDIM), then deriving that ODE as a byproduct of an SDE introduces unnecessary constraints:

"the learning of ODEs can be approached directly in a very simple way, allowing us to identify the canonical case of linear rectified flow and open the door of further improvements with flexible and decoupled choices of the interpolation curves $X_t$ and initial distributions $\pi_0$."

The paper argues that the SDE-to-ODE conversion inherited from [73] forces choices of $\alpha_t, \beta_t$ that are suboptimal for ODE simulation — specifically, exponential schedules that concentrate updates in later time steps and coupling between $\alpha_t$ and $\beta_t$ (e.g., $\beta_t = \sqrt{1-\alpha_t^2}$ for VP ODE) that produces curved rather than straight trajectories.

3. ODE-Based Methods: MLE Approach and Probability Flow ODEs

The MLE approach for neural ODEs [6] trains continuous-time models by maximizing the likelihood of the ODE's terminal distribution under the data. While avoiding architectural constraints of discrete normalizing flows through the instantaneous change of variables formula, this approach has fundamental limitations:

• Computational expense: Training requires repeated ODE simulation with backpropagation through the solver, which is costly at scale and suffers from gradient vanishing/exploding problems.
• Theoretical under-specification: MLE only constrains the terminal distribution $\text{Law}(Z_1)$ to match $\pi_1$; there are infinitely many ODEs achieving this with different intermediate paths. The learned paths are implicitly determined by initialization and optimizer hyperparameters rather than by design. Some works [54, 55] add transport cost regularization to favor shorter paths, but this adds complexity without a principled resolution.

Probability flow ODEs (PF-ODEs) [73] and DDIM [70] addressed the MLE limitations by providing a training objective that does not require ODE simulation during training. However, the paper argues these methods are unnecessarily constrained by their SDE origins:

"because PF-ODEs and DDIM were derived as the side product of learning the mathematically more involved diffusion/SDE models, their theories and algorithm forms were made unnecessarily restrictive and complicated."

Specifically, the paper identifies three issues (Section 2.3.1):

1. Non-straight paths: The $\beta_t$ choices in VP ODE ($\beta_t = \sqrt{1-\alpha_t^2}$) and sub-VP ODE ($\beta_t = 1-\alpha_t^2$) produce curved trajectories that cannot be straightened by reflow, fundamentally limiting few-step performance.

2. Non-uniform speed: The exponential $\alpha_t$ in (7), inherited from the Ornstein-Uhlenbeck process derivation, causes the flow to move slowly in early time steps ($t \lesssim 0.5$) and concentrate most updates in the later phase, as shown in Figure 5. This makes large-step simulation particularly inaccurate.

3. Coupled choices of initial distribution and interpolation: In these methods, the initial distribution $\pi_0$ is forced to be approximately Gaussian with variance determined by $\beta_0$, rather than being freely chosen. For VE ODE, $\pi_0$ must be $\mathcal{N}(0, \sigma_{\max}^2 I)$ with $\sigma_{\max}$ set as the maximum pairwise distance in the training data — an unnatural constraint that the paper argues is unnecessary.

The paper's re-analysis in Proposition 3.11 shows that PF-ODEs are instances of a more general nonlinear rectified flow framework with $X_t = \alpha_t X_1 + \beta_t \xi$, but that the standard choices of $\alpha_t, \beta_t$ from the SDE literature are suboptimal for the ODE setting. This motivates the paper's recommendation of the canonical linear interpolation $X_t = tX_1 + (1-t)X_0$ as default.

4. Domain Transfer Methods: Separate Methodologies from Generation

A significant gap the paper identifies is that generative modeling and domain transfer are typically treated as separate problems requiring distinct methods:

"In existing approaches, generative modeling and domain transfer are typically treated separately. It often requires to extend or customize a generative learning techniques to solve domain transfer problems; see e.g., Cycle GAN [100] and diffusion-based image-to-image translation [e.g., 75, 97]."

CycleGAN [100] uses adversarial losses with a cycle-consistency constraint, requiring two generators and two discriminators trained in a minimax game. Diffusion-based translation methods adapt pretrained diffusion models through guidance mechanisms or latent space manipulations, each requiring problem-specific engineering. Optimal transport (OT) [85, 2, 15, 59] provides a unified mathematical framework, but:

"the existing OT techniques are slow for problems with high dimensional and large volumes of data [59]. Furthermore, as the transport costs do not perfectly align with the actual learning performance, methods that faithfully find the optimal transport maps do not necessarily have better learning performance [34]."

This creates an opportunity: a method that handles both generation and transfer with essentially the same algorithm — simply by changing what $\pi_0$ is (Gaussian noise for generation, source domain data for transfer) — would represent a significant unification.

Reconciling Conflicting Desiderata

The paper's motivation can be understood as an attempt to simultaneously achieve four properties that existing methods trade off against each other:

Rectified flow aims to achieve all four: stable regression-based training (like diffusion), straight paths enabling one-step inference (like GANs), a unified framework for generation and transfer (unlike existing methods), and principled, geometry-aware path selection (the linear interpolation as Euclidean geodesic).

The Core Insight: Straight Paths Are Both Optimal and Computationally Ideal

The paper's key motivating insight is that straight-line paths are uniquely privileged for transport problems:

1. Geometric optimality: Straight lines are the shortest paths between two points in Euclidean space. A flow following straight paths minimizes the distance traveled, which connects naturally to optimal transport theory.

2. Computational ideal: A perfectly straight flow satisfies $Z_t = tZ_1 + (1-t)Z_0$, meaning the velocity $v(Z_t, t) = Z_1 - Z_0$ is constant along each trajectory. Such a flow can be simulated exactly with a single Euler step: $Z_1 = Z_0 + v(Z_0, 0)$. This eliminates time-discretization error entirely.

3. The challenge: The linear interpolation $X_t = tX_1 + (1-t)X_0$ is already straight, but it is non-causal — computing the update $dX_t = (X_1 - X_0)dt$ requires knowing the final point $X_1$, which is not available at inference time. The rectified flow solves this by learning a velocity field $v(X_t, t)$ that predicts the direction $X_1 - X_0$ from only the current state $X_t$ and time $t$, effectively "causalizing" the straight paths.

This insight — that we can achieve straight-path transport without requiring future information through a simple regression objective — is what distinguishes rectified flow from both the diffusion-based approaches (which follow curved paths by construction) and optimal transport methods (which target specific cost functions rather than the geometric property of straightness).

How the Paper Positions Itself

The paper positions rectified flow as a fundamental re-thinking of how to learn ODE-based transport models, rather than an incremental improvement within existing frameworks. Several aspects of this positioning are notable:

Independence from diffusion noise: The paper explicitly questions whether diffusion noise is necessary for the success of recent generative models:

"the role of diffusion mechanisms should be re-examined and clearly decoupled from the other merits of denoising diffusion models. The success of the denoising diffusion models may be mainly attributed to the simple and stable optimization-based training procedure that allows us to avoid the instability issues and the need of case-by-case tuning of GANs, rather than the presence of diffusion noises."

This is a provocative claim: it suggests that the key innovation of diffusion models was not the stochastic process itself, but rather the regression-based training objective. By demonstrating that an ODE trained with a similarly simple regression objective can match or exceed diffusion performance, the paper makes the case that the SDE machinery is dispensable for the core task of learning transport maps.

A new axis of improvement: straightness via reflow: The paper introduces the concept that flows can be iteratively straightened through the reflow procedure — a novel mechanism with no analog in prior generative modeling literature. Each reflow step takes the coupling produced by the previous flow (which is deterministic and has lower transport cost) and learns a new flow on that coupling's linear interpolation, producing an even straighter flow. The theoretical guarantee that straightness improves at rate $O(1/K)$ (Theorem 3.7) provides principled motivation for this recursive procedure.

Unification through the transport mapping problem: By framing both generation and transfer as instances of finding a transport map between two distributions, the paper provides a single algorithm that applies to both without modification. This contrasts sharply with the current landscape where generation (noise → data) and translation (data → data) are handled by different method families with different training procedures and theoretical foundations.

Practical, not just theoretical: While the paper provides theoretical results on transport cost reduction and straightening, it emphasizes that the straightness property has direct computational consequences — fewer sampling steps, faster inference, simpler deployment — making it a theoretically grounded practical contribution rather than an abstract mathematical exercise.

3. Technical Approach

3.1 Reader Orientation

This is fundamentally a methodology paper with strong theoretical backing that introduces a new way to train continuous-time generative models. The core idea is deceptively simple: learn an ODE whose velocity field follows the straight-line direction between randomly paired source and target samples, then iteratively refine this ODE to make its trajectories increasingly straight, enabling accurate simulation with very few time-discretization steps. The system being built is a neural ODE model $dZ_t = v(Z_t, t)dt$ that can transport samples from any source distribution $\pi_0$ to any target distribution $\pi_1$ by solving a simple least-squares regression problem — no adversarial training, no MLE, no SDE machinery required. The solution's "shape" is: pair points randomly from the two distributions, learn to predict the straight-line direction connecting them at every intermediate point, then simulate the resulting ODE to produce a new (deterministic, lower-transport-cost) coupling, and recursively apply this procedure to obtain flows with increasingly straight paths that approach one-step models.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components that interact in a pipeline:

1. Data coupling $(X_0, X_1)$ — an arbitrary pairing of samples from source distribution $\pi_0$ and target distribution $\pi_1$, typically independent draws $\pi_0 \times \pi_1$ since we lack meaningful paired observations. This defines the "roads" that the flow will attempt to follow.

2. Linear interpolation process $X_t = tX_1 + (1-t)X_0$ for $t \in [0, 1]$ — for each coupled pair, this traces the straight-line path connecting $X_0$ to $X_1$. The velocity along this path is $\dot{X}_t = X_1 - X_0$, which is constant but non-causal because computing it requires knowing the endpoint $X_1$.

3. Velocity field $v_\theta(z, t)$ — a neural network (or non-parametric estimator) that takes a position $z \in \mathbb{R}^d$ and time $t \in [0, 1]$ and predicts the expected direction $X_1 - X_0$ of all linear interpolation paths passing through $z$ at time $t$. This "causalizes" the non-causal linear interpolation: at inference time, the network only sees the current state $Z_t$, not the future endpoint.

4. Rectified flow ODE $dZ_t = v_\theta(Z_t, t)dt$ — the ODE whose drift is the learned velocity field. Starting from $Z_0 \sim \pi_0$ and simulating forward to $t=1$ produces $Z_1 \sim \pi_1$ (by the marginal preserving property). The key property: this ODE avoids self-intersections, "rewiring" the crossing paths of the linear interpolation into non-crossing flow trajectories.

5. Reflow procedure — a recursive outer loop: simulate the current rectified flow to obtain new pairs $(Z_0, Z_1)$ (which are deterministically coupled via the ODE), then train a new rectified flow on these pairs. Each reflow step straightens the trajectories and reduces convex transport costs. After $k$ reflow steps, the $k$-rectified flow can optionally be distilled into a one-step model $\hat{T}(z_0) = z_0 + v(z_0, 0)$.

Information flows as follows: random pairs $(X_0, X_1) \sim \pi_0 \times \pi_1$ → train $v_\theta$ by regressing $X_1 - X_0$ against $X_t = tX_1 + (1-t)X_0$ for random $t$ → simulate ODE from $\pi_0$ to obtain $(Z_0^1, Z_1^1)$ → (optional) use these new pairs to train $v_\theta^2$ for 2-rectified flow → repeat → (optional) distill final flow into one-step model.

3.3 Roadmap for the Deep Dive

I will explain the technical approach in this order, which mirrors how the system is both motivated and implemented:

1. The training objective and optimal velocity field — how the least-squares regression in Equation (1) arises from the goal of following straight paths, what the Bayes-optimal solution $v^X(z, t) = \mathbb{E}[X_1 - X_0 \mid X_t = z]$ represents, and why this particular form of conditional expectation is both computable and meaningful.

2. The marginal preserving property — why the rectified flow ODE $dZ_t = v^X(Z_t, t)dt$ is guaranteed to transport $\pi_0$ to $\pi_1$ exactly (when solved perfectly), through the continuity equation argument showing $\text{Law}(Z_t) = \text{Law}(X_t)$ for all $t$. This is the theoretical foundation that makes the method valid.

3. Transport cost reduction — the proof that $\mathbb{E}[c(Z_1 - Z_0)] \leq \mathbb{E}[c(X_1 - X_0)]$ for all convex costs $c$, establishing that rectification produces a "better" coupling than the random independent pairing we started with.

4. The reflow procedure and straightening — the recursive application $\mathbf{Z}^{k+1} = \text{RectFlow}((Z_0^k, Z_1^k))$, its $O(1/K)$ convergence rate for straightness, and why straight flows are computationally ideal (single Euler step suffices).

5. The nonlinear extension and connection to diffusion models — how the general framework with arbitrary interpolation curves $X_t = \alpha_t X_1 + \beta_t \xi$ encompasses probability flow ODEs and DDIM as special cases, and why the canonical linear interpolation $X_t = tX_1 + (1-t)X_0$ is the recommended default.

6. Practical implementation: neural network training, non-parametric estimators, and distillation — the concrete algorithms, architecture choices, and training procedures used in the experiments.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methodology paper with strong theoretical analysis whose core idea is that straight-path ODEs can be learned through simple regression, and that recursive application of this learning procedure straightens the paths.

---

The Training Objective and Optimal Velocity Field

The fundamental operation in rectified flow is learning a velocity field $v: \mathbb{R}^d \times [0, 1] \to \mathbb{R}^d$ that drives an ODE to follow the straight-line paths between paired samples. The training objective is:

$\min_v \int_0^1 \mathbb{E} \left[ \left\| (X_1 - X_0) - v(X_t, t) \right\|^2 \right] dt, \quad \text{with} \quad X_t = tX_1 + (1 - t)X_0$

Here, $(X_0, X_1)$ is a random pair drawn from some coupling of $\pi_0$ and $\pi_1$ (typically independent, i.e., $\pi_0 \times \pi_1$); $t \sim \text{Uniform}([0, 1])$ is a random time; $X_t = tX_1 + (1-t)X_0$ is the linear interpolation point at time $t$ along the straight line connecting $X_0$ to $X_1$; and $X_1 - X_0$ is the constant direction vector of this straight-line path. The expectation $\mathbb{E}[\cdot]$ is taken over the joint distribution of $(X_0, X_1, t)$.

What it computes: For each sampled time $t$ and pair $(X_0, X_1)$, the objective takes the current interpolation point $X_t$ (which is a convex combination of the source and target) and asks the velocity model $v$ to predict the direction vector $X_1 - X_0$ that points from the source to the target. The squared error $\|(X_1 - X_0) - v(X_t, t)\|^2$ penalizes deviations between the predicted velocity and the true straight-line direction. The integral over $t \in [0, 1]$ ensures the model learns to predict this direction at all points along the trajectory, not just at the start or end. The result of minimizing this objective is a velocity field $v$ that, when used in the ODE $dZ_t = v(Z_t, t)dt$, attempts to drive the flow along straight paths.

Why this form: The choice is motivated by the geometry of straight-line transport. The linear interpolation $X_t$ already traces perfect straight lines and would be the ideal transport process — if it were causal. But it is not: the update $dX_t = (X_1 - X_0)dt$ requires knowing $X_1$, the final destination. The regression objective "causalizes" these paths by training $v$ to predict $X_1 - X_0$ from only the current state $X_t$ and time $t$. The squared-error loss is the natural choice because: (1) it is the maximum-likelihood objective if we assume Gaussian errors around the true direction, (2) it yields a closed-form optimal solution as a conditional expectation (enabling theoretical analysis), and (3) it is computationally simple to optimize with stochastic gradient descent. No adversarial loss, no likelihood evaluation, no SDE simulation — just standard supervised regression.

The paper explicitly notes that this objective is time-symmetric: exchanging $X_0$ and $X_1$ and flipping the sign of $v$ yields an equivalent problem. This means the trained velocity field works equally well for forward transport ($\pi_0 \to \pi_1$) and backward transport ($\pi_1 \to \pi_0$) by simply reversing the time direction and negating the velocity: $d\tilde{Z}_t = -v(\tilde{Z}_t, t)dt$ starting from $\tilde{Z}_0 \sim \pi_1$.

The Bayes-optimal solution. If the model $v$ has unlimited capacity and the optimization is solved exactly, the optimal velocity field is the conditional expectation:

$v^X(z, t) = \mathbb{E}[X_1 - X_0 \mid X_t = z]$

Here, $v^X(z, t)$ is the optimal velocity at position $z$ and time $t$; the conditioning event $X_t = z$ means "among all linear interpolation paths that pass through point $z$ at time $t$"; and $\mathbb{E}[X_1 - X_0 \mid X_t = z]$ is the average direction of those paths.

What this means operationally: At any location $z$ and time $t$, there may be multiple straight-line paths connecting different $(X_0, X_1)$ pairs that all happen to cross through $z$ at time $t$. The conditional expectation averages the direction vectors $X_1 - X_0$ of all such paths, weighted by their probability of passing through $z$ at time $t$. This averaging has a crucial geometric consequence: it "rewires" the intersecting paths into a single, consistent velocity at each point, ensuring that the resulting ODE trajectories do not cross each other (by the uniqueness of ODE solutions). Figure 2 illustrates this: the linear interpolation paths (Figure 2a) cross at intersection points, but the rectified flow (Figure 2b) resolves these crossings into non-intersecting trajectories.

Why this conditional expectation form matters: It is what makes the rectified flow a valid transport. Because $v^X$ averages the velocities of the linear interpolation paths at each point, the resulting ODE $dZ_t = v^X(Z_t, t)dt$ preserves the marginal distributions of $X_t$ at every time $t$ (Theorem 3.3). Intuitively, the expected flux of probability mass through any infinitesimal volume element is identical under the $X_t$ process and the $Z_t$ process, which is sufficient to guarantee $\text{Law}(Z_t) = \text{Law}(X_t)$. Since $\text{Law}(X_0) = \pi_0$ and $\text{Law}(X_1) = \pi_1$, this guarantees $Z_0 \sim \pi_0$ and $Z_1 \sim \pi_1$.

Practical approximation. In practice, $v^X$ is not computed exactly; it is approximated by a neural network $v_\theta$ trained on the regression objective. The paper uses a U-Net architecture (DDPM++ from [73]) for the velocity model. Training hyperparameters for CIFAR-10 experiments follow the configurations from [73]. The loss is estimated by Monte Carlo sampling: draw a batch of pairs $(X_0^{(i)}, X_1^{(i)})$, sample random times $t^{(i)} \sim \text{Uniform}([0, 1])$, compute $X_{t^{(i)}}^{(i)} = t^{(i)} X_1^{(i)} + (1-t^{(i)})X_0^{(i)}$, and minimize $\frac{1}{B}\sum_i \|X_1^{(i)} - X_0^{(i)} - v_\theta(X_{t^{(i)}}^{(i)}, t^{(i)})\|^2$ using Adam.

---

The Conditional Density Representation

For theoretical analysis, the paper provides an alternative representation of the optimal velocity field when $X_0$ admits a conditional density $\rho(x_0 \mid x_1)$ given $X_1$:

$v^X(z, t) = \mathbb{E} \left[ \frac{X_1 - z}{1 - t} \eta_t(X_1, z) \right], \quad \eta_t(X_1, z) = \frac{\rho\left(\frac{z - tX_1}{1 - t} \mid X_1\right)}{\mathbb{E}\left[\rho\left(\frac{z - tX_1}{1 - t} \mid X_1\right)\right]}$

Here, $\rho(\cdot \mid X_1)$ is the conditional density of $X_0$ given $X_1$; the expression $\frac{z - tX_1}{1 - t}$ recovers the $X_0$ value that would produce $X_t = z$ when interpolating with a given $X_1$ (solving $z = tX_1 + (1-t)X_0$ for $X_0$); $\eta_t(X_1, z)$ is a normalized weight that sums to 1 over $X_1$; and $\mathbb{E}[\cdot]$ is taken with respect to $X_1 \sim \pi_1$. The factor $\frac{X_1 - z}{1 - t}$ represents the direction from the current point $z$ to the target $X_1$, scaled by the remaining time.

What this representation reveals: The optimal velocity at $(z, t)$ is a weighted average of vectors pointing from $z$ to every possible target point $X_1$ in the support of $\pi_1$, where the weight $\eta_t(X_1, z)$ measures how likely it is that $X_1$ is the true target, given that we observe $X_t = z$. This likelihood is computed through the conditional density $\rho$: if many $X_0$ values near $\frac{z - tX_1}{1 - t}$ are paired with this $X_1$ in the training data, then $X_1$ gets high weight. This representation shows explicitly that the velocity field always points toward points in the support of $\pi_1$, providing an inductive bias toward generating valid samples.

Regularity conditions. The paper notes that if $\rho$ is positive and continuous everywhere, then $v^X$ is well-defined and continuous on $\mathbb{R}^d \times [0, 1)$. The ODE $dZ_t = v^X(Z_t, t)dt$ has a unique solution if $v^X$ is uniformly Lipschitz continuous on $[0, a]$ for any $a < 1$. However, if $X_0 \mid X_1 = x_1$ does not have a conditional density (e.g., if the coupling is deterministic), then $v^X$ may be undefined or discontinuous, making the ODE ill-behaved. A simple fix proposed is to add independent Gaussian noise $\xi \sim \mathcal{N}(0, \sigma^2 I)$ to $X_0$, yielding a smoothed variable $\tilde{X}_0 = X_0 + \xi$, and transport $\tilde{X}_0$ to $X_1$ instead. This gives a randomized mapping $T(X_0 + \xi)$ that still transports $\pi_0$ to $\pi_1$.

---

The Marginal Preserving Property (Theorem 3.3)

This is the foundational guarantee that makes rectified flow work as a transport method. It states: if $\mathbf{X} = \{X_t = tX_1 + (1-t)X_0\}$ is the linear interpolation process and $\mathbf{Z}$ is the rectified flow solving $dZ_t = v^X(Z_t, t)dt$ with $Z_0 = X_0$, then for every time $t \in [0, 1]$,

$\text{Law}(Z_t) = \text{Law}(X_t)$

The proof mechanism (Section 3.1, generalized to arbitrary interpolation curves) uses the continuity equation from fluid dynamics. For any smooth test function $h: \mathbb{R}^d \to \mathbb{R}$ with compact support, the time derivative of the expected value under $X_t$ is:

$\frac{d}{dt}\mathbb{E}[h(X_t)] = \mathbb{E}[\nabla h(X_t)^\top \dot{X}_t] = \mathbb{E}[\nabla h(X_t)^\top v^{\mathbf{X}}(X_t, t)]$

The first equality is the chain rule applied to $h(X_t)$. The second equality uses the law of total expectation (conditioning on $X_t$): $\mathbb{E}[\nabla h(X_t)^\top \dot{X}_t] = \mathbb{E}[\nabla h(X_t)^\top \mathbb{E}[\dot{X}_t \mid X_t]] = \mathbb{E}[\nabla h(X_t)^\top v^{\mathbf{X}}(X_t, t)]$, since $v^{\mathbf{X}}(X_t, t) = \mathbb{E}[\dot{X}_t \mid X_t]$ by definition.

This equation implies that the probability density $\pi_t$ of $X_t$ satisfies the continuity equation with drift $v^{\mathbf{X}}$:

$\dot{\pi}_t + \nabla \cdot (v_t^{\mathbf{X}} \pi_t) = 0$

The continuity equation describes how a probability distribution evolves when its particles move with velocity field $v^{\mathbf{X}}$. The rectified flow $Z_t$, by construction, moves with exactly this same velocity field $v^{\mathbf{X}}$. Therefore, its marginal distribution $\text{Law}(Z_t)$ satisfies the same continuity equation with the same initial condition ($Z_0 = X_0 \sim \pi_0$). If the solution to the continuity equation is unique (which holds when the ODE has unique solutions, per Corollary 1.3 of Kurtz [37]), then $\text{Law}(Z_t) = \text{Law}(X_t)$ for all $t$.

What this property means operationally: The rectified flow ODE $dZ_t = v^X(Z_t, t)dt$ is guaranteed to transform $\pi_0$ into $\pi_1$ when simulated from $t=0$ to $t=1$. The marginal distributions at intermediate times match those of the linear interpolation process. However, the joint distribution of the entire trajectory $\{Z_t\}_{t \in [0,1]}$ differs from that of $\{X_t\}_{t \in [0,1]}$: the rectified flow is a Markov process (the future depends only on the present) and deterministic (given $Z_0$, the entire trajectory is determined by the ODE), while the linear interpolation process is non-Markov and stochastic (the pair $(X_0, X_1)$ is random, and intermediate points depend on both endpoints). The rectified flow thus "causalizes, Markovianizes, and derandomizes" the linear interpolation while preserving marginal distributions.

Why this is non-trivial. The mere fact that $v^X$ is defined as a conditional expectation does not automatically guarantee that simulating the ODE produces the correct marginals. The proof requires showing that the continuity equation with this particular drift field has the same solution for both processes. The non-crossing property of ODEs is what makes this work: at any intersection point of the linear interpolation paths, $v^X$ assigns a single, averaged velocity, which the flow follows consistently, avoiding the ambiguity that would cause distribution mismatch.

Generalization to nonlinear interpolations (Theorem 3.3, full version). The theorem holds for any time-differentiable interpolation process $X_t$ connecting $X_0$ and $X_1$, not just the linear one. The general procedure defines $v^{\mathbf{X}}(z, t) = \mathbb{E}[\dot{X}_t \mid X_t = z]$ and solves $dZ_t = v^{\mathbf{X}}(Z_t, t)dt$. The marginal preserving property still holds: $\text{Law}(Z_t) = \text{Law}(X_t)$ for all $t$. This generalization is what allows the framework to encompass probability flow ODEs (Section 3.5). However, using nonlinear interpolation curves loses the transport cost reduction and straightening properties, which are specific to the linear case.

---

Transport Cost Reduction (Theorem 3.5)

The linear rectified flow has a remarkable property: it reduces all convex transport costs simultaneously. Formally, if $(Z_0, Z_1) = \text{Rectify}((X_0, X_1))$ is the coupling obtained by simulating the rectified flow, then for any convex function $c: \mathbb{R}^d \to \mathbb{R}$:

$\mathbb{E}[c(Z_1 - Z_0)] \leq \mathbb{E}[c(X_1 - X_0)]$

What a transport cost means: $\mathbb{E}[c(Z_1 - Z_0)]$ measures the expected "expense" of transporting mass from $\pi_0$ to $\pi_1$ according to the pairing specified by the coupling $(Z_0, Z_1)$, using cost function $c$. Common choices are $c(\cdot) = \|\cdot\|^\alpha$ for $\alpha \geq 1$: $\alpha=1$ is the expected Euclidean distance traveled, $\alpha=2$ is the expected squared distance (related to the $L_2$ optimal transport problem). The theorem says that the rectified coupling is Pareto-superior to the original coupling across all convex costs simultaneously.

The proof (Section 3.2) is a chain of equalities and inequalities:

\mathbb{E}[c(Z_1 - Z_0)] &= \mathbb{E}\left[c\left(\int_0^1 v^{\mathbf{X}}(Z_t, t) dt\right)\right] && \text{(1) } Z_1 - Z_0 = \int_0^1 dZ_t = \int_0^1 v^{\mathbf{X}}(Z_t, t)dt \\ &\leq \mathbb{E}\left[\int_0^1 c(v^{\mathbf{X}}(Z_t, t)) dt\right] && \text{(2) Jensen: } c(\int f) \leq \int c(f) \text{ for convex } c \\ &= \mathbb{E}\left[\int_0^1 c(v^{\mathbf{X}}(X_t, t)) dt\right] && \text{(3) } \text{Law}(Z_t) = \text{Law}(X_t), \text{ so expectations of } c(v(Z_t)) \text{ and } c(v(X_t)) \text{ are equal} \\ &= \mathbb{E}\left[\int_0^1 c(\mathbb{E}[X_1 - X_0 \mid X_t]) dt\right] && \text{(4) definition of } v^{\mathbf{X}} \\ &\leq \mathbb{E}\left[\int_0^1 \mathbb{E}[c(X_1 - X_0) \mid X_t] dt\right] && \text{(5) Jensen again: } c(\mathbb{E}[\cdot]) \leq \mathbb{E}[c(\cdot)] \text{ for convex } c \\ &= \int_0^1 \mathbb{E}[c(X_1 - X_0)] dt && \text{(6) tower property: } \mathbb{E}[\mathbb{E}[\cdot \mid X_t]] = \mathbb{E}[\cdot] \\ &= \mathbb{E}[c(X_1 - X_0)] && \text{(7) integrand is constant in } t \end{aligned}$$ **Why each step matters, operationally:** - **Step (1):** The displacement `$Z_1 - Z_0$` is the integral of instantaneous velocities along the rectified flow trajectory. This decomposes the total transport distance into a sum of local moves. - **Step (2):** Jensen's inequality penalizes "curved" paths more than straight ones: a path that wanders (changing direction) accumulates the same total integrated velocity magnitude but pays a higher convex cost on the **net** displacement. This is the geometric reason straight paths minimize convex costs. - **Step (3):** This is where the marginal preserving property pays off. Even though the `$Z_t$` trajectories differ from the `$X_t$` trajectories, the **distribution** of positions at each time is identical. So the expected cost of the velocity at each time is computable from the `$X_t$` distribution, which we understand directly from the training data. - **Step (5):** This is the critical inequality. The velocity `$v^{\mathbf{X}}(X_t, t)$` is an **average** of directions `$X_1 - X_0$` over all paths through `$X_t$`. By Jensen's inequality, the convex cost of this average is less than or equal to the average of the convex costs of the individual directions. This is where the "rewiring" at intersection points reduces cost: instead of paths crossing (each following its own `$X_1 - X_0$` direction), the flow takes the average direction, which is a shorter path in terms of convex cost. **The geometric intuition** provided in the paper for `$c(\cdot) = \|\cdot\|$` (expected Euclidean distance) makes this concrete: the rectified flow paths are rewired versions of the straight-line paths. Where two straight-line paths cross, the flow takes a path that is no longer than the average of the two crossing paths (by the triangle inequality). The expected length of the flow trajectory is therefore less than or equal to the expected length of the original straight lines. **Why this matters beyond theory.** The transport cost reduction means that `$\text{Rectify}(\cdot)$` produces a coupling that is "more deterministic" and "closer to optimal" than the random independent pairing `$\pi_0 \times \pi_1$`. The recursive application (reflow) further reduces costs at each step (though not to zero in general — the fixed point is a straight coupling, not necessarily a `$c$`-optimal coupling for any specific `$c$`; see Section 3.4). This increasing determinism is practically useful: it means the latent space `$Z_0$` becomes more tightly coupled to the output `$Z_1$`, enabling better interpolation and manipulation. **Extension to time-changed linear paths.** If `$X_t = \alpha_t X_1 + \beta_t X_0$` with `$\beta_t = 1 - \alpha_t$` and `$\dot{\alpha}_t \geq 0$` (straight paths with non-constant speed), the transport cost reduction still holds for `$m$`-homogeneous convex costs where `$c(ax) = |a|^m c(x)$`. This covers `$\|\cdot\|^\alpha$` costs and means the result is robust to time reparameterization as long as the paths remain straight. --- #### The Reflow Procedure and Straightening (Theorem 3.7) The reflow procedure is the paper's key mechanism for improving the computational properties of the rectified flow. It is defined recursively: $$\mathbf{Z}^{k+1} = \text{RectFlow}((Z_0^k, Z_1^k))$$ where `$\mathbf{Z}^0$` is the linear interpolation of the original random pairing `$(X_0, X_1)$`, `$\mathbf{Z}^1$` is the first rectified flow trained on `$(X_0, X_1)$`, and `$(Z_0^k, Z_1^k)$` is the coupling produced by simulating `$\mathbf{Z}^k$` from `$t=0$` to `$t=1$`. Each step: (1) simulate the current rectified flow to obtain a new deterministic coupling, (2) train a new rectified flow on this coupling's linear interpolation. The result is a sequence of flows with increasingly straight trajectories. **What "straight" means formally.** A flow is **straight** if `$Z_t = tZ_1 + (1-t)Z_0$` almost surely for all `$t \in [0, 1]$`, or equivalently, `$v(Z_t, t) = Z_1 - Z_0 = \text{const}$` along each trajectory. Such a flow has constant velocity along each path. **Why straight flows are computationally ideal.** For a straight flow, a single Euler step `$Z_1 = Z_0 + v(Z_0, 0)$` computes the exact terminal point `$Z_1$`. There is **zero time-discretization error**. The paper notes that achieving straightness is non-trivial because it requires `$v$` to satisfy the inviscid Burgers' equation `$\partial_t v + (\partial_z v)v = 0$` (the condition that velocity is constant along trajectories), which is not automatically satisfied by the optimal `$v^X$`. **The straightness metric.** The paper quantifies straightness for any continuously differentiable process `$Z = \{Z_t\}$` as: $$S(Z) = \int_0^1 \mathbb{E}\left[\|(Z_1 - Z_0) - \dot{Z}_t\|^2\right] dt$$ Here, `$Z_1 - Z_0$` is the net displacement (a constant vector for each trajectory), `$\dot{Z}_t = v(Z_t, t)$` is the instantaneous velocity at time `$t$`, and the squared norm `$\|(Z_1 - Z_0) - \dot{Z}_t\|^2$` measures the deviation between the instantaneous direction and the overall direction. `$S(Z) = 0$` if and only if the velocity is constant along every trajectory (exact straightness). Small `$S(Z)$` means the flow is "nearly straight" and can be simulated accurately with few time steps (since the velocity doesn't change much, large Euler steps remain accurate). **The non-crossing condition for straight couplings.** Theorem 3.6 establishes that a coupling `$(X_0, X_1)$` is straight (i.e., a fixed point of `$\text{Rectify}(\cdot)$`) if and only if its linear interpolation paths do not intersect, as measured by: $$V((X_0, X_1)) = \int_0^1 \mathbb{E}\left[\|X_1 - X_0 - \mathbb{E}[X_1 - X_0 \mid X_t]\|^2\right] dt = 0$$ Here, `$\|X_1 - X_0 - \mathbb{E}[X_1 - X_0 \mid X_t]\|^2$` measures the variance of the direction `$X_1 - X_0$` among all paths passing through `$X_t$`. If this variance is zero, then all paths through `$X_t$` have the same direction, meaning no paths cross at `$X_t$`. `$V = 0$` implies no crossings anywhere. **Theorem 3.7 — the `$O(1/K)$` convergence rate.** This is the paper's central theoretical guarantee for the reflow procedure. Let `$\mathbf{Z}^k$` be the `$k$`-th rectified flow. Then: $$\sum_{k=0}^K S(\mathbf{Z}^{k+1}) + V((Z_0^k, Z_1^k)) \leq \mathbb{E}[\|X_1 - X_0\|^2]$$ This implies `$\min_{k \leq K} (S(\mathbf{Z}^k) + V((Z_0^k, Z_1^k))) = O(1/K)$`: the minimum straightness metric plus crossing measure across the first `$K$` reflow steps decays at rate `$1/K$`. **The proof** uses telescoping sums on the `$L_2$` transport cost. Taking `$c(x) = \|x\|^2$` in the proof of Theorem 3.5 yields the identity: $$\mathbb{E}[\|X_1 - X_0\|^2] - \mathbb{E}[\|Z_1 - Z_0\|^2] = S(\mathbf{Z}) + V((X_0, X_1))$$ This decomposes the reduction in squared transport cost into exactly two terms: the straightness of the new flow `$S(\mathbf{Z})$` and the non-crossing measure of the old coupling `$V((X_0, X_1))$`. Applying this at each reflow step and telescoping gives the bound. Since the `$L_2$` transport cost is non-negative, the sum of all straightness and crossing measures cannot exceed the initial `$\mathbb{E}[\|X_1 - X_0\|^2]$`, forcing the average (and hence the minimum) to decay. **Practical interpretation.** The theorem guarantees that reflow makes the flows straighter. In practice (Figure 3), one reflow step already produces a dramatic straightening effect. The paper notes that too many reflow steps can accumulate estimation error in `$v^X$` (since each step uses a finite neural network approximation), so in practice 1–3 reflows are used, with distillation applied at the final step. **Why reflow works geometrically.** The first rectified flow takes the random independent pairing and produces a deterministic coupling `$(Z_0^1, Z_1^1)$` with lower transport cost. The linear interpolation of this new coupling has fewer path intersections because the deterministic mapping tends to preserve ordering: nearby `$Z_0$` points map to nearby `$Z_1$` points (a consequence of ODE continuity). Fewer intersections mean the next rectified flow is even straighter. Iterating this process drives the coupling toward a fixed point where the linear interpolation paths are entirely non-intersecting — a straight coupling. --- #### Straight vs. Optimal Couplings (Section 3.4) This section clarifies the relationship between two desirable coupling properties: straightness and `$c$`-optimality. It addresses a potential misconception that rectified flow might recover optimal transport maps. **Theorem 3.8:** If a coupling `$(X_0, X_1)$` is `$c$`-optimal for some strictly convex cost `$c$`, then it is a straight coupling (a fixed point of `$\text{Rectify}(\cdot)$`). **The converse is not true** in dimensions `$d \geq 2$`. **Proof of the forward direction:** If `$(X_0, X_1)$` is `$c$`-optimal, then no other coupling can achieve strictly lower `$c$`-cost. Since `$(Z_0, Z_1) = \text{Rectify}((X_0, X_1))$` satisfies `$\mathbb{E}[c(Z_1 - Z_0)] \leq \mathbb{E}[c(X_1 - X_0)]$`, we must have equality. By Theorem 3.6, this equality for a strictly convex `$c$` implies straightness. **The one-dimensional special case (Lemma 3.9, Theorem 3.10).** On `$\mathbb{R}$` (but not `$\mathbb{R}^d$` for `$d \geq 2$`), straightness and optimality coincide. The unique straight coupling is the **deterministic monotonic coupling** characterized by: if `$x_0 < x_0'$` in the support, then `$x_1 \leq x_1'$`. This coupling is simultaneously optimal for all non-negative convex costs `$c$`. The reason is geometric: in 1D, the non-crossing condition (paths don't intersect) is equivalent to monotonicity (paths preserve order), and monotonicity is exactly the condition for joint optimality across all convex costs [65]. This is a special property of 1D: the total order on `$\mathbb{R}$` aligns all convex costs. **The multi-dimensional case** (`$d \geq 2$`): Different convex costs `$c$` do not share a common optimal coupling. A straight coupling is not guaranteed to be `$c$`-optimal for any specific `$c$`. The paper argues this is expected: `$\text{Rectify}(\cdot)$` does not depend on a particular choice of `$c$`, so it cannot tailor its coupling to optimize a specific cost. This is a **feature, not a bug**: the goal is fast inference (straight paths are equally good regardless of which `$c$` they minimize), not solving optimal transport for a specific cost. The paper notes in passing that [42] shows `$c$`-optimality can be achieved by restricting `$v$` to be a gradient field `$v(x, t) = \nabla f(x, t)$`, which removes the rotational component that causes sub-optimal transport cost. **Connection to a false conjecture.** The paper mentions that [30] conjectured VP ODE (equivalently DDIM) yields `$L_2$`-optimal couplings, which was disproved in [39, 78]. The paper argues that even straight couplings (which VP ODE does not produce, since its paths are curved) would not guarantee `$L_2$`-optimality in high dimensions. This underscores the paper's reframing: straightness, not optimal transport, should be the goal for fast simulation. --- #### The Reflow + Distill Pipeline The full procedure for obtaining a one-step generative model is a two-phase process: **Phase 1: Reflow (iterative straightening).** Starting from independent draws `$(X_0, X_1) \sim \pi_0 \times \pi_1$`: 1. Train `$v_\theta^1$` by minimizing `$\mathbb{E}[\|X_1 - X_0 - v_\theta^1(tX_1 + (1-t)X_0, t)\|^2]$` with `$t \sim \text{Uniform}([0, 1])$`. 2. Simulate `$dZ_t^1 = v_{\hat{\theta}}^1(Z_t^1, t)dt$` from `$Z_0^1 \sim \pi_0$` to obtain pairs `$(Z_0^1, Z_1^1)$`. 3. Train `$v_\theta^2$` on `$(Z_0^1, Z_1^1)$` using the same objective. 4. Repeat `$k$` times. **Phase 2: Distillation (one-step model).** After `$k$` reflows, the `$k$`-rectified flow `$\mathbf{Z}^k$` is nearly straight. A one-step model `$\hat{T}$` is trained to predict `$Z_1^k$` directly from `$Z_0^k$`: $$\hat{T}(z_0) = z_0 + v(z_0, 0)$$ where `$v(z_0, 0)$` is trained by minimizing: $$\mathbb{E}\left[\|(Z_1^k - Z_0^k) - v(Z_0^k, 0)\|^2\right]$$ This is exactly the `$t=0$` term of the original rectified flow objective (1). The distillation loss only requires evaluating `$v$` at `$t=0$`, not simulating the ODE. **Why distillation differs from an additional reflow.** The paper emphasizes a crucial distinction. Distillation attempts to **faithfully approximate** the coupling `$(Z_0^k, Z_1^k)$` — it aims to reproduce the same mapping. Reflow (`$\text{Rectify}(\cdot)$`) produces a **different** coupling `$(Z_0^{k+1}, Z_1^{k+1})$` with lower transport cost and straighter flow. Distillation should only be applied after the flow is already straight enough that one-step simulation is accurate; applying it earlier would produce a poor one-step model because the flow is curved and a single Euler step is a poor approximation. **Algorithmic summary (Algorithm 1 from the paper):** ``` Procedure: Z = RectFlow((X_0, X_1)): Inputs: Draws from coupling (X_0, X_1) of π_0 and π_1; velocity model v_θ Training: θ̂ = arg min_θ E[‖X_1 - X_0 - v_θ(tX_1 + (1-t)X_0, t)‖²], t ~ Uniform([0,1]) Sampling: Draw (Z_0, Z_1) following dZ_t = v_θ̂(Z_t, t)dt from Z_0 ~ π_0 (or backward Z_1 ~ π_1) Return: Z = {Z_t: t ∈ [0,1]} Reflow (optional): Z^{k+1} = RectFlow((Z_0^k, Z_1^k)), starting from (Z_0^0, Z_1^0) = (X_0, X_1) Distill (optional): Learn T̂ to distill k-rectified flow: Z_1^k ≈ T̂(Z_0^k) ``` --- #### The Nonlinear Extension and Connection to Diffusion Models (Sections 2.3 and 3.5) The paper presents a generalization that encompasses existing methods and clarifies the advantages of the canonical linear rectified flow. **General nonlinear rectified flow.** Let `$\mathbf{X} = \{X_t: t \in [0, 1]\}$` be **any** time-differentiable process connecting `$X_0 \sim \pi_0$` to `$X_1 \sim \pi_1$`, with time derivative `$\dot{X}_t$`. The rectified flow induced from `$\mathbf{X}$` is defined as: $$dZ_t = v^{\mathbf{X}}(Z_t, t)dt, \quad v^{\mathbf{X}}(z, t) = \mathbb{E}[\dot{X}_t \mid X_t = z]$$ The training objective becomes: $$\min_v \int_0^1 \mathbb{E}\left[w_t \|v(X_t, t) - \dot{X}_t\|^2\right] dt$$ where `$w_t: (0, 1) \to (0, +\infty)$` is a positive weighting function (`$w_t = 1$` by default). The linear case is the special case where `$X_t = tX_1 + (1-t)X_0$` and `$\dot{X}_t = X_1 - X_0$`. **Key property.** Theorem 3.3 still holds: `$\text{Law}(Z_t) = \text{Law}(X_t)$` for all `$t$`. However, if `$\mathbf{X}$` does not follow straight paths, the transport cost reduction (Theorem 3.5) and the straightening effect (Theorem 3.7) no longer hold. The reflow procedure will not straighten the flow if applied to a nonlinear interpolation. **A useful class: `$X_t = \alpha_t X_1 + \beta_t X_0$`.** With `$\alpha_t, \beta_t$` differentiable and `$\alpha_1 = \beta_0 = 1$`, `$\alpha_0 = \beta_1 = 0$`, we have `$\dot{X}_t = \dot{\alpha}_t X_1 + \dot{\beta}_t X_0$`. If `$\beta_t = 1 - \alpha_t$`, the paths are straight (just not constant-speed). If `$\beta_t \neq 1 - \alpha_t$`, the paths are curved. The paper recommends **against** curved choices for ODE-based generative modeling. **Connection to probability flow ODEs (Proposition 3.11).** The paper proves that all variants of probability flow ODEs from [73] (VE ODE, VP ODE, sub-VP ODE) and DDIM [70] are instances of the nonlinear rectified flow with `$X_t = \alpha_t X_1 + \beta_t \xi$` where `$\xi \sim \mathcal{N}(0, I)$`. The derivation requires showing that the target `$\tilde{Y}_t$` in the PF-ODE training objective (18) equals `$\dot{X}_t$`. **The PF-ODE training procedure (review).** In denoising diffusion, an SDE `$dU_t = b(U_t, t)dt + \sigma_t dW_t$` is trained by: $$\min_v \int_0^1 \mathbb{E}\left[w_t \|v(V_t, t) - Y_t\|^2\right] dt, \quad V_t = \alpha_t X_1 + \beta_t \xi_t, \quad Y_t = -\eta_t V_t - \frac{\sigma_t^2}{\beta_t} \xi_t$$ The hyperparameters `$\eta_t, \sigma_t$` determine `$\alpha_t, \beta_t$` through: $$\alpha_t = \exp\left(\int_t^1 \eta_s ds\right), \quad \beta_t^2 = \int_t^1 \exp\left(2\int_t^s \eta_r dr\right) \sigma_s^2 ds$$ The SDE is then converted to a probability flow ODE `$dZ_t = \tilde{b}(Z_t, t)dt$` with `$\tilde{b}(z, t) = \frac{1}{2}(b(z, t) - \eta_t z)$`, which can also be directly trained using the modified target `$\tilde{Y}_t = -\eta_t V_t - \frac{\sigma_t^2}{2\beta_t}\xi_t$`. **The equivalence proof (Proposition 3.11).** The paper verifies that `$\tilde{Y}_t = \dot{X}_t$` when `$X_t = \alpha_t X_1 + \beta_t \xi$`. Using the relations derived from (16) — specifically `$\eta_t = -\dot{\alpha}_t / \alpha_t$` and `$\sigma_t^2 = 2\beta_t^2(\dot{\alpha}_t/\alpha_t - \dot{\beta}_t/\beta_t)$` — substituting into `$\tilde{Y}_t$` and simplifying yields `$\dot{\alpha}_t X_1 + \dot{\beta}_t \xi = \dot{X}_t$`. Therefore, the PF-ODE training objective (18) is exactly the nonlinear rectified flow objective (6) with `$X_t = \alpha_t X_1 + \beta_t \xi$` and `$w_t$` set appropriately. **Why this connection matters.** It reveals that PF-ODEs and DDIM are **not fundamentally different** from rectified flow — they are all ODEs trained by regressing on `$\dot{X}_t$`. The differences lie entirely in the choice of `$\alpha_t, \beta_t$`: - **VP ODE:** `$\alpha_t = \exp(-\frac{1}{4}a(1-t)^2 - \frac{1}{2}b(1-t))$` with `$a = 19.9, b = 0.1$`, and `$\beta_t = \sqrt{1 - \alpha_t^2}$`. - **sub-VP ODE:** Same `$\alpha_t$`, but `$\beta_t = 1 - \alpha_t^2$`. - **VE ODE:** `$\alpha_t = 1$`, `$\beta_t = \sigma_{\min}\sqrt{r^{2(1-t)} - 1}$` with `$\sigma_{\min} = 0.01$` and `$r$` set such that `$\sigma_{\max} = r\sigma_{\min}$` is large. **The paper's critique of these choices (Section 2.3.1).** The paper identifies three problems: 1. **Curved paths:** Because `$\beta_t \neq 1 - \alpha_t$` for VP and sub-VP ODE (specifically, `$\beta_t = \sqrt{1-\alpha_t^2}$` and `$\beta_t = 1-\alpha_t^2$`), the interpolation trajectories are **not straight lines**. This means the induced ODE follows curved paths, which cannot be straightened by reflow (reflow only straightens flows induced from straight interpolations). VE ODE has straight paths (`$\dot{X}_t = \dot{\beta}_t \xi$` is always in the direction of `$\xi$`), but suffers from the speed issue. 2. **Non-uniform speed:** The exponential `$\alpha_t$` changes slowly for `$t \lesssim 0.5$` and then rapidly for `$t \gtrsim 0.5$` (Figure 6). This concentrates most of the displacement in the later phase of the ODE simulation, making large-step Euler discretization inaccurate because the velocity changes dramatically within a single step. The paper demonstrates this in Figure 5: VP ODE and sub-VP ODE with few steps produce poor results, while changing to `$\alpha_t = t$` (linear schedule, last column of Figure 5) dramatically improves few-step performance. 3. **Coupled initial distribution:** In these methods, `$\pi_0$` must be approximately `$\mathcal{N}(0, \beta_0^2 I)$` because the true `$X_0 = \alpha_0 X_1 + \beta_0 \xi$` is approximated by `$X_0 \approx \beta_0 \xi$` under the assumption `$\alpha_0 X_1 \ll \beta_0 \xi$`. For VE ODE, `$\pi_0 = \mathcal{N}(0, \sigma_{\max}^2 I)$` where `$\sigma_{\max}$` must be set as "the maximum Euclidean distance between all pairs of training data points" (Technique 1 of [72]) — an unnatural constraint. The rectified flow framework decouples these: `$\pi_0$` can be any distribution, independent of the choice of interpolation. **The paper's recommendation.** Use the canonical linear interpolation `$X_t = tX_1 + (1-t)X_0$` with `$\alpha_t = t$` and `$\beta_t = 1 - t$`. This choice ensures: (a) straight paths (enabling straightening via reflow), (b) constant speed `$\dot{X}_t = X_1 - X_0$` (uniform progression in time, good for fixed-step solvers), and (c) decoupling from `$\pi_0$` (any source distribution works). The paper acknowledges that nonlinear choices may be useful when incorporating non-Euclidean geometry (e.g., on manifolds), but this is left to future work. --- #### Non-Parametric Estimator for Low-Dimensional Problems For the toy examples (Figures 2–5), the paper uses a Nadaraya-Watson style kernel estimator instead of a neural network, which allows exact visualization of the theoretical properties: $$v^{X,h}(z, t) = \mathbb{E}\left[\frac{X_1 - z}{1 - t} \omega_h(X_t, z)\right]$$ where `$\omega_h(X_t, z) = \kappa_h(X_t, z) / \mathbb{E}[\kappa_h(X_t, z)]$` and `$\kappa_h(x, z) = \exp(-\|x - z\|^2 / (2h^2))$` is a Gaussian RBF kernel with bandwidth `$h > 0$`. In practice, this is approximated using nearest neighbors: $$v^{X,h}(z, t) \approx \sum_{i \in \text{knn}(z, m)} \frac{x_1^{(i)} - z}{1 - t} \omega_h(x_t^{(i)}, z) / \sum_{i \in \text{knn}(z, m)} \omega_h(x_t^{(i)}, z)$$ where `$\text{knn}(z, m)$` denotes the top `$m$` nearest neighbors of `$z$` among `$\{x_t^{(i)}\}_i$`. The paper uses `$h = 1$` and `$m = 100$` by default, noting that results are not sensitive to these choices (Figure 7). **Why this form.** The estimator is a **mixture of linear functions**: each term `$(x_1^{(i)} - z) / (1 - t)$` points directly from the current position `$z$` toward a specific target point `$x_1^{(i)}$`. The weights `$\omega_h$` are higher for interpolation points `$x_t^{(i)}$` that are close to `$z$`. This means the estimator extrapolates by averaging the directions toward target points whose interpolation paths pass near `$z$`. This has the desirable property that all predicted velocities point toward actual data points in `$\pi_1$`, providing a natural inductive bias. As `$h \to 0^+$`, the estimator converges to the true conditional expectation `$\mathbb{E}[(X_1 - z)/(1 - t) \mid X_t = z]$` at points where `$X_t$` has positive density. --- #### Practical Design Choices and Their Justifications **Why independent coupling `$\pi_0 \times \pi_1$` as starting point?** The paper starts with independent draws because, in both generative modeling (noise `$\perp$` data) and domain transfer (unpaired samples), we lack meaningful paired observations. Any coupling would be equally arbitrary. The independent coupling is the maximum-entropy choice and is trivial to sample. The transport cost reduction property ensures that rectification improves upon this naive starting point. **Why squared-error loss rather than other divergences?** The `$L_2$` loss has several advantages: (1) it admits a clean closed-form optimum `$\mathbb{E}[X_1 - X_0 \mid X_t]$`, enabling the theoretical analysis; (2) it is computationally efficient (no adversarial training, no likelihood computation); (3) it is the natural choice for predicting continuous directions in Euclidean space (the maximum-likelihood objective under Gaussian errors); (4) it connects directly to the `$L_2$` transport cost, enabling the telescoping-sum proof of the straightening rate. **Why uniform time sampling `$t \sim \text{Uniform}([0, 1])$`?** Uniform sampling ensures the velocity field is learned at all points along the trajectories with equal weight. If certain time intervals were under-sampled, the ODE would be poorly approximated in those regions, causing integration error to accumulate. The time-symmetry of the objective means that forward and backward simulation are equally accurate. **Why Euler discretization with constant step size?** The paper uses the simplest ODE solver — explicit Euler with step size `$1/N$` — for most experiments. This is deliberate: the goal is to demonstrate that rectified flow works well even with the coarsest, cheapest solver. Using more sophisticated adaptive solvers (RK45) would mask the benefits of straightness. The fact that 2-rectified flow achieves good results with `$N=1$` Euler step validates the core claim that straightness enables fast simulation. **Why neural networks rather than kernel estimators for high dimensions?** The non-parametric estimator requires computing nearest neighbors in the data space, which scales poorly with dimension and dataset size. Neural networks learn a parametric approximation that generalizes across the space and scales to large datasets. The paper uses the DDPM++ U-Net architecture from [73] for image experiments, demonstrating compatibility with existing diffusion model architectures. **Why the reflow stopping criterion is `$k = 2$` or `$k = 3$`?** The paper finds that 2-rectified flow already achieves near-straight paths (Figure 9) and that 3-rectified flow provides marginal additional improvement at the cost of accumulated estimation error. The `$O(1/K)$` rate means diminishing returns, and in practice the finite-sample error in estimating `$v^X$` eventually dominates. The paper notes that "it is not recommended to apply too many reflow steps as it may accumulate estimation error on `$v^X$`." **Why distillation only at the final reflow stage?** Distillation approximates the current coupling; reflow produces a better coupling. Applying distillation before the flow is straight would produce a poor one-step model because the mapping `$Z_0 \mapsto Z_1$` cannot be accurately captured by a single Euler step (the flow is curved, so `$Z_1 \neq Z_0 + v(Z_0, 0)$`). Waiting until after reflow makes the flow straight enough that `$Z_1 \approx Z_0 + v(Z_0, 0)$` is a good approximation, making distillation effective. **Why the feature-weighted loss for image translation?** For domain transfer, the paper introduces a modified loss (Equation 20) that uses `$\nabla h(X_t)$` to weight the regression error: $$\min_v \int_0^1 \mathbb{E}\left[\|\nabla h(X_t)^\top (X_1 - X_0 - v(X_t, t))\|_2^2\right] dt$$ Here, `$h(x)$` is a classifier's latent representation trained to distinguish domains `$\pi_0$` and `$\pi_1$`. The gradient `$\nabla h(X_t)$` acts as a **saliency map**, identifying which pixels are important for domain-specific style. The weighted loss focuses the velocity model on accurately predicting changes that affect style-relevant features, while being more permissive of errors in style-irrelevant dimensions (e.g., background texture). This is motivated by the fact that the goal of image translation is not perfect distribution matching (which would destroy content identity), but rather style transfer while preserving content. The classifier-based weighting provides a soft, learned notion of which image features matter for the domain distinction. ## 4. Key Insights and Innovations ### Innovation 1: The Reflow Procedure as a New Mechanism for Turning Continuous-Time Models into One-Step Models The most conceptually distinctive contribution of this paper is not rectified flow itself—it's the **reflow** procedure: recursively applying the same rectification operation to its own output to straighten the flow's trajectories. This is fundamentally different from how the field has previously approached the problem of accelerating continuous-time generative models, and it represents a new category of solution. **What the field did before.** The dominant approach to speeding up diffusion and flow models has been **knowledge distillation**: train a student model (typically a one-step generator) to mimic the output of a teacher model (the multi-step diffusion/flow process). This is the strategy behind DDIM distillation [47], progressive distillation, and consistency models. The key assumption is that the teacher already produces the desired distribution, and the student's job is merely to compress this into fewer steps. The reflow procedure rejects this assumption entirely: instead of compressing an existing flow, **reflow changes which coupling the flow represents**, producing a new coupling with provably straighter paths. It is not making the existing flow faster—it is constructing a *different flow* that is inherently easier to simulate. **Why this is a fundamental shift.** Distillation is fundamentally imitative: it tries to reproduce the teacher's mapping. Reflow is *constructive*: each iteration produces a genuinely new transport plan with lower convex transport costs and fewer path intersections. The paper's Theorem 3.7 provides a formal guarantee—straightness improves at rate O(1/K)—that has no analog in distillation approaches. This theoretical grounding transforms what could have been a heuristic trick into a principled procedure with predictable behavior. **The significance beyond metrics.** The reflow concept opens a new axis of optimization for generative models. Rather than treating the learned coupling as fixed and trying to approximate it more efficiently, reflow treats the coupling itself as something that can be iteratively refined toward computational idealness (straightness). This reframes the acceleration problem from "how do we compress this complex mapping?" to "how do we learn a mapping that is inherently simple to simulate?" The distinction matters because it suggests future work might design training procedures that directly target path straightness, rather than treating it as a post-hoc property to be improved. **Evidence anchor.** Figure 3(d) shows straightness decreasing monotonically with reflow steps on a toy example, while Figure 9 demonstrates the same effect on CIFAR-10. The practical payoff is visible in Figure 8(a): 2-rectified flow with N=1 Euler step achieves FID comparable to 1-rectified flow with N≈10 steps—a dramatic improvement that distillation alone cannot achieve starting from the same 1-rectified flow. --- ### Innovation 2: Reframing the ODE Learning Problem as "Causalizing" Straight-Line Interpolation Paths Before this work, learning neural ODEs for generative modeling was approached either through maximum likelihood estimation [6] or through the probability flow ODE framework derived from SDEs [73, 70]. Both approaches treat the ODE's intermediate paths as something the training procedure must *discover*—through likelihood optimization in the MLE case, or through the SDE-to-ODE conversion in the diffusion case. Rectified flow introduces a fundamentally different perspective: **pre-specify the desired paths as the straight-line interpolation between randomly paired points, then learn a velocity field that causalizes these paths.** **What this reframing accomplishes.** The key insight is that the non-causal linear interpolation $X_t = tX_1 + (1-t)X_0$ already traces perfect straight lines—exactly the paths we want. The only problem is that it requires knowing $X_1$ at time $t$, making it non-causal and therefore not a valid ODE. The rectified flow solution is elegantly minimal: learn to predict the direction $X_1 - X_0$ from only the current state $X_t$ and time $t$. This is a standard conditional expectation problem solvable by least-squares regression. The paper demonstrates that this simple procedure produces a valid ODE (Theorem 3.3), reduces convex transport costs (Theorem 3.5), and can be iteratively straightened (Theorem 3.7)—all from what is essentially a supervised learning objective. **How this differs from prior ODE learning.** In the MLE approach [6], the paths are implicitly determined by the optimization landscape—different initializations and optimizers can produce different paths, all achieving the same terminal distribution. There is no mechanism to prefer straight paths over curved ones. In the PF-ODE/DDIM approach, the paths are inherited from an SDE derivation with specific $\alpha_t, \beta_t$ schedules that produce curved trajectories (Figure 5). The paper's reframing makes path geometry a **design choice** rather than an emergent property: we decide that we want straight paths, we construct the interpolation accordingly, and the training objective follows directly. **The significance beyond performance.** This reframing demystifies ODE-based generative modeling. It shows that the complex SDE machinery of diffusion models—Ornstein-Uhlenbeck processes, time reversal, score matching—is not necessary for learning effective transport ODEs. The field's assumption had been that the mathematical structure of diffusion processes was essential to the success of these models. Rectified flow demonstrates that a simple regression objective on straight-line interpolation achieves comparable or better results, suggesting that the key innovation in diffusion models was not the stochastic process itself but the stable, scalable regression-based training procedure. **Evidence anchor.** Proposition 3.11 proves that PF-ODEs are instances of the nonlinear rectified flow framework with specific (suboptimal) choices of $\alpha_t, \beta_t$. Figure 5 demonstrates visually that the canonical linear interpolation produces straight, uniform-speed trajectories while VP ODE and sub-VP ODE produce curved, non-uniform-speed trajectories that perform poorly with coarse discretization. Table 1(a) shows that 1-rectified flow matches or exceeds all PF-ODE variants in both full-simulation quality and few-step performance. --- ### Innovation 3: Separating "Straightness" from "Optimal Transport" as Independent Desiderata Prior work connecting generative models to optimal transport theory [67, 34, 35, 49, 63] has largely assumed that the goal is to find the $c$-optimal coupling for some cost function $c$, typically the quadratic cost $\|\cdot\|^2$. The underlying assumption is that optimal transport maps are the "right" maps to learn—they minimize some notion of distance, they have nice regularity properties, and therefore they should produce better generative models. This paper makes a clean conceptual break: **straightness and $c$-optimality are different properties, and for the purpose of fast inference, straightness is what actually matters.** **The diagnostic move.** Theorem 3.8 establishes that $c$-optimal couplings are straight (they are fixed points of Rectify), but straight couplings are not necessarily $c$-optimal for any specific $c$ in dimensions $d \geq 2$. The one-dimensional case is a special exception where the two concepts coincide (Theorem 3.10), which the paper explains cleanly through the equivalence of non-crossing and monotonicity on $\mathbb{R}$. This separation is not merely theoretical taxonomy—it has practical consequences. The paper explicitly notes that a false conjecture in the literature [30] claimed DDIM yields $L_2$-optimal couplings, which was disproved [39, 78]. The paper goes further: even if DDIM did produce straight paths (which it doesn't—Figure 5 shows they're curved), straightness alone wouldn't guarantee $L_2$-optimality in high dimensions. **Why this separation matters.** It redirects the optimization target. If the goal is fast inference (few-step simulation), then **all straight couplings are equally good** because they all admit exact one-step simulation. There is no need to solve the computationally challenging optimal transport problem for a specific $c$—any straight coupling will do. This is liberating: Rectify produces straight couplings without targeting any specific cost function, and its O(1/K) straightening rate (Theorem 3.7) is independent of which cost function one might care about. **The significance beyond the paper.** This reframing suggests that the community's focus on optimal transport for generative modeling may have been partially misdirected. Optimal transport solves a different problem (matching a specific cost) than what fast generative models need (straight, easy-to-simulate paths). The reflow procedure is Pareto-optimal across all convex costs simultaneously—it reduces them all—but it does not optimize any single one. This is a feature, not a bug: it means reflow works without requiring us to choose and justify a particular cost function. At the same time, the paper acknowledges in a brief forward reference that restricting $v$ to be a gradient field can recover $c$-optimality for the quadratic cost [42], showing that the framework can be specialized when optimal transport is genuinely needed. **Evidence anchor.** The toy example in Figure 3(d) shows that both straightness and relative $L_2$ transport cost improve with reflow, but they are distinct curves. The theory is anchored in Theorem 3.6 (characterizing straight couplings via non-intersecting paths), Theorem 3.8 (optimal $\implies$ straight but not conversely), and the 1D special case in Lemma 3.9 and Theorem 3.10. --- ### Innovation 4: Demonstrating That Diffusion Noise Is Unnecessary for State-of-the-Art Continuous-Time Generative Modeling The empirical success of diffusion models [23, 71, 72, 73] created a strong implicit assumption in the field: that the stochastic nature of the generative process—gradually denoising through many small random steps—was essential to their performance. The mathematical derivations of these methods rely heavily on SDE theory, score matching, and time reversal, reinforcing the sense that diffusion noise is fundamental. This paper provides a clean empirical refutation: **an ODE trained with a simple regression objective, without any diffusion noise in training or inference, achieves comparable or better results than SDE-based diffusion models on standard benchmarks.** **What makes this distinctive beyond just "ODEs work."** The paper doesn't merely show that ODEs can be competitive—it provides a specific, principled alternative training procedure that is simpler than diffusion training. The rectified flow objective (Equation 1) has no noise schedule to tune, no forward diffusion process to design, no score function to estimate, and no SDE-to-ODE conversion step. The only design choices are the coupling of $\pi_0$ and $\pi_1$ (independent by default) and the interpolation curve (linear by default). This is dramatically simpler than the "involved design space" of diffusion models that the paper critiques as "still poorly understood both empirically and theoretically." **The re-thinking of what made diffusion work.** The paper explicitly argues (Section 4 discussion) that "the success of the denoising diffusion models may be mainly attributed to the simple and stable optimization-based training procedure that allows us to avoid the instability issues and the need of case-by-case tuning of GANs, rather than the presence of diffusion noises." This is a provocative hypothesis: it suggests that the field may have been attributing diffusion models' success to the wrong mechanism. If correct, it means the key innovation was not the stochastic process but the regression-based training objective—and rectified flow inherits this exact advantage (regression, no minimax) without the diffusion baggage. **The significance for future work.** If diffusion noise is unnecessary, then the entire design space of $\eta_t$, $\sigma_t$, $\alpha_t$, $\beta_t$ inherited from Ornstein-Uhlenbeck processes can be discarded for ODE-based generative modeling. Future work can focus on designing interpolation curves that incorporate domain-specific geometry (e.g., on manifolds) without being constrained by SDE derivations. This is a simplification that opens design flexibility rather than constraining it. **Evidence anchor.** Table 1(a) shows 1-rectified flow achieving FID 2.58 with RK45 solver, comparable to VP SDE (2.55) and sub-VP SDE (2.61) which require 2000 Euler steps. Figure 5 demonstrates visually that the rectified flow with $\alpha_t = t, \beta_t = 1-t$ produces uniform-speed straight trajectories that work well with N=2 steps, while the SDE-derived VP ODE and sub-VP ODE require N=100 steps for comparable quality. --- ### Innovation 5: Unifying Generative Modeling and Domain Transfer Under a Single Algorithm Without Modification Prior to this work, generative modeling (noise → data) and domain transfer (data → data) were treated as distinct problems requiring distinct methods. Generative modeling used GANs, VAEs, normalizing flows, or diffusion models. Domain transfer used CycleGANs [100], diffusion-based translation [75, 97], or optimal transport [16, 59]—each requiring problem-specific architectural choices, loss functions, and training procedures. The paper's framework provides a unification that is **not merely cosmetic**: the exact same algorithm, with the exact same objective, handles both tasks by simply changing what distribution is assigned to $\pi_0$. **Why this unification is non-trivial.** The difficulty of domain transfer compared to generation is that $\pi_0$ and $\pi_1$ are both complex, high-dimensional data distributions (e.g., human faces and cat faces). In generation, $\pi_0$ is a simple Gaussian—easy to sample, with known density. Many generative methods exploit this simplicity (e.g., VAEs use the Gaussian's tractable density for the KL divergence; diffusion models construct the forward noising process to terminate at a Gaussian). Domain transfer provides no such convenience: both distributions are empirical and complex. The fact that rectified flow handles both without modification is a consequence of its design: the training objective only requires samples from $\pi_0$ and $\pi_1$, never their densities. The independent coupling $\pi_0 \times \pi_1$ is equally trivial to sample whether $\pi_0$ is Gaussian or a dataset of human faces. **The significance beyond the specific results.** This unification suggests that the transport mapping problem—not generative modeling specifically—is the right level of abstraction. Any problem that can be framed as "find a map from distribution A to distribution B given samples from both" is in scope: style transfer, domain adaptation, image editing, super-resolution, inpainting. The paper demonstrates domain adaptation (Table 2) where rectified flow matches or exceeds specialized methods like CORAL [76], showing the framework's breadth. The feature-weighted loss for image translation (Equation 20) shows how the basic framework can be adapted for translation-specific goals (preserving content while transferring style) through a simple modification to the regression weights, rather than a completely different training paradigm. **Evidence anchor.** Figures 1, 13, 14, and 15 show visually high-quality image translation results across diverse domain pairs (human↔cat, cat↔wild, MetFace↔cat). Figure 1 specifically demonstrates that the same reflow procedure works for translation: 2-rectified flow produces good results with a single Euler step even for domain transfer, not just generation. Table 2 shows domain adaptation accuracy matching or exceeding specialized methods. ## 5. Experimental Analysis ### Evaluation Methodology - **Dataset.** Unconditioned image generation uses **CIFAR-10** (50,000 training images, 10 classes, 32×32 resolution) and several high-resolution datasets: **LSUN Bedroom** and **LSUN Church** [93] (both 256×256, scene categories), **CelebA-HQ** [27] (30,000 celebrity faces at 256×256), and **AFHQ Cat** [9] (cat faces at 256×256 from the Animal Faces-HQ dataset). Image-to-image translation uses paired combinations of **AFHQ Cat, AFHQ Wild, MetFaces** [28], and **CelebA-HQ**, with 80% of each dataset used for training and 20% for testing. Domain adaptation uses **DomainNet** [58] (345 object categories across six domains, evaluated following the DomainBed [20] protocol) and **Office-Home** [83] (65 categories across four domains). - **Base model(s).** All image experiments use the **DDPM++ U-Net architecture** from Song et al. [73] for the velocity field `$v_\theta$`, which employs a U-Net backbone with attention layers, residual blocks, and time conditioning. The paper adopts the exact architecture and hyperparameter configurations from [73] without modification, including channel multipliers, attention resolutions, and the number of residual blocks. For the toy examples (Figures 2–5, 7), a **non-parametric kernel estimator** (Equation 5) or a **2-hidden-layer fully connected network** (64 neurons per layer) is used instead to allow exact visualization of the theoretical properties without the confounding effects of neural network approximation error. For domain adaptation experiments, features are first extracted from the final hidden layer of a pre-trained model, and a smaller DDPM++ variant is trained on these latent representations. - **Metrics.** For image generation, the paper reports **Fréchet Inception Distance (FID)** (lower is better; measures the distributional distance between generated and real images in Inception-v3 feature space), **Inception Score (IS)** (higher is better; measures both image quality and class diversity using a pre-trained Inception classifier), and **Recall** [38] (higher is better; estimates the fraction of the real data distribution covered by generated samples, measuring diversity independently of precision). For domain adaptation, **classification accuracy** of a model trained on the source domain and evaluated on the rectified-flow-transferred target domain is reported. For the toy examples, **straightness** `$S(Z)$` (Equation 3, lower means straighter paths) and **relative `$L_2$` transport cost** (computed as the gap between the average squared displacement of the rectified coupling and the discrete `$L_2$` optimal transport solution) are tracked. The **number of function evaluations (NFE)** counts how many times the neural velocity model `$v_\theta$` is called during inference, which coincides with the number of Euler discretization steps `$N$` for fixed-step solvers or the adaptive step count for RK45. - **Baselines.** The paper compares against **VP ODE** and **sub-VP ODE** from Song et al. [73] (probability flow ODEs derived from variance-preserving and sub-variance-preserving SDEs), **VE ODE** [73] (derived from variance-exploding SDE), **VP SDE** and **sub-VP SDE** [73] (the original stochastic diffusion processes requiring 2,000 Euler steps), and **DDPM** [23] (1,000-step denoising diffusion). For one-step generation comparisons, baselines include **DDIM Distillation** [47] (distilling a DDIM teacher into a single-step student model), **TDPM** [99] (truncated diffusion with a GAN prior, T=1 variant), **Denoising Diffusion GAN** [91] (T=1 variant), and several GAN methods: **SNGAN** [52], **StyleGAN2** [28], **StyleGAN2 + ADA** [28], **StyleGAN2 + DiffAug** [98], **TransGAN + DiffAug** [26], and **StyleGAN-XL** [66]. For domain adaptation, baselines are **ERM, IRM, ARM, Mixup, MLDG, and CORAL** [76] as reported in DomainBed [20]. - **Generation budget / compute accounting.** The primary unit of computation is **NFE (number of function evaluations)**, which counts neural network forward passes. For ODE models using Euler discretization with `$N$` uniform steps, NFE = `$N$`. For RK45 adaptive solvers, NFE is the actual number of steps taken (typically 104–176 for CIFAR-10, with tolerances set following [73]). For SDE baselines, NFE = 2,000 (VP SDE, sub-VP SDE) or 1,000 (DDPM). For one-step distilled models, NFE = 1. Training cost is not included in the comparison; the focus is exclusively on inference-time efficiency. For the reflow procedure, each reflow step requires (1) generating a new training dataset by simulating the previous flow (cost proportional to NFE of that flow × number of training samples) and (2) training a new velocity model from scratch. These costs are reported separately and not amortized into the inference budget. - **Cross-validation / statistical protocol.** No formal cross-validation is described for the image generation experiments. Results on CIFAR-10 are reported on the standard test set (10,000 images). For image translation, the 80/20 train/test split per dataset serves as a held-out evaluation. For domain adaptation, results follow the DomainBed evaluation protocol [20] with mean and standard deviation across multiple runs. The paper does not report confidence intervals or statistical significance tests for FID/IS/Recall differences between methods. ### Main Quantitative Results #### Unconditioned Image Generation: Full ODE Simulation The paper first establishes that rectified flow, when solved accurately with an adaptive-step ODE solver (RK45), matches or exceeds the quality of existing ODE and SDE methods while requiring fewer function evaluations. **1-rectified flow achieves the best FID among all ODE methods.** Using the DDPM++ architecture on CIFAR-10 (Table 1a, "ODE Full Simulation" section), 1-rectified flow with RK45 solver attains **FID = 2.58, IS = 9.60, Recall = 0.57** using **NFE = 127** adaptive steps. This outperforms VP ODE (FID = 3.93, IS = 9.37, Recall = 0.51, NFE = 140), sub-VP ODE (FID = 3.16, IS = 9.46, Recall = 0.55, NFE = 146), and VE ODE with NCSN++ architecture (FID = 5.38, IS = 9.35, Recall = 0.56, NFE = 176, Table 1b). The recall score of 0.57 represents a "substantial improvement over existing ODE and GAN methods" per the paper, indicating superior diversity. **Rectified flow is competitive with SDE methods at a fraction of the inference cost.** The fully-simulated VP SDE achieves FID = 2.55, IS = 9.58, Recall = 0.58 at NFE = 2,000 (Table 1a, "SDE Full Simulation"). Sub-VP SDE achieves FID = 2.61, IS = 9.56, Recall = 0.58 at NFE = 2,000. DDPM achieves FID = 3.21, IS = 9.46, Recall = 0.57 at NFE = 1,000 (Table 1b). The 1-rectified flow's FID = 2.58 at NFE = 127 is statistically indistinguishable from VP SDE's 2.55 at NFE = 2,000—a **~16× reduction in inference cost** with no meaningful quality degradation. **Subsequent reflows degrade full-simulation quality modestly.** 2-rectified flow achieves FID = 3.36, IS = 9.24, Recall = 0.54 (NFE = 110). 3-rectified flow achieves FID = 3.96, IS = 9.01, Recall = 0.53 (NFE = 104). The paper attributes this degradation to "accumulation of error on estimating `$v^X$`" at each reflow step, as the finite-capacity neural network cannot perfectly recover the true conditional expectation. The NFE also decreases slightly with reflow, which the paper notes but does not explain mechanistically—it may be because straighter paths are easier for the adaptive solver and require fewer steps to meet the tolerance. **Key takeaway from full simulation:** When simulation accuracy is not constrained, 1-rectified flow alone (without reflow) already produces state-of-the-art ODE-based generation, matching SDE quality at dramatically lower inference cost. The reflow procedure provides no benefit in this regime; it is strictly useful for the low-NFE regime discussed next. #### Unconditioned Image Generation: Few-Step and One-Step Generation The paper's central practical claim is that reflow + distillation enables high-quality generation with very few (1–5) Euler steps. This is evaluated in Figure 8(a) and Table 1a ("ODE One-Step Generation" section). **Reflow dramatically improves few-step performance.** Figure 8(a) plots FID and Recall against the number of Euler discretization steps `$N$` from 1 to 100+. At `$N = 2$` steps, 1-rectified flow achieves FID ≈ 90 (estimated from Figure 8a, left panel), while 2-rectified flow achieves FID ≈ 25—a **~3.6× improvement**. At `$N = 5$` steps, 2-rectified flow achieves FID ≈ 7 while 1-rectified flow remains around FID ≈ 40. The recall curves (Figure 8a, right panel) show a similar pattern: 2-rectified flow at `$N = 5$` achieves recall ≈ 0.50, while 1-rectified flow requires `$N ≈ 80$` to reach the same recall. This means **2-rectified flow achieves comparable diversity to 1-rectified flow with ~16× fewer steps**. **Distillation makes one-step generation competitive.** The distilled models (stars in Figure 8a) evaluated at `$N = 1$`: - **Distilled 1-rectified flow:** FID = 378, IS = 1.13 (essentially no useful generation—the flow is too curved for a single Euler step to approximate). - **Distilled 2-rectified flow:** **FID = 4.85, IS = 8.08, Recall = 0.34** (with distillation raising IS to 9.01 and Recall to 0.50, as shown in Table 1a parenthetical values). - **Distilled 3-rectified flow:** FID = 8.15, IS = 8.47, Recall = 0.41 (with distillation: FID = 5.21, IS = 8.79, Recall = 0.51). The distilled 2-rectified flow with FID = 4.85 is, at the time of publication, **the best reported one-step generative model using U-Net architecture**, beating TDPM (T=1) at FID = 8.91, Denoising Diffusion GAN (T=1) at FID = 14.6, and DDIM Distillation at FID = 9.36 (Table 1). The recall of 0.50 (distilled) and 0.51 (3-rectified + distill) exceed the best GAN recall of 0.49 (StyleGAN2 + ADA), showing particular strength in diversity. **Comparison to VP ODE and sub-VP ODE one-step generation (Table 1a).** Distilled VP ODE achieves FID = 451 (essentially random), IS = 1.20 (distilled to 8.73), Recall = 0.0 (distilled to 0.29). Distilled sub-VP ODE achieves FID = 451, IS = 1.21 (distilled to 8.80), Recall = 0.0 (distilled to 0.35). The paper notes that even with distillation, these methods cannot approach rectified flow's one-step quality because their trajectories are fundamentally curved (see Figure 5)—distillation can compress a curved flow into one step, but the approximation error remains large because the true mapping `$Z_0 \mapsto Z_1$` is not well-approximated by `$Z_0 + v(Z_0, 0)$` when the flow is not straight. **What the distillation parenthetical values reveal.** The notation "1.13 (9.08)" in Table 1a for 1-rectified flow means: the one-step model without distillation achieves IS = 1.13 (terrible), but the one-step model after distillation achieves IS = 9.08 (good). However, for 1-rectified flow, the distilled FID is 6.18—much worse than 2-rectified flow's 4.85, because 1-rectified flow's paths are too curved for even a distilled one-step model to capture well. The distillation procedure for 2-rectified flow is effective precisely because the flow is already nearly straight: the true mapping is close to linear, so the `$t=0$` velocity `$v(Z_0, 0)$` already approximates `$Z_1 - Z_0$` well, and distillation only needs to fine-tune this. **The `$k$`-rectified flow progression measured by FID.** Figure 8(b) tracks FID and Recall across training iterations for different rectification rounds. Each reflow step produces a noticeable jump in quality: at the same training iteration, `$k+1$`-rectified flow consistently outperforms `$k$`-rectified flow on both FID and Recall. This confirms that reflow is not merely faster convergence—it genuinely improves the attainable quality ceiling. #### The Straightening Effect: Empirical Verification The paper validates the theoretical straightening prediction (Theorem 3.7) with both quantitative and qualitative evidence on CIFAR-10 and AFHQ Cat. **Straightness metric decreases with reflow.** Figure 9 (left) plots the straightness measure `$S(Z)$` from Equation (3) across training iterations for 1-, 2-, and 3-rectified flow. The 2-rectified flow achieves substantially lower `$S(Z)$` than 1-rectified flow throughout training, and 3-rectified flow is lower still. This confirms that the reflow procedure progressively straightens the trajectories in practice, not just in the toy examples (Figure 3d). **Trajectory visualization confirms near-perfect straightness at `$k=2$`.** Figure 9 (right) visualizes randomly sampled pixel trajectories following 1- and 2-rectified flow. The 1-rectified flow trajectories show visible curvature, while the 2-rectified flow trajectories appear nearly linear. Figure 10 provides a more detailed visualization on AFHQ Cat: at intermediate times `$t \in \{0.0, 0.1, \dots, 1.0\}$`, the paper displays both the current state `$z_t$` and the extrapolated final state `$\hat{z}_1^t = z_t + (1 - t)v(z_t, t)$`. For a perfectly straight flow, `$\hat{z}_1^t$` should be constant (the same final image) at all `$t$`. For 2-rectified flow, the extrapolated images are indeed "almost independent with `$t$`," visually confirming near-straightness. In contrast, 1-rectified flow shows clear evolution in `$\hat{z}_1^t$` from blurred to sharp, indicating curvature. Sub-VP ODE shows even more extreme curvature: `$\hat{z}_1^t$` remains essentially noise until `$t \approx 0.6$`, reflecting the non-uniform speed problem analyzed in Section 2.3.1. **Early-time image recognizability.** An intriguing observation from Figure 10: even though 1-rectified flow is not straight, it still yields "recognizable and clear images very early (`$t \approx 0.1$`)," while sub-VP ODE requires `$t \approx 0.6$` to produce recognizable images from the extrapolation. This means rectified flow's trajectories are not only straighter but also more uniformly distributed in "semantic progress"—the flow makes steady progress in generating recognizable features throughout the time interval, rather than remaining in a noise-like regime for most of the process and then rushing to form the image at the end. This property is a direct consequence of the constant-speed linear interpolation `$X_t = tX_1 + (1-t)X_0$`, which mixes source and target uniformly over time. **The straightness vs. FID trade-off at high NFE.** Figure 8(a) reveals an important nuance: while reflow dramatically improves FID at low NFE (`$N \lesssim 80$`), it **worsens FID at high NFE**. At NFE ≈ 100+, 1-rectified flow achieves better FID than 2-rectified flow, which in turn beats 3-rectified flow. The paper attributes this to "accumulation of error on estimating `$v^X$`." Each reflow step uses the output of the previous flow as training data, and any errors in the previous flow's coupling (due to finite network capacity, finite simulation steps, or finite training data) become baked into the next flow's training objective. Over multiple reflow rounds, these errors compound, degrading the asymptotic quality. This creates a practical trade-off: reflow improves the **computational efficiency** (quality per NFE) but can degrade **asymptotic quality** (quality at very high NFE). The paper's recommendation of 2 or 3 reflow steps represents an empirical optimum for the few-step regime. #### High-Resolution Image Generation The paper demonstrates that 1-rectified flow scales to 256×256 resolution on four datasets (Figure 11). Generated examples from LSUN Bedroom, LSUN Church, CelebA-HQ, and AFHQ Cat show visually plausible images with recognizable global structure and fine details. However, **no quantitative metrics (FID, IS) are reported for these high-resolution experiments**. The paper states only that "it can generate high quality results across the different datasets" without numerical comparison to baselines. This is a notable gap: the CIFAR-10 results are thoroughly quantified, but the high-resolution results are purely qualitative. The paper also shows (Figure 1, top two rows) that 1-rectified flow produces recognizable cat faces in 2–5 Euler steps and 2-rectified flow produces high-quality faces in a single step, but again without quantitative metrics. **Image editing demonstration.** Figure 12 shows a creative application: an unnatural "chimera" image is created by stitching the upper half of a white cat with the lower half of a black cat. This image `$z_1$` is then mapped backward through the 1-rectified flow ODE to obtain a latent code `$z_0$`. Because `$z_1$` is unnatural (low probability under the cat image distribution), its latent code `$z_0$` should have low probability under `$\pi_0 = \mathcal{N}(0, I)$`. The paper modifies `$z_0$` to increase its likelihood under the Gaussian prior—either deterministically by scaling `$z_0' = \alpha z_0$` with `$\alpha \in (0, 1)$` (contracting toward the origin, where Gaussian density is highest), or stochastically via Langevin dynamics `$z_0' = \alpha z_0 + \sqrt{1-\alpha^2}\xi$` with `$\xi \sim \mathcal{N}(0, I)$`. Forward simulation from the modified `$z_0'$` produces a more realistically blended cat image. This demonstrates that the rectified flow's latent space is semantically meaningful: moving `$z_0$` toward higher prior probability produces more "natural" images, confirming that the ODE has learned a useful latent representation where the Gaussian prior structure corresponds to image naturalness. #### Image-to-Image Translation The paper evaluates rectified flow for unpaired image translation between five domain pairs: AFHQ Cat ↔ AFHQ Wild, AFHQ Cat ↔ MetFaces, and AFHQ Cat ↔ CelebA-HQ (Figures 1, 13, 14, 15), using the feature-weighted loss (Equation 20) with a classifier `$h$` fine-tuned from an ImageNet-pretrained model. **1-rectified flow produces high-quality translations at N = 100.** Figure 13 shows examples of 1-rectified flow simulated with Euler method at N = 100 steps for all domain pairs. The translations are visually compelling: cat→wild produces diverse animal faces (fox, lion, tiger, cheetah) with preserved pose and background structure, wild→cat produces plausible cat faces with fur textures matching the original animal's coloring, MetaFace→cat produces human-cat hybrids with preserved facial structure, and CelebA-HQ→cat produces cat faces that retain the original human's expression and lighting. **2-rectified flow enables single-step translation.** Figure 14 compares 1- and 2-rectified flow at both N = 1 and N = 100 steps for cat ↔ wild and human ↔ cat. At N = 100, both flows produce high-quality results. At N = 1, 1-rectified flow produces degraded, blurry results (the flow is too curved for a single step), while 2-rectified flow produces sharp, recognizable translations comparable to the N = 100 results. This demonstrates that the reflow straightening effect transfers directly to the domain transfer setting: the same procedure that enables one-step generation also enables one-step translation. **Trajectory visualization for translation.** Figure 15 shows the temporal evolution `$\{z_t\}$` for 1-rectified flow across all domain pairs (Figure 15a) and for 1- vs. 2-rectified flow on MetFace → Cat (Figure 15b). The trajectories show smooth, semantically meaningful interpolation: the domain-specific features (fur texture, ear shape, facial proportions) evolve gradually over time while content identity (pose, expression, composition) is preserved throughout. **No quantitative metrics for image translation.** As with high-resolution generation, the paper reports no FID, IS, or other quantitative metrics for the translation experiments. The evaluation is purely qualitative ("visually high-quality"). The paper notes that "as the main goal here is to obtain good visual results, we are not interested in faithfully transferring `$X_0 \sim \pi_0$` to an `$X_1$` that exactly follows `$\pi_1$`," acknowledging a tension between distribution matching and content preservation that makes standard distributional metrics potentially misleading for translation tasks. However, no alternative quantitative evaluation (e.g., user studies, content preservation metrics like LPIPS, or domain classification accuracy) is provided. #### Domain Adaptation The paper applies 1-rectified flow to domain adaptation by training on latent representations from the final hidden layer of a pre-trained model, then evaluating classification accuracy on the transferred target domain (Table 2). **Office-Home results.** 1-rectified flow achieves **69.2 ± 0.5%** accuracy, outperforming all baselines: ERM (66.5 ± 0.3), IRM (64.3 ± 2.2), ARM (64.8 ± 0.3), Mixup (68.1 ± 0.3), MLDG (66.8 ± 0.6), and the previous best method CORAL [76] (68.7 ± 0.3). The improvement over CORAL is +0.5 percentage points, with non-overlapping error bars suggesting statistical significance. **DomainNet results.** 1-rectified flow achieves **41.4 ± 0.1%** accuracy, essentially matching CORAL's 41.5 ± 0.2% (the previous best) within error bars, and outperforming all other baselines: ERM (40.9 ± 0.1), IRM (33.9 ± 2.8), ARM (35.5 ± 0.2), Mixup (39.2 ± 0.1), and MLDG (41.2 ± 0.1). The paper describes this as "state-of-the-art performance on both DomainNet and OfficeHome." **Significance for the unified framework claim.** These domain adaptation results provide quantitative evidence (unlike the qualitative translation results) that rectified flow performs competitively on a task completely distinct from image generation, using the same algorithm without architectural modifications. This supports the paper's framing of rectified flow as a unified solution to transport mapping problems, not merely a generative modeling technique. #### Comparison to the SDE-to-ODE Conversion Baseline An important quantitative comparison buried in the paper's discussion is the few-step performance comparison between rectified flow and the probability flow ODE variants when using the **same** architecture. Figure 8(a) shows that across all Euler step counts from 1 to 100+, rectified flow variants consistently outperform VP ODE and sub-VP ODE on both FID and Recall. At N = 10 steps, VP ODE achieves FID ≈ 25, sub-VP ODE achieves FID ≈ 18, while 2-rectified flow achieves FID ≈ 6 and 1-rectified flow achieves FID ≈ 13. The gap is even larger at N = 2: VP ODE and sub-VP ODE both achieve FID > 100 (essentially random), while 2-rectified flow achieves FID ≈ 25. This comparison is important because it isolates the effect of the interpolation curve choice (`$\alpha_t = t, \beta_t = 1-t$` for rectified flow vs. the exponential `$\alpha_t$` and nonlinear `$\beta_t$` of VP/sub-VP ODE) while holding the neural architecture constant. It provides direct empirical support for the paper's critique (Section 2.3.1) that the SDE-derived schedules are suboptimal for ODE simulation. ### Ablation Studies and Robustness Checks - **Non-parametric vs. neural network estimation (Figure 7):** On a toy 2D example, the non-parametric kernel estimator (`$h = 1$`, `$m = 100$`) produces trajectories that closely follow the piecewise-linear theoretical optimum, while a 2-hidden-layer MLP produces trajectories with slight curvature. The paper notes that increasing L2 regularization on the neural network "can help straighten the flow, in addition to the rectification effect," suggesting that smoothness of the function approximator interacts with path straightness—a practical insight for neural network training that is not explored further. - **Kernel bandwidth sensitivity (Figure 7, right):** The non-parametric estimator's performance on the toy example is shown to be "not sensitive to the choice of `$m$` and the bandwidth `$h$`," with the nearest-neighbor count `$m = 100$` and bandwidth `$h = 1$` used as defaults. This suggests the estimator is robust, though the evaluation is purely qualitative (visual inspection of trajectories). - **Number of reflow steps (Figures 8 and 9):** The paper systematically varies the number of reflow steps `$k \in \{1, 2, 3\}$` on CIFAR-10. The key pattern is a trade-off: higher `$k$` improves few-step FID and recall (Figure 8a, left side of plots) but degrades asymptotic FID at high NFE (Figure 8a, right side). The paper attributes this to error accumulation and recommends against "too many reflow steps." The optimal choice depends on the target NFE budget: `$k=2$` is best for `$N \lesssim 20$`, while `$k=1$` is best for `$N \gtrsim 80$`. - **Distillation effect (Table 1a, parenthetical values):** Comparing the distilled and non-distilled one-step models reveals that distillation provides a large boost for `$k=1$` (IS improves from 1.13 to 9.08) but a smaller boost for `$k=2$` (IS improves from 8.08 to 9.01) and `$k=3$` (IS improves from 8.47 to 8.79). This confirms that reflow makes the flow straighter, reducing the gap between the one-step approximation `$Z_0 + v(Z_0, 0)$` and the true `$Z_1$`, making distillation less necessary. For `$k=2$` and `$k=3$`, the one-step model without distillation already achieves reasonable FID (12.21 and 8.15, respectively), while `$k=1$` without distillation is useless (FID = 378). - **Feature-weighted loss for image translation (Equation 20, Figures 1, 13, 14):** The paper introduces a modified training objective for domain transfer where the regression loss is weighted by `$\nabla h(X_t)$`, the gradient of a domain classifier. No ablation comparing the weighted vs. unweighted loss is reported—the weighted version is simply presented as the method. The paper argues this weighting "re-weights coordinates so that the loss focuses on penalizing the error that causes significant changes on `$h$`" (domain-relevant features), but whether this actually improves translation quality over the standard unweighted loss is not quantitatively or qualitatively demonstrated. This is a missing ablation that would clarify whether the feature weighting is necessary or merely optional. - **Solver choice (Euler vs. RK45):** All few-step experiments use Euler discretization, demonstrating that the method works well with the simplest possible solver. The full-simulation results use RK45 with adaptive step sizing, following [73]'s protocol. The paper does not systematically compare different ODE solvers (e.g., midpoint, Heun's method, DPM-Solver) or step size schedules, which is a missed opportunity to characterize the robustness of the straightness benefit. - **Negative result: 3-rectified flow degrades full-simulation quality (Table 1a, Figure 8a):** This is a genuine negative result that the paper reports honestly. As `$k$` increases from 2 to 3, the full-simulation (RK45) FID worsens from 3.36 to 3.96, and IS drops from 9.24 to 9.01. This demonstrates that the reflow procedure has a practical limit due to error accumulation, and that the `$O(1/K)$` convergence rate of Theorem 3.7 does not guarantee monotonic improvement in finite-sample, finite-capacity settings. The paper does not investigate whether this error accumulation could be mitigated by increasing model capacity, training data, or training duration at each reflow step. ### Critical Assessment **Claim: Rectified flow achieves state-of-the-art one-step generation (FID = 4.85, Recall = 0.51 on CIFAR-10).** The quantitative evidence for CIFAR-10 one-step generation is strong: Table 1 provides direct comparisons against multiple competitive baselines across GAN, ODE, and SDE families under the same NFE = 1 constraint. The FID of 4.85 for 2-rectified flow (+ distill) is clearly superior to TDPM (8.91), Denoising Diffusion GAN (14.6), DDIM Distillation (9.36), and distilled VP ODE (16.23). However, this claim is narrower than it appears: (a) the comparison is restricted to U-Net-based architectures—StyleGAN2 + ADA achieves FID = 2.92 (Table 1b) using a different architecture, meaning rectified flow is not beating all one-step models, only those within a specific architecture class; (b) the claim applies only to CIFAR-10—no quantitative one-step metrics are reported for any higher-resolution dataset; (c) StyleGAN-XL achieves FID = 1.85 with a style-based architecture, showing there remains a substantial gap to the best one-step models overall. **Claim: Reflow straightens flows, enabling accurate simulation with coarse discretization.** The evidence for straightening is multi-modal and convincing. The straightness metric `$S(Z)$` decreases with reflow steps (Figure 9, left). The trajectory visualizations (Figures 9 right, 10) directly show the paths becoming more linear. The practical consequence—improved FID at low NFE (Figure 8a)—is unambiguous. The theoretical guarantee (Theorem 3.7) is consistent with the empirical behavior. However, the paper does not establish a quantitative relationship between `$S(Z)$` and FID or between `$S(Z)$` and the NFE required to reach a target FID. The claim that straightness "enables" accurate few-step simulation is supported directionally but not with a causal or predictive model (e.g., "reducing `$S(Z)$` by X% reduces the NFE needed for FID = 5 by Y steps"). **Claim: Rectified flow matches SDE quality at a fraction of the inference cost (~16× fewer evaluations).** This claim is well-supported for the specific comparison of 1-rectified flow (NFE = 127, FID = 2.58) vs. VP SDE (NFE = 2,000, FID = 2.55). The problem is that this is a comparison against **unoptimized** SDE inference. The SDE baselines use 2,000 Euler steps, but the diffusion literature has since developed numerous accelerated sampling methods (DDIM with 50–100 steps, DPM-Solver with 10–20 steps, etc.). The paper does not compare against these more efficient SDE sampling methods, making the 16× comparison an upper bound on the advantage. The comparison against VP ODE and sub-VP ODE (which already represent the SDE-to-ODE speedup) at similar NFE (110–146) shows a more modest improvement (FID 2.58 vs. 3.16–3.93), and the 2- and 3-rectified flows are slightly worse at full simulation (FID 3.36, 3.96). The headline 16× improvement is relative to the slowest reasonable baseline, not the fastest. **Claim: The same algorithm works unifiedly for generation, translation, and domain adaptation.** The unification is demonstrated across three tasks, but the strength of evidence varies dramatically. For generation, the evidence is quantitative and thorough on CIFAR-10, qualitative on high resolution. For domain adaptation, the evidence is quantitative (Table 2) and shows competitive performance. For image translation, **the evidence is entirely qualitative**—no FID, no user study, no content preservation metric, no comparison to CycleGAN or diffusion-based translation methods. The paper states that "we are not interested in faithfully transferring `$X_0 \sim \pi_0$` to an `$X_1$` that exactly follows `$\pi_1$`," which acknowledges that standard distribution-matching metrics may not capture translation quality, but the absence of any quantitative evaluation makes it impossible to assess whether the translation results are competitive with existing methods or merely visually plausible. A human preference study or automated content/style decomposition metric would have substantially strengthened this claim. **Missing experiments and baselines.** Several comparisons would have strengthened the paper: 1. **No DDIM baseline in the main few-step comparison.** DDIM [70] is the most natural ODE baseline for few-step generation, yet Figure 8(a) only compares against VP ODE and sub-VP ODE. DDIM with the DDPM++ architecture trained on CIFAR-10 would provide a direct comparison at the same NFE values. 2. **No direct CycleGAN comparison for translation.** Given that CycleGAN [100] is the canonical unpaired image translation method and is discussed extensively in the paper, its absence from the experimental comparison is conspicuous. Even qualitative side-by-side comparisons would help situate the results. 3. **No ablation of the feature-weighted loss (Equation 20) vs. the standard loss (Equation 1) for translation.** It is unclear whether the classifier-based weighting is necessary or merely an optional enhancement. 4. **No systematic evaluation of computational cost for reflow.** The paper notes that reflow requires generating a new training dataset and retraining the model, but does not quantify this cost. For the CIFAR-10 experiments, 1-rectified flow training already uses the full dataset; 2-rectified flow requires simulating 1-rectified flow on ~50,000 noise samples to generate training pairs, which costs an additional ~127 × 50,000 = ~6.4 million NFE just for data generation. This is a substantial hidden cost not reflected in the inference-time NFE comparison. 5. **No evaluation on standard generative modeling benchmarks beyond CIFAR-10.** ImageNet (64×64 or 128×128), LSUN, or FFHQ would provide more robust evidence at higher resolutions with standard metrics. 6. **No confidence intervals or statistical testing.** All FID/IS/Recall values are reported as point estimates without error bars, making it difficult to assess whether observed differences (e.g., 1-rectified flow FID 2.58 vs. VP SDE 2.55) are statistically significant or within noise. 7. **No comparison of different ODE solvers at fixed NFE.** The paper uses Euler discretization throughout but does not test whether higher-order solvers (midpoint, Heun) could extract better performance from the same NFE budget, particularly for the non-straight 1-rectified flow. **Claims conditionally supported.** The central claim—that straightness via reflow enables fast, high-quality inference—is well-supported quantitatively for CIFAR-10 in the low-to-moderate NFE regime (1–20 steps) and qualitatively for high-resolution generation and translation. The claim that rectified flow is a unified framework for generation and transfer is supported for generation and domain adaptation (both have quantitative evidence), but only qualitatively for image translation. The claim that rectified flow outperforms diffusion-based methods must be understood relative to the specific (unaccelerated) SDE baselines and the VP/sub-VP ODE baselines—not against the best available diffusion sampling methods. The practical impact of the method would be clearer with a quantitative accounting of the total computational cost (training + reflow data generation + inference) compared to training a single diffusion model and using an accelerated sampler. ## 6. Limitations and Trade-offs ### 6.1 The Hidden Cost of Difficulty Estimation: Generating Training Data for Reflow **The assumption or constraint.** The reflow procedure — which is the primary mechanism enabling one-step generation — requires simulating the current rectified flow on the entire training set to produce a new dataset of `$(Z_0^k, Z_1^k)$` pairs, which then serves as training data for the next rectified flow. The paper acknowledges this cost implicitly but does not quantify it or include it in its headline efficiency comparisons. Section 5 describes the procedure: "The second rectified flow `$\mathbf{Z}^2$` is obtained by the same procedure except with the data replaced by the draws from `$(Z_0^1, Z_1^1)$`, obtained by simulating the first rectified flow `$\mathbf{Z}^1$`." **The consequence.** The reported inference-time speedups (e.g., 1-step generation at NFE = 1) hide a massive upfront computational cost. For CIFAR-10, generating training data for 2-rectified flow requires simulating 1-rectified flow on ~50,000 noise samples. Using the RK45 solver reported in Table 1a (NFE = 127 per sample), this costs roughly 127 × 50,000 = 6.35 million NFE just for data generation — equivalent to generating over 10,000 images at the full SDE cost of 2,000 NFE each. This cost is incurred before training the 2-rectified flow model itself (which requires a full training run from scratch). For domain transfer where both `$\pi_0$` and `$\pi_1$` are large image datasets, the cost doubles (both source and target domains must be simulated). A practitioner deciding whether to deploy this method faces an unquantified trade-off: the headline one-step inference speed comes at the cost of multiple expensive retraining cycles whose total FLOPs may exceed simply running a slower model with more inference steps many times over. **What evidence exists in the paper.** The paper does not report the total computational cost (training + reflow data generation + inference) for any experiment. The NFE metrics in Table 1 and Figure 8 count only inference-time function evaluations. The training cost of each rectified flow is mentioned as using the configurations from Song et al. [73] but is never aggregated or compared against baselines on a total-FLOPs basis. This is particularly important because the paper's own argument against diffusion models is their high inference cost — but if the training cost of rectified flow + reflow dominates, the total cost advantage may be smaller or reversed depending on how many images will ultimately be generated. **Mitigation status.** The paper does not address this limitation. It does not propose any method to reduce the cost of generating reflow training data (e.g., using a subset of the data, amortizing simulation across multiple reflow rounds, or using faster solvers for data generation). The high-resolution experiments (Section 5.2) and image translation experiments (Section 5.3) provide no cost accounting whatsoever. A practitioner reading the paper would need to independently estimate total FLOPs before making deployment decisions. --- ### 6.2 Error Accumulation Across Reflow Steps: Asymptotic Quality Degrades with `$k$` **The assumption or constraint.** The reflow procedure relies on each rectified flow exactly recovering the true conditional expectation `$v^X(z, t) = \mathbb{E}[X_1 - X_0 \mid X_t = z]$`. In practice, neural networks provide only a finite-capacity approximation, and ODE simulation uses finite discretization steps. Each reflow step therefore propagates and compounds these errors. The paper acknowledges this in Section 2.2: "It is not recommended to apply too many reflow steps as it may accumulate estimation error on `$v^X$`." And in Section 5.2: "the reflow procedure substantially improves both FID and recall in the small step regime … even though it worsens the results in the large step regime due to the accumulation of error on estimating `$v^X$`." **The consequence.** There is a fundamental trade-off that the paper surfaces but does not resolve: reflow improves computational efficiency (quality at low NFE) but degrades asymptotic quality (quality at high NFE). This means the method cannot be used to simultaneously achieve the best possible quality AND the fastest inference — the practitioner must choose. On CIFAR-10 (Table 1a, Figure 8a), 1-rectified flow achieves the best full-simulation FID (2.58 at NFE = 127), but requires many steps. 2-rectified flow achieves the best one-step FID (4.85), but its full-simulation FID degrades to 3.36 (26% worse). 3-rectified flow degrades further to 3.96 (53% worse than 1-rectified flow). The paper's `$O(1/K)$` straightening guarantee (Theorem 3.7) ensures straighter paths but does not bound the impact of approximation error on distribution-matching quality. In a deployment where inference budget varies per query (e.g., some queries need fast answers, others can wait for higher quality), it is unclear whether to deploy a single model or maintain separate `$k$`-rectified flows for different latency requirements. **What evidence exists in the paper.** Figure 8(a) clearly shows the crossover: 2-rectified flow is superior for `$N \lesssim 80$` Euler steps, while 1-rectified flow is superior for `$N \gtrsim 80$`. Table 1a shows that full-simulation FID monotonically worsens with `$k$` (2.58 → 3.36 → 3.96 for `$k = 1, 2, 3$`), while one-step FID monotonically improves (378 → 12.21 → 8.15 before distillation). The paper also reports that 3-rectified flow degrades on recall (0.57 → 0.54 → 0.53). No experiment explores whether increasing model capacity or training duration at each reflow step can reduce error accumulation — all reflow steps use the same architecture and training budget. **Mitigation status.** The paper partially mitigates this by recommending 2–3 reflow steps as a practical sweet spot, but provides no principled criterion for selecting `$k$`. It suggests distillation at the final stage but notes this only improves the efficiency-quality trade-off at the target operating point; it does not recover the lost asymptotic quality. Future work on reducing estimation error (better architectures, longer training, larger models) is mentioned only in passing. There is no investigation of whether error accumulation can be reduced by using the same velocity model architecture across reflow steps (e.g., fine-tuning rather than retraining from scratch) or by using more accurate ODE solvers during training data generation. --- ### 6.3 No Quantitative Evaluation for High-Resolution Generation or Image Translation **The assumption or constraint.** The paper's strongest quantitative claims — state-of-the-art one-step FID, `$4\times$` efficiency gains, competitive recall — are based entirely on CIFAR-10 at 32×32 resolution. For high-resolution generation (256×256 on LSUN Bedroom, LSUN Church, CelebA-HQ, AFHQ Cat) and for all image translation experiments, the evaluation is purely qualitative: visual inspection of generated samples. The paper states for translation (Section 5.3): "As the main goal here is to obtain good visual results, we are not interested in faithfully transferring `$X_0 \sim \pi_0$` to an `$X_1$` that exactly follows `$\pi_1$`." **The consequence.** A practitioner interested in high-resolution generation or image-to-image translation cannot assess from this paper whether rectified flow is competitive with existing methods on standard quantitative benchmarks. For high-resolution generation, no FID, IS, or recall scores are reported against baselines like StyleGAN2, ADM [12], or latent diffusion [64]. The paper's Figure 11 shows plausible samples, but visual plausibility of cherry-picked examples does not establish distribution-level quality or diversity — a GAN suffering from mode collapse can produce individually plausible images while achieving terrible FID and recall. For image translation, the absence of quantitative comparison against CycleGAN [100], UNIT [45], MUNIT [25], diffusion-based translation [75, 97], or even simple baselines (e.g., histogram matching) means a practitioner cannot determine whether the visual quality shown in Figures 13–15 represents a meaningful advance or merely the fact that the method produces some plausible-looking outputs. The domain adaptation results (Table 2) provide the only quantitative evidence beyond CIFAR-10, but domain adaptation is a different task with different metrics (classification accuracy) that do not measure generation or translation quality. **What evidence exists in the paper.** For high-resolution generation: Figure 11 shows 12 example images across 4 datasets (3 per dataset) with no metrics. For image translation: Figures 13–15 show example translations with no metrics and no comparison to any baseline method. The domain adaptation results (Table 2) show quantitative metrics but on a completely different evaluation protocol (latent space classification accuracy, not image quality). The paper does compare rectified flow against VP ODE and sub-VP ODE quantitatively on CIFAR-10 (Figure 8a, Table 1a), demonstrating that at this resolution, the method performs well under standard metrics — but this provides no guarantee of scaling behavior to higher resolutions. **Mitigation status.** Not addressed. The paper does not explain why quantitative metrics are omitted for high-resolution generation (where standard FID computation is routine) or for image translation (where metrics like FID, KID, LPIPS, or user studies are standard practice). The statement about not being "interested in faithfully transferring" for translation is a claim about goals, not a justification for omitting evaluation. Even if distribution-matching metrics are imperfect for translation, reporting them alongside qualitative results would allow readers to calibrate the trade-offs. --- ### 6.4 The Framework Provides No Mechanism for Content Preservation in Domain Transfer **The assumption or constraint.** In standard rectified flow (Equation 1), the training objective learns to transport `$\pi_0$` exactly to `$\pi_1$`. This is appropriate for generation (where exact distribution matching is the goal) but problematic for domain transfer, where the goal is to change style while preserving content identity. If the flow perfectly matches `$\pi_1$`, it may discard all information about the specific source image `$X_0$`, producing a random sample from the target domain rather than a content-preserving translation. The paper introduces the feature-weighted loss (Equation 20) to address this: `$\min_v \int_0^1 \mathbb{E}[\|\nabla h(X_t)^\top (X_1 - X_0 - v(X_t, t))\|_2^2] dt$`, where `$h$` is a domain classifier. The gradient `$\nabla h$` acts as a saliency map, focusing the velocity model on accurately predicting changes that affect domain-relevant features while being more permissive of errors in domain-irrelevant dimensions. **The consequence.** The feature-weighted loss provides a soft preference rather than a hard constraint. There is no formal guarantee that content is preserved — the method relies on the inductive bias that `$\nabla h(X_t)$` will assign low weight to content-preserving directions and high weight to style-changing directions. If the classifier `$h$` uses spurious features to distinguish domains (e.g., background color, image sharpness, or watermark artifacts rather than semantic style), the weighting may emphasize the wrong dimensions, producing translations that fail to preserve identity. Moreover, the paper provides no ablation comparing the weighted loss to the standard unweighted loss for translation, so the reader cannot assess whether the feature weighting is actually necessary or merely an optional enhancement. A practitioner implementing domain transfer with rectified flow has no guidance on how to train `$h$`, what architecture to use, or how to validate that the weighting is functioning as intended. **What evidence exists in the paper.** Only qualitative visual results (Figures 13–15). The paper shows that translations look plausible — cats become wild animals while retaining pose, humans become cats while retaining facial structure — but provides no metric quantifying content preservation (e.g., LPIPS distance between source and translated images, facial recognition accuracy across domains, or keypoint correspondence). There is no ablation showing translations with and without the feature-weighted loss. The domain adaptation results (Table 2) use standard rectified flow (no feature weighting) on latent representations, which is a different setup that does not evaluate content preservation. **Mitigation status.** Not addressed. The paper does not discuss failure modes of the feature-weighted loss, does not provide criteria for selecting or training the classifier `$h$`, and does not propose alternative mechanisms for content preservation (such as cycle consistency, mutual information constraints, or explicit content-style decomposition). The claim that rectified flow "provides a unified solution to generative modeling and domain transfer" must therefore be qualified: the generation pipeline transfers directly, but the translation pipeline requires task-specific modification (the feature-weighted loss) whose design and validation are not systematized. --- ### 6.5 Single Benchmark and Architecture: CIFAR-10 with DDPM++ U-Net Only **The assumption or constraint.** All quantitative results are on a single dataset (CIFAR-10, 32×32) using a single neural architecture (DDPM++ U-Net from Song et al. [73]). The paper states in Section 1 that rectified flow provides "a unified solution to generative modeling and domain transfer," but provides no evidence that the method works across different data modalities (text, audio, video), different resolutions (the high-resolution results are qualitative only), different model architectures (transformers, S4, alternatives to U-Nets), or different dataset scales (CIFAR-10 has 50,000 training images; large-scale datasets like ImageNet are not tested). **The consequence.** The paper's claims of state-of-the-art performance and broad applicability rest on a narrow empirical foundation. Several aspects of the results could be specific to this particular combination: - CIFAR-10 images are small (32×32) and relatively low-dimensional, meaning the straightness benefits may be more pronounced than in higher dimensions where path intersections are rarer (the curse of dimensionality could make even random paths nearly straight, reducing the benefit of reflow). - The DDPM++ architecture was designed for diffusion models and may have specific inductive biases (e.g., sensitivity to noise levels) that interact favorably or unfavorably with rectified flow's training objective in ways that differ from other architectures. - The toy examples (Figures 2–5) use kernel estimators, not neural networks, so the theoretical properties demonstrated there (marginal preservation, transport cost reduction, straightening) are shown in a setting without function approximation error — the gap between these theoretical demonstrations and the neural network experiments is bridged only by the CIFAR-10 results. A practitioner working with text, audio, high-resolution images, or transformer architectures cannot determine from this paper whether rectified flow will be effective, whether the same number of reflow steps (`$k = 2$` or `$3$`) is optimal, or whether the straightening rate `$O(1/K)$` will translate to practical improvements in their domain. **What evidence exists in the paper.** CIFAR-10 results (Table 1, Figure 8) are quantitative and thorough. High-resolution results (Figure 11) are qualitative only, with 3 examples per dataset. Domain adaptation (Table 2) is on latent representations, not raw data, and uses a smaller version of the DDPM++ architecture. No experiments on text, audio, video, 3D data, or other modalities. No experiments with transformer-based velocity models (despite their growing importance in generative modeling). No experiments on datasets larger than 50,000 samples (CIFAR-10) for quantitative evaluation. **Mitigation status.** The paper acknowledges none of these scope limitations explicitly. The domain adaptation results (Table 2) provide the only evidence of the method working outside the CIFAR-10 image generation setting, but these experiments use latent features rather than raw data and a classification accuracy metric that is different from the generative quality metrics used elsewhere. The high-resolution qualitative results suggest the method scales to larger images, but without quantitative metrics, the evidence is suggestive at best. --- ### 6.6 Sequential Dependency of Reflow Precludes Parallel Training and Imposes Wall-Clock Overhead **The assumption or constraint.** The reflow procedure is inherently sequential: each rectified flow `$\mathbf{Z}^{k+1}$` depends on the coupling `$(Z_0^k, Z_1^k)$` produced by the previous flow. Training `$\mathbf{Z}^2$` cannot begin until `$\mathbf{Z}^1$` is fully trained and simulated on the training set. Training `$\mathbf{Z}^3$` cannot begin until `$\mathbf{Z}^2$` is complete. This creates a pipeline with serial dependencies that cannot be parallelized across the `$k$` reflow steps. The paper describes the procedure in Section 5: "The second rectified flow `$\mathbf{Z}^2$` is obtained by the same procedure except with the data replaced by the draws from `$(Z_0^1, Z_1^1)$`, obtained by simulating the first rectified flow `$\mathbf{Z}^1$`. This process is repeated for `$k$` times to get the `$k$`-rectified flow `$\mathbf{Z}^k$`." **The consequence.** The wall-clock time to produce a `$k$`-rectified flow is roughly `$k$` times the time to train a single flow, plus the simulation time for data generation at each step. For a practitioner with limited computational resources, this sequential dependency means that deploying a 2-rectified flow requires at minimum: (1) train 1-rectified flow, (2) simulate it on the full training set, (3) train 2-rectified flow from scratch on the new data, (4) optionally distill. Each training step is a full model training run (the paper uses the configurations from [73], which for CIFAR-10 represents substantial GPU-hours). This cost is not accounted for anywhere in the paper's comparisons. In contrast, a practitioner deploying a diffusion model with an accelerated sampler (e.g., DDIM with 50 steps) incurs only one training run. The headline speedup — one-step inference vs. 1,000-step diffusion — ignores the fact that achieving one-step capability required multiple full training cycles. Depending on the ratio of training cost to inference volume, the total-FLOPs advantage may only materialize after generating a very large number of images. **What evidence exists in the paper.** The paper does not report training time, number of training iterations, GPU-hours, or wall-clock time for any experiment. The NFE metric used throughout counts only inference-time function evaluations. Figure 8(b) shows FID and recall vs. training iterations, but the x-axis is "Training Iterations × `$10^4$`" within a single reflow round — it does not aggregate across reflow rounds. The toy experiments (Figure 3) use non-parametric estimators that have no training cost, making them uninformative about practical scaling. The paper notes the sequential dependency implicitly when describing the procedure but does not discuss its practical implications. **Mitigation status.** Not addressed. The paper does not discuss strategies to reduce the wall-clock overhead of reflow, such as: using checkpointed models across reflow steps (fine-tuning rather than retraining), using fewer training iterations for later reflow steps (if convergence is faster from the straightened data), generating reflow training data with fewer simulation steps (trading off data quality for speed), or stopping reflow early based on a straightness criterion. The distillation step (Phase 2) adds additional training cost on top of reflow. A practitioner weighing rectified flow against alternative acceleration methods (progressive distillation, consistency models, GAN hybrid approaches) lacks the information needed to compare total computational budgets. ## 7. Implications and Future Directions ### How This Work Changes the Landscape This paper fundamentally reframes the problem of learning continuous-time transport models by separating the *path geometry* of an ODE from its *marginal-matching property*. Before rectified flow, the dominant approaches to neural ODE-based generative modeling — probability flow ODEs [73] and DDIM [70] — inherited their path geometry as a byproduct of SDE derivations. The field largely accepted that the curved, non-uniform-speed trajectories produced by these methods (documented in Figure 5) were an unavoidable consequence of the training framework. Rectified flow demonstrates that this coupling is artificial: **any time-differentiable interpolation curve determines valid ODE training, and the straight-line geodesic is the canonical choice for Euclidean data**. This is not merely a new method — it is a conceptual simplification that reveals prior approaches as instances of a more general framework with suboptimal parameter choices (Proposition 3.11). The magnitude of this shift is **moderate but specific**. It does not overturn the core insight of diffusion models (regression-based training avoids GAN instability), but it **decouples that insight from the stochastic process machinery** that had become intertwined with it. The paper's re-analysis in Section 2.3.1 makes explicit what was previously implicit: the choices of α_t and β_t in VP ODE (exponential schedule, β_t = √(1-α_t²)) produce curved paths and non-uniform speed, while the trivial linear schedule (α_t = t, β_t = 1-t) yields straight, constant-speed trajectories that dramatically improve few-step simulation. This is a diagnostic contribution that will likely redirect how researchers design interpolation-based training objectives. The paper also resolves a tension that had been accumulating in the literature around optimal transport and generative modeling. Prior work [30] conjectured that DDIM yields L₂-optimal couplings, which was disproved [39, 78]. Rectified flow provides a clean resolution: **straightness and c-optimality are different properties** (Theorem 3.8), and for the purpose of fast inference, straightness — not optimal transport — is the relevant target. This reframing makes the negative results from [39, 78] less damaging: DDIM's failure to achieve L₂-optimality doesn't matter if straightness is the appropriate objective. It also redirects the optimal transport community toward a more nuanced question: when is c-optimality worth the computational cost versus settling for straight (but not optimal) couplings? Most consequentially, the **reflow procedure introduces a new axis of model improvement** — iterative straightening via recursive application of the same training procedure — that has no direct analog in prior generative modeling. This is distinct from distillation (which compresses an existing model) and from progressive training (which gradually increases resolution or complexity). Reflow produces a genuinely different coupling at each step, with provably decreasing convex transport costs (Theorem 3.5) and O(1/K) straightening rate (Theorem 3.7). This opens the door to considering the coupling itself as an optimizable quantity, not a fixed target determined by the initial random pairing. ### Follow-Up Research This Work Enables **Cheap difficulty estimation for adaptive reflow depth.** The paper identifies but does not resolve a practical tension: reflow depth k creates a trade-off between few-step quality (improves with k) and asymptotic quality (degrades with k, as seen in Table 1a where full-simulation FID worsens from 2.58 at k=1 to 3.96 at k=3). A natural follow-up would train a lightweight "straightness predictor" that, given a partially trained velocity field, estimates how many reflow steps are needed to reach a target straightness threshold — or alternatively, stops reflow early when the marginal improvement in S(Z) falls below some criterion. The experiment would measure total FLOPs (training + reflow data generation + inference) to reach a target FID at a target NFE budget, comparing adaptive-depth reflow against fixed k = 1, 2, 3. This directly addresses the practical question the paper leaves open: "It is not recommended to apply too many reflow steps as it may accumulate estimation error" — but how many is too many, and can we detect it automatically? **Scaling reflow to transformer-based velocity models on ImageNet.** The paper's quantitative evidence is limited to CIFAR-10 (32×32, 50K images) with a U-Net architecture. DiT [61] and related work have shown that transformers can serve as effective backbones for diffusion models at scale. A strong follow-up would replicate the CIFAR-10 reflow experiments (measuring FID vs. NFE for k = 1, 2, 3 with and without distillation) using a DiT-style transformer velocity model on ImageNet 64×64 or 128×128. The key question: does the O(1/K) straightening rate transfer to transformer architectures and higher-dimensional data, where the curse of dimensionality might make random paths naturally less intersecting (reducing the benefit of reflow)? This experiment would stress-test whether straightness benefits are architecture-dependent or truly general. **Combining reflow with higher-order ODE solvers.** The paper uses only Euler discretization for few-step evaluation and RK45 for full simulation. Recent work on accelerated diffusion sampling (DPM-Solver [46], UniPC [96]) has shown that higher-order solvers can extract dramatically better quality from the same NFE budget. A natural follow-up would evaluate rectified flow (k = 1, 2, 3) using DPM-Solver or a similar exponential-integrator method, measuring FID at NFE = 1, 2, 5, 10, 20. The hypothesis: higher-order solvers will benefit 1-rectified flow more than 2-rectified flow (since curved paths require more accurate integration), potentially narrowing the gap that reflow currently fills. This would clarify whether reflow is strictly necessary or whether better solvers can partially substitute for straightness — a question with direct practical implications for deployment engineering. **Failure mode analysis: when does reflow straightening fail?** The paper demonstrates straightening on toy examples and CIFAR-10 but does not explore boundary conditions. A systematic failure analysis would test reflow on distribution pairs deliberately constructed to resist straightening: (a) multi-modal distributions where π₀ and π₁ have incompatible topological structures (e.g., π₀ is a single Gaussian and π₁ is two separated clusters — can reflow produce a straight coupling, or does it get stuck at a curved fixed point?), (b) distributions with very different intrinsic dimensionality (e.g., π₀ on a low-dimensional manifold embedded in high dimensions), and (c) the empirical limit of error accumulation by running reflow to convergence (k = 10, 20, 50 steps) and measuring when S(Z) stops decreasing and FID begins to degrade. This would provide the first empiricalcharacterization of the reflow fixed point in practical settings, informing whether the theoretical O(1/K) rate is tight or conservative. **Rectified flow for discrete or structured data.** The paper's framework assumes continuous Euclidean data amenable to linear interpolation. Extending rectified flow to discrete domains (text tokens, molecular graphs, code) requires defining meaningful interpolation paths between discrete objects. A concrete experiment would use embeddings from a pre-trained language model as the interpolation space: given text from two distributions (e.g., formal vs. informal writing), embed sentences into a continuous space using a frozen encoder, train rectified flow on the embeddings, then decode the transported embeddings back to text. The evaluation would measure style transfer accuracy (via a classifier) and content preservation (via BLEU or semantic similarity). This tests the paper's claim that rectified flow provides a "unified solution" beyond image domains, while also surfacing the challenges of decoding from the continuous flow back to discrete output — a problem the paper does not address. **Test-time compute scaling laws for reflow depth vs. solver steps.** The paper's central empirical finding is a trade-off: reflow improves low-NFE quality but degrades high-NFE quality. This suggests an optimal operating point that depends on the inference budget. A systematic scaling study would measure FID on CIFAR-10 for a 2D grid of (reflow depth k = 0, 1, 2, 3, 4) × (NFE = 1, 2, 4, 8, 16, 32, 64, 128, 256) and produce a "compute-optimal frontier" analogous to the Chinchilla scaling laws for pretraining. The result would be a practical lookup table: given a target NFE budget, which k minimizes FID? Given a target FID, which (k, NFE) pair minimizes total FLOPs (including reflow training cost)? This would transform reflow from a heuristic ("use k=2 or 3") into a principled resource allocation decision. ### Practical Applications and Downstream Use Cases **On-device one-step image generation for mobile applications.** The distilled 2-rectified flow achieves FID = 4.85 on CIFAR-10 with a single neural network evaluation — a one-step model competitive with multi-step diffusion and GAN methods (Table 1). For mobile deployment where GPU memory and latency are severely constrained, this is directly actionable: a 2-rectified flow distilled into a single forward pass eliminates the iterative ODE solver entirely, reducing inference latency by ~100× compared to a 100-step diffusion model and ~10× compared to DDIM with 10 steps. The key deployment scenario is on-device image generation (e.g., photo editing, avatar creation) where cloud round-trips are unacceptable. The paper's evidence for this is CIFAR-10-specific; practitioners would need to validate on target-resolution datasets, but the 256×256 qualitative results (Figure 11) and the 2-rectified flow's one-step translation results (Figure 14) suggest the approach scales. The primary implementation requirement is training a 2-rectified flow followed by distillation, which the paper's Algorithm 1 specifies completely. **Fast domain adaptation for production ML pipelines with distribution shift.** The domain adaptation results (Table 2) show 1-rectified flow achieving 69.2% on Office-Home and 41.4% on DomainNet, matching or exceeding specialized adaptation methods like CORAL. For a production ML pipeline where the test distribution drifts from the training distribution (common in e-commerce, content moderation, and sensor-based applications), rectified flow provides a simple adaptation mechanism: train a flow between the source and target domain features at deployment time, then transport target samples to the source domain before running the existing classifier. The paper's method requires no adversarial training, no cycle-consistency constraints, and no architectural modifications to the downstream classifier — it operates purely on the feature representations. The inference cost is configurable: 100 Euler steps for full quality or fewer with some quality trade-off. The key practical advantage over CORAL and other moment-matching methods is that rectified flow learns a full nonlinear transport map rather than just aligning feature covariances, potentially capturing more complex distribution shifts, as the Office-Home improvement (+0.5% over CORAL) suggests. **Fast image-to-image translation for creative tools.** The translation results (Figures 13–15) demonstrate that 2-rectified flow produces high-quality domain translations (cat↔wild, human↔cat, MetFace↔cat) in a single Euler step. For creative applications (style transfer in photo editing software, real-time video filters), the ability to translate between domains in one forward pass — without running an iterative ODE solver — makes deployment on consumer hardware feasible. The 1-step 2-rectified flow translation quality (Figure 14) appears comparable to 100-step 1-rectified flow, representing a ~100× speedup for interactive applications. The feature-weighted loss (Equation 20) provides a mechanism to control the style-content trade-off by choosing the domain classifier h. A practical implementation would pre-train 2-rectified flows for a library of domain pairs (e.g., photo→sketch, day→night, summer→winter) and deploy the distilled one-step models as filters in a creative tool, with the classifier h trained on user-provided examples of each style. ### When to Prefer This Method The paper explicitly positions rectified flow against both diffusion/SDE-based methods and GAN-based one-step models, providing clear criteria for method selection: - **Prefer rectified flow with reflow + distillation when** inference latency is the primary constraint and the target application can tolerate a more expensive training procedure. The distilled 2-rectified flow achieves one-step generation quality (FID = 4.85) that significantly exceeds other one-step U-Net models (TDPM at 8.91, DDIM Distillation at 9.36) and approaches the diversity of multi-step methods (Recall = 0.50 vs. SDE Recall = 0.58). This is the appropriate choice when generating large volumes of images at deployment time amortizes the extra training cost of reflow data generation and retraining. The paper's evidence for this trade-off is concentrated in Figure 8 and Table 1a. - **Prefer 1-rectified flow (no reflow) when** asymptotic quality at moderate-to-high NFE is the priority. 1-rectified flow achieves the best full-simulation FID among all ODE methods (2.58 at NFE = 127, Table 1a), outperforming VP ODE (3.93), sub-VP ODE (3.16), and VE ODE (5.38), and matching SDE quality (VP SDE at 2.55) at ~16× lower inference cost. Training requires only one round of the simple regression objective, with no reflow data generation overhead. This is the appropriate choice when some inference-time compute is available (100–150 NFE) and training simplicity is valued, or when the model will be evaluated across a wide range of NFE budgets (since 1-rectified flow degrades more gracefully at high NFE than 2- or 3-rectified flow). - **Prefer rectified flow over CycleGAN for domain transfer when** a unified training procedure for both generation and transfer is desired, and GAN stability is a concern. Rectified flow uses the same regression objective for both tasks, avoiding the minimax optimization and cycle-consistency tuning required by CycleGAN. The ODE reversibility automatically provides cycle consistency without an explicit loss term. The paper does not provide quantitative comparison to CycleGAN, so this preference is based on training simplicity and stability rather than demonstrated output quality superiority. - **Prefer SDE-based diffusion over rectified flow when** the data has inherent stochasticity that a deterministic ODE cannot capture (e.g., modeling Brownian motion in physical systems, financial time series with intrinsic noise). The paper acknowledges this limitation explicitly: "SDEs may be preferred if we need to capture richer time-correlation structures." The rectified flow coupling (Z₀, Z₁) is deterministic, which is appropriate for most image generation but constrains the expressiveness of the latent representation. - **Avoid rectified flow with reflow when** the total computational budget for training is severely limited relative to inference volume. The reflow procedure requires generating training data by simulating the previous flow on the full dataset (e.g., ~6.4M NFE for CIFAR-10 2-rectified flow data generation, as calculated from 50K samples × 127 NFE/sample) plus a full retraining cycle per reflow step. If the total number of images to be generated at deployment is small, the amortized cost per generated image may exceed simply running a 100-step diffusion model or 1-rectified flow. The paper provides no break-even analysis for this trade-off; practitioners should estimate their own total-FLOPs budget before committing to the reflow pipeline.