On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers

ArXiv: 2603.28762

🎯 Pitch

Modern diffusion models like Flux are remarkably consistent—they collapse onto nearly identical images for the same prompt across different seeds. This paper shows that a 20–30% runtime overhead is enough to break that habit by injecting a repulsive force directly into the text-image attention channels mid-generation, yielding strikingly varied outputs even in distilled Turbo models where prior diversity tricks fail.

---

1. Executive Summary

This paper introduces on-the-fly repulsion in the Contextual Space, a novel framework for increasing diversity in Diffusion Transformer (DiT) text-to-image models by intervening in the multimodal attention channels during the forward pass rather than modifying inputs or image latents. The approach applies repulsive forces directly to the enriched text tokens that emerge from bidirectional text-image attention blocks — what the authors term the Contextual Space (operationalized as flattening the enriched text token sequence and computing a batch-level Vendi Score diversity loss) — using the particle guidance framework with on-the-fly gradient computation that requires no backpropagation through the model. Evaluated on Flux-dev, SD3.5-Large, and SD3.5-Turbo using the MS-COCO 2017 validation set, the method achieves a superior Pareto frontier across diversity, image quality, and prompt alignment metrics while imposing only a 20–30% runtime overhead, establishing that meaningful diversity can be recovered in modern distilled "Turbo" models where traditional trajectory-based interventions fail.

2. Context and Motivation

The Core Problem: Text-to-Image Models Have a Diversity Problem

The fundamental issue this paper addresses is deceptively simple: state-of-the-art text-to-image (T2I) models like Flux, SDXL, and Stable Diffusion 3.5 generate high-quality images that strongly align with prompts, but they produce remarkably little variety across different random seeds. Given the same prompt — "a jazz musician playing saxophone in a dimly lit club" — the base Flux model will generate images that share nearly identical compositions, lighting conditions, camera angles, and subject appearances across multiple seeds (Figure 11 in the paper). The model converges on what the paper describes as a "narrow set of typical visual solutions" — a phenomenon the field has begun calling typicality bias (Teotia et al., 2025).

This is not merely a cosmetic concern. The paper argues that what separates a genuine generative engine from a "sophisticated retrieval mechanism" is precisely the ability to explore the manifold of possible visual realizations for a given concept. When a user prompts "a phoenix rising from ashes," a useful creative tool should offer genuinely different interpretations — perhaps varying artistic style, composition, the phoenix's design, the environment, and the emotional tone. Instead, modern T2I models gravitate toward a single "preferred" visual template, dramatically limiting their utility for creative exploration, ideation, and applications where a wide range of candidate outputs is valuable (design iteration, concept art, content variation at scale).

The paper locates a structural reason for this collapse: Classifier-Free Guidance (CFG) (Ho and Salimans, 2022). CFG is the dominant technique that sharpens a model's probability distribution around modes that strongly satisfy the conditioning signal. It works by computing a linear combination of the conditional and unconditional score estimates:

$\epsilon_\theta(x_t, c) = \epsilon_\theta(x_t, \emptyset) + w \cdot (\epsilon_\theta(x_t, c) - \epsilon_\theta(x_t, \emptyset))$

where $w > 1$ amplifies the conditioning effect. While this dramatically improves prompt alignment and image quality, it does so by suppressing nearby semantically valid alternatives — secondary modes of the distribution that would satisfy the prompt equally well but differ in composition, style, or subject appearance. The model's probability mass concentrates around the single most "typical" solution. Restoring diversity, then, requires an efficient mechanism to overcome this CFG-induced mode collapse without degrading the structural integrity or semantic faithfulness of the output.

Why Prior Diversity Approaches Fall Short

The paper organizes existing diversity-enhancement methods into two categories based on where in the generative pipeline they intervene, and systematically identifies the limitations of each. This taxonomy (Figure 2) is central to understanding the paper's motivation.

Upstream Methods: Modifying Inputs Before Generation

Upstream approaches attempt to induce diversity by altering initial conditions — either the noise seed, the prompt embedding, or both — before the denoising process commits to a particular visual structure. The idea is intuitive: if the model always converges to the same mode from a given starting point, change the starting point.

CADS (Sadat et al., 2023) injects prompt-agnostic noise directly into the text conditioning embeddings. The mechanism is simple and requires no optimization, but it suffers from a critical weakness: lack of structural feedback. Because the intervention is decoupled from the actual generation process — the injected noise doesn't "know" what image structure is emerging — CADS frequently causes semantic drift, where diversity is achieved not through genuine conceptual variation but through weakened prompt alignment. The paper's qualitative results (Figure 12) show striking examples: for "a camera with old photographs," CADS replaces the scene with images of people entirely; for "a phoenix rising from ashes," it produces a bonfire instead. The diversity is real, but it comes at the cost of the prompt's meaning — a Pyrrhic victory.

MinorityPrompt (Um and Ye, 2025) and Scalable Group Inference (SGI) (Parmar et al., 2025) take a more sophisticated approach: they search for diversity-inducing initial conditions through optimization. SGI, for instance, filters a large pool of initial noise candidates ($N \in \{8, 16, 32, 64\}$), selecting only those that lead to diverse outputs according to a learned diversity metric. This achieves both high quality and strong prompt adherence because the intervention itself is minimal — the method essentially cherry-picks diverse seeds rather than actively steering the model. However, the paper identifies two critical drawbacks: computational overhead and bounded diversity. As Table 1 shows, SGI with 64 candidates requires 113.99 seconds on Flux-dev versus 10.34 seconds for the base model — more than 11× slower. More subtly, SGI can only find diversity that already exists in the base model's sampling distribution. For prompts where the base model has an inherent lack of diversity — "a red London double-decker bus" produces nearly identical vehicles across all seeds (Figure 12) — SGI's filtering strategy hits a ceiling: it can't induce diversity the base model doesn't already produce.

Contrastive Noise Optimization (CNO) (Kim et al., 2025) attempts to bridge the feedback gap by using the very first timestep's predicted clean image ($\hat{x}_0$) to estimate diversity and force divergence. But as the paper points out, these early $\hat{x}_0$ estimates are often structurally unformed at high noise levels — the model hasn't yet resolved what it's generating — providing an unstable signal for steering conceptual variety.

The paper synthesizes the upstream failure mode: early interventions lack structural feedback. Without knowing what image is actually forming, upstream methods cannot distinguish between diversity that comes from genuine conceptual variation and diversity that comes from breaking prompt adherence. Optimization-based methods partially address this by evaluating downstream outputs, but at the cost of substantial computation that makes them impractical for real-time or few-step models.

Downstream Methods: Manipulating Image Latents During Denoising

Downstream methods take the opposite approach: they intervene during the denoising process, applying repulsive forces directly to the image latents $z_t$ to push multiple samples apart from each other. The intuition is that by actively preventing samples from converging to the same region of pixel space, the model is forced to explore alternative visual realizations.

Particle Guidance (PG) (Corso et al., 2023) is the foundational approach. It treats a batch of $B$ samples as interacting particles and applies a repulsive potential at each denoising step:

$z_t^{(i)} \leftarrow z_t^{(i)} + \eta \nabla_{z_t^{(i)}} \mathcal{L}_{rep}(\{z_t^{(j)}\}_{j=1}^B)$

where $\mathcal{L}_{rep}$ measures pairwise similarity in latent space (typically using a kernel-based distance) and $\eta$ controls the repulsion strength. By pushing latents apart in pixel space, PG forces variation between samples. Subsequent works — SPARKE (Jalali et al., 2025) and DiverseFlow (Morshed and Boddeti, 2025) — refined the repulsion objectives, proposing better distance measures and more principled diversity losses.

However, the paper identifies a fundamental mismatch between where repulsion is applied (pixel-level latents) and where diversity is desired (semantic concepts). Image latents encode spatially-structured pixel information, not disentangled semantic attributes. The paper points out that "semantic concepts in the image latent space are spatially entangled and not aligned across samples." Concretely: if one sample places a bus on the left side of the image and another places it on the right, the pixel-level distance between these latents is large — but this doesn't reflect meaningful semantic diversity. Conversely, two samples might have the same overall composition but differ in subtle semantic attributes (the bus's color, the lighting), yet have small pixel-level distance.

This mismatch manifests in two failure modes:

1. Artifacts from manifold violation. Because pixel-level repulsion doesn't respect semantic boundaries, it can push latents outside the learned data manifold — regions of latent space where the decoder produces incoherent or unnatural images. The paper's qualitative results (Figure 10, 12) show striking examples: SPARKE produces "patterned holes" in images where objects have been moved (the bus example), and PG creates unnatural blending artifacts (the sea blending into the road in the "street" example). The paper's space ablation (Figure 9) quantifies this: repulsion in the image attention space ($\hat{f}_I$) shows sharp degradation in all quality metrics as diversity increases.

2. Insufficient trajectory depth in Turbo models. This is perhaps the most practically significant limitation. In modern distilled "Turbo" models (SD3.5-Turbo uses only 4 denoising steps; Flux-dev uses 20), the generative path is decided almost instantly. As the paper explains, downstream repulsion arrives too late — by the time the intervention applies enough force to meaningfully steer samples apart, the composition is already fixed. Traditional trajectory-based interventions like PG simply don't have enough steps to work with. This is particularly damaging because Turbo models, with their rapid inference, are precisely where efficient diversity methods would be most valuable in production settings.

Interval Guidance (Kynkäänniemi et al., 2024) takes a different downstream approach: rather than applying repulsive forces, it modulates the CFG scale itself during denoising — increasing guidance early to establish semantic alignment, then reducing it later to prevent over-convergence. But the paper points out that these fixed schedules are independent of the model's internal state, reducing prompt influence before the model has sufficiently established what it's generating. It's a one-size-fits-all solution that can't adapt to per-prompt or per-timestep needs.

The Interventional Trade-Off

The paper distills its analysis into a clean formulation of the core difficulty: there is an interventional trade-off. Upstream methods act too early, before the model has formed any visual structure to guide diversity in semantically meaningful directions. Downstream methods act too late, when the visual mode is already committed and the denoising trajectory is too short to meaningfully redirect it (compounded in Turbo models). As the paper states:

"upstream intervention must incur significant computational overhead to find valid diversity-inducing paths, while downstream interventions occur on a committed visual mode where the composition is already fixed, often producing noise-level variance that pushes samples outside the learned manifold and results in unnatural artifacts."

This trade-off is the paper's central motivation. The goal is to find a point of intervention that is structurally informed (like downstream methods, it knows what the image is becoming) but conceptually flexible (like upstream methods, it can redirect high-level intent without being locked into pixel-level commitments). The paper's core claim is that the Contextual Space — the enriched text tokens $\hat{f}_T$ emerging from multimodal attention blocks in DiT architectures — sits precisely at this sweet spot.

How Diffusion Transformers Enable a New Intervention Point

The paper's motivation is inseparable from the architectural shift from U-Net-based diffusion models (Stable Diffusion 1.x, 2.x, SDXL) to Diffusion Transformers (DiTs). Understanding this architectural difference is essential to understanding why the Contextual Space exists and why it hasn't been exploited before.

In U-Net architectures, text conditioning enters through cross-attention layers: the image features query the text embeddings, but the text embeddings themselves remain static throughout the denoising process. There is no mechanism for the emerging image to influence the text representation. Diversity interventions in U-Net models are therefore confined to the two categories already discussed — modify inputs or modify latents — because there is no "intermediate" space where text and image information is jointly represented.

In DiT architectures (Flux, SD3.x), text and image tokens are processed concurrently within multimodal attention blocks. In each such block, the model computes:

$\hat{f}_T^{(l)}, \hat{f}_I^{(l)} = \text{MM-Attn}(f_T^{(l-1)}, f_I^{(l-1)})$

where $f_T$ are text features and $f_I$ are image features. The attention mechanism allows text to guide image formation (as in cross-attention) but also allows image features to update text features. This creates a bidirectional exchange: the text tokens become "enriched" with information about the emerging image — they know what spatial layout is forming, what objects are appearing, what the composition looks like — while still encoding the original semantic intent from the prompt.

Critically, these enriched text tokens have fixed semantic alignment across the sequence index. Unlike image latent pixels, where the same conceptual attribute can shift spatially across samples (a "dog" might appear anywhere in the image), each text token index corresponds to the same conceptual component regardless of where it manifests spatially in different generations. This property means that repulsive forces applied in this space operate on semantic content rather than pixel position — pushing apart high-level concepts (style, composition, subject attributes) rather than low-level spatial features.

A growing body of research has confirmed that these internal text-image representations in DiTs encode interpretable semantic features (Helbling et al., 2025's ConceptAttention work). The present paper builds on this insight not to analyze what these features represent, but to exploit them as an intervention point for diversity control.

How This Paper Positions Itself

The paper positions its contribution as resolving the interventional trade-off by identifying and exploiting a previously overlooked intervention space. This is framed as a middle path (Figure 2(c)):

• Not upstream: Unlike CADS or SGI, the intervention does not modify initial conditions. It operates during the forward pass, after the model has begun forming visual structure. This means it has access to structural feedback — the intervention "knows" what the image is becoming and can steer diversity in semantically coherent directions.

• Not downstream: Unlike PG or SPARKE, the intervention does not act on image latents. It acts on semantic representations that are not tied to a specific spatial grid. This means pushing samples apart doesn't tear geometry or create artifacts — the model's attention mechanisms can resolve the repulsion into coherent visual alternatives because the repulsion operates at the level of "what to generate" rather than "where to place pixels."

• Uniquely efficient: Because the repulsion is computed in a space that is not the final image latent, it does not require backpropagation through the entire model — only through the diversity loss with respect to the contextual tokens themselves. This is what enables "on-the-fly" computation. The paper explicitly contrasts this with optimization-based upstream methods (SGI, MinorityPrompt) that require expensive iterative search or gradient-based optimization through the full denoising trajectory.

• Compatible with Turbo models: Because the Contextual Space is accessible even in the earliest denoising steps (the bidirectional exchange begins immediately), the intervention has sufficient trajectory depth to influence the generative path even in 4-step Turbo models where downstream interventions fail. As the paper states in the abstract: "our method... remains effective even in modern 'Turbo' and distilled models where traditional trajectory-based interventions typically fail."

The paper also positions itself within the broader discourse on the diversity-alignment gap in generative models. Rather than advocating for de-emphasizing prompt alignment (which interval guidance approaches do implicitly), the paper argues for a framing where diversity and alignment can be simultaneously achieved — the Pareto frontier analysis (Figure 6) is central to this positioning. The goal is not to maximize diversity at the cost of quality (as CADS does) or maximize quality at the cost of diversity (as the base model does), but to push outward the frontier that trades off between them — achieving more diversity at any given quality level, or higher quality at any given diversity level.

This positioning reflects a maturing of the diversity research agenda. Earlier work often treated diversity as something to be restored at the cost of other desiderata. The present paper argues for diversity as a design parameter that can be controlled without degradation — a shift from "how much quality are we willing to sacrifice for diversity?" to "how can we have both?"

3. Technical Approach

3.1 Reader Orientation

This paper presents a training-free, inference-only intervention system that plugs into the forward pass of existing Diffusion Transformer (DiT) models to increase the visual diversity of generated images when sampling multiple outputs from the same text prompt. The system solves the problem of typicality bias — the tendency of modern T2I models to converge on near-identical visual solutions across different random seeds — by identifying a previously overlooked representation space inside the transformer where semantic intent can be redirected before the image composition crystallizes, without requiring any model retraining, extra memory, or backpropagation through the full denoising process.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four conceptual components, all operating within a standard DiT inference pipeline:

1. Base Diffusion Transformer (Flux-dev, SD3.5-Large, or SD3.5-Turbo) — the frozen pretrained model that performs the standard denoising process. The user provides a text prompt and a batch of $B$ random initial noise seeds, and the model would normally produce $B$ independent images.

2. Contextual Space Extractor — at each multimodal attention block $l$ within the DiT's forward pass, this component captures the enriched text tokens $\hat{f}_T^{(l)}$ that have been updated by bidirectional text-image attention. These tokens form the Contextual Space: a set of $N$ vectors, each of dimension $D$, that encode the model's current semantic intent for each conceptual component of the prompt.

3. Batch-Level Diversity Computer — for the current batch of $B$ samples, this component flattens each sample's $N \times D$ contextual token matrix into a single feature vector $\mathbf{c}_i^{(l)}$, computes a $B \times B$ cosine similarity kernel matrix $\mathbf{K}$ across the batch, and evaluates a diversity loss $\mathcal{L}_{div}$ based on the von Neumann entropy of the normalized kernel. This loss measures how effectively the batch spans the space of possible semantic realizations.

4. On-the-Fly Repulsion Applier — using the gradient of the diversity loss with respect to the contextual tokens, this component applies a repulsive update to each sample's $\hat{f}_T^{(l)}$ tokens, pushing them apart in Contextual Space. The updated tokens then continue through the rest of the transformer block normally. This update is computed on-the-fly (no backpropagation through downstream layers) and applied only during the first $\tau$ denoising timesteps.

Information flows as follows: a text prompt enters the DiT → the model initializes $B$ noisy latents from different random seeds → at each denoising timestep $t$, the noisy latents and text embeddings pass through successive transformer blocks → at each multimodal attention block, the enriched text tokens are extracted → if $t$ is within the intervention window ($t \leq \tau$), the repulsive update is computed and applied to these tokens → the modified tokens continue through the block → the denoising process completes → $B$ diverse images are produced.

3.3 Roadmap for the Deep Dive

• First, the formal definition of the Contextual Space — what exactly these enriched text tokens are, why they exist in DiTs but not U-Nets, and what properties (fixed token ordering, spatial disentanglement) make them viable for semantic repulsion.
• Second, the repulsion mechanism — how the particle guidance framework is adapted from latent-space repulsion to Contextual Space repulsion, including the update equation, why on-the-fly computation works, and the crucial design choice of restricting intervention to early timesteps.
• Third, the diversity objective — how the Vendi Score is computed from the kernel matrix of batch-level similarity vectors, why von Neumann entropy is a principled diversity measure, and what the loss function penalizes.
• Fourth, the intervention scheduling — why repulsion is applied only during early timesteps and how the timestep window $\tau$ and repulsion scale $\eta$ jointly control the diversity-quality trade-off.
• Fifth, implementation details — the exact hyperparameter ranges per model (Flux-dev, SD3.5-Large, SD3.5-Turbo), the number of inner-block iterations $M$, and the design decision to keep $\eta$ constant within the intervention window rather than scheduling it.
• Sixth, the architectural generality — how the method adapts to different DiT variants (Flux's dual-stream vs. single-stream blocks, guidance-distilled vs. standard models) and why it transfers seamlessly to image editing models like Flux-Kontext.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a method paper with an empirical validation component. Its core novel idea is to shift the locus of diversity intervention from initial conditions or image latents to the Contextual Space inside DiT multimodal attention blocks, exploiting the bidirectional text-image exchange that continuously updates text representations with emergent visual structure. The method does not train any new parameters; it computes a repulsive update on-the-fly during the forward pass, using only the contextual tokens that already exist inside the architecture.

---

The Contextual Space: Definition and Properties

The Contextual Space is the high-dimensional manifold formed within the Multimodal Attention (MM-Attention) blocks of a Diffusion Transformer. To understand what this space is and why it exists, we need to contrast DiT conditioning with the U-Net conditioning used in earlier diffusion models like Stable Diffusion 1.x and SDXL.

In a U-Net architecture, text conditioning enters through cross-attention layers. At each such layer, the image features act as queries and the text embeddings act as keys and values. The attention output updates the image features, but the text embeddings themselves are never modified — they are read-only. This means the text representation remains exactly the same at every denoising timestep and at every spatial location in the U-Net. It encodes the static meaning of the prompt, nothing more. There is no mechanism for the emerging image structure to feed back into the text representation.

In a DiT architecture, text tokens and image tokens are processed concurrently as a single sequence within transformer blocks. The attention mechanism is fully bidirectional: image tokens can attend to text tokens, and text tokens can attend to image tokens. In each transformer block $l$, the model computes:

$\hat{f}_T^{(l)}, \hat{f}_I^{(l)} = \text{MM-Attn}(f_T^{(l-1)}, f_I^{(l-1)})$

where $f_T^{(l-1)}$ are the text features entering block $l$ (a matrix of shape $N_T \times D$, where $N_T$ is the number of text tokens and $D$ is the model's hidden dimension), $f_I^{(l-1)}$ are the image features entering block $l$ (a matrix of shape $N_I \times D$, where $N_I$ is the number of image tokens), and $\hat{f}_T^{(l)}$ and $\hat{f}_I^{(l)}$ are the updated text and image features after bidirectional attention.

What it computes: for each text token $i$ and each image token $j$, the MM-Attn operation computes attention weights that describe how relevant image patch $j$ is to text token $i$. The text token $i$'s representation is then updated as a weighted sum of the image token values, where the weights are these relevance scores. Simultaneously, each image token $j$ is updated as a weighted sum of the text token values. The outputs $\hat{f}_T^{(l)}$ and $\hat{f}_I^{(l)}$ are the transformed representations — the text tokens now encode information about the emerging image structure, and the image tokens now encode refined semantic guidance from the text.

Why this happens block by block: the MM-Attn operation occurs in every multimodal attention block along the DiT's depth, not just at a single cross-attention layer. This means the text tokens are continuously updated as the image tokens evolve — in early blocks, they capture coarse layout information; in later blocks, they capture fine-grained visual details. This is what makes the Contextual Space "structurally informed": by the time the enriched text tokens exit block $l$, they have absorbed information about the image's composition up to that point.

The paper identifies the enriched text tokens $\hat{f}_T^{(l)}$ — not the static prompt embeddings at the input, and not the image tokens that encode pixel-level structure — as the elements of the Contextual Space. These are the tokens that the paper intervenes on.

Two properties of this space are critical for the method's effectiveness:

Property 1: Fixed semantic alignment across the token sequence index. The sequence of text tokens has a fixed ordering that corresponds to the original prompt tokens (and padding). Token index $i$ always represents the $i$-th conceptual component of the prompt, regardless of where that component manifests spatially in different generated images. For example, if the prompt is "A red London double-decker bus on a rainy street," the token index corresponding to "bus" will always represent the bus concept, even though in one generated image the bus might appear on the left side, in another on the right, and in another centered. This is fundamentally different from the image latent space, where pixel $(x, y)$ in sample 1 and pixel $(x, y)$ in sample 2 may correspond to entirely different semantic objects because the spatial layout varies between generations.

Property 2: Spatial disentanglement from image structure. Because the enriched text tokens encode "what" the model intends to generate (semantic concepts) rather than "where" (spatial positions), repulsive forces applied in this space push apart high-level concepts — style, composition, subject attributes, object types — rather than low-level pixel arrangements. The paper's interpolation experiment (Figure 3) empirically demonstrates this property: blending Contextual Space features produces smooth semantic transitions between two images (e.g., a pet gradually shifting from dog-like to cat-like appearance), while blending VAE latent space features produces structural blur and ghosting artifacts because the two images' spatial grids are misaligned.

This spatial disentanglement is what makes repulsion in the Contextual Space fundamentally different from repulsion in the image latent space. Latent-space repulsion (as in PG or SPARKE) pushes pixel values apart, which can mean pushing semantically meaningless high-frequency detail or spatially shifting objects. Contextual Space repulsion pushes semantic intent apart — it tells the model "generate a different kind of bus" rather than "move the bus a few pixels to the left."

---

The Repulsion Mechanism: Particle Guidance Adapted to the Contextual Space

The paper adopts the particle guidance framework introduced by Corso et al. (2023) but redirects it from the image latent space to the Contextual Space. The core idea of particle guidance is to treat a batch of $B$ simultaneously generated samples as an interacting particle system, where each particle (sample) experiences a repulsive force from every other particle based on their similarity in some feature space. The original particle guidance applied this force to the noisy image latents $z_t$ at each denoising step. This paper applies it instead to the enriched text tokens $\hat{f}_T^{(l)}$ at each multimodal attention block.

For a given transformer block $l$, a given sample $i$ in the batch of size $B$, and a given inner iteration step within that block, the repulsive update is:

$\hat{f}_{T,i}^{(l)\prime} = \hat{f}_{T,i}^{(l)} + \frac{\eta}{M} \nabla_{\hat{f}_{T,i}^{(l)}} \mathcal{L}_{div}(\{\hat{f}_{T,j}^{(l)}\}_{j=1}^{B})$

where $\hat{f}_{T,i}^{(l)}$ is the enriched text token representation for sample $i$ at block $l$ (a matrix of shape $N \times D$, with $N$ text tokens each of dimension $D$), $\hat{f}_{T,i}^{(l)\prime}$ is the updated representation after applying repulsion, $\eta$ is the overall repulsion scale (a scalar hyperparameter that controls the strength of the diversity-inducing force), $M$ is the number of inner-block iterations (gradient steps applied within this block before the tokens continue forward), and $\nabla_{\hat{f}_{T,i}^{(l)}} \mathcal{L}_{div}$ is the gradient of the batch diversity loss with respect to sample $i$'s contextual tokens.

What it computes: for each sample $i$ in the batch, compute the gradient of the diversity loss $\mathcal{L}_{div}$ with respect to $\hat{f}_{T,i}^{(l)}$. This gradient is a matrix of the same shape $N \times D$ as the contextual tokens. It points in the direction in Contextual Space that would most increase the batch's diversity — essentially, it tells sample $i$ how to move its semantic representation to make the batch more spread out. Scale this gradient by $\eta / M$ and add it to the current tokens. The updated tokens $\hat{f}_{T,i}^{(l)\prime}$ now encode a version of the semantic intent that has been nudged away from the other samples' intents.

Why the $\eta / M$ scaling: dividing by $M$ ensures that after $M$ inner iterations, the total accumulated update is approximately $\eta$ times the initial gradient (assuming the gradient doesn't change dramatically across iterations). This makes the effective repulsion strength independent of the number of inner iterations, which is a standard practice when using multiple gradient steps with a fixed step size. The parameter $\eta$ therefore directly controls "how strongly do we push samples apart" regardless of how many small steps we break that push into.

Why on-the-fly computation works without backpropagation: this is a crucial efficiency property of the method. The gradient $\nabla_{\hat{f}_{T,i}^{(l)}} \mathcal{L}_{div}$ is computed only with respect to the contextual tokens themselves — it does not require backpropagating through the downstream layers of the network. Why? Because the diversity loss $\mathcal{L}_{div}$ is computed directly from the current batch of contextual tokens $\{\hat{f}_{T,j}^{(l)}\}_{j=1}^B$. There is no dependency on later network outputs. The computation is:

1. Extract $\hat{f}_{T,j}^{(l)}$ for all $j \in \{1, \dots, B\}$ from the current forward pass.
2. Compute $\mathcal{L}_{div}$ as a function of these tokens.
3. Compute $\nabla_{\hat{f}_{T,i}^{(l)}} \mathcal{L}_{div}$ using standard automatic differentiation, but stopping the computation graph at the token level.
4. Apply the update and continue the forward pass with the modified tokens.

Step 3 is the only "extra" computation beyond the normal forward pass. It requires computing gradients through the diversity loss function, but not through any neural network layers. The diversity loss function itself is a simple function of the kernel matrix and its eigenvalues — it has a closed-form gradient that can be computed efficiently. This is what keeps the overhead at 20–30% rather than the 400–1100% overhead of optimization-based methods like SGI.

The inner-block iteration loop: the paper applies $M = 100$ gradient steps for Stable Diffusion models and $M = 50$ for Flux-dev within each block. Each step computes the gradient and updates the tokens according to the equation above. The purpose of multiple inner steps is to allow the tokens to find a configuration that satisfies the diversity objective well before they continue to the next block. A single gradient step might not be sufficient to push tokens meaningfully apart, especially if the initial configuration is highly concentrated (as it will be when CFG is collapsing diversity). The multiple inner steps approximate finding a local minimum of the diversity loss within the Contextual Space of the current block.

The paper explicitly states that this is more computationally efficient than optimization-based methods like MinorityPrompt and SGI because it does not require backpropagation through model layers. However, it does still require computing and applying the gradient update within each multimodal attention block, which is what accounts for the 20–30% runtime increase relative to the base model.

A critical design choice: the conditioning is re-initialized from the unmodified prompt embedding at every timestep. At each denoising step $t$, the model starts its forward pass with the original, unmodified text embeddings $f_T^{(0)}$ (the output of the text encoder, before any multimodal attention). The Contextual Space intervention modifies $\hat{f}_T^{(l)}$ within the blocks at timestep $t$, but at timestep $t+1$, the model begins again from $f_T^{(0)}$. This design has a subtle but important consequence: the repulsive force at each timestep is a "correction" to the guidance signal, not a permanent modification of the conditioning. As the paper states:

"Since the conditioning for each sample is initialized from the same unmodified prompt encoding at every timestep, the intervention mitigates the risk of permanent semantic drift."

In other words, even if the repulsion at a given timestep pushes the contextual tokens into a region that slightly misaligns with the prompt, the next timestep will restart from the correct prompt embedding. The repulsion acts as a per-timestep perturbation that steers the guidance trajectory, not as a cumulatively diverging modification. This is in contrast to upstream methods like CADS, which permanently modify the text embeddings before generation begins — any semantic drift introduced by CADS persists throughout the entire denoising process and can accumulate.

The fact that all samples start from the same unmodified prompt encoding at each timestep also ensures that the contextual features remain "directly comparable across the batch," as the paper puts it. Because the starting point is identical, differences in the enriched text tokens $\hat{f}_T^{(l)}$ are attributable to differences in the emerging image structures (which differ across samples due to different noise seeds), not to differences in the conditioning. This makes the similarity computation in the diversity loss meaningful — if two samples have similar contextual tokens, it's because their images are converging semantically, not because their conditioning was similar to begin with.

---

The Diversity Objective: Vendi Score via von Neumann Entropy

The paper chooses the Vendi Score (Friedman and Dieng, 2022) as the quantitative measure of batch diversity, and maximizes it by minimizing the negative von Neumann entropy of the batch's similarity kernel. This choice is motivated by the Vendi Score's principled interpretation as the "effective number of distinct samples" in a batch — it takes into account not just pairwise distances but the full eigenvalue spectrum of the similarity matrix, capturing both how spread out the samples are and how redundantly they cluster.

The computation proceeds in three steps:

Step 1: Construct per-sample feature vectors. For each sample $i$ at block $l$, the contextual tokens $\hat{f}_{T,i}^{(l)}$ (a matrix of shape $N \times D$) are flattened into a single vector:

$\mathbf{c}_i^{(l)} \in \mathbb{R}^{ND}$

where $N$ is the number of text tokens (the sequence length of the enriched text representation) and $D$ is the per-token hidden dimension. Flattening concatenates all $N$ token vectors of dimension $D$ into a single $ND$-dimensional feature vector that represents the entirety of sample $i$'s semantic intent at block $l$.

What this flattening does: it collapses the token sequence dimension into a single flat vector, treating the entire enriched text representation as one holistic feature. This is important because it means the diversity loss operates on the global semantic intent of each sample — not on individual tokens independently. The method does not apply per-token repulsion (e.g., pushing the "bus" token of sample 1 away from the "bus" token of sample 2); instead, it pushes the entire semantic representation of sample 1 away from that of sample 2. This is consistent with the goal of producing samples that are globally different in their realization of the prompt, rather than samples that differ on some tokens but are identical on others.

The paper notes that the framework is flexible and can adopt various diversity losses defined in prior work (Morshed and Boddeti, 2025; Jalali et al., 2025), but specifically chooses the Vendi Score as the primary objective. It does not provide an ablation comparing alternative diversity losses (e.g., simple pairwise cosine distance, RBF kernel-based repulsion), leaving open the question of how much the choice of loss matters.

Step 2: Build the batch similarity kernel matrix. For a batch of size $B$, compute the $B \times B$ kernel matrix $\mathbf{K}$ where each entry is the cosine similarity between two samples' flattened contextual vectors:

$K_{ij} = \frac{\langle \mathbf{c}_i^{(l)}, \mathbf{c}_j^{(l)} \rangle}{\|\mathbf{c}_i^{(l)}\| \|\mathbf{c}_j^{(l)}\|}$

where $\langle \cdot, \cdot \rangle$ denotes the dot product (the sum over all $ND$ dimensions of the elementwise product), and $\|\cdot\|$ denotes the Euclidean norm ($\ell^2$ norm, the square root of the sum of squared elements).

What it computes: for each pair of samples $i$ and $j$, $K_{ij}$ is a scalar between $-1$ and $1$ (since cosine similarity is bounded). $K_{ij} \approx 1$ means sample $i$ and sample $j$ have nearly identical semantic intent — their flattened contextual vectors point in the same direction. $K_{ij} \approx 0$ means they are orthogonal — their semantic intents are unrelated. $K_{ij} \approx -1$ means they are opposite — their semantic intents point in opposite directions (though this is rare in practice for samples from the same prompt). The diagonal entries $K_{ii}$ are always exactly $1$ because a vector's cosine similarity with itself is always $1$.

Why cosine similarity as the kernel: cosine similarity measures angular distance, which is invariant to the overall magnitude (norm) of the feature vectors. This is important because the magnitude $\|\mathbf{c}_i^{(l)}\|$ might vary across samples for reasons unrelated to semantic content — for instance, some images might simply produce stronger activations overall. Cosine similarity normalizes out the magnitude, focusing purely on the direction of the semantic intent vector. If two samples have the same "shape" of semantic intent but different intensities, cosine similarity will still identify them as similar. The paper does not ablate alternative kernel choices (RBF, linear, polynomial), so the sensitivity to this design choice is unknown.

The normalized kernel is then computed as:

$\tilde{\mathbf{K}} = \frac{1}{B} \mathbf{K}$

Dividing by $B$ ensures that the eigenvalues of $\tilde{\mathbf{K}}$ sum to $1$, which is a necessary condition for interpreting them as a probability distribution (which the von Neumann entropy requires). Without this normalization, the eigenvalues could sum to an arbitrary value depending on the batch size, making the entropy scale-dependent.

Step 3: Compute the diversity loss as negative von Neumann entropy. Compute the eigenvalues $\{\lambda_k\}_{k=1}^{B}$ of the normalized kernel $\tilde{\mathbf{K}}$. Since $\tilde{\mathbf{K}}$ is symmetric and positive semi-definite (it's a normalized cosine similarity matrix scaled to have unit trace), all eigenvalues are real and non-negative. The von Neumann entropy of the kernel is:

$H(\tilde{\mathbf{K}}) = -\sum_{k=1}^{B} \lambda_k \log \lambda_k$

The diversity loss is the negative of this entropy:

$\mathcal{L}_{div} = -H(\tilde{\mathbf{K}}) = -\sum_{k=1}^{B} \lambda_k \log \lambda_k$

where the convention $0 \log 0 = 0$ is used (since $\lim_{\lambda \to 0^+} \lambda \log \lambda = 0$).

What it computes: the von Neumann entropy measures how "spread out" the eigenvalue distribution is. Consider three scenarios:

• Complete collapse (all samples identical): the kernel matrix has all entries equal to $1$, which means all eigenvalues are $0$ except for one eigenvalue equal to $1$ (since the trace is $1$). Then $H = -(1 \cdot \log 1 + (B-1) \cdot 0 \cdot \log 0) = 0$. The entropy is zero — there is no diversity.

• Maximal diversity (all samples orthogonal): the kernel matrix is the identity matrix (all off-diagonal entries $0$), which means all eigenvalues are $1/B$. Then $H = -B \cdot \frac{1}{B} \log \frac{1}{B} = \log B$. The entropy is maximal — the effective number of distinct samples equals the batch size.

• Intermediate diversity: the eigenvalues fall somewhere between these extremes, with the entropy reflecting how many "effectively distinct" modes the samples represent. If the batch clusters into $k$ groups of near-identical samples, approximately $k$ eigenvalues will be large and the rest will be near zero, yielding an entropy close to $\log k$.

The Vendi Score is defined as $\exp(H)$, giving the "effective number of distinct samples." The Vendi Score ranges from $1$ (all samples identical) to $B$ (all samples fully distinct).

Why minimize $\mathcal{L}_{div}$: the objective is to maximize diversity, which means maximizing the von Neumann entropy $H$, which means minimizing its negative $-\sum \lambda_k \log \lambda_k$. This form is convenient because gradient-based optimization typically minimizes a loss function. When we compute $\nabla_{\hat{f}_{T,i}^{(l)}} \mathcal{L}_{div}$, we are computing the direction to move the contextual tokens to decrease $\mathcal{L}_{div}$, which increases diversity.

Why von Neumann entropy rather than simpler alternatives: the Vendi Score captures the full spectral structure of the similarity matrix, not just pairwise distances. Simpler diversity losses based on the sum of pairwise distances (e.g., $\mathcal{L} = \sum_{i \neq j} K_{ij}$) only measure whether pairs are similar, but don't account for higher-order redundancies — a batch where three samples cluster together and two samples are far apart might have the same average pairwise distance as a batch where all five samples are equally spread out. The Vendi Score's eigenvalue-based formulation penalizes clustering: if samples form tight groups, the kernel matrix has low-rank structure (few large eigenvalues), which reduces entropy even if the average pairwise distance is moderate. This is a more principled measure for the goal of maximizing the number of genuinely distinct visual solutions.

The paper does not provide a derivation of the gradient $\nabla_{\hat{f}_{T,i}^{(l)}} \mathcal{L}_{div}$ or discuss the computational cost of the eigenvalue decomposition. For a batch size of $B = 4$ (the primary evaluation setting), the eigenvalue decomposition of a $4 \times 4$ matrix is trivial. For the batch size scalability experiment (Table 6), the paper evaluates up to $B = 16$, where the eigenvalue decomposition remains inexpensive. The paper does not discuss whether the method would remain efficient at much larger batch sizes (e.g., $B = 100$), where the $O(B^3)$ eigenvalue decomposition might become noticeable.

---

Intervention Scheduling: Early Timesteps Only

The paper restricts the repulsion intervention to a chosen interval of the first few timesteps of the denoising process, controlled by a hyperparameter $\tau$. After timestep $t > \tau$, the repulsion is disabled and the model denoises normally. The choice of $\tau$ is model-specific:

• SD3.5-Large: $\tau = 4$ (out of 28 total denoising steps, so approximately the first 14% of the trajectory).
• SD3.5-Turbo: $\tau = 1$ (out of 4 total steps, so the first 25% of the trajectory).
• Flux-dev: $\tau = 1$ (out of 20 total steps, so the first 5% of the trajectory).

Why restrict to early timesteps: the motivation draws on well-established findings in the diffusion model literature. The initial stages of the denoising trajectory are where the global composition and semantic meaning are decided (Dahary et al., 2024, 2025; Patashnik et al., 2023; Balaji et al., 2023; Cao et al., 2025; Huberman et al., 2025; Yehezkel et al., 2025). These early steps transform the random noise into a rough layout — where objects go, what the overall scene looks like, the high-level color palette and lighting. Later steps refine details and textures but don't fundamentally change the composition. As the paper states:

"the initial stages of the denoising trajectory are the most crucial for the eventual semantic meaning and global composition, and are also where strong guidance signals such as CFG most strongly bias the generative path."

By intervening in the first few steps, the repulsion acts at the stage where the model is deciding what to generate, not just how to render it. This is precisely where diversity matters most — different high-level compositional choices (different layouts, different subject types, different environments) produce genuinely distinct visual outcomes, whereas differences in fine details (texture, lighting nuance) produce images that look largely similar.

Additionally, CFG's mode-collapsing effect is strongest in early steps because the guidance signal is most influential when the image is still unformed. Intervening during these early steps pushes back against CFG precisely when it is most aggressively narrowing the distribution.

Consequence of stopping early: after timestep $\tau$, the model completes the denoising process without any repulsion. The semantic intent has already been diversified — the model has committed to different high-level compositional paths for each sample. The remaining denoising steps refine these different compositions into detailed images, but the diversity is already "locked in" by the early-stage intervention.

Why the $\tau$ values differ across models: the denoising trajectories differ in length (28, 4, and 20 steps for the three models), but also in how quickly the composition is decided. Turbo models (4 steps) decide the entire image in very few steps, so the intervention must be applied in the very first step to have any effect. Standard models with longer trajectories (28 steps) have more room for the composition to evolve, so the intervention can span a few early steps without interfering with detail refinement.

The paper's timestep ablation (Table 7) provides empirical support for this design. For SD3.5-Turbo, applying repulsion in the interval $[0, 1/4]$ (the first 25% of the trajectory) yields Vendi 1.764 and ImageReward 0.829, while applying it in $[3/4, 1]$ (the last 25%) yields Vendi 1.988 and ImageReward 0.660 — much higher diversity but at severe cost to quality. For SD3.5-Large, the pattern is similar: late intervals ($[6/7, 1]$) produce Vendi 1.960 with ImageReward 0.852, while middle intervals ($[4/7, 5/7]$) produce Vendi 1.848 with ImageReward 0.954 — a much better trade-off. This confirms that early intervention balances diversity and quality, while late intervention (when the composition is already fixed) forces diversity through unnatural means that degrade visual fidelity.

The paper notes that the repulsion scale $\eta$ is kept constant within the intervention window:

"For simplicity, $\eta$ remained constant throughout the intervention window."

This is a simplification — a varying schedule (e.g., high repulsion in the very first step, tapering off toward $\tau$) might yield better trade-offs, but the paper doesn't explore this. The constant schedule means the same repulsive force is applied at every timestep within the window, which could be suboptimal if the sensitivity of the composition to perturbations varies across timesteps (as the timestep ablation suggests it does).

---

Hyperparameter Configuration and Model-Specific Adaptations

The paper tunes three hyperparameters per model architecture: the repulsion scale $\eta$, the intervention window $\tau$, and the number of inner-block iterations $M$. The number of denoising steps and the CFG guidance scale follow the standard settings for each model.

Flux-dev:

• Denoising steps: 20
• CFG guidance scale: 3.5
• Repulsion scale range: $\eta \in [2.5 \times 10^8, 5 \times 10^{10}]$
• Intervention window: $\tau = 1$ (first denoising step only)
• Inner-block iterations: $M = 50$
• Special handling: repulsion is applied to text tokens in the dual-stream multimodal attention blocks and to all tokens in the later single-stream blocks (which are specific to Flux architectures; these blocks process text and image tokens jointly without a separate text stream)

SD3.5-Large:

• Denoising steps: 28
• CFG guidance scale: 3.5
• Repulsion scale range: $\eta \in [2.5 \times 10^7, 5 \times 10^8]$
• Intervention window: $\tau = 4$ (first four denoising steps)
• Inner-block iterations: $M = 100$
• Special handling: since SD3.5-Large is not distilled for classifier-free guidance, it computes both a conditional and unconditional denoising path. The repulsion is applied to the text tokens in both the conditional and unconditional branches of the multimodal attention blocks. This ensures the diversity-inducing signal affects both the guided and unguided paths consistently.

SD3.5-Turbo:

• Denoising steps: 4
• CFG guidance scale: 0.0 (Turbo models do not use CFG; they are distilled to directly output the guided prediction)
• Repulsion scale range: $\eta \in [5 \times 10^6, 1 \times 10^8]$
• Intervention window: $\tau = 1$ (first denoising step only)
• Inner-block iterations: $M = 100$

The repulsion scale ranges differ dramatically across models — Flux-dev uses $\eta$ values on the order of $10^{9}$ to $10^{10}$, while SD3.5-Turbo uses values on the order of $10^{6}$ to $10^{8}$. This is not surprising because the contextual token representations have different magnitudes and dimensionalities in different architectures, so a given $\eta$ produces different effective repulsive forces. The paper does not provide a principled normalization — the ranges were determined empirically by sweeping and selecting values that produced a useful diversity-quality trade-off.

Why apply to both conditional and unconditional branches in SD3.5-Large: standard (non-distilled) diffusion models compute both a conditional score estimate $\epsilon_\theta(x_t, c)$ and an unconditional score estimate $\epsilon_\theta(x_t, \emptyset)$, then combine them via CFG: $\epsilon = \epsilon_\theta(x_t, \emptyset) + w(\epsilon_\theta(x_t, c) - \epsilon_\theta(x_t, \emptyset))$. The contextual tokens in the conditional branch encode prompt-specific semantic intent, while those in the unconditional branch encode a generic "no prompt" representation. Applying repulsion only to the conditional branch would diversify the prompt-conditional signal but leave the unconditional signal unchanged; the CFG combination might then partially cancel the diversity effect. By applying repulsion to both branches, the method ensures that the diversity signal appears in both the conditional and unconditional paths, making it more robust to the CFG combination.

Flux-dev and SD3.5-Turbo are guidance-distilled, meaning they directly output the CFG-equivalent prediction without computing separate conditional and unconditional paths — hence this consideration doesn't apply to them.

Why apply to single-stream blocks in Flux models: Flux architectures have dual-stream blocks (where text and image tokens are processed in separate attention streams before being combined) followed by single-stream blocks (where all tokens are processed jointly). The dual-stream blocks are where the enriched text tokens $\hat{f}_T$ are formed; the single-stream blocks further refine these representations. The paper applies repulsion in both block types, with the single-stream application affecting all tokens (text and image), not just text tokens. The paper does not provide a detailed ablation of single-stream vs. dual-stream application; the design choice appears to have been made empirically.

The repulsion scale $\eta$ directly controls the diversity-quality trade-off. The paper sweeps $\eta$ across a range to produce the Pareto frontiers in Figure 6: higher $\eta$ increases diversity but degrades quality (lower ImageReward, worse KID). The "sweet spot" depends on the application — a user who wants maximum creative exploration can use higher $\eta$, while a user who wants subtle variation can use lower $\eta$. The paper doesn't propose an automatic mechanism for selecting $\eta$; it treats it as a user-tunable parameter.

The number of inner-block iterations $M$ is not extensively ablated. The paper states $M = 100$ for Stable Diffusion and $M = 50$ for Flux without analyzing sensitivity to this choice. More iterations would allow the tokens to better optimize the diversity loss, but at higher computational cost. The chosen values presumably balance diversity gains and runtime — the paper's Table 1 shows 20–30% overhead, and reducing $M$ would reduce this overhead.

---

Architectural Generality and Transfer to Image Editing

A notable property of the method is that it transfers without any modification to image editing models. The paper demonstrates this with Flux-Kontext (Figure 5), an in-context image generation and editing model:

"we apply the exact same Contextual Space repulsion within the editing instruction stream."

The base Flux-Kontext model, when given an input image and an editing instruction (e.g., "a person running a marathon"), produces nearly identical edits across different random seeds — the same person in the same pose, just with slightly different backgrounds. Applying Contextual Space repulsion produces diverse yet coherent edits: different running poses, different camera angles, different environments (urban street, forest trail, stadium track), all while preserving the identity of the person from the input image and following the editing instruction.

This transfer works because Flux-Kontext is built on the same DiT architecture as Flux-dev, with the same multimodal attention blocks that create enriched text tokens. The editing instruction stream undergoes the same bidirectional exchange with image tokens, producing contextual representations that encode the model's intent for how to modify the image. By applying repulsion to these representations, the method pushes the model toward different editing realizations — different "ways" to satisfy the instruction — without modifying the architecture or the intervention strategy.

This result is significant because it suggests the Contextual Space is a general intervention locus for steering generative intent, not just a hack specific to text-to-image generation. Any DiT-based model that processes instructions or prompts through multimodal attention blocks — including video generation (HunyuanVideo), multi-concept personalization (TokenVerse), and other emerging applications — might be amenable to the same repulsion technique. The paper doesn't explore these extensions, but the Flux-Kontext result provides a proof of concept for broader applicability.

4. Key Insights and Innovations

Innovation 1: The Interventional Trade-Off as a Unifying Diagnostic Framework

The paper's most conceptually distinctive contribution is not the method itself but the diagnostic framework that motivates it. By taxonomizing all prior diversity approaches into upstream (modify inputs before generation) and downstream (modify image latents during denoising) categories, the paper identifies a structural tension that previous work had experienced but never named: the interventional trade-off. Upstream methods act before visual structure exists, lacking feedback about what the image is becoming. Downstream methods act after the composition is committed, on representations tied to a specific spatial grid. Each category's failure mode is the inverse of the other's: upstream methods need structural feedback they can't access; downstream methods need semantic flexibility that pixel-level repulsion can't provide.

Prior work had documented specific failures — CADS causes semantic drift (Sadat et al., 2023), Particle Guidance struggles in few-step models (Corso et al., 2023) — but treated these as implementation weaknesses rather than symptoms of a deeper architectural constraint. The paper's reframing converts a list of disconnected failure cases into a single diagnostic lens: the distance from the intervention point to the generative decision boundary. Intervene too early, and you lack structural feedback; intervene too late, and you face a committed visual mode. This lens explains why optimization-based upstream methods (SGI, MinorityPrompt) need high computational cost — they're implicitly compensating for the feedback gap by searching — and why downstream methods produce artifacts — they're pushing on a non-semantic representation that can't express high-level variation without leaving the data manifold.

The framework is fundamental rather than incremental because it defines the solution space. It doesn't just say "our method is better"; it says "here is the axis along which all methods differ, and here is why existing methods occupy points along that axis that are structurally disadvantaged." This makes it generative: the framework doesn't just explain existing failures, it predicts where successful interventions must lie — at a point in the architecture that is structurally informed (has access to emerging visual structure) but conceptually flexible (not tied to a spatial grid). The Contextual Space is then identified as the natural candidate satisfying both criteria, but the framework is more general than the specific instantiation. Future work on diversity in generative models — including video, 3D, and audio — can use the same diagnostic lens to identify analogous intervention points in their respective architectures.

Innovation 2: The Contextual Space as a Previously Unrecognized Semantic Control Locus in DiTs

While the existence of enriched text tokens in DiT multimodal attention blocks was known — ConceptAttention (Helbling et al., 2025) analyzed their interpretability — no prior work had recognized them as a control surface for steering generative behavior. The paper's key move is to pivot from a descriptive question ("what do these representations encode?") to an instrumental one ("can we manipulate them to change what the model generates?"), and to demonstrate that the answer is yes with a degree of semantic coherence that surpasses any previously available intervention point.

This is a fundamental insight about DiT architectures, not an incremental refinement. Prior work on representation manipulation in generative models focused overwhelmingly on two loci: the initial noise latent (for controlling layout and composition; e.g., Dahary et al., 2024) and the cross-attention maps (for controlling object placement and appearance in U-Net models; e.g., Hertz et al., 2022). Both of these are spatially committed — they encode where things go, not what things are. The Contextual Space is qualitatively different: it encodes semantic intent in a spatially disentangled form, meaning that modifying it changes high-level conceptual attributes (subject appearance, style, composition type) without breaking the spatial coherence that the transformer's attention mechanisms maintain.

The interpolation experiment (Figure 3) provides the empirical anchor. The fact that linear interpolation in Contextual Space produces smooth semantic transitions — a dog gradually becoming cat-like, facial features shifting naturally — while the same operation in VAE latent space produces structural blur and ghosting, is diagnostic evidence that these representations encode a different type of information. This isn't just "Contextual Space works better for diversity"; it's evidence that the DiT architecture has created an internal bottleneck where semantic planning and spatial execution are partially decoupled, and that this bottleneck is accessible for external steering. This has implications beyond diversity: any application requiring semantic-level control — style transfer, concept interpolation, attribute editing — might benefit from operating in this space rather than in pixel latents or attention maps.

The paper's naming of this space — "Contextual Space" — is itself a conceptual contribution. It crystallizes a property (semantically rich, structurally informed, spatially disentangled) into a term that future work can reference, analogous to how the "latent space" of GANs and VAEs became a standard concept for representation manipulation.

Innovation 3: Diversity as a Pareto-Optimal Frontier Rather Than a Single-Metric Trade-Off

The paper implicitly advances a normative reframing of the diversity problem: diversity is not something to be maximized in isolation, nor is it a simple dial that trades off linearly against quality. It is a Pareto frontier — the set of points where you cannot increase diversity without reducing some quality metric, and vice versa. The goal of a diversity method is not to achieve higher diversity at any cost (as CADS does, with catastrophic ImageReward scores), but to push the frontier outward — to achieve more diversity at the same quality level, or higher quality at the same diversity level.

This is a methodological contribution to evaluation practice. Prior diversity papers often reported diversity and quality metrics separately, leaving it to readers to mentally weigh the trade-off. The paper's Pareto frontier plots (Figure 6) — Vendi Score vs. ImageReward, VQAScore, and KID on three separate axes — make the trade-off explicit and visual. A method that lies strictly inside another method's frontier is unambiguously worse; a method on the frontier represents a point on the achievable trade-off curve. The analysis shows that the paper's method achieves a superior frontier to all baselines: at any given diversity level, it achieves higher ImageReward, higher VQAScore, and lower KID than competitors.

This framing also clarifies what "better" means in a way that single-number summaries cannot. SGI with 64 candidates achieves the highest raw Vendi Score (1.916) but at the cost of lower ImageReward (1.042) than the base model — it's on the diversity-quality frontier but at a different point than the base model, not beyond it. The paper's method achieves Vendi 1.898 with ImageReward 1.070 — essentially matching SGI's diversity while maintaining quality closer to the base model (ImageReward 1.075). The frontier framing makes this visible: the method isn't just "more diverse," it's operating at a point on the trade-off curve that was previously inaccessible.

The user study (Figure 7, Table 5) provides convergent evidence that this frontier is perceptually meaningful: human raters prefer the paper's outputs to baselines on diversity (61.1% average preference), quality (58.0%), and overall (65.1%), with the strongest advantage against PG and SPARKE (78.3% and 71.9% overall preference respectively). The only close competitor is SGI on diversity and overall preference (56.7% and 31.1% preference for Ours vs. Comp, with 32.2% and 41.1% ties), consistent with SGI occupying a nearby point on the frontier but at drastically higher computational cost.

Innovation 4: Efficient Diversity in Turbo Models as a Implicitly Validated Capability Claim

The paper makes a strong empirical claim that its method "remains effective even in modern 'Turbo' and distilled models where traditional trajectory-based interventions typically fail" (Abstract). This is not just a performance result — it's a diagnostic validation of the core insight. Turbo models represent an extreme case of the interventional trade-off: with only 4 denoising steps (SD3.5-Turbo), the generative path is decided almost instantly, leaving essentially no trajectory depth for downstream repulsion to act on. That the method works in this regime — Figure 14 shows it achieving a comparable trade-off frontier on SD3.5-Turbo as on Flux-dev — is evidence that intervening in the Contextual Space genuinely resolves the trade-off rather than merely shifting it to a more forgiving regime.

To see why this is significant, compare with Particle Guidance: PG applies repulsion at each denoising step, so with only 4 steps, it has at most 4 opportunities to influence the generation. By the second or third step in a Turbo model, the image is already substantially formed; the first step's $\hat{x}_0$ prediction is structurally unformed. PG's failure in this regime (Table 4 shows Vendi 1.926 with ImageReward crashing to 0.221 at high repulsion scales) is structural, not a matter of tuning. The Contextual Space method, by contrast, applies repulsion within the first denoising step's forward pass — at multiple transformer blocks, with 100 inner iterations per block — effectively "unfolding" the intervention across the depth of the network even when the temporal sampling budget is minimal. The paper's timestep ablation (Table 7) confirms this interpretation: the first 25% of the trajectory ($[0, 1/4]$) achieves the best diversity-quality balance on SD3.5-Turbo.

This is important not just for the specific method but for the broader research agenda on distilled generative models. As the field moves toward ever-faster inference (LCM, SD Turbo, SD3.5-Turbo, Flux Schnell), methods that assume long denoising trajectories will become increasingly irrelevant. The paper demonstrates that architectural depth (transformer blocks) can substitute for temporal depth (denoising steps) as a locus for intervention — a finding that may generalize to other control and editing tasks in distilled models. The transfer to Flux-Kontext (Figure 5) reinforces this: the method works in editing without modification because the Contextual Space exists in any DiT that processes instructions through multimodal attention, regardless of whether the task is generation or editing.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary quantitative evaluation uses 1,000 prompts randomly sampled from the MS-COCO 2017 validation set. For each prompt, 4 images are generated (4,000 total images per configuration). Additional evaluations use the 100 longest prompts from the "Complex" and "Fine-Grained Detail" categories of PartiPrompts (Yu et al., 2022), and the user study uses 40 prompts generated by ChatGPT across diverse categories.

• Base model(s). Three Diffusion Transformer architectures spanning distinct design choices and sampling regimes: Flux-dev (Black Forest Labs, 2024), a guidance-distilled model generating 20 denoising steps with CFG scale 3.5; SD3.5-Large (Esser et al., 2024), a standard non-distilled model generating 28 steps with CFG scale 3.5; and SD3.5-Turbo (Esser et al., 2024), distilled for 4-step inference with CFG scale 0.0 (Turbo models directly output the guided prediction without separate conditional/unconditional branches). These three were chosen to span the spectrum of modern DiT variants — standard, guidance-distilled, and few-step distilled — enabling a test of architectural generality.

• Metrics. Four metrics are used to capture different dimensions of the diversity-quality trade-off:

  • Vendi Inception Score (Friedman and Dieng, 2022; Szegedy et al., 2017): measures high-level semantic diversity by computing the von Neumann entropy of the similarity matrix of Inception features, then exponentiating to obtain the "effective number of distinct samples." Values range from 1 (all identical) to the batch size (all distinct). Computed per 4-image group and averaged.
  • ImageReward (Xu et al., 2023): measures human preference alignment on a learned scale. Higher is better (↑). This captures perceptual quality and aesthetic appeal.
  • VQAScore (Lin et al., 2024): measures fine-grained prompt adherence by evaluating whether a visual question-answering model can confirm that the image contains the prompted elements. Higher is better (↑). This captures whether diversity comes at the cost of semantic drift.
  • Kernel Inception Distance (KID) (Bińkowski et al., 2018): measures distributional fidelity between the generated set and a reference distribution. Lower is better (↓). This captures whether diversity pushes samples outside the data manifold (degrading the overall distribution).

• Baselines. Five diversity-enhancing methods spanning both upstream and downstream categories:

  • CADS (Sadat et al., 2023): injects prompt-agnostic noise into text embeddings with scale varied between 0.1 and 0.7, $\tau_1 = 0.3$, $\tau_2 = 0.8$, $\psi = 1$.
  • Particle Guidance (PG) (Corso et al., 2023): applies kernel-based repulsion in image latent space during denoising. Repulsion scales varied between 1 and 150.
  • SPARKE (Jalali et al., 2025): applies repulsion in latent space using a refined diversity objective. Scales varied between 0.01 and 0.14 depending on the model.
  • SGI (Scalable Group Inference) (Parmar et al., 2025): filters a large pool of initial noise candidates through optimization-based diversity selection. Evaluated with $N \in \{8, 16, 32, 64\}$ initial candidates using default hyperparameters from the official implementation. For qualitative comparisons and the user study, $N = 64$ was used.
  • Base model: the unmodified model with no diversity intervention, serving as the lower bound on diversity and upper bound on quality/alignment.

• Generation budget / compute accounting. The universal unit of compute is wall-clock runtime measured in seconds on an NVIDIA A100 GPU for generating a group of 4 images. The paper reports runtime for the base model, the proposed method, and SGI at multiple candidate set sizes (8, 16, 32, 64). For the proposed method, runtime includes the cost of computing and applying the repulsive updates in all multimodal attention blocks across the intervention window. Table 1 provides the full comparison: for Flux-dev, the base model requires 10.34 seconds, the proposed method requires 12.80 seconds (23.8% overhead), and SGI requires 47.47 to 113.99 seconds depending on candidate count (359% to 1002% overhead). No FLOPs accounting or parameter counting is reported — the runtime comparison is purely empirical.

• Cross-validation / statistical protocol. The paper does not report cross-validation for hyperparameter selection. Hyperparameters ($\eta$, $\tau$, $M$) were set per model based on empirical sweeps across the ranges reported in Appendix A. The Pareto frontiers (Figures 6, 13, 14) are mapped by systematically varying each baseline's control hyperparameter (guidance scale for PG and SPARKE, noise intensity for CADS, number of initial candidates for SGI, repulsion scale $\eta$ for the proposed method) and plotting the resulting diversity-quality pairs. For the user study, 450 responses were collected from 45 participants evaluating 40 prompts (each participant evaluated a subset), with standard side-by-side preference judgments across four metrics. No confidence intervals, standard errors, or statistical significance tests are reported for any metric — the results are presented as point estimates without uncertainty quantification.

---

Main Quantitative Results

Diversity-Quality Trade-off on Flux-dev (Figure 6, Table 2)

The headline result is that the proposed method achieves a superior Pareto frontier across all three quality axes on Flux-dev. The Pareto frontier plots in Figure 6 show the Vendi Score (x-axis) against ImageReward, VQAScore, and KID (three separate y-axes). Each method's frontier is the envelope of points achievable by varying its control hyperparameter. The proposed method's frontier (red) lies above/outside the frontiers of all baselines on all three axes.

The base Flux-dev model (no intervention) achieves Vendi 1.780, ImageReward 1.075, VQAScore 0.883, and KID 0.066 (Table 2). At comparable diversity levels, the proposed method maintains substantially higher quality. For instance, at $\eta = 2.5 \times 10^{10}$, the method achieves Vendi 1.898 with ImageReward 1.070 — a diversity increase of +0.118 over the base model with only a 0.005 decrease in ImageReward. In contrast, CADS, at the low end of its repulsion scale ($s = 10^{-20}$), achieves Vendi 1.908 but with ImageReward crashing to 0.377 and VQAScore dropping to 0.719 — the diversity is real but comes at the cost of catastrophic prompt alignment degradation. CADS's frontier lies far to the upper-left in the Vendi-ImageReward plot: it achieves diversity but at quality levels far below any other method.

Particle Guidance shows a very limited diversity range on Flux-dev. At its lowest repulsion scale ($s = 1$), PG achieves Vendi 1.753 (actually lower diversity than the base model's 1.780) with ImageReward 0.991 — the offset in diversity is because PG's pixel-level repulsion can interfere with the normal stochasticity of sampling, slightly reducing variation before it starts forcing it. At $s = 80$, Vendi reaches only 1.759, and at $s = 150$, it reaches 1.787 but with ImageReward dropping to 0.846. PG's frontier barely extends beyond the base model's diversity level — it cannot meaningfully increase the Vendi Score on Flux-dev without severe quality degradation. This is consistent with the paper's claim that downstream methods lack sufficient trajectory depth in 20-step models.

SPARKE shows a slightly better trade-off than PG. At $s = 0.01$, it achieves Vendi 1.790 with ImageReward 1.094 and VQAScore 0.884 — nearly identical to the base model. At $s = 0.02$, Vendi reaches 1.850 but ImageReward drops to 1.067 and KID spikes to 1.079 (more than 16× the base model's KID of 0.066). The sharp KID degradation indicates SPARKE pushes samples outside the data manifold at higher repulsion scales, producing unnatural artifacts — consistent with the qualitative observation of "patterned holes" in images.

SGI with 64 candidates achieves the highest raw Vendi Score among all methods at 1.916, exceeding the proposed method's best Vendi of 1.898. However, this comes at a notable quality cost: ImageReward drops to 1.042 (vs. 1.075 for the base model) and KID reaches 0.297 (vs. 0.066 for the base). More importantly, the computational cost is enormous: SGI-64 requires 113.99 seconds on Flux-dev versus 12.80 seconds for the proposed method — 8.9× slower. SGI's frontier does push further to the right in diversity than the proposed method, but at significantly worse quality and drastically higher cost.

The proposed method's quantitative advantage is clearest in the KID metric. At high diversity ($\eta = 2.5 \times 10^{10}$), the method achieves KID 0.172 — elevated over the base model's 0.066 but far below SPARKE's 1.079 and PG's 2.650 at comparable diversity levels. This is quantitative evidence that Contextual Space repulsion keeps samples on-manifold, whereas latent-space repulsion (PG, SPARKE) pushes them off-manifold. The method's VQAScore is almost perfectly flat across all $\eta$ values: 0.884 at $\eta = 2.5 \times 10^8$ to 0.880 at $\eta = 2.5 \times 10^{10}$, a drop of only 0.004. This confirms that diversity in the Contextual Space does not cause semantic drift — the prompt adherence remains essentially unchanged as diversity increases.

Diversity-Quality Trade-off on SD3.5-Large (Figure 13, Table 3)

The pattern of results on SD3.5-Large mirrors Flux-dev but with some differences in baseline behavior. The base SD3.5-Large model achieves Vendi 1.819, ImageReward 1.051, VQAScore 0.905, and KID 0.619 (Table 3). The proposed method at $\eta = 2.5 \times 10^7$ achieves Vendi 1.941 (+0.122 diversity) with ImageReward 0.988 (−0.063 quality) and KID 0.625 (nearly unchanged). At $\eta = 2.5 \times 10^8$, Vendi reaches 1.980 with ImageReward 0.940 and KID actually improving to 0.445 — a counterintuitive result where stronger repulsion improves distributional fidelity. The paper does not explain this phenomenon, but it may reflect that the base SD3.5-Large model's distribution was itself concentrated (low diversity), and pushing samples toward a more diverse but still on-manifold distribution better matches the reference distribution used to compute KID.

CADS again achieves high diversity but catastrophic quality: Vendi 2.004–2.025 with ImageReward dropping to 0.051–0.131 and VQAScore to 0.692–0.717. The KID values (0.941–0.953) are elevated but not catastrophic, likely because the CADS-generated distribution is still recognized as "images" by the Inception network even though they don't match the prompts — the images are of something, just not what was asked for.

PG reaches higher diversity on SD3.5-Large than on Flux-dev: Vendi up to 1.924 at $s = 80$, but with ImageReward collapsing to 0.632 and KID spiking to 5.930 — the highest KID among all methods and models, indicating severe off-manifold generation. This is consistent with the paper's claim that pixel-level repulsion in longer denoising trajectories (28 steps for SD3.5-Large) has more opportunity to accumulate and push latents far outside the data manifold.

SGI shows a strong trade-off on SD3.5-Large. At 64 candidates, it achieves Vendi 1.915 with ImageReward 1.004 and KID 0.421 — a better image quality at similar diversity compared to the proposed method's $\eta = 2.5 \times 10^7$ (Vendi 1.941 with ImageReward 0.988). However, the runtime penalty remains: SGI-64 requires 145.14 seconds versus 18.12 seconds for the proposed method — 8.0× slower (Table 1).

SPARKE shows a more extended frontier on SD3.5-Large than on Flux-dev, with Vendi reaching 1.989 at $s = 0.04$ but ImageReward dropping to 0.735 and KID to 2.918. The ability to push diversity higher on the 28-step model (vs. 20-step Flux) is consistent with downstream methods benefiting from longer trajectories, but the quality cost remains severe.

Diversity-Quality Trade-off on SD3.5-Turbo (Figure 14, Table 4)

The SD3.5-Turbo results are the most important test of the paper's claim about effectiveness in few-step models. The base Turbo model achieves Vendi 1.724, ImageReward 0.978, VQAScore 0.891, and KID 0.078 (Table 4). The proposed method at $\eta = 5 \times 10^7$ achieves Vendi 1.914 (+0.190 diversity) with ImageReward 0.864 (−0.114) and KID 1.897. At $\eta = 2.5 \times 10^7$, the trade-off is more favorable: Vendi 1.879 with ImageReward 0.899.

The critical comparison is with PG and SPARKE on this 4-step model. PG at $s = 2$ achieves Vendi 1.765 — barely above the base model — with ImageReward 0.915. At $s = 40$, Vendi reaches 1.926 but ImageReward crashes to 0.221 and KID explodes to 14.128 — by far the worst distributional fidelity across all experiments. This is quantitative confirmation that pixel-level repulsion is fundamentally incompatible with few-step models: the 4 denoising steps provide insufficient trajectory depth for the repulsive forces to steer samples toward diverse but on-manifold configurations. Instead, PG simply pushes latents into regions the decoder cannot handle.

SPARKE shows a similar pattern: at $s = 0.12$, Vendi 1.869 with ImageReward 0.629; at $s = 0.14$, Vendi 1.970 with ImageReward 0.231 and KID 7.037. The diversity can be forced, but at catastrophic quality cost.

SGI on SD3.5-Turbo shows its best performance relative to the proposed method. At 64 candidates, SGI achieves Vendi 1.879 with ImageReward 0.913 — nearly identical diversity to the proposed method at $\eta = 2.5 \times 10^7$ (Vendi 1.879, ImageReward 0.899), but with slightly better ImageReward. However, the runtime gap remains: SGI-64 requires 91.30 seconds versus 5.52 seconds for the proposed method — 16.5× slower (Table 1). On the 4-step Turbo model, the runtime gap is even more pronounced because the base inference is so fast (4.18 seconds), making SGI's optimization overhead proportionally larger.

CADS shows typical behavior: Vendi up to 1.958 with ImageReward 0.127 — high diversity, zero alignment.

The Turbo results validate the paper's central claim: Contextual Space repulsion can meaningfully increase diversity in few-step models where downstream methods fail. The architectural depth of the transformer (multiple blocks within a single timestep) provides the intervention depth that the temporal trajectory lacks. While the quality cost at high repulsion scales is real (ImageReward dropping to 0.864 at $\eta = 5 \times 10^7$), moderate scales ($\eta = 2.5 \times 10^7$) achieve a genuine improvement in the diversity-quality Pareto frontier over all baselines at a fraction of the computational cost of SGI.

Runtime Comparison (Table 1)

The runtime comparison across all three models quantifies the efficiency advantage. Across Flux-dev, SD3.5-Large, and SD3.5-Turbo, the proposed method increases generation time by 23.8%, 31.0%, and 32.1% respectively relative to the base model. SGI, the strongest competing method, increases generation time by factors ranging from 2.5× (SGI-8 on SD3.5-Turbo: 13.15s vs. 4.18s base) to 21.8× (SGI-64 on SD3.5-Turbo: 91.30s vs. 4.18s base). The proposed method's overhead is relatively consistent across models (20–32%), while SGI's overhead grows with denoising steps and candidate count. Table 1 does not include runtime for CADS, PG, or SPARKE, though these methods have lower overhead than SGI (CADS requires no optimization; PG and SPARKE require computing forces in latent space at each timestep but with fewer iterations and no candidate filtering).

The paper does not provide a breakdown of where the 20–30% overhead comes from — how much is due to the Vendi Score computation (kernel matrix + eigenvalue decomposition), how much to the gradient computation with $M$ inner iterations, and how much to the token manipulation itself. The Vendi Score computation on a $4 \times 4$ kernel is negligible; the dominant cost is likely the $M$ gradient steps inside each multimodal attention block, multiplied by the number of blocks and the number of timesteps in the intervention window.

User Study (Figure 7, Table 5)

The user study provides convergent validity beyond quantitative metrics. Across 450 pairwise comparisons from 45 participants evaluating 40 prompts:

• Overall preference: The proposed method is preferred over the base model by 57.8% vs. 13.8% (28.4% tie), over CADS by 74.4% vs. 15.6%, over SGI by 31.1% vs. 27.8% (41.1% tie), over PG by 83.3% vs. 10.0%, and over SPARKE by 87.5% vs. 9.4%. Averaged across all competitors, overall preference is 65.1% for the proposed method vs. 15.6% for competitors.

• Diversity perception: Users rate the proposed method as more diverse than competitors in 61.1% of cases on average, with PG as the strongest competitor (53.1% prefer SPARKE diversity; interestingly, this is the only metric where the proposed method is not preferred against a specific baseline — users find SPARKE's outputs more diverse, which is consistent with SPARKE's higher raw Vendi Scores in Table 4, but the quality cost is severe as shown in the quality preference: 85.9% prefer the proposed method on quality vs. 3.1% for SPARKE).

• Quality perception: 58.0% average preference for the proposed method. The strongest competitor on quality is SGI (31.1% prefer SGI, 15.6% prefer the proposed method, 53.3% tie), consistent with SGI achieving high ImageReward at its chosen operating point but at high computational cost.

• Prompt adherence: 48.9% average preference, with SGI again the closest competitor (22.2% prefer SGI, 13.3% prefer the proposed method, 64.4% tie). The high percentage of ties (37.1% average across all metrics) suggests that for many prompts, the differences are subtle enough that non-expert raters cannot reliably distinguish methods.

The user study results for SGI reveal the most nuanced comparison: preferences are closely matched (31.1% Ours, 27.8% Comp, 41.1% Tie overall), with the proposed method having a slight edge. Given SGI's 8–16× runtime penalty, a pragmatic interpretation is that the proposed method achieves statistically indistinguishable user preference at drastically lower cost. However, the user study uses SGI-64 (the highest-quality, highest-cost configuration), so the comparison is between the best SGI can offer and the proposed method at a single $\eta$ — the paper does not specify which repulsion scale was used for the user study outputs.

Scalability Across Batch Sizes (Table 6)

The batch size ablation on SD3.5-Turbo shows that the method scales favorably: going from batch size 4 to 16, the average Vendi Score per pair increases from 1.393 to 1.404, and ImageReward actually improves from 0.914 to 0.928. The raw Vendi Score increases from 1.819 to 2.768, but this is partly mechanical (Vendi is bounded by batch size). The per-pair normalization confirms that the repulsion mechanism benefits from denser batch coverage: with more samples to push against, each sample can find a more distinct region of the semantic space while remaining on-manifold.

The paper does not evaluate batch sizes larger than 16, leaving open questions about saturation — does the diversity-per-pair eventually plateau as the batch approaches the effective dimensionality of the Contextual Space? — and computational scaling — the $B \times B$ kernel matrix and eigenvalue decomposition become non-trivial at very large batch sizes.

Evaluation on Detailed Prompts

The evaluation on 100 longest PartiPrompts (constrained prompts with extensive detail) provides a robustness check: even when prompts are highly specific and leave little room for interpretation, the method finds remaining semantic degrees of freedom. The Vendi Score increased by +0.08 and ImageReward by +0.05, while VQAScore was nearly constant (−0.01). The paper describes this as demonstrating that the Contextual Space "effectively identifies and navigates remaining semantic degrees of freedom, even in the presence of extensive conditioning." This is an important stress test because diversity is trivially easy to achieve when prompts are vague; the method's ability to extract variation from highly constrained prompts suggests it operates on fine-grained semantic attributes rather than simply relaxing prompt adherence. The modest magnitude of the gains (+0.08 Vendi) relative to the MS-COCO results (+0.118 at $\eta = 2.5 \times 10^{10}$ on Flux-dev) is consistent with the interpretation that heavily constrained prompts offer less room for variation.

---

Ablation Studies and Robustness Checks

Repulsion scale ($\eta$) ablation (Figure 8): The qualitative response to increasing $\eta$ is demonstrated on the prompt "A breathtaking view of a distant house in beautiful scenery." At $\eta = 0$ (base model), all five images show near-identical architectural styles, environments, and compositions — a house on a grassy hill with similar lighting and framing. At $\eta = 5 \times 10^{10}$, subtle variations appear (slightly different house designs, similar environments). At $\eta = 1 \times 10^{11}$, structural changes emerge: one house gains a tower, the landscape varies more significantly. At $\eta = 2.5 \times 10^{11}$, seasonal and environmental shifts become pronounced: one scene is wintry with snow, another is autumnal, a third features a lake. At $\eta = 4 \times 10^{11}$, the variations become more dramatic — different architectural styles, different terrains, different weather conditions — while maintaining high visual quality and prompt alignment. The figure does not show degradation at the highest $\eta$, suggesting either that the chosen range doesn't reach the over-optimization regime for this particular prompt, or that quality degradation manifests in ways not visible in a small qualitative sample.

Repulsion space ablation (Figure 9, Figure 10): The paper compares repulsion applied to the text attention tokens (Contextual Space, $\hat{f}_T$) versus repulsion applied identically to the image attention tokens ($\hat{f}_I$) within the same multimodal blocks. Figure 9 shows Pareto frontiers for both variants on Flux-dev across Vendi vs. ImageReward, VQAScore, and KID. The Contextual Space (red) frontier is consistently and substantially superior: at any given diversity level, it achieves higher ImageReward, higher VQAScore, and lower KID. The image-token baseline shows steep degradation — as diversity increases, all quality metrics collapse much faster than for the Contextual Space variant. This confirms that the semantic properties of the text tokens (fixed sequence alignment, spatial disentanglement) are responsible for the method's effectiveness, not merely the fact of intervening inside the multimodal attention blocks. Figure 10 provides qualitative examples: for "A city street scene with a green bus coming up a street, with ocean," image-space repulsion produces artifacts (sea blending unnaturally into the road) due to spatial rigidity forcing repulsion to manifest as unnatural texture modifications, while Contextual Space repulsion produces varied but coherent compositions (different bus positions, different street layouts, different ocean visibility).

Timestep interval ablation (Table 7): On SD3.5-Turbo, applying repulsion in different quarters of the 4-step trajectory reveals a clear monotonic relationship: later intervals produce higher diversity but worse quality. The first quarter ($[0, 1/4]$) yields Vendi 1.764 with ImageReward 0.829; the last quarter ($[3/4, 1]$) yields Vendi 1.988 with ImageReward 0.660. Applying repulsion throughout the entire trajectory ($[0, 1]$) produces maximum diversity (Vendi 2.064) but minimum ImageReward (0.501). The sweet spot is clearly in the early timesteps. On SD3.5-Large, the pattern is similar but the quality cliff is less extreme: early intervals ($[0, 1/7]$ through $[4/7, 5/7]$) maintain ImageReward in the 0.932–0.954 range with Vendi between 1.847 and 1.854, while late intervals ($[6/7, 1]$) push Vendi to 1.960 but drop ImageReward to 0.852. The entire trajectory ($[0, 1]$) reaches Vendi 2.135 with ImageReward 0.535. The paper's choice of $\tau = 4$ out of 28 steps for SD3.5-Large and $\tau = 1$ out of 4 for SD3.5-Turbo is supported by these patterns: the earliest intervals provide the best quality-diversity trade-off, while limiting the intervention window avoids the quality degradation associated with late-timestep repulsion.

Transformer block ablation (Table 8): Restricting the repulsion to the first, middle, or last third of the transformer blocks reveals different effects on the two model architectures. On SD3.5-Turbo, the middle third produces the highest diversity (Vendi 1.947 vs. 1.878 for first third and 1.765 for last third) with reasonable ImageReward (0.844). The last third produces the highest quality (ImageReward 0.913) but the lowest diversity (Vendi 1.765). All three partitioned groups underperform the "all blocks" configuration on SD3.5-Large (Vendi 1.960 with ImageReward 0.852), but on SD3.5-Turbo, the middle-third configuration achieves a better trade-off than "all blocks" (Vendi 1.947/ImageReward 0.844 vs. 1.764/0.829). This architectural difference is notable but not discussed in detail by the paper — it may reflect differences in how information propagates through the shallower Turbo model's blocks versus the deeper SD3.5-Large.

Batch size ablation (Table 6): Already covered under scalability results. The key finding is that per-pair diversity and ImageReward both modestly improve as batch size increases from 4 to 16, suggesting the repulsion mechanism benefits from denser sampling of the conditional manifold.

Negative result — Flux-Kontext transfer (Figure 5): While presented as a positive result (successful generalization to image editing), the paper does not provide quantitative metrics for the editing setting. The evidence is purely qualitative — four example images for the base editing model and four for the proposed method applied to the same model. Without quantitative diversity or quality metrics, the claim that the method "can be successfully integrated" is supported only anecdotally. This is an explicit gap: the editing transfer is a proof-of-concept demonstration, not a rigorously validated result.

---

Critical Assessment

Claim 1: The method achieves a superior Pareto frontier across diversity, quality, and alignment metrics. This claim is well-supported on Flux-dev (Figure 6) and replicated across SD3.5-Large (Figure 13) and SD3.5-Turbo (Figure 14). The Pareto frontier framing is the correct way to evaluate diversity methods, and the paper's method consistently dominates or is competitive with all baselines. However, the support is strongest at moderate diversity levels. At the high-diversity extreme, SGI-64 achieves higher raw Vendi Scores on Flux-dev (1.916 vs. 1.898) and SD3.5-Large (1.915 vs. 1.980 — note the proposed method actually exceeds SGI here, but at lower ImageReward), though at substantially worse quality and drastically higher cost. The frontier framing correctly captures that the proposed method dominates most of the trade-off space, but there are edge cases where SGI pushes further in diversity at acceptable quality. The user study (Table 5) provides convergent evidence that these Pareto improvements are perceptually meaningful to human raters across multiple evaluation dimensions.

Claim 2: The method imposes only 20–30% runtime overhead. Supported by Table 1 across all three models. The overhead is remarkably consistent: 23.8% (Flux-dev), 31.0% (SD3.5-Large), 32.1% (SD3.5-Turbo). The comparison with SGI (359–2183% overhead) makes the efficiency advantage stark. However, the paper does not break down what fraction of this overhead is due to the Vendi Score computation, the gradient steps, or the token manipulation — an ablation of $M$ (inner iterations) would be informative. Additionally, runtime is measured for groups of 4 images; the overhead percentage might differ for different batch sizes, as the gradient computation and eigenvalue decomposition scale with batch size while the forward pass also scales. The paper's batch size scalability experiment (Table 6) shows performance trends but does not report runtime scaling with batch size.

Claim 3: The method works in Turbo/distilled models where traditional interventions fail. Supported with strong evidence. The SD3.5-Turbo results (Figure 14, Table 4) show that PG and SPARKE either barely increase diversity or do so at catastrophic quality cost (KID reaching 14.128 for PG). The proposed method achieves meaningful diversity increases (Vendi from 1.724 to 1.879 at $\eta = 2.5 \times 10^7$) with manageable quality impact (ImageReward from 0.978 to 0.899). This is the most empirically robust claim in the paper because it demonstrates not just that the method works, but why it works where others fail: architectural depth (transformer blocks with inner iterations) substitutes for temporal trajectory depth. The timestep ablation (Table 7) provides mechanistic evidence: the first 25% of Turbo's trajectory already achieves the best trade-off, confirming that the method intervenes at the right representational level for few-step models.

Claim 4: Diversity is achieved without sacrificing visual fidelity or semantic adherence. Supported with qualifications. The VQAScore is remarkably stable across repulsion scales — dropping by only 0.004 on Flux-dev as $\eta$ increases from $2.5 \times 10^8$ to $2.5 \times 10^{10}$ (Table 2). This is strong quantitative evidence that diversity does not come from breaking prompt alignment, unlike CADS where VQAScore drops from 0.883 to 0.674. ImageReward shows more degradation — from 1.075 (base model) to 1.070 at $\eta = 2.5 \times 10^{10}$ — but this is minimal compared to competitors (CADS drops to 0.208, PG to 0.846). However, on SD3.5-Turbo, ImageReward drops more substantially from 0.978 to 0.864 at $\eta = 5 \times 10^7$, indicating that the quality cost is model-dependent and non-trivial at high repulsion scales. The paper's phrasing ("without sacrificing") overstates the case: some quality cost exists, particularly at higher diversity regimes, but it is substantially smaller than for competitors.

Genuine weaknesses in the experimental design:

• No confidence intervals or statistical significance. All metrics are reported as single-point estimates without standard errors, confidence intervals, or significance tests. For a test set of 1,000 prompts generating 4,000 images, sampling variability is likely non-trivial. The differences between methods at closely spaced points on the Pareto frontier (e.g., the proposed method at $\eta = 2.5 \times 10^{10}$ vs. SGI-64 on Flux-dev: Vendi 1.898 vs. 1.916, ImageReward 1.070 vs. 1.042) are small enough that statistical uncertainty could affect the ordering. The user study reports percentage preferences but no confidence intervals on these proportions.

• Single hyperparameter sweep for baselines. The Pareto frontiers for baselines are mapped by varying a single control parameter (scale for PG/SPARKE/CADS, candidate count for SGI). These baselines may have additional tunable hyperparameters (e.g., CADS's timing parameters $\tau_1$, $\tau_2$; PG's kernel bandwidth; SPARKE's loss formulation) that could shift their frontiers. The paper states that baselines were "tuned to ensure competitive diversity levels," but this tuning is not systematically reported or ablated. It is possible that more careful tuning of baseline hyperparameters would close the gap with the proposed method.

• Fixed prompt set from MS-COCO. While 1,000 prompts is a reasonable test set, MS-COCO captions have a particular distribution (descriptive, natural imagery, common objects). The paper's findings might not generalize to prompts with different characteristics — abstract concepts, highly creative or unusual scenarios, long-form narrative descriptions. The PartiPrompts experiment partially addresses this, but only with 100 prompts from a specific subset.

• Batch size of 4 for primary evaluation. All quantitative metrics are computed on groups of 4 images. This is standard practice (Vendi Score is batch-size-dependent), but it means the diversity being measured is diversity within a small batch, not diversity across the full population of possible outputs. A method that produces 4 images that are diverse relative to each other but all drawn from a narrow region of the output distribution (compared to what's possible) would score well on Vendi-4 but poorly on population-level diversity metrics. The batch size scalability experiment (Table 6) shows the method continues to produce per-pair diversity at batch sizes up to 16, but population-level diversity is not directly measured.

• No comparison with prompt-level diversity methods that work by augmenting prompts (e.g., generating multiple variant prompts from a single user prompt and sampling one image per variant). This is a qualitatively different approach to diversity but addresses the same user need. The paper's claim to achieve "richer diversity" should ideally be benchmarked against this alternative paradigm, at minimum qualitatively.

• Missing ablation of diversity loss formulation. The paper uses the Vendi Score but states the framework is compatible with alternative losses. No ablation compares Vendi against simpler objectives (pairwise cosine distance, RBF kernel repulsion, minimum pairwise similarity) to determine whether the eigenvalue-based formulation matters in practice. If a simpler loss achieved comparable results, the eigenvalue decomposition (which could become costly at large batch sizes) would be unnecessary.

• The Flux-Kontext editing result is qualitative only. Transfer to image editing is presented as a key generalization result (Figure 5), but no quantitative metrics are reported. This remains an anecdotal demonstration, not a validated claim.

Missing experiments that would strengthen the paper:

• An end-to-end comparison of total cost: the paper treats runtime as the cost metric but does not account for the fact that SGI's candidate filtering produces 4 final images from $N$ generated candidates, while the proposed method produces exactly $B$ final images from $B$ generations. If a user needs 4 diverse images and SGI-64 generates 64 candidates to select 4, the actual "cost per final image" includes the wasted generations. A metric like "Vendi Score per FLOP" or "ImageReward per second" would capture this more holistically.

• Diversity on the same prompt across multiple independent batches: since the repulsion operates within a batch, two independent batches of 4 images each (different random seeds) might converge to similar overall modes. The paper does not evaluate cross-batch diversity — whether the method consistently explores different regions of the output distribution across independent runs.

• Ablation of $M$ (inner iterations): the number of gradient steps per block is fixed at 50 or 100 without sensitivity analysis. Reducing $M$ would reduce runtime overhead; an ablation showing the diversity-quality trade-off as a function of $M$ would guide practitioners in balancing cost and performance.

• Effect of CFG scale on the diversity-quality trade-off: the paper attributes typicality bias to CFG but uses fixed CFG scales (3.5 for Flux-dev and SD3.5-Large, 0.0 for Turbo). An experiment showing how the proposed method's frontier shifts at different CFG scales — and whether it can compensate more effectively than baselines at high CFG — would strengthen the mechanistic claims.

• Intervention in unconditional branch ablation for SD3.5-Large: the paper applies repulsion to both conditional and unconditional branches but provides no ablation showing the contribution of each. This is relevant for understanding the mechanism and for potential simplification (applying only to the conditional branch might suffice and reduce cost).

6. Limitations and Trade-offs

Capability Ceiling: Diversity Is Bounded by the Base Model's Inherent Expressiveness

The assumption or constraint. The method operates by pushing apart internal representations during the forward pass, but it cannot create semantic variation that the base model does not already encode somewhere in its learned distribution. The paper implicitly acknowledges this in its description of SGI's limitation: "SGI can only find diversity that already exists in the base model's sampling distribution." The same constraint applies to the proposed method — repulsion steers the model toward alternative semantic configurations, but only those that lie within the conditional manifold learned during training. If the base model has never learned to render a particular concept in a particular way, no amount of contextual repulsion will induce that variation.

The consequence. For prompts where the base model has a strong, narrow prior — the paper's examples include "a red London double-decker bus" producing nearly identical vehicles across all seeds (Figure 12) — the method can force structural variation (different camera angles, different weather) but may never produce fundamentally different types of buses (e.g., different historical eras, different vehicle designs). The diversity is real but bounded by what the model can represent. In the extreme case, for highly specific prompts where only one visual interpretation dominates the training data, diversity methods can only create superficial variation (lighting, framing, background) rather than genuine conceptual alternatives. The paper's evaluation on detailed PartiPrompts captures a milder version of this: the diversity gain is modest (+0.08 Vendi) compared to the gain on MS-COCO prompts (+0.118), consistent with heavily constrained prompts leaving fewer degrees of semantic freedom to explore.

What evidence exists in the paper. The paper does not directly quantify this ceiling or characterize the relationship between a prompt's "diversity capacity" and the achievable Vendi gain. The qualitative results show prompts with varying degrees of success — "a breathtaking view of a distant house" shows genuine compositional variety (different seasons, architectural styles, landscapes) at high $\eta$ (Figure 8), while "a red London double-decker bus" shows more constrained variation (Figure 12). But no systematic analysis measures, for a given prompt, how much diversity is possible from the base model (e.g., by generating thousands of samples with different seeds and computing the population-level Vendi Score) versus how much the method actually achieves. Without this baseline, it is impossible to know whether the method is extracting near-maximal diversity or leaving substantial variation on the table.

Mitigation status. The paper does not address this limitation directly. The conclusion gestures toward more controlled diversity — "whether a user-provided textual cue, such as 'color' or 'size,' can be used to guide the repulsion along a specific semantic direction" — which might partially address the boundedness problem by focusing repulsion on attributes the model can vary, but this is presented as future work. The limitation is structural: the method is a steering mechanism, not a generative one, and cannot create capabilities the base model lacks.

---

Practical Usability: The Repulsion Scale $\eta$ Is a Non-Interpretable, Model-Specific Hyperparameter

The assumption or constraint. The method's primary control knob is the repulsion scale $\eta$, which determines the strength of diversity-inducing forces applied in Contextual Space. The values of $\eta$ differ by orders of magnitude across models: $[5 \times 10^6, 1 \times 10^8]$ for SD3.5-Turbo, $[2.5 \times 10^7, 5 \times 10^8]$ for SD3.5-Large, and $[2.5 \times 10^8, 5 \times 10^{10}]$ for Flux-dev. There is no principled normalization — the ranges were determined by empirical grid search — and the paper provides no guidance for how a practitioner would choose $\eta$ for a new model or a new prompt distribution without conducting a full hyperparameter sweep.

The consequence. A user deploying this method on a different DiT-based model (e.g., a custom fine-tune, a newer version of SD, a DiT-based video model) faces an open-ended tuning problem. The $\eta$ ranges are not transferable across architectures because the magnitude of contextual token representations depends on model depth, hidden dimension, and training specifics. Setting $\eta$ too low produces negligible diversity gain; setting it too high degrades image quality (as the ImageReward and KID trajectories in Tables 2–4 show). For a developer integrating this into a product, the practical workflow becomes: generate a grid of $\eta$ values, visually evaluate diversity-quality trade-offs for a representative prompt set, and select a single operating point. This is labor-intensive and prompt-set-dependent — an $\eta$ that works well for MS-COCO prompts may be suboptimal for creative or highly specific prompts. The paper's user study uses a single unspecified $\eta$, implying that a fixed operating point was chosen, but how this choice generalizes to new prompt distributions is unknown.

What evidence exists in the paper. The evidence is in the hyperparameter ranges themselves (Appendix A) and the Pareto frontier sweeps (Figures 6, 13, 14). The ranges differ by $10^4\times$ between the smallest $\eta$ on SD3.5-Turbo ($5 \times 10^6$) and the largest on Flux-dev ($5 \times 10^{10}$). Within each model, the usable range spans roughly $20\times$ to $200\times$. The paper does not provide a normalized measure (e.g., repulsion strength per token per dimension) that would make these values comparable or transferable. The constant-$\eta$ design choice ("for simplicity, $\eta$ remained constant throughout the intervention window") adds another degree of freedom — a time-varying schedule could, in principle, achieve better trade-offs but would further complicate the tuning problem.

Mitigation status. The paper treats $\eta$ as a user-tunable parameter but does not provide selection heuristics, normalized units, or an automatic selection mechanism. The Pareto frontier is presented as a tool for understanding the trade-off, but selecting a specific $\eta$ from this frontier requires a practitioner to specify their relative preference for diversity versus quality in a way that current diversity metrics do not easily express. The authors do not suggest a default or recommend an automatic selection strategy.

---

Generalization Gap: Evaluated on a Single Prompt Distribution with Fixed Batch Size

The assumption or constraint. All quantitative results are computed on 1,000 prompts from the MS-COCO 2017 validation set, generating exactly 4 images per prompt. The Vendi Score is batch-size-dependent, and the method's optimization (repulsion forces across the batch) is inherently batch-aware — the repulsive force on a sample depends on the other samples in its batch. The paper's stated justification for the 4-image batch is standard practice, but it means that all quantitative diversity metrics measure diversity within a batch of 4, not diversity across the full output distribution.

The consequence. There are two distinct concerns. First, cross-batch diversity is unevaluated. If a user generates two independent batches of 4 images with the same prompt and $\eta$ but different random seeds, the two batches might explore the same regions of the output distribution. Within-batch repulsion pushes the 4 samples apart, but nothing prevents batch 1 and batch 2 from landing in the same semantic modes. The method measures and optimizes within-batch diversity, but users generating many images across multiple batches care about total diversity across all batches, which could be substantially lower than within-batch diversity. Second, MS-COCO prompts may not represent the diversity-relevant prompt regimes. MS-COCO captions describe natural, everyday scenes with concrete objects. The paper's own qualitative results show prompts where diversity is most striking — "a phoenix rising from ashes," "a futuristic warrior on a neon-lit cliff" — are creative and fantastical, not photographic descriptions of common scenes. The PartiPrompts experiment partially addresses this but with only 100 prompts and modest diversity gains.

What evidence exists in the paper. The batch size ablation (Table 6) provides partial evidence for within-batch scalability (per-pair Vendi improves slightly from 1.393 to 1.404 as batch size goes from 4 to 16), but this is still within-batch diversity. No experiment evaluates inter-batch diversity — e.g., generating 5 independent batches of 4 images each for the same 100 prompts and measuring the Vendi Score over the 20-image set versus the within-batch average. The quantitative metric tables (Tables 2–4) report only within-batch averages. The qualitative results (Figures 4, 11–12, 15–19) show single batches without comparison across independent runs with the same prompt.

Mitigation status. Not addressed. The paper's evaluation framework follows existing practice in the diversity literature (Vendi Score on groups of 4), but this practice was established for evaluating unconditional or weakly-conditional generative models where batch size is less critical. For a method that explicitly optimizes batch-level diversity, the batch size is a parameter of both the algorithm and the evaluation, and the paper does not discuss the relationship between the two. The claim that the method produces "richer diversity" should be understood as "richer within-batch diversity at batch size 4 on MS-COCO-style prompts," not as a guarantee of population-level diversity richness.

---

Latency-Agnostic Efficiency: The 20–30% Overhead Only Applies to the Specific Generation-Hardware-Regime Tested

The assumption or constraint. The headline efficiency claim — "imposing only a small computational overhead" of 20–30% (Table 1) — is based on wall-clock measurements on a single hardware configuration (NVIDIA A100 GPU) for batch generation of 4 images. This overhead includes the cost of the Vendi Score eigen-decomposition, the $M$ gradient steps per multimodal attention block, and the token manipulation itself, but it does not decompose these costs, and it measures runtime for end-to-end batch generation where all 4 samples are generated simultaneously on a single GPU.

The consequence. The overhead percentage is not guaranteed to generalize to different deployment scenarios. Specifically: (1) Single-sample generation: the method requires a batch of samples to compute the diversity loss (the Vendi Score is undefined for batch size 1). If a user wants to generate one image at a time interactively, they must either maintain a running batch (generating 4 images when only 1 is needed) or find an alternative diversity objective that works on single samples. The paper designs the method around batch processing; latency-sensitive single-image scenarios are not addressed. (2) Different hardware: the relative cost of the forward pass versus the gradient computation and eigenvalue decomposition may shift on different GPUs, at different batch sizes, or at different precision levels. The 20–30% figure is a point estimate, not a guaranteed bound. (3) Comparison with SGI's "wasted generation" cost: Table 1 measures SGI's runtime for generating 4 final images from $N$ candidates, but SGI-64 actually generates 64 images and selects 4 — a factor of $16\times$ more total image content generated. The proposed method generates exactly 4 images from 4 seeds. A fairer comparison might be: "SGI-64 can output 64 images in its runtime; the proposed method would need to generate $16$ independent batches to output 64 images, costing $16 \times 12.80 = 204.8$ seconds on Flux-dev versus SGI's 113.99 seconds." On a total-output basis, SGI's efficiency depends on how many final images the user needs and whether the candidate generation is seen as waste or as additional useful content. The paper's framing of "20–30% overhead for 4 images" implicitly assumes the user only needs precisely 4 images.

What evidence exists in the paper. Table 1 provides the raw numbers. The paper does not report runtime for batch size 1, for larger batch sizes, or at different precision levels. It does not break down the overhead by component. It does not discuss the single-sample limitation — the method is presented as a batch-generation technique without clarifying that batch generation is a hard requirement.

Mitigation status. Not addressed. The limitation is inherent to the particle guidance framework the paper adopts: repulsion requires a population of interacting samples. The paper does not discuss how a user generating images one at a time would use the method, whether through maintaining a running buffer of previous generations, using synthetic "anti-target" samples, or other workarounds. The Flux-Kontext editing demonstration also uses a batch, suggesting the batch requirement generalizes to the editing use case.

---

Qualitative-Only Validation for Image Editing and Architectural Generalization

The assumption or constraint. The paper claims that the method is architecturally general — applicable to "any DiT-based model that processes instructions or prompts through multimodal attention blocks" — and supports this with a demonstration on Flux-Kontext for image editing (Figure 5). However, the Flux-Kontext evaluation is purely qualitative (4 images from the base editing model, 4 images from the proposed method) with no quantitative metrics, no comparison against editing-specific diversity baselines, and no user study for the editing setting.

The consequence. The editing result is a proof-of-concept, not a validated capability. Without quantitative evaluation, it is impossible to know whether the method's diversity gains in editing come at the cost of editing accuracy (does the edit correctly follow the instruction across all diverse outputs?), identity preservation (do the diverse edits maintain the identity of the subject from the input image?), or other editing-specific desiderata. The paper's core T2I evaluation showed that contextual repulsion can maintain VQAScore nearly constant while increasing Vendi, but editing tasks have different "alignment" metrics (e.g., how well the edit reflects the instruction while preserving irrelevant parts of the image) that were not measured. The claimed architectural generality — extending to video generation, multi-concept personalization, etc. — is a hypothesis supported by a single qualitative demonstration on one editing model.

What evidence exists in the paper. Figure 5 shows 5 images: 1 input image, 4 base model edits, and 4 proposed method edits for the prompt "a person running a marathon." The images show varied running environments (stadium, street, forest trail versus only street in the base model), but no metrics are reported. The paper states the method was applied "out of the box" with "the exact same Contextual Space repulsion within the editing instruction stream," but does not report whether hyperparameters ($\eta$, $\tau$, $M$) were tuned for the editing case or reused from the Flux-dev T2I configuration.

Mitigation status. The paper presents the editing result as a demonstration of generality, not as a fully evaluated contribution, and does not make strong quantitative claims about editing performance. However, the broader architectural generality claim — that the Contextual Space is a "general intervention locus" — rests partly on this editing result, and the lack of quantitative validation weakens that claim. The paper does not specify what a rigorous editing evaluation would entail or flag this as future work.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a genuinely new intervention point for generative model control — not an incremental improvement to existing latent-space or prompt-space methods, but a qualitatively different locus of manipulation that resolves the structural tension between early and late interventions that had previously forced diversity methods into unfavorable trade-offs. The core shift is from thinking of diversity as something you induce by modifying inputs or outputs to something you induce by modifying the internal reasoning process of the model — its semantic intent, as encoded in the continuous bidirectional exchange between text and image representations inside DiT multimodal attention blocks.

This is a conceptual reframing with practical consequences, not a paradigm shift in the Kuhnian sense. The underlying framework — particle guidance, von Neumann entropy diversity objectives, diffusion sampling — remains unchanged. What changes is where these tools are applied, and the paper makes a compelling case that the "where" matters enormously. The diagnostic taxonomy of upstream vs. downstream interventions (Figure 2) will likely become a standard reference point for future work on generative diversity, much as the proposer-scorer decomposition became standard in the MCMC literature. It provides a shared language for describing why methods fail that was previously absent: upstream methods lack structural feedback; downstream methods face spatially committed representations. This language makes it possible to reason about new intervention points without trial-and-error empiricism — a researcher can ask "does this representation encode semantic intent in a spatially disentangled form?" and "does it have access to emerging visual structure?" to assess whether it sits at the interventional sweet spot.

The paper also reconciles conflicting intuitions about diversity in distilled models. Prior to this work, it was reasonable to believe that few-step models like SD3.5-Turbo were simply incompatible with diversity enhancement — their trajectories are too short, their compositions decided too quickly, and existing downstream methods like Particle Guidance confirmed this pessimism with catastrophic KID scores (14.128 on Turbo, Table 4). The paper demonstrates that this pessimism was about a specific class of methods, not about few-step models in general. By exploiting architectural depth (transformer blocks within a single timestep) as a substitute for temporal depth (denoising steps), the method opens the door to diversity control in the increasingly dominant regime of distilled, fast-inference models. This will likely accelerate the development of other control and editing methods for Turbo-class models by shifting researchers' attention from the denoising trajectory to the transformer forward pass as the locus of intervention.

Perhaps most significantly, the paper elevates the Contextual Space to the status of a first-class control surface in DiT architectures. Prior work like ConceptAttention (Helbling et al., 2025) had analyzed what these enriched text tokens represent. The present paper demonstrates that they are not merely interpretable but actionable — modifying them produces coherent, semantically meaningful changes in the output. This opens a broad research program around representation manipulation in DiT internal states that extends far beyond diversity. Any task that requires semantic-level steering — style control, concept interpolation, attribute editing, debiasing — might be addressed by intervening in the Contextual Space rather than in pixel latents or attention maps. The paper's Flux-Kontext editing result (Figure 5), though qualitative, provides an existence proof that this generality is real: the same intervention works for both generation and editing without modification, suggesting the Contextual Space encodes a task-invariant representation of semantic intent.

The paper also implicitly redefines the diversity evaluation standard from single-metric comparisons to Pareto frontier analysis. By plotting diversity (Vendi) against multiple quality axes (ImageReward, VQAScore, KID) and showing that different methods occupy different points on an achievable trade-off curve, the paper makes visible what single-number summaries obscure: that the goal is not "more diversity" but "more diversity at the same quality" — shifting the frontier outward. Subsequent diversity work will likely be expected to report Pareto frontiers rather than isolated diversity or quality numbers, which will raise the bar for method evaluation and make it harder for methods that achieve diversity through semantic drift (like CADS) to appear competitive.

Follow-Up Research This Work Enables

Characterizing the effective capacity of the Contextual Space for diversity. The paper demonstrates that repulsion works but provides essentially no characterization of how much diversity the Contextual Space can encode for a given prompt. A natural follow-up would measure the maximum achievable population-level diversity for a prompt set by generating thousands of independent batches with the same $\eta$ and computing aggregate Vendi Scores across batches, not just within them. This would establish an empirical upper bound on what Contextual Space repulsion can achieve and identify prompts where the method hits a ceiling — where even large $\eta$ produces saturating diversity because the base model simply lacks alternative visual modes. The experiment could also measure the correlation between this maximum achievable diversity and simple prompt properties (length, number of concrete nouns, abstractness) to predict a priori which prompts will benefit most from repulsion. The paper already provides the necessary infrastructure — the evaluation pipeline on MS-COCO and PartiPrompts — and this extension would transform the method from "it works empirically" to "here is how well it works and for what kinds of prompts."

Single-sample diversity via synthetic anti-targets or a learned diversity critic. The method's requirement for batch generation is a significant practical limitation: generating 4 images when a user wants 1 wastes computation and breaks interactive workflows. A follow-up could ask whether diversity can be induced for a single sample by providing one or more synthetic "anti-target" representations — either fixed vectors from a pre-computed memory bank of diverse contextual tokens, or adversarially constructed representations designed to push the current sample away from typical modes. The key experiment would measure: (1) per-image diversity compared to batch-4 diversity, (2) whether the diversity gain is stable across different prompts (since a fixed anti-target that works for "a phoenix" might be irrelevant for "a red bus"), and (3) the computational overhead relative to the batch-4 method. A more ambitious variant would train a lightweight diversity critic that predicts, from a single sample's contextual tokens in early blocks, whether that sample will converge to a typical mode — and applies repulsion with strength proportional to this predicted typicality. This would transform the method from a fixed-strength batch intervention to an adaptive single-sample intervention, addressing both the batch-size limitation and the non-interpretable $\eta$ tuning problem simultaneously.

Combining Contextual Space repulsion with controlled attribute variation. The paper's conclusion identifies "whether a user-provided textual cue, such as 'color' or 'size,' can be used to guide the repulsion along a specific semantic direction" as a key future direction. This can be made concrete: the idea is to modify the diversity loss so that rather than maximizing unconstrained von Neumann entropy, it maximizes entropy within a subspace of the Contextual Space that corresponds to the user-specified attribute. The practical approach would use ConceptAttention or a similar method to identify which contextual token dimensions encode a given concept (e.g., "style," "pose," "lighting"), then compute the Vendi Score only on the projection of contextual tokens onto those dimensions. The experiment would compare: (1) unconstrained repulsion (the current method), (2) repulsion constrained to style-relevant dimensions, (3) repulsion constrained to composition-relevant dimensions, and (4) repulsion in the orthogonal complement (everything except the target attribute). The strongest evidence for controllable diversity would be: the method produces variation in the target attribute without affecting other attributes, and re-applying with a different target attribute produces variation in that attribute without affecting the first. The paper already demonstrates that the axes of variation adapt to the prompt (Figure 12: "phoenix" alternates artistic styles, "bus" varies weather and pose) — constraining this variation to user-specified axes would make the method useful for creative workflows where unconstrained variation is distracting.

Generalization to video generation DiTs. The paper's architectural generality claim — that the Contextual Space exists in any DiT that processes instructions through multimodal attention — is testable on video generation models like HunyuanVideo (Kong et al., 2025), which the paper cites. A follow-up could apply the identical repulsion mechanism (no modification to the diversity loss, the $\eta$ range, or the intervention scheduling) to a DiT-based video model and measure: (1) whether batch diversity increases for the same prompt as measured by frame-level Vendi Score and video-level perceptual diversity metrics, (2) whether temporal coherence is preserved across the diverse outputs (the paper's concern about spatial artifacts has a temporal analog — does repulsion break frame-to-frame consistency?), and (3) whether the 20–30% runtime overhead holds for video where the forward pass is substantially more expensive (potentially making the relative overhead even smaller). A positive result would validate the Contextual Space as a genuinely medium-agnostic control surface, not just a text-to-image trick. A negative result — e.g., video diversity fails because the Contextual Space in video DiTs encodes temporal as well as semantic information, and repulsion disrupts the temporal dimension — would be equally informative, defining the boundary conditions of the intervention space.

Stress-testing verifier over-optimization: how far can $\eta$ go before collapse? The paper sweeps $\eta$ to produce Pareto frontiers but stops before severe quality degradation (the lowest ImageReward reported for the method is 0.864 on SD3.5-Turbo at $\eta = 5 \times 10^7$, Table 4 — far better than baselines' catastrophic drops to ~0.2). This suggests the method has a "graceful degradation" regime that baselines lack, but the paper does not systematically characterize when Contextual Space repulsion starts to produce artifacts. A follow-up stress test would push $\eta$ to extreme values (beyond the reported ranges) on a fixed prompt set and document: (1) what failure modes emerge (do they resemble downstream artifacts like spatial blur, or upstream failures like semantic drift, or something new?), (2) whether failure is prompt-dependent (do simple prompts break at lower $\eta$ than complex ones?), and (3) whether there is a detectable signal — e.g., a sudden increase in the variance of per-token repulsion magnitudes — that could be used for automatic early stopping. This would provide practical guidance for setting $\eta$ without visual inspection and would characterize the method's robustness boundary, which is currently unknown.

Cross-model transferability of the Contextual Space. The paper evaluates on three DiT variants with different training procedures (Flux-dev is guidance-distilled, SD3.5-Large is standard, SD3.5-Turbo is few-step distilled) but does not ask whether the Contextual Space of one model can guide diversity in another. A cross-model transfer experiment would extract contextual token representations from a "teacher" DiT during generation and use them to compute repulsive forces applied to a "student" DiT's contextual tokens. If the semantic structure of the Contextual Space is sufficiently universal across independently trained DiTs, this would provide strong evidence that it encodes model-invariant semantic properties rather than model-specific artifacts. The experiment would measure whether cross-model repulsion produces diversity gains comparable to within-model repulsion, and whether failure modes differ. A positive result would enable interesting applications — e.g., using a large DiT to plan diverse semantic layouts that guide a smaller, faster DiT — while a negative result would clarify that the Contextual Space is tightly coupled to each model's learned representations.

Practical Applications and Downstream Use Cases

Creative exploration interfaces for design and concept art. The method's most immediate application is in tools where a user wants rapid visual exploration of a concept — generating 4–16 diverse candidate images from a single prompt with a single click, rather than manually tweaking prompts and seeds. The 20–30% overhead (12.80 seconds for 4 Flux-dev images vs. 10.34 seconds base, Table 1) is negligible in an interactive workflow where the dominant cost is human inspection and iteration time. A designer seeking inspiration for "a futuristic warrior on a neon-lit cliff" currently sees near-identical outputs from the base model (Figure 18) and must either write variant prompts or generate dozens of batches hoping for lucky seeds. With the method, a single batch of 8 images at $\eta \approx 5 \times 10^9$ produces genuinely different stylistic interpretations (cyberpunk vs. fantasy armor, different poses, different environments) while maintaining visual quality — compressing what would otherwise be multiple prompting cycles into one generation. The user study's 57.8% overall preference over the base model (Table 5) supports that this diversity is perceptually valuable to end users, not just metric-friendly.

Data augmentation for training robust vision models. When training classifiers, object detectors, or vision-language models, synthetic data from T2I models is increasingly used to supplement real datasets. However, the typicality bias of modern T2I models means that synthetic data generated from the same prompt can be unrealistically homogeneous — all "red London double-decker buses" look identical — reducing the effective diversity of the synthetic training set and limiting the robustness gains. Applying Contextual Space repulsion during synthetic data generation (e.g., generating 100,000 diverse images across 10,000 COCO captions with $\eta$ set to produce roughly Vendi 1.9–1.95) would produce training data that spans a wider range of visual realizations for each concept — different camera angles, lighting conditions, backgrounds, and object appearances — without requiring prompt engineering or post-hoc filtering. The paper's VQAScore stability (dropping only 0.004 across $\eta$ ranges on Flux-dev, Table 2) is critical here: the diverse images still correctly reflect the prompt semantics, so the synthetic labels remain accurate. The 20–30% generation overhead would be amortized across the training pipeline and would likely be cost-effective compared to collecting equivalent real-world data diversity.

Diverse candidate generation for self-improvement and RLHF data pipelines. In reinforcement learning from human feedback (RLHF) and related self-improvement pipelines, the quality of preference data depends on presenting human raters with genuinely distinct candidate outputs — if all candidates are near-identical, preference judgments become noise. Contextual Space repulsion could be integrated into the candidate generation stage of these pipelines to ensure that the $K$ candidates presented for comparison span the space of possible outputs rather than clustering around the typical mode. For a fixed generation budget, using the method to produce $K$ diverse candidates (rather than $K$ i.i.d. samples from the base model) would likely yield preference data with higher signal-to-noise ratio, as raters would be comparing meaningfully different outputs rather than subtly different renderings of the same concept. The paper's batch size scalability (Table 6: Vendi per pair improves from 1.393 to 1.404 as batch increases from 4 to 16) suggests this benefit would scale to the larger batch sizes typical in RLHF pipelines.

Diverse image editing for content creation at scale. The Flux-Kontext editing result (Figure 5) points toward a use case where a creator provides a single reference image and an editing instruction, and the method produces multiple diverse edited versions — different "takes" on the same instruction. For applications like virtual try-on, interior design visualization, or photo retouching where showing a client multiple options is standard practice, this would replace the current workflow of generating one edit, then re-generating with different seeds until sufficient variety is found. The paper's qualitative demonstration shows the method varying editing-relevant attributes (running pose, environment) while preserving the image content that should remain unchanged (the person's identity). Quantitative validation of editing accuracy across diverse outputs — does the edit instruction hold equally well across all diverse edits? — would determine whether this application is production-ready or requires additional control mechanisms.