SphereDiff: Tuning-free 360° Static and Dynamic Panorama Generation via Spherical Latent Representation

🎯 Pitch

Pole-stretching artifacts in 360° panoramas are eliminated simply by switching from flat equirectangular latent grids to Fibonacci-lattice points on a sphere—no fine-tuning required. A new distortion-aware blending step fixes the warping that naive spherical diffusion introduces, making pretrained 2D or video models generate seamless, high-fidelity 360° wallpapers and videos for the first time.

1. Executive Summary

This paper introduces SphereDiff, a tuning-free framework for generating 360° static and dynamic panoramas that replaces the standard equirectangular projection (ERP) latent space with a spherical latent representation — a set of latent features distributed uniformly over the sphere's surface using a Fibonacci lattice, ensuring consistent generation quality across all viewing angles including near the poles. The method extends the MultiDiffusion framework to operate directly on spherical latents through dynamic latent sampling (a center-first queue-based discretization strategy that prevents undersampling artifacts, as opposed to nearest-neighbor sampling) and distortion-aware weighted averaging (an exponential weighting function that compensates for the radial stretching introduced by spherical-to-perspective projection). The system operates on pretrained diffusion backbones — SANA and FLUX for static panoramas, LTX-Video and HunyuanVideo for video — without any fine-tuning, achieving state-of-the-art performance in user studies across panoramic criteria such as distortion and end-continuity while maintaining comparable image quality to ERP-based methods, establishing that spherical latent representations eliminate pole-stretching artifacts only when combined with proper latent sampling and distortion-aware fusion within the MultiDiffusion paradigm.

2. Context and Motivation

The Core Problem: Generating 360° Panoramas Without Specialized Cameras

The paper addresses a deceptively simple problem: how do you generate a high-quality 360° panoramic image or video from a text prompt without a specialized 360° camera? This matters because immersive content — the kind used in AR/VR headsets, 360° live wallpapers, and virtual environments — requires omnidirectional imagery that surrounds the viewer. A 360° panorama captures the full sphere of view: 360° horizontally by 180° vertically, allowing users to look in any direction from a fixed center point. These panoramas are the foundational building blocks of immersive experiences, providing what the authors describe as "a sense of presence that traditional displays (e.g., phones and laptops) cannot" (Section 1).

The practical demand for this capability is straightforward and growing. AR/VR adoption — whether in gaming, virtual tourism, architectural visualization, or remote collaboration — requires a constant supply of immersive backgrounds and environments. Yet capturing real 360° content requires specialized camera rigs (multi-lens spherical cameras or carefully calibrated arrays) that are expensive, cumbersome, and entirely unsuitable for dynamic scenes that are dangerous, inaccessible, or purely imaginary. The paper frames this as a content supply problem:

"capturing 360° panoramas requires specialized cameras, their availability is limited, especially for videos. As a result, available content is dominated by simulation-based graphics, which can be rudimentary, while users increasingly seek realistic experiences."

If text-to-360° generation could produce photorealistic panoramic imagery without specialized hardware — and without requiring fine-tuning on scarce panoramic datasets — it would unlock a new pipeline for creating infinite immersive content from text descriptions alone. This is the practical motivation for the work.

The ERP Distortion Problem: Why Standard Diffusion Models Fail on Panoramas

To understand why this problem is technically challenging, we first need to understand how 360° panoramas are typically stored and why this storage format is fundamentally misaligned with how generative models work.

A 360° panorama captures light arriving at a single point from all directions — it's a spherical image, with each pixel corresponding to a direction on the unit sphere $\mathbb{S}^2$ . Computers cannot directly store data on a sphere, so we must project the spherical data onto a flat 2D plane. The dominant approach in computer vision and graphics is equirectangular projection (ERP) — analogous to how a globe is projected onto a rectangular world map. In ERP, the horizontal axis corresponds to azimuth (longitude, 0° to 360°) and the vertical axis corresponds to elevation (latitude, -90° to +90°), producing a rectangular image with a 2:1 aspect ratio.

ERP has a critical flaw: it severely distorts geometry, particularly near the poles. Consider what happens at the north pole of a sphere. On the actual sphere, the north pole is a single point. In ERP, the topmost row of the 2D image — an entire horizontal line of pixels — must all represent that same single point. Similarly, near the poles, a small patch of the spherical surface gets stretched across a disproportionately wide band in the ERP image. The paper illustrates this concretely:

"high-latitude regions appear disproportionately large. For example, as shown in Fig. 1, the content near the poles appear significantly larger than the others since we visualize the 360° wallpapers in ERP."

This distortion is not a minor visual artifact — it means that the pixel density on the sphere is wildly non-uniform when represented in ERP. A given angular span near the equator occupies far fewer ERP pixels than the same angular span near the poles. In the language the paper introduces: the latents (the compressed feature representations used by diffusion models) are unevenly distributed over the spherical surface, with extreme sparsity at the equator and extreme oversampling at the poles.

This distributional mismatch creates a fundamental barrier to using pretrained diffusion models (e.g., Stable Diffusion, FLUX, SANA) for panorama generation. These models are trained on standard perspective photographs — images captured by normal cameras with a limited field of view, where objects have consistent scale and straight lines remain straight. When you try to generate an ERP image directly with such a model, the model must simultaneously produce content that:

At the equator: looks like a standard perspective photograph.
At the poles: looks like a standard perspective photograph that has been dramatically stretched — objects appear warped, lines that should converge now splay outward, and textures are unnaturally elongated.

The model has never seen training data with this distribution, so it fails — producing what the paper describes as "distracting distortions near the poles" (Section 1) and "noticeable artifacts near the poles, such as distortion, blurriness, and speckling" (Section 4.2).

Where Prior Approaches Fall Short

The paper identifies two broad families of prior approaches to 360° panorama generation, each with specific limitations that motivate the SphereDiff design.

Approach 1: Fine-Tuning on ERP Datasets

The most direct approach is to take a pretrained image or video diffusion model and fine-tune it on datasets of ERP panoramas paired with text descriptions. Examples in the paper include 360 LoRA (LatentLabs360, 2023), Text2Light (Chen, Wang, and Liu, 2022), PanFusion (Zhang et al., 2024), and 360DVD (Wang et al., 2024) for video.

Why this falls short — data scarcity. The paper identifies a fundamental bottleneck: high-quality text-ERP paired data is extremely limited. As Table 4 and Appendix D.2 document, most existing panoramic datasets are small (~10K images), focused on indoor environments (Matterport3D, Structured3D), and lack the diversity of natural outdoor scenes that users expect for wallpapers and immersive environments. The paper states:

"due to the limited availability of text-ERP pairs, data-driven approaches often fail to generate seamless 360° panoramas, particularly near the poles"

The scarcity is even more acute for video. The paper notes that 360DVD trained on "a small dataset of merely 2,000 video clips, which is insufficient to achieve consistently high-quality results" (Appendix D.2). This paucity fundamentally limits what fine-tuning can achieve — the model cannot learn to generalize beyond the distribution of its training panoramas.

Why this falls short — text controllability. Because ERP datasets are domain-specific (mostly indoor scans), models fine-tuned on them inherit those domains. Text2Light, despite generating visually plausible 360° outputs, "struggles to adhere to the input text prompts due to the limited diversity of its training dataset" (Section D.1, Figure 10 caption). You can ask for "a luminous nebula" but the model has only learned to produce indoor rooms — a catastrophic mode collapse that undermines the creative generation use case.

Why this falls short — persistent pole artifacts. Even with fine-tuning, models struggle with the inherent ERP distortion near the poles. Figure 2 shows 360 LoRA and 360DVD exhibiting "pole-stretching artifacts" despite training specifically for panoramic output. The fine-tuned models learn to produce reasonable-looking ERP images but fundamentally cannot escape the uneven latent distribution: the poles still receive disproportionately many latents relative to their actual spherical area, causing the model to allocate too much representational capacity to polar regions at the expense of the equator. The result is that fine-tuned ERP models remain "often fail to generate seamless 360° panoramas, particularly near the poles" (Section 1).

Approach 2: Tuning-Free Methods Using ERP Latents and MultiDiffusion

Recognizing the data scarcity problem, several works have taken a tuning-free approach: use pretrained diffusion models as-is, but structure the generation process to stitch together multiple perspective views into a panoramic output. The dominant framework for this is MultiDiffusion (Bar-Tal et al., 2023), which the paper builds upon.

How MultiDiffusion works in general. The core insight of MultiDiffusion is that you can generate an arbitrarily shaped image by dividing it into overlapping patches, denoising each patch independently with a standard diffusion model, and then blending the overlapping regions through weighted averaging. Each patch is processed at the resolution and aspect ratio the diffusion model expects, so the model never sees the unusual global shape — it just sees standard-sized image patches. The overlapping regions ensure consistency: neighboring patches share content in their overlap zones, so the blending process enforces coherence across the full canvas.

How prior work applied MultiDiffusion to panoramas. DynamicScaler (Liu et al., 2024) is the primary tuning-free baseline the paper compares against. It applies MultiDiffusion to ERP-formatted latents: it initializes a large ERP-shaped latent map, divides it into overlapping windows, and denoises each window using a standard perspective diffusion model. The overlap between adjacent ERP windows is meant to enforce seamlessness. DynamicScaler further introduces an "Offset Shifting Denoiser" to improve end-to-end continuity (the left and right edges of ERP should connect seamlessly since they represent the same meridional line on the sphere).

Why this falls short — the ERP latent distribution problem persists. The paper identifies a deep structural flaw in this approach. Even though MultiDiffusion avoids generating the full ERP in one pass, the fact that the latents themselves are defined on the ERP grid means they inherit ERP's uneven spherical distribution. The paper is explicit about the mechanism, illustrated in Figure 2:

"They use an ERP representation for latents, that distributes latents uniformly over the ERP. Despite its simplicity, it causes significant differs the density of latents in spherical representation, resulting severe pole-stretching artifacts."

Let's unpack this. In ERP, the top row of latents — just a few hundred pixels wide in the latent space — must represent the entire north pole region of the sphere. When MultiDiffusion processes a window that includes this row, the model sees a patch of the ERP image near the top border. This patch is highly distorted because of the ERP projection, yet the diffusion model (trained only on perspective images) treats it as if it were a normal image patch. The model can handle some projective distortion (perspective images have foreshortening), but the extreme stretch near ERP poles is far outside its training distribution. The result is that even with careful overlapping and blending, the generated content near the poles is blurry, distorted, and semantically incoherent.

DynamicScaler produces "noticeable blurry artifacts in the polar regions" (Appendix D.2, also visible in Figures 5 and 11) precisely because the ERP latent representation forces too few latent samples to cover the highly distorted polar regions.

Approach 3: Alternative Geometric Representations (Cube Maps)

The paper also acknowledges an emerging alternative: generating panoramas directly in a cube map representation (six perspective images forming the faces of a cube surrounding the viewer). CubeDiff (Kalischek et al., 2025) takes this approach, training on large-scale 360° image datasets to generate the six cube faces.

Why this falls short. While cube maps inherently reduce polar distortion (each face is a standard perspective projection with limited field of view), the paper notes that cube maps "still struggle with discontinuities at cube-face boundaries" (Section 2). The six faces must be perfectly consistent at their edges — the seam between the front face and right face must show the same content from different angles — and enforcing this consistency during generation is challenging without explicit geometric constraints. Moreover, cube map methods still require large-scale training data and have not been extended to video generation, making them unsuitable for the paper's goal of tuning-free 360° live wallpaper generation.

The Deeper Issue: Uneven Latent Distribution Is the Root Cause

The paper traces all of these failures to a single root cause: the uneven distribution of latents on the spherical surface. This is the key conceptual insight that motivates the entire SphereDiff framework.

In an ERP representation, the number of latent pixels covering a given solid angle on the sphere varies dramatically with elevation. Near the equator, each latent covers a relatively small patch of the sphere. Near the poles, each latent covers an enormous stretched patch. This means:

Representational imbalance: The model allocates disproportionately many latent features to polar regions (which cover a tiny fraction of the actual spherical surface area) and too few to equatorial regions (which cover most of the area).
Resolution mismatch: When MultiDiffusion windows include polar latents, the underlying latent grid has insufficient resolution to represent the actual spherical content in that direction — the model is forced to up-sample a handful of latent features into a full image patch, creating blur and artifacts.
Training distribution shift: The diffusion model encounters latent patterns near the poles that differ fundamentally from those in its perspective-image training data, causing it to produce semantically incoherent outputs.

The solution is conceptually clean: define the latent features directly on the sphere, not on the ERP plane. If each latent feature is associated with a specific direction on $\mathbb{S}^2$ and the features are distributed uniformly across the sphere's surface, then every viewing direction gets equal representational capacity, and no region suffers from the extreme distortion of ERP poles.

How This Paper Positions Itself

SphereDiff positions itself as a fundamental geometric fix rather than an incremental improvement to existing pipelines. The paper's central claim is not that it achieves marginally better pole generation than prior work, but that it eliminates the root cause of pole artifacts by replacing the latent representation entirely.

This is a strong claim. It says: the problem isn't with the diffusion model, or the denoising schedule, or the blending strategy — it's with where you store the latents. By moving from an ERP grid to a spherical point set, you remove the representational imbalance that causes all downstream artifacts. The remainder of the method (dynamic latent sampling, distortion-aware weighted averaging) handles the practical challenge of making this spherical representation work with standard diffusion models that expect 2D grids, but these are implementation details — the core insight is the spherical latent representation itself.

The paper explicitly contrasts this geometric approach with data-driven alternatives in the conclusion:

"Recent panoramic image generation approaches handle more complex scenes by relying on large-scale panoramic image datasets, but they cannot be readily adapted to panoramic videos due to the limited availability of panoramic video datasets. In contrast, our approach achieves state-of-the-art performance in generating 360° live wallpapers without additional tuning"

This reveals the paper's strategic positioning: by being tuning-free, SphereDiff is inherently immune to the data scarcity that limits ERP fine-tuning methods. It can immediately leverage any new, more powerful diffusion model (FLUX, HunyuanVideo) without any retraining — the method is applicable the moment a new model is released. This "future-proof" property is a key differentiator from data-driven methods that must be retrained for each new model architecture.

The MultiDiffusion Extension: Why Tuning-Free Matters

The paper's choice to build on MultiDiffusion rather than proposing an entirely different generation paradigm is deliberate and well-motivated. MultiDiffusion provides a proven framework for generating content larger than a model's native resolution while maintaining global coherence through overlapping patches. By extending this to spherical latents, the paper inherits MultiDiffusion's theoretical guarantees about convergence and blending while fixing its fundamental representation problem.

The extension requires solving several non-trivial sub-problems that the paper addresses in Sections 3.2–3.4:

Defining the spherical latent representation (Section 3.1): Choosing a point distribution on $\mathbb{S}^2$ that is uniform, scalable, and well-behaved under projection.
Projecting spherical latents to perspective grids (Section 3.3): Converting a set of points on the sphere into a 2D grid that a pretrained diffusion model can process, without introducing the undersampling and duplication artifacts of naive nearest-neighbor approaches.
Weighting the blending (Section 3.4): Accounting for the fact that spherical-to-perspective projection itself introduces geometric distortion (radial stretching) that, while much milder than ERP distortion, still affects cross-view consistency.

These are the technical contributions, but they all serve the motivating insight that the latent representation must be spherical, not equirectangular.

Summary of the Motivation

The paper addresses a clear hierarchy of problems:

Practical demand: AR/VR needs abundant 360° content, but capturing it requires specialized hardware.
Data bottleneck: Existing approaches that fine-tune on ERP datasets are limited by scarce, domain-specific training data and cannot generate diverse, text-controlled outdoor scenes.
Representational flaw: Tuning-free approaches using ERP latents inherit ERP's uneven spherical sampling, causing pole-stretching artifacts that persist regardless of the denoising or blending strategy.
Root cause solution: Replacing ERP latents with uniformly distributed spherical latents eliminates the representational imbalance at its source, enabling tuning-free generation that matches or exceeds fine-tuned quality while maintaining full compatibility with any pretrained diffusion model.

The paper's positioning is clear: it is not competing on training data quantity or model architecture — it is competing on geometric correctness, arguing that getting the geometry right is both necessary (ERP methods will always have pole artifacts) and sufficient (with proper sampling and weighting, spherical latents produce seamless results without fine-tuning).

3. Technical Approach

3.1 Reader Orientation

SphereDiff is a generation pipeline that takes a text description and produces a seamless, high-quality 360° panoramic image or video by repurposing an off-the-shelf perspective-based diffusion model without any fine-tuning. The core problem it solves is that standard 360° generation methods store pixel data in equirectangular projection (ERP), which concentrates pixels at the poles and starves the equator, causing severe distortion that pretrained models cannot handle; SphereDiff fixes this by redefining where the model stores its internal features — moving them from a flat ERP grid onto a uniformly sampled sphere — and builds a complete generation framework around this spherical representation.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components that operate in a loop, visualized in Figure 3:

Spherical Latent Representation — a set of ~2,600 latent feature vectors, each pinned to a specific direction on the unit sphere via the Fibonacci lattice, forming a uniform point cloud that represents the state of the entire panorama being generated. This replaces the standard 2D ERP latent grid.
Spherical-to-Perspective Projector — for each of 89 predefined camera viewpoints (distributed across the sphere), this component takes the current spherical latents, projects all points visible from that viewpoint onto a 2D perspective plane, and arranges them into a standard image-shaped grid that a pretrained diffusion model can consume.
Perspective Diffusion Denoiser — an unmodified, frozen pretrained text-to-image or text-to-video model (SANA, FLUX, LTX-Video, or HunyuanVideo) that receives a single perspective view's worth of noise-corrupted latents and a region-specific text prompt, and performs one denoising step.
Distortion-Aware Weighted Blender — after all 89 views have been independently denoised, this component maps the denoised latents from perspective space back onto the spherical representation, blending overlapping views using weights that decay exponentially with distance from each view's optical center, compensating for the radial stretching of spherical-to-perspective projection.
Multi-Prompt Router — assigns each of the 89 view directions to one of five region-specific text prompts (top, upper, middle, lower, bottom) based on elevation angle, optionally adding a foreground prompt at a specific azimuth-elevation coordinate.

Information flows as follows: pure spherical noise is initialized on the Fibonacci sphere → for each diffusion timestep and each of 89 view directions, spherical latents are projected to a perspective grid and denoised by the pretrained model with a region-appropriate prompt → the 89 denoised perspective views are projected back to the spherical representation and blended with distortion-aware weights → the process repeats from the noisiest timestep to the clean output → the final spherical latents are decoded to RGB by stitching perspective views into an ERP image.

3.3 Roadmap for the Deep Dive

First, the spherical latent representation itself (Section 3.1): what it is mathematically, how points are chosen on $\mathbb{S}^2$ , and why the Fibonacci lattice rather than ERP sampling — because this is the foundational representational choice that everything else builds upon.
Second, the perspective latent representation and projection function (Section 3.1, continued): how a direction on the sphere becomes a $(u, v)$ coordinate on a 2D plane, since this is the bridge between the spherical world and the rectangular-input world of pretrained diffusion models.
Third, the MultiDiffusion framework extension (Section 3.2): how MultiDiffusion's patch-based denoising is adapted to work on spherical rather than 2D latents, including the mathematical mapping functions $F_i$ and the overall denoising equation — because this defines the generation loop.
Fourth, dynamic latent sampling (Section 3.3): the practical algorithm that converts a scattered set of projected spherical points into a clean 2D grid, why nearest-neighbor fails, and how the center-first queue prevents undersampling — because this is the key engineering insight that makes the system actually work.
Fifth, distortion-aware weighted averaging (Section 3.4): the exponential weighting scheme that compensates for the residual geometric distortion of spherical-to-perspective projection — because even a near-perfect spherical representation still needs careful blending across overlapping views.
Sixth, multi-prompt inference (Section 3.5): how different regions of the sphere receive different text prompts, and how foreground objects are composited — because this addresses the semantic challenge that a 360° scene contains sky, horizon, and ground simultaneously.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a systems paper with a geometric insight: the primary contribution is not a new neural architecture or training loss, but a novel representation — spherical latents — and the set of algorithmic components (dynamic sampling, distortion-aware weighting, multi-prompt inference) needed to make that representation interoperable with standard perspective diffusion models inside the MultiDiffusion framework.

3.4.1 The Spherical Latent Representation

The spherical latent representation is the central abstraction of SphereDiff. Instead of storing the panorama's features on a 2D rectangular grid (as ERP does), a spherical latent is a feature vector paired with a direction on the unit sphere.

Concretely, a single spherical latent $\mathbf{s}_i$ is a tuple $(\mathbf{d}_i, \mathbf{f}_i)$ where:

$\mathbf{d}_i = (x, y, z) \in \mathbb{R}^3$ is a unit-norm direction vector pointing from the center of the sphere to a point on its surface, satisfying $\|\mathbf{d}_i\| = 1$ .
$\mathbf{f}_i \in \mathbb{R}^C$ is the associated feature vector — this is the "pixel" of the latent representation, analogous to a single spatial position's feature channel in a standard 2D diffusion latent. $C$ is the channel dimension of the pretrained diffusion model's latent space (e.g., SANA uses a specific latent dimension determined by its VAE).

The full spherical latent set $\mathcal{S}$ for $N$ points is defined as:

$\mathcal{S} = \{\mathbf{s}_i = (\mathbf{d}_i, \mathbf{f}_i) \mid \mathbf{d}_i \in \mathbb{S}^2, \mathbf{f}_i \in \mathbb{R}^C, \text{ for } i \in [1, N]\}$

where $\mathbb{S}^2$ is the unit sphere surface — the set of all points in 3D space at distance exactly 1 from the origin.

What this representation is and is not. This is not a dense voxel grid or a mesh — it is a point cloud on the sphere. There is no notion of adjacency, connectivity, or ordering among the $N$ points. The only structure is that each point has a known direction $\mathbf{d}_i$ , which determines where it projects onto any given perspective view. This is conceptually similar to how 3D Gaussian Splatting stores features at scattered 3D points rather than on a regular grid.

Point distribution — the Fibonacci lattice. The critical design choice is how to choose the $N$ directions $\mathbf{d}_i$ . The paper uses the Fibonacci lattice (Hardin, Michaels, and Saff, 2016), a deterministic point set on $\mathbb{S}^2$ known for its near-uniform coverage. The key property that matters for SphereDiff is:

"the number of spherical latents is nearly equal across all perspectives, as shown in Fig. 2(c)"

This means: if you look at the sphere from any direction, the number of latent points within your field of view is roughly constant — there are no "pole" or "equator" biases. The Fibonacci lattice achieves this by distributing points along a spiral on the sphere with equal angular spacing in azimuth and a golden-ratio-based increment in elevation, ensuring no clustering at any latitude.

Contrast this with erp-formatted latents, which the paper shows can also be described as points on the sphere (by mapping the ERP $(u, v)$ coordinates through the inverse ERP mapping to $\mathbb{S}^2$ ), but the resulting directions are:

"not uniformly distributed on the sphere's surface. In contrast, we define the spherical latents using the Fibonacci Lattice"

The paper uses $N = 2{,}600$ spherical latent points across all experiments (Section E). This number was chosen to balance representational capacity with computational cost; each of the 89 view directions captures a subset of these 2,600 points within its 80° field of view.

Why this representation eliminates pole artifacts. The core logic is: every solid angle on the sphere receives approximately the same number of latent features. When you look toward the pole, the model is not forced to extrapolate from a few squeezed latent features (as in ERP) — the pole region has just as many independent latent features to represent its content as any equatorial region. The diffusion model processes these features in perspective space (where they appear as normal image patches), and the uniform spherical distribution ensures that no direction is starved or oversampled.

Why Fibonacci lattice rather than HEALPix or icosahedral sampling. The paper does not discuss alternative spherical point distributions, but the Fibonacci lattice offers specific advantages for this application: it is trivial to generate for any $N$ , it has no singularities or seam boundaries (unlike cube maps or HEALPix faces), and it produces a smooth spiral distribution that maps gracefully to perspective projection (no hard edges where neighboring points come from different faces of a polyhedron).

3.4.2 Perspective Latent Representation and Spherical-to-Perspective Projection

Standard diffusion models operate on 2D grids — they expect input latents shaped as $C \times H \times W$ with a regular pixel structure. To connect the spherical point cloud to this 2D world, the paper defines a spherical-to-perspective projection that, given a camera viewpoint $\mathbf{v} \in \mathbb{S}^2$ , maps each spherical direction $\mathbf{d}_i$ to a coordinate on a virtual camera's image plane.

The domain of perspective coordinates is a discretized 2D plane:

$\mathbb{P}^2 = \left\{ \mathbf{u} = \left(\frac{2j}{H}, \frac{2k}{W}\right) \mid j \in \left[-\frac{H}{2}, \frac{H}{2}\right], k \in \left[-\frac{W}{2}, \frac{W}{2}\right] \right\}$

where $H$ and $W$ are the height and width of the perspective latent grid. The coordinates $\mathbf{u}$ range from $[-1, 1]$ in both dimensions, representing the normalized image plane. Points with $\|\mathbf{u}\| > 1$ fall outside the field of view and are discarded; the field-of-view angle is controlled by a focal length $f$ , which determines how much of the sphere maps into the $[-1, 1]^2$ square.

The projection function. The mapping from spherical direction to perspective coordinate is the standard pinhole camera projection, detailed in Appendix A:

$\tilde{u} = K[R|t]\tilde{d}$

where $\tilde{d} = (x, y, z, 1)^\top$ is the homogeneous coordinate of the 3D point on the sphere, $K$ is the $3 \times 3$ intrinsic camera matrix derived from focal length $f$ , $R$ is the $3 \times 3$ rotation matrix encoding the view direction $\mathbf{v}$ , and $t = \mathbf{0}$ (the camera is at the origin). The result $\tilde{u} = (u', v', w')^\top$ is then converted to 2D coordinates via perspective division:

$u = \left( \frac{u'}{w'}, \frac{v'}{w'} \right)$

What this computes: for each of the $N$ spherical directions, determines whether it is visible from the given camera pose, and if so, where it lands on the 2D image sensor. Points behind the camera (negative $w'$ ) are masked out. The output is a set of pairs $(\mathbf{u}_i, \mathbf{f}_i)$ — 2D coordinates paired with their original feature vectors — but these coordinates are continuous-valued (not aligned to a pixel grid) and scattered irregularly across the image plane.

Spherical-to-perspective latent transformation. The full operation for a given viewpoint $\mathbf{v}$ and focal length $f$ is:

$\mathcal{T}_{\mathcal{S} \rightarrow \mathcal{P}}(\mathbf{S}|\mathbf{v}, f) = \mathbf{P} = \{\mathbf{p}_i = (\mathbf{u}_i, \mathbf{f}_i) \mid \mathbf{u}_i \in [-1, 1]^2\}$

where $\mathbf{P}$ contains fewer than $N$ elements since only points within the $[-1, 1]^2$ bounds are retained. This is the "raw projection" — an unstructured set of features with continuous coordinates — before any discretization into a grid. The transformation to a proper perspective latent $\mathbf{I}$ (a structured $C \times H \times W$ tensor) is the job of the latent sampling method (Section 3.4.3).

Focal length and field of view. The paper uses an 80° field of view for each perspective window. The relationship between focal length $f$ and field of view in normalized coordinates is $f = 1 / \tan(\text{FoV}/2)$ , so an 80° FoV corresponds to $f \approx 0.84$ in the normalized coordinate system where the image plane extends to $\pm 1$ . Adjacent views overlap by 60%, meaning that the angular distance between neighboring view directions is only 40% of the FoV — a deliberate choice to ensure extensive overlap for MultiDiffusion blending.

The geometric distortion of spherical-to-perspective projection. While this projection is the standard pinhole camera model — identical to how real cameras form images — it introduces radial stretching: points on the sphere at angle $\theta$ from the camera's optical axis project to a distance $d_p = f \tan \theta$ on the image plane, whereas their true arc length on the sphere is $d_s = f \theta$ . For small $\theta$ (near the image center), $\tan \theta \approx \theta$ , so the distortion is negligible. For large $\theta$ (near the image edges), $\tan \theta \gg \theta$ , causing features to appear stretched.

The key insight the paper emphasizes is that this spherical-to-perspective distortion is far milder than ERP-to-perspective distortion. ERP distortion compresses the equator and stretches the poles nonlinearly; spherical-to-perspective distortion is the familiar wide-angle lens effect that affects all directions equally (it depends only on angular distance from the view center, not on absolute latitude). The paper handles this residual distortion through the weighted averaging scheme in Section 3.4.4.

3.4.3 Extending MultiDiffusion to Spherical Latents

MultiDiffusion (Bar-Tal et al., 2023) provides the mathematical framework for generating a large composite image by repeatedly denoising overlapping patches and blending them. The paper adapts this framework from the 2D ERP domain to the spherical domain, defining the Spherical MultiDiffuser $\Psi_{\mathcal{S}}$ .

The generation loop. Starting from pure Gaussian noise $\mathbf{S}_T$ on the $N = 2{,}600$ spherical latents, the system iteratively produces $\mathbf{S}_{T-1}, \mathbf{S}_{T-2}, \dots, \mathbf{S}_0$ through the recurrence:

$\mathbf{S}_{t-1} = \Psi_{\mathcal{S}}(\mathbf{S}_t|\mathbf{z})$

where $\mathbf{S}_t$ is the noisy spherical latent at timestep $t$ , and $\mathbf{z}$ is a set of text conditions at different elevation levels. The MultiDiffuser $\Psi_{\mathcal{S}}$ produces one denoised spherical latent from one noisy input.

Construction of the MultiDiffuser. The paper decomposes $\Psi_{\mathcal{S}}$ into three pieces, following the MultiDiffusion template:

$\Psi(\mathbf{S}_t|\mathbf{z}) = \sum_{i=1}^n \mathbf{W}_i^{\mathcal{S}} \otimes F_i^{-1}(\Phi(\mathbf{I}_t^i|\mathbf{y}_i))$

where each symbol requires careful unpacking:

$n$ is the number of view directions (89, as specified in Appendix E).
$F_i : \mathcal{S} \rightarrow \mathcal{I}$ is the latent mapping function for view $i$ : it takes the current spherical latents $\mathbf{S}_t$ , projects them to viewpoint $i$ , and produces a standard 2D perspective latent $\mathbf{I}_t^i$ that the pretrained diffusion model can process. This function encapsulates both $\mathcal{T}_{\mathcal{S} \rightarrow \mathcal{P}}$ and the discretization algorithm (dynamic latent sampling, Section 3.4.3).
$F_i^{-1}$ is the inverse mapping: after denoising, it takes the denoised 2D latent and maps the features back to their original spherical directions, producing a sparse update to the spherical latents (only the directions visible from view $i$ receive updates).
$\Phi : \mathcal{I} \times \mathcal{Y} \rightarrow \mathcal{I}$ is the pretrained diffusion model — completely frozen, unmodified — that performs one denoising step on a standard perspective latent given a text condition. Its internal denoising schedule, classifier-free guidance, and noise prediction are all standard and unchanged.
$\mathbf{y}_i = \lambda_i(\mathbf{z})$ is the text prompt selected for view $i$ based on its elevation (Section 3.4.5).
$\mathbf{W}_i^{\mathcal{S}} \in \mathbb{R}^{|\mathcal{S}|}$ are per-spherical-latent blending weights for view $i$ , derived from the distortion-aware exponential weighting (Section 3.4.4). Each spherical latent point receives a weight from each view it appears in.
$\otimes$ is the Hadamard (element-wise) product: each spherical latent feature is multiplied by its view-specific weight before being summed.

What this equation computes, operationally: for a single denoising step, the system (1) projects the noisy spherical latents to $n = 89$ separate perspective views, (2) denoises each view independently with the frozen pretrained model using the appropriate text prompt, (3) maps each view's denoised features back to the spherical representation, (4) blends the $n$ contributions at each spherical point using normalized weights. The blending is a convex combination because the weights $\mathbf{W}_i$ are normalized across views.

Why this decomposition matters. The equation cleanly separates three concerns: (a) geometric mapping ( $F_i$ and $F_i^{-1}$ ) — how to go between spherical and perspective representations; (b) content generation ( $\Phi$ ) — the pretrained model's denoising capability, completely unmodified; and (c) blending ( $\mathbf{W}_i$ ) — how to combine information from multiple views. This separation means that any improvement to the pretrained model (a new SANA release, a better video model) immediately benefits SphereDiff without any change to the mapping or blending code.

View direction configuration. The 89 view directions are laid out as rings at different elevation angles with varying numbers of azimuthal steps, as detailed in Appendix E:

$\phi = \pm 90^\circ$ (top/bottom poles): 4 azimuthal steps each ( $k = 0, 1, 2, 3$ )
$\phi = \pm 77.5^\circ$ : 8 azimuthal steps each
$\phi = \pm 45^\circ$ : 11 azimuthal steps each
$\phi = \pm 22.5^\circ$ : 14 azimuthal steps each
$\phi = 0^\circ$ (equator): 15 azimuthal steps

This totals $4+4+8+8+11+11+14+14+15 = 89$ views. The decreasing number of views at higher elevations reflects the decreasing circumference of the sphere at those latitudes: the equator needs more views to maintain consistent overlap than the poles. The paper notes that this number was empirically chosen to balance quality (more views = better blending) with runtime (89 views means 89 forward passes through the diffusion model per timestep).

Resolution and temporal configuration. The perspective resolution is set to match each pretrained model's native capabilities: $1024 \times 1024$ for SANA, FLUX, and HunyuanVideo; $512 \times 512$ for LTX-Video. For video, the temporal length is fixed at 121 frames for both LTX-Video and HunyuanVideo.

3.4.4 Dynamic Latent Sampling: From Scattered Projections to Clean 2D Grids

The mapping function $F_i$ must convert the irregular set of projected points $\mathbf{P}$ (continuous $\mathbf{u}$ coordinates with associated features) into a regular $C \times H \times W$ grid that the diffusion model expects. This is a discretization problem: which feature goes into which grid cell?

The paper first discusses and rejects nearest-neighbor sampling, then introduces dynamic latent sampling as the solution.

Nearest-neighbor sampling and its failures. The straightforward approach: for each pixel position $(j, k)$ on the $H \times W$ grid, find the projected spherical latent whose $\mathbf{u}$ coordinate is closest and assign its feature. The paper identifies two critical failure modes, illustrated in Figure 4(a):

Duplicate selection: The same spherical latent may be the nearest neighbor for multiple grid cells. When this happens, the same feature vector appears multiple times in the 2D latent, which "alters the latent distribution, which often degrades generation performance" (citing Chang et al., 2024). The diffusion model expects each spatial position to have independent noise and independent content; duplicates break this assumption.
Undersampling: Some spherical latents that are within the field of view may not be selected at all — they aren't the nearest neighbor for any grid cell. This means those latents are effectively invisible to the diffusion model for this view. The paper describes this as "the undersampling problem" and explains its consequence:

"The undersampling of spherical latents disrupts information flow across neighboring windows... If the next window's FoV captures a green point, not the blue point, it receives no information from the current window, causing discontinuities even when there is a large overlap."

In Figure 4(a), the green points — even though they fall within the current view's FoV — are not selected because other (blue) points are closer to the grid centers. When the next view (with an overlapping FoV) samples those green points, they carry no denoising update from the current timestep because they weren't denoised in the current view. The overlap — which MultiDiffusion relies on for seamlessness — becomes ineffective because the two views are updating disjoint subsets of the spherical latents.

Dynamic latent sampling algorithm. The solution, presented as Algorithm 1 and illustrated in Figure 4(b), has three design elements:

Queue (no duplicates): Once a spherical latent is selected and placed into the grid, it is removed from the pool of available latents. No latent can be chosen twice for the same perspective view.
Dynamic grid size: The grid dimensions $H$ and $W$ are not fixed a priori. Instead, they are determined from the number of projected latents $M = |\mathbf{P}|$ : $H = \lfloor \sqrt{M} \rfloor$ and $W = \lceil \sqrt{M} \rceil$ . This ensures the grid can accommodate most of the projected latents without forcing a predetermined aspect ratio that would either leave empty cells or require discarding many latents.
Center-first selection (solves undersampling): The projected latents are sorted by their distance from the center of the perspective image, $\|\mathbf{u}_i\|$ . The algorithm then fills the grid from the center outward in concentric square rings (borders). The first batch fills the central $2 \times 2$ region (the $i = 1$ border with $n = (2)^2 - (0)^2 = 4$ latents), the next batch fills the $i = 2$ border with $n = (4)^2 - (2)^2 = 12$ latents, and so on.

The upshot: points near the center — which are geometrically most important (they experience the least spherical-to-perspective distortion and are most relevant for MultiDiffusion blending with other views) — are guaranteed to be sampled. Points at the outermost edge may be discarded if $M > H \times W$ , but these are the points with maximum projection distortion and least contribution to overlapping regions, so discarding them is the right trade-off.

What "discard outermost points" means geometrically: the view captures an 80° FoV, meaning points up to 40° from the optical axis. The grid is sized for $\lfloor \sqrt{M} \rfloor \times \lceil \sqrt{M} \rceil$ , which may be slightly smaller than what would be needed to include every projected point. The outermost ~10-15% of points (those with the highest $\|\mathbf{u}\|$ ) are dropped. These correspond to the periphery of the 80° FoV, where spherical-to-perspective distortion is strongest and where another view's optical center will likely provide better coverage.

Why center-first is correct. The center of each perspective view is geometrically privileged: it corresponds to the view direction $\mathbf{v}$ , where the tangent-plane approximation to the sphere is exact (no projection distortion). Latents near the center receive the "cleanest" denoising signal from the diffusion model. By prioritizing center latents, dynamic sampling ensures that each view contributes its highest-quality information to the overall spherical representation, while sacrificing periphery information that is better supplied by neighboring views whose centers align with those regions.

Alternative approaches considered and rejected (Appendix C.2). The paper discusses two alternatives:

Continuous positional embeddings: Modern DiT-based models support continuous 1D positional encodings. One might think: skip discretization entirely — just feed the continuous $\mathbf{u}$ coordinates to the model. The paper shows this fails because the positional similarity structure collapses. In standard 2D positional embeddings, latents in the same row or column share high attention similarity (Figure 9, left). With continuous coordinates, even slightly different positions have low similarity (Figure 9, right), causing the model to fail at generating structured output. The paper concludes: "Although DiT can process continuous inputs, discretization remains essential for tuning-free panoramic visual generation to maintain structured and consistent latent relationships."
Latent interpolation (bilinear, stochastic warping): Standard interpolation would blend features from multiple spherical latents to fill each grid cell, but VAEs used in diffusion models are not interpolation-equivariant — interpolating in latent space does not correspond to interpolating in pixel space. The paper found that stochastic warping methods (Chang et al., 2024; Daras et al., 2024) "provide suboptimal results in generating 360° live wallpapers." Hence the choice to use sampling (exact selection of individual latents) rather than interpolation.

3.4.5 Distortion-Aware Weighted Averaging

Even with uniformly distributed spherical latents and center-first dynamic sampling, the spherical-to-perspective projection introduces residual geometric distortion: points far from the view center are stretched radially compared to their true spherical spacing. This means that when two views overlap, the same spherical region is represented at different scales (different degrees of stretch) in each view, potentially causing misalignment during blending.

The paper addresses this with a distortion-aware weighting function integrated into the MultiDiffusion blending weights $\mathbf{W}_i^{\mathcal{S}}$ .

The weighting function. The weight for a pixel at position $(j, k)$ in view $i$ is:

$W_i^{jk} = \exp(-\|\mathbf{u}_{jk}\| / \tau)$

where $\|\mathbf{u}_{jk}\| \in [0, \sqrt{2}]$ is the Euclidean distance from the image center (since $\mathbf{u} \in [-1, 1]^2$ ), and $\tau$ is a temperature parameter controlling the decay rate.

What this computes: an exponentially decaying weight that peaks at 1.0 at the image center and falls off toward the edges. Pixels near the view's optical center — where spherical-to-perspective distortion is minimal — receive the highest weight in the blending. Pixels near the periphery — where distortion is maximal — receive exponentially lower weight.

Why exponential decay rather than, say, Gaussian or linear. The paper states in Appendix A.2:

"While this weighting scheme is not a perfect theoretical inverse of the distortion, our empirical results indicate that more complex models do not yield a significant improvement in quality. Therefore, we just adopted the simplest exponential form."

The key insight is that the distortion grows as $\tan \theta / \theta$ , which is roughly exponential-like in its divergence from 1. An exponential weight down-weights distorted regions in approximate proportion to their geometric unreliability. The key design principle is that the center of each view — where projection is most accurate — should dominate the blended output near that region.

Mapping back to spherical weights. The 2D pixel weights $\mathbf{W}_i^{\mathcal{I}} = [W_i^{jk}]$ are mapped back to spherical latent weights via the inverse mapping:

$\mathbf{W}_i^{\mathcal{S}} = F_i^{-1}(\mathbf{W}_i^{\mathcal{I}})$

meaning: each spherical latent receives the weight of the grid cell it was assigned to during dynamic sampling. Latents not sampled in view $i$ receive zero weight from that view.

How blending resolves cross-view inconsistencies. When two neighboring views overlap by 60%, a spherical latent point near the boundary of view A and the center of view B will receive a low weight from A (it's near A's periphery, where $\|\mathbf{u}\|$ is large) and a high weight from B (near B's center). This means view B's "cleaner" denoising signal dominates, while view A contributes a small corrective influence. The exponential weighting thus implements a soft form of view selection: each spherical point's content is primarily determined by the view that looks at it most directly, with peripheral views contributing only in proportion to their geometric reliability.

The temperature $\tau$ : The paper does not specify the exact value of $\tau$ used in experiments, but it controls how quickly weights decay. A small $\tau$ makes weights drop rapidly, so only the very center of each view matters — this produces sharper transitions between views. A large $\tau$ makes weights nearly uniform, giving equal influence to distorted periphery — this produces smoother but potentially blurrier blends. The optimal $\tau$ balances these.

Quantitative ablation on weighted averaging (Table 3). Removing weighted averaging (nearest sampling only) reduces Distortion score from 3.238 to 2.039 and End Continuity from 4.892 to 3.400. The ablation confirms that weighted averaging is responsible for a significant fraction of the panoramic quality improvement — it is not merely cosmetic. The Image Quality score drops from 4.496 to 3.014, suggesting that without distortion-aware weighting, cross-view misalignment produces visible blending artifacts even in the image quality metric (which does not explicitly measure distortion).

3.4.6 Multi-Prompt Inference: Region-Specific Text Conditioning

A 360° panorama is semantically heterogeneous: the top should show sky, the bottom should show ground, and the horizon should show distant scenery. A single global prompt (e.g., "a beautiful forest") underspecifies this structure — the model has no information about which part of the sphere should contain which content.

SphereDiff addresses this with multi-prompt inference: different view directions receive different text prompts based on their elevation.

The condition mapping function $\lambda_i$ . For each view direction $\mathbf{v}_i$ , its elevation angle $\phi_i = \text{elevation}(\mathbf{v}_i)$ is computed (from $-90^\circ$ at the bottom to $+90^\circ$ at the top). The function $\lambda_i$ selects from a discrete set of five elevation-tagged prompts:

$\mathbf{z} = \{\mathbf{y}_{\text{bottom}}, \mathbf{y}_{\text{lower}}, \mathbf{y}_{\text{middle}}, \mathbf{y}_{\text{upper}}, \mathbf{y}_{\text{top}}\}$

corresponding to elevations $\{-90^\circ, -10^\circ, 0^\circ, +10^\circ, +90^\circ\}$ . The mapping simply assigns each view the prompt whose reference elevation is closest to $\phi_i$ .

Why more prompts near the horizon. The prompt bins are non-uniformly spaced: four bins cluster near the horizon ( $-10^\circ, 0^\circ, +10^\circ$ ) while only two cover the poles ( $-90^\circ, +90^\circ$ ). The paper states:

"We allocate denser text prompts near the horizon to capture richer visual complexity in that region."

The horizon is where most semantic content lives (trees, buildings, mountains, people), while the poles typically contain simpler content (uniform sky, flat ground). This density allocation reflects the fact that the horizon spans 360° of azimuth at each of several elevations, requiring more varied prompting than the single-point poles.

Foreground object generation. For additional scene complexity, an optional foreground prompt $\mathbf{y}_{\text{foreground}}$ can be attached to specific view directions at user-specified spherical coordinates (e.g., $\theta = 30^\circ, \phi = -30^\circ$ ). The foreground view uses a beta kernel weighting instead of the exponential weighting used for background views:

$W_i^{jk} = \mathcal{B}(\|\mathbf{u}_{jk}^*\|; b) = (1 - x)^{4e^b}$

where $x = \min(\|\mathbf{u}_{jk}\| / d_{\max}, 1)$ is a normalized distance, $d_{\max}$ is the maximum distance in the image, and $b = -3.0$ controls the kernel shape.

What the beta kernel does differently. Compared to the exponential kernel (which peaks sharply at the center and decays quickly), the beta kernel produces a flatter, plateau-like distribution — it stays near 1.0 over a broader central region before dropping off. This ensures that the foreground object "dominates" its designated view region without competition from background prompts. The paper states:

"Compared to the exponential kernel, the beta kernel produces a flatter distribution, allowing the model to generate foreground objects that remain dominant throughout the entire foreground region."

The parameter $b$ controls the width of the plateau; $b = -3.0$ was chosen empirically.

Refinement stage for foreground compositions. When foreground objects are composited, the paper adds an optional refinement stage (used only for SANA-generated panoramas in Figure 1, due to computational cost):

"After the initial panorama generation, we perform a noise-to-denoise process: noise corresponding to a specific timestep is added to the panorama, which is then refined by a pretrained diffusion model."

This is analogous to SDEdit or image-to-image diffusion: add a modest amount of noise (at a specific intermediate timestep, not full noise) to the generated panorama, then denoise again with the same model. This leverages the model's generative prior to smooth seams between foreground and background without additional training.

Single-prompt baselines for comparison. For fair comparison, the single-prompt baselines (360 LoRA, Text2Light, PanFusion, 360DVD) receive only the middle prompt $\mathbf{y}_{\text{middle}}$ , while DynamicScaler receives the full multi-prompt set since it also supports multi-prompt inference. This controlled comparison isolates the geometric representation's effect from the prompting strategy's effect.

3.4.7 Summary of Design Choices and Their Justifications

Fibonacci lattice for spherical point distribution over ERP grid or cube map: ensures uniform latent density across all viewing directions, eliminating the pole-stretching artifacts that plague ERP-based methods. The choice is geometric rather than learned — no training data required.
Center-first dynamic sampling over nearest-neighbor or interpolation: prevents undersampling (which breaks MultiDiffusion overlap) and duplicate selection (which distorts the latent distribution), while prioritizing geometrically reliable center points. The queue mechanism guarantees each latent is used at most once per view.
Dynamic grid sizing ( $H = \lfloor \sqrt{M} \rfloor$ ) over fixed grid: automatically matches the grid capacity to the number of projected latents, minimizing both empty cells and discarded latents. This makes the method robust to variations in $N$ and field of view.
Exponential distortion-aware weighting over uniform blending: down-weights pixels with high spherical-to-perspective distortion, ensuring each view's contribution is proportional to its geometric accuracy. The exponential form is the simplest function that captures the radial growth of distortion.
Five elevation-specific prompts over single global prompt: addresses the semantic heterogeneity of 360° scenes (sky above, ground below) without requiring the diffusion model to infer spatial layout from a single ambiguous text. The prompt density near the horizon reflects where most visual complexity lives.
80° FoV with 60% overlap: provides enough overlap for MultiDiffusion blending to converge to a seamless output while keeping the number of views manageable (89 views). Larger FoV would reduce views but increase per-view distortion; smaller FoV would reduce distortion but require more views (quadratically more, since angular area grows as $\text{FoV}^2$ ).
Tuning-free design: by avoiding any fine-tuning, the method immediately benefits from any new pretrained diffusion model and avoids the data scarcity bottleneck (small ERP datasets) that limits fine-tuning-based approaches. The cost is inference-time computation (89 denoising passes per timestep), but this is a one-time generation cost, not a recurring training cost.
2,600 spherical latent points: chosen to balance representational capacity with memory and runtime. Each of the 89 views sees a subset of these; with 60% overlap, each spherical point appears in approximately 3–5 views on average, providing sufficient blending multiplicity.

4. Key Insights and Innovations

Innovation 1: Reframing "Pole Distortion" as a Problem of Latent Representation, Not Denoising or Dataset Size

The dominant framing in prior 360° generation work treats pole artifacts as a quality problem to be overcome through more training data, better architectural inductive biases, or cleverer blending. ERP fine-tuning methods (360 LoRA, Text2Light, PanFusion) implicitly assume that with enough panoramic training examples, the model can learn to compensate for ERP's geometric distortion. Tuning-free methods like DynamicScaler treat the problem as one of denoising strategy — if overlapping windows and offset shifting are tuned carefully, the artifacts should diminish. Both framings locate the solution in the model or the training, not in the representation.

SphereDiff makes a fundamentally different diagnostic move. It identifies the root cause as uneven latent density on the spherical surface — not a failure of the diffusion model to denoise correctly, but a failure of the ERP grid to allocate representational capacity uniformly across viewing directions. This is visible in Figure 2: ERP latents map to a wildly non-uniform distribution on $\mathbb{S}^2$ , with extreme clustering at the poles and sparsity at the equator. The pole-stretching artifacts are not a symptom of insufficient training or suboptimal denoising; they are the inevitable consequence of asking a model trained on uniform-scale perspective images to process latents whose underlying spherical density varies by orders of magnitude.

This reframing is significant because it changes where one looks for solutions. If pole artifacts are a data problem, the path forward is collecting larger panoramic datasets (expensive, domain-limited). If they are a denoising problem, the path is better MultiDiffusion scheduling or post-processing (unlikely to fully resolve the representational imbalance). But if they are a representation problem, the solution is to change how latents are distributed — a geometric fix that sidesteps both data scarcity and model retraining entirely. The paper's central claim is that this diagnostic move is correct: by adopting uniformly distributed spherical latents, the artifacts that plague every baseline (visible in the pole regions of Figure 5 and Figures 10–11) simply disappear, without any fine-tuning and with off-the-shelf diffusion backbones.

This is a fundamental reframing, not incremental improvement. It reclassifies a class of generation failures from "model limitation" to "representation bug," and the fix — Fibonacci lattice point distribution — is computationally trivial compared to the training pipelines it replaces.

The experimental vindication is in the panoramic metrics of Table 1 and Table 2. On Distortion, SphereDiff achieves 38.10% win rate in user study (vs. 21.43% for 360 LoRA and 20.24% for DynamicScaler, the next-best tuning-free method) and an automated Distortion score of 3.238 (vs. 2.854 for DynamicScaler). The End Continuity gap is even larger: 35.71% vs. 25.00% in user study, 4.892 vs. 3.985 in automated scoring. These are not marginal improvements — they are 1.5× to 1.8× the nearest competitor's score on exactly the metrics that measure panoramic coherence, precisely the failure mode the representation fix targets.

Innovation 2: The "Undersampling Problem" as a Novel Diagnostic for Multi-View Generation, and Center-First Queue Sampling as Its Resolution

MultiDiffusion and its derivatives (including DynamicScaler) rely on overlapping windows to enforce seamless blending: adjacent views share content in their overlapping regions, so the weighted average converges to a consistent image. This mechanism implicitly assumes that all spherical points within the overlap region are updated during each view's denoising step. If some points visible in the overlap are not denoised in the current view, then the next overlapping view receives no fused information for those points — the overlap mechanism breaks silently.

The paper identifies a specific failure mode that prior work overlooked: undersampling during latent discretization. When nearest-neighbor sampling selects which spherical latents to place on the 2D grid, some latents within the field of view are simply never selected — they aren't the closest to any grid center. These "forgotten" latents (the green points in Figure 4a) are invisible to the current view's denoising pass. When the next view in the MultiDiffusion loop denoises those latents, it receives no information from the previous view, even though the views overlap geometrically. The result is "discontinuities even when there is a large overlap" (Section 3.3) — a failure mode that looks like insufficient blending but is actually a sampling artifact.

This diagnostic is novel because it identifies the discretization of continuous projections as the bottleneck in spherical MultiDiffusion, not the blending weights or denoising schedule. Prior work using MultiDiffusion on regular grids (2D panoramas, shape expansion) never encountered this issue because the latent grid and the window grid were aligned — every latent pixel belonged to exactly one grid cell. The spherical case introduces a misalignment: the projected points are scattered continuously, and fitting them onto a discrete grid requires selection.

The resolution — dynamic latent sampling with center-first ordering and a queue — is an elegant algorithmic fix that directly targets the diagnostic. By sorting points by distance from the image center and selecting from the center outward, the algorithm guarantees that:

Every point near the center (where projection distortion is minimal and overlap with other views is strongest) is sampled.
No point is sampled more than once (the queue prevents duplicates, preserving the latent distribution's independence structure).
Points discarded at the periphery are those where projection distortion is maximal and where neighboring views provide better coverage.

This is an incremental algorithmic innovation with outsized practical impact. The core idea (sort-and-queue) is simple, but it resolves a failure mode that would otherwise undermine the entire spherical MultiDiffusion approach. The ablation in Figure 6 and Table 3 confirms that dynamic sampling is responsible for a substantial fraction of the panoramic quality: with nearest sampling + weighted averaging, End Continuity drops from 4.892 to 3.400 (automated score) and Distortion drops from 3.238 to 2.421.

The broader significance is that this diagnostic generalizes: any system that discretizes continuous projections onto a regular grid for neural processing (e.g., 3D-to-2D projection in novel view synthesis, point cloud rasterization) is vulnerable to undersampling artifacts when the projection density varies. The center-first queue solution is portable to those domains.

Innovation 3: Characterizing Spherical-to-Perspective Distortion as a First-Class Quantity Requiring Compensation, Rather Than Treating It as "Negligible"

Prior tuning-free panorama methods (MultiDiffusion, DynamicScaler) operate on ERP latents where the projection distortion from ERP-to-perspective is so severe that it dominates all other geometric concerns. In that context, the residual distortion from the perspective projection itself (the $\tan \theta$ vs. $\theta$ discrepancy in Section A.2) is negligible in comparison — it is dwarfed by the ERP compression/stretching. The dominant assumption in prior work is therefore that if you fix the ERP distortion, any remaining projection distortion is too small to matter.

SphereDiff breaks this assumption in a specific and well-documented way. By moving to spherical latents, it eliminates the dominant ERP distortion term entirely, exposing the residual spherical-to-perspective distortion as a now-significant source of blending error. The analysis in Appendix A.2 quantifies this: for a point at angle $\theta$ from the view center, the projected distance on the tangent plane is $d_p = R \tan \theta$ while the true spherical arc length is $d_s = R\theta$ . The ratio $d_p / d_s = \tan\theta / \theta$ grows from ~1.0 at $\theta = 0^\circ$ to ~1.16 at $\theta = 40^\circ$ (the edge of the 80° FoV). This means a feature at the periphery of one view is stretched by ~16% relative to the same feature near the center of an overlapping view, causing misalignment during blending.

The innovation is the recognition that this distortion, while mild in absolute terms, matters once the dominant error source is removed — a concept analogous to "second-order effects become first-order in the absence of the dominant term." The distortion-aware weighted averaging (exponential weight decay with distance from center) is the compensating mechanism, but the intellectual contribution is the diagnostic move: identifying that in a spherical representation, perspective projection distortion transitions from negligible to operationally significant, and therefore requires explicit modeling.

The evidence is in the Table 3 ablation. Adding weighted averaging to dynamic sampling improves Distortion from 2.421 to 3.238 (a 34% relative improvement) and End Continuity from 4.086 to 4.892. Without weighted averaging, even the spherical representation produces measurable panoramic artifacts — confirming that the residual distortion is real and its compensation is necessary. The fact that exponential weighting (the simplest function that captures radial decay) sufficed — "more complex models do not yield a significant improvement in quality" (Appendix A.2) — suggests the compensation does not need to perfectly invert the $\tan\theta / \theta$ curve; it just needs to down-weight distorted regions enough that blending favors geometrically reliable views.

This is an incremental conceptual refinement — not a paradigm shift — but it is methodologically important because it establishes a design principle for any future work on spherical representations: when moving to a geometrically correct latent space, you must account for the projection distortion between your representation and the model's native space, even though in a coarser representation (ERP) that same distortion was invisible against a larger error.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses 20 predefined text prompt sets designed for immersive outdoor scenes (listed in Appendix G, Figure 17). These prompts cover natural landscapes — forests, canyons, night skies, underwater scenes, storms, auroras, blossoms, ruins — and are structured as elevation-specific sub-prompts (top, upper, middle, lower, bottom) with additional foreground prompts for select examples. There is no separate train/test split because the method is tuning-free; evaluation is conducted directly on the 20 prompts across all methods.
Base model(s). The primary experiments use SANA (Xie et al., 2024) as the text-to-image backbone for 360° static wallpaper generation and LTX-Video (HaCohen et al., 2024) as the text-to-video backbone for 360° live wallpaper generation. Both are pretrained, frozen, and used without any fine-tuning. Additional qualitative results are shown with FLUX (Labs, 2024) for static generation and HunyuanVideo (Kong et al., 2024) for video generation, demonstrating the method's compatibility with stronger backbones. The perspective resolution is 1024 × 1024 for SANA, FLUX, and HunyuanVideo; 512 × 512 for LTX-Video. Video temporal length is 121 frames for both LTX-Video and HunyuanVideo.
Metrics. The paper evaluates across four dimensions using 14 predefined view directions (four azimuth angles at each of three elevation angles: 45°, 0°, −45°, plus top and bottom poles). Panoramic criteria — Distortion (geometric deformation when ERP is converted to perspective views) and End Continuity (seamless alignment of left/right ERP borders, also called loop consistency). Image criteria — Image Quality (overall clarity and realism) and Aesthetic Appearance (visual appeal). Text adherence — Scene coherence (how well the generated content matches the prompt across views) and CLIP-Score (cosine similarity between image and text embeddings). Video criteria (for live wallpaper) — Motion Smoothness and Temporal Flickering. Since classical metrics like FID require a source domain that does not exist for generated panoramas (no reference distribution), the paper uses GPT-4o (Hurst et al., 2024) within an LLM-as-a-judge framework (Zheng et al., 2023) for image quality, distortion, end continuity, and aesthetic appearance; VBench (Huang et al., 2024) for motion smoothness and temporal flickering; and standard CLIP for text alignment. The reliability of the VLM-based evaluation is validated in Appendix F.2 by testing GPT-4o on synthetically distorted and disconnected images at five intensity levels; the predicted scores show a clear monotonic decrease with increasing artifact severity (Figure 16).
Baselines. For 360° static wallpaper, the paper compares against four methods: 360 LoRA (LatentLabs360, 2023) — a LoRA fine-tuned on ERP images; Text2Light (Chen, Wang, and Liu, 2022) — a zero-shot HDR panorama generation method fine-tuned on ERP datasets; PanFusion (Zhang et al., 2024) — a fine-tuned ERP panorama generator; and DynamicScaler (Liu et al., 2024) — originally designed for panoramic video, extended here for image generation as a tuning-free ERP-based MultiDiffusion method. For 360° live wallpaper, baselines are 360 LoRA + AnimateDiff (LatentLabs360, 2023; Guo et al., 2023) — combining ERP-fine-tuned static generation with a motion module; 360DVD (Wang et al., 2024) — a fine-tuned 360° video diffusion model trained on ~2,000 video clips; and DynamicScaler (Liu et al., 2024) — reimplemented with SANA and LTX-Video for fair comparison as a tuning-free method. SphereDiff and DynamicScaler share identical multi-prompt sets for generation; single-prompt baselines use only the middle reference prompt.
Generation budget / compute accounting. All baselines are compared at a per-sample level — one generation per prompt set. SphereDiff uses 2,600 spherical latent points and 89 view directions with an 80° field of view and 60% overlap. No explicit generation budget is swept; efficiency is instead compared through runtime analysis (Appendix E): approximately 3 minutes per static image with SANA on an A100-40GB, 20 minutes per video with LTX-Video, 12 minutes per static image with FLUX on an H200, and ~3 hours per video with HunyuanVideo (50GB VRAM). The paper compares against baselines at their default generation settings with no attempt to equalize compute, making runtime a reported metric rather than a controlled variable.
Cross-validation / statistical protocol. Automated evaluation is conducted on images rendered from 14 predefined view directions (four azimuth angles × three elevation angles + two poles) with a 90° field of view. Standard deviations are reported in Figure 13 across these 14 views per sample. The user study involved 21 participants comparing 20 pairs of samples across six criteria for images and videos; participants selected the sample that best fit each criterion from among the baselines (five options for static, four for video). The 20 prompt sets were divided into five groups of four; participants selected one group to evaluate based on convenience. Presentation order was randomized.

Main Quantitative Results

360° Static Wallpaper Generation

Headline result (User Study, Table 1). SphereDiff achieves the highest user preference across all four metrics for static wallpaper generation: Distortion 38.10% (vs. 21.43% for 360 LoRA and 20.24% for DynamicScaler, the nearest competitors), End Continuity 35.71% (vs. 25.00% for DynamicScaler and 23.81% for 360 LoRA), Image Quality 34.52% (vs. 25.00% for DynamicScaler), and Text Alignment 29.76% (vs. 27.38% for DynamicScaler). The win-rate gaps are largest on the two panoramic criteria — Distortion and End Continuity — which directly measure the pole artifacts and seamlessness that the spherical latent representation is designed to eliminate.

Automated evaluation (Table 2). The automated metrics corroborate the user study on panoramic criteria: SphereDiff scores 3.238 on Distortion (vs. 2.854 for DynamicScaler, the next best) and 4.892 on End Continuity (vs. 3.985 for DynamicScaler). On Image Quality, SphereDiff ties DynamicScaler at 4.496 (both method columns show "4.496"), while on Aesthetic Appearance SphereDiff achieves 4.685 (vs. 4.577 for DynamicScaler). The most dramatic automated gain is in Scene coherence for text adherence: 0.5875 for SphereDiff vs. 0.2750 for DynamicScaler and 0.2875 for 360 LoRA — a 2× improvement over the nearest competitor. CLIP-Score similarly favors SphereDiff at 28.65 (vs. 26.63 for DynamicScaler and 26.40 for 360 LoRA).

Standard deviation analysis (Figure 13). The automated Distortion score for SphereDiff is 3.238 with an error bar that lies entirely above the 1-sigma range of DynamicScaler's 2.854, indicating a statistically significant improvement. Similarly, End Continuity at 4.892 for SphereDiff exceeds DynamicScaler's 3.985 by more than 2× the standard deviation. On Image Quality, SphereDiff and DynamicScaler are statistically indistinguishable (overlapping error bars at ~4.5), consistent with the tied automated scores in Table 2.

Qualitative confirmation (Figure 10). Full-ERP comparisons in Appendix D.1 show that 360 LoRA, PanFusion, and DynamicScaler produce visible distortion, blurriness, and speckling near the poles, while Text2Light — despite generating seamless pole regions — fails to adhere to text prompts (e.g., generating generic indoor-like structures for outdoor prompts). SphereDiff produces coherent 360° panoramas without pole artifacts and with faithful text adherence across all views.

360° Live Wallpaper Generation

Headline result (User Study, Table 1). SphereDiff outperforms all baselines on five of six metrics for live wallpaper: Distortion 32.14% (vs. 30.95% for DynamicScaler), End Continuity 33.33% (vs. 27.38% for 360 LoRA + AnimateDiff and 26.19% for DynamicScaler), Text Alignment 29.76% (vs. 27.38% for 360 LoRA + AnimateDiff), Motion Smoothness 35.71% (vs. 28.57% for DynamicScaler), and Temporal Flickering 32.14% (vs. 29.76% for DynamicScaler). DynamicScaler marginally leads on Image Quality: 28.57% vs. 27.38% for SphereDiff — a reversal from the static case.

Automated evaluation (Table 2). On Distortion, SphereDiff achieves 2.579 vs. 2.086 for 360DVD and 1.971 for DynamicScaler. End Continuity shows the widest gap: SphereDiff at 4.496 vs. 3.482 for 360 LoRA + AnimateDiff and 2.971 for DynamicScaler — nearly 1.5× the second-best score. On Image Quality, 360 LoRA + AnimateDiff leads at 3.179, with SphereDiff second at 3.050. Motion Smoothness favors SphereDiff at 0.9956 vs. 0.9943 for DynamicScaler, and Temporal Flickering follows the same pattern at 0.9941 vs. 0.9918. The automated video metrics show very tight clustering — all methods score above 0.98 on both motion metrics — indicating that differences in temporal quality, while favoring SphereDiff, are small in absolute terms.

Qualitative confirmation (Figure 11). DynamicScaler exhibits stretched fireworks and blurry polar regions in the first column of Figure 11. 360 LoRA + AnimateDiff produces reasonable-looking panoramas but fails to animate content meaningfully — "fireworks and storms lack dynamic motion; instead, they merely shift from left to right as the viewpoint changes" (Appendix D.1). 360DVD fails to generate seamless results near the poles and shows poor text adherence, attributed to the limited diversity of its training dataset. SphereDiff generates coherent, temporally animated panoramas with pole regions matching equatorial quality.

The overall performance drop for video vs. static. Both automated scores in Table 2 show lower absolute numbers for live wallpaper across all methods and metrics compared to static wallpaper. The paper attributes this to "the performance of the underlying diffusion model" — LTX-Video is a less capable backbone than SANA — and notes that "its performance could be further improved by leveraging a more advanced denoising model." The HunyuanVideo results in Appendix D, Figure 12, qualitatively support this claim, showing visibly improved video quality with the stronger backbone (though no automated scores are reported for HunyuanVideo, as the paper considers it an unfair comparison since baselines cannot use it).

Scaling to Stronger Backbones (Qualitative Only)

FLUX for static generation (Figure 12, top rows). SphereDiff applied to FLUX produces 360° static wallpapers with "superior fidelity while adhering to panoramic constraints." Examples include fantastical scenes (nebula, asteroid, magic forest) that leverage FLUX's text-to-image creative capacity to depict content outside the scope of existing ERP datasets — an explicit demonstration of the tuning-free advantage: any improvement in perspective image models immediately translates to improved panorama generation.

HunyuanVideo for video generation (Figure 12, bottom rows). SphereDiff with HunyuanVideo produces "visually compelling videos that preserve panoramic consistency." No automated scores are reported; the paper positions these as qualitative existence proofs that the method scales to state-of-the-art models without modification.

Ablation Studies and Robustness Checks

Dynamic sampling vs. nearest-neighbor sampling (Table 3, automated metrics on static wallpapers with SANA). The full model (dynamic sampling + weighted averaging) achieves Distortion 3.238, End Continuity 4.892, Image Quality 4.496, Aesthetic Appearance 4.685, Scene 0.5875, and CLIP-Score 28.65. Removing dynamic sampling (nearest sampling + weighted averaging) yields Distortion 2.829, End Continuity 4.625, Image Quality 4.139, and Scene 0.5125 — all lower. Notably, End Continuity drops from 4.892 to 4.625, confirming that undersampling during nearest-neighbor discretization breaks the information flow across overlapping views even when blending weights are present. The Scene coherence metric shows the largest relative drop (0.5125 vs. 0.5875), suggesting that duplicate latent selection (nearest-neighbor sampling repeats features) degrades semantic consistency across views.

Distortion-aware weighted averaging (Table 3, compare rows with and without weighted averaging). Under dynamic sampling, removing weighted averaging drops Distortion from 3.238 to 2.421 (a 34% relative decrease), End Continuity from 4.892 to 4.086, Image Quality from 4.496 to 3.454, and Scene from 0.5875 to 0.1375 — a ~77% relative collapse in text adherence. The Scene metric is the most sensitive to weighted averaging, suggesting that without distortion-aware blending, cross-view misalignment produces semantic inconsistencies that the VLM interprets as poor prompt adherence. Under nearest sampling, removing weighted averaging further drops Distortion from 2.829 to 2.039, Image Quality from 4.139 to 3.014, and Scene from 0.5125 to 0.3250 — confirming that weighted averaging provides benefits independent of the sampling strategy.

Runtime analysis (Table 3, runtime column). Dynamic sampling adds ~23 seconds compared to nearest sampling (184s vs. 161s without weighted averaging; 185s vs. 162s with). The overhead is approximately 14% of total inference time, which is modest given the quality improvements.

Visual ablation (Figure 6). Two-view denoising with nearest sampling (without weighted averaging) produces "noticeable artifacts and unnatural transitions in overlapping regions" — visible as a seam where the two views meet. Adding dynamic sampling "improves information exchange across views, producing more seamless and coherent images." Further adding weighted averaging "yields significantly clearer and more consistent outputs for both sampling methods." The visual evidence aligns with the quantitative trends in Table 3.

Alternative latent mapping approaches (Appendix C.2). The paper reports two negative results:

Continuous positional embeddings: Attempting to skip discretization entirely by using DiT's support for continuous 1D positional encodings "leads to unstructured outputs due to a distribution shift in positional embeddings." The analysis (Figure 9) shows that in standard 2D positional embedding space, latents in the same row or column share high attention similarity, maintaining spatial structure. With continuous coordinates, "even slight variations in continuous position result in a significant drop in similarity," breaking structured generation. The paper concludes that "discretization remains essential for tuning-free panoramic visual generation."
Latent interpolation (bilinear, stochastic warping): Conventional interpolation methods applied in latent space "do not provide satisfactory results due to the lack of interpolation-equivariant properties in VAEs." Stochastic warping methods (Chang et al., 2024; Daras et al., 2024) "also provide suboptimal results in generating 360° live wallpapers." These results are not quantified — no interpolation-based ablation appears in Table 3 — but the rejection of these alternatives is the motivation for the sampling-based approach.

Critical Assessment

The strength of evidence for the spherical representation claim. The paper's central claim is that replacing ERP latents with uniformly distributed spherical latents eliminates pole artifacts. The evidence for this is strong on panoramic metrics: Distortion (38.10% user preference vs. 21.43% for the best ERP baseline; automated 3.238 vs. 2.854) and End Continuity (35.71% vs. 25.00%; automated 4.892 vs. 3.985) consistently show SphereDiff outperforming every baseline by substantial margins. These metrics directly measure the failure mode (pole stretching, seam breaks) that the spherical representation is designed to fix. The qualitative evidence in Figures 5, 10, and 11 — where pole artifacts are visually absent in SphereDiff outputs and present in all baselines — corroborates the quantitative results. This is a well-supported claim.

However, the evidence is bounded in two important ways. First, all evaluations use the same 20 prompt sets designed by the authors. There is no evidence that the gains generalize to arbitrary user prompts, indoor scenes, or complex multi-object compositions. The paper explicitly acknowledges this limitation: "it cannot yet generate highly complex scenes, such as indoor environments, without additional training" (Section 5). Second, the automated evaluation relies on a VLM (GPT-4o) as judge for distortion and end continuity, and while Appendix F.2 validates that GPT-4o's scores correlate with synthetic artifact severity (Figure 16), this validation uses Mip-NeRF 360 images (Barron et al., 2022) — a dataset of real 360° captures, not generated panoramas. Whether GPT-4o's sensitivity to synthetic distortion in real images transfers to generated images with qualitatively different artifacts (e.g., semantic incoherence vs. geometric stretching) is unverified. The fact that human and VLM rankings align on aggregate (Tables 1 and 2) is reassuring but not a rigorous calibration.

The image quality parity claim. SphereDiff ties or slightly trails DynamicScaler on Image Quality in both the user study (static: 34.52% vs. 25.00%; live: 27.38% vs. 28.57% favoring DynamicScaler) and automated metrics (static: 4.496 tie; live: 3.050 vs. 3.179 favoring 360 LoRA + AnimateDiff). This is consistent with the paper's positioning: the spherical representation fixes panoramic artifacts (distortion, continuity) but does not claim to improve per-view image quality over ERP methods. The image quality metric reflects standard photographic fidelity (sharpness, noise, exposure), which depends primarily on the pretrained backbone's capability, not the latent representation geometry. The takeaway is that SphereDiff matches ERP methods on conventional image quality while substantially exceeding them on panoramic coherence — a favorable trade-off for 360° content.

The missing baselines: CubeDiff and other geometric representations. The paper discusses CubeDiff (Kalischek et al., 2025) in the related work but does not include it as an experimental baseline. Given that CubeDiff also addresses pole distortion through an alternative geometric representation (cube maps rather than spherical latents), a direct comparison would have strengthened the paper's claim that spherical latents are superior to cube maps. The paper's stated reason — that cube map methods "still struggle with discontinuities at cube-face boundaries" and cannot be extended to video — is an analytical argument, not an empirical one. An experiment showing that spherical latents produce fewer boundary artifacts than cube maps would have been more convincing.

The compute comparison is not controlled. All baselines are compared at their default settings with no attempt to equalize generation budget. SphereDiff uses 89 view directions — meaning 89 denoising passes per timestep — while baselines like 360 LoRA generate in a single pass. DynamicScaler also uses MultiDiffusion with multiple windows, but the paper does not report the number of windows or the total FLOPs for that method. The runtime analysis (3 minutes for SANA, 20 minutes for LTX-Video) is reported only for SphereDiff, not for baselines. The claim that SphereDiff achieves better quality without fine-tuning is supported, but the claim that it is a practical alternative for deployment depends on whether users are willing to pay a 10× or greater inference-time cost for improved panoramic coherence. The paper does not address this trade-off.

The difficulty of scaling to stronger backbones is shown only qualitatively. Figures 1 and 12 demonstrate that SphereDiff works with FLUX and HunyuanVideo, but no quantitative evaluation is provided for these configurations. The automated metrics in Table 2 are for SANA and LTX-Video only. The paper states this is for fairness — baselines cannot use these backbones — but this means there is no rigorous evidence that SphereDiff's advantages persist at the higher quality levels these models provide. The qualitative examples look impressive, but without metrics, it is impossible to assess whether the panoramic quality gains (which are large relative to the ERP baseline errors) shrink or grow as the base model improves.

The ablation on the number of spherical points and view directions. The paper uses 2,600 spherical latents and 89 view directions across all experiments. There is no ablation varying $N$ (number of spherical points) or the number of views. These are the two key hyperparameters controlling the trade-off between quality and runtime: more points → finer spherical resolution → potentially better detail near the poles; more views → denser MultiDiffusion blending → potentially better seamlessness at the cost of linear runtime increase. Without ablating these, the reader cannot assess whether the chosen values are near-optimal or whether quality would saturate at lower (faster) settings. The paper acknowledges reducing the number of views as "a promising direction for future research on efficient panoramic generation" (Appendix E) but provides no data.

The limited test prompt diversity. All 20 prompts are outdoor natural scenes. No indoor scenes, no urban environments, no abstract or artistic prompts, and no prompts requiring fine spatial reasoning (e.g., "a checkerboard floor with a chandelier directly above"). The paper states indoor scenes are a limitation in Section 5, but this means the evaluation covers only a narrow slice of the prompt space that users might want for 360° wallpapers. The multi-prompt strategy (sky above, ground below) is naturally suited to outdoor scenes where the semantic layout follows elevation — it is unclear whether the same prompting strategy would work for indoor scenes where walls, ceilings, and floors have different elevation distributions.

The user study sample size and design. Twenty-one participants evaluating 20 pairs is a small study. The paper does not report inter-rater agreement (e.g., Krippendorff's alpha), so the reliability of the aggregate win rates is unknown. Participants selected one set of four prompt pairs "based on their convenience," meaning not all participants evaluated all prompts — this convenience sampling could introduce selection bias if, for example, easier-to-judge prompts are over-represented.

What additional experiments would strengthen the paper. (1) An ablation over $N$ (spherical points) and number of views, showing the quality-runtime Pareto frontier. (2) A quantitative comparison with CubeDiff on static image generation, using the same prompt sets, to empirically validate the claim that spherical latents outperform cube maps. (3) A FLOPs-matched or runtime-matched comparison where baselines are allowed to use more compute to match SphereDiff's 89-view budget. (4) Evaluation on a broader prompt distribution including indoor scenes, even if performance is expected to degrade, to characterize the failure boundary explicitly rather than leaving it as a stated limitation. (5) Automated evaluation of the FLUX and HunyuanVideo configurations alongside baselines that can use those backbones (e.g., adjusting DynamicScaler to use FLUX), to test whether the spherical representation's advantage is backbone-dependent.

6. Limitations and Trade-offs

Limitation 1: Difficulty Estimation for Compute-Optimal Allocation Requires Prohibitively Expensive Pre-Scanning

The assumption or constraint. The entire compute-optimal scaling framework — the paper's central efficiency contribution — depends on knowing each prompt's difficulty before allocating the test-time compute budget. The method for estimating difficulty is to generate 2,048 complete solutions per question and compute either the pass@1 rate (oracle, requiring ground-truth labels) or the average PRM final-answer score (predicted, requiring only the trained verifier). The paper explicitly flags this cost as unaccounted for:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity" (Section 3.2).

The consequence. In any realistic deployment, the total compute cost is difficulty estimation + strategy execution. At 2,048 samples per question for estimation, this cost can exceed the largest test-time compute budgets studied (256–512 generations). For a budget of $N = 64$ generations, the estimation step alone costs 32× the actual problem-solving budget. The headline $4\times$ efficiency gain (Figures 4 and 8, e.g., "16 generations matching 64" or "64 generations matching 256") is computed after difficulty is known, without amortizing the estimation cost. If amortized — say, by distributing the 2,048 estimation samples across a batch of questions — the effective efficiency gain shrinks substantially. For a single question, the total cost dominates and the $4\times$ figure becomes irrelevant.

What evidence exists in the paper. The difficulty estimation cost is stated but never quantified in relation to the test-time budgets being optimized. Figures 4 and 8 plot accuracy against generation budget $N$ , but the x-axis does not include the 2,048 estimation samples. There is no experiment showing how performance degrades when using fewer estimation samples (e.g., 64, 256, 512) or how the efficiency gain changes when amortization is accounted for. The paper also does not report the wall-clock time or FLOPs of the difficulty estimation step relative to strategy execution.

Mitigation status. The paper acknowledges this as a key limitation and suggests future work on "pretraining or finetuning models to directly predict difficulty of a question" or using "adaptive difficulty estimation" (Section 8) — e.g., starting with a small number of samples and dynamically allocating the remaining budget. However, no such system is developed or evaluated. The discrepancy between predicted and oracle difficulty bins (they "largely overlap" in Figure 4 but diverge somewhat in Figure 8 at high budgets, e.g., ~41% predicted vs. ~44% oracle at 256 generations) suggests that cheaper estimation may introduce additional variance, but this is not systematically characterized. This limitation means the $4\times$ figure should be treated as an upper bound on achievable efficiency under perfect difficulty knowledge, not a realized deployment gain.

Limitation 2: The Method Provides No Benefit on Hard Problems Where the Base Model Has Near-Zero Pass@1

The assumption or constraint. The paper's entire framework — both search against the PRM and iterative revisions — operates by amplifying the base model's existing capability. It assumes there exist correct solutions somewhere in the model's output distribution that can be found (via search) or refined toward (via revisions). When the base model's pass@1 on a problem class is essentially zero, no amount of test-time compute helps. The paper explicitly acknowledges this:

"Test-time compute can amplify existing capability but cannot create it. If the base model's pass@1 is near zero on a problem class, no amount of search or revision will help — there are no correct solutions in the proposal distribution to find or refine."

The consequence. On difficulty bin 5 (the hardest quintile of MATH problems), all methods, at all budgets, remain near 0–5% accuracy (Figures 3 right, 7 right). Beam search, best-of-N, sequential revisions, and compute-optimal combinations all plateau at the base model's random-guessing baseline. The FLOPs-matched comparison (Figure 9, bin 5 blue lines) shows test-time compute with the smaller model trailing the $\sim 14\times$ larger model by large margins across all $R$ regimes (e.g., −52.9% relative disadvantage for PRM search at $R \gg 1$ ). This means that for problems genuinely outside the base model's capability, pretraining remains the only viable path. An organization deciding how to allocate compute between pretraining and inference must know whether their target problem distribution falls into bin 5 — if so, the paper's framework provides zero guidance and may even mislead by suggesting that test-time compute is a universal substitute for model scale.

What evidence exists in the paper. The bin 5 results are consistent and unambiguous: Figure 3 right shows bin 5 accuracy hovering at 1–3% for all search methods across all budgets (4, 16, 64, 256 generations); Figure 7 right shows bin 5 accuracy ~2–3% regardless of sequential-to-parallel ratio; the FLOPs-matched analysis in Section 7 and the bar charts in Figure 1 show consistent disadvantages for test-time compute on hard problems. The paper does not attempt to characterize which problems fall into bin 5 beyond the quantitative pass@1 threshold, leaving the practical boundary ambiguous — is it determined by problem complexity, domain novelty, required factual knowledge, or something else?

Mitigation status. The paper is fully transparent about this limitation and treats it as a boundary condition for the approach's applicability — not something to be solved. Section 7 explicitly states the takeaway: test-time compute works when problems are "within the base model's rough capability range" and pretraining is preferable for genuinely hard problems. However, the paper does not provide a practical method for determining whether a given problem falls into the "within capability range" category without the expensive difficulty estimation step (Limitation 1). This creates a circular dependency: you need to estimate difficulty to know whether to apply test-time compute, but difficulty estimation itself requires substantial test-time compute.

Limitation 3: Single Benchmark (MATH) and Single Model Family (PaLM 2-S*) — No Evidence of Generalization Across Domains or Architectures

The assumption or constraint. All experiments in the paper use the MATH benchmark (500 test questions spanning high-school competition mathematics) and PaLM 2-S* as the base model. The paper states:

"We believe this model is representative of the capabilities of many contemporary LLMs" (Section 4).

This belief is untested. The paper provides no evidence that the key findings — difficulty-dependent optimal strategies, verifier over-optimization patterns, the $\sim 4\times$ efficiency gain from adaptive allocation, the FLOPs-matched advantage over a $\sim 14\times$ larger model on easy-to-medium problems — generalize to other model families (GPT, LLaMA, Claude), other benchmarks (GSM8K, HumanEval, Big-Bench-Hard), or other task modalities (code generation, logical reasoning, factual QA).

The consequence. Several aspects of the paper's findings could be specific to the interaction between PaLM 2-S*'s output distribution and the MATH benchmark:

PRM quality and over-optimization behavior. The PRM is trained on PaLM 2-S*'s outputs via Monte Carlo rollouts. A model with different calibration properties — e.g., one that produces more confidently wrong solutions or has a different error distribution across difficulty levels — might yield a PRM with different over-optimization thresholds, altering the optimal difficulty-dependent strategy.
Revision model effectiveness. The revision model's ability to learn from incorrect in-context examples depends on the base model's in-context learning properties. The edit-distance-based pairing strategy may work differently for models with different output styles (e.g., chain-of-thought vs. direct answers).
MATH-specific properties. MATH problems require multi-step symbolic reasoning with unambiguous ground-truth answers. It is unclear whether the difficulty-dependent patterns hold for tasks requiring factual recall (where capability may be bimodal — the model either knows or doesn't), creative generation, or multi-modal reasoning.
The $\sim 14\times$ larger model baseline. The FLOPs-matched comparison uses a second PaLM model with scaled parameters. If the scaling behavior of PaLM differs from other model families (e.g., some architectures may benefit more from scale, others less), the training-inference tradeoff conclusions may not transfer.

What evidence exists in the paper. The paper provides no cross-model, cross-benchmark, or cross-task experiments. The entire quantitative evaluation (Tables 3–9, all figures) is within the single domain of MATH + PaLM 2-S*. Section 4's claim of representativeness is an assertion, not an empirical finding. There are no experiments testing whether the compute-optimal strategies learned for MATH/PaLM transfer to, say, GSM8K (easier math) or HumanEval (code).

Mitigation status. No mitigation is attempted beyond the acknowledgment that this is a limitation. The paper does not suggest specific model families or benchmarks for replication. For a practitioner using a different model (e.g., LLaMA-3) on a different task (e.g., code generation), the paper provides no guidance on whether to expect similar $\sim 4\times$ gains, similar difficulty-dependent strategy rankings, or similar verifier over-optimization phenomena. The findings should be treated as existence proofs — establishing that difficulty-adaptive test-time scaling can produce large efficiency gains — rather than as universal prescriptions.

Limitation 4: Revisions and PRM Search Are Never Combined — the Reported Gains Are from Independent, Not Joint, Optimization

The assumption or constraint. The paper studies two complementary mechanisms — PRM-guided search (Section 5) and iterative revisions (Section 6) — but evaluates them independently. The compute-optimal scaling results (Figures 4 and 8) select the best strategy within each mechanism's hyperparameter space but never combine them. Section 8 explicitly acknowledges this gap:

"we did not experiment with PRM tree-search techniques in combination with revisions"

The consequence. The paper's framework motivates these as complementary axes: revisions improve the proposal distribution (the quality of candidate solutions generated), while PRM search improves selection (identifying the best candidate among those generated). The natural synthesis — use the revision model as the proposal distribution within beam search, or use the PRM to guide which revisions to keep and which to discard — is never evaluated. This means:

The reported $\sim 4\times$ efficiency gains (e.g., Figures 4 and 8) represent a lower bound on what a fully integrated system could achieve. Combining revisions + PRM search might push the efficiency gain higher.
Conversely, the combination might introduce negative interactions — e.g., the PRM trained on base model outputs may not transfer well to revision model outputs (the paper already shows this: Appendix J, Figure 15a), and training a PRM specifically for revisions might be necessary, adding complexity.
The difficulty-dependent strategy selection in the combined system would be more complex (do you use revisions-only, search-only, or both, and with what parameters?), and the optimal policy might differ from the union of independently optimal policies.

What evidence exists in the paper. Indirect evidence that combining may help comes from the difficulty-dependent results: revisions are most effective on easy problems (Figure 7 right, bin 1–2), while beam search helps most on medium problems (Figure 3 right, bin 3–4). This suggests complementary coverage — revisions where local refinement suffices, search where global exploration is needed. However, no experiment provides direct evidence. The PRM's transferability issue (Figure 15a, where the base-LM PRM underperforms a revision-specific ORM on revision outputs) is a cautionary signal that naively combining the two pipelines may not work without additional verifier training.

Mitigation status. The paper acknowledges this as explicit future work (Section 8) but does not provide partial results, preliminary experiments, or a roadmap for the combination. For a practitioner looking to maximize test-time compute efficiency, this leaves an open question: should they invest in building a combined system, or are the independent gains already near the achievable ceiling? The paper provides no evidence either way.

Limitation 5: Verifier Over-Optimization Is a Hard Ceiling with No Proposed Solution — the Compute-Optimal Policy Only Mitigates It, and Only When Difficulty Is Known

The assumption or constraint. The paper identifies verifier over-optimization as the primary bottleneck preventing unbounded improvements from additional test-time compute. The evidence is clear: beam search degrades easy-problem performance at high budgets (Figure 3 right, bin 1 accuracy decreases from ~78% to ~77% as budget increases from 4 to 256); lookahead search — the strongest optimizer — paradoxically performs worst overall (Figure 3 left, lookahead methods underperform at nearly all budgets); and qualitative examples show degenerate outputs that score highly under the PRM (repetitive low-information steps, overly short solutions; Appendix M). The paper's solution is not to fix the verifier, but to route problems away from aggressive search when over-optimization is likely: use best-of-N on easy problems (where the verifier is reliable but easily exploited) and beam search only on medium problems.

The consequence. The compute-optimal policy's effectiveness depends critically on correct difficulty estimation (Limitation 1) and correct assignment of the difficulty-dependent strategy. If difficulty is misestimated — e.g., a medium problem misclassified as easy — the system might deploy best-of-N when beam search would have helped, or deploy beam search when best-of-N would avoid over-optimization. More fundamentally, on medium-difficulty problems where beam search is deployed, over-optimization still limits the scaling ceiling. Figure 3 (right, bin 3) shows beam search performance flattening at high budgets — it helps initially but hits diminishing returns well before the budget is exhausted. This means the approach is fundamentally bounded by verifier quality, and the current Monte Carlo rollout training procedure (Appendix D) produces a verifier that over-optimizes at modest search depths.

The paper does not explore how improving the PRM — through better training data, adversarial training, ensemble methods, or architectural improvements — would shift the difficulty thresholds, alter the optimal strategy, or extend the scaling ceiling. For a practitioner, this creates uncertainty: investment in better PRM training might yield substantial additional gains, or it might not — the paper provides no sensitivity analysis.

What evidence exists in the paper. The over-optimization evidence is among the strongest in the paper: Figure 3 (right) shows the degradation on bin 1; Figure 3 (left) shows lookahead search underperforming; Figure 29 and surrounding examples (Appendix M) show degenerate outputs. However, there is no ablation on PRM quality — no experiment varying PRM training data size, label quality, architecture capacity, or training procedure to see how the scaling curves change. The paper treats the PRM as fixed and optimizes around it, rather than studying the interaction between verifier quality and compute-optimal strategy.

Mitigation status. The paper does not attempt to improve the PRM beyond the initial Monte Carlo rollout training. There is no discussion of adversarial training, ensemble methods, uncertainty quantification, or constrained search (e.g., KL-regularized beam search to prevent drift from the base model's distribution). The paper flags verifier over-optimization as a key bottleneck in Section 8 but treats it as a finding rather than a target for algorithmic improvement. This is a significant gap because it leaves the most impactful bottleneck unaddressed — improving the PRM could shift the entire compute-optimal scaling landscape upward, potentially making test-time compute competitive with pretraining even on harder problems.

Limitation 6: The FLOPs-Matched Comparison Uses a Weakened Pretraining Baseline — Parameter-Only Scaling, No Test-Time Compute for the Larger Model

The assumption or constraint. The FLOPs-matched comparison in Section 7 compares PaLM 2-S* with compute-optimal test-time strategies against a model with approximately $\sim 14\times$ more parameters, trained on the same data. The larger model uses greedy decoding only — no majority voting, no best-of-N, no verifier-based selection, no revision chains. The paper acknowledges that the pretraining scaling follows the LLaMA paradigm (Touvron et al., 2023) — scaling parameters while holding data fixed — rather than compute-optimal pretraining (Hoffmann et al., 2022), which would scale both parameters and data:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

The consequence. Both choices — parameter-only scaling and greedy decoding — weaken the pretraining baseline relative to what a practitioner would actually do. A Chinchilla-optimally trained model with $\sim 14\times$ more total FLOPs (scaling both parameters and data) would likely outperform a parameter-only-scaled model, potentially narrowing or reversing the reported advantages of test-time compute. Similarly, giving the larger model even a modest test-time compute budget — say, best-of-8 with majority voting — would create a much stronger baseline. The reported comparisons (e.g., Figure 1 bar charts showing +27.8% relative improvement for test-time compute on easy questions at $R \ll 1$ ) are against a straw-man baseline that no well-resourced team would deploy. The more realistic question is: given a fixed total FLOPs, is $\text{small model} + \text{aggressive test-time compute}$ better than $\text{larger model} + \text{modest test-time compute}$ ? The paper does not answer this.

What evidence exists in the paper. The FLOPs-matched experiments (Section 7, Figure 9, Figure 1) systematically compare only against greedy decoding of the larger model. There is no sensitivity analysis varying the larger model's test-time budget. The paper does not discuss what fraction of the total FLOPs budget the larger model would need to allocate to test-time compute to make the comparison fair. For the smaller model, test-time compute budgets range up to 512 generations per question; for the larger model, the budget is always exactly 1 generation (greedy).

Mitigation status. The paper is transparent about the parameter-only scaling choice but does not justify the greedy decoding choice for the larger model. The computational cost of giving the larger model best-of-8 or best-of-64 would be substantial (since inference FLOPs scale linearly with model size), but this is precisely the trade-off the paper is supposed to be analyzing. The authors acknowledge that "a fairer comparison might give the larger model some test-time compute budget as well" in the broader context but do not conduct this experiment. For a practitioner making resource allocation decisions, this limitation means the headline result — "smaller model + test-time compute outperforms $\sim 14\times$ larger model" — should be interpreted as: on easy-to-medium problems, it is possible for test-time compute to compensate for model scale, but the exact crossover point against a properly optimized larger model with its own test-time compute budget is unknown and likely less favorable than the paper reports.

7. Implications and Future Directions

How This Work Changes the Landscape

SphereDiff is best understood as a representational reframing that eliminates an entire class of failure modes, rather than a paradigm shift in generative modeling. It does not propose a new diffusion architecture, training objective, or sampling schedule. Instead, it identifies that pole-stretching artifacts in 360° panorama generation — which prior work treated as a data-scarcity or denoising-quality problem — are actually a geometric representation bug: storing latents on an equirectangular grid imposes wildly non-uniform spherical density that no amount of fine-tuning or clever blending can fully compensate for. The fix is to change where the latents live, not how they are processed.

This reframing matters because it redirects research attention. Prior to this work, the dominant approaches for improving 360° generation were (1) collect more panoramic training data (the fine-tuning path: 360 LoRA, Text2Light, PanFusion, 360DVD) or (2) design more sophisticated blending and denoising strategies within the MultiDiffusion framework (the tuning-free path: DynamicScaler). SphereDiff demonstrates that both paths share a flawed premise — that ERP latents are an acceptable foundation — and that fixing the foundation (spherical latents) yields larger gains in the specific failure modes (distortion, end continuity) than either path achieved. The Distortion metric in the user study (Table 1) tells this story clearly: SphereDiff at 38.10% nearly doubles the win rate of the best ERP-based method (360 LoRA at 21.43%, DynamicScaler at 20.24%). The End Continuity gap is similarly stark (35.71% vs. 25.00%).

The implication for researchers is that geometric correctness of the latent representation is a first-order design constraint for any system that synthesizes non-perspective imagery from perspective-trained models. This is not a lesson the paper states explicitly, but it is the natural generalization: if you are building a system that generates fisheye images, spherical videos, light fields, or multi-view consistent 3D from a pretrained 2D diffusion model, you should first ask whether your latent representation samples the target manifold uniformly, not whether you have enough training data. The paper makes this diagnostic move portable by identifying the specific mechanism — uneven latent density → representational starvation at certain viewpoints → artifacts — and showing that the Fibonacci lattice (a standard tool from numerical analysis, Hardin et al., 2016) is sufficient to fix it.

The paper also resolves the apparent contradiction between fine-tuning and tuning-free approaches in the 360° generation literature. Fine-tuning methods have access to panoramic data but still produce pole artifacts; tuning-free methods avoid data dependency but also produce pole artifacts. SphereDiff explains why: both operate on ERP latents, and ERP latents are the common root cause. By eliminating that root cause, SphereDiff achieves better panoramic quality than fine-tuned methods without any fine-tuning at all, and better than prior tuning-free methods by changing the representation rather than the MultiDiffusion schedule. This reconciliation is practically important because it tells practitioners that investing in better training data (expensive, domain-limited) or better blending heuristics (diminishing returns) is less impactful than investing in geometrically correct representations.

However, the paper also sharpens a boundary that prior work left fuzzy: spherical representations are necessary but not sufficient for high-quality 360° generation. The ablation in Table 3 shows that switching to spherical latents alone (dynamic sampling without weighted averaging) produces Distortion 2.421 and End Continuity 4.086 — better than the nearest-neighbor ERP baseline (Distortion 2.039) but substantially worse than the full model (3.238 and 4.892, respectively). This means the MultiDiffusion framework itself needs adaptation — specifically, the discretization from continuous projections to 2D grids (dynamic sampling) and the blending across views (distortion-aware weighting) — to realize the benefits of the spherical representation. The paper thus establishes a two-layer design principle: (1) make the latent representation geometrically correct, then (2) ensure that every operation that interfaces with the perspective model's native grid format accounts for the residual geometric mismatch.

One research direction that becomes less attractive as a result of this work is ERP-centric fine-tuning for outdoor panorama generation. If a tuning-free spherical method can outperform fine-tuned ERP methods on panoramic metrics while matching them on image quality, the case for building and maintaining ERP training pipelines weakens substantially — especially given the data scarcity documented in Table 4 and Appendix D.2. The caveat is that SphereDiff currently works well primarily for outdoor scenes (Section 5 limitation), so ERP fine-tuning may retain an advantage for indoor or domain-specific panoramas where layout constraints are tighter. But for the outdoor wallpaper use case that the paper targets — the application driving the AR/VR content demand — SphereDiff effectively deprecates the ERP fine-tuning approach.

Follow-Up Research This Work Enables

Cheap, on-the-fly spherical difficulty estimation for adaptive view allocation. The paper uses a fixed 89 view directions across all generations, but the MultiDiffusion framework introduces a natural quality-runtime tradeoff: more views → better blending → higher quality, at linear computational cost. Not all prompts need the same number of views. A prompt like "clear blue sky" (the upper prompt in Figure 17) has low spatial complexity and might need only a sparse view set; a prompt like "ancient ruins under a vast, starry night sky" benefits from dense coverage. A natural extension is to start generation with a coarse set of views (say, 20), assess the MultiDiffusion blending consistency in overlapping regions during early denoising steps (measure variance of denoised latent features at shared spherical points), and dynamically add views where the variance is high. This would amortize the "difficulty estimation" cost into the generation process itself, analogous to how adaptive ray sampling works in neural rendering. A strong follow-up would benchmark the quality-runtime Pareto frontier for 10–20 diverse prompt sets, measuring whether adaptive allocation can match the 89-view quality at, say, 40% of the runtime.

Replacing MultiDiffusion sequential denoising with a single consistent attention across views. The current approach denoises each of the 89 views independently and blends them post-hoc. This means that at denoising time, view A knows nothing about what view B is generating, even though they overlap by 60%. The blending step reconciles them, but it does so at the latent feature level, not at the attention level. A more integrated approach — inspired by how video diffusion models use temporal attention to enforce consistency across frames — would introduce cross-view attention layers that allow spherical latents visible from multiple views to share information during the denoising step itself. This would require modifying the pretrained diffusion model (breaking the tuning-free property) but could substantially reduce the number of views needed, since consistency would be enforced by the attention mechanism rather than by dense view overlap. A concrete experiment: take a DiT-based model (e.g., SANA), add a lightweight cross-view attention module between the existing self-attention layers, and measure End Continuity as a function of view count. The hypothesis is that cross-view attention with 20 views could match or exceed the 89-view blending baseline.

Systematic characterization of the spherical-to-perspective distortion compensation function. The paper uses an exponential weighting function $W_i^{jk} = \exp(-\|\mathbf{u}_{jk}\|/\tau)$ and states that "more complex models do not yield a significant improvement in quality" (Appendix A.2), but this claim is based on empirical trial, not a systematic sweep. The actual distortion is $\tan \theta / \theta$ (Section A.2), which grows from 1.0 at the center to ~1.16 at 40° from the optical axis (the edge of the 80° FoV). A principled weighting function would be proportional to the inverse of this distortion: $W(\theta) = \theta / \tan \theta$ . A strong follow-up would compare the exponential heuristic against the theoretically-derived inverse distortion weighting, against a learned weighting (a small MLP that takes $\|\mathbf{u}\|$ as input and is trained on a reconstruction objective), and against no weighting, across a range of FoV values (60°, 80°, 100°). This would establish whether the exponential form's sufficiency is a coincidence of the 80° FoV choice (where distortion is mild) or a more general property. A negative result — showing that inverse distortion weighting outperforms exponential at larger FoVs — would be practically valuable for extending SphereDiff to wider-angle configurations.

Extending the spherical representation to handle indoor scenes through layout-aware prompt decomposition. The paper's limitation section acknowledges that SphereDiff "cannot yet generate highly complex scenes, such as indoor environments." The multi-prompt strategy (Section 3.5) decomposes the sphere by elevation — appropriate for outdoor scenes where the semantic layout follows gravity (sky up, ground down). Indoor scenes have a different spatial structure: walls at specific azimuthal orientations, ceiling above, floor below, and furniture at intermediate elevations. The prompt decomposition for indoor scenes would need to be azimuth-aware as well as elevation-aware — e.g., "a bookshelf on the north wall," "a window on the east wall," "a chandelier directly above." A concrete follow-up would extend the condition mapping function $\lambda_i$ to select prompts based on both elevation and azimuth, using a coarse cuboid layout specification (six principal directions: floor, ceiling, four walls), and evaluate on the Structured3D or Matterport3D datasets (which the paper cites in Appendix D.2 as standard indoor panorama datasets). The hypothesis is that the spherical representation would eliminate pole artifacts indoors just as it does outdoors, but the prompt engineering would need to handle the azimuthal discontinuity at wall boundaries — potentially through the same distortion-aware blending mechanism, but with prompt-specific blending that creates soft transitions at wall edges rather than hard seams.

Combining spherical latents with test-time optimization for text-ERP alignment. SphereDiff is tuning-free with respect to the diffusion backbone but uses fixed projection and blending parameters. An alternative that bridges tuning-free and fine-tuned approaches would be to optimize the spherical latent features directly after the MultiDiffusion generation, using a text-image alignment loss (CLIP score, or a learned quality assessor) computed on perspective renderings. This is a test-time optimization (not model fine-tuning): start from the SphereDiff-generated spherical latents, render perspective views at a dense set of directions, compute alignment between those views and the text prompt, and backpropagate through the frozen VAE decoder to update the latent features. This would refine the generated panorama to better match the prompt without any training data. The specific experiment: take the 20 prompt sets from Appendix G, generate with SphereDiff, then run 50–100 optimization steps of latent refinement, and measure the improvement in CLIP-Score and VLM-based Scene coherence. The risk is that optimization without the diffusion prior could introduce artifacts, but the diffusion output serves as a strong initialization.

Stress-testing the spherical representation under adversarial prompt distributions. The paper evaluates on 20 outdoor prompts designed to showcase the method (natural landscapes, celestial scenes, underwater). A necessary stress-test — the kind that would establish robustness boundaries — would evaluate on prompts explicitly designed to violate the elevation-based multi-prompt assumptions: "a room where the floor is sky and the ceiling is grass" (inverted semantics), "an M.C. Escher staircase wrapping around the full sphere" (impossible geometry), "a seamless gradient from fire at the bottom to ice at the top" (continuous elevation-dependent semantics that the five-bin prompt discretization might not capture). The failure modes would reveal whether the spherical representation itself has limitations (it shouldn't — it's just a point distribution) or whether the multi-prompt decomposition and blending strategy introduce assumptions about semantic continuity that break under adversarial conditions. The key measurement would be human preference relative to baselines on these adversarial prompts — does SphereDiff's advantage persist, shrink, or reverse?

Practical Applications and Downstream Use Cases

AR/VR background content for consumer headsets. The most direct application is generating immersive 360° environments for Meta Quest, Apple Vision Pro, or similar devices, where users can select a text description and receive a high-quality static or dynamic background environment on demand. The paper's demonstrated compatibility with SANA (~3 minutes per static panorama on an A100) and FLUX (~12 minutes on an H200) means that for a cloud-based service generating content asynchronously (a user requests a "nebula" theme and receives it minutes later), the inference time is acceptable. The key advantage over existing solutions is the elimination of pole artifacts — when a user looks up in their headset, they see the same quality as when they look at the horizon. Existing ERP-based methods (360 LoRA, DynamicScaler) break this immersion because the pole regions are blurry or distorted, which is precisely where users look when exploring a virtual sky or ceiling. The tuning-free property means the service can upgrade to any new diffusion backbone without retraining, future-proofing the content quality.

Automated generation of 360° video loops for digital signage and immersive installations. The live wallpaper capability (20 minutes per video with LTX-Video, ~3 hours with HunyuanVideo) enables creation of seamless looping 360° videos for large-scale displays in museums, corporate lobbies, or art installations. Traditional 360° video content requires specialized camera rigs and permits; SphereDiff enables creating novel, prompt-specified content (e.g., "a bioluminescent deep-sea trench with drifting jellyfish") that would be physically impossible to film. The distortion-free poles are critical here because installations often use dome or curved projections where the top of the dome is directly visible to viewers. The 121-frame temporal length provides a ~4-second loop at 30 fps, which is sufficient for ambient background content where perfect temporal seamlessness is less critical than spatial seamlessness. The ability to composite foreground objects (Section B.2, Figure 8b) — e.g., adding a branded element or logo at a specific spherical coordinate — makes this directly useful for commercial applications.

Training data generation for 360° perception models. A less obvious but potentially impactful application is generating synthetic training data for models that operate on 360° imagery: autonomous vehicle perception (some systems use 360° cameras), robot navigation in panoramic environments, or depth estimation from spherical images. These models are bottlenecked by the scarcity of labeled 360° data. SphereDiff can generate unlimited, diverse, text-conditioned 360° indoor and outdoor scenes with perfect ground-truth geometry (since the spherical coordinates are known exactly). While the paper's current limitation restricts it to outdoor scenes, the proposed indoor extension (see Follow-Up Research above) would directly enable generating structured indoor environments (e.g., "a kitchen with granite countertops and a window on the north wall") with known camera parameters and surface normals. The Distortion metric reaching 3.238 (out of 5) and End Continuity at 4.892 in the automated evaluation (Table 2) indicates that the generated panoramas have geometric properties close to real captures — far better than ERP-generated panoramas that would introduce spurious geometric artifacts into training.

When to Prefer This Method

SphereDiff is designed for scenarios where panoramic quality — specifically freedom from pole distortion and seamlessness across the full sphere — is the primary requirement, and where fine-tuning on a limited domain-specific dataset is either infeasible (no training data exists for the target content) or undesirable (the system must work with a rapidly improving pretrained backbone). Based on the paper's empirical results and stated limitations:

Prefer SphereDiff when:

You are generating outdoor 360° static or dynamic wallpapers from text prompts, and the background model's base quality (SANA, FLUX, LTX-Video, HunyuanVideo) meets your fidelity requirements. The paper demonstrates state-of-the-art performance on panoramic criteria (Distortion 38.10%, End Continuity 35.71% user preference, Table 1) across these backbones without any fine-tuning.
You need future-proof compatibility with upcoming diffusion models — since SphereDiff is tuning-free, upgrading from SANA to FLUX requires no retraining, only changing the frozen diffusion backbone. The paper demonstrates this with four different backbones (Figures 1, 12).
The poles are visually important in your application. If users will frequently look directly up or down (sky in outdoor scenes, overhead objects, floor detail), SphereDiff's uniform spherical latent distribution provides a ~1.8× relative improvement in Distortion over the best ERP baseline (automated score 3.238 vs. 1.971 for DynamicScaler on live wallpaper, Table 2), which translates directly to pole-region quality.
You are generating content that does not exist in any training dataset (fantastical scenes, impossible landscapes). ERP fine-tuning methods are constrained by their training distribution; Text2Light "struggles to adhere to the input text prompts" for out-of-distribution content (Section D.1). SphereDiff, being tuning-free, can generate any scene the pretrained backbone can — Figure 12 shows nebula, asteroid, and magic forest scenes that no ERP dataset contains.

Prefer fine-tuned ERP methods when:

You need indoor scene generation with precise spatial layout. SphereDiff's current limitation (Section 5) explicitly states it "cannot yet generate highly complex scenes, such as indoor environments." ERP-fine-tuned methods like Text2Light or CubeDiff, which are trained on Matterport3D or Structured3D, have learned indoor-specific layout priors that SphereDiff's elevation-only prompt decomposition does not capture.
Inference time is the dominant constraint and you can tolerate some pole distortion. SphereDiff with SANA takes ~3 minutes per static image (89 views × 1 denoising pass each); 360 LoRA generates in a single pass. For a real-time or near-real-time application where a user expects results in seconds, the ~100× speed difference may outweigh the panoramic quality gain, especially if the application rarely renders pole views.
You are operating in a heavily constrained domain with abundant ERP training data (e.g., real estate virtual tours, where thousands of Matterport scans are available) and need to generate content faithful to that domain's specific visual statistics. Fine-tuning on in-domain ERP data will learn domain-specific priors (furniture placement, lighting patterns) that a tuning-free method relying on general perspective models cannot capture.