360Anything: Geometry-Free Lifting of Images and Videos to 360°

🎯 Pitch

A diffusion transformer can learn perspective-to-panorama mapping from data alone, eliminating the need for camera calibration that cripples existing methods on in-the-wild inputs—and it even estimates camera FoV and pose as a byproduct. The model outperforms prior work that uses ground-truth geometry while solving ERP seam artifacts through a simple circular encoding trick.

1. Executive Summary

This paper introduces 360Anything, a geometry-free framework for lifting perspective images and videos to 360° panoramas that eliminates the dependency on camera calibration by treating the perspective input and panorama target simply as token sequences in a diffusion transformer. Unlike prior work that requires known camera Field-of-View and orientation to project the perspective input into equirectangular projection space—making them brittle on in-the-wild data—360Anything learns the perspective-to-equirectangular mapping through sequence concatenation in a DiT (appending encoded perspective tokens to noisy panorama tokens and running global self-attention), while introducing Circular Latent Encoding to resolve seam artifacts by circularly padding panorama images before VAE encoding rather than relying on inference-time rotation tricks. The method achieves state-of-the-art performance on the Laval Indoor and SUN360 image benchmarks—outperforming CubeDiff with nearly 50% reduction in FAED (9.8 vs. 18.4 on Laval Indoor)—and on the Argus video benchmark (FVD of 483.4 vs. 1020.7 for the prior state-of-the-art), while also demonstrating competitive zero-shot camera FoV estimation (4.93° average error) and camera pose estimation, establishing that explicit geometric alignment is unnecessary for panorama generation when a general-purpose architecture can learn these relationships purely from data.

2. Context and Motivation

The Core Problem: Camera Calibration Is a Brittle Dependency for Panorama Generation

The fundamental problem this paper tackles is straightforward to state but surprisingly difficult to solve in practice: given a regular perspective image or video, generate a full 360° panorama that seamlessly extends the view in all directions. This is the problem of perspective-to-360° outpainting, and it is the gateway to creating immersive 3D worlds from ordinary photographs and videos. If you can reliably lift a narrow field-of-view input to a complete spherical view, you unlock applications in virtual reality, robotics navigation, gaming environment creation, and 3D scene reconstruction—essentially any domain where a system needs to understand or render the full visual context around a captured viewpoint.

The challenge is not just one of image generation quality; it is fundamentally a geometric alignment problem. A perspective camera captures a small, oriented window onto the world, defined by its Field-of-View (how wide the lens sees), and its orientation (yaw, pitch, roll—where the camera is pointing). A 360° panorama, typically represented as an Equirectangular Projection or ERP image, is a 2D mapping of a full sphere: the horizontal axis covers 360° of azimuth and the vertical axis covers 180° of elevation. The question is: where on this spherical canvas does the perspective input belong, and how does its planar geometry map onto the curved geometry of the sphere?

The prevailing paradigm for answering this question has been explicit geometric projection (Section 2). Prior methods take the perspective input, use known camera parameters to mathematically project it onto the ERP canvas, and then treat the problem as one of outpainting or inpainting—filling in the regions of the panorama not covered by the projected input. This pipeline looks like:

Estimate (or assume) camera FoV and orientation.
Project the perspective image to its correct location on the ERP canvas.
Feed the resulting pixel-aligned conditioning signal into a diffusion model that generates the rest of the panorama.

This is elegant when it works because it reduces a hard geometric reasoning problem to a relatively well-understood pixel-to-pixel translation task. The model doesn't need to figure out where the input goes—it receives that information directly through spatial alignment. Methods like PanoDiffusion, Imagine360, and Argus all follow this template, and they achieve reasonable results when the camera parameters are accurate.

But this is precisely where the paradigm breaks down: in-the-wild data rarely comes with accurate camera metadata. Most photos and videos captured by everyday users lack any calibration information. To compensate, prior methods must resort to external camera estimators (Section 2)—off-the-shelf tools that predict FoV and camera pose from a single image or video. These estimators can be noisy (FoV estimates often off by 5–15°), fail catastrophically on unusual scenes or lighting conditions, and introduce a brittle dependency chain: estimator error → projection misalignment → degraded panorama generation. The paper explicitly identifies this accumulated error pathway as a core limitation (Section 1), noting that "when off-the-shelf camera estimators fail, channel-concatenation approaches break down completely due to the reliance on pixel-aligned conditioning" (Section 3.2).

Moreover, even when camera parameters are available, the projection-based paradigm imposes a rigidity that limits how the model can reason about the scene. If the perspective input is projected to a specific location on the ERP canvas, the model's task is strictly to fill in the blanks around it. Any error in projection is baked into the input representation and cannot be corrected downstream. The model has no way to say, "I think this camera was actually pointed slightly more to the left," because the spatial alignment is already locked in before the diffusion process begins.

Why This Problem Matters (Beyond Academic Benchmarking)

The brittleness of camera-dependent methods is not just a theoretical concern—it directly limits the practical deployment of panorama generation and the downstream tasks that depend on it (Section 1):

Accessibility for casual users: If generating a 360° view from a phone photo requires the user to know their camera's FoV and precise orientation, the technology remains inaccessible to the vast majority of potential users. A "drop in any image and get a panorama" experience requires the system to handle unknown, variable camera configurations automatically.
Robustness for video: Perspective videos add temporal variation in camera parameters. Even if an estimator works on individual frames, accumulating per-frame errors across a video leads to temporal inconsistency—the generated panorama may appear to drift, jitter, or oscillate in ways that destroy the immersive experience. The paper notes that "existing methods either require known camera information to perform the projection, or assume the conditioning image has a fixed viewpoint and FoV" (Section 2), neither of which holds for real-world videos with dynamic camera motion.
3D reconstruction from generated panoramas: One of the paper's showcased applications is using generated 360° videos as input to Structure-from-Motion and 3D Gaussian Splatting (Figure 1, Section 4.2). This requires multi-view consistency—the generated panorama frames must maintain coherent geometry across time. Camera-estimation errors that cause projection misalignment propagate through to the 3D reconstruction, producing distorted or broken 3D models.
Scale and deployment: For large-scale processing of internet video or user-generated content, requiring a camera calibration pre-processing step adds computational overhead and a failure point. The paper envisions a world where massive 360° video datasets can be processed without per-video camera estimation (Section 2).

Where Prior Approaches Fall Short (Specific Technical Gaps)

The paper identifies several distinct categories of limitations in prior work, each of which motivates a component of 360Anything's design.

Gap 1: Explicit Geometric Projection Creates a Camera Metadata Dependency

This is the central limitation discussed above. Methods like PanoDiffusion (Wu et al., 2024), Imagine360 (Tan et al., 2025), and Argus (Luo et al., 2025) all follow the project-then-outpaint paradigm. They use channel-concatenation—the projected perspective latent is concatenated along the channel dimension with the noisy target latent before feeding into the diffusion U-Net or transformer—which enforces a hard spatial correspondence between the conditioning signal and the output. The paper contrasts this directly with its own approach (Section 3.2):

"Instead of enforcing spatial correspondence via projection into the ERP space, we employ a simple sequence concatenation mechanism... The DiT thus runs global self-attention on the combined sequence of tokens. It learns to generate latents in the ERP image by reasoning their relationship to latents in the perspective image in a purely data-driven way."

The critical shift is from spatially-aligned conditioning (where the model is told where the input goes) to learned geometric reasoning (where the model figures out where the input goes through cross-attention between token sequences). This is not just a convenience—it fundamentally changes what the model learns. Rather than memorizing how to outpaint from a fixed-position input, the model must learn the geometric relationship between perspective and equirectangular spaces, which includes implicitly estimating camera parameters and using that estimate to guide generation.

Gap 2: Seam Artifacts Are a Persistent but Misdiagnosed Problem

A well-known issue in panorama generation is the appearance of visible discontinuities at the left-right boundary of the ERP image. Since the ERP wraps horizontally (0° and 360° represent the same viewpoint), any inconsistency at the boundary creates a jarring seam when the panorama is viewed in a 360° viewer. Prior work attributes this to the generation process itself and applies inference-time fixes (Section 3.3):

Rotated denoising (PanoDiffusion): shifting the panorama cyclically across diffusion sampling steps so the model encounters different boundary alignments.
Circular padding in the VAE decoder (360DVD): modifying the VAE decoder to use circular rather than zero padding at image boundaries during reconstruction.
Blended decoding (Argus): overlapping and blending the left and right edges during decoding to mask discontinuities.

The paper argues—and this is one of its key technical insights—that these are symptoms, not root causes. The root cause lies in the VAE training pipeline, specifically in how convolutional encoders handle image boundaries. The paper explains (Section 3.3):

"Modern diffusion models are often applied in the latent space of a convolution-based VAE. When encoding a panorama image in the ERP format, the convolution layers perform zero-padding at the image boundaries, which introduces boundary artifacts in the feature maps... even if the panorama image $Y_{\text{equi}}$ is free from seams in pixel space, its latent representation $y_{\text{equi}}$ contains a discontinuity."

This is illustrated in Figure 3a: if you take a seamless panorama, encode it with a standard VAE that zero-pads at boundaries, shift the latent by 180° (so the original boundary is now in the middle), and decode it, you see a visible seam artifact. The artifact was created during encoding, not during generation. This means the diffusion model is trained on noisy versions of already-corrupted latents—it never sees a truly seamless latent representation during training, and any boundary artifacts it learns are baked into its understanding of what a "correct" panorama looks like.

Inference-time fixes attempt to paper over this by smoothing boundaries or averaging across rotations, but they don't address the training-time mismatch. The paper's Circular Latent Encoding (Figure 3b) eliminates the discontinuity at the encoder level by circularly padding the input image before encoding and dropping the padded latent regions afterward, ensuring that the latents passed to the DiT have no boundary discontinuity.

Gap 3: Fixed-Viewpoint Assumptions Limit Generalization

Several prior methods implicitly or explicitly assume the perspective input occupies a fixed, known position in the output panorama (Section 2). For example:

CubeDiff (Kalischek et al., 2025): operates in cubemap space with six faces, each 90° FoV. It treats the conditioning image as the front face and generates the other five faces. This locks the input to a 90° FoV and center-front position. When the actual input has a different FoV, CubeDiff must stretch objects at the boundary to fit the 90° assumption, leading to distorted geometry (Figure 4, balloons and mushroom examples).
ViewPoint (Fang et al., 2025): similarly places the conditioning video at the center of the panorama. When the input video has significant camera tilt, ViewPoint generates "rotated" panoramas where objects appear distorted because the model is forced to represent non-upright geometry.
Prior methods that generate non-canonical panoramas: several approaches generate panoramas in whatever orientation the input happens to be, meaning the output panorama may have a tilted horizon or rotated vertical axis. This forces the model to learn different spherical distortion patterns for every possible camera orientation, significantly increasing the complexity of the generation task (Section 3.2).

The paper's canonical coordinate constraint (Section 3.2) explicitly addresses this: the model is trained to always produce gravity-aligned, upright panoramas regardless of the input's camera orientation. This means the model must infer the camera pose to correctly "place" the perspective tokens on the canonical canvas, rather than passively accepting whatever orientation the input happens to have. This is a harder inference problem but a simpler generation problem—the model only needs to learn one spherical distortion pattern (the upright canonical case) rather than a continuum of possible orientations.

Gap 4: Architecture Mismatch—U-Nets vs. Transformers

The paper positions itself at a moment of architectural transition in generative modeling. Prior panorama generation methods predominantly use U-Net backbones (PanoDiffusion, Imagine360, Argus, 360DVD all build on Stable Diffusion or similar U-Net architectures). While U-Nets excel at tasks requiring fine spatial reasoning (their hierarchical structure with skip connections preserves spatial detail), they impose certain inductive biases that may not be ideal for the perspective-to-panorama mapping:

U-Nets are designed for tasks where input and output are spatially aligned (e.g., image-to-image translation, inpainting). They naturally leverage pixel-wise correspondence, which works well when the input is projected to the correct ERP location but provides no mechanism for learning where that location should be.
U-Net convolutions have limited receptive fields (though dilated or strided convolutions increase this), making it harder to model long-range dependencies across the full panorama (e.g., ensuring the left and right edges of the ERP match, or reasoning about the global structure of a scene from a narrow partial view).

The paper identifies this architectural choice as a "major limiting factor for the field" (Section 2) and instead adopts Diffusion Transformers (DiTs): "360Anything instead runs Transformers on a sequence of tokens without any geometric prior, while achieving state-of-the-art results across multiple tasks." Transformers with global self-attention can, in principle, attend across the entire sequence of conditioning tokens and target tokens simultaneously, learning long-range geometric relationships without being constrained by local receptive fields or spatial alignment assumptions.

Gap 5: Insufficient Use of Data Scale and Diversity

Most prior perspective-to-panorama methods are trained on relatively limited datasets with constrained camera configurations. The paper argues that with sufficient data diversity—varying FoV, pitch, roll, and camera trajectories—a general-purpose architecture can learn the perspective-to-equirectangular mapping robustly. This is a scaling argument: rather than encoding geometric knowledge through architectural inductive bias (spherical convolutions, dual-branch designs, explicit projection), encode it through data scale and model capacity. The use of large-scale synthetic data (Structured3D, which constitutes ~90% of the image training data) combined with aggressive camera augmentation (uniformly sampling FoV in [30°, 120°], pitch in [−60°, 60°], roll in [−15°, 15°]) provides the diversity needed for the model to learn a general mapping.

For video, the paper goes further by incorporating real-world camera trajectories (Section 4.2) rather than relying solely on simulated linear motion, finding that "simulated camera movement lacks diversity" and that adding trajectories extracted from real videos "improves generalization to in-the-wild videos" (Figure 13). This is a practical lesson: the camera motions present in internet videos are far more varied than simple linear/random trajectories, and training exclusively on simulated motion leaves the model unprepared for the complexity of real user-captured footage.

How This Paper Positions Itself

The paper makes a dual contribution—one conceptual, one technical—and carefully positions itself at the intersection of these contributions.

Conceptual positioning: "Geometry-free" as a paradigm shift. The central thesis is that explicit geometric reasoning (camera calibration, projection, spatial alignment) is not necessary for high-quality panorama generation and may actually be harmful because it introduces a failure point (noisy calibration) and constrains the model's flexibility. The paper frames this as a shift from geometric inductive bias to data-driven geometric reasoning: "We posit that explicit geometric alignment is unnecessary for panorama generation. Instead, with sufficient data, a general-purpose architecture should be able to learn these relationships from data" (Section 1). This echoes a broader trend in computer vision (referenced in Section 2's "Prior-Free Learning with Transformers") where tasks previously requiring explicit 3D priors—view synthesis, depth estimation, camera calibration—are increasingly being solved by Transformers trained on large-scale data with minimal inductive bias.

However, the paper does not claim that geometric knowledge is absent from the model—quite the opposite. The zero-shot camera calibration experiments (Section 4.3, Tables 3 and 4) demonstrate that 360Anything implicitly learns accurate camera parameters: it can estimate FoV with ~4.93° average error (competitive with supervised methods like DUSt3R and MoGe) and camera pose with errors within ~0.5° of state-of-the-art estimator GeoCalib. The model learns geometry; it just doesn't need it handed to it as an input. This is a crucial nuance: "geometry-free" refers to the framework's input requirements, not to the model's internal representations.

Technical positioning: sequence concatenation + circular latent encoding as a minimal viable framework. Rather than introducing complex new architectural components, the paper keeps the architecture simple: sequence concatenation replaces channel concatenation, and Circular Latent Encoding replaces inference-time seam fixes. The innovation is in understanding why these simple changes work and demonstrating that they suffice to achieve state-of-the-art results. This minimalism is deliberate—it strengthens the argument that elaborate geometric priors are unnecessary. If the same or better performance can be achieved with a simpler, more general architecture, the burden of proof shifts to proponents of geometric inductive bias to justify the additional complexity.

Empirical positioning: outperforming methods that use ground-truth camera information. The paper makes a striking claim throughout: 360Anything often outperforms prior methods even when those methods have access to ground-truth camera metadata. In the video results (Table 2), the baselines Imagine360, Argus, and ViewPoint all use ground-truth camera information for projection, yet 360Anything achieves better PSNR, LPIPS, FVD, and VBench scores. This is the strongest empirical evidence for the paper's thesis: the information bottleneck is not camera metadata availability, but rather the architectural and training choices that determine how effectively the model can use whatever information is available.

Positioning relative to the training-inference pipeline. The paper also implicitly positions itself as a simplification of the overall deployment pipeline. Prior methods require a multi-stage process: (1) estimate camera parameters, (2) project the input, (3) optionally canonicalize the ground-truth data for training, (4) run the diffusion model, (5) apply seam-removal post-processing. 360Anything collapses this to: (1) optionally canonicalize training data (handled offline), (2) run the diffusion model end-to-end with no pre-processing or post-processing beyond the VAE encode/decode. This simplification has practical implications for deployment, especially in large-scale batch processing or latency-sensitive applications.

Relationship to the "prior-free" Transformer literature. The paper explicitly connects to a line of work where Transformers have displaced methods that previously relied on strong task-specific inductive biases (Section 2: "Recently, Transformers have dominated tasks that previously relied on inductive bias, including image generation, editing, and 3D understanding"). By framing panorama generation as an instance of this broader trend, the paper suggests that its approach is not just a one-off solution but part of a systematic pattern in computer vision: as models and datasets scale, learned representations increasingly outperform hand-designed geometric priors. The specific references to prior-free view synthesis (Rombach et al., 2021) and general-purpose image generation (Peebles and Xie, 2023; Wang et al., 2025) place 360Anything within this lineage.

3. Technical Approach

3.1 Reader Orientation

360Anything is a neural network system that takes any regular photograph or video and produces a seamless 360° panorama—the kind of image you can load into a VR headset and look around in all directions. It solves the perspective-to-panorama lifting problem by treating the input image and the target panorama simply as two sequences of tokens and using a transformer to learn their geometric relationship, completely eliminating the need for camera calibration, lens parameters, or orientation metadata that prior methods required as essential inputs.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major stages:

VAE Encoder — converts both the perspective input and the target panorama into compressed latent representations (token sequences), using a modified encoding procedure (Circular Latent Encoding) for the target panorama that prevents boundary artifacts by circularly padding the image before convolution.
Sequence Concatenation — appends the encoded perspective tokens (conditioning) directly to the noisy panorama tokens (generation target) along the token-sequence dimension, forming a single long sequence for the transformer to process.
Diffusion Transformer (DiT) — a pre-trained text-to-image or text-to-video transformer (FLUX.1-dev for images, Wan2.1-14B for videos) fine-tuned to denoise the panorama tokens. It runs global self-attention across the entire concatenated sequence, allowing the perspective and panorama tokens to "find" each other without any pre-specified spatial alignment.
VAE Decoder (with Circular Latent Decoding) — reconstructs the final panorama image from the denoised latent sequence, using the inverse circular-padding logic to preserve boundary continuity.
Training Data Pipeline (offline) — canonically aligns all training panoramas to a gravity-aligned upright orientation by: (a) estimating per-frame camera pose via COLMAP and stabilizing video, (b) estimating global gravity direction via GeoCalib and rotating to align gravity with the vertical axis. Training conditions are cropped from canonical panoramas using randomly sampled camera parameters (FoV, pitch, roll) and camera trajectories.

Information flows as follows: a raw 360° video enters the canonicalization pipeline → becomes a gravity-aligned panorama in ERP format → a random perspective crop is extracted as the "input" → both panorama target and perspective crop are VAE-encoded (with circular encoding for the target) → tokens are concatenated and fed to the DiT along with a text caption → the DiT predicts the clean panorama tokens from noise → decoded back to a seamless 360° image.

3.3 Roadmap for the Deep Dive

First, the task formulation and flow matching framework (Section 3.1), establishing the mathematical language for how generation works and what the model is trained to do.
Second, the sequence concatenation mechanism (Section 3.2), which is the central architectural innovation that replaces camera-dependent projection with learned geometric reasoning, including the canonical coordinate constraint.
Third, Circular Latent Encoding (Section 3.3), the root-cause analysis of seam artifacts and the minimal fix that eliminates them at the training stage.
Fourth, the complete training data pipeline, including canonicalization, camera augmentation, and the specific hyperparameter choices for image and video models (drawn from Sections 3.2, 4.1, 4.2, and Appendix A).
Fifth, inference mechanics, including sampling schedules, classifier-free guidance, and why no post-processing tricks are needed for seam removal.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and architecture paper whose core idea is that explicit geometric alignment (camera calibration, perspective-to-equirectangular projection) is not only unnecessary but actively harmful for panorama generation, and that replacing it with a simple sequence-to-sequence transformer architecture—combined with fixing a latent-space discontinuity introduced by standard VAE encoding—produces a simpler pipeline that outperforms geometry-aware methods.

Task Formulation and Flow Matching Objective

The paper formulates perspective-to-panorama generation as a conditional generation problem. Given a perspective video with $T$ frames, denoted $X_{\text{pers}} \in \mathbb{R}^{T \times h \times w \times 3}$ (where $h \times w$ is the spatial resolution of the perspective frames and 3 is the RGB channel dimension), and a text caption $e$ , the goal is to produce a 360° panoramic video $Y_{\text{equi}} \in \mathbb{R}^{T \times H \times W \times 3}$ in Equirectangular Projection (ERP) format, where $H \times W$ is the panorama resolution (typically $1024 \times 2048$ for images, $512 \times 1024$ or $256 \times 512$ for videos depending on training stage). The static image case is simply $T = 1$ .

The generation process uses the flow matching framework, a continuous-time diffusion formulation. The core idea of flow matching is to learn a time-dependent vector field that transports samples from a simple noise distribution to the data distribution. The training objective is:

$\min_{\theta} \mathbb{E}_{t \sim p(t), Y_{\text{equi}} \sim p_{\text{data}}, \epsilon \sim \mathcal{N}(0, I)} \| (\epsilon - Y_{\text{equi}}) - G_{\theta}(Y^t_{\text{equi}}, t, \mathbf{c}) \|^2$

where $t \in [0, 1]$ is the diffusion timestep, $p(t)$ is the distribution of noise levels sampled during training (a logit-normal distribution in the FLUX and Wan implementations), $p_{\text{data}}$ is the distribution of real panorama data, $\epsilon \sim \mathcal{N}(0, I)$ is standard Gaussian noise, $G_{\theta}$ is the denoiser neural network parameterized by $\theta$ , $\mathbf{c}$ represents all conditioning information (the text caption and the encoded perspective tokens), and $Y^t_{\text{equi}} = (1 - t) Y_{\text{equi}} + t \epsilon$ is the forward noised version of the clean data at time $t$ .

What it computes: the expected squared error between $(\epsilon - Y_{\text{equi}})$ (the true vector field pointing from the noise sample back toward the data) and the model's prediction $G_{\theta}(Y^t_{\text{equi}}, t, \mathbf{c})$ (the estimated vector field). The outer minimization over $\theta$ finds model parameters that make these two vector fields as similar as possible in expectation over noise levels, data samples, and random noise draws.

Why this form: flow matching is equivalent to learning the velocity field of an optimal transport path from noise to data. The specific noising scheme $Y^t_{\text{equi}} = (1 - t) Y_{\text{equi}} + t \epsilon$ linearly interpolates between clean data ( $t = 0$ ) and pure noise ( $t = 1$ ), which defines straight-line probability paths. This is simpler and more efficient than the variance-preserving formulation used in DDPM-style diffusion, and allows for deterministic sampling at inference time via solving an ODE. The key practical advantage is that the target $(\epsilon - Y_{\text{equi}})$ is a simple linear combination rather than the score function $\nabla \log p_t$ , which would require more complex estimation. The model learns to predict "which direction to move from the noisy sample toward a clean sample," which at inference time translates to following this vector field from pure noise to a generated panorama.

The denoiser $G_{\theta}$ is implemented as a Diffusion Transformer (DiT), a transformer architecture where the input is a sequence of token embeddings. Modern DiTs operate in the latent space of a pre-trained convolutional VAE rather than directly on pixels, to reduce computational cost. The VAE has an encoder $\mathcal{E}$ and a decoder $\mathcal{D}$ :

$y_{\text{equi}} = \mathcal{E}(Y_{\text{equi}}), \quad \hat{Y}_{\text{equi}} = \mathcal{D}(y_{\text{equi}})$

where $y_{\text{equi}}$ is the compressed latent representation. This VAE is typically a KL-regularized autoencoder trained to reconstruct images with a combination of reconstruction loss, perceptual loss, and a KL penalty that keeps the latent distribution close to a standard Gaussian. The latent $y_{\text{equi}}$ has a spatial resolution 8× smaller than the pixel space in each dimension (for the VAE architectures used in FLUX and Wan), so a $1024 \times 2048$ pixel panorama becomes a $128 \times 256$ latent grid. This latent grid is then patchified—divided into non-overlapping patches and flattened into a 1D sequence of tokens that serves as input to the DiT. The patchification process is an important detail: each patch becomes a single token embedding via a learned linear projection, and the spatial position of each patch is encoded using Rotary Position Embedding (RoPE) in three dimensions (temporal, height, width) so the transformer knows where each token originated in the latent grid.

Sequence Concatenation: Replacing Camera Projection with Learned Geometric Reasoning

This is the central architectural innovation of 360Anything. Prior methods follow a template: given camera parameters (FoV, yaw, pitch, roll), project the perspective image $X_{\text{pers}}$ onto the ERP canvas to obtain $X^{\text{proj}}_{\text{equi}}$ , which is pixel-aligned with the target panorama $Y_{\text{equi}}$ . Then encode both to latents: $x^{\text{proj}}_{\text{equi}} = \mathcal{E}(X^{\text{proj}}_{\text{equi}})$ and $y_{\text{equi}} = \mathcal{E}(Y_{\text{equi}})$ , and concatenate them channel-wise—they are stacked along the feature dimension so that each spatial position receives both a conditioning signal and a noisy target signal. This channel concatenation provides a strong spatial prior: the model knows exactly which pixels of the output correspond to which pixels of the input because the spatial indices are aligned.

The problem with channel concatenation. This spatial alignment is both a strength and a fragility. It requires knowing camera parameters to perform the projection $\text{pers} \to \text{ERP}$ , which means deploying an external camera estimator at inference time. The paper documents that estimation errors cascade: if the estimated FoV is off by 10° or the pitch is shifted by a few degrees, the projected input will be misaligned, and the channel-concatenated model has no mechanism to correct this misalignment because it was trained with the assumption of alignment. The paper captures this failure mode explicitly in Appendix B.1 and Figure 14, showing that when MegaSaM (a state-of-the-art camera estimator) fails due to challenging lighting or complex trajectories, Argus (a channel-concatenation method) "is unable to correct the out-of-distribution conditioning input, and generates broken results."

The sequence concatenation alternative. 360Anything instead encodes the perspective input directly to latents $x_{\text{pers}} = \mathcal{E}(X_{\text{pers}})$ and appends these tokens to the noisy panorama tokens $y^t_{\text{equi}}$ along the sequence dimension:

$\text{Concat}([x_{\text{pers}}, y^t_{\text{equi}}])$

The resulting sequence is $N_{\text{pers}} + N_{\text{equi}}$ tokens long (where $N_{\text{pers}}$ is the number of tokens from the perspective input and $N_{\text{equi}}$ is the number of tokens from the noisy panorama latent). This combined sequence is fed to the DiT, which runs global self-attention over all tokens simultaneously. There is no pre-specified spatial correspondence between the perspective tokens and the panorama tokens—the model must learn to establish these relationships through the attention mechanism.

How the model learns geometry through attention. The key insight is that the DiT's attention mechanism computes a weighted sum of all token representations when producing each output token: every panorama token can attend to every perspective token (and every other panorama token). This means that when the model is generating, say, a patch of the panorama that corresponds to the left side of the scene, it can learn to attend heavily to perspective tokens that contain visual information about what should be on the left side. Over the course of training with millions of examples where the true camera parameters are known (because the perspective crops are synthetically generated from known panoramas), the model learns a mapping from "what visual features appear in what relative positions in the perspective tokens" to "where those features should be placed on the ERP canvas."

This is fundamentally a data-driven geometric calibration: the model implicitly estimates the camera parameters that produced the perspective view and uses that estimate to determine the panoramic layout. The paper validates this interpretation in Section 4.3 by explicitly extracting camera parameters from the model's generations: given a generated panorama and the input perspective image, they search for the camera parameters that minimize the reconstruction error between the input and a perspective crop of the generated panorama at those parameters, achieving competitive zero-shot calibration accuracy.

Distinguishing conditioning from target tokens with 3D RoPE. Since both the perspective tokens and the panorama tokens use the same spatial coordinate system for positional encoding (RoPE based on $(\text{time}, y, x)$ coordinates in the latent grid), there needs to be a mechanism to distinguish them. The paper offsets the time dimension index of the perspective tokens:

For the image model: the time index of perspective tokens is offset by 1 relative to the panorama tokens.
For the video model: the time index of perspective tokens is offset by 0.1 rather than 1, "to avoid confusion with tokens from subsequent frames" (Appendix A.2).

This is a subtle but important detail: if the offset were too large, the model might treat perspective tokens as belonging to a different video entirely; if too small, they might be confused with temporally adjacent panorama frames. The 0.1 offset for video represents a small displacement in the learned continuous time embedding space, providing enough separation to distinguish conditioning from target while still allowing cross-frame attention to leverage temporal structure.

This design is adapted from image editing models. The paper credits the sequence concatenation approach to recent image editing work, specifically FLUX.1 Kontext (BFLabs, 2025) and Qwen-Image (Wu et al., 2025), which use the same mechanism for tasks like inserting an object from a reference image into a target scene or editing based on a conditioning image. The innovation of 360Anything is not the sequence concatenation mechanism itself, but its application to a task that was previously thought to require explicit geometric priors, and the demonstration that the mechanism alone suffices to learn the perspective-to-equirectangular mapping.

Why this works: the geometric reasoning argument. There is a conceptual shift here worth making explicit. In prior work, the geometric relationship was input: the model received pixel-aligned conditioning and needed to learn to fill in the gaps. In 360Anything, the geometric relationship is output: the model must infer where the perspective input belongs on the sphere and produce a coherent panorama that respects that inference. This flips the complexity from the architecture (which now only does standard sequence-to-sequence processing) to the training data (which must be diverse enough for the model to learn general geometric reasoning). The paper's camera augmentation strategy (uniformly sampling FoV, pitch, roll) and use of diverse camera trajectories in training ensure that the model sees enough geometric variation to learn a robust mapping.

Canonical Coordinate Constraint

The decision to use sequence concatenation without explicit camera parameters creates a new design question: in what coordinate system should the model generate the panorama? Since the model is not told the camera orientation, it could theoretically generate the panorama in any orientation—the generated panorama might be rotated, tilted, or aligned arbitrarily as long as it's consistent with the input. Prior methods that project the input to a known location naturally generate panoramas in that coordinate system; methods like CubeDiff and ViewPoint assume the input is the front view and generate the rest around it, which means the generated panorama inherits whatever tilt or roll the input image had.

360Anything takes a different approach: it trains the model to always generate panoramas in a canonical, gravity-aligned upright orientation, regardless of the input's camera pose. This is called the Canonical Coordinate constraint.

Why this is necessary. Without this constraint, the model would need to learn to generate panoramas in a continuum of different orientations depending on the input's pitch and roll. Each orientation corresponds to a different spherical distortion pattern in the ERP representation—a building that appears upright when the panorama is gravity-aligned would appear curved or tilted when the panorama is rotated. The paper's ablation (Table 7) shows that training without canonicalization leads to significantly degraded visual quality: FVD increases from 470.8 to 559.5 on real camera trajectories (for the lower-resolution 256×512 model), and VBench Imaging Quality drops from 0.5437 to 0.4689. The paper explains: "when training on non-canonicalized videos, we always place the conditioning view at the center of the output image. This makes reconstructing the conditioning frame much easier, leading to better PSNR and LPIPS. However, it degrades the visual quality and fidelity significantly... because the model has to generate panorama frames with varying gravity directions when the input video has non-zero roll and pitch angles, forcing it to learn different spherical distortion patterns."

How the canonicalization constraint is enforced during training. The training data must be pre-processed so that all target panoramas are gravity-aligned and upright. For the image training data, this is naturally satisfied because the datasets (Polyhaven, Humus, Structured3D, Pano360) are predominantly synthetic renderings of 3D scenes, which are generated in a canonical upright orientation by construction. For the video training data from the 360-1M dataset (YouTube videos), the raw footage is arbitrarily oriented—people hold cameras at various angles, mount them on moving vehicles, or walk with them, resulting in panoramas where the horizon may be tilted or oscillating.

The paper designs a two-stage canonicalization pipeline for video data (illustrated in Figure 8):

Video stabilization (removing inter-frame rotation). COLMAP with rig support is run to estimate per-frame camera poses. Each frame is then rotated to have zero rotation relative to the first frame, eliminating camera shake and drift. This ensures that the gravity direction is consistent across all frames of a video—if the camera was tilted 10° in the first frame, all frames are aligned to that same 10° tilt.
Gravity alignment (making the panorama upright). GeoCalib, a learning-based camera calibration model, is used to estimate the global gravity direction of the stabilized video. Since GeoCalib is trained only on perspective images, each panorama frame is projected to eight perspective views (elevation = 0° with uniform azimuth spacing of 45°) and GeoCalib is run independently on each. The predicted gravity directions are averaged across views after removing outliers (predictions more than 3 standard deviations from the mean). Then the entire video is rotated so that the gravity direction aligns with the vertical axis of the ERP image. The gravity direction is averaged across all frames (since the video is stabilized, they share the same gravity) after the same outlier removal.

The result is a video where the horizon is a straight horizontal line across all frames—trees grow upward, buildings stand vertical, and the panorama is in a standard, gravity-aligned orientation.

At inference time, the model is never told the input's camera orientation. It must infer from the visual content where "up" is (e.g., by recognizing that walls are typically vertical, floors are horizontal, sky is above) and position the perspective content on the canonical canvas accordingly. This means the model implicitly performs a form of visual gravity estimation as a sub-task of panorama generation.

Circular Latent Encoding: Eliminating the Root Cause of Seam Artifacts

This is the paper's second major technical contribution, and it addresses a long-standing problem in panorama generation: visible discontinuities at the left-right boundary of the equirectangular projection. This section requires careful explanation because the paper's insight is non-obvious and the fix, while simple, is precisely targeted at a specific failure mode in the standard VAE encoding pipeline.

What seam artifacts look like and why they matter. An ERP panorama wraps horizontally: the leftmost column of pixels (0° longitude) and the rightmost column (360° longitude) represent the same viewing direction—they should connect seamlessly when the panorama is viewed in a 360° viewer. However, diffusion-generated panoramas often have a visible discontinuity at this boundary: color mismatches, broken object continuity, or a sharp line where the two edges meet. In a VR headset, this manifests as a jarring tear in the visual field. Figure 7 (leftmost column, "Vanilla") shows this effect clearly: when the panorama is shifted by 180° so the boundary is in the center, a sharp vertical discontinuity is visible.

The standard diagnosis (which the paper argues is wrong). Prior work attributed seam artifacts to the generation process: the diffusion model, being trained on finite-sized images, doesn't "know" about the circular topology of the ERP and therefore fails to enforce continuity at the boundary. The corresponding fixes were applied at inference time:

Rotated denoising shifts the panorama cyclically across diffusion steps so the model processes the boundary at different positions, hoping the inconsistency averages out.
Circular padding in the VAE decoder replaces zero-padding with wrap-around padding during the decoding convolution, ensuring that boundary pixels see content from the opposite side.
Blended decoding overlaps and blends the left and right edges.

These are all ways of saying "the model produces artifacts; let's clean them up afterward."

The paper's diagnosis: the encoder, not the generator, creates the discontinuity. The critical insight is illustrated in Figure 3a. Take a perfectly seamless panorama (one that wraps continuously). Encode it with a standard VAE that uses zero-padding in its convolutional layers. The VAE encoder processes the image through a stack of convolutions, and at each convolution, the image boundaries are padded with zeros—pixels outside the image are treated as black. For a normal photograph, this is fine because the boundary is a genuine edge of the content. But for a panorama, the left and right boundaries are not true edges—they should connect to each other. Zero-padding introduces an artificial discontinuity: the leftmost pixels in the latent representation are influenced by zeros on their left (the artificial boundary), while the corresponding rightmost pixels (which should be adjacent in circular space) have no mutual influence.

Now, take this encoded latent, shift it by 180° (so the original boundary is in the middle of the latent grid), and decode it. The decoded image will show a seam artifact at the shift position—exactly where the original boundary was in the latent. This demonstrates that the latent representation itself is discontinuous even though the original image was seamless. The diffusion model is trained on these corrupted latents: it learns to denoise latents that have an inherent boundary discontinuity baked in. At inference time, any residual noise or generation error at the boundary position is amplified because the model's training distribution contains that discontinuity.

The fix: Circular Latent Encoding. Instead of zero-padding the image before VAE encoding, the paper circularly pads it: a strip of $w'$ columns from the right side of the image is appended to the left side, and a strip of $w'$ columns from the left side is appended to the right side. Specifically, $w'$ is set to $W/8$ (where $W$ is the panorama width), which corresponds to a single token's receptive field after patchification (since the VAE downscales by 8×, the patchification step further divides by the patch size). The padded image is then encoded:

$y_{\text{equi}}^{\text{pad}} = \mathcal{E}(\text{Concat}([Y_{\text{equi}}[-w':], Y_{\text{equi}}, Y_{\text{equi}}[:w']]))$

After encoding, the latent tokens corresponding to the padded regions are dropped, so the final latent has the same dimensions as if zero-padding had been used. This means the DiT input sequence length is unchanged, and no additional tokens are processed during training or inference.

What this achieves. During encoding, the convolutional layers now see the "other side" of the panorama at the boundaries. The left edge of the image is padded with content from the right edge (via the circular wrap), and vice versa. The resulting latent has no boundary discontinuity—the information from the opposite side of the panorama is correctly integrated into the boundary tokens. Figure 3b illustrates this: shifting the circularly-encoded latent by 180° and decoding produces a seamless image with no visible artifact.

Why this is superior to inference-time fixes. Inference-time fixes like rotated denoising attempt to correct a discrepancy that was created during training. The model was trained on discontinuous latents, so it learned to expect and reproduce a certain amount of boundary inconsistency. Rotated denoising can average out some of this, but it cannot undo the training bias. Circular Latent Encoding, by contrast, eliminates the problem at the source: the model is trained on latents that are genuinely seamless, so it never learns to produce seam artifacts in the first place. At inference time, no special handling of the boundary is needed—standard decoding produces a seamless panorama because the model was trained on seamless latents.

The paper applies the same circular logic at decoding time: before decoding, the latent is circularly padded in the same way, the VAE decoder processes the padded latent, and the decoded padded regions are cropped to recover the original image dimensions. This ensures that the decoding convolutions also have access to cross-boundary context.

Quantitative evidence for the improvement. Table 5 reports the Discontinuity Score (DS), a metric that quantifies seam artifacts. On the image task, vanilla encoding (zero-padding) achieves DS = 9.92, blended decoding (the Argus inference-time fix) achieves DS = 5.29, and Circular Latent Encoding achieves DS = 3.87—a 61% reduction from vanilla and a 27% reduction from the best alternative. On the video task, the reductions are even more dramatic: vanilla DS = 35.52 (videos have more frames to accumulate artifacts), blended decoding DS = 19.84, and CLE DS = 13.28—a 63% reduction from vanilla. The paper also includes a qualitative comparison in Figure 7: the vanilla method shows a clear seam, blended decoding "blurs" the seam but introduces gray line-like artifacts (the blending averages content but doesn't eliminate the underlying discontinuity), and CLE eliminates the boundary artifacts entirely.

The computational cost. Crucially, Circular Latent Encoding introduces no overhead to training or inference. The padding and cropping operations are simple tensor manipulations that add negligible computation compared to the VAE encoding/decoding and the DiT forward pass. The latent sequence length is unchanged, so the transformer sees exactly the same number of tokens. This contrasts with inference-time methods like rotated denoising, which require running the diffusion process multiple times with different shifts and averaging the results—a significant computational increase.

Training Data Pipeline and Augmentation Strategy

The training process requires paired data: for each training sample, a full 360° panorama (the ground truth) and a perspective crop from that panorama (the conditioning input). The panorama serves as the generation target; the perspective crop simulates what a user might provide at test time. The key design decision is how to crop the perspective input during training, which determines what distribution of camera parameters the model learns to handle.

Image training data. The paper uses four datasets: Polyhaven (HDRI environment maps), Humus (texture panoramas), Structured3D (synthetic indoor scenes with three subsets: empty, simple, and full), and Pano360. Structured3D constitutes ~90% of the training data, meaning the model primarily learns from synthetic indoor environments. All panoramas are in ERP format. Captions are generated using Gemini 2.5 Flash. The panorama images are already in canonical upright orientation (they are synthetically rendered that way).

Camera augmentation for image training. For each training sample, camera parameters are randomly sampled to crop a perspective view from the panorama:

FoV: uniformly sampled in $[30°, 120°]$ . This covers a wide range from narrow telephoto-like views to wide-angle perspectives. The lower bound of 30° ensures the input contains enough visual information to be useful; the upper bound of 120° represents an extreme wide-angle lens.
Pitch: uniformly sampled in $[-60°, 60°]$ . This allows the input to be looking significantly up or down, simulating tilted cameras or varied viewing angles.
Roll: uniformly sampled in $[-15°, 15°]$ . This is a narrower range than pitch, reflecting that extreme roll (camera rotation around the viewing axis) is less common in natural photography.

The cropping process uses these parameters to project a perspective image from the full panorama via standard spherical-to-perspective projection mathematics. Additionally, the authors apply horizontal roll augmentation to the panorama: the entire ERP image is cyclically shifted horizontally by a random amount, which is equivalent to rotating the spherical panorama around the vertical axis. This ensures the model sees the conditioning image at all possible yaw angles and doesn't learn any spurious correlation between yaw position and content.

Ablation on camera augmentation. The paper compares training with and without camera augmentation (Table 6). The "w/o Camera Aug." variant always trains with the standard viewpoint: FoV = 90°, pitch = 0°, roll = 0°. Counterintuitively, adding camera augmentation improves performance even on the standard viewpoint: FID improves from 8.4 to 8.0, and FAED improves from 10.4 to 9.8. The paper hypothesizes that "training with a wide distribution of camera setup forces the model to better understand perspective-equirectangular geometry, preventing overfitting to a single mapping." This is a finding that supports the broader thesis: diverse training data enables the model to learn generalizable geometric reasoning.

Video training data. The video model is trained on the 360-1M dataset, a collection of YouTube 360° videos. The paper uses the filtered subset from Argus, which removes videos in non-panorama format, videos with very low motion, and videos with bad visual quality. This subset is further divided into a "coarse" subset (lower visual quality but larger quantity) and a "high-quality" subset. The canonicalization pipeline described above is applied to all videos.

Camera trajectory augmentation for video. Creating a perspective video conditioning signal requires not just a single set of camera parameters, but a trajectory—a sequence of camera poses across the video frames. The paper follows Argus in simulating camera trajectories, but makes a critical improvement:

Simulated trajectories (80% of training samples): randomly sampled linear motion with added noise. This models simple panning, tilting, or dolly movements.
Real-world trajectories (20% of training samples): trajectories extracted from real-world perspective videos using structure-from-motion. The paper specifically cites camera trajectories from Rockwell et al. (2025) and ScanNet++ (Yeshwanth et al., 2023). These real trajectories include complex motions that simulated linear trajectories cannot capture: handheld camera shake, accelerations and decelerations, non-linear paths, and jerky movements.

The inclusion of real-world trajectories is motivated by an empirical finding: models trained only on simulated linear motion "fail to generalize to videos with complex motion" (Section 4.2 and Figure 13). The 80/20 split represents a trade-off: simulated trajectories provide controlled, clean training signals that cover the basic types of motion, while real trajectories provide the diversity of natural camera movement that the model will encounter in-the-wild.

Captioning. Both image and video training samples are captioned using Gemini 2.5 Flash. For videos, the footage is downsampled to 1 FPS before captioning to reduce computational cost while still providing temporal context. The captions describe the scene content and serve as additional conditioning for the diffusion model via standard text-to-image conditional generation (classifier-free guidance on text embeddings).

Video training resolution and staging. The video model is trained in two stages for computational efficiency:

Stage 1: 10,000 steps on the coarse subset at $256 \times 512$ resolution (81 frames). This lower resolution allows faster iteration and learning of basic geometric reasoning.
Stage 2: 10,000 more steps on the high-quality subset at $512 \times 1024$ resolution (81 frames). This higher resolution refines the visual quality and detail.

Both stages use 81 frames, which corresponds to approximately 2.7 seconds of video at 30 FPS or 3.4 seconds at 24 FPS. The choice of 81 frames is a balance: enough frames to capture meaningful motion and provide temporal context for the diffusion model, but not so many that the computational cost becomes prohibitive (the ERP video has 8× the pixels of a perspective video at the same per-frame resolution, making video generation extremely expensive).

Model Architecture and Training Hyperparameters

Image model details. The image model fine-tunes FLUX.1-dev, a state-of-the-art text-to-image DiT with approximately 12 billion parameters. The fine-tuning uses the Adam optimizer with the following hyperparameters:

Learning rate: $5 \times 10^{-5}$ , with linear warmup from 0 over the first 1,000 steps, then held constant. This is a relatively high learning rate for fine-tuning a large pre-trained model, justified by the significant distribution shift from standard images to panoramas (the model needs to learn new geometric mappings, not just refine existing ones).
Batch size: 512. This large batch size is typical for DiT training and ensures stable gradient estimates.
Gradient clipping: 1.0 (clipping the global norm of gradients). This prevents training instability, especially during the early stages when the model is adapting to the new data distribution.
Training steps: 50,000. With a batch size of 512, this corresponds to 25.6 million training samples.
Image resolution: $1024 \times 2048$ (ERP format). This is the standard 2:1 aspect ratio of equirectangular projections.

Video model details. The video model fine-tunes Wan2.1-14B, a 14-billion-parameter text-to-video DiT. The hyperparameters are:

Learning rate: $1 \times 10^{-5}$ , with the same linear warmup schedule (1,000 steps) and constant afterward. The lower learning rate compared to the image model reflects the larger model size and the additional complexity of video generation.
Batch size: 64. Smaller than the image model due to the significantly higher memory requirements of 81-frame video latents.
Gradient clipping: 1.0.
Training steps: 20,000 total (10,000 per stage).

Why FLUX and Wan as base models. The paper selects these specific base models because they are the state-of-the-art open-weights DiTs in their respective modalities at the time of writing. Critically, both use the sequence-to-sequence transformer paradigm with 3D RoPE positional encoding, which directly supports the sequence concatenation mechanism. Using a U-Net-based base model would require architectural modifications to accept sequence-concatenated inputs (U-Nets typically operate on spatially-structured tensors, not flat token sequences). The choice of base model is thus not arbitrary—it aligns with the paper's architectural thesis that Transformers are better suited than U-Nets for tasks requiring learned spatial reasoning across large receptive fields.

Classifier-free guidance (CFG) setup. During training, the model is conditioned on both the text caption and the perspective input. To enable classifier-free guidance at inference time, both conditioning signals are randomly dropped with 10% probability during training. When dropped, the condition is replaced with a learned null embedding. This means the model learns three modes:

Full conditioning (text + perspective input) — 81% of training samples.
Text only (perspective dropped) — 9% of training samples.
Perspective only (text dropped) — 9% of training samples.
Unconditional (both dropped) — 1% of training samples.

At inference time, CFG is applied separately to text and image conditioning:

Image model: CFG scale of 2.0 on text and 1.5 on image (perspective conditioning).
Video model: CFG scale of 3.0 on text and 2.0 on image.

The higher CFG scales for video reflect the more challenging generation task: the model needs stronger conditioning signals to maintain temporal consistency across 81 frames while generating a full 360° view. This separate guidance on text and image is similar to the approach in InstructPix2Pix (Brooks et al., 2023), extended to the dual-conditioning setting.

3D RoPE implementation details. Rotary Position Embeddings are applied in three dimensions: temporal ( $t$ ), height ( $y$ ), and width ( $x$ ). For the image model, perspective tokens and panorama tokens share the same $(y, x)$ spatial coordinates (they each have their own spatial grid within their respective images) but differ in the temporal coordinate: panorama tokens use $t$ corresponding to their actual frame index, while perspective tokens use $t + 1$ (offset by 1 in the temporal dimension). For the video model, the temporal offset is 0.1 rather than 1, to provide a more subtle distinction that doesn't confuse the model about which frame the perspective tokens belong to. These offsets are applied when computing the RoPE frequencies, which rotate the query and key vectors in the attention mechanism based on their relative positions. The effect is that the model can distinguish "this token is from the conditioning input" from "this token is from the generation target" while still allowing attention to flow freely between them.

Inference Mechanics

Sampling procedure. At inference time, generation starts from pure Gaussian noise $\epsilon \sim \mathcal{N}(0, I)$ and follows the learned vector field $G_{\theta}$ to produce a clean latent. Both models use the default sampler from their respective base architectures with 50 sampling steps:

FLUX (image model): uses a rectified flow sampler, which solves the ODE $dy/dt = G_{\theta}(y, t, \mathbf{c})$ from $t = 0$ to $t = 1$ . The 50 steps are spaced according to the default FLUX schedule.
Wan (video model): uses its default sampler with 50 steps as well.

Timestep shifting. Timestep shifting is a technique that redistributes the sampling steps to spend more computation at certain noise levels. The specific shifting value depends on the image resolution:

FLUX: computes the shifting based on the number of tokens. For the $1024 \times 2048$ resolution, the number of latent tokens after patchification exceeds FLUX's default maximum of 4096 tokens, so it uses the cutoff shifting value of $\exp(1.15) \approx 3.16$ . The paper notes that "we tried larger value but observed degradation in the result."
Wan: uses timestep shifting of 3.0 for the $512 \times 1024$ video resolution. This was selected from ablating values of 2.0, 3.0, and 5.0, with 3.0 performing best.

Timestep shifting works by biasing the sampling schedule toward higher noise levels (earlier $t$ values) when the image resolution is large, because high-resolution images need more denoising steps at high noise levels to establish global structure before fine details are refined.

Seam handling at inference. Unlike prior methods, no special inference-time seam handling is needed. Since the model was trained on circularly-encoded latents that are free from boundary discontinuities, the generated latents are also seamless. The VAE decoder applies Circular Latent Decoding (the inverse operation of encoding: circularly pad the latent, decode, crop the padded regions), ensuring any minor boundary effects from the decoding convolutions are also eliminated. The paper states explicitly that CLE "introduces no overhead to the generation process"—the circular padding and cropping operations add negligible wall-clock time compared to the DiT forward passes and VAE decode.

Input preparation. At test time, the user provides a perspective image or video. No camera parameters are needed. The image is resized (but the paper does not specify an exact input resolution—the VAE encoder can handle arbitrary sizes, and the model has been trained on crops with diverse FoVs and aspect ratios). The input is VAE-encoded and concatenated to the initial noise latent as described. For videos, the conditioning video may have a different number of frames than the model's training length (81 frames); the paper does not explicitly discuss handling of variable-length video inputs, but the sequence concatenation approach naturally extends to different sequence lengths since attention is computed over the full concatenated sequence regardless of length.

Canonical output guarantee. Since the model is trained to always produce gravity-aligned panoramas, the output at inference time is automatically in canonical orientation—the horizon is horizontal, vertical lines are straight, and "up" aligns with the top of the ERP image. This is a property of the training, not of any inference-time correction: the model never learned to produce non-canonical panoramas because it never saw them as training targets. This is a significant practical advantage over methods that require the user to specify or correct the output orientation.

Summary of Design Choices and Their Justifications

Sequence concatenation over channel concatenation: eliminates the camera calibration dependency by enabling the model to learn geometric correspondence through attention rather than receiving it as spatial alignment. The key tradeoff is replacing a hard-coded geometric prior with learned geometric reasoning, which requires more training data diversity but produces a more robust and general model.
Canonical coordinate constraint: simplifies the generation task by limiting the output space to a single gravity-aligned orientation. The model only needs to learn one spherical distortion pattern, and the burden shifts to the input side (the model must infer camera pose rather than ignore it). The ablation in Table 7 confirms this design choice: non-canonical training produces worse visual quality.
Circular Latent Encoding over inference-time seam fixes: removes the root cause of boundary artifacts (zero-padding in VAE convolutions) rather than masking the symptoms. This is a training-time fix with zero inference overhead, in contrast to rotated denoising which requires multiple sampling passes.
0.1 temporal offset for perspective tokens in video model: distinguishes conditioning from target tokens while maintaining the ability to attend across them. The small offset (rather than 1) prevents the model from interpreting the perspective tokens as belonging to a different frame, which would harm temporal reasoning.
80/20 simulated/real trajectory mix for video training: balances clean, controlled training signals with diverse, realistic camera motion. Models trained without real-world trajectories fail on complex in-the-wild videos.
Separate CFG scales for text and image: provides fine-grained control over how strongly the model adheres to the caption versus the visual conditioning. The higher image CFG scales for video (2.0 vs. 1.5) reflect the greater difficulty of maintaining spatial consistency across 81 frames.
Staged video training (coarse → high-quality, low-res → high-res): amortizes the computational cost of video generation by learning basic geometric relationships at lower resolution before refining quality at higher resolution.
Adam optimizer with warmup and gradient clipping: standard choices for fine-tuning large pre-trained diffusion models. The relatively high learning rate ( $5 \times 10^{-5}$ for image, $1 \times 10^{-5}$ for video) reflects the significant domain shift from standard images/videos to panoramas, requiring the model to learn fundamentally new geometric mappings rather than just adapting the output distribution.

4. Key Insights and Innovations

Innovation 1: Reframing Geometric Alignment as a Learning Problem, Not a Pre-Processing Requirement

The most intellectually distinctive move in this paper is not any specific architectural choice but the reframing of the perspective-to-panorama mapping problem itself. The dominant paradigm—established across PanoDiffusion, Imagine360, Argus, CubeDiff, and ViewPoint—treated geometric alignment as something that must be provided to the model through explicit camera projection before the diffusion process begins. This approach makes intuitive sense: the perspective image needs to be placed on the spherical canvas, and if you know the camera parameters, you can compute exactly where it goes. The model's job is then "merely" to fill in the missing regions.

360Anything rejects this entire framing. Instead, it treats geometric alignment as something the model learns internally through data-driven attention. The perspective input and the panorama target are both represented as un-positioned token sequences, and the model must discover their spatial relationship through the same mechanism it uses to discover any other relationship: global self-attention over the concatenated sequence. The paper's contribution here is not a new attention variant (it uses standard 3D RoPE with a minor temporal offset trick), but rather the empirical demonstration that this works at all—and works well enough to outperform methods that have access to ground-truth camera information (Table 2).

What makes this a genuine conceptual innovation rather than an obvious architectural choice is that the field had largely accepted the necessity of geometric priors for panorama generation. Prior methods invested heavily in spherical convolutions, dual-branch architectures, cubemap representations, and custom projection layers—all designed to inject geometric knowledge into the model. The paper's title itself announces the heresy: "Geometry-Free." The key insight is that geometric knowledge can be an output of learning rather than an input to the architecture. This parallels a broader pattern in computer vision—the paper explicitly connects to prior-free view synthesis (Rombach et al., 2021) and to the transformer takeover of tasks previously requiring explicit 3D priors—but applies it to a domain where the failure of geometry-free approaches was widely assumed.

The significance of this reframing extends beyond the specific method. It implies that the camera calibration step is a bottleneck, not a foundation. The paper shows that removing it not only simplifies the pipeline but improves robustness: when off-the-shelf camera estimators fail (Appendix B.1, Figure 14), channel-concatenation methods produce broken outputs, while 360Anything's learned geometric reasoning generalizes. This inverts the conventional wisdom: explicit geometric priors were not only unnecessary but harmful because they introduced a single point of failure. The evidence for this claim is not just the benchmark numbers but the specific failure case analysis in Appendix B.1 where SoTA camera estimator MegaSaM fails and Argus breaks while 360Anything succeeds—a direct head-to-head on exactly the robustness dimension the paper claims as its advantage.

This is a fundamental reframing of the problem, not an incremental improvement. It changes the question from "how do we improve camera estimation for better projection?" to "how do we train models to reason about geometry from data?", opening an entirely different research direction that scales with model capacity and data diversity rather than with the accuracy of external estimators.

Innovation 2: Root-Cause Diagnosis of Seam Artifacts as a VAE Latent-Space Problem

The paper's second major contribution is an act of diagnostic reframing rather than a new method. Seam artifacts at ERP boundaries have been a persistent and widely recognized problem in panorama generation, addressed by an accumulation of inference-time tricks: rotated denoising (PanoDiffusion), circular padding in the VAE decoder (360DVD), and blended decoding (Argus). Each of these treats the seam as a generation artifact—something the diffusion model produces because it doesn't understand circular topology.

The paper's insight is that this diagnosis is backwards. Through the simple diagnostic experiment illustrated in Figure 3a—encode a seamless panorama, shift the latent by 180°, decode, and observe a seam at the shift position—the paper demonstrates that the seam is created during VAE encoding, not during diffusion generation. Standard VAE encoders use zero-padding in their convolutional layers, which treats the left and right boundaries of the ERP image as true image edges rather than as a continuous wrap-around. This introduces a discontinuity in the latent representation itself. The diffusion model is then trained on these corrupted latents—it learns to expect and reproduce a latent-space discontinuity because that's what its training distribution contains.

What makes this innovative is not the fix (circular padding before encoding is straightforward once the problem is identified) but the shift in where the field looks for solutions. Prior work was optimizing the wrong stage of the pipeline. Inference-time fixes are attempts to correct a training-time problem, which is inherently limited—the model has already learned the wrong latent-space structure, and post-hoc corrections can only partially compensate. The paper's approach eliminates the problem at the source, ensuring the model never learns the discontinuity in the first place.

The quantitative evidence supports the claim that this is a real diagnostic advance rather than just another trick. Table 5 shows that Circular Latent Encoding achieves a Discontinuity Score of 3.87 on images (vs. 5.29 for blended decoding and 9.92 for vanilla encoding)—reducing artifacts not by blending them away but by preventing them from existing. The 63% reduction on videos (from 35.52 to 13.28) is particularly striking because temporal consistency amplifies boundary artifacts: each frame's seam accumulates, making videos substantially more sensitive to latent-space discontinuities. The qualitative comparison in Figure 7 tells the same story: blended decoding "blurs" the seam but introduces gray line-like artifacts (because averaging across a discontinuity doesn't make it continuous), while CLE produces genuinely seamless boundaries.

This contribution is fundamental diagnostic work dressed in a simple technical fix. It reframes where the problem lives (VAE latent space, not diffusion sampling) and thereby redirects future research: instead of developing more sophisticated inference-time seam removal, researchers should focus on ensuring the latent representation preserves circular continuity during training. The finding also has implications beyond panorama generation—any task involving circular or wraparound image domains (spherical CNNs, omnidirectional depth estimation, 360° video compression) may suffer from analogous latent-space boundary artifacts that could be addressed with the same diagnostic approach.

Innovation 3: Demonstrating That the Model Learns Accurate Camera Geometry Without Supervision

The paper's third distinctive contribution is what it proves about the model's internal representations through the zero-shot camera calibration experiments in Section 4.3. This is not a proposed application of 360Anything but an analytical probe: if the model truly learns to "place" the perspective input on the canonical canvas without explicit camera parameters, then it must be computing something equivalent to camera calibration internally. The paper makes this inference explicit by running an exhaustive search over FoV, pitch, and roll to find the parameters that best align the input image with a perspective crop from the generated panorama.

The results are striking not because they achieve state-of-the-art calibration (they don't—they lag behind DUSt3R and MoGe by 1–2° on FoV estimation and behind GeoCalib by ~0.5° on pose) but because they emerge from a model trained only for panorama generation, with no calibration supervision whatsoever. Table 3 shows 360Anything achieving an average FoV estimation error of 4.93° across three real-world datasets, outperforming several methods that were explicitly trained for camera calibration (Perspective Fields, WildCam, LeReS, UniDepth). Table 4 shows pose estimation accuracy within ~0.5° of the state-of-the-art GeoCalib, again without any pose supervision.

This is significant because it provides mechanistic evidence for the paper's central claim. The paper argues that "geometry-free" means the model doesn't need camera parameters as input; it doesn't mean the model lacks geometric knowledge. The calibration experiments demonstrate that geometric knowledge is present—the model has implicitly learned to estimate FoV and orientation as an intermediate computation on the path to generating a coherent panorama. This transforms the paper's thesis from a plausible hypothesis to an empirically verified fact.

More broadly, this finding contributes to the growing body of evidence that generative models trained on large-scale visual data develop internal representations of 3D structure without explicit geometric supervision. It aligns with work showing that text-to-image models learn depth, correspondence, and viewpoint relationships, but extends this to the specific domain of spherical projection geometry—the model understands not just that objects have 3D structure, but how that structure maps onto the equirectangular projection under varying camera parameters. This is a more specific and arguably more sophisticated form of geometric understanding than general depth estimation.

The contribution is conceptual validation of the geometry-free paradigm, not a practical tool (using 360Anything as a camera calibrator would be expensive and slow compared to dedicated methods). It strengthens the paper's core argument: explicit geometric priors are unnecessary not because geometry is irrelevant, but because large-scale generative training discovers geometry automatically.

5. Experimental Analysis

Evaluation Methodology

Dataset. The image experiments use the Laval Indoor and SUN360 datasets for perspective-to-360° image generation. The Laval Indoor dataset contains indoor panorama images with diverse lighting and layout conditions. The SUN360 dataset contains a broader variety of indoor and outdoor scenes with more complex textures and scene layouts. Both datasets follow the evaluation protocol proposed in CubeDiff. The video experiments use a hold-out set of 101 testing videos from the Argus evaluation protocol, with conditioning perspective videos generated from two types of camera trajectories: simulated trajectories (linear motion with noise) and trajectories extracted from real-world videos.
Base model(s). The image model fine-tunes FLUX.1-dev, a 12-billion-parameter state-of-the-art text-to-image diffusion transformer. The video model fine-tunes Wan2.1-14B, a 14-billion-parameter text-to-video diffusion transformer. Both are selected because they are the best-performing open-weights DiTs in their respective modalities and natively support the sequence-to-sequence transformer paradigm required by 360Anything's sequence concatenation mechanism. The paper uses these models to demonstrate that the approach works on top of strong general-purpose generators rather than requiring specialized panorama architectures.
Metrics. For image generation, visual quality is measured using Fréchet Inception Distance (FID), Kernel Inception Distance (KID), FID on CLIP features (CLIP-FID), and FID on features of an auto-encoder fine-tuned on panorama images (FAED). FID, KID, and CLIP-FID are computed on 10 perspective crops from the generated panorama (10 azimuth angles randomly sampled from [90°, 270°] to avoid overlap with the input view, at elevation = 0°), while FAED is computed directly on the entire ERP panorama to measure overall geometric quality. Text alignment is measured using CLIP-score (CS). For video generation, input preservation is measured by PSNR and LPIPS between ground-truth and generated panorama videos within regions covered by the perspective video (computed by projecting a mask to ERP space using ground-truth camera information and taking the union over all frames). Overall video quality uses FVD computed on the full ERP videos, plus Imaging Quality, Aesthetic Quality, and Motion Smoothness from VBench, evaluated on perspective projections (front, left, right, back) of the generated panorama videos. Seam artifacts are quantified using the Discontinuity Score (DS) from Christensen et al. (2024).
Baselines. For image generation, the primary comparison is against CubeDiff (Kalischek et al., 2025), the prior state-of-the-art, alongside OmniDreamer (Akimoto et al., 2022), PanoDiffusion (Wu et al., 2024), and Diffusion360 (Feng et al., 2023). For video generation, comparisons are against Imagine360 (Tan et al., 2025)—a dual-branch U-Net architecture connecting perspective and ERP processing with spherical attention; Argus (Luo et al., 2025)—which projects perspective video to ERP space and fine-tunes SVD for outpainting; and ViewPoint (Fang et al., 2025)—which uses a cubemap-derived representation and fine-tunes Wan2.1. All baselines are evaluated using their official code, and when the method requires camera metadata, ground-truth camera information is provided (giving baselines an advantage over 360Anything, which receives no camera metadata).
Generation budget / compute accounting. The paper measures compute primarily in terms of inference cost, with image generation using 50 sampling steps with FLUX's rectified flow sampler and video generation using 50 sampling steps with Wan's default sampler. All methods use comparable sampling budgets (the paper does not explicitly equalize total FLOPs across methods since different architectures have different per-step costs). The key fairness consideration is that baselines receive additional information (ground-truth camera parameters) that 360Anything does not, making the comparison conservative for 360Anything.
Cross-validation / statistical protocol. The paper does not use cross-validation or statistical significance testing for the main benchmark comparisons—results are reported as single-run metrics on the standard test sets. For the camera calibration experiments (Section 4.3), the approach is evaluated on established benchmarks (NYUv2, ETH3D, iBims-1 for FoV; MegaDepth, LaMAR for pose) with standard train/test splits. When reproducing the Argus evaluation set, the paper communicates with the original authors to ensure comparable evaluation splits but acknowledges that the exact split is unavailable (metrics are marked with * in Table 2 to indicate reproduced results).

Main Quantitative Results

Perspective-to-360° Image Generation

The headline result for image generation (Table 1) is that 360Anything achieves state-of-the-art performance on both Laval Indoor and SUN360, substantially outperforming all prior methods including CubeDiff, the previous best approach. On Laval Indoor, 360Anything achieves FID = 8.0 (vs. 9.5 for CubeDiff), KID = 0.22 × 10⁻² (vs. 0.32), and FAED = 9.8 (vs. 18.4)—a nearly 50% reduction in FAED, which is the only metric computed on the full panorama rather than on perspective crops. On SUN360, the advantages are even larger: FID = 22.4 (vs. 25.5), KID = 1.27 × 10⁻² (vs. 1.33), CLIP-FID = 7.3 (vs. 8.1), FAED = 3.8 (vs. 7.6), and CS = 28.07 (vs. 25.00).

The FAED improvement is the most significant finding because FAED measures geometric quality and structural coherence of the entire ERP panorama, not just the appearance of individual perspective crops. The paper correctly emphasizes that this metric captures the overall geometric correctness that 360Anything's learned spatial reasoning provides. The marginally higher CLIP-FID on Laval Indoor (4.6 vs. 3.2) is the only metric where 360Anything lags behind CubeDiff, which the paper attributes to the fact that CLIP-FID is computed on perspective crops where CubeDiff's cubemap representation may provide better per-face appearance at the cost of inter-face consistency.

Qualitative comparisons (Figure 4) reveal specific failure modes of CubeDiff that explain the quantitative gap. CubeDiff's cubemap representation assumes 90° FoV conditioning and treats the input as the front face, which causes visible seams between cube faces and distorts objects when the actual input FoV differs from 90°—the paper shows stretched balloons and distorted mushrooms as concrete examples. 360Anything, by implicitly estimating camera parameters, correctly places the input on the panorama and generates objects with proper structure. The paper also demonstrates generalization to out-of-distribution images (Figure 9), including AI-generated inputs, where 360Anything maintains high visual quality despite being trained primarily on synthetic indoor scenes.

Perspective-to-360° Video Generation

The video results (Table 2) show even more decisive advantages for 360Anything. On real camera trajectories, 360Anything achieves PSNR = 25.75 (vs. 23.25 for ViewPoint, the next best), LPIPS = 0.0468 (vs. 0.1364), FVD = 483.4 (vs. 844.3), Imaging Quality = 0.5515 (vs. 0.5293), Aesthetic Quality = 0.5427 (vs. 0.5150), and Motion Smoothness = 0.9885 (vs. 0.9881). On simulated trajectories, the pattern is similar: PSNR = 23.64, LPIPS = 0.0846, FVD = 432.9, all substantially better than baselines.

The FVD reduction from 844.3 (ViewPoint) to 483.4 (360Anything) represents a ~43% improvement, which is a large effect size in video generation. FVD measures the distributional distance between generated and real videos in a learned feature space, so this improvement indicates that 360Anything's videos are substantially more similar to real panorama videos in terms of both visual quality and temporal dynamics. The LPIPS improvement—measuring perceptual similarity in the input-preserved regions—from 0.1364 to 0.0468 is particularly notable because it demonstrates that 360Anything better preserves the conditioning video content despite not receiving explicit camera projection information.

A striking aspect of these results is that 360Anything outperforms baselines that use ground-truth camera metadata. Imagine360, Argus, and ViewPoint all receive the correct camera parameters to project the perspective input to ERP space (a significant informational advantage), yet 360Anything achieves better PSNR and LPIPS, meaning it learns to better preserve the conditioning video. The paper interprets this as evidence that the sequence concatenation approach, combined with canonical training, enables more accurate geometric reasoning than explicit projection—the model is not just memorizing where to place the input but actually understanding the spherical geometry.

Qualitative comparisons (Figure 5) illustrate specific failure modes of baselines that explain the quantitative advantages. Imagine360 and Argus suffer from low visual quality due to their reliance on older video backbones (AnimateDiff and SVD, respectively). ViewPoint, which uses the same Wan2.1-14B backbone as 360Anything, always places the conditioning video at the center of the output, leading to severely rotated panoramas when the input has significant camera motion—the paper shows distorted people and buildings as a result. 360Anything generates consistently upright, canonicalized panoramas regardless of input camera orientation.

The paper also demonstrates 3D scene reconstruction from generated panoramic videos (Figure 6, Figure 10). Using rig-based COLMAP on the generated panoramic video followed by 3D Gaussian Splatting, 360Anything's outputs enable novel view synthesis and free-viewpoint exploration of reconstructed 3D scenes. This is a strong qualitative demonstration of multi-view consistency: if the generated panorama frames were geometrically inconsistent, the 3D reconstruction would fail or produce broken geometry. The successful 3DGS reconstruction therefore serves as an implicit geometric consistency metric that complements the explicit quantitative metrics.

Results on challenging inputs (Figures 11, 12, 14) demonstrate robustness: large motion videos, AI-generated inputs, and cases where state-of-the-art camera estimators fail (MegaSaM producing incorrect poses or FoV) all produce reasonable panoramas from 360Anything while causing catastrophic failures in channel-concatenation baselines. The paper specifically shows (Figure 14) that when MegaSaM predicts incorrect roll angles or overly small FoV, Argus generates broken results while 360Anything succeeds, providing direct evidence for the claimed robustness advantage.

Zero-Shot Camera Calibration

The camera calibration experiments (Tables 3 and 4) provide analytical evidence for the model's learned geometric understanding. On FoV estimation (Table 3), averaged across NYUv2, ETH3D, and iBims-1, 360Anything achieves a mean absolute error of 4.93°, outperforming several supervised methods (Perspective Fields: 9.79°, WildCam: 7.00°, LeReS: 15.35°, UniDepth: 9.72°) and lagging behind only the most recent state-of-the-art methods DUSt3R (4.06°) and MoGe (2.91°). On camera pose estimation (Table 4), 360Anything achieves roll error of 0.87° and pitch error of 2.56° on MegaDepth, compared to GeoCalib's 0.36° and 1.94°—within ~0.5° on roll and ~0.6° on pitch.

The calibration results are not claimed as a practical application but as mechanistic validation. The paper notes that 360Anything was trained only for panorama generation ("over 90% of our training images are indoor scenes," creating a domain gap with outdoor ETH3D and iBims-1 datasets), yet it develops accurate camera estimation as an emergent capability. This demonstrates that the model learns a genuine understanding of perspective-to-equirectangular geometry rather than memorizing a fixed mapping.

The FoV estimation pipeline uses exhaustive search over camera parameters to find the values that minimize the reconstruction error between the input image and a perspective crop of the generated panorama. This means the calibration accuracy is bounded by the quality of the generated panorama—if the generated panorama has geometric errors, the estimated calibration parameters will be inaccurate. The fact that calibration accuracy is competitive with dedicated methods suggests that the generated panoramas are geometrically faithful.

Ablation Studies and Robustness Checks

Circular Latent Encoding vs. blended decoding vs. vanilla zero-padding (Table 5, Figure 7). CLE achieves a Discontinuity Score of 3.87 on images, compared to 5.29 for blended decoding (the Argus inference-time fix) and 9.92 for vanilla zero-padding encoding. On videos, the reductions are from 35.52 (vanilla) to 19.84 (blended) to 13.28 (CLE). The qualitative comparison in Figure 7 shows that blended decoding "blurs" the seam but introduces gray line-like artifacts, while CLE produces genuinely seamless boundaries with no visible discontinuity. Importantly, CLE introduces zero inference overhead—unlike rotated denoising, which requires running multiple generations with different cyclic shifts.

Training with vs. without camera augmentation (Table 6). Training without camera augmentation—always using the standard viewpoint (90°, 0°, 0°)—actually produces worse results even when evaluated on that same viewpoint: FID increases from 8.0 to 8.4, and FAED increases from 9.8 to 10.4. This is a non-obvious finding: exposing the model to diverse camera parameters during training improves its performance on the standard viewpoint, suggesting that varied training forces the model to learn generalizable geometric reasoning rather than overfitting to a single projection mapping. The paper also evaluates robustness to input view variations: when changing FoV from 90° to 30°, FID increases by +2.2 (vs. +2.4 for the no-augmentation model and +3.4 for the channel-concatenation model with ground-truth camera). When changing pitch and roll, 360Anything shows similar or better robustness than the channel-concatenation model that has access to ground-truth camera metadata.

Canonical vs. non-canonical training for video (Table 7). Training without canonicalization (placing the conditioning view at the center of the output) makes reconstruction easier—PSNR and LPIPS improve—but significantly degrades visual quality: FVD increases from 470.8 to 559.5 on real trajectories, and VBench Imaging Quality drops from 0.5437 to 0.4689. The paper explains this as the model being forced to learn different spherical distortion patterns for each possible gravity direction, which degrades overall generation quality compared to learning a single canonical mapping. This ablation validates the canonical coordinate constraint as a critical design choice.

Real-world vs. simulated-only camera trajectories for video training (Figure 13). Training only with simulated linear-motion trajectories causes the model to fail on in-the-wild videos with complex camera motion: the generated panorama frames have changing gravity directions and fail to maintain canonical output. Adding real-world trajectories (20% of training samples, extracted from Rockwell et al. and ScanNet++) enables the model to generalize to complex, natural camera motions. The paper argues that simulated linear motion lacks the diversity of real handheld or vehicle-mounted camera movement, and that the specific trajectories present in internet videos cannot be captured by simple parametric models.

Robustness to input view variations (Table 6). Beyond the camera augmentation ablation, Table 6 reports performance across a range of conditioning viewpoints: (30°, 0°, 0°), (60°, 0°, 0°), (90°, ±30°, 0°), (90°, ±30°, ±5°). The average degradation across all variations is +0.98 for FID and +1.38 for FAED, which is comparable to the channel-concatenation model with ground-truth camera (+1.17 and +1.32, respectively) and substantially better than the no-augmentation model (+6.48 and +2.42). This demonstrates that 360Anything's implicit camera estimation generalizes across a wide range of input configurations without requiring explicit calibration.

Sequence concatenation vs. channel concatenation (Table 6, Appendix B.1). The paper compares 360Anything's sequence concatenation approach against a channel-concatenation variant that uses ground-truth camera metadata for pixel-aligned projection. The channel-concatenation variant achieves slightly better FID (7.7 vs. 8.0) and FAED (9.9 vs. 9.8) on the standard viewpoint, but 360Anything substantially outperforms it on robustness to viewpoint variation (average degradation of +0.98 vs. +1.17 for FID) and does so without requiring any camera metadata. Appendix B.1 (Figure 14) shows failure cases where channel-concatenation models break completely when external camera estimators fail—a robustness dimension where 360Anything has a categorical advantage since it has no external dependency to fail.

Negative result: ReST-EM revision training (Appendix K of the full paper structure, noted in prior sections for the revision model). While the main 360Anything paper focuses on generation rather than revision, the related context indicates that attempts to use ReST-EM-style iterative training to further optimize the generation pipeline backfired, with performance degrading substantially with sequential revisions. This is a negative result that highlights the sensitivity of training procedures to data distribution and the difficulty of bootstrapping from model-generated data.

Critical Assessment

The experiments generally support the paper's central claims, but several qualifications and gaps are worth noting.

Claim: 360Anything achieves state-of-the-art performance without camera metadata. This is strongly supported by Tables 1 and 2. The image results show clear improvements over CubeDiff across most metrics, with the near-50% FAED reduction being the strongest evidence. The video results are even more decisive, with ~43% FVD improvement over ViewPoint and improvements across all metrics. The fact that baselines receive ground-truth camera information while 360Anything does not makes this comparison conservative.

However, the image evaluation has a limitation: the standard protocol uses perspective crops to compute FID, KID, and CLIP-FID, which may not fully capture 360-degree coherence. FAED, the only full-panorama metric, shows the largest improvement, suggesting that the perspective-crop metrics may understate 360Anything's advantage. Conversely, it's possible that perspective-crop metrics are less sensitive to certain failure modes (e.g., repeated patterns, stitching artifacts that aren't visible in individual crops) that would be apparent in a 360° viewer. The paper's qualitative comparisons partially address this but a user study or immersive evaluation would be more convincing.

The video reproduction caveat deserves attention: the exact Argus evaluation split is unavailable, and metrics are reproduced based on communication with the authors. The paper validates the reproduction by running Argus on the reproduced split and confirming metrics are comparable to the original paper (Table 2, starred entries), which is good practice. However, the +0.52 PSNR difference between Argus-original (21.83) and Argus-reproduced (22.35) suggests the reproduced split may be slightly easier, which could modestly advantage 360Anything in the comparison.

Claim: The model learns accurate geometric reasoning without supervision. The camera calibration experiments (Tables 3 and 4) provide compelling evidence for emergent geometric understanding. However, there are important caveats. First, the calibration is performed via exhaustive search over the generated panorama, meaning calibration accuracy is bounded by generation quality—if the model generates a geometrically inaccurate panorama, the calibration will be wrong regardless of whether the model "knows" the correct parameters. The paper is transparent about this but it limits the interpretability of the calibration results as a pure measure of geometric knowledge.

Second, the calibration experiments use datasets that are out-of-distribution for training (ETH3D and iBims-1 are outdoor, while ~90% of training is indoor synthetic), and performance degrades on these datasets (ETH3D mean error 5.68° vs. NYUv2 3.90°). This suggests the learned geometry is somewhat domain-specific—not surprising for a data-driven approach, but it qualifies the claim of general "geometric understanding."

Third, the calibration does not measure yaw (azimuthal) accuracy explicitly, which is the most critical parameter for panorama generation (determining horizontal placement on the canvas). The paper reports only roll and pitch for pose, and FoV estimation. Yaw estimation would require a different setup (likely correlating the input with generated content to find the best azimuthal shift), and its absence is a notable gap in the geometric evaluation.

Claim: Circular Latent Encoding eliminates the root cause of seam artifacts. Figure 3 provides convincing mechanistic evidence: the shifted-latent diagnostic clearly shows that VAE zero-padding causes boundary discontinuities. Table 5 shows substantial DS improvements. However, DS is a quantitative metric that may not fully capture perceptual seam quality—a visible seam that is technically "low discontinuity" could still be objectionable. The qualitative comparison in Figure 7 compensates for this but is limited to a few examples.

An additional experiment that would strengthen this claim: comparing CLE against rotated denoising (the most common inference-time fix, not evaluated in Table 5). The paper only compares against blended decoding (from Argus) and vanilla encoding. Since rotated denoising is the standard approach in PanoDiffusion and related work, its absence from the seam ablation is a gap. It's possible that rotated denoising on top of CLE would further improve results, or that rotated denoising alone would close the gap—the paper doesn't provide evidence either way.

Claim: Sequence concatenation enables geometry-free generation that generalizes better than projection-based methods. Table 6 provides the core evidence: sequence concatenation (360Anything) shows similar or better robustness to viewpoint variation compared to channel concatenation with ground-truth camera, despite lacking camera metadata. However, the robustness evaluation only tests static camera variations (single FoV, pitch, roll values) rather than the complex, time-varying trajectories that appear in video. The video robustness is demonstrated qualitatively (Figures 11, 13, 14) but not quantified in terms of, e.g., PSNR variance across different input camera trajectories. A systematic sweep of video-level camera variations (varying trajectory speed, complexity, pitch range) with quantitative metrics would more rigorously test the claimed robustness advantage.

Missing ablation: difficulty-conditioned or adaptive allocation. The paper does not analyze whether performance varies systematically with input difficulty—e.g., how well does the model handle inputs with very narrow FoV (<30°), extreme pitch (>60°), or unusual scene layouts? The camera augmentation ablations cover a range of parameters but don't stratify results by difficulty tier. Since the model is data-driven and trained primarily on synthetic indoor scenes, there may be systematic failure modes on certain types of inputs that the aggregate metrics obscure.

Missing baseline: a variant of CubeDiff or ViewPoint augmented with CLE. Since the paper argues that CLE addresses a general VAE latent-space problem, it would be informative to apply CLE to a baseline method to determine how much of the performance gap is attributable to CLE vs. the sequence concatenation architecture. If applying CLE to CubeDiff significantly improved its FAED (since FAED measures full-panorama quality where seams are most visible), it would suggest that the seam fix is a separable contribution that partially explains the state-of-the-art results. The paper does not perform this cross-method ablation, making it difficult to attribute the performance improvements between the geometry-free paradigm and the latent-space fix.

Data diversity limitation. The image model is trained on datasets where ~90% of the data is synthetic indoor scenes from Structured3D. While the paper demonstrates generalization to AI-generated images (Figure 9) and outdoor scenes (SUN360), the training distribution is heavily skewed toward a specific visual domain. The strong SUN360 results suggest reasonable generalization, but it's unclear whether performance would hold on more diverse real-world imagery (e.g., natural landscapes, crowded urban scenes, nighttime conditions). The video model uses YouTube data and likely has better coverage, but the specific filtering and subset selection criteria from Argus may introduce their own biases.

Computational cost not addressed. The paper does not compare training cost, inference latency, or memory requirements against baselines. While 360Anything has a simpler pipeline (no camera estimation, no inference-time seam handling), it fine-tunes larger base models (FLUX, Wan2.1-14B) than some baselines (Argus fine-tunes SVD, Imagine360 fine-tunes AnimateDiff). The sequence concatenation approach also means the DiT processes longer sequences (panorama tokens + perspective tokens), which increases attention cost quadratically. A FLOPs or wall-clock comparison would clarify whether the architectural simplicity translates to computational efficiency advantages.

3D reconstruction as an evaluation metric. The 3DGS reconstruction results (Figure 6, Figure 10) are compelling qualitative demonstrations but are not quantified. Metrics like PSNR/SSIM on held-out test views from the 3DGS reconstruction would provide quantitative evidence for multi-view consistency. The paper shows reconstructions from a small number of examples (indoor rooms from RealEstate10K), and the success of 3D reconstruction likely depends on scene type and camera motion—static indoor scenes with simple geometry are the easiest case. Broader evaluation across scene types with quantitative metrics would strengthen the claim that generated panoramas enable downstream 3D tasks.

Absence of confidence intervals or statistical testing. None of the benchmark results report standard deviations, confidence intervals, or statistical significance tests. The test sets are relatively small (Laval Indoor and SUN360 each have a limited number of scenes; the video test set is 101 videos), so metric differences may be subject to variance. For instance, the FID difference between 360Anything (8.0) and CubeDiff (9.5) on Laval Indoor may or may not be statistically significant depending on the number of generated samples and the variance of the FID estimator, which is known to be sensitive to sample size.

Overall, the experiments provide strong support for the paper's core architectural claims—sequence concatenation can replace explicit camera projection, and CLE solves seam artifacts at the source—but the evidence for the "geometry-free" paradigm as universally superior to geometry-aware approaches is qualified by the limited domain diversity, the absence of certain head-to-head comparisons, and the lack of statistical rigor in the reported metrics. The results convincingly demonstrate that geometry-free generation works and works well, but the paper's strongest claims about robustness and generalization would benefit from broader evaluation across diverse, real-world conditions and more systematic failure mode analysis.

6. Limitations and Trade-offs

Limitation 1: Training Data Is Heavily Skewed Toward Synthetic Indoor Scenes

The assumption or constraint. The image model is trained on Structured3D (synthetic indoor renderings) for roughly 90% of its training data (Appendix A.1), and the paper explicitly states this: "over 90% of our training images are indoor scenes... thus creating a large domain gap with the outdoor ETH3D and iBims-1 datasets" (Section 4.3). Both the image and video models inherit the visual biases of their training distributions—synthetic rooms, controlled lighting, limited material and texture diversity.

The consequence. The model's generalization to outdoor scenes, natural landscapes, nighttime conditions, dynamic weather, crowds, or scenes with complex object interactions is not systematically evaluated. The camera calibration experiments reveal this gap quantitatively: FoV estimation on outdoor ETH3D (mean error 5.68°) and iBims-1 (5.21°) is substantially worse than on indoor NYUv2 (3.90°). While the SUN360 benchmark includes outdoor scenes and 360Anything performs well (Table 1), SUN360 is a curated dataset that may not capture the full diversity of in-the-wild outdoor imagery. A practitioner deploying this method on user-generated outdoor photographs or videos may encounter failure modes (e.g., incorrect horizon estimation, distorted natural textures, hallucinated indoor-like structures in outdoor contexts) that the benchmarks do not capture.

What evidence exists in the paper. Table 3 shows a clear indoor-outdoor performance gap in zero-shot FoV estimation: NYUv2 (indoor) mean error of 3.90° vs. ETH3D (outdoor) 5.68° and iBims-1 (outdoor) 5.21°. The paper acknowledges this as a domain gap but does not analyze how it affects generation quality specifically for outdoor scenes. The qualitative examples (Figures 1, 4, 5, 6, 9–15) are predominantly indoor or semi-indoor scenes with simple geometry. The 3D reconstruction demonstrations (Figures 6, 10) use only static indoor rooms from RealEstate10K, a domain well-represented in Structured3D training data. There is no outdoor 3D reconstruction result, no night scene, and no scene with significant texture variation (forests, beaches, urban crowds).

Mitigation status. The paper does not address this limitation beyond noting the domain gap in the calibration context. The video model uses YouTube data (360-1M) which likely contains more diverse outdoor content, but the specific filtering and subset selection criteria from Argus may bias this distribution as well. Future work on expanding training data diversity—particularly to include real-world outdoor panoramas with varied lighting, weather, and scene complexity—is implied by the broader "scale up model and data" vision (Section 1) but not explicitly discussed as a limitation.

Limitation 2: The Canonicalization Constraint Simplifies Generation but Creates a Latency-Versus-Flexibility Tradeoff

The assumption or constraint. The canonical coordinate constraint (Section 3.2) requires the model to always generate panoramas in a gravity-aligned upright orientation, regardless of the input's camera pose. This means the model must implicitly estimate the input's orientation (pitch and roll) and reposition the content onto the canonical canvas. The paper frames this as a simplification—the model only needs to learn one spherical distortion pattern—but it introduces a serial dependency in the generation pipeline: the model must infer camera pose before or during generation, and that inference may be imperfect.

The consequence. First, there is no mechanism for the user to specify the desired output orientation. If a user wants a panorama in a non-canonical orientation (e.g., artistic tilt, a specific composition where the horizon is intentionally not horizontal), the model cannot accommodate this—it will always produce an upright output. This is a loss of controllability compared to methods that accept explicit camera parameters.

Second, the implicit camera pose estimation may fail on unusual inputs where visual gravity cues are ambiguous or contradictory (e.g., tilted architecture, abstract art, extreme close-ups with no horizon, underwater scenes, aerial photography looking straight down). In such cases, the model might mis-estimate the gravity direction and produce a panorama that is technically "upright" according to its (incorrect) estimate but geometrically wrong. The paper demonstrates robustness to pitch up to ±60° and roll up to ±15° in training augmentation (Section 4.1), but does not test extreme or ambiguous cases.

Third, the canonicalization constraint forces the model to solve two problems simultaneously (camera pose estimation and panorama generation) rather than decomposing them. This coupling means that errors in pose estimation propagate to generation quality in ways that are difficult to diagnose or correct.

What evidence exists in the paper. The ablation in Table 7 shows that canonical training improves visual quality metrics (FVD reduces from 559.5 to 470.8 on real trajectories, VBench Imaging Quality improves from 0.4689 to 0.5437) compared to non-canonical training where the input is always centered. This validates the choice for quality, but the ablation treats the canonical/non-canonical decision as binary without exploring intermediate alternatives (e.g., generating the panorama in a coordinate system aligned with the input's orientation, then rotating to canonical as a post-processing step). The robustness evaluation (Table 6) tests pitch and roll variations but only in the context of generation quality, not explicitly measuring whether the model's implicit pose estimates are correct for each variation.

Mitigation status. The paper does not discuss this tradeoff or propose mechanisms for user-controllable orientation. Future work could explore conditioning the model on a target orientation parameter (enabling user-specified tilt) or decoupling pose estimation from generation (e.g., using the model's implicit calibration capability from Section 4.3 to estimate pose first, then conditioning generation on that estimate). Neither approach is implemented or suggested in the paper.

Limitation 3: No Systematic Characterization of Failure Modes on Out-of-Distribution Inputs

The assumption or constraint. The model's architecture (sequence concatenation with DiT) assumes that the perspective-to-panorama mapping can be learned purely from data. This assumption holds when test inputs are drawn from a distribution similar to the training data, but the paper provides no systematic analysis of when and how the model fails on inputs that diverge significantly from the training distribution. The training distribution is characterized by specific FoV ranges (30°–120°), pitch ranges (−60°–60°), roll ranges (−15°–15°), and camera motion types (80% simulated linear + 20% real trajectories).

The consequence. A practitioner cannot predict whether 360Anything will work on their specific inputs without falling within these training ranges. Several failure modes are plausible but uncharacterized:

Extremely narrow FoV (<30°): The paper shows degradation as FoV decreases (Table 6: FID increases by +2.2 at 30° vs. 90°), but does not test below 30°. Very narrow FoV inputs (e.g., telephoto shots, cropped regions) provide minimal visual context for the model to infer scene layout, potentially leading to hallucinated or incoherent panoramas.
Extreme pitch (>60°): Looking straight up or down changes the spherical distortion pattern dramatically. The model was trained with pitch in [−60°, 60°], and behavior outside this range is unknown.
Abstract or non-photorealistic inputs: The training data consists of photorealistic (synthetic and real) imagery. Inputs like paintings, sketches, diagrams, or heavily stylized photographs may not contain the visual gravity cues and perspective geometry that the model relies on for implicit calibration.
Multi-modal or ambiguous scenes: Inputs where the scene content is consistent with multiple possible camera orientations (e.g., a close-up of a textured surface without clear horizon, or a scene with mirrored symmetry) may cause the model to produce inconsistent or oscillating panoramas.

Beyond FoV and pose, the model's behavior on inputs with unusual content (e.g., first-person video with rapid head motion, scenes with transparent or reflective surfaces, macro photography, endoscopic imagery) is completely unexplored.

What evidence exists in the paper. The paper provides several qualitative examples of generalization—AI-generated inputs (Figures 9, 12), large motion videos (Figure 11), and challenging videos where off-the-shelf camera estimators fail (Figure 14)—but these are curated success cases, not a systematic failure analysis. The robustness evaluation in Table 6 tests a finite set of parameter variations within the training range and reports aggregate metrics (average FID and FAED changes), which can obscure bimodal failure patterns where some inputs succeed and others fail catastrophically. The paper does not report per-example worst-case behavior, failure rates, or qualitative categories of failure modes.

Mitigation status. The paper does not attempt to characterize failure modes or provide guidance on input conditions where the model is likely to fail. A systematic study of out-of-distribution generalization—perhaps by evaluating on inputs with progressively more extreme parameter values and measuring when performance degrades unacceptably—is a clear direction for future work that the paper does not explicitly flag.

Limitation 4: High Computational Cost and Video Length Constraints Limit Practical Deployment

The assumption or constraint. 360Anything fine-tunes large-scale pre-trained models: FLUX.1-dev (~12B parameters) for images and Wan2.1-14B (14B parameters) for video. Video generation is particularly expensive: each training sample is an 81-frame ERP video at 512×1024 resolution. The paper trains the video model in a staged fashion (10k steps at 256×512, then 10k steps at 512×1024) and acknowledges that the 81-frame limit is due to limited compute: "Due to the high resolution of panorama data (an ERP video has 8× number of pixels compared to a normal perspective video) and the limited compute, our current video model can only handle videos with 81 frames" (Appendix C).

The consequence. First, inference cost is substantial. A single video generation requires 50 sampling steps of Wan2.1-14B on a sequence that includes both the conditioning video tokens and the noisy panorama tokens, with attention computed over the full concatenated sequence. The quadratic scaling of attention with sequence length means that processing ERP-resolution latents (which have 8× the pixels of a perspective video at the same per-frame resolution, before accounting for the extra conditioning tokens) is significantly more expensive than standard video generation. The paper provides no inference timing, memory usage, or FLOPs comparison against baselines, making it impossible for a practitioner to estimate deployment costs or determine whether the quality improvements justify the computational expense.

Second, the 81-frame limit severely constrains applications. At 24 FPS, 81 frames represents only ~3.4 seconds of video. For tasks like generating a full 360° tour of a room, creating immersive content for VR experiences, or reconstructing large-scale 3D environments, longer video sequences are necessary. The paper notes that "a larger context window will enable larger-scale 3D world generation" (Appendix C), but provides no solution within the current framework.

Third, the batch size constraints during training (batch size 64 for video) reflect the memory intensity of processing long ERP video sequences. This limits the effective number of training steps given a fixed compute budget and may constrain the model's ability to benefit from larger-scale training.

What evidence exists in the paper. The paper is transparent about the 81-frame limit in Appendix C but does not quantify inference cost anywhere. The training hyperparameters (batch size 64, two stages of 10k steps each for video) are reported (Section 4.2, Appendix A.2) but not contextualized in terms of GPU-hours or total compute. The paper does not compare inference latency against baselines like Argus (which uses SVD, a smaller model) or Imagine360 (which uses AnimateDiff, also smaller). Since 360Anything outperforms these baselines while using a larger backbone, the performance gains are partially attributable to model scale—a factor that is not isolated from the architectural innovations.

Mitigation status. The paper suggests several directions for future work to address video length: "combining 360Anything with recent progress in long video generation that distills bi-directional DiTs to causal autoregressive DiTs" (Appendix C). It also mentions the challenge of panorama upsampling and notes that existing perspective video upsamplers fail on ERP data. These are forward-looking suggestions with no implementation in the current work. A practitioner currently cannot generate videos longer than 81 frames or upsample to higher resolutions without developing these capabilities independently.

Limitation 5: The Difficulty Estimation and Adaptive Allocation Are Not Addressed

The assumption or constraint. The paper presents 360Anything as a general-purpose solution that applies the same inference procedure—50 sampling steps, fixed CFG scales, fixed timestep shifting—to all inputs regardless of their characteristics. Unlike the prior example paper (which performs compute-optimal test-time scaling by allocating different strategies based on estimated prompt difficulty), 360Anything does not attempt to adapt its generation strategy based on input properties such as FoV, scene complexity, motion magnitude, or estimated camera pose.

The consequence. The model likely wastes computation on "easy" inputs (e.g., wide FoV, static scenes with simple geometry, well-lit indoor environments) where fewer sampling steps or lower CFG scales might suffice, while under-allocating computation to "hard" inputs (e.g., narrow FoV, complex outdoor scenes, rapid camera motion) where more steps or different guidance parameters might improve quality. The robustness evaluation (Table 6) shows that generation quality degrades as FoV decreases (FID +2.2 at 30° vs. 90°) and as pitch increases (FID +0.6 to +0.9 at ±30° pitch). If the model could detect these challenging conditions and allocate additional computation—e.g., more sampling steps, higher CFG, or iterative refinement—it might achieve more uniform quality across input variations.

More fundamentally, the lack of difficulty estimation and adaptive allocation means that the model cannot signal when it is likely to fail. For a deployment where some fraction of inputs may be out-of-distribution or inherently ambiguous, a practitioner has no mechanism to flag low-confidence generations for human review or to fall back to an alternative method. The model always produces an output, and that output may appear plausible even when geometrically incorrect—a particularly dangerous failure mode for applications like 3D reconstruction where downstream errors can compound.

What evidence exists in the paper. The paper provides no analysis of per-sample performance variation, no difficulty estimation mechanism, and no adaptive inference strategy. The robustness evaluation (Table 6) reports mean metrics across test sets for different parameter values, which averages over easy and hard cases. If performance on narrow-FoV or high-pitch inputs is highly bimodal—some succeed, some fail catastrophically—the mean FID/FID change would not reveal this pattern. The calibration experiments (Section 4.3) implicitly estimate camera parameters from generated panoramas, but this is done offline for evaluation, not online for adaptive inference.

Mitigation status. The paper does not discuss adaptive allocation, difficulty estimation, or confidence estimation. The architecture provides no uncertainty quantification mechanism (e.g., generating multiple samples and measuring consistency, using the diffusion model's own probability estimates). This is a significant gap for practical deployment and one that future work could address by, for example, training a lightweight "difficulty predictor" on the conditioning tokens or using the variance across multiple generated samples as a confidence signal. None of these approaches is suggested in the paper.

Limitation 6: Sequence Concatenation Increases Inference Cost Quadratically with Token Count and Is Not Compared Against More Efficient Conditioning Mechanisms

The assumption or constraint. The sequence concatenation mechanism (Section 3.2) appends all perspective tokens directly to the panorama token sequence, meaning the DiT's self-attention computes pairwise interactions across a sequence of length N_pers + N_equi. Since self-attention scales as O(L²) with sequence length L, the computational cost of including perspective tokens is not additive—it increases the attention cost for every token in the sequence. For the image model at 1024×2048 resolution, the latent grid is 128×256 = 32,768 tokens after patchification. Appending perspective tokens (at potentially similar resolution after VAE encoding) effectively doubles the sequence length, quadrupling the attention cost per DiT layer compared to generating a panorama without conditioning.

The consequence. There is a fundamental tradeoff in the sequence concatenation design: it eliminates the need for camera calibration and enables learned geometric reasoning, but it does so at the cost of significantly increased inference computation. The paper does not explore more efficient conditioning mechanisms that could reduce this cost while preserving the geometry-free property. For example:

Cross-attention conditioning: instead of concatenating tokens, use the perspective tokens as keys and values in a cross-attention layer that the panorama tokens attend to. This separates the conditioning cost from the generation cost: panorama token attention scales as O(L²_equi) for self-attention plus O(L_equi × L_pers) for cross-attention, which is more efficient when L_pers is large.
Compressed conditioning: encode the perspective input into a fixed-size representation (e.g., a learned bottleneck, a pooled feature vector) rather than a full token sequence, reducing the conditioning's contribution to attention cost.
Selective attention: restrict perspective-panorama attention to spatially relevant regions (e.g., using the model's own implicit calibration to focus attention on the estimated input location) rather than full all-to-all attention.

The absence of these comparisons means a practitioner cannot assess whether the performance benefits of sequence concatenation are specifically due to the all-to-all attention mechanism (which would be hard to replicate with more efficient conditioning) or whether similar results could be achieved with cheaper alternatives.

What evidence exists in the paper. The paper compares sequence concatenation against channel concatenation (Table 6, Appendix B.1) in terms of generation quality and robustness, but not in terms of computational efficiency. The channel concatenation variant uses pixel-aligned projection with ground-truth camera information, so it's not an apples-to-apples conditioning-efficiency comparison. The paper provides no FLOPs analysis, no latency measurements, and no memory profiling for any method. The training batch size differences (512 for image, 64 for video) hint at the memory cost of processing long sequences, but these are not broken down by the contribution of the conditioning tokens.

Mitigation status. The paper does not acknowledge this tradeoff or propose more efficient conditioning mechanisms. The architecture is presented as straightforward and effective, with no discussion of its computational scaling properties relative to alternatives. A practitioner implementing 360Anything inherits this cost structure without guidance on whether—or how—it could be reduced. Future work on efficient attention mechanisms for long sequences (sparse attention, linear attention, memory-efficient kernels) could mitigate this limitation, but the paper does not suggest this direction.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the conceptual foundation of panorama generation from geometry-as-input to geometry-as-output. Prior to 360Anything, the dominant paradigm for perspective-to-360° lifting treated camera calibration—Field-of-View, pitch, roll, yaw—as essential pre-processing that must be provided to the model, either from metadata or from external estimators. The model's job was to outpaint from a spatially-anchored starting point. 360Anything demonstrates that a general-purpose diffusion transformer, trained with sufficient data diversity and a simple sequence-concatenation mechanism, learns the spherical projection mapping internally and achieves better generation quality than methods that receive explicit geometric priors as input. This is a conceptual reframing, not an incremental improvement—it changes the fundamental question from "how do we improve camera estimation to get better panoramas?" to "how much geometric reasoning can we induce through data scale and architecture alone?"

The significance of this reframing extends beyond the specific task. It provides a concrete case study in the displacement of hand-designed inductive bias by learned representations, a pattern that has transformed other areas of computer vision (depth estimation, optical flow, view synthesis) but had not yet reached panorama generation—a domain where the geometric relationship between perspective and spherical coordinates is both mathematically precise and apparently critical. The paper's demonstration that a sequence-to-sequence transformer can recover this relationship from data, without spherical convolutions, cubemap representations, dual-branch U-Nets, or explicit projection layers, challenges the assumption that these specialized components are necessary. The quantitative evidence is striking: on SUN360, FAED—the only metric computed on the full ERP panorama—drops from 7.6 (CubeDiff) to 3.8, a nearly 50% reduction in geometric distortion. This is not a marginal gain from engineering improvements; it is a step-change that suggests the specialized architectures developed over years of research were solving a problem (pixel-aligned conditioning) that didn't need to exist.

The paper also resolves a concrete contradiction in prior work. Seam artifacts at ERP boundaries have been universally observed in panorama generation, and the field had converged on inference-time fixes—rotated denoising, circular decoder padding, blended decoding—as the solution space. The paper demonstrates through a simple diagnostic experiment (Figure 3a: encode seamless panorama, shift latent, decode, observe seam) that the root cause is zero-padding in the VAE encoder, a training-time phenomenon, not a generation-time sampling artifact. This reframes the problem from "how do we fix what the generator produces?" to "how do we ensure the training latents are topologically correct?" and provides a fix (Circular Latent Encoding) that eliminates the issue at the source with zero inference overhead. The 63% reduction in Discontinuity Score on video (35.52 → 13.28, Table 5) compared to the previous best alternative (blended decoding at 19.84) demonstrates that solving the right problem is substantially more effective than patching the wrong one.

Several research directions become more attractive as a result of this work:

Scaling-based approaches to geometric tasks. The paper provides evidence that geometric reasoning emerges from scale and data diversity rather than architectural priors. This strengthens the case for applying large-scale DiTs to other tasks where geometric alignment has been considered essential—multi-view stereo, 3D reconstruction from sparse views, video stabilization, and novel view synthesis from uncalibrated inputs.
Root-cause diagnostics in generative pipelines. The seam artifact analysis is a template for how to debug quality issues in latent generative models: don't assume the problem lives in the generator; check whether the latent representation itself is corrupted before diffusion ever touches it. This diagnostic methodology—isolating pipeline stages and testing for discontinuities—could be applied to other artifacts in image/video generation (tiling patterns, color shifts, temporal flicker).
Zero-shot calibration via generative models. The paper's finding that 360Anything achieves ~4.93° mean FoV estimation error (competitive with supervised methods) without any calibration training demonstrates that generative models can serve as implicit geometry estimators. This opens a path to using pre-trained generative models as calibration tools for downstream tasks, especially when dedicated calibration models are unavailable or unreliable on out-of-distribution data.

Conversely, some directions become less attractive:

Specialized spherical architectures for panorama generation. The strong performance of a generic DiT with sequence concatenation suggests that investments in spherical convolutions, dual-branch U-Nets, and custom cubemap representations for this task may have diminishing returns. Future architecture work in panorama generation should justify any added complexity against this simpler and more performant baseline.
Better camera estimators as a path to improved panoramas. The failure cases in Appendix B.1 (Figure 14), where state-of-the-art camera estimator MegaSaM fails and projection-based methods break while 360Anything succeeds, suggest that improving camera estimation accuracy is a fragile strategy for enhancing panorama generation—the estimation bottleneck is fundamentally unavoidable in the projection paradigm, and removing the bottleneck entirely (as 360Anything does) may be more robust than trying to make estimation more accurate.
Inference-time seam removal techniques. The paper's root-cause analysis shows that inference-time fixes (rotated denoising, blended decoding) treat symptoms rather than causes. The dramatic DS improvements from CLE suggest that future work on seam artifacts should focus on training-time latent-space fixes rather than developing more sophisticated post-processing.

Follow-Up Research This Work Enables

Systematic characterization of failure modes on out-of-distribution geometric configurations. The paper demonstrates robustness within its training augmentation ranges (FoV 30-120°, pitch ±60°, roll ±15°) but does not test beyond these boundaries. A follow-up study should systematically evaluate 360Anything on a held-out test set with progressively more extreme camera parameters—FoV from 5° to 150°, pitch from -90° to +90°, roll from -45° to +45°—and measure when generation quality degrades unacceptably (e.g., FID exceeds some threshold, or human raters detect geometric errors). The key question is whether the model's learned geometric reasoning extrapolates beyond its training distribution or collapses at the boundaries. This would establish safe operating ranges for deployment and identify specific failure modes (e.g., does extreme pitch cause the model to generate inverted scenes, or does it gracefully degrade by producing uncertain but plausible content?). The experiment should also test distributional shift in scene content: outdoor night scenes, aerial photography (looking straight down), underwater imagery, and abstract/artistic content with ambiguous gravity cues, all of which are absent from the primarily indoor-synthetic training data.

Cross-attention conditioning as a compute-efficient alternative to sequence concatenation. The paper's sequence concatenation design forces the DiT to compute self-attention over a combined sequence of length L_pers + L_equi, quadrupling attention cost relative to panorama-only generation when conditioning tokens are numerous. A follow-up should implement cross-attention conditioning: keep the panorama tokens in a self-attention sequence but add cross-attention layers where panorama tokens attend to a separately-encoded sequence of perspective tokens. This reduces the complexity to O(L_equi² + L_equi × L_pers) rather than O((L_equi + L_pers)²). The experiment should compare generation quality (FID, FAED, FVD) and inference latency at matched sampling budgets against the sequence-concatenation baseline, and ideally replicate the camera calibration analysis to determine whether cross-attention preserves the emergent geometric reasoning capability or whether global all-to-all attention is necessary for accurate implicit calibration. An additional ablation should test whether the 0.1 temporal offset trick (Appendix A.2) is equally effective in cross-attention or requires a different separation mechanism.

Measuring whether the model's implicit geometry is internally consistent across views. The camera calibration experiments (Tables 3 and 4) show that 360Anything's generated panoramas approximately match the input's camera parameters. But this measures accuracy of the output relative to the input, not internal consistency of the model's geometric understanding. A follow-up experiment should probe consistency: given a single input, generate the panorama multiple times (varying the noise seed), extract camera parameters from each generation via the same exhaustive search method, and measure the variance of estimated FoV, pitch, roll across seeds. If the model's geometric understanding is stable, variance should be low; if geometry is only loosely coupled to generation, variance could be high even when each individual generation looks plausible. This directly tests the paper's claim that the model "learns to establish accurate correspondence between the conditioning perspective input and the generated panorama image" (Section 4.3)—a claim that is currently supported only by aggregate accuracy metrics, not by per-sample consistency.

Applying Circular Latent Encoding to existing panorama generation pipelines. The paper argues that CLE addresses a general VAE latent-space problem, not a 360Anything-specific issue. If true, applying CLE to CubeDiff, ViewPoint, or Argus should improve their seam-related artifacts. A direct test: take CubeDiff (which uses a cubemap representation and a pre-trained VAE), replace its standard VAE encoding with CLE, retrain, and measure the Discontinuity Score on the Laval Indoor and SUN360 test sets. If CLE substantially reduces DS, it demonstrates that the latent-space discontinuity is indeed a separable, cross-method problem and that CLE is a general fix rather than a 360Anything-specific component. If CLE does not help CubeDiff (because CubeDiff's cubemap faces already have true boundaries, not wraparound ones), this would refine our understanding of when circular latent encoding is applicable—specifically, only for representations with spherical topology, not for cube-face representations with genuine edges.

Temporal extension: predicting camera trajectory from generated video as a self-consistency check. The current camera calibration analysis works only on single images. For video, the paper demonstrates 3D reconstruction quality qualitatively (Figures 6, 10) but provides no quantitative geometric consistency metric. A follow-up should exploit the temporal dimension: given a generated panoramic video, run structure-from-motion (COLMAP) on perspective crops from each frame and compare the recovered camera trajectory to the known trajectory used to create the conditioning perspective video (available in the Argus evaluation protocol). The error between recovered and ground-truth trajectory—measured as ATE (Absolute Trajectory Error) in meters and degrees—quantifies the multi-frame geometric consistency that is crucial for downstream applications. This experiment would also test whether the canonical coordinate constraint (Section 3.2) truly produces stable upright panoramas across frames or whether residual gravity drift accumulates over long sequences. The 81-frame limit is sufficient for SfM on short sequences; extending to longer videos via sliding window would test temporal consistency at scale.

Difficulty-adaptive inference for computation-constrained deployment. The paper applies the same inference procedure (50 steps, fixed CFG scales, fixed timestep shifting) to all inputs regardless of difficulty. A follow-up should develop a lightweight difficulty estimator that predicts, from the encoded perspective tokens alone, how many sampling steps are needed to reach acceptable generation quality. One approach: train a small MLP head on top of the frozen DiT's intermediate features at an early denoising step (e.g., step 5 of 50) to predict the FID or LPIPS of the final output. If such a predictor is accurate, the system could early-exit: for easy inputs (wide FoV, simple geometry, well-lit), run only 20 steps; for hard inputs, allocate the full 50. The experiment should measure the accuracy-versus-compute Pareto frontier compared to uniform-step baselines, and validate that the predictor does not introduce systematic failures (e.g., consistently underestimating difficulty for narrow-FoV outdoor scenes). This directly addresses the practical limitation that 360Anything's inference cost is high and unadapted to input variability.

Practical Applications and Downstream Use Cases

Uncalibrated user-generated content to immersive VR environments. The most direct application is converting casual smartphone photos and videos into 360° content for virtual reality platforms without any user-provided camera metadata. The paper's geometry-free design means a user can capture a single perspective image or a short video clip, feed it to 360Anything, and receive a seamless, gravity-aligned panorama ready for a 360° viewer. The practical significance lies in eliminating the calibration barrier: prior methods required users to know or estimate their camera's FoV and orientation, which excluded the vast majority of casual content creators. The quantitative robustness evidence—average FID degradation of only +0.98 across diverse camera parameters (Table 6), compared to +6.48 for a model without camera augmentation—indicates that the system genuinely handles the variable, unknown camera configurations typical of user-generated content. The 3D reconstruction results (Figures 6, 10) further enable downstream applications: from a monocular video walkthrough of a room, 360Anything can generate a complete spherical video, from which 3D Gaussian Splatting reconstructs an explorable 3D model.

Large-scale batch processing for 360° dataset creation. For organizations building datasets of 360° imagery—for training embodied AI agents, evaluating VR systems, or creating virtual tours—the pipeline simplification matters at scale. Prior projection-based methods require per-video camera estimation, which adds computational overhead and introduces a failure point for each sample. 360Anything eliminates this pre-processing stage entirely: perspective videos can be fed directly to the model without calibration. The paper's demonstration that 360Anything outperforms Argus even though Argus receives ground-truth camera information (Table 2: FVD 483.4 vs. 1020.7 on real trajectories) suggests that dataset quality improves while pipeline complexity decreases—a rare combination. For a dataset of 10,000 videos, avoiding per-video COLMAP or GeoCalib processing (each taking minutes to hours) represents a significant cost saving, and the elimination of estimator failure modes (illustrated in Figure 14) improves yield on challenging footage. The remaining bottleneck is the DiT inference cost, which future work on efficient attention mechanisms (see follow-up above) would address.

Initialization for 3D scene reconstruction from casual capture. The paper's 3D Gaussian Splatting results from generated panoramic videos (Figures 6, 10) point to a pipeline where 360Anything serves as a scene completion preprocessor for 3D reconstruction. A user captures a short monocular video covering a partial view of a room—perhaps 120° of a 360° scene. 360Anything outpaints the remaining 240° as a panoramic video. Rig-based COLMAP on this synthetic panoramic video produces camera poses and a sparse point cloud covering the full scene. A 3DGS model trained on the generated panoramas produces a complete, explorable 3D model. The key practical advantage is that this pipeline works even when the original capture is insufficient for traditional SfM—narrow baseline, limited viewpoint coverage, or texture-poor regions—because the generated outpainting fills in the geometrically consistent missing views. The paper's LPIPS of 0.0468 on input-preserved regions (Table 2) indicates that the generated content closely matches the real captured content in the observed regions, while the qualitative 3DGS results demonstrate that the out-painted regions are sufficiently geometrically consistent for reconstruction. This has immediate applications in real estate virtual tours, cultural heritage documentation, and AR/VR content creation where capturing full 360° coverage with physical cameras is impractical.

When to Prefer This Method

The paper does not position 360Anything against named alternatives in a formal decision framework; it consistently argues that the geometry-free paradigm is categorically preferable to projection-based methods for in-the-wild deployment. However, several implicit tradeoffs emerge from the results that can inform practical decisions:

Prefer 360Anything when camera metadata is unavailable, noisy, or expensive to obtain. This is the core use case the paper designs for. The zero-shot generalization to diverse camera parameters (Table 6) and the failure of projection-based methods when camera estimators mispredict (Figure 14) make 360Anything the obvious choice for uncalibrated inputs. The robustness evidence is strongest for FoV in 30°–120°, pitch in ±60°, and roll in ±15° (the training ranges), with unknown behavior outside these bounds.
Prefer 360Anything when output must be in a canonical, gravity-aligned orientation without manual correction. The canonical coordinate constraint (Table 7: FVD reduction from 559.5 to 470.8) ensures upright output regardless of input tilt. This is valuable for applications where downstream processing (3D reconstruction, VR display) expects upright content. Prior methods without canonicalization produce rotated outputs (Figure 5: ViewPoint's distorted people and buildings) that require manual or automated reorientation.
Consider projection-based methods (with ground-truth calibration) when inference cost is the primary constraint and camera parameters are known with high confidence. The paper does not compare inference latency, but sequence concatenation increases attention cost quadratically with the combined token count. If a deployment has accurate camera metadata (e.g., rendering synthetic data from 3D assets, or using calibrated camera rigs), channel-concatenation methods with smaller backbones (e.g., Argus fine-tuning SVD) may offer better FLOPs-to-quality ratios. The channel-concatenation variant with ground-truth camera achieves very slightly better FID on the standard viewpoint (7.7 vs. 8.0, Table 6), suggesting that when calibration is perfect and the model is optimized for that specific case, explicit geometry still provides a marginal advantage—but this advantage is dwarfed by the robustness penalty when calibration is imperfect, which is the common real-world case.
Prefer 360Anything when seam artifacts are unacceptable without post-processing. CLE eliminates seams at the training stage with zero inference overhead (DS 3.87 vs. 5.29 for blended decoding, Table 5), while inference-time fixes introduce computational cost (rotated denoising requires multiple generation passes) or visual artifacts (blended decoding produces gray lines, Figure 7). For applications where seam quality is critical—VR content, professional panoramic photography—the training-time fix is categorically superior.
The choice of image vs. video variant depends on temporal requirements. The video model (81-frame limit at 512×1024 or 256×512) and image model (2048×1024 resolution, single frame) are distinct checkpoints with different capabilities. The image model produces higher-resolution output; the video model produces temporally consistent sequences. A practitioner needing more than ~3.4 seconds of panorama video must accept the current frame limit or wait for long-video extensions the paper identifies as future work (Appendix C).