Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model

ArXiv: 2312.13252

🎯 Pitch

A diffusion model trained with log-depth parameterization and field-of-view conditioning slashes zero-shot depth error by up to 70% without any domain-specific heads, proving that explicit camera geometry can resolve scale ambiguity far better than specialized architectures.

---

1. Executive Summary

This paper proposes Diffusion for Metric Depth (DMD), a generic diffusion-based model for zero-shot metric depth estimation that jointly handles indoor and outdoor scenes without domain-specific architectural components. DMD introduces three key mechanisms — log-scale depth parameterization (e.g., mapping depths via log rather than linear scaling to balance representation capacity across shallow indoor and deep outdoor ranges), field-of-view (FOV) conditioning (e.g., using tan(θ/2) as an input signal to disambiguate the depth-scale ambiguity caused by unknown camera intrinsics), and synthetic FOV augmentation during training (e.g., cropping or uncropping images with noise padding to simulate diverse camera intrinsics beyond those in the training datasets) — combined with v-parameterization for efficient inference requiring as few as one or two denoising steps. Trained on a mixture of NYU, KITTI, nuScenes, and Taskonomy, DMD achieves a 25% reduction in relative error on zero-shot indoor benchmarks and a 33% reduction on zero-shot outdoor benchmarks over the prior state-of-the-art ZoeDepth, while also outperforming on in-distribution data, establishing that a task-agnostic diffusion framework with appropriate depth and camera modeling can surpass specialized multi-head architectures across diverse environments only when the training mixture is sufficiently broad and FOV is properly conditioned.

2. Context and Motivation

The Core Problem: Zero-Shot Metric Depth in the Wild

The fundamental problem this paper addresses is simple to state but has resisted solution: given a single RGB image taken with an arbitrary camera in an arbitrary environment, estimate the absolute distance (in meters) of every pixel from the camera. This is the problem of monocular metric depth estimation, and the "zero-shot" qualifier means the model must generalize to images from cameras and environments it has never seen during training.

This problem sits at an uncomfortable intersection of two difficulties that have historically been managed separately. The paper identifies these as the twin barriers in Section 1:

First, indoor and outdoor scenes have radically different depth distributions. Indoor environments (homes, offices) typically span 0–10 meters, with dense structure at close range. Outdoor environments (streets, highways) can extend to 80 meters or more, with large expanses at distance. A model must allocate its representational capacity across both regimes simultaneously, and naïvely training on mixed data produces models that perform poorly in one domain or the other.

Second, depth is intrinsically ambiguous from a single image without camera intrinsics. The same scene, photographed with two different cameras at different fields of view, produces images where objects occupy different numbers of pixels. A person who appears 50 pixels tall in one image might appear 100 pixels tall in another, yet be at the same physical distance. Without knowing the camera's field of view, the mapping from image appearance to metric depth is fundamentally underdetermined.

These two challenges are not merely theoretical nuisances — they are the central reasons why, as the paper states in its opening sentence, "zero-shot metric depth estimation remains unsolved" despite significant progress on narrower formulations of the depth estimation problem.

Why This Problem Matters

The practical motivation is straightforward. Metric depth is a foundational signal for applications that interact with the physical world:

• Mobile robotics: A robot navigating an office building needs to know obstacle distances in meters, not relative ordering, to plan safe trajectories. It cannot assume it will only operate in environments seen during training.
• Autonomous driving: A self-driving car on an unfamiliar road must estimate distances to vehicles, pedestrians, and infrastructure with metric precision, regardless of which camera rig captured the training data for its perception system.
• Augmented reality: Placing virtual objects consistently in real scenes requires knowing absolute depth at each pixel, and AR systems run on diverse consumer cameras with varying intrinsics.

The theoretical motivation is equally significant. Monocular metric depth from arbitrary images is a canonical test of whether computer vision systems can learn geometric understanding (the relationship between 2D appearance and 3D structure) rather than merely memorizing dataset-specific statistics. A model that solves this problem in a generic, task-agnostic framework — as this paper attempts with diffusion — demonstrates that the underlying visual reasoning generalizes beyond the training distribution.

The Landscape of Prior Approaches and Where They Fall Short

The paper organizes existing work into three categories, and the progression between them tells the story of why zero-shot metric depth has been so elusive.

Domain-specific models trained and evaluated in-domain. The first and most straightforward approach is to sidestep generalization entirely: train a model on a single dataset captured with a fixed camera, and evaluate on held-out images from that same dataset. Models like BTS, AdaBins, LocalBins, NeWCRFs, and BinsFormer (all cited in Tables 1 and 4) follow this paradigm, training separately on NYU (indoor, Kinect camera) or KITTI (outdoor, LiDAR+camera rig). These models achieve impressive in-distribution numbers — DPT reaches 0.110 REL on NYU, BinsFormer reaches 0.052 REL on KITTI — but they fundamentally do not address the problem this paper targets. They succeed by overfitting to a specific camera's intrinsics and a specific environment's depth distribution. A model trained only on NYU will produce nonsense depths when shown a KITTI image, and vice versa.

The limitation is not just that these models don't generalize — it's that their very design assumes away the hard parts of the problem. By fixing the camera intrinsics, they never learn to handle scale ambiguity. By restricting to one domain, they never learn to handle diverse depth distributions. The strong in-distribution numbers in Table 4 serve as a useful ceiling: if a domain-specific model can achieve 0.052 REL on KITTI, then a joint model that approaches this performance while also generalizing zero-shot is genuinely impressive.

Joint indoor-outdoor models using scale-invariant depth. To handle both environments within a single model, the predominant strategy has been to abandon metric depth estimation and instead predict scale- and shift-invariant depth, where depth values are normalized to be consistent up to an unknown global scale and offset. MiDaS, the landmark work in this category, trains on diverse indoor-outdoor datasets and demonstrates broad zero-shot generalization, but its outputs are in arbitrary units — you know which surfaces are closer than others, but not by how much in meters.

This is a pragmatic but deeply unsatisfying compromise. For robotics and autonomous driving, "relative" depth is often useless: a planner needs to know whether an obstacle is at 2 meters or 5 meters to decide between braking and steering. MiDaS's success at generalization proves that a single model can learn joint indoor-outdoor representations, but its failure to predict metric depth leaves the hardest part of the problem unsolved.

ZoeDepth and the multi-head bridge between relative and metric depth. The immediate predecessor to this paper, and its primary baseline, is ZoeDepth (Bhat et al., 2023). ZoeDepth's architecture embodies a key insight: rather than training a joint metric model end-to-end, first train a scale-invariant MiDaS-style model, then attach domain-specific heads — one for indoor, one for outdoor — that convert the relative depth predictions into metric depth. A scene type classifier (the "auto router") determines which head to use at inference time.

ZoeDepth represents the state of the art that this paper sets out to surpass, and understanding its limitations reveals the specific gaps DMD aims to fill:

1. Architectural specialization as a crutch: ZoeDepth requires knowing whether the input image is indoor or outdoor (either via an explicit label or via the auto router). This is a domain-specific inductive bias that the paper argues should not be necessary — a truly generic model ought to handle both environments from a unified representation. The paper demonstrates the fragility of this dependence in Table 8: when ZoeDepth's scene type supervision is removed (auto router), its in-domain performance degrades significantly (NYU REL rises from 0.077 to 0.102; KITTI REL rises from 0.057 to 0.075), while DMD trained without scene-type information maintains strong performance (NYU REL 0.074; KITTI REL 0.055).

2. Limited training data diversity: ZoeDepth fine-tunes its metric heads only on NYU (indoor) and KITTI (outdoor). While this enables the comparison in Table 4, it restricts the model's exposure to diverse camera intrinsics and scene types. The paper's ablation in Tables 1-2 systematically demonstrates that expanding the training mixture to include nuScenes and Taskonomy (the DMD-MIX variant) yields dramatic improvements on zero-shot benchmarks: SUN RGB-D REL drops from 0.109 to 0.091, DDAD REL drops from 0.122 to 0.108, DIML Outdoor REL drops from 0.300 to 0.190.

3. No explicit mechanism for handling unknown intrinsics: ZoeDepth does not condition on camera parameters. While it may learn some implicit cues from the training data's limited intrinsics variation, it lacks a principled way to resolve scale ambiguity when presented with a dramatically different camera. The paper's FOV conditioning (Table 5) demonstrates the value of explicit intrinsics handling: adding FOV conditioning reduces DIML Outdoor REL from a catastrophic 1.257 to 0.300 — DIML Outdoor has a significantly different FOV from the training cameras, and without conditioning, the model cannot adapt.

The paper's critical observation about ZoeDepth is not that it performs poorly — it performs well — but that its performance is achieved through domain-specific engineering that limits its generality. The paper's mission is to show that a simpler, more generic framework can outperform this specialized approach, provided the right modeling choices are made around depth representation and camera conditioning.

Broader Context: Diffusion Models for Vision Tasks

The paper positions itself within an emerging line of work applying denoising diffusion models to dense visual prediction tasks. Diffusion models were originally developed for image generation (Ho et al., 2020; Dhariwal and Nichol, 2022), but recent work has demonstrated their surprising effectiveness on tasks like semantic segmentation, panoptic segmentation, optical flow, and monocular depth estimation (Saxena et al., 2023; Ji et al., 2023; Chen et al., 2023).

The immediate predecessor in the diffusion-for-depth line is DDVM (Saxena et al., 2023), from the same research group, which showed that a task-agnostic Efficient U-Net diffusion model could achieve competitive depth estimation results without specialized architectures, loss functions, or training procedures. However, DDVM's depth estimation was limited — it used 128 denoising steps for inference, making it computationally expensive, and it did not specifically address the challenges of zero-shot metric depth across indoor and outdoor environments.

DMD inherits DDVM's architectural framework but targets specifically the metric depth generalization problem that DDVM did not solve. The paper's innovations — log-scale depth, FOV conditioning, FOV augmentation, v-parameterization — are all motivated by the specific challenges of metric scale and domain generalization, not by architectural novelty. This is characteristic of the paper's philosophy: rather than designing a clever architecture for depth, design the training procedure and data representation so that a generic architecture can learn the right thing.

The Training Data Bottleneck

Underlying the technical discussion is an implicit argument about data. The paper observes that most prior work trains on limited datasets (NYU for indoor, KITTI for outdoor) captured with fixed camera intrinsics. This makes generalization impossible because the model never sees the variation it needs to learn robust features. The paper's training strategy is unusually broad for the depth estimation literature: unsupervised pre-training on ImageNet and Places365, supervised pre-training on ScanNet, SceneNet-RGBD, Waymo, and DIML Indoor, and final fine-tuning on a mixture of NYU, KITTI, nuScenes, and Taskonomy with FOV augmentation applied. This staged approach mirrors the strategy from DDVM but adds crucial diversity — the inclusion of nuScenes (a large-scale autonomous driving dataset) and Taskonomy (a diverse indoor scene dataset) in the fine-tuning mixture is what enables DMD-MIX's substantial zero-shot gains over DMD-NK (Tables 1-2, Figure 5).

The paper does not frame this as a contribution per se — it is presented as implementation detail (Section 4.1) — but it is arguably essential to the results. The message is that zero-shot metric depth generalization requires not just clever modeling (log depth, FOV conditioning) but also exposure to sufficient training diversity that the model can learn to disentangle depth from camera intrinsics and scene type.

How DMD Positions Itself

The paper's positioning is explicit in its abstract and reinforced throughout: DMD is "a generic, task-agnostic diffusion model" that achieves state-of-the-art zero-shot metric depth despite having "no task-specific inductive biases" and "no specialized architectures for handling diverse indoor and outdoor scenes." This is a deliberate contrast with ZoeDepth's multi-head architecture and domain-specific design.

The paper argues that the apparent need for domain-specific components in prior work was actually masking deficiencies in three areas: (1) depth representation (linear scaling starves indoor scenes of representational capacity — fixed by log-scale depth), (2) camera awareness (without FOV information, scale ambiguity cannot be resolved — fixed by FOV conditioning), and (3) training diversity (models overfit to training camera intrinsics — fixed by FOV augmentation and expanded training mixtures). Once these are addressed, a generic diffusion model surpasses the specialized approach.

This framing matters because it implies that the right way forward for zero-shot metric depth is not more complex architectures but better modeling of the underlying geometric realities — that depth spans orders of magnitude, that scale depends on intrinsics, and that the training distribution must reflect the diversity of deployment conditions. The paper's evidence for this claim is the systematic ablation in Section 4.4, where each component (log depth, FOV conditioning, FOV augmentation) is removed and the resulting degradation in zero-shot performance is measured.

3. Technical Approach

Framing. This is primarily a methodology paper that recasts monocular depth estimation as a conditional image-to-image generation problem within a denoising diffusion framework, then layers on domain-agnostic mechanisms — log-scale depth parameterization, field-of-view conditioning, and synthetic FOV augmentation — to overcome the twin barriers of joint indoor-outdoor modeling and scale ambiguity from unknown camera intrinsics. The core insight is that zero-shot metric depth generalization does not require specialized architectures but rather requires the model to be given the right representational tools (log depth to balance capacity, explicit FOV information to resolve scale) and trained with sufficient diversity (FOV augmentation, broad dataset mixture) that it can disentangle depth from camera-specific and scene-specific biases.

3.1 Reader Orientation

What the system is, in plain language. DMD is a neural network that takes an RGB image and a camera field-of-view value as input, and produces a metric depth map — an image where each pixel stores the absolute distance in meters from the camera to that surface — by iteratively refining a random noise pattern into the depth map through a learned denoising process, like a generative model that "imagines" the 3D structure consistent with the 2D appearance and the camera geometry.

The problem shape and solution shape. The problem is that a single image alone does not contain enough information to uniquely determine metric depth — the same scene photographed with different cameras looks different in ways that affect scale, and indoor/outdoor environments span different depth ranges that tax a model's representational capacity — so the solution must inject the missing geometric information (FOV as a conditioning signal) and reparameterize the output space (log depth) so that the model allocates its learning capacity proportionally across shallow and deep ranges, while also exposing the model to enough synthetic variation during training (FOV augmentation, broad datasets) that it learns to generalize rather than memorize training-camera statistics.

3.2 Big-Picture Architecture (Diagram in Words)

The DMD system has five major components:

1. Efficient U-Net Denoiser Backbone — a convolutional neural network inherited from DDVM [40] that takes as input a noisy depth map, the conditioning RGB image, and conditioning signals (timestep embedding, FOV embedding), and outputs a prediction of the "velocity" $v$ from which the clean depth map can be recovered. This is the only learned component that performs the core computation.

2. FOV Embedding Module — converts the scalar vertical field-of-view $\theta$ (specifically $\tan(\theta/2)$) into a sinusoidal positional embedding followed by a linear projection, producing a vector that modulates layers of the denoiser backbone via FiLM (Feature-wise Linear Modulation) layers, enabling the network to condition its depth predictions on camera geometry.

3. FOV Augmentation Pipeline — during training only, randomly crops or uncrops (with Gaussian noise padding for RGB, near-neighbor in-filling for missing depth) each training image to simulate a synthetic vertical FOV in the range $[0.8, 1.5]$ times the original, forcing the network to learn depth estimation across diverse virtual camera intrinsics rather than overfitting to training-camera FOV.

4. Log-Depth Parameterization — converts raw metric depth values $d_r$ in meters into targets for the diffusion process via $d_{\text{log}} = \text{normalize}(\frac{\log(d_r / d_{\min})}{\log(d_{\max} / d_{\min})})$ where $d_{\min} = 0.5$ m and $d_{\max} = 80$ m, mapping the output space to $[-1, 1]$ in a way that allocates more dynamic range to shallow indoor depths and compresses deep outdoor depths, then at inference time the predicted $[-1, 1]$ values are inverted back to meters.

5. DDPM Sampler with v-Parameterization — the inference procedure that starts from pure Gaussian noise $z_1 \sim \mathcal{N}(0, I)$ and iteratively applies the trained denoiser for $T$ steps (typically $T = 8$ for indoor, $T = 2$ for outdoor, with $T = 1$ also viable) to produce a clean depth map, using the v-parameterization training objective where the network predicts $v = \alpha_t \epsilon - \sigma_t x$ rather than predicting the noise $\epsilon$ directly, enabling dramatically fewer denoising steps than the $\epsilon$-parameterization would require.

Information flow. At inference time: the RGB image enters as conditioning $y$ → the FOV value $\theta$ is embedded and summed with the timestep embedding → a noise sample $z_T$ is drawn → for $t = T, T-1, ..., 1$, the denoiser predicts $v_\theta(z_t, y, \text{FOV\_emb}, t)$ → the clean depth estimate $\hat{x}_t$ is recovered from $v_\theta$ → the DDPM update produces $z_{t-1}$ → after $T$ steps, $z_0$ is the predicted depth map in log-scale $[-1, 1]$ → inverted to meters via $d_r = d_{\min} \cdot \exp(\text{unnormalize}(z_0) \cdot \log(d_{\max}/d_{\min}))$. Optionally, this process is repeated 8 times with different noise seeds and the results are averaged pixelwise to produce the final depth map.

3.3 Roadmap for the Deep Dive

• First, the diffusion framework and v-parameterization, because everything else builds on understanding how a denoising diffusion model is trained and sampled for dense regression. We need to establish what $v$-parameterization is, why it enables few-step inference, and how the training loss is formulated before we can discuss how depth is represented within this framework.
• Second, the log-depth parameterization, because it defines the target space the diffusion model operates in. This determines what the network actually predicts and how reconstruction error translates to physical depth error — a prerequisite for understanding why linear scaling fails on indoor scenes.
• Third, the FOV augmentation pipeline, because it modifies the training data to simulate diverse camera intrinsics. We need to understand how cropping and uncropping with noise padding works, and how missing depth values from uncropping are handled, before we can discuss why conditioning on FOV is necessary.
• Fourth, FOV conditioning and the embedding mechanism, because FOV augmentation alone is insufficient without explicit FOV information provided to the network. This section explains how the scalar FOV is embedded, how it modulates the denoiser backbone, and why FOV conditioning is essential for resolving scale ambiguity.
• Fifth, the training procedure and dataset strategy, because the staged training — unsupervised pre-training, supervised pre-training, final fine-tuning with FOV augmentation and expanded dataset mixture — is what enables the zero-shot generalization gains. This section ties together all the mechanisms into a concrete training recipe with specific datasets, hyperparameters, and schedules.
• Sixth, inference-time details and design choices, including the number of denoising steps, sample averaging, resolution handling, and evaluation protocol, because these practical choices affect the reported results and distinguish DMD's efficiency from prior diffusion-based depth estimators.

3.4 Detailed, Sentence-Based Technical Breakdown

Diffusion Framework and v-Parameterization

DMD is built on denoising diffusion probabilistic models (DDPMs) as its generative framework. A diffusion model defines two processes: a forward process that gradually corrupts a clean data sample $x$ (here, a depth map) into pure Gaussian noise by repeatedly adding small amounts of noise, and a reverse process learned by a neural network that iteratively removes noise to recover the clean sample. The forward process is fixed and simply defined by a noise schedule; the reverse process is where learning happens.

In standard diffusion notation, the forward process at timestep $t$ produces a noisy version $z_t$ of the clean depth map $x$ according to:

$z_t = \alpha_t x + \sigma_t \epsilon$

where $t \in [0, 1]$ is the continuous time index (sampled uniformly during training), $\epsilon \sim \mathcal{N}(0, I)$ is isotropic Gaussian noise, $\alpha_t > 0$ is a signal scaling coefficient that decreases from 1 to 0 as $t$ goes from 0 to 1, and $\sigma_t = \sqrt{1 - \alpha_t^2}$ is the noise scaling coefficient that increases from 0 to 1, following a pre-determined cosine noise schedule. At $t = 0$, $z_0 = x$ (clean data). At $t = 1$, $z_1 = \epsilon$ (pure noise).

What this equation computes operationally: Given a clean depth map $x$, it produces a corrupted version $z_t$ by mixing the signal and noise with coefficients that depend on the timestep. Early in training ($t$ near 0), $z_t$ is mostly signal with slight noise; late in training ($t$ near 1), $z_t$ is mostly noise with barely any signal. The network sees examples at all corruption levels, learning to denoise under varying amounts of uncertainty.

Why this form: The $\alpha_t$-$\sigma_t$ parameterization with the constraint $\sigma_t^2 = 1 - \alpha_t^2$ ensures that if $x$ has unit variance, then $z_t$ also has unit variance for all $t$ (since $\text{Var}(z_t) = \alpha_t^2 \cdot 1 + (1 - \alpha_t^2) \cdot 1 = 1$), keeping the input distribution to the network stable across noise levels. This variance-preserving property is crucial for training stability.

The conventional approach in diffusion models — called $\epsilon$-parameterization — trains the denoising network $\epsilon_\theta$ to predict the noise $\epsilon$ that was added to produce $z_t$, given $z_t$ and conditioning $y$ (the RGB image). The clean data estimate is then recovered as $\hat{x}_t = (z_t - \sigma_t \epsilon_\theta(z_t, y, t)) / \alpha_t$. However, DMD adopts a different parameterization called v-parameterization (Salimans & Ho, 2022), where the network $v_\theta$ predicts:

$v = \alpha_t \epsilon - \sigma_t x$

What this equation computes physically: The target $v$ is a velocity vector — a weighted difference between the noise that was added and the clean signal, where the weights $\alpha_t$ and $\sigma_t$ ensure that $v$ has a consistent interpretation across noise levels. At $t = 0$ (no noise, $\alpha_t = 1, \sigma_t = 0$), $v = -x$ (the network predicts the negated clean image). At $t = 1$ (pure noise, $\alpha_t = 0, \sigma_t = 1$), $v = \epsilon$ (the network predicts the noise). At intermediate timesteps, $v$ interpolates smoothly between these extremes.

Why this form matters deeply: The $\epsilon$-parameterization suffers from a fundamental asymmetry: at low signal-to-noise ratios (SNR, i.e., $t$ near 1 where $\alpha_t \ll \sigma_t$), predicting $\epsilon$ is easy because the noise dominates the input, but recovering $\hat{x} = (z_t - \sigma_t \hat{\epsilon}) / \alpha_t$ involves dividing by a near-zero $\alpha_t$, amplifying any small errors in the noise prediction catastrophically. Conversely, at high SNR ($t$ near 0), the noise prediction is numerically unstable. The v-parameterization sidesteps both problems: $v$ has comparable magnitude across all timesteps (since $\alpha_t^2 + \sigma_t^2 = 1$), and recovering $\hat{x}_t = \alpha_t z_t - \sigma_t v_\theta(z_t, y, t)$ does not involve any division by near-zero quantities. The paper demonstrates this empirically in Table 9: a model trained with $\epsilon$-parameterization requires 64 denoising steps to achieve 0.077 REL on NYU, while the v-parameterized model achieves 0.077 REL with just 1 step. This is a ~64× reduction in inference compute for the same accuracy, which is crucial for making diffusion-based depth estimation practical.

Training objective. Under v-parameterization, the conventional approach is to minimize the expected $\ell_2$ distance with truncated SNR weighting:

$\mathbb{E}_{x, y, t, \epsilon} \left[ \max\left( \|x - \hat{x}_t\|_2^2, \|\epsilon - \hat{\epsilon}_t\|_2^2 \right) \right]$

where $\hat{x}_t = \alpha_t z_t - \sigma_t v_\theta(z_t, y, t)$ is the clean depth estimate and $\hat{\epsilon}_t = \sigma_t z_t + \alpha_t v_\theta(z_t, y, t)$ is the noise estimate.

However, motivated by the superior performance of the $\ell_1$ loss in DDVM [40] compared to $\ell_2$, the paper replaces $\ell_2$ with $\ell_1$:

$\mathcal{L}_{\text{DMD}} = \mathbb{E}_{x, y, t, \epsilon} \left[ \max\left( \|x - \hat{x}_t\|_1, \|\epsilon - \hat{\epsilon}_t\|_1 \right) \right]$

where $x$ is the clean log-depth map, $y$ is the conditioning RGB image, $t \sim \mathcal{U}(0, 1)$ is the randomly sampled timestep, and $\epsilon \sim \mathcal{N}(0, I)$ is the noise.

What this loss computes operationally: For each training example, the network sees a noisy depth map $z_t$ and the RGB image $y$, predicts $v_\theta$, then computes both the implied clean depth $\hat{x}_t$ and implied noise $\hat{\epsilon}_t$. The loss is the element-wise $\ell_1$ distance to whichever ground truth ($x$ or $\epsilon$) produces the larger error. The expectation is over training images, timesteps, and noise draws.

Why the max and $\ell_1$: The $\max$ selects the harder of the two reconstruction targets at each pixel, preventing the network from "cheating" by focusing on whichever target is easier at a given noise level. The $\ell_1$ norm, compared to $\ell_2$, is more robust to outliers and produces sharper depth predictions in practice — the paper inherits this choice directly from DDVM's empirical finding that $\ell_1$ consistently outperforms $\ell_2$ for dense regression tasks in the diffusion framework.

Log-Depth Parameterization: Why Linear Scaling Fails

The naive approach to representing metric depth in a diffusion model (whose output space is conventionally $[-1, 1]$) is linear scaling:

$d_{\text{lin}} = \text{normalize}\left( \frac{d_r}{d_{\max}} \right)$

where $d_r$ is the raw depth in meters, $d_{\max}$ is chosen to accommodate the maximum expected depth (the paper uses 80 m, the typical outdoor upper bound), and $\text{normalize}(d) = \text{clip}(2d - 1, -1, 1)$ maps $[0, d_{\max}]$ to $[-1, 1]$.

What this computes: Each meter of physical depth gets an equal-sized bin in the representation space. An indoor scene with depths 0–10 m occupies only $10/80 = 12.5\%$ of the $[-1, 1]$ range (approximately $[-1, -0.75]$). An outdoor scene with depths 10–80 m occupies the remaining $87.5\%$.

Why this is catastrophic for indoor scenes: The diffusion model's loss is computed uniformly over the $[-1, 1]$ output space. When an indoor scene's depths are compressed into 12.5% of that range, the model's representational capacity — its ability to distinguish fine depth variations — is starved. A 10 cm depth error at 2 m and a 10 cm error at 70 m produce the same loss contribution, but the former is a much larger relative error ($0.1/2 = 5\%$ vs. $0.1/70 \approx 0.14\%$). The model has no incentive to be precise at close range, even though close-range accuracy is what matters most for indoor applications. Empirically, Figure 4 and Table 3 show that linear scaling produces "noisy predictions for images with shallow depth" and damages indoor performance: on iBims-1, linear scaling gives REL 0.146 vs. 0.130 for log scaling; on DIML Outdoor (which features relatively shallow outdoor depths), REL drops from 0.467 to 0.300.

The paper's solution is to parameterize depth in the log domain:

$d_{\text{log}} = \text{normalize}\left( \frac{\log(d_r / d_{\min})}{\log(d_{\max} / d_{\min})} \right)$

where $d_{\min} = 0.5$ m and $d_{\max} = 80$ m are the supported depth range.

What this computes operationally: Instead of mapping raw meters linearly to $[-1, 1]$, it maps the logarithm of meters linearly. For a depth of 0.5 m, $\log(0.5 / 0.5) / \log(80 / 0.5) = 0 / \log(160) = 0$, mapping to $\text{normalize}(0) = -1$. For a depth of 80 m, the ratio is 1, mapping to $\text{normalize}(1) = 1$. For a depth of 10 m, $\log(10 / 0.5) / \log(160) = \log(20) / \log(160) \approx 0.59$, mapping to $\approx 0.18$. The critical property is that equal ratios of physical depth occupy equal intervals in representation space: the interval from 1 m to 2 m (a factor of 2) spans the same representation-space distance as the interval from 40 m to 80 m (also a factor of 2).

Why this form: The log mapping allocates representational capacity in proportion to relative depth precision, which is the natural error metric for depth estimation (we care about percentage error, not absolute meters). Indoor scenes (0.5–10 m, a 20× range) now occupy roughly $\log(20)/\log(160) \approx 0.59$ of the representation space (nearly 60%), compared to 12.5% under linear scaling. Outdoor scenes (10–80 m, an 8× range) occupy the remaining ~41%. This is a much more balanced allocation that reflects the relative precision needs of each domain. The choice of $d_{\min} = 0.5$ m and $d_{\max} = 80$ m sets the supported log-range to $\log(160) \approx 5.08$ natural log units; all depths outside this range are clipped during the normalize step, which is acceptable because training data rarely contains depths below 0.5 m or above 80 m.

At inference time, the predicted $d_{\text{log}} \in [-1, 1]$ is inverted:

$d_r = d_{\min} \cdot \exp\left( \text{unnormalize}(d_{\text{log}}) \cdot \log(d_{\max} / d_{\min}) \right)$

where $\text{unnormalize}(z) = (z + 1) / 2$ maps $[-1, 1]$ back to $[0, 1]$.

Field-of-View Augmentation Pipeline

Most depth estimation datasets are captured with a single camera or a narrow range of cameras, meaning the training data have essentially fixed field-of-view. An NYU image always has the FOV of a Microsoft Kinect; a KITTI image always has the FOV of that specific camera rig. A model trained solely on such data can learn a spurious shortcut: "this visual appearance corresponds to this absolute depth" without ever learning to factor out the camera intrinsics. When deployed on an image from a camera with a different FOV, the shortcut breaks and depth estimates are systematically wrong — typically, depth scale is misestimated because the model implicitly assumes the training camera's intrinsics.

FOV augmentation attacks this problem by synthetically diversifying the virtual camera intrinsics during training. The procedure operates as follows:

1. Random scale selection. For each training image, uniformly sample a scale factor $s \in [0.8, 1.5]$. This determines the simulated FOV relative to the original: $s > 1$ means "uncrop" (simulate a wider FOV by zooming out), $s < 1$ means "crop" (simulate a narrower FOV by zooming in).

2. Cropping ($s < 1$). The image and its ground-truth depth map are both center-cropped to $1/s$ times the original dimensions. For example, at $s = 0.8$, the image is cropped to $1/0.8 = 1.25\times$ the original size, then resized back to the original resolution — effectively zooming in and simulating a narrower FOV. Both the RGB and depth maps have valid pixel values everywhere, so this case is straightforward.

3. Uncropping ($s > 1$). The image is shrunk to $1/s$ times the original dimensions, creating empty borders on all sides. The empty RGB border is filled with Gaussian noise (mean zero, variance one), which the paper found through preliminary experiments to work as well as much more sophisticated generative uncropping using Palette [37], while being simpler and faster. For the depth map borders where ground truth is missing, the paper adopts the approach from DDVM [40]: near-neighbor in-filling fills the empty border pixels with the depth value of the nearest valid pixel (essentially extrapolating the depth at the image boundary outward), and during training, the diffusion model is trained with step-unrolled denoising on these border regions — meaning the denoising network sees partial noisy inputs and must predict the missing regions, learning to inpaint plausible depths at the expanded boundaries. The paper states this technique is "effective in coping with the inherent distribution shift between training and testing when ground truth data are noisy or incomplete."

Why noise padding works for RGB but near-neighbor for depth: For the RGB image, the uncropped border corresponds to areas outside the original camera's field of view — regions the camera never saw. Filling with Gaussian noise is a form of data augmentation that tells the network "you don't have information here, don't hallucinate" while still providing a valid input tensor. For depth, near-neighbor in-filling provides a plausible (if not perfect) depth continuation, giving the diffusion model something to condition on during training; the step-unrolled denoising further refines these extrapolations. The paper's preliminary experiments comparing noise padding against Palette-based generative uncropping found no significant difference in final depth estimation performance, validating the simpler approach.

The key effect of FOV augmentation: By applying random FOV transformations, the training data now exhibits the same RGB image at multiple effective FOVs (and corresponding depth scales), forcing the network to learn that depth must be inferred with reference to camera geometry rather than image appearance alone. The augmentation range $[0.8, 1.5]$ was chosen empirically to provide sufficient diversity without creating unrealistic extreme FOVs that would harm training stability.

Augmentation is only applied during the final fine-tuning stage, not during the earlier pre-training stages (Section 4.1), because FOV augmentation is specifically targeted at the metric depth (scale-aware) learning phase. The paper reports (Table 7) that removing FOV augmentation and conditioning causes catastrophic degradation on certain zero-shot outdoor datasets: DIML Outdoor REL jumps from 0.300 to 1.399, iBims-1 REL degrades from 0.130 to 0.180, and Hypersim REL worsens from 0.382 to 0.479. The pattern is clear: datasets with FOV distributions far from the training cameras suffer most when augmentation is removed.

FOV Conditioning Mechanism

While FOV augmentation diversifies the training distribution, it does not by itself resolve the fundamental ambiguity: a given image could have been produced by many different (scene, camera) pairs, and without knowing the camera intrinsics, the model cannot uniquely determine depth scale. The paper and others [48] observe that FOV augmentation alone "is not sufficient in itself" — the model needs explicit FOV information as an input to condition its predictions.

The conditioning signal is the vertical field-of-view $\theta$, encoded as:

$c_{\text{FOV}} = \tan\left( \frac{\theta}{2} \right)$

Why this specific encoding: The tangent of the half-angle is proportional to the ratio of the image plane half-height to the focal length. In the pinhole camera model, depth $Z$ relates to image-plane displacement $\Delta y$ and physical height $H$ as $Z = (f \cdot H) / \Delta y$. The FOV determines $f / \text{sensor\_height}$, and $\tan(\theta/2) = (\text{sensor\_height}/2) / f$. By conditioning on $\tan(\theta/2)$, the network receives exactly the geometric quantity that relates pixel-space measurements to metric-scale depth, making the scale disambiguation problem explicit. The paper explored conditioning on horizontal FOV as well but "that did not improve results substantially," suggesting that vertical FOV (which affects all rows equally in the common case of square pixels) provides sufficient disambiguation.

How the conditioning signal is incorporated into the network (Section 4.2):

1. The scalar $c_{\text{FOV}}$ is first mapped to a sinusoidal positional embedding, identical in form to the standard transformer positional encodings [47]: for embedding dimension $d$, the $i$-th pair of dimensions encodes $\sin(c_{\text{FOV}} \cdot \omega_i)$ and $\cos(c_{\text{FOV}} \cdot \omega_i)$ for logarithmically-spaced frequencies $\omega_i$. This expands the scalar into a high-dimensional vector that can represent fine-grained FOV-dependent patterns.

2. The sinusoidal embedding is passed through a learned linear projection to produce the FOV embedding vector $\mathbf{e}_{\text{FOV}} \in \mathbb{R}^{d_{\text{emb}}}$.

3. Separately, the diffusion timestep $t$ undergoes the same sinusoidal embedding and linear projection to produce the timestep embedding $\mathbf{e}_t$.

4. The two embeddings are summed: $\mathbf{e} = \mathbf{e}_{\text{FOV}} + \mathbf{e}_t$.

5. This combined embedding modulates different layers of the Efficient U-Net denoiser backbone using FiLM (Feature-wise Linear Modulation) layers [33]. A FiLM layer takes the embedding $\mathbf{e}$, passes it through small learned networks to produce scale $\gamma$ and shift $\beta$ parameters per feature channel, then applies $\text{FiLM}(h) = \gamma \odot h + \beta$ to the feature maps $h$ at that layer. This allows the FOV and timestep information to condition the network's computations everywhere in the hierarchy — from early low-level features to late high-level semantic features — rather than being injected only at the input.

Why FiLM conditioning rather than concatenation: Concatenating the conditioning signal to the input would only influence the first layer; FiLM enables the conditioning to directly modulate every layer's feature activations, which is important because how FOV affects depth estimation is not a simple global scale factor — different depth ranges, object scales, and scene regions interact with FOV in different ways that need to be resolved at multiple levels of abstraction.

The effect of FOV conditioning is empirically dramatic (Table 5): On DIML Outdoor — a dataset with significantly larger FOV than the training cameras — adding FOV conditioning reduces REL from 1.257 to 0.300, a >4× improvement. On iBims-1, REL drops from 0.180 to 0.130. On Hypersim, from 0.400 to 0.382. The pattern is consistent: FOV conditioning provides the largest benefits exactly where FOV generalization is hardest. Figure 6 further validates that the model genuinely uses the FOV signal: when the conditioning FOV is systematically perturbed away from the true FOV during inference, performance degrades monotonically, confirming that the network has learned to interpret the FOV correctly and relies on it for accurate depth prediction.

Training Procedure and Dataset Strategy

DMD adopts a staged training regimen adapted from DDVM [40], with three distinct phases designed to build increasingly task-specific representations while maximizing data diversity:

Stage 1: Unsupervised pre-training on ImageNet and Places365. The denoiser backbone is first trained without depth supervision, using self-supervised tasks proposed in Palette [37]. These likely include inpainting (predicting masked image regions), colorization, and uncropping — tasks that force the network to learn general visual representations without requiring depth labels. ImageNet provides 1.2M object-centric images; Places365 provides 1.8M scene-centric images. Together they expose the model to an enormous diversity of visual content before any depth-specific learning occurs. The paper does not specify the exact pre-training tasks or duration for this stage, referring only to the Palette [37] framework.

Stage 2: Supervised pre-training on diverse depth datasets. The model is fine-tuned for depth estimation on ScanNet [11], SceneNet-RGBD [31], Waymo [45], and DIML Indoor [10] — all datasets with ground-truth depth that span indoor (ScanNet, SceneNet-RGBD, DIML Indoor) and outdoor (Waymo) environments. This stage uses 1.5M training steps with a learning rate of $1 \times 10^{-4}$, optimized with the same settings as DDVM [40] (likely AdamW with default betas, though exact optimizer config is not specified in the paper). No FOV augmentation or conditioning is used in this stage because the goal is to learn robust depth features that can later be adapted to handle FOV variation. The inclusion of DIML Indoor — which was not used in DDVM [40] — adds additional indoor diversity beyond ScanNet and SceneNet-RGBD.

Why these datasets: ScanNet (1,500+ indoor scenes with Kinect depth), SceneNet-RGBD (synthetic indoor scenes with perfect depth), and DIML Indoor (diverse indoor with DSLR + depth sensor) together cover a broad range of indoor environments and depth sensor characteristics. Waymo (autonomous driving data with LiDAR depth) provides large-scale outdoor data with metric depth. The combination ensures the model has seen both domains with metric supervision before the final stage.

Stage 3: Final fine-tuning with FOV augmentation, FOV conditioning, and expanded dataset mixture. The model is fine-tuned for 50,000 steps with a lower learning rate of $3 \times 10^{-5}$ on a mixture of NYU [42], KITTI [19], nuScenes [7], and Taskonomy [51]. This stage enables three critical changes:

• FOV augmentation is activated: Random cropping/uncropping with scale $s \in [0.8, 1.5]$ is applied to NYU, KITTI, and nuScenes, but not to Taskonomy. The paper explains that Taskonomy is omitted from FOV augmentation because it is "large and has substantial FOV diversity" already — Taskonomy consists of images from hundreds of buildings captured with varying camera setups, so its natural FOV diversity is sufficient without synthetic augmentation.

• FOV conditioning is added: The $\tan(\theta/2)$ embedding is provided as input. The model learns to use this signal during the final stage.

• Expanded dataset mixture: nuScenes contributes a massive amount of diverse outdoor driving data (1,000 scenes, 40k keyframes, multiple cameras); Taskonomy provides broad indoor diversity (4.5M images from 600+ buildings). The paper demonstrates that this expanded mixture (DMD-MIX) dramatically outperforms the restricted mixture (DMD-NK, which uses only NYU and KITTI like ZoeDepth). As shown in Tables 1-2 and Figure 5, DMD-MIX improves REL on SUN RGB-D from 0.109 to 0.091, on DDAD from 0.122 to 0.108, and most dramatically on DIML Outdoor from 0.300 to 0.190. The qualitative difference is visible in Figure 5: DMD-MIX produces "significantly improved depth scale and fine-grained depth details near object boundaries" compared to DMD-NK.

Why staged training: The staged approach reflects an important practical insight. Training directly on the final mixture with FOV augmentation from scratch would be challenging because the network has to simultaneously learn depth estimation, handle diverse scene types, and adapt to variable FOV. By first learning depth estimation on large, consistent datasets (Stage 2), then adapting to FOV augmentation and conditioning on a carefully curated mixture (Stage 3), the model can build on stable depth representations rather than struggling with everything at once.

Additional augmentations: Throughout all supervised stages, the paper applies random horizontal flip augmentation, "like many prior works." No other augmentations (color jitter, rotation, scaling) are mentioned.

Training resolution: The model is trained at $384 \times 512$ resolution (height $\times$ width), following ZoeDepth [5]. This is a relatively modest resolution for depth estimation — many domain-specific models train at higher resolutions — but it reflects the diffusion model's computational requirements and the desire to keep training and inference efficient. At inference time, predicted depth maps are resized to the ground-truth resolution for evaluation, again following ZoeDepth's protocol.

Model architecture details (Section 4.2): DMD adopts the "modifications of the Efficient U-Net proposed in DDVM [40]" with one further modification for FOV conditioning (the FiLM layers described above). The Efficient U-Net [38] is a convolutional architecture originally designed for image generation with reduced computational cost compared to standard U-Nets. The specific modifications from DDVM are not detailed in this paper but are presumed to include task-specific adjustments for dense prediction. The FOV embedding dimension, number of FiLM-modulated layers, and exact architecture depth are not specified — the paper treats the architecture as a constant inherited from prior work, with the innovations focused on data representation and conditioning rather than architecture design.

Inference-Time Procedure and Design Choices

Sampling. At inference, the trained denoiser $v_\theta$ is used with the DDPM sampler [23] to produce a clean depth map from random noise. The number of denoising steps depends on the domain:

• Indoor datasets: 8 denoising steps. More steps are beneficial for indoor scenes because they typically contain fine geometric detail (furniture edges, thin structures, cluttered layouts) that requires careful refinement.
• Outdoor datasets: 2 denoising steps. Outdoor scenes often have smoother depth variation (ground planes, building facades, distant sky) that can be captured with coarser sampling. The paper finds that "two denoising steps suffice" for outdoor data.

Why this domain-dependent step count: The choice reflects empirical performance characteristics rather than a theoretical requirement. The v-parameterization enables accurate results with very few steps in both domains (Table 9 shows that even 1 step achieves competitive performance), but the optimal tradeoff between computation and quality differs. Indoor scenes benefit more from additional steps because the relative depth error is more sensitive to fine details; outdoor scenes, where errors are dominated by large-scale structure rather than fine detail, saturate faster. The paper does not provide a formal ablation of step count per domain, but the practical choice of 8/2 steps is justified by the strong quantitative results.

Sample averaging. Following DDVM [40], DMD generates 8 independent samples (each from a different random noise seed) and averages them pixelwise to produce the final depth map. Table 6 quantifies the effect: on NYU, averaging 8 samples reduces REL from 0.077 (single sample) to 0.076 — a small but consistent improvement. On iBims-1, REL drops from 0.135 to 0.130. The effect is systematic but modest, suggesting that the model's predictions are already quite stable and averaging primarily smooths out residual noise.

Why averaging works: Diffusion models are stochastic generators — different noise seeds produce different plausible depth maps consistent with the input image. Averaging across samples can be seen as a Monte Carlo estimate of the posterior mean of the depth given the image, which tends to be more accurate (lower variance, similar bias) than any single sample. However, the paper does not explore whether the gains saturate beyond 8 samples or whether 8 is optimal.

Handling unknown FOV at inference. While FOV is typically known for most practical applications (phone cameras, robot platforms, autonomous vehicles report their intrinsics), the paper also addresses the case where FOV is unknown (e.g., internet images). In Appendix B, the authors train a simple FOV estimator: the encoder of a pre-trained Palette [37] model followed by spatial average pooling and a linear head that regresses to $\tan(\theta/2)$ using an $\ell_1$ loss, trained on NYU and KITTI with FOV augmentation. Table 10 shows that using this estimated FOV with DMD produces depth estimates that are competitive with using the true FOV on most datasets — except DIML Outdoor, where the FOV estimation error is large (28 degrees) and the resulting depth REL degrades from 0.190 to 1.049 for DMD-MIX. The paper attributes this to DIML Outdoor's "significantly larger FOV" that falls outside the estimator's training distribution. The discussion suggests incorporating more sophisticated camera calibration models [25, 27] but defers this to future work.

Evaluation protocol (Section 4.2, reproduced from ZoeDepth [5]). The paper closely follows ZoeDepth's evaluation protocol, including:

• Depth range filtering: Evaluation is restricted to pixels with ground-truth depth less than a specified maximum, which differs per dataset: 8 m for SUN RGB-D, 10 m for iBims-1 and DIODE Indoor, and 80 m for Hypersim and outdoor datasets. This reflects the reliable range of the ground-truth sensors and the typical depth extents of each environment.
• Cropping: The evaluation uses the same center-cropping as ZoeDepth, likely to exclude boundary artifacts from sensor limitations.
• Metrics: Standard depth estimation metrics are reported: $\delta_1$ (percentage of pixels where $\max(\text{pred}/\text{gt}, \text{gt}/\text{pred}) < 1.25$), REL (mean absolute relative error: $|\text{pred} - \text{gt}| / \text{gt}$), RMSE (root mean squared error in meters), and for outdoor datasets, $\text{RMS}_{\log}$ and Sq-rel. The paper primarily emphasizes REL as the headline metric because relative error is scale-invariant and comparably meaningful across environments.
• Resolution handling: Predicted depth maps are resized to the ground-truth resolution for evaluation, matching the protocol of prior work [5]. This means the model operates at $384 \times 512$ internally but is evaluated at the native resolution of each benchmark, with no train-time adjustments for output resolution.

One important detail about resolution: Because the diffusion model operates at a fixed internal resolution, resizing the log-depth predictions to ground-truth resolution for evaluation involves interpolation. The paper does not discuss whether this interpolation is bilinear, nearest-neighbor, or some other method, but standard practice in depth estimation is bilinear interpolation. The fact that DMD still outperforms ZoeDepth after this resizing step suggests that the resolution gap is not a major bottleneck for the tasks evaluated.

4. Key Insights and Innovations

Innovation 1: Difficulty-Agnostic Architecture Can Surpass Difficulty-Specific Architecture — If You Fix the Representation, Not the Model

The paper's most intellectually distinctive move is its refusal to add architectural complexity in response to domain heterogeneity. The dominant assumption in the field, crystallized by ZoeDepth, was that jointly modeling indoor and outdoor metric depth requires domain-specific architectural components — separate output heads, scene-type classifiers, or mixture-of-experts routing. ZoeDepth's two-headed design was a natural, almost inevitable-seeming response to the observation that indoor and outdoor depth distributions differ by an order of magnitude. If the data has two modes, the reasoning went, the model should have two modes.

DMD argues the opposite: the problem is not that the architecture is insufficiently specialized, but that the data representation is poorly matched to the task geometry. The paper demonstrates that three representation-level fixes — log-scale depth to balance capacity across depth ranges, FOV conditioning to resolve scale ambiguity, and FOV augmentation to prevent overfitting to training-camera intrinsics — enable a generic, task-agnostic diffusion model to outperform ZoeDepth's specialized design on every zero-shot benchmark (Tables 1-2: 25% REL reduction indoor, 33% outdoor).

What makes this more than a "simple tricks beat complex models" story is the diagnostic insight it embodies: architectural specialization was masking a failure of geometric reasoning. ZoeDepth's indoor and outdoor heads were not solving the scale ambiguity problem — they were memorizing the scale statistics of their respective training domains. The moment a test image has different camera intrinsics (as in DIML Outdoor or DDAD), those memorized scales break. DMD's approach — conditioning on the actual FOV rather than inferring domain from appearance — is conceptually cleaner and empirically more robust because it addresses the underlying geometric ambiguity rather than the surface-level domain mismatch.

The significance of this reframing extends beyond depth estimation. It suggests a general principle: when a task exhibits domain-specific failure modes, the first diagnostic question should be "are we providing the model with the right input representation to solve the geometric/statistical problem?" rather than "how should we partition the architecture across domains?" The paper's evidence is particularly strong here because the ablation in Table 8 shows that DMD without FOV augmentation or conditioning already outperforms ZoeDepth without scene-type supervision on in-domain data (NYU REL 0.074 vs. 0.102; KITTI REL 0.055 vs. 0.075), indicating that the diffusion framework's inherent flexibility is part of the story, not just the representation fixes. This is a fundamental conceptual shift — from architecture-as-solution to representation-as-solution — not an incremental refinement.

---

Innovation 2: Log-Scale Depth as a Capacity Allocation Strategy, Not Just a Numerical Trick

Log-depth parameterization is not new in depth estimation — it has been used in various forms for years — but the paper's contribution is reframing log depth from a numerical convenience to a principled capacity allocation mechanism for joint indoor-outdoor modeling, and providing the first clear evidence that it is the key enabler of domain-agnostic architecture.

Prior work using log depth (e.g., Eigen & Fergus, 2015) typically motivated it as a way to compress the output range or handle long-tailed depth distributions. The paper's insight is sharper: linear depth scaling allocates model capacity — measured by the fraction of the [-1, 1] output range devoted to each depth interval — in proportion to absolute depth range, while relative depth error (the metric that matters for evaluation) is invariant to absolute depth. An indoor scene spanning 0–10 m gets 12.5% of the capacity under linear scaling at d_max = 80 m, despite contributing disproportionately to the final relative error metric because errors at close range have larger relative magnitude. Log scaling reallocates capacity in proportion to multiplicative depth range — each factor-of-2 interval gets equal representation space — which directly aligns the training loss landscape with the evaluation metric.

The evidence for this as a genuine insight rather than just a hyperparameter choice comes from Table 3 and Figure 4. Linear scaling does not merely produce slightly worse numbers — it produces qualitatively different failure modes: "noise artifacts in the depth estimates for indoor scenes" (Figure 4), where the model's predictions visibly degrade in textureless regions. This is exactly what one would predict if the model's representational capacity were starved at close range: it cannot resolve fine depth variations because the differences are compressed into a tiny fraction of the output range, so it defaults to noisy, high-variance predictions. Log scaling eliminates these artifacts, and the improvement is concentrated precisely on shallow-depth datasets: iBims-1 REL drops from 0.146 to 0.130, DIML Outdoor from 0.467 to 0.300.

The broader implication is that output space parameterization is a first-class design decision for dense regression tasks, not an afterthought. The standard practice of linearly normalizing targets to [-1, 1] (ubiquitous in diffusion models and regression networks) implicitly assumes that equal absolute error matters equally everywhere in the output range. When the evaluation metric is scale-invariant (relative error, ordinal metrics), this assumption is violated, and the model will systematically underperform in the regime where small absolute errors translate to large relative errors. Log parameterization is one solution; the deeper point is that the output encoding should be designed to match the evaluation metric's sensitivity, and this paper provides a clean case study of the principle in action. This is an incremental refinement of existing practice elevated to a diagnostic insight by the paper's systematic ablation and qualitative analysis.

---

Innovation 3: FOV Conditioning as the Missing Piece That Makes Augmentation Work

The paper's treatment of field-of-view reveals a subtle but crucial insight about the relationship between data augmentation and conditioning: augmentation diversifies the training distribution, but without explicit conditioning, the model learns to ignore the augmented variation rather than disentangle it. This distinction — between merely exposing the model to diverse FOVs and teaching it to use FOV as a predictive feature — is the paper's key conceptual contribution to the camera-aware depth estimation literature.

Prior work on intrinsics-conditioned depth estimation (Facil et al., 2019; He et al., 2018; Guizilini et al., 2023) had explored providing camera parameters as input, but typically in small-scale experiments or with limited analysis of the interaction between augmentation and conditioning. The paper's innovation is demonstrating that FOV augmentation without conditioning is largely ineffective for generalization, because the model treats FOV variation as noise to be averaged out rather than signal to be exploited. Table 7 shows the evidence: a model trained with FOV augmentation but without FOV conditioning achieves DIML Outdoor REL of 1.399 (catastrophic) vs. 0.300 with conditioning — the augmentation alone provides essentially no benefit on this challenging generalization target.

What's happening mechanistically is subtle. When FOV is varied through augmentation but not provided as a conditioning signal, the model faces an impossible learning problem: the same RGB image (at different crops) must map to different depth maps (scaled by the FOV factor), but the model has no way to know which depth map is correct because it doesn't know which FOV produced the crop. The optimal strategy for minimizing expected loss is to predict the average depth across all FOVs seen during training, which is precisely the wrong thing for zero-shot generalization to a new FOV. By providing FOV as an explicit conditioning signal, the model can learn the mapping (image, FOV) → depth rather than image → depth, separating the intrinsic scene geometry from the camera projection.

Figure 6 provides the cleanest evidence that the model genuinely uses the FOV signal rather than memorizing it: when the conditioning FOV is systematically perturbed away from the true FOV at inference, depth accuracy degrades monotonically, with optimal performance at the true FOV. This is a strong causal test — if the model were ignoring the FOV conditioning and relying purely on image appearance, perturbing it would have no effect.

The broader implication is a design principle for any domain where augmentation is used to improve robustness: if the augmented variable affects the input-output mapping, it must be provided as a conditioning signal, or else augmentation becomes a form of label noise rather than a robustness intervention. This principle is well-known in some subfields (e.g., camera pose augmentation for view synthesis) but underappreciated in depth estimation, where the dominant approach had been to either fix camera intrinsics (in-domain models) or ignore them (scale-invariant models). The paper's contribution is demonstrating that the principle applies decisively to metric depth, and that the combination of augmentation with conditioning is what unlocks zero-shot generalization. This is a fundamental conceptual advance in understanding the interplay between data augmentation and input conditioning for geometric tasks.

---

Innovation 4: v-Parameterization Makes Diffusion Practical for Dense Regression — And the Reason Why Matters

The paper's use of v-parameterization (Salimans & Ho, 2022) is not itself novel, but its application to dense visual prediction with an analysis of why it matters specifically for regression tasks constitutes a significant practical contribution. DDVM (Saxena et al., 2023) used ε-parameterization and required 128 denoising steps for depth estimation, making diffusion-based depth estimation computationally prohibitive for deployment. DMD achieves competitive or superior performance with 1–2 denoising steps — a reduction of up to 128× in inference compute — by switching to v-parameterization and showing that the benefits are particularly pronounced for regression tasks where accurate signal recovery at all noise levels is critical.

The intellectual contribution here is understanding why v-parameterization matters so much for depth estimation specifically. The paper provides a crisp explanation: under ε-parameterization, recovering the clean signal x̂ = (z_t - σ_t ε̂) / α_t requires dividing by α_t, which becomes arbitrarily small as t → 1 (high noise levels). Small errors in the noise prediction ε̂ are amplified into large errors in x̂, and this amplification is most severe precisely where the model is most uncertain (high noise). v-parameterization avoids this division entirely by predicting v = α_t ε - σ_t x and recovering x̂ = α_t z_t - σ_t v̂, which involves no numerically unstable operations at any noise level.

The empirical evidence in Table 9 is striking: an ε-parameterized model requires 64 denoising steps to reach 0.077 REL on NYU; the v-parameterized model reaches the same accuracy with 1 step. This is not a small efficiency gain — it is the difference between diffusion being a research curiosity for depth estimation and being a practical method. The paper's finding that v-parameterization enables high-quality depth from 1–2 steps without requiring progressive distillation (the more complex approach from Salimans & Ho) is an important practical observation that simplifies the training pipeline.

The significance extends beyond depth estimation: dense regression tasks (optical flow, surface normals, depth) share the property that the full range of the output space is meaningful and must be recovered accurately, unlike image generation where high-frequency detail at intermediate noise levels may be perceptually forgivable. For these tasks, the ε-parameterization's instability at high noise levels is particularly damaging because it corrupts the large-scale structure (low frequencies) that provides the foundation for finer details. v-parameterization ensures that the model can recover global structure accurately even with very few denoising steps, making diffusion viable for regression at scale. This is an incremental refinement of an existing technique that becomes a practical enabler in the context of dense prediction — the paper's contribution is demonstrating and explaining the magnitude of the effect specifically for depth estimation.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All experiments train on and evaluate against multiple depth estimation benchmarks. The primary training datasets for the final fine-tuning stage are NYU [42] (indoor, Kinect depth), KITTI [19] (outdoor, LiDAR+camera), nuScenes [7] (outdoor autonomous driving, multi-camera), and Taskonomy [51] (large-scale indoor, diverse camera intrinsics). Pre-training stages additionally use ImageNet [12], Places365 [52], ScanNet [11], SceneNet-RGBD [31], Waymo [45], and DIML Indoor [10] (Section 4.1). Zero-shot evaluation is performed on eight unseen datasets drawn from ZoeDepth's protocol (Section 4.3): SunRGBD [44], iBims-1 [44], DIODE Indoor [46], HyperSim [36], Virtual KITTI 2 [6], DDAD [20], DIML Outdoor [10], and DIODE Outdoor [46]. In-distribution evaluation uses NYU and KITTI test splits. All datasets are used with their standard training/testing splits as established in prior work; the paper does not modify dataset splits.

• Base model. The core architecture is the Efficient U-Net from DDVM [40] with modifications to support FOV conditioning via FiLM layers (Section 4.2). The architecture is a convolutional denoising U-Net originally designed for efficient image generation [38] and adapted in DDVM for dense prediction tasks. The paper does not specify the exact parameter count, but the model is described as task-agnostic, implying it lacks depth-specific architectural components such as binning heads, multi-scale fusion modules, or domain-specific output branches. The choice is motivated by the paper's thesis that architectural specialization is unnecessary when the representation and conditioning are correctly designed.

• Metrics. Following the evaluation protocol of ZoeDepth [5], the paper reports standard depth estimation metrics (Section 4.2): δ₁ (fraction of pixels where max(pred/gt, gt/pred) < 1.25, i.e., the percentage of pixels with relative error under 25%), REL (mean absolute relative error, computed as |pred − gt| / gt), RMSE (root mean squared error in meters), and for certain datasets, RMS_log and Sq-rel. Among these, REL receives the most emphasis as the headline metric because it is scale-invariant and directly comparable across environments with different depth ranges. For in-distribution results on KITTI (Table 4), additional standard metrics are reported: δ₂, δ₃ (thresholds at 1.25² and 1.25³), and log₁₀ error. Evaluation is restricted to pixels with ground-truth depth below dataset-specific maxima: 8 m for SUN RGB-D, 10 m for iBims-1 and DIODE Indoor, and 80 m for Hypersim and all outdoor datasets (Section 4.2, Table 1 notes). Center-cropping is applied following ZoeDepth [5], and predicted depth maps are resized to the ground-truth resolution for metric computation.

• Baselines. The primary baseline is ZoeDepth (ZoeD-M12-NK) [5], the prior state-of-the-art for zero-shot metric depth, which uses a MiDaS backbone with domain-specific metric heads (indoor/outdoor) and an auto-router for scene type classification. Both DMD and ZoeDepth are compared under matched training data conditions (DMD-NK vs. ZoeD-M12-NK, both fine-tuned on NYU and KITTI) to enable fair comparison. Additional baselines from prior work are reported for completeness in Tables 1-2 and 4: BTS [29], AdaBins [3], LocalBins [4], NeWCRFs [50], DPT [35], BinsFormer [30], PixelFormer [1], IEBins [41], MIM [49], and DDVM [40]. These represent a mix of domain-specific models (trained on single datasets with fixed intrinsics) and joint models, providing context for DMD's performance relative to both specialized and general approaches.

• Generation budget / compute accounting. Test-time compute for DMD is measured in number of denoising steps and number of independent samples generated. The number of denoising steps is domain-dependent: 8 steps for indoor datasets and 2 steps for outdoor datasets at inference time (Section 4.2). Additionally, the paper follows DDVM's practice of generating 8 independent samples (each from a different random noise seed) and averaging them pixelwise to produce the final depth map (Section 4.2). Table 6 shows the effect of sample averaging on REL and RMS. No formal FLOP counting is performed to compare DMD's inference cost against ZoeDepth or other baselines — compute is discussed qualitatively in terms of steps and samples rather than in absolute FLOPs or wall-clock time. The paper's efficiency argument rests on v-parameterization enabling far fewer denoising steps than ε-parameterized alternatives (Table 9: 1 step vs. 64 steps for equivalent accuracy), but no head-to-head latency or FLOP comparison with ZoeDepth is reported.

• Cross-validation / statistical protocol. The paper does not report explicit cross-validation or statistical significance testing. The zero-shot evaluation protocol directly follows ZoeDepth [5], applying models trained on specific training sets to held-out test datasets and reporting standard metrics. For in-distribution results (Table 4), the model is evaluated on the standard NYU and KITTI test splits. The two model variants (DMD-NK and DMD-MIX) differ only in training data, not in architecture or hyperparameters, and both are evaluated on the same test sets. The paper does not report confidence intervals, standard deviations across runs, or statistical tests comparing DMD against ZoeDepth — all comparisons are based on point estimates of metrics. The only cross-validation-like element is the FOV estimator experiment (Appendix B, Table 10), where a separate FOV prediction model is trained on NYU+KITTI and evaluated on all test datasets, but this is not used for the main DMD results and no cross-validation of the depth model itself is performed.

---

Main Quantitative Results

Zero-Shot Indoor Generalization (Tables 1, Figure 2, Figure 7)

The central claim is that DMD achieves substantially lower relative error than ZoeDepth on all four zero-shot indoor benchmarks. Under matched training data (DMD-NK vs. ZoeD-M12-NK, both fine-tuned on NYU and KITTI only), DMD-NK produces (Table 1):

• SUN RGB-D: REL 0.109 vs. 0.123 for ZoeDepth (11.4% reduction), δ₁ 0.914 vs. 0.856
• iBims-1: REL 0.130 vs. 0.186 for ZoeDepth (30.1% reduction), δ₁ 0.801 vs. 0.615
• DIODE Indoor: REL 0.298 vs. 0.331 for ZoeDepth (10.0% reduction), δ₁ 0.402 vs. 0.386
• Hypersim: REL 0.382 vs. 0.419 for ZoeDepth (8.8% reduction), δ₁ 0.356 vs. 0.274

When the training mixture is expanded to include Taskonomy and nuScenes (DMD-MIX), the gains become substantially larger (Table 1):

• SUN RGB-D: REL drops from 0.109 to 0.091 (additional 16.5% improvement over DMD-NK), δ₁ rises to 0.930
• iBims-1: REL drops from 0.130 to 0.118 (additional 9.2%), δ₁ rises to 0.859
• Hypersim: REL drops from 0.382 to 0.318 (additional 16.8%), δ₁ rises to 0.497

The qualitative results in Figure 2 (and extended Figure 7) visualize these differences: DMD's depth maps exhibit "more accurate scale over diverse datasets," with noticeably better depth ordering and fewer scale errors compared to ZoeDepth, particularly on iBims-1 and SUN RGB-D where ZoeDepth shows visible scale miscalibration (e.g., assigning incorrect absolute depths to walls and furniture).

The combined indoor improvement cited in the abstract — 25% reduction in REL — is computed as the mean across the four indoor zero-shot benchmarks. For DMD-MIX vs. ZoeDepth: (0.123−0.091)/0.123 = 26.0% on SUN RGB-D, (0.186−0.118)/0.186 = 36.6% on iBims-1, (0.331−0.291)/0.331 = 12.1% on DIODE Indoor, (0.419−0.318)/0.419 = 24.1% on Hypersim; the average of these per-dataset improvements is approximately 24.7%, consistent with the claimed 25%. However, this aggregate masks substantial variation: the improvement is dramatic on iBims-1 (36.6%) but more modest on DIODE Indoor (12.1%), suggesting particular sensitivity to structured indoor evaluation (iBims-1 tests planar accuracy, which DMD handles well) versus general indoor scenes (DIODE Indoor, where both methods struggle with extreme close-range variation).

Zero-Shot Outdoor Generalization (Tables 2, Figure 3, Figure 8)

On outdoor benchmarks under matched training (DMD-NK vs. ZoeD-M12-NK), Table 2 reports:

• Virtual KITTI 2: REL 0.093 vs. 0.105 for ZoeDepth (11.4% reduction), δ₁ 0.872 vs. 0.850
• DDAD: REL 0.122 vs. 0.138 for ZoeDepth (11.6% reduction), δ₁ 0.842 vs. 0.824
• DIML Outdoor: REL 0.300 vs. 0.641 for ZoeDepth (53.2% reduction), δ₁ 0.544 vs. 0.292 — by far the largest relative gain
• DIODE Outdoor: REL 0.627 vs. 0.757 for ZoeDepth (17.2% reduction on REL), but δ₁ degrades (0.162 vs. 0.208), and RMSE worsens (9.577 vs. 7.569). This is the only benchmark where DMD-NK does not uniformly dominate — it wins on REL but loses on RMSE and δ₁. The likely explanation is that DIODE Outdoor has large depth ranges (up to 80 m) where absolute RMSE is dominated by distant-region errors, and DMD-NK may trade off close-range accuracy (improving REL) against distant-region stability (worsening RMSE).

With expanded training data (DMD-MIX), outdoor performance further improves (Table 2):

• Virtual KITTI 2: REL drops to 0.092, δ₁ rises to 0.890
• DDAD: REL drops to 0.108, δ₁ rises to 0.907
• DIML Outdoor: REL drops dramatically from 0.300 to 0.190 (additional 36.7%), δ₁ rises to 0.602
• DIODE Outdoor: REL improves from 0.627 to 0.553 (11.8%), though RMSE remains higher than ZoeDepth (8.943 vs. 7.569)

The aggregated outdoor improvement of 33% comes from averaging the per-dataset REL reductions of DMD-MIX over ZoeDepth: (0.105−0.092)/0.105 = 12.4% on Virtual KITTI 2, (0.138−0.108)/0.138 = 21.7% on DDAD, (0.641−0.190)/0.641 = 70.4% on DIML Outdoor, (0.757−0.553)/0.757 = 26.9% on DIODE Outdoor; the average is approximately 32.9%, consistent with the claimed 33%.

The DIML Outdoor result deserves special attention. It is the dataset where FOV conditioning produces its most dramatic effect — without FOV conditioning, DMD-NK achieves REL 1.257 (Table 5), which is worse than random guessing at large depths. With FOV conditioning, this drops to 0.300. DIML Outdoor was captured with a significantly wider FOV than typical training cameras (the FOV estimator in Appendix B shows 28° error on this dataset), making it a stress test for camera generalization. The 70.4% improvement over ZoeDepth demonstrates that DMD's FOV conditioning genuinely resolves scale ambiguity on out-of-distribution cameras, not merely on cameras similar to those seen during training.

Figure 3 (and extended Figure 8) shows qualitative outdoor comparisons. The visual differences mirror the quantitative pattern: DMD produces depth maps with more accurate overall scale — particularly visible in the consistent depth assignation to ground planes and distant structures — while ZoeDepth sometimes exhibits systematic scale drift (e.g., assigning depths of 40 m where ground truth is 25 m). Figure 5 specifically compares DMD-NK vs. DMD-MIX on three examples (iBims-1 indoor, DDAD outdoor), showing that the expanded training mixture "significantly improves depth scale and fine-grained depth details near object boundaries."

In-Distribution Performance (Table 4)

While zero-shot generalization is the paper's primary contribution, in-distribution results contextualize whether the gains come at the cost of degrading on training-distribution data — a common failure mode for generalization-focused methods. Table 4 reports:

NYU (indoor, in-distribution):

• ZoeD-M12-NK: REL 0.077, δ₁ 0.953, RMSE 0.277
• DMD-NK: REL 0.076, δ₁ 0.944, RMSE 0.313 — slightly better on REL, slightly worse on δ₁ and RMSE
• DMD-MIX: REL 0.072, δ₁ 0.953, RMSE 0.296 — better on REL, matching on δ₁, slightly worse on RMSE

KITTI (outdoor, in-distribution):

• ZoeD-M12-NK: REL 0.057, δ₁ 0.966, RMSE 2.362
• DMD-NK: REL 0.055, δ₁ 0.964, RMSE 2.527 — slightly better on REL, competitive on δ₁ and RMSE
• DMD-MIX: REL 0.053, δ₁ 0.967, RMSE 2.411 — better on REL and δ₁, competitive on RMSE

The key takeaway is that DMD does not sacrifice in-distribution performance for zero-shot gains — it is competitive or superior on both. Interestingly, DMD-MIX improves in-distribution performance over DMD-NK (NYU REL: 0.072 vs. 0.076; KITTI REL: 0.053 vs. 0.055), despite the expanded training mixture including out-of-domain data (nuScenes, Taskonomy). This contradicts the intuition that training on broader data degrades domain-specific performance and suggests that the additional diversity acts as a regularizer that prevents overfitting to NYU/KITTI idiosyncrasies.

Compared against domain-specific models (which are trained and evaluated on single datasets), DMD-MIX is competitive but not dominant: on NYU, MIM achieves REL 0.083 (higher error than DMD-MIX at 0.072) but better δ₁ (0.949 vs. 0.953); DDVM achieves REL 0.074 (competitive). On KITTI, BinsFormer achieves REL 0.052 and RMSE 2.098, both better than DMD-MIX's REL 0.053 and RMSE 2.411. The paper's contribution is not surpassing domain-specific specialists on their home turf — it is approaching their performance while simultaneously generalizing zero-shot, which no prior joint model achieves.

FOV Conditioning Perturbation Analysis (Figure 6)

Figure 6 provides a causal test of whether the model genuinely uses FOV information: at inference, the conditioning FOV value is systematically perturbed by adding offsets ranging from approximately −30° to +30° to the true vertical FOV, and depth estimation accuracy is measured. The resulting curves show a clear inverted-U shape for all datasets — performance peaks at or very near the true FOV (offset = 0) and degrades monotonically as the perturbation magnitude increases in either direction. This confirms that the model has learned to interpret the FOV conditioning signal correctly and relies on it for accurate depth prediction, rather than memorizing or ignoring it.

The steepness of the degradation varies by dataset: indoor datasets (SUN RGB-D, DIODE Indoor) show relatively symmetric, gradual degradation; certain outdoor datasets show sharp drops when FOV is underestimated. This asymmetry likely reflects the fact that underestimating FOV (telling the model the camera is narrower than it actually is) causes systematic depth underestimation (objects appear larger in pixels, so the model interprets them as closer), which produces larger relative errors at far distances where absolute depth values are large.

Sample Averaging Effect (Table 6)

Generating 8 independent samples and averaging them produces "small but consistent improvement" over using a single sample. Table 6 quantifies the effect: on NYU, REL improves from 0.077 to 0.076; on iBims-1, from 0.135 to 0.130 (−3.7%); on DIODE Indoor, from 0.304 to 0.298 (−2.0%); on DIML Outdoor, from 0.311 to 0.300 (−3.5%). The improvements are systematic but modest — sample averaging is not a major driver of DMD's performance advantage. The largest absolute gains appear on datasets where single-sample predictions are noisier (iBims-1, DIML Outdoor), consistent with averaging reducing variance while leaving bias largely unchanged.

Scene-Type Supervision Comparison (Table 8)

Table 8 compares ZoeDepth without scene-type supervision (Auto Router) against DMD trained without FOV augmentation or conditioning, both on in-distribution data:

• NYU: ZoeDepth Auto Router REL 0.102, DMD REL 0.074 — DMD substantially better
• KITTI: ZoeDepth Auto Router REL 0.075, DMD REL 0.055 — DMD substantially better
• NYU RMSE: ZoeDepth 0.377, DMD 0.310
• KITTI RMSE: ZoeDepth 2.584, DMD 2.597

This comparison reveals that ZoeDepth's performance is fragile to the removal of domain-specific supervision — REL degrades from 0.077 to 0.102 on NYU (32% worse) when the auto router is used instead of ground-truth scene type. In contrast, DMD, even without its core innovations (FOV augmentation/conditioning), learns a joint indoor-outdoor representation that does not depend on knowing the scene type a priori. This supports the paper's claim that domain-specific architectural components mask an underlying failure of geometric reasoning: when the architecture is forced to handle both domains without explicit domain routing, ZoeDepth struggles while DMD's generic framework adapts naturally.

---

Ablation Studies and Robustness Checks

Log vs. linear depth scaling (Table 3, Figure 4): Log-scale depth parameterization improves performance primarily on indoor and shallow-depth datasets by allocating more representation capacity to close-range depths. On DMD-NK: NYU REL improves from 0.082 to 0.076 (−7.3%), iBims-1 REL from 0.146 to 0.130 (−11.0%), DIML Outdoor REL from 0.467 to 0.300 (−35.8%), DIODE Indoor RMS from 1.526 to 1.407. Outdoor datasets with predominantly large depths (KITTI, Virtual KITTI 2, DDAD) show minimal differences (KITTI REL: 0.056 vs. 0.055). Figure 4 provides visual evidence: linear scaling produces "noise artifacts in the depth estimates for indoor scenes," visible as speckled, high-variance predictions in textureless regions like walls and floors, which log scaling eliminates. The mechanism is that linear scaling compresses the 0–10 m range into ~12.5% of the [−1, 1] output space, starving the model of precision where relative error is most sensitive; log scaling expands this to ~59% of the output range.

Field-of-view conditioning (Table 5): Adding FOV conditioning produces the largest single-component improvement among all ablations, and the benefit is concentrated on datasets with FOV distributions far from the training cameras. On DMD-NK: DIML Outdoor REL improves from 1.257 to 0.300 — a >4× reduction — because DIML Outdoor has a substantially larger FOV than KITTI or NYU. iBims-1 REL improves from 0.180 to 0.130 (−27.8%), Hypersim REL from 0.400 to 0.382 (−4.5%), SUN RGB-D REL from 0.116 to 0.109 (−6.0%). In-distribution datasets (NYU: 0.081 vs. 0.076, KITTI: 0.057 vs. 0.055) show small improvements consistent with reduced overfitting. The DIODE Outdoor RMS result (8.582 vs. 9.577, i.e., conditioning worsens RMS) is anomalous and may reflect that FOV conditioning improves close-range accuracy (helping REL) at the expense of slightly noisier distant predictions (hurting RMS, which is dominated by large absolute errors at far depths).

No FOV augmentation or conditioning (Table 7): Training without both FOV augmentation and FOV conditioning hurts generalization to out-of-domain data due to overfitting on training camera intrinsics. The degradation is most severe on DIML Outdoor (REL worsens from 0.300 to 1.399), followed by iBims-1 (0.130 to 0.180), Hypersim (0.382 to 0.479), and DIODE Indoor (0.298 to 0.337). In-distribution performance is largely unaffected (NYU: 0.076 vs. 0.074, KITTI: 0.055 vs. 0.055), confirming that FOV augmentation and conditioning specifically address generalization rather than fundamental depth estimation capability. Table 7 is a critical result because it disentangles the contributions: removing both components causes catastrophic failure on the hardest generalization target (DIML Outdoor) while barely affecting in-distribution accuracy, demonstrating that the problem is not depth estimation per se but handling unknown camera intrinsics.

ϵ vs. v diffusion parameterization (Table 9): v-parameterization dramatically reduces the number of denoising steps required for accurate depth estimation compared to ε-parameterization. On NYU with DMD-NK (no FOV augmentation/conditioning): ε-parameterization achieves REL 2.374 at 1 step, 1.484 at 4 steps, 0.409 at 16 steps, 0.077 at 64 steps; v-parameterization achieves REL 0.077 at 1 step (matching ε at 64 steps). On KITTI: ε-parameterization achieves REL 0.596 at 1 step, 0.406 at 4 steps, 0.141 at 16 steps, 0.056 at 64 steps; v-parameterization achieves REL 0.056 at 1 step. The numerical instability of ε-parameterization is visible in the 1-step results: REL of 2.374 on NYU corresponds to predicting average depth errors 2.4× the ground truth — essentially random predictions. The v-parameterization achieves its best result immediately at 1 step (REL does not improve beyond 0.077 for NYU or 0.056 for KITTI with more steps), suggesting that v-parameterization converges to the optimal prediction in a single denoising step under this training setup.

Estimated vs. true FOV at inference (Table 10): Using a learned FOV estimator (a pre-trained Palette encoder + linear head trained on NYU+KITTI with FOV augmentation) instead of ground-truth FOV produces competitive depth estimates on most datasets, demonstrating that DMD's FOV conditioning does not critically depend on having oracle intrinsics at inference. For DMD-MIX: NYU REL 0.072 with both, SUN RGB-D 0.091 (true) vs. 0.136 (estimated), DDAD 0.108 vs. 0.119, Virtual KITTI 2 0.092 vs. 0.131, DIODE Outdoor 0.553 vs. 0.560. However, DIML Outdoor shows catastrophic degradation (REL 0.190 true vs. 1.049 estimated) because the FOV estimator's error on this dataset is 28° — DIML Outdoor's FOV is far outside the estimator's training distribution, and large FOV estimation errors propagate directly into depth scale errors. The paper notes this limitation and suggests more sophisticated camera calibration models [25, 27] as future work.

Domain-dependent denoising step count: The paper uses 8 denoising steps for indoor datasets and 2 for outdoor, but provides no formal ablation of this choice. Table 9 shows that on the in-distribution NYU test set with v-parameterization, performance saturates at 1 step (REL 0.077) — additional steps provide no benefit. For outdoor KITTI, 1 step achieves REL 0.056, again saturating immediately. This suggests the 8/2 step choice is either (a) conservative, chosen to ensure robustness even if zero-shot datasets are harder than in-distribution data, or (b) an artifact of using sample averaging (8 samples × 1 step might perform slightly better than 1 × 1 step, and the step count is reported per sample). The paper does not clarify the interaction between step count, sample averaging, and the reported metrics.

---

Critical Assessment

Does the paper demonstrate that a generic, task-agnostic diffusion model can surpass specialized architectures for zero-shot metric depth?

Yes, with important caveats about the scope of "generic." The evidence in Tables 1-2 and Table 4 shows that DMD — which the paper characterizes as having "no task-specific inductive biases" and "no specialized architectures for handling diverse indoor and outdoor scenes" — outperforms ZoeDepth's multi-head domain-specific design on all zero-shot benchmarks and is competitive on in-distribution data. The matched-training comparison (DMD-NK vs. ZoeD-M12-NK) isolates the modeling choices from the data scale: under identical training data, DMD's representation choices (log depth, FOV conditioning, FOV augmentation, v-parameterization) produce systematically better zero-shot generalization (indoor: 10–30% REL reduction across benchmarks; outdoor: 11–53% REL reduction).

However, "generic" and "task-agnostic" require qualification. DMD incorporates several depth-specific modeling decisions — log-scale parameterization with chosen d_min = 0.5 m and d_max = 80 m, FOV conditioning using tan(θ/2), domain-dependent denoising step counts (8 indoor, 2 outdoor) — that are not generic to all vision tasks. The architecture is task-agnostic (a U-Net denoiser used identically for optical flow in DDVM and depth here), but the training pipeline and input/output representations are carefully tailored to depth estimation's specific geometric challenges. A more precise characterization would be "architecturally generic with geometrically principled representation design," which is a genuine insight about where specialization belongs (in data representation, not model structure) rather than a claim of complete task agnosticism.

The strongest evidence for the "architecture doesn't need specialization" thesis is Table 8: DMD learns joint indoor-outdoor depth without scene-type labels and outperforms ZoeDepth in the same setting, while ZoeDepth's performance degrades substantially when the auto router replaces ground-truth scene type (NYU REL: 0.077 → 0.102). This demonstrates that domain-specific heads were compensating for a representational deficiency rather than providing a necessary inductive bias.

Does the paper demonstrate that log-scale depth parameterization enables joint indoor-outdoor modeling?

Supported, and the qualitative evidence strengthens the quantitative case. Table 3 shows that log scaling improves REL on indoor and shallow-depth datasets (iBims-1: −11.0%, DIML Outdoor: −35.8%) while minimally affecting deep outdoor datasets. This asymmetry is exactly what the capacity allocation argument predicts: indoor scenes benefit most because linear scaling starves their narrow depth range of representational precision. Figure 4 provides a mechanistic explanation by showing qualitative differences — linear scaling produces noisy, artifact-ridden depth maps for indoor scenes, which log scaling cleans up. The visual evidence is important because it rules out alternative explanations: if the improvement were merely from better numerical conditioning, we would see uniform gains across all datasets rather than domain-concentrated improvements.

However, the paper does not explore whether the specific log-base and d_min/d_max choices are optimal or merely sufficient. An ablation over different d_min and d_max values (e.g., d_max = 40 m, 80 m, 120 m) would clarify whether performance is sensitive to these hyperparameters or whether the log transformation's benefit is robust to the exact range. Similarly, comparing log against other capacity-rebalancing transformations (square root, power-law, learned nonlinearities) would strengthen the claim that log scaling specifically is the right choice.

Does the paper demonstrate that FOV conditioning is necessary for resolving scale ambiguity — that augmentation alone is insufficient?

Yes, and this is the paper's strongest causal claim. Table 7 and Table 5 together form a clean 2×2 test: FOV augmentation without conditioning (Table 5, "No FOV cond" row, but augmentation is active since this table uses DMD-NK which applies FOV augmentation) → DIML Outdoor REL 1.257; FOV conditioning with augmentation → 0.300. The mechanism is explained conceptually in Section 3.2: without conditioning, the model faces an underdetermined mapping problem and defaults to averaging across FOVs, producing systematically wrong depth scale on out-of-distribution cameras. Figure 6 provides the causal manipulation: systematically perturbing the conditioning FOV away from the true FOV causes monotonic degradation, confirming that the model relies on the FOV signal rather than ignoring it.

The DIML Outdoor result is particularly informative because it is an extreme test. DIML Outdoor's FOV is far from the training distribution (the FOV estimator's 28° error confirms this), making it the hardest case. The fact that FOV conditioning alone reduces REL from 1.257 to 0.300 — without any other changes — is strong evidence that the conditioning genuinely resolves scale ambiguity rather than providing a marginal improvement.

A missing experiment is training with FOV conditioning but without FOV augmentation. This would test whether the model can learn to use FOV from the limited natural variation in training datasets (NYU and KITTI have fixed intrinsics, so without augmentation there is essentially zero FOV variation) or whether augmentation is necessary to teach the FOV-to-depth mapping. If FOV conditioning helps even without augmentation, that would suggest the model learns the geometric relationship analytically; if it doesn't help, it confirms that augmentation is the necessary training signal for the conditioning to be useful. The paper does not run this ablation.

Does the paper demonstrate that the expanded training mixture (DMD-MIX) is essential for the strongest zero-shot results?

Supported, but the contribution is underanalyzed. Tables 1-2 clearly show that DMD-MIX substantially outperforms DMD-NK on almost every zero-shot benchmark: iBims-1 REL 0.130 → 0.118, Hypersim 0.382 → 0.318, DIML Outdoor 0.300 → 0.190, DDAD 0.122 → 0.108, DIODE Outdoor 0.627 → 0.553. Figure 5 visualizes these differences, showing improved scale and boundary detail.

However, the training mixture is not ablated systematically. DMD-MIX differs from DMD-NK by adding both Taskonomy and nuScenes — datasets that differ in multiple ways (Taskonomy: diverse indoor scenes with variable cameras; nuScenes: multi-camera outdoor driving with diverse weather/lighting). We cannot determine whether the gains come from: (a) more total training data, (b) the specific diversity of Taskonomy's camera intrinsics, (c) nuScenes' outdoor scene diversity, or (d) the combination. An ablation training with only one additional dataset at a time would clarify which dataset contributes what. Further, ZoeDepth could potentially be fine-tuned on the same expanded mixture, which would control for whether DMD's advantage is from the data scale or the modeling choices. The paper does not report ZoeDepth-MIX results, which means the DMD-MIX vs. ZoeDepth comparison conflates modeling differences with data differences.

The paper also does not discuss whether the improved in-distribution performance of DMD-MIX over DMD-NK (NYU REL: 0.072 vs. 0.076) is surprising or expected. Typically, training on broader data trades off in-distribution accuracy for out-of-distribution robustness; here, both improve. Understanding why — whether it's regularization, better feature learning, or simply Taskonomy being similar enough to NYU to provide beneficial transfer — would strengthen the data diversity argument.

Does the paper demonstrate that v-parameterization enables practical (few-step) inference for depth estimation?

Yes, convincingly. Table 9 is unambiguous: v-parameterization matches ε-parameterization's 64-step accuracy at 1 step on both NYU and KITTI. This is a 64× reduction in denoising steps for equivalent accuracy, making diffusion-based depth estimation practical where DDVM's 128-step requirement was prohibitive. The explanation — that v-parameterization avoids division by near-zero α_t at high noise levels — is clear and mechanistically sound.

Two limitations reduce the practical impact: First, the paper does not report actual inference latency or wall-clock time. While 64× fewer denoising steps implies 64× less neural network forward passes, the v-parameterization model may have a slightly different architecture that affects per-step cost, and the 8-sample averaging adds an 8× multiplier. A wall-clock comparison against ZoeDepth would contextualize whether DMD is genuinely more efficient or whether the diffusion framework's inherent cost still makes it slower despite v-parameterization. Second, Table 9 uses models trained without FOV augmentation or conditioning; it is not confirmed that v-parameterization's 1-step saturation holds when these components are added. The main results use 8 steps for indoor datasets, suggesting that with FOV conditioning and augmentation, additional steps do provide marginal benefit not captured in Table 9.

What genuine weaknesses exist in the experimental design?

1. No statistical significance or confidence intervals. All comparisons are point estimates; with test sets of varying sizes (SUN RGB-D: 5,050 images, iBims-1: 100 images, Hypersim: ~75K images across scenes), the reliability of comparisons differs substantially. The paper should at minimum report per-dataset test set sizes and ideally standard deviations or confidence intervals on the metrics, especially for the smaller benchmarks where sampling error could affect the ranking.

2. The FOV estimator experiment (Table 10) reveals fragility on out-of-distribution FOV. The 28° error on DIML Outdoor and resulting REL degradation (0.190 → 1.049) shows that DMD's FOV conditioning is only as good as the FOV estimate provided. In practice, if FOV is unknown and must be estimated, DMD's performance degrades to below ZoeDepth on challenging FOVs. The paper acknowledges this but treats it as a limitation of the FOV estimator rather than a limitation of the approach — yet in the intended zero-shot deployment scenario, FOV may genuinely be unknown.

3. No baseline giving ZoeDepth the same training data and augmentation. DMD-MIX vs. ZoeDepth trained only on NYU+KITTI is not a fair comparison if the claim is that DMD's modeling choices are responsible for the gains. A ZoeDepth variant trained on the DMD-MIX mixture would control for data scale — without it, we cannot separate data-driven gains from modeling-driven gains in the DMD-MIX results.

4. Single model family, single resolution. All experiments use the Efficient U-Net from DDVM at 384 × 512 resolution. We cannot determine whether the representational insights (log depth, FOV conditioning) transfer to other architectures (ViT-based, convolutional with different backbones) or higher resolutions where fine geometric detail becomes more important. The paper's claim that the architecture is "generic" would be strengthened by showing results with at least one alternative backbone.

5. No exploration of failure cases or limitations beyond the DIML Outdoor FOV issue. The paper does not analyze which types of scenes or conditions cause DMD to fail — low light, reflective surfaces, thin structures, severe occlusions, or images with unusual aspect ratios. A qualitative error analysis or per-category breakdown would contextualize where the remaining errors come from and whether they are fundamentally addressable.

6. Limited ablation of log-depth hyperparameters. The choice of d_min = 0.5 m and d_max = 80 m is stated without justification or sensitivity analysis. If d_max were set to 40 m (closer to typical outdoor training data maximum), would indoor performance improve further at the cost of saturating distant outdoor depths? If d_min were 0.1 m (for extreme close-up scenes), would the representation allocation shift enough to harm outdoor performance? These choices affect the log-range compression and may interact with the FOV augmentation strategy.

7. The 8-sample averaging contributes modestly to accuracy (Table 6) but adds 8× inference cost. The paper does not discuss whether sample averaging is necessary or whether a single sample with slightly more denoising steps would achieve the same accuracy at lower total cost. Since v-parameterization saturates at 1 step (Table 9), the relationship between number of samples and number of steps is underexplored: is 8 samples × 1 step better than 1 sample × 8 steps? The paper uses 8 samples × 8 steps for indoor, which suggests some redundancy that could be optimized.

6. Limitations and Trade-offs

Field-of-View Must Be Known (or Accurately Estimated) at Inference Time — and the Provided Estimator Fails on Out-of-Distribution Cameras

The assumption or constraint. DMD conditions its depth predictions on the vertical field-of-view tan(θ/2), which must be provided at inference time. The paper acknowledges that while camera intrinsics are "available for most practical uses" (phones, robots, self-driving cars), they "may sometimes be unknown (e.g. internet images or generative imagery)" (Appendix B). The proposed solution is a learned FOV estimator trained on NYU and KITTI, but this estimator inherits the same generalization limitations that FOV conditioning was designed to solve — it was trained on a limited range of camera intrinsics and cannot predict FOV values far outside that distribution.

The consequence. When the FOV estimator encounters a camera substantially different from its training distribution, depth estimation degrades catastrophically. Table 10 (Appendix B) reports that on DIML Outdoor — where the FOV estimation error is 28 degrees — DMD-MIX relative error jumps from 0.190 (with true FOV) to 1.049 (with estimated FOV). This is a 5.5× increase, rendering the depth estimates essentially useless. A practitioner deploying DMD on unconstrained imagery (e.g., user-uploaded photos from arbitrary cameras) cannot rely on the provided FOV estimator for scenes where the camera FOV is far from typical smartphone or automotive cameras. The model inherits a circular dependency: it needs FOV to generalize zero-shot, but the FOV estimator itself does not generalize zero-shot.

What evidence exists in the paper. Table 10 is the only evidence, and it clearly shows the failure. FOV estimation error in degrees is reported alongside depth REL for all 10 evaluation datasets. The correlation is instructive: on datasets where FOV error is under ~5° (NYU: 1.5°, DIODE Indoor: 0.7°, KITTI: 0.8°, Virtual KITTI 2: 4.1°), depth REL using estimated FOV is close to the true-FOV result. On DIML Outdoor with 28° error, depth REL collapses. Interestingly, on other datasets with moderate FOV error (SUN RGB-D: 5.1°, DDAD: 4.4°), the depth degradation is present but less severe (SUN RGB-D: 0.091 → 0.136; DDAD: 0.108 → 0.119), suggesting the relationship between FOV error and depth error is nonlinear and becomes severe beyond some threshold. The paper does not characterize this threshold or provide guidance on when the estimator is trustworthy.

Mitigation status. The paper partially acknowledges this limitation, stating that "incorporating more sophisticated camera intrinsic prediction models, such as those proposed by [25, 27], could lead to further improvements in FOV estimation accuracy" (Appendix B), but "defers a thorough investigation of this approach to future work." In practice, a deployer would need to either (a) ensure camera intrinsics are always known (restricting deployment to calibrated systems), (b) integrate a more robust third-party FOV estimator, or (c) accept that DMD's accuracy is bounded by FOV estimation quality on unknown cameras. None of these is a solved problem within the paper's scope, and the headline zero-shot results all assume oracle FOV access, which is the best-case scenario.

---

The Expanded Training Mixture (DMD-MIX) Is Not Ablated, Confounding Data-Scale Gains with Modeling Gains

The assumption or constraint. The paper's strongest results (DMD-MIX) are obtained by training on NYU, KITTI, nuScenes, and Taskonomy — two more datasets than ZoeDepth uses (NYU and KITTI only). The paper claims these gains demonstrate DMD's superiority over ZoeDepth, but the comparison conflates two variables: the modeling choices (log depth, FOV conditioning, FOV augmentation, diffusion framework, v-parameterization) and the training data scale/diversity (Taskonomy's 4.5M diverse indoor images, nuScenes' multi-camera outdoor data). The matched-training comparison (DMD-NK vs. ZoeD-M12-NK, Tables 1-2) controls for data, but the headline 25%/33% indoor/outdoor REL reduction uses DMD-MIX against ZoeDepth trained on only NYU+KITTI.

The consequence. We cannot determine what fraction of DMD-MIX's gains comes from the modeling innovations versus the additional data. Table 1 shows that on SUN RGB-D, DMD-NK achieves REL 0.109 (vs. ZoeDepth 0.123, a 11.4% reduction), while DMD-MIX achieves 0.091 (26.0% reduction) — more than half the total improvement over ZoeDepth comes from expanding the training mixture rather than the core method. If ZoeDepth were trained on the same expanded mixture, it might close much of the gap. The paper provides no ZoeDepth-MIX baseline, which means the central claim — that DMD's generic diffusion framework outperforms ZoeDepth's domain-specific architecture — is supported only under the matched-training condition (DMD-NK vs. ZoeDepth), not under the stronger condition where both methods have equal access to training data scale. A practitioner deciding between DMD and ZoeDepth cannot tell whether DMD's modeling innovations or simply training on more data is responsible for the improvement, and therefore cannot assess whether the method's complexity is justified.

What evidence exists in the paper. The ablation gap is explicit in Tables 1-2, where DMD-NK and DMD-MIX are reported separately but ZoeDepth is only reported in its NYU+KITTI-trained variant (ZoeD-M12-NK). The paper does not discuss why ZoeDepth-MIX was not trained or reported. Figure 5 shows qualitative improvements of DMD-MIX over DMD-NK, but these demonstrate the effect of more training data on DMD — not a comparison with ZoeDepth given the same data. The improved in-distribution performance of DMD-MIX over DMD-NK (NYU REL: 0.072 vs. 0.076; KITTI REL: 0.053 vs. 0.055, Table 4) further suggests data scale is a major factor, since in-distribution metrics benefit from the expanded mixture even though NYU and KITTI are already in the training set.

Mitigation status. Not addressed. The paper treats the expanded training mixture as part of the method ("our variant trained on a mixture... dubbed DMD-MIX") rather than as a separate experimental variable. The matched-training comparison (DMD-NK) partially mitigates the concern — DMD-NK does outperform ZoeDepth on nearly all zero-shot benchmarks (Tables 1-2), establishing that the modeling choices matter independently — but the magnitude of DMD-MIX's additional gains cannot be attributed to modeling, and the paper's headline numbers implicitly credit the method with gains that may be data-driven. A fairer evaluation would require either (a) reporting a ZoeDepth variant trained on the same mixture, or (b) restricting the primary claim to the DMD-NK comparison and treating DMD-MIX as an orthogonal scaling result.

---

No Analysis of Inference Latency or Wall-Clock Time, Making Deployment Feasibility Unknown

The assumption or constraint. The paper measures inference cost solely in "number of denoising steps" and "number of independent samples," and argues that v-parameterization enables efficiency by reducing steps from 128 (DDVM) to 1-8. However, the actual inference latency — the wall-clock time a user waits for a depth map — involves factors that the paper does not measure or report: the per-step cost of the Efficient U-Net (which processes a 384 × 512 image through FiLM-modulated convolutional layers), the 8× multiplier from sample averaging, the overhead of DDPM sampling (even with v-parameterization, each step involves a full forward pass), and the cost of log-to-linear conversion and resolution resizing. No comparison is made against ZoeDepth's inference time, which uses a single forward pass through a MiDaS backbone with two lightweight metric heads.

The consequence. A practitioner cannot determine whether DMD is fast enough for their use case. The paper emphasizes 1-2 step inference as "efficient" (abstract) and "remarkably efficient inference" (Section 3.1), but if the Efficient U-Net denoiser is substantially larger or more expensive per forward pass than ZoeDepth's MiDaS backbone, DMD could be slower at 1 denoising step than ZoeDepth at 1 forward pass. The sample averaging requirement multiplies any per-sample cost by 8×, meaning even a 1-step DMD might require 8 forward passes (one per sample, plus averaging). For latency-sensitive applications — augmented reality (needs depth within ~16 ms for real-time frame rates), mobile robotics (needs depth for control loops at 10-30 Hz), or autonomous driving (safety-critical latency budgets) — this unquantified cost could be prohibitive regardless of accuracy gains. The domain-dependent step count (8 indoor, 2 outdoor) further complicates deployment: an application handling mixed indoor-outdoor scenes needs to decide step count per frame, adding a routing overhead not discussed in the paper.

What evidence exists in the paper. There is none. Table 9 reports number of denoising steps vs. accuracy for ε and v-parameterization, but does not report GPU-hours, FLOPs, or milliseconds per step. The sample averaging table (Table 6) shows accuracy vs. number of samples but not the time cost. No hardware specifications are provided for any timing benchmark. The paper inherits the Efficient U-Net from DDVM [40] and mentions it is "efficient" but provides no architectural detail (parameter count, layer count, FLOPs per inference) that would allow a reader to estimate latency from prior work. The paper's efficiency claim — central to the v-parameterization contribution — rests entirely on the step-count reduction, which is a necessary but not sufficient condition for practical efficiency.

Mitigation status. Not addressed. The paper presents v-parameterization as an efficiency breakthrough without measuring the metric that matters for deployment (wall-clock time). The closest the paper comes to acknowledging this gap is the qualitative statement that "DDVM, for example, uses 128 denoising steps for depth estimation which can be prohibitive" (Section 4.4), implying that reducing steps solves the prohibitive cost — but the per-step cost, which is what ultimately determines whether even 1 step is fast enough, remains unknown. An implementer would need to profile the model on their own hardware before deciding whether DMD meets their latency budget.

---

Hardest Scenes and Failure Cases Are Not Characterized, Leaving the Robustness Envelope Unknown

The assumption or constraint. The paper reports aggregate metrics (REL, RMSE, δ₁) averaged over entire zero-shot test sets, but provides no analysis of when or why DMD fails. Unlike the study of difficulty-dependent behavior for test-time compute scaling in the reference example (where Figure 3 and Figure 7 break performance into difficulty quintiles and show that the hardest problems receive near-zero benefit), DMD does not stratify results by scene type, depth range, lighting condition, camera pose, or any other failure-relevant variable. The qualitative examples in Figures 2, 3, 7, 8 are curated success cases — they show DMD outperforming ZoeDepth on selected images, not the images where DMD performs worst. There is no analysis of what the remaining errors look like, what scene factors correlate with high error, or whether there are systematic failure modes (e.g., transparent surfaces, thin structures, specular reflections, extreme lighting, unusual aspect ratios).

The consequence. A practitioner deploying DMD cannot anticipate when it will fail. The aggregate REL numbers (0.091 on SUN RGB-D, 0.318 on Hypersim) represent average performance; the worst-case performance on some subset of images could be dramatically worse. For safety-critical applications (robotics, autonomous driving), knowing that the average error is low is insufficient — one needs to know whether the model makes catastrophic errors on specific scene types, and if so, with what frequency. The paper provides no failure rate metric (e.g., percentage of images with REL > 0.5, or maximum pixel error), making it impossible to assess worst-case behavior. Additionally, the domain-dependent step count (8 indoor, 2 outdoor) suggests the model behaves differently on indoor vs. outdoor scenes, but the paper does not analyze whether there are indoor scenes that would benefit from fewer steps (e.g., simple empty rooms) or outdoor scenes that need more (e.g., cluttered street scenes with many depth discontinuities). Without failure mode analysis, a deployer cannot implement safeguards (e.g., uncertainty estimation, fallback to a simpler model, human-in-the-loop review) for the regimes where DMD breaks.

What evidence exists in the paper. None systematic. The paper reports only aggregate metrics and curated qualitative comparisons. Table 5 shows that DIML Outdoor performance degrades dramatically without FOV conditioning (REL 1.257), which hints that large-FOV outdoor scenes are a failure mode for the unconditioned model, but the conditioned model's failure modes are not analyzed. Table 10 shows that DMD degrades on out-of-distribution FOV when using estimated intrinsics, but this is a failure of the FOV estimator, not of DMD's depth estimation on in-distribution FOV. The DIODE Outdoor RMSE result — where DMD-NK achieves 9.577 vs. ZoeDepth 7.569 despite better REL — suggests DMD may produce outlier predictions at far distances that inflate RMSE, but this is not investigated. The paper does not include per-image error histograms, per-depth-range error analysis, or per-scene-category breakdowns.

Mitigation status. Not addressed. The paper treats uniform improvement across all benchmarks as sufficient evidence of robustness, but aggregate metrics on standard benchmarks can mask systematic failures on minority sub-populations. This limitation is common in the depth estimation literature — ZoeDepth similarly reports only aggregate metrics — but it is nonetheless a gap for anyone considering deployment. Future work could address this by analyzing failure modes, perhaps through depth-range-stratified metrics, per-semantic-category error analysis, or worst-case performance characterization. The paper offers no path toward such analysis.

---

FOV Augmentation Range and Parameters Are Chosen Heuristically, With Unknown Sensitivity

The assumption or constraint. The FOV augmentation strategy — which the paper argues is essential for preventing overfitting to training camera intrinsics — uses a uniformly sampled scale factor s ∈ [0.8, 1.5] applied during the final fine-tuning stage only, with uncropped RGB borders filled with Gaussian noise (mean 0, variance 1) and missing depth filled via near-neighbor in-filling with step-unrolled denoising (Section 3.2). The paper states that the scale range was chosen empirically and that Gaussian noise padding "works as well" as generative uncropping based on preliminary experiments, but provides no ablation over these choices: different scale ranges (e.g., [0.5, 1.5], [0.9, 1.1], or asymmetric ranges), different padding strategies (constant value, reflection padding, learned inpainting), whether FOV augmentation should be applied in earlier training stages, or the interaction between augmentation range and the d_min/d_max log-depth parameters.

The consequence. A practitioner cannot determine whether DMD's generalization depends critically on the specific augmentation hyperparameters or is robust to them. If DMD were deployed in a setting with cameras even further from the training distribution than DIML Outdoor (e.g., 180° fisheye lenses, extreme telephoto), the chosen augmentation range [0.8, 1.5] may be insufficient to prepare the model. The paper does not characterize whether performance degrades gracefully or catastrophically when test FOV exceeds the augmentation range — the DIML Outdoor result (Table 5: REL 0.300 with conditioning) shows the model handles FOV outside the training range better than without augmentation, but we do not know the limits. If a practitioner wanted to adapt DMD to, say, drone imagery with downward-facing wide-angle cameras, they would need guidance on whether to adjust the augmentation range or training procedure, and the paper provides none.

What evidence exists in the paper. Minimal. The only ablation involving FOV augmentation is Table 7, which compares training with full FOV augmentation and conditioning against training with neither. This establishes that some augmentation is beneficial but does not explore sensitivity to the augmentation parameters. The paper mentions preliminary experiments comparing Gaussian noise padding to Palette-based generative uncropping (Section 3.2), finding them equivalent, but does not present these results or explore alternative padding methods. The interaction between FOV augmentation range and log-depth range selection is entirely unexplored — for example, uncropping to s = 1.5 expands the visible area but does not add new depth values (depths in the uncropped border are extrapolated from the image boundary), which may create a mismatch between the apparent FOV and the depth distribution that the log mapping is designed to handle.

Mitigation status. The paper does not address this directly. The choice to apply augmentation only during the final fine-tuning stage is motivated implicitly (Section 4.1: earlier stages focus on learning depth features without FOV variation), but the sensitivity to the specific augmentation range and padding strategy is not discussed as a limitation. A robustness analysis — sweeping augmentation range and measuring zero-shot performance — would clarify whether these choices are critical or merely convenient. The paper's suggestion that more sophisticated FOV estimation models could improve robustness (Appendix B) implicitly acknowledges that the current augmentation strategy leaves some generalization gaps, but does not quantify them.

---

Single Architecture, Single Resolution, Single Task Domain — No Evidence of Transferability

The assumption or constraint. All experiments use a single model architecture — the Efficient U-Net denoiser inherited from DDVM [40] with FiLM layers for FOV conditioning — trained and evaluated at a single resolution (384 × 512), on a single task (monocular depth estimation from RGB images), using a single dataset family (real and synthetic indoor/outdoor scenes from standard depth estimation benchmarks). The paper claims the approach is "generic" and "task-agnostic" (abstract, Section 1), but provides no evidence that the key innovations — log-scale parameterization, FOV conditioning, v-parameterization for few-step inference — transfer to other architectures (ViT-based backbones, discriminative rather than generative models), other resolutions (high-resolution 1024 × 2048 common in autonomous driving, or low-resolution for embedded deployment), or other dense regression tasks (surface normals, optical flow, multi-view stereo).

The consequence. A practitioner cannot determine whether DMD's design principles are broadly applicable or specific to the Efficient U-Net diffusion framework at 384 × 512 for depth estimation. If a team has an existing depth estimation pipeline based on a different architecture (e.g., a transformer-based model like DPT), they cannot assess whether adding log-depth parameterization and FOV conditioning would yield similar improvements without reimplementing the entire diffusion framework. If a team needs higher-resolution depth maps (e.g., for close-range robotic manipulation where millimeter-level precision matters at 1024 × 1024 resolution), they do not know whether the method scales — the diffusion model's memory and compute requirements grow with resolution, and the v-parameterization's 1-step saturation was demonstrated only at 384 × 512. If a team works on a related task like surface normal estimation, they do not know whether log-parameterization (which specifically addresses the indoor/outdoor depth range imbalance) has an analog for normal estimation, or whether the principle of capacity-aware output encoding generalizes.

What evidence exists in the paper. None beyond the depth estimation results. The paper's claim of "task-agnostic" refers to the fact that the architecture lacks depth-specific heads or loss functions, not to demonstrated performance across multiple tasks. The Efficient U-Net was shown to work for optical flow and depth in DDVM [40], but DDVM did not use log-depth, FOV conditioning, or v-parameterization, so the transferability of these innovations — which constitute DMD's contributions — is untested. The resolution dependence is not explored at all.

Mitigation status. Not addressed. The paper's contributions (log depth, FOV conditioning, FOV augmentation, v-parameterization, training mixture) are all evaluated exclusively in the context of the Efficient U-Net diffusion model for depth estimation. The claims of genericity and task-agnosticism should be interpreted as applying to the method within the depth estimation domain (i.e., it handles indoor and outdoor, diverse cameras, without domain-specific components), not as evidence that the approach transfers to other architectures, tasks, or resolutions. A practitioner interested in adapting DMD's ideas to a different setting would need to run their own validation experiments, as the paper provides no guidance on expected transferability.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper fundamentally reframes what is necessary to solve zero-shot metric depth estimation, shifting the field's focus from architectural specialization toward representation design and geometric conditioning. The dominant assumption codified by ZoeDepth — that joint indoor-outdoor metric depth requires domain-specific model components (separate metric heads, scene-type routers) — is convincingly overturned. DMD demonstrates that a single, architecturally generic denoising diffusion model can outperform ZoeDepth's multi-head design by 25% on indoor and 33% on outdoor zero-shot benchmarks (Tables 1-2) when three representation-level fixes are applied: log-scale depth to balance capacity across shallow and deep ranges, FOV conditioning to resolve scale ambiguity, and FOV augmentation to prevent overfitting to training-camera intrinsics.

The magnitude of this reframing is significant but bounded — it is not a paradigm shift in the sense of introducing a new model class (diffusion for depth was established by DDVM), but it is a diagnostic breakthrough that reveals why prior joint models failed. The paper's evidence resolves a tension that had persisted in the literature: MiDaS showed that a single model could learn joint indoor-outdoor representations (albeit scale-invariant), while ZoeDepth showed that converting these to metric depth seemed to require domain-specific heads. DMD's ablation in Table 8 cuts through this apparent contradiction — ZoeDepth's domain-specific heads were compensating for a failure of scale disambiguation, not a fundamental incompatibility of indoor and outdoor depth distributions. When DMD is provided with FOV as a conditioning signal, it learns a unified representation that handles both domains without architectural routing, and its in-distribution performance actually improves with broader training data (DMD-MIX outperforms DMD-NK on both NYU and KITTI, Table 4), confirming that domain-specific heads were limiting rather than enabling generalization.

The paper also recalibrates the perceived difficulty of diffusion-based dense prediction. DDVM established that diffusion models could perform depth estimation competitively but required 128 denoising steps, making them computationally prohibitive. DMD's demonstration that v-parameterization achieves state-of-the-art accuracy with 1-2 denoising steps (Table 9: v-parameterization at 1 step matches ε-parameterization at 64 steps) removes the primary barrier to practical adoption. This finding makes diffusion-based depth estimation genuinely competitive with single-forward-pass discriminative models on an efficiency basis, opening the door to diffusion's advantages — uncertainty quantification, sample diversity, principled handling of ambiguity — without the previously prohibitive computational cost. The explanation of why v-parameterization matters specifically for dense regression (avoiding division-by-near-zero α_t at high noise levels, which would amplify errors in large-scale structure recovery) provides a transferable insight for other dense prediction tasks like optical flow and surface normal estimation.

Research directions that become more attractive after this work:

• Representation-first model design for geometric tasks, where the primary research question shifts from "what architecture handles domain X?" to "what input/output parameterization makes the geometric relationship learnable by a generic architecture?" The paper's log-depth and FOV conditioning results provide a template for this approach that could be applied to related ambiguities in other tasks.
• Camera-aware monocular estimation broadly, since FOV conditioning is shown to be the key enabler of zero-shot generalization. The principle that augmentation without conditioning turns diversity into label noise (Table 5: without FOV conditioning, DIML Outdoor REL is 1.257; with conditioning, 0.300) is a design rule that should transfer to any task where the augmented variable affects the input-output mapping.
• Efficient diffusion for dense regression, since v-parameterization's 64× step reduction has now been demonstrated for depth.

Research directions that become less attractive:

• Architectural specialization for domain adaptation in depth estimation. The paper shows that adding domain-specific heads (ZoeDepth's approach) provides no benefit over proper geometric conditioning when evaluated on diverse zero-shot benchmarks. Future work on handling new domains (underwater, aerial, medical imaging) should invest in understanding the geometric conditioning signals relevant to those domains rather than designing new output heads.
• Scale-invariant depth as a necessary intermediate representation. DMD achieves state-of-the-art metric depth without first predicting scale-invariant depth then converting (as MiDaS→ZoeDepth does). The log-depth parameterization directly handles the indoor-outdoor range imbalance without the information loss that scale normalization introduces, suggesting that scale-invariant pretraining may be an unnecessary detour.

Follow-Up Research This Work Enables

Ablating the interaction between FOV conditioning and FOV augmentation in isolation. The paper demonstrates that FOV conditioning with FOV augmentation is dramatically better than neither (Table 7), and that FOV conditioning with augmentation is better than augmentation without conditioning (Table 5, comparing DMD-NK with and without FOV conditioning, where FOV augmentation is active in both). What remains unknown is whether FOV conditioning without augmentation would provide any benefit — i.e., if the model is trained only on NYU and KITTI (fixed intrinsics, zero FOV variation) but receives the FOV conditioning signal, can it learn the geometric relationship between FOV and depth scale analytically from the pinhole camera model, or does it require exposure to variable FOV during training to learn to use the signal? A pair of experiments — DMD trained on NYU+KITTI with FOV conditioning but without FOV augmentation, evaluated on DIML Outdoor — would distinguish between these hypotheses. If FOV conditioning without augmentation helps (reducing DIML Outdoor REL from 1.399 to, say, 0.800), the model has learned some geometric reasoning; if it doesn't help (staying near 1.399), FOV augmentation is a necessary training signal, confirming that the model learns purely empirically from data diversity rather than from an inductive bias toward the pinhole projection geometry.

Testing whether DMD's design principles transfer to discriminative architectures. The paper's thesis — that log-depth parameterization, FOV conditioning, and FOV augmentation enable a generic architecture to surpass specialized ones — is demonstrated only for the diffusion-based Efficient U-Net. A critical test of generality would be to apply the same three mechanisms to a discriminative architecture, such as a DPT (Vision Transformer) backbone or a standard convolutional encoder-decoder, and measure whether the zero-shot gains replicate. Specifically: take a DPT model trained on the same DMD-MIX datasets, add FOV conditioning via FiLM or concatenation, adopt log-depth output parameterization with an appropriate loss function (scale-invariant log loss, or L1 in log space), and apply the same FOV augmentation during training. If the DPT variant matches DMD's zero-shot performance, the representation innovations generalize across architectures; if it underperforms, the diffusion framework's generative formulation provides complementary benefits (uncertainty handling, multi-modality) beyond the representation choices. This experiment would clarify whether practitioners can adopt DMD's ideas incrementally into existing depth estimation pipelines or whether the full diffusion framework is necessary.

Systematically characterizing what DMD's remaining errors look like and where the FOV-conditioned model fails. The paper reports aggregate metrics on zero-shot benchmarks but provides no failure mode analysis. A high-value follow-up would stratify errors by scene geometry: depth range (0-2m, 2-5m, 5-10m, 10-30m, 30-80m), semantic category (sky, building, vegetation, vehicle, person), camera pitch angle (downward-looking, forward-looking, upward-looking), and FOV distance from training distribution. The DIODE Outdoor result — where DMD-NK achieves better REL than ZoeDepth but worse RMSE (Table 2: REL 0.627 vs. 0.757, but RMSE 9.577 vs. 7.569) — hints that DMD may trade off close-range accuracy (which dominates REL) against distant-region stability (which dominates RMSE). A per-depth-range error analysis would confirm or refute this hypothesis and identify the specific regimes where DMD underperforms. Additionally, running DMD on the DIML Outdoor test set and reporting per-image REL histograms (rather than just the mean) would reveal whether the 0.300 REL is achieved uniformly across images or is driven by a few catastrophic failures offsetting many near-perfect predictions. This analysis would give practitioners concrete guidance on DMD's safe operating envelope.

Exploring whether log-depth parameterization transfers to other dense regression tasks with imbalanced output distributions. The paper's log-depth insight — that the output parameterization should be matched to the evaluation metric's sensitivity (relative error cares about ratios, so the output space should be logarithmic) — has natural analogs in other tasks. Surface normal estimation: angular error is the standard metric, which is invariant to vector magnitude but sensitive to direction, suggesting that predicting normalized vectors with an appropriate spherical parameterization might outperform direct xyz regression. Optical flow: endpoint error is measured in pixels, but large displacements and small displacements have different perceptual significance, suggesting a log-flow or scale-adaptive parameterization. A follow-up study could apply the log-parameterization principle to one of these tasks, train a DMD-style diffusion model with the adapted output encoding, and measure whether zero-shot generalization improves over standard linear parameterization. The specific hypothesis is that tasks where the evaluation metric is scale-invariant or relative will benefit from logarithmic-like output encodings, while tasks where absolute error matters uniformly (e.g., object detection bounding box regression in pixels) will not. This would elevate the paper's log-depth finding from a depth-specific trick to a general design principle for dense regression.

Training and evaluating a FOV estimator that generalizes to the FOV range DMD encounters at test time. Appendix B shows that the provided FOV estimator fails catastrophically on DIML Outdoor (28° error, causing depth REL to degrade from 0.190 to 1.049 for DMD-MIX), which is precisely the kind of out-of-distribution camera that zero-shot deployment will encounter. A practical follow-up would train a FOV estimator on a deliberately broad mixture of datasets spanning diverse camera types — wide-angle action cameras (GoPro, ~120° horizontal FOV), smartphone cameras (varying focal lengths across models), DSLR lenses, fisheye lenses (180°+), and the existing NYU/KITTI cameras — with FOV augmentation applied during training to interpolate between seen FOVs. The evaluation would measure FOV estimation error and downstream DMD depth REL on held-out cameras not seen during FOV estimator training, including DIML Outdoor. A strong result would be FOV estimation error under 5° on all test cameras and downstream DMD depth REL that stays within 10% of the oracle-FOV result. This would close the most significant deployment gap in the current method.

A FLOPs-matched or latency-matched comparison between DMD and ZoeDepth at equivalent accuracy levels. The paper's efficiency argument is entirely qualitative (v-parameterization reduces denoising steps, therefore DMD is efficient). A rigorous comparison would: profile wall-clock inference time for DMD (at 1, 2, 4, and 8 denoising steps, with and without 8-sample averaging) and ZoeDepth (single forward pass, with and without auto-router overhead) on identical hardware; measure FLOPs per inference for both models; and plot the accuracy-vs-latency Pareto frontier for both methods across the zero-shot benchmarks. The specific question is whether DMD at 1 denoising step with 1 sample (the fastest configuration) matches or exceeds ZoeDepth's accuracy, and if not, how much additional compute (more steps, more samples) is needed to surpass it. This analysis would replace the paper's suggestive but incomplete efficiency narrative with actionable deployment guidance: for a given latency budget, which method should a practitioner choose?

Practical Applications and Downstream Use Cases

Autonomous driving on roads not seen during training, using novel camera rigs. For self-driving companies deploying across geographies with different sensor configurations than their training fleet, DMD's FOV conditioning enables a single model to produce metric depth from any camera without per-rig recalibration or fine-tuning. The DIML Outdoor result — where DMD-MIX achieves REL 0.190 vs. ZoeDepth's 0.641, a 70.4% reduction on a camera with significantly different FOV than common autonomous driving datasets (KITTI, nuScenes) — demonstrates that DMD maintains depth accuracy even when the camera intrinsics differ substantially from training. A deployment team could integrate DMD into their perception stack, provide the known camera intrinsics as the FOV conditioning input, and expect consistent depth estimation across vehicle generations or sensor suppliers without retraining.

Mobile robotics operating in mixed indoor-outdoor environments with consumer-grade cameras. For delivery robots, vacuum robots, or inspection drones that transition between indoor and outdoor settings and carry cameras with known but variable intrinsics (e.g., different smartphone models used as robot vision sensors), DMD's joint indoor-outdoor architecture eliminates the need for separate models per domain or scene-type classifiers that can fail during transitions. A single DMD model, conditioned on the device's reported vertical FOV, produces metric depth in both environments — the indoor results (SUN RGB-D REL 0.091, iBims-1 REL 0.118 for DMD-MIX) and outdoor results (Virtual KITTI 2 REL 0.092) show strong performance in both regimes. The 1-2 step inference capability (Table 9) is critical here: mobile robots have tight power and latency budgets, and the v-parameterization makes diffusion-based depth potentially viable where DDVM's 128-step requirement would exceed the computational budget.

Augmented reality on consumer smartphones with known camera intrinsics. AR applications — virtual furniture placement, measurement tools, navigation overlays — require metric-scale depth to correctly position virtual content in the real world. Smartphone APIs typically provide camera intrinsics (focal length, sensor size) that can be converted to vertical FOV, making DMD directly deployable on devices without needing the FOV estimator. The strong indoor performance (NYU in-distribution REL 0.072 for DMD-MIX, Table 4) and the ability to handle diverse indoor environments zero-shot (SUN RGB-D, iBims-1, Hypersim) means a single DMD model could serve AR applications across many device models and room types without per-device or per-scene calibration. The sample averaging cost (8× inference, Table 6) may be prohibitive for real-time AR at 30-60 fps, but the paper shows single-sample performance is only marginally worse (NYU REL 0.077 vs. 0.076 for 8 samples), making a 1-sample, 1-step configuration viable for real-time use if the Efficient U-Net's per-step latency is low enough.

Monocular depth as a pre-training target for general-purpose visual representations. The paper's training strategy — unsupervised pre-training on ImageNet and Places365, followed by supervised depth pre-training on diverse datasets (ScanNet, SceneNet-RGBD, Waymo, DIML Indoor), followed by task-specific fine-tuning — mirrors the successful paradigm in NLP where language models are pre-trained on broad data then fine-tuned. DMD's demonstration that the depth pre-training stage produces features that transfer to diverse zero-shot benchmarks (indoor and outdoor, multiple camera types) suggests that metric depth estimation, when properly conditioned on camera geometry, may serve as an effective pre-training task for visual representations that require 3D understanding — such as navigation, manipulation, or visual question answering about spatial relationships. The key enabling insight is that FOV conditioning prevents the depth pre-training from overfitting to training-camera statistics, producing features that genuinely encode scene geometry rather than camera-specific depth heuristics. The specific experiment this suggests: take DMD's pre-trained denoiser backbone (after Stage 2, before final fine-tuning), freeze it, attach task-specific heads for surface normal estimation, semantic segmentation, or object detection, and fine-tune on limited target data. If DMD's depth pre-training provides a better initialization than ImageNet pre-training for 3D-aware tasks, it would position metric depth estimation as a foundational pre-training objective for embodied vision.