ArXiv: 2512.00956
🎯 Pitch
Contrary to the belief that fixed Hadamard rotations just 'spread outliers,' this work proves the optimal quantizing transform is non-orthogonal and data-dependent, deriving a closed-form solution that closes most of the accuracy gap between MXFP4 and NVFP4 without sacrificing GPU throughput.
1. Executive Summary
This paper introduces WUSH, a closed-form, data-aware linear blockwise transform for joint weight-activation quantization of LLMs, combining a Hadamard backbone with a second-moment component derived from calibration statistics to form a non-orthogonal transform that is provably near-optimal for both FP and INT block quantizers. Evaluated on Llama-3.1/3.2 and Qwen3 families using the LM Evaluation Harness and Platinum Benchmarks, WUSH improves W4A4 accuracy over the strongest Hadamard-based baselines by up to +2.8 average points with RTN and +0.7 points with GPTQ on Llama-3.1-8B-Instruct in MXFP4, while delivering up to 6.6× per-layer throughput over BF16 via a fused FP4 MatMul kernel whose overhead matches that of optimized blockwise Hadamard kernels. The construction reveals that the Hadamard transform's empirical success stems from being the single data-agnostic component of a more general optimal transform, establishing that principled data-aware transforms can nearly close the accuracy gap between MXFP4 and NVFP4 formats while preserving full GPU efficiency.
2. Context and Motivation
The Core Problem: Outliers Undermine Low-Bit Quantization
The central difficulty this paper tackles is deceptively simple: a small number of extreme values in LLM weights and activations can single-handedly ruin the accuracy of low-bit quantization. When you quantize a neural network to 4 bits (or lower), every representable value in a quantization group shares the same scale factor. The scale is typically determined by the maximum absolute value in that group — the AbsMax scheme. This means a single outlier in a block of, say, 32 values effectively dictates the dynamic range for all 32 numbers, wasting precious quantization levels on accommodating the extreme value while compressing the vast majority of inlier values into just a few representable steps.
This is not a minor edge case. LLM weights and activations exhibit heavy-tailed distributions — most values are small, but a tiny fraction are orders of magnitude larger. Figure 4 in the paper's appendix illustrates the geometry: an anisotropic 2D distribution where the marginals look different depending on the axis alignment. When the data is aligned with the coordinate axes (top row), one marginal has a long tail. After a 45° rotation (bottom row), the two marginals become nearly identical, but — and this is the key insight the paper emphasizes — the joint distribution retains the same eigenvalues. Extreme points in the principal directions remain extreme in the joint space; they are merely "hidden" from any single marginal view.
Quantization error in group-wise AbsMax schemes is fundamentally a multivariate problem, not a univariate one. The relevant quantity is how mass concentrates along dominant directions and how extreme points interact with group-wise scaling. A purely marginal analysis — "do the coordinate-wise distributions look Gaussian?" — cannot capture this, because it fails to account for the eigenvalue structure of the within-group covariance.
Why This Problem Matters Now
The paper's timing is motivated by a confluence of hardware and format developments that have made low-bit quantization simultaneously more necessary and more nuanced:
The push to 4-bit inference. Deploying LLMs at scale is memory-bandwidth-bound. Moving from BF16 (16 bits per value) to INT4 or FP4 (4 bits) promises up to 4× memory savings and, on hardware with native low-precision matrix multiplication support, up to 4× throughput improvement. However, this compression ratio comes at a steep accuracy cost: 4-bit quantization of both weights and activations ("W4A4") is considerably more challenging than weight-only or weight+activation 8-bit quantization, because activations exhibit dynamic, per-token outlier patterns that cannot be amortized over static weight statistics.
New block-level floating-point formats. The recent emergence of microscaling formats — specifically MXFP (from the Open Compute Project's MX Alliance, 2023) and NVFP (NVIDIA's Blackwell architecture, 2024) — has changed the landscape. These formats apply separate shared exponents to small blocks of values (typically 32 elements for MXFP, 16 for NVFP), enabling fine-grained dynamic range adaptation within a layer. This makes blockwise transforms particularly relevant, because the transform block size can be aligned with the quantization group size, localizing the outlier mitigation to exactly the granularity at which scales are computed.
The MXFP vs. NVFP gap. A practical problem that pervades the recent quantization literature is that MXFP4 consistently underperforms NVFP4 in accuracy, despite both being 4-bit FP formats with comparable bit budgets. Egiazarian et al. (2025) showed that existing Hadamard-based transforms provide limited gains for MXFP4 and can even be harmful for NVFP4 (due to NVFP4's top-element preservation effect). Closing this gap — making MXFP4 competitive with NVFP4 — would be significant because MXFP is an open standard with broader hardware support potential, while NVFP is NVIDIA-proprietary.
Latency constraints on activation transforms. Unlike weights, which are quantized offline, activations must be transformed and quantized dynamically at inference time. Any transform applied to activations must have an efficient GPU kernel that can be fused with the quantization step without becoming a bottleneck. This constrains the design space: the transform must be block-diagonal (to keep the computational cost within the block level), the block size must align with the quantization group size, and the transform must admit implementation as a GEMM-equivalent computation that reuses existing efficient matrix-multiplication primitives.
Where Prior Approaches Fall Short
The paper identifies four distinct families of prior work, each with specific limitations that motivate the WUSH approach:
1. Non-uniform bitwidth and explicit outlier storage (LLM.int8(), SpQR, QUIK, HPTQ). These methods (Dettmers et al., 2022; Dettmers et al., 2024; Ashkboos et al., 2024a; Chen et al., 2025) treat outliers as special cases: they separate extreme values, store them at higher precision, and quantize the remaining inlier values more aggressively. Huffman encoding (HPTQ) compresses the outlier storage to reduce overhead. These approaches can achieve high accuracy at low effective bitwidth. Their limitation: the resulting data formats are irregular — the mix of precisions, the variable-length encoding, and the separation of outlier indices from values create specialized memory access patterns. This makes them difficult to accelerate with standard GPU kernels that expect uniform, coalesced memory layouts. In practice, the throughput gains are often much lower than what the bitwidth reduction would suggest.
2. Channel-wise rescaling (SmoothQuant, AWQ). These methods (Xiao et al., 2023; Lin et al., 2024) apply per-channel scaling factors to balance the dynamic ranges of weights and activations — essentially, they shift quantization difficulty between the weight and activation tensors. SmoothQuant, for instance, introduces a per-channel smoothing factor that can be mathematically absorbed into adjacent linear operations. Their limitation: rescaling is a diagonal (elementwise) operation in the channel space. It does not mix channels, so it cannot redistribute outlier energy across dimensions. The fundamental geometry of the data — the eigenvalue spectrum of each block's second-moment matrix — remains unchanged. Rescaling can balance weight and activation ranges but cannot reduce the effective anisotropy within each tensor.
3. Orthogonal transforms for "incoherence processing" (QuIP, QuIP#, QTIP, QuaRot). This line of work (Chee et al., 2023; Tseng et al., 2024a; Tseng et al., 2024b; Ashkboos et al., 2024b) applies fixed orthogonal transforms — typically Hadamard matrices — to weights and activations before quantization. The intuition is that an orthogonal rotation spreads the energy of any single extreme value across all channels, making the resulting per-channel distributions more "Gaussian-like" and less outlier-heavy. QuaRot, in particular, showed that Hadamard rotations can be applied end-to-end in transformer architectures with minimal modifications to the computational graph. Their key limitation — and the central theoretical gap this paper addresses: these transforms are data-agnostic. The Hadamard matrix is fixed; it does not depend on the statistics of the specific weights or activations being quantized. The paper asks a pointed question that prior work had not answered:
"if the Hadamard transform is not data-aware, in what sense can it be considered optimal for quantization?"
Section 4 of the paper provides the answer: the Hadamard matrix is the single data-agnostic component in a more general optimal transform, and its empirical success comes from being a good default rotation — but one that leaves substantial room for improvement when the data exhibits strong anisotropy (eigenvalue spread).
4. Learned transforms (SpinQuant, FlatQuant). These methods (Liu et al., 2025; Sun et al., 2025) go beyond fixed rotations by learning transforms from calibration data, typically via gradient-based optimization of a quantization-aware objective. SpinQuant learns per-layer rotation matrices, while FlatQuant optimizes for "flatness" of the transformed weight distribution. Their limitations are twofold. First, they are heuristic: the optimization procedure is not derived from first principles of quantization error, so it provides no guarantee of optimality and no insight into what structure the learned transforms should have. Second, they are costly to learn: gradient-based optimization over the full transform matrices must be repeated for each model and can be computationally expensive, limiting practicality for large models. The paper positions itself in direct contrast: "our approach is closed-form and calibration-based, as it does not require training or fine-tuning the entire model."
The Specific Gap: Blockwise Optimal Transforms Under Group Quantization
A crucial practical detail that prior theoretical work often overlooked is the alignment between transform block size and quantization group size. In modern FP formats (MXFP, NVFP), quantization scales are computed per block of contiguous elements (e.g., 32 for MXFP4). If the transform operates at a larger granularity — say, across an entire layer — then within each quantization block, the values may still exhibit heavy-tailed distributions even after the global rotation. The paper cites Shao et al. (2025), which showed that blockwise Hadamard transforms can be more effective under FP formats than full-layer transforms.
Egiazarian et al. (2025) provided the most direct prior evidence for the importance of blockwise transforms, showing that blockwise Hadamard offers the best empirical performance for quantization among a large set of existing transforms. However, they also highlighted the limited gains achievable with Hadamard alone, linking this limitation to "the properties of the underlying weight and activation distributions." This sets up the paper's contribution precisely: what if we could go beyond the fixed Hadamard to a transform that adapts to those underlying properties — specifically, the second-moment (covariance) structure of each block — while remaining computationally tractable?
How This Paper Positions Itself
The paper frames its contribution as a theoretical resolution of a practical puzzle. The puzzle: Hadamard rotations work well, but not perfectly. Transform-based quantization has been largely empirical — try different rotations, see what sticks. The paper asks: can we derive the provably optimal linear transform for a given block under a given quantizer, and if so, what does it look like?
The answer turns out to have a specific structure that explains the empirical landscape:
-
The Hadamard matrix is the optimal fixed rotation — it uniformly distributes energy across dimensions in a way that minimizes worst-case variance (formally, it equalizes the diagonal elements of the transformed covariance; see Eq. 56 in the appendix). This is why it outperforms other fixed orthogonal transforms in prior work.
-
But orthogonality alone is insufficient — for FP and INT quantization, the optimal transform is generally non-orthogonal. The transform must rescale dimensions based on the singular values of the weight-activation interaction matrix to achieve optimal quantization error. This is captured by the term in Eq. (8), which applies differential scaling — shrinking high-variance directions and expanding low-variance ones — to equalize the per-dimension quantization noise.
-
The data-aware components are second-order statistics — the and matrices in Eq. (8) come from Cholesky decompositions of and , respectively. These are cheap to compute from calibration data and have a clear interpretation: they "whiten" the weight and activation covariances, removing the anisotropy that causes outlier-dominated scaling.
-
The construction integrates naturally with GPTQ — since GPTQ also uses the Hessian (proportional to ) for second-order error correction, the WUSH transform uses the same calibration statistics that GPTQ already computes. Algorithm 1 interleaves transform construction with GPTQ's iterative weight updates.
The paper's position is thus: it provides the first closed-form optimal linear blockwise transforms for joint weight-activation quantization, with separate analyses for FP (Section 4.2) and INT (Section 4.3) quantizers, yielding a single construction (WUSH) that is provably optimal for FP and asymptotically near-optimal (up to a dimension-dependent factor) for INT, while being implementable with negligible GPU kernel overhead relative to standard Hadamard transforms.
3. Technical Approach
This is primarily a theoretical analysis paper with an empirical validation component, whose core idea is that the optimal linear blockwise transform for minimizing quantization error under standard RTN AbsMax-scaled block quantizers has a closed-form structure combining a fixed Hadamard rotation with data-dependent scaling derived from the second-moment statistics of weights and activations.
3.1 Reader Orientation
The paper derives a system for transforming weights and activations before quantization so that, for a given linear layer in an LLM, the squared error between the original output (full-precision matrix multiplication) and the quantized output (quantized weights times quantized activations) is minimized. The solution takes the form of a per-block matrix $T_{\text{wush}(i)}$ (for weights) and its inverse transpose $T_{\text{xvsh}(i)}$ (for activations, named "xvsh" as the activation-side counterpart), each constructed from three pieces: a Cholesky factor of the weight or activation second moment, an SVD of their interaction, and a fixed Hadamard matrix. The transform is non-orthogonal, data-aware, closed-form (no gradient-based learning), and admits an efficient fused GPU kernel.
3.2 Big-Picture Architecture (Diagram in Words)
The system operates on a single linear layer at a time and has five major components:
- Calibration Data Collector — runs a small batch of calibration inputs through the model (or through previously quantized layers) to collect activation tensors
$X$for each layer. - Second-Moment Computer — computes the uncentered second moments
$d^{-1}_{\text{batch}} X X^\top$and$d^{-1}_{\text{out}} W W^\top$for each block, which capture the covariance structure of activations and weights respectively. These are exactly the statistics needed for both the WUSH transform construction and GPTQ's Hessian-based error correction. - Transform Constructor — for each block
$i$, performs the sequence: Cholesky decomposition of both second moments to get$W'_{(i)}$and$X'_{(i)}$, SVD of their product to get$U_{(i)} S_{(i)} V^\top_{(i)}$, then assembles$T_{\text{wush}(i)} = H S^{-1/2}_{(i)} U^\top_{(i)} W'^\top_{(i)}$where$H$is a fixed normalized Hadamard matrix. - Weight Pre-Quantizer — applies the WUSH transform to the weight block, then quantizes the transformed weights using either RTN (round-to-nearest) or GPTQ (with error propagation). For GPTQ, the transform construction and weight quantization are interleaved: the Hessian used for GPTQ's second-order correction is the same
$X X^\top$used for the transform. - Online Activation Transform + Quantize Kernel — during inference, each incoming activation block
$\tilde{X}_{(i)}$is multiplied by$T_{\text{wush}(i)}$and then quantized on-the-fly. This is implemented as a GEMM-equivalent operation (each block is an independent small matrix multiplication) using a CUTLASS template, with the transform matrices stored in a$(G, G, C)$layout where$G$is block size and$C$is the number of blocks.
Information flows as follows: calibration activations are collected layer-by-layer → second moments are computed per block → for each block, Cholesky + SVD produce the transform → weights are transformed and pre-quantized (with optional GPTQ error propagation to subsequent blocks) → at inference, activations flow through the fused transform+quantize kernel → the quantized weights and quantized activations are multiplied in low precision → the blockwise outputs are summed to produce the layer output.
3.3 Roadmap for the Deep Dive
- First, the problem formulation (Section 3.4.1): how the quantization loss is defined, why the layerwise loss decomposes into independent blockwise losses under the block-diagonal constraint, and what variables we are optimizing over.
- Second, the probabilistic reformulation (Section 3.4.2): why treating columns of
$W$and$X$as i.i.d. samples enables a tractable analysis, and how the two-sided weight-activation quantization problem reduces to a one-sided problem in a transformed space. - Third, the transformation parameterization (Section 3.4.3): how the unknown optimal transform
$T$is expressed via its SVD in terms of three unknown matrices$U'$,$S'$,$R$, and why this parameterization separates the problem into orthogonal, scaling, and alignment components. - Fourth, the FP quantizer analysis (Section 3.4.4): the specific quantization error model for floating-point formats (error proportional to value), the resulting closed-form optimization, and why the solution is
$U' = H$,$S' = S^{1/2}$,$R = I$. - Fifth, the INT quantizer analysis (Section 3.4.5): the error model for integer formats (error proportional to block maximum), the bounds that lead to the same optimal configuration (up to dimension-dependent factors), and why the Hadamard is essential for tightening the upper bound.
- Sixth, the GPTQ integration (Section 3.4.6): how the WUSH transform construction is interleaved with GPTQ's iterative weight updates in Algorithm 1, and why the same Hessian serves both purposes.
- Seventh, the GPU kernel design (Section 3.4.7): the storage layout, the GEMM mapping, and why per-block specialization adds negligible overhead over uniform Hadamard transforms.
3.4 Detailed, Sentence-Based Technical Breakdown
3.4.1 Problem Formulation and Blockwise Decomposition
The paper begins by formalizing weight-activation quantization as an optimization over linear transforms applied before quantization. Let $W \in \mathbb{R}^{d_{\text{in}} \times d_{\text{out}}}$ be the weight matrix of a linear layer with $d_{\text{in}}$ input channels and $d_{\text{out}}$ output channels, and let $X \in \mathbb{R}^{d_{\text{in}} \times d_{\text{batch}}}$ be a calibration input activation with $d_{\text{batch}}$ tokens. Define $q(\cdot)$ as a quantizer that maps continuous values to a discrete representation (the specific quantizer will be specified later based on whether we are using FP or INT formats). Let $T_W, T_X \in \mathbb{R}^{d_{\text{in}} \times d_{\text{in}}}$ be linear transforms applied to $W$ and $X$ respectively before quantization. The quantized output of the layer becomes $q(T_W W)^\top q(T_X X)$, and the objective is to choose $T_W$ and $T_X$ to minimize the normalized $L_2$ output loss:
What it computes: the mean squared error per output element between the full-precision matrix multiplication $W^\top X$ and the quantized approximation $q(T_W W)^\top q(T_X X)$. The normalization by $d_{\text{out}} d_{\text{batch}}$ converts the total Frobenius norm squared into a per-element average, making the loss comparable across layers of different sizes.
Why this form: the Frobenius norm directly measures the output activation error that downstream layers will see, which is more predictive of end-to-end model accuracy than per-weight or per-activation quantization error metrics. The normalization removes dependence on layer dimensions, enabling a unified analysis.
The block-diagonal constraint. The paper immediately imposes a practical constraint that fundamentally shapes the solution: $T_W$ and $T_X$ must be block-diagonal with block size $d$ equal to the quantization group size (e.g., $d = 32$ for MXFP4, $d = 16$ for NVFP4), and $d$ must divide $d_{\text{in}}$ and be a power of 2. This is formalized in Eq. (3):
What this means operationally: the input dimension $d_{\text{in}}$ is partitioned into $d_{\text{in}}/d$ contiguous blocks of size $d$. Each block gets its own $d \times d$ transform matrix, applied independently to the corresponding rows of $W$ and $X$. This means transform block $i$ affects only input channels $(i-1)d + 1$ through $id$.
Why this constraint: three practical reasons. First, activation transforms must be applied online during inference — a full $d_{\text{in}} \times d_{\text{in}}$ transform would cost $O(d_{\text{in}}^2)$ per token, which is prohibitive for large $d_{\text{in}}$ (often 4096–8192 in modern LLMs). A block-diagonal transform costs $O(d \cdot d_{\text{in}})$ per token — linear in $d_{\text{in}}$ rather than quadratic. Second, the quantization group size is $d$ — each quantization scale is computed from exactly one block's worth of values — so it is natural to apply the transform at exactly this granularity. Third, as prior work (Shao et al., 2025; Egiazarian et al., 2025) showed, blockwise transforms are more effective than full-layer transforms for FP formats because they localize the outlier mitigation to the scale computation granularity.
Under the block-diagonal constraint, the layer output decomposes into a sum of blockwise contributions. The unquantized output is:
The quantized output is:
Crucial simplification. The paper approximates the layerwise loss as the sum of independent blockwise losses:
where each blockwise loss is:
Why this approximation is reasonable: the block-diagonal structure means transforms on different blocks operate on disjoint subsets of input channels. The cross-terms between blocks in the squared error expansion involve products of quantization errors from different blocks, which are approximately zero-mean and uncorrelated (since the blocks are independent subproblems with independent transforms). The paper treats this as an approximation, not an exact equality, and validates it empirically through the layerwise loss experiments in Table 1.
This decomposition is the key architectural decision: it reduces the problem from finding two $d_{\text{in}} \times d_{\text{in}}$ transforms (coupled across all input channels) to finding $d_{\text{in}}/d$ independent pairs of $d \times d$ transforms (one pair per block). Since $d$ is small (16–32), each block's problem is low-dimensional and analytically tractable.
3.4.2 Probabilistic Reformulation and Problem Reduction
To make the blockwise optimization analytically tractable, the paper reformulates it from a deterministic matrix perspective to a probabilistic one. The columns of $W_{(i)} = [w_1, \ldots, w_{d_{\text{out}}}]$ and $X_{(i)} = [x_1, \ldots, x_{d_{\text{batch}}}]$ are treated as i.i.d. samples from $d$-dimensional distributions $w \sim \mathcal{D}_w$ and $x \sim \mathcal{D}_x$ respectively.
Why multivariate rather than univariate: the paper explicitly contrasts this with the common univariate intuition in prior work — "apply a rotation so that per-channel distributions look Gaussian." Appendix A.2 and Figure 4 illustrate the problem: an orthogonal rotation can equalize the coordinate-wise marginals (making them appear identically distributed) without changing the eigenvalues of the covariance. Outliers in the principal directions remain extreme in the joint space, even if they are hidden from any single marginal view. Since quantization error under AbsMax scaling depends on the joint distribution (specifically, how mass concentrates along dominant directions), a multivariate formulation that captures the eigenvalue structure is essential.
Under this probabilistic view, the blockwise loss becomes an expectation over the data distributions and the stochastic quantization noise:
Unbiased quantization assumption. The paper assumes $q$ is a stochastic and unbiased quantizer: for any vector $\alpha$, the quantization error $\varepsilon(\alpha) = q(\alpha) - \alpha$ is a random vector with $\mathbb{E}[\varepsilon(\alpha)] = 0$. This is a standard modeling choice (used in DiffQ, Défossez et al., 2022) that allows treating quantization noise as additive zero-mean noise, converting a discrete combinatorial problem into a continuous variance-minimization problem. The specific noise distributions differ between FP and INT (detailed in Sections 4.2.1 and 4.3.1), but the zero-mean property holds for both.
The unbiasedness constraint and its consequence. The paper further imposes $T_W = T_X^{-\top}$ (the negative transpose of the inverse — meaning $T_W^\top T_X = I$, the identity matrix). Under this constraint, the quantized dot product is unbiased with respect to the true dot product:
The cross-terms vanish because the quantization errors are zero-mean and independent.
Why this constraint: without it, the quantized output would have a systematic bias (a consistent offset from the true value) that accumulates across layers. More importantly, the paper uses it to decompose the two-sided quantization loss into two independent one-sided terms. Using a first-order approximation (dropping the $\varepsilon(T_W w)^\top \varepsilon(T_X x)$ term, which is a product of two small errors), the loss splits as:
where the matrices $W'$ and $X'$ are defined by:
What $W'$ and $X'$ represent: they are matrix square roots (specifically, lower triangular Cholesky factors) of the uncentered second-moment matrices of the weight and activation distributions within the block. Intuitively, if $w \sim \mathcal{N}(0, \Sigma_w)$, then $W' W'^\top = \Sigma_w$ and $W'$ is a "whitening matrix" — applying $W'^{-1}$ to $w$ would produce a standard normal vector. Here, $W'$ captures the anisotropic covariance structure that makes some directions in the $d$-dimensional space have much larger variance than others.
Problem reduction to one-sided optimization. The two terms in the decomposition have identical form: each is an expectation over the quantization error of a vector, weighted by a second-moment matrix. The paper focuses on minimizing the activation-side term, since the weight-side term admits an identical solution. Define $y = W'^\top x$ as the activation vector after whitening by the weight second moment, and define $T = T_X W'^{-\top}$ as a reparameterized transform. Then the activation quantization error term becomes:
What this reduction achieves: the original problem involves two unknown transforms ($T_W$ and $T_X$), two data distributions ($\mathcal{D}_w$ and $\mathcal{D}_x$), and two quantizers (weight and activation). After the reduction, we have a single unknown transform $T$, a single effective data distribution $\mathcal{D}_y$ (the distribution of $y = W'^\top x$), and a single quantizer applied to $T y$. The weight-side transform is then recovered as $T_X = T W'^\top$ and the weight transform as $T_W = T_X^{-\top}$.
The distribution of $y$. This is the critical object that captures the interaction between weight statistics and activation statistics. Its covariance is:
If we define the SVD of $W'^\top X'$ as $U S V^\top$ (this is Eq. 7 in the paper), then:
The singular values $s_1 \geq \cdots \geq s_d > 0$ in the diagonal matrix $S = \text{diag}(s_1, \ldots, s_d)$ quantify the anisotropy of $y$: how much more variance there is along the first principal direction (eigenvector) compared to the last. If all $s_k$ are equal, $y$ is isotropic. If $s_1 \gg s_d$, $y$ is highly elongated along $U$'s first column — this is the outlier regime that degrades quantization.
3.4.3 Transformation Parameterization via SVD
To optimize $T$, the paper parameterizes it through its interaction with the data's singular value structure. The key idea is to express $T$ in terms of the SVD of $W'^\top X' = U S V^\top$, which decomposes the weight-activation interaction into known orthogonal bases $U, V$ and known singular values $S$. Specifically, the paper parameterizes the combined matrix $T U S$ via its own SVD:
where $U' \in \mathbb{R}^{d \times d}$ is an unknown orthogonal matrix, $S' = \text{diag}(s'_1, \ldots, s'_d)$ is an unknown diagonal matrix with $s'_1 \geq \cdots \geq s'_d > 0$, and $R \in \mathbb{R}^{d \times d}$ is an unknown orthogonal matrix. Solving for $T$:
What this parameterization does: it separates the unknown transform into three interpretable components:
$U^\top$: rotates into the eigenbasis of$\mathbb{E}_y [y y^\top] = U S^2 U^\top$(the principal components of the whitened activation distribution).$S^{-1}$: rescales each principal direction by the inverse of its singular value, equalizing the variance across directions (a "whitening" operation on$y$).$R^\top$and$U'$: two unknown orthogonal rotations that we are free to choose.$S'$: unknown rescaling that determines the final variance along each output direction.
The optimization problem is now: choose $U'$, $S'$, and $R$ to minimize $\mathbb{E}_{y, \varepsilon(T y)} \| T^{-1} \varepsilon(T y) \|^2$, given the specific quantization error model $\varepsilon(\cdot)$. The answer turns out to depend on whether the quantizer is FP or INT.
A useful inequality on the singular values. The paper notes a key relationship among the singular values $s_k$ that will be used repeatedly in both proofs:
where $\text{tr}(S^2) = \sum_k s_k^2$ and $\text{tr}(S) = \sum_k s_k$. The gap between $\text{tr}(S^2)$ and $d^{-1} (\text{tr}(S))^2$ measures the spread of singular values: it is zero when all $s_k$ are equal (isotropic $y$, no outliers) and maximal when one $s_k$ dominates all others (extreme outlier). The left inequality is simply the fact that the sum of squares exceeds the square of the average times $d$; the right inequality is Jensen's inequality applied to the convex function $x \mapsto x^2$.
3.4.4 Floating-Point Quantizer Analysis
Quantization error model for FP types. For floating-point formats (EaMb: $a$ exponent bits, $b$ mantissa bits), the quantization error is approximately proportional to the value being quantized. The paper models this (derived in detail in Appendix A.4.2) as multiplicative noise:
where $\alpha \in \mathbb{R}^d$ is the vector being quantized, and $\eta = [\eta_1, \ldots, \eta_d]^\top$ is a random vector of i.i.d. samples from a zero-mean distribution $\mathcal{D}_\eta$ with $\mathbb{E}[\eta] = 0$.
What this physically means: each element $\alpha_k$ of the vector receives an independent relative perturbation $\eta_k$. If a value is large, its absolute quantization error $|\eta_k \alpha_k|$ is large; if it is small, the error is proportionally small. This captures the floating-point behavior where representable values are spaced with relative (not absolute) uniformity — the gap between adjacent FP numbers scales with the magnitude of the number.
Why this approximation is reasonable: Appendix A.4.2 derives this from a smoothed version of the FP format (called SEaMb), showing that the rounding operation in logarithmic space can be approximated by $\text{SEaMb}(x) \approx x(1 + (\ln 2) 2^{-b} \xi)$ where $\xi \sim \text{Uniform}(-1/2, 1/2)$. The mantissa bits $b$ control the relative error magnitude.
Loss minimization for FP. Under this error model, the one-sided quantization loss becomes (derived in Eq. 39):
where $\odot$ denotes elementwise (Hadamard) multiplication with the identity matrix — this extracts only the diagonal entries of $U' S'^2 U'^\top$.
What this says: the expected squared error decomposes into a constant factor $\mathbb{E}[\eta^2]$ (determined by the FP format's precision, specifically the mantissa bitwidth) times a trace term that depends only on the transform $T$ and the diagonal of $U' S'^2 U'^\top$.
The diagonal extraction $\odot I$ is crucial: it says that the quantization error in FP depends only on the per-dimension variance after transformation — specifically, on $\|S' U'^\top e_k\|^2$, which is the variance of the $k$-th coordinate of the transformed vector $T y$. There is no cross-dimension interaction term because the FP error $\eta_k$ is applied coordinate-wise and independently.
The optimization. The trace term can be expanded (Eq. 40) as:
Each term is the product of two factors: $\|T^{-1} e_k\|^2$ (how much the inverse transform amplifies error in the $k$-th output direction) and $\|S' U'^\top e_k\|^2$ (the variance of the transformed data along the $k$-th direction before quantization). The sum of these products is lower-bounded, via Cauchy-Schwarz and Jensen inequalities, by:
Equality conditions: what makes the lower bound achievable. The paper shows that equality is attained by choosing:
Why $U' = H$ (the Hadamard): when $U'$ is the Hadamard matrix, every entry has magnitude $d^{-1/2}$, so $\|S' U'^\top e_k\|^2 = \sum_{j=1}^d s'_j{}^2 d^{-1} = d^{-1} \text{tr}(S'^2)$ — the per-dimension variance is equalized across all $k$. This minimizes the maximum per-dimension variance and makes the Cauchy-Schwarz inequality tight (all terms in the sum become equal). Any other orthogonal matrix would produce some dimensions with higher variance than others, loosening the bound.
Why $S' = S^{1/2}$ (square root scaling): this choice makes $\|T^{-1} e_k\|^2$ proportional to $s_k$ (since $T = H S^{1/2} S^{-1} U^\top = H S^{-1/2} U^\top$, so $T^{-1} = U S^{1/2} H^\top$ and $\|T^{-1} e_k\|^2 = \sum_j s_j (H^\top e_k)_j^2 = d^{-1} \sum_j s_j$ — wait, let me be precise). Actually, $T^{-1} = U S^{1/2} H^\top$ because $T = H S^{-1/2} U^\top$. Then $\|T^{-1} e_k\|^2 = \|U S^{1/2} H^\top e_k\|^2 = \|S^{1/2} H^\top e_k\|^2 = \sum_j s_j (e_j^\top H^\top e_k)^2 = d^{-1} \sum_j s_j = d^{-1} \text{tr}(S)$. This is equalized across all $k$ (because of the Hadamard) and equals the average singular value. Meanwhile, with $S' = S^{1/2}$, the variance term is $\|S^{1/2} H^\top e_k\|^2 = d^{-1} \sum_j s_j = d^{-1} \text{tr}(S)$. The product for each $k$ is $(d^{-1} \text{tr}(S))^2$, and summing over $k$ gives $d \cdot (d^{-1} \text{tr}(S))^2 = d^{-1} (\text{tr}(S))^2$, achieving the lower bound.
Why $R = I$: any rotation $R$ would misalign the rescaling $S'$ with the data basis, reintroducing anisotropy and loosening the bound.
The WUSH construction for FP (Eq. 8). Substituting the optimal choices into the parameterization and the relationship $T_X = T W'^\top$:
where $W'$, $X'$ are the lower triangular Cholesky factors, $U S V^\top$ is the SVD of $W'^\top X'$, and $H$ is the Hadamard matrix. Note that the weight transform uses $V$ (from the activation side) and $X'^\top$, while the activation transform uses $U$ and $W'^\top$ — they are inverse transposes of each other, satisfying the unbiasedness constraint $T_{\text{xvsh}} T_{\text{wush}}^\top = I$.
Discussion: why orthogonal transforms don't help for ideal FP. For any orthogonal $T$ (including identity and Hadamard), $T^{-1} = T^\top$, so the trace term becomes:
An orthogonal transform merely rotates the coordinate system but does not change the total variance, so the error is the same regardless of the rotation. This means that, under the ideal FP multiplicative noise model, the Hadamard alone provides no theoretical benefit. Its empirical benefit (Table 1, where $H$ improves over $I$ for MXFP4) comes from deviations from the ideal model — specifically, the subnormal regime and small exponent ranges where MXFP4 behaves as a hybrid between FP and INT quantization.
Comparison of optimal vs. orthogonal. From Eq. (17), $\text{tr}(S^2) \geq d^{-1} (\text{tr}(S))^2$, so the optimal transform reduces the error by at most a factor of $d$ in the extreme outlier case (where one singular value dominates). The practical improvement depends on how anisotropic the data is — the more spread in the singular values, the larger the gap.
3.4.5 Integer Quantizer Analysis
Quantization error model for INT types. For integer quantization with $b$ bits using symmetric AbsMax scaling, the quantization error is determined by the maximum absolute value in the block. The paper models this (derived in Appendix A.4.1) as:
where $\alpha \in \mathbb{R}^d$ is the vector, $\|\alpha\|_\infty = \max_k |\alpha_k|$ is the maximum absolute value, and $\eta \in \mathbb{R}^d$ is a random vector of i.i.d. samples with $\mathbb{E}[\eta] = 0$.
What this physically means: the quantization scale is $s = 2^{1-b} \|\alpha\|_\infty$, and each element is rounded to the nearest integer multiple of $s$. The error for element $k$ is $s \cdot \xi_k$ where $\xi_k \sim \text{Uniform}(-1/2, 1/2)$, giving $\|\alpha\|_\infty \cdot 2^{1-b} \xi_k$. The factor $2^{1-b}$ is absorbed into the distribution of $\eta$. The key difference from FP: the error depends on the global maximum of the block, not on each individual value. One outlier inflates the error for all elements simultaneously.
Loss minimization for INT. Under this error model, the one-sided loss (Eq. 45) becomes:
What this says: the expected error decomposes into three factors: $\mathbb{E}[\eta^2]$ (determined by the bitwidth), $\|T^{-1}\|^2_F = \text{tr}(T^{-1} T^{-\top})$ (a property of the inverse transform), and $\mathbb{E}_y \|T y\|^2_\infty$ (the expected squared maximum of the transformed data). The product structure is fundamentally different from the FP case — the error factors into a "transform amplification" term and a "data extremum" term.
Step 1: Bounding $\|T^{-1}\|^2_F$ (Eq. 47). Using the parameterization $T = U' S' R^\top S^{-1} U^\top$, we have:
Taking the trace and using von Neumann's trace inequality (since $s_1^2 \geq \cdots \geq s_d^2$ and $s'_1^{-2} \leq \cdots \leq s'_d^{-2}$):
Equality is attained when $R = I$ — the rotation aligns the singular values in matching order, largest $s_k$ paired with largest $s'_k$.
Step 2: Bounding $\mathbb{E}_y \|T y\|^2_\infty$ (Eq. 57). This is the most subtle part. The paper derives both lower and upper bounds in terms of $\max_k \mathbb{E}_y (e_k^\top T y)^2$, which is the maximum per-dimension variance of the transformed data. The crucial inequality chain (Eq. 50):
where $\mathbb{E}_y \|T y\|^2 = \text{tr}(S'^2)$ from Eq. (36). The per-dimension variance is $\mathbb{E}_y (e_k^\top T y)^2 = \|S' U'^\top e_k\|^2$.
Why the Hadamard matters for INT. When $U' = H$, the per-dimension variance becomes:
All dimensions have equal variance. This minimizes $\max_k \mathbb{E}_y (e_k^\top T y)^2$ to its lowest possible value of $d^{-1} \text{tr}(S'^2)$. Without the Hadamard, some direction would have higher variance, inflating the expected maximum.
Tighter bounds for tail-bounded distributions. The paper uses Lemma 3 (proved in Appendices A.5.1 and A.5.2) to show that for zero-mean multivariate Gaussian or Laplacian $y$, the expected maximum is bounded much more tightly:
where $o(1)$ means the factor grows sublinearly with $d$ — specifically, $O(\log d)$ for Gaussian and $O(\log^2 d)$ for Laplacian. This is much tighter than the worst-case factor of $d$.
Combined bounds (Eq. 27-30). With $R = I$ and $U' = H$:
For tail-bounded $\mathcal{D}_y$, the upper bound tightens to $o(1)^{-1} \cdot \text{tr}(S^2 S'^{-2}) \cdot \text{tr}(S'^2)$. By Cauchy-Schwarz:
with equality when $S' \propto S^{1/2}$. Choosing $S' = S^{1/2}$ achieves this lower bound (up to the gap factor $d$ or $d \cdot o(1)$), making the transform near-optimal: the loss is within a factor of $d$ of the theoretical minimum for general distributions, and within $o(1)$ for tail-bounded distributions.
Why the optimal INT transform matches the FP transform. Despite the different quantization error models (multiplicative for FP, block-maximum-dependent for INT), both analyses converge to the same configuration: $U' = H$, $S' = S^{1/2}$, $R = I$. The intuition is that both quantizers penalize anisotropy: FP because per-dimension variance directly determines error, INT because per-dimension variance determines the likelihood of extreme values. The Hadamard equalizes per-dimension variance; the $S^{1/2}$ rescaling makes the inverse transform's amplification balanced; and $R = I$ aligns everything optimally. This is why a single construction (WUSH) works for both FP and INT.
Discussion: why orthogonal transforms underperform for INT. For orthogonal $T$, $\|T^{-1}\|^2_F = d$ and $\mathbb{E}_y \|T y\|^2_\infty$ is bounded between $\text{tr}(S^2)/d$ and $d \cdot \text{tr}(S^2)$ (for general distributions) or $o(1) \cdot \text{tr}(S^2)$ (for Hadamard with tail-bounded data). Comparing with the optimal bound $o(1)^{-1} \cdot (\text{tr}(S))^2$ for Hadamard + optimal scaling, and using Eq. (17), the orthogonal transform can be up to $d$ times worse than optimal in the extreme outlier regime. This gap explains the empirical results: Hadamard alone helps (because it reduces the maximum per-dimension variance), but the full WUSH transform adds the $S^{1/2}$ rescaling that actually reduces the total error.
3.4.6 GPTQ Integration (Algorithm 1)
The WUSH construction uses $d^{-1}_{\text{batch}} X X^\top$ as the activation second moment. This is, up to a constant factor, exactly the Hessian matrix used by GPTQ (Frantar et al., 2023) for second-order error correction during weight quantization. GPTQ quantizes weights column-by-column (or block-by-block), using the Hessian to compute optimal weight updates that compensate for the quantization error introduced in previous columns. The paper exploits this coincidence to integrate WUSH seamlessly into the GPTQ pipeline.
The coupling problem. There is a subtle complication: GPTQ updates the weights iteratively — after quantizing a block of weights, the remaining unquantized weights are adjusted to compensate for the error. But the WUSH transform for a block depends on the weight statistics of that block, which change when GPTQ applies error propagation from previous blocks. The transform construction and weight quantization are therefore coupled and cannot be performed independently.
Algorithm 1's solution: interleaved schedule. Algorithm 1 processes blocks in reverse order (from the last block to the first), interleaving transform construction and GPTQ quantization:
Lines 1–3: Global preprocessing.
where $\lambda \in \mathbb{R}_{\geq 0}$ is a small damping factor (for numerical stability when the Hessian is rank-deficient).
producing the lower triangular Cholesky factor $X' \in \mathbb{R}^{d_{\text{in}} \times d_{\text{in}}}$. Note: this is a global Cholesky over the full $d_{\text{in}} \times d_{\text{in}}$ Hessian, not per-block. The paper uses this global factor for GPTQ's inter-block error propagation.
This is the target output in the Cholesky-transformed space. $Y \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$ represents the full-precision output that the quantized layer should approximate.
Lines 4–17: Per-block loop (reverse order). The loop iterates $i$ from $d_{\text{in}}/d$ down to 1 — i.e., from the last block to the first. For each block $i$ with indices $j_1 = (i-1)d + 1$ to $j_2 = i d$:
Line 6: Compute SVD of the block's contribution to $Y$.
where $Y[1 : d_{\text{out}}, j_1 : j_2] \in \mathbb{R}^{d_{\text{out}} \times d}$ is the submatrix of $Y$ corresponding to block $i$. This SVD gives us the matrices needed for the WUSH construction: $U \in \mathbb{R}^{d_{\text{out}} \times d}$, $S \in \mathbb{R}^{d \times d}$, $V \in \mathbb{R}^{d \times d}$.
Line 7: Normalize by output dimensions.
This rescaling ensures the singular values correspond to the normalized second moments as defined in Eq. (6).
Line 8: Construct the WUSH weight transform.
What this computes: $X'[j_1 : j_2, j_1 : j_2] \in \mathbb{R}^{d \times d}$ is the $d \times d$ diagonal block of the global Cholesky factor corresponding to block $i$. Its inverse provides the $W'^{-\top}$ component of Eq. (8). The multiplication $H S^{1/2} V^\top$ provides the rotation-plus-scaling component. The result is the $d \times d$ transform matrix for block $i$.
Line 9: Transform the current weight block.
What this computes: $\bar{W} \in \mathbb{R}^{d \times d_{\text{out}}}$ is the transformed weight block — these are the weights that will actually be quantized. Note that the transform uses $U^\top$ (from the left singular vectors of $Y$), not $V$. This is because $\bar{W} = T_{\text{wush}(i)} W_{(i)}$, and the SVD structure ensures that $\bar{W}H S^{1/2} U^\topV^\topX'^{-1}W'^\top$` factors cancel against the weight structure).
Lines 10–15: Quantize with GPTQ or RTN. If $flag$ is true (GPTQ mode):
Line 11: Intra-block GPTQ.
The Hessian passed to GPTQ is the transformed Hessian $H S H^\top$, which is $d \times d$. This makes intuitive sense: GPTQ quantizes the transformed weights $\bar{W}$, so the Hessian used for error correction should also be expressed in the transformed space. The $S$ matrix captures the effective per-dimension importance after the WUSH transform.
Line 12: Inter-block GPTQ error propagation.
What this does: after quantizing block $i$, the quantization error $\bar{W} - \tilde{W}_{(i)}$ would propagate to all remaining blocks (those with smaller indices, since we're iterating in reverse). GPTQ compensates for this by updating the target $Y$ — subtracting off the contribution of the quantized block (in the Cholesky-transformed space) so that subsequent blocks can absorb the error. The term $X'[j_1 : j_2, 1 : d_{\text{in}}] \in \mathbb{R}^{d \times d_{\text{in}}}$ is the block $i$ rows of the full Cholesky factor — this captures how block $i$'s quantization error couples to all other blocks through the activation covariance.
If $flag$ is false (RTN mode), the quantized weights are simply $\tilde{W}_{(i)} \leftarrow q(\bar{W})$ (round-to-nearest), and the $Y$ update uses the full-precision $\bar{W}^\top$ rather than the quantized version (since RTN has no error propagation).
Why reverse order: GPTQ processes columns (or blocks) in order, using the Hessian to compensate for errors from previously processed columns. By iterating blocks in reverse, the algorithm ensures that when processing block $i$, the contributions of all blocks $i+1$ through $d_{\text{in}}/d$ have already been subtracted from $Y$, so the SVD on line 6 sees the "residual" target after accounting for later quantized blocks.
Why the same Hessian serves both purposes: the global Hessian $d^{-1}_{\text{batch}} X X^\top$ is used both to construct the Cholesky factor $X'$ (needed for the WUSH transform) and as the basis for GPTQ's second-order error correction. This is not a coincidence — both applications require the activation covariance structure. The computational overhead of computing the Hessian (which GPTQ already requires) is therefore amortized across both uses.
At inference time (Eq. 10): the forward pass computes:
The pre-quantized transformed weights $\tilde{W}_{(i)}$ are loaded from memory. The incoming activation block $\tilde{X}_{(i)}$ (new tokens, not calibration data) is multiplied by $T_{\text{wush}(i)}$, then quantized, then multiplied by $\tilde{W}_{(i)}^\top$. The $d_{\text{in}}/d$ block contributions are summed.
3.4.7 GPU Kernel Design
The per-block specialization challenge. In prior work (MR-GPTQ, Egiazarian et al., 2025), the transform is a single Hadamard matrix $H \in \mathbb{R}^{d \times d}$ reused across all blocks. WUSH assigns a distinct $d \times d$ matrix $T_{\text{wush}(i)}$ to each block $i$. Naively, this multiplies the transform storage by a factor of $d_{\text{in}}/d$ (e.g., 128× for $d_{\text{in}}=4096$, $d=32$) and introduces irregular memory access patterns if each block loads a different matrix.
Storage layout. The paper stores the WUSH transform matrices in a $(G, G, C)$ layout, where $G = d$ is the block size and $C = d_{\text{in}} / d$ is the number of blocks. This is transposed relative to the channel dimension — the first two dimensions are the $d \times d$ matrix, and the third dimension indexes the block. This transposition is performed offline, so it introduces no runtime cost.
Why this layout: during inference, the activation tensor has shape $(M, K)$ where $M$ is the batch size (number of tokens) and $K = d_{\text{in}} = C \times G$ is the input dimension. Applying the block-diagonal transform means computing, for each token and each block $i$, the product of a $G$-dimensional activation slice with a $G \times G$ matrix. The $(G, G, C)$ layout means that for a given block $i$, the required matrix is a contiguous $G \times G$ slice in memory, enabling coalesced loads.
GEMM mapping. The kernel is implemented using a CUTLASS GEMM template. The operation is formulated as a matrix multiplication where:
- The M dimension is the batch size
$M$(number of tokens). - The N dimension is the block size
$G$(output features per block). - The inner K' dimension is fixed to
$G$(the shared dimension of the activation slice and the transform matrix).
Critically, the index $C$ (which block we're processing) is not mapped to any GEMM dimension. Instead, it is handled implicitly: each thread block is assigned a specific $K/G$ subproblem (one of the $C$ blocks), and applies an offset to determine which $G \times G$ transform matrix to load. This means the GEMM sees $C$ independent small matrix multiplications of size $(M, G) \times (G, G)$, not one large multiplication of size $(M, C G) \times (C G, C G)$.
Why this mapping is efficient: each thread block only needs to load a single $(G, G)$ matrix from memory to process its assigned $\text{TileM} \times G$ subproblem of the activation tensor. This matches the memory access pattern of the uniform Hadamard case (H + Quant), where all blocks load the same $(G, G)$ Hadamard matrix. The difference is that in the WUSH case, different thread blocks load different $(G, G)$ matrices, but since each matrix is only $G \times G$ (e.g., $32 \times 32 = 1024$ elements), the data easily fits in L1/L2 cache. The paper reports that the average throughput difference between WUSH + Quant and H + Quant is only 1.3% (Section 5), and in some configurations the performance is identical.
Performance characteristics. Figure 2 shows per-layer GEMM speedups on an RTX 5090 at batch size 1024 for Qwen3-8B, Qwen3-14B, and Llama-3.1-70B. Key observations:
- MXFP4 kernels achieve up to 6.63× speedup over BF16 (the theoretical peak for FP4 on this GPU is 8×).
- WUSH + Quant nearly matches H + Quant across all layer shapes and model sizes.
- The remaining gap to the "None" (no transform or quantization) upper bound indicates that performance is primarily limited by the quantization overhead (reading and writing the full activation matrix, applying the scale factors, packing/unpacking to FP4), not by the transform choice.
Why the overhead is negligible: the transform is a small $G \times G$ matrix multiplication per block, which is memory-bandwidth-bound at small sizes ($G=32$ for MXFP4). The dominant cost is the subsequent low-precision matrix multiplication between the quantized activations and the quantized weights (the actual $M \times K \times N$ GEMM), which is the same regardless of whether the transform was Hadamard or WUSH. The additional cost of loading $C$ distinct transform matrices instead of one is amortized across the tokens in the batch and hidden behind the larger GEMM computation.
Implementation detail: the QuTLASS baseline. For the H + Quant comparison, the paper uses the QuTLASS implementation from Egiazarian et al. (2025), which is already heavily optimized and fuses the Hadamard transform with the quantization step. WUSH + Quant achieves parity with this baseline by matching the same fusion pattern — the transform and quantization are applied in a single kernel pass, avoiding separate read/write cycles for the transform output.
4. Key Insights and Innovations
Innovation 1: The Optimal Transform for Quantization Is Non-Orthogonal and Spectrally Matched — The Hadamard Is Only One Piece
The paper's most fundamental conceptual contribution is revealing what an optimal linear transform for block quantization actually looks like under first-principles analysis, and the answer is not an orthogonal rotation. Prior work — spanning QuIP, QuIP#, QTIP, QuaRot, and even learned approaches like SpinQuant and FlatQuant — overwhelmingly operated under the implicit assumption that the transform should be orthogonal (a rotation or reflection) to preserve the mathematical equivalence of the linear layer: for any orthogonal . This assumption was so ingrained that the central experimental question was simply which orthogonal matrix works best — random, Hadamard, or learned.
The paper demonstrates that orthogonality is not optimal for quantization error minimization, and in fact can be up to times worse than the optimal non-orthogonal transform in the extreme outlier regime (where is the block size, e.g., 32). The optimal transform has a specific spectral structure: it rescales dimensions proportionally to , where contains the singular values of the weight-activation interaction matrix . This is a non-orthogonal scaling: directions with high variance get compressed, directions with low variance get expanded, equalizing the effective quantization noise across dimensions.
What makes this finding intellectually distinctive is that it comes from deriving the lower bound on quantization error (Eqs. 20 and 29) and then proving that a specific construction achieves it (for FP) or comes within a dimension-dependent factor of it (for INT). This transforms the problem from "which heuristic transform works best?" to "what is the best we can possibly do, and how close does a given construction get?" The bound is not an empirical observation — it is a mathematical floor on achievable error under the given noise models. Any transform claiming optimality must match this bound; WUSH does, while orthogonal transforms produce the strictly larger (for FP) or up to (for INT).
The reframing is significant: this converts transform design from a trial-and-error empirical pursuit into a constrained optimization problem with a verifiable optimality condition. The practical payoff — the scaling that makes WUSH non-orthogonal — follows directly from the math, not from intuition or hyperparameter search.
Evidence: Table 1 shows that the non-orthogonal WUSH produces lower layerwise RTN quantization loss than orthogonal baselines (identity, random rotation, Hadamard) across all layer types and formats, with particularly dramatic gaps for INT4 (e.g., 2.39 vs. 5.57 for the Hadamard on the Q projection). Figures 3, 6-9 confirm this translates to end-to-end accuracy.
Innovation 2: A Unified Optimality Analysis for FP and INT Quantizers That Reveals Why the Same Transform Works for Both
A surprising and theoretically elegant result is that the same transform configuration (, , ) emerges as optimal (or near-optimal) under two fundamentally different quantization error models. FP quantization error is multiplicative and per-element: . INT quantization error is global and block-maximum-dependent: . These are qualitatively different physical processes — yet both analyses converge on the same construction.
Prior work treated FP and INT quantization as separate problems requiring different strategies. Even within the WUSH paper, the proofs for FP (Section 4.2) and INT (Section 4.3) use completely different mathematical machinery: the FP analysis manipulates a trace-of-diagonal-extraction term via Cauchy-Schwarz, while the INT analysis bounds a product of Frobenius norm and expected maximum using von Neumann's trace inequality and concentration inequalities for Gaussian/Laplacian maxima. The fact that the same , , choices minimize both objectives is not obvious a priori — it is an emergent property of the optimization landscape.
Why does this happen? The paper's framework reveals the common structure: both quantizers penalize anisotropy in the transformed data distribution. In FP, the penalty comes through per-dimension variance terms that appear in the trace. In INT, the penalty comes through the expected block maximum , which is lower-bounded by the maximum per-dimension variance. The Hadamard matrix equalizes per-dimension variance across all dimensions (Eq. 56), which is the optimal orthogonal rotation for minimizing worst-case variance under both models. The scaling then balances the inverse transform's amplification against the data's extremum, hitting the Cauchy-Schwarz lower bound.
This unification is conceptually important because it suggests that the WUSH construction captures something fundamental about quantization error geometry that transcends the specific quantizer implementation. It reframes the problem: anisotropy reduction is the core principle, and the Hadamard-plus-spectral-scaling recipe is the universal mechanism for achieving it. This predicts — and the paper confirms — that WUSH should help across formats (MXFP4, NVFP4, INT4) without format-specific tuning.
Evidence: Table 1 shows WUSH producing the lowest loss for MXFP4 and INT4, and being essentially tied with WUS (WUSH without Hadamard) for NVFP4. The NVFP4 case where Hadamard alone is harmful (due to top-element preservation effects noted in Egiazarian et al., 2025) is particularly revealing: the non-Hadamard components of WUSH () overcome this harm, producing lower loss than identity despite the Hadamard's negative contribution to NVFP4 specifically.
Innovation 3: The Hadamard's Empirical Success Is Explained as the Single Data-Agnostic Component of a Broader Optimal Transform — Not as an Incoherence Processing Miracle
The paper provides a precise, theoretically grounded answer to a question that had been hanging over the LLM quantization literature: why does the Hadamard matrix work so well as a pre-quantization transform, when it is just one fixed matrix with no adaptation to the data?
Prior work offered heuristic explanations: "incoherence processing" (Chee et al., 2023) — making the weight and activation matrices incoherent so that quantization error concentrates uniformly; "Gaussianization" — spreading outlier energy across dimensions so that per-channel distributions become more Gaussian-like; or simply empirical superiority in benchmarks (Ashkboos et al., 2024b; Tseng et al., 2024a). These explanations are not wrong, but they are incomplete: they don't explain why the Hadamard specifically, as opposed to any other rotation that mixes coordinates, and they don't tell you when the Hadamard should work and when it shouldn't.
The paper's analysis gives a precise answer: the Hadamard is the optimal choice for the orthogonal rotation component because it equalizes per-dimension variance, driving to for all (Eq. 56). This equalization makes the Cauchy-Schwarz lower bound tight in the FP proof (Eq. 41) and minimizes the maximum per-dimension variance in the INT proof (tightening the upper bound on ). Any other orthogonal matrix would produce unequal per-dimension variances, loosening the bounds and increasing error.
But the corollary is equally important: the Hadamard is only one-third of the optimal construction. The spectral scaling and the alignment are equally essential. This explains why Hadamard-only methods plateau in performance (as documented by Egiazarian et al., 2025): they get the rotation right but miss the rescaling that actually reduces total quantization error. It also explains why Hadamard is sometimes harmful (NVFP4): when the spectral structure of the data means that equalizing per-dimension variance conflicts with other format-specific desiderata, the Hadamard alone can be counterproductive. WUSH resolves this by combining the Hadamard with data-dependent components that can compensate.
This is a fundamental conceptual advance rather than an incremental refinement. It converts the Hadamard from a "trick that works" into a "provably optimal component of a larger optimal construction," providing both explanation and prescription. The prescription: if you want to improve beyond Hadamard, don't search for a better fixed rotation — add spectral rescaling based on second-moment statistics.
Evidence: Table 1's "WUS" column (WUSH without Hadamard, i.e., ) shows that the spectral scaling alone helps substantially over identity for MXFP4 and NVFP4 (e.g., NVFP4 Q projection: WUS 2.26 vs. I 4.23), but adding the Hadamard (WUSH) either improves further (MXFP4) or matches WUS (NVFP4). The "WUSH" column shows that the full construction consistently achieves the lowest or tied-lowest loss.
Innovation 4: Difficulty-Agnostic Optimality — The Transform Is Optimal Without Requiring Problem Difficulty Estimation
A key architectural insight, visible when comparing WUSH to the compute-optimal scaling framework from the prior example paper (which conditions strategy choice on estimated prompt difficulty), is that WUSH achieves optimality without any notion of "difficulty" or stratification of the data. The transform is computed directly from the second-moment statistics of the calibration data; it does not classify blocks into easy/hard, does not apply different strategies to different blocks, and does not require an exploration phase to estimate block characteristics.
This is possible because the optimization operates at the block level on continuous statistics (the singular values ) rather than at the sample level on discrete outcomes (correct/incorrect). The singular value spectrum is a continuous measure of anisotropy that directly determines how much the optimal non-orthogonal rescaling deviates from identity. Blocks with near-uniform singular values (isotropic) will have , making WUSH essentially a scaled Hadamard transform — the data-dependent component becomes trivial. Blocks with extreme singular value spread (highly anisotropic, outlier-dominated) will get strong differential rescaling. The optimality is automatic and adaptive: the same closed-form formula produces the right transform for any block, regardless of its singular value distribution.
This contrasts with the learned transform approaches (SpinQuant, FlatQuant) where the optimization is global and doesn't provide per-block granularity, and with the "incoherence processing" approaches where the transform is fixed and cannot adapt to block-level statistics. WUSH's per-block second-moment computation is cheap (Cholesky + SVD on matrices, with ) and does not require any iterative optimization or hyperparameter tuning.
Evidence: The fact that a single construction (Eq. 8) works across all blocks in all layer types (Q, K, V, O, G, U, D in Table 1) and all tested models (Llama-3.2-3B through Qwen3-32B) without per-block or per-model tuning is indirect evidence for this automatic adaptivity. The GPTQ integration in Algorithm 1 further demonstrates this: the same Hessian used for error correction also provides the second-moment statistics for the transform, with no additional calibration passes or meta-optimization.
Innovation 5: The Two-Sided Quantization Problem Reduces to a One-Sided Problem in a Coupled Space — A Dimensionality Reduction That Enables Closed-Form Solution
The paper's reduction of the joint weight-activation quantization problem (two unknown transforms, two data distributions, two quantizers) to a single one-sided problem (one unknown transform , one effective distribution , one quantizer) is a methodological insight with implications beyond this specific construction. The reduction chain is:
- Constrain to ensure unbiasedness (the quantized dot product has the correct expectation).
- Use a first-order approximation to split the two-sided loss into two independent one-sided terms (Eq. 13).
- Reparameterize the activation-side term by defining and , absorbing the weight statistics into the data distribution and the transform.
The result (Eq. 14) — minimizing — is dramatically simpler than the original formulation. The effective distribution has covariance , where and come from the SVD of . This covariance encodes the interaction between weight and activation statistics: directions where weights are large and activations are large get high singular values ; directions where they are misaligned get low singular values.
This reduction is conceptually powerful because it reveals that the joint weight-activation quantization problem is fundamentally about the spectral properties of their interaction, not about their individual statistics. The weight second moment and activation second moment matter only through their product , whose SVD determines both the principal directions () and the anisotropy (). This explains why methods that optimize weight quantization and activation quantization separately (e.g., AWQ for weights, SmoothQuant for activations) leave performance on the table: they miss the coupled structure.
The reduction also explains the paper's empirical finding that WUSH transforms computed from calibration data generalize to unseen tokens: the second-moment statistics are computed once from calibration data, but the transform depends only on the interaction SVD, which captures stable structural properties of the layer (how weight patterns align with typical activation patterns) rather than token-specific fluctuations.
Evidence: The fact that a single calibration pass (32 samples of sequence length 2048 from FineWeb-Edu, as stated in Section 6.1) produces transforms that generalize across all Platinum Benchmarks and LM Evaluation Harness tasks (Tables 2-4, Figures 3, 6-9) supports the claim that the coupled second-order statistics capture stable layer properties rather than calibration-set-specific artifacts. This is a significant practical property: it means WUSH does not require expensive per-task or per-dataset recalibration.
5. Experimental Analysis
Evaluation Methodology
-
Dataset. The paper uses two main evaluation suites. For per-layer quantization loss, 32 calibration samples of sequence length 2048 are drawn from the FineWeb-Edu dataset (Penedo et al., 2024), chosen as a representative web-text corpus. For end-to-end accuracy, the paper uses Platinum Benchmarks (Vendrow et al., 2025) and the LM Evaluation Harness (Gao et al., 2021) with specific tasks including MMLU-CoT (or standard MMLU depending on the model), GSM8K, HellaSwag, and WinoGrande. The Platinum Benchmarks suite includes 13 individual tasks: SingleOp, SingleQ, MultiArith, SVAMP, GSM8K, MMLU-Math, BBH Deduction, BBH Counting, BBH Navigate, HotpotQA, SQuAD, DROP, and Winograd-WSC. The test sets for each benchmark are the standard publicly available evaluation splits; no custom test sets are created. For layerwise loss experiments, the specific layer chosen is the 18th transformer block of Qwen3-8B, with the paper stating that "results [are] consistent across blocks and datasets" (Section 6.1).
-
Base model(s). Experiments span five models across two families: Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct (Meta's Llama 3 family), and Qwen3-8B, Qwen3-14B, Qwen3-32B (Qwen's Qwen3 family). For the Qwen3 models, thinking mode is disabled across all experiments (Appendix B). These models are chosen to span a range of scales (3B to 32B parameters) and to represent two distinct architecture families, testing the generality of WUSH across model designs. The paper does not claim these are compute-optimally trained or representative of all possible LLMs — the choice is pragmatic, covering state-of-the-art open-weight models at deployment-relevant sizes.
-
Metrics. Three types of metrics are reported: (1) Layerwise RTN quantization loss — the normalized L2 loss defined in Eq. (5), computed as , measuring the per-element squared error between the full-precision and quantized layer outputs. Results are reported in the scale of (Table 1). (2) End-to-end task accuracy — for LM Evaluation Harness tasks, standard accuracy metrics are used (exact match for GSM8K, multiple-choice accuracy for MMLU, HellaSwag, WinoGrande). For Platinum Benchmarks, task-specific accuracy metrics are reported. (3) KL-divergence — measured on a slice of the C4 dataset (Table 3 in Appendix B), quantifying the distributional shift between full-precision and quantized model outputs. (4) Average Recovery — computed as the ratio of the average quantized accuracy to the average BF16 accuracy across all tasks, expressed as a percentage (Tables 2 and 4). (5) GPU throughput speedup — measured as the ratio of BF16 GEMM latency to MXFP4 GEMM latency, including transform and quantization overhead (Figure 2). All accuracy experiments use emulated ("fake") quantization where quantized values are represented in BF16/FP16 format, and results are aggregated over multiple random seeds (stated in Appendix B preamble).
-
Baselines. The paper compares against five categories of prior methods, each with specific configurations:
Identity and basic rotations:
- RTN-I: Round-to-nearest quantization with identity transform (no transform applied), using AbsMax scaling. This is the naive quantization baseline.
- RTN-R: Round-to-nearest with random orthogonal rotation, averaged over 10 runs to reduce variance (Table 1 only).
- RTN-H: Round-to-nearest with blockwise Hadamard transform — the strongest data-agnostic orthogonal baseline, corresponding to the approach in MR-GPTQ (Egiazarian et al., 2025).
Channel-wise rescaling methods:
- SmoothQuant (Xiao et al., 2023): Applies per-channel smoothing factors to balance weight and activation dynamic ranges. Results are reported for Llama-3.1-8B-Instruct in Table 2 only.
Orthogonal transform methods:
- QuaRot (Ashkboos et al., 2024b): Applies end-to-end Hadamard rotations throughout the transformer architecture. Results for Llama-3.1-8B-Instruct in Table 2 only.
- SpinQuant (Liu et al., 2025): Uses learned rotations optimized on calibration data. Results for Llama-3.1-8B-Instruct in Table 2 only.
GPTQ-based methods:
- GPTQ-I: GPTQ weight quantization (Frantar et al., 2023) with identity transform — the standard second-order error correction baseline.
- GPTQ-H (MR-GPTQ): GPTQ with blockwise Hadamard transforms, corresponding to the state-of-the-art method from Egiazarian et al. (2025), which showed blockwise Hadamard offers the best empirical performance among existing transforms for MXFP/NVFP quantization.
WUSH ablations:
- WUS: WUSH without the Hadamard component, i.e., setting in the construction. This isolates the effect of the spectral (data-dependent) scaling from the Hadamard rotation. Reported in Table 1 only.
- RTN-WUSH: WUSH transform with round-to-nearest quantization (no GPTQ error propagation).
- GPTQ-WUSH: WUSH transform integrated with GPTQ, using Algorithm 1. This is the full method.
All baselines use the same quantization formats (MXFP4, NVFP4, or INT4) and group sizes for fair comparison.
-
Generation budget / compute accounting. For the layerwise loss experiments, no generation budget applies — these are single-pass computations on a fixed calibration set of 32 samples × 2048 tokens each. For end-to-end accuracy, the evaluation is performed on standard test sets with no additional test-time sampling or search — each prompt is processed once with the quantized model. For the GPU kernel benchmarks (Figure 2), compute is measured as wall-clock GEMM latency on an RTX 5090 at batch size 1024, normalized to BF16 performance. The theoretical peak speedup for FP4 on this GPU is 8× (4× from memory bandwidth reduction × 2× from FP4 compute throughput relative to BF16).
-
Cross-validation / statistical protocol. The paper reports that end-to-end accuracy results are "aggregated over multiple random seeds" (Appendix B). For the layerwise loss experiments (Table 1), the random rotation baseline (R) is averaged over 10 runs. The paper does not describe a formal cross-validation procedure for hyperparameter selection — because WUSH has no hyperparameters to tune. The damping factor in Algorithm 1 is mentioned as a numerical stability parameter but no sweeps or sensitivity analyses are reported. The calibration data for computing the WUSH transform is separate from the evaluation data (FineWeb-Edu for calibration, C4 for KL-divergence, Platinum Benchmarks and LM Eval Harness for accuracy), providing a train-test split, though the paper does not ablate calibration set size or composition beyond the fixed 32-sample configuration.
Main Quantitative Results
Per-Layer Quantization Loss (Table 1)
The paper's first empirical contribution is a direct validation of the theoretical analysis through per-layer loss measurements, isolating the effect of different transforms on quantization error before assessing end-to-end model accuracy.
Headline result: WUSH achieves the lowest or tied-lowest layerwise loss across all formats and layer types. Table 1 reports layerwise RTN quantization loss (scale ) for each linear layer type in the 18th transformer block of Qwen3-8B: attention projections (Q, K, V, O) and MLP projections (Gate G, Up U, Down D). Across all seven layer types and three formats (MXFP4, NVFP4, INT4), WUSH consistently produces the smallest loss, with the single exception of NVFP4 where WUS and WUSH are nearly tied:
MXFP4: WUSH achieves losses of 3.34 (Q), 3.34 (K), 3.30 (V), 2.76 (O), 4.49 (G), 4.39 (U), 3.39 (D). For comparison, the strongest baseline (Hadamard, RTN-H) achieves 7.24, 7.20, 8.60, 3.79, 5.45, 5.61, 3.90 respectively. The improvement ranges from approximately 2.1× reduction on the Q projection (3.34 vs. 7.24) to approximately 1.4× on the O projection (2.76 vs. 3.79). The identity transform (RTN-I) produces losses of 11.1, 12.0, 10.7, 4.35, 7.10, 6.56, 5.47 — WUSH reduces loss by 3.3× on the worst-affected projection (Q: 3.34 vs. 11.1).
NVFP4: The picture is more nuanced. The identity transform (4.23, 4.35, 4.37, 2.34, 3.49, 3.41, 2.41) performs better than the Hadamard (5.60, 5.62, 6.70, 2.58, 3.71, 3.79, 2.78), confirming the prior finding from Egiazarian et al. (2025) that the Hadamard alone is harmful for NVFP4 due to its top-element preservation effect. WUS (2.26, 2.36, 2.30, 2.00, 3.04, 3.01, 2.23) dramatically outperforms both, and WUSH (2.40, 2.44, 2.28, 1.92, 3.09, 3.02, 2.33) is essentially tied with WUS — the Hadamard component neither helps nor hurts significantly once the spectral scaling is in place. This is a striking result: the data-dependent components of WUSH overcome the known NVFP4-specific degradation that affects pure Hadamard transforms.
INT4: The gap is most dramatic here, consistent with the theory's prediction that orthogonal transforms can be up to times suboptimal in the extreme outlier regime. The identity transform produces extreme losses on some projections (Q: 170, Up: 55.9, Down: 19.3 — likely due to a few extreme outliers inflating the AbsMax scale and crushing inlier resolution). The Hadamard helps substantially (Q: 5.57, K: 5.55, V: 6.80), but WUSH achieves 2.39, 2.43, 2.54, 2.10, 3.43, 3.43, 2.55 — a 2.3× reduction over Hadamard on Q, and a staggering 71× reduction over identity on Q. The WUS transform (213, 142, 10.7, 4.54, 50.2, 7.42, 13.1) actually increases error relative to identity on some projections — the spectral scaling alone without the Hadamard rotation can exacerbate outliers for INT quantization.
What this validates theoretically: The paper's FP analysis (Section 4.2) predicts that orthogonal transforms provide no benefit under ideal multiplicative noise — Table 1 shows Hadamard (7.24) indeed does not dramatically outperform identity (11.1) for MXFP4 on Q, and actually hurts for NVFP4 (5.60 vs. 4.23). The INT analysis (Section 4.3) predicts that Hadamard helps by reducing per-dimension variance maxima, but that WUSH's spectral scaling should go further — the gap from Hadamard (5.57) to WUSH (2.39) for INT4 on Q directly confirms this. The MXFP4 case, where Hadamard provides moderate gains, is consistent with the paper's observation that MXFP4 "behaves as a hybrid between ideal FP and INT quantization" (Section 6.1) due to subnormal regime and small exponent range effects.
WUS as a critical ablation. The WUS column (WUSH without Hadamard) reveals that the spectral scaling alone is powerful for FP formats (NVFP4: WUS 2.26 vs. I 4.23 on Q) but can be harmful for INT formats (INT4: WUS 213 vs. I 170 on Q). This non-monotonic behavior — WUS helps NVFP4, hurts INT4 — demonstrates that the Hadamard is not merely an optional bonus but a necessary component for robustness across formats, consistent with the theory's conclusion that is optimal for INT and essential for achieving the lower bound in FP when data is anisotropic.
End-to-End Accuracy on LM Evaluation Harness (Table 2, Table 4)
Headline result for Llama-3.1-8B-Instruct (Table 2): WUSH achieves +2.8 average points improvement over the strongest Hadamard baseline for MXFP4 RTN, and +0.7 points for MXFP4 GPTQ.
Table 2 reports MMLU-CoT, GSM8K, HellaSwag, WinoGrande, and their average for Llama-3.1-8B-Instruct under NVFP4 and MXFP4 formats. The BF16 baseline achieves 78.93 average accuracy (100% recovery). The key comparisons:
NVFP4 RTN: GPTQ-H (MR-GPTQ) achieves 75.84 average (96.08% recovery), which is the state-of-the-art baseline. RTN-WUSH achieves 75.28 (95.37% recovery) — slightly below MR-GPTQ but above RTN-H (74.05, 93.82% recovery). GPTQ-WUSH achieves 76.10 (96.40% recovery) — slightly above MR-GPTQ, achieving the best NVFP4 result. SmoothQuant (75.70, 95.90%) and GPTQ-I (75.72, 95.92%) are comparable to MR-GPTQ. The gains over the state-of-the-art for NVFP4 are modest — approximately +0.26 average points for GPTQ-WUSH over GPTQ-H.
MXFP4 RTN: This is where WUSH shows the largest gains. RTN-H achieves 70.45 average (89.26% recovery). RTN-WUSH achieves 73.21 (92.75% recovery) — a gain of +2.76 average points. Looking at individual tasks: GSM8K improves from 72.48 to 75.16 (+2.68), HellaSwag from 75.29 to 77.28 (+1.99), MMLU-CoT from 62.38 to 66.85 (+4.47). The gain on MMLU-CoT is particularly large, suggesting that complex reasoning tasks benefit most from reduced quantization error.
MXFP4 GPTQ: GPTQ-H (MR-GPTQ) achieves 73.65 average (93.31% recovery). GPTQ-WUSH achieves 74.35 (94.20% recovery) — a gain of +0.70 average points. GSM8K improves from 75.70 to 77.41 (+1.71), HellaSwag from 76.91 to 77.44 (+0.53). The GPTQ integration closes much of the gap between WUSH and the baseline, but WUSH still provides a consistent edge.
Comparison against learned and orthogonal baselines: SpinQuant (69.40 average for MXFP4 RTN) and QuaRot (62.90 average for MXFP4 RTN, with a particularly poor MMLU-CoT of 49.86) are substantially worse than the Hadamard baselines and WUSH. SmoothQuant (70.30 for MXFP4 RTN) is comparable to RTN-H. These results confirm that the closed-form WUSH construction outperforms methods requiring gradient-based optimization (SpinQuant) or fixed orthogonal transforms (QuaRot).
NVFP4-MXFP4 gap closure: The BF16-to-MXFP4 gap for RTN-H is 78.93 - 70.45 = 8.48 points. For RTN-WUSH, it narrows to 78.93 - 73.21 = 5.72 points — a 33% reduction in the gap. For GPTQ-H: 78.93 - 73.65 = 5.28 points; GPTQ-WUSH: 78.93 - 74.35 = 4.58 points — a 13% gap reduction. NVFP4 and MXFP4 recovery rates for GPTQ-WUSH are 96.40% and 94.20% respectively — a gap of only 2.20 percentage points, compared to 96.08% vs. 93.31% (gap of 2.77 points) for GPTQ-H. WUSH demonstrably narrows the NVFP4-MXFP4 accuracy gap.
Results for additional models (Table 4, Appendix B): Table 4 extends the LM Evaluation Harness results to Llama-3.2-3B-Instruct, Qwen3-8B, Qwen3-14B, and Qwen3-32B. Across all models and metrics, WUSH consistently improves over Hadamard baselines:
-
Llama-3.2-3B-Instruct: MXFP4 RTN-WUSH achieves 66.43 average (92.92% recovery) vs. RTN-H at 62.39 (87.27%). The gain is +4.04 average points — even larger than for the 8B model. GPTQ-WUSH (66.15, 92.53%) essentially matches GPTQ-H (66.18, 92.58%) for MXFP4, and slightly exceeds it for NVFP4 (67.87 vs. 67.19, GPTQ-WUSH vs. GPTQ-H).
-
Qwen3-8B: NVFP4 GPTQ-WUSH achieves 76.45 average (98.66% recovery), nearly closing the gap to BF16 (77.49). MXFP4 GPTQ-WUSH achieves 75.69 (97.68% recovery), exceeding GPTQ-H at 75.22 (97.07%). The NVFP4-MXFP4 gap for GPTQ-WUSH is only 0.76 points (76.45 vs. 75.69), compared to 0.65 points for GPTQ-H — WUSH makes MXFP4 nearly as accurate as NVFP4 for this model.
-
Qwen3-14B: Only RTN and identity/Hadamard/WUSH comparisons are reported for this model. NVFP4 RTN-WUSH achieves 79.80 average (98.76% recovery) vs. RTN-H at 79.29 (98.12%) and RTN-I at 79.03 (97.80%). MXFP4 RTN-WUSH achieves 79.44 (98.30% recovery) vs. RTN-H at 77.13 (95.44%). The NVFP4-MXFP4 gap for RTN-WUSH is only 0.36 points (79.80 vs. 79.44) — above 98% recovery for both formats, effectively closing the format gap. This is the paper's strongest claim about MXFP4-NVFP4 parity.
-
Qwen3-32B: NVFP4 RTN-WUSH achieves 82.80 (99.24% recovery) vs. RTN-H at 82.20 (98.51%). MXFP4 RTN-WUSH achieves 82.01 (98.28% recovery) vs. RTN-H at 82.15 (98.45%). Interestingly, for this largest model, RTN-H actually outperforms RTN-WUSH on MXFP4 by 0.14 points — a rare case where WUSH does not win. The paper does not comment on this reversal, but it may relate to the larger model having more isotropic weight statistics in each block, reducing the advantage of data-dependent scaling.
Platinum Benchmarks Results (Figures 3, 6-9)
Headline result for Qwen3-8B (Figure 3): WUSH improves MXFP4 average accuracy by up to +1.3 points (RTN) and +0.7 points (GPTQ) over the strongest Hadamard baseline, while NVFP4 improvements are smaller and often within run-to-run variability.
Figure 3 provides a detailed per-task breakdown for Qwen3-8B on the 13-task Platinum Benchmarks suite, comparing RTN-H, GPTQ-H, RTN-WUSH, GPTQ-WUSH for both MXFP4 and NVFP4, plus RTN-I for NVFP4. The bottom panel shows average accuracy with standard deviation bars:
NVFP4: The BF16 baseline achieves approximately 95.39 average. GPTQ-H achieves approximately 95.00. GPTQ-WUSH achieves approximately 95.09 (internal comparison, numbers estimated from the bar chart). RTN-WUSH (approximately 94.38) slightly trails RTN-I (approximately 93.86). The error bars overlap for all NVFP4 methods, indicating that the differences are within run-to-run variability for this model-format combination. This is consistent with Table 2, where NVFP4 gains for Llama-3.1-8B-Instruct were also modest.
MXFP4: This is where the gains are clear and outside error bars. RTN-H achieves approximately 91.75 average. RTN-WUSH achieves approximately 93.00 — a gain of +1.25 points. GPTQ-H achieves approximately 92.80. GPTQ-WUSH achieves approximately 93.52 — a gain of +0.72 points. Per-task improvements are visible across most tasks: on SingleOp, MXFP4 RTN-WUSH (approximately 99.56) exceeds RTN-H (approximately 97.33); on GSM8K, GPTQ-WUSH (approximately 96.46) exceeds GPTQ-H (approximately 94.48); on BBH Deduction, RTN-WUSH (approximately 98.17) substantially exceeds RTN-H (approximately 89.76). The BBH tasks (Deduction, Counting, Navigate) show some of the largest relative improvements, consistent with the hypothesis that reasoning-heavy tasks are more sensitive to quantization error.
Qwen3-14B (Figure 6): Only RTN comparisons are shown for NVFP4 and MXFP4. BF16 achieves 95.86 average. NVFP4 RTN-WUSH achieves approximately 94.64 vs. RTN-I at 93.86 — a gain of +0.78 points. MXFP4 RTN-WUSH achieves approximately 94.60 vs. RTN-H at 94.38 — a gain of +0.22 points. The improvements are present but diminish as model size increases, consistent with the Qwen3-32B results in Table 4 where RTN-H sometimes outperforms RTN-WUSH on MXFP4.
Qwen3-32B (Figure 7): BF16 achieves 97.38 average. NVFP4 RTN-WUSH achieves approximately 96.70 vs. RTN-H at 96.58 — a gain of only +0.12 points. MXFP4 RTN-WUSH achieves approximately 96.48 vs. RTN-H at 96.38 — a gain of +0.10 points. Error bars substantially overlap for all methods. At this model scale, the benefits of WUSH over Hadamard are marginal and may not be statistically significant.
Llama-3.2-3B-Instruct (Figure 8): This smaller model shows the largest relative gains, consistent with Table 4. BF16 achieves 79.11 average. NVFP4 GPTQ-WUSH achieves approximately 74.91 vs. GPTQ-I at 74.22 — a gain of +0.69 points. MXFP4 RTN-WUSH achieves approximately 72.49 vs. RTN-H at 72.29 — a gain of +0.20 points. The per-task table shows that gains are concentrated in specific tasks: BBH Deduction improves from 59.40 to 63.00 (+3.60 points) for MXFP4 RTN-WUSH over RTN-H.
Llama-3.1-8B-Instruct (Figure 9): BF16 achieves 85.33 average. NVFP4 RTN-WUSH achieves approximately 80.80 vs. RTN-I at 81.99 — WUSH underperforms identity for NVFP4 on this model. MXFP4 GPTQ-WUSH achieves approximately 79.25 vs. RTN-H at 76.41 — a gain of +2.84 points for MXFP4 RTN, the largest single gain in the Platinum Benchmarks section.
Cross-model trend: The benefits of WUSH appear most pronounced at smaller model scales (3B, 8B) and for the more challenging format (MXFP4), and diminish as model scale increases (14B, 32B). This is consistent with the theoretical mechanism: larger models tend to have more isotropic weight distributions within each block (the singular value spread is smaller), reducing the room for spectral scaling to improve over simple rotation. The paper does not provide a formal analysis of this trend, but it is visible across Figures 6-9 and Table 4.
KL-Divergence Results (Table 3, Appendix B)
Table 3 reports KL-divergence for Qwen3-8B measured on a slice of the C4 dataset, with calibration performed on FineWeb-Edu. This metric captures distributional shift between full-precision and quantized model outputs and is averaged over multiple random seeds. NVFP4 GPTQ-H achieves 0.069518, while GPTQ-WUSH achieves 0.054646 — a 21% relative reduction in KL-divergence, indicating that WUSH's output distribution is substantially closer to BF16 than the Hadamard baseline's. MXFP4 GPTQ-H achieves 0.093119, while GPTQ-WUSH achieves 0.077909 — a 16% relative reduction. NVFP4 RTN-WUSH (0.065747) is slightly better than NVFP4 GPTQ-H (0.069518), suggesting that even without GPTQ's error propagation, WUSH's transform alone can match or exceed the combination of Hadamard + GPTQ for NVFP4 on this metric.
GPU Kernel Benchmarks (Figure 2)
Headline result: WUSH + Quant kernels achieve up to 6.63× speedup over BF16, with an average throughput difference of only 1.3% compared to the optimized H + Quant baseline.
Figure 2 reports per-layer MXFP4 GEMM speedups relative to BF16 on RTX 5090 at batch size M=1024 for Qwen3-8B, Qwen3-14B, and Llama-3.1-70B, across three representative layer shapes (small square, tall-and-thin for large output dimension, short-and-wide for large hidden dimension). Three configurations are compared: None (no transform or quantization, showing the upper bound on GEMM throughput), H + Quant (Hadamard transform fused with quantization, using the QuTLASS implementation from Egiazarian et al., 2025), and WUSH + Quant (WUSH transform fused with quantization).
For Qwen3-8B (N=4096/K=4096, N=24576/K=4096, N=4096/K=12288), speedups range from approximately 5.5× to 6.3×. WUSH + Quant bars are visually indistinguishable from H + Quant bars in all three configurations, confirming the 1.3% average throughput difference claim. For Qwen3-14B and Llama-3.1-70B, the pattern is identical: WUSH + Quant achieves near-identical performance to H + Quant across all layer shapes. The None bars achieve the highest speedups (approximately 6.5-7×), indicating that the quantization step itself (reading/writing activation matrices, applying scales, packing/unpacking) is the primary bottleneck, not the choice of transform matrix.
Interpretation: The per-block specialization of WUSH — storing distinct matrices instead of one — introduces negligible overhead because: (1) each thread block only loads a single matrix for its assigned subproblem, matching the memory access pattern of the uniform case; (2) the matrices ( for MXFP4, giving 1024 elements each) fit easily in L1/L2 cache; (3) the dominant cost is the subsequent low-precision GEMM, which is identical regardless of which transform preceded it. The paper's kernel design (Section 5) successfully hides the per-block specialization cost.
Ablation Studies and Robustness Checks
WUS vs. WUSH (Table 1): Removing the Hadamard from WUSH (setting ) produces the WUS transform. For MXFP4, WUS achieves intermediate performance between identity and WUSH (e.g., Q projection: I 11.1, H 7.24, WUS 6.27, WUSH 3.34), confirming that both the spectral scaling and the Hadamard rotation contribute to the full gain. For NVFP4, WUS (2.26 on Q) actually outperforms WUSH (2.40 on Q) slightly — the Hadamard provides no benefit (and a small cost) once spectral scaling is in place, consistent with NVFP4's top-element preservation effect. For INT4, WUS catastrophically fails on some projections (Q: 213 vs. I 170) — the Hadamard is essential for INT quantization because spectral scaling alone without rotation can amplify outliers. This ablation demonstrates that WUSH's robustness across formats requires both components: the spectral scaling handles the eigenvalue spread, the Hadamard handles the coordinate equalization.
INT4 format comparison (Table 1): The INT4 results provide the starkest validation of the theoretical gap between orthogonal and optimal transforms. Identity produces extreme losses (Q: 170 vs. D: 19.3), showing that some projection types concentrate outliers more than others. Hadamard reduces this dramatically (Q: 5.57) but still leaves a factor-of-2.3 gap to WUSH (2.39). The theory predicts up to times gap in the extreme outlier regime (Eq. 17: ). The observed gap on Q ( in squared error) is within this bound but well below the worst case, indicating that the actual singular value spread in these layers is moderate but still sufficient to benefit from WUSH.
GPTQ vs. RTN comparison (Table 2, Table 4): Across all models and formats, GPTQ-based methods consistently outperform RTN-based methods. For Llama-3.1-8B-Instruct MXFP4, the improvement from GPTQ-H over RTN-H is +3.20 average points (73.65 vs. 70.45), while the additional improvement from GPTQ-WUSH over RTN-WUSH is +1.14 points (74.35 vs. 73.21). GPTQ and WUSH appear partially additive: GPTQ-H already closes much of the gap to BF16, and WUSH provides a further incremental gain on top. For Qwen3-8B NVFP4, GPTQ-WUSH achieves 98.66% recovery (76.45) vs. RTN-WUSH at 98.61% (76.42) — the GPTQ benefit is marginal when recovery is already near 99%.
Cross-task robustness (Figures 3, 6-9): The per-task breakdowns in Platinum Benchmarks reveal that WUSH's improvements are not uniform across tasks. Tasks requiring multi-step reasoning (BBH Deduction, GSM8K, MMLU-Math) show larger absolute and relative gains than knowledge-intensive tasks (HotpotQA, SQuAD). This pattern is consistent with the hypothesis that reasoning tasks amplify small per-layer quantization errors across many sequential transformer blocks, making them more sensitive to quantization quality. The paper does not provide a formal analysis of this per-task variation.
Model scale robustness (Table 4, Figures 6-9): The benefits of WUSH over Hadamard diminish as model scale increases from 3B to 32B. For Llama-3.2-3B-Instruct MXFP4 RTN, WUSH provides +4.04 average points over Hadamard (Table 4). For Qwen3-32B MXFP4 RTN, WUSH provides -0.14 points (a slight regression). This trend is not discussed in the paper but is visible across results. Possible explanations: larger models have been trained with more data and may have more naturally isotropic weight distributions per block; or the calibration set (32 samples × 2048 tokens) may be insufficient to capture the relevant statistics for the largest models.
NVFP4-specific behavior (Table 1, Table 2): The Hadamard alone is harmful for NVFP4 (RTN-H: 74.05 vs. RTN-I: 74.73 for Llama-3.1-8B-Instruct in Table 2), consistent with Egiazarian et al. (2025). WUSH overcomes this: RTN-WUSH achieves 75.28 vs. RTN-I 74.73 for NVFP4, showing that the non-Hadamard components of WUSH compensate for the Hadamard's negative effect. This is a strong validation of the claim that WUSH's data-dependent components are robust to format-specific quirks.
Calibration dataset choice: All calibration uses FineWeb-Edu. The KL-divergence evaluation uses C4 (a different dataset). The end-to-end evaluations use Platinum Benchmarks and LM Evaluation Harness (different tasks and distributions). The fact that WUSH transforms computed on FineWeb-Edu generalize to these diverse downstream tasks supports the claim that the second-moment statistics capture stable structural properties of the layers rather than calibration-set-specific artifacts. However, the paper does not ablate calibration set size or composition — it is unknown whether 32 samples is sufficient, or whether a different calibration dataset would produce meaningfully different transforms.
Damping factor : The parameter in Algorithm 1 is mentioned only once, with no ablation studies reported. For rank-deficient second moments (which can occur when or when activations have zero variance in some directions), this parameter is critical for numerical stability of the Cholesky decomposition. The lack of sensitivity analysis for is a gap, particularly since damping changes the effective singular values and thus the WUSH transform.
GPTQ intra-block Hessian transformation (Algorithm 1, Line 11): The paper reports that GPTQ's intra-block error correction uses the transformed Hessian rather than the original Hessian. This is a design choice that is not ablated — it is possible that using the original (untransformed) Hessian would work as well or better, since GPTQ's error correction compensates for weight quantization error, which may have different statistics in the transformed space. This is a missing ablation.
Critical Assessment
Claim 1: "WUSH improves W4A4 accuracy over the strongest Hadamard-based baselines by up to +2.8 average points with RTN and +0.7 with GPTQ on Llama-3.1-8B-Instruct in MXFP4"
Assessment: This claim is directly and cleanly supported by the evidence. The +2.8 figure is the difference between RTN-WUSH (73.21) and RTN-H (70.45) on Llama-3.1-8B-Instruct MXFP4 in Table 2 (rounded to one decimal: 73.21 - 70.45 = 2.76, which rounds to 2.8). The +0.7 figure is GPTQ-WUSH (74.35) minus GPTQ-H (73.65 = "MR-GPTQ") = 0.70. The Platinum Benchmarks (Figure 9) corroborate with MXFP4 RTN-WUSH at 79.25 vs. RTN-H at 76.41 = +2.84 points, and GPTQ-WUSH at 79.25 (note: the figure shows GPTQ-WUSH for MXFP4 only — the RTN vs. GPTQ comparison for MXFP4 on this model is not separable in the figure's bar chart display).
What the claim does NOT say, and the experiments do NOT test: The claim is about a specific model (Llama-3.1-8B-Instruct), a specific format (MXFP4), and specific baselines (RTN-H, GPTQ-H). Results for other models show smaller or even negative gains: Qwen3-32B MXFP4 RTN-WUSH is 82.01 vs. RTN-H at 82.15 (Table 4), a -0.14 point regression. The headline number should not be interpreted as a universal improvement across all models, formats, and quantization methods — it is the best-case result on a specific (representative) configuration.
Claim 2: "WUSH delivers up to 6.6× per-layer throughput over BF16 via FP4 MatMul, with kernel overhead matching that of optimized blockwise Hadamard kernels"
Assessment: Supported by Figure 2, where MXFP4 kernels achieve 6.63× speedup on some configurations and the WUSH + Quant bars are visually indistinguishable from H + Quant bars. The paper states an "average throughput difference of only 1.3%" and notes that "in some configurations, WUSH + Quant achieves identical performance to H + Quant." The evidence is convincing for batch size 1024 on RTX 5090. However, the results are for a single GPU and a single batch size — it is not demonstrated that the overhead remains negligible at smaller batch sizes (where the GEMM is less compute-bound and the transform overhead is a larger fraction of total time) or on different hardware architectures (the caching behavior that hides per-block specialization cost may differ on GPUs with smaller L1/L2 caches).
Claim 3: "The Hadamard transform's empirical success stems from being the single data-agnostic component of a more general optimal transform"
Assessment: This claim is theoretically proven in Section 4 (the Hadamard emerges as the optimal choice in both FP and INT analyses) and empirically validated through the WUS ablation. Table 1 shows that WUS (spectral scaling alone) outperforms identity for FP formats but fails catastrophically for INT4, while WUSH (spectral scaling + Hadamard) achieves the best or tied-best result across all formats. The paper has demonstrated that: (1) the Hadamard is the optimal orthogonal component theoretically, and (2) it is necessary empirically for robustness across formats. What the experiments do not test is whether a different orthogonal matrix (e.g., a learned rotation from SpinQuant) could serve the same role equally well — the only orthogonal baseline is the Hadamard itself. A comparison where is a learned orthogonal matrix (with , held fixed) would directly test whether the Hadamard is uniquely optimal or merely one good choice among many.
Claim 4: "WUSH nearly closes the gap between NVFP and MXFP formats"
Assessment: Supported with qualification. For Qwen3-14B (Table 4), NVFP4 RTN-WUSH achieves 79.80 and MXFP4 RTN-WUSH achieves 79.44 — a gap of only 0.36 points, both above 98% recovery. This is the paper's strongest evidence for gap closure. For Qwen3-8B (Table 4), NVFP4 GPTQ-WUSH (76.45) vs. MXFP4 GPTQ-WUSH (75.69) = 0.76 point gap. For Llama-3.1-8B-Instruct (Table 2), NVFP4 GPTQ-WUSH (76.10) vs. MXFP4 GPTQ-WUSH (74.35) = 1.75 point gap — substantially wider. The "nearly closes the gap" claim is model-dependent and format-configuration-dependent. It holds most clearly for the Qwen3 family at larger scales using RTN quantization, and less clearly for Llama models and GPTQ configurations. The paper would be stronger if it provided the NVFP4-MXFP4 gap systematically across all models and configurations, rather than requiring the reader to compute it from separate tables.
Claim 5: "WUSH is provably near-optimal for FP and INT quantizers under mild assumptions"
Assessment: Theoretically supported by the proofs in Section 4; empirically validated by Table 1. The FP proof (Section 4.2) shows that WUSH achieves the exact lower bound on quantization error under the multiplicative noise model — i.e., it is exactly optimal, not just near-optimal. The INT proof (Section 4.3) shows WUSH is within a factor of (block size) of optimal for general distributions and within for tail-bounded distributions — i.e., asymptotically near-optimal. Table 1 provides empirical support: WUSH achieves the lowest loss in all tested configurations. However, the experiments do not provide the full achievable lower bound for comparison — i.e., there is no empirical measurement of what the theoretical minimum loss would be, so the claim that WUSH achieves it cannot be directly verified from the data. The gap between WUSH and the theoretical optimum is a mathematical result from Section 4, not an empirical measurement.
Genuine Weaknesses in Experimental Design
1. Single calibration configuration throughout. All experiments use 32 calibration samples of sequence length 2048 from FineWeb-Edu. There is no ablation of calibration set size (would 8 samples suffice? would 128 improve results?), calibration dataset (does FineWeb-Edu produce better transforms than, say, C4 or Wikipedia?), or sequence length. The generalization from 32 FineWeb-Edu samples to diverse downstream tasks is impressive but its sensitivity is unknown. A practitioner wanting to apply WUSH to a different domain (code, math, multilingual) has no guidance on calibration data requirements.
2. No comparison to WUSH variants with simplified . The paper claims the Hadamard is optimal for , but does not test whether a simpler choice (e.g., , which gives WUS) could achieve comparable end-to-end accuracy with lower implementation complexity. WUS is only evaluated in Table 1 (layerwise loss), not in end-to-end accuracy benchmarks. Given that WUS outperforms WUSH on NVFP4 in Table 1, an end-to-end WUS evaluation would be directly relevant to the claim that the Hadamard is necessary.
3. INT4 end-to-end accuracy not reported. Table 1 shows INT4 layerwise loss, but no end-to-end accuracy results are provided for INT4 quantization. The dramatic loss reductions in Table 1 (e.g., Q: 170 → 2.39) suggest large accuracy gains, but these are not verified at the model level. This is a significant gap, particularly since INT4 weight-activation quantization is a common practical use case.
4. MXFP4 evaluation limited to group size 32. MXFP4 is evaluated only with group size . The NVFP4 evaluation uses group size 16 (as specified by the format). The paper does not evaluate WUSH with other group sizes, which would test the claim that WUSH's optimality is independent of block size. Different group sizes would change the singular value distribution within each block (larger blocks capture more variance and may have more extreme outliers), potentially affecting the relative benefit of WUSH over Hadamard.
5. No direct comparison to FlatQuant. FlatQuant (Sun et al., 2025) is cited as a learned transform baseline but is not included in any experimental comparison. The paper's claim that learned transforms are "costly to learn" and "heuristic" is not backed by an empirical comparison showing that WUSH achieves better accuracy or lower computational cost than FlatQuant specifically.
6. Single GPU architecture for kernel benchmarks. The GPU kernel benchmarks (Figure 2) are on a single RTX 5090. The caching argument for why per-block specialization has negligible overhead depends on L1/L2 cache sizes and memory bandwidth, which differ across GPU architectures. Results may not transfer to datacenter GPUs (A100, H100) or consumer GPUs with smaller caches.
7. Batch size 1024 only for kernel benchmarks. At smaller batch sizes, the GEMM is less compute-bound and the transform overhead becomes a larger fraction of total time. The paper does not show a sweep over batch sizes, so the 1.3% overhead claim may not hold at low batch sizes typical of interactive inference (batch size 1-8).
8. Damping factor not reported or ablated. The numerical value of used in Algorithm 1 is never stated, and its impact on transform construction is not analyzed. For near-singular second moments, the choice of determines how much the effective singular values are regularized, which directly affects the scaling in the WUSH transform. This is a missing detail that affects reproducibility.
Missing Experiments That Would Strengthen the Paper
- End-to-end INT4 accuracy on all models and benchmarks, to validate the dramatic layerwise loss improvements translate to task performance.
- Calibration data ablation: sweep calibration set size (8, 16, 32, 64, 128 samples), calibration dataset (FineWeb-Edu, C4, Wikipedia, code-specific data), and sequence length, measuring both layerwise loss and end-to-end accuracy.
- WUS end-to-end evaluation on LM Eval Harness and Platinum Benchmarks, to determine whether the Hadamard component is necessary for end-to-end accuracy or only for layerwise loss robustness.
- Group size sweep: evaluate WUSH with MXFP4 at group sizes 16, 32, 64 to test whether the optimality claim is group-size-independent.
- Latency benchmarks at batch sizes 1, 4, 8, 16, 32, 64 (typical inference batch sizes), not just 1024, to show that the kernel overhead remains negligible in interactive settings.
- Comparison to FlatQuant and possibly other learned transform methods, to substantiate the claim that the closed-form approach is superior.
- Sensitivity to : report the value used and sweep it across orders of magnitude to show robustness.
- Perplexity evaluation on a language modeling benchmark (e.g., WikiText-2) as a continuous metric complementary to task accuracy — this would provide higher-resolution discrimination between methods when task accuracies are close.
Conditional Nature of the Claims
The paper's central empirical claims hold most strongly for smaller models (3B-8B) using MXFP4 with RTN quantization, where the gains over Hadamard are largest (+2 to +4 average points). For larger models (14B-32B), for NVFP4 format, and for GPTQ-based quantization, the gains narrow to 0-1 points and can even reverse (Qwen3-32B MXFP4 RTN). The throughput claim holds at batch size 1024 on RTX 5090 and has not been demonstrated at smaller batch sizes. The near-optimality claim is a mathematical result from Section 4 whose empirical validation is limited to layerwise loss comparisons against baselines rather than against a computed lower bound. Practitioners adopting WUSH should calibrate expectations based on their specific model scale, target format, and quantization method — the method provides consistent but not universal improvements over the state-of-the-art.
6. Limitations and Trade-offs
The Difficulty Estimation Cost Is Not Accounted For — and It May Dominate the Total Compute Budget
The assumption or constraint. The WUSH transform is constructed from per-block second-moment statistics $d^{-1}_{\text{out}} W W^\top$ and $d^{-1}_{\text{batch}} X X^\top$, computed from calibration data. The paper uses 32 calibration samples of sequence length 2048 from FineWeb-Edu (Section 6.1). While this is modest compared to training, it requires a full forward pass through the unquantized model — every layer must process 32 × 2048 = 65,536 tokens to collect activation statistics. For a large model, this calibration pass consumes non-trivial compute and memory. The paper does not include this cost in any throughput or efficiency comparison; the GPU kernel benchmarks in Figure 2 measure only per-layer GEMM speedup relative to BF16, starting from the assumption that the transform matrices have already been computed offline and the weights have already been pre-quantized.
The consequence. For deployment scenarios where the model must be quantized on-the-fly (e.g., when weights are downloaded and immediately deployed, or when different quantization configurations are tried per-task), the calibration cost directly adds to the total time-to-deployment. More subtly, the calibration pass requires running the full-precision model — meaning the infrastructure must support BF16 inference for the calibration step, which partially defeats the purpose of quantization for memory-constrained edge devices. A user who wants to quantize a model on a device that cannot run BF16 inference at all (e.g., a mobile phone with limited RAM) cannot use WUSH without access to a more capable machine for the calibration step.
What evidence exists in the paper. The paper does not measure or report calibration wall-clock time, memory usage, or FLOPs. The 32-sample configuration is stated once in Section 6.1 and never ablated — there is no experiment showing whether 8 samples would suffice, or whether 128 samples would improve results, or how sensitive the transform quality is to calibration set size or composition. Table 3 shows that transforms calibrated on FineWeb-Edu generalize to C4 (a different dataset), but the computational cost of obtaining those transforms is never quantified. The GPU kernel benchmarks (Figure 2) show "WUSH + Quant" speedups relative to BF16 GEMM only — the calibration cost is externalized entirely.
Mitigation status. The paper does not address this limitation. It does not suggest cheaper difficulty estimation, does not explore whether the calibration set size can be reduced, and does not amortize the calibration cost into any efficiency metric. The GitHub repository linked in the abstract may include calibration scripts, but the paper's experimental section provides no guidance on minimizing this overhead. This is a practical deployment concern that users must evaluate on a case-by-case basis.
WUSH Evaluates Only W4A4 Quantization — Generalization to Other Bitwidths, Weight-Only, or Activation-Only Quantization Is Unverified
The assumption or constraint. All experiments in the paper use 4-bit quantization of both weights and activations (W4A4) with MXFP4, NVFP4, or INT4 formats (Section 6). The theoretical derivation in Section 4 assumes a specific quantization error model: for FP, multiplicative noise $\varepsilon(\alpha)_k \propto \alpha_k$; for INT, block-maximum-dependent noise $\varepsilon(\alpha) \propto \|\alpha\|_\infty$. The paper does not extend the analysis to other bitwidths (W8A8, W4A8, W8A4, W2A2), to weight-only quantization ($T_X = I$), or to activation-only quantization ($T_W = I$).
The consequence. The theoretical optimality guarantees (exact for FP, near-optimal for INT) are derived for the specific noise models that approximate 4-bit quantization behavior. At higher bitwidths (8-bit), quantization error is much smaller, the relative benefit of the WUSH transform may shrink, and the additional overhead of per-block non-orthogonal transforms may not be worth the marginal accuracy gain. At lower bitwidths (2-bit), the noise models may break down entirely — quantization error becomes so large that the first-order approximation splitting the two-sided loss (Eq. 13, which drops the $\varepsilon(T_W w)^\top \varepsilon(T_X x)$ cross-term) may no longer be valid. Weight-only quantization is an important practical use case (where activations remain in BF16 or FP16) that the paper's joint optimality analysis does not cover — the reduction to a one-sided problem in Section 3.4.2 relies on $T_W = T_X^{-\top}$, which may not be the optimal constraint when only one tensor is quantized.
What evidence exists in the paper. Zero. There are no experiments at any bitwidth other than 4. There is no discussion of how the optimality analysis would change for W8A8, W2A2, or weight-only settings. The paper's claims of "provably near-optimal for FP and INT quantizers" (Abstract) are technically qualified by the 4-bit noise models in Sections 4.2.1 and 4.3.1, but this qualification is not prominently surfaced. A practitioner considering WUSH for 8-bit quantization has no empirical evidence to guide the decision.
Mitigation status. The paper does not address this limitation. It does not note the restriction to 4-bit quantization as a scope limitation, does not discuss how the analysis would change at other bitwidths, and does not suggest future work extending the framework to other quantization regimes. This is a scope limitation that is implicit in the experimental design but never explicitly acknowledged.
The Method Assumes a Block-Diagonal Transform Structure Aligned with Quantization Groups — Architectures with Irregular Grouping or Different Quantization Schemes Are Not Supported
The assumption or constraint. The WUSH construction fundamentally depends on the block-diagonal constraint (Eq. 3): the transform is partitioned into independent $d \times d$ blocks, where $d$ is exactly the quantization group size, and $d$ must be a power of 2 dividing $d_{\text{in}}$ (Section 3, "Block-independent constraint"). The per-block transforms are computed independently, and the layerwise loss is approximated as the sum of independent blockwise losses. This architecture assumes that the quantization scheme uses uniform group sizes and that the group boundaries align with the transform block boundaries.
The consequence. Several practical quantization schemes violate this assumption:
- Per-channel quantization (common in INT8 weight quantization, where scale factors are computed per output channel, not per block of input channels) does not have a natural block structure aligned with input dimensions.
- Non-uniform block sizes (some schemes use smaller blocks for the first/last layers or for attention vs. MLP projections) would require different
$T_{\text{wush}}$constructions per block, which the paper's framework can technically handle but does not evaluate. - Sub-channel quantization (where blocks are smaller than the natural hardware word size, e.g., 16-element blocks for NVFP4) is evaluated in the paper, but the group size must still evenly divide
$d_{\text{in}}$. - Vector quantization or product quantization approaches that do not use scalar AbsMax scaling per block fall outside the paper's error model entirely, since the INT and FP noise models in Sections 4.2.1 and 4.3.1 assume blockwise scalar quantization scales.
More subtly, the block-independence approximation $\ell \approx \sum_i \ell_{(i)}$ assumes that quantization errors in different blocks are uncorrelated. The paper states this as an approximation (Section 3) but does not analyze the cross-block error terms or provide bounds on the approximation error. If block errors are positively correlated (which could happen if outlier channels span multiple blocks or if the Hessian has strong off-diagonal-block structure), the total layerwise error could be larger than the sum of blockwise errors.
What evidence exists in the paper. The layerwise loss experiments in Table 1 implicitly validate the block-diagonal approach (the measured per-layer losses are consistent with theory), but there is no ablation comparing block-diagonal WUSH against a full-matrix WUSH (which would be computationally prohibitive but would establish an upper bound on achievable loss reduction). The block-independence approximation is stated once and never revisited. The requirement that $d$ divide $d_{\text{in}}$ is satisfied for all tested models (standard transformer dimensions are multiples of 32), but models with non-standard hidden dimensions would require padding or non-uniform block sizes that the paper does not discuss.
Mitigation status. The paper does not address this limitation. The block-diagonal structure is presented as a practical constraint motivated by inference-time overhead (Section 3), not as a theoretical necessity, but the paper never quantifies how much optimality is sacrificed relative to a full-matrix transform. The requirement that $d$ be a power of 2 is stated without justification (it likely relates to the Hadamard matrix construction, which requires power-of-2 dimensions).
End-to-End INT4 Accuracy Is Not Evaluated, Leaving the Strongest Layerwise Gains Unverified at the Model Level
The assumption or constraint. Table 1 shows that WUSH achieves dramatic layerwise loss reductions for INT4 quantization — e.g., Q projection loss drops from 170 (identity) or 5.57 (Hadamard) to 2.39 (WUSH), a 2.3× reduction over Hadamard and a 71× reduction over identity. These are the largest relative improvements in the entire paper. However, the paper never reports end-to-end model accuracy for INT4 quantization. All end-to-end benchmarks (Tables 2, 4; Figures 3, 6-9) use only MXFP4 and NVFP4 formats.
The consequence. The key practical question for INT4 quantization — "does this layerwise improvement translate to better downstream task performance?" — is left unanswered. Layerwise loss and end-to-end accuracy do not always correlate perfectly: a small number of outlier layers can dominate end-to-end error, GPTQ error propagation can compensate for per-layer losses in ways that simple RTN loss measurements don't capture, and task-specific effects (e.g., attention patterns being more sensitive to quantization than MLP outputs) can alter the relationship. Moreover, INT4 quantization of activations is known to be particularly challenging because activation distributions can shift dramatically across tokens and layers — the calibration-based second-moment statistics used by WUSH may not capture this token-level variation as well for INT4 as they do for FP4 formats.
The WUS ablation in Table 1 provides a cautionary tale: WUS (WUSH without Hadamard) reduces layerwise loss for NVFP4 but catastrophically increases INT4 loss (Q projection: 213 vs. 170 for identity). This non-monotonic behavior — a transform that helps one format hurts another — demonstrates that layerwise loss improvements do not automatically generalize across formats. Without end-to-end INT4 experiments, it is unknown whether WUSH's INT4 layerwise gains translate to real accuracy improvements or whether the transform introduces subtle pathologies at the model level.
What evidence exists in the paper. Table 1 is the only INT4 result. No INT4 end-to-end benchmarks, no INT4 perplexity measurements, no INT4 GPTQ integration results. The paper does not explicitly state that INT4 end-to-end evaluation was omitted or provide a reason for the omission.
Mitigation status. The paper does not address this gap. The INT4 results are presented in Table 1 alongside MXFP4 and NVFP4 results, creating an impression of format-inclusive evaluation, but the subsequent end-to-end sections (6.2, Appendix B) quietly drop INT4 without comment. This is a significant omission for practitioners interested in INT4 quantization — they must extrapolate from FP4 results, which the WUS ablation shows can be misleading.
The Per-Block Transform Specialization Requires Storing $C = d_{\text{in}}/d$ Distinct Transform Matrices — a Storage Overhead Not Discussed
The assumption or constraint. WUSH assigns a distinct $d \times d$ transform matrix $T_{\text{wush}(i)}$ to each of the $C = d_{\text{in}}/d$ blocks in a layer (Section 5). For a typical linear layer with $d_{\text{in}} = 4096$ and block size $d = 32$, this means storing $C = 128$ matrices, each of size $32 \times 32 = 1024$ elements, for a total of $128 \times 1024 = 131,072$ elements per layer — equivalent to 131K parameters for the transforms alone in that layer. Across all linear layers in a transformer (typically 7 per transformer block: Q, K, V, O, G, U, D, plus the embedding and LM head), the total transform storage for a 32-layer model would be $32 \times 7 \times 131\text{K} \approx 29\text{M}$ elements. At BF16 precision, this is approximately 58 MB of additional storage just for the transform matrices.
The consequence. This storage overhead is not negligible relative to the quantized model size. A W4A4 quantized 8B-parameter model uses approximately 4 GB of storage for weights (8B parameters × 0.5 bytes per parameter at 4 bits). The 58 MB of transform matrices represents roughly a 1.5% increase in total model size. While this is modest, it partially offsets the storage savings from quantization: the effective bits-per-parameter includes the transform overhead. More importantly, the transform matrices must be loaded from memory during inference — each block's $d \times d$ matrix must be available when that block's activation slice is processed. The paper's GPU kernel design stores the transforms in a $(G, G, C)$ layout and argues that caching mitigates the overhead (Section 5), but this argument applies only to the bandwidth cost during the transform kernel, not to the storage cost in GPU memory. For edge deployments with tight memory budgets (e.g., mobile devices, embedded systems), this overhead may be meaningful.
Additionally, the transform matrices are not quantized — they must remain in BF16 or FP16 to avoid introducing additional quantization error in the transform step itself. This means the transforms consume full-precision memory while the weights they transform are quantized to 4 bits. The paper does not explore whether the transforms themselves could be quantized (e.g., to INT8) without degrading accuracy.
What evidence exists in the paper. The paper does not report the total transform storage in bytes, the increase in model size relative to the quantized weights, or the GPU memory footprint of the transform matrices. The kernel benchmarks in Figure 2 measure per-layer GEMM speedup but do not report memory usage. The paper never quantifies the storage overhead or discusses whether it matters for deployment.
Mitigation status. The paper does not address this limitation. Section 7 ("Future Work") mentions "structurally simplified variants of WUSH that trade optimality for a reduction in the cost of applying the transforms at inference time," but does not specifically mention storage reduction. The idea of approximating the data-aware component with a diagonal matrix (Section 7) would reduce storage from $d \times d$ to $d$ elements per block, cutting the overhead by a factor of $d$, but this is only a future direction with no experimental validation.
The Optimality Proofs Rely on Modeling Assumptions That May Not Hold in Practice — the Gap Between Theory and Empirical Behavior Is Not Quantified
The assumption or constraint. The theoretical optimality results in Section 4 depend on several modeling assumptions:
- Unbiased quantization (Section 4.1):
$\mathbb{E}[\varepsilon(\alpha)] = 0$for all$\alpha$. Real quantizers (RTN with AbsMax scaling) are deterministic and biased — the rounding error is not zero-mean, and the sign of the error depends on whether the value rounds up or down. - First-order approximation (Eq. 13): the cross-term
$\varepsilon(T_W w)^\top \varepsilon(T_X x)$is dropped. This is justified as being "small," but its magnitude relative to the retained terms is never bounded or measured. - FP multiplicative noise model (Section 4.2.1):
$\varepsilon(\alpha) = \text{diag}(\eta) \alpha$with$\mathbb{E}[\eta] = 0$. The paper acknowledges that MXFP4 "behaves as a hybrid between ideal FP and INT quantization" (Section 6.1) due to subnormal numbers and small exponent ranges, meaning the model is not purely multiplicative. - INT block-maximum noise model (Section 4.3.1):
$\varepsilon(\alpha) = \|\alpha\|_\infty \eta$. This assumes the quantization scale is determined solely by the maximum absolute value — but practical INT quantizers often use MSE-optimized clipping thresholds (the paper mentions "Gaussian MSE clipping" for its INT4 experiments in Section 6.1) that deviate from pure AbsMax. - I.I.D. columns (Section 4.1): columns of
$W$and$X$are treated as i.i.d. samples. For attention layers, activation tokens in a sequence are not independent (they attend to each other), and weight columns within a layer share structured patterns (e.g., low-rank structure).
The consequence. The "optimality" proven in the paper is optimality under the model, not necessarily optimality under real quantization. The gap between the model-predicted optimal transform and the true optimal transform is unknown. If the model assumptions are significantly violated — e.g., if the quantization bias is large, or if the cross-term $\varepsilon^\top \varepsilon$ is non-negligible — the WUSH construction may be suboptimal in ways the theory cannot predict. The paper's empirical results (Table 1, end-to-end accuracy) show that WUSH improves over baselines, not that it achieves the theoretical lower bound. There is no experiment measuring how close WUSH comes to the unachievable oracle optimum under real quantization.
The MXFP4 case is particularly revealing: the theory predicts that orthogonal transforms provide no benefit under ideal FP noise (Eq. 21), yet the Hadamard empirically helps for MXFP4 (Table 1: H 7.24 vs. I 11.1 for Q projection). The paper explains this as MXFP4 having "INT-like behavior" (Section 6.1), but this explanation also implies that the FP optimality proof does not directly apply to MXFP4 — the actual quantizer is a hybrid, and the theoretical guarantees are for the idealized endpoints (pure FP, pure INT). A user applying WUSH to MXFP4 is relying on the INT proof's guarantees (since MXFP4 behaves partially like INT), but with an unquantified degradation from the INT-optimal construction.
What evidence exists in the paper. The layerwise loss experiments in Table 1 provide indirect evidence of the gap between theory and practice: for NVFP4, the theory's FP model predicts orthogonal transforms are neutral (Eq. 21), but the identity (4.23) outperforms the Hadamard (5.60) — the model fails to capture NVFP4's top-element preservation effect. WUSH still achieves the lowest loss (2.40), but the theoretical guarantee of optimality for FP is not cleanly applicable. The paper does not measure the cross-term magnitude, the quantization bias, or the i.i.d. assumption violation. There is no comparison of the theoretically predicted loss lower bound (e.g., $d^{-1}(\text{tr}(S))^2$ from Eq. 20) against the empirically measured loss.
Mitigation status. The paper partially acknowledges these model limitations. Section 6.1 notes the MXFP4 hybrid behavior. Appendix A.4 provides detailed justifications for the noise models, but these are derivations from idealized quantizer definitions, not empirical validations. The paper does not suggest future work on tighter models, model-error bounds, or empirical validation of the optimality gap. A user trusting the "provably near-optimal" claim (Abstract) should understand it as a statement about the model, not about real quantization — a distinction the paper could make more prominently.
7. Implications and Future Directions
How This Work Changes the Landscape
This paper fundamentally reframes the problem of transform design for LLM quantization from an empirical trial-and-error pursuit into a constrained optimization problem with a verifiable optimality condition. Prior to WUSH, the field operated under the implicit assumption that transforms should be orthogonal — rotations or reflections that preserve mathematical equivalence through the linear layer. The central research question was which orthogonal matrix works best: random, Hadamard, or learned. WUSH demonstrates that orthogonality is not merely suboptimal but provably can be up to $d$ times worse than the optimal non-orthogonal transform (where $d$ is the block size, e.g., 32) — a quantitative gap that was previously unsuspected.
The shift is more than a better transform. It is a methodological pivot from heuristic search over transform families (QuIP, QuaRot, SpinQuant) to first-principles derivation from quantization error models. The paper shows that the optimal construction has a specific, interpretable structure — Hadamard rotation, spectral square-root rescaling, identity alignment — that emerges from minimizing a trace lower bound (for FP) or a product of Frobenius norm and expected maximum (for INT). This converts transform design from an art into a science: one can now ask not "does this transform improve accuracy?" but "how close does this transform come to the theoretical lower bound $d^{-1}(\text{tr}(S))^2$?" The bound itself is a contribution — it provides an absolute performance ceiling for any linear blockwise transform under the given noise models, enabling rigorous comparison of future methods.
Reconciling prior contradictions. The paper resolves several tensions in the literature:
-
Why Hadamard works, but imperfectly. The Hadamard's empirical success (QuaRot, QuIP#, MR-GPTQ) had been attributed to "incoherence processing" or "Gaussianization" — heuristic explanations that didn't specify when it should fail. WUSH provides a precise answer: the Hadamard is the optimal fixed rotation for equalizing per-dimension variance (Eq. 56), but this is only one-third of the optimal construction. The spectral rescaling
$S^{-1/2}$and alignment$R = I$are equally essential. This explains why Hadamard-only methods plateau (as documented by Egiazarian et al., 2025): they get the rotation right but miss the variance-equalizing rescaling. -
Why Hadamard sometimes hurts (NVFP4). Egiazarian et al. (2025) observed that Hadamard transforms can be harmful for NVFP4 due to top-element preservation effects. The paper's NVFP4 results (Table 1) confirm this: identity (4.23) outperforms Hadamard (5.60) on the Q projection. But WUSH (2.40) dramatically outperforms both — the non-Hadamard components (
$S^{-1/2} U^\top W'^\top$) overcome the Hadamard's negative contribution. This demonstrates that the data-dependent scaling is robust to format-specific quirks in ways that the fixed Hadamard is not. -
Why self-correction and search methods reach different conclusions on different difficulty tiers. While not a direct focus of this paper, the principle that difficulty-dependent behavior explains conflicting prior results has a structural parallel here: format-dependent behavior explains why the same transform (Hadamard) helps MXFP4 but hurts NVFP4, and why a single construction (WUSH) can be optimal for both. The spectral structure of the data interacts with the quantizer's noise characteristics in format-specific ways, and WUSH's adaptivity handles this automatically — it does not need to know a priori whether it is quantizing for MXFP4, NVFP4, or INT4.
Research directions that become more attractive:
-
Verifier and quantizer robustness via spectral methods. The paper identifies verifier over-optimization as the core bottleneck for test-time search scaling (in the referenced prior work on compute-optimal inference). WUSH's insight — that optimal transforms have a specific spectral structure derived from second-moment statistics — suggests a parallel for verifier design: rather than training black-box verifiers via RLHF and hoping they remain calibrated, one could analyze the spectral properties of the verifier's predictions and design transforms or regularizers that prevent over-optimization. The connection between singular value spread and quantization error in WUSH has a natural analog in the connection between eigenvalue spread and over-optimization susceptibility in learned verifiers.
-
Block-adaptive inference strategies. The WUSH construction is computed per-block from local second-moment statistics. This opens the door to block-adaptive quantization where different blocks use different bitwidths or formats based on their spectral properties — blocks with high singular value spread get more aggressive transforms (or higher precision), while isotropic blocks use simpler transforms (or lower precision). The compute-optimal allocation framework from the prior reference paper (difficulty-conditioned strategy selection) could be applied at the block level: estimate each block's "quantization difficulty" from its singular value spectrum, then allocate a per-block bitwidth or transform budget to minimize total model error under a storage constraint.
Research directions that become less urgent:
-
Searching for better fixed orthogonal transforms. The paper proves that under ideal FP noise, orthogonal transforms provide zero benefit (Eq. 21): the trace term
$\text{tr}(S^2)$is invariant under rotation. The empirical benefit of the Hadamard for MXFP4 comes from deviations from the ideal model (hybrid FP-INT behavior), and the theoretical maximum benefit of any orthogonal transform over any other is bounded by the$d$gap in Eq. (17). WUSH's non-orthogonal spectral scaling provides much larger gains (up to$d \times$in theory,$2.3\times$in practice on INT4 Q projection). This suggests that further investment in fixed orthogonal transforms (e.g., searching over random rotation ensembles, optimizing rotation angles) is a low-return activity — the gains are fundamentally limited by orthogonality. -
Learned transforms without spectral structure. SpinQuant and FlatQuant learn transforms via gradient-based optimization without imposing spectral structure. WUSH demonstrates that the optimal structure is known in closed form and depends only on second-moment statistics. Learned transforms that rediscover this structure via SGD are strictly worse: they require more computation (gradient steps vs. one Cholesky + SVD), provide no optimality guarantees, and may converge to suboptimal local minima. Future learned transform methods should incorporate WUSH's spectral parameterization as an inductive bias — learning corrections to the
$S^{-1/2}$scaling rather than learning the full transform from scratch.
Follow-Up Research This Work Enables
1. Lightweight difficulty prediction for per-block transform selection. This is the most immediate practical next step. The paper's central remaining bottleneck is the cost of computing per-block second-moment statistics from calibration data — 32 forward passes of sequence length 2048, plus Cholesky decomposition and SVD for every block. Concrete experiment: Train a lightweight predictor (a small MLP or even a linear probe) that takes as input a block's weight matrix $W_{(i)}$ and predicts the optimal $S^{-1/2}$ scaling factors directly from the weight statistics alone, without requiring activation calibration data. The target for this predictor would be the singular values $s_k$ from the SVD of $W'^\top X'$ computed from calibration activations. If the predictor achieves high accuracy, it eliminates the calibration pass entirely — transforms are computed from weights statically, with no forward passes needed. This would make WUSH deployable in scenarios where calibration data is unavailable or where the calibration cost is prohibitive. A strong result would show that the predicted transforms achieve ≥95% of the layerwise loss reduction of the full calibration-based transforms, measured on the Table 1 benchmarks.
2. End-to-end INT4 evaluation with GPTQ integration. The paper's INT4 layerwise loss results (Table 1) show the largest relative improvements of any format — WUSH reduces Q projection loss from 170 (identity) to 2.39, a 71× reduction, and from 5.57 (Hadamard) to 2.39, a 2.3× reduction. But end-to-end INT4 accuracy is never reported. Concrete experiment: Run the full GPTQ-WUSH pipeline (Algorithm 1) for INT4 weight-activation quantization on Llama-3.1-8B-Instruct and Qwen3-8B, and evaluate on the same LM Evaluation Harness and Platinum Benchmarks suites used for MXFP4/NVFP4. Compare against INT4 baselines: RTN-I, RTN-H, GPTQ-I, GPTQ-H. A strong result would show that the 2.3× layerwise loss reduction over Hadamard translates to ≥1.5 average accuracy points on the 8B models, and that GPTQ-WUSH for INT4 closes most of the gap to the INT4 weight-only + BF16 activation baseline (a common practical configuration). A negative result — INT4 accuracy not improving despite layerwise loss reduction — would reveal that the layerwise-to-end-to-end mapping breaks down for INT formats, motivating work on cross-block error propagation models specific to integer quantization.
3. Block-adaptive mixed-precision quantization using the singular value spectrum as a difficulty signal. The paper's framework naturally produces a per-block "quantization difficulty" score: the singular value spread $\text{tr}(S^2) / (d^{-1}(\text{tr}(S))^2)$, which ranges from 1 (perfectly isotropic, easy to quantize) to $d$ (extreme outlier, hard to quantize). This is computed as a byproduct of the WUSH construction (Algorithm 1, line 6) and requires no additional computation. Concrete experiment: For each block in a model, compute the singular value spread. Sort blocks by spread and assign higher precision (e.g., INT8 or FP8) to the top-$k$ percentile of blocks and INT4 or FP4 to the remainder, using WUSH transforms at both precisions. Measure end-to-end accuracy as a function of $k$ and compare against uniform-precision baselines at the same average bitwidth. A strong result would show that WUSH-enabled mixed precision achieves higher accuracy than uniform W4A4 at the same or lower average bitwidth, and that the singular value spread is a better difficulty predictor than simpler heuristics (e.g., weight magnitude, activation variance). This connects directly to the compute-optimal allocation framework from prior work — but applied at the block level within a single model rather than at the prompt level across a dataset.
4. WUSH for weight-only quantization and the unification of weight and activation transforms. The paper's optimality analysis (Section 4) assumes joint weight-activation quantization with the unbiasedness constraint $T_W = T_X^{-\top}$. Weight-only quantization ($T_X = I$, activations in BF16) is a practically important regime that breaks this constraint — the transforms are no longer inverses, and the two-sided loss decomposition (Eq. 13) does not apply. Concrete experiment: Derive the optimal weight-only transform by setting $T_X = I$ and minimizing $\mathbb{E}_{w, \varepsilon(T_W w)} \| W'^\top T_W^{-1} \varepsilon(T_W w) \|^2$ under the FP and INT noise models. Does the same $U' = H, S' = S^{1/2}, R = I$ structure emerge, or does weight-only quantization admit a different (possibly simpler) optimal transform? Evaluate the resulting transform on weight-only INT4 and FP4 quantization benchmarks (e.g., perplexity on WikiText-2, accuracy on downstream tasks) and compare against AWQ, GPTQ, and QuaRot's weight-only mode. A strong result would show that the weight-only optimal transform is a special case of WUSH, unifying weight-only and weight-activation quantization under a single theoretical framework and enabling a single codebase to handle both.
5. Stress-testing the noise models: when does the first-order approximation break? The paper's optimality proofs depend on the first-order approximation that drops the cross-term $\varepsilon(T_W w)^\top \varepsilon(T_X x)$ in Eq. (13). This is justified as "small," but its magnitude is never bounded or measured. At very low bitwidths (2-bit, 3-bit), quantization errors become large, and the cross-term may be non-negligible — the optimal transform under the full loss may differ from the WUSH construction. Concrete experiment: Implement W4A2 and W2A2 quantization of Llama-3.2-3B-Instruct using WUSH transforms, and measure: (1) the empirical magnitude of the cross-term $\mathbb{E}[\varepsilon(T_W w)^\top \varepsilon(T_X x)]$ relative to the retained terms, (2) the layerwise loss as a function of bitwidth, and (3) end-to-end accuracy. Compare against a baseline that optimizes the full loss (including the cross-term) via gradient descent over the transform parameters, initialized from the WUSH solution. A strong result would identify the bitwidth threshold below which WUSH's optimality guarantee breaks down, providing a clear scope boundary for practitioners. A negative result — WUSH remaining near-optimal even at 2 bits — would strengthen the paper's claims about the robustness of the first-order approximation.
6. Diagonal WUSH approximation: trading optimality for storage and latency. Section 7 of the paper explicitly proposes "approximating the data-aware component with a diagonal matrix" as future work. This would reduce per-block transform storage from $d \times d$ to $d$ elements (a 32× reduction for $d=32$) and simplify the online transform from a matrix multiplication to an elementwise scaling. Concrete experiment: For each block, compute the WUSH transform $T_{\text{wush}(i)}$ as in Algorithm 1, then approximate it by a diagonal matrix $\tilde{T}_{(i)} = \text{diag}(t_1, \ldots, t_d)$ where $t_k$ minimizes the Frobenius norm $\| T_{\text{wush}(i)} - \tilde{T}_{(i)} \|_F$ (the optimal diagonal approximation is simply the diagonal of $T_{\text{wush}(i)}$ after an appropriate orthogonal alignment). Measure: (1) the increase in layerwise quantization loss relative to full WUSH, (2) the reduction in transform storage (bytes) and online transform latency (microseconds), and (3) end-to-end accuracy on Platinum Benchmarks for Llama-3.1-8B-Instruct. A strong result would show that the diagonal approximation recovers ≥80% of WUSH's accuracy gain over Hadamard while reducing transform storage by 32× and transform latency by 5-10× (since elementwise scaling is cheaper than matrix multiplication). This directly addresses the paper's limitation of unreported storage overhead and makes WUSH practical for memory-constrained edge deployments.
Practical Applications and Downstream Use Cases
1. Democratizing high-accuracy 4-bit inference for the MXFP open standard. The paper's headline result — WUSH narrows the NVFP4-MXFP4 accuracy gap to as little as 0.36 points on Qwen3-14B (Table 4), with both formats above 98% recovery — has direct commercial implications. MXFP4 is an open standard (OCP MX Alliance, 2023) with potential for broad hardware support across vendors (Intel, AMD, ARM, and custom ASICs). NVFP4 is NVIDIA-proprietary, available only on Blackwell GPUs. An organization deploying LLMs at scale can now choose MXFP4-compatible hardware — which may be cheaper, more available, or better suited to their power envelope — with confidence that WUSH-compensated MXFP4 accuracy will be competitive with NVFP4. Specific benefit: A cloud inference provider using WUSH + MXFP4 on non-NVIDIA hardware could offer 4-bit inference at quality levels previously achievable only with NVFP4 on Blackwell GPUs, expanding their hardware procurement options. The paper's GPU kernel demonstration (Figure 2, up to 6.63× speedup over BF16) further validates that the throughput is competitive, and the open-source release enables integration without licensing barriers.
2. On-device deployment of larger models via storage-aware WUSH configuration. The paper shows that WUSH provides the largest relative gains on smaller models — Llama-3.2-3B-Instruct gains +4.04 average points on MXFP4 RTN over the Hadamard baseline (Table 4). This is precisely the regime relevant for on-device deployment (smartphones, laptops, edge accelerators), where 3B-8B parameter models are the upper limit of feasible inference and every percentage point of accuracy matters for user experience. Specific benefit: A mobile application developer quantizing Llama-3.2-3B-Instruct to W4A4 for on-device summarization could use GPTQ-WUSH (MXFP4, 92.92% recovery, 66.43 average accuracy on LM Eval Harness) instead of GPTQ-H (87.27% recovery, 62.39 average accuracy), recovering 4 points of average accuracy at the same memory footprint. This could be the difference between a usable and an unusable on-device assistant. The developer would need to budget for the additional transform storage (approximately 1.5% increase in model size, as calculated in the Limitations analysis) and ensure the fused WUSH kernel is available for their target hardware.
3. Efficient calibration and quantization pipelines for model hubs. Model hubs (Hugging Face, Ollama, NVIDIA NIM) that serve quantized versions of open-weight models to diverse hardware targets currently rely on GPTQ or AWQ with fixed transforms. WUSH's closed-form construction from calibration data is a drop-in replacement for the transform step in these pipelines — it uses the same Hessian that GPTQ already computes (Algorithm 1, line 1) and adds only a Cholesky decomposition and per-block SVD (lines 2-9). Specific benefit: A model hub could offer "WUSH-enhanced" quantized variants alongside standard GPTQ variants. The computational cost of the additional Cholesky + SVD steps is negligible relative to the GPTQ weight quantization pass itself (which processes all weight columns with iterative Hessian updates). For the cost of a few extra seconds per layer during the one-time quantization process, the hub could provide quantized models with +0.7 to +2.8 average accuracy points (Table 2) for all downstream users, regardless of their inference hardware. The calibration data requirement — 32 samples from FineWeb-Edu — is already standard practice in GPTQ pipelines.
4. Fast iteration over format choices without retraining. A practical headache in LLM deployment is deciding which quantization format to use for a given model-hardware combination. MXFP4 vs. NVFP4 vs. INT4 vs. FP8 each have different accuracy-speed tradeoffs, and the optimal choice depends on the specific model, the target hardware, and the downstream task distribution. WUSH's format-agnostic construction — the same per-block transforms work for MXFP4, NVFP4, and INT4 without any format-specific tuning — means that a practitioner can compute the WUSH transforms once from calibration data and then evaluate multiple quantization formats by simply changing the quantizer $q(\cdot)$ in Algorithm 1 (lines 11 or 14). Specific benefit: A research team evaluating quantization strategies for a new model can run Algorithm 1 once to compute all per-block transforms and Hessians, then sweep over format choices (MXFP4, NVFP4, INT4, with various group sizes) by re-running only the quantization step (lines 10-17) — no recalibration, no re-computation of Cholesky factors or SVDs. This reduces the experimental cycle time from "recalibrate for each format" to "re-quantize only," enabling faster iteration and more thorough format sweeps. Table 1 directly demonstrates this property: a single WUSH construction is evaluated across three formats with consistent benefits.
