Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation

ArXiv: 2601.22813

🎯 Pitch

Standard NVFP4 pre-training loses accuracy because the prevailing unbiased estimator—element-wise stochastic rounding—imposes unnecessarily high quantization error. Quartet II moves the stochasticity to coarser microscale factors (MS-EDEN), dropping gradient error by over 2× while preserving unbiasedness, and when combined with an optimized forward pass, delivers up to 4.2× speedup on Blackwell GPUs and >20% better validation loss than prior NVFP4 recipes.

---

1. Executive Summary

This paper introduces MS-EDEN (MicroScaling EDEN), a novel unbiased quantization routine for NVFP4 micro-scaled floating-point formats that reduces quantization error by more than 2× relative to standard element-wise stochastic rounding by moving stochasticity from individual FP4 values to the microscale factors while preserving provable unbiasedness. Integrated into Quartet II, a fully-NVFP4 linear-layer computation graph that pairs a high-capacity forward pass using native NVFP4 scaling with Four-Over-Six scale selection and an unbiased backward pass built on MS-EDEN with inner-dimension randomized Hadamard transforms, the method achieves consistently improved gradient estimation across all major matrix multiplications in transformer training. Validated on Llama-2–style pre-training up to 1.9B parameters on 38B tokens on the C4 and FineWeb-Edu datasets, Quartet II improves validation loss by at least 20% over prior NVFP4 recipes and delivers up to 4.2× speedup over BF16 on NVIDIA Blackwell GPUs, establishing that superior backward-pass gradient estimation through coarser-grouped stochasticity yields meaningful accuracy gains even under the constraint of weight re-quantization—though hardest-evaluated downstream benchmarks show that gains in pre-training perplexity do not reliably translate into statistically significant improvements on zero-shot tasks after short fine-tuning phases.

2. Context and Motivation

The Core Problem: Stable 4-Bit Pre-Training Requires a Better Unbiased Quantizer

The fundamental gap this paper addresses is deceptively specific: current NVFP4 pre-training recipes lose significant accuracy relative to FP16 and FP8 because the prevailing unbiased gradient estimation technique—element-wise stochastic rounding—has unnecessarily high quantization error. The paper's insight is that this error is not an inherent property of 4-bit quantization; rather, it is an artifact of where stochasticity is applied. By moving the stochasticity from individual FP4 values to the coarser microscale factors, one can achieve the same unbiasedness guarantees with substantially lower mean-square error.

This matters because the field currently stands at an inflection point where 4-bit training is transitioning from a research curiosity to a hardware-supported reality. The NVFP4 format is natively supported in NVIDIA Blackwell GPUs, promising 2–4× throughput gains over FP8 for individual matrix multiplications (NVIDIA, 2024). Yet the paper opens by observing that "existing quantized training methods still sacrifice some of the representation capacity of this format in favor of more accurate unbiased quantized gradient estimation by stochastic rounding (SR), losing noticeable accuracy relative to standard FP16 and FP8 training." In other words, we have the hardware to run 4-bit training fast, but not the algorithms to run it accurately enough to replace higher-precision training in production pre-training pipelines.

The gap is not theoretical—it manifests as concrete validation loss degradation in end-to-end LLM pre-training. Several recent works have demonstrated stable fully-quantized training in NVFP4 (NVIDIA et al., 2025; Castro et al., 2025; Chmiel et al., 2025; Chen et al., 2025b), but all report a persistent gap relative to FP8 and FP16 baselines. Closing this gap is the paper's stated objective.

Why This Problem Is Important

Sustainability of foundation model training. The paper frames this argument in its opening paragraph: "The computational cost of training state-of-the-art foundation models has been increasing at a roughly exponential pace, putting into question the sustainability of the area." Pre-training modern transformers is dominated by dense matrix multiplications (GEMMs) in the attention and MLP linear projections, so reducing GEMM precision is one of the most direct levers for reducing end-to-end training costs. A 4-bit GEMM uses half the memory bandwidth and roughly half the compute of an FP8 GEMM, and one-quarter that of BF16. If 4-bit training can be made to match FP16/FP8 accuracy, the economic and environmental implications are enormous.

The backward pass is the bottleneck. The paper notes that stable FP4 training has converged on a two-part principle. The forward pass should maximize representation capacity by minimizing quantization error (typically MSE), since forward-pass errors compound through the network and directly degrade model quality. The backward pass, however, is especially sensitive to bias: "consistent bias in gradient estimation can lead to persistently incorrect descent directions" (Section 3). Biased gradient quantization accumulates systematic error over many training steps, making unbiased (or carefully controlled) gradient quantization essential for stable convergence. The backward pass is therefore subject to a stricter requirement: the quantizer must not only be accurate but also unbiased in expectation. Existing methods satisfy this requirement through element-wise stochastic rounding, but at the cost of increased variance—and it is this variance gap that the paper targets.

Hardware is already here, but algorithms lag. NVIDIA Blackwell GPUs provide native support for micro-scaled FP4 formats with tensor-core acceleration. The paper cites this as a motivating force: the hardware capability exists, but practical adoption requires training recipes that don't sacrifice model quality. Each paper in the prior NVFP4 training literature has progressively improved accuracy, but a gap persists. The paper positions MS-EDEN as a technique that closes this gap by revisiting a fundamental primitive—unbiased quantization—rather than by adding more heuristics on top of existing approaches.

Prior Approaches and Where They Fall Short

The paper surveys a specific lineage of prior work that has shaped the current state of NVFP4 training:

NVIDIA's first end-to-end NVFP4 recipe (NVIDIA et al., 2025) established the baseline: square-block quantization of weights (one FP8 scale per 16×16 block, rather than per 1×16 block), Hadamard rotations on the backward pass, and keeping some layers in higher precision. This work demonstrated that fully-NVFP4 pre-training is possible at scale, but left a significant accuracy gap relative to FP8 training.

TetraJet-v2 (Chen et al., 2025b) enhanced the NVIDIA approach with improved outlier control and oscillation-reduction techniques. It also corrected how activation re-quantization is performed on the backward pass to better align with the chain rule, and added weight re-quantization similar to Castro et al. (2025). However, the paper points out a critical limitation of TetraJet-v2: it introduces intermediate-level FP32 scales and selective outlier channels, which require "substantially more complicated kernel support that is not provided by the authors." This makes TetraJet-v2 difficult to validate in practice and potentially incompatible with efficient GPU execution. When the paper references TetraJet-v2 in later comparisons, it uses a simplified, GPU-feasible version that omits these impractical components.

Four-Over-Six (Cook et al., 2025) proposed an orthogonal improvement: a quantization grid selection heuristic that evaluates two potential scale factors (4.0 and 6.0) for each block of values and picks the one yielding lower MSE. This is purely a forward-pass optimization, and the paper embraces it. However, Cook et al. combined it with stochastic rounding on the backward pass, and the paper identifies a subtle but important issue here (discussed below).

The unifying weakness: element-wise stochastic rounding. All these prior methods rely on element-wise FP4 stochastic rounding to preserve unbiasedness on the backward pass. The paper's Table 1 reveals the core problem numerically: SR achieves unbiasedness at the cost of approximately 2.5× higher MSE over simple round-to-nearest (RTN) quantization on normally-distributed data ($23.5 \times 10^{-3}$ vs. $9.0 \times 10^{-3}$). This is the price of unbiasedness under the standard approach: each individual FP4 value must be randomly rounded up or down proportional to its distance from the quantization boundaries, introducing substantial per-element variance.

The square-block trade-off. One specific design choice in the NVIDIA recipe illustrates the tension the paper seeks to resolve. NVIDIA et al. (2025) uses square-block quantization (16×16 groups) for the weight tensor on the forward pass, specifically so the quantized weights can be re-used without re-quantization on the backward pass for the input gradient computation $\partial L / \partial X \approx Q_{\text{FP4}}(E) \cdot Q_{\text{FP4}}(W^T)^T$. This halves the backward-pass quantization error for this matrix product (visible in Figure 1, comparing panels b and c). However, square-block quantization comes at a cost: having one FP8 scale per 256 FP4 values instead of per 16 values reduces outlier preservation and representation capacity on the forward pass. Figure 2 shows that square blocks ("16×16gs") consistently produce worse validation perplexity than native blocks ("1×16gs"). The paper frames this as a "representation-requantization trade-off"—one that prior work resolved in favor of better backward-pass gradient estimation at the expense of forward-pass model quality. MS-EDEN offers a way out of this trade-off by providing superior gradient estimation without needing the forward-pass square blocks to remain untouched.

How This Paper Positions Itself Relative to Existing Work

A fundamental improvement to the unbiased quantization primitive, not another heuristic. The paper explicitly distinguishes its approach: "All the above techniques employ some variant of stochastic rounding (SR) to preserve unbiasedness on the backward pass. We re-consider this choice, and propose a new unbiased gradient estimator (MS-EDEN) which provides significantly better MSE, and validation loss." The contribution is structural rather than incremental—replacing the element-wise stochasticity mechanism entirely rather than adding corrections around it.

The EDEN connection and why it requires adaptation. The paper draws inspiration from EDEN (Vargaftik et al., 2022), a technique from distributed optimization that uses randomized rotations and a bias-correction factor to achieve near-unbiased quantization with very low error. The EDEN approach uses a scaling factor $S = \langle \boldsymbol{x}, \boldsymbol{x} \rangle / \langle \text{RHT}(\boldsymbol{x}, \omega), Q(\text{RHT}(\boldsymbol{x}, \omega)) \rangle$ to correct the quantized values, ensuring they remain collinear in expectation with the original high-precision vector. However, the paper identifies a critical incompatibility: "the scale correction factor $S$ proposed by EDEN has values in the interval $[0.94, 1.06]$ in practice, requiring a high precision representation for storage." NVFP4's group scales are stored in E4M3 FP8, which has a minimum relative update of $1/16 = 0.0625$—incrementing by roughly $6.25\%$. The EDEN correction factors (up to $\pm 6\%$) are simply too fine-grained to be represented in FP8 group scales. Nor can they be merged into the per-tensor FP32 global scale, because the scaling groups must be subsets of the rotation groups. This is the specific technical obstacle that MS-EDEN solves: how to apply EDEN-style bias correction within the NVFP4 format's coarse scale representation.

Reconciling the Four-Over-Six correctness issue. The paper identifies a notable correctness problem in Cook et al. (2025) that had gone unaddressed. Four-Over-Six evaluates two scale branches (4.0 and 6.0) and picks the one with lower MSE. Even if both branches are individually made unbiased via SR, the act of selecting the lower-MSE branch introduces bias—the selection is deterministic and correlated with the data. The paper validates this empirically in Appendix A, showing that "the application of Four Over Six to the backward pass isn't" unbiased (the error plateaus rather than decreasing as $1/B$ in Figure 9). The paper therefore restricts Four-Over-Six to the forward pass only, where unbiasedness is not required, and this synergy between native group scales and adaptive grid selection is itself presented as a novel finding.

Breaking the representation-requantization trade-off. The paper's key positioning argument is that MS-EDEN fundamentally changes the trade-off calculus. Because MS-EDEN has more than 2× lower quadratic error than SR (Table 1), it can afford to re-quantize weights on the backward pass (breaking the square-block re-use assumption) and still outperform SR without weight re-quantization. This is validated in Figure 1, where panel (e) (MS-EDEN with weight re-quantization) shows a smaller validation loss gap than panel (d) (SR without weight re-quantization). The implication is that Quartet II can use native 1×16 group scaling on the forward pass (better representation capacity) while maintaining superior gradient estimation on the backward pass—achieving both goals simultaneously rather than trading one off against the other.

3. Technical Approach

3.1 Reader Orientation

This paper presents a new computation scheme for training neural networks entirely in 4-bit precision (NVFP4 format) on NVIDIA Blackwell GPUs. The core problem is that existing 4-bit training methods rely on element-wise stochastic rounding to keep gradient estimates unbiased, but this introduces large quantization variance that degrades model quality. The solution reshapes the problem: instead of randomizing each 4-bit value independently, the paper moves the stochasticity to the coarser microscale factors—applying a bias-correction rescaling via stochastic rounding of the 8-bit group scales, while quantizing individual values deterministically to minimize error. This is integrated into a complete linear-layer computation graph (Quartet II) that combines a maximally-expressive forward pass with an unbiased, low-variance backward pass.

3.2 Big-Picture Architecture

Quartet II processes a linear layer $Y = X W^T$ as follows:

1. Forward pass quantization (deterministic, MSE-minimizing): Weights $W$ and activations $X$ are quantized to NVFP4 using round-to-nearest (RTN) with native 1×16 group scales, augmented by the Four-Over-Six grid selection heuristic. The quantized tensors are multiplied in NVFP4 tensor cores to produce $Y$. The quantized $W$ and $X$ are saved for the backward pass.

2. Backward pass quantization (unbiased, through two GEMM operations): The backward pass computes two gradients: the output gradient w.r.t. the input ($\partial L / \partial X$, the "input gradient") and the output gradient w.r.t. the weights ($\partial L / \partial W$, the "weight gradient"). For each, Quartet II de-quantizes the saved forward-pass tensors, transposes as needed, applies randomized Hadamard transforms (RHT) along the inner GEMM dimension, and re-quantizes all operands using MS-EDEN—the novel unbiased quantization primitive. The quantized tensors are multiplied in NVFP4 tensor cores. Because both operands of each GEMM receive the same RHT with the same seed, the rotations cancel along the inner product dimension, requiring no inverse transform.

3. Core primitive (MS-EDEN): Takes a high-precision vector, applies RHT, quantizes to NVFP4 via RTN (low MSE), computes a per-group bias-correction factor $S$, and merges $S$ into the FP8 group scales using stochastic rounding—the only stochastic operation. This guarantees the quantized vector is unbiased in expectation relative to the original, with approximately 2× lower MSE than element-wise SR.

The information flow is: forward pass inputs → deterministic RTN + 4/6 quantization → NVFP4 GEMM → saved quantized weights/activations → backward pass starts → de-quantization → tranposition → RHT on inner dimension → MS-EDEN re-quantization → NVFP4 GEMM for each gradient → output gradients continue backpropagation.

3.3 Roadmap for the Deep Dive

• First, the formal definition of unbiased quantization and why it matters specifically for the backward pass—this establishes the mathematical requirement that MS-EDEN must satisfy.
• Second, the standard NVFP4 format and its stochastic rounding mechanism, so we understand exactly what MS-EDEN replaces and why.
• Third, the EDEN technique from distributed optimization, its bias-correction mechanism, and why it cannot be directly applied to NVFP4—the specific precision bottleneck that motivates the entire design.
• Fourth, MS-EDEN itself: the algorithm, how it resolves the precision bottleneck by merging EDEN correction into group scales via stochastic rounding, and its unbiasedness guarantee.
• Fifth, the practical performance of MS-EDEN: the MSE numbers from Table 1, why moving stochasticity to scales reduces error so dramatically, and the architectural constraints it imposes (rotation groups must subsume scale groups, requiring weight re-quantization on the backward pass).
• Sixth, the forward-pass design: the representation-requantization trade-off, native vs. square-block scales, the Four-Over-Six heuristic, and why Quartet II chooses native scales + 4/6.
• Seventh, the complete Quartet II computation graph: how forward and backward passes connect, which operations are fused, and how the scheme compares analytically to prior approaches (TetraJet-v2, NVIDIA recipe).

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and algorithms paper whose core idea is that unbiased gradient estimation for 4-bit training can be achieved with substantially lower variance by applying stochasticity at the group-scale level rather than at the element level, and that this improvement enables a fully-NVFP4 training recipe that outperforms prior work on both forward-pass representation quality and backward-pass gradient accuracy.

---

Formal Definition of Unbiased Quantization

The paper begins Section 3 by formalizing what it means for a quantization operator to be unbiased, because the entire backward-pass design hinges on this property. Let $Q(\boldsymbol{x}^d, \omega) \to \mathbb{R}^d$ be a (possibly stochastic) quantization mapping, where $\boldsymbol{x}^d \in \mathbb{R}^d$ is the input vector to be quantized and $\omega \in \Omega$ is a random seed drawn from some distribution $\Omega$. The operator is unbiased with respect to $\omega$ if:

$\forall \boldsymbol{x}^d \in \mathbb{R}^d: \mathbb{E}_{\omega}\left[Q(\boldsymbol{x}^d, \omega)\right] = \boldsymbol{x}^d.$

In prose: for any input vector, the expected value of the quantized output over all possible random seeds equals the original unquantized vector. This is a per-element equality: each coordinate's quantized value, averaged over randomness, recovers the exact high-precision value.

The paper states that unbiasedness is "crucial for stable long-term convergence" in LLM pre-training, and cites four prior works (Chmiel et al., 2024; Tseng et al., 2025; Castro et al., 2025; NVIDIA et al., 2025) that independently converged on this principle. Intuitively, the reason is that the backward pass propagates gradient signals through potentially hundreds of layers over millions of training steps. If the quantization introduces a consistent bias—say, systematically under-estimating gradients in certain directions—this bias compounds across layers and over time, pushing the optimizer toward systematically incorrect weight updates. An unbiased estimator, by contrast, has zero expected error: it may be noisy at each individual step, but the noise averages out over mini-batches and training steps, so the long-run trajectory of SGD/Adam is not systematically deflected.

This definition sets up the central tension: unbiasedness is necessary, but the standard mechanism for achieving it (element-wise stochastic rounding) is expensive in variance. The paper's contribution is to show that the same formal guarantee can be achieved with much lower variance by redesigning where the stochasticity is introduced.

---

NVFP4 Format and Standard Stochastic Rounding

Before explaining MS-EDEN, the paper provides the precise definition of the NVFP4 format and the standard stochastic rounding procedure that it replaces. NVFP4 represents tensors using a three-level scale hierarchy:

• FP4 elements ($\boldsymbol{x}^{\text{FP4}} \in \mathbb{R}^d$): Each individual value is stored in the E2M1 floating-point format (2 exponent bits, 1 mantissa bit), which can represent values in the set $\{0, \pm 0.5, \pm 1.0, \pm 1.5, \pm 2.0, \pm 3.0, \pm 4.0, \pm 6.0\}$. This is a very coarse grid—only 16 possible values including sign and zero.
• FP8 group scales ($\boldsymbol{x}^{\text{FP8}} \in \mathbb{R}^{d // 16}$): One E4M3 FP8 scale per block of 16 consecutive elements, providing per-block dynamic range adjustment. FP8 E4M3 can represent values with approximately 7.5% relative precision.
• FP32 global scale ($x^{\text{FP32}} \in \mathbb{R}$): A single high-precision scalar scale for the entire tensor, extending the overall dynamic range.

The dequantized value for element $i$ is $\boldsymbol{x}_i^{\text{FP4}} \times \boldsymbol{x}_{i//16}^{\text{FP8}} \times x^{\text{FP32}}$—the FP4 base value multiplied by its group's FP8 scale and the global FP32 scale.

The standard NVFP4 stochastic rounding procedure $Q_{\text{SR}}$ works in three steps, designed to guarantee that no value is ever clipped (ensuring unbiasedness is possible) and that the FP8 scales can accommodate the maximum possible values:

Step 1: Compute the global FP32 scale. The global scale is set to the absolute maximum element divided by the product of the maximum representable values and a safety factor:

$x^{\text{FP32}} = \max_{i=1 \dots d} |\boldsymbol{x}_i| / (6.0 \times \frac{16}{17} \times 448.0).$

Here $6.0$ is the maximum absolute value representable in FP4 (E2M1), $448.0$ is the maximum absolute value representable in FP8 (E4M3), and $\frac{16}{17}$ is a guard factor ensuring that subsequent RTN rounding of group scales to FP8 cannot push values beyond the representable range. This guard factor accounts for the worst-case rounding scenario: if a computed group-scale value falls exactly halfway between two FP8 levels, RTN rounding up could increase it by up to a factor of $\frac{17}{16}$, potentially causing overflow. Dividing by $\frac{16}{17}$ pre-compensates for this.

Step 2: Compute group FP8 scales. For each block $g$ of 16 elements:

$\boldsymbol{x}^{\text{FP8}}_g = \operatorname{RTN}_{\text{FP8}}\left( \max_{i=16\cdot g \dots 16\cdot g+15} \frac{|\boldsymbol{x}_i|}{x^{\text{FP32}} \times 6.0 \times \frac{16}{17}} \right).$

In prose: for each group of 16 values, compute the maximum absolute value in that group, divide by the global scale and the guard factors, and round to the nearest representable FP8 value. This ensures that after dequantization, every element in the group has magnitude at most $6.0 \times \frac{16}{17}$ times the FP8 scale times the global scale—well within the representable range.

Step 3: Stochastic rounding of FP4 elements. Each element is rounded to FP4 probabilistically:

$\boldsymbol{x}_i^{\text{FP4}} = \operatorname{SR}_{FP4}\left( \frac{\boldsymbol{x}_i}{\boldsymbol{x}^{\text{FP8}}_{i//16} \times x^{\text{FP32}}}, \omega \right).$

Here $\operatorname{SR}_{FP4}$ is the stochastic rounding operator: if the normalized value $v$ falls between two adjacent FP4 levels $a$ and $b$ (with $a < v < b$), it rounds to $b$ with probability $\frac{v - a}{b - a}$ and to $a$ with probability $\frac{b - v}{b - a}$. This ensures $\mathbb{E}[\operatorname{SR}_{FP4}(v)] = v$. Critically, because the scales in steps 1–2 guarantee $v \in [-6.0, 6.0]$ (no clipping), stochastic rounding can be applied to all values without bias from truncated ranges.

The resulting unbiasedness guarantee: Because SR is applied per-element and the scales are deterministic given the data, the expected value of the dequantized representation recovers the original:

$\mathbb{E}_{\omega}\left[\boldsymbol{x}_i^{\text{FP4}} \times \boldsymbol{x}^{\text{FP8}}_{i//16} \times x^{\text{FP32}}\right] = \boldsymbol{x}_i.$

The paper notes that "all existing FP4 training methods" use some variant of this per-element SR mechanism to achieve unbiasedness. The key limitation is variance: each element's quantization is independently randomized, so the total variance is the sum of per-element variances. For the coarse FP4 grid (only 8 positive levels), this per-element variance is substantial—approximately 2.5× larger than the error of deterministic round-to-nearest on normally-distributed data, as shown in Table 1.

The EDEN Technique and Its Precision Bottleneck

The paper draws on EDEN (Vargaftik et al., 2022), a technique from distributed optimization that achieves near-unbiased quantization with much lower error than per-element stochastic rounding. Understanding how EDEN works is essential because MS-EDEN is a direct adaptation that resolves EDEN's incompatibility with NVFP4's coarse scale representation.

EDEN's mechanism. Given an input vector $\boldsymbol{x}$, EDEN applies the following steps:

1. Apply a randomized Hadamard transform (RHT) to $\boldsymbol{x}$, seeded by a random variable $\omega$, producing $\text{RHT}(\boldsymbol{x}, \omega)$. The RHT is an orthogonal linear transformation that "spreads out" the energy of the vector uniformly across all coordinates—a vector that originally had one large outlier and many small values becomes a vector where all entries have roughly similar magnitude. This uniform distribution is much friendlier to quantization.

2. Quantize the rotated vector using any deterministic (non-stochastic) quantizer $Q$—for instance, simple round-to-nearest. This yields a low-MSE quantized representation, but it is biased because RTN creates systematic rounding errors.

3. Compute a scalar bias-correction factor $S$:

$S = \frac{\langle \boldsymbol{x}, \boldsymbol{x} \rangle}{\langle \text{RHT}(\boldsymbol{x}, \omega), Q(\text{RHT}(\boldsymbol{x}, \omega)) \rangle}.$

where $\langle \cdot, \cdot \rangle$ denotes the dot product, and $Q$ is the deterministic quantizer.

4. Scale the quantized values by $S$:

$Q_{\text{EDEN}}(\boldsymbol{x}, \omega) = S \cdot Q(\text{RHT}(\boldsymbol{x}, \omega)).$

What $S$ computes and why: The numerator $\langle \boldsymbol{x}, \boldsymbol{x} \rangle = \|\boldsymbol{x}\|^2$ is the squared L2 norm of the original vector. The denominator is the dot product between the rotated vector and its quantized version. Because the RHT is orthogonal, $\langle \text{RHT}(\boldsymbol{x}), \text{RHT}(\boldsymbol{x}) \rangle = \|\boldsymbol{x}\|^2$ as well. The denominator measures how much of the original rotated vector's energy is preserved in the quantized version—if quantization perfectly preserved the rotated vector, the dot product would equal $\|\boldsymbol{x}\|^2$ and $S = 1$. In practice, quantization loses some information, so the dot product is slightly less than $\|\boldsymbol{x}\|^2$ and $S > 1$, compensating for the energy loss.

The unbiasedness property: EDEN's key theoretical result (their Theorem 2.1) is that as the dimension $d \to \infty$, the inverse-rotated quantized vector becomes unbiased:

$\lim_{d \to \infty} \mathbb{E}_{\omega}\left[\text{RHT}^{-1}\left(Q_{\text{EDEN}}(\boldsymbol{x}, \omega), \omega\right)\right] = \boldsymbol{x}.$

Intuitively, the RHT makes the quantized rotated vector co-linear with the original rotated vector in expectation (by the law of large numbers over the many coordinates), so a single scalar correction $S$ suffices to remove bias. In practice, the paper notes that this sequence "converges fast enough to be unbiased with RHT performed in groups as small as $d = 64$."

Why EDEN cannot be directly applied to NVFP4. The paper identifies a specific, concrete obstacle (Section 3.2, "The Challenge"):

"the scale correction factor $S$ proposed by EDEN has values in the interval $[0.94, 1.06]$ in practice, requiring a high precision representation for storage."

NVFP4's group scales are stored in E4M3 FP8 format. FP8 has approximately 7.5% relative precision—the minimum representable ratio between adjacent values is $\frac{17}{16} \approx 1.0625$. This means an FP8 value can be multiplied by at most $1.0625\times$ or $0.9375\times$ in a single representable step. The EDEN correction factors can be anywhere in $[0.94, 1.06]$, and a factor of, say, $1.03$ cannot be represented precisely in FP8—it would either be rounded to $1.0$ or $1.0625$, introducing additional error.

Furthermore, the correction cannot be absorbed into the per-tensor FP32 global scale because NVFP4 requires that "the scaling groups have to be a subset of the rotation groups." That is, the groups of 16 elements that share an FP8 scale must be fully contained within a single rotation group—you cannot have a scale that spans across two rotation groups and then apply separate corrections. Since the correction factor $S$ is computed per rotation group (not per block of 16), it cannot be applied to the global scale without violating this constraint.

This is the precise technical bottleneck that MS-EDEN solves: how to apply a fine-grained bias correction (with precision finer than FP8's 6.25% granularity) within a format whose scales can only represent coarse increments.

---

MS-EDEN: Microscaling EDEN

MS-EDEN resolves the EDEN precision bottleneck through a single insight: instead of storing the correction factor $S$ precisely, apply it to the FP8 group scales using stochastic rounding. Stochastic rounding of the scales ensures that, in expectation, the correction factor is represented exactly, even though any individual scale can only take FP8-representable values. This preserves the overall unbiasedness guarantee while staying within the NVFP4 format constraints.

The full algorithm is given as Algorithm 1 in the paper. It processes an input vector $\boldsymbol{x}$ in chunks of 128 consecutive elements. I will walk through each step in detail.

Step 1: Randomized Hadamard Transform. The input vector is divided into chunks of 128 elements each. For each chunk $h$ (indexed from 0 to $d/128$):

$\boldsymbol{x}^{\text{RHT}}_{[h:h+128]} = \operatorname{RHT}(\boldsymbol{x}_{[h:h+128]}, \omega_{\text{RHT}}).$

The RHT is a random orthogonal transformation: the Hadamard matrix (a structured orthogonal matrix with $\pm 1$ entries) is multiplied by random diagonal sign flips, producing a fast, hardware-friendly orthogonal transform that "whitens" the vector—spreading energy uniformly across coordinates so that no single coordinate dominates the quantization error. The rotation seed $\omega_{\text{RHT}}$ determines the random sign pattern, and the same seed is used for all chunks within the tensor (so the rotation can be implemented as a single matrix multiplication rather than per-chunk operations—this is a practical optimization, not a mathematical requirement).

Why chunk size 128: The paper notes in Appendix A that this value is chosen to align with the mma.m16n8k16 tensor-core instruction on Blackwell GPUs. It is also large enough for EDEN's asymptotic unbiasedness to hold in practice, but small enough to keep the rotation operations efficient.

Step 2: Deterministic NVFP4 quantization with clipping. The rotated chunks are quantized to NVFP4 using a modified round-to-nearest (RTN) scheme that allows clipping (unlike the standard SR scheme which avoids clipping to maintain unbiasedness). The quantization $Q_{\text{RTN}}(\boldsymbol{x}^{\text{RHT}}, s)$ is parameterized by a clipping factor $s$:

$x^{\text{FP32}} = \max_{i=1 \dots d} |\boldsymbol{x}^{\text{RHT}}_i| / (s \times 256.0),$

$\boldsymbol{x}^{\text{FP8}}_g = \operatorname{RTN}_{\text{FP8}}\left( \max_{i=16\cdot g \dots 16\cdot g+15} \frac{|\boldsymbol{x}^{\text{RHT}}_i|}{x^{\text{FP32}} \times s} \right),$

$\boldsymbol{x}_i^{\text{FP4}} = \operatorname{RTN}_{\text{FP4}}\left( \frac{\boldsymbol{x}^{\text{RHT}}_i}{\boldsymbol{x}^{\text{FP8}}_{i//16} \times x^{\text{FP32}}} \right).$

This differs from the standard SR quantization in several important ways:

• The global scale divisor uses $s \times 256.0$ instead of $6.0 \times \frac{16}{17} \times 448.0$. The value $256.0$ is a conservative cap on FP8 E4M3 scales, chosen because the EDEN correction factor $S$ may sometimes need to increase scales after the initial quantization (the correction factors can be up to $1.06$). Starting with $256.0$ instead of $448.0$ leaves approximately $448.0/256.0 \approx 1.75\times$ headroom for this upward adjustment without overflow.

• The clipping factor $s$ controls how aggressively values are clipped. Setting $s = 6 \times \frac{16}{17}$ (the same as in standard SR) makes the scheme non-clipping, guaranteeing all values fall within the representable range. Smaller values of $s$ allow clipping: values exceeding the representable range are truncated to the nearest representable boundary. The paper uses $s$ as a tunable parameter; in practice, allowing some clipping reduces the MSE of the majority of (in-range) values at the cost of larger error on a small fraction of outliers.

• Rounding uses deterministic RTN rather than stochastic rounding for the FP4 elements. This is possible because unbiasedness will be recovered later through the EDEN correction, not through per-element SR.

Why clipping is now acceptable: In the standard SR approach, clipping destroys unbiasedness because clipped values are systematically underestimated (they are always rounded down to the boundary). EDEN's correction factor $S$ can theoretically compensate for any systematic energy loss—including from clipping—by scaling up the entire quantized representation. However, the paper doesn't explicitly analyze the interaction of clipping with EDEN unbiasedness; the empirical results suggest it's benign when $s$ is chosen appropriately.

The dequantized RTN representation is denoted $\boldsymbol{x}^{\text{RTN}} = \boldsymbol{x}^{\text{FP4}} \times \boldsymbol{x}^{\text{FP8}} \times x^{\text{FP32}}$.

Step 3: Compute EDEN correction factors per rotation group. For each chunk $h$ of 128 elements:

$S_{h//128} = \frac{\langle \boldsymbol{x}^{\text{RHT}}_{[h:h+128]}, \boldsymbol{x}^{\text{RHT}}_{[h:h+128]} \rangle}{\langle \boldsymbol{x}^{\text{RHT}}_{[h:h+128]}, \boldsymbol{x}^{\text{RTN}}_{[h:h+128]} \rangle}.$

This is the EDEN correction factor computed per rotation group: the ratio of the squared L2 norm of the rotated input to the dot product of the rotated input with its quantized version. If quantization preserved all energy, $S = 1$; if energy was lost (the typical case), $S > 1$ (scaling up to compensate); in edge cases with unusual quantization patterns, $S < 1$ is possible.

Step 4: Merge correction into group scales via stochastic rounding. This is the crucial innovation. For each block $g$ of 16 elements (which is fully contained within its parent rotation group $h = 16g // 128$):

$\boldsymbol{x}^{\text{FP8}}_g = \text{SR}_{\text{FP8}}\left( S_{16g // 128} \cdot \boldsymbol{x}^{\text{FP8}}_g, \omega_{\text{SR}} \right).$

In prose: take the existing FP8 group scale (computed in step 2), multiply it by the EDEN correction factor for its parent rotation group, and round the result to FP8 stochastically. The stochastic rounding of the scale ensures:

\mathbb{E}_{\omega_{\text{SR}}}\left[\boldsymbol{x}^{\text{FP8}}_g\right] = S_{16g//128} \cdot \boldsymbol{x}^{\text{FP8}}_g^{\text{(old)}}.

The FP4 elements $\boldsymbol{x}^{\text{FP4}}$ are left unchanged from step 2—their deterministic RTN values remain.

The resulting MS-EDEN quantized representation is the triple $\{\boldsymbol{x}^{\text{FP4}}, \boldsymbol{x}^{\text{FP8}}, x^{\text{FP32}}\}$, where the FP8 scales have been stochastically adjusted and the FP4 values and global scale remain as computed in step 2.

Step 5: Return the NVFP4 representation. The algorithm returns the standard NVFP4 triple. In the backward pass of Quartet II, these quantized tensors are directly multiplied in NVFP4 tensor cores—no further processing is needed because the RHT has been applied on the inner GEMM dimension and the rotations cancel out (see the Quartet II computation graph section below).

Unbiasedness guarantee (Corollary 3.1). The paper formally states:

$\mathbb{E}_{\omega_{\text{RHT}}, \omega_{\text{SR}}} \operatorname{RHT}^{-1}\left(Q_{\text{MS-EDEN}}(\boldsymbol{x}, \omega_{\text{RHT}}, \omega_{\text{SR}}, s), \omega_{\text{RHT}}\right) = \boldsymbol{x}.$

The expectation is taken over both sources of randomness: the RHT seed (which determines the rotation pattern) and the SR seed (which determines how scales are stochastically rounded). The inverse RHT $\operatorname{RHT}^{-1}$ maps the dequantized rotated representation back to the original coordinate system. The paper notes that in practice, the inverse RHT is never explicitly computed—it "naturally cancels out when MS-EDEN is applied on the inner dimension of a matrix multiplication to both tensors with the same rotation seed."

The proof builds on EDEN's Theorem 2.1 (which guarantees unbiasedness for the RHT + EDEN correction in the $d \to \infty$ limit) and the property of stochastic rounding that $\mathbb{E}[\text{SR}_{\text{FP8}}(v)] = v$. Because the EDEN correction is applied in expectation via SR of the scales, the overall estimator retains the EDEN unbiasedness property.

Why the FP4 values use RTN rather than SR: This is the central design choice that gives MS-EDEN its advantage. In standard SR, each of the $d$ FP4 elements is independently randomized, contributing variance proportional to the per-element FP4 grid spacing. In MS-EDEN, only the $d/16$ FP8 group scales are randomized—the FP4 values are deterministic RTN with much lower per-element error. The SR applied to FP8 scales introduces variance that is "an order of magnitude smaller than 4-bit quantization itself" (Section 3.3, "Practical Performance"), because FP8's precision is approximately 7.5% relative, compared to FP4's coarse absolute grid. Trading $d$ randomizations at FP4 precision for $d/16$ randomizations at FP8 precision is what produces the >2× MSE improvement in Table 1.

Compatibility constraints: The reliance on RHT imposes a structural constraint: "Micro-scaling groups have to be sub-divisions of rotation groups." Since rotation groups are 128 elements and micro-scaling groups are 16 elements, this is satisfied (128 is a multiple of 16). However, rotations must be applied along the inner dimension of the multiplied tensors—the dimension that is contracted in the GEMM—so that the rotations cancel. For a GEMM $C_{m,n} = A_{m,k} \times B_{k,n}$, the inner dimension is $k$. This means that in the backward pass, weights must be re-quantized (transposed so that the inner dimension aligns with the rotation), breaking the square-block weight re-use assumption of the NVIDIA recipe. The paper argues this is an acceptable cost given MS-EDEN's superior error.

---

Practical Performance of MS-EDEN

Table 1 provides the core quantitative evidence for MS-EDEN's advantage. The table reports the mean squared error (MSE $\times 10^{-3}$) for various NVFP4 quantization schemes applied to normally-distributed ($\mathcal{N}(0,1)$) random vectors:

Key observations from this table:

• SR vs. RTN: Stochastic rounding with native 1×16 scales has MSE of $23.5 \times 10^{-3}$, approximately 2.6× higher than deterministic RTN at $9.0 \times 10^{-3}$. This is the price of unbiasedness under the standard approach—each element's independent randomization adds substantial variance.

• MS-EDEN vs. SR: MS-EDEN achieves $9.8 \times 10^{-3}$, which is (a) more than 2× lower than SR's $23.5 \times 10^{-3}$, and (b) only marginally higher than deterministic RTN's $9.0 \times 10^{-3}$. In plain terms: MS-EDEN provides unbiasedness at nearly the same error cost as biased RTN, whereas standard SR pays a ~2.6× penalty.

• MS-EDEN vs. RTN: The difference between $9.8$ and $9.0$ represents the residual variance from stochastically rounding the FP8 group scales and the EDEN correction factor. This difference is small (~9% increase over RTN), confirming that scale-level stochasticity introduces minimal additional error.

• Square-block quantization (16×16) is worse: RTN with 16×16 group scales has MSE of $12.4 \times 10^{-3}$, worse than native 1×16 ($9.0 \times 10^{-3}$). This quantifies the forward-pass representation cost of square-block quantization—having only one scale per 256 values instead of per 16 reduces the granularity of per-block range adaptation, increasing error.

• Four-Over-Six helps native scales more than square blocks: Adding 4/6 reduces RTN error from $9.0$ to $7.6$ for native scales (~15% improvement), but has no effect on square-block RTN (remains at $12.4$). This is because square blocks already have coarser scales that can't benefit from the finer grid selection—the scale is shared across 256 elements, so the optimal grid for one sub-block may be suboptimal for another, and 4/6 can't resolve this conflict.

The paper attributes MS-EDEN's advantage to three factors: "(a) per-element stochastic rounding introduces significant variance that is fully avoided in MS-EDEN, (b) the rescaling with $S$ is small for NVFP4 and (c) stochastic rounding for the 8-bit scales introduces variance an order of magnitude smaller than 4-bit quantization itself." Factor (c) is the most subtle: FP8 SR operates on values that are already high-precision (the computed group scales), so the randomization only perturbs them within a ~7.5% relative band. The FP4 values, by contrast, are on a grid where adjacent levels can differ by a factor of 2 or more (e.g., $2.0$ to $3.0$, or $4.0$ to $6.0$). Randomizing which of these coarse levels a value maps to has much larger absolute consequences.

---

Forward Pass Design: The Representation-Requantization Trade-Off

The paper devotes Section 4 to the forward pass design, which is conceptually simpler than the backward pass but involves an important architectural choice.

The trade-off. The NVIDIA NVFP4 recipe (NVIDIA et al., 2025) uses square-block quantization (16×16 groups) for the weight tensor $W$ on the forward pass. The motivation is specific to the backward pass: the forward pass computes $Y = X W^T$, and the backward pass needs to compute the input gradient $\partial L / \partial X = E W$ (where $E$ is the output gradient). If the weight tensor is quantized in square blocks of 16×16, the quantized $Q(W^T)$ can be reused directly in the backward GEMM $Q(E) \cdot Q(W^T)^T$—the transpose of a 16×16 block is still a 16×16 block, and the scale dimensions align correctly. This avoids re-quantizing $W$ on the backward pass, which effectively halves the backward-pass quantization error for this specific GEMM (since $W$ contributes zero additional quantization error in the backward pass, beyond whatever error it absorbs on the forward pass).

However, square-block quantization has a significant forward-pass cost. With 16×16 groups, one FP8 scale governs 256 FP4 values instead of 16 values under native 1×16 scaling. This reduces the granularity of per-block dynamic range adaptation, causing larger quantization error for weights—visible in Figure 2, where "16×16gs" consistently lags "1×16gs" in validation perplexity across model sizes and data-to-parameter ratios. This creates a tension: better forward-pass representation (native scales) vs. better backward-pass gradient estimation (square blocks enabling weight re-use).

Quartet II's resolution. Quartet II chooses native 1×16 group scaling for the forward pass, accepting that weights will need to be re-quantized on the backward pass. The justification is quantitative:

• MS-EDEN has >2× lower MSE than SR (Table 1), so even with weight re-quantization, the backward-pass error is lower than SR without weight re-quantization. This is validated in Figure 1: panel (e) (MS-EDEN with weight re-quantization) shows a smaller validation loss gap than panel (d) (SR without weight re-quantization).

• The forward pass benefits from native scales, which reduces the representation error that propagates through all layers. This forward-pass improvement is visible in Figure 2, where "1×16gs" consistently outperforms "16×16gs."

• MS-EDEN requires re-quantization anyway because the RHT must be applied on the inner GEMM dimension, which requires the tensor to be transposed and its scale groups to be re-aligned with the rotation groups. The square-block weight re-use optimization was thus incompatible with MS-EDEN from the start.

Four-Over-Six ("4/6") for the forward pass. The paper incorporates Cook et al. (2025)'s Four-Over-Six grid selection heuristic, but only on the forward pass (where unbiasedness is not required). The heuristic works as follows:

1. For each block of 16 values, evaluate two candidate quantization grids: one with an effective scale of 4.0 and one with 6.0 (these correspond to different mappings from the normalized values to the FP4 grid points).

2. Quantize the block under both grids and compute the resulting MSE for that block.

3. Select the grid (4.0 or 6.0) that yields lower MSE for that specific block.

4. The chosen grid is encoded implicitly in the FP8 group scale (the scale represents the product of the dynamic-range adjustment and the grid selection).

The paper validates the effectiveness of 4/6 for forward-pass quantization: in Figure 2, adding "4/6" to native 1×16 scales shows a roughly 15% improvement in validation loss gap (similar to the MSE improvement in Table 1). For square-block 16×16 scales, the improvement is smaller (approximately half the relative gain), consistent with the observation that square blocks dilute the per-block grid optimization.

Why 4/6 cannot be used on the backward pass (with SR): The paper identifies a correctness issue that had not been previously addressed. Even if each grid branch (4.0 and 6.0) is individually made unbiased via SR, the selection of which branch to use is deterministic and data-dependent—it is the branch with lower MSE for that specific data. This deterministic selection introduces bias because the estimator is now a function of the data (through the selection) in addition to the SR randomness. Appendix A validates this empirically: Figure 9 shows that when 4/6 is applied to the backward pass, the gradient error plateaus at a non-zero value rather than decreasing as $1/B$ with more averaging, indicating systematic bias. The paper therefore restricts 4/6 to the forward pass (which has no unbiasedness requirement) and uses MS-EDEN for the backward pass.

Synergy of native scales and 4/6. The paper presents this as a novel finding: "4/6 positively synergizes with native NVFP4 scales on the forward pass, showing roughly double the improvement compared to square-block-quantization for LLM pre-training" (Figure 2). The reason is that with native scales, 4/6 optimizes the grid for both weight and activation quantization independently; with square blocks, the weight grid is effectively fixed across the block and only activation quantization benefits from the per-block grid optimization.

---

The Complete Quartet II Computation Graph

Section 5 and Figure 3 describe how the forward and backward pass designs integrate into a complete linear-layer computation scheme for LLM pre-training. The objective is to replace each linear layer $Y = X W^T$ in the transformer with a fully-NVFP4 operation that matches BF16/FP8 training quality.

Forward pass (deterministic).

1. Weights $W$ are quantized using deterministic RTN with native 1×16 group scales, augmented by the 4/6 grid selection heuristic. The quantization uses the standard NVFP4 format: FP4 elements + FP8 E4M3 per-16 group scales + one FP32 global scale for range extension.

2. Activations $X$ are quantized identically: RTN + 4/6 + native 1×16 scales.

3. The quantized $Q(W)$ and $Q(X)$ are multiplied in NVFP4 tensor cores on Blackwell GPUs, producing the output $Y$ in higher precision (the tensor core accumulates products in FP32 before converting to the output format).

4. The quantized weight and activation tensors are saved for use in the backward pass. This is a memory optimization: storing 4-bit quantized tensors rather than the original FP16/BF16 activations reduces activation memory by roughly 4×, which enables larger micro-batch sizes (as the paper demonstrates in Appendix D.2, where FP4 training enables a micro-batch size of 4 vs. 1 for BF16 on a 32GB GPU).

Backward pass (unbiased, with MS-EDEN).

The backward pass computes two gradients:

• Weight gradient $\partial L / \partial W = E^T X$ (GEMM 1): the output gradient $E$ transposed, multiplied by the saved activations $X$.
• Input gradient $\partial L / \partial X = E W^T$ (GEMM 2): the output gradient $E$ multiplied by the transposed weights.

For each GEMM, Quartet II applies the following procedure:

1. De-quantize the saved forward-pass tensors ($W$ and $X$) back to high precision. This is necessary because the tensors need to be transposed and rotated before re-quantization.

2. Transpose the tensors as needed for the GEMM.

3. Generate RHT rotation matrices using pseudo-random seeds. The same seed is used for both operands of each GEMM on their shared inner dimension, so the rotations cancel in the dot products.

4. Re-quantize all operands using MS-EDEN. For each GEMM, this means:

   • For GEMM 1 ($E^T X$): re-quantize $E^T$ and $X$ using MS-EDEN.
   • For GEMM 2 ($E W^T$): re-quantize $E$ and $W^T$ using MS-EDEN.

   The MS-EDEN procedure for each tensor follows Algorithm 1: apply RHT along the inner GEMM dimension, quantize to NVFP4 via RTN (with clipping factor $s$), compute EDEN correction factors per 128-element rotation group, and merge into FP8 group scales via stochastic rounding.

5. Multiply the quantized tensors in NVFP4 tensor cores. Because both operands received the same RHT on their inner dimension, the rotations cancel: $\langle \text{RHT}(a), \text{RHT}(b) \rangle = \langle a, b \rangle$ for any vectors $a, b$, since the RHT is an orthogonal transformation.

6. The product outputs need no further processing—no inverse RHT, no dequantization before the optimizer step. The gradients flow directly into the optimizer (Adam or Muon) and further backpropagation in FP32 (the optimizer states and accumulator are kept in FP32, consistent with standard mixed-precision training).

Comparison to TetraJet-v2 and the NVIDIA recipe. The paper explicitly positions Quartet II relative to its immediate predecessors:

• NVIDIA recipe (NVIDIA et al., 2025): Uses square-block quantization (16×16) for $W$ on the forward pass to enable weight re-use on the backward pass; uses SR for unbiased gradient estimation; applies RHT only on the backward pass for outlier smoothing. Quartet II replaces SR with MS-EDEN, switches to native 1×16 scales on the forward pass (accepting weight re-quantization cost), and adds 4/6 grid selection.

• TetraJet-v2 (Chen et al., 2025b): The paper implements a simplified, GPU-feasible version: NVFP4 quantization with RTN (no square-block-scales) on the forward pass, and SR quantization with RHT on the inner dimension for both GEMMs on the backward pass. The paper omits TetraJet-v2's intermediate FP32 scales and outlier channels due to implementation complexity. Quartet II differs from this simplified TetraJet-v2 in one key respect: the backward pass uses MS-EDEN instead of SR.

Why weight re-quantization is acceptable. The paper argues this empirically (Figure 1): comparing panel (d) (SR without weight re-quantization, the NVIDIA approach) with panel (e) (MS-EDEN with weight re-quantization, the Quartet II approach), MS-EDEN with re-quantization achieves a smaller validation loss gap than SR without re-quantization. The >2× MSE reduction from MS-EDEN more than compensates for the additional error from re-quantizing weights on the backward pass.

Analytic gradient estimation comparison. The paper provides an analytic argument for why Quartet II yields better gradient estimates across all major GEMMs:

• GEMM 1 (weight gradient $E^T X$): In the NVIDIA recipe, only $E^T$ is quantized (since $X$ is kept in high precision or square-block-quantized without re-quantization). In Quartet II, both $E^T$ and $X$ are MS-EDEN quantized, but with >2× lower per-tensor error than SR. The net error depends on the relative contributions, and the paper's Figure 1(a) and 1(b-c) show MS-EDEN's advantage in this GEMM specifically.

• GEMM 2 (input gradient $E W^T$): In the NVIDIA recipe, $W^T$ benefits from square-block re-use (zero additional quantization error), while $E$ is SR-quantized. In Quartet II, both $E$ and $W^T$ are MS-EDEN quantized. Even though $W^T$ now incurs quantization error (where it previously had none), MS-EDEN's lower error on $E$ and the overall lower combined error produces a net improvement—validated in Figure 1(d) vs. 1(e).

The computational graph in Figure 3. The figure illustrates the flow for a single linear layer, with hashed regions indicating operations that can be fused for GPU efficiency:

• Forward pass: $X \to \text{RTN}+4/6 \to Q(X)$, $W \to \text{RTN}+4/6 \to Q(W)$, then NVFP4 GEMM $\to Y$. The quantized $Q(W)$ and $Q(X)$ are saved.

• Backward pass GEMM 1 (weight gradient): Saved $Q(X) \to$ de-quantize $\to$ transpose $\to$ MS-EDEN re-quantize. The output gradient $E \to$ de-quantize (if saved quantized) $\to$ MS-EDEN re-quantize. NVFP4 GEMM $\to \partial L/\partial W$.

• Backward pass GEMM 2 (input gradient): Saved $Q(W) \to$ de-quantize $\to$ transpose $\to$ MS-EDEN re-quantize. The output gradient $E \to$ MS-EDEN re-quantize. NVFP4 GEMM $\to \partial L/\partial X$.

The RHT is applied within the MS-EDEN re-quantization blocks (not shown as a separate step in the figure because it is part of the MS-EDEN primitive). The cancelation of RHT along the inner GEMM dimension means that the GEMM outputs are directly the desired gradients—no inverse RHT is needed.

Fusion considerations. The paper notes (Section 7) that in practice, these operations cannot be fused into a single kernel pass because the global maximum reduction required for NVFP4 quantization acts as a barrier—it needs to see the entire tensor before scales can be computed. This requires a two-kernel implementation: one kernel for the RHT, initial quantization, and EDEN correction factor computation, and a second kernel for the scale correction via stochastic rounding. The paper proposes a "post hoc range alignment" optimization to avoid loading and rotating the tensor twice (discussed in Section 7 of the paper, though not part of the core technical approach description).

Choice of rotation group size (128) and clipping factor. The paper uses rotation groups of 128 elements (exactly 8 NVFP4 scale groups of 16). This is a hardware-driven choice: the mma.m16n8k16 tensor-core instruction on Blackwell GPUs operates on tiles of 16×16×16 for the K dimension, and 128 is a convenient multiple. The clipping factor $s$ is a tunable hyperparameter, but the paper does not specify the exact value used in experiments—this is an implementation detail that may be found in the code release.

Summary of design decisions in Quartet II:

The paper presents Quartet II as an improvement over TetraJet-v2 and the NVIDIA recipe through the single key change of replacing SR with MS-EDEN on the backward pass—the forward-pass design choices (native scales, 4/6) are enabling complements that take advantage of MS-EDEN's ability to handle weight re-quantization without degrading gradient quality.

4. Key Insights and Innovations

Innovation 1: Unbiasedness Need Not Be Paid for with Per-Element Variance—the Stochasticity Can Be Moved to Coarser Structures

The paper's most fundamental conceptual contribution is the demonstration that the unbiasedness requirement in low-precision training does not mandate element-wise stochastic rounding. This overturns an assumption that was implicitly embedded in every prior FP4 training recipe. The field had accepted a clean but expensive trade-off: if you want your gradient estimates to be unbiased (essential for long-run convergence), you must randomize each individual 4-bit value independently, accepting the resulting variance as an inescapable cost of low-precision training. MS-EDEN's core insight is that this decomposition—unbiasedness through per-element randomization—is only one point in a larger design space.

The alternative the paper constructs is to apply stochasticity at the level of the group scales rather than the individual values. Concretely, the FP4 values themselves are quantized deterministically (round-to-nearest, minimizing MSE), while unbiasedness is recovered by stochastically rounding the EDEN bias-correction factor into the FP8 group scales. Because there is one scale per 16 elements rather than one randomization per element, and because FP8's relative precision is an order of magnitude finer than FP4's absolute grid spacing, the variance introduced is dramatically smaller. The quantitative consequence is stark: Table 1 shows MS-EDEN achieves an MSE of $9.8 \times 10^{-3}$ on normally-distributed data, compared to $23.5 \times 10^{-3}$ for element-wise SR—a >2× reduction—while matching the MSE of biased deterministic RTN ($9.0 \times 10^{-3}$) almost exactly.

This is a fundamental architectural insight about quantization granularity, not merely an engineering optimization. Prior work in low-precision training (Castro et al., 2025; Chmiel et al., 2025; NVIDIA et al., 2025; Tseng et al., 2025) had all converged on SR as the default unbiasedness mechanism without questioning whether the stochasticity had to live at the element level. The paper demonstrates that the answer is no: unbiasedness is a property of the composite estimator, and by moving the random perturbation to a coarser, higher-precision component of the representation (the FP8 scales), one can satisfy the same formal guarantee with far less noise. This reframes the problem from "how do we make SR less noisy?" to "where in the quantization hierarchy should stochasticity be applied to minimize variance while preserving unbiasedness?"—a question that did not exist in the literature before this work.

The significance extends beyond the immediate NVFP4 setting. The principle generalizes to any hierarchical quantization format: if a representation decomposes into coarse group-level parameters and fine element-level parameters, unbiasedness can be achieved by randomizing the former deterministically while rendering the latter stochastic-free. This is a design pattern for unbiased quantization that could inform future format and algorithm co-design.

Where the evidence lives: Table 1 (MSE comparison across quantizers) and Corollary 3.1 (formal unbiasedness guarantee). The validation that this theoretical advantage translates to training quality appears in Figure 4, where Quartet II (backed by MS-EDEN) consistently outperforms prior NVFP4 recipes by at least 20% in validation loss gap.

---

Innovation 2: The Representation-Requantization Trade-Off Is False—Better Gradient Estimation Can Coexist with Better Forward-Pass Fidelity

The paper identifies and resolves a structural tension in prior NVFP4 training recipes that the field had treated as an unavoidable trade-off. The NVIDIA recipe (NVIDIA et al., 2025) uses square-block quantization (16×16 groups) for the weight tensor on the forward pass for a specific reason: it enables the quantized weights to be reused without re-quantization on the backward pass for the input gradient computation, effectively halving the quantization error for that GEMM. However, this forward-pass design choice comes at a cost: a single FP8 scale governing 256 values instead of 16, reducing outlier preservation and representation capacity. The field had accepted this as a necessary concession—better backward-pass gradients required sacrificing some forward-pass model expressiveness.

Quartet II demonstrates that this trade-off is an artifact of SR's high variance, not a fundamental constraint. Because MS-EDEN reduces MSE by >2× relative to SR, the paper can afford to re-quantize weights on the backward pass (breaking the square-block reuse optimization) while still achieving better gradient estimation than SR without weight re-quantization. This is validated empirically in Figure 1: panel (e) (MS-EDEN with weight re-quantization) shows a smaller validation loss gap than panel (d) (SR without weight re-quantization). The conclusion is that the correct metric is not "how many tensors are re-quantized" but "what is the total error of the gradient estimator." MS-EDEN's lower per-tensor error more than compensates for the additional quantization step.

This frees the forward pass to use native 1×16 group scales, which provide finer per-block dynamic range adaptation and—as Figure 2 shows—consistently better validation perplexity than square-block quantization across model sizes and data-to-parameter ratios. The paper further enhances forward-pass quality by incorporating Four-Over-Six grid selection, which synergizes particularly well with native scales (approximately double the improvement relative to square blocks). The net result is that Quartet II improves both the forward pass (better representation capacity via native scales + 4/6) and the backward pass (better gradient estimates via MS-EDEN) simultaneously—achieving what prior work treated as mutually exclusive goals.

This is a conceptual reframing with direct practical consequences. It implies that future work on low-precision training should not optimize forward-pass and backward-pass quantization in isolation, but should instead consider the joint optimization: a superior backward-pass quantizer can "pay for" a more aggressive forward-pass design, and vice versa. The paper makes this point analytically by comparing the gradient estimation quality across all major GEMMs, showing that Quartet II's design yields consistently better estimates despite (and in some cases because of) its willingness to re-quantize.

Where the evidence lives: Figure 1 (backward-pass ablation comparing SR with and without weight re-quantization vs. MS-EDEN with re-quantization), Figure 2 (forward-pass validation perplexity for native vs. square-block scales), and the analytic comparison in Section 4.1.

---

Innovation 3: Co-Design with Hardware Constraints as a First-Class Design Principle (Not an Afterthought)

A recurring weakness in the low-precision training literature is that algorithmic innovations often assume idealized hardware capabilities that do not exist. TetraJet-v2 (Chen et al., 2025b) introduces intermediate FP32 scales and selective outlier channels that, as the paper notes, require "substantially more complicated kernel support that is not provided by the authors." The result is a recipe that looks good in emulation but cannot be efficiently executed on real GPUs. Similarly, the original EDEN technique (Vargaftik et al., 2022) assumes arbitrary-precision storage for its correction factors—an assumption that breaks on NVFP4's E4M3 FP8 group scales with their ~6.25% relative granularity.

Quartet II inverts this relationship: the hardware constraints are not obstacles to work around, but design inputs that shape the algorithm from the start. The paper makes explicit, concrete design choices driven by hardware realities:

• Rotation group size of 128 is chosen to align with the mma.m16n8k16 tensor-core instruction on Blackwell GPUs, enabling efficient rotation as a simple GEMM rather than a custom kernel.
• The correction factor precision problem is solved not by demanding more format precision (which would break hardware compatibility) but by exploiting a capability the hardware does provide: stochastic rounding of FP8 values. This turns a format limitation into an algorithmic opportunity—the same mechanism that achieves unbiasedness for the EDEN correction is natively supported by the GPU's FP8 rounding modes.
• Identical rotations across all rotation groups within a tensor are used to enable implementing the RHT as a single matrix multiplication, despite the theoretical EDEN analysis assuming independent rotations per group. Appendix A validates numerically that this compromise does not harm unbiasedness in practice.
• The post hoc range alignment kernel optimization (Section 7) is a hardware-aware implementation heuristic that reduces memory bandwidth by ~20% by decoupling scale computation from value quantization, avoiding redundant tensor loads and rotations.

This is significant because it establishes a methodology for future 4-bit training research: algorithmic proposals should be evaluated not only on emulated accuracy but on their feasibility to implement efficiently on the target hardware. The paper's decision to evaluate only the GPU-feasible subset of TetraJet-v2—omitting its impractical FP32 intermediate scales and outlier channels—is a methodological statement as much as an empirical choice. It implies that prior work that reports accuracy numbers based on full-feature emulation may be overstating what is achievable in practice, and that the field needs a clearer separation between "algorithmic capability" and "deployable system performance."

The kernel benchmarks in Section 7 and Appendix D further demonstrate this co-design philosophy: the paper provides not just accuracy results but latency and throughput measurements on actual Blackwell GPUs (RTX 5090), showing up to 4.2× speedup over BF16 and ~70% improvement over prior FP4 training kernels from Quartet (Castro et al., 2025). The end-to-end training results (Appendix D.2) reveal an additional, hardware-driven benefit: FP4's memory savings enable a 4× larger micro-batch size on a single 32GB GPU, yielding 2.45× training throughput even when individual GEMM speedups are modest.

Where the evidence lives: Appendix A (validation that hardware compromises don't break unbiasedness), Section 7 and Table 2 (kernel design and bandwidth analysis), Figure 6 and Table 7 (speedup measurements and time breakdown), and the explicit discussion of TetraJet-v2's impractical components in Section 2.

---

Innovation 4: Diagnosing and Isolating the Bias Problem in Four-Over-Six

While Four-Over-Six (Cook et al., 2025) is an effective forward-pass heuristic, the paper makes a sharp diagnostic contribution by identifying that the combination of Four-Over-Six with stochastic rounding on the backward pass is not unbiased—the grid selection itself introduces bias, even when both candidate branches are individually unbiased via SR. This is a subtle but important point that had not been recognized in the prior literature.

The mechanism of bias is conceptual, not mechanical: Four-Over-Six evaluates two quantization grids (4.0 and 6.0) for each block, quantizes under both, computes the MSE for each, and selects the grid with lower MSE. Even if SR is applied element-wise under both grids, the act of choosing which grid's quantized values to use is a deterministic function of the data. The estimator is therefore a mixture: with some probability (determined by the data, not by randomness), it uses the 4.0 grid's SR output; with complementary probability, it uses the 6.0 grid's SR output. The expectation over SR randomness does not cancel out the effect of the data-dependent selection, because the selection probability itself depends on the values being estimated. This is distinct from standard SR, where the randomness is independent of the data value (given the value, the rounding probability is determined, but the value itself doesn't influence which randomness is applied).

The paper validates this diagnosis empirically in Appendix A (Figure 9). For unbiased estimators, the relative quadratic error of the averaged quantized gradient should decrease as ~1/B with more averaging (by the Central Limit Theorem). The NVIDIA and TetraJet-v2 methods (pure SR, no 4/6 on backward pass) show this behavior—their error curves slope downward. The NVIDIA+4/6 combination, however, plateaus at a non-zero error, indicating systematic bias that no amount of averaging can remove. Quartet II's MS-EDEN also shows the correct ~1/B scaling, confirming its unbiasedness despite using an entirely different mechanism to achieve it.

This is a negative result with positive implications: it identifies a previously unrecognized correctness issue in a published technique, establishes the diagnostic methodology (the ~1/B scaling test), and informs the design of Quartet II by restricting 4/6 to the forward pass where unbiasedness is not required. It also demonstrates that the combinatorial space of "add SR to each operation individually" is not closed under composition—techniques that are unbiased in isolation can become biased when combined with data-dependent selection or switching. This has implications beyond Four-Over-Six, for any future work that considers adaptive or data-dependent quantization schemes on the backward pass.

The paper's treatment is notable for its precision: rather than making a vague claim that 4/6 "might introduce bias," it provides a concrete mechanistic explanation and an empirical test with a clear expected behavior under the null hypothesis of unbiasedness. This sets a methodological standard for how the field should validate claims of unbiasedness in composite quantization schemes.

Where the evidence lives: Appendix A, Figure 9 (concentration of quantized backward average toward unquantized backward for different methods), and the discussion in Section 4.2 connecting the theoretical issue to the empirical validation.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary pre-training corpus is the C4 dataset (Dodge et al., 2021), a cleaned version of Common Crawl web text. For larger-scale validation, the paper additionally uses FineWeb-Edu (Lozhkov et al., 2024), a curated educational web-text corpus, as part of the Nanochat training pipeline (Section 6.2). All experiments train on language modeling loss—standard next-token prediction cross-entropy.

• Base model(s). The paper trains Llama-2–style transformer models (Touvron et al., 2023) at four scales: 30M, 50M, 100M, and 200M parameters, with data-to-parameter ratios ranging from 25 to 800 tokens per parameter (spanning compute-optimal to heavily over-trained regimes). These are not pre-trained starting points but architectures trained from scratch for each QAT configuration, with all hyperparameters held constant between BF16 baselines and quantized runs. The larger-scale Nanochat experiments use models of 560M parameters (11B tokens) and 1.9B parameters (38B tokens) (Section 6.2), trained with the Muon optimizer and WSD learning rate schedule rather than Adam with cosine decay.

• Metrics. The primary metric is validation loss on held-out C4 data, reported as validation loss gap: the absolute increase in loss relative to a BF16 baseline trained under identical hyperparameters. For Nanochat, the metric is bits-per-byte (BPB) increase relative to BF16 pre-training (Figure 5). Downstream evaluation (Appendix C, Table 5) reports zero-shot accuracy on ARC-Challenge, ARC-Easy, GSM8K, HumanEval, and MMLU after supervised fine-tuning, with confidence intervals (2 standard deviations) reported for the smaller baseline model. The paper also reports MSE ×10^-3 for synthetic quantization error measurements on N(0,1) data (Table 1).

• Baselines. The paper compares against four prior NVFP4 training recipes:

  • NVIDIA (NVIDIA et al., 2025): The first end-to-end NVFP4 pre-training recipe. Square-block quantization (16×16 groups) for weights on the forward pass, SR on the backward pass with RHT for outlier smoothing, and some layers kept in higher precision. The paper reimplements this as described.
  • FourOverSix (Cook et al., 2025): The NVIDIA recipe augmented with the 4/6 grid selection heuristic on the forward pass. The paper applies 4/6 to the forward pass only (not backward, due to the bias issue discussed in Section 4.2).
  • TetraJet-v2 (Chen et al., 2025b): The paper implements a simplified, GPU-feasible version: NVFP4 quantization with RTN on the forward pass (no square-block-scales), and SR quantization with RHT on the inner dimension for both GEMMs on the backward pass. The intermediate FP32 scales and selective outlier channels from the original TetraJet-v2 proposal are omitted because they "require substantially more complicated kernel support that is not provided by the authors" (Section 2).
  • BF16: Full-precision training serving as the accuracy upper bound.

  For the backward-pass ablation (Section 6.1, Figure 1), the baselines are SR-based quantization with and without weight re-quantization, applied to various combinations of the two backward-pass GEMMs.

• Generation budget / compute accounting. All experiments are training throughput and accuracy comparisons, not inference-time compute allocation studies. The key resource is total training FLOPs, dictated by the model size and token count. For kernel benchmarks (Section 7, Figure 6), the metric is speedup over BF16 for individual linear layer operations (forward + backward pass), reported as a multiplier. End-to-end throughput (Appendix D.2) reports thousands of tokens per second (ktok/s) on a single NVIDIA RTX 5090 GPU with 32GB memory. The paper also reports Model FLOPs Utilization (MFU) for the BF16 baseline (68% at 1.1B parameters).

• Cross-validation / statistical protocol. No cross-validation is used for pre-training experiments—each QAT configuration is a single training run to completion, and the validation loss gap is reported as a point estimate. For the downstream benchmarks in Nanochat (Appendix C, Table 5), confidence intervals (±2 standard deviations) are reported for the smaller 560M parameter baseline model to indicate whether differences between QAT methods are statistically significant (the paper notes that "the vast majority of the differences between FP4 QAT methods are not statistically significant" on downstream tasks). For the backward-pass unbiasedness verification (Appendix A, Figure 9), the paper averages quantized gradients over B independent samples and checks whether the relative quadratic error decreases as ~1/B, which is the expected behavior for an unbiased estimator under the Central Limit Theorem.

---

Main Quantitative Results

MS-EDEN vs. SR: Quantization Error on Synthetic Data

Headline: MS-EDEN reduces MSE by more than 2× relative to SR while maintaining unbiasedness, and nearly matches biased RTN. Table 1 reports the mean squared error of various NVFP4 quantization schemes applied to normally-distributed (N(0,1)) random vectors:

• SR (native 1×16 scales): MSE = 23.5 × 10^-3 (unbiased)
• MS-EDEN (native 1×16 scales): MSE = 9.8 × 10^-3 (unbiased)
• RTN (native 1×16 scales): MSE = 9.0 × 10^-3 (biased)
• RTN + 4/6 (native 1×16): MSE = 7.6 × 10^-3 (biased)
• SR + 4/6 (native 1×16): MSE = 17.5 × 10^-3 (the paper argues this is biased for the backward pass)

The key comparison: MS-EDEN's MSE is only 9% higher than biased RTN (9.8 vs. 9.0), while SR's is 2.6× higher (23.5 vs. 9.0). This is the quantitative foundation for the paper's claim that MS-EDEN "reduces quantization error by moving the stochasticity from individual FP4 values to the microscale factors."

Square-block quantization (16×16 groups) increases RTN error to 12.4 × 10^-3, confirming that coarser group scaling degrades representation quality. Four-Over-Six reduces native-scale RTN error from 9.0 to 7.6 (a ~15% improvement) but has no effect on square-block RTN (remains at 12.4), indicating that the per-block grid optimization is diluted when a single scale governs 256 values.

---

Backward Pass Ablation: Which Tensors Benefit from MS-EDEN?

Headline: MS-EDEN consistently outperforms SR across all backward-pass GEMM configurations, and MS-EDEN with weight re-quantization beats SR without weight re-quantization. Figure 1 presents a selective quantization ablation on Llama-2–style models, where specific tensors in the two backward-pass GEMMs are quantized while others are kept in high precision. The figure reports the C4 validation loss gap relative to BF16 pre-training, averaged over model sizes (30M–200M) and data-to-parameter ratios (25–800).

The five tested configurations (panels a–e) are:

(a) Weight gradient GEMM only (E·W^T, Q(E^T)·Q(X^T)^T): Only the GEMM computing dL/dW is quantized. SR and MS-EDEN show similar performance here, since only one of the two GEMMs is affected and the error contribution from weight gradients tends to be smaller than from input gradients (weight gradients are averaged over the batch dimension, reducing variance).

(b) Input gradient GEMM without weight re-quantization (Q(E)·W, E^T·X): Only the input gradient GEMM is quantized, with the weight tensor W kept in high precision (simulating the square-block weight re-use benefit). MS-EDEN is not tested here because it requires weight re-quantization—this configuration is SR-only.

(c) Input gradient GEMM with weight re-quantization (Q(E)·Q(W^T)^T, E^T·X): Both operands of the input gradient GEMM are quantized, but the weight gradient GEMM remains in high precision. MS-EDEN outperforms SR: the validation loss gap for SR is visibly larger than for MS-EDEN at equivalent model scales and data ratios.

(d) Both GEMMs without weight re-quantization (Q(E)·W, Q(E^T)·Q(X^T)^T): Full backward-pass quantization, but the weight tensor W in the input gradient GEMM is not re-quantized (mimicking the NVIDIA recipe's square-block reuse). This is an SR-only configuration; MS-EDEN cannot be evaluated here for the same reason as (b).

(e) Both GEMMs with weight re-quantization (Q(E)·Q(W^T)^T, Q(E^T)·Q(X^T)^T): Full backward-pass quantization with all tensors re-quantized—this is the Quartet II configuration for MS-EDEN, and a "worst-case" configuration for SR (since it incurs weight re-quantization error without the benefits of MS-EDEN's lower variance). The paper's key finding from this figure is that MS-EDEN in panel (e) achieves a smaller validation loss gap than SR in panel (d). In other words, even though Quartet II re-quantizes weights (adding quantization error where the NVIDIA recipe avoids it), MS-EDEN's >2× lower per-tensor error more than compensates, yielding better overall gradient estimates. This is the empirical basis for the claim that MS-EDEN breaks the representation-requantization trade-off.

The paper states this result explicitly in Section 4.1: "this can be seen by comparing SR without weight re-quantization in Figure 1(d) with MS-EDEN with weight re-quantization in Figure 1(e)."

---

Forward Pass Ablation: Native Scales + 4/6 Outperforms Square Blocks

Headline: Native 1×16 group scales consistently outperform square-block 16×16 scales on the forward pass, and Four-Over-Six provides approximately double the improvement when combined with native scales. Figure 2 reports the C4 validation loss gap for forward-pass-only quantization (backward pass kept in high precision), comparing four configurations across model sizes and data-to-parameter ratios:

• 16×16 group scales (square blocks): Consistently the worst-performing configuration, with the largest validation loss gap relative to BF16.
• 16×16gs + 4/6: A modest improvement over square blocks alone—the gap narrows slightly, but remains substantially larger than native-scale configurations.
• 1×16 group scales (native): Outperforms both square-block configurations. The gap to BF16 is notably smaller, confirming that finer per-block dynamic range adaptation improves forward-pass representation quality.
• 1×16gs + 4/6: The best-performing configuration. The improvement from adding 4/6 to native scales is visibly larger (the gap narrows substantially) than the improvement from adding 4/6 to square blocks (where the gap narrows only marginally).

The paper quantifies this as "roughly double the improvement" for native scales over square blocks when 4/6 is added. This result validates Quartet II's choice of native 1×16 scales + 4/6 for the forward pass—it maximizes representation capacity while the backward pass handles gradient estimation quality independently through MS-EDEN.

The paper also notes that with square blocks, 4/6 effectively optimizes only the activation quantization grid (since weight scales are shared across 16×16 blocks), whereas with native scales, both weight and activation quantization benefit independently from per-block grid optimization. This explains the synergy quantitatively: native scales give 4/6 twice as many degrees of freedom to optimize.

---

Full Quantization: Quartet II vs. Prior NVFP4 Recipes

Headline: Quartet II reduces the validation loss gap relative to BF16 by at least 20% compared to all prior NVFP4 recipes across model sizes and data-to-parameter ratios. Figure 4 presents the C4 validation loss gap for fully-quantized training (forward pass + backward pass) comparing four methods:

• NVIDIA (NVIDIA et al., 2025): Square-block weights + SR on backward pass
• FourOverSix (Cook et al., 2025): NVIDIA recipe + 4/6 on forward pass
• TetraJet-v2 (simplified, GPU-feasible version as described in Section 2): RTN on forward pass + SR with RHT on backward pass
• Quartet II: Native scales + 4/6 on forward pass + MS-EDEN on backward pass

The figure shows validation loss gap as a function of data-to-parameter ratio (D/N), with separate panels for each model size (30M–200M). The key observations:

• Quartet II consistently produces the smallest gap across all model sizes and all data ratios. The improvement over the next-best method (typically TetraJet-v2 or FourOverSix) is visibly consistent—the Quartet II curve sits below the others in every panel.

• The improvement is at least 20% relative to prior methods. For example, at the 200M parameter scale with D/N = 200, the NVIDIA recipe shows a gap of roughly 0.04–0.05 in validation loss, while Quartet II shows a gap of roughly 0.03—a ~25–40% reduction. The paper's text states the improvement is "by at least 20% in terms of loss."

• TetraJet-v2 and FourOverSix perform similarly, both improving over the base NVIDIA recipe but leaving a substantial gap to Quartet II. This confirms that the dominant factor is MS-EDEN on the backward pass—the forward-pass improvements (4/6, native scales) are shared between Quartet II and TetraJet-v2 (which also uses RTN with native scales on the forward pass, as simplified by the paper), so the gap between them is attributable primarily to MS-EDEN vs. SR on the backward pass.

• The gap is robust to data volume. Quartet II maintains its advantage across the full range of D/N ratios (25 to 800), from compute-optimal to heavily over-trained regimes. This suggests MS-EDEN's benefit is not limited to a particular phase of training and does not diminish as models see more data.

• Absolute gap sizes vary with model scale. At 30M parameters, the validation loss gap for all methods is larger (roughly 0.06–0.10) than at 200M parameters (roughly 0.02–0.06), consistent with the intuition that larger models are more robust to quantization noise. Quartet II's relative improvement is maintained across scales.

---

Nanochat Pre-Training at Larger Scale

Headline: Quartet II reduces the pre-training BPB increase over BF16 by 15–25% relative to existing NVFP4 methods at 560M and 1.9B parameter scales on higher-quality data, but downstream zero-shot benchmarks show no statistically significant differences between QAT methods after short fine-tuning. Section 6.2 and Figure 5 report results on the Nanochat training pipeline, which differs from the C4 ablations in using the Muon optimizer, WSD learning rate schedule, QK-normalization, ReLU² MLP activations, and FineWeb-Edu data (20 tokens per parameter).

Pre-training loss (Figure 5): The figure shows validation bits-per-byte (BPB) increase relative to BF16 pre-training over the course of training (tokens processed). Multiple QAT methods are plotted:

• Quartet II (blue curve) consistently shows the smallest BPB increase, staying closest to the BF16 baseline throughout training.
• The gap between Quartet II and the next-best method (TetraJet-v2 or FourOverSix, depending on the phase) is visible throughout the training trajectory, not just at the final checkpoint.

Table 5 in Appendix C provides the final BPB values for two training scales:

• 560M parameters, 11B tokens ("Speedrun"):

  • BF16 baseline: 0.7693 BPB
  • NVIDIA: 0.7814 (+1.57%)
  • FourOverSix: 0.7810 (+1.52%)
  • TetraJet-v2: 0.7813 (+1.56%)
  • Quartet II: 0.7787 (+1.22%)

  Quartet II's BPB increase (1.22%) is approximately 22% lower than the next-best method (FourOverSix at 1.52%).

• 1.9B parameters, 38B tokens ("$1000"):

  • BF16 baseline: 0.6925 BPB
  • NVIDIA: 0.7058 (+1.92%)
  • FourOverSix: 0.7047 (+1.76%)
  • TetraJet-v2: 0.7044 (+1.72%)
  • Quartet II: 0.7025 (+1.44%)

  Quartet II's BPB increase (1.44%) is approximately 16% lower than the next-best method (TetraJet-v2 at 1.72%).

The paper notes that this corresponds to a "15–25% relative reduction" in the pre-training loss gap with BF16. The exact percentage depends on which baseline is used for comparison (the range 15–25% covers the improvement relative to both the worst and best prior methods).

Downstream benchmarks (Table 5, Appendix C): After mid-training and SFT (with backward-pass quantization disabled for these short phases), the paper evaluates on ARC-Challenge, ARC-Easy, GSM8K, HumanEval, and MMLU:

• At the 560M scale, Quartet II achieves: ArcC 36.8%, ArcE 53.5%, GSM8K 7.7%, HumanEval 1.2%, MMLU 35.4%. The BF16 baseline achieves: ArcC 38.8%, ArcE 57.4%, GSM8K 11.4%, HumanEval 3.1%, MMLU 35.5%.
• At the 1.9B scale, Quartet II achieves: ArcC 52.1%, ArcE 71.5%, GSM8K 16.9%, HumanEval 5.5%, MMLU 41.6%. The BF16 baseline achieves: ArcC 60.3%, ArcE 78.7%, GSM8K 21.8%, HumanEval 10.4%, MMLU 44.9%.

The paper explicitly states: "the vast majority of the differences between FP4 QAT methods are not statistically significant" on downstream benchmarks. For the 560M baseline, the reported confidence intervals (2 standard deviations) are, for example, ArcC 38.8 ± 2.8%—meaning Quartet II's 36.8% falls outside the BF16 confidence interval, but differences between QAT methods (e.g., TetraJet-v2 at 36.8% vs. Quartet II at 36.8% for ArcC at 560M) are within sampling noise.

Training stability (Figure 5): The paper notes that "loss spikes are observed for both BF16 and QAT around 6T tokens but training stabilizes later." This indicates that the quantization is not introducing additional training instability beyond what is present in the BF16 baseline—a relevant robustness check for 4-bit training, which has historically been sensitive to divergence.

---

Kernel Speedups on NVIDIA Blackwell Hardware

Headline: Quartet II achieves up to 4.2× speedup over BF16 for linear layer training operations, improving by ~70% over prior FP4 training kernels, and enables 2.45× end-to-end training throughput via larger micro-batch sizes. Section 7 and Appendix D provide GPU kernel benchmarks on an NVIDIA RTX 5090 (Blackwell architecture).

Linear layer speedups (Figure 6): The figure shows speedup over BF16 for training layers characteristic of different model sizes (800M, 3B, 7B, 22B parameters). For each size, the speedup is computed as the latency reduction for a single forward pass + backward pass through the attention and MLP linear projections (shapes specified in Table 6, Appendix D):

• Quartet II achieves approximately 4.2× speedup over BF16 at the largest tested shapes (22B-parameter scale), and approximately 3.5–4.0× at smaller scales.
• Quartet II improves over the prior Quartet FP4 training kernels (Castro et al., 2025) by approximately 70% at comparable shapes.
• The paper attributes this improvement to the fused MS-EDEN re-quantization kernel design (post hoc range alignment) which reduces memory bandwidth by ~20% (Table 2) compared to a naïve implementation.

End-to-end throughput (Table 7, Appendix D.2): The paper reports full model training benchmarks on a single RTX 5090 with a 1.1B-parameter Nanochat model (32GB GPU memory):

• The BF16 baseline achieves 20.8 ktok/s at micro-batch size 1 (MFU of 68%), limited by GPU memory.
• Direct FP4 training at micro-batch size 1 does not yield speedup because the matrix dimensions are too small to overcome the overhead of additional quantization operations.
• However, FP4's memory savings (storing only 4-bit preactivations instead of BF16) enable increasing the micro-batch size to 4, yielding 51 ktok/s—a 2.45× throughput increase over the BF16 baseline at micro-batch size 1.

Table 7 provides a time breakdown of the 1.1B parameter model training at 4096 tokens per pass:

• NVFP4 GEMMs consume 43% of total time (24% forward + 19% backward).
• Non-GEMM operations (attention, LM-head, RMSNorm, ReLU², loss) consume approximately 60% of total time, indicating that at this small model scale, the GEMM speedup is partially bottlenecked by operations untouched by FP4 quantization. The paper notes that "this ratio is expected to drastically decrease as model size grows, increasing the usefulness of FP4."

Post hoc range alignment kernel design (Figures 7–8, Table 2): The paper details two kernel implementations for MS-EDEN re-quantization:

• Naïve implementation (Figure 7): Load the original tensor, perform RHT, compute scales with a global AbsMax barrier, apply EDEN correction, stochastic-round the scales—all in separate passes. This requires loading the tensor twice and performing RHT twice, effectively doubling the memory bandwidth and GEMM instruction count.

• Post hoc range alignment (Figure 8): In the first kernel, quantize without global range alignment—compute E8M3 pseudo-scales (extended-range proxy for FP8 stored in BF16) and FP4 values, and compute the global AbsMax and EDEN correction factors in the same pass. In the second kernel, load only the pseudo-scales (not the FP4 values), shift them into the FP8-representable region using the pre-computed AbsMax, apply EDEN correction, and round to FP8 with stochastic rounding. Since the second kernel operates only on scales (1/16 of the data), it is "more than 10× less" expensive in latency than the first kernel.

Table 2 quantifies the bandwidth savings: the naïve approach moves 4.5 + 4.5 = 9.0 bits per element from global memory to shared memory (loading and storing the full tensor twice for the first kernel, plus loading it once for the second), while the post hoc approach moves 4.5 + 1 = 5.5 bits per element for the first kernel and 5 + 0.5 = 5.5 bits per element for the second—a total bandwidth reduction. Additionally, the post hoc approach reduces GEMM instruction calls per NVFP4 group from 2 mma.m16n8k16 operations to 1, since the RHT is performed only once.

---

Unbiasedness Verification

Headline: Quartet II's MS-EDEN produces effectively unbiased gradient estimates in practice, while NVIDIA+4/6 on the backward pass introduces systematic bias. Appendix A and Figure 9 present a numerical verification of unbiasedness using a pre-trained Llama-3.2-1B model (not a model trained with QAT, to avoid confounding from QAT dynamics). The procedure:

1. Perform B repeated quantized backward passes over a batch of sample data using a given quantization scheme, each with different random seeds.
2. Compute the average quantized gradient (1/B) Σ Ĝ(ω) and compare it to the reference unquantized gradient G via relative quadratic error.
3. If the quantized gradient estimator is unbiased (E[Ĝ(ω)] = G), the error should decrease as ~1/B (by the Central Limit Theorem). If biased, the error will plateau at a non-zero value.

Figure 9 shows the error as a function of 1/B for several methods, tested on attention block 0 (the deepest block from the backpropagation perspective, where gradient errors compound the most):

• Quartet II (MS-EDEN): The error decreases roughly linearly with 1/B, confirming unbiasedness. The paper notes this holds despite three practical compromises: using rotation groups of only d = 128 (not d → ∞), applying identical rotations across groups within a tensor (not independent per group), and not stochastically rounding underflowing FP8 values (which can only affect scales ~32,000× smaller than the largest scale, making the effect negligible).

• NVIDIA (SR-based) and TetraJet-v2 (SR-based): Both also show decreasing error with 1/B, confirming their unbiasedness.

• NVIDIA+4/6: The error plateaus at a non-zero value regardless of B, confirming systematic bias. The paper attributes this to the data-dependent grid selection in Four-Over-Six: even though each grid branch (4.0, 6.0) is individually unbiased via SR, the selection of the lower-MSE branch is deterministic given the data, making the composite estimator biased.

This figure provides the empirical validation for the paper's methodological choice to restrict Four-Over-Six to the forward pass and for its claim that MS-EDEN achieves unbiasedness in practice despite the hardware-driven compromises from the idealized EDEN setting.

---

Ablation Studies and Robustness Checks

Square-block vs. native group scales on the forward pass (Figure 2): Native 1×16 scales consistently outperform square-block 16×16 scales across all model sizes (30M–200M) and data-to-parameter ratios (25–800). The improvement is visible as a substantially smaller validation loss gap relative to BF16. Adding Four-Over-Six improves both configurations, but the improvement is roughly twice as large for native scales—the paper attributes this to 4/6 optimizing both weight and activation grids independently with native scales, but only the activation grid with square blocks (since the weight grid is shared across the 16×16 block). This ablation validates Quartet II's forward-pass design choice.

Backward pass GEMM-by-GEMM ablation (Figure 1, panels a–e): Quantizing only the weight gradient GEMM (panel a) has minimal impact on validation loss, regardless of quantizer. Quantizing the input gradient GEMM (panels b–c) has a larger impact, consistent with the input gradient being more sensitive to quantization error (it propagates through subsequent layers). MS-EDEN outperforms SR in every configuration where both can be compared, and the gap is most pronounced when both GEMMs are fully quantized with weight re-quantization (panel e vs. panel d, where SR is tested without re-quantization). This ablation isolates the contribution of MS-EDEN to each backward-pass operation.

Model scale ablation (Figures 1, 2, 4): Results are reported separately for 30M, 50M, 100M, and 200M parameter models. The absolute validation loss gap shrinks with model scale (larger models are more robust to quantization noise), but Quartet II's relative advantage over baselines is maintained across all scales. This suggests the benefit of MS-EDEN is not limited to small models where quantization error is more severe.

Data-to-parameter ratio ablation (Figures 1, 2, 4): Results span ratios from 25 (approximately compute-optimal) to 800 (heavily over-trained). Quartet II maintains its advantage across the full range, indicating that MS-EDEN's improved gradient estimation benefits both the early "learning" phase and the later "compression" phase of training. There is no evidence that the benefit saturates or diminishes with more data.

Optimizer and architecture robustness (Nanochat, Section 6.2): The C4 ablations use Adam with cosine LR schedule and standard Llama-2 architecture. The Nanochat experiments use Muon optimizer with WSD schedule, QK-normalization, and ReLU² activations—a substantially different training setup. Quartet II's advantage persists (15–25% reduction in BPB gap vs. BF16), suggesting MS-EDEN's benefit is not specific to a particular optimizer or architectural variant.

Training stability (Figure 5): Loss spikes are observed around 6T tokens for both BF16 and QAT runs, and training stabilizes afterward. This indicates that the quantization scheme does not introduce additional instability—an important robustness check, since FP4 training has historically been prone to divergence (the paper's own introduction notes that "naive quantization leads to divergence over long pre-training runs").

Four-Over-Six on backward pass bias (Figure 9): The plateauing error for NVIDIA+4/6 validates the paper's claim that the combination is biased, and justifies restricting 4/6 to the forward pass. This is both an ablation (testing whether 4/6 could be used on the backward pass) and a diagnostic contribution.

Downstream task evaluation (Table 5, Appendix C): After mid-training and SFT, differences between QAT methods on zero-shot benchmarks (ARC, GSM8K, HumanEval, MMLU) are not statistically significant for the vast majority of comparisons. The paper reports confidence intervals for the 560M baseline and notes that the differences between QAT methods fall within sampling noise. This is presented as a neutral finding: the pre-training improvement from Quartet II does not reliably translate to downstream task improvements after short fine-tuning phases.

Pre-trained model unbiasedness verification (Figure 9): Using a pre-trained Llama-3.2-1B model (rather than a QAT-trained model) avoids confounding from the fact that QAT models adapt their weights to the quantizer's error characteristics during training. This is a robustness check that the unbiasedness property is inherent to the quantizer, not an artifact of the training process compensating for bias.

Post-training backward-pass precision (Appendix C): For the short mid-training and SFT phases in Nanochat, the paper disables backward-pass quantization entirely ("to get the most out of these very short and data-limited phases"). This is a practical choice that acknowledges the cumulative-benefit argument for unbiased gradient estimation: with very few training steps, the variance introduced by quantization may not be adequately averaged out, so it is safer to use full precision.

---

Critical Assessment

Do the experiments support the claim of >2× lower MSE with preserved unbiasedness?

Yes, on synthetic data; partially validated in training. Table 1 provides clean, controlled evidence that MS-EDEN reduces MSE from $23.5 \times 10^{-3}$ (SR) to $9.8 \times 10^{-3}$ on N(0,1) data while maintaining unbiasedness. The unbiasedness is verified numerically in Figure 9, which shows the correct ~1/B error scaling for MS-EDEN. However, the >2× figure is specific to the N(0,1) distribution—the paper does not report MSE on real gradient distributions from LLM training, which may have different outlier characteristics, sparsity patterns, or correlation structures. The training loss improvements (Figures 1, 4, 5) validate that the synthetic-data advantage translates to real training, but they do not isolate MSE as the causal mechanism—other factors (interaction with optimizer dynamics, effect of RHT on gradient structure, etc.) could contribute to the observed improvements.

Do the experiments support the claim of "at least 20% improvement in validation loss gap"?

Yes, but with important caveats about the baseline definition. Figure 4 clearly shows Quartet II outperforming all prior methods across all model sizes and data ratios. The 20% figure is a lower bound on the relative improvement (e.g., at 200M parameters with D/N = 200, the gap reduction from ~0.04 to ~0.03 represents a 25% improvement). However, the comparison is against a specific set of baselines: the simplified (GPU-feasible) TetraJet-v2 and the reimplemented NVIDIA recipe. The paper does not compare against the full TetraJet-v2 with intermediate FP32 scales and outlier channels—a method that was reported by its authors (Chen et al., 2025b) to achieve better accuracy than the simplified version tested here. If the full TetraJet-v2 closes part of the gap to BF16 that the simplified version leaves open, the 20% improvement figure might be an overestimate relative to the true state of the art. The paper's methodological argument—that unimplementable features shouldn't count—is reasonable, but it means the quantitative improvement claim is conditional on the GPU-feasible baseline definition.

Do the experiments support the claim that Quartet II breaks the representation-requantization trade-off?

Partially. Figure 1(e) vs. 1(d) is the critical comparison: MS-EDEN with weight re-quantization achieves better validation loss than SR without weight re-quantization. This is a direct empirical demonstration that better gradient estimation can compensate for additional quantization steps. However, the comparison is asymmetric—it pits the best MS-EDEN configuration against a suboptimal SR configuration (SR would also benefit from re-quantizing weights if MS-EDEN weren't available, but the comparison in panel (d) specifically tests the NVIDIA-style square-block reuse approach). A more complete demonstration would show MS-EDEN with re-quantization vs. SR with re-quantization at equal budgets—panel (e) includes this comparison implicitly (both methods are tested there), but the paper's headline claim about the trade-off relies on the cross-panel comparison (e) vs. (d), not on the within-panel SR comparison in (e).

Do the experiments support the claim of up to 4.2× speedup over BF16?

Yes, for individual linear layers; end-to-end speedup is more nuanced. Figure 6 demonstrates 4.2× speedup over BF16 for linear layer training operations at larger tensor shapes (22B-parameter scale). This is a clean kernel-level measurement. However, the end-to-end results (Appendix D.2) reveal a more complex picture: at 1.1B parameters, direct layer-by-layer speedup is negligible because the matrix dimensions are too small, and the actual 2.45× throughput gain comes from increased micro-batch size (4 vs. 1) enabled by FP4's memory savings, not from faster GEMM execution. The paper acknowledges this and notes that the fraction of time spent in non-GEMM operations (~60% at 1.1B) "is expected to drastically decrease as model size grows." There is no end-to-end speedup measurement at larger scales where GEMMs dominate—the 4.2× figure for individual layers at 22B-scale shapes is suggestive but not directly validated in full training throughput.

Do the downstream benchmark results undermine the practical significance of Quartet II?

To some degree, though this may be an artifact of short fine-tuning. Table 5 shows that after mid-training and SFT, the differences between QAT methods on zero-shot benchmarks are not statistically significant. Quartet II's 15–25% pre-training loss improvement does not translate into measurable downstream accuracy gains in this experimental setup. This could mean: (a) the downstream tasks are too noisy to detect the improvement (test sets for ARC, GSM8K, HumanEval are small—e.g., HumanEval has only 164 problems); (b) the short fine-tuning phases (which use full-precision backward passes) wash out differences in pre-training quality; or (c) the pre-training perplexity improvement is genuine but does not affect the specific capabilities tested by these benchmarks. The paper does not explore these alternatives, and the downstream results are presented as a neutral observation rather than a negative result. For practitioners, the implications depend on the use case: if the goal is lower perplexity on the training distribution, Quartet II delivers; if the goal is better zero-shot performance after fine-tuning, the evidence is inconclusive.

What experiments are missing that would strengthen the paper?

• Larger-scale pre-training (7B+ parameters, 100B+ tokens): The largest model is 1.9B parameters on 38B tokens—two orders of magnitude smaller than production LLM training runs. The paper's own kernel benchmarks suggest GEMM speedups are more impactful at larger scales, but there is no accuracy validation at those scales. The risk is that MS-EDEN's unbiasedness property might degrade with deeper networks (more layers for gradient bias to compound) or that the RHT-based smoothing might behave differently at larger hidden dimensions.

• Full TetraJet-v2 comparison: Even if the intermediate FP32 scales and outlier channels are difficult to implement, an emulated comparison (accepting that it would be slower) would establish a tighter upper bound on the accuracy achievable with SR-based methods, providing a more honest baseline for Quartet II's improvement claim.

• Ablation of the clipping factor s in MS-EDEN: The paper introduces a clipping factor s in the RTN quantization step but never reports its value or sweeps it. The interaction of clipping with the EDEN correction factor is non-trivial—aggressive clipping might improve MSE for in-range values but could also make the correction factor S larger and more variable, increasing the variance from stochastic rounding of the scales. An ablation over s would clarify how much of MS-EDEN's advantage comes from the stochasticity placement vs. from the clipping tolerance.

• Training curves over time, not just final loss: Figures 1, 2, and 4 report final validation loss gaps, but Figure 5 (Nanochat) shows that the advantage accumulates over training. Training curves for the C4 ablations would reveal whether Quartet II's improvement is consistent throughout training or concentrated in specific phases (early learning vs. late convergence).

• Gradient cosine similarity to BF16: The paper validates unbiasedness (expectation matches reference) but not gradient alignment at individual steps. Measuring the cosine similarity between quantized and BF16 gradients during training would provide additional evidence that MS-EDEN's lower MSE translates to better per-step optimization directions, not just better long-run averages.

• Comparison to FP8 training: The paper positions NVFP4 as a replacement for higher-precision training but never compares Quartet II directly against FP8. The entire motivation is that NVFP4 training currently loses accuracy relative to FP8; showing that Quartet II narrows or closes this gap would be more compelling than comparisons only to other NVFP4 methods.

6. Limitations and Trade-offs

Difficulty Estimation Uses a Method That Costs More Than the Computation It Allocates

The assumption or constraint. Quartet II does not involve difficulty estimation—unlike the paper analyzed in the prior sections, it is a uniform quantization scheme applied identically to all layers, all training steps, and all data samples. There is no adaptive allocation or per-sample decision-making. This limitation is not about Quartet II directly but about the broader ecosystem into which it fits: the paper provides no mechanism for allocating the quantization budget differently across layers or training phases. The paper explicitly acknowledges a related cost asymmetry in its kernel discussion (Section 7), noting that the MS-EDEN re-quantization requires a global maximum reduction that "acts as a global barrier" and must be "performed in a separate kernel," effectively doubling the memory bandwidth and matrix multiplication costs in a naïve implementation. The post hoc range alignment optimization mitigates this specific overhead (reducing bandwidth by ~20%, Table 2), but the paper does not account for the entire cost of MS-EDEN's quantization operations in the headline speedup numbers.

The consequence. The practical consequence is that the 4.2× speedup figure reported for individual linear layers (Figure 6) does not include the full end-to-end overhead of the MS-EDEN quantization pipeline when applied to all layers at every training step. The end-to-end benchmark in Appendix D.2 reveals a more complex reality: at 1.1B parameters on a single RTX 5090, FP4 training at micro-batch size 1 provides no speedup at all over BF16 because "the matrix dimensions are too small to speed up the matrix multiplications beyond the overhead of the additional operations." The actual 2.45× throughput gain comes entirely from memory savings enabling a 4× larger micro-batch size (4 vs. 1), not from faster GEMM execution. Furthermore, Table 7 shows that at 1.1B parameters, quantization-related overhead (gradient re-quantization + activation re-quantization + AbsMax computation) consumes approximately 26% of backward pass time (13% + 8% + 5%), while NVFP4 GEMMs themselves account for only 19% of backward pass time. As model size grows, the GEMM fraction increases and the overhead fraction decreases (the paper notes this ratio "is expected to drastically decrease as model size grows"), but no measurements are provided at larger scales where the 4.2× layer-wise speedup is projected. The headline speedup numbers are therefore best understood as an upper bound, not a realized end-to-end throughput gain at the scales tested.

What evidence exists in the paper. Appendix D.2 (Table 7, end-to-end 1.1B training throughput) reveals the gap between layer-wise and end-to-end speedups. The breakdown shows that only 43% of total training time is spent in NVFP4 GEMMs (24% forward + 19% backward), with the remaining 57% consumed by operations untouched by FP4 quantization (attention, LM-head, RMSNorm, etc.) or by quantization overhead itself. For the 560M and 1.9B parameter models used in the main accuracy experiments (Sections 6.1–6.2), no throughput measurements are reported at all—only validation loss. The paper does not provide scaling curves showing how the GEMM fraction grows with model size, leaving practitioners to extrapolate from the single 1.1B data point.

Mitigation status. The paper partially addresses this through the post hoc range alignment kernel optimization (Section 7, Figures 7–8), which reduces MS-EDEN re-quantization overhead by approximately 20% in memory bandwidth and 2× in GEMM instructions per NVFP4 group (Table 2). The paper also explicitly notes that the non-GEMM fraction is expected to shrink with model size. However, the core gap—no throughput measurement at the largest accuracy-tested scale (1.9B parameters, 38B tokens)—remains unaddressed. A practitioner evaluating Quartet II for production pre-training would need to benchmark their specific model scale to determine the realized speedup.

---

All Accuracy Validation Is at Small Scale Relative to Production LLM Training

The assumption or constraint. The paper's largest accuracy experiments use a 1.9B-parameter model trained on 38B tokens (Nanochat "$1000" configuration, Section 6.2). The C4 ablation experiments (Section 6.1) use models ranging from 30M to 200M parameters with data-to-parameter ratios from 25 to 800—at the upper end, this means 200M parameters trained on 160B tokens. The paper motivates its work by citing the "exponential pace" of foundation model training costs and NVIDIA Blackwell's promise of 2–4× throughput gains for "massive models such as LLMs." Yet the largest model actually tested (1.9B parameters) is two orders of magnitude smaller than production-scale LLMs (LLaMA-2 70B, GPT-4 at ~1.7T). The paper does not provide results for 7B, 13B, or 70B-parameter models—the scales at which FP4 training would deliver the most impactful cost savings.

The consequence. Several aspects of Quartet II's behavior could change at larger scale:

1. Gradient bias accumulation: MS-EDEN is unbiased only asymptotically (as rotation group size $d \to \infty$), and the paper uses $d = 128$ for hardware compatibility (Appendix A). The paper validates numerically that this is "effectively unbiased" for a pre-trained Llama-3.2-1B model (Figure 9), but with more layers (e.g., 80 layers in LLaMA-2 70B vs. ~20 in the tested models), gradient bias from earlier layers must propagate through many more subsequent layers. Even a tiny per-layer bias could accumulate to a meaningful systematic error in very deep networks trained for trillions of tokens.

2. Verification that rotations remain beneficial: The RHT's effectiveness at "whitening" vectors (spreading energy uniformly across coordinates) depends on the statistical properties of the gradients. If gradients at larger scales exhibit different outlier patterns or correlation structures, the RHT's smoothing benefit could change—potentially improving (if outliers become more severe) or degrading (if gradients are already well-behaved).

3. Interaction with large-scale training dynamics: LLM pre-training at scale often involves loss spikes, phase transitions, and optimizer instabilities that are not present or are less severe at smaller scales. Quartet II's interaction with these phenomena is untested. Figure 5 shows that loss spikes at ~6T tokens affect both BF16 and QAT training, which is encouraging, but the spikes occur in a 560M-parameter model—it is unknown whether the behavior generalizes.

4. The paper's own kernel benchmarks (Figure 6) show speedups increasing with model scale—the 4.2× figure is achieved at 22B-parameter-characteristic tensor shapes, not at the tested accuracy scales. The empirical accuracy validation and the kernel speedup validation are performed at different scales, and the paper provides no bridge between them.

What evidence exists in the paper. None at scale. The paper's largest accuracy experiment (1.9B parameters, 38B tokens) is an order of magnitude smaller than the smallest model in the kernel speedup figure (800M parameters is the smallest speedup data point in Figure 6, and even that uses shapes characteristic of 800M-parameter models but measured at batch size 8, sequence length 2048—a different configuration than the accuracy experiments). The gap between accuracy validation scale and production scale is unaddressed.

Mitigation status. Not addressed. The paper acknowledges no scaling limitation and proposes no extrapolation methodology. A practitioner would need to run their own large-scale validation before adopting Quartet II for production training. The paper's claim that MS-EDEN "improves the state of the art for quantized training in NVFP4" is strictly supported only at the tested scales.

---

Pre-Training Perplexity Improvements Do Not Translate to Downstream Task Gains After Fine-Tuning

The assumption or constraint. The paper's primary evaluation metric is validation loss (perplexity or bits-per-byte) during pre-training, and the central claim is that Quartet II reduces the validation loss gap relative to BF16 by "at least 20%" over prior NVFP4 recipes (Figure 4). However, Section 6.2 and Appendix C (Table 5) also report downstream zero-shot benchmark performance after mid-training and supervised fine-tuning, and the paper explicitly states:

"the vast majority of the differences between FP4 QAT methods are not statistically significant" on downstream benchmarks.

For the 560M-parameter model, Quartet II achieves ArcC 36.8% vs. TetraJet-v2 at 36.8% (identical) and BF16 at 38.8 ± 2.8% (within confidence interval). For GSM8K, Quartet II achieves 7.7% vs. NVIDIA at 9.3% (worse) and BF16 at 11.4%. For HumanEval, Quartet II achieves 1.2% vs. FourOverSix at 3.7% (worse). At the 1.9B-parameter scale, the pattern persists: Quartet II is not the best QAT method on any downstream benchmark—TetraJet-v2 leads on HumanEval (8.5%), FourOverSix leads on MMLU (43.6%), NVIDIA leads on ArcC (56.8%).

The consequence. This raises a genuinely uncomfortable question: what does the 15–25% pre-training perplexity improvement actually buy? If the improvement in validation loss does not translate to better performance on the downstream tasks that practitioners ultimately care about, then the practical value of Quartet II depends entirely on whether perplexity itself is the target metric (e.g., for language modeling APIs where next-token prediction accuracy is the product). For the dominant use case of LLMs—pre-training followed by fine-tuning for downstream applications—the paper's own evidence suggests that the choice of QAT method may be substantially less important than the pre-training loss curves suggest.

There are several possible explanations, none of which the paper investigates:

• The downstream benchmarks have small test sets (HumanEval: 164 problems; GSM8K: ~1,319 test examples) and high variance, making it statistically difficult to detect real but small improvements.
• The short fine-tuning phases (mid-training and SFT) use full-precision backward passes and may erase or override differences in pre-training quality.
• Pre-training perplexity improvements may disproportionately improve performance on tokens or patterns that are underrepresented in the downstream benchmarks.
• The improvements may be concentrated in capabilities (e.g., factual recall, syntactic fluency) that are not well-measured by the chosen benchmarks.

What evidence exists in the paper. Table 5 (Appendix C) provides the complete downstream evaluation, with confidence intervals (±2 standard deviations) for the 560M BF16 baseline. The paper's text in Appendix C acknowledges the result: "one can see that the vast majority of the differences between FP4 QAT methods are not statistically significant." However, the main paper body (Sections 1, 6, 8) emphasizes the pre-training loss improvements without discussing the downstream null result in comparable detail. The conclusion (Section 8) states that the paper "validate[s] that MS-EDEN's better guarantees imply better model quality" without qualification.

Mitigation status. The paper does not address this limitation beyond acknowledging it in the appendix. There is no investigation of why the pre-training improvement does not transfer, no ablation of fine-tuning length or data quantity, and no comparison on additional downstream tasks that might be more sensitive to pre-training quality. A practitioner evaluating Quartet II for a typical pre-train-then-fine-tune workflow is left uncertain about whether the method will improve their final application metrics.

---

The Method Depends on a Specific Hardware Feature (NVIDIA Blackwell Tensor Cores) and a Specific Quantization Format (NVFP4)

The assumption or constraint. Quartet II is designed for and validated exclusively on the NVFP4 format as implemented on NVIDIA Blackwell GPUs. The format-specific choices are deep in the design:

• The 128-element rotation group size is chosen to align with the mma.m16n8k16 tensor-core instruction on Blackwell GPUs (Appendix A).
• The EDEN correction factor precision bottleneck (Section 3.2) is specific to NVFP4's E4M3 FP8 group scales with their ~6.25% relative granularity.
• The post hoc range alignment kernel optimization (Section 7) exploits NVFP4's specific scale hierarchy.
• The Four-Over-Six forward-pass heuristic (Section 4.2) is specific to the E2M1 FP4 format's grid structure with scale choices of 4.0 and 6.0.
• All kernel benchmarks (Figure 6, Table 7) are measured on an NVIDIA RTX 5090 (Blackwell architecture).

The paper does not evaluate Quartet II on the alternative MXFP4 microscaling format, on non-Blackwell GPU hardware, or on non-NVIDIA accelerators. It also does not provide results for FP8 training (the natural comparison point that NVFP4 aims to replace), making the accuracy comparison purely relative to other NVFP4 methods rather than to the precision level practitioners would actually be moving from.

The consequence. Quartet II's improvements are not portable to other hardware platforms or quantization formats without substantial redesign. A practitioner using AMD GPUs (which support MXFP4 but not NVFP4), older NVIDIA hardware (pre-Blackwell), or non-GPU accelerators cannot apply this method. More subtly, the paper's central insight—moving stochasticity from element-level to group-scale-level—may not transfer to formats with different scale precisions, group sizes, or element encodings. For example, MXFP4 uses different scale representations and block sizes; INT4 formats use uniform quantization grids rather than floating-point; future 2-bit or 1-bit formats may have scale-to-element ratios that change the variance trade-off calculus entirely.

Additionally, the lack of FP8 comparison means the paper never answers the question: at what scale and budget does Quartet II's NVFP4 training become preferable to simply using FP8? The motivation (Section 1) states that NVFP4 promises "2-4x throughput gains over FP8," but the paper's 4.2× speedup is reported relative to BF16, not FP8. If FP8 training already achieves, say, 2× speedup over BF16 with minimal accuracy loss, then Quartet II's incremental value over FP8 is the additional speedup (roughly 2×) minus any residual accuracy gap—a more nuanced calculus than the paper's framing.

What evidence exists in the paper. The entire paper is NVFP4-on-Blackwell. No comparison to FP8 training accuracy is provided. No measurements on alternative hardware or formats are attempted. The paper's title ("Accurate LLM Pre-Training in NVFP4") correctly scopes the contribution, but the claims in the body about "improving the state of the art for quantized training" (Section 1, Contributions) are ambiguous about whether this is the state of the art in NVFP4 specifically or in low-precision training broadly.

Mitigation status. Not addressed. The paper does not discuss portability, format-agnosticism, or the path to other hardware targets. The code release may include abstractions that facilitate adaptation, but the paper provides no architectural guidance for doing so. A practitioner not already committed to NVIDIA Blackwell hardware has no evidence that Quartet II would benefit their training pipeline.

---

The Square-Block Weight Re-Use Optimization Is Abandoned Without Quantifying Its Value at Scale

The assumption or constraint. The paper's case for abandoning square-block quantization (16×16 group scales for weights on the forward pass, enabling weight re-use on the backward pass without re-quantization) rests on a specific empirical comparison: Figure 1 panel (e) (MS-EDEN with weight re-quantization) vs. panel (d) (SR without weight re-quantization). The paper concludes that MS-EDEN's lower per-tensor error more than compensates for the additional re-quantization cost. However, this comparison is asymmetric: it compares the best MS-EDEN configuration against the NVIDIA recipe's SR configuration, but it does not compare against the best possible SR configuration that could also re-quantize weights (i.e., SR in panel (e) vs. MS-EDEN in panel (e)). More importantly, the square-block optimization provides an additional benefit beyond gradient error: it reduces memory bandwidth by avoiding a full re-load, de-quantize, rotate, and re-quantize of the weight tensor on the backward pass. At large model scales where weight tensors are enormous (e.g., a 70B-parameter model has weight matrices with tens of thousands of elements per dimension), this memory bandwidth saving could be significant—and it is entirely unquantified.

The consequence. A practitioner evaluating Quartet II for large-scale training faces an incomplete trade-off analysis. The paper demonstrates that MS-EDEN improves gradient estimation quality—but this quality comes at the cost of additional backward-pass operations (weight re-quantization, RHT on weights, EDEN correction computation). For small models (30M–200M), the paper shows the quality improvement dominates. For large models (7B+), the relative costs shift: weight re-quantization scales with model size (more parameters to re-process), and the memory bandwidth for weight operations becomes a larger fraction of total training time. The paper's kernel benchmarks (Figure 6) show speedups for Quartet II's re-quantizing approach at 22B-parameter-characteristic shapes, but these benchmarks measure latency for individual layers, not end-to-end training throughput including optimizer steps, gradient accumulation, and communication. The NVIDIA recipe's square-block approach may have throughput advantages at very large scales that are not captured by the accuracy-only comparison in Figure 1.

Additionally, Figure 2 shows that square-block quantization degrades forward-pass quality (higher validation loss). But this degradation is measured at small scales (30M–200M) on C4 data. Larger models with more parameters might tolerate coarser weight quantization better (they have more redundant capacity), potentially narrowing the forward-pass quality gap between square and native blocks. If so, the square-block approach could be competitive at scale—the paper provides no evidence either way.

What evidence exists in the paper. The key comparison is Figure 1(d) vs. 1(e), limited to accuracy on 30M–200M parameter models. The kernel benchmarks in Figure 6 include MS-EDEN re-quantization overhead, but only as layer-wise latency, not in the context of a full training loop that exercises weight re-quantization at every step. The end-to-end training breakdown (Table 7) at 1.1B parameters shows that quantization-related operations (gradient quantization 13%, activation re-quantization 8%, AbsMax 5%) account for 26% of backward-pass time—but this is for Quartet II's approach, with no comparison to what the NVIDIA square-block approach would consume at the same scale.

Mitigation status. Not addressed. The paper does not provide an ablation comparing MS-EDEN against SR in the same configuration (both with weight re-quantization, both with RHT on the inner dimension) at any model scale. It does not measure the throughput impact of weight re-quantization in an end-to-end training loop. It does not discuss whether the square-block weight re-use optimization could be combined with MS-EDEN (e.g., by designing a rotation scheme compatible with square blocks). A practitioner choosing between Quartet II and the NVIDIA recipe for large-scale training cannot determine from the paper whether the accuracy improvement outweighs the throughput cost.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not introduce a new training paradigm, a new optimizer, or a new model architecture. What it introduces is a reconsideration of a fundamental primitive—unbiased quantization—that the entire NVFP4 training field had taken as settled. The prevailing assumption was that element-wise stochastic rounding was the only practical mechanism for achieving unbiased gradient estimates at 4-bit precision; every prior NVFP4 recipe (NVIDIA et al., 2025; Castro et al., 2025; Chmiel et al., 2025; Chen et al., 2025b; Tseng et al., 2025) built on this foundation. The paper's central finding—that moving stochasticity from individual 4-bit elements to 8-bit group scales reduces quantization error by >2× while preserving unbiasedness—shifts the question from "how do we make SR less noisy?" to "where in the quantization hierarchy should stochasticity live to minimize variance while satisfying the unbiasedness constraint?" This is a reframing, not an incremental improvement: it opens a previously unexamined design dimension that applies to any hierarchical quantization format.

The practical magnitude of the reframing is mixed. The paper demonstrates that MS-EDEN's advantage translates to a 15–25% reduction in pre-training validation loss gap relative to prior NVFP4 recipes (Figures 4, 5). This is a meaningful improvement within the NVFP4 ecosystem, but it does not fully close the gap to BF16 training—Quartet II still loses 1.22–1.44% in bits-per-byte relative to BF16 on Nanochat pre-training (Table 5). More significantly, the downstream benchmark results show that the pre-training perplexity improvement does not reliably translate to better zero-shot task performance after fine-tuning (Appendix C, Table 5). From the perspective of a practitioner whose end goal is downstream accuracy, the landscape shift is genuine but narrower than the pre-training metrics suggest: Quartet II improves language modeling fidelity, but whether this matters for a given application depends on how much the application values perplexity versus task-specific capabilities.

The paper also sharpens the field's understanding of what counts as a valid training method in the hardware-software co-design regime. The explicit decision to evaluate only the GPU-feasible subset of TetraJet-v2 (omitting its intermediate FP32 scales and outlier channels because they "require substantially more complicated kernel support that is not provided by the authors," Section 2) makes a methodological statement: accuracy claims in low-precision training should be conditioned on deployability. A method that looks good in emulation but cannot be efficiently implemented on target hardware is not a complete contribution. This is not a new principle—systems researchers have long argued it—but the paper operationalizes it in the NVFP4 training literature by drawing a clear line between what is and is not evaluated. This implicitly raises the bar for future work: proposals that add complexity without providing kernel support will be compared against a baseline that explicitly excludes them.

The paper resolves one specific contradiction: whether Four-Over-Six (Cook et al., 2025) can be applied to the backward pass. The original work combined 4/6 with stochastic rounding on the backward pass, implying (or at least not contradicting) that the combination was valid. The paper provides both a mechanistic argument (data-dependent grid selection introduces bias even when each branch is individually unbiased via SR) and empirical evidence (Figure 9: the error plateaus rather than decreasing as 1/B) that the combination is biased. This diagnostic—the 1/B scaling test for unbiasedness—is a transferable methodology that future work can apply to any composite quantization scheme.

The research directions that become more attractive after this work:

• Co-design of quantization formats and unbiased estimation algorithms. MS-EDEN's design is tightly coupled to NVFP4's specific scale hierarchy (E4M3 FP8 group scales, 16-element blocks, FP32 global scale). If the scale precision were higher (say, FP16 group scales), EDEN correction could be applied directly without stochastic rounding, eliminating the residual variance from scale randomization. If group sizes were different (say, 32 or 8 elements per scale), the variance trade-off between element-level and scale-level stochasticity would shift. The paper demonstrates that this coupling matters, making format-algorithm co-design a productive research direction rather than an afterthought.

• Understanding gradient structure in rotated space. MS-EDEN's effectiveness depends on the RHT producing vectors with near-uniform energy distribution, which makes deterministic RTN quantization nearly optimal and keeps the EDEN correction factor S close to 1. The paper shows this works for gradients in LLM training (Figure 9), but does not characterize why gradient structure in rotated space is so amenable to quantization, or whether this property holds uniformly across layers, training phases, and architectures. A deeper investigation could inform better rotation strategies (beyond RHT) or identify cases where MS-EDEN would degrade.

The research directions that become less attractive:

• Incremental improvements to element-wise SR. The paper demonstrates that SR's variance penalty is structural—it randomizes every element independently—and that a fundamentally different approach (group-scale-level stochasticity) achieves the same guarantee with much lower error. This suggests that further tuning of SR (e.g., better SR probability distributions, per-element variance reduction) is unlikely to close the gap to MS-EDEN, since the gap originates from where randomness is applied, not from how it is applied. The diminishing returns from optimizing SR make MS-EDEN-like approaches the more promising direction for future unbiased quantization research.

• Square-block quantization as a backward-pass optimization. The paper shows that MS-EDEN with weight re-quantization outperforms SR with weight re-use from square blocks (Figure 1d vs. 1e). This undercuts the primary motivation for square-block quantization on the forward pass—the backward-pass weight re-use benefit is obsoleted if a better quantizer makes re-quantization affordable. While square blocks may retain throughput advantages at very large scales (which the paper does not measure), the accuracy case for them is substantially weakened.

Follow-Up Research This Work Enables

1. MS-EDEN at production scale: 7B–70B parameter models trained on 1T+ tokens. The paper's largest accuracy experiment is 1.9B parameters on 38B tokens (Section 6.2), while its kernel speedup measurements project up to 22B-parameter-characteristic shapes (Figure 6). There is a gap between where accuracy is validated and where speedups are projected. A direct follow-up would train a 7B-parameter Llama-2 or Llama-3 architecture model on at least 200B tokens with Quartet II, measuring both pre-training validation loss and a comprehensive downstream benchmark suite (MMLU, HellaSwag, ARC, GSM8K, HumanEval, and at least one long-form generation task). The key question is whether the 15–25% loss gap reduction over prior NVFP4 recipes persists at scale, and whether the downstream benchmark null result (Table 5) was an artifact of small scale, short fine-tuning, or small test sets. This experiment would also stress-test the unbiasedness claim: with 32+ transformer layers, any residual per-layer gradient bias compounds substantially more than in the paper's 6–10 layer models. Monitoring gradient statistics (per-layer gradient norm, cosine similarity to BF16 gradients) throughout training would reveal whether the asymptotic unbiasedness with d = 128 rotation groups (Appendix A, Figure 9) is sufficient for deep networks.

2. Characterizing the relationship between rotation group size, clipping factor s, and MS-EDEN variance. The paper uses rotation groups of 128 elements and introduces a clipping factor s in the RTN quantization step (Section 3.3), but never reports s values or investigates the interaction between group size, clipping aggressiveness, and the resulting EDEN correction factor distribution. A systematic study would sweep group sizes (64, 128, 256, 512) and clipping factors (across the range that produces non-clipping to aggressive-clipping behavior) on real LLM gradient distributions (not synthetic N(0,1) data), measuring: (a) the mean and variance of the EDEN correction factor S, (b) the resulting MSE of the MS-EDEN quantized gradients relative to BF16, and (c) the variance contributed by the FP8 stochastic rounding of scales versus the deterministic FP4 RTN error. This would establish whether the paper's choice of 128 and its unspecified s are near-optimal, or whether further tuning can recover the remaining gap between MS-EDEN's 9.8 × 10^-3 MSE and biased RTN's 9.0 × 10^-3 in Table 1. The experiment could also test whether larger rotation groups reduce the EDEN correction factor variance (as the asymptotic theory predicts), potentially making the scale-level SR variance negligible even at very aggressive clipping.

3. Combining MS-EDEN with selective higher-precision layers or channels. The NVIDIA recipe (NVIDIA et al., 2025) kept some layers in higher precision, and TetraJet-v2 (Chen et al., 2025b) proposed selective outlier channels with FP32 intermediate scales. The paper explicitly excludes these from its TetraJet-v2 baseline due to kernel complexity, but does not investigate whether similar selective-precision ideas could complement MS-EDEN. A natural experiment: apply MS-EDEN to all layers, but keep the first and last transformer layers (which process raw embeddings and produce logits, respectively) in FP8 on the backward pass, since these layers tend to be more sensitive to quantization. Alternatively, identify channels with the largest gradient magnitudes (outlier channels) and use MS-EDEN's clipping tolerance to handle them naturally—since MS-EDEN allows clipping with EDEN correction compensation, it may already handle outliers better than SR, but this has not been demonstrated. The question is whether selective precision and MS-EDEN are substitutes (one obviates the need for the other) or complements (both together outperform either alone). A controlled ablation comparing uniform MS-EDEN vs. MS-EDEN with 1–2 high-precision layers would quantify the marginal benefit of selectivity.

4. Adapting MS-EDEN to other microscaling formats (MXFP4, MXFP6) and to post-training quantization. The paper's design is NVFP4-specific, but the core insight—moving stochasticity to group scales—applies to any format with a hierarchical scale structure. MXFP4 (supported by AMD GPUs and some NVIDIA hardware) uses different scale representations and block sizes; a direct port of MS-EDEN would need to adjust rotation group sizes (to align with MXFP4's block structure), re-derive the scale precision analysis (MXFP4 scales may have different relative granularity than NVFP4's E4M3), and re-validate unbiasedness. More ambitiously, MXFP6 (6-bit microscaling formats with 2-bit group scales) would shift the variance trade-off even further in MS-EDEN's favor, since the element quantization is finer and the scale precision is coarser, making scale-level SR relatively less costly. Conversely, applying MS-EDEN to post-training quantization (where weights are quantized once for inference, not continuously during training) would simplify the problem—the EDEN correction could be applied deterministically to the scales without SR, since unbiasedness is irrelevant for static weight quantization—but the RHT step would need to be folded into the inference graph, and the rotation overhead might dominate the quantization savings. A concrete follow-up would implement MS-EDEN-style quantization for MXFP4 inference and measure the weight-quantization MSE vs. standard RTN, establishing whether the EDEN correction provides benefits beyond the training setting.

5. Understanding why pre-training perplexity improvements do not transfer to downstream benchmarks, and designing fine-tuning protocols that preserve them. The paper's downstream results (Table 5) show that Quartet II's 15–25% pre-training loss improvement relative to other NVFP4 methods does not produce statistically significant gains on ARC, GSM8K, HumanEval, or MMLU after SFT. This is the most practically important open question from the paper: is the null result an artifact of experimental design (short fine-tuning phases with full-precision backward passes, small test sets, specific benchmark choices), or does NVFP4 pre-training genuinely improve language modeling fidelity without improving the extractable capabilities that standard benchmarks measure? A targeted follow-up would vary: (a) fine-tuning duration (from hundreds to tens of thousands of steps) to see whether longer fine-tuning surfaces pre-training quality differences; (b) fine-tuning precision (keeping the backward pass in NVFP4 vs. BF16 during SFT) to test whether the switch to full precision erases the benefit; (c) benchmark selection, including tasks more directly tied to language modeling quality (perplexity on domain-specific corpora, long-range coherence metrics, factual consistency measures) alongside standard QA and reasoning benchmarks. The experiment would also benefit from using a larger base model (7B+) and longer pre-training (100B+ tokens) to ensure that pre-training quality differences are large enough in absolute terms to potentially survive fine-tuning. This follow-up would determine whether Quartet II's practical value is limited to pure language modeling applications or extends to the pre-train-then-fine-tune workflow that dominates LLM deployment.

6. A negative-result study: what happens when MS-EDEN is applied to the forward pass? The paper applies MS-EDEN only to the backward pass, using deterministic RTN + 4/6 on the forward pass. The forward pass has no unbiasedness requirement, so MS-EDEN's key advantage (unbiasedness at near-RTN error) is irrelevant—but MS-EDEN's RHT and EDEN correction might still reduce forward-pass quantization error by smoothing activation distributions before quantization. However, the RHT would need to be inverted after the GEMM (unlike the backward pass where rotations cancel), adding computational overhead. A negative-result study would implement MS-EDEN for forward-pass quantization (with inverse RHT after the GEMM) and measure: (a) validation loss gap vs. RTN + 4/6 on the forward pass, (b) throughput impact from the inverse RHT, and (c) whether any accuracy benefit justifies the overhead. The hypothesis (likely null) is that the forward pass does not benefit from MS-EDEN because RTN + 4/6 is already near-optimal for forward-pass MSE, and the RHT overhead dominates any small accuracy gain. This negative result would be valuable because it would delineate the scope of MS-EDEN's applicability—the backward pass specifically, where unbiasedness creates a binding constraint that MS-EDEN relaxes—and prevent unproductive efforts to apply it where it provides no advantage.

Practical Applications and Downstream Use Cases

1. Training small-to-medium language models under tight GPU memory constraints. The paper's end-to-end benchmark (Appendix D.2) demonstrates that FP4 training with Quartet II enables a 4× larger micro-batch size on a single 32GB GPU (NVIDIA RTX 5090), yielding 2.45× higher training throughput (51 ktok/s vs. 20.8 ktok/s for BF16) for a 1.1B-parameter model. For researchers, startups, or academic labs with limited GPU resources, this is directly actionable: adopting Quartet II allows training models that would otherwise not fit in memory, or training the same model 2.45× faster. The memory savings come from storing 4-bit quantized preactivations instead of BF16 for the backward pass—a feature of the Quartet II computation graph (Figure 3) that is orthogonal to the MS-EDEN quality improvement but enabled by it (since MS-EDEN provides the accurate re-quantization needed for the backward pass). At the scales tested (560M–1.9B parameters), the pre-training quality gap relative to BF16 is 1.22–1.44% in bits-per-byte (Table 5)—a small enough degradation that the throughput gain may justify the quality trade-off for many practical training runs, particularly during development, hyperparameter search, or architecture exploration where many training runs are needed and final model quality matters less than iteration speed.

2. Cost reduction for large-batch LLM pre-training where memory bandwidth dominates. The paper's kernel benchmarks (Figure 6) project up to 4.2× speedup over BF16 for individual linear layer operations at tensor shapes characteristic of 7B–22B parameter models. For organizations running large-scale pre-training (e.g., training a 7B model on 1T+ tokens), linear layers (attention projections and MLP) consume the majority of training FLOPs and memory bandwidth. If Quartet II's projected layer-wise speedups translate to even 3× end-to-end throughput at these scales (assuming the non-GEMM fraction shrinks as the paper predicts), the cost savings are substantial: a training run that would require 1,000 GPU-days in BF16 could be completed in ~333 GPU-days in NVFP4 with Quartet II. The key uncertainty—which the paper does not resolve—is whether the 1.22–1.44% pre-training perplexity degradation observed at 560M–1.9B parameters narrows or widens at 7B+ scale, and whether downstream task performance after fine-tuning shows a measurable penalty. An organization considering this deployment would need to run a validation experiment at their target scale, using the paper's open-source code and kernels. The decision trades off a known throughput gain (projected 2–4×) against an unknown but likely small quality penalty at scale.

3. Enabling training of embedding-heavy or activation-memory-bound architectures on commodity GPUs. Beyond standard decoder-only transformers, architectures that are activation-memory-bound—including encoder-decoder models (T5, BART), models with large vocabulary embeddings relative to hidden size, or models that process long sequences with large intermediate activations—benefit disproportionately from FP4's 4× activation memory reduction. Quartet II's backward pass re-quantization design (Figure 3) stores only the 4-bit quantized versions of all GEMM preactivations, which is the standard practice in FP4 training but is specifically validated by the paper's kernel benchmarks (Table 2, 7). For a practitioner training a model where activations (not weights or optimizer states) are the memory bottleneck, Quartet II provides a drop-in path to 4× larger micro-batch sizes or 4× longer sequence lengths compared to BF16 training, without redesigning the architecture. The quality trade-off is quantified at the paper's tested scales (1.22–1.44% BPB increase for 560M–1.9B parameter models on FineWeb-Edu, Table 5), giving practitioners a concrete estimate for cost-benefit analysis.

When to Prefer This Method

The paper positions Quartet II (and by extension MS-EDEN) as an improvement over prior NVFP4 training recipes, not as a replacement for higher-precision training (BF16 or FP8). The decision rule is therefore contextual: given a commitment to NVFP4 training (because hardware, throughput requirements, or memory constraints make it necessary or strongly beneficial), when should one choose Quartet II over the alternatives?

• Prefer Quartet II over the NVIDIA recipe (NVIDIA et al., 2025) or FourOverSix (Cook et al., 2025) when: (a) Pre-training validation perplexity is the primary quality metric and downstream fine-tuning performance is either not relevant or will be validated independently. The paper shows a 15–25% reduction in validation loss gap relative to BF16 compared to these baselines (Figures 4, 5; Table 5). (b) The model has native 1×16-scaled forward-pass quantization rather than square-block 16×16 scales, since Quartet II's forward-pass design relies on native scales + 4/6 to achieve better representation capacity than square blocks (Figure 2), and MS-EDEN on the backward pass compensates for the resulting need to re-quantize weights. (c) Efficient MS-EDEN kernels (post hoc range alignment, Section 7) are available for the target hardware; the paper provides these for NVIDIA Blackwell GPUs, and the quality-throughput trade-off at other scales needs independent benchmarking. (d) Training runs are long enough (tens of billions of tokens or more) for MS-EDEN's unbiased gradient estimation advantage to accumulate relative to biased alternatives—Figure 5 shows the loss gap widening over training time, suggesting short runs may not realize the full benefit.

• Prefer the simplified TetraJet-v2 (SR-based, GPU-feasible) over Quartet II when: (a) Downstream zero-shot benchmark performance after fine-tuning is the primary quality metric, and the paper's Table 5 shows no statistically significant difference between the methods on ARC, GSM8K, HumanEval, or MMLU. If Quartet II's pre-training perplexity improvement does not translate to the application's target metrics, the additional implementation complexity of MS-EDEN (RHT, EDEN correction computation, two-kernel re-quantization) may not be justified. (b) The training setup already uses square-block quantization or other forward-pass optimizations that Quartet II's native-scale design would require abandoning and re-tuning. (c) The hardware does not support the custom CUDA kernels required for efficient MS-EDEN execution, making the throughput advantage relative to simpler SR-based methods negligible or negative.

• Prefer BF16 or FP8 training over any NVFP4 method (including Quartet II) when: (a) The model is small enough that training throughput is not a binding constraint, or GPU memory is sufficient for desired batch sizes and sequence lengths in higher precision. (b) Downstream task accuracy is paramount and even a 1.22–1.44% pre-training perplexity degradation (Quartet II's best result, Table 5) is unacceptable without evidence that it does not harm final task performance. The paper provides no evidence that Quartet II matches BF16 on downstream benchmarks—in fact, Table 5 shows BF16 consistently outperforming all NVFP4 methods on every downstream metric at both tested scales. (c) Very short fine-tuning phases are planned, since the paper disables backward-pass quantization for short mid-training and SFT stages ("to get the most out of these very short and data-limited phases," Appendix C), implying that NVFP4 training is most appropriate for long pre-training runs where cumulative gradient estimation quality matters, not for short adaptation phases where full precision is affordable.