Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling

ArXiv: 2512.02010

🎯 Pitch

Standard NVFP4 quantization wastes precision on values near the top of each block's range—you simply cannot represent anything between 66.7% and 100% of the block maximum. The paper shows that adaptively scaling some blocks to a peak value of 4 instead of 6 eliminates this representable gap, cutting quantization error by up to 45% and moving NVFP4 training loss 13% closer to BF16 without changing the datatype.

---

1. Executive Summary

This paper introduces Four Over Six (4/6), a modification to the NVFP4 block-scaled quantization algorithm that improves numerical accuracy by adaptively scaling some blocks to a maximum FP4 value of 4 rather than the standard 6, yielding reduced quantization error for near-maximal values that are otherwise unrepresentable in the 66.6%–100% range of a block's largest magnitude. Evaluated on Llama 3 and Qwen 3 models during post-training quantization and on the Nemotron 3 Nano 30B-A3B architecture during pre-training, 4/6 brings NVFP4 training loss 13.0% closer to the BF16 baseline while adding under 15% overhead to quantization kernels, and when combined with existing PTQ methods such as AWQ and SmoothQuant it improves WikiText-2 perplexity across all tested models — establishing that the dominant source of NVFP4 performance degradation is rounding error on values near the top of each block's range, and that this can be substantially mitigated without changing the underlying FP4 datatype or search strategy, provided scale factors have sufficient precision (FP8 E4M3 rather than E8M0) to accommodate the alternate scaling.

2. Context and Motivation

The Core Problem: NVFP4 Quantization Degrades Model Quality, and We Don't Fully Understand Why

The fundamental question this paper tackles is deceptively specific: when you quantize LLM weights, activations, and gradients to NVFP4 — a 4-bit floating point format with block scaling — where exactly does the performance degradation come from, and can you fix it without changing the datatype or adding expensive operations?

This matters because NVFP4 represents a genuine hardware inflection point. NVIDIA's Blackwell GPU architecture provides dedicated hardware support for NVFP4 matrix multiplications, offering up to 2× speed improvements over FP8 and 4× over BF16/FP16 on B200 GPUs (Section 6). Unlike FP8, which is now broadly adopted in training pipelines (DeepSeek-V3, Llama 4, and others all report FP8 training), NVFP4 sits at a precarious position: the precision is low enough that naive quantization introduces substantial quality degradation, yet the speedup is compelling enough that making it work would constitute a major advance in training efficiency. The paper's framing is practical: "to make NVFP4 training viable, more lightweight operations that improve numerical accuracy are necessary" (Section 1).

The problem has two distinct manifestations:

• During pre-training: To benefit from NVFP4 hardware acceleration, both operands of every matrix multiplication must be quantized to NVFP4 — meaning weights, activations, and gradients all go through the narrow FP4 bottleneck. Current state-of-the-art NVFP4 training recipes (the NVIDIA NVFP4 pre-training paper, Quartet, FP4 All the Way) require a suite of auxiliary operations to maintain accuracy: random Hadamard transforms (RHT) to decorrelate gradient dimensions, stochastic rounding (SR) to reduce quantization bias, keeping sensitive layers (attention, normalization, output heads) in high precision, and "healing" the model by switching to high precision near the end of training (Sections 1, 4.1). Each of these operations adds computational overhead. The paper's explicit concern is that "if too much overhead is introduced, it becomes faster to train models using more accurate FP8 formats" — making the entire NVFP4 value proposition collapse.

• During post-training quantization (PTQ): Even when only weights and activations need to be quantized (inference), existing methods — GPTQ, AWQ, SmoothQuant — fail to fully recover BF16 performance when targeting NVFP4 rather than INT4. Table 3 shows this concretely: Llama-3.1-8B at BF16 achieves 7.54 WikiText-2 perplexity, while standard NVFP4 round-to-nearest quantization degrades this to 8.43, a non-trivial gap.

Prior Approaches and Where They Fall Short

Block-scaled FP4 formats (MXFP4 and NVFP4) are a partial solution. The paper acknowledges that FP4 alone — with only 16 representable values: ±{0, 0.5, 1, 1.5, 2, 3, 4, 6} — has far too narrow a dynamic range (12:1, the ratio of 6 to 0.5) for practical LLM tensors. Block scaling addresses this by storing a higher-precision scale factor (FP8 E4M3 for NVFP4, FP8 E8M0 for MXFP4) alongside every block of FP4 values (16 values per block for NVFP4, 32 for MXFP4). This allows different blocks in the same tensor to have vastly different scales, dramatically expanding the representable range. However, this doesn't solve the within-block representation problem: within a single block of 16 values, all values are still quantized to the same FP4 grid, which omits entire regions of the range — notably the gap between 4 (66.6% of the block maximum) and 6 (100%).

Prior PTQ methods were designed for INT4, not FP4, and their core assumption doesn't transfer. The paper observes that most quantization methods — per-channel smoothing in AWQ and SmoothQuant, second-order Hessian-based optimization in GPTQ, rotation-based outlier suppression in QuaRot and SpinQuant — were developed with INT4 in mind. INT4 has uniform step sizes, so the primary challenge is reducing dynamic range to fit values into the representable interval. Block-scaled FP4 formats, by contrast, already solve the dynamic range problem via per-block scaling factors. The remaining error comes from the non-uniform step sizes of the floating point representation itself — specifically, the coarse granularity near the top of each block's range. Existing methods, which focus on suppressing outliers and smoothing distributions, don't directly address this mechanism of error because it's inherent to the FP4 datatype's grid, not to the presence of outliers. The paper's diagnostic in Section 2.2 makes this explicit: when scale factors are kept in high precision while only values are cast to FP4, performance degrades substantially; but when values are kept in high precision while only scale factors are quantized, performance recovers completely (Figure 2a). This cleanly isolates the source of error to the FP4 value quantization step, not the scale factor quantization.

Pre-training with FP4 requires expensive auxiliary operations that the paper aims to complement, not replace. The NVIDIA NVFP4 pre-training recipe (which this work builds on directly — the authors acknowledge following it in Section 4.1) uses stochastic rounding on gradients, random Hadamard transforms on weight gradient inputs, 2D block quantization on weight matrices, and high-precision retention for sensitive layers and operations. Other concurrent work (Quartet, FP4 All the Way, the MXFP4 training paper by Tseng et al.) explores similar techniques. These methods all operate orthogonally to the quantization algorithm itself — they change the distribution of values being quantized (RHT, 2D blocking) or the rounding procedure (stochastic rounding), but they don't modify how the FP4 grid maps values to representable numbers. The gap this paper identifies is that even with all these techniques applied, the fundamental quantization operation — mapping scaled values to the FP4 grid — introduces error that is disproportionately concentrated on values near the top of each block's range, and no prior work addresses this directly.

The Specific Diagnostic Gap: No One Asked Which FP4 Values Cause the Damage

The paper's key insight, laid out in Section 2.2, is that the degradation from NVFP4 quantization is not uniformly distributed across all FP4 values — it's concentrated on specific values that the FP4 grid represents poorly. Figure 2b provides the crucial evidence: the authors simulate a "partial quantization" where only scaled values above a threshold are cast to FP4 (all others stay at high precision), and measure downstream perplexity as this threshold varies. The result is not a smooth, monotonic degradation. Instead, performance degrades gradually as values above 0, 1, 2, 3, and 4 are quantized, but then steeply drops when values above 4 are included — precisely the region between 4 and 6 where FP4's step size jumps to 2 and four entire FP4 values (4, 6, plus their negatives) are covered by just two representable numbers.

This is a subtle and non-obvious finding. The conventional wisdom in quantization — going back to the foundational white paper by Nagel et al. (2021) — is that you should scale values to the full range of the target format to maximize precision. For FP4, that means scaling the largest value in each block to 6. But this convention, the paper argues, is exactly what causes the problem: when a block's largest value is scaled to 6, the value 4 in FP4 represents only 66.6% of that maximum, and no FP4 value exists between 4 and 6. This means any value between 66.6% and 100% of the block maximum gets rounded either to 4 (introducing up to 33.3% relative error) or to 6 (if it's close enough to the top). The simple worked example in Table 1 crystallizes this: the block [10, 20, 30, 40] quantized with the standard M=6 approach introduces an MSE of 4.33, while scaling the same block to M=4 yields MSE of 0 — because 30, which is 75% of the maximum, lands cleanly on the representable FP4 value 3 when scaled to 4 rather than falling in the gap between 4 and 6 when scaled to 6.

How This Paper Positions Itself

The paper frames its contribution not as a new quantization format or a new training recipe, but as a modification to the quantization algorithm itself — a change to how the NVFP4 grid is used — that is orthogonal to and compatible with existing techniques. This positioning has several important implications:

It's a drop-in improvement, not a replacement. The paper explicitly tests 4/6 combined with round-to-nearest (RTN) quantization, GPTQ, AWQ, and SmoothQuant (Tables 5, 6, 7). It also integrates 4/6 into the full NVFP4 pre-training pipeline with stochastic rounding, RHT, and 2D block quantization (Section 4.1, Figure 3). The claim is that 4/6 composes with existing methods — it addresses a source of error that these methods don't touch.

It's specific to NVFP4, not MXFP4. The paper is careful to bound the applicability: 4/6 requires that scale factors have sufficient precision to represent a 50% larger value (since a block scaled to 4 needs a scale factor 1.5× larger than the same block scaled to 6). NVFP4 uses FP8 E4M3 for scale factors, which has enough granularity to accommodate this. MXFP4 uses FP8 E8M0, where each representable value is a factor of 2 from its neighbors — the only way to represent a 1.5× increase is to jump by a full factor of 2, destroying the precision benefit. This is a crucial distinction that explains why prior work on MXFP4 didn't discover this approach, and it positions 4/6 as an argument for E4M3 scale factors over E8M0 in future format designs.

It's motivated by an empirical diagnostic, not theoretical analysis. The paper doesn't derive 4/6 from first principles of information theory or quantization error bounds. It emerges from the empirical observation in Figure 2 — the "steep slope at x=5" — that near-maximal values are the bottleneck. The solution (scale some blocks to 4 instead of 6) follows directly from examining the FP4 grid: the gap between representable values is largest at the top (step size of 2 between 4 and 6), so shrinking the range of values being mapped to that grid — even at the cost of losing the [4,6] range — can reduce worst-case error for blocks where most values cluster below 75% of the maximum.

The claimed improvement is modest but meaningful. The paper does not claim 4/6 makes NVFP4 match BF16 performance. The 13.0% closer-to-BF16 figure in pre-training and the ~0.1–0.4 perplexity improvements in PTQ are incremental gains. This honesty about the magnitude of improvement is important context: 4/6 is sold as a lightweight, low-overhead (<15%) operation that moves the needle in the right direction, not a silver bullet. Combined with the fact that it composes with other methods, this positions it as a practical engineering improvement that, alongside future advances in training recipes and verifier design, could collectively close the gap to high precision.

Connecting to the Prior Sections

The Executive Summary established that 4/6 works by adaptively scaling some blocks to 4 rather than 6, reducing quantization error for near-maximal values, and that it yields measurable improvements in both pre-training and PTQ. What this Context and Motivation section adds is the why: the specific diagnostic that NVFP4 error is concentrated on values around 5 in the scaled representation (Figure 2b), that this error comes entirely from the FP4 value casting step rather than scale factor quantization (Figure 2a), and that the conventional wisdom of always scaling to the full FP4 range is precisely what creates the gap between representable values that causes the problem. Prior methods — both PTQ techniques designed for INT4 and FP4 training recipes using RHT and stochastic rounding — operate orthogonally to this mechanism, which is why 4/6 provides additional benefit when integrated with them. The paper's narrow applicability to NVFP4 (not MXFP4) follows from the need for E4M3-precision scale factors, making this an argument about format design as much as about quantization algorithms.

3. Technical Approach

3.1 Reader Orientation

The paper develops Four Over Six (4/6), a modification to the NVFP4 block-scaled quantization algorithm that makes a per-block decision about whether to scale values to a maximum representable FP4 value of 4 or 6, rather than always scaling to 6 as the standard algorithm does. The problem it solves is that standard NVFP4 quantization introduces disproportionate error on values that fall between 66.6% and 100% of a block's largest magnitude — the gap between FP4 values 4 and 6 where the representable grid is coarsest — and the solution works by sacrificing the ability to represent the extreme values 6 (and -6) in exchange for a denser grid of representable values across the 0–4 range, adaptively applied only to blocks where this tradeoff reduces total quantization error.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components operating within the NVFP4 quantization pipeline of a neural network layer:

1. High-Precision Tensor ($X$) — the BF16 or FP32 weights, activations, or gradients that need to be compressed to NVFP4 format before a matrix multiplication can execute on the hardware accelerator. This is the input that quantization consumes.

2. Standard NVFP4 Quantizer (M=6 path) — the existing quantization algorithm specified in Equations 1–3 of the paper, which computes a tensor-wide FP32 scale factor $\alpha$, per-block FP8 E4M3 scale factors $\Delta_i$, and per-value FP4 representations $\bar{X}^{(6)}$ by scaling the block's largest magnitude to 6. This produces one candidate quantized representation for every block of 16 values.

3. Alternative NVFP4 Quantizer (M=4 path) — the same quantization algorithm but with the block's largest magnitude scaled to 4 instead of 6, producing a second candidate quantized representation $\bar{X}^{(4)}$ for every block. This requires a modified tensor-wide scale factor to prevent FP8 overflow in the per-block scales.

4. Error Comparator and Scale Selector — for each block, both candidate representations ($\bar{X}^{(6)}$ and $\bar{X}^{(4)}$) are dequantized back to high precision, their per-element errors relative to the original values are computed, and the candidate with lower aggregate error (measured by MSE or MAE, depending on context) is selected. The corresponding scale factor $\Delta_i$ and FP4 values are written out.

5. Hardware-Accelerated GEMM — the selected NVFP4 representations for weights and activations (and gradients, during training) are fed into the Blackwell GPU's native NVFP4 matrix multiplication unit, which accumulates in FP32 and outputs in BF16.

Information flows as follows: a high-precision tensor enters → the tensor-wide FP32 scale factor $\alpha$ is computed (once, with the modified maximum FP8 constant) → for each block of 16 values, two quantization paths execute in parallel: one scaling the largest value to 6, one to 4 → both quantized versions are immediately dequantized in-register → per-element squared (or absolute) errors are accumulated → the lower-error candidate wins, and its FP4 values and FP8 scale factor are stored → the packed NVFP4 tensor is passed to the GEMM unit.

3.3 Roadmap for the Deep Dive

• First, the standard NVFP4 quantization algorithm (Equations 1–3) in complete detail, because 4/6 is a modification of this algorithm and the reader must understand what specifically changes and why.
• Second, the diagnostic analysis (Section 2.2) that identifies which FP4 values cause quantization error, since this motivates the entire design — understanding the problem makes the solution's logic transparent.
• Third, the core 4/6 mechanism: the modified tensor scale computation, the dual-path quantization, the error metrics used for selection, and why M=4 was chosen over other candidates like M=2 or M=3.
• Fourth, the hardware-efficient CUDA implementation strategy — how both quantization paths, dequantization, and error computation are kept in the register file using PTX cvt instructions to achieve <15% overhead.
• Fifth, the integration of 4/6 into the full NVFP4 pre-training pipeline (Figure 3), including how it composes with stochastic rounding, random Hadamard transforms, 2D block quantization, and per-tensor decisions about whether to apply 4/6 to weights, activations, gradients, or subsets thereof.
• Sixth, the integration of 4/6 into PTQ methods (GPTQ, AWQ, SmoothQuant), which is architecturally simpler since only weights and activations are quantized, but involves decisions about whether to modify the optimization procedure itself.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems improvement paper whose core idea is that the NVFP4 quantization grid, when always scaled to use its full [-6, 6] range, leaves a large unrepresentable gap between 66.6% and 100% of each block's maximum — and that adaptively shrinking the range to [-4, 4] for blocks where the values cluster below 75% of the maximum reduces quantization error at very low cost.

---

Standard NVFP4 Quantization Algorithm

Before explaining what 4/6 changes, the reader must understand the baseline quantization algorithm that NVFP4 hardware implements. The algorithm operates on a tensor $X$ of high-precision values (BF16 or FP32) and produces three outputs: a single FP32 tensor-wide scale factor $\alpha$, one FP8 E4M3 scale factor $\Delta_i$ for every contiguous block of 16 values, and one FP4 E2M1 value for every element of the original tensor. Together, the quantized representation approximates the original value at position $j$ in block $i$ as $\bar{X}_{i,j} \times \Delta_i \times \alpha$.

Step 1: Tensor-wide scale factor.

$\alpha_{\text{FP32}} = \frac{\max(|X|)}{M_{\text{FP4}} \times M_{\text{FP8}}}$

where $\max(|X|)$ is the largest absolute value in the entire tensor, $M_{\text{FP4}} = 6$ is the largest representable magnitude in FP4 E2M1, and $M_{\text{FP8}} = 448$ is the largest representable magnitude in FP8 E4M3.

What it computes: a single scalar $\alpha$ such that when the tensor's largest value is divided by $\alpha$ and then further divided by the maximum per-block scale factor (which will be computed next), the result lands at exactly 6 — the top of the FP4 range. This coarsest level of scaling ensures that no value in the tensor, regardless of which block it belongs to, will ever need to be represented by an FP4 value greater than 6.

Why this form: the denominator $M_{\text{FP4}} \times M_{\text{FP8}}$ acts as a two-stage normalization. The $M_{\text{FP4}}$ factor accounts for the FP4 range — ensuring the largest value maps to 6 — and the $M_{\text{FP8}}$ factor accounts for the fact that each block will multiply its values by its own FP8 scale factor $\Delta_i$, which can be at most 448. By dividing the global maximum by the product of both maxima, the algorithm guarantees that for the block containing the tensor's largest value, the computed $\Delta_i$ will be exactly 448, and the scaled value will be exactly 6, with no overflow possible for any other block.

Step 2: Per-block scale factors.

$\Delta^{\text{FP8}}_i = \max(|X_{16i...16(i+1)}|) \div (\alpha \times M_{\text{FP4}})$

where $\max(|X_{16i...16(i+1)}|)$ is the largest absolute value within the $i$-th block of 16 consecutive elements, $\alpha$ is the tensor-wide scale from Step 1, and $M_{\text{FP4}} = 6$. The result is then cast (rounded) to the nearest representable value in FP8 E4M3.

What it computes: for each block of 16 values, a scale factor $\Delta_i$ such that when the block's own largest value is divided by both $\alpha$ and $\Delta_i$, the result equals exactly 6 — aligning that block's internal maximum with the top of the FP4 grid, regardless of how much smaller it is than the global tensor maximum. Blocks with smaller values get proportionally smaller $\Delta_i$, keeping their internal dynamic range compressed into the same [-6, 6] FP4 interval.

Why this form: the division by $M_{\text{FP4}}$ inside the scale factor computation (rather than combining it with $\alpha$) is what enables the two-stage scaling to handle tensors with large dynamic range. Without per-block scaling, a tensor containing both very large and very small values would force small values to be quantized to zero because the single global scale would make them fall below FP4's smallest representable magnitude (0.5). The block-wise $\Delta_i$ solves this — small-value blocks get small $\Delta_i$, keeping their values in the representable range.

The FP8 casting introduces error. The computed $\Delta_i$ must be stored in FP8 E4M3 format, which has only 256 representable values (including subnormals) and a maximum of 448. If the ideal $\Delta_i$ falls between two FP8 values, it gets rounded to the nearest one, and this rounding error propagates to every value in the block since all 16 values are multiplied by the same $\Delta_i$ during dequantization. The paper's Figure 2a shows that this error source, however, has minimal impact on downstream model quality compared to the FP4 value rounding in the next step.

Step 3: Per-value FP4 quantization.

$\bar{X}^{\text{FP4}} = \begin{cases} \frac{1}{2} \lceil 2 \frac{X}{\alpha \Delta} \rfloor, & |\frac{X}{\alpha \Delta}| < 2 \\ \lceil \frac{X}{\alpha \Delta} \rfloor, & |\frac{X}{\alpha \Delta}| < 4 \\ 2 \lceil \frac{X}{2 \alpha \Delta} \rfloor, & |\frac{X}{\alpha \Delta}| \leq 6 \end{cases}$

where $X$ is the original high-precision value, $\alpha$ is the FP32 tensor scale, $\Delta$ is the FP8 block scale (already rounded to E4M3), and $\lceil \cdot \rfloor$ denotes rounding to the nearest integer.

What it computes: the three-branch piecewise function maps the scaled value $X / (\alpha \Delta)$ to the nearest representable FP4 E2M1 value. FP4's representable positive values are {0, 0.5, 1, 1.5, 2, 3, 4, 6}, with step sizes that increase as values grow larger: 0.5 between 0 and 2 (first branch, where the factor $\frac{1}{2}$ outside the rounding and $2$ inside together handle the half-integer grid), 1 between 2 and 4 (second branch, direct rounding to integers), and 2 between 4 and 6 (third branch, where the factor $2$ outside and the division by $2$ inside handle the even-integer grid). The three branches correspond to FP4's three precision regimes: high precision near zero, medium precision in the middle, and low precision at the top.

Why this form: the piecewise structure directly reflects the FP4 E2M1 format's encoding. FP4 E2M1 has 1 sign bit, 2 exponent bits, and 1 mantissa bit. The exponent field determines the step size: exponent 00 gives step size 0.5 (values 0, 0.5, 1, 1.5), exponent 01 gives step size 1 (values 2, 3), and exponent 10 gives step size 2 (values 4, 6). The three branches of Equation 3 correspond to these three exponent regimes, and the different pre- and post-rounding scaling factors ($2$ and $\frac{1}{2}$ in the first branch, $2$ and $\frac{1}{2}$ in the third) compensate for the fact that $\lceil \cdot \rfloor$ rounds to integers while FP4 needs half-integer or even-integer values.

Where the error concentrates. The critical region is the third branch ($4 < |X/(\alpha\Delta)| \leq 6$). Here, the step size is 2, meaning that the only representable values are 4 and 6. Any scaled value between 4 and 6 — for instance, 5 — must be rounded either down to 4 (introducing a relative error of 20%) or up to 6 (introducing a relative error of 20% in the other direction). Since round-to-nearest is used, the threshold is 5: values in (4, 5] round to 4, and values in (5, 6] round to 6. This creates a quantization dead zone between 4 and 6 where a full third of the FP4 range (2 out of 6 units) is covered by only two representable values with a gap of 2 between them. In contrast, the region between 0 and 2 has four representable values with a gap of only 0.5 between neighbors — 4× denser.

The same logic applies to negative values: the gap between -6 and -4 (step size 2) is the symmetric dead zone for large-magnitude negative numbers.

---

Diagnostic: Which Values Actually Cause the Degradation?

Section 2.2 of the paper performs a crucial empirical decomposition that motivates the entire 4/6 design. The experiment is a partial quantization simulation: rather than quantizing all values in a tensor to NVFP4, the authors only quantize values whose scaled magnitude exceeds a threshold $x$, leaving all smaller-magnitude values in high precision. They vary $x$ from 0 (all values in high precision — equivalent to BF16) to 6 (all values quantized — equivalent to full NVFP4). For each threshold, they measure the downstream WikiText-2 perplexity of Llama-3.1-8B.

What this experiment reveals. The resulting curve (Figure 2b) is not smooth. Performance degrades gradually as values above 0 through 4 are quantized — the slope is shallow. Then, when the threshold crosses 4 and values above 4 start being quantized, there is a steep drop in perplexity. A secondary steeper region is also observed around scaled values of 2.5 and 3.5. The interpretation is straightforward: values that land in FP4's coarse step-size regions (especially the [4,6] gap) are responsible for a disproportionate share of the total model degradation. Values that land in the fine-grained [0,2] region cause relatively little harm.

A complementary experiment (Figure 2a) decomposes the error into two sources: error from casting scale factors $\Delta_i$ to FP8 E4M3, and error from casting values $\bar{X}$ to FP4 E2M1. By simulating NVFP4 with one source kept in high precision and the other quantized, the authors show that keeping values in high precision while quantizing only scale factors completely recovers BF16 performance, while keeping scale factors in high precision while quantizing values still causes substantial degradation. This cleanly isolates the FP4 value quantization as the sole meaningful source of model quality loss — the FP8 scale factor error is negligible in comparison.

Implication for algorithm design. Any method that improves NVFP4 accuracy must address the FP4 value quantization step specifically, and within that step, must address the representation of values that land in the coarse regions of the FP4 grid — primarily the [4,6] gap, secondarily the gaps around 2.5 and 3.5 (which correspond to the boundaries where step size changes from 0.5 to 1, and from 1 to 2, respectively).

---

The Core 4/6 Mechanism: Dual-Path Quantization with Adaptive Scale Selection

The central idea of 4/6 is to give each block of 16 values a choice: be quantized with the largest value scaled to 6 (standard NVFP4), or with the largest value scaled to 4 (the proposed alternative). The block takes whichever option yields lower reconstruction error. The name "Four Over Six" encodes this logic: the algorithm prefers scaling to 4 over scaling to 6 when — and only when — doing so reduces error.

Why 4 specifically? The FP4 format has representable positive values {0, 0.5, 1, 1.5, 2, 3, 4, 6}. If the largest value in a block is scaled to 4 instead of 6, the grid of representable values (relative to the block's maximum) becomes: {0%, 12.5%, 25%, 37.5%, 50%, 75%, 100%} — compared to {0%, 8.3%, 16.7%, 25%, 33.3%, 50%, 66.7%, 100%} when scaled to 6. The key difference: with M=4, the value 3 represents 75% of the block maximum, filling the crucial gap that M=6 leaves between 66.7% (value 4) and 100% (value 6). The paper argues (Section 3) that other choices — M=2 or M=3 — "only offer subsets of values that can be represented with 4 or 6," meaning they don't add any new representable fractions beyond what M=4 and M=6 already cover. M=4 is the unique alternative that provides a new, useful representable point (75%) without completely collapsing the expressible range.

The tradeoff. Scaling to M=4 means the block can no longer represent values at 6 (or 100% of the maximum). The highest representable value is 4, which maps to 100% of the block's maximum. This means any value that was originally between 75% and 100% of the block maximum now gets squeezed into the [3, 4] range in FP4 space, where it can be represented at 3 (75%) or 4 (100%). The sacrifice of the [83.3%, 100%] representation that M=6 provides (via the value 6) is acceptable precisely when the block doesn't have many values in that top region — which, empirically, many blocks don't, as demonstrated by Table 1 and the worked example.

Worked example from Table 2. Consider the block $X = [10, 20, 30, 40]$. With M=6:

• $\Delta^{(6)} = \max(|X|) / 6 = 40 / 6 = 6.67$, rounded to FP8 E4M3 as $6.5$
• Scaled values: $[1.54, 3.08, 4.62, 6.15]$
• Rounded to FP4: $[1.5, 3, 4, 6]$
• Dequantized: $[9.75, 19.5, 26, 39]$
• MSE = $\frac{1}{4}[(9.75-10)^2 + (19.5-20)^2 + (26-30)^2 + (39-40)^2] = 4.33$

With M=4:

• $\Delta^{(4)} = 40 / 4 = 10$, which is exactly representable in FP8 E4M3 as $10$
• Scaled values: $[1, 2, 3, 4]$
• Rounded to FP4: $[1, 2, 3, 4]$ — note that 30/10 = 3 lands cleanly on an FP4 value, whereas with M=6 it landed on 4.62 and got rounded down to 4, losing 13.3%.
• Dequantized: $[10, 20, 30, 40]$
• MSE = 0

The block $[15, 30, 120, 180]$ shows the opposite case. With M=6, MSE=0 because the values 0.5, 1, 4, and 6 are all exactly representable. With M=4, the scaled values $[0.34, 0.68, 2.73, 4.09]$ produce MSE=96.25 because 0.34 rounds to 0.5 (overestimating 15 as 22) and 2.73 rounds to 3 (overestimating 120 as 136). The key difference is that this block has values at many different fractions of the maximum (8.3%, 16.7%, 66.7%, 100%), and M=6's grid aligns well with all of them, while M=4's grid aligns poorly with the low-end values because its smallest representable fraction (12.5%) is coarser than M=6's (8.3%).

The dual-path procedure. For each block of 16 values, the algorithm:

1. Compute $\Delta^{(6)}$: $\max(|X_{\text{block}}|) \div (\alpha \times 6)$, cast to FP8 E4M3.
2. Compute $\Delta^{(4)}$: $\max(|X_{\text{block}}|) \div (\alpha \times 4)$, cast to FP8 E4M3.
3. Quantize with M=6: for each value in the block, compute $\bar{X}^{(6)}_j = \text{fp4\_e2m1}(X_j / (\alpha \times \Delta^{(6)}))$ using Equation 3.
4. Quantize with M=4: for each value, compute $\bar{X}^{(4)}_j = \text{fp4\_e2m1}(X_j / (\alpha \times \Delta^{(4)}))$. Note that the same quantization function (Equation 3) works for M=4 because the piecewise branches depend on $|X/(\alpha\Delta)|$, not on $M_{\text{FP4}}$ — scaling the largest value to 4 simply means the third branch (step size 2) covers $(4, 4]$, which is an empty interval, so it's never invoked. The FP4 grid still has values at 4 and 6; they just won't be used.
5. Dequantize both candidates: $D^{(6)}_j = \bar{X}^{(6)}_j \times \Delta^{(6)} \times \alpha$, and similarly for $D^{(4)}_j$.
6. Compute error for each candidate: $E^{(6)} = \frac{1}{16} \sum_{j} (D^{(6)}_j - X_j)^2$, and similarly for $E^{(4)}$.
7. Select: if $E^{(4)} < E^{(6)}$, use $\Delta^{(4)}$ and $\bar{X}^{(4)}$; otherwise use $\Delta^{(6)}$ and $\bar{X}^{(6)}$.

Step 6 is where the choice of error metric matters, and the paper evaluates three candidates (Table 4): mean squared error (MSE), mean absolute error (MAE), and maximum absolute error (Abs-Max). For PTQ, MSE works best overall. For pre-training, MAE works better (Figure 5) — the paper does not deeply analyze why, but the Appendix A.1 experiments show the gap consistently across configurations.

The modified tensor-wide scale factor. A subtle but critical implementation detail: the standard $\alpha$ computation (Equation 1) uses $M_{\text{FP8}} = 448$ in the denominator. This ensures that for the block containing the tensor's largest value, $\Delta_i$ will be exactly 448, and the scaled largest value will be exactly 6. However, if this block were to select M=4 instead, its scale factor would need to be $448 \times 6/4 = 672$ to keep the largest value at 4 — but 672 exceeds the maximum representable FP8 E4M3 value of 448, causing overflow. The fix (Section 3.1, final paragraph): replace $M_{\text{FP8}}$ with 256 when computing $\alpha$:

$\alpha_{\text{FP32}} = \frac{\max(|X|)}{M_{\text{FP4}} \times 256}$

The value 256 is chosen because $256 \times 6/4 = 384$, which is the largest E4M3 value that, when multiplied by 1.5 (the ratio 6/4), stays within the 448 maximum. This slightly reduces the available range — the tensor-wide scale is now computed with a smaller denominator, meaning $\alpha$ is larger, which means per-block $\Delta_i$ values will generally be smaller on average — but the paper finds this has a negligible or slightly positive effect on model quality (Figure 6), likely because the affected blocks (those containing the tensor's absolute maximum) are few and the slightly reduced headroom is compensated by the ability to choose M=4.

---

Scale Selection Rules: Comparing Error Metrics

The scale selection rule — how the algorithm decides between the M=4 and M=6 candidates — is evaluated in three variants (Section 3.1, Table 4):

Mean Squared Error (MSE): $E = \frac{1}{16}\sum_{j=1}^{16} (D_j - X_j)^2$. This penalizes large individual errors quadratically — a single value with error 2 contributes 4 to the sum, while four values with error 0.5 each contribute only 1 total. This aligns with the paper's diagnostic that degradation comes from a few badly-quantized near-maximal values: MSE will strongly prefer the candidate that avoids those large errors. Table 4 shows MSE is the best rule for PTQ overall, producing the lowest WikiText-2 and C4 perplexities across the most model-metric combinations.

Mean Absolute Error (MAE): $E = \frac{1}{16}\sum_{j=1}^{16} |D_j - X_j|$. This penalizes errors linearly. In PTQ, MAE slightly underperforms MSE (Table 4), but in pre-training (Figure 5, Appendix A.1), MAE consistently outperforms MSE. The paper speculates that during training, the distribution of values within blocks is different (gradients especially have different statistics than static weights), and MAE's linear penalty may better preserve gradient signal that would be rounded to zero under MSE's quadratic penalty. However, the paper explicitly leaves the theoretical explanation to future work.

Maximum Absolute Error (Abs-Max): $E = \max_j |D_j - X_j|$. This only considers the single worst element in the block. Table 4 shows this is the weakest rule, often producing worse perplexity than standard NVFP4. The reason: a block might have one extreme outlier that dominates the max error metric, forcing the selection of a candidate that poorly represents the other 15 values. For instance, a block with 15 small values and one near-maximal value would see both candidates produce similar max error (on that one value) but very different errors on the small values — and the max-error rule would be blind to this difference.

The paper's final recommendation is MSE for PTQ and MAE for pre-training, with the caveat that the pre-training finding may be architecture- or optimizer-specific.

---

Efficient CUDA Implementation: Under 15% Overhead

The naive implementation of 4/6 would require quantizing each block twice (once for M=6, once for M=4), dequantizing both results, and computing errors — roughly tripling the cost of the quantization step compared to standard NVFP4. The paper achieves the claimed <15% overhead through careful use of GPU hardware features (Section 3.2).

Register-file residency. The key optimization is that all intermediate values — the original high-precision block elements, both sets of quantized FP4 values, both sets of dequantized values, the accumulated errors, and the two candidate scale factors — are kept entirely in the GPU's register file, never spilled to shared memory or global memory. A Blackwell GPU SM has 256 KB of register file (64K 32-bit registers), and a block of 16 FP16 values occupies only 32 bytes — the working set for 4/6 is tiny, fitting comfortably in registers.

PTX cvt instructions for FP4 conversion. NVIDIA Blackwell GPUs provide PTX (Parallel Thread Execution, the GPU's intermediate assembly language) instructions that directly convert between FP16 and the packed FP4 format. The cvt family of instructions handles the piecewise rounding logic of Equation 3 in hardware, executing in a single cycle per value. The paper explicitly states: "we use the cvt family of instructions to perform quantization into the packed FP4 format, and then also dequantization from FP4 to FP16, which is needed to calculate error." This means both the forward quantization (BF16 → FP4, lines 4 and 11 in Table 2's pseudocode) and the inverse dequantization (FP4 → BF16, lines 6 and 12) are single-instruction operations.

Fused error accumulation. The per-element error computation ($(D_j - X_j)^2$ or $|D_j - X_j|$) uses standard FP16 fused multiply-add instructions (FFMA) and accumulates into an FP32 register to avoid precision loss. The reduction across 16 elements (the summation in line 6/12 of Table 2) is a simple loop with 16 iterations, fully unrolled by the compiler.

What the <15% covers. The overhead is measured relative to the standard NVFP4 quantization kernel, which already includes computing $\Delta_i$, performing the per-value FP4 cast, and packing values into the memory format expected by the GEMM unit. The additional work in 4/6 is: computing $\Delta^{(4)}$ (one extra FP8 E4M3 cast), performing a second set of 16 FP4 casts, performing two sets of 16 FP4 dequantizations, computing 16 error terms, and accumulating them. Since the cvt operations are single-cycle and the arithmetic is simple, the total additional latency is modest compared to the memory access and scale factor computation that both kernels share.

Acknowledged optimization headroom. The paper states "we expect that we will be able to reduce this overhead further with more optimization," suggesting the <15% is an initial implementation, not a fundamental lower bound. Potential further optimizations include fusing the error computation with the dequantization step (avoiding a separate pass over the data) and using vectorized cvt instructions where available.

---

Integration into NVFP4 Pre-Training

The pre-training integration (Section 4.1, Appendix A) embeds 4/6 into the full NVFP4 training pipeline illustrated in Figure 3. Understanding this pipeline requires tracing the data flow through a single linear layer during a training step.

The standard NVFP4 training pipeline (without 4/6). Referring to Figure 3:

1. Forward pass (FPROP): BF16 activations from the previous layer are quantized to NVFP4 using the standard algorithm (Equations 1–3 with stochastic rounding for activation quantization — though the paper uses round-to-nearest for activations). FP32 weights (stored in high precision in the optimizer) are quantized to NVFP4 using 2D block quantization. The two NVFP4 tensors are multiplied in the NVFP4 GEMM unit, accumulating in FP32 and outputting BF16 activations to the next layer.

2. Weight gradient computation (WGRAD): The backward pass first computes the gradient of the loss with respect to the layer's output (incoming from layer i+1 in BF16). This gradient tensor and the forward-pass activations (reloaded or recomputed) are both inputs to WGRAD. Before the GEMM, the gradient tensor undergoes a fused transpose + random Hadamard transform (RHT) + quantization to NVFP4 with stochastic rounding. The activation tensor undergoes transpose + quantization to NVFP4 with stochastic rounding. The NVFP4 GEMM computes the weight gradient, accumulating in FP32, and the result is used by the optimizer to update the FP32 weights.

3. Data gradient computation (DGRAD): The incoming gradient (from layer i+1) and the FP32 weights (quantized to NVFP4) are multiplied in the NVFP4 GEMM to compute the gradient with respect to the layer's input, which is passed backward to layer i-1 in BF16.

What 4/6 changes in this pipeline. The paper evaluates 4/6 applied to different subsets of tensors (Appendix A.4, Figure 8):

• Activations only (the recommended configuration): 4/6 is used when quantizing activations in FPROP, WGRAD, and DGRAD. Weights and gradients use standard NVFP4 quantization (with stochastic rounding for gradients). This achieves training loss comparable to applying 4/6 everywhere (Figure 8), with lower overhead.
• Activations + weights: adds 4/6 to weight quantization as well, but Figure 8 shows this provides no additional benefit over activations-only.
• Activations + gradients: adds 4/6 to gradient quantization, but this underperforms activations-only.
• All three: comparable to activations-only, but with more overhead.

The surprising finding is that weights and gradients don't benefit from 4/6. The paper speculates that this may be because weight values, being updated slowly by the optimizer, have different distributional properties than the rapidly-varying activations, and gradient tensors — which are noisy and already receive stochastic rounding treatment — may not have the same concentration of near-maximal values that makes 4/6 useful.

Stochastic rounding interaction (Appendix A.3). A subtle compatibility issue arises when 4/6 is applied to gradient quantization. Stochastic rounding (SR) works by randomly rounding a value up or down with probability proportional to its distance to each neighbor, producing an unbiased estimator of the original value. However, when 4/6 selects between M=4 and M=6 candidates based on which has lower quantization error, it introduces bias: the selection procedure systematically favors the candidate that happens to land closer to the original values, which is exactly the kind of systematic error that SR is designed to eliminate.

The paper evaluates three approaches:

1. Biased 4/6 (the default): apply 4/6 directly to the stochastically rounded values, accepting the bias.
2. No 4/6 on gradients: use standard NVFP4 with SR, no block-scale selection.
3. Unbiased 4/6: select M=4 or M=6 using round-to-nearest quantization error, then independently apply SR with the selected scale.

Figure 7 shows that the biased approach (option 1) works best, slightly outperforming both alternatives. The paper hypothesizes: "the benefits provided by the reduced quantization error outweighs those provided by the unbiased nature of stochastic rounding." In other words, the variance reduction from 4/6's better representation of near-maximal values is larger than the bias introduced by the selection procedure, at least for the gradient distributions encountered during training of this model architecture.

Scale selection rule for pre-training (Appendix A.1, Figure 5): MAE outperforms MSE during pre-training, contrary to the PTQ finding. The paper evaluates this in two configurations — 4/6 on activations and weights only, and 4/6 on all three tensors — and MAE is better in both. The difference is small but consistent, appearing from early in training (around 100B tokens) and persisting through 1T tokens.

Reduced FP32 tensor scale for pre-training (Appendix A.2, Figure 6): the modification described in Section 3.1 (using 256 instead of 448 in the $\alpha$ computation) provides a small but measurable training loss improvement. The paper evaluates this by comparing 4/6 on activations with and without the scale adjustment, finding the adjusted version consistently gives marginally lower loss. Since this adjustment only affects blocks containing the tensor's absolute maximum — a tiny fraction of all blocks — the benefit likely comes from those specific high-magnitude activation values (which can disproportionately influence model behavior) receiving the option of M=4 quantization.

Other pre-training configuration details (Section 4.1): the model is Nemotron 3 Nano 30B-A3B, a Mixture-of-Experts hybrid Mamba-Transformer architecture, trained on 1 trillion tokens of curated and synthetic data. The optimizer is AdamW with $\beta_1 = 0.9$, $\beta_2 = 0.95$, weight decay 0.1, gradient clipping at 1.0, sequence length 8192, and global batch size 3072. The learning rate schedule is Warmup-Stable-Decay with a constant learning rate of $10^{-3}$ decaying to $10^{-5}$ over the last 20% of training. All runs use 384 NVIDIA B200 GPUs. Attention components, the output projection head, normalization layers, non-linearities, and the output projection layer of Mamba-2 blocks are kept in high precision (BF16 or MXFP8). These choices directly follow the NVIDIA NVFP4 pre-training recipe (the paper cites the concurrent NVIDIA NVFP4 pre-training paper as its baseline).

The 13.0% closer-to-BF16 claim (Figure 4): the training loss curve for NVFP4 with 4/6 (on activations, MAE selection, reduced tensor scale) sits between the standard NVFP4 curve and the BF16 curve. The figure's bottom subplot shows the relative difference from BF16 as a percentage. The paper quantifies the improvement as "training loss 13.0% closer to the BF16 baseline," meaning that the gap between NVFP4 and BF16 loss is reduced by 13% when 4/6 is added. This is computed from the loss values after 1T tokens of training. It's an incremental but non-trivial improvement: for a training run costing thousands of GPU-hours, a 13% reduction in the precision gap, achieved with <15% per-step overhead, is a favorable cost-benefit tradeoff.

---

Integration into Post-Training Quantization

The PTQ integration (Section 4.2) applies 4/6 to already-trained models where only weights and activations need to be quantized (inference setting). This is architecturally simpler than the training case because there are no gradients to quantize and no stochastic rounding to worry about.

4/6 as a drop-in quantization backend. The standard PTQ pipeline works as follows: a calibration dataset (typically 128 samples from the training distribution) is passed through the model to collect activation statistics. These statistics inform how weights and activations should be quantized. The paper's approach is to swap out the quantization function: wherever the PTQ method would normally call standard NVFP4 quantization (Equations 1–3), it instead calls the 4/6 dual-path quantization procedure. The PTQ method's own logic (per-channel smoothing, Hessian-based weight optimization, etc.) operates as usual — it only sees the final quantized weights, not the internal M=4/M=6 selection.

Round-to-nearest (RTN) with 4/6 is the simplest case: no calibration or optimization, just direct quantization of each weight matrix and, at inference time, each activation tensor using the 4/6 procedure with MSE selection. Table 5 shows this improves WikiText-2 perplexity over standard NVFP4 RTN for all models tested (Llama-3 1B, 8B, 70B; Qwen-3 1.7B, 8B, 32B), with improvements ranging from 0.01 to 0.44 perplexity points.

AWQ with 4/6. AWQ (Activation-aware Weight Quantization) works by identifying the salient weight channels that correspond to large activation magnitudes, and scaling those weights up (with a compensating inverse scaling of activations) to reduce quantization error on the most impactful values. When combined with 4/6, the AWQ scaling and the NVFP4 block scaling operate at different granularities: AWQ operates per-channel (across the whole weight matrix), while NVFP4 block scaling operates per-block-of-16. After AWQ applies its per-channel scaling, the resulting weight matrix is quantized using 4/6. Table 5 shows AWQ + 4/6 achieves the best overall perplexity: average WikiText-2 PPL of 11.58 and average C4 PPL of 32.36 across all models. 4/6 improves AWQ for every model and every metric tested — there is no case where standard AWQ without 4/6 is better. The average improvement brings perplexity 19.9% closer to BF16.

SmoothQuant with 4/6. SmoothQuant applies per-channel smoothing to both weights and activations: it divides each activation channel by a smoothing factor and multiplies the corresponding weight channel by the same factor, transferring quantization difficulty from activations to weights. When combined with 4/6, the smoothed tensors are then quantized using the dual-path procedure. The improvements are consistent but smaller than with AWQ: 4/6 brings WikiText-2 perplexity 5.3% closer to BF16 on average (Table 5). The difference likely reflects that SmoothQuant's smoothing already addresses some of the distributional skew that causes large quantization errors — so the gap that 4/6 can close is narrower.

GPTQ with 4/6: a mixed result. GPTQ uses second-order Hessian information to optimize which weight values get rounded up vs. down, compensating quantization error across columns. Unlike AWQ and SmoothQuant, which modify the values before quantization, GPTQ modifies the quantization decisions (the rounding direction) based on a global optimization. The paper applies 4/6 as a direct replacement of the quantization function within GPTQ — after GPTQ's Hessian-based optimization selects rounding directions, the weights are quantized with 4/6 instead of standard NVFP4. The results are mixed (Table 5): for Llama-3 1B and 8B, GPTQ + 4/6 improves over GPTQ alone; for Qwen-3 models, it sometimes degrades performance. The paper acknowledges this explicitly: "Modifying the GPTQ optimization process in a way that incorporates Four Over Six is likely to deliver performance improvements in future work" (Section 4.2). The issue is that GPTQ's optimization assumes the quantization grid is fixed and optimizes around it; introducing a per-block adaptive grid choice (M=4 vs. M=6) means the optimization objective changes during the process, and the Hessian-based compensation no longer targets the right error sources. A proper integration would interleave the block-scale selection with the Hessian optimization, which is non-trivial but left as future work.

Downstream task evaluation (Tables 6 and 7) extends beyond perplexity to four established benchmarks: BoolQ (yes/no question answering), ARC-Easy and ARC-Challenge (multiple-choice science reasoning), and HellaSwag (commonsense sentence completion). Normalized accuracy is used for ARC and HellaSwag to control for tokenization differences across models. The results mirror the perplexity findings: 4/6 improves average task performance for nearly every combination of model, PTQ method, and task. For instance, AWQ + 4/6 on Llama-3.1-8B improves average accuracy from 72.2 to 73.1 (Table 6), bringing it closer to the BF16 baseline of 75.0. The consistency across tasks — BoolQ, ARC-E, ARC-C, and HellaSwag measure qualitatively different capabilities — suggests 4/6's benefit is not task-specific but comes from a general improvement in model fidelity after quantization.

---

Why 4/6 Composes with Existing Methods

A key architectural property of 4/6 is its orthogonality to other quantization techniques. This is not accidental — it follows from the abstraction boundary that 4/6 modifies:

• Per-channel scaling methods (AWQ, SmoothQuant) operate on the values before quantization. They multiply channels by scaling factors to reshape the distribution. 4/6 operates on the quantization grid itself — how the FP4 representable values are mapped to the scaled values. These are sequential operations: first redistribute values (AWQ/SmoothQuant), then quantize them (4/6). They don't interfere because they modify different aspects of the pipeline.

• Rotation methods (QuaRot, SpinQuant, the RHT used in pre-training) apply orthogonal transforms to decorrelate dimensions and spread outliers. This changes which values appear in which blocks but doesn't change the block-quantization problem: after rotation, each block still has a largest value that gets scaled to the grid maximum, and values near that maximum still fall in the coarse FP4 regions. 4/6 applies after the rotation, addressing the same grid-gap problem on the transformed values.

• Stochastic rounding changes the rounding direction (probabilistic rather than nearest-neighbor) but not the set of representable values. 4/6 changes the set of representable values (by choosing a different grid scaling) but not the rounding direction per se. As Appendix A.3 shows, they interact — the selection bias from 4/6 conflicts with SR's unbiasedness goal — but the empirical result is that the net effect is still positive.

• 2D block quantization (used in pre-training) partitions weight matrices into 2D tiles rather than 1D strips for computing scale factors. This is a reorganization of which 16 values form a block, not a change to the quantization within a block. 4/6 applies identically regardless of how blocks are formed.

This composability is the paper's strongest architectural argument: 4/6 is not a competitor to existing PTQ or training methods but a complement that addresses a specific, previously unexploited source of error — the FP4 grid gap — that other methods leave untouched.

---

Design Choice: Why Adaptive Per-Block Rather Than Global M=4

A natural question is why the paper chooses adaptive per-block selection rather than simply scaling all blocks to M=4. Table 3 provides the empirical answer: scaling all blocks to M=4 yields worse perplexity than standard NVFP4 (M=6) for every model tested. The reason, explained in Section 3, is that M=4 sacrifices 50% of the representable range: with M=6, a block can represent values from 0.5× to 6× its scale factor (a ratio of 12:1); with M=4, the range is 0.5× to 4× (a ratio of 8:1). Many blocks have values that span a wide enough range to need the full M=6 headroom — forcing them to M=4 would cause underflow-to-zero for their largest values. The adaptive approach lets each block choose the range that fits its internal value distribution.

This is fundamentally different from uniform scaling decisions. In standard quantization, the choice of $M_{\text{FP4}}$ is a global constant (6 for NVFP4) because the hardware expects a single grid for all blocks — the FP4 values {0.5, 1, 1.5, 2, 3, 4, 6} are hardwired. 4/6 doesn't change the grid values; it changes how those values map to the original tensor, block by block. The hardware still sees standard FP4 values in memory; the M=4 decision is encoded in the scale factor $\Delta_i$, which has been made 1.5× larger than it would be for M=6. During dequantization, $\Delta_i \times \bar{X}_j$ reconstructs the value correctly regardless of which path was taken. This is what makes 4/6 a transparent modification — downstream consumers of the NVFP4 tensor (the GEMM unit, the optimizer) don't need to know whether a block used M=4 or M=6.

---

Summary of Design Choices and Their Justifications

• M=4 as the alternative (rather than M=2, M=3, M=5): the FP4 grid at M=4 provides a new representable fraction (75%) that M=6 lacks, filling the critical gap between 66.7% and 100%. Other values of M would only produce grids that are subsets of what M=4 and M=6 jointly cover.
• Per-block adaptive selection (rather than global M=4 or M=6): the optimal scaling depends on each block's internal value distribution. Blocks with values near 75% of maximum benefit from M=4; blocks with values spanning the full range or clustering near 100% benefit from M=6. Adaptivity captures the best of both.
• MSE for PTQ, MAE for pre-training: MSE's quadratic penalty aligns with the diagnostic that a few large errors dominate model degradation in static weights; MAE's linear penalty empirically works better for training-time tensors, possibly due to different distributional properties or interaction with stochastic rounding.
• Reduced tensor scale ($M_{\text{FP8}} = 256$): necessary to prevent FP8 overflow when blocks containing the tensor maximum choose M=4. The value 256 ensures $256 \times 6/4 = 384 \leq 448$.
• Register-resident dual-path execution: achieves <15% overhead by avoiding memory round-trips for the second quantization path, using hardware cvt instructions for fast FP4/FP16 conversion.
• Integration with existing methods via quantization function substitution: 4/6 changes only the mapping from scaled values to FP4; all upstream distribution-shaping (AWQ, SmoothQuant, RHT, 2D blocking) and downstream consumption (GEMM, optimizer) remain unchanged.

4. Key Insights and Innovations

Innovation 1: The Diagnostic Reframing — "Where Does the Error Actually Come From?" as a Methodological Contribution

The paper's most distinctive intellectual move is not the 4/6 algorithm itself, but the diagnostic framework that motivated it. Prior work on low-precision FP4 quantization treated the problem holistically: models degrade when quantized to NVFP4, so the response was to develop methods that make models more "quantization-friendly" — suppressing outliers via rotations (QuaRot, SpinQuant), smoothing activation distributions (SmoothQuant), or optimizing rounding decisions with second-order information (GPTQ). These are all techniques that modify what gets quantized — the values themselves.

This paper does something fundamentally different. It asks a question that, remarkably, no prior NVFP4 work had systematically investigated: which specific values in the FP4 grid cause the degradation, and what is the mechanism? The answer, revealed by the partial quantization simulation in Figure 2b, is that the degradation is not diffuse — it is sharply concentrated on values that fall in FP4's coarse regions, primarily the gap between representable values 4 and 6 where the step size jumps to 2. This is not a statement about outliers or dynamic range (the problems that block scaling and smoothing solve); it's a statement about the within-block alignment between a block's internal value distribution and the fixed FP4 representable grid. The paper's diagnostic decomposition in Figure 2a — showing that scale factor quantization error is negligible while FP4 value error accounts for essentially all degradation — further isolates the problem to a single, addressable source.

This diagnostic reframing is significant beyond the 4/6 solution because it changes how the field should think about FP4 quantization. The dominant assumption in quantization research — codified in the foundational white paper by Nagel et al. (2021) — is that you should scale values to the full range of the target format to maximize precision. This paper provides empirical evidence that, for floating point formats with non-uniform step sizes, that assumption is exactly backwards for a substantial fraction of blocks: using the full range creates the dead zone (66.7%–100% of the block maximum has no representable values) that causes the worst errors. The diagnostic is a conceptual contribution independent of the solution — it tells future researchers where to look when improving FP4 quantization, regardless of whether they adopt 4/6 or develop alternative approaches.

Evidence: The steep slope at threshold x=5 in Figure 2b, and the complete performance recovery when values are kept in high precision (Figure 2a), provide clean, unambiguous support for the claim that near-maximal FP4 values are the bottleneck. This is not a correlation; it's a controlled causal decomposition.

Innovation 2: Recognizing That Per-Block Scale Selection Is a "Free" Degree of Freedom That Prior Work Ignored

The second conceptual innovation is the recognition that the per-block scale factor $\Delta_i$ — which exists in every block-scaled FP4 format by design — can encode not just the scale of the block's values but also a choice of which subset of the FP4 grid to use. In standard NVFP4 quantization, $\Delta_i$ is computed deterministically from the block's maximum value: it is set so that the maximum lands at 6, the top of the FP4 range. This is treated as a constraint — "you must use the full FP4 range for every block" — rather than a design choice.

The paper's insight is that this constraint is self-imposed. The FP4 grid itself has values at {0, 0.5, 1, 1.5, 2, 3, 4, 6} regardless of $\Delta_i$. But the mapping from original values to FP4 values depends on $\Delta_i$: a larger $\Delta_i$ compresses the original values into a smaller portion of the grid, effectively selecting which representable fractions of the block's maximum value are available. With $\Delta_i$ set for M=6, the representable fractions are {8.3%, 16.7%, 25%, 33.3%, 50%, 66.7%, 100%}. With $\Delta_i$ set 1.5× larger (for M=4), the fractions become {12.5%, 25%, 37.5%, 50%, 75%, 100%}. The grid values {0.5, 1, 1.5, 2, 3, 4, 6} haven't changed — but the fractions of the block maximum that they represent have shifted, because the denominator (the block's maximum relative to $\Delta_i$) has changed.

This is a genuinely novel perspective on block-scaled quantization. Prior work treated $\Delta_i$ as a fixed function of the block's statistics (specifically, its maximum). This paper treats it as a selectable parameter that trades off range (the ability to represent values near 100% of the block maximum) against granularity (the ability to represent values near 75% of the block maximum). The key enabling condition — that the scale factor has enough precision to represent a 1.5× increase — is satisfied by NVFP4's FP8 E4M3 scale factors but not by MXFP4's E8M0, which explains why this degree of freedom went unexplored: MXFP4, with its powers-of-two scale factors, cannot express the M=4 alternative without jumping by a full factor of 2.

This reframing is significant because it opens a new axis for optimization in block-scaled formats. Instead of treating the scale factor as a deterministic function of the block maximum, future work could explore learned or optimized scale factor selection that considers the entire block distribution, not just the maximum. The 4/6 dual-choice (M=4 or M=6) is the simplest possible instantiation of this idea; more sophisticated selection policies (continuous optimization of the scale factor, or selection among more than two grid alignments) become natural extensions once the constraint is recognized as artificial.

Evidence: The worked example in Table 2 demonstrates the mechanism concretely: the same block quantized with $\Delta^{(6)}$ and $\Delta^{(4)}$ produces different FP4 values for the same original inputs, and the reconstruction error differs dramatically (MSE 4.33 vs. 0 for the first example). Table 3 shows that naively applying M=4 globally is worse than M=6, confirming that the per-block choice — not the M=4 mapping itself — is what provides the benefit.

Innovation 3: The Composability Argument — 4/6 as a Quantization-Function Modification, Not a Competing Method

The paper's third intellectual contribution is an architectural argument about where in the quantization stack improvements should be made. The field has developed a rich ecosystem of PTQ and training methods — AWQ, GPTQ, SmoothQuant, QuaRot, SpinQuant, RHT, stochastic rounding, 2D block quantization — each addressing different aspects of the low-precision problem. A new method that competes with these would need to outperform them, which is a high bar. The paper's strategy is different: position 4/6 as a modification to the quantization function itself — the final step that maps scaled values to FP4 — which is a shared substrate that all existing methods ultimately invoke.

The evidence for this claim is the comprehensive integration results in Tables 5, 6, and 7, which show that 4/6 provides additive improvements when combined with AWQ (+19.9% closer to BF16 perplexity on average), SmoothQuant (+5.3%), and RTN. Even with GPTQ, where the composition is more complex and results are mixed, the improvements for Llama models suggest that proper integration could extend the benefit. This is not just a practical convenience — it's an argument that the FP4 grid gap is a fundamental source of error that no existing method addresses, because they all operate upstream of the quantizer.

This composability argument has intellectual significance beyond the specific method: it suggests a layered architecture for quantization research, analogous to the abstraction layers in compilers or networking stacks. Methods that modify value distributions (AWQ, SmoothQuant) operate at the "data" layer. Methods that modify block formation (2D blocking, rotation-based decorrelation) operate at the "partitioning" layer. Methods that modify rounding (stochastic rounding, GPTQ's Hessian-based optimization) operate at the "discretization" layer. And 4/6 operates at the "grid alignment" layer — a previously unrecognized layer that sits between discretization and the hardware grid. Recognizing this layer as a distinct optimization target is a conceptual contribution that could guide future format and algorithm design.

Evidence: The consistent improvement when 4/6 is added to AWQ and SmoothQuant across all six models and two perplexity benchmarks (Table 5), and across four downstream tasks (Tables 6, 7), demonstrates that the benefit is additive and robust — it does not depend on specific properties of the PTQ method being combined with.

Innovation 4: The Pre-Training Finding That Activation Quantization Is the Bottleneck — and the Implication for Where to Invest Overhead

The pre-training experiments in Section 4.1 and Appendix A.4 yield a finding that is both surprising and practically important: applying 4/6 only to activation quantization achieves training loss comparable to applying it to all three tensor types (activations, weights, and gradients), as shown in Figure 8. This is counterintuitive because the diagnostic in Figure 2b was performed on weights during PTQ, not on activations during training, and there was no a priori reason to expect that activations would be the primary beneficiary.

This finding has two layers of significance. First, it provides practical guidance: the overhead of 4/6 can be concentrated on a single tensor type (activations), reducing the total per-step overhead to well under the already-low 15% figure, since only one of the three quantization operations in each training step (FPROP activation quantization, WGRAD activation quantization, DGRAD activation quantization) incurs the dual-path cost. Second, it suggests a deeper hypothesis about the dynamics of low-precision training: activation quantization error, rather than weight or gradient quantization error, may be the dominant source of training instability in FP4 regimes. This aligns with the observation that activations vary rapidly during training (they are a function of the current input and all upstream weights), while weights change slowly (via optimizer updates) and gradients are noisy by nature. The rapidly-varying statistics of activations may mean they encounter the FP4 grid gap more frequently or more severely than the relatively stable weight distributions.

This finding also connects to the broader observation in the NVFP4 training literature that certain layers (attention, normalization, output heads) must be kept in high precision. The paper's result suggests that even within the layers that are quantized, not all tensors benefit equally from improved quantization — the bottleneck is asymmetric. This has implications for future hardware-software co-design: if activation quantization is the primary bottleneck, accelerator designers might consider asymmetric precision support (e.g., higher-precision activation pathways while keeping weight storage at FP4), and training recipe designers might invest more heavily in activation-specific improvements.

Evidence: Figure 8 shows that the training loss curve for "4/6 on X only" (activations) tracks very close to "4/6 on X, W, and G" (all tensors), while "4/6 on X and W" underperforms and "4/6 on X and G" underperforms further. The gap between the BF16 baseline and NVFP4 baseline is approximately 0.75%, and 4/6 on activations alone recovers a significant fraction of that gap with minimal overhead.

Innovation 5: The MAE vs. MSE Reversal Between PTQ and Pre-Training as a Signal About Error Metric Design

The paper's observation that the optimal scale selection metric differs between PTQ (where MSE works best, Table 4) and pre-training (where MAE works best, Figure 5) is a small empirical finding with potentially large implications. The standard assumption in quantization research is that minimizing MSE is the correct objective — it aligns with the L2 loss used in training, it penalizes large errors more heavily (which aligns with the intuition that outliers matter more), and it has convenient mathematical properties (differentiability, connection to variance). The fact that this assumption breaks during training suggests that the purpose of quantization error minimization changes between static inference and dynamic training.

During inference, quantized weights are fixed — the only goal is to reconstruct them as accurately as possible, and MSE is the natural metric because large per-value errors translate directly to activation errors in the forward pass. During training, quantized values (especially gradients and activations) serve as signals in an optimization process. A small number of large quantization errors on gradient values, which MSE penalizes heavily, might be less harmful than a larger number of moderate errors that systematically bias the gradient direction — and MAE's linear penalty might better preserve the overall gradient signal distribution. The paper does not explore this mechanism in depth (it's left as future work), but the empirical reversal itself serves as a warning: error metrics that work for PTQ may not transfer to training, and the choice of metric deserves careful empirical validation rather than being assumed.

This finding also has implications for the design of future quantization-aware training methods. If the optimal error metric for quantization is context-dependent (MSE for static tensors, MAE for dynamic tensors, potentially other metrics for other contexts), then the quantization function itself should be parameterized by the error metric — a design principle that 4/6 naturally supports since the error computation in the dual-path procedure (steps 6–7 in Section 3.4) is a modular component that can be swapped.

Evidence: Figure 5 shows two configurations (4/6 on X and W, 4/6 on X, W, and G), and in both cases the MAE curve sits closer to BF16 than the MSE curve. Table 4 shows the opposite for PTQ: MSE produces lower perplexity in 9 of 12 model-benchmark combinations compared to MAE. The reversal is consistent enough across configurations to be unlikely to be noise, but the paper appropriately treats the explanation as speculative and open to future investigation.

5. Experimental Analysis

Evaluation Methodology

• Dataset. Post-training quantization experiments use WikiText-2 and C4 for word perplexity evaluation, and BoolQ, ARC-Easy, ARC-Challenge, and HellaSwag for downstream task evaluation. Pretraining experiments use a "high-quality curated and synthetic data distribution of 1 trillion tokens based on [NVIDIA Nemotron Nano 2]" (Section 4.1). The 500-question MATH test set from Lightman et al. (2022) is not used in this paper — the paper's domain is language modeling, not mathematical reasoning.

• Base model(s). PTQ experiments evaluate Llama 3 models at scales 1B, 8B, and 70B (specifically Llama-3.2-1B and Llama-3.1-8B for downstream tasks; the 70B model appears only in perplexity tables), and Qwen 3 models at scales 1.7B, 8B, and 32B (specifically Qwen3-1.7B and Qwen3-8B for downstream tasks). These represent two different model families from different organizations, providing cross-architecture validation. Pre-training experiments use the Nemotron 3 Nano 30B-A3B architecture — a Mixture-of-Experts hybrid Mamba-Transformer model. The scale range (1B to 70B for PTQ; 30B-A3B for pre-training) covers both small and moderately large models, though no >70B models are evaluated.

• Metrics. For PTQ, the primary metrics are WikiText-2 word perplexity and C4 word perplexity (both lower is better), chosen because "perplexity is often considered a more stable metric for evaluating quantized models" (Section 4.2). Downstream evaluation uses normalized accuracy on BoolQ, ARC-Easy, ARC-Challenge, and HellaSwag, with normalization for ARC and HellaSwag "in order to reduce differences due to tokenization when comparing across different models" (Section 4.2). For pre-training, the metric is training loss, measured continuously throughout training, with the gap to BF16 expressed as a relative percentage difference. No downstream task evaluation is reported for the pre-trained models — the paper only reports training loss curves.

• Baselines. PTQ baselines include: RTN (round-to-nearest quantization, the simplest possible NVFP4 quantization with no calibration), GPTQ (Frantar et al., 2023) as implemented in the FP-Quant library (Egiazarian et al., 2025), AWQ (Lin et al., 2024), and SmoothQuant (Xiao et al., 2024). Each is evaluated both with standard NVFP4 quantization and with 4/6 replacing the quantization function. The high-precision baseline is BF16 (unquantized). For pre-training, the baselines are BF16 (all operations in high precision) and NVFP4 (the current state-of-the-art NVFP4 pre-training recipe from NVIDIA, including stochastic rounding, random Hadamard transforms, 2D block quantization, and keeping sensitive layers in high precision, as described in Section 4.1). MXFP4 is evaluated separately in Table 3 as a format-level comparison.

• Generation budget / compute accounting. For PTQ, there is no generation budget — weights are quantized once offline using a calibration dataset, and inference perplexity is measured. The "compute" metric is the quantization algorithm's per-step cost, measured as overhead percentage relative to standard NVFP4 quantization. The paper reports that 4/6 adds "under 15%" overhead (Section 3.2). For pre-training, compute is measured in tokens processed (1 trillion total), with training loss compared at matched token counts. All NVFP4 training runs use 384 NVIDIA B200 GPUs (Section 4.1), making this a large-scale training experiment, not a simulation.

• Cross-validation / statistical protocol. The paper does not report any cross-validation, statistical significance tests, or confidence intervals. Perplexity results (Tables 4, 5) and downstream task results (Tables 6, 7) are reported as point estimates without error bars. Training loss curves (Figures 4, 5, 6, 7, 8) show single runs with no indication of whether multiple seeds were evaluated. This is a genuine weakness: for downstream tasks on relatively small evaluation sets (BoolQ, ARC, HellaSwag), the absence of statistical testing means the small accuracy differences — often 0.3–0.9 percentage points (Table 6) — could be within noise, and the claim that "4/6 improves average task performance in nearly all cases" rests on interpreting these small differences as signal. The PTQ experiments do evaluate across six different model sizes, providing some replication across architectures, but within each model-benchmark combination, only a single evaluation is reported.

Main Quantitative Results

Post-Training Quantization: Perplexity

The core PTQ result is that 4/6 improves WikiText-2 and C4 perplexity when combined with most PTQ methods across all tested models, with the strongest and most consistent improvements coming from the AWQ + 4/6 combination. Table 5 reports these results in full, and the paper summarizes: "AWQ with 4/6 performs best overall, with an average WikiText-2 word perplexity of 11.58 and an average C4 word perplexity of 32.36 across all models" (Section 4.2).

Baseline NVFP4 gap. Standard NVFP4 round-to-nearest (RTN) quantization introduces a substantial perplexity degradation relative to BF16. For Llama-3.1-8B on WikiText-2, BF16 achieves 7.54 while RTN NVFP4 achieves 8.43 — a degradation of 0.89 perplexity points. For Qwen3-8B, the gap is 12.22 (BF16) to 12.68 (RTN NVFP4). For the largest models, Llama-3-70B degrades from 2.86 to 4.00, and Qwen3-32B degrades from 9.34 to 9.85. These gaps establish the problem that 4/6 aims to address.

4/6 with RTN. Adding 4/6 to RTN quantization reduces perplexity for all six models on WikiText-2, with improvements ranging from 0.01 (Qwen3-32B: 9.85 → 9.84) to 0.43 (Llama-3.1-1B: 14.27 → 13.84). On C4, the improvements are larger in absolute terms: Llama-3.1-1B improves from 36.19 to 35.09 (a 1.10 reduction), Llama-3.1-8B improves from 20.83 to 20.48 (a 0.35 reduction), and Llama-3-70B improves from 14.16 to 13.95 (a 0.21 reduction). However, for Qwen3 models on C4, the improvement is inconsistent: Qwen3-1.7B degrades from 65.54 to 66.32 (worsens by 0.78), and Qwen3-32B degrades from 27.54 to 27.67 (worsens by 0.13). The paper does not comment on these specific degradations, and they contradict the claimed uniform improvement.

4/6 with AWQ: the strongest combination. AWQ + 4/6 achieves the best overall perplexity across the tested configurations. On WikiText-2, it improves over standard AWQ for every model: Llama-3.1-1B (14.04 → 13.67, a reduction of 0.37), Llama-3.1-8B (8.33 → 8.24, reduction of 0.09), Llama-3-70B (3.86 → 3.71, reduction of 0.15), Qwen3-1.7B (22.20 → 21.67, reduction of 0.53), Qwen3-8B (12.68 → 12.57, reduction of 0.11), and Qwen3-32B (9.69 → 9.64, reduction of 0.05). The paper quantifies this as bringing WikiText-2 perplexity "19.9% closer to BF16 model performance" — meaning that across models, the gap between AWQ NVFP4 and BF16 shrinks by approximately one-fifth when 4/6 is added.

On C4, the pattern is the same: consistent improvement for all six models, with Llama-3-70B improving from 13.58 to 13.41 and Qwen3-1.7B improving from 62.50 to 61.78. The paper reports that AWQ + 4/6 brings C4 perplexity closer to BF16, though the specific percentage is not stated.

4/6 with SmoothQuant: consistent but smaller improvements. SmoothQuant + 4/6 improves over standard SmoothQuant on WikiText-2 for all six models, with improvements ranging from 0.01 (Qwen3-32B: 9.65 → 9.63) to 0.17 (Llama-3.1-1B: 14.17 → 14.03, though here SmoothQuant is slightly worse than RTN at 14.27 — a detail the paper doesn't discuss). The paper reports the improvement as bringing WikiText-2 perplexity "5.3% closer to BF16 model performance" — notably smaller than the 19.9% figure for AWQ. This is attributed to SmoothQuant's smoothing already addressing distributional issues that cause some of the same errors 4/6 mitigates, leaving a narrower gap to close.

4/6 with GPTQ: mixed results. GPTQ + 4/6 improves over standard GPTQ for Llama models on WikiText-2: Llama-3.2-1B (13.73 → 13.67) and Llama-3.1-8B (8.33 → 8.30). On C4, the improvements are marginal: Llama-3.1-1B (35.65 → 35.55) and Llama-3.1-8B (20.98 → 20.89). For Qwen3 models, the picture is more negative: Qwen3-1.7B on WikiText-2 degrades from 21.48 to 22.70 (a substantial 1.22 perplexity increase), and Qwen3-32B on C4 degrades from 27.17 to 27.09 (a small change, but in the wrong direction). The paper acknowledges: "GPTQ with 4/6 reduces the performance of models quantized with GPTQ, increasing the gap between NVFP4 and BF16 word perplexity by an average of 34.6%" (Section 4.2). This is the most notable negative result, and the paper attributes it to the fact that GPTQ's Hessian-based optimization assumes a fixed quantization grid — "Modifying the GPTQ optimization process in a way that incorporates Four Over Six is likely to deliver performance improvements in future work" (Section 4.2).

Model size trends. Looking across model sizes within the Llama 3 family (Table 5), RTN + 4/6 on WikiText-2 shows improvements that decrease with model size: 14.27 → 13.84 (improvement of 0.43) for 1B, 8.43 → 8.30 (0.13) for 8B, and 4.00 → 3.83 (0.17) for 70B. Similarly, for AWQ + 4/6 on WikiText-2: 14.04 → 13.67 (0.37) for 1B, 8.33 → 8.24 (0.09) for 8B, and 3.86 → 3.71 (0.15) for 70B. This suggests the benefit of 4/6 is somewhat larger for smaller models, which is intuitive — larger models have more redundancy and can tolerate quantization error better, so the marginal gain from reducing error on near-maximal values is smaller when the baseline performance is already closer to BF16.

Scale selection rule comparison (Table 4). The choice of error metric for the M=4 vs. M=6 decision matters, and MSE is the clear winner for PTQ. Comparing 4/6 with MSE selection to 4/6 with MAE selection on WikiText-2: MSE is better for 8 of 12 model-benchmark combinations (e.g., Llama-3-70B WikiText-2: MSE = 3.83 vs. MAE = 3.86; Qwen3-32B WikiText-2: MSE = 9.84 vs. MAE = 9.82 — the latter being one of the rare MAE wins). Abs-Max selection consistently performs worst of the three, often degrading below standard RTN NVFP4 for larger models (Llama-3-70B: 4.39 vs. RTN's 4.00; Qwen3-32B: 9.97 vs. RTN's 9.85). This confirms the paper's reasoning that a single worst-element error is a poor proxy for overall block reconstruction quality.

Post-Training Quantization: Downstream Tasks

Tables 6 and 7 extend the PTQ evaluation beyond perplexity to four downstream tasks, demonstrating that the improvements observed in language modeling metrics translate — at least directionally — to task performance. The headline finding is that "4/6 improves average task performance in nearly all cases" (Section 4.2).

Llama-3 family (Table 6). For Llama-3.2-1B, the best configuration is AWQ + 4/6, achieving an average normalized accuracy of 54.1 (across BoolQ, ARC-E, ARC-C, HellaSwag), compared to 53.2 for standard AWQ and 56.7 for BF16. The improvement is +0.9 percentage points, representing a partial recovery toward the BF16 baseline. For Llama-3.1-8B, AWQ + 4/6 achieves 73.1 average accuracy, compared to 72.2 for standard AWQ and 75.0 for BF16 — a similar +0.9 point gain.

Breaking down by method:

• RTN + 4/6: For Llama-3.2-1B, average accuracy is 51.9 vs. 52.3 for standard RTN — a slight degradation, notably worse than RTN alone, though the paper's statement of "improved in nearly all cases" glosses over this. For Llama-3.1-8B, RTN + 4/6 achieves 72.2 vs. 72.0 for RTN — a marginal improvement of +0.2 points.

• GPTQ + 4/6: For Llama-3.2-1B, average accuracy is 53.1 vs. 53.0 — negligible. For Llama-3.1-8B, 72.6 vs. 72.4 — also negligible. The near-zero improvements across GPTQ configurations match the mixed perplexity results and support the paper's explanation that GPTQ's optimization procedure is incompatible with per-block adaptive grid selection.

• AWQ + 4/6: This shows the clearest improvement. For Llama-3.2-1B, individual task improvements are: BoolQ 59.8 → 61.0 (+1.2), ARC-E 58.0 → 58.8 (+0.8), ARC-C 34.2 → 35.5 (+1.3), HellaSwag 60.9 → 61.2 (+0.3). For Llama-3.1-8B: BoolQ 81.3 → 80.4 (-0.9, a degradation), ARC-E 78.4 → 80.2 (+1.8), ARC-C 51.7 → 53.6 (+1.9), HellaSwag 77.5 → 78.2 (+0.7). Note that BoolQ performance for 8B actually decreases with 4/6, even though average accuracy improves — this illustrates that the "nearly all cases" claim masks individual task-level degradations.

• SmoothQuant + 4/6: For Llama-3.2-1B, average accuracy improves from 53.7 to 54.2 (+0.5). For Llama-3.1-8B, from 72.3 to 72.9 (+0.6). Individual task results show improvements on most benchmarks but with small magnitudes (typically 0.3–1.0 points).

Qwen3 family (Table 7). The pattern is similar. For Qwen3-1.7B, the best configuration is RTN + 4/6 with average accuracy of 59.8 (vs. 56.6 for RTN and 62.8 for BF16 — a +3.2 point improvement, the largest single-method gain in either table). For Qwen3-8B, GPTQ + 4/6 achieves 73.9 average accuracy (vs. 73.4 for GPTQ and 74.8 for BF16). Notably, AWQ + 4/6 underperforms RTN + 4/6 for Qwen3-1.7B (59.2 vs. 59.8), which is different from the Llama results where AWQ + 4/6 was consistently best.

Magnitude and practical significance. The improvements are consistently small — typically 0.3–1.0 percentage points on average accuracy, with individual task improvements rarely exceeding 2 points. At Llama-3.1-8B scale, the gap between the best quantized configuration (AWQ + 4/6 at 73.1) and BF16 (75.0) is 1.9 percentage points, meaning 4/6 closes approximately 32% of the gap between standard AWQ (72.2, 2.8 point gap) and BF16. For smaller models (Llama-3.2-1B), the closure is proportionally larger: standard AWQ to BF16 gap is 3.5 points (53.2 to 56.7), and AWQ + 4/6 closes 0.9 of that, or about 26%. These are meaningful but incremental gains — 4/6 does not bring quantized models near BF16 parity, it simply moves them incrementally closer.

Important caveat: The paper does not report whether task evaluation uses the same calibration data as perplexity evaluation, the number of shots used, or any details of the evaluation protocol beyond "normalized accuracy." The absence of statistical testing means that task-level differences of 0.3–0.5 points — which account for many of the reported improvements — cannot be meaningfully distinguished from noise without knowing the evaluation set size and variance.

Pre-Training

The main pre-training result appears in Figure 4 and the surrounding discussion in Section 4.1: 4/6 applied to activation quantization reduces NVFP4 training loss, bringing it 13.0% closer to the BF16 baseline after 1 trillion tokens of training on Nemotron 3 Nano 30B-A3B. The figure shows three training loss curves — BF16 (lowest loss), NVFP4 baseline (highest loss), and NVFP4 with 4/6 (intermediate, closer to BF16) — along with a bottom subplot that shows the relative difference from BF16 as a percentage throughout training. The 13.0% figure is computed from the loss values at the end of training (1T tokens), and it represents the fraction of the BF16-to-NVFP4 loss gap that is recovered by adding 4/6.

Reading from Figure 4: The absolute training loss values are approximately 1.4 for BF16, roughly 1.41 for NVFP4 + 4/6, and roughly 1.42 for NVFP4 baseline at 1T tokens (exact values are not tabulated — these are read from the y-axis of the log-scale plot). The bottom subplot shows the relative difference from BF16 stabilizing at approximately -0.75% for standard NVFP4 and -0.65% for NVFP4 with 4/6 — a reduction of 0.10 percentage points, or approximately 13% of the 0.75% gap.

Activation-only application matches full application. Appendix A.4 (Figure 8) shows a critical finding for practical deployment: applying 4/6 only to activation quantization achieves training performance comparable to applying it to all tensor types (activations, weights, and gradients). The three curves in Figure 8 — "4/6 on X only," "4/6 on X, W, and G," and "4/6 on X and W" (a slightly different configuration) — all show similar trajectories, with "4/6 on X only" tracking very close to "4/6 on X, W, and G" throughout training. In contrast, "4/6 on X and G" (activations and gradients only) underperforms, sitting closer to the NVFP4 baseline. The implication is straightforward: the computational overhead of 4/6 can be limited to activation quantization (affecting fewer operations per training step), and the benefit is preserved.

Scale selection rule for pre-training. Figure 5 (Appendix A.1) shows that MAE selection consistently outperforms MSE selection during training, regardless of whether 4/6 is applied to activations and weights only or to all three tensor types. The MAE curves sit visibly closer to the BF16 baseline throughout the 1T token training run, with the gap widening slightly in the second half of training. This is the opposite of the PTQ finding, where MSE was best (Table 4). The paper does not provide a mechanistic explanation for this reversal but notes it consistently.

Reduced tensor scale provides a marginal benefit. Figure 6 (Appendix A.2) shows that the modified FP32 tensor scale computation (using 256 instead of 448 as the maximum FP8 value, see Section 3.1) provides a small improvement in training loss compared to using the standard tensor scale computation. The improvement is visible but small — the two curves ("4/6 on X only" with and without the scale adjustment) largely overlap, with the scale-adjusted version being marginally lower, particularly evident between 200B and 500B tokens.

Stochastic rounding interaction. Figure 7 (Appendix A.3) evaluates three approaches for handling the interaction between 4/6 and stochastic rounding on gradients. The "biased" approach (standard 4/6 applied directly to stochastically rounded values) outperforms both alternatives: using NVFP4 without 4/6 on gradients (which provides no benefit over the baseline, confirming that the gradient-related benefit must come from 4/6 specifically), and an "unbiased" selection approach where M=4/M=6 is chosen using round-to-nearest error, then stochastic rounding is applied independently with the selected scale. The unbiased approach performs worse than the biased approach, which the paper attributes to the variance reduction from better near-maximal value representation outweighing the bias introduced by the error-minimizing scale selection.

Ablation Studies and Robustness Checks

Scale selection rule (MSE vs. MAE vs. Abs-Max) for PTQ (Table 4): MSE is best overall, producing the lowest perplexity in 9 of 12 model-benchmark combinations. MAE is a close second, sometimes matching or slightly exceeding MSE for specific models (e.g., Qwen3-32B WikiText-2: MAE = 9.82, MSE = 9.84). Abs-Max is unambiguously worse, often degrading below standard RTN NVFP4 for larger models — it produces WikiText-2 perplexity of 4.39 for Llama-3-70B compared to 4.00 for RTN NVFP4, a substantial degradation. This confirms that a single worst-element error is a poor proxy for block-level reconstruction quality when the goal is downstream model performance.

Scale selection rule (MSE vs. MAE) for pre-training (Figure 5): MAE consistently outperforms MSE during pre-training, a reversal from the PTQ finding. The MAE curves sit closer to the BF16 baseline regardless of whether 4/6 is applied to (X, W) or (X, W, G). The gap between MAE and MSE becomes more pronounced after approximately 400B tokens, suggesting the benefit accumulates as training progresses. This is one of the paper's most interesting negative results — it shows that the "obviously correct" error metric (MSE, which aligns with training loss) is not optimal for the quantization selection decision during training, pointing to a non-trivial interaction between quantization error structure and gradient signal quality.

Per-tensor application of 4/6 during pre-training (Figure 8): Applying 4/6 only to activation quantization achieves performance comparable to applying it to all three tensor types (activations, weights, and gradients). Applying it to activations and weights (no gradients) slightly underperforms activations-only. Applying it to activations and gradients (no weights) underperforms further. Applying it to all three is comparable to activations-only. This is a practically important result: the overhead of 4/6 can be concentrated on a single tensor type, reducing the total per-step cost while retaining the training loss benefit.

Reduced FP32 tensor scale (Figure 6): Using 256 instead of 448 as the maximum FP8 E4M3 value in the tensor scale computation provides a marginal training loss improvement. The improvement is small (visible as a slight offset between the two "4/6 on X only" curves in Figure 6) but consistent, suggesting that the few blocks containing the tensor's absolute maximum value — which this adjustment affects — have a disproportionate impact on training dynamics.

Stochastic rounding interaction with gradient quantization (Figure 7): The biased 4/6 selection mechanism (selecting between M=4 and M=6 based on reconstruction error of the already-stochastically-rounded values) outperforms an unbiased alternative and matching the performance of not applying 4/6 to gradients at all (which provides no improvement). The unbiased selection, which tries to preserve stochastic rounding's unbiased property, performs worst of the three approaches evaluated. This suggests that for gradient quantization, reducing overall quantization error is more important than maintaining strict unbiasedness, at least for the optimizer and architecture tested.

Format-level comparison (MXFP4 vs. NVFP4) (Table 3): NVFP4 (M=6) consistently outperforms MXFP4 across all six models on WikiText-2. For Llama-3.1-8B, MXFP4 achieves 9.66 perplexity vs. 8.43 for NVFP4 and 7.54 for BF16. NVFP4 with all blocks scaled to M=4 performs worse than standard NVFP4 M=6 for every model — confirming that global M=4 is not a viable strategy and that adaptive per-block selection is essential. This table also establishes that the format itself (NVFP4's FP8 E4M3 scale factors with 16-value blocks vs. MXFP4's E8M0 scale factors with 32-value blocks) makes a meaningful difference in baseline quantization quality, independent of 4/6.

GPTQ integration approach (Table 5, discussed in Section 4.2): The paper's "drop-in replacement" approach — swapping the quantization function in GPTQ's optimization loop without modifying the optimization itself — produces mixed results, with significant degradations for some Qwen3 models (e.g., Qwen3-1.7B WikiText-2 degrades from 21.48 to 22.70). This is a meaningful negative result: it demonstrates that the benefit of 4/6 is not automatic and that proper integration with Hessian-based optimization requires modifying the optimization to account for the adaptive grid choice. The paper explicitly acknowledges this limitation and positions it as future work.

Model scale ablation (Tables 5, 6, 7): The paper evaluates across a range of model sizes (1B, 1.7B, 8B, 32B, 70B), but the patterns across scales are not systematically analyzed. The relative improvement tends to be larger for smaller models (e.g., Llama-3.2-1B sees larger perplexity reductions than Llama-3-70B), which is intuitively plausible — larger models have more redundancy and tolerate quantization error better, leaving less room for 4/6 to improve. However, the Qwen3 family does not show this pattern cleanly: Qwen3-1.7B improvements are not consistently larger than Qwen3-8B or Qwen3-32B improvements.

Downstream task diversity (Tables 6, 7): The four tasks — BoolQ, ARC-Easy, ARC-Challenge, HellaSwag — test different capabilities (factual knowledge, science reasoning, commonsense reasoning), and 4/6 improvements appear across all four, suggesting the benefit is not specific to a particular task type. However, the small magnitude of improvements means this claim is suggestive rather than conclusively demonstrated — across all model-method-task combinations, only a handful show improvements exceeding 2 percentage points.

Critical Assessment

The paper's central claim is that 4/6 improves NVFP4 quantization accuracy by adaptively scaling some blocks to an FP4 maximum of 4 rather than 6, reducing quantization error for near-maximal values, and that this translates to improved pre-training loss and PTQ perplexity/downstream performance with minimal overhead. Let us examine what the experiments actually demonstrate and where the evidence falls short.

Claim: NVFP4 error is concentrated on near-maximal values, and the FP4 grid gap between 4 and 6 is the primary bottleneck.

This claim is the most thoroughly supported in the paper. Figure 2b provides clear, controlled evidence through the partial quantization simulation: performance degrades steeply when scaled values above 4 are included in the quantization, with a visibly steeper slope at x=5 than at lower thresholds. Figure 2a cleanly isolates the error source to FP4 value quantization rather than FP8 scale factor quantization. These experiments are well-designed and the results are unambiguous. However, both experiments are performed only on Llama-3.1-8B with WikiText-2 perplexity — a single model, a single metric. The paper does not replicate the diagnostic on other model families (Qwen3), other model sizes (1B, 70B), or other metrics (downstream tasks). We therefore know that the near-maximal-value concentration holds for this specific model-metric combination, but the claim that this is a general property of NVFP4 quantization is an extrapolation, not a demonstrated fact. The diagnostic experiments are also performed only on weights (PTQ setting), not on activations or gradients during training, yet the paper's pre-training configuration applies 4/6 primarily to activations, where the error distribution may differ.

Claim: 4/6 improves PTQ perplexity and downstream task performance.

The PTQ results in Tables 5, 6, and 7 show consistent improvements for AWQ and SmoothQuant across all models tested, with improvements also appearing for RTN on most (but not all) model-benchmark combinations. The magnitude of improvement is small — typically 0.1–0.5 perplexity points and 0.3–1.0 percentage points on average task accuracy — but directionally consistent. The evidence is strengthened by evaluation across two model families (Llama 3, Qwen 3), six model sizes, two perplexity benchmarks, and four downstream tasks, providing a reasonable cross-section of modern LLM architectures and scales.

However, several aspects weaken these results:

• No statistical testing. All reported numbers are point estimates with no confidence intervals, standard deviations, or significance tests. For downstream tasks where accuracy differences are often 0.3–0.5 points, it is impossible to determine whether these represent genuine improvements or sampling noise. The paper's claim that "4/6 improves average task performance in nearly all cases" is based on comparing single numbers without any measure of reliability.

• GPTQ integration shows degradation. The paper's "drop-in" approach to combining 4/6 with GPTQ produces mixed and sometimes negative results (e.g., Qwen3-1.7B WikiText-2 degrades from 21.48 to 22.70). The paper acknowledges this and attributes it to incompatibility with GPTQ's optimization, but this means the claim of broad composability has a clear failure case. The paper's suggestion that "modifying the GPTQ optimization process... is likely to deliver performance improvements in future work" is speculation, not a demonstrated fact.

• The Qwen3-1.7B RTN + 4/6 degradation on C4 perplexity (65.54 → 66.32) contradicts the claim of uniform improvement and is not discussed or explained in the text. This is a non-trivial degradation (0.78 perplexity points) that, if genuine, would mean 4/6 can sometimes harm performance even in simple RTN settings.

• No evaluation on models larger than 70B. While the paper tests up to 70B parameters (a reasonable scale), the most impactful deployment of NVFP4 quantization would be on the largest models (100B+, 400B+) where the memory and speed benefits are most dramatic. The behavior of 4/6 at those scales is unknown.

• Perplexity is an incomplete proxy. While the paper notes that "perplexity is often considered a more stable metric for evaluating quantized models" (citing Dettmers and Zettlemoyer, 2023), the downstream task results (Tables 6, 7) sometimes show discrepancies with perplexity rankings. For instance, Qwen3-1.7B RTN + 4/6 has worse C4 perplexity than standard RTN (66.32 vs. 65.54) but better average task performance (59.8 vs. 56.6). The paper does not explore these discrepancies or discuss which metric should be prioritized for model selection.

Claim: 4/6 improves pre-training loss, bringing it 13.0% closer to BF16.

The pre-training experiment (Figure 4) is the paper's most important result, as it demonstrates 4/6's value in the most computationally expensive and practically significant setting. The experiment is well-configured: it uses a modern architecture (MoE hybrid Mamba-Transformer), trains on 1 trillion tokens with 384 B200 GPUs, and integrates 4/6 into the full NVFP4 training pipeline with stochastic rounding, RHT, and 2D block quantization.

However, the evidence has significant limitations:

• Single architecture, single training run. The pre-training results come from exactly one model — Nemotron 3 Nano 30B-A3B. There is no replication across different model architectures (dense transformers, different MoE configurations), different model sizes, or different training data distributions. The 13.0% figure is specific to this architecture, this data, and this training recipe. Whether the improvement generalizes is entirely unknown.

• Single run per configuration. Each training curve in Figures 4–8 appears to represent a single training run (the paper does not mention multiple seeds or replication). Large-scale training runs have inherent variance from initialization, data ordering, and hardware non-determinism. The gap between BF16 and NVFP4 curves in Figure 4 is small (approximately 0.75% relative difference), and the 4/6 curve sits between them with an even smaller separation from the NVFP4 baseline. Without multiple runs, it is impossible to know whether the observed difference is signal or noise from training variance.

• No downstream evaluation of the pre-trained model. The paper reports only training loss curves for the pre-training experiment. There is no evaluation of the pre-trained models on any downstream task, benchmark, or even held-out validation perplexity. Training loss is a necessary but not sufficient metric — models with identical training loss can have different downstream capabilities, especially when quantization is involved (quantization noise could affect generalization in ways not captured by training loss). A key missing experiment is evaluating the pre-trained NVFP4 + 4/6 model on standard benchmarks (MMLU, HellaSwag, etc.) and comparing it to the BF16 and NVFP4 baselines.

• The 13.0% figure needs careful interpretation. The figure represents the fraction of the gap between NVFP4 and BF16 training loss that is closed by 4/6, not the absolute loss reduction. If the absolute gap is small (0.75% relative difference), then closing 13% of it is a change of approximately 0.10 percentage points in relative loss difference. Whether this improvement translates to meaningful downstream differences is not established.

Claim: 4/6 adds under 15% overhead to the NVFP4 quantization kernel.

The paper reports that the 4/6 CUDA kernel adds under 15% overhead relative to the standard NVFP4 quantization kernel (Section 3.2). This is a critical practical claim — if the overhead were large, it would undermine the value proposition of NVFP4 training. However, the paper provides no detailed benchmarking of this overhead. There is no table of kernel execution times, no comparison across different tensor sizes or shapes, no measurement of end-to-end training step time impact, and no discussion of whether the overhead varies across the three GEMM operations (FPROP, DGRAD, WGRAD) that invoke quantization. The under 15% figure is stated as an observation from the CUDA implementation but is not quantitatively substantiated in the paper.

Additionally, the pre-training optimal configuration — 4/6 on activations only — means the overhead applies only to a subset of quantization operations per training step, which should make the end-to-end overhead substantially less than 15% of the total quantization cost (since weight and gradient quantization use standard NVFP4). The paper does not quantify this end-to-end overhead, nor does it compare the throughput (tokens/second) of NVFP4 + 4/6 training against NVFP4 baseline and BF16 training. This is a significant omission for a systems paper where the practical deployment cost is central to the argument.

Missing experiments that would have strengthened the paper:

1. Downstream evaluation of pre-trained models. The most glaring gap. A table comparing NVFP4 + 4/6 pre-trained models against BF16 and NVFP4 baselines on standard benchmarks would directly answer whether the training loss improvement translates to better models.

2. 4/6 applied to both AWQ and SmoothQuant simultaneously (or with rotations). The paper shows 4/6 combines with individual PTQ methods but does not test combinations of multiple methods — e.g., AWQ + SmoothQuant + 4/6, or QuaRot + 4/6. Since these methods address different sources of error, stacking them with 4/6 could yield larger improvements.

3. 4/6 during fine-tuning, not just pre-training. A natural use case is quantized fine-tuning (QLoRA-style), where 4/6 might provide benefits during the adaptation phase. This is not explored.

4. Evaluation on code generation or long-context tasks. The paper evaluates only on language modeling perplexity and short-form QA/reasoning tasks. Code generation, where token-level accuracy is critical and errors compound, might be more sensitive to quantization quality, making 4/6's impact larger — or might reveal new failure modes.

5. Measurement of end-to-end throughput. The paper reports kernel-level overhead (<15%) but not system-level throughput. A table showing tokens/second for BF16, NVFP4, and NVFP4 + 4/6 training configurations would directly address the practical tradeoff between accuracy and speed.

Overall assessment. The paper provides convincing evidence that the FP4 grid gap between representable values 4 and 6 is a meaningful source of NVFP4 quantization error (Figures 2a, 2b), and that 4/6's per-block adaptive scaling reduces this error in a way that translates to measurable — but small — improvements in PTQ perplexity and task accuracy across two model families (Tables 5, 6, 7). The pre-training result (Figure 4) is promising but rests on a single training run of a single architecture with no downstream evaluation, limiting the strength of conclusions that can be drawn. The overhead claim (<15%) is stated without quantitative benchmarking. The paper's contribution is best understood as a well-motivated, low-cost engineering improvement that incrementally advances the state of NVFP4 quantization, rather than a breakthrough that fundamentally closes the gap between 4-bit and high-precision training.

6. Limitations and Trade-offs

Single Pre-Training Architecture and Single Run

The assumption or constraint. All pre-training results come from a single training run of a single model architecture — Nemotron 3 Nano 30B-A3B, a Mixture-of-Experts hybrid Mamba-Transformer — trained on a specific curated data distribution of 1 trillion tokens. The paper provides exactly one training loss curve per configuration in Figures 4–8, with no mention of multiple random seeds, multiple architectures, or multiple data mixes. The 13.0% closer-to-BF16 figure is an estimate derived from comparing three point values at the end of a single 1T-token run.

The consequence. A practitioner considering adopting 4/6 for pre-training a different architecture — a dense transformer, a different MoE configuration, a vision-language model, a model at a different scale — has essentially no evidence about whether the training loss improvement will transfer. The paper's own PTQ results show model-family-dependent effects: 4/6 improves RTN for Llama models more consistently than for Qwen3 models on C4 (Table 5), and AWQ + 4/6 is the best configuration for Llama while RTN + 4/6 is best for Qwen3-1.7B (Tables 6 vs. 7). These differences suggest that optimal 4/6 configuration may be architecture-dependent, yet the pre-training experiments explore exactly one architecture.

More fundamentally, a single training run per configuration cannot distinguish a genuine algorithmic improvement from training variance. Large-scale training runs exhibit inherent non-determinism from hardware, data ordering, and initialization. The absolute gap between the NVFP4 baseline and the BF16 baseline in Figure 4 is small (the relative difference from BF16 stabilizes around -0.75% for NVFP4 vs. approximately -0.65% for NVFP4 + 4/6 — a difference of roughly 0.10 percentage points in the relative loss metric). Without replication, a practitioner cannot assess whether a 0.10 percentage-point shift in relative loss is reliably attributable to 4/6 or falls within the run-to-run variance of training this architecture at this scale.

What evidence exists in the paper. The pre-training experiment is reported in Section 4.1 and Appendix A, with results in Figures 4–8. The ablation studies in Appendix A — comparing scale selection rules (Figure 5), tensor scale adjustment (Figure 6), stochastic rounding interactions (Figure 7), and per-tensor application (Figure 8) — all use the same architecture and, based on the presented curves, appear to be single runs. The paper reports training hyperparameters in Section 4.1 (AdamW with specific betas, weight decay, gradient clipping, sequence length, batch size, LR schedule) but does not mention multiple seeds.

Mitigation status. The paper does not acknowledge this limitation explicitly for the pre-training results. For PTQ, the evaluation across six models from two families provides some cross-architecture validation, but this does not extend to the pre-training setting where the computational cost of replication is three orders of magnitude larger.

---

No Downstream Evaluation of Pre-Trained Models

The assumption or constraint. The pre-training experiments report only training loss curves (Figures 4–8). There is no evaluation of the pre-trained NVFP4 + 4/6 model on any downstream benchmark, held-out validation set, or task-oriented metric. The paper's central pre-training claim — that "4/6 brings training loss closer to BF16" (Section 4.1) — relies entirely on the assumption that improved training loss implies improved model quality, with no direct evidence.

The consequence. Training loss is a within-distribution metric that measures how well the model fits its training data. It does not necessarily correlate with downstream capabilities, especially under aggressive quantization where the noise introduced by FP4 rounding could affect generalization in ways that training loss does not capture. A practitioner choosing between training with NVFP4 + 4/6 versus allocating budget to a larger BF16 model needs to know the end-to-end model quality, not just the training curve. The paper provides no answer: it is entirely possible that the 13.0% training loss improvement translates to zero improvement (or even degradation) on standard benchmarks like MMLU, HellaSwag, or coding tasks.

This is particularly concerning given the paper's own PTQ results, which show that perplexity improvements do not always translate cleanly to downstream tasks. For example, Table 6 shows that for Llama-3.1-8B, AWQ + 4/6 improves ARC-C from 51.7 to 53.6 (+1.9 points) but degrades BoolQ from 81.3 to 80.4 (-0.9 points). If the relationship between quantization error reduction and downstream performance is non-monotonic even for individual tasks, the assumption that a lower training loss implies a universally better model is risky.

What evidence exists in the paper. None. Section 4.1 discusses only training loss. The paper does not mention held-out validation perplexity, downstream benchmarks, or any form of post-training evaluation for the pre-trained models. This is the most significant missing experiment in the paper.

Mitigation status. Not addressed. The paper does not acknowledge this as a limitation, does not report any pre-training evaluation beyond loss curves, and does not discuss future plans to evaluate the pre-trained models on benchmarks. The gap between training loss and downstream evaluation is a well-known issue in the quantization literature — models trained in low precision can exhibit different generalization behavior than high-precision models with identical training loss — and the paper's silence on this point is a genuine weakness.

---

Overhead Claim is Not Quantitatively Benchmarked

The assumption or constraint. The paper states that 4/6 "can be implemented efficiently... adding less than 15% overhead to the NVFP4 quantization operation" (Section 3.2) and that "we expect that we will be able to reduce this overhead further with more optimization." This is the sole quantitative performance claim, and it refers to the isolated quantization kernel, not to end-to-end training throughput. No kernel execution time measurements, no tables of latency or throughput, and no comparison across tensor shapes are provided.

The consequence. The entire practical argument for 4/6 during pre-training — that it improves accuracy with "minimal computational overhead" — is undermined by the absence of end-to-end throughput data. The paper's own framing in Section 1 emphasizes that "if too much overhead is introduced, it becomes faster to train models using more accurate FP8 formats." But the paper never measures whether NVFP4 + 4/6 training is actually faster than FP8 training on the same hardware, or by how much. A practitioner evaluating whether to adopt 4/6 needs to know:

• The wall-clock time per training step for NVFP4 + 4/6 vs. NVFP4 baseline vs. BF16 and FP8 training.
• Whether the <15% kernel overhead on the quantization operation translates to proportionally smaller end-to-end overhead (since quantization is only one component of a training step that also includes GEMM execution, attention, normalization, and communication).
• Whether the overhead varies across FPROP, DGRAD, and WGRAD — the three GEMM operations that invoke quantization — and how the "activations-only" recommendation (Appendix A.4) affects total overhead.

The paper's recommended configuration — 4/6 on activations only — means only a subset of quantization operations incur the dual-path cost, which should reduce end-to-end overhead well below 15% of total quantization time. But this is not quantified, leaving practitioners to guess at the actual speed-accuracy tradeoff.

What evidence exists in the paper. None beyond the qualitative statement in Section 3.2 and the promise of future optimization. There is no kernel benchmarking table, no roofline analysis, and no measurement of training tokens per second for any configuration. All overhead discussion is speculative.

Mitigation status. The paper partially acknowledges the need for further optimization ("we expect that we will be able to reduce this overhead further with more optimization" — Section 3.2) but does not treat the absence of benchmarking as a limitation. The paper states that quantization and matrix multiplication kernels are "release[d] on GitHub," which could allow independent benchmarking, but the paper itself contains no data. This is a significant omission for a systems paper where the speed-accuracy tradeoff is the central practical question.

---

No Generalization Evidence Beyond Language Modeling on Two Model Families

The assumption or constraint. All experiments — both PTQ and pre-training — are restricted to autoregressive language models evaluated on language modeling perplexity (WikiText-2, C4) and four standard multiple-choice benchmarks (BoolQ, ARC, HellaSwag). The PTQ evaluation covers two model families (Llama 3, Qwen 3) with a maximum size of 70B parameters, and the pre-training evaluation covers a single hybrid Mamba-Transformer architecture at 30B-A3B scale.

The consequence. A practitioner or researcher cannot determine whether 4/6's benefit transfers to:

• Model modalities beyond text: Vision transformers, multimodal models, diffusion models, or speech models might have activation distributions with different outlier patterns or different sensitivity to near-maximal value quantization. The paper's diagnostic (Figure 2b) showing that error concentrates on values near the top of each block's range may not hold for modalities where activation distributions are fundamentally different.
• Model scales beyond 70B: The largest models (100B+, 400B+) are where NVFP4's memory and speed benefits are most dramatic, but also where quantization sensitivity may differ. The PTQ results show a weak trend of larger relative improvements for smaller models (Llama-3.2-1B RTN + 4/6 improves WikiText-2 by 0.43 vs. 0.17 for Llama-3-70B), but the pattern is inconsistent across model families, and the paper does not formally analyze scale dependence.
• Tasks beyond perplexity and short-form QA: Code generation, mathematical reasoning, long-context tasks, and instruction-following all involve different patterns of token-level prediction. Code generation in particular is sensitive to single-token errors that compound across a sequence — if 4/6 reduces quantization error for most values but occasionally introduces larger errors on specific values (which Abs-Max selection might catch but MSE/MAE might not), code generation could be disproportionately affected. The paper's downstream evaluation (Tables 6, 7) covers only short-form tasks, and even there individual task degradations occur (e.g., Llama-3.1-8B BoolQ drops from 81.3 to 80.4 with AWQ + 4/6) despite aggregate improvement.
• Non-English language modeling or specialized-domain models: The training data and evaluation benchmarks are English-centric. Models fine-tuned on code, mathematics, or multilingual data may have different distributional properties in their weight and activation tensors.

What evidence exists in the paper. The PTQ evaluation covers two model families and six model sizes (Llama 3 at 1B/8B/70B, Qwen 3 at 1.7B/8B/32B), providing better cross-architecture validation than most quantization papers. However, all models are autoregressive text LLMs, all evaluation is on standard English benchmarks, and the pre-training uses a single architecture. The paper does not evaluate on code generation, math, long-context, or non-English tasks despite these being important deployment scenarios.

Mitigation status. The paper does not acknowledge the modality and task scope limitation. The abstract's claim that 4/6 yields "performance gains during both pre-training and inference" does not qualify the scope of evaluation. The limitation to language models on standard English benchmarks is implicit in the experimental design but not discussed as a boundary on the claims.

---

GPTQ Integration Produces Degradations, Revealing a Fundamental Composability Gap

The assumption or constraint. The paper positions 4/6 as a "drop-in improvement" compatible with existing PTQ methods by substituting the quantization function. The experimental results partially validate this: AWQ and SmoothQuant show consistent improvements with 4/6. However, GPTQ + 4/6 produces substantial degradations for some configurations. The paper reports that "GPTQ with 4/6 reduces the performance of models quantized with GPTQ, increasing the gap between NVFP4 and BF16 word perplexity by an average of 34.6%" (Section 4.2). Specific failures include Qwen3-1.7B degrading from 21.48 to 22.70 WikiText-2 perplexity (Table 5).

The consequence. This is not a minor integration bug — it reveals a fundamental tension between 4/6's adaptive per-block grid selection and GPTQ's Hessian-based optimization. GPTQ works by using second-order information to compensate quantization error across columns of the weight matrix: when a value in one column is rounded in a way that introduces error, the optimization adjusts the remaining unquantized values in that column to compensate. This optimization assumes a fixed quantization grid — the mapping from scaled values to FP4 representable values is known and constant during the optimization. When 4/6 introduces per-block adaptive grid selection, the grid is no longer fixed: a block's choice between M=4 and M=6 depends on the final quantized values, and the error compensation that GPTQ performed assumed one grid but may get a different one. The optimization objective and the quantization function become circularly dependent.

This matters because GPTQ is one of the most widely used post-training quantization methods, and a practitioner who uses GPTQ (or GPTQ-derived methods) cannot simply swap in 4/6 and expect improvement. The paper's composability claim — that 4/6 "can be easily combined with existing PTQ methods" (Section 4.2) — is therefore not universally true. It holds for methods that modify values before quantization (AWQ, SmoothQuant) but breaks for methods that modify the quantization decisions (GPTQ). This is a boundary condition that limits 4/6's practical applicability.

What evidence exists in the paper. Table 5 provides the raw data showing GPTQ + 4/6 degradations for several Qwen3 configurations. The paper acknowledges the issue: "Modifying the GPTQ optimization process in a way that incorporates Four Over Six is likely to deliver performance improvements in future work" (Section 4.2). However, this treats the problem as an implementation detail of the "drop-in" approach rather than a fundamental algorithmic incompatibility.

Mitigation status. The paper proposes future work on integrating 4/6 into the GPTQ optimization loop but does not provide any concrete approach or preliminary results. The paper does not discuss whether other Hessian-based or optimization-based PTQ methods (e.g., SparseGPT, QuIP) would face similar compatibility issues. A practitioner evaluating 4/6 for their PTQ pipeline must determine experimentally whether their specific quantization method falls into the "compatible" or "incompatible" category, with no guidance from the paper beyond these three tested methods.

---

Difficulty Estimation Is Not Evaluated for the Paper's Target Setting

The assumption or constraint. The paper's core diagnostic framework — identifying which values cause quantization error and designing 4/6 to address them — is grounded in post-training quantization experiments on static weights. The pre-training application (Section 4.1) applies 4/6 primarily to activations, which have fundamentally different statistical properties: they vary per input, they change as the model trains, and they are not amenable to the offline calibration and error analysis that motivated 4/6 in the first place. The paper presents no diagnostic evidence — no analogue of Figures 2a and 2b — for activation tensors during training.

The consequence. A practitioner cannot verify whether the mechanism that makes 4/6 work for PTQ — reducing error on near-maximal values that fall in the FP4 grid gap — actually applies to training-time activation tensors. The paper's finding that 4/6 on activations alone provides most of the benefit (Appendix A.4, Figure 8) is empirical and not mechanistically explained: it could be that activations during training exhibit the same near-maximal-value concentration as PTQ weights, or it could be that 4/6 provides a different, uncharacterized benefit during training (e.g., reducing gradient variance through a different mechanism). Without the equivalent of Figures 2a and 2b for activations during training, the paper's own diagnostic framework cannot be validated for its primary use case.

This matters because the pre-training result — the 13.0% closer-to-BF16 claim — is the paper's headline contribution, and it is less well-understood than the PTQ results. The PTQ results have a clear causal story: Figure 2b shows near-maximal values cause degradation, Table 2 shows 4/6 fixes those values, and Table 5 shows downstream improvement. For pre-training, the story is: Figure 2b (on static weights) suggests near-maximal values matter, Figure 8 shows activations benefit disproportionately from 4/6, and Figure 4 shows training loss improves — but the connection between the diagnostic and the result is bridged by an unvalidated extrapolation from weights to activations.

What evidence exists in the paper. None that is training-specific. Figure 2b and Figure 2a were performed on Llama-3.1-8B weights during PTQ, not on training-time activation tensors. The paper does not present a partial-quantization simulation for activations, does not analyze the distribution of scaled activation values during training, and does not measure whether activation tensors in the Nemotron 3 Nano 30B-A3B architecture actually exhibit the near-maximal-value concentration that 4/6 is designed to address. The pre-training optimization choices (MAE over MSE, activations-only, biased selection over unbiased) are determined empirically without mechanistic explanation.

Mitigation status. The paper does not acknowledge this as a gap. The pre-training design is guided by empirical ablation — Figure 5 selects MAE over MSE, Figure 8 selects activations-only over all-tensor 4/6 — rather than by an analytical understanding of why those choices are optimal. The paper explicitly leaves the analysis of why MAE outperforms MSE during training as future work (Appendix A.1: "we leave this evaluation to future work"). The absence of training-time diagnostic analysis means the paper's central mechanistic claim — that 4/6 works by fixing near-maximal FP4 values — is well-supported for PTQ but largely assumed for pre-training.

This limitation is not fatal to the paper's practical contribution (the pre-training result is an empirical finding regardless of mechanism), but it weakens the paper's scientific contribution: the diagnostic framework that Section 2.2 painstakingly establishes does not directly extend to the setting where the paper's most important result is demonstrated.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not introduce a new quantization format, a new training recipe, or a new model architecture. It introduces something more subtle: a diagnostic reframing of where NVFP4 quantization error comes from, and a correspondingly targeted intervention that exploits a previously unrecognized degree of freedom in block-scaled quantization. The conceptual contribution is not 4/6 the algorithm — which is a straightforward dual-path quantization with a selection rule — but rather the demonstration that the FP4 grid gap between 66.7% and 100% of a block's maximum value is the dominant source of model degradation, and that this gap can be addressed by changing how the grid is aligned to the data, block by block, without changing the underlying FP4 datatype or adding expensive auxiliary operations.

This matters in three ways that together shift how the field should think about low-precision quantization:

First, it breaks the implicit assumption that scaling to the full range of the target format is always optimal. This assumption — codified in Nagel et al. (2021) and embedded in the standard NVFP4 quantization algorithm (Equation 3) — is so fundamental that prior work did not question it. The paper's diagnostic (Figures 2a, 2b) shows that for floating point formats with non-uniform step sizes, using the full range creates the worst errors by leaving a wide unrepresentable gap between the second-largest and largest representable values. The 4/6 solution — adaptively not using the top of the range for blocks where the gap is harmful — inverts the conventional wisdom. This reframing is not specific to FP4 or NVFP4; it applies to any floating point format where the step size is coarsest at the top of the range. Future format designers and quantization algorithm developers will need to consider not just the representable range but also the alignment between that range and the distribution of values within each quantization block.

Second, it identifies a new abstraction layer in the quantization stack. Prior work treated quantization as having three layers: data shaping (AWQ, SmoothQuant, rotations), block formation (2D blocking, tile configuration), and discretization (rounding, stochastic rounding, GPTQ's optimization). The paper cleanly demonstrates that there is a fourth layer — grid alignment — that sits between block formation and discretization. Grid alignment asks: given a block of values and a fixed set of representable grid points (the FP4 values), how should the grid be scaled and shifted relative to the block's values to minimize reconstruction error? Standard NVFP4 answers this with a deterministic rule: align the largest grid point (6) with the block's largest value. 4/6 answers with an adaptive rule: choose between two alignments (M=4 or M=6) based on which yields lower error. The fact that this single-layer change provides consistent improvements across multiple PTQ methods (AWQ, SmoothQuant, and partially RTN) without modifying those methods' internal logic demonstrates that grid alignment is an independent optimization target that prior work was not exploiting. This opens a new axis for research: what are the optimal grid alignment strategies for different formats (FP8, FP6, MX formats), different block sizes, and different tensor types?

Third, it provides a resolution to a latent tension in the FP4 quantization literature. Prior work on FP4 training (the NVIDIA NVFP4 paper, Quartet, FP4 All the Way, the MXFP4 training paper by Tseng et al.) relied on a growing set of auxiliary operations — random Hadamard transforms, stochastic rounding, high-precision layer retention, healing phases — to close the gap to BF16. The implicit narrative was that FP4 is fundamentally too coarse, and the only way forward is to make the values being quantized more "quantization-friendly" through preprocessing. This paper offers a complementary narrative: part of the problem is not the coarseness of the format per se, but the misalignment between the format's grid and the block's value distribution. Fixing this misalignment — at a cost of <15% kernel overhead, and with the recommended activations-only configuration reducing end-to-end overhead further — recovers a meaningful fraction of the gap without adding complexity elsewhere. This does not make RHT or stochastic rounding obsolete, but it does suggest that the path to viable FP4 training involves improvements at both the data-shaping layer and the grid-alignment layer, and that the latter may be substantially cheaper to implement in hardware than the former.

The paper also resolves a specific puzzle: why is NVFP4 (with E4M3 scale factors) empirically better than MXFP4 (with E8M0 scale factors) despite using the same underlying FP4 values? Table 3 quantifies this gap (NVFP4 is 0.43 to 1.53 perplexity points better across Llama-3-8B through Qwen3-8B), and the paper's explanation — that E4M3's non-power-of-two granularity enables adaptive grid alignment while E8M0's powers-of-two factorization cannot — provides a mechanistic reason that was not previously articulated. This has direct implications for future format design: E4M3-level precision in scale factors is not a luxury; it enables a class of optimization (grid alignment) that E8M0 cannot support.

The paper's contribution is an incremental refinement of NVFP4 quantization, not a paradigm shift. It reduces the gap between NVFP4 and BF16 by 13.0% in pre-training loss and 19.9% in AWQ perplexity — meaningful but not revolutionary improvements. Its primary impact is likely to be as a design principle rather than a specific algorithm: the recognition that grid alignment is a first-class optimization target, that floating point formats' non-uniform step sizes create alignment problems that uniform formats (INT4) don't have, and that block-scaled formats with sufficient scale factor precision can adaptively select grid alignments to reduce the largest errors. This principle will influence how the next generation of low-precision formats and quantization algorithms are designed, even if the specific M=4/M=6 dual-choice is superseded by more sophisticated alignment strategies.

Follow-Up Research This Work Enables

1. Continuous grid alignment optimization beyond the M=4/M=6 binary choice. The paper's 4/6 offers a single alternative alignment (M=4) to the standard (M=6), with a simple MSE-based selection rule. A natural and immediate extension is to ask: given a block of 16 values and the FP4 representable grid, what is the optimal scale factor $\Delta_i$ (not constrained to M=4 or M=6) that minimizes reconstruction error? This is a 1D continuous optimization problem per block — the scale factor can be any FP8 E4M3 value, not just those corresponding to M=4 or M=6 — and the objective is the MSE between the original values and their FP4-quantized-and-dequantized versions. Gradient-based or grid-search optimization of $\Delta_i$ during an offline calibration phase (analogous to AWQ's per-channel optimization but applied per-block) could recover additional error reduction beyond the binary choice. A strong experiment would implement per-block scale factor optimization for a Llama-3-8B model during PTQ, measure WikiText-2 perplexity and downstream task accuracy, and compare against both standard NVFP4 and 4/6. The key question is whether the additional degrees of freedom (continuous $\Delta_i$ vs. binary M=4/M=6) yield sufficient improvement to justify the calibration cost and any runtime overhead from non-standard scale factors.

2. Training-time diagnostic analysis of activation and gradient value distributions. The paper's pre-training finding — that 4/6 on activations alone provides most of the benefit (Figure 8), and that MAE outperforms MSE for scale selection during training (Figure 5) — is established empirically but not mechanistically explained. A critical follow-up would replicate the Figure 2 diagnostic (partial quantization simulation and scale-vs-value error decomposition) on training-time activation tensors and gradient tensors, not just static PTQ weights. Do activation tensors during training exhibit the same concentration of near-maximal values that makes 4/6 effective? Does the distribution of scaled activation values shift over the course of training, and if so, does the optimal scale selection rule change? Does the MAE-over-MSE reversal reflect a fundamental difference in how activation quantization error propagates through gradients compared to weight quantization error during inference? A strong experiment would instrument a training run (even at small scale, e.g., 100M parameters) to log the distribution of scaled activation values per block at multiple training steps, measure the relationship between block-level quantization error (under both MSE and MAE selection) and downstream gradient norm, and determine whether the MSE/MAE reversal is driven by gradient bias, variance, or interaction with the optimizer's update rule. This would transform the pre-training results from an empirical observation into a principled design guideline.

3. Proper integration of 4/6 with GPTQ's Hessian-based optimization. The paper's "drop-in" approach to GPTQ + 4/6 produced substantial degradations for Qwen3 models (Table 5), which the paper attributes to incompatibility between adaptive grid selection and GPTQ's fixed-grid optimization. A direct follow-up would modify the GPTQ algorithm to interleave grid alignment selection with the column-wise error compensation. Specifically: during GPTQ's sequential column quantization, when quantizing a column of a weight block, first evaluate both M=4 and M=6 alignments, compute the quantization error for each, select the lower-error alignment, and then apply the standard GPTQ Hessian-based compensation to the remaining columns using the actual error from the selected alignment. This makes the grid selection part of the optimization rather than a post-hoc step that invalidates the compensation. A strong experiment would implement this modified GPTQ + 4/6 algorithm, evaluate it on the same Llama-3 and Qwen3 models used in the paper, and compare against standard GPTQ, standard GPTQ + naive 4/6 (the paper's approach), and AWQ + 4/6 (the paper's best PTQ result). The key metric is whether the proper integration recovers the 4/6 benefit for GPTQ — i.e., does it produce improvements comparable to AWQ + 4/6, or does the fundamental tension between Hessian-based compensation and per-block grid selection make the methods inherently difficult to combine?

4. Evaluating 4/6 at scale: 100B+ models and long-context/code generation tasks. The paper's largest evaluated model is 70B parameters (Llama-3-70B), and evaluations are limited to language modeling perplexity and short-form multiple-choice tasks. Two gaps are particularly important. First, does the benefit of 4/6 persist or change at the 100B–400B scale where NVFP4 deployment is most economically impactful? Larger models have more redundancy and may tolerate quantization error better, potentially reducing 4/6's relative benefit — or, alternatively, the larger number of blocks in bigger models may mean that the fraction of blocks benefiting from M=4 alignment grows, increasing 4/6's impact. Second, how does 4/6 affect tasks where token-level accuracy is critical and errors compound — specifically code generation (HumanEval, MBPP) and long-context reasoning (Needle-in-a-Haystack, long-document QA)? These tasks are known to be more sensitive to quantization error than perplexity or short-form QA, and the paper's downstream evaluation (BoolQ, ARC, HellaSwag) does not stress-test error propagation. A strong experiment would quantize a Llama-3-70B (or larger, if available) model with AWQ + 4/6, evaluate on HumanEval pass@1 and pass@10, on a long-context retrieval benchmark, and on the standard suite from Tables 6/7, comparing against AWQ without 4/6 and BF16. The result would either validate that 4/6's benefit generalizes to deployment-critical tasks, or reveal specific failure modes that bound its applicability.

5. Hardware-algorithm co-design: can the grid alignment choice be made in the GEMM unit rather than the quantization kernel? The paper's implementation performs dual-path quantization and error comparison in the quantization kernel (Section 3.2), then passes the selected FP4 values and FP8 scale factor to the GEMM unit. An alternative design would move the grid alignment decision into the GEMM unit: the GEMM receives both candidates $\Delta^{(4)}, \bar{X}^{(4)}$ and $\Delta^{(6)}, \bar{X}^{(6)}$ for each block, performs two partial dot-product accumulations (one per candidate), and selects the candidate that produces the dot-product result closer to some reference (e.g., a higher-precision partial accumulation). This would be more expensive per GEMM operation but would integrate grid alignment with the actual downstream computation rather than optimizing per-block reconstruction error as a proxy. This question is particularly relevant given the paper's finding that the optimal error metric differs between PTQ and pre-training (MSE vs. MAE) — the GEMM-level decision could adapt the error metric to the computation being performed. A strong experiment would simulate (in a software GEMM, not requiring hardware modifications) the accuracy of GEMM-level grid alignment selection vs. the paper's block-level MSE/MAE selection, measuring both the dot-product error relative to FP32 reference and the downstream model perplexity, for weight-activation and activation-gradient matrix multiplications. This would inform whether future hardware should expose grid-alignment flexibility to the GEMM unit rather than handling it entirely in the quantization pipeline.

6. Grid alignment for other block-scaled floating point formats (MXFP6, FP8, and beyond). The paper's argument that 4/6 is specific to NVFP4 (not MXFP4) rests on MXFP4's E8M0 scale factors lacking the granularity to represent a 1.5× increase. However, the principle of adaptive grid alignment — selecting per-block how the representable grid maps to the block's value distribution — applies to any block-scaled format whose scale factors have sufficient precision. MXFP6 (a proposed 6-bit format with block scaling) and FP8 E4M3 (already used in training) both have E4M3 or better scale factors and non-uniform step sizes. For FP8 E4M3, the representable values are denser than FP4 but still have non-uniform step sizes (the gap between adjacent values grows with magnitude), so a similar near-maximal-value gap may exist, albeit smaller in relative terms. A strong follow-up would: (a) replicate the Figure 2 diagnostic for FP8 E4M3 quantization of weights and activations on a Llama-3-8B model to determine whether a near-maximal-value gap exists and contributes meaningfully to model degradation; (b) if yes, develop an adaptive grid alignment strategy for FP8 (e.g., choosing between scaling the largest value to 448 vs. some smaller value, analogous to the M=4/M=6 choice); (c) measure whether this strategy improves FP8 PTQ perplexity over standard FP8 quantization. This would test whether the grid-alignment principle is FP4-specific or generalizes to higher-precision floating point formats, and would directly inform the design of future block-scaled formats.

Practical Applications and Downstream Use Cases

1. NVFP4 pre-training with reduced overhead. The paper's recommended pre-training configuration — 4/6 applied to activation quantization only, with MAE-based scale selection and reduced FP32 tensor scale — is a drop-in modification to the NVIDIA NVFP4 training recipe that improves training loss by 13.0% toward BF16 at minimal additional cost. Since activation quantization uses 4/6 while weight and gradient quantization use standard NVFP4, the per-step overhead is well below the <15% kernel-level figure and concentrated on a subset of operations. For a team training a 30B-parameter model on 1T tokens with 384 B200 GPUs (the paper's configuration), this translates to a free accuracy improvement — same hardware, same training time, strictly better loss. The training loss gap to BF16 is not closed (the remaining ~87% is still there), but the improvement is obtained without adding any new auxiliary operations, without extending training time, and without changing the optimizer or learning rate schedule. For production training pipelines that already use the NVFP4 recipe (RHT, stochastic rounding, high-precision layers), adding 4/6 is a low-risk incremental improvement. The key deployment consideration is that the MAE selection rule and reduced tensor scale (using 256 instead of 448) should be used, not the MSE rule that worked better for PTQ — a configuration detail that the paper's Appendix A makes clear.

2. AWQ + 4/6 for weight-and-activation quantized inference serving. The paper's strongest PTQ result is AWQ combined with 4/6, which achieves the best overall WikiText-2 perplexity (average 11.58 across Llama-3 and Qwen3 models, Table 5) and improves downstream task accuracy for nearly all model-method-task combinations (Tables 6, 7). For a team deploying a Llama-3-8B or Qwen3-8B model for inference with NVFP4 weight and activation quantization, the pipeline is: apply AWQ per-channel scaling during offline calibration, then quantize weights with 4/6 (MSE selection), and at inference time quantize activations with 4/6 (also MSE selection). This adds negligible latency to the inference quantization step (the <15% kernel overhead applies, but quantization is a small fraction of total inference time) while recovering approximately 19.9% of the perplexity gap between AWQ NVFP4 and BF16. The practical benefit is higher-quality model outputs at the same memory footprint and throughput as standard AWQ-quantized NVFP4 inference. This is a direct upgrade for any serving system using AWQ with NVFP4 — no changes to the serving infrastructure, GEMM kernels, or request batching logic are needed, since the quantized tensors are standard NVFP4 format regardless of whether 4/6 was used during quantization.

3. Edge and on-device deployment of smaller NVFP4-quantized models. The paper's results on smaller models — Llama-3.2-1B and Qwen3-1.7B — show that 4/6's relative benefit is often largest at smaller scales. For Llama-3.2-1B, RTN + 4/6 improves WikiText-2 perplexity from 14.27 to 13.84 (a 0.43 point reduction, Table 4), and AWQ + 4/6 improves average downstream task accuracy from 53.2 to 54.1 (a 0.9 point improvement over standard AWQ, Table 6). While these gaps to BF16 remain substantial (BF16 accuracy is 56.7), the improvements come at zero additional inference cost and may be the difference between acceptable and unacceptable quality for a particular deployment. In an on-device setting where a 1B-parameter model must fit in limited memory (phone, laptop, embedded system), and where NVFP4 quantization is used to reduce the model's footprint below the memory budget, every fraction of a perplexity point matters — it can determine whether the model produces coherent responses or degrades to unusable quality. The zero-cost nature of 4/6 (it only affects the offline quantization step) makes it a strictly dominant choice for these scenarios: there is no reason not to use 4/6 when quantizing a model for NVFP4 inference, since the quantized model is identical in format and inference cost regardless of whether 4/6 was used during quantization.

4. NVFP4 training for self-improvement and distillation pipelines. The paper's pre-training result — even without downstream evaluation — suggests a specific workflow for teams that train models on generated data. In a distillation or self-improvement pipeline (e.g., STaR, ReST, or iterative fine-tuning), a larger teacher model generates training data, and a smaller student model is trained on that data. Training the student model in NVFP4 with 4/6 could substantially reduce training cost (via the 4× speedup of NVFP4 GEMMs over BF16 on B200 GPUs) while keeping the training loss closer to BF16 than standard NVFP4 training would. The 13.0% loss improvement means the student trained with NVFP4 + 4/6 achieves a loss that would require approximately 13% less training compute (in tokens) under standard NVFP4 — or, equivalently, the student reaches a given loss threshold earlier, reducing total training FLOPs. For a pipeline that involves many rounds of student training (iterative distillation, RLHF with multiple policy updates), this per-round improvement compounds. The key enabling paper finding is Figure 8: the benefit can be achieved by applying 4/6 only to activations, keeping the implementation simple and the overhead minimal. The missing piece — which the paper does not provide — is whether the 13.0% training loss improvement translates to better downstream performance of the student model, which would need to be evaluated for the specific distillation setup.

When to Prefer This Method

The paper does not articulate an explicit tradeoff matrix against named alternatives, so a formal "Prefer A when / Prefer B when" matrix would be fabricated. However, the paper does position 4/6 relative to specific existing methods in ways that imply decision boundaries. These are the conditions under which the experimental evidence directly supports preferring 4/6:

• If you are already using NVFP4 with AWQ or SmoothQuant for post-training quantization, adding 4/6 is a strictly dominant choice: it improves perplexity and downstream task accuracy for every model evaluated in the paper, with no downside (Tables 5, 6, 7). The quantized model format is unchanged, so there are no inference-time compatibility issues. The paper provides no evidence that 4/6 ever degrades AWQ or SmoothQuant performance — every entry in Tables 5, 6, and 7 for these two methods shows improvement or, in rare individual task cases, statistical noise-level fluctuation.

• If you are using GPTQ for NVFP4 post-training quantization, the paper's evidence indicates that the naive "drop-in" approach to 4/6 can degrade performance (Section 4.2, Table 5). 4/6 should not be used with GPTQ without a modified integration that incorporates grid alignment into the Hessian-based optimization loop. The paper explicitly identifies this as future work, and until such an integration is developed and validated, standard GPTQ without 4/6 is the safer choice based on the evidence presented.

• If you are pre-training with NVFP4 using the NVIDIA training recipe (RHT, stochastic rounding, 2D block quantization, high-precision layer retention), the paper's evidence supports adding 4/6 on activations only, with MAE-based scale selection and the reduced FP32 tensor scale (using 256, not 448) as the $M_{\text{FP8}}$ constant. This configuration reduces training loss by 13.0% toward BF16 at minimal overhead (Section 4.1, Appendix A). The paper does not provide evidence that 4/6 helps with a fundamentally different training recipe (e.g., without RHT, or with a different optimizer), so the recommendation is tied to the specific recipe evaluated.

• If you are using MXFP4 rather than NVFP4, 4/6 is not applicable. The paper is explicit that MXFP4's E8M0 scale factors lack the granularity to represent the 1.5× increase needed for the M=4 alternative (Section 5.2). The paper does not evaluate 4/6 with MXFP4 and does not claim it would work. For MXFP4 deployments, the paper's contribution is the diagnostic insight (near-maximal values are the bottleneck), not the algorithmic solution — a different grid alignment strategy would be needed.

• If you are training or deploying at a scale substantially beyond the paper's evaluation (models >70B for PTQ, architectures other than Nemotron 3 Nano 30B-A3B for pre-training), the paper provides no direct evidence about 4/6's efficacy or optimal configuration. The PTQ results across six models from two families provide some cross-architecture validation, but the pre-training result is a single architecture and scale. Extrapolation is plausible (the mechanism should generalize) but unvalidated, and a practitioner operating at 100B+ scale should treat 4/6 as an experiment to validate internally rather than a proven method.