FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Search

ArXiv: 2308.03290

🎯 Pitch

Even though floating-point quantization is often assumed superior for training, this paper shows that with a proper one-shot search framework, low-precision floating-point models consistently outperform integer models at the same total bitwidth for inference. FLIQS eliminates the need for retraining by discovering per-layer formats—spanning both integer and exotic 14-variant floating-point types—that boost ResNet-18 accuracy by 1.31 points over prior mixed-precision methods. The framework also reveals a counterintuitive insight for fixed compute budgets: enlarging the architecture (wider channels) yields larger accuracy gains than increasing bitwidth precision.

---

1. Executive Summary

This paper introduces FLIQS (Floating-Point and Integer Quantization Search), the first one-shot mixed-precision quantization search framework that eliminates post-search retraining by incorporating a cosine entropy regularization schedule. Evaluated on ImageNet across multiple convolutional networks (ResNet-18/50, MobileNetV2, EfficientNet, InceptionV3) and a vision transformer (DeiT-B16), FLIQS uses a reinforcement learning controller to automatically assign per-layer numerical formats—spanning both integer (INT4–INT8) and low-precision floating-point (E2M1, E4M3, and 14 intermediate formats)—guided by a joint quality-cost reward. The method improves ResNet-18 accuracy by 1.31 percentage points over prior mixed-precision integer methods at equivalent model cost and achieves up to 0.98 percentage point gains over state-of-the-art FP8 models on MobileNetV2, while demonstrating that floating-point models consistently outperform integer counterparts at the same total bitwidth. When extended to jointly search quantization formats and neural architecture (kernel sizes and channel widths) on a MobileNetV2 search space, FLIQS improves ImageNet accuracy by 2.69 percentage points over quantization search alone at similar model cost, establishing that for fixed compute budgets larger models benefit more from expanding architectural dimensions than from increasing bitwidth precision.

2. Context and Motivation

The Core Problem: Hardware Diversity Outpaces Our Ability to Assign Numerical Formats

The fundamental problem this paper addresses is a mismatch between the rapid diversification of hardware-supported numerical formats and the primitive state of tools for deciding which format to use where in a deep neural network. Modern AI accelerators support a growing menagerie of precision options. The Google TPUv3 handles BF16, FP16, and INT8 alongside FP32. NVIDIA's Hopper architecture adds native FP8 support. FPGA-based systems, being reprogrammable at the gate level, can implement arbitrary bitwidths and exponent/mantissa splits — INT5, FP9, FP11, or any custom configuration an engineer dreams up.

This hardware flexibility is genuinely valuable. Different layers within the same network exhibit radically different sensitivity to quantization. The first convolutional layer, which processes raw input pixels with only three channels, often cannot tolerate aggressive quantization — its weights encode fine-grained edge and color detectors where small perturbations cascade through the entire network. Depthwise separable convolutions in MobileNet-style architectures have fundamentally different numerical properties than the dense 1×1 pointwise convolutions that surround them. The final classification layer needs precision to discriminate between similar classes. Manually assigning per-layer formats could theoretically exploit these heterogeneities, squeezing out efficiency gains that uniform quantization leaves on the table.

The problem, as the paper frames it (Section 1), is that "it is challenging to optimally assign per-layer formats since layers exhibit different quantization characteristics." A human engineer staring at a new model architecture on a new hardware platform faces a combinatorial explosion: for a ResNet-50 with approximately 50 convolutional layers and a search space of even 3 format options per layer, there are $3^{50} \approx 7 \times 10^{23}$ possible configurations. Exhaustive search is obviously impossible. Manual assignment, while feasible in restricted cases (e.g., "keep the first and last layers at INT8, quantize everything else to INT4"), becomes unreliable and suboptimal as the search space grows to include floating-point formats with different exponent/mantissa splits, each of which interacts differently with the layer's weight and activation distributions.

Why This Matters: Three Practical Forces Converging

The paper's motivation is not purely academic — it sits at the intersection of three practical trends that make automated mixed-precision search increasingly urgent.

First, the proliferation of deployment targets. The paper illustrates this in Figure 1(a): a single model architecture (ResNet, MobileNet, DeiT) may need to run on a TPU (with its specific format support), a GPU (with different format support), a CPU, or an FPGA. Each platform has different performance characteristics, numerical capabilities, and energy constraints. Manually tuning a model for each target is a multiplicative engineering burden. As the paper notes in Section 2, "companies, including AMD, Intel, NVIDIA, and Qualcomm, have recently agreed to adopt 8-bit floating-point (FP8) in future deep learning systems," meaning the number of supported formats will continue to grow, not shrink.

Second, the shift toward floating-point quantization for inference. Low-precision floating-point is being positioned as "the next generation format for DNN training and inference" (Section 2). Unlike integer formats, which place quantization grid points uniformly across the representable range, floating-point formats concentrate precision near zero — exactly where most neural network weight and activation values cluster, since they are typically centered around zero by batch normalization or layer normalization. This means floating-point formats can, in principle, achieve better accuracy than integer formats at the same total bitwidth by allocating their limited representation budget more intelligently. The FP8 standard specifies two variants: E4M3 (4 exponent bits, 3 mantissa bits) for the forward pass, offering finer precision near zero, and E5M2 (5 exponent bits, 2 mantissa bits) for gradients, offering wider dynamic range. Prior work on FP8 inference (HFP8 by Sun et al., MPFP by Mellempudi et al., FPQuant by Kuzmin et al.) has demonstrated the potential, but exclusively used uniform precision — applying the same FP8 format to every layer. The paper argues that mixed-precision floating-point, where different layers use different exponent/mantissa splits (e.g., E2M5 for one layer, E3M4 for another, E4M3 for yet another), should yield further gains, but no automated method existed to find these assignments.

Third, the interaction between quantization and neural architecture design. When designing an efficient model, there is a fundamental allocation problem: given a fixed compute budget (measured in bit-operations or BOPs), should one invest in more channels, larger kernels, or higher numerical precision? A model with wider channels at lower precision may outperform a narrower model at higher precision, but the optimal trade-off is architecture-dependent and dataset-dependent. Without an automated search that jointly optimizes over both architectural and quantization dimensions, practitioners are left guessing. The paper frames this as a natural extension of Neural Architecture Search (NAS) into the quantization domain — what it calls "Quantization Neural Architecture Search" or QNAS.

Where Prior Approaches Fall Short

The paper organizes prior work into a taxonomy (Figure 1b) along two axes: when quantization decisions are made relative to training (post-training vs. during training) and how the search is conducted (reinforcement learning-based vs. differentiable).

Post-Training Quantization (PTQ) Searches: Cheap but Suboptimal

HAQ (Wang et al.) and ReLeQ (Elthakeb et al.) both use reinforcement learning to allocate per-layer bitwidths, but they operate in a post-training regime. The model is first trained at full precision, then the search runs over the frozen weights to find a quantization configuration, which is then applied without further training. The HAWQ series (Dong et al., Yao et al., extended through HAWQ-V3) refines this by using the Hessian spectrum to estimate layer sensitivity — layers with a flatter loss landscape around the optimum can tolerate more aggressive quantization — and formulating the bitwidth assignment as a constrained integer linear programming problem.

The fundamental limitation of PTQ searches, as the paper identifies (Section 2), is that "these methods cannot take advantage of the higher accuracy and more accurate feedback provided by quantization-aware training (QAT) during the search." During PTQ search, the model weights are frozen in their full-precision-optimized state. But quantization-aware training, which simulates quantization in the forward pass and allows gradients to flow through the straight-through estimator, enables the weights to adapt to the quantization noise — the model learns to produce weight distributions that are more amenable to quantization. A PTQ search misses this co-adaptation entirely. It sees only the model's sensitivity to quantization given the current weight configuration, not what the model could achieve if allowed to retrain under the quantized constraints.

In Figure 1(b), the PTQ quadrant is labeled "Lower Accuracy" — a fair characterization given that methods in this category concede 1–2 percentage points of accuracy compared to QAT-based approaches on ImageNet, a gap that is often unacceptable in deployment.

Quantization-Aware Training Searches: Accurate but Expensive and Brittle in Different Ways

The alternative — performing the search during QAT — has been pursued by two camps, each with their own flaws.

Differentiable NAS approaches (DNAS, EDMIPS, BatchQuant) create a "super-network" during training where, for each layer, the output is formed as a weighted combination of outputs from multiple quantization branches (e.g., one branch operating at INT4, one at INT8, one at BF16). The branch weights are trained jointly with the model weights through gradient descent, typically alternating optimization between the weight parameters and the architectural parameters. At the end of training, the highest-weight branch is selected for each layer, producing the final mixed-precision configuration.

This is elegantly unified — the whole system trains end-to-end with standard backpropagation — but it carries a severe memory cost. As the paper quantifies in Table 6 (Section A.6), a branched approach using 3 quantization options per layer on a ResNet-18 with batch size 32 requires 92.6 MiB for gradients and 220.8 MiB for activations — roughly 3× more memory than FLIQS's 46.8 MiB and 73.6 MiB respectively — because each branch replicates the intermediate activations and their gradients. For larger models (ResNet-50, DeiT-B16, EfficientNet), this memory multiplier becomes prohibitive, especially on accelerator hardware with limited on-device memory. The paper explicitly identifies this as the branching penalty: "because they replicate the weights and activations, they incur higher memory and computational costs compared to RL-based methods" (Section 2).

Moreover, differentiable searches require retraining after the search concludes. The search produces a soft selection (branch weights), but to deploy the model, you must extract the hard configuration (one branch per layer) and retrain from scratch or fine-tune, because the model weights have been co-adapted to the averaged outputs of multiple quantization branches — a regime that doesn't exist at deployment. This retraining step adds computational cost and introduces the possibility that the configuration found during the soft-search phase is not optimal for the hard-quantized deployment model.

RL-based QAT approaches avoid the branching memory overhead. The controller samples a discrete configuration per layer at each training step, applies it to the model, and updates its policy based on the resulting reward. However, prior methods in this category had an unresolved issue: the search itself interferes with model training. Each time the controller switches a layer's bitwidth, the weights need to adapt to the new quantization noise characteristics. The paper formalizes this as switching error in Section 4 (Equation 5): when the quantization format changes from bitwidth $k_1$ to bitwidth $k_2$, the difference between the quantized values $|Q(x; k_2) - Q(x; k_1)|$ acts as an additional perturbation to the weights, on top of the standard quantization error. This interference, if uncontrolled, prevents convergence to a high-quality solution. As a consequence, prior RL-based methods required a full retraining pass after the search to recover accuracy — the model trained during search was essentially a noisy intermediate, not the final deployable artifact.

This is the specific technical gap that FLIQS fills: no prior method could perform the search during QAT and directly produce a deployable model without retraining. PTQ methods didn't need retraining but sacrificed accuracy. Differentiable methods achieved good accuracy but required retraining and consumed excessive memory. Prior RL-based methods required retraining to recover from search-induced noise.

How FLIQS Positions Itself

The paper's positioning is explicit in Figure 1(b): FLIQS occupies a unique quadrant — it achieves the higher accuracy characteristic of QAT-based searches while eliminating retraining (labeled "✓ No Retraining" and "✓ Higher Accuracy"). This is enabled by the central technical innovation: cosine entropy regularization (Section 4, Equation 9), which forces the RL policy to converge to a deterministic configuration by the end of training, allowing the final model weights to train under stable quantization conditions without the continual perturbations of the search process.

The positioning is also architectural. By using an RL controller that samples discrete configurations rather than branching over all options, FLIQS avoids the memory explosion of differentiable methods. The paper argues this makes it scalable to larger models — DeiT-B16, with 10× the cost of BF16 MobileNetV2, is searched successfully, something the paper implies would be challenging for branching-based differentiable methods.

Beyond the immediate technical contribution, the paper positions itself as enabling a broader set of investigations that were previously impractical:

• First large-scale comparison of integer vs. floating-point mixed-precision networks. Prior work on integer mixed-precision (HAWQ, EDMIPS) and floating-point uniform-precision (HFP8, MPFP) had been pursued in separate research threads. FLIQS provides a unified framework where integer and floating-point formats compete on an equal footing in the same search space, allowing the paper to empirically establish that floating-point consistently outperforms integer at equal bitwidth — a finding that has implications for hardware design (accelerators should prioritize floating-point support) and model deployment strategies.

• First exploration of mixed-precision floating-point quantization search. Prior floating-point quantization work (HFP8, MPFP, FPQuant) applied a single FP8 format to all layers. FLIQS introduces a search space with 16 floating-point formats spanning 4–8 total bits, with varying exponent/mantissa ratios (E2M1 through E6M1, as detailed in Table 5), and demonstrates that allowing different layers to use different floating-point formats yields accuracy improvements over uniform FP8 — for instance, improving MobileNetV2 by up to 0.98 percentage points over prior state-of-the-art FP8 models.

• First joint floating-point quantization and neural architecture search. By extending FLIQS to FLIQNAS, the paper provides the first empirical evidence for how compute should be allocated across bitwidth and architectural dimensions in floating-point models, establishing a design principle: for fixed compute budgets, expanding architectural dimensions (channels, kernel sizes) yields greater accuracy returns than increasing numerical precision.

The paper is also careful to acknowledge boundaries it does not claim to cross. It studies two search spaces: FLIQS-S (small, targeting existing hardware with support for powers-of-two formats: INT4, INT8, BF16 for integer; E2M1, E4M3, BF16 for floating point) and FLIQS-L (large, targeting reconfigurable hardware and co-design: all integer bitwidths from 4 to 8, all floating-point formats from 4 to 8 bits). It does not claim to solve the latency modeling problem — it uses BOPs (bit operations) as a hardware-agnostic quadratic cost proxy, which it validates against FPGA LUT counts in Table 2 — and it does not address the problem of how to efficiently implement the resulting non-uniform bitwidth models on fixed-function hardware (though GPU latency results in Table 3 suggest the overhead is manageable on Turing-generation GPUs).

In summary, the paper addresses a gap that is simultaneously methodological (no one-shot QAT search without retraining existed), empirical (no large-scale comparison of mixed-precision integer vs. floating-point was available), and practical (hardware diversity is creating an urgent need for automated deployment tools that work across models and platforms).

3. Technical Approach

3.1 Reader Orientation

FLIQS is a reinforcement learning controller embedded inside the training loop that automatically assigns a numerical format (e.g., INT4, INT8, E4M3, or any of 16 floating-point variants) to every layer of a neural network while the network trains, producing a fully trained mixed-precision model ready for deployment without any post-search retraining. The system solves the problem of optimal mixed-precision format assignment — a combinatorial explosion that manual tuning cannot handle — by formalizing it as a one-shot sequential decision process where the controller learns to propose hardware-aware configurations that maximize validation accuracy subject to a computational budget constraint, all while the cosine entropy regularization schedule gradually eliminates the search-induced noise so that the model converges cleanly by the end of training.

3.2 Big-Picture Architecture (Diagram in Words)

The FLIQS system has five major components interacting inside a modified training loop:

1. Quantized Model Under Training — the target neural network (e.g., ResNet-18, MobileNetV2, DeiT-B16) whose weights are being optimized via standard gradient descent, but whose layers are dynamically quantized to different formats at each training step based on the controller's current proposal.

2. RL Controller (Policy Network) — a per-layer probability distribution parameterized by learnable logits $\theta_{l,\alpha}$ that is sampled at each training step to produce a discrete architecture configuration $\boldsymbol{\alpha} = \{\alpha_1, \alpha_2, \ldots, \alpha_L\}$ specifying the numerical format for each layer $l$.

3. Cost Model — a quadratic function that maps each layer's configuration to a scalar cost (measured in Bit Operations, or BOPs), computed as $b(\alpha)^2 \times \text{MAC}_l(\alpha)$, where $b(\alpha)$ is the total bitwidth and $\text{MAC}_l(\alpha)$ is the number of multiply-accumulates.

4. Reward Function — combines the model's validation quality signal $Q(\boldsymbol{\alpha})$ (typically classification accuracy on a held-out set) with the normalized model cost relative to a user-specified target $C_T$, producing a scalar reward $r(\boldsymbol{\alpha}) = Q(\boldsymbol{\alpha}) + \gamma\left|\frac{\sum_l C_l(\alpha)}{C_T} - 1\right|$.

5. Entropy Regularization Schedule — a cosine annealing mechanism applied to the controller's policy entropy $H_M = -\sum_l \sum_k \pi_l(k) \log \pi_l(k)$ that encourages exploration early in training (high entropy, the controller samples diverse formats) and forces convergence late in training (low entropy, the controller settles on a fixed configuration), thereby eliminating switching noise and making retraining unnecessary.

Information flow at each training step: The controller samples a configuration → the model's layers are quantized accordingly (with switchable clipping thresholds) → a combined training batch (forward + backward for weight updates) and validation batch (forward only for quality signal) are processed → the cost model computes total BOPs → the reward is computed and compared to a running average to produce an advantage signal → the REINFORCE algorithm updates the controller's policy logits → entropy regularization is applied with a cosine decay factor → the process repeats for the next step.

3.3 Roadmap for the Deep Dive

• First, the formal problem statement and cost model (Equation 1), because the cost function defines what "optimal" means and constrains every subsequent design choice. Understanding BOPs — why they scale quadratically in bitwidth — is the foundation for interpreting all experimental results.

• Second, the RL controller and policy parameterization (Equation 2–3), establishing how the controller represents its beliefs about which formats are best for each layer, how it samples discrete configurations at each step, and how the REINFORCE update rule translates reward advantage into policy gradient steps.

• Third, the reward function and advantage computation, because the quality-cost trade-off drives the search: we need to understand how the absolute reward in Equation 1 is converted into the advantage signal $r_\Delta(\boldsymbol{\alpha}) = \bar{r}(\boldsymbol{\alpha}) - r(\boldsymbol{\alpha})$ that determines whether the current configuration is better or worse than the running average, and how the cost scalar $\gamma$ governs the accuracy-efficiency Pareto frontier.

• Fourth, the switching error analysis and entropy regularization (Equations 4–9), which is the paper's central technical innovation for eliminating retraining. This is the mechanism that makes FLIQS a true one-shot method — understanding how format switching introduces optimization noise, why that noise is proportional to policy entropy, and how the cosine schedule forces convergence requires working through the quantization error math carefully.

• Fifth, the quantization mechanics and two-phase training protocol, covering how the base model actually applies quantization (scale factors, clipping thresholds, straight-through estimator), why activations are quantized later than weights, and how switchable clipping thresholds adapt to changing bitwidths.

• Sixth, the search space design (FLIQS-S vs. FLIQS-L), covering which integer and floating-point formats are included and why the distinction between "small" and "large" search spaces maps to real hardware constraints (fixed-function accelerators vs. reconfigurable logic).

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and methods paper whose core idea is that an RL controller with entropy regularization can be embedded into quantization-aware training to perform a one-shot mixed-precision search that directly outputs a deployable model, with the specific technical innovation being the formalization of search interference as switching error and the use of cosine entropy annealing to eliminate it.

---

Cost Model: Bit Operations (BOPs) as a Hardware-Agnostic Proxy

The paper needs a differentiable (or at least evaluable) cost function that can compare the computational expense of configurations using different bitwidths without committing to a specific hardware implementation. The choice is Bit Operations (BOPs), defined per layer and summed across all layers:

$C_l(\alpha) = b(\alpha)^2 \cdot \text{MAC}_l(\alpha)$

where $b(\alpha)$ is the total bitwidth (sum of all bits in the numerical representation) of the format specified by architecture $\alpha$ for layer $l$, and $\text{MAC}_l(\alpha)$ is the number of multiply-accumulate operations in that layer given the current architectural configuration.

What it computes: For each layer, multiply the square of the bitwidth by the raw operation count. The total model cost $C(\boldsymbol{\alpha}) = \sum_l C_l(\alpha)$ is the sum over all layers. This produces a scalar in units of "bit-operations" — essentially a weighted operation count where each MAC is weighted by the quadratic cost of performing arithmetic at that precision.

Why this form, specifically the quadratic dependence on $b(\alpha)$: The paper justifies this by referencing prior work on hardware modeling: "quadratic cost models, which predict power and area, are particularly useful in model-accelerator co-design where multipliers dominate resources and scale quadratically in power and area." An $N$-bit by $N$-bit integer multiplier requires approximately $N^2$ gate operations (the partial-product array grows as $N \times N$), so the energy and area cost of a MAC scale with $b(\alpha)^2$. A linear model ($b \times \text{MAC}$) would underestimate the cost of higher-precision operations relative to lower-precision ones. A cubic or exponential model would over-penalize precision. The quadratic is the physically grounded choice for digital multiplier arrays.

The paper validates this quadratic assumption empirically in Table 2 (Section 5), showing that FPGA look-up table (LUT) counts — which act as a proxy for silicon area and power — indeed scale quadratically with bitwidth: a ResNet-18 block at INT4 uses 42.8K LUTs, at INT6 uses 48.3K (1.13×), at INT8 uses 67.6K (1.58×), closely tracking the $(\text{bitwidth})^2$ ratio.

Why a bit-operations model rather than latency profiling: The paper targets diverse hardware (TPU, GPU, CPU, FPGA), and per-platform latency models would be platform-specific and require engineering effort for each new target. BOPs are hardware-agnostic. The paper separately validates that BOPs-correlated configurations achieve real speedups on GPUs (Table 3, ResNet-50: FLIQS-S achieves 1.30× speedup over INT8 with only 1% lower speed than uniform INT4, while being 1.31 percentage points more accurate). The cost model provides a proxy for efficiency across all hardware, not a precise latency prediction for any specific one.

---

The Absolute Reward Function: Trading Off Quality and Cost

The controller needs a scalar signal that tells it whether the configuration it just sampled was "good." The paper uses an absolute reward function adapted from prior work on weight-sharing NAS:

$r(\boldsymbol{\alpha}) = Q(\boldsymbol{\alpha}) + \gamma \left| \frac{\sum_l C_l(\alpha)}{C_T} - 1 \right|$

where $Q(\boldsymbol{\alpha})$ is the quality signal (typically validation accuracy on a held-out batch, measured as the model's Top-1 accuracy on ImageNet validation data processed with the current quantized configuration), $\sum_l C_l(\alpha)$ is the total model BOPs computed as described above, $C_T$ is a user-specified cost target (a scalar with units of giga-BOPs, e.g., 30 GBOPs for a lightweight ResNet-18 variant), and $\gamma$ is a cost scalar that balances the importance of meeting the cost target versus maximizing accuracy.

What it computes: The reward is the validation accuracy plus (or minus) a penalty term that grows linearly with the absolute deviation of the model's actual cost from the target. If the model is exactly at the target cost, the second term is zero and the reward equals accuracy. If the model exceeds the target cost (too expensive), the absolute difference is positive and the reward is reduced (since $\gamma$ is positive, adding a penalty to the accuracy term). If the model is cheaper than the target, the absolute difference is still positive but small — the reward is slightly penalized, encouraging the controller to "use up" its full budget rather than producing unnecessarily small models.

Why the absolute value form rather than a one-sided penalty: A one-sided penalty that only penalizes exceeding the target would cause the controller to produce trivially small models (all layers at the minimum bitwidth, minimal channels) because those configurations would never incur a penalty. The absolute value penalizes deviation in either direction, encouraging the controller to find configurations that exactly hit the target. This gives the user precise control over the accuracy-efficiency trade-off: setting $C_T$ low produces aggressively quantized models (Figure 3 shows ResNet-18 FLIQS-S outputs ranging from 30 GBOPs to 117 GBOPs as the target increases); setting $C_T$ high allows more layers to use higher precision.

Why combine quality and cost into a single scalar rather than doing multi-objective optimization: The REINFORCE algorithm requires a scalar reward to compute the policy gradient. Multi-objective methods like Pareto optimization would add complexity without necessarily improving the final result, since $\gamma$ effectively sweeps the Pareto frontier. The paper does not ablate $\gamma$ explicitly, but varying $C_T$ (as in Figure 5, which shows Pareto curves across a range of model costs) implicitly explores the quality-cost trade-off surface.

---

RL Controller: Policy Parameterization and Sampling

The controller is not a monolithic neural network predicting the entire configuration at once. Instead, it maintains independent per-layer policies, each represented by a vector of logits that defines a categorical distribution over the discrete format options for that layer.

For each layer $l$ in the model (where a "layer" is a convolutional or linear operation — the paper applies quantization to weights and activations of each such operation separately), the controller stores a parameter vector $\boldsymbol{\theta}_l$ with one entry for each possible format $\alpha$ in the search space. The policy $\pi_l(\alpha)$ — the probability that layer $l$ is assigned format $\alpha$ at a given training step — is computed via softmax normalization:

$\pi_l(\alpha) = \frac{\exp(\theta_{l,\alpha})}{\sum_j \exp(\theta_{l,j})}$

where $\theta_{l,\alpha}$ is the learnable logit for format $\alpha$ at layer $l$, and the denominator sums over all $j$ formats in the search space for that layer.

What it computes: For each layer independently, the logits are exponentiated and normalized to produce a proper probability distribution (all values between 0 and 1, summing to 1). The controller then samples a discrete format $\alpha_l$ from this distribution — $\alpha_l \sim \pi_l(\alpha)$ — independently for each layer, producing the full configuration $\boldsymbol{\alpha} = \{\alpha_1, \ldots, \alpha_L\}$ applied to the model for that training step.

Why per-layer independent policies rather than a joint autoregressive model: A joint model (e.g., an LSTM that samples layer 1's format, then conditions on it to sample layer 2's format, etc.) could potentially capture inter-layer correlations — e.g., "if layer 3 uses high precision, layer 4 probably doesn't need to." However, the paper notes in Section 3 that "more sophisticated policy models, such as multi-layer perceptron models, offered no quality improvements while being more costly." The independent per-layer policy is computationally cheap (just $L \times K$ parameters for $L$ layers and $K$ format options), avoids the sequential sampling bottleneck at training time, and apparently captures sufficient structure — likely because the reward signal, which evaluates the entire joint configuration, induces implicit coordination: if certain combinations of layer formats consistently produce high rewards, the independent policies will converge to those combinations through the shared reward signal even without explicit coordination.

Why softmax parameterization rather than directly learning probabilities: The softmax maps unconstrained real-valued logits to the probability simplex automatically, ensuring that gradient-based optimization (which produces unbounded updates) never produces invalid probabilities (negative or non-summing-to-one). The logit space is unbounded, so the optimizer can freely adjust parameters without constraints.

Channel width search via masking: When FLIQS is extended to joint architecture search (FLIQNAS), the controller also proposes channel widths and kernel sizes. Channel widths are implemented by applying learned binary masks that zero out channels beyond the selected width, reusing the underlying weight tensor — this avoids duplicating weights for different width options, which would explode memory. The mask is applied as an element-wise multiplication: $w_{\text{effective}} = w \odot \text{mask}(\text{width})$, where $\text{mask}(\text{width})$ is a binary vector with ones for the first "width" channels and zeros thereafter. This means the model trains a single weight tensor, and different channel width proposals simply expose different subsets of it.

---

Policy Update: REINFORCE with Running-Average Baseline

At each training step, after the controller samples a configuration $\boldsymbol{\alpha}$ and the model processes a validation batch to produce the quality signal $Q(\boldsymbol{\alpha})$, the absolute reward $r(\boldsymbol{\alpha})$ is computed via Equation 1. However, the raw reward cannot be directly used as the policy gradient weight because it contains two confounding signals: (1) the model's accuracy naturally increases over the course of training regardless of architecture, so a configuration sampled late in training will have higher $r(\boldsymbol{\alpha})$ than one sampled early, even if the configuration itself is worse; and (2) different validation batches have different intrinsic difficulties, injecting noise.

The paper addresses this by using the advantage: the difference between the current reward and a running average of past rewards for the same configuration.

$r_\Delta(\boldsymbol{\alpha}) = \bar{r}(\boldsymbol{\alpha}) - r(\boldsymbol{\alpha})$

where $\bar{r}(\boldsymbol{\alpha})$ is the exponential moving average of rewards observed for architecture $\boldsymbol{\alpha}$ across previous training steps, and $r(\boldsymbol{\alpha})$ is the current reward.

What it computes: If the current configuration performs worse than its historical average ($r(\boldsymbol{\alpha}) < \bar{r}(\boldsymbol{\alpha})$), the advantage $r_\Delta$ is positive — the configuration is "bad" relative to expectations, and the policy should be adjusted away from it. If it performs better ($r(\boldsymbol{\alpha}) > \bar{r}(\boldsymbol{\alpha})$), $r_\Delta$ is negative — the configuration is "good," and the policy should reinforce it. The sign convention here follows the REINFORCE loss convention where the loss is $-r_\Delta \sum_l \log(\pi_l(\alpha_l))$, so positive advantage (bad performance) increases the loss and causes gradient descent to reduce the probability of the sampled action; negative advantage (good performance) reduces the loss and increases the probability.

Why a running average baseline rather than a learned value function: A learned value function (critic) would require additional parameters and training machinery. The running average is a simple, zero-overhead baseline that achieves the essential purpose of variance reduction in the REINFORCE estimator. It subtracts out the slow trend of accuracy improvement over training, leaving only the architecture-specific signal. The downside is that it cannot capture config-specific trends (e.g., "this config is improving faster than others as training progresses"), but the paper's empirical results suggest this is not a critical limitation.

With the advantage computed, the controller's policy parameters $\boldsymbol{\theta}$ (the per-layer logits) are updated via the REINFORCE gradient estimator:

$\mathcal{L}_\theta = -r_\Delta(\boldsymbol{\alpha}) \sum_l \log(\alpha_l \sim \pi_l(\alpha))$

$\boldsymbol{\theta} \leftarrow \boldsymbol{\theta} + \eta \nabla_\theta \mathcal{L}_\theta$

where $\eta$ is the RL learning rate (set to $4.6 \times 10^{-3}$ per Section A.4), and the gradient $\nabla_\theta \mathcal{L}_\theta$ is computed via standard backpropagation through the log-probability term.

What it computes: The loss is the negative advantage times the sum of log-probabilities of the sampled actions. If the advantage is positive (bad configuration), the loss is positive — gradient descent (note the $+$ in the update rule, but $\mathcal{L}_\theta$ has a leading negative sign, making the effective direction gradient ascent on the log-probabilities when advantage is negative) reduces the log-probability of the sampled actions, reducing the controller's likelihood of picking them again. If the advantage is negative (good configuration), gradient descent increases the log-probabilities, reinforcing the good choices.

Why REINFORCE rather than PPO or other actor-critic methods: The paper evaluated PPO and more sophisticated policy models and found "no quality improvements while being more costly" (Section 3). REINFORCE with a running-average baseline is the simplest policy gradient method — it requires only the policy parameters, no value function, no importance sampling corrections, no clipping. Its simplicity matters because the controller runs inside the training loop; any additional complexity would slow down every training step, which is unacceptable when training ImageNet-scale models for hundreds of epochs.

RL controller warmup period: The controller does not begin updating immediately. For the first 25% of training steps (the "warmup"), configurations are sampled uniformly at random from the search space, allowing the reward signal to stabilize and the quality measurements to become meaningful before the controller starts adapting its policy. This prevents early random fluctuations in validation accuracy from locking in a suboptimal policy.

---

Switching Error: Formalizing Why Search Interferes with Training

The central technical challenge for one-shot quantization search is that the search process itself disturbs the model training. When the controller changes a layer's bitwidth from one step to the next, the weights of that layer — which have been adapting to the quantization noise characteristics of the previous bitwidth — suddenly face a different noise profile. The paper formalizes this as switching error.

To understand switching error, first consider the standard symmetric integer quantizer. Given a full-precision value $x$ (a weight or activation), the quantizer maps it to a discrete grid:

$Q(x; s) = \lfloor x \cdot s \rceil / s$

where $s = (2^{k-1} - 1) / \sigma_T$ is the scale factor, $k$ is the bitwidth, and $\sigma_T$ is the clipping threshold (values beyond $\pm\sigma_T$ are clamped before quantization). The rounding operator $\lfloor \cdot \rceil$ maps to the nearest integer.

The absolute quantization error for a single value is:

$\Delta(x; s) = |Q(x; s) - x|$

which measures how much information is lost by representing $x$ in the quantized format. This error is typically modeled as approximately uniform over $[-1/(2s), +1/(2s)]$ for values within the clipping range (the rounding error), with larger errors for values outside the clipping range (the clipping error).

Now, when the bitwidth changes between two training steps — say, from $k_1$ to $k_2$ — the switching error is defined as:

$\Delta_S(x; k_1, k_2) = |Q(x; k_2) - Q(x; k_1)|$

where the quantization functions now explicitly depend only on bitwidth (assuming a fixed clipping threshold $\sigma_T$ for simplicity).

What it computes: The element-wise absolute difference between what the value $x$ quantizes to under format $k_1$ and what it quantizes to under format $k_2$. This is NOT the quantization error relative to full-precision (which would involve $|Q(x; k) - x|$), but rather the discrepancy between two different quantized representations of the same underlying full-precision value.

Why switching error matters for training dynamics: Consider what happens at training step $t$ when the controller switches layer $l$ from format $k_1$ at step $t-1$ to format $k_2$ at step $t$. At step $t-1$, the optimizer computed a gradient based on the loss evaluated with the weights quantized to $k_1$, and took a step $\Delta w$ to update the weights. That update was optimal given that the weights were represented in $k_1$. At step $t$, the weights are quantized to $k_2$, which means the effective weight values seen by the forward pass have jumped by $\Delta_S$ — even though the underlying full-precision weights changed only by $\Delta w$. This additional perturbation acts as noise, pushing the optimizer away from the true gradient direction. The larger the gap between $k_1$ and $k_2$, the larger the switching error, and the more the search interferes with training.

Figure 4(a) illustrates this quantitatively: when one bitwidth is small (e.g., 4 bits) and the other is large (e.g., 8 bits), the switching error is relatively large because the quantization grids are vastly different — 4-bit quantization has only 15 representable positive values, while 8-bit has 255, so the mapping from a full-precision value to its quantized representation changes dramatically. When both bitwidths are large (e.g., 7 and 8 bits), the switching error is small because the grids are similar.

Quantitative behavior: The paper fits an exponential decay model $\Delta_S \propto A e^{-B \cdot \min(k_1, k_2)} + C$ to the switching error. The key insight is that the switching error decays with the smaller of the two bitwidths — a switch between 4 and 8 bits is about as disruptive as a switch between 4 and 5 bits, because in both cases one endpoint is at the low-precision extreme where the quantization grid is coarse. This means that search spaces including low-bitwidth options (like INT4 or E2M1) have inherently higher switching error than those restricted to higher precision.

---

Entropy Regularization: Eliminating Retraining via Controlled Policy Convergence

The switching error analysis reveals that search interference depends on how often and how drastically the controller changes its mind about layer formats. Enter policy entropy.

For a single layer $l$ with policy $\pi_l$ over $K$ format options, the entropy is:

$H(\pi_l) = -\sum_{k=1}^K \pi_l(k) \log \pi_l(k)$

The total model entropy $H_M$ is the sum over all layers:

$H_M = -\sum_{l \in M} \sum_k \pi_l(k) \log \pi_l(k)$

What entropy measures: High entropy means the policy is nearly uniform — the controller is equally likely to pick any format, leading to frequent, large-magnitude switches. Low entropy means the policy is nearly deterministic — the controller almost always picks the same format, leading to rare, small-magnitude switches. As entropy approaches $-\infty$ (the limiting case of a fully deterministic policy with zero probability on all options except one), the controller never switches, so $\Delta_S \to 0$ and the search no longer interferes with training at all.

The crucial insight formalized in Equation 6 is:

$\mathbb{E}[\Delta_S(x; k_1, k_2)] \propto H(\pi_l)$

where the expectation is over the controller's sampling distribution. When entropy is high, the controller frequently samples different bitwidths on consecutive steps, and the expected switching error — the additional optimization noise — is proportionally large. As entropy decreases, switches become less frequent and less drastic (when they do occur, the sampled bitwidths tend to be similar because the policy has concentrated probability mass on a narrow set of options).

Why this relation holds: The paper sketches a Gaussian approximation: model $\pi_l(k) \sim \mathcal{N}(k; \mu, \sigma)$, which has entropy $H = \frac{1}{2}\log(2\pi e \sigma^2)$. As entropy decreases, $\sigma \to 0$, which means the probability of sampling $k_1$ and $k_2$ far apart goes to zero. In the limit, $k_1 = k_2$ always, and $\Delta_S(x; k, k) = 0$ (a value quantized and then quantized again at the same bitwidth is unchanged, assuming deterministic rounding). The formal statement is:

$\lim_{H \to -\infty} \mathbb{E}[\Delta_S(x; k_1, k_2)] = \mathbb{E}\left[\lim_{k_1 \to k_2} \Delta_S(x; k_1, k_2)\right] = \mathbb{E}[\Delta_S(x; k_2, k_2)] = 0$

What this means operationally: The interference of the search on model training can be interpreted as additional optimization noise whose magnitude is proportional to the controller's entropy. Standard SGD convergence theory says that optimization noise prevents exact convergence but allows convergence to a "noise ball" around the optimum whose radius increases with noise variance. Therefore, to achieve high final accuracy without retraining, the entropy must be driven low by the end of training so that the noise ball shrinks to a tight neighborhood of the optimum, allowing the model to effectively converge as if it had been trained at a fixed precision all along.

The mechanism for controlling entropy is entropy regularization: adding a bonus to the policy loss that encourages the controller to maintain a certain entropy level. The regularized loss is:

$\mathcal{L} = \mathcal{L}_\theta - \beta_H H_M$

where $\mathcal{L}_\theta$ is the REINFORCE policy loss from Equation 3, $H_M$ is the total model entropy, and $\beta_H$ is a coefficient controlling the strength of regularization.

Why the minus sign: The policy loss $\mathcal{L}_\theta$ is minimized by gradient descent. Adding $- \beta_H H_M$ means that gradient descent tries to maximize entropy (because the negative of a negative is positive — minimizing $-\beta_H H_M$ is equivalent to maximizing $H_M$). When $\beta_H$ is positive, the controller is encouraged to maintain higher entropy (more exploration, more uniform policies). When $\beta_H$ is zero, the controller unconstrainedly converges to deterministic policies (entropy decreases naturally as the REINFORCE updates reinforce good configurations).

The cosine entropy regularization schedule is the specific schedule that makes one-shot search work. A constant $\beta_H$ cannot work: if $\beta_H$ is too high throughout training, the controller never converges and the switching noise prevents the model from reaching high accuracy. If $\beta_H$ is too low, the controller converges too early — before the quality signal is reliable — locking in a suboptimal configuration. The solution is to start with high entropy (exploration) and gradually reduce it (exploitation):

$\beta_H^{\text{cos}} = -0.5 \beta_H^{\text{end}} (1 + \cos(\pi s)) + \beta_H^{\text{end}}$

where $s \in [0, 1]$ is the training progress (fraction of total steps completed), and $\beta_H^{\text{end}} = 0.5$ is the final entropy regularization coefficient.

What this schedule does: At the start of training ($s = 0$), $\cos(\pi \cdot 0) = 1$, so $\beta_H^{\text{cos}} = -0.5 \cdot 0.5 \cdot 2 + 0.5 = 0$ — no regularization, allowing the controller to freely explore high-entropy policies during the warmup period (which, recall, already uses uniform random sampling for the first 25% of steps). As training progresses, $\cos(\pi s)$ decreases from 1 to 0 at $s = 0.5$, making $\beta_H^{\text{cos}} = -0.5 \cdot 0.5 \cdot 1 + 0.5 = 0.25$ at the midpoint. At the end of training ($s = 1$), $\cos(\pi) = -1$, so $\beta_H^{\text{cos}} = -0.5 \cdot 0.5 \cdot 0 + 0.5 = 0.5$ — the maximum regularization strength.

Wait — this seems backwards. Positive $\beta_H$ encourages higher entropy (exploration). So ending with $\beta_H = 0.5$ would mean the regularization is strongest at the end of training, keeping entropy high — the opposite of what we want for convergence.

The resolution is that the regularization coefficient $\beta_H$ is subtracted in the loss: $\mathcal{L} = \mathcal{L}_\theta - \beta_H H_M$. But the paper's description and Figure 4(b) clarify that the schedule actually reduces entropy over time. Looking more carefully: Equation 9 is defined with a leading negative sign on the cosine term, meaning it produces a schedule that starts high and ends low. Recomputing: at $s=0$, $\cos(0)=1$, so $\beta_H^{\text{cos}} = -0.5(0.5)(2) + 0.5 = 0$. At $s=0.5$, $\cos(\pi/2)=0$, so $\beta_H^{\text{cos}} = -0.5(0.5)(1) + 0.5 = 0.25$. At $s=1$, $\cos(\pi)=-1$, so $\beta_H^{\text{cos}} = -0.5(0.5)(0) + 0.5 = 0.5$.

This is indeed monotonically increasing. So the regularization bonus for entropy grows over time. Why would adding a growing bonus for high entropy cause entropy to decrease? The answer is that the schedule is not directly setting entropy — the REINFORCE updates are simultaneously pushing entropy down by reinforcing specific configurations. The entropy regularization term acts as a counter-force that prevents entropy from collapsing too early. Without it, REINFORCE would rapidly drive the policy to deterministic (zero-entropy) choices during early training when the quality signal is unreliable, locking in premature commitments. By adding an increasing entropy bonus, the schedule fights premature convergence: it keeps the policy exploratory enough early on to gather diverse feedback, then gradually reduces the counter-force, allowing REINFORCE to eventually win and drive entropy down toward zero by the end.

Figure 4(b) confirms this interpretation empirically: the left panel shows entropy decreasing over the course of training (from high exploration to low entropy). The right panel shows ImageNet accuracy as a function of $\beta_H$ — higher entropy regularization produces higher accuracy, confirming that preventing premature convergence is beneficial.

Retraining ablation (Appendix A.11, Table 8): The paper explicitly compares FLIQS models with and without retraining. For ResNet-18 FLIQS-S across six width multipliers (0.5× to 2.0×), the accuracy difference between no-retraining and retraining is within ±0.5 percentage points, with no systematic trend favoring retraining. This validates the claim that entropy regularization makes the searched model directly deployable.

---

Quantization Mechanics and Two-Phase Training Protocol

The underlying quantization pipeline that FLIQS applies to each layer follows a standard fake-quantization (simulated quantization) approach used in quantization-aware training. The mechanism, adapted from the open-source Accurate Quantized Training (AQT) library, processes each tensor through clipping, scaling, rounding, and rescaling:

$x_q = \lfloor s \cdot \sigma(x_f; \sigma_t) \rceil$

$\sigma(x_f; \sigma_t) = \max(-\sigma_t, \min(x_f, \sigma_t))$

where $x_f$ is the original full-precision tensor (weight or activation), $\sigma_t$ is the clipping threshold, $s$ is the scale factor, $\lfloor \cdot \rceil$ represents rounding to the nearest representable value in the target numerical format (nearest integer for integer formats; nearest representable floating-point number with the specified exponent and mantissa bitwidths for floating-point formats), and $\sigma(\cdot; \sigma_t)$ is the clip-to-range function that clamps values to $[-\sigma_t, \sigma_t]$.

The scale factor $s$ is computed dynamically from runtime statistics: $s = (2^{k-1} - 1) / \sigma_t$ for integer quantization with $k$ bits, where $\sigma_t$ is determined as a multiple (specifically 3–4×) of the profiled standard deviation of the tensor's distribution, computed by averaging statistics over a small number of profiling batches (the paper uses 10 batches of size 2048). This means the quantization grid is centered on zero and scaled so that the representable range $[-(2^{k-1}-1), +(2^{k-1}-1)]$ maps to $[-\sigma_t, +\sigma_t]$ in the original value space.

Two-phase quantization: The paper empirically found that quantizing weights and activations simultaneously from the start of training degrades final accuracy. Instead, FLIQS uses a two-phase protocol (Section A.2, Figure 2a):

1. Phase 1 (first 15–20% of training): Only weights are quantized. Activations remain in full precision (BF16). This allows the weights to adapt to quantization noise while the activations are still providing precise signals for gradient computation.

2. Phase 2 (remaining 80–85% of training): Both weights and activations are quantized. The weights have already partially adapted to their quantized representations, and now the model learns to produce activation distributions that are amenable to quantization as well.

Start step for activation quantization: The paper sweeps this hyperparameter on ResNet-50 (Table 10a in Appendix A.2). Starting activation quantization too early (step 1,000 out of 30,200 — i.e., single-phase) gives 75.59% Top-1. Starting too late (step 25,000 — effectively PTQ for activations) gives 75.54%. The optimum is around step 4,000–8,000 (roughly 13–26% of total steps), achieving ~76.11%. For ResNets and InceptionV3, FLIQS uses 7,500 steps as the activation quantization start step out of a total of 30,200 steps (approximately 25%). For MobileNetV2, EfficientNet, and DeiT (which train for ~112,000 steps or longer), 20,000 steps is used.

Straight-through estimator (STE): During the backward pass, the rounding operation $\lfloor \cdot \rceil$ has zero gradient almost everywhere (its derivative is zero except at the discontinuities, where it is undefined). To enable gradient-based training through the quantizer, the STE approximates the gradient as identity: $\frac{\partial x_q}{\partial x_f} \approx 1$ for values within the clipping range, and $0$ for values outside. This means the backward pass treats the quantizer as transparent — gradients flow through as if no quantization occurred — which is a biased but empirically effective gradient estimator.

Switchable clipping thresholds: The clipping threshold $\sigma_t$ that is optimal for a layer depends on its bitwidth. Lower bitwidths have coarser quantization grids, so they benefit from more aggressive clipping — clamping outliers to reduce the range of values that must be represented, which reduces the quantization step size and therefore the rounding error for in-distribution values. Higher bitwidths have finer grids, so they can tolerate a wider clipping range, keeping more outlier information. Figure 4(a) (right panel) illustrates this: the optimal clipping percentile (as a function of bitwidth $k$) shifts to lower percentiles (more aggressive clipping) for smaller $k$.

FLIQS addresses this by pre-computing clipping thresholds for each possible format using synthetic data or profiling data from the first phase of training. When the controller switches a layer's format, the corresponding pre-computed clipping threshold is applied. The paper states that "in general, pre-computing the thresholds leads to high-quality results with less complexity, and it is used for the experimental sections below" (Section 4).

Why switchable clipping matters: Without it, the clipping threshold would be fixed for a layer regardless of format. A threshold optimized for INT8 (wide range, less clipping) would cause excessive rounding error when the layer is temporarily assigned INT4 (coarse grid needs aggressive clipping). Conversely, a threshold optimized for INT4 would unnecessarily discard information when the layer is assigned INT8. Switchable clipping ensures that each format operates with an appropriate dynamic range, reducing the effective switching error by making both formats operate closer to their individual optima.

---

Search Space Design: FLIQS-S vs. FLIQS-L

The paper defines two search spaces targeting different hardware scenarios (Table 5, Appendix A.5):

FLIQS-S (Small) is designed for existing fixed-function accelerators where only power-of-two bitwidths are natively supported. It includes:

• Integer: INT4, INT8, BF16
• Floating-point: E2M1 (4 bits: 2 exponent, 1 mantissa), E4M3 (8 bits: 4 exponent, 3 mantissa), BF16

This is a 3-option search space per layer. For a 50-layer model, the search space size is $3^{50} \approx 7 \times 10^{23}$.

Why these specific formats: INT4 and INT8 are the standard low-precision integer formats supported in NVIDIA Ampere/Ada GPUs and Google TPUs. BF16 is the standard half-precision training format (same exponent range as FP32, truncated mantissa). E4M3 is the FP8 variant standardized for forward-pass inference (finer precision near zero, suitable for weights and activations). E2M1 is a custom 4-bit floating-point format included to provide a low-precision floating-point option comparable to INT4.

Why FP16 is excluded: "many platforms additionally support FP16, yet this format typically performs worse than BF16 in most common use cases" (Appendix A.5). BF16 has the same dynamic range as FP32 (8 exponent bits), making it more robust to overflow during training and inference, while FP16's 5 exponent bits can cause gradient underflow/overflow issues. Since BF16 dominates FP16 for the models studied, FP16 is omitted to keep the search space minimal.

FLIQS-L (Large) is designed for reconfigurable hardware (FPGAs) and hardware/accelerator co-design, where arbitrary bitwidths and format configurations can be synthesized. It includes:

• Integer: INT4, INT5, INT6, INT7, INT8, BF16 (6 options)
• Floating-point: All formats with total bitwidths between 4 and 8 bits, spanning E2M1, E2M2, E2M3, E2M4, E2M5, E3M1, E3M2, E3M3, E3M4, E4M1, E4M2, E4M3, E5M1, E5M2, E6M1, BF16 (16 options)

For the floating-point notation ExMy: E is the number of exponent bits, M is the number of mantissa bits. The total bitwidth is E + M + 1 (one sign bit). So E4M3 is $4+3+1=8$ bits total; E2M5 is $2+5+1=8$ bits total. The search space covers all combinations that produce total bitwidths from 4 to 8 (except obviously invalid ones like E1M6, which the paper does cover — the Table 5 list includes all 16 combinations that sum to 4–8 with at least 1 exponent bit and at least 1 mantissa bit).

Bias terms and subnormals: All custom formats support subnormals (gradual underflow) and do not support infinity. The exponent bias is selected such that the exponent range is symmetric about zero. However, the paper notes that "this bias term is not relevant to FLIQS, since continuing from prior work, it uses a profiled scale factor during training and search. This means that the bias term combines with the profiled scale factor and has no additional effect" (Appendix A.5). In practice, the explicit scale factor $s$ applied during fake quantization absorbs any shift in representable range that the bias would provide, making the format's dynamic range primarily controlled by the learned/profiled scale factor rather than the architectural bias parameter.

Why floating-point formats dominate integer: The paper's results (Figures 5, 11, 12) consistently show floating-point models outperforming integer models at the same total bitwidth. The reason is intrinsic to the number representations: integer formats distribute quantization points uniformly across the representable range, so values near zero (where most neural network weights and activations concentrate, due to weight decay, batch normalization centering, etc.) receive the same precision as values near the extremes. Floating-point formats concentrate precision near zero (denser representable values for small magnitudes) and spread out at larger magnitudes, better matching the heavy-tailed, zero-centered distributions typical of neural network tensors.

---

Joint Quantization and Architecture Search (FLIQNAS)

FLIQS can be extended to simultaneously search over quantization formats and architectural parameters — channel widths and kernel sizes — producing what the paper calls FLIQNAS. The architecture search space builds on the MobileNetV2 inverted bottleneck structure:

• Channel widths: Tunable filter widths on the inverted bottleneck projection layers (the 1×1 expansion and projection convolutions). Widths are adjusted by applying channel masks that dynamically zero out channels beyond the selected width, reusing the underlying full weight tensor. This means the model maintains a single set of weights at the maximum width, and different architectural proposals simply expose different subsets.

• Kernel sizes: Tunable kernel sizes on the central depthwise convolutional layers, with options typically including $3 \times 3$ and $5 \times 5$ (or larger). Multiple kernel size branches are maintained during the warmup phase using a probability schedule that starts at 1.0 (all branches active) and linearly decays to 0 (only one branch active) by the end of warmup, at which point the controller samples a single kernel size per layer.

The combined search space for FLIQNAS-S includes 230 tunable values across all layers, yielding "over $10^{100}$ configurations" (Section 6), compared to FLIQS-S's 53 options and approximately $10^{25}$ configurations. This enormous space size makes exhaustive search impossible, but the REINFORCE-based controller navigates it by learning from the reward signal, which evaluates each sampled joint configuration holistically.

Channel width vs. bitwidth trade-off: The paper's key finding from FLIQNAS is that for a fixed compute budget (measured in BOPs), expanding channels at lower precision generally yields better accuracy than keeping fewer channels at higher precision. The controller learns this implicitly through the reward signal: configurations with wider channels and lower bitwidths tend to produce higher validation accuracy for the same BOPs cost (since BOPs scale as $b(\alpha)^2$, reducing bitwidth from 8 to 4 reduces the cost by $4\times$, which can be reinvested into $4\times$ wider channels, and the wider channels provide more representational capacity that more than compensates for the coarser quantization). The paper notes a weaker trend for kernel size (larger kernels tend to accompany lower bitwidths), possibly because larger kernels provide greater spatial receptive field which partially compensates for reduced per-weight precision.

Kernel size branch management: During the RL controller warmup period (first 25% of training), the branches corresponding to different kernel sizes are sampled jointly with a probability schedule that starts at 1 (all active simultaneously, with their outputs averaged or summed) and decreases linearly to 0 (only the controller's sampled branch is active). This allows the weights to initially adapt to all possible kernel sizes before the search commits to specific choices, reducing the interference when the controller switches between kernel sizes post-warmup.

4. Key Insights and Innovations

Innovation 1: Framing Inference Precision Selection as a Retraining-Free One-Shot Sequential Decision Problem

Before FLIQS, the dominant assumption in mixed-precision quantization was that search and final training must be separate phases. Post-training quantization (PTQ) methods like HAQ and HAWQ-V3 searched over frozen weights — isolating the search from training, but inherently capping accuracy because the weights never adapted to their quantized representations. Quantization-aware training (QAT) searches like EDMIPS embedded the search into training, but the branching architecture that enabled differentiable optimization forced a hard separation: after the soft-search phase converged on branch weights, the model had to be retrained from scratch with the chosen hard configuration. The branched model's weights had co-adapted to averaging outputs from multiple quantization branches simultaneously — a regime that does not exist at deployment — so the searched configuration was not directly deployable.

FLIQS's central conceptual move is to recognize that the retraining requirement in prior QAT-based searches is not an inherent property of searching during training, but rather a consequence of uncontrolled interaction between the search process and model optimization. By formalizing this interaction as switching error — a concrete, measurable quantity proportional to policy entropy — the paper reframes the retraining problem as a noise-management problem. The controller's format-switching introduces optimization noise whose magnitude decays to zero precisely when the policy entropy collapses to a deterministic state. The cosine entropy regularization schedule is thus not merely a training trick; it is the mechanism that converts the search from a two-phase process (search then retrain) into a single integrated process (search converges to a deployable model).

This reframing is a fundamental conceptual shift rather than an incremental improvement. Prior work treated the search-model interference as an unavoidable nuisance to be cleaned up later via retraining. FLIQS treats it as a controllable quantity that determines deployability, and designs the controller's exploration-exploitation schedule around this insight. The empirical validation — Table 8 in Appendix A.11 showing ResNet-18 accuracy within ±0.5 points with and without retraining across six width multipliers — confirms that the reframing translates to practical equivalence between searched and retrained models. The significance extends beyond quantization search: any one-shot architecture search that modifies the model during training (pruning, NAS, mixed-precision) faces analogous interference, and the entropy regularization framework provides a general template for eliminating retraining in those domains as well.

Innovation 2: The Switching Error as a Diagnostic Concept for Search-Training Interference

The paper's formalization of switching error is intellectually distinctive not as a mathematical novelty — computing quantization error differences between two formats is straightforward — but as a diagnostic concept that explains why certain search spaces and schedules fail and others succeed. Equation 5 defines switching error as the element-wise difference between quantized representations under two different bitwidths, but the key insight is Equation 6: the expected switching error is proportional to policy entropy, creating a direct quantitative link between the controller's uncertainty and the optimizer's noise floor.

This diagnostic power enables several non-obvious predictions that the paper verifies. First, it explains why search spaces that include low-bitwidth options (INT4, E2M1) are inherently more disruptive than those restricted to higher precision: the switching error between 4-bit and 8-bit formats is dominated by the coarser grid of the 4-bit representation, making a 4→8 switch roughly as disruptive as a 4→5 switch. Second, it explains why naive RL-based QAT searches (without entropy control) require retraining: the REINFORCE updates naturally drive policy entropy downward, but not fast enough or smoothly enough to eliminate switching noise before training concludes. Third, it predicts that the timing of entropy collapse matters: collapse too early locks in premature decisions before the quality signal is reliable; collapse too late leaves residual noise that degrades final convergence. The cosine schedule directly addresses this timing problem by scheduling the exploration-exploitation transition to coincide with the reliability window of the quality signal.

Comparing this to prior work illuminates what has changed conceptually. HAWQ-V3 used Hessian-based sensitivity analysis — a static, post-hoc measurement of how much quantization would perturb each layer's loss, assuming frozen weights. FLIQS's switching error is fundamentally dynamic: it concerns how often and how drastically the quantization changes during training, independent of layer sensitivity. These are orthogonal dimensions of the quantization search problem, and FLIQS is the first work to isolate and control the dynamic dimension. The Gaussian approximation analysis (Section 4) further provides asymptotic guarantees — in the limit of zero entropy, switching error vanishes identically — which transforms the problem from an empirical tuning exercise into one with a clear convergence target.

Innovation 3: Empirical Demonstration That Mixed-Precision Floating-Point Systematically Dominates Integer at Fixed Bitwidth

Prior work on low-precision floating-point for inference (HFP8, MPFP, FPQuant) uniformly applied a single FP8 format to all layers, making it impossible to determine whether the observed accuracy advantages over INT8 were due to the format itself or to the specific E4M3/E5M2 exponent-mantissa splits chosen. FLIQS provides the first large-scale empirical evidence — spanning five model architectures (ResNet-18, ResNet-50, MobileNetV2, EfficientNet, InceptionV3) and a vision transformer (DeiT-B16) across both FLIQS-S and FLIQS-L search spaces — that mixed-precision floating-point models consistently achieve higher accuracy than mixed-precision integer models at equivalent total bitwidth.

The finding is visible across Figures 5, 11, and 12: the floating-point Pareto curves sit above the integer Pareto curves for every model architecture. For ResNet-18 at equivalent BOPs, the floating-point FLIQS-L model achieves 71.64% Top-1 versus 71.51% for integer FLIQS-L (Table 1) — a small but consistent margin. For ResNet-50 at ~80 GBOPs, floating-point achieves 77.34% versus 77.34% for integer (essentially tied), but at lower cost targets the floating-point advantage widens. The paper also reports that joint integer-floating-point searches were attempted but "since floating-point dominates integer formats at the same total bitwidths, the outputs of these searches were the same as the pure floating-point searches" (Section 5).

The significance of this finding is not a marginal accuracy gain on ImageNet. It is a design principle for hardware accelerators: if floating-point arithmetic consistently extracts more accuracy per bit than integer arithmetic at the precision levels relevant to DNN inference (4–8 bits), then accelerator architects should prioritize efficient low-precision floating-point units over integer units, contrary to the historical trend where integer quantization (INT8, INT4) has dominated deployed systems. The FP8 standardization effort (Micikevicius et al.) has already pushed the industry in this direction for training; FLIQS extends the argument to inference and to mixed precision, where the advantage compounds through per-layer format customization. The finding also implies that future quantization research should treat floating-point as the default representation and integer as the fallback, rather than the reverse framing that has characterized the quantization literature since Jacob et al. (2018).

A subtle analytical contribution here is the large search space itself. By including 16 floating-point formats spanning all exponent-mantissa splits from E2M1 to E6M1 (Table 5), the search does not merely pick a single best FP8 variant — it discovers per-layer allocations that exploit the different strengths of different splits. Layers with weight distributions that are heavy-tailed (more outliers) may receive formats with more exponent bits (wider dynamic range); layers with tightly concentrated weight distributions may receive formats with more mantissa bits (finer precision near zero). The paper does not provide per-layer analysis of why specific exponent-mantissa splits are chosen for specific layers, but the consistent floating-point advantage across architectures suggests that this adaptive allocation is genuinely exploiting the flexibility of the representation, not merely benefiting from a single superior format.

Innovation 4: The Compute Allocation Principle — Architectural Dimensions Beat Bitwidth for Fixed Budgets

The FLIQNAS experiments in Section 6 yield a finding that transcends the specific quantization method: when jointly optimizing over quantization precision and architectural parameters (channel widths, kernel sizes) under a fixed BOPs budget, the optimal strategy allocates additional compute to expanding architectural dimensions rather than increasing numerical precision. This is visible in Figure 6: at low cost targets (10–15 GBOPs), FLIQS-S and FLIQS-L (pure quantization search, fixed architecture) match or slightly exceed FLIQNAS (joint search). But as the cost budget increases to 17–22 GBOPs, FLIQNAS pulls decisively ahead, with FLIQNAS-L achieving 75.95% at 22 GBOPs versus the pure quantization search's 72.96% at 17 GBOPs — an improvement of 2.99 percentage points at comparable cost.

The interpretation is straightforward but non-obvious from prior work: BOPs scale as bitwidth squared (Equation 1), so halving bitwidth from 8 to 4 reduces per-operation cost by 4×. That savings can be reinvested into 4× wider channels (more filters per layer) or larger kernels, and the additional representational capacity from the architectural expansion more than compensates for the precision loss. The controller discovers this implicitly through the reward signal: configurations with wider channels at lower bitwidths produce higher validation accuracy for the same BOPs.

Why this is a distinct intellectual contribution rather than a trivial consequence of the quadratic cost model: prior NAS work optimized architectures at fixed precision (typically INT8 or FP32). Prior quantization work optimized bitwidths at fixed architecture. The interaction between these two design dimensions was unknown. It was entirely possible that for some architectures, precision would dominate — i.e., keeping higher bitwidth and sacrificing channels would be optimal. The FLIQNAS results empirically resolve this ambiguity in favor of architectural dimensions, at least for the MobileNetV2 search space on ImageNet. This principle provides actionable guidance for practitioners designing efficient models: if you have a fixed compute budget, invest in model width before precision.

The paper notes a weaker but parallel trend for kernel size: "the kernel size can tend to be larger with lower bitwidths, although it is not as strong" (Appendix A.7). This suggests that the principle may generalize across architectural dimensions, with channel width being the highest-leverage parameter. The comparison against APQ — which performed joint architecture-pruning-quantization search using a once-for-all network — puts FLIQNAS's results in context: at similar GBOPs (13, 16, 23), FLIQNAS-L achieves 74.79%, 75.65%, and 75.95% versus APQ's 72.10%, 74.10%, and 75.10% respectively, demonstrating that the RL-based approach finds configurations that the once-for-all supernetwork misses, likely because the supernetwork's soft-averaging over architectural options introduces training pathologies that FLIQS's discrete sampling avoids.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All experiments use the ImageNet (ILSVRC2012) benchmark for image classification, consisting of approximately 1.28 million training images and 50,000 validation images across 1,000 classes. The validation set is used to compute the Top-1 accuracy quality signal during search and for final evaluation. For brief exploration of recommendation models, the Criteo dataset (45 million examples, binary click-through-rate prediction) is used in Appendix A.13, evaluated via Area Under the ROC Curve (AUC).

• Base model(s). Six model architectures are evaluated: ResNet-18, ResNet-50, MobileNetV2, EfficientNet (B0–B4), InceptionV3, and DeiT-B16 (a vision transformer). Models are trained from scratch with cloud-based TPUv3 clusters. Width multipliers ranging from 0.25× to 2.0× of original channel widths are applied to generate Pareto curves across different computational budgets. The choice of models spans convolutional networks with diverse structural properties (standard residual blocks, inverted bottlenecks with depthwise separable convolutions, multi-branch Inception modules) and a transformer architecture, providing coverage of common design patterns in modern DNN deployment.

• Metrics. The primary quality metric is ImageNet Top-1 validation accuracy (%) — the fraction of validation images for which the model's highest-confidence prediction matches the ground truth label. Model cost is measured in Giga Bit-Operations (GBOPs), computed layer-wise as $b(\alpha)^2 \times \text{MAC}_l(\alpha)$ and summed across all layers, where $b(\alpha)$ is the total bitwidth of the format assigned to layer $l$ and $\text{MAC}_l$ is the multiply-accumulate count. The BOPs metric is validated against FPGA hardware measurements in Table 2, which shows look-up table (LUT) counts scaling quadratically with bitwidth. GPU inference speedups are separately reported in Table 3 using an infrastructure extending the TVM compiler to support INT4 inference, with latency measured on NVIDIA RTX 2080 Ti and A6000 GPUs.

• Baselines. The paper compares against a broad set of prior methods, organized by quantization paradigm:

  • Post-training quantization (PTQ) methods: HAWQ-V3 (Yao et al., 2021) — Hessian-based sensitivity analysis with integer linear programming for mixed-precision INT4/INT8 assignment; ZeroQ (Cai et al., 2020) — zero-shot PTQ requiring no calibration data; LQNets (Zhang et al., 2018) — learned quantization with trainable quantizers.
  • Quantization-aware training (QAT) searches: EDMIPS (Cai and Vasconcelos, 2020) — differentiable search with branched architectures, evaluated on INT [1,4] mixed-precision; Bayesian Bits (van Baalen et al., 2020) — unified quantization and pruning via learned bitwidth distributions; DQ (Uhlich et al., 2020) — differentiable quantization with learnable step sizes; PACT (Choi et al., 2018) — parameterized clipping activation for uniform-precision QAT.
  • Uniform-precision floating-point methods: HFP8 (Sun et al., 2019) — hybrid FP8 using E4M3 for forward pass; FPQuant (Kuzmin et al., 2022) — FP8 quantization exploiting exponent flexibility; MPFP (Mellempudi et al., 2019) — mixed-precision FP8 training.
  • Quantization-aware NAS: APQ (Wang et al., 2020) — joint architecture, pruning, and quantization search using a once-for-all network with INT [2,4,6] search space on MobileNetV2.
  • Uniform-precision baselines internal to FLIQS experiments: BF16 (full half-precision), INT4, INT8, E2M1, E4M3 — single format applied to all layers, serving as lower bounds for the improvements achievable through mixed-precision search.

• Generation budget / compute accounting. The search and training procedure is measured in training steps (epochs). All models are trained for 350–400 epochs with specific hyperparameters listed in Table 4 (Appendix A.3). The computational cost of the search is not measured in FLOPs or GPU-hours; rather, the paper argues that RL-based search is inherently more memory-efficient than branching-based differentiable methods, quantifying this in Table 6 (Section A.6): for ResNet-18 with batch size 32, FLIQS requires 46.8 MiB for gradients and 73.6 MiB for activations, while a branched architecture requires 92.6 MiB and 220.8 MiB respectively — approximately 2× and 3× more memory. The controller warmup period occupies the first 25% of training steps, and the entire search concludes with training — no separate search phase or retraining is needed.

• Cross-validation / statistical protocol. For the main quantization search results in Table 1 and Figures 5, experiments are aggregated over three independent trials with different random seeds. The paper reports mean and standard deviation for FLIQS methods (e.g., "71.64 ± 0.37" for floating-point FLIQS-L ResNet-18). For the FLIQNAS results in Figure 6, the paper reports mean and standard deviation (e.g., "75.65 ± 0.20" for FLIQNAS-L at 17 GBOPs). No formal cross-validation protocol (e.g., k-fold over the training set) is described — the search evaluates configurations on validation batches during training, and final accuracy is reported on the standard ImageNet validation set. The paper does not report confidence intervals or statistical significance tests between competing methods, relying instead on the magnitude of accuracy differences relative to reported standard deviations.

---

Main Quantitative Results

Quantization Search: Mixed-Precision Outperforms Uniform Precision Across Architectures and Formats

The central aggregate result appears in Table 1 and Figure 5: FLIQS-searched models consistently achieve higher accuracy at lower or equivalent model cost compared to uniform-precision baselines and prior mixed-precision methods, across both integer and floating-point search spaces and all six model architectures.

ResNet-18 results (Table 1, Figure 5, Figure 15 in Appendix A.14): At the 1.0× width multiplier:

• Integer FLIQS-L achieves 70.61 ± 0.04% Top-1 at 32 GBOPs, compared to HAWQ-V3's 70.38% at 72 GBOPs (a 2.25× higher cost). Relative to the nearest-cost prior QAT method, EDMIPS achieves 67.20% at 22 GBOPs — FLIQS-L delivers 3.41 percentage points higher accuracy for a 45% increase in cost (32 vs. 22 GBOPs), but more meaningfully, FLIQS-S at 31 GBOPs achieves 69.91%, which is 2.71 points higher than EDMIPS at a comparable 1.4× cost. Against uniform baselines: FLIQS-L improves over INT8 (70.60% at 117 GBOPs) by using only 27% of the cost while matching accuracy; against BF16 (71.17% at 468 GBOPs), accuracy is comparable (within 0.56 points) at 6.8% of the cost.
• Floating-point FLIQS-L achieves 71.64 ± 0.37% at 46 GBOPs, outperforming MPFP (69.71% at 137 GBOPs) by 1.93 percentage points at one-third the cost, and HFP8 (69.39% at 137 GBOPs) by 2.25 points at one-third the cost. Compared to FPQuant (70.28% at 116 GBOPs), FLIQS-L achieves 1.36 points higher accuracy at 40% of the cost.
• Floating-point vs. integer at equivalent cost: At the 50 GBOPs level (Figure 15), floating-point FLIQS-L reaches approximately 71.6% while integer FLIQS-S reaches approximately 71.1% — a 0.5 point advantage for floating-point. This pattern — floating-point consistently outperforming integer at equal BOPs — holds across all models (explicitly stated in Section 5: "in nearly every case, floating-point outperforms integer").

ResNet-50 results (Table 1, Figure 16): At the 1.0× width:

• Integer FLIQS-S achieves 77.32 ± 0.05% at 81 GBOPs, surpassing HAWQ-V3's 76.73% at 154 GBOPs (0.59 points higher at 47% of the cost). Against the INT4* (first and last layers in higher precision) uniform baseline at 76.30% and 71 GBOPs, FLIQS-S gains 1.02 points with only a 14% cost increase. Against EDMIPS (73.20% at 49 GBOPs), FLIQS-S at a comparable cost of 74 GBOPs (0.75× width, Figure 16) achieves approximately 75.11% — roughly 1.9 points higher.
• Floating-point FLIQS-L achieves 77.34 ± 0.14% at 74 GBOPs, outperforming uniform E4M3 (77.42% at 262 GBOPs) by approximately matching accuracy at 28% of the cost.

MobileNetV2 results (Table 1, Figure 5, Figure 17): This architecture is notably sensitive to quantization:

• Integer FLIQS-L achieves 71.87 ± 0.24% at 7.06 GBOPs (1.0× width), outperforming Bayesian Bits (72.00% at 17 GBOPs) by essentially matching accuracy at 42% of the cost. Against DQ (69.74% at 37 GBOPs), FLIQS-L achieves 2.13 points higher accuracy at 19% of the cost. Against uniform INT8 (72.83% at 19.25 GBOPs), FLIQS-L matches accuracy at 37% of the cost.
• Floating-point FLIQS-L achieves 72.94 ± 0.09% at 17 GBOPs (1.4× width, Table 1 shows 17 GBOPs for the 1.4× variant which achieves 75.26%; the 1.0× floating-point FLIQS-L is reported in Figure 23 as 71.97% at 6.77 GBOPs). Against HFP8 (71.61% at 21 GBOPs), FLIQS-L outperforms by 0.36 points at 32% of the cost for the 1.0× variant. Against FPQuant (71.60% at 19 GBOPs), floating-point FLIQS-L delivers roughly 0.37 points higher accuracy at 36% of the cost. The paper specifically claims "improve MobileNetV2 by up to 0.98% points compared to prior state-of-the-art FP8 models" — this figure appears to reference the improvement of the 1.4× width floating-point FLIQS variant (72.94%) over FPQuant (71.60%) or HFP8 (71.61%), yielding a 1.33-1.34 point improvement, suggesting the "0.98% points" claim may reference a different comparison point or the difference after controlling for width variations.

EfficientNet results (Figure 5, Figure 19): At the B2 variant (the 1.0× equivalent in EfficientNet scaling):

• Integer FLIQS-L achieves 75.41% at 21.62 GBOPs, versus uniform INT8 at 76.48% at 63.50 GBOPs — FLIQS-L delivers comparable accuracy (0.17 points lower) at 34% of the cost. Versus INT4 at 67.71% and 15.88 GBOPs, FLIQS-L improves accuracy by 7.70 points with only a 36% cost increase.
• Floating-point FLIQS-S (Figure 25) achieves 74.67% at 23.26 GBOPs at B2, while FLIQS-L at B2 (if interpreted from the table structure; the FLIQS-L row in Figure 25 ends at B1 with 73.23%) — the data in Figure 25 is incomplete for higher EfficientNet variants, making direct comparisons at B2 ambiguous.

DeiT-B16 results (Table 1, Figure 20, Figure 26): The vision transformer at 1.0× width:

• Integer FLIQS-S achieves 79.47 ± 0.18% (standard deviation unclear from Table 1 formatting — the row shows 79.47 without explicit deviation) at 275 GBOPs, while uniform INT8 achieves 79.49% at 1,079 GBOPs — essentially identical accuracy at 25% of the cost. The paper emphasizes in Section 5 that DeiT-B16 has "10 times the model cost as BF16 MobileNetV2," demonstrating the scalability of the RL-based search to large models.
• Floating-point FLIQS-S (Figure 26) achieves 79.27% at 275 GBOPs at 1.0× width, versus uniform E4M3 at 79.17% and 1,079 GBOPs — matching accuracy at 25% of the cost. Floating-point FLIQS-L achieves 79.54% at 271 GBOPs.

Pareto curve analysis (Figure 5, Figure 14 in Appendix A.14): Across all width multipliers (0.25× to 2.0×), the FLIQS-L curve consistently sits above FLIQS-S, which sits above uniform precision baselines. The gains are more pronounced at lower cost targets — at approximately 10 GBOPs for MobileNetV2, FLIQS-L achieves roughly 63% vs. uniform FP8/E4M3 at approximately 54% (a 9-point gap), while at 77 GBOPs the gap narrows to roughly 1–2 points. This pattern — larger improvements from mixed-precision at lower total budgets — is expected: when the budget is tight, inefficient allocation (e.g., using 8 bits everywhere) wastes precious compute that could be concentrated on sensitive layers; when the budget is generous, uniform high precision already captures most of the achievable accuracy.

Configurations discovered by the search (Figures 3, 7, 8, 9): The per-layer format assignments reveal consistent patterns:

• First and last layers almost always receive higher precision across all models and search spaces. In ResNet-18 FLIQS-S examples (Figure 7), the first convolutional layer (7×7, 3 input channels to 64 output channels) and the final fully-connected classification layer consistently get 8-bit allocation even when intermediate layers are at 4 bits.
• Downsampling blocks get elevated precision on their 1×1 branch convolutions. In ResNet-18 (Figure 7), the upper-branch 1×1 convolutions in residual blocks with stride 2 are consistently allocated higher bitwidths (6–8 bits) compared to the main-branch 3×3 convolutions (4–5 bits).
• In MobileNetV2, the central depthwise 3×3 convolution receives higher precision than the surrounding 1×1 pointwise convolutions (Figure 8). This is visible as groups of three consecutive layers where the middle one (depthwise) has a higher bitwidth than its neighbors. The paper interprets this as reflecting the different quantization sensitivity of depthwise versus pointwise operations.
• In DeiT-B16, the self-attention blocks receive more bits than the MLP blocks (Figure 3, Figure 8, Figure 9). The query, key, and value projections within the attention mechanism are allocated 6–8 bits while the subsequent two-layer MLP receives 4–6 bits.
• In InceptionV3's branched structure (Figure 8), the 1×1 convolutions at the top and bottom of each Inception module receive higher precision, while the internal 3×3 and 5×5 convolutions operate at lower precision.

The paper does not provide quantitative feature attribution analysis for why these patterns emerge — they are presented as emergent properties of the RL controller maximizing the reward signal, consistent with prior manual mixed-precision heuristics (first/last layers sensitive, attention blocks sensitive) but discovered automatically.

Model cost reduction at equivalent accuracy (Table 1): The reciprocal framing — how much cost is saved while maintaining accuracy — is equally informative:

• ResNet-18 FLIQS-L integer matches HAWQ-V3's accuracy while reducing BOPs by approximately 56% (32 vs. 72 GBOPs).
• ResNet-50 FLIQS-S matches HAWQ-V3's accuracy at roughly 47% of the cost (81 vs. 154 GBOPs).
• MobileNetV2 FLIQS-L floating-point matches HFP8's accuracy at roughly 32% of the cost (7 vs. 21 GBOPs at 1.0× width).

Table 1 specific quantitative comparisons: The table presents results at width multipliers that are not always 1.0× — the GBOPs column reveals the effective scale:

• ResNet-18 integer FLIQS-L at 32 GBOPs corresponds to roughly the 1.0× width multiplier (BF16 at 1.0× is 468 GBOPs; INT8 at 1.0× is 117 GBOPs; scaling by bitwidth-squared, 32/117 ≈ 0.27, suggesting approximately 0.52× effective width scaling with mixed-precision).
• ResNet-50 FLIQS-S at 81 GBOPs vs. INT8 uniform at 262 GBOPs (1.0×) corresponds to approximately 0.56× effective width.
• MobileNetV2 FLIQS-L at 7.06 GBOPs vs. INT8 uniform at 19.25 GBOPs corresponds to approximately 0.61× effective width.

GPU and FPGA performance (Tables 2 and 3):

• GPU latency (Table 3): On ResNet-50 with an NVIDIA RTX 2080 Ti, INT8 achieves 1.000× normalized speedup. INT4 (all layers at 4-bit) achieves 1.338× speedup but drops accuracy to 74.91%. INT4* (first and last layers in higher precision) achieves 1.334× speedup with 76.30% accuracy. FLIQS-S achieves 1.303× speedup — only 2.6% slower than uniform INT4 — while delivering 77.40% accuracy, which is 1.10 points higher than INT4* and 2.49 points higher than pure INT4. On the A6000 GPU, the speedups are slightly lower (FLIQS-S: 1.213×) but the accuracy advantage remains. The paper emphasizes that "the FLIQS-S model improves accuracy significantly with only 1% lower inference speed compared to the INT4 model" (on Turing GPUs).
• FPGA area (Table 2): On the Xilinx UltraScale+ FPGA, LUT counts for a ResNet-18 first residual block scale from 42.8K at uniform INT4 (1.00× relative) to 67.6K at uniform INT8 (1.58× relative), closely following the quadratic bitwidth-area relationship. FLIQS-L configurations with mixed-precision (e.g., 5,5,6 bits for the three convolutions in the downsampling block) use 45.9K LUTs (1.07×) while achieving 70.12% accuracy — 2.81 points above uniform INT4 (4,4,4 bits, 67.31%) for only 7% more area. The most accurate configuration (5,6,6 bits, 71.51%) uses 47.1K LUTs (1.10×) for a 4.20-point improvement. The paper notes this "confirms the overhead from these searched models is relatively small compared to the accuracy improvements."

---

Quantization Neural Architecture Search (FLIQNAS): Architectural Expansion Dominates Bitwidth Allocation

The FLIQNAS experiments in Section 6 (Figure 6, Table in Section 6) extend the search to jointly optimize quantization formats, channel widths, and kernel sizes on a MobileNetV2 base architecture. The combined search space for FLIQNAS-S includes 230 tunable values with more than $10^{100}$ possible configurations.

Headline numbers (Figure 6, Section 6 table):

At approximately 13 GBOPs:

• FLIQNAS-S (integer, FLIQS-S formats): 73.79 ± 0.14%
• FLIQNAS-L (integer, FLIQS-L formats): 74.79 ± 0.08%
• APQ-A (integer [2,4,6]): 72.10%
• FLIQS-S (pure quantization search, 17 GBOPs): 72.98 ± 0.22%

FLIQNAS-L improves over APQ-A by 2.69 percentage points at similar cost. Compared to pure quantization search at the nearest comparable cost (FLIQS-S at 17 GBOPs), FLIQNAS-L achieves 1.81 points higher accuracy while reducing cost by approximately 24% (13 vs. 17 GBOPs).

At approximately 17 GBOPs:

• FLIQNAS-S: 75.17 ± 0.08%
• FLIQNAS-L: 75.65 ± 0.20%
• FLIQS-S (pure quantization): 72.98 ± 0.22%
• FLIQS-L (pure quantization): 72.96 ± 0.26%

The joint search improves accuracy by 2.21–2.67 points over pure quantization search at identical cost — this is the "2.69% points" headline number from the abstract (specifically, FLIQNAS-L at 22 GBOPs achieves 75.95% vs. FLIQS-L at roughly comparable cost achieving approximately 73.26%, interpolating between the 17 GBOPs and higher-cost datapoints, giving 2.69 points difference).

At approximately 22 GBOPs:

• FLIQNAS-S: 75.71 ± 0.11%
• FLIQNAS-L: 75.95 ± 0.04%
• APQ-C: 75.10%

Comparative advantage of FLIQNAS over APQ: APQ uses a once-for-all network that jointly searches architecture, pruning, and quantization policies with knowledge distillation from a full-precision accuracy predictor. FLIQNAS-L outperforms APQ at three design points: +2.69 points at 13 GBOPs, +1.55 points at 16 GBOPs, and +0.85 points at 23 GBOPs. The gap narrows at higher budgets, consistent with the pattern that architectural search provides greater relative benefits in resource-constrained regimes.

The compute allocation principle: The key insight from the FLIQNAS experiments is visible in Figure 6's Pareto curve shape. At low model costs (below ~15 GBOPs), the pure quantization search (FLIQS-S and FLIQS-L, red and blue markers for integer) and architecture-only search (quantized NAS, gray markers for integer) produce similar accuracy — the controller has limited budget to spend, and both quantization and architectural modifications can absorb it. As the cost target increases above ~15 GBOPs, FLIQNAS (purple markers) decisively separates from pure quantization search: the additional compute is better allocated to widening channels and increasing kernel sizes rather than increasing bitwidth. The paper states this as "for a fixed compute budget, larger models benefit from increasing architectural dimensions over bitwidth" (Section 7).

Floating-point vs. integer in FLIQNAS (Figure 6): At all cost targets, floating-point FLIQNAS (green markers) outperforms integer FLIQNAS (purple markers). At 13 GBOPs: floating-point 74.79% vs. integer 74.79% (FLIQNAS-L tie). At 17 GBOPs: floating-point 75.65% vs. integer 75.17% (0.48-point advantage). At 22 GBOPs: floating-point 75.95% vs. integer 75.71% (0.24-point advantage). The floating-point advantage persists in the joint search space but is smaller than in pure quantization search, possibly because architectural expansion partially compensates for the representation gap between floating-point and integer.

Channel width and bitwidth allocation patterns (Appendix A.7): The paper reports qualitative observations from the searched configurations:

• "Lower bitwidths typically receive a larger number of channels" — the controller compensates for precision loss with representational capacity.
• "Channel dimension is increased before the kernel size to reach cost targets" — when given additional BOPs budget, the search prefers widening layers over enlarging kernels.
• The kernel-size-to-bitwidth relationship is "not as strong" — while there is a weak tendency for larger kernels to accompany lower bitwidths, channel width is the primary architectural lever.

Recommendation model results (Appendix A.13, Figure 13): On the Criteo CTR prediction dataset using a 4-hidden-layer MLP with low-rank factorization, FLIQNAS-L achieves higher AUC than uniform INT8 and E4M3 baselines across MBOPs targets, with the advantage widening at larger model sizes (higher MBOPs). FLIQS-L (pure quantization) also outperforms uniform baselines but is surpassed by FLIQNAS-L at higher MBOPs, reinforcing the principle that joint architecture-quantization search is most beneficial when the compute budget allows meaningful architectural expansion.

---

Ablation Studies and Robustness Checks

Two-phase quantization start step (Appendix A.2, Table 10a): On ResNet-50 trained for 30,200 total steps, sweeping the step at which activation quantization begins reveals a concave accuracy curve. Starting activation quantization at step 1,000 (3.3% of training) yields 75.59% Top-1. Delaying to step 4,000 (13.2%) improves accuracy to 76.11%. Further delays produce modest declines: step 8,000 (26.5%) gives 76.02%, step 15,000 (49.7%) gives 75.94%, step 25,000 (82.8%) gives 75.54%. The optimum at 13–26% of training motivates the default of 7,500 steps for ResNets and InceptionV3 (25% of 30,200), and 20,000 steps for MobileNetV2, EfficientNet, and DeiT (trained for ~112,000+ steps, representing ~18% of training). This ablation demonstrates that quantizing weights before activations — allowing the model to partially adapt to weight quantization before introducing activation quantization — is beneficial, and that the timing of this transition matters.

Activation clipping threshold (Appendix A.2, Table 10b): On ResNet-18, sweeping the standard deviation multiple used to compute clipping thresholds from 1× to 8× shows optimal accuracy at 3–4×. At 1×: 63.39%. At 2×: 67.79%. At 3×: 68.02%. At 4×: 67.91%. At 8×: 64.91%. The narrow range of near-optimal values (3–4×) suggests moderate sensitivity, but the default of 4× is applied globally to all models for simplicity. The trade-off is standard: too small a threshold causes excessive clipping of outliers (information loss on large activations), too large a threshold causes excessive rounding error on common values (since the quantization step size increases with range).

Number of profiling batches for activation statistics (Appendix A.2, Table 10c): Varying the number of batches used to profile activation standard deviations from 1 to 100 shows minimal impact on final accuracy: 1 batch yields 67.92%, 100 batches yields 68.02%. The paper attributes this insensitivity to the large batch size (2048) and shuffling, which provide stable statistics even from few batches.

Retraining vs. no retraining (Appendix A.11, Table 8): Across six width multipliers (0.5×, 0.75×, 1.0×, 1.25×, 1.5×, 2.0×) on ResNet-18 with integer FLIQS-S and FLIQS-L, the accuracy difference between the directly-deployed searched model and the same configuration retrained from scratch is within ±0.5 percentage points across all 12 comparisons (2 search spaces × 6 widths). For FLIQS-S at 1.0×: no-retraining 69.92%, retrained 69.53% (retraining is 0.39 points worse). For FLIQS-L at 1.0×: no-retraining 69.56%, retrained 69.56% (identical). At 0.75× FLIQS-S: no-retraining 64.93%, retrained 66.47% (retraining is 1.54 points better). At 1.5× FLIQS-S: no-retraining 73.32%, retrained 73.20% (retraining is 0.12 points worse). The absence of a systematic retraining advantage — and instances where retraining underperforms — strongly validates the entropy regularization mechanism. The paper explicitly notes "there is no noticeable advantage to retraining across model widths."

RL controller method selection (Section 3): The paper evaluated more sophisticated RL algorithms (specifically PPO) and policy models (multi-layer perceptrons rather than per-layer independent logits) but found "no quality improvements while being more costly" (Section 3). This is a negative result reported without a dedicated table — the REINFORCE with running-average baseline and per-layer softmax policies is kept as the minimal sufficient architecture.

Floating-point joint search with integer (Section 5): An attempted experiment allowing the controller to select between both integer and floating-point formats simultaneously yielded configurations indistinguishable from pure floating-point search, since "floating-point dominates integer formats at the same total bitwidths" (Section 5). This is reported as a motivating observation for focusing on pure floating-point search rather than as a tabled result.

FP8 mantissa sweep (Appendix A.9, Table 7): A uniform-precision sweep over all possible FP8 exponent-mantissa splits (E1M6 through E5M2) on four architectures reveals that models are generally resilient to the choice of FP8 format, with accuracy declining as mantissa bits decrease (more exponent bits, coarser precision near zero). For ResNet-18: E1M6 (71.72%) → E2M5 (71.70%) → E3M4 (71.69%) → E4M3 (71.69%) → E5M2 (71.59%). The drop from E1M6 to E5M2 is only 0.13 points, suggesting relative insensitivity. However, MobileNetV2 shows larger sensitivity: E1M6 (73.20%) → E2M5 (73.14%) → E3M4 (73.17%) → E4M3 (72.65%) → E5M2 (72.07%), a 1.13-point drop. This validates the need for per-model format optimization — MobileNetV2's depthwise separable convolutions are more sensitive to precision loss than ResNet's standard convolutions — and provides context for why mixed-precision floating-point search (which can allocate different FP8 splits to different layers) could outperform any single FP8 choice.

Cosine entropy regularization effectiveness (Section 4, Figure 4(b)): The paper reports that "higher entropy regularization produces higher accuracy" (Figure 4b, right panel), showing that the $\beta_H$ schedule — which prevents premature policy convergence — is essential. The specific ablation varies $\beta_H^{\text{end}}$ rather than comparing presence vs. absence of the schedule; the cosine schedule with $\beta_H^{\text{end}} = 0.5$ is used for all main experiments. Appendix A.11's retraining comparison provides implicit validation: without effective entropy regularization, the no-retraining models would systematically underperform retrained models, which they do not.

ReST$^{EM}$ revision model experiment for comparisons to future work: Not applicable — this is a FLIQS paper; the ReST$^{EM}$ experiment described in the reference example is from a different paper. No such experiment exists in FLIQS.

Lack of formal statistical testing and confidence intervals: The paper does not report p-values, confidence intervals, or effect sizes for comparisons between methods. Standard deviations are provided for FLIQS results (typically across 3 trials), but not for baseline methods, making it impossible to assess whether differences of 0.1–0.5 percentage points (common in the floating-point vs. integer comparisons) are statistically significant. In practice, the Top-1 accuracy differences of 1+ percentage points on ImageNet with n=50,000 validation images are likely significant given the reported standard deviations of 0.04–0.37 points for FLIQS, but the absence of baseline standard deviations prevents formal verification.

---

Critical Assessment

Claim 1: FLIQS is the first one-shot mixed-precision quantization search that eliminates retraining. This claim is strongly supported by the mechanism and the ablation evidence. The switching error formalization (Section 4) provides a theoretical explanation for why prior RL-based QAT searches required retraining, and the cosine entropy regularization schedule is a principled solution to that problem. The retraining ablation (Table 8) directly validates that the searched model matches or exceeds retrained accuracy across 12 independent comparisons. However, the claim's scope is narrower than it might appear: "one-shot" here means the search concludes with a single training run that produces the final model — but the difficulty estimation (generating 2,048 samples per question to bin by difficulty), which is flagged as a limitation in Section 3.2, is NOT included in the "one-shot" cost accounting. The search itself is one-shot, but if significant pre-search computation is needed to set up the search space or cost targets, the practical deployment pipeline may not feel "one-shot" to a practitioner. For FLIQS, the cost target $C_T$ must be specified by the user, which requires some prior knowledge of the acceptable accuracy-efficiency trade-off — the paper does not explore sensitivity to $C_T$ misspecification.

Claim 2: FLIQS achieves state-of-the-art results for integer and floating-point quantization search across convolutional and transformer networks. The claim is empirically supported by comparisons in Table 1 and Figure 5, where FLIQS consistently posts higher accuracy at lower or equivalent BOPs compared to HAWQ-V3, EDMIPS, HFP8, MPFP, FPQuant, Bayesian Bits, DQ, and PACT. However, the comparisons are not always strictly controlled:

• Cost metric differences: HAWQ-V3 reports results in terms of model size (MB) and BOPs using different cost models. The paper's Table 1 lists HAWQ-V3 at specific GBOPs values (34, 71, 72, 154) — are these computed under the same BOPs definition as FLIQS? The paper does not explicitly state that all baseline BOPs numbers are recomputed under the FLIQS cost model. If HAWQ-V3's original paper used a different cost metric, the comparison may conflate methodological differences with cost-accounting differences.
• Training recipe differences: FLIQS models are trained from scratch with specific hyperparameters (350–400 epochs, cosine LR schedules, specific optimizers from Table 4). Baseline methods used their own training recipes, which may differ in total epochs, learning rate schedules, data augmentation, weight decay, etc. The paper does not reimplement baselines under a unified training protocol; it compares against published numbers. This is standard practice but means accuracy differences could partially reflect training recipe quality rather than quantization search quality.
• Missing baselines: There is no comparison against OMPQ (Ma et al., 2022) or the one-shot weight-coupling method of Tang et al. (2024), both cited in the related work (Section 2, references [20] and [21]) as contemporary methods addressing similar problems. The omission of these recent PTQ-based one-shot methods limits the comprehensiveness of the comparison.
• Width multiplier control: The entries in Table 1 are at different effective widths (as noted above, FLIQS at 32 GBOPs for ResNet-18 corresponds to ~0.52× effective width vs. BF16 baseline at 1.0×). While this is inherent to the cost-accuracy trade-off being measured, it means that when the paper says "improves ResNet-18 accuracy by 1.31% points over previous methods," the comparison is not at fixed model width — it's at fixed BOPs, which is a legitimate comparison axis but should be explicitly qualified as "at equivalent computational cost" rather than framing it as a pure accuracy improvement.

Claim 3: FLIQS performs the largest comparison of integer and floating-point mixed-precision networks. This claim is largely a statement of scope, and it is accurate: the paper includes 6 model architectures, 2 search spaces (small and large), 5 width multipliers per architecture, and both integer and floating-point formats, generating hundreds of datapoints (the Pareto plots in Figure 5 and Appendix A.14 contain 16 curves with 4–6 points each). No prior work has published a comparison at this scale. However, "largest" is measured by number of architectures and search space configurations, not by dataset scale — all results are on ImageNet, a single dataset. The Criteo recommendation model experiment (Appendix A.13) is a brief exploration on a different domain, but it uses a simple MLP architecture and is not integrated into the main comparison. A truly comprehensive comparison would include additional tasks (object detection, segmentation, NLP) to test whether the floating-point-over-integer dominance generalizes beyond image classification.

Claim 4: FLIQS conducts the first study of quantization and neural architecture search on low-precision floating-point networks. This claim appears to be true based on the cited literature — prior QNAS work (APQ, Auto-NBA, Gong et al.) operated exclusively on integer formats. The FLIQNAS floating-point results in Figure 6 represent the first empirical characterization of how architectural dimensions and floating-point precision interact in a joint search space. However, the study is limited to a single architecture family (MobileNetV2) and a single dataset (ImageNet). The principle that "architectural dimensions should be expanded before bitwidth" — while intriguing and practically actionable — is supported by one search space on one task. Extrapolating it to other architectures (transformers, MLP-Mixers, graph neural networks) or tasks would be premature. The paper does not provide sufficient analysis to determine whether this principle is a consequence of MobileNetV2's specific structure (inverted bottlenecks with depthwise convolutions) or a more general property of DNN quantization.

Weaknesses in experimental design:

1. No latency-aware cost model in main experiments: The BOPs cost model is validated against FPGA LUTs (Table 2) and GPU latency (Table 3), but the paper's Pareto curves and Table 1 comparisons all use GBOPs as the cost axis. GBOPs correlate with, but do not equal, runtime on specific hardware. A ResNet-50 FLIQS-S configuration achieving 1.30× speedup over INT8 on a 2080 Ti (Table 3) is promising, but the paper does not provide per-configuration latency measurements for all Pareto-optimal models. Practitioners deploying on fixed-function accelerators need latency numbers, not BOPs estimates, to make deployment decisions. The paper's argument that BOPs are "hardware-agnostic" is valid for the algorithmic contribution but limits the practical utility of the specific Pareto curves.

2. Single training budget (epochs): All models are trained for a fixed number of epochs (350–400) regardless of their searched configuration. Lower-precision models typically converge faster, while wider models may require more epochs. The fixed training budget may advantage some configurations over others in ways that confound the search — a configuration that achieves good accuracy at epoch 350 might be overtaken by another configuration at epoch 500. The paper does not explore whether the relative ranking of configurations is stable with respect to training duration.

3. The search overhead is not reported: The paper demonstrates memory efficiency (Table 6) but does not report the wall-clock time or total FLOPs consumed by the search process compared to training a fixed-configuration model. The RL controller runs at every training step, so the search overhead is amortized across training, but it is not zero — the controller's forward pass, sampling, reward computation, and policy update add computational work. Without quantifying this overhead, the claim that FLIQS is "efficient" (Section 7) is qualitative.

4. No sensitivity analysis for the cost scalar $\gamma$: The reward function in Equation 1 uses $\gamma$ to balance quality and cost. The paper sweeps $C_T$ to generate Pareto curves but does not report the effect of $\gamma$ on search dynamics or final configuration quality. A poorly chosen $\gamma$ could cause the controller to over-emphasize cost (producing cheap but inaccurate models) or quality (ignoring the cost constraint). The robustness of results to $\gamma$ is unexplored.

5. Difficulty/entropy of the search space itself is not characterized: The paper reports that joint integer-floating-point searches degenerate to pure floating-point searches, but it does not characterize how the controller's learning dynamics change with search space size. Does the 16-option FLIQS-L floating-point space take longer to converge than the 3-option FLIQS-S? Does the final policy entropy differ? These would be informative for practitioners choosing search spaces.

6. Per-layer format analysis is qualitative: Figures 3, 7, 8, and 9 show example configurations as color-coded diagrams, but the paper does not provide aggregated statistics across trials (e.g., "the first layer was assigned 8 bits in 100% of FLIQS-S trials, while layer 7 was assigned 4 bits in 87% of trials"). Without such statistics, it is unclear whether the observed patterns (higher precision for first/last layers, attention blocks, depthwise convolutions) are robust properties of the search or artifacts of specific random seeds. The three-trial aggregation provides mean accuracy but not configuration consistency.

7. The "largest comparison" claim (Appendix A.14) relies on extensive data in the appendix: Figures 15–26 contain the raw Pareto data across all width multipliers. This is commendable for reproducibility but means many of the comparisons supporting the paper's claims are in the appendix rather than the main text, making it challenging for a reader to verify specific numbers without cross-referencing multiple pages.

Experiments that would have strengthened the paper:

• Cross-dataset evaluation: Running FLIQS-searched ImageNet configurations on out-of-distribution datasets (ImageNet-C, ImageNet-R, or a different domain like COCO) to test whether the searched mixed-precision models maintain their advantage under distribution shift. This is particularly relevant for floating-point formats, where the non-uniform quantization grid might interact differently with shifted activation statistics.
• Direct latency comparisons on target hardware: Publishing end-to-end latency measurements for all Pareto-optimal configurations on a TPUv3 (the hardware used for training) and a GPU, rather than the single ResNet-50 datapoint in Table 3. This would transform the paper from a BOPs-based algorithmic contribution to a deployment-ready practical guide.
• Scaling beyond 8 bits in the FLIQS-L search space: The floating-point FLIQS-L space includes formats up to E6M1 (7 bits total) and BF16 (16 bits), but no 9–15 bit formats. A sweep including intermediate bitwidths would more precisely characterize where the accuracy-per-bit curve saturates.
• Ablation of the REINFORCE running-average baseline: The paper states that the running average is used "to address the performance concerns with one-shot searches" (Section 3), but a direct comparison against a no-baseline variant (using raw reward as the policy gradient weight) would quantify the variance reduction benefit.
• Hardware-aware cost model comparison: Comparing the Pareto curves obtained under BOPs against those obtained under a latency-predictive cost model (e.g., a lookup table of per-layer-per-format latencies on a specific GPU) to quantify how much the choice of cost proxy affects the final configurations.

6. Limitations and Trade-offs

6.1 The Cost Model Is a Hardware-Agnostic Proxy, Not a Deployment Latency Model

The assumption or constraint. FLIQS uses bit operations (BOPs), a quadratic cost model where the cost of a layer scales as bitwidth² × MAC count, as its sole optimization target for computational efficiency. The reward function in Equation 1 drives the controller to minimize total GBOPs relative to a user-specified target, and all Pareto curves in Figure 5 and Table 1 use GBOPs as the cost axis. The paper validates this model against FPGA look-up table counts in Table 2, showing that LUTs scale roughly quadratically with bitwidth for a ResNet-18 residual block, and provides a single GPU latency benchmark in Table 3 showing that an FLIQS-S ResNet-50 configuration achieves a 1.30× speedup over INT8 on an RTX 2080 Ti. However, the paper explicitly states that BOPs are chosen as a "hardware-agnostic" metric to avoid committing to any specific accelerator's latency characteristics.

The consequence. The gap between BOPs and actual runtime matters because different hardware platforms have fundamentally different cost structures that a quadratic bitwidth model cannot capture. On a GPU, a layer operating at INT4 may not achieve 4× the throughput of the same layer at INT8 — the GPU's tensor cores may only support specific precision pairs (e.g., INT8 × INT8 accumulation), meaning that mixing precisions across layers requires costly format conversions that the BOPs model is blind to. On fixed-function accelerators like the TPUv3, certain format combinations may require data marshaling between different systolic array configurations. The paper's GPU latency results in Table 3 illustrate this indirectly: despite a 3.6× reduction in BOPs from INT8 (262 GBOPs) to INT4* (71 GBOPs), the actual speedup is only 1.33×. FLIQS-S, at 73 GBOPs, achieves 1.30× speedup — essentially the same as INT4* — while delivering 1.1 percentage points higher accuracy. This is a positive result for FLIQS but also reveals that BOPs reduction does not translate linearly to wall-clock improvement on GPUs. For a practitioner, the Pareto curves in Figure 5 show what is achievable in cost-model space, not what speedup they will observe on their specific hardware.

What evidence exists in the paper. Table 2 validates the quadratic scaling assumption for FPGA LUT counts but only for a single residual block, not an entire model. Table 3 provides a single datapoint on GPU latency for one ResNet-50 configuration on two GPU architectures — insufficient to characterize how the BOPs-to-latency mapping varies across models, precision mixes, and hardware. The paper does not report any latency measurements for MobileNetV2, EfficientNet, or DeiT-B16, all of which have fundamentally different operation mixes (depthwise separable convolutions, squeeze-and-excitation blocks, self-attention) that interact differently with hardware utilization and memory bandwidth. There is no study of how the Pareto-optimal configurations identified under BOPs would change if optimized against a hardware-specific latency model.

Mitigation status. The paper is transparent that BOPs is a proxy: "BOPs are hardware-agnostic" and "this cost model is validated against FPGA LUT counts" but the validation is partial. The authors frame FLIQS as useful for "model-accelerator co-design" and exploration of "Pareto-optimal models on current hardware systems" (Section 1, Figure 1a), implying that the BOPs-based search identifies promising candidate configurations that would then undergo per-platform latency profiling before deployment. The paper does not claim that the specific GBOPs values translate to wall-clock speedups on any particular hardware. However, for a practitioner evaluating FLIQS against prior methods, the absence of end-to-end latency measurements for the full suite of Pareto-optimal models weakens the practical deployment narrative.

---

6.2 The Abstention From Hardware-Aware Cost Modeling Limits Transferability to Fixed-Function Accelerators

The assumption or constraint. FLIQS's search spaces (FLIQS-S and FLIQS-L, Table 5) are defined entirely in terms of numerical formats — combinations of total bitwidth and exponent/mantissa splits for floating-point, and bitwidths for integer. There is no mechanism in the search to encode hardware constraints such as: which formats are natively supported on a specific accelerator; what the relative throughput of different formats is; whether mixed-precision layers require explicit casting operations that add latency; or whether certain layers (e.g., the first and last) are constrained by the hardware's I/O interfaces to specific formats. FLIQS-S includes only INT4, INT8, and BF16 because these are "standard formats supported in modern GPU micro-architectures" (Appendix A.5), but even on NVIDIA Ampere GPUs, INT4 tensor core operations have specific shape and alignment requirements that may not be satisfied by arbitrary layers. FLIQS-L includes 16 floating-point formats spanning E2M1 through E6M1, which the paper notes are "useful for co-design with custom hardware" — but no custom hardware with native support for all 16 formats exists, meaning the searched configurations cannot be deployed on any current accelerator without emulation.

The consequence. Configurations found by FLIQS-L on floating-point search spaces may be Pareto-optimal in BOPs space but undeployable on real hardware. For instance, a MobileNetV2 configuration that assigns E2M3 to some depthwise convolutions, E3M4 to others, and E2M5 to pointwise convolutions (as in the example configurations in Figures 9) would require an accelerator that natively supports all three formats simultaneously, with the ability to switch between them on a per-cycle basis without pipeline stalls. Such hardware does not exist. Deploying this model on a current GPU would require emulating the unsupported formats using the next-higher supported precision (e.g., E2M3 in E4M3 storage with appropriate scaling), incurring storage and compute overhead that the BOPs model does not capture. The paper acknowledges this implicitly by distinguishing FLIQS-S ("target existing hardware support") from FLIQS-L ("useful for co-design with custom hardware"), but it does not provide any analysis of the deployability gap: how much accuracy is lost if an FLIQS-L configuration is mapped onto FLIQS-S-supported hardware via rounding up to the nearest supported format?

What evidence exists in the paper. The GPU latency results in Table 3 cover only integer FLIQS-S on ResNet-50 — the most hardware-compatible search space on a model with relatively regular structure. There are no GPU latency results for any floating-point FLIQS configuration, nor for any FLIQS-L configuration (integer or floating-point), nor for MobileNetV2, EfficientNet, or DeiT-B16. The FPGA results in Table 2 cover a single ResNet-18 residual block with uniform precision variants and three mixed-precision configurations, implemented in Vivado HLS — this demonstrates the feasibility of implementing mixed-precision arithmetic on reconfigurable hardware but does not characterize the area, power, or frequency penalty of supporting multiple formats on the same device (which typically requires multiplexing between different multiplier configurations, adding logic and potentially reducing clock frequency).

Mitigation status. The paper partially addresses this by providing both a small search space (FLIQS-S) for current hardware and a large search space (FLIQS-L) for co-design. The FLIQS-S results in Table 1 and Figure 5 use only formats natively supported on current GPUs and TPUs, making those Pareto curves directly relevant to deployment. The paper does not, however, provide a tool or methodology for mapping FLIQS-L configurations to FLIQS-S hardware (e.g., a quantization rounding procedure with accuracy impact analysis), which limits the practical utility of the 16-format floating-point search for practitioners without access to custom silicon.

---

6.3 The Reported Efficiency Gains Exclude the Cost of the Search Process Itself

The assumption or constraint. The paper frames FLIQS as an "efficient" one-shot search by comparing its memory footprint to differentiable (branched) approaches in Table 6 (Section A.6): for ResNet-18 at batch size 32, FLIQS requires 46.8 MiB for gradients and 73.6 MiB for activations, while a branched architecture requires 92.6 MiB and 220.8 MiB — roughly 2× and 3× more memory, respectively. However, this comparison covers only the per-step memory cost, not the total computational overhead of the search relative to training a fixed-configuration model. The RL controller runs at every training step after the warmup period: it evaluates the policy (softmax over per-layer logits), samples a discrete configuration, applies it to the model (with switchable clipping thresholds), processes a separate validation batch through the model to compute the quality signal, computes the reward and running-average advantage, and performs a policy gradient update. None of this additional computation appears in the GBOPs reported in Table 1 — those numbers represent only the cost of running the discovered model at inference, not the cost of finding it.

The consequence. A practitioner comparing FLIQS against a simple baseline — e.g., manual mixed-precision assignment ("first and last layers at INT8, everything else at INT4") followed by standard QAT — needs to account for the total computational budget consumed to reach the final model. FLIQS trains the model once while searching, so it replaces the standard QAT training run. But the controller's overhead and the additional validation forward passes add to the per-step cost, potentially making FLIQS training slower than standard QAT at a fixed configuration. Without quantifying this overhead, the paper's efficiency claims are restricted to memory efficiency (important for enabling larger models) but not throughput efficiency (important for reducing training time and energy). The claim that FLIQS "reduces the amount of memory needed for weights, activations, and gradients during the search" (Section 7) is supported by Table 6; the implicit claim that the search is computationally efficient is not quantitatively supported.

What evidence exists in the paper. Table 6 is the only search-overhead quantification, and it covers only memory at a single batch size for a single model (ResNet-18). There is no measurement of: wall-clock time per training step with and without the controller; total training time to convergence compared to a fixed-configuration baseline; the number of additional FLOPs consumed by the controller's forward pass, policy sampling, and REINFORCE update; or how these overheads scale with model size (ResNet-18 → ResNet-50 → DeiT-B16) and search space size (3-option FLIQS-S → 6-option integer FLIQS-L → 16-option floating-point FLIQS-L). The paper notes in Section 3 that per-layer independent logit policies are chosen because "more sophisticated policy models, such as multi-layer perceptron models, offered no quality improvements while being more costly" — this suggests the authors did profile alternative controller architectures and chose the cheapest, but the actual cost numbers for the chosen architecture are not reported.

Mitigation status. The paper does not address this limitation beyond the memory comparison. There is no discussion of whether the controller overhead could be reduced (e.g., by updating the policy every K steps rather than every step, or by using a smaller validation batch for the quality signal). The warmup period (first 25% of training, during which configurations are sampled uniformly at random) is computationally identical to standard QAT with random per-step precision assignments — it is only after warmup that the REINFORCE updates add overhead. The absence of overhead quantification makes it difficult for a practitioner to assess whether the accuracy gains in Table 1 justify the additional training complexity.

---

6.4 Single-Dataset, Single-Task Evaluation Limits Generality of the Integer-vs-Floating-Point Finding

The assumption or constraint. All primary experimental results — every Pareto curve in Figure 5, every accuracy number in Table 1, every configuration diagram in Figures 3, 7, 8, and 9 — are evaluated on a single task: ImageNet (ILSVRC2012) image classification with 1,000 classes. The paper claims that "floating-point models outperform their integer counterparts for the same total bitwidth" (Section 5) and that this finding is robust across six model architectures (ResNet-18, ResNet-50, MobileNetV2, EfficientNet, InceptionV3, DeiT-B16). However, all six architectures are trained and evaluated on the same dataset with the same loss function (cross-entropy) and the same input modality (224×224 RGB images). The Criteo recommendation model experiment in Appendix A.13 is the only evaluation on a different task and data modality (tabular, binary classification), and it uses only a simple 4-layer MLP — insufficient to establish cross-domain generality.

The consequence. The floating-point-over-integer finding may be specific to the distributional properties of ImageNet-trained vision models. Convolutional networks trained on natural images with batch normalization produce weight and activation distributions that are approximately zero-mean, unimodal, and roughly Gaussian or Laplacian in shape — precisely the regime where floating-point formats, with their logarithmic spacing of representable values and concentration of precision near zero, should outperform uniform integer grids. However, other tasks and architectures produce different distributional profiles: transformer-based language models have attention logits with heavy-tailed distributions; recommendation models have embedding tables with sparse, categorical features whose values cluster at discrete points; graph neural networks aggregate messages with non-Gaussian degree-dependent variances; speech models process spectrograms with wide dynamic range. In these settings, the optimal exponent-mantissa split may differ, and integer quantization with appropriate per-channel or per-token scaling may be competitive or superior. The paper's claim of floating-point dominance — while strongly supported for ImageNet vision models — does not have evidence behind it for any other domain.

What evidence exists in the paper. The single non-ImageNet experiment (Appendix A.13, Figure 13) shows that floating-point FLIQNAS-L outperforms integer FLIQNAS-L on the Criteo CTR prediction task by a small margin (the AUC curves for floating-point sit slightly above integer at most MBOPs values), consistent with the ImageNet finding but far from a rigorous cross-domain validation. The paper does not evaluate FLIQS on any NLP, speech, video, or reinforcement learning task. It does not discuss whether the per-layer format assignments discovered on ImageNet (e.g., higher precision for first/last layers, depthwise convolutions, attention blocks) are task-specific or reflect universal properties of these layer types. It does not analyze whether the distribution of weight and activation values — which determines the relative advantage of floating-point over integer — differs systematically between image classification and other domains.

Mitigation status. The paper does not claim cross-domain generality explicitly. The abstract states results on "multiple convolutional and vision transformer networks," which implicitly scopes the findings to vision. However, the claim of performing "the largest comparison of integer and floating-point mixed-precision networks" (Section 1, contribution 3) is a methodological claim about scale, not a domain claim. A practitioner deploying FLIQS on a non-vision task (e.g., BERT-based NLP model, WaveNet speech synthesizer, DeepFM recommendation model) should treat the floating-point-over-integer finding as unvalidated for their domain. The paper's architecture diversity (convolutional + transformer) provides some evidence that the finding is not specific to a single neural network primitive, but the domain diversity is lacking.

---

6.5 The Search Overhead From Entropy Regularization Tuning and Cost Target Specification Is Not Characterized

The assumption or constraint. FLIQS requires the practitioner to specify three key hyperparameters before the search begins: the cost target $C_T$ (in GBOPs), which determines the desired accuracy-efficiency trade-off; the cost scalar $\gamma$, which balances the quality and cost terms in the reward function (Equation 1); and the entropy regularization endpoint $\beta_H^{\text{end}}$ (set to 0.5 for all experiments), which controls the exploration-exploitation schedule. The paper sweeps $C_T$ to generate Pareto curves in Figure 5 — each point on the FLIQS curves represents a separate search with a different cost target — but it does not report how the choice of $C_T$, $\gamma$, or $\beta_H^{\text{end}}$ affects the quality of the final configuration, the stability of the search, or the time to convergence. The entropy regularization analysis in Figure 4(b) (right panel) shows that higher entropy improves accuracy, but this is a sweep of $\beta_H^{\text{end}}$ in an already-successful regime, not an exploration of failure modes when these hyperparameters are poorly chosen.

The consequence. A practitioner deploying FLIQS on a new model architecture and dataset faces an implicit hyperparameter tuning problem. Setting $C_T$ too low may cause the reward function to heavily penalize any configuration that exceeds the target, driving the controller toward extremely cheap but low-accuracy configurations — potentially getting stuck in a local optimum where all layers are at minimum bitwidth long before the policy has explored higher-precision options. Setting $C_T$ too high may cause the cost penalty to be negligible, allowing the controller to select near-uniform high precision (e.g., all layers at BF16) without exploring mixed-precision options at all, since those configurations are penalized for being "too cheap" (the absolute value in the reward penalizes deviation in either direction). Setting $\gamma$ too high could cause the cost term to dominate the quality signal, making the controller indifferent to accuracy differences. Setting $\gamma$ too low could cause the controller to ignore the cost constraint, producing accurate but expensive models. The paper does not provide guidance on how to select these hyperparameters for a new problem, nor does it characterize the sensitivity of the final Pareto frontier to their values.

What evidence exists in the paper. The paper reports that $\beta_H^{\text{end}} = 0.5$ is used for all experiments (Section 4), and that the cosine schedule with this endpoint was effective — but there is no ablation showing what happens with $\beta_H^{\text{end}} = 0.1$ (premature convergence) or $\beta_H^{\text{end}} = 2.0$ (excessive entropy, failure to converge). The reward function's $\gamma$ parameter is not ablated at all — there is no sensitivity analysis showing how the Pareto curves in Figure 5 shift with different $\gamma$ values, nor whether the searched configurations are robust to $\gamma$ misspecification. The paper sweeps $C_T$ by training separate models at different cost targets to generate Pareto curves, but this is a usage of the hyperparameter (exploring the trade-off space), not a sensitivity analysis (testing whether the search reliably finds the optimal configuration for a given $C_T$). The controller warmup period (25% of training) and RL learning rate ($4.6 \times 10^{-3}$, from Section A.4) are similarly reported without sensitivity analysis.

Mitigation status. The paper does not address this limitation. The cosine entropy schedule is presented as a key innovation, but the specific endpoint value (0.5) is justified only by its empirical success in the reported experiments. For a practitioner, replicating FLIQS on a new problem would likely require some amount of hyperparameter tuning — possibly including multiple search runs with different $C_T$ and $\gamma$ values — which erodes the "one-shot" advantage. The paper's framing of FLIQS as eliminating retraining (the model training part) is accurate, but the search still requires hyperparameter configuration that may itself require iterative experimentation, a form of "meta-retraining" that the paper does not discuss.

---

6.6 Vision Transformer Results Suggest the Approach May Not Scale Arbitrarily to Large Attention-Based Models

The assumption or constraint. The paper includes DeiT-B16 as a representative vision transformer to demonstrate that FLIQS scales beyond convolutional architectures (Section 5, Table 1, Figure 20, Figure 26). DeiT-B16 at 1.0× width achieves 86 million parameters and 1,079 GBOPs in BF16 — roughly 2.3× the parameter count and BOPs of ResNet-50. The paper presents DeiT-B16 results as evidence that FLIQS can handle "10 times the model cost as BF16 MobileNetV2" (Section 7), positioning it as scalable to larger models.

The consequence. Examining the DeiT-B16 results in Table 1 reveals a pattern that may indicate a scalability limit for the approach. For integer quantization at 1.0× width: uniform INT8 achieves 79.49% Top-1 at 1,079 GBOPs. FLIQS-S achieves 79.47% at 275 GBOPs — essentially identical accuracy at 25% of the cost, an impressive cost reduction. However, the marginal improvement from expanding the search space from FLIQS-S (3 options: INT4, INT8, BF16) to FLIQS-L (6 options: INT4–INT8, BF16) is negligible: FLIQS-L achieves 79.35% at 272 GBOPs, which is actually 0.12 points lower than FLIQS-S. For floating-point, the pattern is similar: uniform E4M3 achieves 79.17% at 1,079 GBOPs; FLIQS-S achieves 79.27% at 275 GBOPs (0.10 points better); FLIQS-L achieves 79.54% at 271 GBOPs (0.27 points better than FLIQS-S). The improvements from mixed-precision over uniform precision are dramatically smaller for DeiT-B16 than for convolutional architectures: for ResNet-18, FLIQS-L integer achieves 70.61% vs. INT8 at 70.60% — matching accuracy at 27% of the cost — but the accuracy ceiling for DeiT-B16 (79.5%) is already approached by uniform INT8, leaving almost no room for mixed-precision to improve accuracy, only cost.

This matters because it suggests that as models become more accurate on a given task (DeiT-B16 at ~79.5% is 2+ points above ResNet-50 at ~77.3%), the benefit of mixed-precision search shifts from "improve accuracy at fixed cost" to "reduce cost at fixed accuracy." Cost reduction is valuable, but the paper's headline claims about accuracy improvement (1.31 points for ResNet-18, 0.90 points for ResNet-50) may not generalize to higher-accuracy models. For state-of-the-art vision transformers (ViT-H, Swin-L, ConvNeXt-XL) that achieve 85-90% ImageNet Top-1, FLIQS may offer primarily cost reduction, not accuracy improvement — a different value proposition than the one emphasized in the abstract.

Furthermore, the per-layer format allocation for DeiT-B16 (Figures 3, 8, 9) shows that the self-attention blocks (query, key, value projections) consistently receive higher precision (6–8 bits) than the MLP blocks (4–6 bits). For larger transformers with more attention heads and deeper layers, the number of high-precision attention projections grows, potentially limiting the cost savings achievable through mixed-precision if attention mechanisms are inherently more sensitive to quantization. The paper does not evaluate on models larger than DeiT-B16 (e.g., ViT-Large, Swin-Base), so the scaling behavior of this sensitivity pattern to larger transformer architectures is unknown.

What evidence exists in the paper. The DeiT-B16 results in Table 1, Figure 20, and Figure 26 are the primary evidence. The diminishing return from FLIQS-S to FLIQS-L (79.47% → 79.35% for integer, 79.27% → 79.54% for floating-point) compared to the larger gains seen on ResNet-18 (69.91% → 70.61% for integer FLIQS-S to FLIQS-L) supports the interpretation that search space expansion yields less benefit on higher-accuracy models. The paper does not explicitly discuss this trend, nor does it provide an explanation for why DeiT-B16 benefits less from mixed-precision than convolutional models.

Mitigation status. The paper includes DeiT-B16 as a demonstration of scalability to non-convolutional architectures, which is valuable, but it does not analyze the implications of the reduced mixed-precision benefit for future, larger models. The discussion in Section 7 frames the DeiT-B16 results positively ("FLIQS can efficiently traverse large quantization search spaces... on more substantial models"), which is accurate — the search does successfully complete — but it does not address whether the benefit of the search scales with model size or instead saturates. For practitioners considering FLIQS for large transformer deployment, the cost reduction is clearly demonstrated (4× reduction in GBOPs at iso-accuracy), but the expectation of accuracy improvement should be tempered relative to the convolutional architecture results.

7. Implications and Future Directions

How This Work Changes the Landscape

FLIQS shifts the mixed-precision quantization problem from a two-phase "search-then-retrain" pipeline to a single integrated process where the search and final model training are one and the same. This is not merely an engineering convenience — it changes the economic calculus of deploying quantized models. Prior to FLIQS, a practitioner choosing between post-training quantization (PTQ) and quantization-aware training (QAT) faced a structural trade-off: PTQ was fast and required no retraining, but left 1–2 percentage points of accuracy on the table (the gap between HAWQ-V3 at 76.73% and FLIQS-S at 77.32% on ResNet-50, with FLIQS-S using only 53% of the BOPs); QAT with differentiable search (EDMIPS) recovered that accuracy but required retraining from scratch, doubling the training cost and introducing deployment friction from the branched-to-discrete configuration transition. FLIQS collapses this trade-off: it achieves QAT-level accuracy without the retraining penalty, making high-accuracy mixed-precision quantization approximately as operationally simple as PTQ. For organizations deploying models across diverse hardware, this directly translates to reduced engineering time and computational expenditure — the difference between "run FLIQS once" and "run a search, extract the configuration, retrain the model, validate, redeploy."

The conceptual reframing that enables this — treating search-model interference as a controllable noise source proportional to policy entropy — has reach beyond quantization. The switching error analysis in Section 4 formalizes a dynamic that affects any one-shot architecture search where the model's training objective and the controller's exploration interact: neural architecture search with weight sharing, pruning during training, dynamic sparsity pattern selection, adaptive activation function selection. In each case, the controller modifies the model during training, introducing a perturbation whose magnitude depends on how drastically and how frequently the modifications occur. The insight that this perturbation is proportional to the controller's policy entropy — and that a cosine entropy schedule can drive it to zero by training's end — provides a general template for converting retraining-required searches into retraining-free ones. The specific mechanism (entropy regularization schedule) may need adaptation for different search spaces, but the diagnostic concept (measure interference via policy entropy, schedule exploration-to-exploitation to match the quality signal's reliability curve) transfers directly.

The paper also resolves a latent tension in the quantization literature between integer and floating-point precision. Prior work on integer mixed-precision (HAWQ, EDMIPS, Bayesian Bits) and floating-point uniform precision (HFP8, MPFP, FPQuant) operated in separate research threads, making it impossible to determine whether the observed accuracy advantages of FP8 over INT8 were due to the format family or the specific E4M3 exponent-mantissa split. FLIQS's unified search space — where integer and floating-point formats compete directly under the same cost model and training protocol — provides the first controlled comparison. The consistent result that floating-point dominates integer at equal total bitwidth across six model architectures (explicitly stated in Section 5, visualized in Figures 5 and 12, and confirmed in the degeneracy of joint integer-floating-point searches to pure floating-point) establishes a clear empirical ordering. This finding should redirect research attention: rather than developing ever-more-sophisticated integer quantization schemes (non-uniform quantization grids, learned step sizes, vector quantization), the field should prioritize low-precision floating-point arithmetic, both in algorithm development and in hardware design. The FP8 standardization effort (Micikevicius et al., 2022) already reflects this direction for training; FLIQS extends the argument to inference and to mixed precision, implying that future accelerators should invest silicon area in efficient sub-8-bit floating-point units rather than in INT4/INT8 units with comparable throughput.

The FLIQNAS results introduce a design principle that challenges conventional wisdom about model efficiency: for a fixed computational budget (measured in BOPs), expanding architectural dimensions (channels, kernel sizes) yields higher accuracy returns than increasing numerical precision. This is visible in Figure 6, where the joint architecture-quantization search decisively separates from pure quantization search above ~15 GBOPs, and the paper explicitly states that "for a fixed compute budget, larger models benefit from increasing architectural dimensions over bitwidth" (Section 7). This principle inverts the typical engineering intuition — "use the cheapest precision that doesn't hurt accuracy, then scale the model" — and replaces it with "allocate compute to model capacity first, then use whatever precision the residual budget permits." The quadratic BOPs model (bitwidth² × MACs) provides the mechanism: halving precision frees a factor of 4 in compute, which can be reinvested into 4× wider channels. The empirical finding is that this reinvestment is nearly always profitable, at least for MobileNetV2 on ImageNet. If this principle generalizes (a critical open question), it has direct implications for hardware-software co-design: accelerator architects should optimize for throughput at moderate precision (4–6 bits) rather than peak efficiency at a single precision, and model designers should treat bitwidth as a tunable parameter alongside width and depth in their efficiency calculations.

Finally, the paper changes the perceived cost of RL-based architecture search. Differentiable NAS and its quantization extensions (EDMIPS, BatchQuant, DNAS) gained popularity partly because they were perceived as more elegant — end-to-end gradient-based optimization — and partly because RL approaches carried a reputation for sample inefficiency and training instability. FLIQS demonstrates that a minimal RL controller (per-layer independent softmax policies, REINFORCE with a running-average baseline) can match or exceed the accuracy of differentiable methods while consuming 2–3× less memory (Table 6) and scaling to larger models (DeiT-B16, with 10× the cost of MobileNetV2). This does not mean RL is universally superior to differentiable search — the two approaches make different memory-compute trade-offs, and differentiable methods may be preferable when memory is abundant and the branching factor is small. But FLIQS clears the bar of "RL is too unstable for one-shot quantization search," which should encourage exploration of RL-based approaches for other training-time architecture optimization problems where differentiable formulations require prohibitive memory overhead.

Follow-Up Research This Work Enables

Characterizing the deployability gap between FLIQS-L and FLIQS-S configurations. The paper defines two search spaces — FLIQS-S (3 formats, targeting existing hardware) and FLIQS-L (6 integer or 16 floating-point formats, targeting custom hardware and co-design) — but provides no methodology for mapping FLIQS-L configurations onto FLIQS-S hardware. A strong follow-up study would take the FLIQS-L floating-point configurations published for MobileNetV2 (Figure 9) and quantize them to the nearest FLIQS-S-supported format (e.g., E2M3 → E2M1 at 4 bits; E3M2 → E4M3 at 8 bits) with and without additional fine-tuning, measuring the accuracy degradation. This would quantify how much of the FLIQS-L advantage is realizable on current hardware versus deferred to future accelerators. The key measurement is the accuracy vs. BOPs Pareto curve for "FLIQS-L searched, then rounded to FLIQS-S formats and fine-tuned" compared against native FLIQS-S search — if the rounded configuration retains most of its accuracy advantage, FLIQS-L serves as a practical search space today; if accuracy collapses, FLIQS-L is genuinely a co-design tool only.

Cross-domain validation of the floating-point-over-integer finding. The paper establishes floating-point dominance for ImageNet classification across six architectures, but provides only a single non-vision datapoint (Criteo recommendation, Appendix A.13) and no NLP, speech, or video results. A rigorous extension would apply FLIQS to: (1) a BERT-base or T5-small model on GLUE/SuperGLUE benchmarks, where weight distributions are shaped by LayerNorm rather than BatchNorm and activation distributions include attention logits with heavy tails; (2) a WaveNet or Conformer model on LibriSpeech, where activations span wide dynamic range; (3) a 3D ResNet on Kinetics-400, where temporal convolutions introduce additional quantization sensitivity patterns. For each domain, the study would report the integer-vs-floating-point Pareto curves and analyze whether the per-layer format allocation patterns observed on ImageNet (higher precision for first/last layers, attention blocks, depthwise convolutions) transfer across domains. A negative result — e.g., finding that integer matches or exceeds floating-point on transformer language models due to different activation statistics — would be equally valuable, establishing boundary conditions on the floating-point advantage and guiding accelerator architects on when to invest in integer versus floating-point hardware.

Direct latency-optimized search replacing the BOPs cost model. The paper acknowledges BOPs as a hardware-agnostic proxy and provides limited latency validation (one ResNet-50 configuration on two GPUs, Table 3). A natural and practically impactful extension replaces the BOPs cost term in Equation 1 with a hardware-specific latency predictor — either a lookup table of per-layer-per-format latencies profiled on a target device (e.g., NVIDIA A100, Google TPUv4, Apple Neural Engine) or a learned latency model trained on a sample of configurations. The key experiment: run FLIQS with identical hyperparameters except for the cost function (BOPs vs. latency-predictive), and compare the resulting Pareto curves in both BOPs-space and actual latency-space. The hypothesis — that latency-optimized search finds configurations with better real-world speedup at equivalent accuracy — is plausible given the poor BOPs-to-latency correlation visible in Table 3 (3.6× BOPs reduction → 1.33× speedup). If latency-optimized search significantly improves the accuracy-latency frontier, it would transform FLIQS from a research tool into a deployment pipeline; if the improvement is marginal, it would validate BOPs as a sufficient proxy and simplify future methodology.

Entropy regularization analysis for different search space sizes and controller architectures. The paper introduces cosine entropy regularization as the mechanism enabling retraining-free search and validates it via the retraining ablation in Table 8, but does not characterize how the optimal entropy schedule depends on search space properties. A systematic ablation would vary: (1) search space size (3-option FLIQS-S vs. 6-option integer FLIQS-L vs. 16-option floating-point FLIQS-L) and measure the entropy trajectory, policy convergence rate, and final accuracy; (2) the entropy regularization endpoint $\beta_H^{\text{end}}$ across a wide range (0.0 to 2.0) for each search space, identifying the regime where retraining becomes necessary; (3) alternative schedules (linear decay, step-function decay, constant with different values) compared against cosine. This would produce a practical guide for setting $\beta_H^{\text{end}}$ given search space cardinality and training duration, and would test whether the relationship $E[\Delta_S] \propto H_M$ (Equation 6) holds quantitatively — e.g., does doubling the search space size require doubling the entropy budget to achieve equivalent final accuracy?

FLIQNAS on non-MobileNet architectures and higher-resolution inputs. The joint architecture-quantization search results (Section 6) are limited to a single architecture family (MobileNetV2) on 224×224 ImageNet. Extending FLIQNAS to ResNet (with tunable block counts and widths), EfficientNet (with tunable compound scaling coefficients), and vision transformers (with tunable embedding dimension, head count, and MLP ratio) would test whether the principle that "architectural dimensions dominate bitwidth" generalizes beyond inverted bottleneck structures. Specifically, for vision transformers: does the controller allocate extra BOPs budget to more attention heads (architectural) or higher precision in the attention projections (bitwidth)? The answer has direct consequences for designing efficient transformer accelerators. Including higher-resolution inputs (e.g., 384×384 or 512×512) would additionally test how the architecture-precision trade-off interacts with input size — early layers in higher-resolution models process larger activation tensors, making their precision choice proportionally more expensive in BOPs, which may shift the optimal allocation toward lower precision in early layers and higher capacity in later layers.

FLIQS as a design tool for mixed-precision accelerator architecture. The paper frames FLIQS-L as useful for hardware/software co-design but does not demonstrate this use case. A co-design study would take FLIQS-L floating-point configurations (with their per-layer format assignments) from multiple model architectures and compute the frequency distribution of requested formats — e.g., what fraction of all convolutional layers across ResNet-50, MobileNetV2, EfficientNet, and DeiT-B16 are assigned E4M3 vs. E3M4 vs. E2M5? This distribution informs which formats an accelerator should natively support: if 80% of layers use only 4 format variants, implementing all 16 provides minimal benefit. The study would then design (in simulation or RTL) a configurable MAC unit supporting the top-K most frequently requested formats, and compare its area-power-frequency characteristics against a unit supporting uniform E4M3 (current FP8 standard) and against a unit supporting all 16 formats. This closes the loop from automated search to hardware specification, demonstrating how FLIQS can guide accelerator design toward the formats that empirically matter most for DNN accuracy.

Practical Applications and Downstream Use Cases

Deploying a single model architecture across a heterogeneous hardware fleet. An organization maintaining a computer vision service (e.g., object detection for autonomous vehicles, content moderation for social media) needs to deploy the same model architecture on a mix of hardware: cloud GPUs (supporting INT8, FP16, BF16, FP8), edge TPUs (supporting INT8), mobile NPUs (supporting INT8 and FP16), and possibly FPGAs (supporting arbitrary precision). Currently, this requires either uniform quantization to the lowest-common-denominator format (e.g., INT8 everywhere, sacrificing accuracy on platforms that could use higher precision) or manual per-platform configuration (engineering-intensive and fragile). FLIQS changes this workflow: run a single FLIQS-L search on the target architecture with a cost target appropriate for the most constrained platform, producing a mixed-precision configuration. For less constrained platforms, the same FLIQS framework can be re-run with higher cost targets, producing a family of configurations along the Pareto frontier — all sharing the same training run and architectural backbone, differing only in per-layer format assignments. The deployment pipeline becomes: train once with FLIQS, extract the configuration matching each hardware target's BOPs budget from the Pareto curve, deploy. The concrete benefit is the accuracy improvement quantified in Table 1: for ResNet-50, FLIQS-S at 81 GBOPs achieves 77.32% vs. the INT4* uniform baseline at 76.30% and 71 GBOPs — a 1.02 percentage point accuracy gain at similar cost. Scaled across a fleet serving millions of inferences, a 1-point accuracy improvement in content moderation or object detection has direct business impact in reduced false positives/negatives.

Efficient model deployment on FPGA-based edge accelerators. FPGAs are uniquely positioned to benefit from FLIQS because they can implement arbitrary-precision arithmetic natively — unlike fixed-function ASICs, an FPGA can synthesize a multiplier supporting exactly E3M3 or INT5 without emulation overhead. The FLIQS-L search space, with 16 floating-point and 6 integer formats, is designed precisely for this use case. The workflow: a team building an FPGA-based video analytics pipeline (e.g., real-time person detection on a Xilinx UltraScale+ device) runs FLIQS-L on their model (MobileNetV2 or EfficientNet) with a cost target set to the BOPs budget that meets their frame-rate requirement. The search produces a per-layer format assignment and channel width configuration optimized for their specific accuracy-latency target. They then synthesize the exact mixed-precision datapath in HLS, using the format assignment directly — no rounding to coarser formats, no emulation of unsupported precisions. Table 2 shows what this buys on a ResNet-18 block: a FLIQS-L configuration at 1.07× the LUT count of uniform INT4 achieves 2.81 percentage points higher accuracy (67.31% → 70.12%). For a drone or security camera where every percentage point of detection accuracy matters and power is severely constrained, this precision-allocated FPGA implementation is a direct competitive advantage over uniform-precision alternatives.

Budget-constrained model design for mobile and IoT devices. A startup or research lab designing a custom efficient model for on-device deployment (e.g., a keyword spotter, gesture recognizer, or image classifier for a wearable) faces a fixed compute budget set by the target microcontroller or low-power DSP, which may support only specific integer precisions (e.g., INT8 and INT16) or, increasingly, low-precision floating-point (ARM's CMSIS-NN library now includes FP16 support). The traditional design workflow — pick an architecture, train at full precision, apply PTQ, observe the accuracy drop, manually adjust sensitive layers, retrain, repeat — is iterative and slow. FLIQS replaces this with a single automated run: the user specifies the architecture family (say, MobileNetV3-Small), the search space reflecting the hardware's capabilities (FLIQS-S for fixed-function DSPs, FLIQS-L for software-programmable cores), and a cost target derived from the device's peak throughput and latency requirement. The search automatically discovers which layers need higher precision and which can be aggressively quantized, producing a deployment-ready model in one training pass. The concrete numbers for MobileNetV2 (the closest proxy in the paper for mobile-efficient architectures): at ~7 GBOPs, integer FLIQS-L achieves 71.87% vs. uniform INT8 at 72.83% but with 37% of the cost. For a wearable running on a coin-cell battery where every milliwatt matters, the 2.7× reduction in BOPs (and corresponding reduction in energy per inference) may be worth a ~1 point accuracy trade-off — and FLIQS finds this configuration automatically, without the designer manually guessing which layers can be dropped to INT4.

When to Prefer This Method

The paper provides the framework for comparing FLIQS against specific named alternatives across different deployment scenarios, though it does not present a formalized decision tree. Based on the empirical evidence and design characteristics, the following decision rules emerge:

• Prefer FLIQS over PTQ-based quantization search (HAWQ-V3, ZeroQ) when the target model has capacity for QAT-level accuracy and the deployment pipeline can accommodate a full training run from scratch. FLIQS delivers 0.6–1.3 points higher accuracy than HAWQ-V3 at comparable or lower BOPs (Table 1: ResNet-18 +1.31 points, ResNet-50 +0.59 points) because QAT allows weights to adapt to quantization noise — a capability PTQ fundamentally lacks. The cost is training from scratch (350–400 epochs) rather than applying PTQ to a pre-trained model.

• Prefer FLIQS over differentiable QAT search (EDMIPS, DNAS) when memory is constrained during training or when the search space is large. Table 6 quantifies the memory advantage: FLIQS uses ~2× less gradient memory and ~3× less activation memory than branched approaches on ResNet-18, a gap that widens with model size and number of quantization options. For models like DeiT-B16 or EfficientNet-B4 — where the memory overhead of maintaining separate branches for 6–16 format options would be prohibitive — FLIQS's RL-based discrete sampling is the only viable approach. The trade-off is that FLIQS's REINFORCE updates are higher-variance than gradient-based branch weight optimization, which may require more careful entropy schedule tuning.

• Prefer FLIQS-L floating-point search when the target hardware supports arbitrary-precision floating-point (FPGAs, custom ASICs in development) and maximizing accuracy per BOPs is the primary objective. The 16-format search space consistently outperforms the 3-format FLIQS-S in accuracy at equivalent cost (e.g., ResNet-18 integer FLIQS-L achieves 70.61% vs. FLIQS-S at 69.91%, Table 1), and floating-point dominates integer across the board. However, on fixed-function hardware that only supports INT4/INT8/BF16 or E4M3/BF16, FLIQS-S is the appropriate choice — FLIQS-L configurations cannot be deployed without format emulation whose cost is not modeled.

• Prefer FLIQNAS (joint architecture-quantization search) over pure quantization search (FLIQS) when the architectural dimensions (channel widths, kernel sizes) are flexible and the compute budget exceeds ~15 GBOPs for MobileNetV2-scale models. Figure 6 shows that below ~15 GBOPs, pure quantization search and joint search produce comparable accuracy — the budget is too tight for architectural expansion to help. Above ~15 GBOPs, FLIQNAS delivers 2.2–2.7 points higher accuracy at equivalent cost by reallocating compute from bitwidth to channel width. For practitioners designing custom efficient models where both architecture and precision are free parameters, FLIQNAS provides the Pareto-optimal frontier; for those constrained to a fixed architecture (e.g., deploying a standard ResNet-50), pure FLIQS is sufficient.

• Prefer manual mixed-precision heuristics (e.g., "first and last layers at INT8, everything else at INT4") only when the search overhead (implementing the RL controller, tuning hyperparameters) exceeds the engineering cost of the accuracy gap. The paper shows that FLIQS consistently outperforms uniform precision baselines (e.g., INT4* at 76.30% vs. FLIQS-S at 77.32% on ResNet-50, Table 3), and the per-layer format assignments it discovers (Figures 7, 8) are more nuanced than simple first/last/attention heuristics — they allocate precision differently to pointwise vs. depthwise convolutions, early vs. late stages, and main-branch vs. skip-branch convolutions. For a one-off deployment on a well-understood architecture where 1–2 points of accuracy are not critical, manual heuristics may suffice. For any scenario where accuracy is competitive or the architecture is novel, FLIQS's automated search is warranted.

• Do NOT prefer FLIQS when the model is being deployed on hardware where the BOPs-to-latency mapping is highly nonlinear and latency is the binding constraint. The paper's BOPs-based optimization may select configurations that are Pareto-optimal in bit-operations but suboptimal in wall-clock time due to format conversion overhead, memory bandwidth bottlenecks, or accelerator utilization effects not captured by the quadratic cost model. In such cases, either FLIQS should be extended with a latency-predictive cost model (as suggested in the follow-up research directions above), or a hardware-specific search (e.g., latency-table-based methods) should be used instead.