Overcoming Simplicity Bias in Deep Networks using a Feature Sieve

🎯 Pitch

Deep networks stubbornly favor simple, spurious features over more predictive ones—but a new 'feature sieve' automatically erases these lazy shortcuts in lower layers, forcing the network to learn richer representations. The method outperforms systems that explicitly use bias labels, without ever needing to know what the spurious features are.

1. Executive Summary

This paper proposes SiFER (Sieving Features for Robust learning), a direct interventional method for mitigating simplicity bias in deep neural networks by automatically identifying and suppressing easily-computable spurious features in lower network layers, thereby enabling higher layers to extract richer representations. Experiments on controlled datasets (CMNIST, CIFAR_MNIST) and real-world debiasing benchmarks (BAR, CelebA, NICO, ImageNet-9, ImageNet-A) using ResNet-18 demonstrate that SiFER's alternating identify-and-erase workflow — operationalized through an auxiliary network that predicts labels from intermediate representations (identifying simple predictive features) and then applies a forgetting loss to erase those features at earlier layers — achieves substantial gains over competitive baselines without using any prior knowledge of spurious attributes. The method delivers relative improvements of 3.2% on BAR, 4% on ImageNet-A, and 11.1% on ImageNet-A over the best comparison methods, while crucially establishing that SiFER can outperform baselines that explicitly incorporate bias attribute labels, though only when a suitable validation set — either in-distribution or out-of-distribution — is available to tune the feature sieve's hyperparameters via generalization error.

2. Context and Motivation

The Core Problem: Deep Networks Over-Depend on Simple Features

The central problem this paper addresses is simplicity bias in deep neural networks — the systematic tendency of DNNs trained with stochastic gradient descent to latch onto simple, easily-computable features while ignoring more complex but potentially stronger predictive features. Shah et al. (2020) formally identified this phenomenon, providing theoretical and empirical evidence that neural networks exhibit a preference for the simplest decision boundary consistent with the training data, even when more sophisticated features would yield better generalization. Pezeshki et al. (2021) advanced an analytical understanding through the lens of "gradient starvation," showing that during training, gradients from simpler features dominate the learning dynamics and effectively starve the gradients that would otherwise encourage the network to develop more complex representations.

This bias is not merely a theoretical curiosity — it manifests in concrete, practically harmful ways. Figure 1 of the paper illustrates three canonical failure modes:

Color over shape: When both color and shape are perfectly predictive of class labels in synthetic datasets, DNNs overwhelmingly rely on color at the expense of shape, since color is computationally cheaper to extract in early convolutional layers.
Texture over object identity: Geirhos et al. (2018) demonstrated that ImageNet-trained CNNs make classification decisions primarily based on texture rather than object shape. Figure 1b shows a cat image with an elephant-skin texture overlaid — standard classifiers predict "elephant" despite clear feline morphology, revealing a catastrophic reliance on superficial texture cues.
Spurious demographic correlations: Classifiers trained on face datasets learn to associate hair color with gender because of dataset imbalances. Figure 1c shows that blond-haired male faces are routinely misclassified as female, since the training data contains overwhelmingly more blond-haired women than blond-haired men (fewer than 1% of blond-haired individuals in CelebA are male).

These examples share a common mechanism: the network discovers a computationally cheap feature that happens to correlate with the label in the training distribution, builds its decision boundary around that feature, and then fails catastrophically when deployed on data where the spurious correlation breaks. The learned model is not "wrong" in the narrow sense — it achieves high training and validation accuracy on data drawn from the same distribution — but it is brittle and biased, failing on out-of-distribution examples, under-represented subgroups, or adversarial perturbations that break the spurious correlation.

Why This Problem Matters: Real-World Consequences and Theoretical Depth

The practical stakes of simplicity bias are enormous because machine learning models are increasingly deployed in high-stakes domains where brittleness and bias are unacceptable:

Face recognition and surveillance: Models that over-index on hair color, skin tone, or background context rather than facial geometry produce systematically higher error rates for demographic minorities. This compounds existing inequities in law enforcement, hiring, and access control.
Healthcare: Diagnostic models trained on data from specific hospitals may learn to associate image artifacts, scanner types, or patient positioning with disease labels rather than genuine pathological features. When deployed at a different hospital with different equipment or protocols, these models fail silently.
Credit scoring and lending: Models may discover that zip code, browser type, or purchase history correlate with creditworthiness in training data, encoding discriminatory proxies for race, class, or age into their decision boundaries.
Autonomous systems: Self-driving perception systems that rely on background context (e.g., "cars appear on roads, not on water") rather than robust visual features of objects themselves can fail in novel environments or edge cases.

Beyond the immediate ethical and safety concerns, simplicity bias threatens the fundamental promise of deep learning: that neural networks can automatically discover rich, hierarchical representations from data without human feature engineering. If networks consistently collapse onto the simplest available features, they fail to extract the full predictive power latent in the training data. This means that even with abundant data and compute, learned representations may be substantially impoverished relative to what the architecture is theoretically capable of learning.

The problem also has deep theoretical significance for understanding neural network training dynamics. Simplicity bias connects to fundamental questions about implicit regularization in gradient descent — why do networks prefer simple solutions when the loss landscape contains many valid parameter configurations? Shah et al. (2020) connect this to the spectral bias of neural network training (neural networks learn low-frequency functions first), while Pezeshki et al. (2021) frame it as a consequence of gradient competition during optimization. Understanding and overcoming simplicity bias therefore advances our theoretical grasp of deep learning itself, not merely its practical reliability.

Prior Approaches and Where They Fall Short

The paper identifies several families of prior work on debiasing, each with significant limitations that the feature sieve sets out to address.

Approaches Requiring Explicit Bias Attribute Labels

A substantial body of prior work assumes access to explicit labels for the spurious attribute — that is, the practitioner knows in advance which feature the model should not rely on, and provides labeled examples or group annotations accordingly. Methods in this category include:

Group-DRO (Sagawa et al., 2019) minimizes the worst-case loss over pre-defined groups (e.g., "blond-haired women," "blond-haired men," "non-blond-haired women," "non-blond-haired men"), requiring the data to be partitioned by the spurious attribute.
EnD (Tartaglione et al., 2021) uses an information bottleneck regularizer that explicitly depends on bias attribute labels to prevent the network from encoding spurious information.
CSAD (Zhu et al., 2021) explicitly extracts separate representations for target and bias features using attribute labels, then minimizes their mutual information.
LearnedMixin and RUBi (Clark et al., 2019; Cadene et al., 2019) were designed for visual question answering where the spurious modality (question-only bias) is known in advance.

The critical weakness of these approaches is practical infeasibility. In real-world deployments, practitioners rarely know which features will become spuriously correlated with labels until after the model has failed. Even if suspected spurious features are identified, manually annotating them across large datasets is expensive and error-prone. For many applications, the spurious feature may be an abstract, emergent property of the data (e.g., texture statistics, background context, image resolution) that is difficult to label at the instance level. The paper emphasizes this limitation in Section 2.2:

"This reduces their practicality since both identifying, and manually labeling biased instances and dimensions in real-life data are significant barriers."

The paper explicitly positions SiFER in contrast to this family of methods by never using attribute labels in any of its real-world experiments (as indicated by the "Spurious Attribs" column showing ✗ throughout Tables 3–6).

Approaches That Do Not Use Attribute Labels

Recognizing the impracticality of manual bias labeling, a more recent line of work attempts debiasing without explicit attribute supervision. The paper surveys several technical strategies within this category and identifies specific limitations in each:

Alternate network approaches (LfF, Debian, LWBC): Nam et al. (2020) proposed Learning from Failure (LfF), which trains a "biased" network first (using generalized cross-entropy, which encourages the network to focus on the easiest predictive features), then trains a second "debiased" network that upweights samples where the biased network disagrees with the label — operating on the intuition that these are samples violating the spurious correlation. Debian (Li et al., 2022) alternates between a "discoverer" network that identifies multiple unknown biases and a classifier that unlearns them. LWBC (Kim et al., 2022) uses a committee of biased classifiers to assign higher weights to bias-conflicting samples.

The fundamental limitation of these alternate-network approaches is that they require bias-conflicting examples — training instances where the spurious feature is not correlated with the label — in order to identify and upweight them. If the training data contains zero conflicting examples (e.g., all climbing images contain rocks, all blond-haired people are women), the biased network's errors on the training set provide no signal about what the bias is, since every training sample is consistent with the biased feature. The paper's Table 1 quantifies this: BAR (ID val), NICO, and ImageNet-9 all have 0% bias-conflicting examples in the training set. In such settings, methods that rely on reweighting conflicting training samples are fundamentally limited.

Ensemble methods (ESB, LWBC): Teney et al. (2022) proposed Evading the Simplicity Bias (ESB), which trains a diverse ensemble of classifiers with a gradient diversity regularizer, hoping that different ensemble members latch onto different features and the ensemble as a whole captures richer representations. LWBC also uses a committee to identify biased samples. While ensemble methods can be effective, they are computationally expensive — training multiple complete models multiplies the training cost — and they do not directly intervene on the simplicity bias mechanism. They create diversity at the output level but do not actively suppress the propagation of simple features through the network. The paper contrasts SiFER's targeted "surgical" intervention at specific network depths against the ensemble-based approach of generating diversity across independently trained models.

Architecture design approaches (NAS-OoD, OccamNet): Bai et al. (2021b) proposed NAS-OoD, which incorporates an out-of-distribution generalization metric into neural architecture search, searching over network architectures to find ones inherently more robust to spurious correlations. OccamNet (Shrestha et al., 2022) adds inductive biases such as spatial localization assumptions and bounded depth to filter spurious features. The limitation here is that these methods require architecture changes or search procedures that are tied to specific vision tasks — spatial localization assumptions about unbiased features, for instance, may not transfer to non-vision domains. They also represent "one-shot" choices rather than an adaptive training-time intervention that could be plugged into any standard architecture.

Multiple environment approaches (IRM, REx, EIIL): Arjovsky et al. (2019) proposed Invariant Risk Minimization (IRM), which uses causal Bayesian network theory to learn feature representations that are invariant across multiple training environments with different bias correlations. REx (Krueger et al., 2021) extends this to optimize for the worst-case linear combination of risks across environments. EIIL (Creager et al., 2021) attempts to infer environment assignments automatically.

The critical practical limitation is that these methods require multiple training environments — essentially, multiple datasets or deliberately constructed data partitions where the relationship between spurious features and labels varies. In many real-world settings, only a single training dataset is available, and constructing artificial environments requires domain knowledge of what the biases might be, which circles back to the attribute-label problem.

Adversarial augmentation approaches (Roadblock, DecAug): Roadblock (Niu et al., 2022) applies adversarial augmentations to images during training to prevent the network from over-relying on spurious visual cues. DecAug (Bai et al., 2021a) uses semantic augmentation and feature decomposition to disentangle context features from category features. The paper identifies Roadblock as conceptually closest to SiFER, noting that both create a competitive dynamic where previous model components challenge a new classifier. However, the paper argues that Roadblock operates at the output/prediction level (building an ensemble where previous components compete with a new classifier) while SiFER directly addresses the representational level — "the competitive development of features within a network (the 'heart' of the simplicity bias challenge)."

The Fundamental Gap: No Direct, Intra-Network Intervention

Synthesizing these limitations, the paper identifies a clear gap in the literature: no prior method directly intervenes inside the network to identify and suppress simple features where they first emerge (early layers) in order to create representational space for more complex features to develop in later layers. All prior approaches operate either:

At the data level (resampling, reweighting, augmentation) — manipulating what the network sees rather than how it processes.
At the output level (ensemble diversity, worst-case optimization) — manipulating what the network produces rather than its internal representations.
At the architecture level (architecture search, inductive bias) — making one-time design choices rather than adaptive, training-time interventions.

None of these approaches directly target the mechanism of simplicity bias as described by Shah et al. (2020) and Pezeshki et al. (2021): simple features are learned early, become available in lower network layers, and then proliferate throughout deeper layers, dominating the gradient signal and preventing complex features from being learned. SiFER proposes to break this mechanism at its source.

How SiFER Positions Itself

The paper frames SiFER as a direct, interventional method that operationalizes the mechanistic understanding of simplicity bias without requiring any a priori knowledge of what the simple or spurious features are. Several key design principles define this positioning:

Principle 1: Intervene at the representational level, not the data or output level. Rather than modifying training data (data augmentation, resampling) or combining predictions (ensembles), SiFER directly modifies the features available in intermediate network layers. The auxiliary network attached at some depth identifies what predictive features are computable at that depth, and the forgetting loss erases them from earlier layers. This is a surgical intervention targeting the specific mechanism by which simplicity bias propagates — early features crowding out later ones.

Principle 2: No prior knowledge of spurious features or their complexity class. The paper explicitly rejects any dependence on hand-specified definitions of what constitutes a "simple" feature. Instead, simple features are defined operationally: whatever is computable by an auxiliary network of limited capacity attached at a shallow depth that achieves low generalization error. The paper states this commitment clearly in Section 3.1:

"do not leverage any a priori information of simple features, or even the function class / degree of complexity of simple features."

This is a crucial differentiator from methods like OccamNet, which hardcodes assumptions about what unbiased features look like (e.g., spatially localized), or bias-label-dependent methods, which assume the spurious attribute is known and named.

Principle 3: Control through generalization error, not bias heuristics. The hyperparameters controlling the feature sieve — auxiliary network depth and position, forgetting weight, interleaving frequency — are tuned by minimizing validation error. This ensures that the sieve does not blindly suppress all "simple" features (some of which may be genuinely useful and unbiased) but rather suppresses features to the extent that doing so improves generalization. The paper notes (Section 3.1):

"Depending on the dataset, a 'simple' feature may in fact be the best / most unbiased predictive feature. For this reason, too, depending upon generalization error for controlling the feature sieve is strongly preferred to the use of any stronger inductive bias."

Principle 4: The feature sieve is controllable and general. Section 3.3 emphasizes that SiFER offers "many mechanisms to control this discovery & tradeoff" between simple and complex features via hyperparameters (auxiliary depth, position, forgetting weight, interleaving frequency). This controllability means the same core method can be adapted to different datasets and bias structures simply through hyperparameter selection guided by validation error, without changing the algorithm itself. Section 5.2 demonstrates this controllability directly by showing that SiFER can be steered to focus on either simple or complex features depending on which feature is made predictive in the validation set.

Principle 5: No dependence on bias-conflicting examples. Unlike LfF, LWBC, and similar alternate-network approaches that require at least some training examples where the spurious correlation breaks, SiFER's core mechanism (identifying features via the auxiliary network's predictions, then erasing them via the forgetting loss) operates entirely on the training distribution itself. The paper demonstrates this explicitly in the BAR ID-val setting (Table 3, bottom half), where the training data has 0% conflicting examples and the validation set provides no OOD signal — yet SiFER achieves 65.75% accuracy compared to 35.32% for ERM and outperforms all baselines that also lack OOD validation.

In positioning itself against the landscape of prior work, the paper does not claim that prior approaches are wrong — rather, it argues they address the problem at the wrong level. The feature sieve represents a shift from indirect methods (manipulating data to discourage simplicity bias) to a direct method (manipulating representations to prevent it). This is analogous to the difference between curriculum learning (which orders training examples to guide learning) and architectural constraints (which shape what can be learned). SiFER is a dynamic, training-time architectural constraint — a computational barrier that the network must overcome, forcing it to develop more sophisticated representations because the easy ones are actively erased.

The paper also positions its contribution as opening a new research direction rather than solving the problem definitively. The concluding section (Section 6) explicitly frames SiFER as "a small, important first step" and calls for "follow-up work [to] build on the notion of the feature sieve, developing effective computational barriers that encourage deep networks to discover and utilize richer, more powerful featural representations." This honest acknowledgment of scope — combined with the demonstrated gains across five real-world benchmarks — establishes SiFER as a proof-of-concept for a new approach to debiasing rather than a final solution.

3. Technical Approach

3.1 Reader Orientation

SiFER is a training-time intervention that modifies how a deep neural network learns internal representations: during training, an auxiliary network attached at some intermediate depth alternately identifies which features the main network is using to make predictions and then actively erases those features from earlier layers, forcing deeper layers to develop more sophisticated representations from whatever residual information survives the sieve. The problem this solves is simplicity bias—the tendency of DNNs to converge on the easiest computable features and ignore more complex ones—and the "shape" of the solution is a dynamic, adversarial competition within a single network: one component tries to predict labels from whatever features exist at a given depth, while another component deletes those very features, creating a computational barrier that only more complex, higher-layer features can overcome.

3.2 Big-Picture Architecture (Diagram in Words)

The SiFER system has four interacting components, all operating on a standard deep convolutional network (ResNet-18 in all experiments):

The primary network ( $\mathcal{M}$ ) — a standard ResNet-18 with ImageNet-pretrained weights, split conceptually into two regions: the early layers ( $\mathcal{M}_d$ , layers before the auxiliary attachment point) and the later layers (everything after). The early layers are where the feature sieve operates; the later layers are where richer representations should develop once simple features are suppressed.
The auxiliary network ( $\mathcal{A}$ ) — a small classification head (using the same BasicBlock structure as ResNet-18, with variable depth ranging from 1 to 9 layers as a hyperparameter) attached at one of three possible positions (after layer 1, 2, or 3 of ResNet-18, but never after layer 4). Its job is to identify what predictive features exist at that depth by trying to classify inputs from the intermediate representation.
The forgetting loss mechanism — when activated, freezes the auxiliary network's parameters, keeps only the early main-network layers trainable, and applies a cross-entropy loss that pushes the auxiliary network's predictions toward a uniform distribution. This forces the early layers to erase the features the auxiliary network was using, since uniform predictions mean the representation contains no label-discriminative information accessible to $\mathcal{A}$ .
The interleaving scheduler — controlled by hyperparameter $\mathcal{F}$ (forget_after_iters), which determines how many standard training iterations occur between forgetting steps. This creates an alternating dynamic: standard training (where both the main classifier and auxiliary network learn to use available features) interleaved with forgetting steps (where features used by the auxiliary network are deliberately suppressed).

Information flows as follows: an input image $\mathbf{x}$ enters the primary network → forward propagation produces representations at each layer → at the auxiliary attachment point, the auxiliary network $\mathcal{A}$ makes its own label prediction $\hat{\mathbf{y}}_{aux}$ from those intermediate features (without backpropagating to the main network during the identify phase) → during standard iterations, both the main classifier's loss $\mathcal{L}_1$ (cross-entropy on final predictions) and the auxiliary loss $\mathcal{L}_2$ (cross-entropy on auxiliary predictions) are backpropagated, but $\mathcal{L}_2$ only updates the auxiliary network, not the main network → every $\mathcal{F}$ iterations, the forgetting loss $\mathcal{L}_f$ is added, which freezes $\mathcal{A}$ , restricts trainability to $\mathcal{M}_d$ , and pushes auxiliary predictions toward uniformity, erasing the identified features.

3.3 Roadmap for the Deep Dive

First, the precise training algorithm (Algorithm 1 from the paper), including every loss term, every gradient flow decision, and the exact interleaving logic — because the dynamic between identification and erasure is the heart of the method.
Second, the auxiliary network design — its architecture, attachment point logic, capacity control, and the critical design decision to not backpropagate its loss to the main network during identification — because this is what forces it to work with already-available features rather than creating new ones.
Third, the forgetting loss mechanism — the pseudo-label construction using uniform probabilities, the cross-entropy formulation, the freezing of $\mathcal{A}$ and restriction of trainability to $\mathcal{M}_d$ , and why this constitutes "erasure" rather than just noise injection — because this is the direct interventional mechanism.
Fourth, the interleaving dynamics — why alternating identification and erasure is necessary rather than simultaneous optimization, how $\mathcal{F}$ controls the frequency, and what happens if this hyperparameter is set poorly — because the temporal structure of the training procedure is what makes the adversarial competition stable.
Fifth, the controllability mechanisms — how auxiliary depth ( $\mathcal{A}_D$ ), auxiliary position ( $\mathcal{A}_P$ ), forgetting weight ( $\alpha_3$ ), interleaving frequency ( $\mathcal{F}$ ), and auxiliary loss weight ( $\alpha_2$ ) collectively determine which features get sieved and how aggressively, and why all of these are tuned via validation error rather than hand-coded heuristics — because this is what enables SiFER to work without prior knowledge of spurious features.
Sixth, the validation-guided hyperparameter selection protocol — the search space for each hyperparameter (Table 7), the validation metric used for each dataset (Table 1), and why tuning by generalization error ensures SiFER suppresses features only when doing so genuinely improves robustness — because this closes the loop on the "no prior knowledge" design goal.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methods paper whose core idea is that simplicity bias can be mitigated by dynamically identifying and erasing easily-computed features at early network layers, creating a computational sieve that forces deeper layers to develop richer representations, with all control parameters tuned automatically via validation error rather than requiring prior knowledge of which features are spurious.

The Alternating Identify-and-Erase Algorithm

Algorithm 1 in the paper defines the complete training loop for SiFER. The algorithm interleaves two modes of operation — standard supervised training and targeted feature erasure — controlled by the hyperparameter $\mathcal{F}$ (forget_after_iters), which specifies how many standard iterations occur between forgetting steps.

Standard training iterations (the "identify" phase): For every mini-batch $(\mathbf{x}, \mathbf{y})$ sampled from the training data $\mathcal{D}$ , the algorithm performs a forward pass through the main network with the auxiliary network attached at position $\mathcal{A}_P$ and depth $\mathcal{A}_D$ . This produces two predictions:

$\hat{\mathbf{y}}$ — the main classifier's final prediction from the last layer.
$\hat{\mathbf{y}}_{aux}$ — the auxiliary network's prediction from the intermediate representation at $\mathcal{A}_P$ .

Two cross-entropy losses are computed:

$\mathcal{L}_1 = \text{CE}(\hat{\mathbf{y}}, \mathbf{y})$ — the standard supervised loss on the main classifier.
$\mathcal{L}_2 = \text{CE}(\hat{\mathbf{y}}_{aux}, \mathbf{y})$ — the auxiliary network's supervised loss, measuring how well labels can be predicted from the intermediate representation.

The combined loss for standard iterations is:

$\mathcal{L} = \alpha_1 \mathcal{L}_1 + \alpha_2 \mathcal{L}_2$

where $\alpha_1$ is the main loss weight (fixed to 10 in all experiments to reduce hyperparameter search space) and $\alpha_2$ is the auxiliary loss weight (tuned from a log-uniform distribution over $[10^{-1}, 10^2]$ ).

What this computes: The total loss for standard iterations is a weighted sum of two independent cross-entropy terms. $\alpha_1 \mathcal{L}_1$ drives the main network to correctly classify inputs using whatever features are available throughout the entire network. $\alpha_2 \mathcal{L}_2$ drives the auxiliary network — attached at an intermediate depth — to correctly classify inputs using only the features available at that shallower depth.

Why this form: The two losses serve fundamentally different purposes. $\mathcal{L}_1$ is the primary task — we want the network to classify correctly. $\mathcal{L}_2$ is not about improving auxiliary accuracy per se; it is about identifying what features exist at the auxiliary attachment depth. By training the auxiliary network to maximize its own accuracy, we force it to discover and exploit whatever predictive features are computationally accessible at that depth — which, by the paper's core assumption (Section 3.1), will include simple, spurious features. The weight $\alpha_2$ controls how aggressively the auxiliary network is optimized relative to the main task. A critical design detail is that the gradient from $\mathcal{L}_2$ only flows to the auxiliary network's parameters, not to the main network. The main network parameters $\mathbf{W}$ are updated using gradients from $\mathcal{L}_1$ during standard iterations. This is explicitly stated in the paper as a "conscious decision choice to force the auxiliary layer to learn from already-available features rather than create or reinforce them in the main network." If $\mathcal{L}_2$ gradients were allowed to flow into the main network, the auxiliary network could shape the intermediate representations to make its own job easier — creating a self-reinforcing loop where early layers develop features specifically for the auxiliary network, defeating the purpose of identifying what naturally emerges at that depth.

Forgetting iterations (the "erase" phase): Every $\mathcal{F}$ iterations, an additional step is inserted. The auxiliary network's parameters are frozen, and only the main network layers before the auxiliary attachment point (denoted $\mathcal{M}_d$ ) are made trainable — later layers are frozen. A forgetting loss $\mathcal{L}_f$ is computed:

$\hat{\mathbf{y}}_{aux} = \mathcal{A}(\mathcal{M}_d(\mathbf{x}))$

$\mathbf{y}_{ep} = \left[ \frac{1}{n}, \frac{1}{n}, \dots \right] \text{ (} n \text{ entries)}$

$\mathcal{L}_f = \text{CE}(\hat{\mathbf{y}}_{aux}, \mathbf{y}_{ep})$

where $\mathbf{x}$ is the input image, $\mathcal{M}_d(\mathbf{x})$ is the intermediate representation at the auxiliary attachment point (produced by only the early layers of the main network), $\hat{\mathbf{y}}_{aux}$ is the frozen auxiliary network's prediction, $\mathbf{y}_{ep}$ is a vector of length $n$ (the number of classes) with every entry equal to $1/n$ , and $\text{CE}$ is the standard categorical cross-entropy.

What this computes: The forgetting loss is a cross-entropy between the auxiliary network's predicted class distribution and a uniform distribution over all classes. Minimizing this loss forces the auxiliary network to output equal probability to every class — i.e., to become maximally uncertain about the label. Since the auxiliary network's parameters are frozen and only the early main-network layers are updated, this forces $\mathcal{M}_d$ to transform its input such that the features the auxiliary network previously relied on for prediction are no longer discriminative.

Why this form — the pseudo-label choice: The uniform pseudo-label $\mathbf{y}_{ep}$ is not arbitrary; it encodes the goal of complete feature erasure. If instead we minimized cross-entropy against a random class label, the gradient would push the network to produce confidently wrong predictions, which could be achieved by simply inverting the feature (e.g., making the representation of "blond" look like "not-blond"). A clever auxiliary network re-attached later could potentially recover the original feature by inverting again. By targeting the uniform distribution, we force the network to make the representation genuinely non-discriminative — there is no linear readout that can extract the label from it, because for any class, the predicted probability is the same $1/n$ . This is information-theoretic erasure: the mutual information between the intermediate representation and the label is driven toward zero for any classifier with the auxiliary network's architecture. The choice of $1/n$ rather than an arbitrary distribution with non-zero entropy ensures symmetry across classes — the representation is equally uninformative for all labels.

Why this form — the freezing and restricted trainability: Freezing $\mathcal{A}$ is essential because if the auxiliary network could adapt during forgetting, it could simply learn to ignore the erased features and find new ones — turning the forgetting step into an adversarial game where features are continuously shifted but never eliminated. By freezing $\mathcal{A}$ , we fix the "definition" of what constitutes the simple feature (it is whatever $\mathcal{A}$ was using at the moment it was frozen), and the erasure targets that specific feature. Restricting trainability to $\mathcal{M}_d$ (layers before the attachment point) ensures that the erasure happens at the source — the early layers where simple features are first computed — rather than being compensated for by later layers that could simply re-extract the feature from a differently-coded but informationally-equivalent representation.

The interleaving dynamic: The total loss during a forgetting iteration is:

$\mathcal{L} = \alpha_1 \mathcal{L}_1 + \alpha_2 \mathcal{L}_2 + \alpha_3 \mathcal{L}_f$

where $\alpha_3$ is the forgetting weight, tuned from a log-uniform distribution over $[10^{-1}, 10^2]$ . After backpropagating this combined loss and updating parameters (only $\mathcal{M}_d$ for the forgetting component), the algorithm resumes standard iterations for another $\mathcal{F}$ steps, during which the auxiliary network re-adapts to whatever features remain or have newly emerged. The cycle then repeats for $N$ total training iterations.

Why interleaving is necessary rather than simultaneous optimization: If the forgetting loss were applied at every iteration simultaneously with the auxiliary training loss, the system would receive contradictory gradient signals: $\mathcal{L}_2$ pulls the early layers to produce features that help $\mathcal{A}$ predict the label, while $\mathcal{L}_f$ pulls to destroy those same features. The result would be either a stalemate (gradients cancel) or a noisy equilibrium where features oscillate. By interleaving, the auxiliary network is given $\mathcal{F}$ iterations to "catch up" to the current representation — identifying what features are now available after the previous forgetting step — before those features are erased again. This turns a conflicting simultaneous optimization into a cooperative sequential process: the auxiliary network reveals what features exist, and the forgetting step removes them, after which the auxiliary network can discover the next set of available features (which, ideally, are the more complex ones that remained because they were not yet computable at that depth).

The hyperparameter $\mathcal{F}$ controls the timescale of this dynamic. If $\mathcal{F}$ is too small (e.g., 1), the auxiliary network never has time to fully identify features before they are erased, leading to unstable training. If $\mathcal{F}$ is too large (e.g., the entire training duration), features become deeply entrenched before any erasure occurs, and the forgetting step may not be strong enough to dislodge them. The paper sweeps $\mathcal{F}$ in multiples of 10 from $[1, 9] \times 10$ (i.e., 10, 20, ..., 90 iterations between forgetting steps, as shown in Table 7).

Auxiliary Network Design

The auxiliary network $\mathcal{A}$ is a small classification module attached at an intermediate layer of the main ResNet-18. Its design is governed by two hyperparameters — auxiliary depth $\mathcal{A}_D$ (number of layers in the auxiliary network) and auxiliary position $\mathcal{A}_P$ (which ResNet layer block it attaches to) — and one critical gradient-flow constraint.

Architecture: The auxiliary network uses the same BasicBlock structure as ResNet-18, meaning each "layer" in $\mathcal{A}$ consists of two convolutional layers with batch normalization and a residual connection, identical in form to the building blocks of the main network. The depth $\mathcal{A}_D$ ranges from 1 to 9 (Table 7), controlling the capacity of the auxiliary network to compute features from the intermediate representation it receives.

Why this architecture: Using the same basic building block as the main network ensures architectural compatibility and makes the auxiliary network's capacity interpretable in terms familiar from the ResNet literature. A depth-1 auxiliary network is essentially a linear classifier on top of the intermediate representation (with a single residual block providing minor non-linearity), while a depth-9 auxiliary network is a substantial sub-network capable of computing moderately complex features from whatever representation it receives. By sweeping this depth, the paper controls the complexity of features that the auxiliary network can identify: shallow auxiliary networks can only detect features that require minimal additional computation beyond what the main network's early layers already provide, while deeper auxiliary networks can compose these into more sophisticated patterns.

Attachment position: ResNet-18 has four layer blocks (each with two BasicBlocks). The auxiliary network can be attached at the output of layer 1, layer 2, or layer 3 — but never at layer 4 (the final layer), because the goal is to sieve features from earlier representations where simple features are expected to dominate. The position $\mathcal{A}_P \in \{1, 2, 3\}$ (Table 7) controls the depth in the main network at which features are being identified and subsequently erased.

Why position matters: The paper's core assumption (Section 3.1) is that simple features are computable earlier in the network. Attaching at layer 1 (very early, just after the first few convolutions) means the auxiliary network operates on low-level features like edges, textures, and color blobs — the kinds of features that drive texture bias and color bias. Attaching at layer 3 (much deeper, after significant hierarchical processing) means the auxiliary network operates on mid-to-high-level features that may already encode object parts or semantic attributes. By sweeping $\mathcal{A}_P$ , the paper can control what class of features gets sieved: shallow attachment sieves the most basic, easily-computed features; deeper attachment sieves features that require more processing but may still be spurious (e.g., background context that becomes recognizable at mid-level). The optimal position is dataset-dependent and determined by validation error.

The no-backpropagation constraint: During standard training iterations, the gradient from $\mathcal{L}_2$ (auxiliary loss) is not backpropagated into the main network. This is explicitly motivated in the paper (Section 3.2):

"Note that feedback from the auxiliary layer does not back-propagate to the main network. This is a conscious decision choice to force the auxiliary layer to learn from already-available features rather than create or reinforce them in the main network."

In implementation terms, this means that when computing $\nabla \mathbf{W}$ from $\mathcal{L} = \alpha_1 \mathcal{L}_1 + \alpha_2 \mathcal{L}_2$ , the $\alpha_2 \mathcal{L}_2$ term contributes gradients only to the parameters of $\mathcal{A}$ (which are part of $\mathbf{W}$ but form a separate subset), while the main network parameters (excluding $\mathcal{A}$ ) receive gradients only from $\alpha_1 \mathcal{L}_1$ . During forgetting iterations, the reverse happens: $\mathcal{A}$ is frozen, $\mathcal{L}_f$ propagates gradients only into $\mathcal{M}_d$ (the main network layers before $\mathcal{A}_P$ ), and $\mathcal{L}_1$ and $\mathcal{L}_2$ continue to provide gradients to their respective parameter subsets.

Why this constraint matters: If $\mathcal{L}_2$ gradients could flow into the main network, the auxiliary network would not merely identify features — it would create them. The main network would be trained to produce representations that are easy for $\mathcal{A}$ to classify, which would amplify exactly the simple features we want to sieve. The auxiliary network would become a self-fulfilling prophecy: it expects simple features, so it trains the network to produce them, then it confirms they exist. By blocking this feedback, $\mathcal{A}$ is forced to work with whatever features naturally emerge from standard supervised training ( $\mathcal{L}_1$ alone), making it an honest detector rather than a sculptor of early representations.

The Forgetting Loss: Mechanism and Semantics

The forgetting loss $\mathcal{L}_f = \text{CE}(\hat{\mathbf{y}}_{aux}, \mathbf{y}_{ep})$ where $\mathbf{y}_{ep} = [1/n, ..., 1/n]$ is the core intervention that distinguishes SiFER from all prior debiasing methods. To understand what this loss actually does to the network, we must examine the gradient signal it produces and the representational change it induces.

The gradient of cross-entropy with respect to the pre-softmax logits: Let $\mathbf{z} = \mathcal{A}(\mathcal{M}_d(\mathbf{x}))$ be the vector of logits output by the frozen auxiliary network, and let $\hat{\mathbf{y}}_{aux} = \text{softmax}(\mathbf{z})$ be the predicted probabilities. The cross-entropy with uniform target $\mathbf{y}_{ep}$ is:

$\mathcal{L}_f = -\sum_{i=1}^n \frac{1}{n} \log(\hat{y}_i)$

where $\hat{y}_i$ is the predicted probability for class $i$ . The gradient of this loss with respect to the logit $z_i$ is:

$\frac{\partial \mathcal{L}_f}{\partial z_i} = \hat{y}_i - \frac{1}{n}$

What this gradient means operationally: For each class $i$ , the gradient pushes the logit $z_i$ in the direction opposite to the deviation of $\hat{y}_i$ from $1/n$ . If the auxiliary network currently predicts class $i$ with high probability ( $\hat{y}_i \gg 1/n$ ), the gradient on $z_i$ is positive, meaning gradient descent will increase $z_i$ — but wait, that would make the prediction more confident, which is the opposite of what we want. This apparent paradox is resolved by remembering that $\mathcal{L}_f$ is minimized, so we take a step in the negative gradient direction: $z_i \leftarrow z_i - \eta (\hat{y}_i - 1/n)$ . If $\hat{y}_i > 1/n$ , the update is $z_i \leftarrow z_i - \eta(\text{positive})$, which *decreases* $z_i$ , pulling the predicted probability toward $1/n$ . If $\hat{y}_i < 1/n$ , the update increases $z_i$ `, again pulling toward uniformity. The net effect is to drive all logits toward equality, making the softmax output uniform.

Why the gradient propagates to $\mathcal{M}_d$ : The chain rule extends this gradient back through the frozen $\mathcal{A}$ into the early main-network layers $\mathcal{M}_d$ . For a weight $w$ in $\mathcal{M}_d$ :

$\frac{\partial \mathcal{L}_f}{\partial w} = \sum_{i=1}^n \frac{\partial \mathcal{L}_f}{\partial z_i} \cdot \frac{\partial z_i}{\partial w}$

The term $\partial z_i / \partial w$ encodes how changes to $w$ affect the auxiliary network's logit for class $i$ , which depends on the frozen parameters of $\mathcal{A}$ . Since $\mathcal{A}$ was trained (during the preceding standard iterations) to extract label-discriminative features from $\mathcal{M}_d(\mathbf{x})$ , these partial derivatives are precisely the directions in representation space that $\mathcal{A}$ uses to distinguish classes. The forgetting gradient thus pushes $\mathcal{M}_d$ to actively suppress those directions — to transform its output so that the specific feature patterns $\mathcal{A}$ relies on are no longer present or are counterbalanced.

Why this constitutes "erasure" rather than obfuscation: A natural question is whether the forgetting loss merely teaches $\mathcal{M}_d$ to produce a confusing representation that fools $\mathcal{A}$ specifically, while leaving the label information intact in a slightly different coding that a different classifier could recover. The paper argues this is not the case for two reasons. First, the auxiliary network's architecture (BasicBlock) is representative of the kinds of computations that subsequent layers of the main network would perform — if the representation becomes unreadable to $\mathcal{A}$ , it is likely unreadable to the next ResNet block as well, since they share similar inductive biases. Second, and more subtly, the forgetting step operates in alternation with standard training: after features are erased, the main classifier $\mathcal{L}_1$ continues to demand accurate predictions, which forces later layers (those after $\mathcal{A}_P$ ) to extract whatever residual information remains. The early layers, subject to recurring forgetting pressure, cannot simply re-encode the same information in a rotated basis because the auxiliary network will re-learn to read that basis during the next identification phase and trigger another round of erasure. Over time, the early layers are shaped to genuinely lack label-discriminative information accessible to shallow computations.

Controllability Mechanisms: The Five Hyperparameters

Section 3.3 of the paper identifies four primary hyperparameters that control the feature sieve's behavior, plus the fixed main loss weight $\alpha_1 = 10$ . Each provides a distinct control dimension, and together they determine which features get sieved, how aggressively, and at what temporal granularity.

Auxiliary position $\mathcal{A}_P \in \{1, 2, 3\}$ : This controls the depth in the main network at which features are identified and erased. Position 1 attaches after the first ResNet layer block, operating on features roughly equivalent to low-level image statistics (edges, textures, colors). Position 2 attaches after the second block, operating on mid-level features. Position 3 attaches after the third block, operating on relatively high-level features that may encode object parts or semantic attributes.

Why position controls feature complexity: Deeper positions provide the auxiliary network with representations that have undergone more hierarchical processing. The representations at position 3 are the product of ~12 convolutional layers of processing; the features they encode are more abstract and compositional than those at position 1 (~4 layers). By choosing $\mathcal{A}_P$ , we select which stage of the feature hierarchy to target. For datasets where the spurious feature is genuinely low-level (e.g., color in CMNIST, texture in ImageNet-9), a shallow position may be optimal — erase the color channel before shape processing even begins. For datasets where the spurious feature is more semantic (e.g., background context in BAR, gender in CelebA), a deeper position may be needed because the feature only becomes discriminatively decodable at mid-to-high levels.

Auxiliary depth $\mathcal{A}_D \in [1, 9]$ : This controls the capacity of the auxiliary network, i.e., how much additional computation it can perform on top of the intermediate representation to extract predictive features. A depth-1 auxiliary network is essentially a linear probe with minor non-linearity; it can only identify features that are already linearly separable in the intermediate representation. A depth-9 auxiliary network can compose features through multiple non-linear transformations, potentially discovering more subtle or distributed patterns.

Why depth matters for the sieve: The auxiliary depth sets the threshold of feature complexity that gets detected and erased. A shallow auxiliary network only detects the most blatantly discriminative features — the ones that require minimal computation beyond what the main network already provides. These are precisely the "simple" features that Shah et al. (2020) showed dominate neural network learning. A deeper auxiliary network can detect more sophisticated features, potentially including genuinely useful ones. Tuning $\mathcal{A}_D$ via validation error ensures we find the sweet spot: deep enough to catch the spurious features but not so deep that we erase everything and destroy in-distribution accuracy. The paper's results (Table 8) show that the tuned $\mathcal{A}_D$ varies substantially across datasets: 4 for BAR-ID and ImageNet-9 (suggesting moderately complex spurious features), 2 for BAR-OOD, CelebA, and NICO (suggesting simpler spurious features are the main problem).

Forgetting weight $\alpha_3 \in [10^{-1}, 10^2]$ (log-uniform): This controls the strength of the erasure gradient relative to the standard training gradient. When $\alpha_3$ is small (e.g., 0.1), the forgetting step makes only a gentle correction — features are partially suppressed but may persist in attenuated form. When $\alpha_3$ is large (e.g., 100), the forgetting step aggressively overwrites the early-layer features, potentially causing the main classifier to lose accuracy on the training set before later layers can compensate.

Why weight matters: The forgetting weight must be balanced against $\alpha_1$ (main loss weight, fixed at 10) and the natural learning dynamics. If $\alpha_3$ is too low, the forgetting gradient is drowned out by the main loss gradient, and simple features survive the sieve. If $\alpha_3$ is too high, the forgetting step destabilizes training — early layers are violently reconfigured, and the main classifier's performance collapses because later layers receive representations that change faster than they can adapt. The paper uses a log-uniform search (Table 7) because the effective dynamic range spans orders of magnitude, and the optimal value depends on dataset-specific factors like the relative strength of spurious vs. genuine features and the network's learning rate.

Auxiliary loss weight $\alpha_2 \in [10^{-1}, 10^2]$ (log-uniform): This controls how strongly the auxiliary network is trained to predict labels during standard iterations. A small $\alpha_2$ means the auxiliary network is weakly optimized and may not fully identify the available features, making the subsequent forgetting step less targeted. A large $\alpha_2$ means the auxiliary network is aggressively trained, potentially overfitting to noise or finding features that are only weakly predictive.

Why this weight matters: $\alpha_2$ determines the quality of feature identification that precedes each forgetting step. If the auxiliary network is poorly trained, the forgetting step erases a noisy, partially-random set of features, which may not correspond to the actual spurious features driving the bias. If the auxiliary network is over-trained, it may identify features that are genuinely useful and should not be erased. Tuning $\alpha_2$ via validation error ensures the auxiliary network identifies features at the right level of granularity — discriminative enough to be the ones the network is actually using, but not so fine-grained that the sieve becomes destructive.

Interleaving frequency $\mathcal{F} \in [10, 90]$ (multiples of 10): This controls the temporal granularity of the alternating dynamic. At $\mathcal{F} = 10$ , forgetting steps occur very frequently (every 10 iterations), giving the auxiliary network little time to adapt between erasures. At $\mathcal{F} = 90$ , forgetting steps are infrequent, allowing features to become deeply entrenched before each erasure attempt.

Why frequency matters: The interleaving frequency determines the timescale separation between feature identification (auxiliary learning) and feature erasure (forgetting). A small $\mathcal{F}$ creates a rapid back-and-forth where features are barely learned before being erased — this may prevent any stable features from forming in early layers, forcing the network to route all discriminative computation through later layers. A large $\mathcal{F}$ creates a slower dynamic where features are well-established, then suddenly erased, forcing a more dramatic reorganization. The optimal $\mathcal{F}$ depends on the learning dynamics of the specific dataset: how quickly simple features are learned (which depends on their signal strength relative to noise) and how quickly later layers can adapt to fill the gap left by erased features.

The validation-error tuning loop: All five hyperparameters ( $\mathcal{A}_D$ , $\mathcal{A}_P$ , $\alpha_2$ , $\alpha_3$ , $\mathcal{F}$ ) are tuned jointly by searching the space defined in Table 7 and selecting the configuration that minimizes the validation metric appropriate for each dataset (Table 1). The search space includes the trivial solution of setting $\alpha_3 = 0$ (or, equivalently, making $\mathcal{F}$ larger than the total number of training iterations), which would reduce SiFER to standard ERM training — this is included in the search space by virtue of $\mathcal{F}$ and $\alpha_3$ both having ranges that can effectively disable the forgetting mechanism. If no configuration of the sieve outperforms ERM, the validation error will select the ERM-like configuration, ensuring that SiFER never hurts relative to standard training. This is the paper's answer to the concern that blindly suppressing features could degrade performance: the validation loop acts as a safety mechanism.

Validation-Guided Hyperparameter Selection Protocol

The paper emphasizes that SiFER's hyperparameters are set entirely by minimizing validation error, with no manual tuning based on knowledge of which features are spurious. This is both a practical necessity (real-world practitioners lack such knowledge) and a conceptual commitment (the sieve should discover what to suppress based on generalization, not human priors).

Validation metric per dataset (Table 1):

BAR (both ID and OOD val settings): Accuracy — standard average accuracy on the validation set. For BAR-OOD, the validation set is drawn from the test distribution (20% of test data), containing bias-conflicting examples; for BAR-ID, the validation set is drawn from the training distribution, containing zero conflicting examples.
CelebA: Unbiased Accuracy — accuracy averaged over each label-context group separately and then averaged, preventing the majority group from dominating the metric. This is critical because CelebA has extreme group imbalance (fewer than 1% of blond-haired individuals are men), so standard accuracy would be dominated by the majority group and fail to detect gender bias.
NICO: Accuracy — standard average accuracy, since the validation set naturally contains unseen contexts that test for spurious correlation robustness.
ImageNet-9 / ImageNet-A: Accuracy on ImageNet-9 validation set — since ImageNet-A has no separate validation split, hyperparameters are tuned on ImageNet-9 validation accuracy, and the selected model is then evaluated on ImageNet-A as an OOD robustness test. This tests whether suppressing texture bias on in-distribution data transfers to improved robustness on naturally adversarial images.

The paper does not use the test set for hyperparameter selection in any experiment — all tuning uses only the designated validation data.

Search procedure: The paper uses a grid-like search over the discrete-continuous hybrid space defined in Table 7. $\mathcal{A}_D$ and $\mathcal{F}$ are discrete choices from small sets (1–9 and 10–90 in steps of 10, respectively). $\mathcal{A}_P$ is discrete with 3 choices. $\alpha_2$ and $\alpha_3$ are sampled from log-uniform distributions over $[10^{-1}, 10^2]$ , acknowledging that the optimal values could span orders of magnitude and that relative (rather than absolute) changes matter for loss weighting. The paper does not specify the exact search strategy (random search, grid search, Bayesian optimization) or the number of trials, which is a minor methodological omission.

Why validation-guided tuning is essential to the "no prior knowledge" claim: The paper's central methodological claim is that SiFER requires no foreknowledge of which features are spurious. But the hyperparameters $\mathcal{A}_P$ and $\mathcal{A}_D$ implicitly encode assumptions about feature complexity and depth. The validation tuning loop resolves this tension: instead of the practitioner guessing where simple features live in the network, the optimization process empirically determines which attachment point and capacity lead to the best generalization. If attaching at layer 1 and aggressively forgetting improves validation accuracy, that is evidence that low-level features were indeed the problem. If attaching at layer 3 works better, that suggests the spurious features were higher-level. The machine discovers this, not the human practitioner.

Hyperparameter values actually used (Table 8): The paper reports the tuned hyperparameters for each dataset, which provides insight into the nature of the biases in each:

BAR-ID val ( $\mathcal{A}_D = 4$ , $\mathcal{A}_P = 2$ , $\alpha_2 = 2$ , $\alpha_3 = 4.5$ , $\mathcal{F} = 70$ ): Moderate auxiliary depth and middle attachment position suggest that spurious background features are detected at mid-level representations. Relatively infrequent forgetting ( $\mathcal{F} = 70$ ) indicates that features take time to become established before erasure is effective.
BAR-OOD val ( $\mathcal{A}_D = 2$ , $\mathcal{A}_P = 2$ , $\alpha_2 = 1$ , $\alpha_3 = 3$ , $\mathcal{F} = 30$ ): Shallower auxiliary network but same position, with more frequent forgetting. The availability of OOD validation data allows more aggressive sieving.
CelebA ( $\mathcal{A}_D = 2$ , $\mathcal{A}_P = 2$ , $\alpha_2 = 25$ , $\alpha_3 = 15$ , $\mathcal{F} = 50$ ): Very high auxiliary loss weight ( $\alpha_2 = 25$ ) indicates that the auxiliary network needs strong optimization to reliably detect gender-correlated features in face representations. High forgetting weight ( $\alpha_3 = 15$ ) suggests aggressive erasure is needed.
NICO ( $\mathcal{A}_D = 2$ , $\mathcal{A}_P = 1$ , $\alpha_2 = 1$ , $\alpha_3 = 75$ , $\mathcal{F} = 70$ ): Shallow attachment ( $\mathcal{A}_P = 1$ ) with extremely high forgetting weight ( $\alpha_3 = 75$ ). This suggests the spurious features in NICO (context-background correlations) are very low-level and require aggressive early-layer erasure, since they are detectable almost immediately in the network.
ImageNet-9 / ImageNet-A ( $\mathcal{A}_D = 4$ , $\mathcal{A}_P = 3$ , $\alpha_2 = 1$ , $\alpha_3 = 4.5$ , $\mathcal{F} = 70$ ): Deep attachment ( $\mathcal{A}_P = 3$ ) suggests that texture bias in ImageNet is not purely a low-level phenomenon — texture features are computed and become discriminative at relatively deep layers of ResNet-18, where they can be identified and erased.

Key insight from the variation in tuned hyperparameters: There is no single "correct" configuration of the feature sieve — the optimal parameters are strongly dataset-dependent. This validates the paper's design choice to make the sieve controllable and to tune via validation error rather than hardcoding assumptions. A method that always erased features at layer 1 with depth 2 and $\alpha_3 = 1$ would work well on NICO but fail on ImageNet-9, where the spurious features manifest deeper in the network.

The Training Dynamics: What Happens Over the Course of Training

Understanding SiFER requires not just the static algorithm description but the dynamic trajectory of representations over the course of training. The paper provides evidence for this trajectory through the layerwise decodability experiments (Figures 3 and 5 for synthetic and real datasets, respectively), and the logic of the alternating optimization implies a specific sequence of representational changes.

Phase 1 — Standard ERM learning (early training): At the start of training, both the main classifier and the auxiliary network are randomly initialized (the main network from ImageNet pretraining, the auxiliary from scratch). Standard training iterations ( $\mathcal{L}_1 + \alpha_2 \mathcal{L}_2$ ) rapidly establish simple predictive features in early layers because these features are easy to compute and provide strong gradient signal. The auxiliary network, attached at some intermediate depth, quickly learns to exploit these features, achieving high accuracy on the auxiliary classification task. Meanwhile, the main classifier learns to use both simple features (from early layers) and whatever additional features are available in later layers — but because the early-layer simple features already provide good accuracy, the gradient pressure to develop complex features in later layers is weak. This is the standard simplicity bias dynamic described by Shah et al. (2020): the network has found a "good enough" solution using simple features and stops investing representational capacity in complex ones.

Phase 2 — First forgetting step (the intervention): When the first forgetting step triggers (after $\mathcal{F}$ iterations), the auxiliary network is frozen, and $\mathcal{L}_f$ backpropagates through it into $\mathcal{M}_d$ . The gradient actively suppresses the feature directions that $\mathcal{A}$ was using. This causes a temporary drop in both auxiliary accuracy (since the features it relies on are being erased) and main classifier accuracy (since the main classifier was also using those simple features, either directly in early layers or through later-layer computations that depended on early-layer features).

Phase 3 — Re-adaptation (post-forgetting standard training): After the forgetting step, training resumes normally for another $\mathcal{F}$ iterations. The main classifier $\mathcal{L}_1$ now faces a dilemma: the early-layer features it previously relied on have been degraded, so to maintain accuracy, it must extract discriminative information from elsewhere. This forces the later layers (after $\mathcal{A}_P$ ) to develop richer features — features that are computable from the residual information that survived the sieve. Simultaneously, the auxiliary network re-trains on the modified early-layer representations, discovering whatever predictive features remain or have re-emerged. Because the most blatant simple features were erased, the auxiliary network may now latch onto slightly more complex features that survived because they required more computation than the auxiliary network's shallow readout could fully exploit before.

Phase 4 — Repeated sieving (convergence): As the cycle repeats, the early layers are progressively stripped of label-discriminative features accessible to shallow readouts. Each cycle targets the features that the auxiliary network can most easily exploit at that stage. Over time, this drives a representational reorganization: early layers become increasingly "neutral" with respect to the label (containing general-purpose visual features like edges, textures, and shapes that are not individually sufficient for classification), while later layers accumulate the complex, composed features that actually drive correct predictions. The paper's Figure 3 shows exactly this dynamic for CIFAR_MNIST: under standard training, simple MNIST digit features become highly decodable at all layers (especially deeper ones), while complex CIFAR features remain weakly decodable and are suppressed at deeper layers. Under SiFER, simple digit features are actively suppressed in early-to-mid layers, and complex CIFAR features become increasingly decodable at deeper layers as the network is forced to rely on them.

Phase 5 — Equilibrium behavior: The system reaches a dynamic equilibrium where the auxiliary network can still extract some label information from the early-layer representations (since complete information erasure is likely impossible without destroying all useful visual processing), but this information is weak and distributed, not concentrated in easily-exploitable simple features. The forgetting step continues to apply pressure against any feature that becomes too discriminative, but the pressure is balanced by the main loss $\mathcal{L}_1$ which requires the early layers to retain enough visual information for later layers to build upon. The equilibrium point — how much early-layer discriminability is tolerated — is determined by the relative weights $\alpha_1$ , $\alpha_2$ , and $\alpha_3$ , and the interleaving frequency $\mathcal{F}$ .

Design Choice: Why Not Just Train Without the Auxiliary Network?

A natural question is whether the auxiliary network is necessary at all — could one simply add a regularization term that directly penalizes label-decodability from intermediate representations (e.g., by training a linear probe and minimizing its accuracy)? The paper's auxiliary network design embeds several non-obvious choices that are worth making explicit.

Why use a trained auxiliary network rather than a random projection: If the auxiliary network were randomly initialized and frozen, $\mathcal{L}_f$ would push early representations toward a random target, which might not correspond to erasing predictive features. The spurious features are precisely those that are useful for prediction on the training data; a random projection might not align with these features, and the forgetting gradient would erase irrelevant directions while leaving the actual spurious features intact. By training the auxiliary network to maximize its own classification accuracy before each forgetting step, we ensure that the features being erased are exactly those that are most discriminative at that depth — which, under the simplicity bias hypothesis, are the spurious ones.

Why freeze the auxiliary network during forgetting: If $\mathcal{A}$ remained trainable during the forgetting step, it could adapt to the changing representation in real-time, learning to extract label information from whatever new basis the early layers adopt. The forgetting step and the auxiliary network would enter an adversarial game where features are continuously shifted but never eliminated — similar to the "cat and mouse" dynamic in GAN training. By freezing $\mathcal{A}$ , we fix the definition of "the feature we're erasing" at the moment of freezing, ensuring the gradient targets that specific feature unambiguously. The auxiliary network can then re-adapt during subsequent standard training iterations, discovering what new features have become available.

Why not simply penalize mutual information directly: More sophisticated approaches (e.g., using a mutual information estimator or adversarial domain classifier) could theoretically achieve the same goal of reducing label information in early layers. The paper's approach using cross-entropy with a uniform target is notably simpler: it requires no additional networks beyond $\mathcal{A}$ (which is already needed for identification), uses the standard classification loss, and has a clear operational interpretation (make the auxiliary network maximally uncertain). This simplicity makes the method easy to implement and tune, and the uniform target has the information-theoretic property of minimizing mutual information for a given classifier family.

Why attach the auxiliary network at a specific depth rather than at every layer: Attaching at a single, tuned position provides a focused intervention. Attaching at every layer simultaneously and applying forgetting losses at all of them would likely be too aggressive — early layers would have no chance to develop any useful visual features, and the network might fail to learn entirely. The single attachment point, combined with the position hyperparameter, allows the practitioner to target the specific depth where spurious features become problematically dominant, which can be discovered empirically through validation-guided tuning. The experiments show that different datasets require different attachment points (Table 8: NICO at layer 1, CelebA at layer 2, ImageNet-9 at layer 3), confirming that a single fixed attachment would not be optimal across all datasets.

4. Key Insights and Innovations

Innovation 1: Reframing Debiasing as a Representational-Competition Problem Within a Single Network

The dominant conceptual framing across prior debiasing work treats the problem as one of output-level correction: the network learns biased predictions, and some auxiliary mechanism — a second network, a reweighting scheme, an ensemble diversity constraint, or a worst-case optimization objective — adjusts either the training data or the final decision boundary to compensate. LfF (Nam et al., 2020) trains a biased network, then trains a debiased network to disagree with it. LWBC (Kim et al., 2022) uses a committee to upweight bias-conflicting samples. ESB (Teney et al., 2022) enforces gradient diversity across ensemble members. In all these cases, the intervention operates after features have been learned, at the level of how those features are combined into predictions.

SiFER makes a fundamentally different conceptual move: it locates the problem of simplicity bias inside the network's representational hierarchy and intervenes there directly. The key insight is that simple features are not merely preferred by the final classifier — they actually crowd out the development of complex features in later layers because the gradient signal from simple features dominates early in training, and once a "good enough" solution is found using those features, there is insufficient gradient pressure to develop richer representations. This reframes debiasing from a post-hoc correction problem to a representational competition problem: the challenge is not to avoid using simple features at the output level, but to prevent them from monopolizing the representational capacity of early layers, thereby starving later layers of the gradient signal needed to learn complex features.

This reframing connects directly to the theoretical literature on simplicity bias (Shah et al., 2020) and gradient starvation (Pezeshki et al., 2021) in a way that prior debiasing methods do not. Those theoretical works described why neural networks over-rely on simple features — the gradient dynamics favor features that are learned quickly — but prior practical debiasing methods did not operationalize this understanding. They treated simplicity bias as a black-box tendency to be counteracted by data manipulation or output regularization, rather than as a specific mechanistic failure (early-layer features dominating the gradient signal) that could be directly targeted. SiFER's intervention — erasing discriminative features at the exact depth where they first become computationally accessible — is the first method that directly operationalizes the mechanistic account from the simplicity bias literature.

The significance of this reframing extends beyond the specific method. It suggests a broader class of "representational barrier" techniques: rather than trying to force networks not to use certain features (which is difficult to enforce at the output level, since features can be linearly decoded from distributed representations even when not explicitly used by the classifier), one can actively degrade those features at the representational level, creating a training dynamic where later layers are forced to find alternative features to maintain accuracy. This opens a research direction orthogonal to the data-manipulation and output-regularization paradigms that have dominated the debiasing literature.

Innovation 2: Operationalizing "Simple Feature" Without Defining It

A deep conceptual challenge in the simplicity bias literature is that "simple" is difficult to define rigorously in a way that generalizes across datasets, architectures, and tasks. Prior work has addressed this by either (a) hand-specifying which features are spurious based on domain knowledge (Group-DRO, EnD, CSAD, and many others require explicit bias attribute labels), (b) making architectural assumptions about what unbiased features look like (OccamNet assumes spatially localized features), or (c) requiring multiple training environments where spurious correlations vary (IRM, REx, EIIL). All of these embed some form of a priori knowledge about what constitutes a simple or spurious feature.

SiFER's approach is conceptually novel: it defines "simple features" purely operationally, as whatever features an auxiliary classifier of limited capacity can extract from an intermediate representation at a given depth that achieves low generalization error. There is no human judgment about what counts as simple; there is no hand-coded definition of the function class or complexity threshold. Instead, the auxiliary network's architecture (its depth and attachment position) implicitly defines a complexity budget, and the features that happen to be computable within that budget — and that happen to be predictive on the training data — are the ones that get identified and subsequently erased.

This operational definition has a subtle but important property: it is self-calibrating to the dataset and model. The same auxiliary architecture attached at the same depth might identify color as the "simple" feature on CMNIST but texture as the "simple" feature on ImageNet-9, because those are the features that are computationally accessible at that depth in those specific training setups. The definition of simplicity is not absolute — it is relative to what the specific network architecture can compute at what depth from the specific training data. This is why the paper can claim the method works "without prior knowledge of spurious attributes or features": the auxiliary network discovers what is simple through the act of trying to classify from a constrained representational budget.

The operationalization is validated by the controllability experiments in Section 5.2 (Table 2), which show that SiFER can be steered to focus on either the "simple" or "complex" feature in CMNIST and CIFAR_MNIST simply by changing which feature is predictive in the validation set. When the validation set randomizes color (making digit shape the only reliable predictor), SiFER suppresses color and enhances digit decodability. When the validation set randomizes digits (making color the only reliable predictor), SiFER suppresses digits and enhances color decodability. The method does not "know" that color is simpler than shape — it simply erases whatever features are identified as predictive by the auxiliary network at the chosen depth, and the validation loop selects the configuration that generalizes best. This demonstrates that the feature sieve is genuinely a feature tradeoff tool rather than a "simple-feature suppressor" — it can enhance either simple or complex features depending on what the validation data demands.

This operationalization represents a genuine conceptual advance over prior work, which either required explicit bias labels (tying the method to human judgments of what is spurious) or required multiple environments (tying the method to data-collection choices). SiFER's approach, while not eliminating all inductive bias (the auxiliary network's architecture and attachment point still embody assumptions about where "simple" features live in the network), transfers these choices from the human practitioner's domain knowledge to the validation loop — the machine discovers what needs to be suppressed through empirical optimization of generalization error.

Innovation 3: The Alternating Identify-and-Erase Dynamic as a New Training Paradigm

Prior debiasing methods that involve multiple networks or training phases typically operate in one of two modes: sequential (train a biased model first, then use it to guide a debiased model — as in LfF, Debian, and LWBC) or parallel/joint (train multiple models simultaneously with a diversity or fairness constraint — as in ESB and Group-DRO). SiFER introduces a fundamentally different temporal structure: alternating interleaved optimization where identification and erasure happen at different timescales within a single network.

The conceptual significance of this alternation is that it resolves a tension that would otherwise make the identify-and-erase goal self-defeating. If identification and erasure were attempted simultaneously (e.g., by applying both the auxiliary classification loss and the forgetting loss at every iteration), the two objectives would be directly contradictory: one pulls the early layers to produce discriminative features, the other pulls to destroy them. The resulting gradient conflict would either produce a noisy stalemate or, if one loss dominates, collapse to one extreme (either no erasure, or complete destruction of all early-layer representations). The alternating schedule avoids this by enforcing a temporal separation: the auxiliary network is given time to discover what features are currently available before those features are erased, and then the main network is given time to adapt to the erasure before the next identification cycle begins.

This is not merely an implementation detail — it represents a novel training paradigm that could generalize beyond debiasing. The core idea is that when a network needs to be forced to not use certain easily-learned solutions, one can create an adversarial dynamic where an auxiliary module periodically identifies what solutions the network is currently exploiting and erases them, with the temporal separation between identification and erasure preventing the gradient conflict that would otherwise arise. This is conceptually related to adversarial training (where a generator and discriminator compete), but with a crucial difference: in GANs, the two networks are opponents with opposing objectives optimized simultaneously; in SiFER, the identification network is a collaborator that helps the main network discover and overcome its own lazy tendencies, and the alternation ensures the collaboration remains productive rather than destructive.

The hyperparameter F (forget_after_iters) controls the timescale of this dynamic, and the paper shows that the optimal value varies substantially across datasets (Table 8: F = 30 for BAR-OOD, F = 50 for CelebA, F = 70 for BAR-ID, NICO, and ImageNet-9). This variation itself is informative: it suggests that different datasets require different temporal granularity for the identify-erase cycle, likely because the speed at which simple features become entrenched in early layers depends on their signal strength and the complexity of the genuine features. The fact that this hyperparameter matters — and that its optimal value is dataset-dependent — indicates that the temporal dynamics of feature competition are themselves an important dimension of the simplicity bias problem that prior work had not explicitly addressed.

Innovation 4: Demonstrating That Debiasing Can Succeed Without Bias-Conflicting Training Examples

A largely unstated assumption across much of the debiasing literature is that some access to bias-conflicting examples is necessary for effective debiasing. LfF requires bias-conflicting examples to identify what the biased network gets wrong. LWBC adds 1% of conflicting examples from the test set to the training data. EIIL and similar environment-inference methods require at least some variation in the spurious correlation across training subsets. Even methods that don't explicitly use conflicting examples often rely on validation sets that contain them (implicitly providing signal about the bias structure during hyperparameter tuning) or on data augmentation that artificially creates conflicting examples.

SiFER challenges this assumption directly. The method's core mechanism — identifying features via the auxiliary network's predictions on the training data, then erasing them — requires no bias-conflicting examples whatsoever. The auxiliary network identifies whatever features are predictive on the training distribution, which are precisely the spurious features (since they are perfectly correlated with the label in the training data). The forgetting loss erases these features regardless of whether they are genuinely causal or merely correlated. The validation loop then determines how aggressively to sieve by measuring generalization — and crucially, the paper shows in the BAR-ID val setting (Table 3, bottom half) that this works even when the validation set contains 0% conflicting examples (Table 1).

The BAR-ID result is the clearest test of this claim. With in-distribution validation (no OOD signal), no bias-conflicting training examples, and no attribute labels, SiFER achieves 65.75% accuracy compared to 35.32% for ERM. This is a ~30 percentage point gain in a setting where most prior debiasing methods would have no signal to work with — there are no conflicting examples to reweight, no attribute labels to condition on, and no OOD validation data to reveal the bias structure. The method outperforms LWBC (62.03%), ESB (64.40%), and other strong baselines that also lack OOD validation.

This finding has both practical and conceptual significance. Practically, it means SiFER can be deployed in settings where the practitioner has no access to bias-conflicting data and cannot even identify what the bias is — a common real-world scenario where datasets are collected without awareness of spurious correlations. Conceptually, it demonstrates that the inductive bias encoded in the auxiliary network's architecture and attachment point — that simple, easily-computed features are more likely to be spurious — is a sufficiently powerful prior to enable debiasing without any explicit examples of the bias being violated. The network's own training dynamics (what features emerge early vs. late) provide enough signal to distinguish spurious from genuine features, at least for the visual biases present in the evaluated datasets.

This is not to say the method is completely assumption-free — the auxiliary network's depth and attachment position encode assumptions about the complexity and depth of spurious features, and the validation metric (e.g., unbiased accuracy on CelebA) may still encode knowledge about protected groups. But the demonstrated ability to debias with zero conflicting training examples and zero attribute labels represents a significant loosening of the data requirements for practical debiasing.

Innovation 5: Empirical Discovery That Spurious Feature Depth Varies Dramatically Across Datasets

While the paper's primary contribution is methodological, the experimental results in Table 8 contain a finding that has independent conceptual significance: the optimal depth for sieving spurious features varies dramatically across datasets, ranging from the earliest layers (NICO: A_P = 1) to relatively deep layers (ImageNet-9: A_P = 3). This is not an obvious or expected result. The simplicity bias literature (Shah et al., 2020; Hermann & Lampinen, 2020) generally treats "simple features" as a monolithic category — low-level texture, color, or edge statistics that are universally learned early in training. If this were true, one would expect the optimal auxiliary attachment position to consistently be at shallow layers across all datasets.

The data tell a more nuanced story. On NICO, where the spurious correlation is between object class and background context (e.g., certain animals appearing in certain landscapes), the optimal position is layer 1 — the spurious features are genuinely low-level and can be identified almost immediately in the network. On CelebA, where the spurious correlation is between hair color and gender (an abstract semantic attribute), the optimal position is layer 2 — the spurious features require some hierarchical processing to become decodable. On ImageNet-9, where the spurious correlation is between object class and texture, the optimal position is layer 3 — texture-based classification, counter to the common intuition that texture is a "low-level" feature, actually requires relatively deep processing in ResNet-18 to become discriminatively useful at the level of super-class classification.

This finding has implications beyond SiFER. It suggests that "spurious feature" is not a single category with a fixed computational profile — different types of spurious correlations manifest at different depths in the feature hierarchy, and understanding where in the network a particular bias emerges may be essential to designing effective interventions. A method that always targets layer 1 would be effective against color and simple texture biases but fail against gender bias in face recognition or background context bias in activity recognition. Conversely, a method that only targets deep layers might miss low-level texture biases that drive ImageNet-trained CNN failures on shape-based recognition tasks.

The finding also validates a key design choice in SiFER: the controllability of the auxiliary attachment position is not a minor hyperparameter but a structurally essential feature of the method. If the optimal position were always the same (e.g., always layer 1), the method could be simplified by hardcoding that choice. The fact that the optimal position varies — and that the variation aligns with intuitive notions of which biases are "low-level" vs. "semantic" — suggests that SiFER's position hyperparameter is genuinely doing work: it is discovering where in the network the spurious correlation becomes computationally accessible, which is a form of automated bias characterization that could inform future debiasing efforts even beyond the SiFER framework itself.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper evaluates on two controlled synthetic datasets and five real-world debiasing benchmarks. CMNIST: a 2-class dataset constructed from MNIST digits 0 and 1 with an added color channel (red for 0, green for 1), where both color and digit shape are perfectly predictive. CIFAR_MNIST: binary classification of composite images pairing MNIST 0s with CIFAR automobiles (class 0) and MNIST 1s with CIFAR trucks (class 1), again with both features fully predictive. BAR (Biased Activity Recognition; Nam et al., 2020): 6-class human activity classification with spurious background correlations in training (e.g., climbing always appears with rocks) and no bias-conflicting training examples; test set uses different backgrounds. CelebA (Liu et al., 2018): hair color prediction heavily correlated with gender — fewer than 1% of blond-haired individuals are men in training. NICO (He et al., 2021): Animal subset with 10 object classes and 10 context labels; training contains only 7 contexts per class, while validation and test include 3 unseen contexts. ImageNet-9 (Xiao et al., 2020): 9 super-class subset of ImageNet with established texture bias; train/val split follows Kim et al. (2022) and Bahng et al. (2020). ImageNet-A (Hendrycks et al., 2021): handpicked real-world images misclassified by standard ImageNet models due to over-reliance on spurious color and texture features; used as an OOD test set for models trained on ImageNet-9. The training data composition for each dataset's bias-conflicting examples is quantified in Table 1: BAR-ID val, NICO, and ImageNet-9 all have 0% conflicting examples in training, making them particularly challenging for methods that rely on reweighting bias-violating samples.
Base model(s). All real-world experiments use ResNet-18 loaded with ImageNet pre-trained weights. The controlled dataset experiments (CMNIST, CIFAR_MNIST) also use ResNet-based architectures, with the layerwise decodability experiments in Figure 3 specifically using ResNet-50 to provide finer-grained layer resolution for tracking feature emergence across network depth. The auxiliary network uses the same BasicBlock structure as ResNet-18, with depth treated as a hyperparameter (range 1–9). The choice of ResNet-18 as the primary architecture is pragmatic: it is a standard, widely-used backbone that allows direct comparison against the majority of baselines in the debiasing literature, which also use ResNet-18 or ResNet-50. The ImageNet pre-training is consistent across all real-world experiments, providing a common starting point that reflects practical deployment scenarios where practitioners fine-tune pre-trained models rather than training from scratch.
Metrics. The paper uses several dataset-specific evaluation metrics, summarized in Table 1. Accuracy: standard average classification accuracy on all test examples — used for BAR, NICO, and ImageNet-9/ImageNet-A. Unbiased Accuracy: accuracy computed separately for each label-context group and then averaged, preventing majority groups from dominating — used specifically for CelebA due to extreme group imbalance (blond-haired men constitute <1% of the training set, so standard accuracy would be insensitive to gender bias). Conflicting Accuracy: accuracy restricted to only the bias-conflicting examples — reported for CelebA alongside Unbiased Accuracy to provide a direct measure of performance on the hardest subgroup. For validation-guided hyperparameter selection, the metric used is Accuracy for BAR, NICO, and ImageNet-9/ImageNet-A, and Unbiased Accuracy for CelebA (Table 1). Additionally, the controlled dataset experiments report Simple Rate (SR) and Complex Rate (CR) — accuracy when only the simple feature (or only the complex feature) is predictive at test time — to measure which features the classifier actually relies on (Table 2, Section 5.2). Feature decodability is measured by freezing the trained classifier, training a linear decoder on its representation at a specified layer using validation data labeled by the feature of interest (e.g., digit identity or CIFAR object class), and reporting the decoder's accuracy on test data — this quantifies whether a given feature is extractable from the representation at that layer (Section 4.3, Figures 3 and 5).
Baselines. The paper compares against an extensive set of prior methods, which can be grouped into several categories (detailed descriptions provided in Appendix B): Output-level alternate network methods: LfF (Nam et al., 2020) trains a biased network then a debiased network that upweights bias-conflicting samples; Debian (Li et al., 2022) alternates between a bias discoverer and a debiased classifier; LWBC (Kim et al., 2022) uses a committee of biased classifiers to reweight training instances. Ensemble methods: ESB (Teney et al., 2022) enforces gradient diversity across an ensemble to evade simplicity bias. Multiple environment methods: IRM (Arjovsky et al., 2019) seeks invariant representations across training environments; REx (Krueger et al., 2021) optimizes worst-case risk over environments; EIIL (Creager et al., 2021) infers environment assignments automatically; PGI (Ahmed et al., 2021) follows EIIL with a small network for bias group identification. Augmentation and adversarial methods: Roadblock (Niu et al., 2022) adds adversarial augmentations to prevent over-reliance on spurious visual cues; DecAug (Bai et al., 2021a) uses semantic augmentation and feature decomposition to disentangle context from category features; BiaSwap (Kim et al., 2021) swaps bias features between bias-guiding and bias-conflicting groups. Architecture and inductive bias methods: ReBias (Bahng et al., 2020) trains de-biased representations by encouraging difference from biased-by-design representations; OccamNet (Shrestha et al., 2022) adds spatial localization and bounded depth inductive biases; NAS-OoD (Bai et al., 2021b) incorporates OOD generalization into neural architecture search. Domain generalization methods: JiGen (Carlucci et al., 2019) jointly classifies and solves jigsaw puzzles; DANN (Ganin et al., 2016) uses adversarial domain classification; CORAL (Sun & Saenko, 2016) aligns second-order statistics across domains; MMD (Li et al., 2018) uses maximum mean discrepancy for distribution alignment; MTL (Blanchard et al., 2021) augments feature space with marginal distributions; Cumix (Mancini et al., 2020) mixes data and labels across domains; CNBB (He et al., 2021) uses causal inference-based sample reweighting. Methods using explicit bias attribute labels: Group-DRO (Sagawa et al., 2019) minimizes worst-case loss over pre-defined groups; EnD (Tartaglione et al., 2021) uses an information bottleneck with bias attribute regularization; CSAD (Zhu et al., 2021) disentangles target and bias features using explicit bias labels; StylisedIN (Geirhos et al., 2018) augments training with stylized images to improve shape bias; LearnedMixin and RUBi (Clark et al., 2019; Cadene et al., 2019) were designed for visual question answering with known spurious modalities; CaaM (Wang et al., 2021) learns causal attention by data partitioning. The standard ERM (Empirical Risk Minimization) baseline trains a standard classifier with cross-entropy loss and no debiasing intervention. The paper explicitly notes which baselines use spurious attribute labels (marked with ✓ in Tables 3–6) versus those that do not (marked with ✗), highlighting that SiFER belongs to the latter category and yet outperforms many methods in the former.
Generation budget / compute accounting. SiFER does not use generation budgets in the LLM sampling sense — the method is a training-time intervention for image classifiers. Computational cost is measured implicitly through the standard training paradigm: all models use the same ResNet-18 backbone with the same optimizer (SGD, learning rate 0.001), same number of training epochs, and identical data processing. SiFER adds computational overhead in the form of the auxiliary network's forward and backward passes, which is modest — the auxiliary network uses BasicBlock structures with depth 1–9, a small fraction of the main ResNet-18's total parameters. The forgetting step adds an additional backward pass through the frozen auxiliary network and the early main-network layers every F iterations, but this cost is not quantified in FLOPs or wall-clock time. Compared to ensemble methods like ESB (which trains multiple complete models) or LWBC (which trains a committee), SiFER's single-network approach with a lightweight auxiliary module is computationally parsimonious. The paper does not provide explicit runtime comparisons, which is a notable omission for practitioners weighing the cost-benefit tradeoff.
Cross-validation / statistical protocol. All real-world experiments are repeated with 5 different random seeds, and results are reported as mean ± standard deviation (Tables 3, 4, 5, 6; also the controlled experiments in Table 2). For hyperparameter selection, the paper uses a validation set separate from the test set, with the specific validation split varying by dataset: for BAR-ID val, 20% of training images are held out as validation (containing 0% bias-conflicting examples); for BAR-OOD val, 20% of test images are used as validation (containing 100% bias-conflicting examples); for CelebA, NICO, and ImageNet-9, the pre-existing validation splits provided by the respective dataset authors are used. Hyperparameters (A_D, A_P, α₂, α₃, F) are tuned by searching the ranges specified in Table 7 and selecting the configuration that maximizes the dataset-appropriate validation metric (Table 1). The final hyperparameter values used are reported in Table 8. For the controlled dataset experiments (CMNIST, CIFAR_MNIST), hyperparameters are selected based on the validation set constructed to encode the desired feature preference (e.g., randomizing color to force focus on digit shape), demonstrating the controllability of the sieve. The paper does not perform cross-validation in the strict k-fold sense across the real-world datasets — the validation set is fixed per dataset, and results are reported on a separate held-out test set with multiple random seeds to assess variance.

Main Quantitative Results

Controlled Datasets: Suppressing and Controlling Features

Suppression of simple features (Figure 3): The paper first demonstrates that SiFER successfully suppresses the simple feature (MNIST digit) and enhances the complex feature (CIFAR object) on the CIFAR_MNIST dataset. Figure 3 tracks layerwise decodability of both simple and complex features across training epochs, contrasting ERM (top row) against SiFER (bottom row) using ResNet-50. Under ERM, the simple MNIST feature becomes highly decodable at all layers — including deep layers — while the complex CIFAR feature is decodable to some extent in early layers but becomes suppressed in deeper layers, exactly the dynamic predicted by the simplicity bias hypothesis: simple features dominate and complex features are starved. Under SiFER, this pattern is reversed: the simple MNIST feature is actively suppressed in early-to-mid layers (its decodability drops substantially compared to ERM), and the complex CIFAR feature shows enhanced decodability in deeper layers, indicating that the network is now forced to rely on the CIFAR component for classification. The paper emphasizes that no prior knowledge of which feature is "simple" or "complex" was used — the auxiliary network organically discovered that MNIST digits were the easily-computed feature and the forgetting loss targeted them.

Feature controllability (Table 2): Section 5.2 demonstrates that SiFER is not merely a "simple feature suppressor" but a controllable feature tradeoff tool. On both CMNIST and CIFAR_MNIST, the training data contain perfectly predictive simple and complex features. The validation set is then manipulated to determine which feature the classifier should prioritize: by randomizing the non-relevant feature in the validation set, the hyperparameter selection process (which optimizes validation accuracy) steers SiFER toward the feature that remains predictive. The results in Table 2 report Simple Rate (SR) — accuracy when only the simple feature is predictive — and Complex Rate (CR) — accuracy when only the complex feature is predictive.

For CMNIST with Complex (Digit) as target (validation set randomizes color, making digit shape the only reliable predictor): SiFER achieves SR = 99.54 ± 0.19% and CR = 58.14 ± 10.69%, compared to ERM's SR = 56.96 ± 6.59% and CR = 92.21 ± 3.92%. This means SiFER successfully prioritizes the complex digit feature (nearly perfect accuracy when digit is predictive) while largely ignoring color (only 58% accuracy when color alone is predictive). ERM, by contrast, achieves high accuracy on color (92.2%) and poor accuracy on digit (57.0%), reflecting its default simplicity bias toward the easier color feature.

For CMNIST with Simple (Color) as target (validation set randomizes digits, making color the only reliable predictor): the pattern reverses. SiFER achieves SR = 52.44 ± 1.22% and CR = 99.64 ± 1.30%, while ERM achieves SR = 49.20 ± 2.60% and CR = 96.27 ± 0.99%. This demonstrates that SiFER can be steered to prioritize either the simple or complex feature based purely on validation set construction — the method suppresses whatever features are identified as spurious by the auxiliary network at the chosen depth, and the validation loop selects the configuration that aligns with the generalization objective. ERM, regardless of which feature the validation set emphasizes, consistently prioritizes the simple feature (higher CR numbers in both rows).

For CIFAR_MNIST with Complex (CIFAR) as target: SiFER achieves SR = 62.37 ± 4.62% and CR = 48.93 ± 1.92%, compared to ERM's SR = 58.14 ± 1.60% and CR = 100%. The 100% CR for ERM indicates that ERM's performance is entirely driven by the simple MNIST feature — when the CIFAR component is predictive and the MNIST component is randomized, ERM achieves perfect accuracy because it relies exclusively on MNIST digits for classification, which remain intact when only CIFAR is randomized (this is a property of the composite image construction: randomizing CIFAR does not affect the MNIST digit, so a digit-reliant classifier maintains accuracy). SiFER reduces this dependence (CR drops from 100% to 48.9%) and increases reliance on CIFAR (SR rises from 58.1% to 62.4%).

For CIFAR_MNIST with Simple (Digit) as target: SiFER achieves SR = 47.17 ± 0.14% and CR = 99.83 ± 0.29%, while ERM achieves SR = 49.20 ± 2.60% and CR = 100%. The near-100% CR in both cases reflects that digit accuracy is high regardless, but the SR numbers show that SiFER slightly reduces reliance on the complex CIFAR feature when the validation set demands focus on digit shape.

The key headline: SiFER's diagonal terms (SR for digit-target, CR for color-target, etc.) are consistently higher than ERM's, while SiFER's off-diagonal terms (reliance on the non-target feature) are substantially lower than ERM's. This demonstrates that the feature sieve is genuinely a controllable feature tradeoff mechanism — it can be steered to prioritize either simple or complex features depending on which is made relevant by the validation set, and it consistently outperforms ERM at directing representational capacity toward the intended feature.

Debiasing Real-World Datasets

BAR — Biased Activity Recognition (Table 3): BAR represents a challenging real-world debiasing benchmark where training data has 0% bias-conflicting examples (all climbing images contain rocks, all skiing images contain snow, etc.) and the test set systematically breaks these correlations. The paper evaluates under two settings: with OOD validation (20% of test data used for validation, providing signal about the bias structure) and with in-distribution (ID) validation (20% of training data used for validation, providing zero explicit signal about the bias).

Under the OOD validation setting (Table 3, top half), SiFER achieves 72.08 ± 0.38% accuracy, outperforming all baselines. The next best method is Debian (Li et al., 2022) at 69.88 ± 2.92%, followed by Roadblock (Niu et al., 2022) at 69.51 ± 2.43%. The gain over the best baseline is approximately 2.2 percentage points (absolute), which translates to a relative improvement of about 3.2% over Debian. Notably, SiFER outperforms methods that explicitly use the OOD validation signal for bias identification (EIIL, PGI) and ensemble methods that train multiple models (ESB). ERM achieves only 51.85 ± 5.92%, meaning SiFER provides a ~39% relative improvement over standard training — a dramatic gain that underscores the severity of the background bias in BAR and the effectiveness of SiFER's intervention.

Under the ID validation setting (Table 3, bottom half), which is the more realistic and harder scenario, SiFER achieves 65.75 ± 1.84% accuracy. This is compared to ERM's 35.32 ± 0.46% — a gain of over 30 percentage points, representing an 86% relative improvement. SiFER outperforms all baselines that also operate without OOD validation: ESB (64.40 ± 0.20%), LWBC (62.03 ± 0.74%), SSL+ERM (60.88 ± 0.80%), LfF (48.15 ± 0.93%), and ReBias (37.02 ± 0.26%). The gap over ESB (the next best method) is approximately 1.35 percentage points. Crucially, SiFER with ID validation (65.75%) even outperforms several baselines that do use OOD validation: BiaSwap (52.44%), LfF with OOD val (62.98%), PGI (65.19%), and EIIL (65.44%). This is a striking result — SiFER without any access to bias-conflicting validation data surpasses methods that explicitly leverage knowledge of the bias structure through OOD validation. The standard deviation of 1.84% under ID validation is notably larger than the 0.38% under OOD validation, indicating that the ID setting introduces more seed-dependent variance, likely because the validation signal is weaker for hyperparameter selection.

CelebA — Hair Color Prediction (Table 4): On the CelebA Hair dataset, which tests for gender bias in hair color prediction, SiFER achieves Unbiased Accuracy = 89.00 ± 0.92% and Conflicting Accuracy = 88.04 ± 1.25%. This is compared to ERM's Unbiased Accuracy of 70.25 ± 0.35% and Conflicting Accuracy of 52.52 ± 0.19% — the low conflicting accuracy for ERM reflects near-total failure on blond-haired men, the minority group.

SiFER's Unbiased Accuracy is competitive with the best method that uses explicit spurious attribute labels: EnD (Tartaglione et al., 2021) achieves 91.21 ± 0.22%, and CSAD (Zhu et al., 2021) achieves 89.36%. SiFER slightly trails EnD (~2.2 percentage points) but matches or exceeds CSAD, and does so without using gender labels that EnD and CSAD require. Among methods that do not use attribute labels (marked ✗ in Table 4), SiFER achieves the highest Unbiased Accuracy: 89.00% vs. LWBC's 88.90 ± 1.55%, LfF's 84.24 ± 0.37%, and SSL+ERM's 80.48 ± 0.91%. SiFER's Conflicting Accuracy (88.04%) is the highest among all methods — including those using attribute labels — slightly exceeding LWBC (87.22 ± 1.14%), CSAD (87.53%), and EnD (87.45 ± 1.06%). This means that on the most challenging subset (the minority group where bias must be overcome), SiFER slightly outperforms even methods that are explicitly given the gender attribute.

The proximity of SiFER's Unbiased and Conflicting accuracies (89.00% vs. 88.04%) indicates that the model treats majority and minority groups nearly equally — there is less than a 1% gap. In contrast, ERM shows a 17.7 percentage point gap (70.25% vs. 52.52%), and even the best bias-label-using method (EnD) shows a 3.76 point gap (91.21% vs. 87.45%). This suggests that SiFER's feature sieve effectively removes the gender-correlated signal from the representations used for classification, resulting in a model that is genuinely fairer across demographic groups rather than merely improving aggregate metrics.

NICO — Domain-Shift Generalization (Table 5): NICO tests generalization to unseen contexts (e.g., object classes appearing in backgrounds never seen during training). SiFER achieves 86.20 ± 0.85% accuracy, outperforming all baselines except NAS-OoD (88.72%), which the paper notes uses architecture search optimization and is "hence unfair to compare directly" due to its fundamentally different computational requirements (architecture search trains and evaluates many candidate architectures). Among standard single-model methods, SiFER leads with a 0.97 percentage point margin over DecAug (85.23%), and substantial margins over JiGen (84.95%), Mixup (80.27%), CORAL (80.27%), and ERM (75.87%).

The paper emphasizes that many of the baselines in Table 5 "use spurious attribute labels for training, while we do not" — specifically, DecAug, DRO, and several others incorporate context information. Despite lacking this privileged information, SiFER surpasses them. The fact that SiFER outperforms domain generalization methods (DANN, CORAL, MMD, MTL) by large margins (5–15 percentage points) suggests that directly sieving spurious features is more effective than domain-adversarial or distribution-matching approaches, at least when the spurious feature is a low-level background context that can be identified and erased at early network layers (Table 8: NICO uses A_P = 1, supporting this interpretation).

ImageNet-9 and ImageNet-A — Texture Bias Robustness (Table 6): This experiment tests SiFER's ability to overcome the well-documented texture bias of ImageNet-trained CNNs. Models are trained on ImageNet-9 and evaluated both on the in-distribution ImageNet-9 validation set (measuring whether accuracy is maintained) and on ImageNet-A, a challenging OOD test set of natural images that are systematically misclassified by standard models due to texture over-reliance.

On ImageNet-9 validation, SiFER achieves 97.78 ± 0.12%, compared to ERM's 90.8 ± 0.6%. The previous best methods were CaaM at 95.70%, SSL+ERM at 94.18 ± 0.07%, and LWBC at 94.03 ± 0.23%. SiFER's gain over CaaM is approximately 2.1 percentage points, and over SSL+ERM is approximately 3.6 percentage points. Notably, SiFER achieves this high in-distribution accuracy without sacrificing it for robustness — a common concern with debiasing methods is that suppressing features might degrade overall performance on the training distribution, but SiFER actually improves ImageNet-9 validation accuracy by ~7 percentage points over ERM. This suggests that the texture features being sieved were not just spurious but actively harmful even within the training distribution, perhaps by causing confusion between super-classes with similar textures but different object identities.

On ImageNet-A, SiFER achieves 39.98 ± 0.81%, compared to ERM's 24.9 ± 1.1%. This represents an 11.1% relative improvement over the previous best method (LWBC at 35.97 ± 0.49%) and an 11.4% relative improvement over SSL+ERM (34.21 ± 0.49%). The absolute gain over LWBC is approximately 4.0 percentage points. This is the largest relative gain reported in the paper and arguably the most practically significant, since ImageNet-A consists of naturally occurring adversarial examples that specifically exploit texture bias — the very failure mode that SiFER is designed to address. The fact that SiFER transfers robustly from ImageNet-9 training to ImageNet-A evaluation (despite hyperparameters being tuned only on ImageNet-9 validation accuracy) provides strong evidence that the sieve is genuinely removing texture bias rather than overfitting to the validation distribution.

Comparing across the four real-world datasets, a consistent pattern emerges: SiFER provides substantial gains over ERM across all benchmarks (ranging from ~19 percentage points on CelebA Unbiased to ~30 percentage points on BAR-ID), and it consistently outperforms the best competing methods that also lack attribute labels. The magnitude of gain varies by dataset — largest on BAR and ImageNet-A where the spurious correlation is particularly egregious, modest on CelebA where even ERM achieves some baseline accuracy — but the direction is uniformly positive.

Feature Decodability on Real-World Data

CelebA layerwise decodability (Figure 5): The paper extends the controlled-dataset decodability analysis to a real-world setting by measuring the decodability of the target feature (hair color) and the spurious feature (gender) across layers of ResNet-18 trained on CelebA. Under ERM (Figure 5, left), gender — the spurious feature — is more decodable than hair color at all layers, and especially at deeper layers (layers 3–4), where gender decodability reaches approximately 0.7–0.8 while hair color decodability is approximately 0.5–0.6. This means the network's internal representations are more informative about gender than about the actual target attribute, directly quantifying the simplicity bias: the network has learned to route gender information through its feature hierarchy more strongly than hair color information because gender is an easier predictive signal in the biased training data.

Under SiFER (Figure 5, right), this relationship is reversed. Gender decodability is substantially suppressed across all layers, dropping to approximately 0.3–0.4 in deeper layers, while hair color decodability rises to approximately 0.6–0.7 — now higher than gender at every layer depth. The suppression is particularly pronounced in the deepest layers (layer 4), where the gap between hair color and gender decodability is largest. This mirrors precisely the pattern observed on controlled datasets (Figure 3): SiFER identifies and erases the spurious feature (gender) from early and intermediate representations, forcing later layers to rely on the target feature (hair color) for classification. The paper notes that this is particularly remarkable because gender is an "abstract concept" — not a low-level perceptual feature like color or texture — yet SiFER automatically identifies it as spurious and suppresses it without being given any explicit information about gender being a protected attribute or a bias variable.

Feature Importance Visualizations

GRAD-CAM comparison (Figure 4): To verify that SiFER's debiasing corresponds to semantically meaningful changes in what the classifier attends to, the paper uses GRAD-CAM (Selvaraju et al., 2017) to visualize feature importance maps for ERM and SiFER classifiers on BAR and NICO images. Figure 4 shows six example images (first three from BAR, last three from NICO) with input images on the top row, ERM feature importance on the middle row, and SiFER feature importance on the bottom row.

The results show a consistent qualitative pattern. ERM's attention maps are diffuse, often covering large portions of the background — for BAR images, ERM highlights the climbing wall, the snowy slope, and the indoor gym background as much as or more than the person performing the action; for NICO, ERM highlights trees, grass, and water backgrounds alongside the animal. This is consistent with ERM relying on background context as a spurious predictive feature (e.g., recognizing "climbing" by detecting rocks rather than by analyzing the human pose). SiFER's attention maps, in contrast, are tightly focused on the central object of interest — the person performing the action in BAR images (the climber, the skier) and the animal in NICO images — while effectively suppressing background regions that ERM heavily weights. The paper describes this as SiFER's ability to "correctly identify important visual features of a scene, while suppressing irrelevant but spuriously-label-correlated background features."

The critical point emphasized in the paper is that SiFER is not simply trading one feature class for another in a blind manner — it is genuinely focusing on semantically relevant objects (the human performing the action, the animal being classified) while ignoring background context that happens to correlate with labels in training but is not causally relevant. This qualitative evidence supports the quantitative gains on BAR and NICO and provides a visual explanation for why SiFER works: the feature sieve removes background texture and context cues from early representations, forcing the network to attend to the foreground objects that actually determine the class label.

Ablation Studies and Robustness Checks

The paper does not include a dedicated ablations section in the traditional sense — there is no systematic removal of components or variation of individual hyperparameters with all others held fixed. However, several experimental choices and analyses function as implicit ablations and robustness checks:

Validation set composition (BAR-ID vs. BAR-OOD, Table 3): The comparison between SiFER with in-distribution validation (65.75%) and with OOD validation (72.08%) quantifies the value of having bias-revealing validation data. Using OOD validation provides an additional 6.33 percentage points — significant but not dramatic relative to the 30-point gap over ERM in both settings. This suggests that while OOD validation helps (by enabling more aggressive hyperparameter selection), the core mechanism of SiFER (identifying features on training data and erasing them) is sufficiently powerful that even ID validation — which provides no direct signal about the bias structure — yields large gains. This is effectively an ablation of the validation set quality.

Feature controllability experiment as an ablation of the "simple features only" assumption (Table 2): The controllability results demonstrate that SiFER is not hardcoded to suppress simple features — it can be steered to suppress either simple or complex features depending on the validation signal. This ablates the concern that SiFER merely inverts the simplicity bias (always suppressing simple features and always enhancing complex ones) rather than genuinely discovering and targeting spurious features.

Hyperparameter variation across datasets as an implicit ablation of fixed-configuration SiFER (Table 8): The fact that optimal hyperparameters vary substantially across datasets — A_P ranges from 1 (NICO) to 3 (ImageNet-9), α₃ ranges from 3 (BAR-OOD) to 75 (NICO), F ranges from 30 (BAR-OOD) to 70 (BAR-ID, NICO, ImageNet-9) — demonstrates that no single fixed configuration would work well across all datasets. The validation-guided tuning is essential, and the paper implicitly ablates the alternative ("one-size-fits-all SiFER") by showing that the optimal configuration is dataset-specific.

GRAD-CAM visualizations as a qualitative ablation of the "SiFER might be doing something unrelated to feature suppression" concern (Figure 4): The attention map comparisons show that the accuracy gains are indeed accompanied by the expected behavioral change (shifted attention from background to foreground), providing evidence that the mechanism is working as intended rather than through some unrelated effect.

The paper does not ablate the following, which would be informative: (1) the effect of removing the auxiliary network entirely and applying a generic regularization (e.g., dropout, weight decay) at the same layers; (2) the effect of varying α₃ while holding all other hyperparameters fixed — this would isolate the contribution of the forgetting loss strength; (3) the effect of the no-backpropagation constraint (allowing L₂ gradients to flow into the main network during standard iterations) — this would test whether the forced separation between identification and creation is essential; (4) training with only identification (α₃ = 0) to confirm that the auxiliary network alone (without erasure) does not produce the gains; (5) a runtime or FLOPs comparison quantifying the computational overhead of the alternating optimization relative to standard ERM training and relative to ensemble baselines.

Critical Assessment

The experimental results provide strong evidence for SiFER's effectiveness on the specific benchmarks evaluated, but several aspects of the experimental design warrant careful scrutiny regarding the scope and robustness of the paper's central claims.

Claim: SiFER mitigates simplicity bias without prior knowledge of spurious features

What the experiments demonstrate: The paper convincingly shows that SiFER, with hyperparameters tuned via validation error, achieves substantial gains over ERM on all five real-world benchmarks (Tables 3–6) without using explicit spurious attribute labels. The feature decodability analyses (Figures 3 and 5) and GRAD-CAM visualizations (Figure 4) provide convergent evidence that the mechanism by which these gains occur is indeed the suppression of spurious features and the enhancement of target features. The controllability experiment (Table 2) further demonstrates that the sieve is not blindly suppressing all simple features but can be steered by validation data construction — an elegant validation of the operational definition of spuriousness.

What the experiments do NOT demonstrate: The "no prior knowledge" claim requires careful qualification. While SiFER does not use explicit bias attribute labels, the method does encode prior knowledge in two important ways: (1) the auxiliary network's architecture and attachment-point hyperparameter space encode the assumption that spurious features are computable at some intermediate depth with limited capacity — this is a weaker assumption than knowing the specific spurious attribute, but it is still an assumption about the nature of spurious features (they are simple enough to be detected by a shallow network); (2) the validation metric for CelebA (Unbiased Accuracy, Table 1) requires group annotations — knowing which examples belong to which demographic groups — which is precisely the kind of bias-relevant metadata that the paper claims not to require. Computing Unbiased Accuracy requires knowing the joint distribution of hair color and gender for every validation example, which means gender labels are used during hyperparameter selection even though they are not used during training. This weakens the claim that SiFER "does not depend on prior knowledge of spurious attributes" — for CelebA at least, the method does depend on gender labels in the validation set to select hyperparameters. The BAR-ID val setting (no group annotations, no OOD validation) is the cleanest test of the no-prior-knowledge claim, and SiFER performs well there (65.75%), but the CelebA result (where Unbiased Accuracy is the tuning metric) does not have the same purity. The paper would be strengthened by reporting CelebA results with hyperparameters tuned on standard accuracy (which does not require group labels) to isolate the contribution of the group-aware validation metric.

Additional concern about the validation set requirement more broadly: All reported results depend on a validation set that provides some signal about generalization — either in-distribution accuracy (BAR-ID, ImageNet-9) or OOD/unbiased metrics (BAR-OOD, CelebA, NICO). In truly blind deployment scenarios where no held-out data of any kind is available, SiFER's hyperparameters would need to be set by some other heuristic, and the reported gains might not transfer. This is a practical limitation rather than a conceptual one, but it matters for real-world use.

Claim: SiFER outperforms baselines that use explicit bias attribute labels

What the experiments demonstrate: The paper provides specific instances: on BAR-OOD, SiFER (72.08%) outperforms Debian (69.88%), which uses bias discovery but not explicit labels — though Debian, like SiFER, does not use explicit attribute labels per se; the Table 3 methods with OOD validation are a mix of label-using and label-free approaches. On CelebA, SiFER's Conflicting Accuracy (88.04%) exceeds that of EnD (87.45%) and CSAD (87.53%), which do use explicit gender labels. On NICO, SiFER (86.20%) outperforms DecAug (85.23%), which uses context information. These are valid point comparisons that support the claim for specific dataset-metric pairs.

What the experiments do NOT demonstrate: The claim of general superiority over label-using methods is not uniformly supported. On CelebA Unbiased Accuracy, EnD achieves 91.21% vs. SiFER's 89.00% — a 2.2 percentage point gap in the opposite direction. The paper acknowledges this but frames it as SiFER being "competitive" with label-using methods. The fair interpretation is that SiFER matches or slightly trails the best label-using methods on CelebA Unbiased while exceeding them on CelebA Conflicting, and exceeds label-using methods on BAR and NICO. This is a strong but not uniform result. Additionally, the comparison is against published baseline numbers from prior papers, not re-implemented baselines run under identical conditions (same pretrained weights, same optimizer settings, same data splits). Small differences in experimental setup could account for some of the reported gaps.

An important missing baseline is giving the label-using methods the same validation-guided hyperparameter tuning budget as SiFER. If EnD or CSAD were also tuned on Unbiased Accuracy with the same hyperparameter search effort, they might close or reverse the gaps. The paper compares SiFER's tuned performance against baseline numbers that may have been tuned on different metrics or with less extensive hyperparameter search. This is a standard practice in the debiasing literature (using published numbers) but limits the strength of comparative claims.

Claim: SiFER is a "controllable feature tradeoff tool" (not just a simple-feature suppressor)

What the experiments demonstrate: The controllability experiment (Table 2) is clean and convincing. By manipulating the validation set (randomizing color vs. randomizing digit), SiFER is steered toward the feature that remains predictive. The diagonal/off-diagonal pattern in the table shows that SiFER consistently prioritizes the validation-set-relevant feature, while ERM always prioritizes the simple feature regardless of validation set. This directly supports the claim of controllability on synthetic data.

What the experiments do NOT demonstrate: This controllability is demonstrated only on two synthetic datasets with two perfectly predictive, separable features. Whether the same controllability extends to real-world datasets — where features are correlated, overlapping, and not cleanly separable into "simple" and "complex" categories — is not tested. The real-world experiments use a single validation metric per dataset (as appropriate for the task), not a manipulation of which feature to prioritize. The paper's claim that SiFER is "more broadly a controllable feature tradeoff tool" (Section 1) is supported primarily by the synthetic experiments, and its applicability to real-world feature tradeoffs (e.g., trading off accuracy on one subgroup vs. another) is not demonstrated.

Methodological strengths

The experimental design has several notable strengths. The use of five diverse real-world benchmarks covering different bias types (background bias, gender bias, context bias, texture bias) provides a broader evaluation than most debiasing papers, which typically focus on 1–2 datasets. The inclusion of both ID and OOD validation settings for BAR tests the method's robustness to validation set quality — an important practical consideration that many debiasing papers ignore. The feature decodability analyses on both synthetic (Figure 3) and real (Figure 5) data provide mechanistic evidence that complements the accuracy gains, showing why the method works rather than just that it works. The GRAD-CAM visualizations (Figure 4) similarly provide qualitative evidence of the intended behavioral change. The reporting of hyperparameters in Table 8 is commendable for reproducibility and for the insights it provides about bias structure across datasets.

Methodological weaknesses and missing experiments

Single architecture (ResNet-18/ResNet-50): All experiments use ResNet architectures. Whether SiFER's effectiveness transfers to other architectures (Vision Transformers, EfficientNets, mobile-scale networks) is untested. The simplicity bias mechanism (simple features learned early in the hierarchy) is well-characterized for CNNs but may manifest differently in transformer architectures where the notion of "early layers" is less straightforward. Testing on at least one non-ResNet architecture would substantially strengthen the generality claim.

No computational cost analysis: The paper claims SiFER is "more computationally parsimonious" than ensemble methods (Section 2.2), but provides no runtime measurements, FLOPs counts, or memory comparisons. The auxiliary network adds parameters and the alternating training schedule adds complexity (freezing/unfreezing, selective gradient propagation). Quantifying this overhead and comparing it against the cost of ensemble methods (ESB trains N models, LWBC trains a committee) or data augmentation methods (Roadblock generates adversarial examples) would allow practitioners to assess the cost-benefit tradeoff. An ideal comparison would show accuracy as a function of total training FLOPs or wall-clock time for SiFER vs. the most competitive baselines.

Limited hyperparameter search description: The paper specifies the search ranges (Table 7) but not the search strategy (random search? grid search? how many trials?) or the computational budget for tuning. If SiFER required substantially more hyperparameter trials than baselines to achieve its reported performance, the comparison would be unfair. The small standard deviations on most results (e.g., BAR-OOD: 72.08 ± 0.38%) suggest that the method is stable once hyperparameters are chosen, but the cost of finding those hyperparameters is not accounted for.

No ablation of individual SiFER components: As noted in the Ablation Studies section, the paper lacks crucial component ablations. Most critically, there is no experiment where α₃ = 0 (disabling the forgetting loss entirely) while keeping the auxiliary network attached — this would test whether the auxiliary network alone (perhaps acting as a regularizer or providing additional supervisory signal) accounts for any of the gains, independent of the feature erasure mechanism. There is no experiment comparing interleaved vs. simultaneous optimization of identify and forget losses. There is no experiment testing whether the uniform pseudo-label (y_ep = [1/n, ..., 1/n]) is superior to alternative forgetting targets (e.g., a random class label, a constant incorrect class, maximum entropy with a different distribution).

Single validation set per dataset: The reported results use a single validation split per dataset, with hyperparameters selected on that split. There is no cross-validation across multiple validation folds to quantify the variance due to validation set composition — important for BAR-ID val where the validation set is constructed by subsampling training data, and the specific 20% held out could affect hyperparameter selection.

Test set sizes are moderate but not tiny: BAR uses a test set of unspecified size (likely a few hundred images); CelebA has thousands of test images but the minority group (blond-haired men) is extremely small; ImageNet-A has 7,500 images but they are handpicked adversarial examples rather than a representative sample. The statistical power for detecting small differences between methods varies substantially across datasets, and the reported standard deviations (based on 5 seeds) capture training stochasticity but not test-set sampling variance (since the test set is fixed).

Bottom-line assessment

The experiments provide strong evidence that SiFER is an effective debiasing method across a diverse set of visual recognition benchmarks, with particularly impressive results on BAR (where training data has zero conflicting examples) and ImageNet-A (where texture bias is the dominant failure mode). The mechanistic evidence (decodability analyses, GRAD-CAM) corroborates the accuracy gains and shows that SiFER works through the intended mechanism of suppressing spurious features and enhancing target features.

The claims about "no prior knowledge" and "controllable feature tradeoff" are supported with important qualifications: the method still requires a validation set that provides some generalization signal (and for CelebA, that signal requires group annotations), and the controllability is demonstrated only on synthetic data. The superiority over label-using baselines is supported in specific cases but is not uniform — on CelebA Unbiased Accuracy, the best label-using method (EnD) retains a small advantage.

The most significant missing experiments are component ablations (especially α₃ = 0), architectural generality tests (beyond ResNet), and computational cost quantification. Without the α₃ = 0 ablation, we cannot be certain that the feature erasure mechanism — as opposed to some other aspect of the auxiliary network setup — is responsible for the gains. This is not a fatal weakness, since the decodability and GRAD-CAM evidence strongly suggests that feature suppression is occurring, but it leaves a gap in the causal chain between the method and the results.

For a practitioner deciding whether to adopt SiFER, the key takeaways from the experimental analysis are: (1) SiFER provides substantial debiasing gains across multiple bias types, with particularly large improvements when training data is heavily biased (BAR: +30 percentage points over ERM) and when the bias is low-level (ImageNet-A: +15 points); (2) SiFER does not require explicit bias attribute labels for training, though a validation set with appropriate coverage of bias-conflicting examples or group annotations substantially helps hyperparameter tuning; (3) the computational overhead (auxiliary network, alternating optimization) is likely modest compared to ensemble methods but has not been quantified; (4) the method is architecturally specific to ResNet-style CNNs in its current form, and generalization to other architectures is untested.

6. Limitations and Trade-offs

The Method Requires a Validation Set That Provides Signal About the Bias Structure

The assumption or constraint. SiFER's entire hyperparameter selection protocol depends on a validation set whose metric (accuracy, unbiased accuracy, or OOD accuracy) provides a meaningful signal about which features are spurious and should be suppressed. The paper explicitly structures the choice of validation metric per dataset (Table 1): BAR uses accuracy on an OOD or ID validation split; CelebA uses Unbiased Accuracy, which requires group annotations; NICO uses accuracy on a validation set containing unseen contexts; ImageNet-9 uses ID validation accuracy. There is no mechanism for SiFER to operate when no held-out validation data of any kind is available, or when the available validation data is drawn from the same biased distribution as the training data and provides no generalization signal whatsoever. The paper acknowledges that SiFER's controllability is "driven by the choice of the validation data" (Section 5.2), but does not explore what happens when that choice is uninformative.

The consequence. In truly blind deployment scenarios — where a practitioner has a single biased dataset with no held-out data, no group annotations, and no knowledge of what the bias might be — SiFER's hyperparameters cannot be tuned via validation error, and the reported gains may not transfer. The validation loop is not merely a convenience; it is load-bearing. It selects which features to suppress (via A_P, A_D), how aggressively (via α₃), and at what temporal granularity (via F). Without a validation signal that rewards suppressing the right features, the method has no way to distinguish between sieving genuine spurious features and sieving genuinely useful ones. The paper's BAR-ID val experiment comes closest to this regime — using 20% of training data as validation, containing 0% bias-conflicting examples — and SiFER still achieves 65.75% (Table 3). But even there, the validation set provides signal: it measures accuracy on in-distribution data, and suppressing background features apparently improves ID accuracy on BAR because the background is genuinely unhelpful even within the training distribution. This may not hold for other biases where the spurious feature is useful in-distribution and only harmful OOD — if ID validation accuracy cannot distinguish between a model that uses the spurious feature and one that uses the genuine feature, the tuning loop cannot steer SiFER away from the spurious feature.

What evidence exists in the paper. The comparison between BAR-OOD and BAR-ID val settings (Table 3) provides indirect evidence: using OOD validation (which directly reveals the bias structure) yields 72.08% accuracy, while using ID validation (which does not) yields 65.75%. The 6.33 percentage point gap quantifies the value of having a bias-revealing validation set. The CelebA experiment is the clearest case of a strong validation requirement: the Unbiased Accuracy metric used for tuning (Table 1) requires group annotations (knowing both hair color and gender for every validation example). The paper does not report CelebA results using standard accuracy for hyperparameter tuning — an experiment that would reveal whether SiFER's gains on CelebA depend on having group-labeled validation data. The controllability experiments (Table 2) also depend on a constructed validation set that deliberately randomizes one feature — this is a clean demonstration of the principle, but it underscores that the validation set must be informative about which feature is relevant.

Mitigation status. The paper does not address the scenario of fully blind deployment. There is no proposed heuristic for setting hyperparameters without a validation set, no sensitivity analysis showing how hyperparameter choices affect performance (e.g., how much worse is SiFER with "reasonable default" hyperparameters vs. tuned ones), and no discussion of whether the gains are robust to validation set composition. The BAR-ID result partially mitigates the concern for cases where the spurious feature is harmful even in-distribution, but does not guarantee SiFER would work when the spurious feature is only harmful OOD and ID validation provides no distinguishing signal. The paper does not suggest future work on validation-free hyperparameter selection or on training a difficulty/bias predictor that could replace the validation loop.

The Method Is Evaluated on a Single Model Architecture (ResNet) Across All Real-World Experiments

The assumption or constraint. Every real-world experiment in the paper (BAR, CelebA, NICO, ImageNet-9/ImageNet-A) uses ResNet-18 as the base architecture, with ResNet-50 used only for the layerwise decodability analysis on CIFAR_MNIST (Figure 3). The auxiliary network uses the same BasicBlock structure as ResNet. The paper does not test SiFER on Vision Transformers, EfficientNets, mobile-scale architectures, or any non-ResNet CNN family. The core mechanistic assumption — that simple features are computed early in the network and proliferate to deeper layers — is well-characterized for convolutional networks with a clear hierarchical structure, but may not transfer straightforwardly to architectures with different representational organizations (e.g., transformers, where "depth" corresponds to self-attention layers rather than spatial resolution reduction, and features at different layers may not map cleanly to a simple-to-complex hierarchy).

The consequence. A practitioner using a non-ResNet architecture cannot know whether SiFER's gains will transfer. Vision Transformers, for instance, process images through self-attention from the first layer, potentially making global texture statistics and shape information accessible at similar depths — the "simple features at early layers" assumption may not hold. Similarly, architectures with skip connections that bypass intermediate layers (DenseNet, U-Net) may route simple features around the sieve, making it ineffective. The paper's decision to always attach the auxiliary network at the output of a ResNet layer block (positions 1, 2, or 3) presupposes a block-structured architecture with clearly defined intermediate representations. How to attach the auxiliary network to a ViT (between self-attention blocks? after the patch embedding?) is not obvious. The lack of architectural diversity means that the reported generality of SiFER — the paper frames it as a general method for "deep networks" (title, abstract) — is empirically supported only for one family of deep networks.

What evidence exists in the paper. Table 8 shows that the optimal auxiliary attachment position varies across datasets (A_P = 1 for NICO, A_P = 2 for BAR and CelebA, A_P = 3 for ImageNet-9), indicating that the depth at which spurious features become discriminatively accessible is dataset-dependent even within ResNet-18. This variation itself suggests architectural sensitivity: if the optimal position depends on the specific depth of ResNet layers, it will likely depend on the architecture's representational structure as well. The paper provides no ablation testing SiFER on a different CNN family (e.g., VGG, MobileNet) with the same datasets, and no discussion of challenges in adapting the auxiliary attachment mechanism to non-block-structured architectures.

Mitigation status. Not addressed. The paper does not claim architectural generality explicitly, but the title ("Overcoming Simplicity Bias in Deep Networks") and abstract imply it. No future work is suggested on architectural generalization. The paper does not discuss which properties of an architecture are required for SiFER to be effective (hierarchical organization? block structure? convolutional inductive bias?), which would help practitioners assess whether SiFER is applicable to their architecture of choice.

The Computational Cost of the Alternating Training Procedure Is Not Quantified

The assumption or constraint. SiFER adds several sources of computational overhead to standard ERM training: (1) the auxiliary network, which has 1–9 additional BasicBlock layers (each with two convolutions, batch norms, and a residual connection), must be trained during every standard iteration; (2) every F iterations, an additional backward pass through the frozen auxiliary network and the early main-network layers is performed for the forgetting loss; (3) the freezing/unfreezing of parameter subsets (auxiliary network parameters, early vs. late main network parameters) requires bookkeeping that may reduce training throughput. The paper does not report training time, FLOPs per iteration, peak memory usage, or total training cost for any experiment. The paper describes SiFER as "more computationally parsimonious" than ensemble methods (Section 2.2) and Roadblock (Section 2.2), but provides no numbers to support this comparison.

The consequence. A practitioner cannot assess whether SiFER's accuracy gains justify its computational overhead relative to simpler debiasing methods. If SiFER takes 2× longer to train than ERM but provides a 30-point gain on BAR, the tradeoff is clearly worthwhile. If it takes 5× longer and provides only a 1–2 point gain over a simpler method (e.g., data augmentation, which has no training-time overhead beyond the forward passes for augmented samples), the cost-benefit calculus shifts. The lack of cost quantification also makes it impossible to compare SiFER fairly against ensemble methods: ESB trains N complete models (a clear N× cost multiplier), while LWBC trains a committee. If SiFER's overhead approaches that of training 2–3 models, the claim that it is "more computationally parsimonious" may not hold. Additionally, the hyperparameter search cost (sweeping 5 parameters over the ranges in Table 7, multiplied by the number of datasets and validation splits) is not reported, but it represents a substantial hidden computational cost that is not amortized into the reported results.

What evidence exists in the paper. None. The paper provides no runtime, FLOPs, parameter count, or memory measurements for SiFER, ERM, or any baseline. The hyperparameter values in Table 8 indicate that F ranges from 30 to 70, meaning forgetting steps occur every 30–70 iterations — frequent enough that the additional backward passes could meaningfully increase training time, but the magnitude is unknown. The auxiliary network depth ranges from 1 to 4 in the tuned configurations (Table 8: most use A_D = 2 or 4), representing a modest parameter overhead relative to ResNet-18's ~11M parameters, but the impact on training throughput (which also depends on memory access patterns, gradient flow, and the freezing/unfreezing overhead) is not measured.

Mitigation status. Not addressed. The paper makes unquantified claims about computational parsimony relative to Roadblock and ensemble methods (Section 2.2) without evidence. No future work is suggested on reducing the computational cost of the alternating training procedure or on amortizing the hyperparameter search. For a methods paper, this is a significant omission — deployability depends not only on accuracy but on the resources required to achieve it.

The Method Has No Mechanism for Handling Hard Problems Where the Base Model Lacks Fundamental Capability

The assumption or constraint. SiFER operates by suppressing features that the auxiliary network identifies as predictive at a given network depth, forcing later layers to extract discriminative information from whatever residual signal survives the sieve. This mechanism fundamentally depends on the existence of alternative predictive features somewhere in the network's representational capacity — the sieve can only force the network to discover and amplify features that are already latent in the data and learnable by the architecture. If the base model genuinely lacks the capacity or the architectural inductive biases to extract the genuine feature (e.g., the model cannot recognize object shape because its receptive fields are too small, or the training data contains no shape signal), then erasing the spurious feature will simply destroy accuracy without enabling the learning of a replacement. The paper observes this failure mode implicitly in the context of the hardest difficulty bins — but in the image debiasing setting, the analogous limitation would be datasets where the "genuine" feature is genuinely not learnable by ResNet-18 from the available training data.

The consequence. SiFER is not guaranteed to improve robustness — it is only guaranteed to force the network not to use the features identified at the auxiliary attachment depth. Whether this forcing results in improved generalization depends on whether the network can find alternative features that are actually generalizable. In cases where the spurious feature is the only learnable signal in the training data (e.g., a medical imaging dataset where disease labels are spuriously correlated with hospital-specific scanner artifacts, and genuine pathological features are too subtle for ResNet-18 to detect from the available data), SiFER will suppress the spurious feature and the network may fail to learn anything useful, collapsing to near-chance accuracy. The paper's validation-guided tuning partially mitigates this: if suppressing features hurts validation accuracy, the tuning loop will select weaker suppression (lower α₃, deeper A_P, higher F) or effectively disable the sieve. But this mitigation only works when the validation set correctly signals that the suppressed features were genuinely necessary — if the validation set shares the same bias as the training data, suppressing spurious features may improve validation accuracy (because the spurious features are noisy even in-distribution, as on BAR), but the network may still fail to learn the genuine feature if it is not learnable, leading to a model that is simply inaccurate everywhere.

What evidence exists in the paper. The paper does not report any experiment where SiFER fails to improve over ERM or where it degrades performance. This is a conspicuous absence: the method is presented as uniformly beneficial across all five real-world benchmarks. There is no negative result, no dataset where SiFER's gains are negligible or negative, and no analysis of the conditions under which the sieve might be counterproductive. The controllability experiments (Table 2) show that SiFER can be steered to suppress either feature, but both features are perfectly predictive and learnable by design — this does not test the regime where one feature is not learnable. The decodability analyses (Figures 3, 5) show that SiFER enhances the decodability of the complex/target feature, but these datasets were specifically chosen because the target feature is learnable — the paper does not test on a dataset where the target feature is genuinely beyond the model's capability.

Mitigation status. Partially addressed through the validation tuning loop. If suppressing features genuinely harms generalization (as measured on the validation set), the tuning procedure selects a configuration that applies less suppression. But this safety mechanism depends on the validation set being representative of the deployment distribution — if both training and validation share a spurious correlation that is the only learnable signal, the tuning loop has no way to detect that the sieve is destroying the model's ability to learn anything. The paper does not discuss this failure mode, does not propose diagnostics for detecting it (e.g., monitoring training accuracy to see if the network is failing to fit even the training data), and does not suggest future work on adaptive suppression that backs off when the network cannot find alternative features.

The "No Prior Knowledge" Claim Is Weakened by the Requirement for Group-Annotated Validation Data on CelebA

The assumption or constraint. The paper's central methodological claim is that SiFER "do[es] not depend on prior knowledge of spurious attributes or features" (Abstract) and "do[es] not use foreknowledge of biased features / input dimensions" (Section 1). This claim is qualified in the experimental setup: for the CelebA Hair dataset, the hyperparameter selection metric is Unbiased Accuracy (Table 1). Computing Unbiased Accuracy requires knowing the joint distribution of the target label (hair color) and the spurious attribute (gender) for every validation example — specifically, it requires computing accuracy separately for each of the four hair-color×gender groups and averaging them. The paper is transparent about this: "Unbiased means accuracy averaged over each label-context group" (Section 4.3). This means that gender labels are used during hyperparameter selection, even though they are not used during training. This is a form of prior knowledge of the spurious attribute — not as strong as using gender labels in the training loss (as EnD and CSAD do), but still a dependency on knowing what the spurious attribute is and having it labeled in the validation set.

The consequence. The claim that SiFER outperforms methods using explicit bias attribute labels (Section 1, Section 5.3) is partially undermined for the CelebA benchmark. SiFER's 89.00% Unbiased Accuracy on CelebA (Table 4) is achieved using gender labels for hyperparameter tuning — making it not strictly comparable to a hypothetical method that uses no gender information whatsoever. A fairer comparison would test SiFER with hyperparameters tuned on standard accuracy (which does not require group labels) on CelebA, and report both standard accuracy and the resulting Unbiased/Conflicting accuracy on the test set. If SiFER tuned on standard accuracy still achieves 89.00% Unbiased Accuracy, the "no prior knowledge" claim is vindicated. If the Unbiased Accuracy drops significantly (e.g., to the mid-80s), then the reported result depends on group-labeled validation data, and the comparison against methods that do not use group labels is less clean. More broadly, the "no prior knowledge" framing conflates two distinct claims: (a) SiFER does not use spurious attribute labels during training (true for all experiments), and (b) SiFER does not use spurious attribute labels at all (false for CelebA due to the Unbiased Accuracy metric). The paper would benefit from distinguishing these claims explicitly and reporting both.

What evidence exists in the paper. The paper's Table 1 explicitly states that the validation metric for CelebA is Unbiased Accuracy, which requires group annotations. Table 4 marks SiFER with ✗ in the "Spurious Attribs" column, indicating no spurious attribute labels are used — but this refers to training-time usage, not validation-time usage. The BAR-ID val experiment (using standard accuracy on an ID validation split with no group annotations) provides a clean test of the no-prior-knowledge claim, and SiFER achieves 65.75% there (Table 3). However, the CelebA result — one of the paper's four headline real-world benchmarks — does not have the same purity. The paper does not discuss this distinction, does not report CelebA results with standard-accuracy-tuned hyperparameters, and does not acknowledge the reliance on group-annotated validation data as a qualification to the "no foreknowledge" claim.

Mitigation status. Not addressed. The paper could partially mitigate this by noting that the BAR-ID result provides cleaner evidence for the no-prior-knowledge claim, and that group-annotated validation data is a weaker requirement than group-annotated training data (since validation sets are typically orders of magnitude smaller and can be manually annotated with protected attributes at acceptable cost). But the paper does not make this argument. The CelebA Unbiased Accuracy metric is presented without comment on its implications for the "no prior knowledge" framing. A straightforward future experiment — tuning SiFER on standard CelebA accuracy and reporting the resulting Unbiased Accuracy — would directly address this concern but is not performed.

The Feature Sieve's Mechanism Has No Inherent Guard Against Erasing Genuinely Useful Features That Are Also "Simple"

The assumption or constraint. SiFER's core mechanism identifies features that are predictive at a given network depth and erases them, regardless of whether those features are spurious or genuinely useful. The auxiliary network, trained to maximize its classification accuracy on the training data, will latch onto any feature that helps it predict the label — if a simple, low-level feature is actually the causally correct signal for the classification task (e.g., color is genuinely diagnostic in a task classifying ripe vs. unripe fruit; texture is genuinely diagnostic in a task classifying fabric types), the auxiliary network will identify it, and the forgetting loss will erase it. The paper acknowledges this in Section 3.1: "Depending on the dataset, a 'simple' feature may in fact be the best / most unbiased predictive feature." The validation tuning loop is supposed to prevent this — if erasing a genuinely useful simple feature hurts validation accuracy, the tuning will select a configuration that erases less aggressively. But this safety mechanism relies entirely on the validation set correctly reflecting the deployment distribution. If the validation set is drawn from the same biased distribution as the training data, suppressing a genuinely useful feature that happens to be simple will not be penalized (since the feature is useful in-distribution too), and the sieve may inadvertently degrade the model's ability to use a valid predictive signal.

The consequence. SiFER embodies a structural inductive bias: that features computable at earlier network layers with limited capacity are more likely to be spurious than features requiring deeper processing. This bias is empirically well-motivated by the simplicity bias literature (Shah et al., 2020; Geirhos et al., 2018), but it is not universally true. In domains where the genuinely causal features are simple and low-level (e.g., certain medical imaging tasks where disease indicators are local texture patterns; certain quality control tasks where defects are color-based), SiFER will work against the correct solution, suppressing the valid signal and forcing the network to find alternative, potentially less reliable features. The validation tuning loop can only detect this if the validation set provides a countervailing signal (e.g., OOD examples where the simple feature is not predictive). Without such a signal, SiFER could produce a model that is less accurate and less robust than ERM — not because it fails to suppress spurious features, but because it successfully suppresses features that were actually correct.

What evidence exists in the paper. None directly. All five real-world benchmarks were chosen because they have known, well-characterized spurious correlations where the "simple" feature is indeed spurious (color/texture/background/gender context is not causally related to the target label). The paper does not include a dataset where the simple feature is genuinely the correct signal — a "negative control" that would test whether SiFER degrades performance in that regime. The controllability experiment (Table 2) comes closest: when the validation set is constructed to make the simple feature (color) the target, SiFER can be steered to preserve it. But this still relies on the validation set providing explicit signal about which feature is desired — it does not test what happens when the validation set is neutral (both features are predictive in both training and validation) and the simple feature happens to be the correct one. The paper does not discuss how to diagnose whether SiFER is suppressing useful vs. spurious features in a given deployment, and does not propose interpretability tools or diagnostic metrics for this purpose.

Mitigation status. Partially addressed through the validation tuning loop, but the paper does not discuss the limitations of this mitigation. The observation in Section 3.1 — that a simple feature may be the best predictive feature — is noted but not explored experimentally. No experiment tests SiFER on a task where the simple feature is the correct one. No diagnostic is proposed for detecting when the sieve is being counterproductive. The paper does not suggest future work on methods that could distinguish between useful-simple and spurious-simple features without relying on a validation set that already encodes this distinction.

7. Implications and Future Directions

How This Work Changes the Landscape

SiFER introduces a representational-level reframing of debiasing that shifts the field's focus from output-level correction to intra-network feature competition. This is not merely an incremental improvement over prior debiasing methods — it represents a qualitatively different class of intervention. Prior work operates almost exclusively at the data level (resampling, reweighting, augmentation), the output level (ensemble diversity, worst-case optimization, alternate-network disagreement), or the architecture level (inductive biases, architecture search). SiFER is the first method to directly intervene inside the network's representational hierarchy to identify and suppress specific features at the depth where they first become computationally accessible, based on a mechanistic understanding of how simplicity bias propagates — simple features learned early crowd out complex features in later layers through gradient competition.

This reframing has three cascading effects on the field:

First, it provides a unified explanation for contradictory findings in the debiasing literature. Prior work produced conflicting results about whether debiasing without attribute labels is possible: LfF and LWBC showed gains on some benchmarks but struggled when training data lacked bias-conflicting examples; ESB showed that ensemble diversity helps but provided no mechanistic account of why; methods requiring multiple environments (IRM, REx) showed promise but were brittle to environment construction choices. SiFER's framework reconciles these findings by identifying where in the network different biases manifest. The paper's Table 8 provides direct evidence: NICO's background-context bias is detectable at layer 1 (very early, low-level features), CelebA's gender bias manifests at layer 2 (mid-level semantic features), and ImageNet-9's texture bias requires layer 3 (relatively deep processing). A method that only intervenes at the output level — upweighting bias-conflicting samples, enforcing prediction diversity — would treat all three biases identically, missing the opportunity for targeted suppression at the specific depth where each bias emerges. SiFER's success across diverse bias types (Table 3–6) while using a unified mechanism suggests that depth-targeted feature erasure is a more fundamental solution than task-specific debiasing heuristics.

Second, it opens a new axis for neural network training methodology: dynamic, adversarial representational barriers. The alternating identify-and-erase dynamic (Algorithm 1) is not merely a trick for debiasing — it is a general training paradigm where an auxiliary module periodically identifies what solutions the network is currently exploiting and erases them, with temporal separation preventing gradient conflict. This paradigm could generalize to problems beyond debiasing: preventing shortcut learning in reinforcement learning (where agents exploit simulator quirks), encouraging compositional generalization in language models (where models memorize surface statistics), or forcing hierarchical feature learning in self-supervised representation learning (where models collapse to low-level pixel similarities). The key insight — that you can force a network to develop richer representations by periodically deleting the easiest features and giving the network time to adapt before the next deletion — is transferable to any domain where "easy" solutions exist and "hard" solutions are learnable but under-exploited.

Third, it challenges the assumption that bias-conflicting training examples or explicit bias labels are necessary for effective debiasing. The BAR-ID val result (65.75% accuracy vs. ERM's 35.32%, Table 3) demonstrates a ~30 percentage point gain in a setting with zero bias-conflicting training examples, zero attribute labels, and zero OOD validation signal. This is achieved through a purely structural inductive bias: features that are computable early in the network with limited capacity and that are predictive on the training distribution are more likely to be spurious, and suppressing them forces the network to discover alternative features. This structural inductive bias — grounded in the theoretical literature on simplicity bias (Shah et al., 2020) and gradient starvation (Pezeshki et al., 2021) — proves sufficient to drive substantial debiasing without requiring any examples where the bias is violated. For practitioners in domains where bias-conflicting data is impossible to collect (e.g., historical datasets where the spurious correlation was universal in the data-generating process), this is a transformative finding: the network's own training dynamics provide enough signal to identify and suppress spurious features, at least for the visual biases evaluated in this paper.

The paper also changes which research directions look promising versus dead-end. Prior to this work, a natural trajectory was to build increasingly sophisticated bias-identification modules — better methods for inferring bias groups from data (EIIL, PGI), better reweighting schemes (LWBC, Debian), better diversity constraints (ESB). SiFER's results suggest that further sophistication in bias identification at the output level may yield diminishing returns compared to depth-targeted representational interventions. The fact that SiFER outperforms Debian (which uses an explicit bias discoverer network) on BAR (72.08% vs. 69.88%, Table 3) and matches or exceeds label-using methods on CelebA (Table 4) suggests that knowing what feature to suppress with high precision matters less than knowing where in the network to intervene. This redirects research attention from better bias classifiers to better characterizations of the depth profile of spurious features — understanding at what layer different types of spurious correlations become discriminatively accessible, and designing architectures or training procedures that systematically inhibit feature propagation at those depths.

Follow-Up Research This Work Enables

Characterizing the depth profile of spurious features across architectures, tasks, and modalities. The paper's Table 8 provides an intriguing but preliminary finding: the optimal auxiliary attachment position varies from layer 1 (NICO, background context) to layer 3 (ImageNet-9, texture bias). This raises a fundamental question: can we predict, from the nature of the spurious correlation, at what network depth it will become discriminatively accessible? A systematic study would train probes at every layer of multiple architectures (ResNet, ViT, ConvNeXt) on multiple bias types (color, texture, background context, demographic attributes, synthetic features of controlled complexity) and measure the layerwise decodability of spurious vs. target features throughout training. This would produce a "feature depth map" that could guide practitioners in setting A_P without expensive hyperparameter search, and would test whether the relationship between bias type and emergence depth is architecture-invariant or architecture-specific. A strong negative result — finding that depth profiles are highly architecture-dependent with no consistent mapping from bias type to depth — would indicate that validation-guided A_P tuning is essential and cannot be replaced by heuristics, an important practical finding in its own right.

Ablation of the interleaving dynamic: alternating vs. simultaneous vs. scheduled-free optimization. SiFER's core innovation is not just the forgetting loss but the interleaving of identification and erasure (controlled by F). The paper motivates this choice conceptually (Section 3.2: preventing gradient conflict) but provides no ablation testing it. A direct follow-up would compare three training regimes on BAR and CelebA: (1) SiFER's alternating schedule (standard for F iterations, then forget), (2) simultaneous optimization (apply L_1 + L_2 + L_f at every iteration, with L_f using a frozen copy of the auxiliary network updated periodically), and (3) a scheduled-free approach where L_f weight is annealed from zero to α_3 over training. The comparison would reveal whether the temporal dynamics are load-bearing or whether the same representational outcome can be achieved through simpler means. If simultaneous optimization fails (gradients conflict, training diverges), this validates a key design choice and establishes the alternating paradigm as necessary for representational-barrier methods. If it succeeds, SiFER can be simplified substantially.

Extending the feature sieve to generative models and self-supervised learning. The paper focuses exclusively on supervised classification, but simplicity bias manifests in generative and self-supervised settings as well — e.g., VAEs that ignore latent factors of variation in favor of pixel-level reconstruction, or contrastive learning methods that collapse to texture-based invariances. The natural extension is to attach an auxiliary network to an intermediate layer of a VAE encoder or a SimCLR backbone, train it to predict some known attribute (or a surrogate task) from the intermediate representation, and apply the forgetting loss to erase that attribute. The goal would be to produce representations that are invariant to specified spurious factors without requiring paired or group-labeled data — the auxiliary network discovers the spurious factor from the training data itself. The NICO and ImageNet-9 results (Tables 5–6, showing gains on domain-shift and texture-shift generalization) suggest that the sieve can suppress features that are not explicitly labeled but are discovered through their predictiveness at specific depths. Testing this in self-supervised settings would assess whether the sieve's discovery mechanism generalizes beyond the supervised cross-entropy signal to contrastive or reconstruction-based training objectives.

Combining the feature sieve at multiple depths simultaneously. The paper attaches the auxiliary network at a single depth (A_P) and erases features only at that depth and earlier layers. But the results in Table 8 show that different biases manifest at different depths in different datasets — what if a single dataset contains multiple spurious correlations at different depths? For instance, a face recognition dataset might have both low-level color bias (skin tone correlated with background) and mid-level attribute bias (gender correlated with occupation). Attaching sieves at multiple depths — say, one at layer 1 targeting low-level texture/color features and another at layer 3 targeting semantic attribute features — and applying forgetting losses at both positions could suppress multiple bias types simultaneously. The key design challenge is preventing the sieves from interfering with each other: if the layer-1 sieve erases color features, the layer-3 sieve may never form gender features because its input is already degraded. A systematic study with synthetic data containing two spurious features at known complexity levels (e.g., color at low complexity, shape at high complexity, with both spuriously correlated with labels in different ways) would reveal whether multi-depth sieving is feasible or whether sieves at different depths compete destructively.

Negative result: deploying SiFER on a dataset where the simple feature is the correct feature. The paper acknowledges (Section 3.1) that a simple feature may be the best predictive feature, but tests only on datasets where the simple feature is known to be spurious. A definitive stress-test would construct a dataset where the correct feature is simple (e.g., classifying ripe vs. unripe fruit where color is causally diagnostic, with shape or background as spurious distractors) and the spurious feature is complex (e.g., the fruit appears in different contexts that are correlated with ripeness in training but not at test time). If SiFER, tuned on in-distribution validation accuracy, suppresses the simple-but-correct color feature and forces the network to rely on the complex-but-spurious context feature, this would reveal a fundamental limitation: the sieve's structural inductive bias (simple features are likely spurious) can be actively harmful when the causal structure of the task inverts this relationship. If SiFER's validation loop correctly preserves the simple feature (because suppressing it hurts ID accuracy, since color is genuinely diagnostic), this would demonstrate that the validation loop is a sufficient safeguard and that SiFER does not blindly suppress all simple features. Either outcome is scientifically valuable — the former reveals a boundary condition, the latter strengthens the case for deployment without manual bias specification.

Theoretical analysis of the alternating optimization dynamics. The paper provides an intuitive account of why interleaving identification and erasure works (Section 3.2, 3.4), but offers no formal analysis. A theoretical follow-up could model SiFER's training dynamics in a simplified setting — e.g., a two-layer linear network with two features (one simple, low-rank; one complex, high-rank) where the simple feature is spuriously correlated with the label in training but not at test time. Analyzing the gradient flow under alternating identify-erase dynamics vs. standard ERM could characterize: (1) under what conditions the simple feature is suppressed (as a function of F, α_3, and the feature learning timescales), (2) whether the complex feature's learning is genuinely accelerated or merely unblocked, and (3) the equilibrium representational structure (how much simple-feature information survives in the converged network). This would connect SiFER to the theoretical literature on simplicity bias and gradient starvation, potentially yielding principled guidelines for setting hyperparameters based on measurable properties of the data (feature strength ratios, noise levels) rather than expensive validation sweeps.

Practical Applications and Downstream Use Cases

Fairness-sensitive face recognition and attribute classification. The CelebA result (Table 4: 89.00% Unbiased Accuracy, 88.04% Conflicting Accuracy) demonstrates that SiFER can substantially reduce gender bias in hair color prediction without requiring gender labels during training — only group-annotated validation data for hyperparameter tuning, which is typically orders of magnitude smaller than training data and can be manually annotated for protected attributes at acceptable cost. This has direct applicability to biometric systems, hiring platforms, and content moderation tools where models must make predictions about individuals without encoding sensitive demographic information. The GRAD-CAM visualizations (Figure 4) provide additional confidence: SiFER's focus on the central object rather than background context suggests that face attribute classifiers trained with SiFER would attend to facial features rather than hairstyle, clothing, or background — all of which can encode demographic information spuriously correlated with target attributes. A concrete deployment scenario: a company building an age-estimation system for age-gated content could train with SiFER on a dataset where age is spuriously correlated with image quality (older photos tend to be lower resolution), and the sieve would suppress resolution-based features at early layers, forcing the network to rely on genuine facial aging markers.

Medical imaging with scanner-specific artifacts. Medical imaging datasets frequently contain spurious correlations between scanner type, hospital site, or imaging protocol and disease labels — for example, if a hospital uses a particular MRI machine primarily for cancer patients, the model may learn to detect the machine's signature rather than pathological features. These biases are particularly dangerous because they are invisible to standard validation (both training and validation data come from the same hospitals) and only manifest when the model is deployed at a new site with different equipment. SiFER's BAR-ID val result (Table 3: 65.75% vs. ERM's 35.32%) is directly relevant: it shows that the sieve can achieve large debiasing gains even when validation data contains zero bias-conflicting examples and the practitioner has no knowledge of what the spurious features are. A radiologist deploying a chest X-ray classifier could train with SiFER using only in-distribution validation data (from the same hospitals as training), and the auxiliary network — attached at an early layer — would identify and suppress scanner-specific texture or intensity features that are spuriously correlated with disease labels in the training data. The paper's ImageNet-A result (39.98% vs. 24.9% for ERM, Table 6) demonstrates that suppressing texture bias improves robustness to naturally occurring distribution shifts, which is the exact requirement for cross-site medical imaging deployment.

Self-driving perception systems robust to environmental context. Autonomous vehicle perception models face a simplicity bias problem analogous to BAR: objects are spuriously correlated with backgrounds in training data (cars appear on roads, boats appear on water, pedestrians appear on sidewalks), and models that rely on background context fail on edge cases (a car in a flood, a boat on a trailer, a pedestrian in an unusual location). SiFER's BAR results (Table 3) and NICO results (Table 5: 86.20%, ~10 points over domain generalization baselines) demonstrate that the sieve can suppress background-context features and force the network to attend to the foreground object — exactly the visual attention shift shown in Figure 4, where SiFER focuses on the person rather than the climbing wall, snowy slope, or indoor gym. A perception team could integrate SiFER into their object detection training pipeline: attach the auxiliary network at an intermediate layer of the detection backbone, train it to predict object classes from intermediate features, and apply the forgetting loss to erase background-context information. The GRAD-CAM evidence suggests the resulting detector would focus on object shape and appearance rather than environmental context, improving robustness to rare scenes, adverse weather, and domain shifts between training geographies and deployment geographies.

When to Prefer This Method

The paper explicitly positions SiFER against two categories of alternatives — methods that require explicit bias attribute labels (Group-DRO, EnD, CSAD) and methods that require bias-conflicting training examples or OOD validation (LfF, LWBC, EIIL) — and argues for SiFER when these resources are unavailable. The decision rule can be articulated from the paper's own framing and experimental design:

Prefer SiFER over bias-label-requiring methods (Group-DRO, EnD, CSAD) when:

The spurious attributes are unknown, difficult to define, or expensive to label at scale — as in most real-world deployments where practitioners discover bias only after model failure (Section 2.2: "both identifying, and manually labeling biased instances and dimensions in real-life data are significant barriers").
The spurious feature is abstract, emergent, or continuous (e.g., texture statistics, image resolution, background context) rather than a discrete, nameable attribute — SiFER's operational definition of spuriousness (whatever is predictive at a given depth) handles these cases without human categorization.
The practitioner has a validation set with group annotations for protected attributes (as on CelebA, where Unbiased Accuracy tuning requires gender labels) but does not want to use those labels during training — SiFER restricts attribute-label usage to the validation set, which is typically much smaller and can be manually annotated.

Prefer SiFER over methods requiring bias-conflicting examples or OOD validation (LfF, LWBC, EIIL, ESB) when:

The training data contains zero or near-zero bias-conflicting examples — BAR-ID val (Table 3: 65.75% vs. ERM's 35.32%, with 0% conflicting train examples) directly demonstrates SiFER's effectiveness in this regime, while LfF, LWBC, and similar methods fundamentally require at least some conflicting examples to identify the bias.
Only in-distribution validation data is available — the BAR-ID result shows that SiFER can tune hyperparameters using standard accuracy on ID validation data and still achieve large gains, outperforming ESB (64.40%) and LWBC (62.03%) which also lack OOD validation.
The bias type is suspected to be low-level or texture-based — SiFER's ImageNet-A result (39.98% vs. LWBC's 35.97%, Table 6) shows particular strength against texture bias, likely because texture features are identified and erased at early layers where they first emerge.

Prefer bias-label-requiring methods over SiFER when:

The spurious attribute is known, easily labeled, and available for all training examples — EnD achieves 91.21% Unbiased Accuracy on CelebA vs. SiFER's 89.00% (Table 4), retaining a small advantage when gender labels are fully exploited during training.
The validation set cannot provide any bias-revealing signal — even group annotations for Unbiased Accuracy, or OOD examples for standard accuracy — because SiFER's hyperparameter tuning depends on a validation metric that rewards suppressing spurious features.

Prefer SiFER over ensemble methods (ESB, LWBC) when computational efficiency matters: The paper claims SiFER is "more computationally parsimonious" than Roadblock and ensemble methods (Section 2.2), though no quantitative cost comparison is provided. Qualitatively, SiFER trains a single model with a lightweight auxiliary network, while ESB trains an ensemble of diverse models and LWBC trains a committee — the single-model approach is intrinsically cheaper at equivalent hyperparameter tuning budgets. The ImageNet-9 result (Table 6: SiFER 97.78% vs. ESB not reported on this benchmark, but SiFER beats the best single-model methods) and BAR result (Table 3: SiFER 65.75% vs. ESB 64.40%) suggest that the single-model sieve matches or exceeds ensemble diversity approaches without the multiplicative training cost.