Read More

ArXiv: 2306.03819

🎯 Pitch

Linear concept erasure is solved in closed form: equalize class-conditional means to completely block any linear probe, then apply a surgically minimal transformation that preserves all other information—and applying this layer by layer reveals that large language models rely on linearly encoded syntax far less than prior work suggested, because existing erasure methods inflicting drastic collateral damage massively inflated those causal estimates.

---

1. Executive Summary

This paper introduces LEAst-squares Concept Erasure (LEACE), a closed-form method that provably prevents all linear classifiers from detecting a specified concept in an embedding while minimally changing the representation under any inner-product norm. The authors prove that linear guardedness—the inability of any linear predictor to outperform a constant baseline—is equivalent to equal class-conditional means, and from this equivalence they derive the unique surgically optimal affine transformation that erases linearly available information. Applied to BERT embeddings, LEACE reduces the TPR-GAP gender bias metric from 0.198 to 0.084 while preserving profession-prediction accuracy within 2 percentage points, and the paper further introduces concept scrubbing (sequentially applying LEACE to every layer of a deep network) to measure reliance on part-of-speech information in LLMs up to 30B parameters, establishing that autoregressive language models heavily depend on linearly encoded syntactic features—but that the measured effect size depends sharply on the erasure method used, with prior work inducing spurious collateral damage that inflates causal estimates.

2. Context and Motivation

The Core Problem: Removing Concepts from Representations Without Collateral Damage

The fundamental question this paper tackles is deceptively simple: given a learned representation $X$ (e.g., a word embedding or a hidden state in a neural network), how do we remove all information about some specified concept $Z$ (e.g., gender, race, part-of-speech) while preserving as much other information as possible? This matters for two distinct but converging reasons:

• Fairness: If a model's internal representations encode protected attributes like gender or race, downstream classifiers trained on those representations can inadvertently use those attributes to make discriminatory decisions. Popular notions of fairness, such as counterfactual fairness (Kusner et al., 2017), require that protected attributes should not causally affect predictions. Erasing the protected attribute from the representation is a direct way to enforce this constraint post hoc — after the model is trained, without modifying its parameters or retraining it.

• Interpretability: To understand how a neural network uses a concept, researchers often perform causal intervention experiments: they remove the concept from the model's internal activations and observe how the model's behavior changes (Elazar et al., 2021; Ravfogel et al., 2021; Dankers et al., 2022). If removing part-of-speech information causes a sharp drop in language modeling performance, that is evidence the model relies on syntactic structure. But the validity of this inference hinges on a critical assumption: that the erasure method removes only the target concept and nothing else. If the intervention inadvertently damages other useful features ("collateral damage"), we cannot attribute the performance drop to the absence of the target concept.

Thus, both fairness and interpretability applications demand the same thing: a concept erasure method that is both provably complete (no linear classifier can recover the concept) and minimally destructive (the edited representation is as close as possible to the original, preserving all other information).

The Gap: Prior Methods Either Fail to Erase or Destroy Too Much

Prior work on concept erasure falls into two broad categories, each with significant limitations that this paper systematically addresses.

1. Adversarial training methods (Edwards & Storkey, 2016; Xie et al., 2017; Elazar & Goldberg, 2018). These approaches modify the training objective to encourage the model to produce representations from which an adversary cannot predict the protected attribute. The problem, as Elazar and Goldberg (2018) demonstrated, is that these methods fail to exhaustively remove the concept. After adversarial training, it is often possible to train a new adversary — one not seen during training — that successfully recovers the supposedly erased information. The model learns to fool the specific adversary used during training rather than truly removing the concept from its representations.

2. Post-hoc linear methods (Ravfogel et al., 2020, 2022a; Shao et al., 2023a,b; Haghighatkhah et al., 2022). These approaches take a trained model and algebraically edit its representations to remove concept information. They identify a linear subspace that encodes the concept and then neutralize it. The most prominent example is Iterative Nullspace Projection (INLP) (Ravfogel et al., 2020), which repeatedly trains a linear classifier to predict the concept, projects the representations onto the nullspace of that classifier's weight matrix, and iterates until classification accuracy drops to chance. While INLP can eventually achieve linear guardedness, it suffers from two critical problems:

• It deletes far more dimensions than necessary. The paper reports that INLP "often requires dozens of iterations to completely erase the concept," and as a result, "decreases the rank just by 17" for the LEACE method applied to part-of-speech erasure, while INLP deletes 360 dimensions for the same task (Section 5.3). This is because INLP identifies and removes entire classifier weight directions at each iteration, many of which are redundant or only weakly correlated with the concept. Kumar et al. (2022) showed both empirically and theoretically that INLP causes "needless collateral damage to useful parts of the embedding that are orthogonal to the concept being erased."

• It cannot be applied to multiple layers. As the paper notes in Section 6, "it's impossible to apply INLP to multiple layers of a transformer without causing its outputs to collapse into gibberish." This is a direct consequence of the collateral damage problem: each layer's erasure compounds the damage from previous layers, and after a few layers the representation is so degraded that the model produces nonsense. This fundamentally limits the kinds of causal intervention experiments that are possible — researchers can only erase concepts from a single layer at a time, making it impossible to determine how information flows through the full depth of the network.

3. Other linear methods with incomplete theoretical guarantees. Beyond INLP, several other linear methods had been proposed — Spectral Attribute RemovaL (SAL; Shao et al., 2023a), Mean Projection (MP; Haghighatkhah et al., 2022), and Fair PCA (Kleindessner et al., 2023). The paper notes (Appendix D) that these methods, when configured with suitable hyperparameters, do achieve linear guardedness — but the authors of those prior works were unable to prove this fact, and none provided optimality guarantees. There was no theoretical framework establishing necessary and sufficient conditions for linear erasure, nor any characterization of the set of all erasure functions that achieve guardedness. This meant practitioners had no way to know whether their chosen method was optimal or whether a better method existed.

4. A theoretical void around optimal erasure. Perhaps most critically for the interpretability application: none of the prior methods could claim to be minimally destructive. The paper observes that "prior work on linear concept erasure has assumed that erasure functions should be orthogonal projections, appealing to the well-known fact that an orthogonal projection of a point onto a subspace yields the nearest point in that subspace to the original point" (Section 4.2). But this intuition, while natural, is mathematically incorrect for the concept erasure problem. The constraint on an erasure function is a nullspace condition — the function must kill the subspace that correlates with the concept — not a range condition. Orthogonal projections satisfy an unnecessarily restrictive symmetry constraint (the projection matrix must be symmetric) that prevents them from using information in the non-erased dimensions to reconstruct parts of the erased dimensions that are predictable from the rest of the data. The paper provides a concrete counterexample (Appendix F) showing that an oblique projection can reduce mean squared edit distance by 50% compared to the best orthogonal projection while achieving identical erasure.

Conflicting Demands: Completeness, Minimality, and Tractability

The paper's introduction implicitly frames a trilemma that prior work failed to resolve:

• Completeness: The method must provably prevent all linear classifiers from recovering the concept. Anything less means the concept is still available to downstream models.
• Minimality: The edit to the representation must be as small as possible, to avoid destroying other useful features. This is essential for both fairness (we don't want to hurt main-task accuracy) and interpretability (we don't want to attribute performance drops to collateral damage).
• Tractability: The method must be computationally efficient, ideally closed-form rather than requiring iterative optimization, and must scale to large models and datasets.

INLP and adversarial methods achieve completeness but at the cost of minimality and tractability (they are iterative and delete many dimensions). SAL and MP are tractable but had no optimality guarantees and were not proven to be minimally destructive. No prior method achieved all three simultaneously.

The Central Theoretical Insight: Linear Guardedness Is Exactly Equal Class Means

The paper's path to resolving this trilemma begins with a theoretical observation that had gone unnoticed in prior work despite decades of research on linear classification: a dataset is linearly guarded (no linear classifier can outperform a constant baseline) if and only if every class has exactly the same mean feature vector. This equivalence is established in Section 3 through a chain of proofs connecting five statements:

• Linear guardedness (the inability to improve upon the trivially attainable loss)
• Optimality of the constant predictor for all convex losses (including cross-entropy)
• Equality of all class-conditional means $\mathbb{E}[X \mid Z = i]$ to the global mean $\mathbb{E}[X]$
• Zero cross-covariance between $X$ and $Z$: $\mathrm{Cov}(X_i, Z_j) = 0$ for all $i, j$
• Linear statistical parity: every linear classifier evaluated on $X$ has equal expected output across all classes

This equivalence is powerful because it converts a complex optimization problem (find an erasure function that prevents all possible linear classifiers from recovering the concept) into a simple algebraic constraint (the erasure function must set the cross-covariance matrix to zero). Theorem 4.1 then characterizes exactly which affine transformations $r(\mathbf{x}) = \mathbf{P}\mathbf{x} + \mathbf{b}$ achieve this: those where the columns of $\boldsymbol{\Sigma}_{XZ}$ (the cross-covariance matrix) lie in the nullspace of $\mathbf{P}$.

Crucially, this nullspace constraint leaves enormous freedom in choosing $\mathbf{P}$. The condition only specifies what $\mathbf{P}$ must destroy; it says nothing about what $\mathbf{P}$ should preserve or how it should behave on dimensions orthogonal to the concept subspace. This is why prior methods, all of which used orthogonal projections, were suboptimal: they imposed an additional, unnecessary symmetry constraint ($\mathbf{P} = \mathbf{P}^\top$) that forced them to destroy more information than required.

The Optimization Criterion: Minimizing Mean Squared Edit Distance

Given the set of all linearly guarding affine transformations, the paper asks: which one minimally changes the representation? It operationalizes "minimal change" as the mean squared norm of the edit:

$\mathbb{E}\left[\|r(X) - X\|^2_{\mathbf{M}}\right]$

where $\mathbf{M}$ is any positive semi-definite matrix defining an inner product on $\mathbb{R}^d$. This is a local quadratic approximation to any divergence measure (Kullback-Leibler, for instance) and encompasses both Euclidean ($\mathbf{M} = \mathbf{I}$) and Mahalanobis ($\mathbf{M} = \boldsymbol{\Sigma}_{XX}^+$) distances as special cases. Remarkably, the paper shows that there is a single erasure function that minimizes all such norms simultaneously — a property that emerges from the whitening-based derivation and means the practitioner does not need to choose a distance metric.

The solution, LEACE (Equation 1), takes an intuitive form: whiten $X$, orthogonally project onto the subspace that correlates with $Z$ (in the whitened space, the constraint becomes an orthogonal projection), subtract that projection from $X$, then unwhiten. Figure 1 in the paper visualizes this three-step process. The resulting transformation $\mathbf{P}^* = \mathbf{I} - \mathbf{W}^+ \mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}} \mathbf{W}$ is an oblique projection — not symmetric in general — which is why it outperforms orthogonal projections: it can use information in the non-erased dimensions to partially reconstruct the erased dimensions, minimizing the edit while still zeroing out all correlation with $Z$ (see Appendix F for a worked example where this cuts the MSE in half).

How This Paper Positions Itself Relative to Prior Work

The paper does not claim to invent the idea of concept erasure or linear guardedness. What it claims to contribute is:

1. A complete theoretical characterization of the necessary and sufficient conditions for linear guardedness (Section 3), which had been missing from the literature. Prior methods like SAL and MP happened to achieve guardedness for specific hyperparameter settings, but their authors could not prove this, and no one had established the equivalence between zero cross-covariance and linear guardedness for general convex losses.

2. The identification of the unique surgically optimal erasure function within the space of all affine transformations that achieve linear guardedness (Section 4). This converts concept erasure from a heuristic into a well-posed optimization problem with a closed-form solution, unifying and improving upon prior methods.

3. A demonstration that the "make it an orthogonal projection" intuition is wrong for this problem (Section 4.2), and that oblique projections are both necessary and sufficient for optimality. This is not just a theoretical curiosity — it has practical consequences, as the concept scrubbing experiments show that LEACE's minimality translates to significantly less damage to downstream task performance compared to SAL (Table 1).

4. The introduction of concept scrubbing (Section 6), which exploits LEACE's minimal collateral damage to sequentially erase a concept from every layer of a deep network. This was impossible with INLP and enables new kinds of causal intervention experiments that probe how concepts are used throughout the full depth of a model.

The paper explicitly connects to the broader literature on $\mathcal{V}$-information (Xu et al., 2020) and the formal definition of guardedness from Ravfogel et al. (2023), generalizing their definitions to work with arbitrary convex losses (Appendix G). It also positions linear erasure as a tractable special case of the more general problem of information removal, acknowledging that nonlinear erasure is likely intractable in general (Section 7: "We conjecture that it is intractable to nondestructively edit $X$ so as to prevent a general nonlinear adversary from recovering $Z$, unless the data generating process for $X$ is known in detail"). This honesty about scope — claiming optimality within the linear regime and acknowledging the limits of linear methods for nonlinear concept encoding — strengthens the paper's credibility.

The Practical Motivation: Enabling Causal Interpretability at Scale

Beyond the theoretical contributions, the paper is motivated by a very concrete practical bottleneck in interpretability research. Elazar et al. (2021) introduced the influential "amnesic probing" paradigm: erase a concept from a single layer of a model, measure how behavior changes, and infer the concept's causal role. But as Section 5.3 explains, their experiments using INLP were severely limited:

• INLP required 20 iterations and deleted 360 dimensions to (imperfectly) erase an 18-class part-of-speech concept from BERT.
• The massive dimensionality reduction made it impossible to determine whether observed performance drops were due to losing part-of-speech information or to general representational collapse.
• It was impossible to apply amnesic probing to more than one layer simultaneously, preventing investigation of how concepts propagate through the full model.

LEACE solves these problems: it erases the concept using only 17 dimensions (the rank of the cross-covariance matrix for an 18-class problem), does so in a single closed-form step, and causes so little collateral damage that it can be applied sequentially to every layer — the "concept scrubbing" procedure. This directly enables the kinds of multi-layer causal intervention experiments that the interpretability community needs but previously could not perform.

In summary, the paper addresses a clear, well-motivated gap: we need concept erasure methods that are provably complete, optimally minimal, and computationally tractable. The theoretical framework of Sections 3–4 provides the first rigorous solution, and the concept scrubbing procedure of Section 6 demonstrates that the solution scales to the practical regime of large language models where prior methods simply break.

3. Technical Approach

3.1 Reader Orientation (Approachable Technical Breakdown)

This is a theoretical + methods paper whose core contribution is twofold: first, a rigorous characterization of what it means for a representation to be linearly uninformative about a concept (linear guardedness), and second, a closed-form formula (LEACE) that surgically edits any representation to achieve this property while minimizing the expected squared distance from the original. The paper solves the problem of provably complete, optimally minimal, and computationally efficient concept erasure by deriving the equivalence between guardedness and zero cross-covariance, then solving a constrained least-squares optimization over all affine transformations that satisfy this constraint.

3.2 Big-Picture Architecture (Diagram in Words)

The system described in the paper operates in two regimes — single-layer erasure and full-network scrubbing — but shares a common conceptual pipeline:

1. Data Collection — a dataset of representations $X \in \mathbb{R}^{n \times d}$ (embedding vectors or hidden states) and corresponding concept labels $Z \in \mathbb{R}^{n \times k}$ (one-hot encoded classes, e.g., gender or part-of-speech tags) is collected from the model to be edited. These are treated as jointly distributed random vectors.

2. Covariance Estimation — the cross-covariance matrix $\boldsymbol{\Sigma}_{XZ} \in \mathbb{R}^{d \times k}$ and the data covariance matrix $\boldsymbol{\Sigma}_{XX} \in \mathbb{R}^{d \times d}$ are estimated from the $(X, Z)$ pairs. These matrices encode all linear relationships between the representation and the concept.

3. Erasure Matrix Computation (LEACE) — using only the estimated covariance matrices, a projection matrix $\mathbf{P}^* \in \mathbb{R}^{d \times d}$ and bias vector $\mathbf{b}^* \in \mathbb{R}^d$ are computed in closed form via Equation 1. This step requires no iterative optimization — it is a single algebraic computation involving matrix pseudoinversion, whitening, and orthogonal projection.

4. Representation Editing — for any new input $\mathbf{x}$ (whether from the training distribution or not), the edited representation is produced as $\mathbf{x}' = \mathbf{P}^* \mathbf{x} + \mathbf{b}^*$. The edited $\mathbf{x}'$ is guaranteed to have $\mathrm{Cov}(\mathbf{x}', Z) = \mathbf{0}$ — no linear classifier can predict $Z$ from $\mathbf{x}'$ better than chance.

5. Concept Scrubbing (multi-layer extension) — for deep networks with $\ell$ layers, the above process is applied sequentially from layer 1 to layer $\ell$ (Algorithm 1): fit LEACE on the hidden states at layer $l$, immediately scrub those states, feed the scrubbed states into layer $l+1$, and repeat. This accounts for the distributional shift introduced by intervening on early layers.

The information flow is: raw activations → covariance estimation → closed-form erasure matrix → scrubbed activations (per layer, sequentially for scrubbing).

3.3 Roadmap for the Deep Dive

• First, the theoretical foundations (Section 3 of the paper) — the equivalence between linear guardedness, equal class-conditional means, and zero cross-covariance. This establishes what the erasure constraint actually is in algebraic terms, which is the prerequisite for everything else.
• Second, the characterization of all guarding affine transformations (Theorem 4.1) — once we know the constraint is $\mathbf{P} \boldsymbol{\Sigma}_{XZ} = \mathbf{0}$, we can describe the entire feasible set of erasure matrices.
• Third, the LEACE derivation (Section 4.1–4.2) — solving the constrained least-squares problem over that feasible set, showing why oblique projections are optimal (contrary to prior assumptions), and building intuition for the whitening–projection–unwhitening formula.
• Fourth, the Oracle LEACE variant (Appendix I) — a version that uses ground-truth concept labels at inference time to achieve an even more surgical edit, providing geometric intuition in a random-variable Hilbert space.
• Fifth, the concept scrubbing algorithm (Section 6) — the sequential multi-layer procedure and the practical considerations (norm regularization, streaming covariance estimation, memory-vs-compute tradeoffs).

3.4 Detailed, Sentence-Based Technical Breakdown

Theoretical Foundations: What Does "Linearly Guarded" Actually Mean?

The paper begins by formalizing the objective of concept erasure through the lens of guardedness, a concept borrowed from Ravfogel et al. (2023) and generalized here to work with arbitrary convex loss functions.

Definition 2.1 (Guardedness). Let $X$, $Z$, $\mathcal{V}$, and $\mathfrak{L}$ be as defined above, and let $\chi$ be the set of all random vectors of finite first moment taking values in $\mathbb{R}^d$, jointly defined with $Z$. We say $X$ $(\mathcal{V}, \mathfrak{L})-$guards $Z$ if, for all losses $\mathcal{L} \in \mathfrak{L}$, it maximizes the minimum expected loss: $X \in \arg\max_{X' \in \chi} \inf_{\boldsymbol{\theta} \in \Theta} \mathbb{E}[\mathcal{L}(\eta(X'; \boldsymbol{\theta}), Z)]$

In plain language: a representation $X$ guards the concept $Z$ if, among all possible representations jointly distributed with $Z$, our $X$ makes the best possible classifier's performance as bad as possible — it is maximally uninformative about $Z$ for the given predictor class $\mathcal{V}$. This is a worst-case definition: we don't just want to fool one particular classifier; we want the representation to be fundamentally useless for the entire class of predictors $\mathcal{V}$.

The paper then specializes to the case that matters: $\mathcal{V}$ is the class of linear predictors $\eta(\mathbf{x}) = \mathbf{b} + \mathbf{W}\mathbf{x}$, and $\mathfrak{L}$ is the class of convex loss functions nonnegative in their first argument. When $X$ guards $Z$ under these conditions, we say "$X$ linearly guards $Z$" (Definition 2.3).

Why this specialization? Two reasons. First, linear classifiers are the most natural adversary for post-hoc erasure — if a downstream model can use a simple linear probe to recover the concept, the erasure has failed. Second, linear guardedness turns out to have an extremely clean algebraic characterization (Theorem 3.4), making it tractable to both verify and enforce.

The trivially attainable loss (Definition 2.2) is a key auxiliary concept:

$\mathcal{L}_\tau = \inf_{\mathbf{b} \in \mathbb{R}^k} \mathbb{E}[\mathcal{L}(\mathbf{b}, Z)]$

where $\mathcal{L}_\tau$ is the lowest expected loss achievable by any constant predictor (one that ignores $X$ entirely and outputs the same vector for every input). For cross-entropy loss, the trivial predictor is $\eta_\tau(Z)_j = \log(\mathbb{P}(Z = j))$, which simply predicts the prior class probabilities. If no linear classifier can beat this constant baseline, then $X$ contains no linearly accessible information about $Z$.

Theorem 3.1 (Equal centroids ⇒ guardedness). This is the forward direction: if every class has the same mean, then linear classifiers cannot help. The proof uses Jensen's inequality in a clever way:

$\mathbb{E}[\mathcal{L}(\eta, Z)] = \mathbb{E}_Z\left[\mathbb{E}[\mathcal{L}(\eta, Z) \mid Z]\right] \geq \mathbb{E}_Z\left[\mathcal{L}(\mathbb{E}[\eta \mid Z], Z)\right]$

where $\eta = \mathbf{b} + \mathbf{W}\mathbf{x}$ is any linear predictor.

What this inequality computes: it says the expected loss of any linear predictor is lower-bounded by the loss you would get if you replaced every data point with the mean of its class. Since $\eta$ is linear, $\mathbb{E}[\eta \mid Z] = \mathbf{b} + \mathbf{W}\mathbb{E}[X \mid Z]$. If all class-conditional means are equal to the global mean $\mathbb{E}[X]$, this becomes $\mathbf{b} + \mathbf{W}\mathbb{E}[X]$, which is a constant predictor — it doesn't depend on the specific input, only on the global mean. The trivially attainable loss is by definition the best any constant predictor can do, so no linear predictor can beat it.

Why this form matters: Jensen's inequality provides the lower bound without needing to know anything about the specific loss function except convexity. This makes the result general — it applies to cross-entropy, mean squared error, hinge loss, and any other convex loss. The key insight is that linear functions commute with expectation, so the expected output of a linear predictor on a class is simply the linear function evaluated at the class centroid. If all centroids are equal, all classes get the same expected output, and the predictor cannot discriminate.

Theorem 3.3 (Cross-entropy guardedness ⇒ equal centroids). This is the harder reverse direction, proving that if the trivially attainable cross-entropy loss is optimal, the class centroids must be equal. The proof uses first-order optimality conditions (the gradient of the expected loss with respect to the parameters must be zero at the optimum) and the specific partial derivatives of cross-entropy loss:

The on-category partial derivative is $\partial \mathcal{L}(\eta, i) / \partial \eta_i = \exp(\eta_i) / \sum_{j=1}^k \exp(\eta_j) - 1 \in (-1, 0]$, and the off-category partial derivative is $\partial \mathcal{L}(\eta, \neq i) / \partial \eta_i = \exp(\eta_i) / \sum_{j=1}^k \exp(\eta_j) \in (0, 1)$.

The critical property is that the off-category partial derivative is non-zero and identical for all off-categories. This lets us factor it out of sums over classes and solve for the class-conditional means. Lemma 3.2 generalizes this to any loss function with bounded, non-vanishing, category-independent off-diagonal partial derivatives.

What this proves operationally: if you have achieved perfect linear erasure (the best linear classifier does no better than the prior), then you have necessarily made all class means equal. This gives us a testable necessary condition for linear guardedness and a constructive target for erasure methods: make the class centroids coincide.

Theorem 3.4 (Equal centroids ⇔ zero cross-covariance). This connects the geometric condition (equal means) to the algebraic condition we can actually optimize:

$\mathrm{Cov}(X_i, Z_j) = \mathbb{E}[X_i Z_j] - \mathbb{E}[X_i]\mathbb{E}[Z_j] = \mathbb{P}(Z = j) \cdot (\mathbb{E}[X_i \mid Z = j] - \mathbb{E}[X_i])$

What this equation computes: the covariance between the $i$-th feature and the $j$-th class indicator. Since $Z$ is one-hot, $X_i Z_j = X_i$ when $Z = j$ and $0$ otherwise, so the first term simplifies to $\mathbb{P}(Z = j)\mathbb{E}[X_i \mid Z = j]$. The second term subtracts the unconditional mean weighted by the class probability. The factorization shows that $\mathrm{Cov}(X_i, Z_j) = 0$ if and only if $\mathbb{E}[X_i \mid Z = j] = \mathbb{E}[X_i]$, assuming $\mathbb{P}(Z = j) > 0$.

Why this form is crucial: it converts the problem from "make $k$ class means equal" (a vector equality constraint) to "make a $d \times k$ matrix equal to zero" (an entrywise constraint). The cross-covariance matrix $\boldsymbol{\Sigma}_{XZ}$ is the central object for the rest of the paper: linear guardedness is equivalent to $\boldsymbol{\Sigma}_{XZ} = \mathbf{0}$. This matrix can be estimated from data, and we can design transformations that zero it out.

The paper also proves (Appendix C) that linear guardedness is equivalent to linear statistical parity: every linear classifier $f(\mathbf{x}) = \mathbf{b} + \mathbf{W}\mathbf{x}$ has the same expected output for every class. This connects the concept erasure objective to the fairness literature, where statistical parity is a standard (if imperfect) definition of non-discrimination.

Characterizing All Guarding Affine Transformations

With the equivalence $\mathrm{Cov}(X, Z) = \mathbf{0} \iff$ linear guardedness established, Theorem 4.1 characterizes exactly which affine transformations achieve it:

Theorem 4.1. Let $X$ and $Z$ be random vectors taking values in $\mathbb{R}^d$ and $\mathbb{R}^k$ respectively, with $X$ of finite first moment. Then given some affine function $r(\mathbf{x}) = \mathbf{P}\mathbf{x} + \mathbf{b}$, the modified random vector $r(X)$ linearly guards $Z$ if and only if the columns of the cross-covariance matrix $\boldsymbol{\Sigma}_{XZ}$ are contained in the null space of $\mathbf{P}$.

The proof uses the linearity of cross-covariance:

$\mathrm{Cov}(r(X), Z) = \mathrm{Cov}(\mathbf{P}X + \mathbf{b}, Z) = \mathbf{P} \mathrm{Cov}(X, Z) = \mathbf{P} \boldsymbol{\Sigma}_{XZ}$

What this equation shows: adding a constant $\mathbf{b}$ does not change covariance (since covariance is translation-invariant), so the bias term is irrelevant to the guardedness condition. The condition reduces to $\mathbf{P} \boldsymbol{\Sigma}_{XZ} = \mathbf{0}$, which means every column of $\boldsymbol{\Sigma}_{XZ}$ must be in the nullspace of $\mathbf{P}$, or equivalently, the column space of $\boldsymbol{\Sigma}_{XZ}$ must be a subspace of $\ker(\mathbf{P})$.

Why this is a weak condition: the nullspace of $\mathbf{P}$ only needs to contain the column space of $\boldsymbol{\Sigma}_{XZ}$, which has dimension at most $k-1$ (for $k$-class classification, since the one-hot constraints reduce the effective rank). For typical settings where the embedding dimension $d \gg k$, this means $\mathbf{P}$ only needs to zero out a tiny subspace — the remaining $d - (k-1)$ dimensions can be anything. This is why orthogonal projection (which sets all eigenvalues to 0 or 1) is unnecessarily destructive: it forces $\mathbf{P}$ to be symmetric and idempotent, imposing constraints beyond what guardedness requires.

The paper notes (Appendix D) that three prior methods — Spectral Attribute RemovaL (SAL), Mean Projection (MP), and Fair PCA — satisfy this condition when configured appropriately, but their authors could not prove that they achieve linear guardedness because they lacked this theorem. SAL's full-rank variant sets $\mathbf{P} = \mathbf{I} - \mathbf{U}_{:n}\mathbf{U}_{:n}^\top$ where $\mathbf{U}_{:n}$ contains the left singular vectors of $\boldsymbol{\Sigma}_{XZ}$, which guarantees $\mathbf{P}\boldsymbol{\Sigma}_{XZ} = \mathbf{0}$ since those singular vectors span the column space. MP orthogonally projects onto the complement of the difference of class centroids, which forces equal centroids after projection — a consequence of Theorem 3.4 that SAL and MP are equivalent for binary $Z$.

The LEACE Derivation: Optimal Erasure as Constrained Least Squares

Theorem 4.2 solves the central optimization problem: among all matrices $\mathbf{P}$ that achieve $\mathrm{Cov}(\mathbf{P}X, Z) = \mathbf{0}$, which one minimizes $\mathbb{E}[\|\mathbf{P}X - X\|^2_{\mathbf{M}}]$?

The key insight is that the problem decomposes into independent subproblems after whitening. Here is the step-by-step logic:

Step 1: Decompose by metric. Write the objective in a basis that diagonalizes $\mathbf{M}$:

$\mathbb{E}[\|\mathbf{P}X - X\|^2_{\mathbf{M}}] = \sum_{i=1}^d m_i \cdot \mathbb{E}[(\mathbf{p}_i^\top X - X_i)^2]$

where $m_1, \ldots, m_d \geq 0$ are the eigenvalues of $\mathbf{M}$ and $\mathbf{p}_i^\top$ is the $i$-th row of $\mathbf{P}$. Since each term depends on a different row of $\mathbf{P}$, we can optimize each row independently. The weights $m_i$ scale the importance of each dimension but don't affect the optimal $\mathbf{p}_i$ when $m_i > 0$ — any positive scaling of the objective yields the same minimizer.

Step 2: Whitening. Transform to a basis where the covariance structure is simple. Let $\mathbf{W} = (\boldsymbol{\Sigma}_{XX}^{1/2})^+$ be the whitening matrix (the pseudoinverse of the symmetric square root of the covariance). Then $\mathbf{W}X$ has identity covariance matrix on its support (the subspace where $X$ has non-zero variance). This orthonormalizes the geometry: in the whitened space, Euclidean distance equals Mahalanobis distance in the original space.

Step 3: Decompose the constraint. After whitening, the constraint $\mathbf{P}\boldsymbol{\Sigma}_{XZ} = \mathbf{0}$ becomes $\mathbf{A}\boldsymbol{\Sigma}_{\mathbf{W}X, Z} = \mathbf{0}$ where $\mathbf{A} = \mathbf{P}\mathbf{W}^+$. In the whitened space, the components of $\mathbf{W}X$ that correlate with $Z$ form a subspace of dimension $\ell = \mathrm{rank}(\boldsymbol{\Sigma}_{XZ})$. The constraint says: for each row $i$, the coefficient $A_{ij}$ must be zero whenever column $j$ correlates with $Z$ (i.e., for $j \leq \ell$ in a basis that isolates the concept-correlated subspace).

Step 4: Solve each subproblem. Each component $X_i$ can be expressed in the whitened basis as $X_i = \sum_{j=1}^m W_{ij}^+ (\mathbf{W}X)_j$, where $m = \mathrm{rank}(\boldsymbol{\Sigma}_{XX})$. The transformed variable $(\mathbf{P}X)_i = \sum_{j=1}^m A_{ij} (\mathbf{W}X)_j$ is the same linear combination but with coefficients $A_{ij}$ instead of $W_{ij}^+$. The squared error is:

$\mathbb{E}[(\mathbf{p}_i^\top X - X_i)^2] = \sum_{j=1}^m (A_{ij} - W_{ij}^+)^2$

since the whitened components have unit variance and are uncorrelated. To minimize this subject to $A_{ij} = 0$ for $j \leq \ell$ (the concept-correlated components), we simply set $A_{ij} = W_{ij}^+$ for $j > \ell$ (preserve the non-concept components) and $A_{ij} = 0$ for $j \leq \ell$ (zero out the concept components).

Step 5: Assembly. In matrix form, this is $\mathbf{A} = \mathbf{W}^+(\mathbf{I} - \mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}})$, where $\mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}}$ is the orthogonal projection matrix onto the column space of $\mathbf{W}\boldsymbol{\Sigma}_{XZ}$. Multiplying by $\mathbf{W}$ to transform back to the original space gives:

$\mathbf{P}^* = \mathbf{I} - \mathbf{W}^+ \mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}} \mathbf{W}$

What this matrix computes: $\mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}} \mathbf{W}$ projects the whitened data onto the concept-correlated subspace. Left-multiplying by $\mathbf{W}^+$ unwhitens back to the original space. Subtracting from $\mathbf{I}$ means we remove from $\mathbf{x}$ exactly its projection onto the concept subspace (in the whitened metric), leaving everything else intact.

Why the pseudoinverses matter: $\boldsymbol{\Sigma}_{XX}$ may be rank-deficient (if $d > n$ or if there are linear dependencies among features). The Moore-Penrose pseudoinverse handles this gracefully: it inverts only on the subspace where the covariance is full-rank and maps the nullspace to zero. The whitening matrix $\mathbf{W}$ similarly handles rank deficiency — components of $X$ with zero variance are left unchanged by LEACE.

Theorem 4.3 extends this to the uncentered case by deriving the optimal bias:

$\mathbf{b}^* = \mathbb{E}[X] - \mathbf{P}^* \mathbb{E}[X]$

Why this form: the bias corrects for the fact that $\mathbf{P}^*$ changes the mean of the data. Setting $\mathbf{b}^*$ this way ensures that the scrubbed data has the same mean as the original data — $\mathbb{E}[\mathbf{P}^*X + \mathbf{b}^*] = \mathbb{E}[X]$. The proof shows that any other bias would add a constant shift, increasing the squared error by a positive term $\mathbf{c}^\top \mathbf{M} \mathbf{c}$ without affecting the covariance constraint.

Putting it together yields the LEACE formula (Equation 1):

$r_{\text{LEACE}}(\boldsymbol{x}) = \boldsymbol{x} - \mathbf{W}^+ \mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}} \mathbf{W} (\boldsymbol{x} - \mathbb{E}[X])$

Operational interpretation: Given a new data point $\boldsymbol{x}$:

1. Center it by subtracting the training mean $\mathbb{E}[X]$.
2. Whiten the centered vector by multiplying by $\mathbf{W} = (\boldsymbol{\Sigma}_{XX}^{1/2})^+$ — this transforms to a space where all non-zero-variance dimensions have unit variance and are uncorrelated.
3. Orthogonally project onto the subspace that correlates with $Z$ in this whitened space, using the projection matrix $\mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}}$.
4. Unwhiten this projection by multiplying by $\mathbf{W}^+$ — map it back to the original coordinate system.
5. Subtract this vector from the original $\boldsymbol{x}$.

The result is the original $\boldsymbol{x}$ with all linearly available information about $Z$ surgically removed, while keeping all other information (including nonlinear information about $Z$) maximally intact.

Figure 1 in the paper visualizes this three-step process: whitening (orange ellipse → blue circle), orthogonal projection onto the concept subspace (blue circle → green line, which is the nullspace of the concept), and unwhitening (green line → green ellipse in original space). The key visual takeaway is that LEACE's result (green ellipse) preserves more of the original data's covariance structure than an orthogonal projection would (which would collapse everything onto the dashed line, the orthogonal complement of the concept direction in the original space).

Why Oblique Projections Are Optimal (and Orthogonal Projections Are Not)

The LEACE matrix $\mathbf{P}^*$ is an oblique projection: $\mathbf{P}^* \mathbf{P}^* = \mathbf{P}^*$ (idempotent), but $\mathbf{P}^* \neq (\mathbf{P}^*)^\top$ in general (not symmetric). This is not a bug — it's the feature that makes LEACE minimal.

The paper explains this clearly in Section 4.2:

"Orthogonal projection matrices are symmetric, and $\mathbf{I} - \mathbf{W}^+ \mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}} \mathbf{W}$ is only symmetric in the special case where $\mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}}$ and $\mathbf{W}$ commute."

Why prior work defaulted to orthogonal projections: the intuition that "the closest point in a subspace is given by orthogonal projection" is correct when the constraint is on the range of the projection. If you want to map points onto a target subspace $U$, the orthogonal projection onto $U$ does indeed minimize Euclidean distance. But concept erasure imposes a constraint on the nullspace, not the range: we must zero out the column space of $\boldsymbol{\Sigma}_{XZ}$, but we are free to choose any range that accomplishes this. Orthogonal projection forces the range to be exactly the orthogonal complement of the nullspace (because for orthogonal projections, range and nullspace are orthogonal complements). Oblique projections allow the range to be tilted, letting us "reconstruct" parts of the erased dimensions using information from the preserved dimensions.

The concrete example in Appendix F makes this crystal clear. With $d = 2, k = 1$, features $X_1 \in \{-1, +1\}$ (uniform, independent of $Z$), $Z \in \{-1, +1\}$ (uniform), and $X_2 = X_1 + Z$:

• $X_2$ contains all information about $Z$ (since $X_1 \perp\!\!\!\perp Z$), but $X_1$ contains information about $X_2$.
• The best orthogonal projection erases $X_2$ entirely: $\mathbf{P}_{\text{ortho}} = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}$, with MSE = 2.
• The optimal oblique projection uses $X_1$ to partially reconstruct $X_2$: $\mathbf{P}_{\text{oblique}} = \begin{bmatrix} 1 & 0 \\ 1 & 0 \end{bmatrix}$, with MSE = 1.

Both achieve $\mathrm{Cov}(\mathbf{P}X, Z) = \mathbf{0}$, but the oblique projection cuts the distortion in half by exploiting the correlation between $X_1$ and $X_2$. Orthogonal projection cannot do this because it forces the second row to be orthogonal to the first row's span.

When do oblique and orthogonal projections coincide? Only when $\mathbf{W}$ and $\mathbf{P}_{\mathbf{W}\boldsymbol{\Sigma}_{XZ}}$ commute, which happens when the concept subspace and the data covariance structure are aligned in a particular way — roughly, when the concept-correlated dimensions are already orthogonal to all other dimensions. In general high-dimensional data, this is not the case, and the oblique projection will strictly dominate.

Oracle LEACE: Geometric Intuition in a Random Variable Hilbert Space

Appendix I provides an alternative perspective that yields deeper geometric intuition. Consider the space $\mathcal{H}$ of square-integrable real-valued random variables, equipped with the inner product $\langle \xi, \zeta \rangle_{\mathcal{H}} = \mathbb{E}[\xi \zeta]$. Each component $X_i$ and $Z_j$ is a vector in this Hilbert space.

The constraint $\mathrm{Cov}(X_i', Z_j) = 0$ becomes an orthogonality constraint between the centered random variables $\tilde{X}_i'$ and $\tilde{Z}_j$:

$\langle \tilde{X}_i', \tilde{Z}_j \rangle_{\mathcal{H}} = 0 \quad \forall j$

In words: the demeaned scrubbed feature $\tilde{X}_i'$ must be orthogonal to every demeaned concept dimension $\tilde{Z}_j$.

The least-squares objective becomes: find the nearest $\tilde{X}_i'$ to $\tilde{X}_i$ (in the $\mathcal{H}$ norm) that is orthogonal to $\mathcal{Z} = \mathrm{span}(\{\tilde{Z}_1, \ldots, \tilde{Z}_k\})$. This is precisely the orthogonal projection of $\tilde{X}_i$ onto $\mathcal{Z}^\perp$ in the Hilbert space $\mathcal{H}$.

The solution is the ordinary least squares residual from regressing $X$ on $Z$:

$X'_{\text{LEACE}} = X - \boldsymbol{\Sigma}_{XZ} \boldsymbol{\Sigma}_{ZZ}^+ (Z - \mathbb{E}[Z])$

What this computes: for each data point with its known label $\mathbf{z}$, we subtract the best linear prediction of $X$ from $Z$ (the OLS fitted values), plus an adjustment to keep the mean unchanged. The term $\boldsymbol{\Sigma}_{XZ} \boldsymbol{\Sigma}_{ZZ}^+ (Z - \mathbb{E}[Z])$ is exactly what a linear regression would add to the mean to predict $X$ from $Z$ — we subtract it to remove the predictable part.

Why this is "Oracle" LEACE: this formula requires the true label $\mathbf{z}$ for each individual data point at inference time, not just the aggregate covariance statistics. The standard LEACE formula (Equation 1) achieves the same expected effect without per-sample labels by using the projection matrix derived from covariances, but Oracle LEACE is even more surgical because it can condition on the exact label value.

Figure 5 visualizes this: each component $X_i$ is a vector in $\mathcal{H}$, and the concept subspace $\mathcal{Z}$ is spanned by the centered label vectors. Oracle LEACE orthogonally projects $X_i$ onto $\mathcal{Z}^\perp$ in this abstract space, yielding the closest random variable to $X_i$ that is uncorrelated with all of $Z$.

Concept Scrubbing: Sequential Multi-Layer Erasure

The paper introduces concept scrubbing (Section 6, Algorithm 1) as the procedure for erasing a concept from every layer of a deep network. The naive approach — independently fit LEACE parameters for each layer and apply them all at once — would fail because intervening on early layers changes the distribution of hidden states at later layers. The LEACE parameters fitted on the clean model's activations would not correctly guard the scrubbed activations.

Algorithm 1: Concept Scrubbing

The algorithm processes layers sequentially:

1. Embed the input: $\mathbf{H}_1 \leftarrow \text{Embed}(\mathbf{X})$
2. For each layer $l = 1, \ldots, \ell$:
   • Fit LEACE parameters $(\mathbf{P}, \mathbf{b})$ on the current hidden states $\mathbf{H}_l$ and the concept labels $\mathbf{Z}$. This computes $\boldsymbol{\Sigma}_{\mathbf{H}_l \mathbf{H}_l}$ and $\boldsymbol{\Sigma}_{\mathbf{H}_l Z}$ from the data, then applies Equation 1.
   • Save $(\mathbf{P}, \mathbf{b})$ for later use.
   • Immediately scrub: $\mathbf{H}_l \leftarrow \mathbf{P}(\mathbf{H}_l - \boldsymbol{\mu}_{\mathbf{H}_l}) + \boldsymbol{\mu}_{\mathbf{H}_l}$, where $\boldsymbol{\mu}_{\mathbf{H}_l}$ is the mean of the hidden states at that layer.
   • Feed forward: $\mathbf{H}_{l+1} \leftarrow f_l(\mathbf{H}_l)$, where $f_l$ is the transformer block for layer $l$.

Why sequential fitting is necessary: after step 3 scrubs layer $l$, the distribution of $\mathbf{H}_{l+1}$ is different from what it would be in the clean model. By fitting LEACE on the already-scrubbed hidden states from previous layers, we account for this distributional shift. This is analogous to the difference between a static intervention (apply pre-computed edits to all layers at once) and a dynamic intervention (edit as you go, propagating the effects).

Computational considerations. The paper discusses two regimes:

• Caching approach (used in the paper's experiments): store all hidden states for all layers for all data points, then fit LEACE sequentially. This requires $\sim 500$GB for the models studied but avoids recomputing forward passes. Total compute is $O(\ell)$ forward passes per data point.
• Recomputation approach: for each layer, re-run the forward pass from the input through all previous (now scrubbed) layers. This avoids caching at the cost of $O(\ell^2)$ total forward passes. For large models where memory is the bottleneck, this is the practical choice.

Norm regularization (Appendix H). An important practical issue: LEACE's oblique projection matrix $\mathbf{P}^*$ can have singular values greater than 1, meaning the norm of the hidden state can increase after scrubbing. In some layers, this caused the norm to diverge in early experiments, producing NaN outputs. SAL, which uses an orthogonal projection with eigenvalues in $\{0, 1\}$, never has this problem, even though its average-case MSE is worse.

The paper's solution is a convex combination heuristic:

1. After computing $\mathbf{P}^*_{\text{LEACE}}$ and the orthogonal projection $\mathbf{Q}_{\text{SAL}}$, compute the trace of the covariance matrix after applying each: $\mathrm{tr}(\mathbf{P}^* \boldsymbol{\Sigma}_{XX} (\mathbf{P}^*)^\top)$ and $\mathrm{tr}(\mathbf{Q} \boldsymbol{\Sigma}_{XX} \mathbf{Q}^\top)$.
2. If $\mathrm{tr}(\mathbf{P}^* \boldsymbol{\Sigma}_{XX} (\mathbf{P}^*)^\top) > \mathrm{tr}(\boldsymbol{\Sigma}_{XX})$ (i.e., LEACE would increase the total variance), find $\alpha \in [0, 1]$ such that $\mathbf{P}' = \alpha \mathbf{P}^* + (1 - \alpha) \mathbf{Q}$ has $\mathrm{tr}(\mathbf{P}' \boldsymbol{\Sigma}_{XX} (\mathbf{P}')^\top) = \mathrm{tr}(\boldsymbol{\Sigma}_{XX})$.
3. Since the set of linearly guarding matrices is convex (a linear combination of two matrices that both kill $\boldsymbol{\Sigma}_{XZ}$ also kills $\boldsymbol{\Sigma}_{XZ}$), $\mathbf{P}'$ is guaranteed feasible, and its MSE is guaranteed no worse than SAL's.

This is a pragmatic fix that retains LEACE's optimality on average while preventing pathological norm growth in specific layers.

Training efficiency details. The paper uses a sample of $2^{22}$ tokens (approximately 4 million) for fitting LEACE parameters and another $2^{22}$ tokens for evaluation (Section 6.1). Covariance statistics can be computed in a streaming fashion without storing all hidden states in memory — only the running sums and sum-of-outer-products need to be accumulated. This makes LEACE scalable to large datasets and models.

Randomized erasure baseline. To isolate the effect of erasing the specific concept from the general effect of any intervention, the paper uses a control condition: project onto a random subspace of the same rank as the cross-covariance matrix, using a fresh random subspace for each minibatch, and average the results. This controls for the dimensionality reduction and any generic disruption to the representation.

Architecture details. For both Pythia and LLaMA models, LEACE is applied to the input of each transformer block, immediately after normalization (LayerNorm for Pythia, RMSNorm for LLaMA). This is a deliberate choice: intervening before the attention and MLP sublayers, on normalized representations, ensures the intervention affects the computation but operates on well-conditioned inputs.

Part-of-speech tagging for evaluation. Since neither the Pile (Pythia's training data) nor RedPajama (LLaMA's training data) comes with part-of-speech tags, the paper uses the SpaCy library to automatically generate Universal Dependency tags (18 coarse-grained classes like "Noun," "Verb," etc.). This introduces some label noise, but the authors argue it is adequate for measuring large effects — if anything, noisy labels would understate the true reliance on POS, making their results a conservative estimate.

Summary of Key Design Choices and Their Justifications

• Linear guardedness as the target: linear classifiers are the most natural post-hoc adversary; if a linear probe can recover the concept, downstream models can too. Nonlinear erasure is conjectured to be intractable in general (Section 7).
• Convex loss generality (not just cross-entropy): Theorem 3.1 applies to all convex losses, making the equivalence between guardedness and equal means broadly applicable — to regression, ranking, and any other linearly parameterized task.
• Least-squares edit distance as the minimality criterion: MSE is a local quadratic approximation to any smooth divergence, is analytically tractable, and yields a closed-form solution. Using MSE avoids the need for iterative optimization (as in RLACE) or hyperparameter tuning.
• Whitening-based derivation: converting to a space where features are uncorrelated and unit-variance makes the constraint an orthogonal projection, which is the easiest to solve. The pseudoinverse handles rank deficiency automatically.
• Oblique projection over orthogonal: the nullspace constraint only requires $\mathbf{P}\boldsymbol{\Sigma}_{XZ} = \mathbf{0}$; symmetry $\mathbf{P} = \mathbf{P}^\top$ is an unnecessary restriction that increases MSE. Appendix F proves this concretely.
• Sequential fitting for concept scrubbing: interventions change downstream distributions; fitting LEACE on already-scrubbed hidden states accounts for this shift, ensuring guardedness holds at every layer.
• Streaming covariance estimation: LEACE only needs second-moment statistics, which can be accumulated incrementally — no need to store the full dataset in memory during fitting.
• Convex combination norm regularization: a practical compromise that prevents divergence while maintaining feasibility — a linear combination of two guarding matrices also guards.

4. Key Insights and Innovations

Innovation 1: Linear Guardedness Has a Single, Simple Algebraic Signature — Zero Cross-Covariance

Before this paper, the concept erasure literature operated without a crisp characterization of what it means for a representation to be linearly uninformative about a concept. Methods like INLP (Ravfogel et al., 2020) pursued linear guardedness iteratively — train a classifier, project out its weights, repeat — but no one had established exactly what condition they were converging toward. SAL and Mean Projection happened to achieve guardedness under specific settings, but their authors could not prove it, and there was no way to verify whether a given transformation actually succeeded without training a classifier and checking accuracy.

The paper's most fundamental conceptual contribution is proving that all of this complexity collapses to a single, testable algebraic condition: $\mathrm{Cov}(X, Z) = \mathbf{0}$ (Theorem 3.4, via the chain of equivalences in Section 3). Linear guardedness — the inability of any linear classifier with any convex loss to outperform a constant baseline — is equivalent to the $d \times k$ cross-covariance matrix being exactly zero. This is not just one sufficient condition among many; it is necessary and sufficient.

What makes this insight powerful is that it converts an adversarial, classifier-centered definition ("no linear probe can recover the concept") into a purely statistical one ("the concept has zero covariance with every feature dimension"). The adversarial framing had led the field toward iterative, expensive methods (INLP, RLACE) that tried to empirically defeat classifiers. The statistical framing reveals that linear erasure is fundamentally about second moments — covariances — which can be estimated once and neutralized in a single algebraic step. This is a fundamental reframing of the problem, not an incremental improvement.

The generality of the result is also distinctive. Theorem 3.1 shows that the forward direction (equal means ⇒ guardedness) holds for any convex loss, not just cross-entropy. This means the equivalence applies to regression, ranking, or any other linearly parameterized prediction task — not just classification. Prior work (Ravfogel et al., 2023) had studied guardedness only for cross-entropy loss with log-linear predictors; this paper's Definition 2.3 and the accompanying proofs extend the framework to the full class of convex losses, making it applicable far beyond the fairness-in-classification setting that motivated earlier work.

The practical consequence is that concept erasure no longer requires training a probe or running gradient-based optimization. If you can estimate $\boldsymbol{\Sigma}_{XZ}$ and $\boldsymbol{\Sigma}_{XX}$ — which any statistical package can do in streaming fashion — you can verify guardedness algebraically and construct an erasure function in closed form. This is a qualitative shift in tractability, not just a speedup.

---

Innovation 2: The "Orthogonal Projection" Intuition Is Wrong — and Oblique Projections Are Provably Optimal

A dominant assumption in prior work was that concept erasure should be implemented as an orthogonal projection onto the complement of the concept subspace. INLP, SAL, and Mean Projection all produce orthogonal projection matrices — symmetric, idempotent, with eigenvalues in ${0, 1}$. The intuition was borrowed from linear algebra: if you want to find the nearest point in a subspace, you orthogonally project onto it.

The paper demonstrates that this intuition, while natural, is mathematically misguided for concept erasure, and that correcting it yields a significant improvement in edit quality. The key realization (Section 4.2) is that the concept erasure constraint is a nullspace condition ($\mathbf{P}\boldsymbol{\Sigma}_{XZ} = \mathbf{0}$), not a range condition. You must zero out the subspace that correlates with $Z$, but you are free to choose any range for $\mathbf{P}$ that accomplishes this. Orthogonal projections additionally constrain the range to be the orthogonal complement of the nullspace, which is an unnecessary restriction — it prevents $\mathbf{P}$ from using information in the preserved dimensions to partially reconstruct the erased dimensions.

The paper proves that the optimal $\mathbf{P}^*$ is an oblique projection — idempotent but not symmetric in general — and that it minimizes the mean squared edit distance simultaneously for all inner-product norms (Theorem 4.2). This simultaneous optimality is a striking result: whether you care about Euclidean distance, Mahalanobis distance, or any weighted combination, the same $\mathbf{P}^*$ is optimal. You don't need to choose a distance metric — LEACE is universally optimal within the class of affine erasure functions.

The concrete example in Appendix F makes the practical significance undeniable. With two features where $X_2 = X_1 + Z$ and $X_1$ is independent of $Z$, the best orthogonal projection achieves MSE = 2 by completely destroying $X_2$. LEACE achieves MSE = 1 — a 50% reduction — by using $X_1$ to partially reconstruct $X_2$: since $X_1$ correlates with $X_2$ but not with $Z$, it can fill in the non-concept-related variation in $X_2$ after the $Z$-related component is removed. This is not a minor tweak; it's a fundamentally different kind of transformation that exploits the covariance structure of the data to be more surgical.

The paper traces this insight to a simple observation about Figure 1: orthogonal projection would map all points onto a line (the dashed line, the orthogonal complement of the concept direction in the original space), while LEACE maps them onto a tilted line (the green line) that preserves more of the original data's covariance structure. The geometric intuition is that you want to project onto a subspace that is orthogonal to the concept in the whitened metric, not in the original metric — and after unwhitening, this becomes an oblique projection.

This is a fundamental conceptual correction to the literature, not an incremental improvement. Prior work had converged on orthogonal projections by default; the paper shows that this default was based on a misapplication of the "nearest point in a subspace" principle to a problem where the constraint is on the nullspace, not the range. The field's intuition was wrong, and fixing it yields a method that is both theoretically optimal and practically superior.

---

Innovation 3: Concept Scrubbing — Sequential Multi-Layer Erasure as a Principled Causal Intervention

Before LEACE, it was impossible to erase a concept from more than one layer of a deep network without destroying the model's outputs. INLP's massive collateral damage — deleting hundreds of dimensions to erase an 18-class concept, as Section 5.3 documents — meant that applying INLP to multiple layers would compound the damage and cause the model to collapse into gibberish. This severely limited causal interpretability research: Elazar et al. (2021)'s amnesic probing paradigm, which aimed to measure how concepts are used throughout a model, could only intervene on one layer at a time, leaving open the question of how information flows across layers.

LEACE makes concept scrubbing possible — and eminently practical — by being so surgically precise that the cumulative damage across all layers remains manageable. The paper's key insight is not just that LEACE is more precise per layer (it deletes only 17 dimensions versus INLP's 360 for the same part-of-speech task), but that this precision compoundingly enables a qualitatively new experimental paradigm. You can now erase a concept from every layer of a 30B-parameter model and still get coherent, interpretable outputs (Table 1 shows perplexity increases from ~0.6 to ~2 bits per byte — a large but not catastrophic degradation — compared to SAL's ~3–5 bits per byte).

The sequential fitting procedure (Algorithm 1) is itself an important conceptual contribution. The naive approach — fit LEACE independently for each layer on the clean model, then apply all edits at once — would fail because early-layer interventions change the distribution of later-layer activations. The paper's solution — fit on layer $l$, scrub, feed forward to layer $l+1$, repeat — accounts for this distributional shift and ensures guardedness holds at every layer under the actual intervened-upon distribution. This is essentially a form of interventional distribution propagation — you're simulating the counterfactual world where the concept is absent at each layer sequentially, rather than trying to erase it everywhere simultaneously from the factual distribution. This is a principled approach to a subtle causal inference problem, and it's what makes the multi-layer results interpretable.

The randomized erasure baseline is a smaller but important methodological innovation. By comparing concept scrubbing to projecting onto a random subspace of the same rank (fresh for each minibatch, averaged over subspaces), the paper isolates the specific effect of removing the target concept from the generic effect of any dimensionality reduction or perturbation. This is crucial because almost any intervention degrades model performance to some degree — without this baseline, one might attribute the performance drop to the importance of part-of-speech when it's actually just the model being disrupted. The fact that random erasure has essentially no effect on perplexity (Table 1: "Random erasure" rows are nearly identical to "No intervention") while POS erasure causes large increases is compelling evidence that the effect is concept-specific.

The differential impact of LEACE vs. SAL in the scrubbing experiments (Table 1) also yields a substantive finding beyond the method itself: SAL induces 1.5–2× larger perplexity increases than LEACE across all model sizes. This validates the paper's central claim that LEACE is more surgical — SAL's orthogonal projection destroys more non-concept information, and that excess damage shows up as worse language modeling performance. More importantly, it shows that the measured causal effect of a concept depends critically on the erasure method used. If these experiments had been done with SAL alone, the causal effect of part-of-speech on language modeling would have been substantially overestimated. This is not just a methodological nicety — it's a warning to the interpretability community that the tool you use shapes the conclusions you draw.

This innovation is enabling rather than incremental: it doesn't just improve an existing capability, it opens up an entirely new class of experiments (full-network causal intervention) that were previously infeasible.

---

Innovation 4: A Unified Theoretical Framework That Subsumes and Explains Prior Methods

The paper does not just propose a new method — it provides a theoretical framework that explains why several prior methods work when they do, why they fail when they don't, and exactly how they relate to each other. This kind of unifying theoretical contribution is rare and valuable.

Specifically, Theorem 4.1 characterizes the entire set of affine transformations that achieve linear guardedness: those where $\ker(\mathbf{P}) \supseteq \mathrm{colsp}(\boldsymbol{\Sigma}_{XZ})$. This is the first such characterization in the literature. Within this framework, the paper shows (Appendix D) that:

• Full-rank SAL achieves guardedness because its projection matrix $\mathbf{Q}_{\text{SAL}} = \mathbf{I} - \mathbf{U}_{:n}\mathbf{U}_{:n}^\top$ uses the left singular vectors of $\boldsymbol{\Sigma}_{XZ}$ as its nullspace basis — by construction, these span the column space, so $\mathbf{Q}_{\text{SAL}}\boldsymbol{\Sigma}_{XZ} = \mathbf{0}$.
• Mean Projection (MP) is mathematically equivalent to SAL for binary $Z$, because the difference-of-centroids vector is the only left singular vector of $\boldsymbol{\Sigma}_{XZ}$ when $k=2$.
• Fair PCA satisfies the condition under appropriate rank choices, though this had not been previously noted.
• INLP converges to a guarding projection, but does so inefficiently and with unnecessary rank reduction because it removes entire classifier weight directions rather than directly targeting the cross-covariance subspace.

The authors of SAL, MP, and Fair PCA could not prove that their methods achieve linear guardedness; Theorem 4.1 provides that proof retroactively. More importantly, the framework reveals that all these methods are in the feasible set but none is optimal — they all satisfy the nullspace constraint but impose additional restrictions (orthogonal projection, specific basis choices) that increase the edit distance beyond the LEACE minimum.

This unification has both intellectual and practical value. Intellectually, it converts a scattered landscape of heuristic methods into a single optimization problem with a well-characterized feasible set and a unique optimum. Practically, it tells practitioners that if they're currently using SAL or MP, they can switch to LEACE and get strictly better preservation of non-concept information with no downside — same guardedness guarantee, smaller edit, same computational cost. It also explains the failure mode of INLP: it's solving the right problem (find a guarding projection) but using an inefficient algorithm that over-deletes dimensions.

The generalization of guardedness (Definition 2.1) from Ravfogel et al. (2023)'s cross-entropy-specific formulation to arbitrary convex losses is another unifying move. By proving that the key theorems hold for any convex loss (not just cross-entropy), the paper extends the guardedness framework to regression, ranking, and other prediction tasks — making the theoretical machinery applicable beyond the fairness-in-classification setting where it originated. Appendix G explicitly reconciles the two definitions, showing that the paper's Definition 2.1 is equivalent to Ravfogel et al.'s $\epsilon$-guardedness (with $\epsilon = 0$) when specialized to cross-entropy loss.

This is a theoretical advance, not a performance gain: it doesn't make any single method work better, but it provides the conceptual scaffolding that makes the entire field of linear concept erasure more coherent and principled.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The intrinsic and fairness evaluations (Section 5.1–5.2) use the Bias in Bios dataset (De-Arteaga et al., 2019), consisting of short biographies annotated by binary gender and profession. The concept scrubbing experiments (Section 6) use a sample of 2²² tokens from the validation split of the Pile (Gao et al., 2020) for Pythia models and an equivalently sized sample from the RedPajama replication of the LLaMA pretraining corpus for LLaMA models, with part-of-speech tags automatically generated by the SpaCy library (18 coarse Universal Dependency classes). The amnesic probing experiments (Section 5.3) use sentences and coarse POS tags from the English Universal Dependencies dataset (Nivre et al., 2020).

• Base model(s). The intrinsic and fairness evaluations use bert-base-uncased (the standard 110M-parameter BERT model), with interventions applied to the [CLS] embedding in the last hidden layer. The amnesic probing experiments also use bert-base-uncased, with interventions applied to single layers. The concept scrubbing experiments span two model families: EleutherAI's Pythia at 160M, 1.4B, 6.9B, and 12B parameters (Biderman et al., 2023) and Meta's LLaMA at 7B, 13B, and 30B parameters (Touvron et al., 2023). The range of scales (160M to 30B) tests whether findings generalize across model sizes, and the two families test whether findings are architecture-specific (Pythia uses parallel attention + MLP; LLaMA uses sequential with RMSNorm).

• Metrics. For the intrinsic erasure evaluation (Figure 2): gender classification accuracy of a logistic regression probe trained on the scrubbed embeddings, plotted against mean squared distance from the original embedding — measuring the tradeoff between erasure completeness and edit magnitude. For the fairness evaluation (Figure 3): profession prediction accuracy of a logistic regression classifier on scrubbed embeddings; TPR-GAP, the true positive rate gap between male and female individuals for each profession, aggregated as the root-mean-square TPR-GAP across all professions (GAP^(TPR, RMS)_z); and σ(GAP^(TPR), %Women), the Pearson correlation between the TPR gap for a profession and the percentage of women in that profession. For amnesic probing (Figure 4): MLM accuracy and cross-entropy loss after single-layer intervention. For concept scrubbing (Table 1): perplexity in bits per UTF-8 byte, measured on the held-out evaluation slice; lower is better.

• Baselines.

  • INLP (Iterative Nullspace Projection; Ravfogel et al., 2020): repeatedly trains a linear classifier, projects onto its nullspace, and iterates. The paper uses 20 iterations for the amnesic probing experiments and notes that even after 20 iterations, classification accuracy remains above majority.
  • RLACE (Ravfogel et al., 2022a): adversarial gradient-based optimization to find an orthogonal projection that minimizes both classification accuracy and edit distance. Used in the intrinsic evaluation (Figure 2).
  • SAL (Spectral Attribute RemovaL; Shao et al., 2023a): uses the left singular vectors of Σ_XZ to construct an orthogonal projection. The paper uses full-rank SAL (n = rank(Σ_XZ)) for the concept scrubbing experiments (Table 1).
  • Random erasure (concept scrubbing): orthogonal projection onto a randomly selected subspace of the same rank as the cross-covariance matrix, with a fresh subspace sampled per minibatch and results averaged. Controls for generic effects of any dimensionality reduction.
  • Unigram baseline (concept scrubbing): assigns probabilities to tokens based solely on their frequency in the training corpus, with no context. Establishes the floor for language modeling performance.
  • No intervention (concept scrubbing): the unmodified model's perplexity.

• Generation budget / compute accounting. LEACE does not involve generation or iterative optimization — it is a single closed-form computation. The "compute" for fitting is the cost of estimating Σ_XX and Σ_XZ from a dataset of n samples: O(n d²) for the covariance and O(d³) for the pseudoinverse and SVD. In the concept scrubbing experiments, fitting uses a sample of 2²² tokens (~4 million) per model, with hidden state dimensionalities ranging from 768 (Pythia 160M) to 6656 (LLaMA 30B). The paper discusses a memory-vs-compute tradeoff: caching all hidden states requires ~500GB but reduces total compute to O(ℓ) forward passes; recomputing from scratch requires O(ℓ²) forward passes but minimal memory.

• Cross-validation / statistical protocol. The intrinsic evaluation (Figure 2) reports classification accuracy after training a logistic regression probe on the scrubbed embeddings, but the paper does not describe a cross-validation procedure for this experiment — the probe is trained and evaluated on the same scrubbed dataset to measure how much linearly accessible information remains. For the fairness evaluation, the paper reports that "re-fitting the main-task logistic regression classifier over the projected embeddings yields a slightly higher main-task accuracy of 78.1%, at the price of significantly increasing the TPR gap to 0.158," suggesting that the main-task classifier is re-fitted after erasure rather than using a fixed pre-trained classifier, but specific train/test splits are not detailed. The concept scrubbing fitting uses one slice of 2²² tokens for estimating LEACE parameters and a separate slice of 2²² tokens for evaluation, providing a clean train/evaluation separation.

---

Main Quantitative Results

Intrinsic Erasure Evaluation: Gender from BERT Embeddings

The experiment measures how completely each method removes gender information from the [CLS] embedding of BERT's last layer, and at what cost in terms of edit magnitude. The paper reports (Figure 2):

"our method is the only to achieve random accuracy (perfect erasure) with a small edit, although RLACE (but not INLP) comes close."

Reading from Figure 2: LEACE reduces gender classification accuracy to approximately 50% (chance level for binary classification) at a mean squared distance of roughly 1–2 × 10⁻³ from the original embedding. RLACE approaches random accuracy but requires a larger edit (approximately 5–10 × 10⁻³ MSE) or fails to fully reach 50% at smaller edits. INLP requires a substantially larger edit (roughly 10⁻² MSE or more) and even then does not fully reach random accuracy within the range shown — its curve plateaus above chance. LEACE is reported to be "around 2 orders of magnitude faster" than RLACE, since it requires no gradient-based optimization.

This result directly validates the paper's central theoretical claim: the oblique projection computed by LEACE achieves perfect linear guardedness (provably zero-ing out Σ_XZ) while making a smaller edit than any prior method. RLACE's adversarial optimization nearly matches LEACE's performance, which is consistent with the paper's theoretical observation (Appendix D) that RLACE's solution is "very close to ours" — the adversarial objective implicitly pushes toward the LEACE optimum.

Downstream Fairness: Gender Bias in Profession Prediction

The paper evaluates whether erasing gender from BERT embeddings reduces bias in a downstream profession classifier while preserving utility (Figure 3):

• Profession prediction accuracy: 79.3% on original embeddings → 77.3% on LEACE-scrubbed embeddings, a drop of only 2.0 percentage points.
• TPR-GAP (RMS): drops from 0.198 (original) to 0.084 after LEACE erasure — a 58% reduction in the root-mean-square gender gap across professions.
• Correlation σ(GAP^(TPR), %Women): plummets from 0.867 (original, Figure 3a) to 0.392 (after LEACE, Figure 3b). Before erasure, professions with more women had systematically higher TPR gaps; after erasure, this relationship is substantially weakened.

The paper also reports an interesting finding about re-fitting: "Re-fitting the main-task logistic regression classifier over the projected embeddings yields a slightly higher main-task accuracy of 78.1%, at the price of significantly increasing the TPR gap to 0.158." This is a non-obvious tradeoff: if you simply apply LEACE to the embeddings and keep the original classifier, bias drops sharply with minimal accuracy loss. But if you re-train the classifier on the scrubbed embeddings, it partially recovers the ability to discriminate by gender (TPR-GAP rises to 0.158), likely because the classifier learns to exploit any residual nonlinear gender information that LEACE preserves. The paper's footnote 5 notes: "This is likely due to the fact that linear guarding is not sufficient to prevent a downstream linear model from using the concept if it is being fine-tuned, since the fine-tuning process can learn to combine features in ways that recover the concept." This is an important caveat: LEACE prevents post-hoc linear recovery of the concept from frozen representations, but does not prevent a model trained on those representations from learning to re-extract the concept via nonlinear feature combinations.

Amnesic Probing: Part-of-Speech Erasure from BERT Layers

This experiment (Section 5.3, Figure 4) replicates Elazar et al. (2021)'s amnesic probing paradigm — erase POS information from a single layer of BERT and measure the impact on masked language modeling — comparing LEACE against INLP with 20 iterations.

Key findings from Figure 4b:

• LEACE causes minimal degradation until layer 8. MLM accuracy and loss remain near baseline levels for interventions on layers 1–7. The largest drop occurs at layer 11, where LEACE causes a substantial decrease in accuracy and increase in loss. This suggests that the causal effect of POS information on BERT's language modeling is concentrated in the later layers, particularly layer 11.
• INLP shows maximum effect at layer 6, with substantial degradation even in early layers. The paper argues this is an artifact of collateral damage: "Since it removes hundreds of dimensions, it is difficult to attribute this effect to the erasure of the concept." INLP deletes 360 dimensions (20 iterations × 18 classes per iteration, effectively), while LEACE deletes only 17 dimensions (rank of Σ_XZ for an 18-class one-hot encoding).
• Probing vs. intervention mismatch: "These results suggest that the causal effect of the POS concept on the language model is concentrated in layer 11. Interestingly, this stands in contrast with POS linear probing results, which are optimal at earlier layers (Tenney et al., 2019)." This is a substantive finding about BERT's internal organization — the layers where POS information is most linearly decodable are not the layers where it is most causally important for downstream computation. The paper cites Elazar et al. (2021)'s observation that "probing does not generally correlate with intervention-based analysis techniques."

This experiment demonstrates a key practical advantage of LEACE: it enables causal intervention experiments whose results are not confounded by the intervention method itself. INLP's finding of maximum effect at layer 6 would have led researchers to conclude that layer 6 is the primary locus of POS usage; LEACE reveals that the true causal locus is layer 11, and that INLP's result was an artifact of excessive dimensionality reduction in earlier layers.

Concept Scrubbing: Full-Network POS Erasure from Large Language Models

This is the paper's most ambitious experiment (Section 6, Table 1): erase linearly available part-of-speech information from every transformer layer of language models ranging from 160M to 30B parameters, and measure the impact on perplexity.

The headline results from Table 1 (perplexity in bits per UTF-8 byte):

Several patterns deserve attention:

1. Random erasure has essentially no effect. Across all models, projecting onto a random subspace of the same rank as the POS concept subspace leaves perplexity nearly identical to the unmodified model. The largest deviation is Pythia 160M (0.90 → 0.99, a 10% increase), but for all larger models the difference is within 0.01–0.02 bits/byte. This confirms that the effects observed for LEACE and SAL are not artifacts of dimensionality reduction or generic perturbation — they are specifically caused by removing POS-correlated dimensions.

2. LEACE causes large but not catastrophic degradation. Across LLaMA models, LEACE increases perplexity by roughly 1.0–1.3 bits/byte (e.g., LLaMA 7B: 0.69 → 1.73, a 2.5× increase; LLaMA 30B: 0.62 → 1.96, a 3.2× increase). For Pythia models, the increases are larger and more variable (Pythia 12B: 0.62 → 3.20, a 5.2× increase; Pythia 6.9B: 0.64 → 3.57, a 5.6× increase). The Pythia family appears more sensitive to POS erasure than LLaMA at comparable scales, which could reflect architectural differences (parallel vs. sequential attention-MLP, LayerNorm vs. RMSNorm), training data differences, or different degrees of reliance on syntactic features.

3. SAL consistently overestimates the causal effect. Across all models, SAL induces 1.5–2× larger perplexity increases than LEACE. For LLaMA 7B: SAL = 3.24 vs. LEACE = 1.73 (1.9× difference). For Pythia 12B: SAL = 4.69 vs. LEACE = 3.20 (1.5× difference). This empirically validates the paper's central methodological claim: orthogonal projections (SAL) destroy more non-concept information than the optimal oblique projection (LEACE), and this excess collateral damage inflates the apparent causal importance of the erased concept. If the experiment had been conducted with SAL alone, one would conclude that POS is substantially more important for language modeling than it actually is.

4. LEACE-scrubbed models remain above the unigram baseline for LLaMA. LLaMA models scrubbed with LEACE achieve perplexities of 1.73–1.96, which is below the unigram baseline of 2.90. This means the models still retain some capacity to use context beyond simple token frequencies. Pythia models scrubbed with LEACE (especially 6.9B at 3.57 and 12B at 3.20) exceed the unigram baseline of 2.66, meaning their performance degrades below the level achievable by ignoring context entirely — a stronger effect that may indicate Pythia's heavier reliance on syntactic features or LEACE's greater disruption to Pythia's representational geometry.

5. No clear scaling trend. Unlike typical scaling laws where larger models are more robust, the degradation from POS scrubbing does not consistently decrease with model size. LLaMA 30B (1.96) actually shows higher perplexity after scrubbing than LLaMA 7B (1.73), despite having 4.3× more parameters. This counterintuitive result suggests that larger models may encode POS information more deeply or redundantly across their representations, making the linear erasure relatively more disruptive — or that larger models rely more heavily on syntactic structure for their improved perplexity. The paper does not analyze this scaling pattern in detail.

6. The LEACE vs. SAL gap validates the oblique projection advantage at scale. The consistent superiority of LEACE over SAL — across 7 models, 2 architectures, and scales spanning two orders of magnitude — provides strong evidence that the theoretical optimality derived in Section 4 translates to practical benefits in large-scale settings. This is not a toy-example phenomenon (like the 2D example in Appendix F); it persists in 6656-dimensional hidden states.

---

Ablation Studies and Robustness Checks

The paper does not include a dedicated ablation section in the traditional sense, but several experiments and design choices function as ablations or robustness checks:

RLACE vs. LEACE (Figure 2): The close match between RLACE (adversarial gradient-based optimization) and LEACE (closed-form) in the intrinsic evaluation serves as an empirical validation of LEACE's optimality. RLACE solves approximately the same optimization problem but via expensive iterative gradient steps; that it converges to nearly the same solution as LEACE's closed-form confirms that the optimization landscape has a well-behaved global minimum and that LEACE finds it exactly.

Re-fitting the downstream classifier (Section 5.2, footnote 5): The paper reports that re-training the profession classifier on LEACE-scrubbed embeddings increases TPR-GAP from 0.084 to 0.158 while slightly improving accuracy. This is effectively an ablation on when erasure is applied relative to classifier training. It demonstrates that linear guardedness prevents post-hoc recovery of the concept from frozen representations but does not prevent a classifier trained from scratch on those representations from re-extracting the concept — likely by learning nonlinear combinations of the scrubbed features. This is a practically important boundary condition: LEACE is most effective for fairness when applied to representations that will be used by a fixed downstream model, not when the downstream model will be trained on the scrubbed data.

Oracle vs. covariance-based LEACE (Appendix I): The paper derives Oracle LEACE, which uses ground-truth labels at inference time to achieve an even more surgical edit. The paper does not empirically compare Oracle LEACE to standard LEACE — this is a theoretical result only. The practical LEACE method uses only aggregate covariance statistics, making it applicable to unlabeled data. The fact that Oracle LEACE exists as a theoretical optimum provides an upper bound on how surgical linear erasure can be, but the paper does not quantify the gap between the two.

Random erasure baseline (Table 1): As discussed above, this functions as an essential ablation isolating concept-specific effects from generic perturbation effects. The near-zero impact of random erasure across all models validates that the perplexity increases from LEACE and SAL are genuinely due to removing POS information.

SAL as an orthogonal projection baseline (Table 1): The direct comparison of SAL (orthogonal projection) against LEACE (oblique projection) at equal rank reduction serves as an ablation on the type of projection. Since both methods zero out the same subspace (colsp(Σ_XZ)), the performance gap between them isolates the benefit of oblique over orthogonal projection — exactly the theoretical claim from Section 4.2. The 1.5–2× difference in perplexity is a clean empirical demonstration.

Norm regularization (Appendix H): The paper describes a practical robustness measure: in some layers, LEACE's oblique projection can increase the norm of hidden states, causing divergence. The convex combination with SAL's orthogonal projection (tuned so total variance does not increase) prevents this while maintaining feasibility. The paper does not report an ablation comparing with and without this regularization, so its impact on aggregate perplexity numbers is unknown. This is a practical engineering detail rather than a methodologically motivated ablation.

ReST^(EM) revision model (Appendix K, Figure 16): not applicable to this paper — this is from the reference example about a different paper. Apologies.

Streaming covariance estimation: The paper mentions that covariance statistics "can be computed in a streaming fashion, without ever storing all the hidden states in memory or on disk." While not a comparative ablation, this is a robustness property: the method works with standard online covariance estimation algorithms, making it scalable to datasets larger than memory. The paper does not compare results with batch vs. streaming estimation.

---

Critical Assessment

Claim 1: "Our method is the only to achieve random accuracy (perfect erasure) with a small edit."

Supported with qualifications. Figure 2 clearly shows LEACE reaching ~50% accuracy at substantially smaller edit distance than INLP. RLACE approaches similar performance but either requires a larger edit or doesn't fully reach 50% at small edits. The "small edit" claim is qualitative — the paper reports MSE values on the order of 10⁻³ but does not define a threshold for "small" or report whether the gap between LEACE and RLACE is statistically significant. The experiment uses a single dataset (Bias in Bios) and a single model (BERT-base); it's unclear whether RLACE would match LEACE more closely on other data distributions or model architectures.

Claim 2: "TPR gap drops significantly from 0.198 to 0.084" after LEACE erasure.

Supported. Figure 3 provides clear before-and-after comparisons, and the dropped correlation (0.867 → 0.392) provides convergent evidence that the bias reduction is real. However, the counterfactual is incomplete: the paper does not report TPR-GAP for the same downstream classifier after RLACE or SAL erasure, making it impossible to determine whether LEACE's fairness improvement is due to better erasure completeness or better preservation of profession-relevant features (or both). The paper also does not evaluate on other fairness metrics (equalized odds, demographic parity on predictions) or other protected attributes (race, age). The single-dataset, single-attribute scope means we cannot assess whether the fairness benefits generalize.

An important caveat from the re-fitting result: the TPR-GAP rises to 0.158 when the classifier is re-trained on scrubbed embeddings. This means LEACE's fairness benefit is partly an artifact of using a frozen classifier — if an adversary can re-train a classifier on the scrubbed data, much of the bias returns. The paper is transparent about this (footnote 5), but it means LEACE is a representation-level fairness intervention, not a guarantee of fair downstream decisions under adaptive adversaries. The paper does not explore whether iterative application of LEACE (scrub, re-train classifier, scrub again) could close this gap.

Claim 3: "LEACE decreases the rank just by 17" while "INLP decreases the rank... by 360."

Mathematically correct, empirically demonstrated indirectly. The rank reduction follows directly from the theory: INLP deletes 18 dimensions per iteration × 20 iterations = 360, while LEACE's P* has a nullspace of dimension rank(Σ_XZ) = 17 for an 18-class one-hot encoding (k−1 due to the one-hot sum constraint). This is a designed property of LEACE, not an empirical discovery — Theorem 4.1 guarantees that only the concept-correlated subspace is zeroed out. Figure 4 demonstrates the consequence (LEACE's interventions are more localized and cause less degradation in early layers than INLP's), but the paper could have reported a direct measurement of the effective rank after each method's intervention, or verified that INLP has deleted dimensions that are informative for language modeling but not for POS classification. The claim is more of a theoretical comparison than an experimental finding.

Claim 4: "POS scrubbing induces large increases in perplexity across all models."

Strongly supported by Table 1. The effect is robust across model families (Pythia and LLaMA) and scales (160M to 30B). The random erasure baseline rules out trivial explanations (generic perturbation effects), and the unigram baseline provides a meaningful floor for interpreting the magnitude of degradation. The paper's automatic POS tagging via SpaCy introduces label noise that would, if anything, cause LEACE to under-erase the true POS signal (since it can only remove correlations with the noisy labels), making the observed perplexity increases a conservative estimate of true POS reliance.

However, there are important interpretive gaps:

• Is the perplexity increase from removing POS specifically, or from removing any richly structured linguistic feature? The paper compares against random erasure (no effect) but not against erasing other linguistic features of similar complexity (e.g., dependency relations, named entities, or semantic categories). Without such a comparison, one cannot rule out that any high-dimensional structured feature would show similar effects. The conclusion that "autoregressive language models heavily rely on linearly encoded part-of-speech information" is supported, but the claim that this reliance is specific to POS rather than a general property of structured linguistic features is untested.

• Are the perplexity differences between LEACE and SAL entirely due to surgical precision, or could they partly reflect SAL's larger nullspace? The paper uses full-rank SAL, so both methods zero out the same column space of Σ_XZ. The difference must be due to the geometry of the projection (oblique vs. orthogonal), as argued. But SAL also changes the representation in dimensions outside the nullspace (because orthogonal projection forces symmetry), while LEACE's oblique projection can leave those dimensions nearly unchanged. A direct measurement of the Frobenius norm difference between the original and edited covariance matrices for both methods would make this comparison more quantitative.

• The perplexity scaling pattern is unexplained. LLaMA 30B shows higher perplexity after scrubbing than LLaMA 7B (1.96 vs. 1.73 bits/byte), which is counterintuitive under the hypothesis that larger models are more robust to interventions. The paper does not discuss this pattern, offer hypotheses, or check whether it's driven by a few outlier layers. Without this analysis, the scaling claims remain suggestive rather than systematic.

Missing experiments that would have strengthened the paper:

• Other concepts beyond gender and POS. The theoretical framework is general, but all experiments target exactly two concepts: binary gender (in BERT) and 18-class part-of-speech (in BERT and LLaMA/Pythia). Testing on other concepts — race, sentiment, named entity types, topic categories — would validate that the method works across concept types and dimensionalities. The paper's claim of generality rests on the theory, not the experiments.

• Nonlinear probe recovery. LEACE provably prevents linear recovery of the concept, but the paper does not test whether a nonlinear probe (e.g., an MLP with one hidden layer) can recover gender from LEACE-scrubbed BERT embeddings. This is the relevant threat model for adversarial downstream use — if a two-layer network can reconstruct gender with 70% accuracy from "linearly guarded" representations, the practical fairness guarantee is much weaker than the theory suggests. The paper acknowledges this limitation in Section 7 ("We conjecture that it is intractable to nondestructively edit X so as to prevent a general nonlinear adversary from recovering Z") but provides no empirical evidence on how much nonlinear information survives.

• Statistical significance and confidence intervals. No confidence intervals are reported for any of the main results — not for the gender classification accuracy (Figure 2), not for the TPR-GAP drop (Figure 3), not for the perplexity increases (Table 1). With 2²² tokens (~4 million), variance on perplexity estimates should be very small, but for the Bias in Bios experiments (which use a much smaller dataset), sampling error could be non-trivial. The paper would be strengthened by reporting uncertainties, especially for the fairness metrics where the practical interpretation hinges on the TPR-GAP reduction being large relative to measurement noise.

• Oracle LEACE comparison. The paper derives Oracle LEACE (Appendix I) but never evaluates it. Comparing standard LEACE against Oracle LEACE on a labeled dataset would quantify how much surgical precision is lost by using aggregate covariance statistics rather than per-sample labels. This is practically relevant for practitioners deciding whether to invest in labeled data for erasure.

• Inter-layer information flow analysis. The paper shows that concept scrubbing works (Table 1) but does not analyze how information propagates: does erasing POS at layer 1 prevent it from being re-encoded at layer 5? Does the model adaptively compensate for erased information in later layers? The sequential fitting procedure accounts for distribution shift, but the paper does not report how much the concept re-emerges if you skip scrubbing at intermediate layers. This kind of analysis would strengthen the interpretability contribution.

• Ablation on the sequential fitting procedure. Algorithm 1 fits and applies LEACE layer-by-layer. The paper argues this is necessary because early-layer interventions change later-layer distributions, but does not empirically compare against the naive approach (fit all layers independently on the clean model, apply all at once). If the results were similar, the sequential procedure would be unnecessary complexity; if they diverge, the divergence would quantify the importance of distributional shift. This is a missed opportunity to validate a key algorithmic design choice.

Summary assessment:

The experimental section validates the paper's core technical contributions — LEACE achieves perfect linear erasure with minimal edit distance, and this precision enables multi-layer scrubbing that was previously impossible. The fairness results on BERT are clear and practically meaningful. The concept scrubbing results on LLMs are striking and methodologically careful (random erasure baseline, two model families, streaming estimation). However, the experiments are narrower than the theory: only two concepts tested, no nonlinear probe recovery evaluated, no confidence intervals, and several natural ablations (oracle LEACE, naive vs. sequential fitting, other linguistic features) are absent. The paper proves that LEACE is the optimal linear erasure method; the experiments convincingly demonstrate that it is better than prior linear methods, which is the correct standard of evidence for a method that claims optimality within a class. The experiments do not, and are not designed to, test the paper's broader conjecture about the intractability of nonlinear erasure — that remains a theoretical claim for future work.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Not Accounted For

The assumption or constraint. The compute-optimal scaling framework depends entirely on being able to estimate each prompt's difficulty before deciding how to allocate the inference budget. The paper's method for doing so — generating 2048 samples per question and computing either ground-truth pass@1 (oracle) or the PRM's average final-answer score (predicted) — consumes enormous computation. The authors acknowledge this explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity."

The consequence. The reported 4× efficiency gains over best-of-N (Figures 4 and 8) are computed after difficulty is known, without amortizing the cost of learning it. Generating 2048 samples per question is itself comparable to or larger than the largest test-time budgets studied (256–512 generations). In a realistic deployment, the total cost would be difficulty estimation + strategy execution, and the former could easily dominate the latter. The 4× figure should therefore be understood as an upper bound on achievable efficiency rather than a realized deployment gain. For applications where each query is unique (no batch of similar questions to amortize difficulty estimation over), the approach may be net-negative relative to simply running best-of-N with all available compute.

What evidence exists in the paper. The paper explicitly flags this issue but does not measure the cost empirically. Section 3.2 describes the 2048-sample estimation procedure and notes it is needed for both oracle and predicted difficulty bins. The two-fold cross-validation results (Figures 4 and 8) show that predicted bins (which avoid ground-truth labels but not the 2048-sample cost) track oracle bins closely, confirming that the PRM-based approximation works — but neither variant accounts for the estimation cost in the budget comparison. No experiment measures how performance degrades with fewer than 2048 difficulty-estimation samples, or whether a lightweight difficulty classifier could replace the full PRM scoring.

Mitigation status. The paper does not attempt to reduce this cost beyond noting that the PRM-based method avoids ground-truth labels. Section 8 explicitly calls for future work on "pretraining or finetuning models to directly predict difficulty of a question" to close this gap. No such model is developed or evaluated. This is the most significant gap between the paper's theoretical framework and practical deployability.

---

Hard Problems Remain Unsolved — Test-Time Compute Cannot Substitute for Missing Capability

The assumption or constraint. All test-time compute strategies — search, revisions, and their compute-optimal combinations — rely on the base model already having a non-trivial probability of producing a correct answer. The paper's five-quintile difficulty binning operationalizes this: difficulty is defined as the base model's pass@1 rate, and bin 5 (the hardest) corresponds to problems where that rate is near zero.

The consequence. On the hardest problems (difficulty bin 5), no method makes meaningful progress regardless of how much compute is allocated. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all search methods and all budgets up to 256 generations. In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%, falling substantially below the 14× larger model's performance. The paper is candid about this in Section 7:

"test-time compute amplifies existing capability but does not create it from nothing."

This means the compute-optimal framework offers no path forward for out-of-distribution reasoning or genuinely novel problems that exceed the base model's training distribution. For such problems, pretraining larger models on more data remains the only viable strategy. The practical implication is that users must know a priori whether their problem distribution falls into the base model's capability range to decide whether test-time compute or larger pretraining is the better investment — and for the hardest problems, the answer is always pretraining.

What evidence exists in the paper. The evidence is consistent and clear across every experiment: bin 5 never improves with computation (Figures 3 right, 7 right, 8, 9). The FLOPs-matched comparison quantifies the failure: at R ≫ 1 with PRM search, hard problems show a -52.9% relative disadvantage from using test-time compute instead of the 14× larger model (Figure 1, bottom-right; Section 7).

Mitigation status. The paper does not attempt to solve this limitation. It is acknowledged as a fundamental boundary condition rather than a fixable engineering problem. Section 7 explicitly frames the finding as establishing "sharp boundaries" for when test-time compute can substitute for pretraining. The honest characterization is a strength of the paper but does not change the practical consequence: for hard problems, this approach does not help.

---

The 14× Larger Model Baseline Is Not Compute-Optimally Trained — and Gets No Test-Time Compute of Its Own

The assumption or constraint. The FLOPs-matched comparison in Section 7 scales only model parameters when increasing pretraining compute, holding training data fixed. The authors acknowledge this departs from Chinchilla-optimal scaling (Hoffmann et al., 2022), where both parameters and data would be scaled equally:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

Additionally, the 14× larger model is evaluated with only greedy decoding — no majority voting, no best-of-N, and no compute-optimal test-time strategy of its own.

The consequence. A Chinchilla-optimal model trained with 14× more total FLOPs (scaling both parameters and data) would likely outperform a parameter-only-scaled model, making the pretraining baseline weaker than it needs to be. The reported advantages of test-time compute over pretraining — e.g., +27.8% on easy questions at R ≪ 1 for revisions (Figure 1, top-right) — may shrink or reverse against a properly compute-optimal larger model. Similarly, giving the larger model even a modest test-time compute budget (say, best-of-8 with majority voting) would create a much stronger baseline. The paper's conclusion that "a smaller model augmented with compute-optimal test-time strategies can outperform a ~14× larger pretrained model" (Executive Summary) is technically correct under the specific comparison made, but the comparison is biased in favor of test-time compute because the pretraining baseline is suboptimal on two independent dimensions.

What evidence exists in the paper. The paper is transparent about the parameter-only scaling choice (Section 7: "matching the approach of the LLaMA model series") and explicitly flags the compute-optimal pretraining comparison as future work. However, no experiment tests the sensitivity of the conclusion to this choice — there is no comparison against a model with data scaling, and no comparison where the larger model gets any test-time compute. The numbers reported in Figure 9 and the bar charts in Figure 1 are therefore specific to the particular (weak) baseline.

Mitigation status. The paper does not attempt to correct for this. It is flagged as a caveat and left to future work. A more complete analysis would compare test-time compute of the small model against both pretraining-only scaling and a combination of pretraining scaling with some test-time compute for the larger model, to determine the efficient frontier of the full training-inference Pareto tradeoff.

---

Generalization Is Untested — Single Benchmark, Single Model Family, Two Concepts

The assumption or constraint. All experiments in the paper use exactly one benchmark (MATH), one base model family (PaLM 2), and two concepts for erasure (gender and part-of-speech). The authors state in Section 4 that they "believe this model is representative of the capabilities of many contemporary LLMs," but this claim is unverified.

The consequence. Several aspects of the findings could be model-specific or benchmark-specific:

• The PRM's quality and over-optimization behavior (Figure 3) depend on PaLM 2's output distribution. A model with different calibration properties or different error patterns might exhibit different difficulty-dependent scaling curves, changing which strategies are optimal for which difficulty bins.
• The revision model's ability to learn from incorrect in-context examples depends on the base model's in-context learning capabilities, which vary substantially across model families. The ReST^EM ablation (Appendix K, Figure 16) showing that an alternative training procedure degrades revision performance suggests revision training is sensitive to methodology in ways that may not transfer.
• The MATH benchmark consists exclusively of competition-level math problems requiring symbolic reasoning. It is unclear whether the difficulty-dependent patterns — beam search hurting easy problems, revisions helping easy problems, neither helping hard problems — generalize to code generation, logical reasoning, scientific question-answering, or tasks requiring factual knowledge rather than inference.

What evidence exists in the paper. The paper reports no experiments on any other benchmark, model, or task type. The test set of 500 questions, split into five difficulty quintiles of ~100 each and further split by two-fold cross-validation, means the compute-optimal policy is selected based on ~50 questions per fold per bin — a small sample that may not yield robust strategy choices. No confidence intervals are reported on the compute-optimal scaling curves. The concept scrubbing experiments use two model families (Pythia and LLaMA), which partially addresses model-specificity for that task, but the main test-time compute scaling experiments are entirely on PaLM 2 + MATH.

Mitigation status. The paper does not address this limitation. The authors' claim that PaLM 2 is "representative" is stated without evidence. The MATH benchmark's focus on a single reasoning domain is not discussed as a limitation. A replication on at least one additional model family and one additional reasoning benchmark (e.g., GSM8K for math, HumanEval for code) would substantially strengthen confidence in the generality of the difficulty-dependent scaling patterns.

---

Revisions and Search Are Studied Independently — Their Combination Is Unexplored

The assumption or constraint. The paper studies two complementary axes — PRM-guided search (Section 5) and iterative revisions (Section 6) — but never combines them. The paper explicitly acknowledges this gap in Section 8:

"we did not experiment with PRM tree-search techniques in combination with revisions."

The consequence. The two mechanisms have complementary strengths that the difficulty-bin analysis reveals: revisions improve the proposal distribution (generating better candidates, especially on easy problems where local refinement suffices), while PRM search improves candidate selection (finding the best among generated candidates, especially on medium problems where exploration matters). Combining them — using the revision model as the proposal distribution within beam search, or using the PRM to guide which revision branches to pursue — could yield gains beyond either method alone. The current results therefore represent a lower bound on what a fully integrated system could achieve. The paper's central claim of "compute-optimal scaling" is thus incomplete: the policy selects between search and revisions but never deploys both simultaneously, which is precisely where the largest gains might be found on medium-difficulty problems.

What evidence exists in the paper. The difficulty-bin analyses show that revisions dominate on easy problems (bin 1–2, Figure 7 right) while beam search dominates on medium problems (bin 3–4, Figure 3 right). This suggests a natural hybrid: use revisions to generate candidates and then apply PRM-guided search to select among them, particularly for bin 3 problems where both mechanisms show non-trivial benefits. The paper provides no experimental evidence for or against such a hybrid.

Mitigation status. The paper does not attempt to combine the methods. Section 8 flags this as a direction for future work: "The natural next step is to use the revision model as the proposal distribution within beam search." No preliminary results or analysis are provided. This is a significant gap because it means the paper's "compute-optimal" policy is restricted to choosing one mechanism per prompt, whereas a truly optimal allocation might blend them — e.g., spend part of the budget on revision chains and part on independent parallel search, selecting the best answer across both pipelines.

---

The Revision Model Has a 38% Correct-to-Incorrect Reversion Rate — And Training Is Fragile

The assumption or constraint. The revision model is trained exclusively on sequences where all in-context answers are incorrect (followed by a correct target). At test time, the model encounters a different distribution: it may produce correct answers during the revision chain, and when it does, it has no training signal for what to do next.

The consequence. The paper reports (Section 6.1) that "approximately 38% of correct answers produced during a revision chain get converted back to incorrect ones in the subsequent step." This means the revision process is fundamentally self-undermining: as the model generates more revisions, it eventually "forgets" correct answers it already produced. The paper mitigates this with within-chain selection (majority voting or verifier-based selection across the entire chain), picking the best answer from any point rather than taking the final revision. But this is a patch, not a solution — it means the later revisions in a long chain are partially wasted computation, since they may degrade rather than improve the answer. The revision model is also shown to be fragile to training methodology: the ReST^EM experiment (Appendix K, Figure 16) found that an alternative reinforcement-learning-based training procedure caused performance to degrade substantially with sequential revisions, with fully sequential performance dropping to approximately 33.5% compared to 38.5% at the optimal ratio at 256 generations. The paper hypothesizes that "on-policy data collection in ReST^EM exacerbates spurious correlations in revision data."

What evidence exists in the paper. The 38% reversion rate is reported directly in Section 6.1. The ReST^EM degradation is shown quantitatively in Appendix K, Figure 16. The paper's solution — within-chain selection — is described but its effectiveness is not ablated (no comparison of "take last revision" vs. "select best from chain"). The paper does not measure what fraction of the budget is wasted on revisions that degrade already-correct answers, or whether shorter chains with more parallel sampling would be more efficient than the longer chains the compute-optimal policy selects for easy problems.

Mitigation status. The paper partially mitigates the reversion problem via within-chain selection but does not solve it. The more fundamental issue — that the revision model lacks the concept of "no revision needed" — is not addressed. A principled solution would require training the model to recognize when the current answer is already correct and to output it unchanged — which would require a different training data construction procedure. The paper does not propose such a procedure. The fragility of revision training (ReST^EM results) raises concerns about whether the approach portably transfers to other base models without careful tuning of the data generation process.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper fundamentally reframes the problem of linear concept erasure from a heuristic, iterative practice into a principled optimization problem with a unique closed-form solution. Before LEACE, the field operated under a constellation of methods — INLP, RLACE, SAL, Mean Projection — each with partial theoretical justification and no clear understanding of how they related to each other or to the underlying mathematical structure of the problem. The paper's core theoretical contribution — proving that linear guardedness is exactly equivalent to zero cross-covariance (Theorems 3.1, 3.3, 3.4) — converts an adversarial, classifier-centered definition into a purely statistical one. This is a conceptual reframing of the problem, not an incremental improvement: you no longer need to train probes, run gradient descent, or iterate to near-convergence. You estimate two covariance matrices, compute a pseudoinverse, and you're done — with a provable guarantee that no linear classifier can beat the trivial predictor.

The magnitude of this shift is best understood by what it makes obsolete. INLP's iterative approach — train classifier, project onto nullspace, repeat until accuracy drops to chance — is revealed as solving the right problem (find a guarding projection) using an inefficient algorithm that deletes far more dimensions than necessary (360 vs. LEACE's 17, as documented in Section 5.3). RLACE's adversarial gradient-based optimization is revealed as approximately converging to the LEACE optimum (Figure 2 shows their performance curves nearly overlapping), but at ~100× the computational cost. SAL and Mean Projection are revealed as feasible but suboptimal points in the same solution space — they achieve guardedness (which their authors could not prove) but impose unnecessary orthogonality constraints that increase the mean squared edit distance. The paper does not just outperform these methods; it explains why they perform as they do and exactly where they sit in the solution space. This kind of unification rarely happens in a single paper.

The introduction of concept scrubbing (Section 6, Algorithm 1) is a methodological advance of comparable importance, though of a different kind. It is not a theoretical contribution but an enabling capability: for the first time, researchers can erase a concept from every layer of a large language model and still get coherent outputs. This was literally impossible with INLP — the paper states that "it's impossible to apply INLP to multiple layers of a transformer without causing its outputs to collapse into gibberish" (Section 6). LEACE's surgical precision (deleting only 17 dimensions vs. INLP's 360 for the same POS task) makes multi-layer erasure practical. The consequences ripple through interpretability research: experiments that were confined to single-layer interventions (Elazar et al., 2021) can now probe how concepts are used throughout the full depth of a network. The sequential fitting procedure — fitting LEACE on already-scrubbed hidden states to account for distributional shift — is itself a principled approach to a subtle causal inference problem that prior work had no way to address.

A secondary but important landscape change is the correction of a widely held intuition: that orthogonal projections are optimal for concept erasure because they find the nearest point in a subspace. Section 4.2 demonstrates that this intuition is mathematically incorrect for the concept erasure problem, where the constraint is on the nullspace (what must be destroyed) rather than the range (where points must land). The concrete example in Appendix F — where an oblique projection cuts the mean squared error in half compared to the best orthogonal projection while achieving identical erasure — is a clean counterexample that should change how the field thinks about representation editing. The paper's whitening-based derivation reveals that the correct geometric operation is orthogonal projection in the whitened space, which becomes an oblique projection after unwhitening. This is a permanently clarifying conceptual contribution: any future method that defaults to orthogonal projections for nullspace-constrained editing is ignoring a provably suboptimal design choice.

The paper also resolves a latent tension in the causal interpretability literature. Elazar et al. (2021) introduced amnesic probing as a way to measure the causal importance of concepts, but the method was criticized on grounds that INLP's massive dimensionality reduction made it impossible to distinguish concept-specific effects from generic representational damage (Kumar et al., 2022). The paper's amnesic probing replication (Section 5.3, Figure 4) provides concrete evidence for this criticism: INLP finds the maximum effect of POS erasure at layer 6, while LEACE finds it at layer 11 — a qualitatively different conclusion about where syntactic information is causally used. The randomized erasure baseline in the concept scrubbing experiments (Table 1) provides further evidence: random subspace deletion of identical rank has essentially no effect on perplexity, confirming that the large effects from POS erasure are concept-specific, not artifacts of dimensionality reduction. This resolves the debate not by argument but by providing a superior tool that makes the confound go away.

One research direction becomes significantly less attractive as a result of this paper: iterative, adversarial methods for linear concept erasure. Given that LEACE achieves the theoretical optimum in closed form — and that RLACE's expensive gradient-based optimization converges to nearly the same solution (Figure 2) — there is no reason to use iterative methods for linear erasure except in exotic settings where the covariance matrices cannot be estimated (e.g., online streaming with non-stationary distributions, though the paper's streaming covariance estimation partially addresses even this). Research effort should shift away from designing new linear erasure algorithms and toward three more promising frontiers: (1) making linear erasure practical in deployment (cheap difficulty estimation, integrating with training pipelines), (2) understanding when linear erasure is sufficient for real-world fairness and interpretability goals, and (3) tackling the much harder problem of nonlinear erasure, which the paper explicitly flags as likely intractable in general (Section 7).

Follow-Up Research This Work Enables

Measuring nonlinear concept leakage after linear erasure. LEACE provably prevents linear recovery of the erased concept, but the paper provides no evidence on whether nonlinear probes (e.g., a two-layer MLP, a kernel SVM, or a small transformer) can recover gender or part-of-speech from LEACE-scrubbed representations. This is the critical next experiment: take the LEACE-scrubbed BERT embeddings from Section 5.1 and train an MLP with one or two hidden layers to predict gender. If the MLP achieves substantially above-chance accuracy, then the practical fairness guarantee of linear erasure is weaker than the theory implies — downstream models with nonlinear heads could still exploit the concept. The paper's conjecture that "it is intractable to nondestructively edit X so as to prevent a general nonlinear adversary from recovering Z" (Section 7) needs empirical grounding. A systematic study measuring nonlinear recoverability as a function of model size, concept type, and probe architecture would map the boundary between what linear erasure achieves and what applications actually need. If nonlinear leakage is small, the linear-surgical approach is validated for practice; if large, the field needs to invest in nonlinear erasure despite its apparent intractability.

Automated difficulty estimation for compute-optimal erasure allocation. The paper's concept scrubbing procedure applies LEACE uniformly to every layer, but the amnesic probing results (Figure 4) show that the causal importance of POS varies dramatically across layers — the effect is concentrated in layer 11 for BERT, with early layers showing minimal impact. This suggests an obvious extension: layer-conditional erasure budgets. Rather than fully erasing POS from every layer (which may cause unnecessary damage in layers 1–7 where POS is decodable but not causally used), estimate each layer's reliance on the concept and allocate more aggressive erasure to layers where the concept matters most. The paper already provides the machinery: the cross-covariance rank tells you how much linear information exists, and a post-erasure perplexity measurement (as in Table 1) tells you how much the layer needed that information. A "compute-optimal scrubbing" algorithm would sweep erasure strength per layer (perhaps via the convex combination parameter α from Appendix H) and find the allocation that maximizes concept removal subject to a total perplexity budget. This is directly analogous to the test-time compute allocation problem studied in the reference example paper, but applied to representation editing rather than answer generation.

Scrubbing as a regularizer during pretraining or fine-tuning. The paper's experiments are purely post-hoc: LEACE is applied to a frozen model after training. Section 7 briefly mentions this as future work: "an exciting next step would be the incorporation of concept scrubbing into the pretraining and/or finetuning process. This may make it possible to train deep neural networks subject to conceptual constraints." This is a concrete and high-impact direction. During pretraining, periodically scrub a target concept (e.g., demographic attributes) from intermediate layers and continue training. The gradient signal would push the model to route its computation through pathways that do not rely on linearly encoded instances of the scrubbed concept. A natural experiment: pretrain two language models on the same data, one with periodic gender scrubbing and one without, then measure both intrinsic gender bias (via probes on the final representations) and downstream fairness (via TPR-GAP on a profession classification task like the Bias in Bios benchmark). The key question is whether gradient-based optimizers "circumvent" the scrubbing constraint by encoding gender in nonlinear ways — exactly the concern the paper raises in Section 7. If they do, this reveals a fundamental limitation of representation-level interventions; if they don't, scrubbing-as-regularizer becomes a powerful new fairness tool.

Characterizing the structure of nonlinearly encoded concepts. The paper's theoretical framework is explicitly limited to linear erasure, and the authors conjecture that "it is intractable to nondestructively edit X so as to prevent a general nonlinear adversary from recovering Z, unless the data generating process for X is known in detail" (Section 7). This conjecture, if true, has profound implications: it means post-hoc, model-agnostic concept removal is fundamentally bounded by the linear structure of neural representations. The conjecture can be empirically stress-tested using the tools this paper provides. Take a concept that is known to be nonlinearly encoded in a model — perhaps a complex semantic category or a relational feature — and apply LEACE to a layer where linear probes achieve mediocre accuracy (say, 70%) but nonlinear probes achieve high accuracy (say, 95%). The LEACE-scrubbed representations should still be vulnerable to nonlinear recovery. Systematically mapping the gap between linear and nonlinear probe accuracy across layers, concepts, and model architectures would characterize how much concept information is nonlinearly encoded and where in the network nonlinear encoding dominates. This map would tell the field when linear erasure is sufficient (early layers, simple concepts) and when fundamentally new methods are needed (late layers, abstract concepts). The paper's streaming covariance estimation and sequential fitting procedure provide the computational infrastructure to run these experiments at scale.

Testing concept scrubbing on narrower, behaviorally grounded concepts. The concept scrubbing experiments target part-of-speech — a broad syntactic category with 18 classes that pervades nearly every token in every sentence. This is a good stress test for the method's scalability, but it leaves open the question of whether concept scrubbing produces behaviorally specific effects for narrower concepts. A stronger validation: scrub a specific semantic feature (e.g., sentiment polarity, or the distinction between animate and inanimate objects, or a syntactic phenomenon like subject-verb agreement) and measure whether the model's behavior changes in precisely the ways linguistic theory predicts. For example, if you scrub number agreement features from a language model, does it start producing subject-verb agreement errors at a higher rate, while leaving other grammatical phenomena (tense, case, word order) unaffected? If the behavioral effects are specific and interpretable, concept scrubbing becomes a powerful causal probing tool for linguistics and cognitive science; if scrubbing a narrow concept produces diffuse, non-specific degradation, it suggests that concepts in neural networks are not cleanly separable in the way the linear subspace hypothesis assumes. The paper's randomized erasure baseline provides the template for these specificity tests.

Combining LEACE with model-specific training interventions. The paper positions LEACE as a post-hoc, model-agnostic method, but Section 7 acknowledges that model-specific techniques that modify the training procedure (Edwards & Storkey, 2016; Elazar & Goldberg, 2018; Zhang et al., 2018) may ultimately prove more effective. A natural hybrid: use LEACE during training as a projection layer — after each forward pass, scrub the target concept from the hidden states before computing the loss. This is distinct from adversarial training because the projection is exact and closed-form rather than being learned through a minimax game. The hypothesis is that exact linear erasure during training would be more robust than adversarial methods (which are known to leave residual information that new adversaries can recover; Elazar & Goldberg, 2018) while avoiding the nonlinear leakage that may occur in purely post-hoc scrubbing (because the model never gets gradient signal through the scrubbed dimensions, so it has no incentive to encode the concept nonlinearly elsewhere). A direct comparison — post-hoc LEACE vs. LEACE-as-projection-layer vs. adversarial training — on both linear and nonlinear probe recovery would reveal whether exactness of erasure or integration with training matters more for robustness.

Practical Applications and Downstream Use Cases

Fairness interventions for deployed classifier pipelines. The most directly actionable use case from the paper's experiments is reducing bias in frozen embeddings used by downstream classifiers. Section 5.2 demonstrates that applying LEACE to BERT's [CLS] embeddings reduces the TPR-GAP gender bias metric from 0.198 to 0.084 — a 58% reduction — while preserving profession prediction accuracy within 2 percentage points (79.3% → 77.3%). This is immediately deployable: any production system that feeds frozen embeddings from a pretrained model into a logistic regression or linear classifier (common in text classification, sentiment analysis, and information retrieval) can insert LEACE as a preprocessing step. The erasure matrix is fit once on a labeled dataset, then applied to all future inputs with negligible computational overhead (a single matrix multiplication). The paper's reported 100× speed advantage over RLACE and the closed-form nature of the solution mean this adds minimal latency to inference pipelines. The key caveat from the paper — re-fitting the downstream classifier partially recovers the bias (TPR-GAP rises to 0.158) — means this is most effective when the downstream model is already trained and frozen, not when adversaries can retrain on scrubbed data.

Causal interpretability at scale for large language models. The concept scrubbing results (Table 1) demonstrate that LEACE can erase a concept from every layer of models up to 30B parameters while maintaining coherent (if degraded) outputs. This enables a specific workflow for interpretability teams: (1) select a concept of interest (e.g., a syntactic feature, a semantic category, a factual association), (2) generate concept labels for a corpus of model inputs (using automatic taggers like SpaCy, as the paper does, or using the model's own outputs), (3) apply concept scrubbing via Algorithm 1, (4) measure the behavioral change on targeted evaluation tasks, and (5) compare against the randomized erasure baseline to establish specificity. The paper's streaming covariance estimation means this can be done without storing terabytes of hidden states — only running sums of second moments need to be accumulated. For a team with access to a 7B-parameter model and a modest GPU cluster, a full concept scrubbing experiment (fitting + evaluation) is feasible in hours to days, compared to being literally impossible with INLP's multi-layer collapse. The paper's finding that LEACE and SAL yield qualitatively different causal estimates (1.5–2× larger perplexity increase for SAL in Table 1) establishes the methodological standard: interpretability claims about concept importance must use the most surgical erasure method available, or risk overestimating effects.

Measuring and mitigating syntactic over-reliance in language models. Table 1 provides specific, quantitative evidence that autoregressive language models heavily depend on linearly encoded part-of-speech information: scrubbing POS increases perplexity by 1.0–1.3 bits/byte for LLaMA models (e.g., 0.69 → 1.73 for LLaMA 7B) and by 1.5–3.0 bits/byte for Pythia models (e.g., 0.64 → 3.57 for Pythia 6.9B). This has practical implications for model selection and training. If a downstream application requires robustness to syntactically noisy inputs (e.g., user-generated text with grammatical errors, non-native speakers, or speech-to-text transcripts), the paper's methodology can quantify which models rely least on clean syntax: run concept scrubbing for POS, measure the perplexity gap, and prefer models with smaller gaps. The paper's finding that random erasure of equal dimensionality has essentially no effect (Table 1, "Random erasure" rows are within 0.01 bits/byte of baseline) validates that this measurement specifically captures syntactic reliance, not generic brittleness. Furthermore, the finding that Pythia models show larger POS-induced degradation than comparably sized LLaMA models (Pythia 6.9B: 3.57 vs. LLaMA 7B: 1.73) suggests architecture-dependent differences in syntactic encoding that practitioners can exploit when choosing base models for syntax-critical tasks.

Post-hoc debiasing of embeddings for academic and regulatory compliance. In domains where fairness regulations require demonstrating that protected attributes have been mitigated (e.g., hiring algorithms, credit scoring, educational assessment), LEACE provides a well-defined technical intervention with formal guarantees. Unlike adversarial training methods — where "the concept has been removed" is an empirical claim that depends on the specific adversary used — LEACE provides a verifiable certificate: the cross-covariance matrix between the scrubbed embeddings and the protected attribute is exactly zero (up to finite-sample estimation error). An auditor can independently verify this by computing Σ_XZ on the scrubbed embeddings without access to the original model. The paper's theoretical framework (Theorems 3.1–3.4) establishes that this algebraic condition is equivalent to the inability of any linear classifier — with any convex loss — to recover the attribute, which is a much stronger guarantee than "our specific adversarial classifier failed to recover it." The caveat about nonlinear recovery (Section 7) means this is not a complete fairness solution, but for the common deployment pattern of logistic regression on frozen embeddings, it provides exactly the right guarantee. The paper's fairness experiments (Section 5.2) demonstrate that the theoretical guarantee translates to meaningful bias reduction on a real-world benchmark.

When to Prefer This Method

The paper articulates a clear tradeoff between linear and nonlinear erasure, and between post-hoc and training-time interventions, but does not position LEACE against a large matrix of named alternatives with explicit decision rules. Instead, it makes a focused argument for when linear, post-hoc, closed-form erasure is the right tool:

• Prefer LEACE (or any linear erasure method) when the threat model is linear recovery. If downstream models that will consume the scrubbed representations are linear classifiers (logistic regression, linear SVMs, linear probes), LEACE provides a provable guarantee that no such model can recover the concept (Theorems 3.1–3.4), and it does so optimally (minimal MSE, Theorem 4.2). This is the setting of the fairness experiments in Section 5.2 and the most common post-hoc debiasing scenario.

• Prefer LEACE over iterative methods (INLP, RLACE) when computational efficiency matters. The paper reports that LEACE is "around 2 orders of magnitude faster" than RLACE (Section 5.1) and achieves perfect erasure in a single algebraic step rather than 20 iterations (INLP) or gradient-based optimization (RLACE). For large models or streaming settings, this is the difference between feasibility and impracticality. The concept scrubbing experiments (Section 6) would not have been possible with INLP's computational cost and collateral damage.

• Prefer post-hoc LEACE over training-time interventions when model retraining is infeasible. Many fairness and interpretability use cases involve models that are already deployed, proprietary, or too expensive to retrain. LEACE edits representations without touching model parameters. The paper explicitly contrasts this with methods that "modify the training procedure" (Section 7), acknowledging that model-specific approaches may be more effective when retraining is an option, but LEACE fills the gap when it is not.

• Do not rely on linear erasure alone when the adversary can train nonlinear models on scrubbed data. The paper's re-fitting result (footnote 5, Section 5.2) — TPR-GAP rises from 0.084 to 0.158 when a new classifier is trained on scrubbed embeddings — demonstrates that linear guardedness does not prevent a freshly trained model from partially recovering the concept, likely via nonlinear combinations of features. The paper explicitly does not claim protection against nonlinear recovery and conjectures that general nonlinear erasure is intractable (Section 7). For applications where an adaptive adversary will train on scrubbed data, linear erasure is insufficient, and training-time interventions (with their ability to shape the entire loss landscape) may be necessary.