Read More — Paper Graph

🎯 Pitch

Sparse autoencoders can recover human-understandable concepts hidden inside language models—like a feature that fires only for apostrophes in possessive phrases—without any labeled data. When you patch these learned features, you can precisely flip a model’s grammatical predictions using far fewer dimensions than previous methods required. The secret is forcing the dictionary to be overcomplete and sparse, because dense representations simply won’t yield the same clean, localizable features.

1. Executive Summary

This paper introduces a scalable, unsupervised method for disentangling the internal representations of language models by training sparse autoencoders to decompose activation vectors into sparse linear combinations of dictionary features—learned directions in activation space that aim to recover the ground-truth features hypothesized to be in superposition. Experiments on Pythia-70M and Pythia-410M residual streams and MLP layers demonstrate that these dictionary features are substantially more interpretable than neurons, PCA components, or ICA directions as measured by automated interpretability scores, and that they enable more precise causal localization of model behavior on the indirect object identification task—requiring fewer feature patches and smaller edit magnitudes to achieve a given KL divergence from target outputs than PCA decompositions. Case studies reveal individual dictionary features that are monosemantic—activating only on specific concepts like apostrophes in possessive contexts—and have predictable effects on output logits, establishing that sparse autoencoders can recover human-understandable features from activation spaces only when the learned dictionary is trained with an explicit sparsity penalty, as non-sparse dictionaries fail to produce the same localization benefits.

2. Context and Motivation

The Core Problem: Polysemanticity Blocks Understanding of Neural Networks

The fundamental obstacle this paper tackles is polysemanticity: the empirically observed phenomenon that individual neurons in neural networks activate in multiple, semantically unrelated contexts. A single neuron might fire strongly for both car wheels and the word "however" in text, with no obvious single concept uniting these activation patterns. This is not an occasional quirk—it appears to be a widespread property of trained neural networks.

Polysemanticity matters because mechanistic interpretability—the research program of reverse-engineering how neural networks compute their outputs—fundamentally requires decomposing the network into smaller, independently analyzable units called features. If the natural units (neurons) are polysemantic, they resist clean explanation. You cannot say "this neuron detects X" when X is a disjunction of unrelated concepts, and downstream computations that depend on this neuron become correspondingly harder to trace. As the paper puts it in Section 1:

"Polysemanticity prevents us from identifying concise, human-understandable explanations for what neural networks are doing internally."

This is not merely an aesthetic problem. The paper situates itself within a broader concern about AI safety and trustworthiness. If we deploy AI systems that make decisions for reasons we do not understand, we cannot reliably predict their behavior in novel situations, detect when they are pursuing unintended goals, or make targeted corrections to their internal processing. The introduction cites concerns about catastrophic AI risks (Hendrycks et al., 2023) and specifically the possibility that "AIs might deceive humans in order to accomplish undesirable goals" (Ngo et al., 2022). Mechanistic interpretability is presented as a mitigation strategy: if we can reverse-engineer model computations, we can potentially verify safety properties or surgically edit problematic behaviors.

The paper also identifies a second, related problem: even if neurons were not polysemantic, there is no principled reason to expect features to align with the neuron basis in certain components of modern architectures. Specifically, the residual stream of a transformer—the primary pathway through which information flows between attention and MLP sublayers—lacks any nonlinearity that would privilege the standard basis. As Elhage et al. (2023) argued, the residual stream is better thought of as a vector space where meaningful features can point in arbitrary directions, not necessarily aligned with individual coordinates. This means that even perfectly monosemantic features might be distributed across many neurons, making neuron-level analysis fundamentally insufficient for understanding residual stream computations.

Why This Problem Is Important: The Superposition Hypothesis and Its Implications

The theoretical significance of polysemanticity runs deeper than the practical inconvenience of hard-to-interpret neurons. Elhage et al. (2022b) proposed the superposition hypothesis to explain why polysemanticity arises in the first place. The argument, which this paper accepts as its theoretical foundation, runs as follows:

Step 1: Models learn more features than dimensions. A neural network layer with dimensionality $d$ can represent at most $d$ orthogonal directions. However, real-world concepts that the model needs to track—syntactic categories, semantic roles, factual associations, etc.—likely number in the thousands or millions. The model cannot afford to dedicate one dimension per feature.

Step 2: Sparsity enables compression via non-orthogonal coding. If features are sparse—meaning that only a small fraction of all possible features are active on any given input—then the model can represent more than $d$ features by encoding them in an overcomplete basis of non-orthogonal directions. The key intuition: even though non-orthogonal vectors interfere with each other, if features activate rarely enough, the interference is manageable and the model benefits from the increased representational capacity. This is directly analogous to compressed sensing in signal processing, where sparse signals can be recovered from fewer measurements than Nyquist-Shannon sampling theory would suggest.

Step 3: Superposition produces polysemantic-looking neurons. When features are represented as non-orthogonal directions rather than individual neurons, probing any single neuron will capture a mixture of multiple features. A neuron might partially align with several different feature directions, leading to the observed polysemantic activation patterns.

If this hypothesis is correct—and Elhage et al. (2022b) provided evidence through toy models—then the interpretability problem has a specific structure: the features exist in the activation space as directions, but they are entangled with each other in the neuron basis. The task is therefore one of disentanglement: finding an alternative basis (a dictionary of directions) where each direction corresponds to a single, interpretable feature, despite the dictionary being larger than the ambient dimensionality.

This framing elevates the problem from "neurons are messy" to a principled challenge with a clear recovery objective. It also explains why previous interpretability work has been simultaneously promising and frustrating: neuron-level analysis works sometimes (when features happen to align with neurons) but fails generally (when superposition spreads features across the basis).

The real-world implications extend beyond academic interpretability research. If superposition is how models achieve their impressive capabilities within finite parameter budgets, then understanding and potentially controlling the features in superposition becomes essential for:

Model auditing: detecting whether a model has learned undesirable features (e.g., deceptive reasoning, biased associations) that are hidden in superposition.
Model editing: making targeted changes to specific features without disrupting unrelated computations that happen to share the same neurons.
Safety verification: providing guarantees about model behavior by exhaustively characterizing its feature set—what Elhage et al. (2022b) termed "enumerative safety."

Where Prior Approaches Fall Short

The paper identifies a landscape of existing approaches, each of which addresses part of the problem but leaves fundamental gaps:

Neuron-level analysis is insufficient, not just unreliable. The dominant approach in early mechanistic interpretability work (Olah et al., 2020; Bills et al., 2023) was to treat individual neurons as the units of analysis—visualizing what inputs maximally activate each neuron, generating natural language descriptions of neuron function, and tracing circuits through neuron-level connectivity. While this yielded genuine insights (e.g., curve detectors in vision models, syntax-sensitive neurons in language models), it suffers from a structural limitation: in the residual stream, there is no reason for features to align with neurons. The paper notes this explicitly:

"for some types of network activations, such as the residual stream of a transformer, there is little reason to expect features to align with the neuron basis" (Section 1, citing Elhage et al., 2023).

Even in components with nonlinearities (MLP layers), where the neuron basis has special status as the basis in which the nonlinearity is applied, superposition can still cause neurons to be polysemantic—the nonlinearity just means that the mixing of features is constrained in certain ways.

Training-time sparsity methods face practical barriers. An alternative approach is to change the model architecture or training procedure to encourage features to align with neurons from the start. Methods include modifying attention mechanisms for sparsity (Correia et al., 2019), adding $\ell_1$ penalties to neuron activations (Kasioumis et al., 2021; Georgiadis, 2019), pruning neurons (Frankle & Carbin, 2018), and using softmax nonlinearities in MLP layers to encourage feature-neuron alignment (Elhage et al., 2022a, the SoLU activation). The paper acknowledges these efforts but identifies a critical limitation:

"training a state-of-the-art foundation model with these additional constraints is difficult, and improvements to interpretability are not always realized" (Section 6.1, citing Meister et al., 2021).

This is the key practical bottleneck. Training large language models from scratch is enormously expensive, and adding architectural constraints that might hurt performance or training stability is a high-risk proposition. Moreover, even if future models adopt interpretability-friendly architectures, the vast number of already-deployed models would remain opaque. A post-hoc method—one that can be applied to any trained model without modifying its architecture or retraining—has clear practical advantages.

Existing post-hoc decomposition methods lack sparsity or scale. The paper compares its approach against several matrix decomposition baselines in Section 3.2:

Principal Component Analysis (PCA) finds orthogonal directions that capture maximal variance. However, the directions are dense (all features active for every input), and variance maximization does not correspond to semantic meaningfulness—the directions that explain the most variance may be mixtures of many underlying features.
Independent Component Analysis (ICA) finds directions that are statistically independent, which aligns with the intuition that real-world features should activate independently. The paper finds ICA to be "the best performing of the alternatives that we considered" (Appendix C.4), partly because it maximizes non-Gaussianity (kurtosis, skew), which correlates with sparsity. However, ICA is fundamentally limited to finding at most $d$ independent components in a $d$ -dimensional space—it cannot recover an overcomplete basis.
Random directions and the default neuron basis serve as baselines but provide no meaningful decomposition.

None of these methods can recover more features than the ambient dimensionality. If the superposition hypothesis is correct and models represent an overcomplete set of features, these methods are structurally incapable of full disentanglement.

Yun et al. (2021) applied dictionary learning but did not validate interpretability. The paper cites Yun et al. (2021) as the closest prior work, which applied sparse dictionary learning to all residual stream layers in a language model simultaneously. However, that work primarily demonstrated that the dictionary could reconstruct activations with sparse coefficients—it did not systematically evaluate whether the learned directions were more interpretable than alternatives, whether they enabled causal manipulation, or whether individual features were monosemantic. The present paper positions itself as building on this line of work while providing the interpretability validation that was missing.

Sharkey et al. (2023) provided initial evidence but at limited scale and depth. This paper directly builds on Sharkey et al. (2023), which first proposed using sparse autoencoders to take features out of superposition. The current work expands on this in several dimensions: (a) applying the method to larger models (Pythia-410M vs. smaller models in the prior work), (b) providing multiple forms of validation (automated interpretability, causal intervention, case studies), and (c) comparing against a broader set of baselines (PCA, ICA, random directions) with quantitative metrics.

How This Paper Positions Itself

The paper's positioning emerges clearly from the synthesis of the above gaps:

Methodologically: It proposes sparse autoencoders as a scalable, unsupervised, post-hoc method for feature disentanglement. "Scalable" means it can be applied to models with hundreds of millions of parameters using modest compute—the paper notes that training runs complete in under an hour on a single A40 GPU (Appendix B). "Unsupervised" means it requires only unlabeled model activations, with no task-specific annotations or human labels. "Post-hoc" means it can be applied to any trained model without modifying architecture or retraining. These three properties together address the practical limitations of prior approaches.

Theoretically: It adopts the superposition hypothesis (Elhage et al., 2022b) as its motivating framework. The sparse autoencoder is explicitly framed as an attempt to reverse the superposition process—to recover the overcomplete feature dictionary that the model is hypothesized to have learned. The $\ell_1$ sparsity penalty in Equation 4 is theoretically motivated by results showing "that reconstruction with an $\ell_1$ penalty can recover the ground-truth features that generated the data" (Section 2, citing Sharkey et al., 2023 and Wright & Ma, 2022).

Empirically: It provides multiple independent lines of evidence that the learned features are meaningful, addressing different facets of what "meaningful" means:

Interpretability scores (Section 3): Automated measurement of whether features correspond to human-understandable concepts, using the protocol from Bills et al. (2023). This addresses the gap that Yun et al. (2021) left open.
Causal localization (Section 4): Demonstration that features enable precise manipulation of model behavior on the indirect object identification task. This goes beyond correlation to show that the features are causally relevant to model computation.
Case studies (Section 5): Qualitative analysis showing individual features are monosemantic, have predictable output effects, and participate in interpretable circuits. This provides intuitive validation that complements the quantitative metrics.

Pragmatically: The paper explicitly situates itself within the broader project of mechanistic interpretability for AI safety. The conclusion invokes the "ambitious dream" of "enumerative safety"—producing a complete, human-understandable explanation of a model's computations that could provide safety guarantees. The sparse autoencoder method is positioned as "a step towards achieving this ambition" (Section 6.3).

A key nuance in the paper's positioning is that it does not claim to fully solve the superposition problem or to achieve perfect reconstruction. Section 6.2 acknowledges that "we do not achieve 0 reconstruction loss," meaning the dictionaries fail to capture all information in the activations. Replacing layer 2 residual stream activations with reconstructions increases perplexity from 25 to 40 on the Pile. The claim is more measured: sparse autoencoders recover features that are substantially more interpretable and causally relevant than alternative decompositions, representing progress toward the eventual goal rather than a complete solution.

The paper also positions its findings as task-agnostic and therefore potentially general. The autoencoders are trained on random web text activations, not on any specific task. The fact that features learned this way enable precise localization of IOI behavior (Section 4) suggests—though does not prove—that the same features will be useful for understanding many other behaviors. This contrasts with approaches that learn task-specific decompositions, which might not transfer.

3. Technical Approach

3.1 Reader orientation (approachable technical breakdown)

The system is a sparse autoencoder—a neural network with a single hidden layer and a sparsity penalty—that takes the internal activation vectors of a frozen language model and learns to reconstruct them as sparse weighted sums of a learned dictionary of feature directions. The problem it solves is the hypothesized superposition of features in language models: the idea that models represent more distinct concepts than they have dimensions by encoding them in non-orthogonal directions, making individual neurons polysemantic mixtures of multiple features. The shape of the solution is to train a larger-than-original dictionary of sparsely-activating directions on unlabelled activation data, such that each direction in the dictionary corresponds (approximately) to a single, human-interpretable concept that was previously entangled in the neuron basis.

3.2 Big-picture architecture (diagram in words)

The system has four major components:

Activation extraction pipeline: Run a frozen language model (Pythia-70M or Pythia-410M) over text from the Pile, caching the activation vectors at a specific layer (residual stream, MLP sublayer, or attention head sublayer). These cached vectors form the training dataset—no labels, no task-specific data, just raw internal activations.
Sparse autoencoder: A single-hidden-layer neural network with tied encoder-decoder weights, ReLU hidden activations, and an $\ell_1$ sparsity penalty on the hidden layer. The encoder weights are the feature dictionary—each row is a learned direction in activation space that detects a specific feature. The hidden layer coefficients are the sparse encoding of the input. The decoder reconstructs the input from these sparse coefficients using the transposed dictionary.
Interpretability evaluation pipeline: For each learned dictionary feature, sample text fragments where it activates strongly, use GPT-4 to generate a natural language interpretation, then use GPT-3.5 to simulate the feature's activation pattern on new text given that interpretation. The correlation between simulated and actual activations is the autointerpretability score.
Causal validation through activation patching: On the IOI task, identify which dictionary features are causally important by replacing their coefficients with those from a counterfactual input and measuring how much the output logits shift toward the counterfactual prediction.

Information flows as follows: text corpus → frozen language model → cached activation vectors → sparse autoencoder training (reconstruct x from sparse c) → learned dictionary M → interpretability scoring + causal patching experiments + qualitative case studies.

3.3 Roadmap for the deep dive

First, the superposition hypothesis and why sparse dictionary learning is the natural recovery method — this establishes the theoretical motivation for the entire technical approach.
Second, the sparse autoencoder architecture and loss function (Equations 1–4) — the core mathematical machinery, including the crucial design choices of tied weights, row normalization, and ReLU activation.
Third, the training procedure and hyperparameter selection — data sources, optimization settings, the tradeoff between sparsity and reconstruction accuracy, and how hyperparameters are chosen.
Fourth, the autointerpretability protocol — how interpretability is measured at scale without human annotators, building on Bills et al. (2023).
Fifth, the activation patching methodology for causal validation — how the paper tests whether dictionary features are not just interpretable but causally relevant to model behavior.
Sixth, the baseline comparison methods (PCA, ICA, random directions, default basis) and what each controls for.

This order moves from theoretical motivation → mathematical definition → practical implementation → evaluation methodology → validation → comparison, which mirrors how the system is built and reasoned about.

3.4 Detailed, sentence-based technical breakdown

This is primarily a methods-development and empirical-validation paper whose core idea is that training sparse autoencoders on language model activations with an $\ell_1$ sparsity penalty can recover an overcomplete set of interpretable feature directions from superposition, without requiring any labels, task-specific data, or architectural modifications to the original model.

The Superposition Hypothesis as Theoretical Motivation

Before describing the autoencoder architecture, the paper grounds its approach in the superposition hypothesis from Elhage et al. (2022b). Understanding this hypothesis is essential because it dictates why an overcomplete dictionary with sparsity is expected to work where other decompositions fail.

The representation bottleneck. A neural network layer with activation dimensionality $d$ can represent at most $d$ orthogonal directions in its activation space. However, the network likely needs to track many more distinct features than $d$ —syntactic categories, semantic roles, entity references, factual associations, etc. If features were forced to be orthogonal, the model could represent only $d$ of them, which is almost certainly insufficient for the complexity of language.

Sparsity as an escape hatch. The superposition hypothesis proposes that models evade this bottleneck by exploiting a statistical property of real-world features: sparsity. On any given input, only a tiny fraction of all possible features are relevant. A sentence about "Alice giving Bob a book" does not simultaneously activate features for mathematical integration, French verb conjugation, and the capital of Mongolia. If features are sparse—meaning at any moment, most features have zero or near-zero activation—then the model can represent more than $d$ features by encoding them in an overcomplete basis of non-orthogonal directions.

Here is the crucial intuition. Two non-orthogonal vectors interfere with each other: the projection of one feature onto another creates noise. However, if features are sparse, this interference is negligible on average. When feature A is active, feature B is usually inactive, so the interference term from A bleeding into B's direction doesn't corrupt B's representation (because B has nothing to corrupt). When both are rarely active simultaneously, the model absorbs the occasional interference as acceptable noise in exchange for massively increased representational capacity.

What superposition looks like in the neuron basis. When features are stored in non-orthogonal directions rather than individual neurons, probing any single neuron captures a weighted mixture of all features whose directions overlap with that neuron. This produces the polysemantic activation patterns that motivated the work: a neuron that fires for both "car wheels" and "the word however" is not encoding some mysterious unified concept; it is simply aligned partially with two different feature directions in an overcomplete basis.

The recovery problem. If this hypothesis is correct, then the model's computations are fundamentally organized around feature directions that are not aligned with neurons, and there are more of these directions than dimensions. The task of interpretability becomes: given the activation vectors produced by the model, can we recover the set of ground-truth feature directions $\{g_j\}_{j=1}^{n_{gt}}$ that the model is using, where $n_{gt} > d$ ? This is exactly the problem of sparse dictionary learning (also called sparse coding), classically studied by Olshausen & Field (1997) in the context of explaining receptive fields in the visual cortex.

The paper formalizes this in Section 2: given a dataset of activation vectors $\{x_i\}_{i=1}^{n_{\text{vec}}} \subset \mathbb{R}^d$ , the goal is to find a dictionary $\{f_k\}_{k=1}^{n_{\text{feat}}} \subset \mathbb{R}^d$ where $n_{\text{feat}} = R \cdot d$ (with $R$ typically $\geq 1$ , often $R > 1$ for overcompleteness), such that each activation vector can be approximated as a sparse linear combination of dictionary features:

$x_i \approx \sum_{k=1}^{n_{\text{feat}}} c_{i,k} f_k$

where only a small number of coefficients $c_{i,k}$ are non-zero for any given $i$ . If successful, each learned dictionary feature $f_k$ should approximately correspond to one ground-truth feature $g_j$ , disentangling the superposition.

Sparse Autoencoder Architecture and Loss Function

The paper implements sparse dictionary learning using a single-hidden-layer autoencoder with tied weights, ReLU activations, and an $\ell_1$ sparsity penalty. This is not the only possible approach to sparse coding (alternatives include iterative thresholding methods), but it has the advantage of being simple, scalable to GPU training, and directly optimizable via gradient descent.

The encoder step. Given an input activation vector $x \in \mathbb{R}^{d_{\text{in}}}$ from the language model (e.g., a residual stream vector of size 512 for Pythia-70M or 1024 for Pythia-410M), the encoder produces the hidden representation $c \in \mathbb{R}^{d_{\text{hid}}}$ via:

$c = \text{ReLU}(Mx + b)$

where $M \in \mathbb{R}^{d_{\text{hid}} \times d_{\text{in}}}$ is the encoder weight matrix, $b \in \mathbb{R}^{d_{\text{hid}}}$ is a bias vector, and ReLU is the element-wise rectified linear unit: $\text{ReLU}(z) = \max(0, z)$ .

What this computes: The vector $x$ is projected through the matrix $M$ to produce $d_{\text{hid}}$ scalar scores. The bias $b$ is added to each score. Then the ReLU clips all negative values to exactly zero, while passing positive values through unchanged. The result $c$ is a non-negative vector where each element $c_k$ represents how strongly feature $k$ in the dictionary is activated by input $x$ . Critically, the ReLU enforces that features either have zero or positive activation—there is no notion of a "negative feature activation" (a feature being anti-present).

The dictionary interpretation of weights. The matrix $M$ has $d_{\text{hid}}$ rows. Each row $f_k \in \mathbb{R}^{d_{\text{in}}}$ is a dictionary feature—a direction in the language model's activation space. These rows are normalized to unit $\ell_2$ norm (row-wise normalization, enforced as part of the training). The dot product $f_k \cdot x$ (which is what the matrix multiplication $Mx$ computes before adding bias $b_k$ ) measures the alignment between input $x$ and feature direction $f_k$ . The bias $b_k$ effectively sets a threshold: the feature activates (produces a non-zero coefficient) only when the alignment exceeds $-b_k$ .

The decoder step. Given the sparse hidden representation $c$ , the autoencoder reconstructs the input as:

$\hat{x} = M^T c = \sum_{k=1}^{d_{\text{hid}}} c_k f_k$

where $M^T$ is the transpose of the encoder weight matrix. The output $\hat{x}$ is a linear combination of the dictionary feature vectors, weighted by their activation coefficients.

What this computes: Each dictionary feature $f_k$ contributes to the reconstruction in proportion to its coefficient $c_k$ . If $c_k = 0$ (the feature is inactive—below the ReLU threshold), that feature contributes nothing. The reconstruction is sparse: only features with positive coefficients participate. The decoder effectively says, "to reconstruct this activation vector, take a little bit of feature direction $f_1$ , a little bit of $f_5$ , a lot of $f_{42}$ , and so on."

Why tied weights? The paper uses the same matrix $M$ for encoding and decoding (up to transposition). This is a deliberate design choice with three justifications given in the footnote in Section 2:

Symmetry intuition: the direction that detects a feature (encoder) should be the same direction that expresses it (decoder). If the model represents a feature as a particular direction in activation space, then "reading" that feature and "writing" that feature should use the same vector.
Memory efficiency: tying weights halves the parameter count, which matters when $d_{\text{hid}}$ is in the thousands.
Interpretation clarity: with separate encoder and decoder matrices, it is ambiguous whether the learned feature direction should be identified with the encoder row or the decoder column. Tying eliminates this ambiguity.

The paper notes that tied weights "do not reduce performance when training on residual stream data but we have observed some reductions in performance when using MLP data" (Section 2, footnote 2). This asymmetry is informative: the residual stream is a linear space with no privileged basis, so there is no structural reason why read and write directions should differ. The MLP layers, however, have nonlinearities that might create asymmetries between how features are detected versus how they are expressed.

Why ReLU? The ReLU activation has a specific property crucial for sparse dictionary learning: it produces exact zeros. This is not just a convenience—it is essential. The sparsity penalty (described next) encourages the model to use few dictionary features. With a ReLU, cutting a feature's contribution to exactly zero means its coefficient genuinely contributes nothing to the reconstruction. With a smooth activation like sigmoid or tanh, "nearly zero" is not the same as zero, and the reconstruction would always include tiny contributions from all features, eliminating the conceptual cleanness of sparsity. Additionally, ReLU enforces non-negativity of coefficients: features are either present (positive) or absent (zero), with no notion of "negative presence." This matches the intuition that features are properties that either apply or do not apply to an input.

Why row normalization? The rows of $M$ (the dictionary feature vectors) are constrained to have unit $\ell_2$ norm. The paper explains in Section 2, footnote 3: "Normalisation of the rows (dictionary features) prevents the model from reducing the sparsity loss term $\|c\|_1$ by increasing the size of the feature vectors in $M$ ." Without normalization, the autoencoder could cheat: it could make the feature vectors larger in magnitude, which would allow smaller coefficients to achieve the same reconstruction (since $x \approx \sum c_k f_k$ , doubling all $f_k$ magnitudes would let all $c_k$ halve while maintaining the same reconstruction). This would artificially reduce the $\ell_1$ penalty without making the representation genuinely sparser in terms of which features are used. Row normalization closes this loophole by fixing the scale of each feature direction.

The loss function. The autoencoder is trained to minimize:

$\mathcal{L}(x) = \underbrace{\|x - \hat{x}\|_2^2}_{\text{Reconstruction loss}} + \alpha \underbrace{\|c\|_1}_{\text{Sparsity loss}}$

where $\alpha > 0$ is a hyperparameter controlling the tradeoff between reconstruction fidelity and sparsity.

What each term computes:

Reconstruction loss $\|x - \hat{x}\|_2^2$ : The squared Euclidean distance between the original activation vector $x$ and the reconstructed vector $\hat{x} = M^T c$ . This measures how much information is lost in the sparse encoding. If the reconstruction is perfect, this term is zero. If the autoencoder fails to capture important variation in the activations, this term is large.
Sparsity loss $\alpha\|c\|_1$ : The $\ell_1$ norm of the hidden representation—the sum of all coefficient values $c_k$ , multiplied by the hyperparameter $\alpha$ . This penalizes having many active features or having features with large coefficients. Since all $c_k \geq 0$ (due to ReLU), $\|c\|_1 = \sum_k c_k$ .

Why $\ell_1$ specifically? The paper appeals to both empirical evidence and theory for this choice. Empirically, "it can be shown empirically (Sharkey et al., 2023) and theoretically (Wright & Ma, 2022) that reconstruction with an $\ell_1$ penalty can recover the ground-truth features that generated the data" (Section 2). The theoretical motivation extends beyond intuition: in compressed sensing and sparse recovery, the $\ell_1$ penalty is a convex relaxation of the $\ell_0$ "norm" (which counts the number of non-zero coefficients). The $\ell_0$ penalty is the ideal sparsity measure—it directly penalizes the count of active features—but optimizing with an $\ell_0$ penalty is computationally intractable (combinatorial). The $\ell_1$ penalty is the tightest convex relaxation of $\ell_0$ , meaning it is the best computationally tractable proxy for true sparsity. It produces sparse solutions (many exact zeros, many small values) in a way that the $\ell_2$ penalty (which penalizes the squared sum of coefficients and favors many small, non-zero values) does not.

Operational summary of the entire autoencoder forward pass. An input activation vector from the language model enters. The encoder computes alignments with all $d_{\text{hid}}$ dictionary directions by matrix multiplication $Mx$ , adds per-feature biases $b$ , and clips negatives to zero via ReLU. The resulting sparse coefficient vector $c$ specifies which features are present and how strongly. The decoder multiplies each feature direction $f_k$ by its coefficient $c_k$ and sums them to produce the reconstruction $\hat{x}$ . The loss function penalizes both reconstruction error (encouraging faithful representation) and total activation magnitude (encouraging sparsity). The learned dictionary $M$ is the output of training—each row is a candidate ground-truth feature direction.

Training Procedure and Hyperparameter Selection

The training pipeline transforms raw language model activations into a learned feature dictionary through several stages, each with specific design choices.

Data collection. The activation dataset is constructed by running a frozen, pre-trained language model over text from the Pile (Gao et al., 2020), a large public web-text corpus consisting of approximately 800GB of diverse text. As the model processes text token by token, the activations at a specified layer are cached and saved. The paper focuses primarily on the residual stream activations of Pythia-70M ( $d_{\text{in}} = 512$ ) and Pythia-410M ( $d_{\text{in}} = 1024$ ), though it also experiments with MLP sublayer activations. The key property of this data is that it is unsupervised and task-agnostic: the autoencoder sees only the internal representations produced during normal language modeling, with no task-specific filtering or labeling. This is crucial for the claim that the learned features will be useful for understanding many different model behaviors, not just the ones present in a curated dataset.

Autoencoder hyperparameters. The hidden layer size is $d_{\text{hid}} = R \cdot d_{\text{in}}$ , where $R$ is the dictionary size ratio—a critical hyperparameter controlling overcompleteness. The paper experiments with $R \in \{0.5, 1, 2, 4, 8\}$ . When $R < 1$ , the dictionary is undercomplete (fewer features than dimensions), which tests whether sparsity alone produces interpretable features even without overcompleteness. When $R > 1$ , the dictionary is overcomplete (more features than dimensions), which is the regime where superposition can be disentangled. The maximum $R$ the paper reports successfully training is 8 for the residual stream, though for MLP layers, many learned features become "dead" (never activating) even at lower $R$ values.

Optimization details. The autoencoder is trained using the Adam optimizer with a learning rate of $1 \times 10^{-3}$ , on 5–50 million activation vectors, for 1–3 epochs. "Larger dictionaries taking longer to converge" (Appendix B). A single training run completes in under an hour on a single A40 GPU when using this quantity of data—this is orders of magnitude less computation than training the original language model, which is essential for the method's practicality.

The sparsity-accuracy tradeoff. The hyperparameter $\alpha$ in Equation 4 controls the balance between reconstruction fidelity and sparsity. The paper finds "consistently a smooth tradeoff between the sparsity and accuracy of our autoencoder, as shown in Figure 6" (Appendix B). There is no sharp "knee" or phase transition in this tradeoff curve—no natural point where the benefits of additional sparsity suddenly diminish. The authors interpret this as evidence that "there is not a single correct way to decompose activation spaces into a sparse basis, though to confirm this would require many additional experiments" (Appendix B). This is a philosophically interesting observation: it suggests that feature decomposition is inherently resolution-dependent, analogous to how an image can be described at different levels of granularity. A very sparse decomposition might capture only the most prominent features, while a less sparse one captures finer detail—both are "correct" at different scales.

Hyperparameter selection for reported results. For the residual stream experiments reported in Section 3, the paper uses $\alpha = 8.6 \times 10^{-4}$ and $R = 2$ (10 epochs of training). For MLP experiments, it uses $\alpha = 3.2 \times 10^{-4}$ and $R = 1$ . These values were selected based on the number of active (non-dead) features in the learned dictionary, as shown in Figure 16 (Appendix E). The goal was to find parameters where the dictionary retained most of its capacity (few dead features) while still achieving meaningful sparsity.

The dead feature problem in MLP layers. Appendix E, Figure 16 reveals a stark asymmetry between residual stream and MLP training. For the residual stream, most dictionaries remain largely active even at $R = 8$ with appropriate $\alpha$ . For MLP layers, "you see large numbers of dead features even with hyperparameter $\alpha = 0$ " (Appendix E). The paper explains this as follows: MLP activations are produced after a nonlinearity (typically GELU in Pythia models), which means that a large part of the activation space is never populated—many directions simply cannot occur as MLP outputs. Dictionary features initialized in these unreachable regions will never receive gradient updates and will remain permanently dead. The paper suggests reinitializing dead features in future work but does not implement this in the current experiments.

Tied vs. untied weights for MLP layers. The paper notes that for MLP layers, using separate encoder and decoder matrices (replacing Equations 1-3 with Equations 5-6 in Appendix C.3) helps retain more active features. Specifically:

$c = \text{ReLU}(M_e x + b)$

$\hat{x} = M_d^T c$

where $M_e$ and $M_d$ are independent matrices. This suggests that the symmetry assumption (read direction equals write direction) breaks down in nonlinear layers, plausibly because the MLP nonlinearity creates an asymmetry in how features are embedded versus how they are read out.

Autointerpretability Protocol

To evaluate whether learned dictionary features are more interpretable than alternative decompositions at scale (without hiring human annotators for thousands of features), the paper adapts the automated interpretability protocol from Bills et al. (2023). This protocol is fully described in Appendix A.

The core idea. An automated system (GPT-4 for explanation generation, GPT-3.5 for simulation) takes each dictionary feature, looks at the text that activates it, generates a natural language interpretation, and then tests how well that interpretation predicts the feature's behavior on new text. The entire pipeline requires no human intervention beyond the initial setup.

Step-by-step procedure.

Step 1: Activation collection. On each of the first 50,000 lines of OpenWebText, extract a 64-token sentence fragment. Run the language model on this fragment and record the dictionary feature's activation value at each token position. Rescale all activation values to integers between 0 and 10 for interpretability.

Step 2: Interpretation generation. From the 50,000 fragments, identify the 20 fragments with the highest activation scores for the feature. Randomly select 5 of these top-20 fragments and pass them to GPT-4, along with the rescaled per-token activation values. GPT-4 receives an instruction to suggest an explanation for when the feature fires, producing a natural language interpretation (e.g., "parts of individual names, especially last names" for feature 1-0000, as shown in Table 1).

Step 3: Simulation. Take another 5 highly activating fragments (from the remaining top-20, not used in Step 2) and 5 randomly selected fragments (with non-zero variation in activation—completely flat activations are excluded). For each fragment, present the interpretation from Step 2 to GPT-3.5 and ask it to simulate the per-token activations the feature would produce given that description. GPT-3.5 is used here rather than GPT-4 because the simulation protocol requires access to the model's logprobs for scoring, and at the time of the experiments, OpenAI's public API supported returning logprobs for GPT-3.5 but not GPT-4 (Appendix A, footnote 6).

Step 4: Scoring. Compute the correlation between the simulated activations (from Step 3) and the actual activations (from Step 1). This correlation is the autointerpretability score of the feature. Two variants are reported:

Top-random score: uses an even mixture of the 5 highly activating fragments and the 5 random fragments for the correlation computation.
Random-only score: uses only the 5 random fragments. This is stricter because the random fragments will mostly contain low or zero activations, so the interpretation must correctly predict absence of the feature, not just recognize obvious high-activation patterns.

Step 5: Filtering. If fewer than 20 fragments among the 50,000 contain non-zero variation in activation (meaning the feature almost never fires or fires at constant values), the feature is skipped entirely from the interpretability analysis. This excludes dead features and near-dead features.

Rationale for top-random scoring. The paper explains why top-random scoring (mixing high-activation and random fragments) is used as the primary metric rather than random-only scoring: "the small sample sizes of a total of 640 tokens used for analysis mean that a random sample will likely not contain any highly activating examples for all but the most common features, making top-random scoring a desirable alternative" (Appendix A). With only 10 fragments total (5 high + 5 random), random-only scoring has very low statistical power for rare features—the interpretation might be perfectly correct but receive a low score because none of the 5 random fragments happen to activate the feature. Top-random scoring ensures that at least some of the evaluation fragments contain the feature, testing whether the interpretation correctly identifies when the feature should fire.

What the score measures (and does not measure). An autointerpretability score of 0.33 (like feature 1-0000 in Table 1) means that the GPT-3.5 simulation's correlation with actual activations is 0.33. This is a conservative measure: it tests not just whether GPT-4 can produce a plausible-sounding description, but whether that description actually predicts held-out behavior. A score of -0.11 (like feature 1-0001) suggests the interpretation is actually anti-predictive—the feature fires in the opposite of the described contexts. However, the paper acknowledges limitations of automated interpretation: "current LLMs are limited in the kinds of patterns that they can find, sometimes struggling to find patterns that center around next or previous tokens rather than the current token" (Section 3.2). Features that are about predicting upcoming tokens (rather than classifying the current token) may be poorly described by GPT-4 and thus receive artificially low scores.

Comparative baselines. The paper computes interpretability scores for four comparison methods (described in Section 3.2):

Default basis (neurons): The standard basis vectors of the residual stream or MLP layer. For the residual stream, negative activations are replaced with zeros so all feature activations are non-negative (to match the ReLU property of the autoencoder features).
Random directions: Randomly sampled unit vectors in activation space, with negative activations similarly replaced by zeros.
Principal Component Analysis (PCA): Directions of maximal variance computed via online estimation on the same quantity of data used for autoencoder training (5–50 million activation vectors). PCA produces orthogonal directions sorted by explained variance.
Independent Component Analysis (ICA): Directions that maximize statistical independence. Due to slower convergence, ICA is run on only 2GB of data (approximately 4 million activations for the residual stream, 1 million for MLPs).

For PCA and ICA, the authors later introduce a "top-K" variant (Appendix G) to control for an important confound: autoencoder features activate on less than a full half-space (because the bias $b$ shifts the ReLU threshold away from the origin), whereas PCA/ICA directions naturally activate on an entire half-space. Top-K forces PCA/ICA to also have only a fixed number $K$ of active directions per datapoint, where $K$ is set to the average number of active features for the autoencoder at $R=1, \alpha=8.6\times10^{-4}$ . The paper shows this adjustment "does not explain more than a small fraction of the improvement in scores" (Appendix G, Figure 18), confirming that the interpretability gains come from the learned directions themselves, not from the artificial restriction on activation extent.

Activation Patching for Causal Validation

Interpretability scores measure correlation between feature descriptions and activation patterns, but they do not demonstrate that features are causally involved in model computation. A feature might be interpretable in isolation but irrelevant to the model's actual decision-making. Section 4 introduces an activation patching protocol to test causal relevance on the indirect object identification (IOI) task.

The IOI task. The model is given sentences like "Then, Alice and Bob went to the store. Alice gave a snack to ___", where the correct completion is "Bob" (the indirect object of "gave"). This task was chosen because it captures a simple, previously-studied model behavior (Wang et al., 2022) that is well-understood enough to serve as a testbed for causal analysis. The task is not part of the autoencoder training—the features are learned from general web text, not from IOI examples—so any causal relevance they show for IOI is evidence that the features capture generally useful representations.

The activation patching procedure. The core idea is to run the model on two versions of an IOI sentence: a base sentence (e.g., "...Alice gave a snack to Bob") and a target sentence with the indirect object changed (e.g., "...Alice gave a snack to Vanessa"). For a fixed layer $\ell$ , the procedure is:

Run the model on the target sentence and save (a) the output logits $y$ that the model produces, and (b) the encoded features $\bar{c}_1, \dots, \bar{c}_k$ at each of the $k$ token positions in layer $\ell$ .
Run the model on the base sentence up through layer $\ell$ (the intervention layer). At each token position, compute the encoded features $c_1, \dots, c_k$ from the base sentence.
At each token position $i$ , replace the residual stream vector $x_i$ with a patched vector:

$x_i' = x_i + \sum_{j \in F} (\bar{c}_{i,j} - c_{i,j}) f_j$

where $F$ is the subset of dictionary features selected for intervention.

Continue running the model from layer $\ell + 1$ onward on the patched residual stream $x_1', \dots, x_k'$ , producing output logits $z$ .
Compute the KL divergence $D_{\text{KL}}(z \| y)$ , which measures how much the patched model's output distribution has shifted toward the target model's output distribution.

What the patching equation does in plain terms. For each feature $j$ in the intervention set $F$ , we look at how much that feature's coefficient differs between the base and target sentences ( $\bar{c}_{i,j} - c_{i,j}$ ). We then add that difference, multiplied by the feature direction $f_j$ , to the base activation. The effect is to "copy" the activation state of selected features from the target sentence into the base sentence, while leaving all other features unchanged. If the selected features are the ones causally responsible for the model's decision on the IOI task, then the patched model should produce outputs resembling the target sentence (small KL divergence). If the selected features are irrelevant, the patching has no effect (KL divergence remains large).

Selecting which features to patch. To choose the subset $F$ of features to intervene on, the paper uses the Automated Circuit Discovery (ACDC) algorithm from Conmy et al. (2023). ACDC treats the features as a flat computational graph where each feature contributes an independent change to the output KL divergence. It uses a greedy backward-elimination procedure: start with all features in $F$ , compute each feature's marginal contribution to the KL divergence when patched, remove the least important features, and repeat. The result is an ordering of features from most to least causally important. The subsets $F$ tested are then the first $k$ features under this ordering for various values of $k$ .

Comparison against PCA. The same patching procedure is applied to PCA components, using identical methodology (same IOI sentences, same ACDC ordering). The comparison reveals that sparse dictionary features require fewer patches to achieve a given KL divergence and achieve the same divergence with smaller total edit magnitude (measured as the $\ell_2$ norm of $x_i' - x_i$ ) compared to PCA, as shown in Figure 3.

Why this comparison matters. PCA is the baseline most similar in spirit to sparse autoencoders—both learn linear decompositions of the activation space from data. The fact that sparse autoencoder features are more causally efficient (fewer features needed, smaller edits) than PCA features at the same reconstruction quality demonstrates that sparsity is doing meaningful work in separating causally relevant from irrelevant directions, not just capturing variance.

Non-sparse dictionary control. The paper includes a crucial control: an autoencoder trained with $\alpha = 0$ (no sparsity penalty). This non-sparse dictionary does not show the same patching efficiency as sparse dictionaries (Figure 3, left panel, shows the $\alpha = 0$ curve performing worse than sparser dictionaries). This demonstrates that the causal localization benefit specifically comes from the sparsity constraint, not from any property of the autoencoder architecture itself.

Layer-wise results. The paper reports patching results primarily on layer 11 of the 24-layer Pythia-410M (Figure 3), with results for layers 3, 7, 11, 15, 19, and 23 provided in Appendix F, Figure 17. The sparse dictionary advantage over PCA is consistent across layers, though the exact magnitude varies.

Qualitative Feature Analysis Methods

Beyond automated metrics and causal patching, the paper conducts manual case studies (Section 5) to provide intuitive validation that individual features are monosemantic and mechanistically meaningful. The methods used for this qualitative analysis are described in Section 5 and Appendix D.

Input analysis: token activation histograms. For a given dictionary feature, the paper plots histograms showing which tokens in the vocabulary cause the feature to activate, broken down by activation magnitude range. For example, Figure 4 (left) shows that feature 556 activates primarily on apostrophes, with higher activations almost exclusively on apostrophes and lower activations including conceptually similar tokens (other punctuation). This histogram method is most informative for features that activate on a small set of specific tokens, not for features representing abstract concepts that can be expressed in many lexical forms. The paper acknowledges this limitation implicitly by focusing the case studies on token-level features (apostrophes, periods, "the", newlines).

Output analysis: logit effects from ablation. To understand what a dictionary feature does causally, the paper performs "less-than-rank-one ablation" (Section 5.2, footnote 5): subtract the feature's contribution from the residual stream only up to the point where the feature is no longer active, rather than removing the entire feature direction. The effect on output logits reveals which token predictions depend on the feature. For the apostrophe feature, ablation primarily reduces the logit for the "s" token (Figure 4, right), consistent with the feature being used to predict "'s" in possessives and contractions.

Intermediate feature analysis: automatic circuit detection. To trace how features causally depend on each other across layers, the paper develops a recursive ablation procedure (Section 5.3):

Choose a target dictionary feature in layer $\ell$ (e.g., a closing parenthesis feature in layer 5).
Find its maximum activation $M$ across the dataset.
Sample 20 contexts that cause the target feature to activate in the range $[M/2, M]$ (high but not maximal activations).
For each dictionary feature in the previous layer $\ell - 1$ , rerun the model while ablating that feature and measure how much the target feature's activation decreases.
Sort the previous-layer features by their causal impact on the target feature.
Recursively apply this procedure to the most impactful features in layer $\ell - 1$ , tracing dependencies back through earlier layers.

The result is a causal tree (Figure 5) where edges indicate the strength of causal influence between features in successive layers. The paper notes that "many dictionary features across layers correspond to similar real-world features and often point in similar directions in activation space, as measured by cosine similarity" (Section 5.3, Figure 5 caption). This cross-layer consistency provides independent confirmation that the features capture genuine model computations: if a feature in layer 3 detects dates-in-parentheses and causally influences a feature in layer 5 that predicts closing parentheses, and both point in similar directions (high cosine similarity), the features are likely tracing a coherent computational pathway.

Feature search methodology. The paper details how specific features were identified for case studies (Appendix D.3). For the apostrophe feature, they crafted a probe sentence: "I don't know about that. It is now up to Dave'" and checked which dictionary feature activated most strongly for the final apostrophe token. The top activating feature was not the pure apostrophe feature; it was first an "outlier dimension feature" (a feature that mainly reads from outlier dimensions of the residual stream, noted by Elhage et al., 2023 as dimensions with unusually large magnitude), then an "apostrophe after O" feature (predicting O'Brien, O'Donnell, etc.), and finally the possessive apostrophe-preceding-s feature. This illustrates that even in a learned dictionary, features can have fine-grained specializations—the model has separately learned to handle apostrophes in Irish surnames versus possessive constructions.

Failed alternative: weight-based circuit tracing. Appendix D.4 documents an attempt to trace circuits by multiplying a feature vector from layer 4 by the MLP weight matrix and computing cosine similarity with features in layer 5, rather than using ablation. This "weight-based method" failed: "There were no meaningful connections. Additionally, it's unclear how to apply this to the Attention sublayer since we'd need to see which position dimension the feature is in." The paper attributes the failure to the approach being "out of distribution"—the MLP weights were trained on full activation vectors, not on individual feature directions in isolation, so projecting a single feature through the MLP does not simulate how the feature actually behaves in context.

This negative result is informative because it demonstrates that ablation-based causal tracing is necessary: the interactions between features in a layer are nonlinear (due to attention mixing across positions and MLP nonlinearities), so simple linear projection through the model's weight matrices does not correctly capture how features causally influence each other. The ablation approach, which runs the actual model with and without a feature, respects these nonlinearities by computing the true counterfactual effect.

Summary of Design Choices and Their Justifications

$\ell_1$ penalty over $\ell_0$ : The $\ell_1$ norm is the tightest convex relaxation of the $\ell_0$ "number of non-zero elements" penalty, making it computationally tractable while still inducing sparsity. The $\ell_2$ penalty would encourage many small coefficients rather than sparse exact zeros.
ReLU activation over sigmoid/tanh: ReLU produces exact zeros, which is essential for genuine sparsity. It also enforces non-negativity, matching the intuition that features are either present or absent.
Tied encoder-decoder weights over separate matrices: Based on the symmetry assumption that feature detection and expression should use the same direction; validated empirically for residual streams but acknowledged as potentially suboptimal for MLP layers.
Row normalization of dictionary: Prevents the autoencoder from reducing the $\ell_1$ penalty by scaling up feature magnitudes rather than genuinely reducing the number or magnitude of active features.
Autointerpretability over human evaluation: Scales to thousands of features without requiring expensive expert annotation, though it inherits the limitations of the LLM evaluator (may miss patterns involving next-token prediction or abstract concepts).
ACDC for feature selection over manual selection: Provides an unbiased, principled ordering of causal importance for activation patching experiments, avoiding experimenter degrees of freedom in choosing which features to patch.
Ablation-based circuit tracing over weight-projection methods: Ablation respects the nonlinearities of the model's computation (attention mixing, MLP activations), which weight-projection ignores, making it the more faithful method for tracing causal dependencies between features.

4. Key Insights and Innovations

Innovation 1: Reframing Polysemanticity as a Recovery Problem Rather Than an Architectural Flaw

Before this paper, the dominant framing of polysemanticity was essentially diagnostic: "neurons respond to multiple unrelated things, and this makes them hard to interpret." The implicit remedy was either to change the model architecture to force feature-neuron alignment (e.g., SoLU activations from Elhage et al., 2022a; sparsity penalties during training from Kasioumis et al., 2021) or to accept polysemanticity as an inherent messiness and work around it through labor-intensive manual analysis.

This paper makes a conceptually sharper move. By grounding their work in the superposition hypothesis (Elhage et al., 2022b), they reframe polysemanticity not as a failure of the model to organize its representations cleanly, but as an optimal compression strategy under a representational bottleneck. The model learns more features than it has dimensions and encodes them in non-orthogonal directions. Individual neurons appear polysemantic because they partially align with multiple feature directions. The problem, then, is not "neurons are messy"—it is "features are entangled, and we need to disentangle them."

This reframing matters because it transforms the interpretability challenge from a qualitative complaint into a well-posed recovery problem: given activation vectors, recover the overcomplete basis of feature directions that generated them. This is sparse dictionary learning (Olshausen & Field, 1997), a problem with decades of theoretical and algorithmic development. The paper's technical contribution—training sparse autoencoders on language model activations—is a specific solution to this recovery problem. But the deeper intellectual move is establishing that this is the right problem to solve. If polysemanticity is a symptom of superposition, then improving interpretability requires finding the superposed features, not just documenting neuron behavior.

The significance of this reframing extends beyond the paper's specific method. It provides a principled explanation for why PCA fails to produce interpretable directions (PCA maximizes variance, but the features of interest may be low-variance directions in a non-orthogonal basis). It explains why ICA performs better among baselines (ICA maximizes non-Gaussianity, which correlates with sparsity because sparse activations are heavy-tailed—Appendix C.4, Table 2 shows correlations of 0.19 and 0.24 between interpretability scores and skew/kurtosis). And it gives a clear criterion for success: the quality of a decomposition method should be judged by whether it recovers directions that correspond to individual features, not just whether it represents variance efficiently.

This is a fundamental reframing, not an incremental refinement, because it changes what the field should optimize for. Under the old framing, a method that produced "more interpretable neurons" was competing with other ways of looking at neurons. Under the new framing, the goal is to find the latent feature basis, and any method—autoencoders, ICA, or future alternatives—should be evaluated on how well it achieves that recovery.

Evidence for the power of this reframing appears throughout the paper, but the most direct support comes from the activation patching results in Figure 3: the fact that patching a few sparse dictionary features can shift model outputs as much as patching many more PCA components demonstrates that the autoencoder has recovered directions that align with causally coherent features, not just statistically convenient ones. This is exactly what the superposition recovery framing predicts: the ground-truth features should be causally relevant to model computation, whereas variance-maximizing directions need not be.

Innovation 2: Establishing Causal Efficiency as an Additional, Independent Validation Criterion Beyond Interpretability Scores

Prior interpretability work—including Bills et al. (2023), which introduced the autointerpretability protocol—focused almost exclusively on descriptive metrics: can a human (or an LLM proxy) produce a natural language description of what a neuron or feature does, and does that description predict held-out activation patterns? This is a sensible first-pass evaluation, but it has an obvious limitation: a feature could be highly interpretable in isolation yet computationally irrelevant. The model might have learned a direction that correlates with some real-world concept but never actually uses that direction in downstream computation.

The paper introduces a second, independent validation axis: causal efficiency. Section 4 asks not "can we describe this feature?" but "can we manipulate model behavior by editing this feature?" The activation patching protocol tests whether intervening on dictionary features moves model outputs toward counterfactual targets. The key finding—that sparse dictionary features achieve a given KL divergence with fewer features patched and smaller edit magnitudes than PCA components (Figure 3)—demonstrates that the features are not just interpretable descriptors but computationally active components that the model's downstream layers actually read from and depend on.

This is a diagnostic innovation, not merely a different experiment. It separates two properties that are often conflated in interpretability research: semantic coherence (does this direction correspond to a human-understandable concept?) and causal relevance (does the model actually use this direction to compute its outputs?). A method might excel at the former while performing poorly on the latter, or vice versa. By evaluating both, the paper provides a richer picture of decomposition quality and catches failure modes that either metric alone would miss.

The non-sparse dictionary control (α = 0 in Figure 3) makes this point sharply. A non-sparse autoencoder still learns a complete basis—it can perfectly reconstruct the activations—but its features are causally inefficient. This tells us that good reconstruction alone is insufficient for useful decomposition. Sparsity is doing work here: it forces the autoencoder to identify the individual causal factors in the data rather than arbitrary linear combinations that happen to span the space.

The significance of this dual-criterion approach extends to how future interpretability methods should be evaluated. Typical workflow: propose a decomposition, measure interpretability scores, claim progress. This paper suggests that interpretability scores are necessary but insufficient—causal validation is equally important, and methods that score well on one axis may fail on the other. This sets a higher bar for the field and provides a template for more rigorous evaluation.

Evidence: Figure 3 (left) shows the causal efficiency advantage quantitatively, while Figure 3 (right) shows it holds even when controlling for edit magnitude, establishing that sparse features achieve more targeted interventions rather than simply making larger changes to the model. The cross-layer replication in Appendix F (Figure 17, showing the effect holds across layers 3, 7, 11, 15, 19, and 23) strengthens the claim that this is a general property of the learned dictionary, not a layer-specific artifact.

Innovation 3: The Diagnostic Power of Negative Results—Where and Why Sparse Autoencoders Fail Reveals Structural Properties of the Network

Scientific papers often bury negative results or treat them as mere limitations. This paper does something more interesting: it treats failures as diagnostic tools that reveal structural properties of transformer internals. Several negative results are reported, and each carries a specific implication.

The dead feature problem in MLP layers (Appendix E, Figure 16). The paper finds that autoencoders trained on MLP activations produce large numbers of "dead features" that never activate, even when trained without a sparsity penalty (α = 0). This is not a generic training failure—it is diagnostic of the constrained geometry of MLP activation spaces. Because MLP outputs pass through a nonlinearity (GELU in Pythia models), they inhabit only a subset of the ambient vector space. Dictionary features initialized outside this reachable subset receive zero gradient and become permanently dead. This reveals that MLP layers, despite their nonlinearity (which one might naively think would make features more aligned with the neuron basis), actually present a harder sparse coding problem than the linear residual stream because their activation manifold is lower-dimensional and irregularly shaped. This finding suggests that interpretability methods developed for residual streams cannot be naively ported to MLP layers—different architectures or training procedures are needed.

The failure of weight-based circuit tracing (Appendix D.4). The paper attempted to trace feature circuits by multiplying a feature vector from one layer through the model's weight matrices and measuring cosine similarity with features in the next layer. This failed to produce meaningful connections. The diagnostic implication: feature interactions are inherently nonlinear in ways that simple weight projection cannot capture. Attention mixes information across token positions, and MLP nonlinearities create dependencies between features that only manifest when the full activation vector passes through the network. This negative result validates the necessity of the paper's ablation-based approach (Section 5.3) and implies that interpretability methods relying on linear projections through weight matrices (a common shortcut in early circuit analysis work) will miss genuine causal structure.

The declining interpretability advantage in later layers (Figure 2 and Section 3.2). The sparse autoencoder's advantage over ICA and PCA shrinks in deeper layers, with "minimal improvement in the final layer." This is not a failure of method quality but a diagnostic of changing representational structure: either features in later layers are genuinely more complex and harder for automated interpretation to capture, or later layers encode information about future tokens (predictions about what comes next) rather than current token properties, and the autointerpretability protocol—which only shows the current token context—cannot identify such features. The paper explicitly notes this possibility: "current LLMs are limited in the kinds of patterns that they can find, sometimes struggling to find patterns that center around next or previous tokens rather than the current token." This diagnostic failure points to a specific limitation of the evaluation protocol, not necessarily the decomposition method, and suggests that future work should incorporate output-side information (e.g., how ablating a feature changes the model's predictions) into the interpretation generation step.

The perplexity gap under reconstruction (Section 6.2). Replacing residual stream activations with their sparse reconstructions increases perplexity from 25 to 40 on the Pile. This is a genuine limitation—the dictionary does not capture all information. But it is also diagnostic: the gap between reconstruction accuracy (how well the autoencoder reproduces the activation vector) and behavioral fidelity (how well the reconstructed activations preserve model outputs) reveals that some of the unrecovered variance corresponds to features the model actually uses for prediction. The future work suggestion—minimizing output change rather than reconstruction error—follows directly from this diagnostic insight.

These negative results collectively signal that the paper is engaged in a systematic scientific investigation, not just a method demonstration. The failures constrain the space of viable approaches and point to specific directions for improvement. This is a methodological innovation in how interpretability research papers should present limitations: not as apologies, but as empirical findings that advance understanding of the system being studied.

Evidence: The dead feature data appears in Appendix E, Figure 16, showing the stark asymmetry between residual stream and MLP training. The weight-based tracing failure is documented in Appendix D.4. The layer-wise interpretability decline is visible in Figure 2 and the main-text discussion in Section 3.2. The perplexity measurement is reported in Section 6.2.

Innovation 4: Demonstrating That Task-Agnostic Decomposition Enables Task-Specific Causal Analysis Without Per-Task Retraining

A standard tension in interpretability research is between general-purpose methods (which decompose activations once into a basis that should be useful for many tasks) and task-specific methods (which identify the specific features relevant to a particular behavior, often by training probes or learning task-conditioned decompositions). The former risks finding features that are interpretable but irrelevant to any particular task of interest. The latter risks overfitting to the analyzed task and failing to generalize.

This paper resolves the tension by showing that a single, task-agnostic decomposition (trained on random web text activations from the Pile) produces features that are sufficiently causally precise for task-specific analysis—specifically, localizing the indirect object identification (IOI) behavior—without any IOI-specific training. The features are learned once, from unlabeled, general-domain data. Then, when the researchers want to understand IOI, they simply apply ACDC to select the relevant features from the existing dictionary (Section 4).

This is methodologically significant because it makes the approach amortized: the expensive step (training the sparse autoencoder) is paid once per model per layer, and the resulting dictionary can then be queried for many different behaviors. If each new behavior required retraining a specialized decomposition, the approach would not practically scale to understanding the full range of model capabilities. The paper's claim that "we expect this result to generalize to similar tasks and behaviours" (Section 6.2) is a bet on this amortization property.

The evidence for this claim is necessarily limited—the paper demonstrates only one task (IOI). But the task was chosen for a principled reason: it is a well-understood behavior with known neural correlates (Wang et al., 2022), making it a strong test of whether task-agnostic features align with known causal structure. The fact that they do—sparse features require fewer patches than PCA to redirect the model's output—suggests that the dictionary has recovered features that are genuinely fundamental to the model's computations, not artifacts of the training data distribution.

This innovation connects to a broader question in interpretability: is there a "natural" basis for a model's representations? If a model's knowledge and capabilities can be decomposed into a fixed set of features that are useful across tasks, then the interpretability problem reduces to finding that basis. If instead features are fundamentally task-dependent (the model reconfigures its representations for each new behavior), then task-agnostic decomposition is inherently limited. The paper's results—particularly the causal efficiency on IOI despite training on general text—provide evidence favoring the "natural basis" hypothesis, at least for the features involved in this specific task.

Evidence: Figure 3 shows IOI localization results using features trained on the Pile, not on IOI data. The comparison with PCA (which is also task-agnostic but lacks sparsity) shows that the advantage comes specifically from the learned sparse directions, not merely from having a task-agnostic decomposition. Section 6.2 contains the explicit generalization claim.

5. Experimental Analysis

Evaluation Methodology

Dataset. Activations are extracted from text in the Pile (Gao et al., 2020), a large, public web-text corpus of approximately 800GB of diverse text. For autointerpretability evaluation, the first 50,000 lines of OpenWebText are used to sample activating text fragments for each feature (Appendix A).
Base model(s). The primary models are Pythia-70M (residual stream dimension $d_{\text{in}} = 512$ ) and Pythia-410M ( $d_{\text{in}} = 1024$ ), both from the Pythia suite (Biderman et al., 2023). These models were chosen as representative, publicly available language models with accessible internal activations. The focus is on residual stream activations, with some experiments on MLP sublayer activations and attention head sublayers.
Metrics. Three distinct metrics are used across different validation approaches:
- Autointerpretability score (Sections 3, Appendix A): The correlation between a language model's simulated per-token activations (given a natural language interpretation of a feature) and the feature's actual activations on held-out text fragments. Both "top-random" (mixing high-activation and random fragments) and "random-only" (random fragments only) variants are reported. Scores range from -1 to 1, with higher values indicating more interpretable features.
- KL divergence from target output (Section 4): $D_{\text{KL}}(z \| y)$ , where $y$ is the output logit distribution on a counterfactual target sentence and $z$ is the output distribution after patching selected features from the target into the base sentence. Lower values indicate more complete behavioral redirection.
- Mean edit magnitude (Section 4): The $\ell_2$ norm of the difference between original and patched residual stream vectors, averaged across token positions. Smaller magnitudes for a given KL divergence indicate more precise (less disruptive) interventions.
Baselines. Four comparison methods are evaluated (Section 3.2):
- Default basis (neurons): The standard basis vectors of the residual stream or MLP layer, with negative activations zeroed out to match the non-negativity of autoencoder features.
- Random directions: Randomly sampled unit vectors in activation space, with negative activations similarly zeroed.
- Principal Component Analysis (PCA): Directions of maximal variance computed via online estimation on the same quantity of data used for autoencoder training (5–50 million activation vectors). PCA directions are orthogonal and sorted by explained variance.
- Independent Component Analysis (ICA): Directions maximizing statistical independence, run on approximately 4 million activations for residual stream (1 million for MLPs) due to slower convergence.
Generation budget / compute accounting. For autoencoder training, compute is measured in activation vectors processed (5–50 million) for 1–3 epochs, with larger dictionaries taking longer to converge. A single training run completes in under an hour on a single A40 GPU (Appendix B). For autointerpretability, 50,000 lines of OpenWebText are processed per feature, with 20 top-activating fragments identified, 10 used for interpretation generation and evaluation (5 for generation, 5 for simulation + 5 random for evaluation). For activation patching, the budget is measured in number of features patched (variable) and total edit magnitude ( $\ell_2$ norm of the residual stream change).
Cross-validation / statistical protocol. For autointerpretability scores, the paper reports means across 150 features per method per layer, with 95% confidence intervals around the mean (Figure 2). The scoring uses held-out fragments not seen during interpretation generation—5 highly activating fragments and 5 random fragments are used for simulation and scoring, with the interpretation generated from a different set of 5 top-activating fragments. For activation patching, ACDC (Conmy et al., 2023, Algorithm 4.1) provides a principled feature ordering on a test set of 50 IOI data points, averaged over these points to produce an ordering from most to least causally important.

Main Quantitative Results

Interpretability Comparison Against Baselines (Section 3.2)

The headline result appears in Figure 2: sparse autoencoder dictionary features achieve substantially higher average top-and-random autointerpretability scores than all four baselines (default neuron basis, random directions, PCA, ICA) across early residual stream layers. The error bars show 95% confidence intervals around the mean for 150 features per method.

At layer 1 of the residual stream, the sparse dictionary features achieve a mean interpretability score above 0.2, while the neuron basis and random directions cluster near 0.0–0.05, PCA achieves roughly 0.0–0.05, and ICA reaches approximately 0.1–0.15. The sparse dictionary advantage is therefore roughly 2–4× that of ICA and substantially larger compared to PCA or the default basis at this layer.

However, the paper explicitly notes that "the strength of this effect declines as we move through the model, being comparable to ICA in layer 4 and showing minimal improvement in the final layer" (Section 3.2). By layers 4–5, ICA and sparse autoencoder scores largely overlap within error bars, and all methods show low absolute scores (below 0.1 for the final layer).

The paper interprets this layer-wise decline cautiously. It may indicate that sparse autoencoders genuinely work less well in later layers—perhaps because features become more abstract or context-dependent. But it may also reflect limitations of the autointerpretability protocol itself: "later features may be more complex, and because they are often best explained by their effect on the output. Bills et al. (2023) showed that GPT-4 is able to generate explanations that are very close to the average quality of the human-generated explanations given similar data. However, they also showed that current LLMs are limited in the kinds of patterns that they can find, sometimes struggling to find patterns that center around next or previous tokens rather than the current token, and in the current protocol are unable to verify outputs by looking at changes in output or other data" (Section 3.2). In other words, later-layer features might be about predicting future tokens, and the autointerpretation protocol—which only shows current-token context—cannot capture this.

The random-only variant of the scoring (Appendix C.2, Figure 9) shows the same pattern of sparse dictionary advantage in early layers, though absolute scores are lower across all methods. This is expected because random fragments will rarely contain high activations for sparse features, making the interpretation task harder. The paper notes that "since our learned features are more sparse than the baselines and thus, activate less on a given fragment, this is likely to underestimate the performance of sparse coding relative to baselines" (Appendix C.2).

The top-K comparison in Appendix G (Figure 18) addresses the concern that autoencoder features activate on less than a full half-space (due to the ReLU bias), whereas PCA and ICA directions naturally activate on an entire half-space. Forcing PCA and ICA to also have only $K$ active directions per datapoint (where $K$ matches the average number of active features for the sparse autoencoder at $R=1$ , $\alpha=8.6\times10^{-4}$ ) produces only a small improvement in their scores. The sparse dictionary advantage persists, confirming that "this change does not explain more than a small fraction of the improvement in scores" (Appendix G).

Dictionary Size and Interpretability (Appendix C.1)

Figure 8 shows that interpretability scores are not limited to overcomplete dictionaries ( $R > 1$ ). Even undercomplete dictionaries ( $R = 0.5$ ) with the same sparsity coefficient ( $\alpha = 8.6 \times 10^{-4}$ ) outperform baselines, and interpretability remains relatively stable across $R \in \{0.5, 1, 2, 4, 8\}$ . The paper notes that "these small dictionaries are able to reconstruct the activation vectors less accurately, so with each feature being similarly interpretable, the larger dictionaries will be able to explain more of the overall variance" (Appendix C.1). This implies that sparsity, not overcompleteness per se, is the primary driver of the interpretability improvement.

MLP Sublayer Results (Appendix C.3, Figure 10)

Applying the same approach to MLP sublayers yields "mixed success" (Appendix C.3). Early MLP layers produce interpretable features that outperform baselines, with top-random scores showing a clear advantage over neurons, random directions, PCA, and ICA at layers 0–1. However, the method learns "many dead features, which never activate across the entire corpus. In some cases, there are so many dead features that the set of living features does not form an overcomplete basis. For example, in a dictionary with twice as many features as neurons, less than half might be active enough to perform automatic interpretability" (Appendix C.3). The paper reports that using separate encoder and decoder matrices (untied weights) helps retain more active features for MLP layers, though "we are currently working on methods to overcome this and find truly overcomplete bases in the middle and later MLP layers" (Appendix C.3).

Causal Localization on the IOI Task (Section 4)

Figure 3 presents the key patching results for layer 11 of the Pythia-410M residual stream. The left panel plots number of features patched against KL divergence from the target output, using the feature ordering provided by ACDC.

The sparse dictionary ( $\alpha = 8.6 \times 10^{-4}$ ) achieves a given KL divergence with substantially fewer features patched than PCA or the non-sparse dictionary ( $\alpha = 0$ ). For example, at approximately 10 features patched, the sparse dictionary achieves a KL divergence roughly an order of magnitude lower than PCA at the same number of features. The non-sparse dictionary ( $\alpha = 0$ ) performs worse than the sparse dictionary at all feature counts, and its curve largely overlaps with or underperforms PCA—demonstrating that the causal efficiency benefit specifically comes from the sparsity constraint, not from the autoencoder architecture itself.

However, the paper notes a tradeoff: "dictionaries with a larger sparsity coefficient $\alpha$ have lower overall reconstruction accuracy which appears in Figure 3 as a larger minimum KL divergence" (Section 4.2). The $\alpha = 8.6 \times 10^{-4}$ dictionary reaches a lower minimum KL divergence (approximately $10^{-2}$ ) than the very sparse $\alpha = 5 \times 10^{-3}$ dictionary (which plateaus above $10^{-1}$ ). This is the sparsity-reconstruction tradeoff manifesting in behavioral terms: very sparse dictionaries capture fewer features overall, so they cannot fully redirect the model's output even when all available features are patched.

The right panel of Figure 3 shows mean edit magnitude against KL divergence, addressing the concern that sparse features might simply make larger edits rather than more targeted ones. The sparse dictionary improves the Pareto frontier: for a given KL divergence, the edit magnitude is smaller than for PCA or the non-sparse dictionary. This means sparse features achieve more precise interventions, changing only the behaviorally relevant directions rather than making large, diffuse changes to the residual stream.

These results generalize across layers. Appendix F, Figure 17 shows the same pattern for layers 3, 7, 11, 15, 19, and 23 of Pythia-410M. At every layer tested, the sparse dictionary ( $\alpha = 8.6 \times 10^{-4}$ , $R = 2$ ) achieves lower KL divergence at given feature counts than PCA or the non-sparse dictionary. The absolute KL divergence levels vary by layer—some layers are more causally central to IOI behavior than others—but the relative advantage of sparse features is consistent.

The paper reports these patching experiments using dictionaries trained on only the first 10,000 elements of the Pile (approximately 7 million activations), not the full 5–50 million used for interpretability experiments. This is a deliberate choice that demonstrates the method works with relatively little data.

Feature Activation Statistics (Appendix C.4, Table 2)

To understand what properties of learned features drive interpretability, the paper computes correlations between feature moments and interpretability scores across all layers and dictionary sizes ( $R \in \{0.5, 1, 2, 4, 8\}$ ). Skew shows a correlation of 0.20 with top-random interpretability scores, and kurtosis shows a correlation of 0.15. Mean and variance show correlations near zero (-0.09 and 0.02 respectively). The paper interprets this as evidence that "the degree of interference due to other active features will be roughly normally distributed by the central limit theorem. If this is the case, then features will be notable for their heavy-tailedness" (Appendix C.4). This also explains ICA's relative strength among baselines—it explicitly maximizes non-Gaussianity, which corresponds to heavy-tailed activation distributions.

Ablation Studies and Robustness Checks

Sparsity coefficient $\alpha$ sweep: Appendix B, Figure 7 shows the tradeoff between sparsity (fraction of variance unexplained) and the $\ell_1$ penalty weight $\alpha$ , with each series representing a different number of training epochs (1–3). The tradeoff is smooth with no sharp phase transition—the paper notes that "the lack of a 'bump' or 'knee' in these plots provides some evidence that there is not a single correct way to decompose activation spaces into a sparse basis" (Appendix B). The $\alpha$ values used for main experiments ( $8.6 \times 10^{-4}$ for residual stream, $3.2 \times 10^{-4}$ for MLP) were selected based on the number of active features (Figure 16) rather than from this tradeoff curve directly.
Dictionary size ratio $R$ : Figure 8 (Appendix C.1) varies $R$ from 0.5 to 8 while holding $\alpha = 8.6 \times 10^{-4}$ constant. Interpretability scores remain relatively stable across this range, with all dictionary sizes substantially outperforming baselines. This is a robustness check confirming that the interpretability benefit does not depend on a carefully tuned dictionary size.
Non-sparse dictionary control ( $\alpha = 0$ ): Figure 3 (left panel) includes a dictionary trained with no sparsity penalty. This dictionary performs comparably to or worse than PCA on causal localization, establishing that sparsity—not the autoencoder architecture itself—drives the causal efficiency improvement. The same control appears in Figure 3 (right panel) for edit magnitude.
Top-K PCA and ICA baselines: Appendix G, Figure 18 controls for the concern that autoencoder features activate on less than a full half-space (due to ReLU bias), while PCA/ICA directions activate on an entire half-space. Forcing PCA and ICA to have exactly $K$ active directions per datapoint (matching the autoencoder's average active feature count) produces only a small fraction of the interpretability improvement. The sparse dictionary advantage is therefore attributable to the learned directions themselves, not to the restricted activation extent.
Random-only vs. top-random scoring: Appendix C.2, Figure 9 recomputes interpretability scores using only randomly selected fragments for the correlation computation (no highly-activating fragments). Absolute scores drop substantially—"matching Bills et al. (2023), we find that random-only scores are significantly smaller than top-and-random scores" (Appendix C.2)—but the relative advantage of sparse features over baselines persists, especially in early layers. The paper acknowledges this as a conservative test: sparse features activate rarely, so random fragments mostly contain zeros, making correlation harder to measure with limited sample sizes.
Tied vs. untied weights for MLP layers: Appendix C.3 reports that using separate encoder and decoder matrices (Equations 5–6) helps retain more active features when training on MLP activations, compared to tied weights. This is presented as an empirical finding rather than a formal ablation with quantitative scores, but it informs the recommendation for MLP training.
Layer-wise replication of patching results: Appendix F, Figure 17 replicates the causal localization advantage (sparse dictionary vs. PCA vs. non-sparse) across layers 3, 7, 11, 15, 19, and 23 of Pythia-410M. The sparse dictionary consistently outperforms PCA at all layers, though the absolute KL divergence levels vary by layer, with some layers showing larger gaps between methods than others.
Feature moment correlations (skew and kurtosis): Table 2 (Appendix C.4) provides correlational evidence that learned features with higher skew and kurtosis tend to be more interpretable, consistent with the theoretical expectation that heavy-tailed activation distributions indicate reduced interference from other features. No causal claim is made—this is purely correlational evidence.

Critical Assessment

Claim from the executive summary: "dictionary features are substantially more interpretable than neurons, PCA components, or ICA directions."

This claim holds for early residual stream layers where the gap is large and outside error bars (Figure 2, layers 1–3). However, the claim weakens substantially in later layers, where sparse autoencoder features are "comparable to ICA in layer 4" and show "minimal improvement in the final layer" (Section 3.2). The paper's abstract and introduction present this as a general finding, but the body text reveals important layer-wise caveats: the method's advantage over ICA is concentrated in early layers, and absolute interpretability scores are low across all methods in later layers (<0.1 for the final layer). This is not a failure of the method per se—it may reflect genuine differences in representational structure or limitations of the evaluation protocol—but the claim of general interpretability improvement needs to be qualified by layer.

A deeper issue: the paper does not establish whether the measured interpretability scores correspond to practically useful interpretability. A score of 0.2–0.3 is low in absolute terms—it means that GPT-3.5's simulation predictions correlate only weakly with actual activations. Table 1 shows individual feature scores ranging from -0.11 to 0.57, with most clustered below 0.6. The paper does not provide a human baseline for what score represents "good enough" interpretability, nor does it compare to human-generated interpretations. Bills et al. (2023) established that GPT-4 interpretations approach human quality for neurons, but the absolute correlation levels reported here suggest that even the best features remain substantially below what a human expert could achieve with full context. Without a calibration point, it is unclear whether a score of 0.33 (feature 1-0000) means the feature is genuinely understood or merely partially characterized.

The random-only scores in Figure 9 are substantially lower than top-random scores—often below 0.1 even in early layers. Since random-only scoring tests whether the interpretation captures the feature's behavior outside its highest-activation contexts, the low scores suggest that the interpretations may be overfitting to obvious patterns in the top-activating examples while failing to characterize the feature's full activation range.

Claim from the executive summary: dictionary features "enable more precise causal localization of model behavior on the IOI task—requiring fewer feature patches and smaller edit magnitudes to achieve a given KL divergence from target outputs than PCA decompositions."

This claim is well-supported by Figure 3 and replicated across layers in Figure 17. The data unambiguously show that sparse autoencoder features are more causally efficient than PCA components. The non-sparse dictionary ( $\alpha = 0$ ) control is crucial and cleanly shows that sparsity drives the effect.

However, there is an important scope limitation: the paper tests this claim on exactly one task (IOI) with one prompt structure (sentences about giving objects between named entities). The IOI task has known, well-localized circuitry in transformer models (Wang et al., 2022), which makes it a good testbed but also a potentially favorable case for any decomposition method. Features might localize cleanly on IOI precisely because IOI is implemented by a small number of specialized features, and the same dictionary might not provide similarly clean localization on a more distributed or complex task. The paper acknowledges this in Section 6.2: "more work is needed to confirm this suspicion." It is a single positive data point, not a systematic demonstration of task-generality.

A subtler issue: ACDC selects features by their marginal causal contribution to the output. The fact that sparse features require fewer patches than PCA features could partly reflect that ACDC is implicitly optimizing for sparsity in its feature selection (it greedily selects features that maximally reduce KL divergence), and sparse features happen to be better aligned with this greedy selection criterion. The paper does not compare against a non-greedy feature selection method or investigate whether the advantage holds for different feature subsets.

The edit magnitude comparison (Figure 3, right panel) addresses one important confound—that sparse features might simply make larger edits—but there is another confound it does not address: the sparse dictionary has $R \times d_{\text{in}}$ features versus PCA's $d_{\text{in}}$ components. With $R=2$ (the setting used in Figures 3 and 17), the sparse dictionary has twice as many directions to choose from. ACDC might simply be selecting from a larger pool, giving the sparse dictionary an unfair advantage. The paper does not compare against PCA with the same number of components (e.g., top- $2d_{\text{in}}$ PCA directions, if that were meaningful). The patching protocol uses the same number of features patched for both methods, but the selection pool for ACDC differs in size. This is not necessarily a flaw—after all, the sparse dictionary's ability to provide more directions is part of its claimed benefit—but it complicates the interpretation of "fewer features" as a measure of precision.

Claim from the executive summary: "non-sparse dictionaries fail to produce the same localization benefits."

Fully supported by the $\alpha = 0$ control in Figure 3. The non-sparse dictionary performs comparably to or worse than PCA, establishing sparsity as the causal factor. This is a clean, well-controlled result with a clear takeaway.

What is missing from the experimental analysis:

Zero-shot behavioral evaluation. The paper never measures how well the reconstructed activations preserve model behavior beyond the single perplexity measurement in Section 6.2 (which shows degradation from 25 to 40). For all the causal patching experiments, the patched activations are partial interventions (only some features are changed), not full reconstructions. The question of whether the full dictionary recovery (using all features simultaneously) can faithfully reproduce model behavior—rather than just few-feature interventions—is never addressed experimentally.
Comparison to probe-based methods. The paper compares against matrix decomposition baselines (PCA, ICA) but not against supervised probing methods that learn to detect specific features from labeled data. This is a fair scope limitation (the paper is about unsupervised decomposition, not supervised feature detection), but it means the claims about "more interpretable features" are relative to a specific set of baselines rather than to all possible approaches.
Scale to larger models. All experiments use Pythia-70M and Pythia-410M. These are very small by contemporary standards. The paper claims the method is "scalable" (title, abstract, Section 6.3), but this claim is never tested on models with billions of parameters, where the activation dimensionality is thousands rather than hundreds and training data requirements might be substantially different.
Dependence on ACDC for feature selection. The causal localization results depend on ACDC to identify causally important features. Without ACDC, it is unclear how an interpretability researcher would identify which features are relevant to a given behavior from the dictionary of thousands. The paper provides case studies showing that features can be manually identified by searching for high activations on probe sentences (Appendix D.3), but this manual approach was only demonstrated for simple, token-level features. The scalability of feature identification for complex, distributed behaviors remains unaddressed.
No direct comparison to supervised dictionary learning methods. Alternative approaches to sparse dictionary learning exist (e.g., matching pursuit, iterative hard thresholding) that might produce different dictionaries. The autoencoder architecture is one implementation choice, and the paper does not ablate this architectural choice against alternatives.
Training data quantity ablation. The paper uses 5–50 million activation vectors for training (Appendix B), but there is no systematic study of how interpretability scores or causal efficiency vary with training data quantity. For the patching experiments, a smaller dataset (7 million activations from 10,000 Pile elements) is used, suggesting the method works with less data, but this is not systematically varied.
The dead feature problem is described but not quantitatively characterized. Appendix E, Figure 16 shows the number of active features across different $\alpha$ and $R$ values, but the paper does not report what fraction of variance is explained by only the active features versus the full dictionary, nor does it measure whether dead features harm interpretability (by wasting capacity) or are merely neutral. The dead feature problem for MLP layers is identified as a limitation, but no ablation attempts to fix it (e.g., feature reinitialization, as suggested for future work).

These gaps do not undermine the paper's core contributions, which are substantial and well-supported by the experiments that were run. But they constrain the generality of the claims. The demonstrated benefits—improved interpretability in early residual stream layers, more efficient causal localization on IOI—are real but are established for specific models (Pythia-70M/410M), specific layers (primarily residual stream), and specific tasks (IOI for causal validation). The paper's claim that the method "may serve as a foundation for future mechanistic interpretability work" (abstract) is well-supported by the evidence presented. The stronger implicit claim that these benefits will generalize to larger models, other layers, and arbitrary tasks remains an open hypothesis that the paper does not experimentally test.

6. Limitations and Trade-offs

6.1 The Method Fails to Recover Features in Later Layers of the Residual Stream and in MLP Sublayers

The assumption or constraint. The paper's abstract and introduction claim that sparse autoencoders produce dictionary features that "are more interpretable and monosemantic than directions identified by alternative approaches," with the strong implication that this holds broadly across model components. However, the body of the paper reveals a sharply bounded scope. The interpretability advantage over baselines is concentrated in early residual stream layers. By layer 4, sparse autoencoder scores are "comparable to ICA" and the final layer shows "minimal improvement" (Section 3.2). For MLP sublayers, the results are explicitly described as "mixed success" (Appendix C.3): many learned features become "dead" (never activating), and in dictionaries with $R = 2$ , "less than half might be active enough to perform automatic interpretability." The paper acknowledges this: "the training pipeline used to train the dictionaries in this paper is not able to robustly learn overcomplete bases in the intermediate layers of the MLP" (Section 6.2).

The consequence. This limitation restricts the method's practical utility in two important ways. First, the later layers of transformer models are where high-level reasoning, task integration, and output prediction happen. If the method cannot recover interpretable features in these layers, it cannot provide end-to-end understanding of model computation—it can only characterize early-stage, often token-level, feature extraction. Someone attempting to trace a circuit through a model using this method would lose visibility in the layers closest to the output, which are precisely where task-specific computations are often consolidated. Second, the failure on MLP layers means that a large fraction of the model's nonlinear computation remains opaque. The MLP sublayers perform substantial transformation of representations, and the paper provides no working method to decompose these computations into features. The residual stream decomposition captures information flowing between sublayers but not the transformations occurring within sublayers.

What evidence exists in the paper. The layer-wise decline is visible in Figure 2, where the gap between sparse autoencoders and ICA closes from layer 1 through layer 5, converging to near-zero in the final layer. The autointerpretability scores for all methods in the final layer are below 0.1 (on a scale where 1.0 would be perfect prediction), meaning even the best features are barely interpretable by this metric. The dead feature problem is documented in Appendix E, Figure 16, which shows the number of active features for MLP layers dropping sharply compared to residual stream layers at the same hyperparameter settings. Appendix C.3 notes: "in some cases, there are so many dead features that the set of living features does not form an overcomplete basis." The weight-tying ablation (using separate encoder/decoder for MLPs) is mentioned as a partial mitigation but no quantitative improvement is reported.

Mitigation status. The paper acknowledges this limitation explicitly in Section 6.2: "We're excited by future work investigating what changes can be made to better understand the computations performed by the attention heads and MLP layers, each of which poses different challenges." For MLP layers specifically, the paper mentions plans to reinitialize dead features during training: "In future we plan to reinitialise such 'dead features' to ensure that we learn as many useful dictionary features as possible" (Appendix E). For later layers, the paper hypothesizes that the autointerpretability protocol itself may be the bottleneck rather than the decomposition—features in later layers may be best explained by their effect on output logits, which the current protocol does not provide to the interpreting model. Section 3.2 notes: "in the current protocol are unable to verify outputs by looking at changes in output or other data." However, no alternative protocol or experiment is implemented in this paper. The limitations are diagnosed but not resolved.

6.2 The Difficulty Estimation Cost for Practical Deployment Is Not Accounted for in Efficiency Measurements

The assumption or constraint. The entire approach requires training a separate sparse autoencoder for each layer of interest in each model. The paper reports that a single training run completes in under an hour on an A40 GPU using 5–50 million activation vectors (Appendix B). While this is modest relative to pretraining costs, it is a non-trivial fixed overhead that must be paid for each model-layer pair that one wishes to analyze. More importantly, to use the learned dictionary for a new task or behavior, a practitioner must run the model on the task inputs, cache the activations, encode them through the sparse autoencoder, and then identify which features are causally relevant—this typically requires some form of feature selection algorithm (like ACDC, used in Section 4) or manual search through large numbers of features (as in the case studies, Section 5). The paper provides no systematic method for efficiently identifying which features are relevant to a new behavior from the dictionary of potentially thousands of features.

The consequence. A practitioner attempting to apply this method to understand a new model behavior faces an uncharacterized search cost. The paper demonstrates that features can be identified (via probe sentences in Appendix D.3), but this manual approach was only shown for simple, token-level features where the relevant concept is obvious to the researcher. For a complex, abstract behavior—say, a model's tendency to produce biased completions or its capacity for analogical reasoning—it is unclear how one would locate the relevant features among $R \times d_{\text{in}}$ candidates. ACDC provides one principled approach (Section 4), but ACDC requires a specific counterfactual task setup (base and target sentences with measurable output divergence), a known causal metric (KL divergence from the target output), and repeated forward passes to estimate marginal contributions. This is feasible for the IOI task but may not generalize to behaviors without clean counterfactual pairs. The paper does not characterize the computational cost of ACDC-based feature selection relative to the autoencoder training cost, nor does it provide guidance on selecting ACDC hyperparameters for new tasks.

What evidence exists in the paper. The IOI results in Section 4 demonstrate that feature selection can be automated using ACDC, but this is a single demonstration on a well-characterized task. The case studies in Section 5 use manual search: the apostrophe feature was found by crafting a specific probe sentence ("I don't know about that. It is now up to Dave'") and checking which feature activated most strongly. This manual approach required the researcher to know in advance that apostrophe features exist and to design an appropriate probe. For a researcher approaching a novel behavior without prior knowledge of what features to expect, the search problem is unaddressed. The paper notes this implicitly in the discussion of automated circuit detection: "we expect this result to generalize to similar tasks and behaviours, but more work is needed to confirm this suspicion" (Section 6.2).

Mitigation status. The paper does not claim to provide a general feature identification method and is transparent about the current manual nature of feature search. The ACDC integration in Section 4 is a proof of concept that automated causal selection is possible for at least one task class. The paper's vision—"tracing the causal dependencies between features in different layers, with the overarching goal of providing a lens for viewing language models under which causal dependencies are sparse" (Section 6.2)—implicitly assumes that automated feature selection will scale. However, no experiments test this scalability, and no guidance is provided for practitioners attempting to apply the method to new tasks. This limitation is unmitigated in the current work and represents a gap between the method's demonstrated capability (finding interpretable features) and its practical deployability (efficiently finding the relevant features for a given question).

6.3 Causal Validation Is Demonstrated on Exactly One Task, Providing Weak Evidence for Task-Generality

The assumption or constraint. The paper's central practical claim is that the learned dictionary features enable precise causal manipulation of model behavior. The evidence for this claim comes entirely from Section 4, which tests a single task: indirect object identification (IOI) on Pythia-410M. The IOI task involves sentences like "Then, Alice and Bob went to the store. Alice gave a snack to ___" where the model must predict the indirect object. This task was chosen because "it captures a simple, previously-studied model behaviour" (Section 4) with known circuitry from Wang et al. (2022). The paper asserts that "because our dictionary is trained in a task-agnostic way, we expect this result to generalize to similar tasks and behaviours" (Section 6.2), but this expectation is asserted, not tested.

The consequence. A practitioner cannot assume that the causal efficiency observed on IOI will transfer to other behaviors of interest. IOI has specific properties that may make it unusually favorable for sparse feature analysis: (1) the relevant features involve concrete, name-like tokens with clear semantic content (person names), which are likely represented by distinct features in early-to-middle layers; (2) the task has a clean structure where a single token must be predicted based on syntactic role assignment, meaning the causal pathway is relatively short and localized; (3) the counterfactual setup (swapping one name for another) provides a clean intervention target that maps naturally to feature-level edits. It is entirely possible that behaviors involving more abstract reasoning, longer-range dependencies, or distributed representations across many features would not show the same clean localization. The paper provides no evidence to distinguish between the hypothesis that sparse features will generally be causally efficient across many tasks and the hypothesis that IOI is a cherry-picked favorable case.

What evidence exists in the paper. Section 4 contains the entirety of the causal patching evidence. Figure 3 shows results for layer 11 of Pythia-410M. Appendix F, Figure 17 replicates this pattern across layers 3, 7, 11, 15, 19, and 23, confirming that the causal efficiency advantage over PCA holds at multiple layers—but all on the same IOI task. There are no experiments on any other task, no analysis of how feature selection quality varies with task type, and no investigation of whether features identified as causally important for IOI are also important for other tasks or are IOI-specific. The paper's claim about generalization is explicitly hedged: "more work is needed to confirm this suspicion" (Section 6.2).

Mitigation status. The paper does not mitigate this limitation. It acknowledges the single-task scope explicitly and frames generalization as a hypothesis for future investigation. The choice of IOI is justified on scientific grounds—it is "a simple, previously-studied model behaviour" that provides a clean testbed—but this justification cuts both ways: the very properties that make IOI a good testbed (simplicity, prior characterization, localized circuitry) make it a poor basis for generalization claims. A stronger case would require at minimum one additional task from a different domain (e.g., factual recall, sentiment analysis, or code generation) to demonstrate that the causal efficiency advantage is not IOI-specific. The paper's title and framing emphasize broad applicability ("Sparse Autoencoders Find Highly Interpretable Features in Language Models"), but the causal validation—arguably the strongest form of evidence that the features matter for model behavior—is narrow.

6.4 The Perplexity Degradation Under Full Reconstruction Indicates That Important Information Is Not Captured

The assumption or constraint. Sparse autoencoders are trained to minimize reconstruction error of activation vectors (Equation 4: $\|x - \hat{x}\|_2^2$ ). However, the paper explicitly acknowledges that this objective does not guarantee behavioral preservation: "we do not achieve 0 reconstruction loss... indicating that our dictionaries fail to capture all the information in a layer's activations" (Section 6.2). The paper measures the behavioral impact of this information loss: replacing the residual stream activations in layer 2 of Pythia-70M with their sparse reconstruction increases perplexity on the Pile from 25 to 40—a substantial degradation.

The consequence. This perplexity gap reveals a fundamental tension: the reconstruction loss directly optimizes for geometric accuracy (making $\hat{x}$ close to $x$ in Euclidean distance), but the model's downstream behavior depends on the information content of $x$ , not its precise vector value. Two activation vectors that differ in Euclidean distance might carry identical behavioral information if the difference lies in directions that subsequent layers ignore. Conversely, two vectors that are close in Euclidean distance might produce very different outputs if the difference lies in behaviorally sensitive directions. The sparse autoencoder, by optimizing geometric reconstruction, may be allocating its representational capacity suboptimally from a behavioral perspective: it may spend capacity precisely reconstructing high-magnitude but behaviorally irrelevant directions while losing low-magnitude but behaviorally crucial information. The perplexity increase from 25 to 40 represents a genuine loss of model capability under full reconstruction, meaning that while individual features may be interpretable and causally relevant in isolation, the full dictionary does not faithfully capture the model's computation when used as a complete replacement.

What evidence exists in the paper. The perplexity measurement is reported in Section 6.2 as a single sentence with a single number: "replacing the residual stream activations in layer 2 of Pythia-70M with our reconstruction of those activations increases the perplexity on the Pile from 25 to 40." No additional layers, dictionary sizes, sparsity settings, or models are tested for perplexity impact. There is no decomposition of which features or which variance components account for the lost predictive power. The paper does not report perplexity for the reconstructed models used in the patching experiments (which use partial interventions, not full reconstruction), making it unclear whether the patching results would hold in a model where all features were simultaneously reconstructed rather than selectively edited.

Mitigation status. The paper acknowledges this limitation and proposes a direction for future work: "to try minimizing the change in model outputs when replacing the activations with our reconstructed vectors, rather than the reconstruction loss" (Section 6.2). This would be a behaviorally-aware training objective that replaces or augments the geometric reconstruction loss with a term that penalizes downstream output changes. However, no experiments in this direction are reported. The paper also suggests exploring "other sparse autoencoder architectures" that might achieve better reconstruction-behavior tradeoffs. This limitation is therefore diagnosed and a mitigation sketched, but no solution is implemented or validated.

6.5 All Experiments Are on Small Models (70M–410M Parameters), Leaving Scalability to Modern LLMs Unverified

The assumption or constraint. The paper is titled "Sparse Autoencoders Find Highly Interpretable Features in Language Models" and its abstract claims the method is "scalable." All experiments are conducted on Pythia-70M and Pythia-410M (Biderman et al., 2023), with residual stream dimensions of 512 and 1024 respectively. These are two to three orders of magnitude smaller than current frontier models (e.g., Llama-2-70B at 70 billion parameters with hidden dimensions of 8192, or GPT-4 at an estimated 1.7 trillion parameters). The paper states that training runs "complete in under an hour on a single A40 GPU" (Appendix B), but this is for models with $d_{\text{in}} = 512$ –1024 and dictionary sizes up to $R = 8$ (i.e., up to 8192 features). The computational and memory scaling of the method to $d_{\text{in}} = 8192$ or larger (where $R=2$ would mean ~16,000 features, $R=4$ would mean ~32,000 features) is not analyzed.

The consequence. Several aspects of the method may not scale linearly or at all to larger models. First, the dictionary size $d_{\text{hid}} = R \times d_{\text{in}}$ grows linearly with $d_{\text{in}}$ , meaning the parameter count of the autoencoder scales quadratically ( $M$ is $d_{\text{hid}} \times d_{\text{in}} = R \times d_{\text{in}}^2$ ). For a model with $d_{\text{in}} = 8192$ and $R = 2$ , the autoencoder would have approximately 134 million parameters—comparable to the size of the full Pythia-410M model, and training it would require proportionally more data and compute. Second, the activation dataset must be collected by running the (potentially very large) language model on text, which may itself be computationally expensive. Third, larger models may exhibit different superposition structure: the ratio of features to dimensions, the degree of sparsity, and the interference patterns between features may all change with scale, making hyperparameters (particularly $\alpha$ , the sparsity coefficient) non-transferable. Fourth, the autointerpretability protocol uses GPT-4 and GPT-3.5 as evaluation models, and the quality of their interpretations may degrade if the features in larger models represent more abstract or compositional concepts that are harder to describe in brief natural language.

What evidence exists in the paper. There is no evidence on scaling. The paper does not experiment with any model larger than Pythia-410M (which has 410 million parameters and residual stream dimension 1024). No scaling law experiments are conducted (e.g., measuring interpretability scores as a function of $d_{\text{in}}$ , model parameters, or training compute). No analysis of autoencoder training time or memory consumption as a function of model size is provided. The claim of scalability appears in the abstract and Section 6.3 ("Sparse autoencoders are a scalable, unsupervised approach") but is not experimentally supported.

Mitigation status. The paper makes no attempt to mitigate this limitation experimentally. The "scalable" claim appears to rest on the observation that autoencoder training is orders of magnitude cheaper than language model pretraining, which is true but does not address whether the method's performance scales to larger models. Section 6.2 proposes future work to "explore other sparse autoencoder architectures" and to incorporate "information about the weights of the model or dictionary features found in adjacent layers into the training process," which could potentially improve scaling properties, but no analysis of current scaling limitations is provided. A practitioner considering applying this method to a 7B, 13B, or 70B parameter model would have no experimental guidance on expected training costs, hyperparameter transfer, or interpretability quality at scale.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper makes a methodological pivot rather than a paradigm shift: it reframes the interpretability bottleneck from "neurons are messy" to "features are entangled, and sparse dictionary learning can disentangle them." The magnitude of the shift is moderate but consequential—it does not introduce fundamentally new mathematics (sparse coding dates to Olshausen & Field, 1997) or a new model architecture, but it establishes that a post-hoc, unsupervised, computationally cheap method can recover directions that are simultaneously more interpretable, more causally efficient, and more monosemantic than the neuron basis or standard matrix decompositions. This is the first paper to provide all three forms of validation (automated interpretability scores, causal patching efficiency, and qualitative monosemanticity) for learned dictionary features in language models, and the combination of evidence is what makes the case compelling.

The most significant conceptual shift is the reorientation from architecture modification to post-hoc recovery. Prior work on reducing polysemanticity focused on changing how models are built or trained (SoLU activations from Elhage et al., 2022a; $\ell_1$ penalties during training from Kasioumis et al., 2021; attention sparsity from Correia et al., 2019). This paper provides evidence that such modifications may be unnecessary: a standard, unmodified trained model already contains interpretable features in superposition, and a relatively simple autoencoder can extract them. If this result generalizes, it redirects research investment away from designing interpretability-friendly architectures (which has proven difficult and may compromise performance—the paper explicitly notes that "training a state-of-the-art foundation model with these additional constraints is difficult, and improvements to interpretability are not always realized," Section 6.1) and toward developing better post-hoc extraction methods. It also means that the vast number of already-deployed, unmodified models are potentially interpretable with this approach, not just future models designed with interpretability in mind.

The paper also resolves a tension between sparse coding theory and practical interpretability results. Sparse dictionary learning has theoretical guarantees for recovering ground-truth features under certain conditions (Wright & Ma, 2022; Sharkey et al., 2023), but prior to this work, it was unclear whether those conditions held for language model activations or whether the recovered features would correspond to human-understandable concepts. Yun et al. (2021) showed that sparse dictionaries could reconstruct transformer activations, but did not evaluate interpretability. This paper closes that gap: the features are not just mathematically sparse—they are semantically meaningful as measured by automated protocols and causally relevant as measured by activation patching. The fact that the non-sparse dictionary ( $\alpha = 0$ ) fails to produce causally efficient features (Figure 3) provides clean evidence that sparsity is the active ingredient, not the autoencoder architecture per se.

A specific contradiction this work helps reconcile: prior interpretability research found both that neurons can be interpretable (Olah et al., 2020; Bills et al., 2023) and that they are often frustratingly polysemantic (Olah et al., 2020). The superposition framing explains this discrepancy: neurons are interpretable when, by chance, a feature direction happens to align with a neuron basis vector, and they are polysemantic when multiple features project onto the same neuron. The sparse autoencoder approach provides a method for systematically finding the feature-aligned directions regardless of their relationship to the neuron basis, potentially making the interpretability successes the norm rather than the exception.

Research directions that become more attractive after this work:

Post-hoc feature extraction generally: sparse autoencoders are one implementation; other sparse coding algorithms (matching pursuit, iterative hard thresholding, FISTA) may offer different tradeoffs and should be systematically compared.
Verifier/feature quality improvement: the over-optimization analog in this context is "dead features" and reconstruction failure—improving autoencoder training to recover features in later layers and MLP sublayers is now a clearly defined technical challenge with empirical benchmarks (autointerpretability scores, perplexity under reconstruction).
Automated circuit discovery building on features: Section 5.3 provides a proof-of-concept for recursive ablation-based circuit tracing between dictionary features. Scaling this to end-to-end circuits is a natural next step, made more tractable by the causal sparsity observed in the IOI patching experiments (Section 4).

Research directions that become less attractive:

Pure neuron-level analysis without feature disentanglement: if superposition is the dominant organization principle and autoencoders can reliably extract the latent features, continuing to analyze neurons one-by-one without attempting to resolve superposition is increasingly a limited approach.
Architecture modification as the primary interpretability strategy: the success of a post-hoc method on standard architectures weakens the case that architectural changes are necessary, though they may still be valuable for future models if post-hoc methods prove insufficient for the hardest cases (later layers, MLPs, very large models).

The paper does not resolve the question of whether there exists a single "correct" decomposition. The smooth sparsity-accuracy tradeoff (Figure 6, Appendix B) with no phase transition or knee suggests that feature granularity is inherently resolution-dependent—analogous to how an image can be described at different levels of detail. The paper explicitly notes: "the lack of a 'bump' or 'knee' in these plots provides some evidence that there is not a single correct way to decompose activation spaces into a sparse basis" (Appendix B). This has a profound implication: enumerative safety (the "ambitious dream" of Section 6.3)—producing a complete list of a model's features—may be an ill-posed goal if features are fundamentally hierarchical or scale-dependent. The field may need to settle for interpretability at multiple resolutions rather than a single canonical feature list.

Follow-Up Research This Work Enables

Scaling sparse autoencoders to models with hidden dimensions ≥4096 and measuring whether interpretability scores, causal efficiency, and dead feature rates degrade. The paper demonstrates success on Pythia-70M ( $d_{\text{in}} = 512$ ) and Pythia-410M ( $d_{\text{in}} = 1024$ ), but makes no scaling measurements. A natural experiment: train autoencoders on residual streams of Pythia-1.4B, Pythia-6.9B, and Pythia-12B (all from the same suite, controlling for architecture), holding the training data quantity and hyperparameter selection protocol constant, and measure autointerpretability scores per layer as a function of model scale. The key question is whether the layer-wise decline in interpretability advantage (Figure 2) becomes more or less severe at scale. If larger models exhibit more superposition (higher feature-to-dimension ratios), the sparse autoencoder advantage might grow; if features in larger models are more abstract and harder for automated interpretation to capture, the advantage might shrink. The dead feature problem in MLP layers should also be characterized as a function of scale—do larger MLPs have a larger fraction of the activation space that is unreachable, making dead features more prevalent?

Behaviorally-aware autoencoder training that replaces or augments the geometric reconstruction loss with a term penalizing downstream output changes. Section 6.2 reports that full reconstruction of layer 2 activations increases perplexity from 25 to 40, indicating behavioral information loss despite reasonable geometric reconstruction. A direct follow-up: train an autoencoder with a composite loss $\mathcal{L}(x) = \|x - \hat{x}\|_2^2 + \alpha\|c\|_1 + \beta \cdot D_{\text{KL}}(p(\cdot|x) \| p(\cdot|\hat{x}))$ , where $p(\cdot|x)$ is the model's output distribution given the original activation and $p(\cdot|\hat{x})$ is the distribution given the reconstruction. Measure whether (a) the perplexity gap closes at equivalent sparsity levels, (b) the learned features remain interpretable (autointerpretability scores should not degrade), and (c) causal patching efficiency improves (since behaviorally-relevant directions are better preserved). The paper explicitly calls for this: "to try minimizing the change in model outputs when replacing the activations with our reconstructed vectors, rather than the reconstruction loss" (Section 6.2). The computational cost would be higher (requiring forward passes through the full language model during autoencoder training), but for interpretability purposes where the autoencoder is trained once per model-layer pair, this is likely acceptable.

Systematic task-generality testing of causal localization using 4–6 diverse behaviors beyond IOI. The causal patching results (Figure 3, Appendix F Figure 17) demonstrate that sparse autoencoder features efficiently localize IOI behavior across multiple layers, but this is a single task with specific properties (name tokens, syntactic role assignment, short causal pathway). To test whether the "task-agnostic dictionaries enable task-specific analysis" claim generalizes, select 4–6 behaviors spanning different cognitive categories: factual recall (e.g., the model's ability to retrieve "Paris" when prompted with "The capital of France is"), sentiment completion ("The movie was absolutely"), code syntax (predicting closing brackets in Python), gender bias (occupation-pronoun associations), and multi-hop reasoning. For each, run the same ACDC-based patching protocol and measure whether sparse features require fewer patches than PCA to redirect outputs. The hypothesis: token-level and syntactic behaviors (code syntax, sentiment tokens) should show strong localization (like IOI), while multi-hop reasoning may show more distributed causal structure, testing the limit of the sparse causal dependency assumption. A negative result (sparse features performing no better than PCA on complex behaviors) would refine the scope of the method's applicability.

Dead feature reinitialization and its effect on MLP layer interpretability. Appendix E documents that MLP dictionaries suffer from many dead features, and Appendix C.3 notes that using untied encoder/decoder weights helps but does not fully solve the problem. A concrete experiment: during autoencoder training, periodically (e.g., every N batches) identify features that have activated fewer than some threshold (e.g., 10 times across the last 100K datapoints), reinitialize their encoder rows to random directions plus the activation vector that would have most activated them (or to PCA directions of the residual data), and continue training. Measure: (a) fraction of dead features at convergence compared to no-reinitialization baselines, (b) autointerpretability scores of the revived features, (c) reconstruction quality (unexplained variance) for the same dictionary size. This is a low-engineering-effort experiment that directly addresses a diagnosed failure mode, and the paper already mentions it as planned future work (Appendix E: "In future we plan to reinitialise such 'dead features'").

Multi-layer joint dictionary learning that incorporates cross-layer consistency constraints. Section 5.3 notes that "many dictionary features across layers correspond to similar real-world features and often point in similar directions in activation space, as measured by cosine similarity" (Figure 5 caption). This observation, combined with the failure of weight-based circuit tracing (Appendix D.4, where multiplying a feature by the MLP weights and measuring cosine similarity with next-layer features produced "no meaningful connections"), suggests an opportunity: train autoencoders on multiple layers simultaneously with a regularization term that encourages features representing the same concept across adjacent layers to have similar encoder directions. The regularization could take the form of a soft constraint penalizing the distance between the feature direction in layer $\ell$ and its projection through the layer $\ell$ MLP/attention weights to layer $\ell + 1$ , evaluated on feature correlations observed in data (not direct weight projection, which failed). A strong follow-up would measure whether joint training (a) improves interpretability scores in later layers (where single-layer training currently underperforms ICA), (b) increases the cosine similarity of corresponding features across layers beyond what single-layer training achieves, and (c) enables more complete circuit tracing without the recursive ablation procedure (since cross-layer feature correspondence would be built into the dictionary). The current paper already uses tied encoder/decoder weights based on symmetry reasoning; cross-layer tying is a natural extension of that philosophy.

Practical Applications and Downstream Use Cases

Model auditing for specific failure modes or undesirable features without task-specific probing. An AI safety researcher wants to check whether a deployed language model has learned a feature for deceptive reasoning or a specific bias (e.g., gender-occupation stereotypes). Using this method, they would train one sparse autoencoder per residual stream layer (cost: <1 hour per layer on an A40 GPU for models up to ~1B parameters), then search for features whose activation patterns correlate with the target behavior by running the model on a curated set of prompt pairs (deceptive vs. honest completions, or stereotype-congruent vs. stereotype-incongruent sentences) and identifying features with systematically different coefficients across conditions. The paper demonstrates the feasibility of feature search via probe sentences in Appendix D.3 (where the apostrophe feature was found by checking which feature activated on "I don't know about that. It is now up to Dave'"), and ACDC in Section 4 provides a more automated alternative for tasks with counterfactual structure. The key practical benefit over neuron-level auditing: features discovered this way are more monosemantic (Figure 4 shows activation histograms concentrated on single token types, vs. the polysemantic neuron basis in Figure 11, Appendix D.1), meaning that editing or monitoring the feature is less likely to have unintended side effects on unrelated model capabilities. The paper's 4× overcompleteness demonstration ( $R=4$ with most features active for residual streams, Appendix E Figure 16) suggests that features are sufficiently disentangled that interventions can target specific concepts without collateral damage.

Targeted model editing for rapid patching of specific undesirable behaviors. A developer discovers that their deployed language model produces toxic completions in response to certain prompts. Using the sparse autoencoder approach, they identify which dictionary features are causally responsible for the toxic behavior (via the patching protocol of Section 4, measuring output changes when those features are ablated), then permanently suppress those features by setting their coefficients to zero at inference time or by editing the model weights to remove the feature directions from the residual stream. The paper's causal efficiency results (Figure 3) show that a small number of dictionary features can substantially redirect model outputs—if the toxic behavior is similarly localized, editing a handful of features could neutralize it. The key advantage over fine-tuning-based editing (which can cause catastrophic forgetting) or prompt-based filtering (which users can circumvent) is precision: the edit targets the specific computational pathway producing the toxicity while leaving unrelated capabilities intact. The paper's finding that sparse features achieve equivalent KL divergence with smaller edit magnitudes than PCA (Figure 3, right panel) directly supports this precision claim: sparse feature edits change less of the residual stream to achieve the same behavioral effect. The current limitation is that feature identification for arbitrary behaviors requires either manual search (as in the case studies) or an ACDC-like automated procedure (which needs a task with counterfactual structure). For toxicity editing to be practical, the field would need to develop efficient feature discovery methods that do not require per-behavior experimental design.

Circuit-level interpretability at scale for safety verification of critical model components. A research team wants to verify that a model's internal computation for a safety-relevant behavior (e.g., refusing harmful requests, following instructions faithfully) does not contain hidden failure modes. Using the recursive ablation-based circuit tracing method demonstrated in Figure 5 (closing parenthesis circuit), the team can systematically map the features and causal dependencies involved in the target behavior across all layers, producing a causal graph of feature interactions. The paper shows this is possible for a simple feature (closing parentheses) and the IOI task; extending it to more complex safety-relevant behaviors is the natural scale-up. The practical benefit over neuron-level circuit analysis (Wang et al., 2022; Conmy et al., 2023) is that dictionary features are monosemantic (Section 5.1), so the discovered circuit has units that can be individually interpreted and verified. The automated circuit detection pipeline from Section 5.3 provides a template: choose a target feature, measure upstream causal influences via ablation, recurse. The computational cost is the forward passes needed for ablation at each layer, which for a 24-layer model with thousands of features per layer is non-trivial but still orders of magnitude less than model retraining. The current bottleneck is the method's poor performance on later layers (where autointerpretability scores decline to near-zero, Figure 2) and MLP sublayers (where dead features dominate, Appendix C.3)—the circuit tracing method would currently lose resolution in the layers closest to the output, which are often the most policy-relevant for safety verification. The behavioral-aware training objective proposed in Section 6.2 would help close this gap by ensuring that behaviorally critical features are preserved in the dictionary even in later layers.