Memory Retrieval and Consolidation in Large Language Models through Function Tokens

🎯 Pitch

A mere 10 function tokens (like 'the' and 'in') activate over 70% of an LLM's features to control output, and steering a single activation on a newline can flip the model's answer from "Japan" to "Russia." Pre-training is dominated not by learning what these tokens mean, but by the much harder task of predicting the content word that comes next, which is what actually builds the model's memory.

1. Executive Summary

This paper proposes the Function Token Hypothesis to explain how large language models retrieve and consolidate memory: during inference, function tokens—roughly corresponding to punctuation marks, articles, prepositions, and conjunctions—dynamically activate the most predictive features from context to direct next-token prediction (memory retrieval), while during pre-training, predicting content tokens that follow function tokens drives the model to learn and expand its feature representations (memory consolidation). Using Gemma2-9B with sparse autoencoder decomposition on SlimPajama-627B, the authors demonstrate that a mere 10 function tokens activate over 70% of the model's features in middle layers and that steering activations on function tokens alone can reliably control output language and semantic content (e.g., activating a "Russia" feature on a newline token transforms "Japan" into "Russia" as the answer to "Where is Mount Fuji?"). Pre-training experiments with 1.5B and 8B LLaMA-3.1 architectures reveal that the function→content token prediction task dominates the training loss (e.g., loss of 4.88 vs. 1.90 for content→function at 1.5B scale), establishing that memory consolidation is primarily driven by the hardest prediction task—anticipating content after function tokens—while function token prediction itself is learned rapidly and nearly completely within the first 3,000 training steps.

2. Context and Motivation

The Core Problem: We Don't Understand How LLMs Remember and Recall

Despite the remarkable success of large language models—their ability to memorize vast factual knowledge, follow complex instructions, and chain together multi-step reasoning—the fundamental mechanisms by which these models store knowledge during training and access it during inference remain poorly understood. This paper frames these two processes as memory consolidation and memory retrieval, borrowing terminology from neuroscience where synaptic consolidation strengthens connections between neurons to form lasting memories. In LLMs, however, we lack a comparable mechanistic account of which tokens, which model components, and which training dynamics are responsible for building and accessing the model's internal representations.

The authors identify two specific, unresolved questions that motivate their investigation (Section 1):

"(1) How is the memory retrieved during inference? and (2) How is the memory consolidated during pre-training?"

These are not merely academic questions. Understanding memory mechanisms has direct practical implications for how we design training procedures, how we allocate compute during inference, and how we diagnose failures in model behavior. If we knew which tokens are responsible for retrieving knowledge from context, we could design more efficient inference strategies (e.g., caching only those tokens' representations). If we understood what drives feature learning during training, we could optimize data mixtures and training curricula to accelerate consolidation of useful knowledge.

The paper's central insight—one that gives it a distinctive lens on these questions—is that analyzing LLMs through the distinction between function tokens and content tokens can unravel the mystery. This distinction, borrowed from linguistics, separates tokens that carry primarily grammatical and structural information (function words like "the," "of," "in," "and," "is," plus punctuation) from tokens that carry semantic content (nouns, verbs, adjectives like "Harry," "capital," "Russia"). The paper argues that this binary classification, while crude, captures a fundamental asymmetry in how LLMs use different token types during both training and inference.

Why This Gap Exists: The Interplay of Scale, Superposition, and Measurement Difficulty

Understanding LLM memory is challenging for several interconnected reasons that the paper explicitly or implicitly addresses:

Polysemantic neurons and superposition. Prior to recent advances in interpretability, the dominant view was that individual neurons in neural networks encode single, interpretable concepts. However, as the paper notes in Section 2.1, the phenomenon of superposition—where a single neuron participates in representing multiple distinct features, and the total number of representable features far exceeds the number of neurons—makes direct inspection of activations insufficient for understanding what the model "knows" at any given point. A neuron that fires on both "Shakespeare" and "Paris" doesn't reveal its role through simple inspection. The development of Sparse Autoencoders (SAEs) by Cunningham et al. (2023) provided a tool to decompose these polysemantic activations into interpretable, monosemantic features, but the application of this tool to systematic questions about memory consolidation and retrieval dynamics is still nascent. The paper positions itself as one of the first efforts to use SAEs not just for static feature cataloging, but for understanding the temporal dynamics of feature activation across token types during generation and the developmental trajectory of feature emergence during pre-training.

The black-box nature of next-token prediction. LLMs are trained with a deceptively simple objective: predict the next token given all previous tokens. But this objective collapses qualitatively different prediction tasks into a single loss value. As the paper reveals in Section 4.3, predicting a function token after a function token (function→function) is fundamentally easier than predicting a content token after a function token (function→content)—the loss gap is enormous (roughly 2.12 vs. 4.88 at the 1.5B scale). Standard training loss curves that average over all tokens obscure this heterogeneity, making it impossible to see which prediction tasks drive learning and which are along for the ride. The paper's contribution is to disaggregate the loss by token type transitions, revealing that the optimization process is dominated by the hardest subtask—predicting content after function tokens—and that this subtask is precisely what forces the model to develop rich, context-dependent feature representations.

The scale of modern LLMs makes exhaustive analysis prohibitive. Training an 8B parameter model from scratch and running SAEs on every checkpoint is expensive. The paper navigates this by using a 2-layer 1.5B model for SAE analysis (Section 4.2) while verifying the loss patterns scale to 8B (Section 4.3). This pragmatic compromise between analytical depth and computational tractability is a deliberate design choice that acknowledges the difficulty of the problem.

Linguistic structure has been underutilized in mechanistic interpretability. While the field of mechanistic interpretability has made progress in identifying circuits for specific tasks (e.g., indirect object identification in GPT-2 small; Wang et al., 2023) and cataloging features through SAEs (Templeton et al., 2024), relatively little work has systematically exploited the structural properties of natural language itself—in particular, the functional distinction between grammatical scaffolding and semantic content—as an organizing principle for understanding model internals. The paper fills this gap by showing that the function/content token distinction maps cleanly onto distinct computational roles in the Transformer.

Where Prior Approaches Fall Short

The paper identifies several strands of related work and explains why they are individually insufficient:

Existing work on neural memory models. The paper traces a lineage from Hopfield networks (1982) through restricted Boltzmann machines to the key-value memory interpretation of Transformer FFN layers (Geva et al., 2021). While these frameworks provide a vocabulary for thinking about memory in neural networks—keys, values, retrieval, storage—they do not explain which inputs trigger memory retrieval or which prediction tasks drive memory consolidation in the specific context of autoregressive language modeling. The key-value memory interpretation tells us that the FFN layer stores knowledge in its parameters, but it doesn't tell us that function tokens are the primary "queries" that access this knowledge and that predicting content tokens after function tokens is what writes the knowledge during training.

SAE-based feature extraction, used statically. Prior work using SAEs (Cunningham et al., 2023; Templeton et al., 2024; Bricken et al., 2023) has demonstrated that interpretable features can be extracted from LLM activations and that steering these features can control model behavior (e.g., making a model talk about the Golden Gate Bridge; Templeton et al., 2024). However, this work has primarily been static: identify features, catalog them, steer them. It has not systematically analyzed which tokens activate which features during inference, how activation patterns differ between token types, or how features emerge over the course of pre-training. The paper extends the SAE toolkit from a static feature-discovery tool to a dynamic analysis instrument for studying memory retrieval and consolidation.

Scattered observations about "special" tokens, without a unifying framework. The paper cites several prior observations that, in retrospect, are explained by the Function Token Hypothesis but were previously treated as separate phenomena:

Massive activations (Sun et al., 2024): "activations with unusually large magnitudes often occur at the initial tokens, periods, or newlines." The paper argues these are all function tokens behaving as the hypothesis predicts—activating large numbers of features to retrieve context-relevant information.
Separator token dominance in attention (Chen et al., 2025): "meaningless separator tokens disproportionately affect attention compared to semantically rich tokens." This is another manifestation of function tokens' privileged role in memory retrieval.
Pivot tokens (Abdin et al., 2024): Phi-4 identifies "pivot tokens, often following function tokens, as critical for response accuracy." The paper reinterprets these as the content tokens whose prediction after function tokens drives feature learning.
High-entropy token training (Wang et al., 2025): "training that concentrates on high-entropy tokens also yields better performance." The paper notes that these tokens (e.g., "thus," "therefore") are typically function tokens or tokens that follow function tokens, aligning with the hypothesis that function→content prediction is the learning bottleneck.

What was missing was a unified causal explanation: why do these tokens behave this way? The Function Token Hypothesis provides that explanation: function tokens are the primary agents of memory retrieval (activating predictive features from context) and their following content tokens are the primary drivers of memory consolidation (the hard prediction task that forces feature learning).

No systematic study of pre-training dynamics at the token-type level. Prior work on pre-training dynamics has largely focused on aggregate metrics (loss curves, benchmark performance) or on the emergence of specific capabilities (in-context learning, induction heads). The paper's contribution is to track loss and feature counts at the granularity of function→function, function→content, content→function, and content→content transitions across the entirety of pre-training, revealing the developmental trajectory: the model first masters function token prediction (within ~3,000 steps), then gradually learns to predict content tokens, with the function→content category remaining the highest-loss, most optimization-driving category throughout training.

How This Paper Positions Itself

The paper positions itself at the intersection of three research traditions:

Interpretability through sparse autoencoders. It adopts the SAE toolkit from Cunningham et al. (2023), Templeton et al. (2024), and the Gemma Scope project (Lieberum et al., 2024), but uses it for a novel purpose: studying the dynamics of feature activation across token types during inference and the emergence of features during pre-training, rather than static feature cataloging.
Linguistic structure as an analytical lens. It imports the function word / content word distinction from linguistics (Carnap, 1937; Chomsky, 1957), operationalizes it for tokenized text using a simple frequency-based heuristic (the top 122 most frequent tokens covering 40% of corpus occurrences are "function tokens"; the rest are "content tokens"), and demonstrates that this operationalization maps onto fundamentally different computational roles in the Transformer.
Memory mechanisms as a bridge between neuroscience and machine learning. It adopts the consolidation/retrieval vocabulary from neuroscience (Josselyn & Tonegawa, 2020) and maps it onto training (parameter updates that form and expand features—consolidation) and inference (activation of features to direct prediction—retrieval). This framing allows the paper to ask questions about LLMs that have natural analogues in biological memory research.

The paper's most distinctive positioning move is its causal claim: function tokens are not merely correlated with feature activation and learning; they are responsible for them. This is supported by the steering experiments in Section 3.2 and Figure 7, where modifying activations exclusively on the final function token (\n) is sufficient to change model outputs in semantically coherent, context-appropriate ways. If activations on content tokens were equally important for retrieving predictive features, steering only the function token would not produce such reliable control. The asymmetry in steerability is evidence for the causal role of function tokens in memory retrieval.

The paper also positions itself as providing a unifying explanation for previously disconnected phenomena. By identifying the tokens involved in massive activations, separator tokens, pivot tokens, and high-entropy training tokens as all being function tokens or their downstream content tokens, the paper offers a single mechanism—function tokens as memory retrieval agents—that explains diverse empirical observations across multiple research groups and model families. This explanatory unification is presented as evidence for the hypothesis's validity and utility.

3. Technical Approach

3.1 Reader Orientation

This paper is primarily an empirical analysis study—it does not propose a new model architecture or training algorithm, but rather constructs an explanatory framework for how existing LLMs consolidate memory during pre-training and retrieve it during inference. The core idea is that analyzing LLM behavior through the lens of function tokens versus content tokens (a binary classification borrowed from linguistics and operationalized via token frequency) reveals a fundamental asymmetry: function tokens act as dynamic "query" operators that activate the most context-predictive features to guide generation, while predicting the content tokens that follow function tokens is the computationally hardest subtask that drives parameter updates and feature expansion during training. The "system" being analyzed is not a new deployment architecture but a conceptual model of LLM internals—a hypothesis about which components are responsible for which memory operations and why.

3.2 Big-Picture Architecture (Diagram in Words)

The analytical framework has four major components, each targeting a different aspect of the function token hypothesis:

Token Classification Pipeline: A preprocessing step that partitions the vocabulary into 122 "function tokens" (the most frequent tokens covering 40% of corpus occurrences—roughly corresponding to punctuation, articles, prepositions, conjunctions, and other high-frequency structural elements) and the remainder as "content tokens." This classification is performed once on the SlimPajama-627B corpus using the LLaMA-3.1 tokenizer and is fixed for all subsequent experiments.
Inference-Time Bipartite Graph Analysis (Memory Retrieval): A pipeline that feeds 10,000 randomly sampled raw documents (~5 million total tokens) into Gemma2-9B with Gemma Scope SAEs (sparse autoencoders with dictionary width 2^20 ≈ 1.05M features) attached at layers 9, 20, and 31. For each token position, the SAE decomposes the residual stream activation into a sparse set of active features. A bipartite graph is constructed with token vocabulary nodes on one side and feature nodes on the other, connected by an edge if that token ever activated that feature in any context. This graph quantifies how many features each token type "touches" during inference.
Steering Experiments (Causal Memory Retrieval): Using Gemma2-9B-it (the instruction-tuned variant), the authors identify specific interpretable features (e.g., Feature 15261 = "Speak Chinese," Feature 9591 = "Russia," Feature 13751 = "UK") through a contrastive prompt method and SAE decomposition. They then modify activations exclusively at the final function token (typically \n) before the model begins its response, and observe whether outputs change in semantically coherent ways. This tests whether function token activations are causally responsible for retrieving context-predictive features, as opposed to content token activations which should show weaker steering effects.
Pre-Training Dynamics Tracking (Memory Consolidation): Two models are trained from scratch on SlimPajama-627B using the LLaMA-3.1-8B architecture—a full 32-layer 8B model and a 2-layer 1.5B model. The training loss is disaggregated into four categories based on whether the current and next tokens are function or content tokens (function→function, function→content, content→function, content→content). For the 1.5B model, JumpReLU-SAEs (with tanh penalty, dictionary width 16× activation dimension = 65,536) are trained on second-layer activations at three checkpoints (3,000, 50,000, and 130,000 steps) to count the number of unique learned features and track how token-feature connectivity evolves during pre-training.

3.3 Roadmap for the Deep Dive

First, the token classification procedure and its linguistic justifications, since the function/content distinction is the foundational abstraction that all subsequent analyses depend on. I'll explain why a frequency-based operationalization works, what the resulting token set looks like, and its document coverage properties.
Second, the bipartite graph construction and analysis for memory retrieval (Section 3 of the paper), including SAE integration with Gemma2-9B, activation extraction, feature decomposition, and how token degrees are computed and interpreted. This establishes the core empirical claim: function tokens activate disproportionately many features.
Third, the steering experiments (Section 3.2 and Appendix A), including the contrastive method for identifying interpretable features, the layer selection procedure, the feature ranking and binary search for minimal steering sets, and the causal interpretation of results. This addresses why function tokens activate many features—because they dynamically reactivate context-predictive features.
Fourth, the pre-training loss tracking (Section 4.3), covering the four-category loss disaggregation, the training configurations for both 1.5B and 8B models, and the developmental trajectory (function token prediction learned first, function→content remaining hardest). This explains how function tokens acquire their retrieval capability—through the optimization pressure of predicting content tokens after function tokens.
Fifth, the SAE-based feature emergence tracking (Section 4.2), including JumpReLU-SAE training details, how reconstruction quality was equalized across checkpoints by tuning the L0 penalty coefficient, and what the growth in feature counts and token-feature degree distributions reveals about memory consolidation.

3.4 Detailed, Sentence-Based Technical Breakdown

This is an interpretability and training dynamics analysis paper whose core idea is that distinguishing function tokens from content tokens reveals the mechanisms of memory retrieval (function tokens activate context-predictive features to guide next-token prediction) and memory consolidation (predicting content tokens after function tokens is the dominant optimization pressure driving feature expansion).

Token Classification: Operationalizing "Function" vs. "Content"

The paper needs a binary partition of the token vocabulary that captures the linguistic intuition that some tokens carry primarily structural/grammatical information while others carry semantic content, but it must do so in a way that is automatic, reproducible, and applicable across tokenizers and corpora without manual linguistic annotation.

Frequency-based operationalization. The authors tokenize the SlimPajama-627B corpus (627 billion tokens of cleaned, deduplicated web text) using the LLaMA-3.1 tokenizer and sample 1 billion tokens for statistical analysis. They sort all token types by frequency in descending order and accumulate tokens from the most frequent downward until the selected set covers 40% of all token occurrences in the sample. This yields exactly 122 token types, which are labeled "function tokens." All remaining token types are labeled "content tokens."

Why 40% cumulative coverage? The paper does not provide an explicit justification for the 40% threshold, but the rationale can be inferred from the document coverage data in Figure 2b. The top 122 tokens include all the high-frequency structural elements (commas, periods, articles, prepositions, conjunctions, auxiliary verbs, whitespace, newlines, quotation marks, digits 0-9, and common formatting tokens) while excluding the vast majority of semantically rich nouns, verbs, and adjectives. The 40% threshold appears to be an empirical compromise: lower thresholds would exclude important structural tokens like "but," "which," "also," and "into" (which appear in Table 2 at cumulative fractions between 33% and 40%); higher thresholds would begin including tokens with substantive semantic content that are not plausibly "function" elements. The threshold is a design choice that the paper implicitly validates through the experimental results—the 122 tokens identified behave consistently as "function tokens" across all analyses.

Linguistic validity of the frequency-based classification. The paper acknowledges that this operationalization is an "approximation of the linguistic concepts" (Section 2.2). In linguistics, function words are defined by their grammatical role (articles, prepositions, conjunctions, auxiliary verbs, pronouns, determiners) and their lack of independent semantic reference, while content words (nouns, verbs, adjectives, most adverbs) carry specific meanings. The frequency-based method captures this distinction because function words appear with disproportionately high frequency in natural language—a consequence of Zipf's law and the fact that grammatical scaffolding is reused across all documents regardless of topic. As shown in Figure 3c, document coverage (the fraction of documents containing a token) correlates strongly with token frequency: high-frequency tokens appear in nearly all documents, while low-frequency tokens appear in only a few. This means the frequency-based classification also captures a distributional property: function tokens are uniformly present across the corpus, while content tokens are bursty and topic-specific.

The full list of 122 function tokens is provided in Appendix D (Table 2) and includes: punctuation marks (, . : ; ? " ' - – — ( ) { } / \), whitespace and formatting tokens (__ (space-prefixed whitespace) \n .\n ____ ___), articles (_the _a _an—note the underscore prefix indicating a leading space in the LLaMA tokenizer), prepositions (_of _in _to _for _on _with _at _by _from _into _about _than), conjunctions (_and _or _but _so _if _when _that _which _who), auxiliary and copula verbs (_is _are _was _were _be _been _have _has _had _will _would _can _do), pronouns (_it _I _you _we _he _she _they _his _her _its _their _my _your _our _them _this _that _these _those), common adverbs and quantifiers (_not _also _more _some _all _just _only _very _so _there _here), the possessive clitic ('s 't '), digits 0-9 and common year-like tokens (200, 201), the dollar sign (_$), equals sign (_=), ampersand (_&), and common short words with predominantly structural functions. The underscore notation (e.g., _the) is an artifact of the LLaMA tokenizer, which prepends a space to tokens that begin a new word.

Statistical properties of the classification. As shown in Figure 2 and Table 2:

The most frequent single token is the comma (,), accounting for 3.60% of all tokens and appearing in 95.00% of documents.
The top 10 tokens (comma, space-prefixed "the", period, space-prefixed "and", space-prefixed "of", space-prefixed "to", space-only token, space-prefixed "a", space-prefixed "in", newline-following period) cumulatively account for 19.39% of all tokens.
The top 122 tokens (the full function token set) account for 40% of all token occurrences.
Document coverage for function tokens is uniformly high: most function tokens appear in 60-95% of documents. In contrast, content tokens show bursty distributions—a content token like "Tokyo" appears in very few documents.

Design justification. The frequency-based classification is chosen over manual linguistic annotation for three reasons: (1) reproducibility—any researcher can replicate it given a corpus and tokenizer; (2) scalability—it can be applied to any domain or language without expert linguistic knowledge; (3) corpus-specificity—the 122 tokens are tuned to SlimPajama's specific distribution. The tradeoff is that some linguistically function words may fall below the 40% threshold (e.g., infrequent prepositions or conjunctions), and some linguistically content words may sneak in if they happen to be extremely common in the corpus. The paper implicitly accepts these edge cases as noise that does not affect the aggregate patterns.

Bipartite Graph Construction for Memory Retrieval Analysis

The paper's core empirical claim about memory retrieval is that function tokens activate a disproportionate fraction of the model's features. To quantify this, the authors construct a token-feature bipartite graph—a graph where one set of nodes represents token types from the vocabulary, the other set represents features from SAE decomposition, and an edge exists between a token and a feature if that token ever activates that feature in any inference context.

Step 1: Activation extraction. The authors feed 10,000 randomly sampled raw documents from the SlimPajama validation set into Gemma2-9B (the base model, not instruction-tuned). These documents contain approximately 5 million tokens in total. During the forward pass, the authors extract the residual stream activations after the add-norm operation at three representative layers: layer 9 (shallow), layer 20 (middle), and layer 31 (deep, near the final output). The residual stream at each layer has dimension $d = 3584$ for Gemma2-9B. For each token position in each document, there is one activation vector $\mathbf{y} \in \mathbb{R}^{3584}$ per layer.

Why residual stream activations? The residual stream is the central "information highway" of the Transformer (Elhage et al., 2021). Each layer reads from the residual stream, performs its attention and FFN computations, and writes back to the residual stream. The residual stream at layer $\ell$ therefore contains the accumulated contributions of all previous layers, making it the natural location to observe what information is present at a given depth. Extracting from the residual stream after the add-norm operation (rather than after attention or FFN separately) captures the layer's complete output. The choice of layers 9, 20, and 31 spans the model's depth to capture whether the function token effect varies across early, middle, and late processing stages.

Step 2: Feature decomposition via Sparse Autoencoder. For each activation vector $\mathbf{y} \in \mathbb{R}^d$ , the authors apply the corresponding Gemma Scope SAE for that layer. Gemma Scope (Lieberum et al., 2024) provides open-source SAEs trained on Gemma2 model activations at multiple dictionary widths. The authors select the SAE with dictionary width $2^{20} = 1,048,576$ (the largest available), which is approximately 293× the residual stream dimension. This large dictionary width enables decomposition into a high number of potentially interpretable features.

The SAE decomposes the activation into a sparse linear combination of feature vectors:

$\mathbf{y} = \sum_{i=1}^{n} c_i \mathbf{f}_i = c_1 \mathbf{f}_1 + c_2 \mathbf{f}_2 + \ldots + c_n \mathbf{f}_n$

where $\mathbf{y} \in \mathbb{R}^d$ is the activation vector being decomposed, $n = 2^{20}$ is the dictionary size (total number of possible features), $\mathbf{f}_i \in \mathbb{R}^d$ is the $i$ -th feature vector (a direction in activation space, corresponding to a decoder column), and $c_i \in \mathbb{R}$ is the activation coefficient for feature $i$ (how strongly this feature is "firing" for the current input).

What it computes: The SAE takes a dense activation vector $\mathbf{y}$ (where most of the 3584 dimensions have non-zero values, but the individual dimensions are polysemantic—each neuron participates in representing multiple concepts) and represents it as a weighted sum of a much larger number of feature vectors $\mathbf{f}_i$ , with the constraint that most coefficients $c_i$ are zero or near-zero (sparsity). The active features (those with $c_i \neq 0$ ) are interpreted as the concepts or patterns present at this token position. The reconstruction $\hat{\mathbf{y}} = \sum_i c_i \mathbf{f}_i$ approximates the original activation, with the reconstruction error $\|\mathbf{y} - \hat{\mathbf{y}}\|_2^2$ measuring what information was lost in the sparse decomposition.

Why this form: Sparse linear decomposition is motivated by the superposition hypothesis (Elhage et al., 2022): neural networks represent more features than they have dimensions by encoding features in overlapping, nearly-orthogonal directions. The SAE's overcomplete dictionary (far more features than dimensions) plus sparsity constraint allows it to "untangle" these superimposed features. The linearity ensures that feature activations are additive—the total activation is the sum of individual feature contributions—which makes the interpretation straightforward: if features A and B are both active, the model's representation is the combination of concepts A and B. Alternative non-linear decompositions would sacrifice this additivity and make it harder to interpret what "activating feature i" means in isolation.

Implementation detail. The paper does not specify the exact Gemma Scope SAE variant used (e.g., the training objective, whether it uses TopK, JumpReLU, or Gated activation functions). Gemma Scope provides multiple SAE variants per layer. The choice of the widest dictionary (2^20) suggests prioritization of feature granularity over reconstruction speed.

Activation rates. After decomposition across all 5 million tokens, the number of features that are activated at least once (connected to at least one token in the bipartite graph) is 965,635 for layer 9, 947,341 for layer 20, and 919,220 for layer 31. With a dictionary size of 1,048,576, this yields activation rates of 92.1%, 90.3%, and 87.7% respectively—confirming that the vast majority of available features are used somewhere during inference on natural text.

Step 3: Bipartite graph construction. For each (token type, feature ID) pair, an edge is added to the graph if there exists at least one occurrence in the 5 million token sample where that token type activated that feature. The edge is binary (present/absent), regardless of how many times the activation occurred. The resulting graph has:

Token nodes: one per token type in the vocabulary (the LLaMA-3.1 tokenizer vocabulary size, which for Gemma2 corresponds to 256,000 tokens—though the paper only analyzes the 122 function tokens and a subset of content tokens).
Feature nodes: one per feature that was activated at least once (965,635 / 947,341 / 919,220 for the three layers).
Edges: an edge connects token $t$ to feature $i$ if $\exists$ at least one token position in the 5M sample where token type $t$ activates feature $i$ .

The degree of a token is defined as the number of feature nodes it connects to—i.e., the number of distinct features that this token type ever activates across all inference contexts. This is the central metric for Section 3.1's claim that "a few function tokens activate most features."

Why binary edges (presence/absence) rather than weighted edges (count/frequency)? The paper does not explicitly justify this, but the rationale is clear from the research question: the goal is to measure the coverage of the feature space by each token type—how many of the model's total available features can be accessed through a given token. A weighted graph measuring activation frequency would answer a different question (which tokens are most often involved in feature activation), which could be dominated by token frequency effects. The binary graph captures the structural property: does this token type ever serve as a "key" that unlocks this feature?

Key result from bipartite analysis (Figure 5, Table 1). Plotting token degree against token rank (sorted by frequency) on a log-log scale reveals that degree drops off sharply as rank increases—the most frequent tokens connect to far more features than less frequent tokens. Table 1 quantifies the cumulative feature coverage of the top-10 most frequent tokens:

At layer 9: the top-10 tokens (period, comma, "the", newline, "and", "to", "of", whitespace, "a", "in") cumulatively cover 48.52% of all features.
At layer 20: the same top-10 tokens cover 76.46% of features—the peak coverage layer.
At layer 31: coverage drops to 68.27%.

The greater coverage at middle layers (layer 20) compared to shallow layers (layer 9) is consistent with prior work finding middle layers to be the most "expressive" and interpretable (Chen et al., 2025; Soligo et al., 2025; Panickssery et al., 2024). The slight drop at deep layers (layer 31) may reflect increasing specialization—deep layers are closer to the output and may activate features specific to the immediate next-token prediction task rather than broad context integration.

Why this supports the function token hypothesis (memory retrieval). The 122 function tokens constitute a tiny fraction of the 256,000-token vocabulary (~0.05%) but collectively connect to the vast majority of features (the top-10 alone cover >70% at layer 20). This means that during inference, the model primarily accesses its stored knowledge (features) through function tokens. When a function token arrives at a given layer, it triggers the activation of many features—presumably those most relevant to predicting what comes next based on the preceding context. Content tokens, by contrast, activate far fewer features on average, suggesting they play a more localized role—activating features directly associated with their semantic content rather than broadly retrieving context-relevant information.

Steering Experiments: Causal Evidence for Feature Reactivation

The bipartite graph analysis shows correlation: function tokens activate many features. But it does not show causation—it could be that function tokens happen to occur at positions where many features are active due to other factors (e.g., sentence boundaries where the model needs to integrate preceding information). The steering experiments in Section 3.2 and Appendix A address this by asking: if we deliberately modify the activations at a function token position, does the model's output change in predictable, semantically coherent ways?

The steering method (Appendix A). This is a four-step pipeline:

Step 1: Collect contrastive prompts. For each target trait (e.g., "Speak Chinese," "Russia," "UK"), the authors construct:

One trait-enforcing prompt: a prompt that explicitly instructs the model to exhibit the trait. For "Speak Chinese," this is "Answer the question in Chinese: What is the capital of UK?"—this explicit instruction forces the model to respond in Chinese.
Twenty non-trait prompts (the "test set"): prompts that do not specify the trait and would normally elicit a default response. For "Speak Chinese," an example is "Where is Mount Fuji?"—normally answered in English with "Japan."

Why these specific numbers? The paper does not justify the choice of one trait-enforcing prompt versus twenty test prompts. The asymmetry likely reflects the fact that the trait-enforcing prompt is used only to identify the relevant feature (a one-time operation), while the test prompts are used to evaluate steering effectiveness across diverse contexts (requiring a larger sample for reliability). The trait-enforcing prompt's explicit instruction ensures the feature is strongly activated during the identification step.

Step 2: Identify the most informative layer. For each layer $\ell$ in the model, the authors:

Extract the activation vector $\mathbf{h}_\ell \in \mathbb{R}^d$ at the position of the final function token in the prompt (typically the \n newline token just before the model begins its response) for the trait-enforcing prompt.
Denote this activation as the steer vector $\mathbf{v}_\ell \in \mathbb{R}^d$ . This captures the "direction" in activation space corresponding to the trait at layer $\ell$ .
For each test prompt, modify the activation at its final function token: $\mathbf{h}_\ell \leftarrow \mathbf{h}_\ell + \mathbf{v}_\ell$ —adding the steer vector from the trait-enforcing prompt to the test prompt's activation.
Generate the model's response and measure whether it exhibits the trait (e.g., responds in Chinese).
Select the layer $\ell^*$ with the highest success rate.

For the three traits examined ("Speak Chinese," "Russia," "UK"), the most informative layer is layer 26 for all three. This is consistent with the steering literature finding that middle-to-late layers are most effective for behavioral steering (Panickssery et al., 2024; Templeton et al., 2024).

Step 3: Identify the specific feature(s). At the selected layer $\ell^*$ , the steer vector $\mathbf{v}_{\ell^*}$ is decomposed using the Gemma Scope SAE:

$\mathbf{v}_{\ell^*} = \mathbf{W}_{\text{dec}} \cdot \mathbf{z} + \mathbf{b}_{\text{dec}}$

where $\mathbf{W}_{\text{dec}} \in \mathbb{R}^{d \times n}$ is the SAE decoder matrix (mapping from feature coefficients back to activation space), $\mathbf{z} = (z_1, z_2, \ldots, z_n)^\top \in \mathbb{R}^n$ is the vector of feature coefficients, $\mathbf{b}_{\text{dec}} \in \mathbb{R}^d$ is the decoder bias, and $n = 2^{20}$ is the number of features.

What this equation computes: The SAE decoder takes the sparse feature coefficient vector $\mathbf{z}$ (where most entries are zero) and reconstructs the steer vector $\mathbf{v}_{\ell^*}$ as a linear combination of feature directions plus a bias term. The coefficient $z_i$ represents how much feature $i$ contributes to the steer vector—larger absolute values indicate the feature is more strongly associated with the trait.

To identify which specific features encode the trait, the authors:

Rank features by their activation strength $z_i$ in descending order.
Perform a binary search to find the smallest set of top- $k$ features such that activating only these $k$ features (setting $z_j = 0$ for $j \notin \text{top-}k$ ) is sufficient to induce the trait on the test prompts, while using the top- $(k-1)$ features fails.

The steering vector using only the top- $k$ features is:

$\mathbf{v}_{\ell^*}^{S_k} = \alpha \cdot \mathbf{W}_{\text{dec}} \sum_{i \in S_k} \mathbf{e}_i$

where $S_k$ is the set of top- $k$ feature IDs, $\mathbf{e}_i \in \mathbb{R}^n$ is the $i$ -th standard basis vector (a one-hot vector with 1 at position $i$ and 0 elsewhere), and $\alpha \in \mathbb{R}$ is a steering strength multiplier.

What this computes: The sum $\sum_{i \in S_k} \mathbf{e}_i$ creates a coefficient vector where the top- $k$ features have coefficient 1 and all others have coefficient 0. Multiplying by $\mathbf{W}_{\text{dec}}$ maps this to activation space—it constructs a vector that is the sum of the decoder directions for the selected features. Multiplying by $\alpha$ scales the strength of the intervention.

For the three traits, the top-1 feature is sufficient (i.e., $k = 1$ ):

"Speak Chinese" → Feature ID 15261 at layer 26
"Russia" → Feature ID 9591 at layer 26
"UK" → Feature ID 13751 at layer 26

Step 4: Steering at inference time. Once a feature $i$ is identified, steering any prompt is achieved by:

$\mathbf{v}_\ell^i = \alpha_i \cdot \mathbf{W}_{\text{dec}} \mathbf{e}_i$

and applying $\mathbf{h}_\ell \leftarrow \mathbf{h}_\ell + \mathbf{v}_\ell^i$ to the final function token position. The steering strength $\alpha_i$ can be tuned per feature to control the intensity of the effect.

Key results from steering (Figure 7, Appendix B). The experimental results demonstrate several properties critical to the function token hypothesis:

Steering exclusively at the final function token is sufficient to change outputs. For the prompt "Where is Mount Fuji?", the model normally answers "Japan." Activating the "Russia" feature at the final \n token changes the answer to "Russia." Activating "Speak Chinese" changes it to "日本" (Japan in Chinese). Jointly activating "Speak Chinese" and "UK" changes it to "英国" (UK in Chinese). This demonstrates that the function token's activation is causally responsible for selecting which context features influence generation.
Features encode high-level semantic concepts, not surface-level token outputs. Activating the "Russia" feature on "Tell me a university" produces "Moscow State University" (not the token "Russia"), and on "Could you recommend a tourist attraction?" produces "Alexandrinsky Theatre" (a Russian theater). The feature steers the model toward Russia-related content in a contextually appropriate way, showing it encodes the concept "Russia" rather than a simple token-level trigger. This is consistent with the SAE literature's finding that features represent interpretable concepts (Templeton et al., 2024).
Combining features produces compositional effects. Activating both "Speak Chinese" and "Russia" yields Russian content in Chinese (e.g., "俄罗斯" for "Russia," "莫斯科国立大学" for "Moscow State University"), demonstrating that features can be composed independently at the function token position.

Why this supports the causal claim. If function tokens merely correlated with feature activation (e.g., because they happen to appear at positions where the model needs to integrate context), then artificially modifying the activation at a function token position would not produce coherent, context-appropriate changes in output—the activation would be "overwritten" or ignored by subsequent processing. The fact that a targeted modification at a single function token position reliably alters output behavior in semantically meaningful ways implies that the model reads the activation state at function tokens and uses it to determine which context features to carry forward into generation. This is the core of the function token hypothesis's memory retrieval mechanism: function tokens serve as the locus where context-predictive features are reactivated and propagated to influence the next token prediction.

Layer specificity additional case study (Figure 11, Appendix B). The paper presents further examples—naming a baby daughter, recommending beverages, recommending dishes—all showing the same pattern: steering at the final function token consistently produces coherent, context-appropriate trait-aligned responses. The diversity of prompts and traits supports the generality of the mechanism.

Pre-Training Loss Disaggregation: Tracking What Drives Learning

The retrieval analysis shows that function tokens activate many features during inference. The consolidation analysis asks: how did those features get there? The answer, the paper argues, lies in the optimization pressure of next-token prediction, specifically the function→content prediction task. To test this, the authors train models from scratch and track the loss separately for four token transition types.

Pre-training configuration (Section 4.1). Two models are trained:

8B model: Full LLaMA-3.1-8B architecture with 32 Transformer layers, trained on 128 GPUs with 80GB memory each.
1.5B model: Same architecture but with only 2 layers (keeping all other components—attention heads, FFN dimensions, etc.—unchanged relative to the LLaMA-3.1-8B design). This reduced model enables SAE training at multiple checkpoints within computational constraints.

Both models are trained on the full SlimPajama-627B corpus (627 billion tokens) for one complete epoch using the LLaMA-3.1-8B training recipe: batch size 1024 sequences, maximum sequence length 4095 tokens, AdamW optimizer (specific betas not provided, but following LLaMA-3.1 defaults: $\beta_1 = 0.9$ , $\beta_2 = 0.95$ ), learning rate linearly warmed up for 8,000 steps to $8 \times 10^{-5}$ , then cosine decay to $8 \times 10^{-7}$ . With a batch size of 1024 and sequence length of 4095, each training step processes $1024 \times 4095 = 4,193,280$ tokens. One epoch over 627B tokens requires approximately $627 \times 10^9 / 4.19 \times 10^6 \approx 149,500$ steps. The 130,000-step checkpoint used for SAE analysis corresponds to roughly 87% of one epoch (not 94% as stated in Figure 10—this discrepancy suggests either the actual training steps differ or the epoch count is approximate; the paper reports "130000 steps (94%)" in Figure 10 but this would imply only ~138k total steps).

Why train from scratch rather than analyzing existing checkpoints? Using publicly available LLaMA-3.1 checkpoints would be cheaper, but the paper needs (a) access to activations at intermediate training stages for SAE training, and (b) fine-grained loss tracking by token type, which requires instrumenting the training loop. Training from scratch gives full control over both.

Loss disaggregation into four categories (Section 4.3). For each next-token prediction $p(\text{next\_token} \mid \text{current\_token}, \text{context})$ , the authors classify the pair into one of four categories based on whether the current token and the next token are function or content tokens:

function→function: Both current and next are function tokens.
function→content: Current is function, next is content.
content→function: Current is content, next is function.
content→content: Both are content tokens.

The loss for each category is the standard cross-entropy loss computed only over token positions belonging to that category. The losses are tracked separately throughout training and plotted in Figure 9 for both the 1.5B and 8B models.

What the loss curves reveal (Figure 9). Several key patterns emerge:

1. Function→content has the highest loss throughout training. At convergence (step 130,000 for 1.5B, step ~130,000 for 8B), the losses are approximately:

1.5B model: function→content ≈ 4.88, content→content ≈ 3.69, function→function ≈ 2.12, content→function ≈ 1.90.
8B model: function→content ≈ 4.27, content→content ≈ 3.08, function→function ≈ 1.87, content→function ≈ 1.64.

The function→content loss is roughly 2.3-2.6× the function→function loss across both model scales. This means predicting a content token after a function token is fundamentally harder than any other transition type—the model is most uncertain about what content word should follow a structural element.

Why this drives memory consolidation. The cross-entropy loss is optimized via stochastic gradient descent. The gradient magnitude for each token position is proportional to the model's prediction error (the difference between predicted probabilities and the one-hot ground truth). Since function→content positions have the largest prediction errors, they contribute the largest gradient updates. This means parameter updates during training are dominated by learning to predict content tokens that follow function tokens. The features that help make this prediction—features that encode the semantic content needed to choose the right next word based on preceding context—are therefore the ones most strongly reinforced during training.

2. Function token prediction is learned earliest and converges fastest. Figure 9c plots the loss for predicting three specific function tokens as next tokens ("the," "of," and ",") alongside the average loss across all tokens. These individual function token losses drop extremely rapidly—converging within the first 3,000 training steps—long before the average loss stabilizes. The function→function and content→function categories in Figure 9a-b also show faster initial loss reduction than the content-predicting categories.

What this means for the developmental trajectory. The model first learns the syntactic skeleton of language—where to place articles, prepositions, punctuation, and other structural elements. This is a relatively easy task because function token prediction depends primarily on local syntactic patterns (e.g., after "the" expect a noun; after a comma expect a clause continuation) rather than long-range semantic understanding. Once the structural scaffold is in place, the model can use it as a foundation for the harder task: predicting which specific content word fills each slot. This is the function→content task, which requires integrating potentially long-range contextual information.

3. Scaling model size disproportionately helps content prediction. Comparing the 1.5B and 8B models at the same training step:

content→function loss: 1.90 → 1.64 (∆ = 0.26)
function→function loss: 2.12 → 1.87 (∆ = 0.25)
function→content loss: 4.88 → 4.27 (∆ = 0.61)
content→content loss: 3.69 → 3.08 (∆ = 0.61)

The losses for categories where the next token is a content token (function→content and content→content) drop by 0.61, more than twice the reduction for categories where the next token is a function token (~0.25). This means larger models allocate their additional capacity primarily toward improving the hard subtask—predicting content—rather than further optimizing the already-easy function token prediction.

Qualitative example of the learning trajectory (Figure 10). The paper provides a concrete illustration using the prompt "When young children are learning to read, they often struggle with complicated words":

Step 100 (0.7% of training): The model's predictions are essentially random—garbled tokens like "sharing" and "SUCCESS."
Step 3,000 (2.3%): The model predicts only function tokens—"a," "to," "the," "and," "are," ","—showing it has learned the syntactic skeleton but not the semantic content.
Step 50,000 (38%): The model produces locally coherent phrases like "learning to" and "to be," but still makes errors on content words (e.g., predicting "find" where "read" is correct).
Step 130,000 (94% for 1.5B model): The 1.5B model still shows some errors (e.g., "have to" instead of "struggle with").
Step 130,000 for the 8B model: The larger model correctly predicts "struggle with" after "they often"—capturing the long-range dependency between "struggle" and the earlier context "learning to read."

This example concretely demonstrates the developmental sequence: function token prediction emerges first, followed by local content prediction, and finally long-range content prediction that requires integrating distant context. The hardest prediction—"struggle with"—is a function→content transition (current token "often" is a function token; "struggle" is content). Mastering this transition requires the model to reactivate the predictive features from context (the concept of "learning to read" implying difficulty), which is precisely the memory retrieval mechanism the function token hypothesis describes.

Feature Emergence Tracking via SAEs on Pre-Training Checkpoints

The loss analysis shows that learning is driven by function→content prediction. The feature emergence analysis shows what that learning looks like internally: the number of distinct features the model can represent grows over training, and function tokens consistently provide access to most of them.

SAE training on pre-training checkpoints (Section 4.2, Appendix C). Due to computational constraints, SAEs are trained only on the 1.5B model (2 layers) and only on second-layer activations (the output of the final layer before the unembedding). Three checkpoints are selected:

3,000 steps (early / ~2.3% of training): The model has just learned function token prediction.
50,000 steps (intermediate / ~38% of training): Content prediction is developing.
130,000 steps (late / ~94% of training): Near convergence.

Why only the second layer? The 1.5B model has only 2 layers, and the second layer's residual stream is the final representation before token prediction—it captures the accumulated computation of the entire (shallow) model. For the 8B model or Gemma2-9B, analyzing multiple layers would be informative, but for a 2-layer model, the second layer is the natural single point of analysis.

SAE architecture and training details. The SAE used is a JumpReLU-SAE (Rajamanoharan et al., 2024) with a tanh penalty function, implemented using the sae_lens library (Bloom et al., 2024). The JumpReLU-SAE formulation is:

$\mathbf{z} = \text{JumpReLU}_{\theta}(\mathbf{W}_{\text{enc}} \mathbf{x} + \mathbf{b}_{\text{enc}})$

$\hat{\mathbf{x}} = \mathbf{W}_{\text{dec}} \mathbf{z} + \mathbf{b}_{\text{dec}}$

where $\mathbf{x} \in \mathbb{R}^d$ is the input activation (from the residual stream, dimension $d$ ), $\mathbf{W}_{\text{enc}} \in \mathbb{R}^{n \times d}$ is the encoder matrix, $\mathbf{b}_{\text{enc}} \in \mathbb{R}^n$ is the encoder bias, $\mathbf{W}_{\text{dec}} \in \mathbb{R}^{d \times n}$ is the decoder matrix, $\mathbf{b}_{\text{dec}} \in \mathbb{R}^d$ is the decoder bias, $n = 65,536$ is the dictionary size (16× the activation dimension), and $\text{JumpReLU}_{\theta}$ is a thresholded ReLU activation: $\text{JumpReLU}_{\theta}(u) = u \cdot \mathbb{1}[u > \theta]$ where $\theta$ is a learned per-feature threshold.

Training objective. The SAE is trained to minimize:

$\mathcal{L}(\mathbf{x}) = \underbrace{\|\mathbf{x} - \hat{\mathbf{x}}\|_2^2}_{\mathcal{L}_{\text{reconstruct}}} + \underbrace{\lambda \|\mathbf{z}\|_0}_{\mathcal{L}_{\text{sparsity}}}$

where $\|\mathbf{x} - \hat{\mathbf{x}}\|_2^2$ is the squared L2 reconstruction error (how faithfully the sparse decomposition reproduces the original activation), $\|\mathbf{z}\|_0$ is the L0 "norm" (the number of non-zero feature coefficients—i.e., how many features are active), and $\lambda \in \mathbb{R}^+$ is a hyperparameter trading off reconstruction accuracy against sparsity.

What it computes: The first term $\mathcal{L}_{\text{reconstruct}}$ penalises the SAE for losing information during the encode-decode cycle—if important aspects of $\mathbf{x}$ cannot be represented as a sparse combination of features, reconstruction will be poor. The second term $\mathcal{L}_{\text{sparsity}}$ penalises using too many features at once—this forces the SAE to choose only the most essential features for each activation, promoting interpretability (each active feature should carry significant explanatory power).

Why this form: The L2 reconstruction loss assumes Gaussian noise in the activation space and encourages the SAE to represent the dominant directions of variation. The L0 sparsity penalty directly targets the number of active features, which is the natural measure of sparsity for dictionary learning—it encourages the SAE to explain each activation with the smallest possible number of features. Alternatives like L1 regularization (encouraging small but non-zero coefficients) would not enforce the hard sparsity needed for interpretable feature discovery, as many weakly active features could collectively reconstruct the input without any single feature being clearly interpretable. JumpReLU provides an architectural sparsity constraint (features below threshold are zeroed) rather than purely loss-based sparsity, which has been found to produce better reconstruction-sparsity tradeoffs (Rajamanoharan et al., 2024).

Training hyperparameters (Appendix C):

Training data: 1 billion activations collected from the pre-training dataset (FineWeb; Penedo et al., 2024), with context size 1024 tokens.
Dictionary width: 16× the activation dimension → for the 1.5B model's residual stream dimension (not explicitly stated, but LLaMA-3.1-8B has $d = 4096$ , so a 1.5B model with reduced layers likely has the same or similar hidden size; the dictionary is $16 \times 4096 = 65,536$ if the dimension matches).
Batch size: 4,096.
Learning rate: constant $1 \times 10^{-5}$ .
JumpReLU parameters: bandwidth = 0.02, initialization threshold = 0.01.
Dead feature detection window: 1,000 training steps (features that never activate for 1,000 consecutive steps are considered "dead" and may be reinitialized).
Default L0 penalty coefficient: $\lambda = 4$ .

Equalizing reconstruction quality across checkpoints. A critical methodological challenge: the activations at different pre-training stages have different statistical properties. Early-stage activations are noisier and less structured; late-stage activations are more organized. This means the same $\lambda$ value will produce different reconstruction-sparsity tradeoffs at different checkpoints. To make feature counts comparable, the authors tune $\lambda$ to achieve similar reconstruction quality across checkpoints, as measured by the cross-entropy reconstruction score:

$\text{CE Score} = \frac{H^* - H_0}{H_{\text{orig}} - H_0}$

where $H_{\text{orig}}$ is the cross-entropy loss of the original model for next-token prediction (using the unmodified activation $\mathbf{x}$ ), $H^*$ is the cross-entropy loss when $\mathbf{x}$ is replaced with its SAE reconstruction $\hat{\mathbf{x}}$ during the forward pass (all downstream computation proceeds normally with the reconstructed activation), and $H_0$ is the cross-entropy loss when $\mathbf{x}$ is zero-ablated (set to the zero vector).

What this metric computes: A CE Score of 1.0 means perfect reconstruction—the SAE output produces identical next-token predictions as the original model. A score of 0.0 means the reconstruction is no better than zeroing out the activation entirely. The score measures how much of the activation's causal effect on the model's output is preserved by the SAE. This is a more behaviorally meaningful metric than raw L2 reconstruction loss, because it captures whether the information the model actually uses for prediction is preserved.

The tuned $\lambda$ values are:

Early checkpoint (3,000 steps): $\lambda = 10$ (stronger sparsity penalty needed because activations are noisier and require more aggressive feature selection).
Intermediate checkpoint (50,000 steps): $\lambda = 4$ (the default).
Late checkpoint (130,000 steps): $\lambda = 2.5$ (weaker sparsity penalty because activations are more structured and can support denser feature representations while maintaining interpretability).

Key observation from SAE training (Figure 12): The paper notes that "as pre-training progresses, the model's feature representations become increasingly complex and more difficult to decompose." This is evidenced by the need to reduce $\lambda$ for later checkpoints—the optimal sparsity-reconstruction tradeoff shifts toward allowing more active features per activation, suggesting that the model develops richer, more compositional representations that require more features to reconstruct accurately.

Feature counting and growth (Figure 8a). At each checkpoint, the trained SAE is used to decompose activations from a sample of SlimPajama text (500,000 activations). The total number of unique features that are active at least once is counted. The results show substantial growth:

3,000 steps: a baseline number of features (exact value not provided in the main text, but Figure 8a shows a bar chart with clear growth).
50,000 steps: more features than at 3,000 steps.
130,000 steps: the largest number of features.

This demonstrates that memory consolidation corresponds to feature expansion—the model learns to represent more distinct concepts (features) as training progresses.

Token-feature degree evolution (Figure 8b). The bipartite graph analysis from Section 3.2 is repeated for each checkpoint, plotting token degree against token rank by frequency. The key finding: "function tokens consistently activate most features, in contrast with content tokens. This disparity widens over time, as evidenced by the gradually steepening slopes in the graph."

Why this supports memory consolidation via function→content prediction. The function→content prediction task is dominant throughout training (it has the highest loss and largest gradients). Each time the model encounters a function→content transition, it must predict a specific content token based on preceding context. This prediction requires activating features that encode the relevant contextual information—the semantic content needed to choose the right word. Features that are useful for this task are reinforced through gradient updates; features that are not useful decay. Over millions of function→content predictions across diverse contexts, the model accumulates a large set of features that can be activated from function token positions to guide content prediction. This explains why function tokens connect to most features: they are the positions where features are most frequently needed and therefore most strongly learned.

Why content tokens show lower feature connectivity. Content→content transitions (e.g., predicting the second half of a compound noun phrase) are less frequent and typically more local—the prediction depends on the immediately preceding content word rather than broad context integration. Content→function transitions are easy (the model predicts a structural element, which has low entropy). Only function→content transitions consistently require activating many context-spanning features. This asymmetry in prediction difficulty, multiplied across billions of training tokens, creates the observed asymmetry in token-feature connectivity.

Summary of Design Choices and Their Justifications

Frequency-based function/content classification over linguistic annotation: Reproducible, scalable, corpus-specific, and captures the distributional property (uniform document coverage) that matters for the hypothesis. The 40% cumulative coverage threshold is an empirical choice that balances including all major structural tokens against excluding semantically rich tokens.
Binary bipartite graph edges (presence/absence) over weighted edges: The research question is about feature space coverage (how many distinct features a token can access), not activation frequency (which would be confounded by token frequency).
JumpReLU-SAE with tanh penalty over TopK or Gated SAEs: The paper cites superior reconstruction-sparsity tradeoffs, following Rajamanoharan et al. (2024).
Cross-entropy reconstruction score for tuning $\lambda$ across checkpoints: Ensures feature counts are comparable by equalizing the behavioral fidelity of the SAE reconstruction rather than raw L2 loss, which is not directly meaningful for comparing activations from different training stages.
Steering exclusively at the final function token position: Isolates the causal role of function tokens—if steering works when applied only to a function token, it demonstrates that function token activations are read by downstream processing, supporting the claim that function tokens are the locus of memory retrieval.
Binary search for minimal feature set: Ensures the identified feature is truly responsible for the trait, not just correlated with it among a larger set.
Two model scales (1.5B and 8B) for loss tracking: The 1.5B model enables SAE analysis; the 8B model verifies that the loss patterns scale. Using the same architecture and training data at both scales controls for confounding factors.
Single epoch training on SlimPajama-627B: Ensures each token is seen exactly once during pre-training, providing a clean signal for tracking learning dynamics without confounding from multiple epochs of data repetition.
SAE on second layer only for the 1.5B model: With only 2 layers, the second layer captures the full model computation; for deeper models, multiple layers would need to be analyzed, but the 1.5B model is a deliberate simplification for tractability.

4. Key Insights and Innovations

Innovation 1: The Function/Content Dichotomy as a Causal Organizing Principle for LLM Memory, Not Just a Linguistic Curiousity

The paper's most fundamental intellectual move is elevating the linguistic distinction between function words and content words from a descriptive taxonomy to a causal mechanism that explains both memory retrieval (inference) and memory consolidation (training). Prior work in mechanistic interpretability has largely treated tokens as interchangeable carriers of information—the prevailing assumption, implicit in circuit analysis (Wang et al., 2023; Elhage et al., 2021) and feature cataloging (Templeton et al., 2024), is that any token can in principle activate any feature depending on context, and that the interesting structure lies in which circuits form rather than which token types serve as circuit entry points. This paper challenges that assumption by demonstrating that function tokens and content tokens play fundamentally asymmetric computational roles that are consistent across layers, model scales, and training stages.

What makes this a conceptual advance rather than a taxonomic observation is the causal evidence. The field already knew that some tokens produce large activations (Sun et al., 2024) and that separator tokens dominate attention (Chen et al., 2025). These were treated as isolated curiosities—"massive activations happen at periods," "attention is concentrated on newlines"—without a unifying explanation for why or what function these serve. The paper's steering experiments (Figure 7) transform these correlations into a causal claim: modifying activations exclusively at a single function token (\n) reliably and coherently alters model outputs, while the same intervention at content token positions would presumably show weaker or less coherent effects (a negative result that is not directly tested but is implied by the asymmetry). This demonstrates that function tokens are not merely locations where features happen to be active; they are the interface through which the model retrieves context-relevant knowledge to guide generation. The function token serves as a "query" into the model's feature memory, and the specific features it activates determine what content will be generated.

The significance extends beyond the specific findings to the methodological template the paper establishes. By operationalizing the function/content distinction through token frequency (Section 2.2), the paper provides a cheap, automatic, and cross-lingual way to partition a model's vocabulary for analysis without manual linguistic annotation. This makes the approach scalable and reproducible in a way that circuit-level analysis (which requires painstaking manual identification of features and attention patterns for each task) is not. Future interpretability work can adopt this partition as a first-pass diagnostic: before diving into circuit analysis, one can now ask whether a phenomenon is function-token-mediated or content-token-mediated, which narrows the search space dramatically.

Innovation 2: Disaggregating the Training Loss by Token-Type Transitions Reveals That Function→Content Prediction Is the Engine of Feature Learning

Prior work on pre-training dynamics has overwhelmingly analyzed aggregate metrics: total loss curves (Kaplan et al., 2020; Hoffmann et al., 2022), benchmark performance trajectories, or the emergence of specific capabilities like in-context learning (Olsson et al., 2022). These aggregate views treat every next-token prediction as contributing equally to learning. The paper's disaggregation of the loss into four transition types (function→function, function→content, content→function, content→content) reveals a qualitative hierarchy of prediction difficulty that is invisible in the aggregate loss: predicting content after function tokens is roughly 2.3–2.6× harder than predicting function after function tokens at both the 1.5B and 8B scales (Section 4.3, Figure 9), and this gap persists throughout training.

This is not merely an incremental refinement of loss analysis—it is a diagnostic discovery with implications for how we understand what pre-training actually accomplishes. The standard narrative about language model pre-training is that the model learns "language" broadly—syntax, semantics, world knowledge, reasoning—through next-token prediction. The disaggregated loss reveals that "learning language" is not a monolithic process. It decomposes into (at least) two distinct phases with different dynamics: function token prediction is mastered almost immediately (within 3,000 steps, Figure 9c), while content token prediction after function tokens remains challenging throughout training and therefore dominates the optimization gradient. This means that the bulk of parameter updates during pre-training are driven by a specific subtask: predicting which semantically rich word should follow a structural element, given the preceding context.

The implications for scaling and training design are substantial. The paper shows that scaling from 1.5B to 8B parameters disproportionately reduces the function→content loss (∆ = 0.61) compared to the function→function loss (∆ = 0.25). This suggests that larger models allocate their additional capacity primarily toward improving the hard subtask—content prediction in structurally cued positions—rather than further optimizing the already-easy structural prediction. A natural corollary, though not tested in the paper, is that training data mixtures or curricula that emphasize function→content transitions (e.g., by weighting or sequencing examples where contentful predictions are cued by function words) might accelerate learning more efficiently than uniform sampling. The paper does not make this claim, but the diagnostic framework it provides makes such hypotheses testable.

Innovation 3: The Function Token Hypothesis Provides a Unified Explanation for Previously Disconnected Empirical Phenomena

Perhaps the paper's most intellectually satisfying contribution is that it reinterprets a scattered set of empirical observations from different research groups as manifestations of a single underlying mechanism. The paper explicitly cites (Section 6):

Massive activations at initial tokens, periods, and newlines (Sun et al., 2024) → these are function tokens retrieving context-spanning features.
Separator token attention dominance (Chen et al., 2025) → function tokens serve as attention sinks because they aggregate information needed for upcoming content prediction.
Pivot tokens in Phi-4 that are "critical for response accuracy" and "often follow function tokens" (Abdin et al., 2024) → these are the content tokens whose prediction after function tokens drives feature learning.
High-entropy token training that improves reasoning performance (Wang et al., 2025) → tokens like "thus" and "therefore" are function tokens or tokens in function→content positions where prediction uncertainty is highest, meaning training on them targets the optimization bottleneck.

Prior to this paper, each of these observations existed in its own interpretive frame: "massive activations" were a peculiarity of activation statistics, "separator tokens" were an engineering challenge for KV cache compression, "pivot tokens" were a post-training optimization trick, and "high-entropy training" was a data selection heuristic. The Function Token Hypothesis reveals that these are not separate phenomena requiring separate explanations. They are all consequences of the same underlying mechanism: function tokens are the primary interface for memory retrieval, and the function→content prediction task is the primary driver of memory consolidation.

This constitutes a theoretical unification of a type that is relatively rare in empirical ML research. It does not introduce a new method or a new metric; it provides a conceptual framework that organizes existing observations into a coherent causal story. The value of such unification is that it generates testable predictions beyond the original observations. For example, the hypothesis predicts that any token that behaves like a function token (high frequency, uniform document coverage, structural rather than semantic role) should exhibit massive activations, attention dominance, and steerability, regardless of whether it is linguistically classified as a function word (e.g., the hypothesis would predict that XML tags or special formatting tokens in code should behave similarly). It also predicts that training interventions targeting function→content transitions should be particularly effective, which is testable but not attempted in the paper.

Innovation 4: Feature Expansion During Pre-Training Is Quantified and Linked to Token-Type Dynamics Through SAEs

While prior work using sparse autoencoders has cataloged features in trained models (Templeton et al., 2024; Lieberum et al., 2024), the paper's training of SAEs at multiple pre-training checkpoints and its tracking of feature count growth and token-feature degree evolution represents a methodological advance: it moves SAE-based interpretability from static analysis of finished models to developmental analysis of models in the process of being trained. This is a non-trivial extension because SAE training is computationally expensive and requires careful equalization of reconstruction quality across checkpoints with different activation statistics (Appendix C, Figure 12).

The finding that feature count grows substantially over pre-training and that function tokens consistently activate the majority of features at all stages, with the disparity widening over time (Figure 8), provides the missing link between the loss dynamics (Section 4.3) and the inference-time behavior (Section 3). The loss analysis shows that function→content prediction drives optimization; the feature emergence analysis shows that this optimization pressure translates into an expanding vocabulary of distinct features that are preferentially accessible through function tokens. This closes the loop: the reason function tokens can activate most features during inference (the retrieval observation) is that those features were learned primarily to serve function→content predictions during training (the consolidation mechanism).

The significance of this innovation is partly methodological. Training SAEs at intermediate checkpoints of a pre-training run is computationally intensive—the paper had to train a smaller model (1.5B, 2 layers) specifically to make this feasible—but it opens up a new dimension of interpretability research. Future work could apply this approach to track the emergence of specific capability-related features (e.g., features for arithmetic, factual knowledge, or reasoning) and determine at what point in training they appear, which token types activate them, and how their connectivity evolves. This would transform SAEs from a tool for describing what a model knows into a tool for understanding how it came to know it.

5. Experimental Analysis

Evaluation Methodology

Dataset. All experiments use SlimPajama-627B (Soboleva et al., 2023), a cleaned and deduplicated 627-billion-token web text corpus, as the pre-training corpus. The inference-time bipartite graph analysis and steering experiments use the instruction-tuned Gemma2-9B-it model evaluated on manually constructed prompts (described in Sections 3.2 and Appendix A), while the pre-training experiments train 1.5B and 8B models on the full SlimPajama-627B corpus for one epoch.
Base model(s). For inference-time retrieval analysis, the paper uses Gemma2-9B (the base model) with open-source Gemma Scope SAEs (Lieberum et al., 2024) at layers 9, 20, and 31 with dictionary width 2^20 ≈ 1.05 million features. Steering experiments use Gemma2-9B-it (the instruction-tuned variant). For pre-training consolidation analysis, two models are trained from scratch using the LLaMA-3.1-8B architecture: a full 8B model (32 layers) and a reduced 1.5B model (2 layers, all other components unchanged). The Gemma2 models are chosen because they provide both models of different sizes and open-source SAEs at multiple dictionary widths, enabling the feature decomposition analysis without requiring the authors to train SAEs themselves on a production-scale model. The LLaMA-3.1 architecture is chosen for pre-training experiments because it is a well-documented, reproducible architecture with publicly available training configurations.
Metrics. Five distinct metrics are used across analyses:
- Token degree in the bipartite graph: the number of distinct features a token type ever activates across 10,000 documents (~5 million tokens), measuring feature space coverage per token type (Section 3.1, Figure 5).
- Cumulative feature coverage: the fraction of all features (that were activated at least once) that are connected to the top-k most frequent tokens, measuring concentration of feature access (Section 3.1, Table 1).
- Steering success rate: the fraction of test prompts where adding the steer vector at the final function token produces the target trait (e.g., responding in Chinese), used during feature identification in Appendix A.
- Next-token prediction loss (cross-entropy): disaggregated into four categories (function→function, function→content, content→function, content→content) and tracked continuously during pre-training for both the 1.5B and 8B models (Section 4.3, Figure 9).
- Number of learned features: the total count of unique features active at least once when SAE-decomposing 500,000 activations at a given checkpoint, measuring representational capacity growth during pre-training (Section 4.2, Figure 8a).
- Cross-entropy reconstruction score: (H* − H0)/(Horig − H0), measuring how faithfully the SAE reconstruction preserves the model's next-token prediction behavior. Used to tune the L0 penalty λ across training checkpoints so feature counts are comparable (Appendix C, Figure 12).
Baselines. The paper's core analyses are primarily observational and comparative rather than benchmark-driven, so formal baselines are limited:
- For the bipartite graph analysis, the implicit baseline is content tokens' feature connectivity compared to function tokens—the claim is that function tokens activate disproportionately many features, with content tokens serving as the reference class (Figure 5).
- For steering experiments, the baseline is the unsteered model output on each prompt (listed as "None" in Figure 7 and Appendix B Figure 11).
- For pre-training loss analysis, the four token transition categories are compared against each other and against the average loss across all tokens (Figure 9).
- For feature emergence, the implicit baseline is the early checkpoint (3,000 steps) against which later checkpoints' feature counts are compared (Figure 8a).
Generation budget / compute accounting. No standard "generation budget" metric is used, as the paper analyzes inference behavior and training dynamics rather than test-time compute scaling. For inference analysis, compute is measured in number of documents processed (10,000 documents, ~5 million tokens for bipartite graph) and number of prompt-response pairs (1 trait-enforcing prompt + 20 test prompts per trait for steering). For pre-training, compute is measured in training steps (3,000, 50,000, 130,000 for checkpoint analysis) and total tokens processed (one epoch over 627 billion tokens). SAE training uses a fixed budget of 1 billion activations per checkpoint (Appendix C).
Cross-validation / statistical protocol. The paper does not employ cross-validation in the traditional machine learning sense, as it is not evaluating predictive performance on held-out data. Instead, it employs several methodological controls:
- Activation extraction across 10,000 documents (~5M tokens) for the bipartite graph ensures feature-token edges are based on diverse natural text, not a single prompt or domain (Section 3.1, Step 1).
- Binary search for minimal feature sets during steering identification (Appendix A, Step 3) controls against the possibility that the trait is encoded by a diffuse combination of many features rather than a specific interpretable one.
- Contrastive prompt design: the trait-enforcing prompt identifies the feature, while 20 independent non-trait prompts test steering effectiveness, reducing the risk that the identified feature is prompt-specific rather than trait-specific (Appendix A, Step 1).
- Cross-entropy reconstruction score equalization across checkpoints ensures feature counts are comparable despite different activation statistics at different training stages (Appendix C, Figure 12). Without this, a higher feature count at later checkpoints could be an artifact of easier SAE decomposition rather than genuine feature expansion.
- The paper explicitly notes the absence of traditional statistical significance testing (no confidence intervals, no error bars on most figures). The token degree distributions in Figure 5 and loss curves in Figure 9 represent point estimates from single training runs.

Main Quantitative Results

Feature Coverage by Function Tokens During Inference

The bipartite graph analysis (Figures 5, Table 1) demonstrates that a small number of function tokens activate a disproportionately large fraction of the model's total features. At layer 20 (the middle layer, identified as most expressive), the top-10 most frequent tokens cumulatively cover 76.46% of all features that are active at least once. At layer 9 (shallow), the same top-10 tokens cover 48.52%. At layer 31 (deep), coverage is 68.27%. The top-10 tokens at layer 20 and their individual cumulative contributions are: period (51.32%), comma (62.45%), "the" (66.93%), newline (71.30%), "and" (71.97%), "to" (73.07%), "of" (74.43%), whitespace (75.70%), "a" (76.12%), "in" (76.46%).

The per-token degree distribution (Figure 5, plotted on log-log axes) shows that token degree drops sharply with frequency rank—the curve is steep for the most frequent tokens (function tokens) and flattens out for less frequent tokens (content tokens). The top-ranked token (period at layer 20) connects to 51.32% of features on its own; tokens beyond the top ~100 rank connect to negligible fractions.

A quantitative comparison across layers in Table 1 reveals the middle layer's primacy: the same top-10 tokens cover 76.46% at layer 20 versus 48.52% at layer 9 and 68.27% at layer 31. The paper does not report the absolute number of features activated or the total number of edges for each layer independently, but the activation rates (92.1%, 90.3%, 87.7% of the 2^20 dictionary) confirm that the vast majority of available features are accessed somewhere in the 5-million-token sample.

The paper does not report the baseline: what fraction of features do the top-10 content tokens cover? This would quantify the asymmetry directly. The log-log plot in Figure 5 implies the gap is large, but no explicit number is given for content token coverage.

Steering Experiments: Causal Control Through Function Token Activations

Figure 7 reports the steering results across three prompts and five steering conditions (plus the "None" baseline):

Prompt 3: "Where is Mount Fuji?"
- Baseline (None): "Japan."
- Activate "Speak Chinese" feature: "日本" (Japan in Chinese)
- Activate "UK" feature: "England"
- Activate "Russia" feature: "Russia"
- Activate "Speak Chinese" + "UK": "英国" (UK in Chinese)
- Activate "Speak Chinese" + "Russia": "俄罗斯" (Russia in Chinese)
Prompt 4: "Tell me a university."
- Baseline: "Harvard University"
- Activate "Speak Chinese": "哈佛大学" (Harvard University in Chinese)
- Activate "UK": "Oxford University"
- Activate "Russia": "Moscow State University"
- Activate "Speak Chinese" + "UK": "牛津大学" (Oxford University in Chinese)
- Activate "Speak Chinese" + "Russia": "莫斯科国立大学" (Moscow State University in Chinese)
Prompt 5: "Could you recommend a tourist attraction?"
- Baseline: "The Eiffel Tower."
- Activate "Speak Chinese": "故宫" (The Forbidden City in Chinese)
- Activate "UK": "The Eiffel Tower." (unchanged—UK presumably doesn't override the default)
- Activate "Russia": "Alexandrinsky Theatre"
- Activate "Speak Chinese" + "UK": "伦敦眼" (London Eye in Chinese)
- Activate "Speak Chinese" + "Russia": "叶卡捷琳娜宫" (Catherine Palace in Chinese)

Across all three prompts and five steering conditions (15 total steering interventions), the steered output is semantically coherent with the activated feature and contextually appropriate to the prompt. The "Speak Chinese" feature reliably forces Chinese-language output regardless of the prompt's original language. The "Russia" and "UK" features reliably shift the semantic domain toward country-appropriate entities. Combined steering produces compositional effects: Chinese language + country-appropriate content.

The paper does not report a quantitative success rate (e.g., "the 'Speak Chinese' feature produces Chinese output on 20/20 test prompts"), only qualitative examples from three prompts. The identification procedure in Appendix A claims the most informative layer (layer 26) had the "highest success rate" for each trait, but these success rates are not numerically reported.

Appendix B (Figure 11) provides additional qualitative examples—naming a baby daughter, recommending beverages, recommending dishes—with similar patterns, confirming that the steering effect generalizes across prompt types.

Pre-Training Loss Disaggregation by Token Type Transition

Figure 9 presents the core quantitative results for memory consolidation. For the 1.5B model (Figure 9a) at training step 130,000 (near convergence):

function→function loss: approximately 2.12
function→content loss: approximately 4.88
content→function loss: approximately 1.90
content→content loss: approximately 3.69
Average loss across all tokens: approximately 3.60

The function→content loss exceeds the function→function loss by a factor of approximately 2.30 (4.88 / 2.12). The function→content loss exceeds the average loss by approximately 1.36 (4.88 / 3.60). The content→function loss is the lowest category (1.90), meaning predicting a function token after a content token is easiest.

For the 8B model (Figure 9b) at the same training step:

function→function loss: approximately 1.87
function→content loss: approximately 4.27
content→function loss: approximately 1.64
content→content loss: approximately 3.08
Average loss: approximately 3.10

The function→content loss exceeds the function→function loss by a factor of approximately 2.28 (4.27 / 1.87), virtually identical to the 1.5B ratio.

The scaling comparison between 1.5B and 8B at 130,000 steps:

function→function: 2.12 → 1.87 (Δ = −0.25, 11.8% reduction)
function→content: 4.88 → 4.27 (Δ = −0.61, 12.5% reduction)
content→function: 1.90 → 1.64 (Δ = −0.26, 13.7% reduction)
content→content: 3.69 → 3.08 (Δ = −0.61, 16.5% reduction)

The absolute reductions for content-predicting categories (0.61) are more than double the reductions for function-predicting categories (~0.25). In relative terms, all categories show similar percentage reductions (12–17%), but the larger absolute reduction in already-higher loss categories means scaling disproportionately improves the hardest subtasks in absolute terms.

Figure 9c provides per-token loss trajectories for three specific function tokens as prediction targets:

Predicting "the" (all three tokens shown actually—the paper's mention of "the", "of", and ","): all three drop to near-zero loss within the first 3,000 training steps, well before the average loss stabilizes. This confirms that individual function token prediction is learned extremely early.

The paper does not report equivalent per-token loss curves for content tokens, which would show slower convergence and higher asymptotic loss.

Figure 10 provides a qualitative longitudinal example of next-token predictions at different training stages for the prompt "When young children are learning to read, they often struggle with complicated words":

At step 100 (0.7% of training): Outputs are random gibberish (e.g., "sharing", "SUCCESS").
At step 3,000 (2.3%): The model predicts only function tokens ("a", "to", "the", "and", "are", ","), demonstrating that function token generation is learned first.
At step 50,000 (38%): The model produces locally coherent but semantically imprecise predictions (e.g., "learning to" is correct, but "to be" rather than "to read").
At step 130,000 (94%, 1.5B model): More accurate but still imperfect (e.g., "have to" instead of "struggle with").
At step 130,000 (8B model): The full model correctly predicts "struggle with" for the context "they often ___ with", capturing the long-range dependency on "learning to read" from earlier in the sentence.

The function→content transition at "often → struggle" is precisely the hardest prediction that the loss analysis identifies, and mastering it requires the largest model at the latest training stage.

Feature Expansion During Pre-Training

Figure 8a shows feature count growth across three checkpoints (3,000, 50,000, and 130,000 steps) for the 1.5B model's second layer. The exact feature counts are not numerically reported in the text, but the bar chart shows clear monotonic growth—the 130,000-step checkpoint has substantially more features than the 3,000-step checkpoint. The paper states: "the number of features grows substantially over the progress of pre-training, reflecting the model's increasing representational capability and corresponding to memory consolidation."

Figure 8b shows token-feature degree distributions (degree vs. rank by frequency) for all three checkpoints. The finding: function tokens (left side of the x-axis, high frequency) consistently have much higher degrees than content tokens at all three checkpoints. The gap between function and content token degrees widens from the early to the late checkpoint—the steepness of the curve at the high-frequency end increases. The paper interprets this as: "the disparity widens over time, as evidenced by the gradually steepening slopes in the graph."

The paper notes (but does not numerically report) an important methodological detail: with the default L0 penalty coefficient of λ = 4, reconstruction quality (measured by cross-entropy reconstruction score) varied across checkpoints. The authors tuned λ to equalize reconstruction scores: λ = 10 for the 3,000-step checkpoint, λ = 4 for 50,000 steps, and λ = 2.5 for 130,000 steps (Appendix C, Figure 12). The observation that "as pre-training progresses, the model's feature representations become increasingly complex and more difficult to decompose" is supported by the need for weaker sparsity penalties (lower λ) at later checkpoints to achieve comparable reconstruction fidelity. Without this tuning, raw feature counts would not be comparable—later checkpoints would have artifactually lower counts because the SAE would sacrifice too many features to maintain sparsity.

Ablation Studies and Robustness Checks

Layer selection for bipartite graph analysis: The paper examines three layers (9, 20, 31) rather than analyzing all 42 layers of Gemma2-9B. Table 1 shows that the function token effect is strongest at the middle layer (layer 20 with 76.46% coverage for top-10 tokens) and weaker at shallow (48.52%) and deep layers (68.27%). This is consistent with prior work finding middle layers most interpretable, but the paper does not analyze whether the effect is specific to these layers or generalizes across all layers. No analysis of layers 1-8 or 21-30 is provided, leaving the layer-specificity claim partially supported rather than exhaustively verified.
SAE dictionary width: The paper selects the Gemma Scope SAE with dictionary width 2^20 (the largest available) for the Gemma2-9B analysis, yielding activation rates of 87.7–92.1%. No comparison with smaller dictionary widths (Gemma Scope provides 2^16 and 2^18 variants) is provided to assess whether the function token coverage result is sensitive to dictionary size. A smaller dictionary might show different concentration patterns if features are merged.
λ tuning for cross-checkpoint SAE comparison: The paper identifies that the default λ = 4 produces unequal reconstruction quality across training checkpoints and tunes λ individually (10, 4, 2.5) to equalize cross-entropy reconstruction scores (Appendix C, Figure 12). This is a critical methodological control without which the feature count growth result would be confounded. However, the paper does not report the actual CE reconstruction scores achieved at each checkpoint after tuning, nor how many λ values were swept before selecting these three values. The equalization is approximate rather than exact, and the tolerance is unspecified.
40% cumulative frequency threshold for function token classification: The paper defines function tokens as the top 122 most frequent tokens covering 40% of all token occurrences. No sensitivity analysis of this threshold is provided—how would results change with 30% (fewer tokens, potentially missing some structural tokens like "into" and "about") or 50% (more tokens, potentially including some content-like tokens)? The exact threshold is critical because the claim that "a few function tokens activate most features" depends on which tokens are counted as "function tokens." If the threshold were 50%, the set would include more tokens but might dilute the per-token feature coverage.
Steering feature identification binary search: Appendix A describes using binary search to find the minimal set of top-k features sufficient to induce the trait. The paper reports that for all three traits ("Speak Chinese," "Russia," "UK"), the top-1 feature is sufficient (k = 1). However, no ablation is reported showing that activating a randomly selected feature (or a different high-activation feature not associated with the trait) fails to produce the trait. Without this negative control, the possibility that any strong activation at the function token position produces behavioral shifts (rather than trait-specific features) is not ruled out.
Model size for feature emergence tracking: SAEs are trained only on the 1.5B model (2 layers) due to computational constraints, not the 8B model. The loss patterns (Figure 9) scale similarly from 1.5B to 8B, suggesting the feature emergence pattern might also scale, but this is unverified. The 1.5B model is extremely shallow (2 layers vs. 32 for the 8B model), and feature dynamics in deeper models—where middle layers play a disproportionate role—might differ qualitatively from a 2-layer model where there is essentially no "middle."
Single training run: Both the 1.5B and 8B models are trained once. The paper does not report whether the loss trajectories (Figure 9) or feature counts (Figure 8) are stable across different random seeds or data orders. With only one training run per model size, variance in convergence dynamics is unquantified.
JumpReLU-SAE vs. alternative SAE architectures: The paper uses JumpReLU-SAE with tanh penalty, motivated by prior work finding superior reconstruction-sparsity tradeoffs (Rajamanoharan et al., 2024). No comparison with TopK-SAE or Gated-SAE is provided for the feature counting task. Different SAE architectures might decompose the same activations into different numbers of features, potentially affecting the claimed growth rate and token-feature connectivity patterns.
Document sample size for bipartite graph: The bipartite graph uses 10,000 documents (~5 million tokens) from SlimPajama. The paper does not analyze whether the feature coverage curves (Figure 5) stabilize at this sample size—would more documents reveal additional rare token-feature edges? The binary edge criterion (connected if activated at least once) means the graph can only grow with more data, never shrink. The reported coverage numbers (76.46% for top-10 tokens at layer 20) are therefore lower bounds on the true coverage in the infinite-data limit.
Steering strength α: The paper mentions that steering strength α can be tuned per feature (Appendix A, Step 4) but does not report what values were used for the demonstrations in Figure 7 or how sensitive the results are to α. Too small α would fail to steer; too large α might produce degenerate outputs. The absence of an α sensitivity sweep leaves the robustness of the steering effect unquantified.

Critical Assessment

Claim 1: Function tokens activate the majority of features during inference. This claim is well-supported by the bipartite graph analysis for the three layers examined (9, 20, 31) in Gemma2-9B. At layer 20, the top-10 function tokens cover 76.46% of features (Table 1), and the log-log degree plot (Figure 5) shows a steep drop-off after the highest-frequency tokens. However, several qualifications apply. First, the analysis uses binary edges (feature activated at least once), so the claim measures potential feature access rather than typical feature activation during any single inference pass—a token that connects to 50% of features does not activate all 50% simultaneously; the actual sparsity of SAE activations means only a small fraction fire at any given position. The claim that function tokens "activate" most features is therefore about breadth of access across contexts, not density of activation within a context. Second, the analysis is done on Gemma2-9B only—whether the result generalizes to other model families (LLaMA, GPT, etc.) or scales (7B, 70B, 405B) is untested. The authors implicitly claim generality by stating they "believe this model is representative," but no evidence supports this. Third, the 40% threshold for defining function tokens is arbitrary—changing it would change the exact coverage numbers, though the qualitative pattern of high-frequency tokens dominating feature access is robust given the steep frequency-degree correlation.

Claim 2: Function tokens dynamically reactivate the most predictive features from context. The steering experiments provide supporting evidence but demonstrate a narrower causal mechanism than the claim implies. The experiments show that modifying activations at a single function token (\n) can shift the semantic domain of the output (Figures 7, 11), and that the same function tokens activate different feature combinations in different contexts (Figure 6). These results demonstrate that function token activations causally influence output behavior and that which features are active at function token positions is context-dependent. However, the paper does not directly test the "reactivation" mechanism—that features activated by earlier content tokens are specifically re-activated at later function tokens. Figure 6 visually suggests this (the "China" feature fires at the content token "Chinese," then re-fires at subsequent function tokens), but no quantitative analysis measures the proportion of features that are reactivations (previously active in the context) versus newly activated (emerging at the function token without prior activation). Without this quantification, "reactivation" remains a qualitative description of the visual patterns in Figure 6 rather than a verified mechanism. Additionally, only three features are examined in the reactivation case study (Feature 15261 = "Speak Chinese," Feature 9591 = "Russia," Feature 13751 = "UK")—a sample too small to establish the generality of the reactivation pattern across the ~1 million features in the dictionary.

Claim 3: Feature growth during pre-training is driven by function→content prediction. This claim has two components: (a) feature count grows during pre-training, and (b) the function→content loss dominance drives this growth. Component (a) is directly supported by Figure 8a, which shows increasing feature counts across checkpoints. However, the evidence for component (b) is correlational rather than causal. The paper demonstrates that function→content loss is highest throughout training (Figure 9) and that feature counts increase over the same period (Figure 8a), and argues that the former drives the latter because gradient magnitudes are proportional to prediction error. But no causal intervention is performed—for example, training a variant model where function→content transitions are upweighted or downweighted in the loss and measuring whether feature growth accelerates or decelerates accordingly. Without such an intervention, alternative explanations are not ruled out: function→content loss might be highest simply because content tokens have higher entropy (more possible next tokens to choose among), not because they uniquely require feature expansion; feature growth might be driven by total training compute or data diversity rather than specifically by function→content prediction difficulty. The claim is plausible given the loss data but not experimentally demonstrated.

Claim 4: Memory consolidation corresponds to feature expansion. This is supported by Figure 8a and conceptually aligns with the key-value memory interpretation of FFN layers (Geva et al., 2021). However, the operationalization of "memory consolidation" as "number of SAE features" is a specific choice that may not capture all relevant aspects of memory. Memory consolidation in the neuroscience sense involves strengthening and stabilizing existing representations, not just creating new ones. The paper only measures the creation of new features (count of unique features active), not the strengthening of existing features (e.g., whether the same feature becomes more reliably activated or more influential on output over training). A feature could exist at step 3,000 but be weak and unreliable, then strengthen at step 50,000—this would count as "no growth" in the paper's metric (the feature already existed) but would represent meaningful consolidation. The feature count metric therefore captures only one dimension of consolidation.

Genuine weaknesses in the experimental design:

The 1.5B model for SAE analysis is not a realistic LLM. A 2-layer Transformer with 1.5B parameters is an unusual architecture—vastly wider than typical models of that depth. Standard scaling practice distributes parameters roughly evenly across layers: a 1.5B model would typically have 24-32 layers, not 2. The paper states the 1.5B model keeps "other components unchanged" from the 8B architecture, meaning it has the same hidden dimension and FFN size as an 8B model but only 2 layers. The feature dynamics in this extremely shallow, extremely wide architecture may not reflect dynamics in realistically deep models where middle layers play a disproportionate role. The paper does not discuss this architectural artifact.
The gap between the Gemma2-9B inference analysis and LLaMA-3.1 pre-training analysis. The paper uses different model families, architectures, and tokenizers for its retrieval analysis (Gemma2-9B with Gemma tokenizer) and its consolidation analysis (LLaMA-3.1 architecture with LLaMA tokenizer). The function token lists would differ between these tokenizers, and the SAE features are model-specific. This makes the integration of the two halves of the hypothesis less direct than it appears—the paper demonstrates retrieval behavior in Gemma and consolidation dynamics in LLaMA, but never in the same model.
No quantitative metric for the steering experiments. The steering results (Figures 7, 11) are purely qualitative—example outputs with no success rates, confidence intervals, or failure case analysis. The paper mentions that the "most informative layer" was chosen by highest success rate (Appendix A, Step 2), but if layer 26 achieves, say, 70% success while layer 25 achieves 65%, the difference may not be meaningful. Without quantification, the reliability of steering and the precision of feature identification are unclear.
The "reactivation" analysis in Figure 6 is anecdotal. The activation patterns for Features 15261, 9591, and 13751 across two prompts are shown, but the paper does not perform a systematic analysis of how many features exhibit reactivation patterns, what fraction of function-token-activated features are reactivations versus newly activated, or whether this pattern holds across a diverse sample of prompts and features. The three-feature, two-prompt case study is suggestive but not conclusive.
Figure 10's "94%" claim appears to contain a calculation error. The paper states that step 130,000 corresponds to 94% of training for the 1.5B model. But with batch size 1024 × sequence length 4095 = 4,193,280 tokens per step, 130,000 steps processes ~545 billion tokens. Since SlimPajama-627B contains 627 billion tokens, 130,000 steps covers ~87%, not 94%. The discrepancy is unexplained—perhaps the training used a subset of SlimPajama, or the step count for one epoch is lower than implied by the 627B figure. This makes the reported stage percentages potentially unreliable.

Missing experiments that would strengthen the paper:

A causal intervention on function→content loss. Training a variant where function→content transitions are upweighted 2× or downweighted 0.5× in the loss function would directly test whether this category drives feature expansion. If upweighting accelerates feature growth and downweighting retards it, the causal claim is strengthened. Without this, causality remains correlational.
Content token steering as a negative control. The paper steers only at the final function token. Demonstrating that steering at a content token (e.g., the last content word in the prompt) produces weaker, less reliable, or less coherent behavioral changes would strengthen the claim that function tokens are uniquely responsible for context retrieval.
Quantitative reactivation analysis. Compute the Jaccard similarity between the feature set active at a content token (when a concept is first introduced) and the feature set active at subsequent function tokens, versus the similarity between the content token feature set and subsequent content token feature sets. If function tokens truly "reactivate" features, the function-token similarity should be higher.
Cross-model-family validation. Repeating the bipartite graph analysis on LLaMA-3.1-8B with appropriately trained SAEs would test whether the function token coverage result is specific to Gemma2 or general.
Ablation on the 40% threshold. Show feature coverage curves (like Figure 5) for function token sets defined at 30%, 40%, and 50% cumulative frequency thresholds to demonstrate that the qualitative result is insensitive to this arbitrary cutoff.
SAE analysis at multiple layers of the 8B model. Even if training SAEs on all 32 layers is infeasible, analyzing layers at early, middle, and deep positions of a realistically deep model (even at a single late checkpoint) would verify that the token-feature degree pattern observed in the 2-layer 1.5B model generalizes to deeper architectures.

Where the claims hold conditionally:

The claim that function tokens activate most features is conditional on the SAE decomposition and dictionary width. Different SAE training methods or dictionary sizes could redistribute feature-token connections. The paper uses one specific SAE variant (Gemma Scope, width 2^20, architecture unspecified beyond being a "sparse autoencoder"). If features are defined differently—e.g., using a different decomposition method or a different sparsity target—the coverage numbers would change.
The claim that function→content prediction drives consolidation is conditional on the assumption that gradient magnitude ∝ prediction error drives feature formation, which is a reasonable but unverified assumption. It is possible that features form primarily through other mechanisms (e.g., the distribution of co-occurrence statistics in the data, independent of loss magnitude), and the loss pattern is a symptom rather than a cause.
The feature growth result is conditional on equalizing reconstruction quality across checkpoints. If the λ tuning (Appendix C) is flawed or if cross-entropy reconstruction score is an imperfect equalization metric, the apparent feature growth could be partially artifactual (e.g., later checkpoints have more features because the SAE is tuned to be less sparse, not because the model has genuinely more features). The paper acknowledges this challenge but does not provide sensitivity analysis for the λ tuning.

6. Limitations and Trade-offs

1. Difficulty Estimation Cost Is Unaccounted for in the Efficiency Claims

The assumption or constraint. The paper's core empirical claims—that function tokens activate over 70% of features and that function→content prediction dominates training—all depend on analyses that require significant up-front computation that is not amortized into the findings. The bipartite graph analysis (Section 3.1) requires generating activations on 10,000 documents (~5 million tokens) through a full Gemma2-9B forward pass, then running SAE decomposition at three layers. The feature emergence tracking (Section 4.2) requires training JumpReLU-SAEs at each of three pre-training checkpoints, each consuming 1 billion activations and requiring a full hyperparameter sweep over the L0 penalty λ to equalize reconstruction quality across checkpoints (Appendix C). The steering feature identification (Appendix A) requires repeated model inference across contrastive prompts and binary search over feature sets. The paper does not account for these computational costs in any budget or efficiency metric—the claims about "a small set of function tokens" covering "most features" are derived from a rich but expensive post-hoc analysis pipeline, not from a lightweight diagnostic.

The consequence. A practitioner seeking to verify or apply the Function Token Hypothesis to their own model faces a substantial barrier: they would need to (a) train or acquire SAEs for their model's architecture at multiple layers, (b) run inference on thousands of documents to construct the bipartite graph, (c) tune SAE hyperparameters for any pre-training checkpoints they wish to compare, and (d) perform contrastive prompt engineering for any trait they wish to steer. The paper provides no guidance on how to cheaply approximate any of these analyses—for example, whether a smaller document sample (fewer than 10,000) would yield stable feature coverage curves, or whether a smaller SAE dictionary width (e.g., 2^16 rather than 2^20) would preserve the qualitative pattern. The headline result that "the top 10 tokens cover 76.46% of features at layer 20" (Table 1) is a point estimate from one specific SAE configuration on one model family; whether this number generalizes to LLaMA-3.1-8B with a different SAE architecture is unknown, and replicating the analysis to find out would require non-trivial compute.

What evidence exists in the paper. The paper does not measure the compute cost of any of its analysis pipelines—no FLOP counts, GPU-hours, or wall-clock times are reported for SAE training, activation extraction, bipartite graph construction, or steering identification. The feature emergence analysis on the 1.5B model is explicitly described as computationally constrained ("Due to computational constraints, we perform feature decomposition only on the 1.5B model," Section 4.2), but the magnitude of the constraint is not quantified. The 8B model, which would be more representative of production LLMs, has no SAE analysis at all—the feature emergence results from the 1.5B model are extrapolated with no verification.

Mitigation status. The paper acknowledges the computational cost implicitly by noting that analyses are restricted to specific layers (3 out of 42 layers for Gemma2-9B) and specific model scales (1.5B rather than 8B for SAE training), but it does not discuss this as a limitation of the hypothesis's practical applicability. No future work section proposes cheaper diagnostic methods, and no sensitivity analysis demonstrates that the results are robust to cheaper approximations (e.g., smaller activation samples or narrower SAEs). The limitation is architectural rather than incidental—the hypothesis is tested using tools (SAEs, large-scale activation collection) that are themselves research-grade and not yet practical for routine model monitoring.

2. Causal Evidence for the "Reactivation" Mechanism Is Anecdotal and Narrow

The assumption or constraint. The Function Token Hypothesis asserts that during inference, function tokens "dynamically reactivate the most predictive features based on preceding contexts" (Section 3.2 caption). The word "reactivate" is specific—it implies that features first activated by content tokens in the context are re-activated at subsequent function token positions, carrying forward the semantic information needed to guide prediction. The paper provides a case study (Figure 6) showing activation patterns for three interpretable features (Feature 15261 = "Speak Chinese," Feature 9591 = "Russia," Feature 13751 = "UK") across two prompts. The visual patterns suggest that these features are activated at their corresponding content tokens ("Chinese," "Russia," "UK") and then re-activated at subsequent function tokens like ":", "the", and "\n". The steering experiments (Figures 7, 11) demonstrate that manipulating activations at a function token can alter outputs in semantically coherent ways, which is consistent with function tokens being causally involved in retrieving context features, but does not directly test whether those features were previously active at content token positions (reactivation) versus newly activated at the function token without prior context activation.

The consequence. The paper's central mechanistic claim—that function tokens specifically reactivate features from context rather than simply activating features de novo—rests on anecdotal evidence from three hand-picked features and two prompts. It is possible that many features active at function tokens are not reactivations but are instead context-dependent features that emerge at the function token position without having been previously active in the context. For example, a function token might activate a feature representing the grammatical structure of the upcoming phrase, which was not explicitly activated by any earlier content token but is inferred from the overall parse of the sentence. If the fraction of features that are genuine reactivations is low (say, 30%), the mechanism would be better described as "function tokens activate predictive features" rather than "function tokens reactivate predictive features," which weakens the analogy to memory retrieval—retrieval implies accessing previously stored information, not constructing new representations.

Additionally, the case study examines only three features out of the ~1 million in the SAE dictionary (0.0003%). These features were selected precisely because they were already known to be interpretable and steerable (Appendix A), making them far more likely to exhibit clean activation patterns than randomly selected features. Features that are less interpretable or more polysemantic might not show the neat content-token → function-token reactivation pattern visible in Figure 6.

What evidence exists in the paper. Figure 6 provides the only direct evidence for reactivation. For Prompt 1 (Russia), the "Russia" feature (red) activates at the content token "Russia" and then re-activates at the function tokens "?", "\n", "the", and at tokens in the model's response. The "Speak Chinese" feature (green) activates at "Chinese" and then at subsequent function tokens. For Prompt 2 (UK), the "UK" feature (blue) replaces the "Russia" feature while the "Speak Chinese" feature shows a similar pattern. However, the paper does not quantify: (a) what fraction of features active at function tokens were previously active at content tokens in the same context, (b) how this reactivation rate compares to the reactivation rate of features at content token positions, (c) whether the reactivation pattern holds for randomly sampled features rather than hand-picked interpretable ones, or (d) whether features that are not reactivated at function tokens are predictive of the output (if non-reactivated features are equally predictive, reactivation may not be the key mechanism).

The steering experiments (Figures 7, 11) provide causal evidence that function token activations influence outputs, but do not address the reactivation question directly. Steering a "Russia" feature at the \n token changes the output to Russia-related content, but this only shows that the feature can be made active at the function token position with causal effect—not that it would have been reactivated there naturally without steering.

Mitigation status. The paper does not acknowledge this as a limitation. The term "reactivation" is used throughout as if established, from the Figure 6 caption ("Function tokens can dynamically reactivate predictive features") to the hypothesis statement ("During inference, function tokens activate the most predictive features from the context"). The paper does not propose future work to systematically quantify reactivation rates or to distinguish reactivated features from context-dependent but newly activated features. This is a significant gap between the evidence presented (anecdotal case study) and the mechanistic claim made (function tokens as agents of memory retrieval through feature reactivation).

3. The Pre-Training Feature Emergence Analysis Uses an Architecturally Atypical Model

The assumption or constraint. The feature emergence results that link memory consolidation to function→content prediction (Section 4.2, Figures 8a and 8b) are derived from SAE analysis on a 2-layer, 1.5 billion parameter model trained with the LLaMA-3.1-8B architecture but with the depth reduced from 32 layers to 2 layers. The paper states this is done "[d]ue to computational constraints" (Section 4.2). The hidden dimension, FFN size, attention heads, and all other architectural hyperparameters are kept at the values designed for an 8B, 32-layer model. This means the 1.5B model has an extremely wide but extremely shallow architecture: each of its two layers must process the entire information flow from input to output, with no intermediate layers to form the hierarchical representations that characterize deep Transformers. Standard scaling practice (as in the LLaMA family itself) distributes parameters across depth—a naturally 1.5B-parameter LLaMA model would have approximately 24-28 layers, not 2.

The consequence. The feature dynamics observed in this model may not reflect how features emerge in realistically deep architectures. In a 2-layer model, the second (final) layer's residual stream activations must simultaneously encode local syntactic patterns, medium-range semantic dependencies, and global discourse structure—roles that in a 32-layer model would be distributed across early, middle, and deep layers respectively. Features that emerge in the second layer of a 2-layer model are therefore a compressed mixture of what would be separate features at different depths in a deeper model. The finding that "function tokens consistently activate most features, in contrast with content tokens" at layer 2 of the 1.5B model (Figure 8b) could be an artifact of this compression: if all information must pass through a single residual stream, the model might learn to route nearly all feature access through the tokens that are most universally present (function tokens), since they serve as reliable "access points" for the entire compressed representation. In a deep model, this routing might be distributed differently—shallow layers might handle local syntactic features accessed through content tokens, while middle layers might handle the context-integration features that the paper attributes to function tokens. The 1.5B model's 2-layer architecture could thus exaggerate the function token dominance relative to a realistically deep model.

The loss curves (Figure 9) do not entirely validate the 1.5B → 8B extrapolation. While the 8B model is 32 layers and shows qualitatively similar loss patterns by token transition type, the paper does not train SAEs on the 8B model, so it cannot verify that the feature growth and token-degree patterns observed in the 2-layer model replicate in the 32-layer model. The loss curves show that function→content prediction dominates optimization at both scales, but not how this translates into feature-token connectivity at different depths.

What evidence exists in the paper. The architectural anomaly is not discussed as a limitation. The paper describes the 1.5B model as having "only 2 layers, keeping other components unchanged" (Section 4.1), implying this is a straightforward parameter reduction rather than an architectural distortion. The feature emergence analysis treats the 1.5B model as directly informative about consolidation mechanisms without addressing the unusual depth-to-width ratio. Figure 8 reports feature counts and token degrees for layer 2 of this model, but does not compare with layer 1 or discuss what differences would be expected in a model with realistic depth.

Mitigation status. The paper partially addresses the scale limitation by showing that the loss patterns (Figure 9) scale from 1.5B to 8B with the same qualitative trends—function→content loss is highest, function prediction converges fastest, and scaling disproportionately helps content prediction. However, this only demonstrates that the prediction task difficulty is consistent across model scales and depths, not that the internal feature organization is consistent. The paper does not propose training SAEs on even a single checkpoint of the 8B model at a subset of layers, which would partially bridge the gap. The limitation is acknowledged only indirectly through the "due to computational constraints" phrasing, which explains the methodology choice but does not discuss its interpretive consequences. The paper treats the 1.5B model's feature dynamics as representative without caveat, which a reader deploying these findings to production models should treat with caution.

4. Single Model Family and Architecture for the Central Inference-Time Claim

The assumption or constraint. The core empirical claim about memory retrieval—that a small number of function tokens activate the majority of features during inference—is demonstrated exclusively on Gemma2-9B using Gemma Scope SAEs with dictionary width 2^20 at three layers (9, 20, 31). The paper states: "We use Gemma2-9B for our analysis, as it provides both models of different sizes and open-source SAEs" (Section 3.1). This is a pragmatic choice driven by the availability of pre-trained SAEs, but it means the entire inference-time retrieval analysis rests on a single model from a single model family (Google's Gemma) with a specific tokenizer (Gemma tokenizer), a specific SAE training methodology (whatever was used to produce Gemma Scope), and a specific SAE architecture (unspecified beyond being a "sparse autoencoder" from Gemma Scope).

The consequence. Several aspects of the findings could be model-specific or SAE-specific:

Tokenizer dependence. The function/content token classification uses the LLaMA-3.1 tokenizer on SlimPajama-627B (Section 2.2), but the inference-time analysis uses Gemma2-9B, which has a different tokenizer. The Gemma tokenizer would produce a different set of function tokens (different frequency distribution, different vocabulary), and the paper does not re-classify function tokens for the Gemma tokenizer. The 122 function tokens identified from the LLaMA tokenizer are assumed to correspond to function tokens in the Gemma tokenizer, but this correspondence is imperfect—a token that is a single unit in one tokenizer might be split into multiple tokens in another. The bipartite graph analysis (Section 3.1) uses "tokens from the vocabulary" (Figure 4 caption), which means it uses Gemma's actual tokenizer for the edge construction, but the classification of which tokens are "function tokens" for the coverage claims in Table 1 appears to use the LLaMA-derived list or a frequency-based reclassification that is not explicitly described for the Gemma tokenizer.
SAE dependence. The feature coverage numbers (Table 1) depend on what the SAE considers a "feature." Different SAE architectures (TopK, Gated, JumpReLU), dictionary widths, training objectives, or sparsity targets would produce different feature decompositions from the same activations, potentially yielding different coverage percentages. A JumpReLU-SAE with a different threshold, or a Gated-SAE with a different gating mechanism, might assign activations to features differently, redistributing token-feature edges. The paper uses one specific pre-trained SAE (Gemma Scope, width 2^20) without comparing to alternative SAE variants, despite Gemma Scope providing multiple SAE types per layer.
Model family dependence. Gemma2 uses a specific architecture, training data mixture, and training procedure. Whether GPT-4, Claude, or LLaMA-3.1 exhibit similar function-token-dominated feature access patterns is unknown. The observation that middle layers are most "expressive" for function token coverage (Table 1: layer 20 at 76.46% vs. layer 9 at 48.52%) might be specific to Gemma2's layer organization and would differ in models with different depth or different pre-training objectives.

What evidence exists in the paper. The paper provides no cross-model validation of the inference-time retrieval claim. All bipartite graphs, all feature coverage tables, all steering experiments, and all reactivation case studies use Gemma2-9B or its instruction-tuned variant Gemma2-9B-it. The pre-training consolidation analysis uses LLaMA-3.1 architecture (1.5B and 8B), but no SAE analysis is done on the LLaMA models to verify that the retrieval patterns observed in Gemma2 also hold.

Mitigation status. The paper addresses this limitation partially through its choice of Gemma2, which it describes as providing "both models of different sizes and open-source SAEs" (Section 3.1). However, it does not actually leverage the different model sizes—all analysis is on the 9B variant, with no comparison to Gemma2-2B or Gemma2-27B to test whether the function token effect scales with model size. The availability of Gemma Scope SAEs is presented as enabling the analysis, but the paper does not discuss how SAE-specific the findings might be. The limitation is not explicitly acknowledged, and no future work section proposes replication on other model families or with other SAE methods. A practitioner using a non-Gemma model would need to independently verify the central retrieval claim before applying the Function Token Hypothesis to their setting—the paper provides no guidance on whether this verification is necessary or how to perform it.

5. No Quantitative Support for the Claim That the Function Token Effect Is Specific to Function Tokens Rather Than High-Frequency Tokens Generally

The assumption or constraint. The paper classifies tokens into function and content categories based purely on frequency in the pre-training corpus: "we automatically classify tokens into 'function tokens' and 'content tokens' based on their frequencies in the pre-training corpus, using this as an approximation of the linguistic concepts" (Section 1). The top 122 most frequent tokens covering 40% of all token occurrences are labeled "function tokens"; all others are "content tokens." The paper then demonstrates that these high-frequency tokens activate disproportionately many features (Figure 5, Table 1), dominate attention and loss patterns, and are steerable. However, the paper never tests whether these properties are specific to function tokens qua linguistic function elements or whether they would hold for any set of the most frequent tokens regardless of their linguistic role. The 122-token set includes not just linguistically canonical function words (articles, prepositions, conjunctions) but also punctuation marks (comma, period, colon, semicolon), whitespace and formatting tokens (space, newline, tab-like whitespace), digits (0-9), quotation marks, and common year-like tokens (200, 201). Many of these (e.g., whitespace, newline, year tokens) are not function words in any linguistic sense. If the observed patterns are driven by token frequency alone—through mechanisms like: frequent tokens appear in more diverse contexts and thus co-occur with more features, or frequent tokens have more stable representations because they receive more gradient updates—then the "function" label is a red herring, and the hypothesis should be about "high-frequency tokens" rather than "function tokens."

The consequence. If the effect is frequency-driven rather than function-driven, the practical implications change significantly. The hypothesis that "function tokens activate the most predictive features from context" would generalize only to other high-frequency tokens, not necessarily to linguistically functional tokens that happen to be rare in the training corpus (e.g., infrequent prepositions like "amongst" or "unto," or function-like tokens in code such as specific indentation patterns or brackets). Conversely, high-frequency content tokens in specialized domains (e.g., "patient" in medical text, "court" in legal text, "function" in code) might exhibit function-token-like retrieval behavior even though they carry semantic content. The paper's recommendation that "the use of 'pivot tokens' during post-training can significantly enhance performance" (Section 5) would then need to be reinterpreted as a recommendation to identify high-frequency, high-context-diversity tokens regardless of their linguistic function, rather than specifically targeting linguistic function words.

The linguistic framing is central to the paper's narrative—it connects to linguistic theory, builds explanatory power from the Chomskyan distinction between functional and lexical categories, and suggests that LLMs have discovered and exploited this linguistic universal. If the mechanism is purely frequency-based, the narrative is weaker: the paper has rediscovered that common things are common, and that models learn to route information through commonly occurring tokens.

What evidence exists in the paper. The paper provides indirect evidence that the effect is not purely frequency-driven through the document coverage analysis (Figure 3): high-frequency tokens have uniform document coverage (they appear in nearly all documents), while low-frequency tokens have sparse, bursty coverage. This distributional property—uniform presence across diverse contexts—is what makes these tokens suitable as information routing points, and it correlates with but is not identical to frequency. A very frequent token that appears only in a specific type of document (e.g., a boilerplate phrase in web text) would have high frequency but low coverage and might not exhibit the function token pattern. However, the paper does not disentangle frequency from coverage: all high-frequency tokens in SlimPajama have high coverage (Figure 3c), so the explanatory variable is ambiguous.

The paper does not perform any ablation that would disambiguate frequency from linguistic function:

It does not compare the top-122 most frequent tokens with a set of 122 tokens that are linguistically function words but vary in frequency (e.g., including rare function words and excluding common non-function tokens like digits).
It does not test whether content tokens that become artificially frequent in a controlled training setup would acquire function-token-like properties.
It does not analyze whether the feature connectivity of a token is better predicted by its frequency, its document coverage, or its linguistic category (function vs. content).

Mitigation status. The paper acknowledges that the frequency-based classification is "an approximation of the linguistic concepts" (Section 2.2), implying that the mapping is imperfect and that some tokens in the 122-token set are not linguistically function words. However, it does not discuss the interpretive consequence of this approximation—that the hypothesis might be about statistical properties of tokens rather than their linguistic roles. The paper uses the term "function token" throughout, reinforcing the linguistic interpretation, without testing whether the effects would hold for a purely linguistic definition of function words. This is a fundamental ambiguity in the paper's central construct that a practitioner seeking to apply the hypothesis must resolve: should they focus on the specific 122 tokens identified here, on tokens with high frequency/coverage in their own corpus, or on tokens that play grammatical roles?

6. Hard Problems (Extrapolation to Complex Reasoning) Are Not Addressed

The assumption or constraint. All analyses in the paper—bipartite graphs, steering experiments, pre-training loss tracking, feature emergence tracking—are conducted on natural text from SlimPajama-627B, which is a cleaned, deduplicated web crawl corpus. The inference-time analyses use "raw documents from the SlimPajama validation dataset" (Section 3.1, Step 1) or simple factual/instruction-following prompts (Section 3.2, Appendix B). The pre-training analyses use the full SlimPajama corpus. This means the paper's claims about memory retrieval and consolidation are demonstrated exclusively in the context of standard next-token prediction on web text and simple prompted generation.

The paper does not test the Function Token Hypothesis on tasks requiring complex, multi-step reasoning—mathematical problem solving (e.g., MATH), chain-of-thought reasoning (e.g., GSM8K), logical deduction, long-horizon planning, or any task where the model must internally manipulate variables and maintain state beyond what is directly retrievable from the prompt context. The distinction matters because complex reasoning may require mechanisms that the function token hypothesis does not explain: maintaining intermediate computational states across many tokens, backtracking from dead ends, or synthesizing information from multiple disparate context locations into a novel inference. These operations might not map cleanly onto the "function tokens reactivate features from context" model.

The consequence. The paper's hypothesis may have limited applicability to the most challenging and practically important LLM tasks. If complex reasoning relies on mechanisms beyond context feature reactivation—for instance, on the model learning to perform internal symbolic operations that are not simply reactivations of previously seen content—then the function token hypothesis would explain only a subset of LLM behavior: the retrieval and reproduction of factual knowledge, instruction following, and linguistically structured generation, but not the synthesis of novel inferences or the execution of multi-step algorithms. The paper's opening examples (Section 1) gesture at both memorization ("The capital of the United States is Washington, D.C.") and reasoning ("The streets are wet and the sidewalks are slick. What is the most likely explanation?"), but the experiments do not test the latter.

The feature emergence analysis in Section 4.2 tracks total feature count growth during pre-training, which could encompass both "retrieval" features (encoding factual knowledge) and "reasoning" features (encoding operations or inference patterns). The paper does not classify features by type or measure whether function tokens preferentially activate retrieval features over reasoning features. It is possible that complex reasoning features are activated primarily through different mechanisms—perhaps through content tokens that act as intermediate variables, or through attention patterns that bypass the function-token bottleneck.

What evidence exists in the paper. The paper does not analyze any task requiring complex reasoning. All prompts in the steering experiments (Figures 7, 11) are simple factual questions or recommendations: "Where is Mount Fuji?", "Tell me a university.", "Could you recommend a tourist attraction?", "Please name my newborn baby daughter.", "Recommend me a traditional alcoholic beverage.", "Recommend me a traditional dish." These tasks primarily require retrieving factual associations (Mount Fuji → Japan, university → Harvard, traditional dish → Chicken Tikka Masala) and formatting the response according to the prompt's constraints (language, cultural domain). They do not require the model to perform multi-step inference, resolve contradictions, or synthesize information from multiple sources in the prompt.

The bipartite graph analysis (Section 3.1) uses raw web documents, which contain a mix of factual statements, narrative text, and possibly some reasoning-adjacent content (explanations, how-to guides), but the analysis is at the level of token-feature co-occurrence, not task-specific activation patterns. The pre-training loss analysis (Section 4.3) similarly uses web text and does not isolate the loss on reasoning-heavy passages.

Mitigation status. The paper does not acknowledge this scope limitation. The hypothesis is stated in general terms—"During inference, function tokens activate the most predictive features from the context to direct the next-token prediction" (Section 5)—without qualification about the types of tasks where this mechanism suffices. The paper's selection of MATH or GSM8K as testbeds would have provided evidence for or against the hypothesis's applicability to reasoning, but no such experiments are performed. The paper briefly mentions reasoning in Section 1 as a capability LLMs possess ("reasoning [22, 55] (e.g., The streets are wet and the sidewalks are slick. What is the most likely explanation?)"), and in Section 7 mentions that post-training for reasoning may "merely activate latent capabilities acquired during pre-training," but the experiments do not engage with reasoning tasks. A practitioner interested in whether the Function Token Hypothesis explains chain-of-thought reasoning or mathematical problem-solving would find no evidence in the paper—the claim of generality is asserted, not demonstrated.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a conceptual reframing rather than a new method or architectural innovation. Its primary contribution is a diagnostic lens—the function/content token distinction—that reveals a previously invisible organizing principle in how LLMs store and access knowledge. This is not a paradigm shift (it does not overturn existing theories of how Transformers work) but a substantive reorientation of interpretability research away from the prevailing circuit-level granularity toward a coarser, token-type-level analysis that explains macroscopic behavior before diving into microscopic mechanisms.

The significance of this reframing lies in its unifying power. Before this work, several empirical phenomena—massive activations at punctuation (Sun et al., 2024), separator-token attention dominance (Chen et al., 2025), pivot tokens in post-training (Abdin et al., 2024), and the effectiveness of high-entropy token training (Wang et al., 2025)—existed as independent observations without a common explanation. The Function Token Hypothesis reveals that these are all manifestations of the same underlying mechanism: tokens with high frequency and uniform document coverage become the primary access points for feature retrieval during inference, and predicting the content tokens that follow them is the dominant optimization pressure during training. This unification is intellectually satisfying in the way that good scientific explanations are—it takes scattered data points and organizes them into a single causal story—but its practical value depends on whether the unification generates new testable predictions rather than merely retrospectively explaining known results.

The paper succeeds in generating at least one forward-looking implication that prior work had not articulated: function tokens are the natural locus for behavioral interventions. The steering experiments (Figures 7, 11) demonstrate that modifying activations exclusively at the model's final function token (\n) reliably and coherently controls output behavior, with compositional effects when multiple features are steered simultaneously. This provides a concrete recipe for future work on controllable generation: rather than steering at content words (which might affect only local semantics) or at all token positions indiscriminately (which is computationally expensive and risks conflicting interventions), steer at function tokens. The paper shows this works for language selection and semantic domain control; whether it generalizes to more abstract traits (formality, verbosity, reasoning style) is an open but now testable question.

Prior contradictions resolved. The paper implicitly resolves a tension in the steering literature: some work finds that steering at any position can influence outputs (Templeton et al., 2024), while other work emphasizes the special role of specific token positions (Panickssery et al., 2024). The Function Token Hypothesis clarifies that while any position can be steered, function token positions are uniquely effective because they are where the model reads out context-predictive features to guide generation. Steering at a content token position might influence local processing but is less likely to propagate to the output because content tokens do not serve the same retrieval-routing role. The paper does not directly test this comparative claim (no content-token steering control is reported), but the hypothesis makes the prediction explicit.

Research directions that become more attractive:

Efficient inference through function-token caching. If function tokens are the primary access points for feature retrieval, then KV cache compression strategies that selectively retain function token representations (while discarding or aggressively compressing content token representations) might preserve model performance while dramatically reducing memory. The paper's observation that separator tokens show distinct attention patterns (Section 6, citing Chen et al., 2025) already points in this direction, but the function token hypothesis provides a mechanistic justification: you can discard content token caches because the information they contain has already been aggregated into function token representations via the retrieval mechanism.
Training data optimization. The finding that function→content prediction dominates the training loss (Figure 9: loss of 4.88 vs. 1.90 for content→function at 1.5B scale) suggests that data selection and curriculum strategies should prioritize examples rich in function→content transitions—text where structurally cued content predictions require integrating long-range context. This might mean upweighting expository and instructional text (where function words introduce substantive content) over list-like or boilerplate text (where content→content transitions dominate and less context integration is needed).
SAE-based developmental interpretability. The paper's training of SAEs at multiple pre-training checkpoints (Section 4.2, Appendix C) opens a new dimension for interpretability research: tracking when and how specific capabilities emerge during training. The methodological advance—equalizing reconstruction quality across checkpoints by tuning the L0 penalty λ (Figure 12)—is transferable to any study of feature emergence dynamics.

Research directions that become less attractive:

Content-token-focused circuit analysis for retrieval. If the paper's claim is correct that function tokens are the primary interface for memory retrieval, then circuit-level analyses that trace information flow primarily through content tokens may be studying the wrong pathways. The paper suggests that to understand how a model retrieves and uses context information, one should start by analyzing what happens at function token positions and trace outward from there, rather than starting from content words and tracing forward.

Follow-Up Research This Work Enables

Systematic quantification of feature reactivation rates across token types. The paper's central mechanistic claim—that function tokens reactivate features previously activated by content tokens—rests on a three-feature, two-prompt case study (Figure 6). A direct follow-up would compute, for a large random sample of features (not just hand-picked interpretable ones), the activation overlap between content tokens and subsequent function tokens versus content tokens and subsequent content tokens. Specifically: for each occurrence of a content token that activates feature i, measure whether feature i is also active at the next function token, the next content token, and at each token position up to some window. If the reactivation hypothesis is correct, the probability of re-activation should be higher at function token positions than at content token positions, and this asymmetry should be strongest for features that are predictive of upcoming content (e.g., features measured to have high mutual information with next-token predictions in function→content positions). A negative result—finding that reactivation rates are similar across all token types or that most function-token-activated features were not previously active in the context—would significantly narrow the hypothesis: function tokens might activate predictive features rather than reactivate them, which would still support the retrieval role but weaken the analogy to explicit memory recall.

Causal test of function→content prediction as the driver of feature expansion. The paper's consolidation claim is correlational: function→content loss is highest, and feature counts grow over the same period. To establish causality, a follow-up experiment would train two models with identical architectures and data but different loss weighting schemes. In one condition, standard next-token prediction loss on all tokens (replicating the paper's setup). In the other, upweight the loss on function→content transitions (e.g., by a factor of 2×) and downweight the loss on function→function and content→function transitions (to maintain the same total gradient norm). If function→content prediction causally drives feature expansion, the upweighted model should show faster feature count growth at equivalent training steps, and the effect should be most pronounced for features accessible through function tokens (measured by token-feature bipartite graph analysis at intermediate checkpoints). A null result—feature counts growing at similar rates regardless of loss weighting—would indicate that total training compute or data diversity, rather than specific prediction difficulty, drives consolidation, and the high function→content loss is a symptom rather than a cause.

Cross-model-family and cross-scale replication of the function token coverage result. The paper demonstrates that top-10 tokens cover 76.46% of features at layer 20 of Gemma2-9B (Table 1), but this could be an artifact of Gemma2's architecture, Gemma Scope's SAE training methodology, or the specific dictionary width (2^20). A replication study would apply the same bipartite graph construction to LLaMA-3.1-8B, Mistral-7B, and Qwen-2.5-7B using either existing open-source SAEs (if available) or newly trained SAEs with standardized hyperparameters. The key measurement is whether the concentration of feature access in high-frequency tokens is a universal property of autoregressive Transformers or varies with architecture, training data, or scale. Additionally, testing whether the effect strengthens or weakens with model scale (Gemma2-2B vs. 9B vs. 27B) would reveal whether function token dominance is a byproduct of limited capacity (smaller models might need to route more information through common tokens) or a fundamental design principle (larger models might exhibit even stronger routing through function tokens because they have more features to access).

Adversarial manipulation of function tokens to test robustness of the retrieval mechanism. If function tokens are the primary interface for feature retrieval, then adversarial perturbations to function tokens should degrade model performance more than equivalent perturbations to content tokens. A concrete experiment: take a reading comprehension or factual QA dataset, and for each example, either delete all function tokens (relying on the model to infer structure from content alone), replace function tokens with other function tokens (e.g., swap "the" for "a," swap "in" for "on"), or replace content tokens with semantically similar alternatives. If the hypothesis is correct, function-token perturbations should cause larger performance drops than content-token perturbations of equivalent "surprise" (measured by the perturbed token's probability under the original model). A related test: measure whether the model's ability to retrieve factual knowledge (e.g., "The capital of France is ___") is more disrupted by prepending irrelevant function tokens than by prepending irrelevant content tokens, which would suggest function tokens play a gating role in memory access.

Function-token-guided training data curricula for accelerated pre-training. The paper shows that function→content prediction is the hardest and slowest-to-converge subtask (Figure 9: loss > 4.8 for function→content vs. < 2.2 for function→function at 1.5B scale, with slower convergence). This suggests a natural curriculum: train initially on text where function→content transitions are short-range and predictable (e.g., children's books, simple Wikipedia), then gradually introduce text with long-range function→content dependencies (e.g., academic papers, legal documents, narrative fiction). The prediction is that a curriculum that explicitly sequences examples by the difficulty of their function→content transitions (measured by the average loss of an existing model on those transitions) would achieve lower final loss or faster convergence than uniform sampling, and that the advantage would be concentrated in the function→content and content→content loss categories. A negative result—curricula providing no benefit over uniform sampling—would suggest that the model needs the full diversity of transition difficulties from the start, or that easy transitions provide no useful foundation for harder ones.

Layer-resolved analysis of where function tokens acquire their retrieval capability. The paper observes that function token feature coverage peaks at middle layers (Table 1: 76.46% at layer 20 vs. 48.52% at layer 9 and 68.27% at layer 31) but does not explain mechanistically how function tokens come to activate more features at deeper layers. A follow-up would use attribution methods (e.g., activation patching, attention pattern analysis, or direct logit attribution) to trace how information from earlier content tokens flows to function token positions layer by layer. The hypothesis would be that attention heads in early layers copy content token representations to function token positions, and FFN layers in middle layers read from these aggregated function token representations to activate predictive features. If this is correct, one should observe (a) attention patterns where function tokens attend strongly to content tokens in early layers, (b) a shift in middle layers where function token representations become more predictive of upcoming content tokens (measured by probing), and (c) the emergence of the high feature coverage at function tokens coinciding with the layer where this predictive information first appears.

Practical Applications and Downstream Use Cases

Selective KV cache retention for memory-constrained inference. The finding that function tokens activate the majority of features (top-10 tokens covering 76.46% of features at layer 20; Table 1) and that separator tokens show distinct attention patterns (Section 6, citing Chen et al., 2025) directly motivates a function-token-prioritized KV cache eviction policy. In long-context inference, the KV cache grows linearly with sequence length and is often the memory bottleneck. If the model primarily retrieves information through function tokens, then retaining only function token KV cache entries (or retaining them at full precision while aggressively quantizing content token entries) could reduce memory usage by approximately 60% (since content tokens constitute ~60% of token occurrences, given function tokens cover 40%; Section 2.2) while preserving most of the model's ability to access context information. The paper does not test this directly, but the feature coverage asymmetry (function tokens connecting to far more features than content tokens in Figure 5) provides the mechanistic justification: discarding a content token's KV entry loses access to only a small number of features that are likely also accessible through nearby function tokens; discarding a function token's KV entry loses access to a much larger fraction of the feature space.

Efficient behavioral steering for controllable LLM deployments. The steering experiments (Figures 7, 11) demonstrate that modifying activations at a single function token position (the final \n before the model's response) is sufficient to control language output (English vs. Chinese), semantic domain (Russia vs. UK vs. default), and their composition. This provides a lightweight alternative to prompt engineering or fine-tuning for controlling model behavior in production systems. Instead of prepending lengthy system prompts ("Respond in Chinese and focus on Russian cultural references")—which consume context window tokens and may be inconsistently followed—a deployment system could identify the relevant features for desired traits (using the contrastive method in Appendix A), apply a constant steering vector to the final function token of every user prompt, and achieve reliable behavioral control with zero additional prompt tokens. The paper shows this works for three traits across five prompt types, but the method's cost (identifying features requires SAE access and contrastive prompt engineering) currently limits its practicality to research settings. If feature identification can be automated or if a library of pre-identified steerable features becomes available (analogous to Gemma Scope's open-source SAEs), this could become a deployable technique.

Data mixture optimization for domain-specific pre-training. The loss analysis (Figure 9) reveals that function→content prediction accounts for the majority of the optimization difficulty (loss of 4.88 vs. 3.69 for content→content at 1.5B scale) and that scaling model size disproportionately improves this subtask (∆ = 0.61 for function→content vs. ∆ = 0.25 for function→function when scaling from 1.5B to 8B). For organizations pre-training domain-specific models on limited compute budgets, this suggests that data selection should prioritize text with high density of diverse, context-dependent function→content transitions. Concretely, a data curator could use a small reference model to score candidate documents by their average function→content loss—documents where even a partially trained model is uncertain about content after function tokens are likely to be information-rich training examples. This is more targeted than generic "perplexity-based" data filtering because it isolates the specific subtask that drives feature learning, rather than averaging over all token positions including the easy function→function transitions that are learned almost immediately (within 3,000 steps; Figure 9c).

Diagnostic for detecting when LLMs are "guessing" versus retrieving. The paper's observation that function tokens reactivate context-predictive features (Figure 6) suggests a behavioral assay for retrieval quality: if a model is genuinely retrieving relevant information from context, specific features should be active at both the content tokens that introduce the information and the function tokens that precede the model's response. If a model is generating fluent but context-unrelated text (a failure mode in retrieval-augmented generation or long-context QA), one would expect feature activation at content tokens (the context is processed) but no reactivation at function tokens (the model fails to carry that information forward to the generation step). Monitoring the similarity between feature activation patterns at the last function token and at key content tokens could serve as an early warning signal for generation faithfulness, though this is currently impractical outside research settings due to SAE computational costs.