ArXiv: 2502.03490

🎯 Pitch

Transformers answering two-hop questions like “Who is Bob’s mother’s boss?” must learn each fact twice, consuming ~1.6 bits per parameter, rather than composing facts as a human would—a fundamental capacity constraint that chain-of-thought reasoning neatly overcomes by requiring only ~2 bits per parameter total.

1. Executive Summary

This paper analyzes why transformers exhibit inconsistent ability to learn latent two-hop question answering by examining how the information content scaling of trained models—the relationship between model size and the amount of dataset information successfully compressed into the weights—varies with different algorithmic hypotheses. Using synthetic two-hop QA datasets (questions of the form "Who is Bob's mother's boss?") and Llama-architecture transformers ranging from 250K to 15M parameters, the authors find that capacity scaling best supports the "two function composition" hypothesis (where transformers learn each fact twice—once for first-hop queries and once for second-hop queries—with an estimated ~1.6 bits per parameter), in contrast to the recurrent alternative where each fact is learned once. This finding is corroborated by generalization experiments showing that models without chain of thought fail to generalize when any component of a hop is held out from training, establishing that latent two-hop QA requires transformers to replicate factual information across layers rather than reusing a single learned function—a capacity constraint that chain-of-thought reasoning substantially alleviates by enabling more efficient, recurrent-like computation with only ~2 bits per parameter.

2. Context and Motivation

The Core Problem: Understanding How Transformers Perform Compositional Reasoning

This paper tackles a specific, well-delineated question about transformer capabilities: when a transformer learns to answer two-hop questions (e.g., "Who is Bob's mother's boss?"), does it learn to compose two independently stored facts, or does it memorize the answers to two-hop questions directly? The answer matters because it reveals something fundamental about whether transformers can perform genuine compositional reasoning—applying learned operations to novel combinations of inputs—when answering questions in a single forward pass, without chain of thought.

The paper's framing is narrower and more mechanistic than the broader "can LLMs reason?" debate. Rather than asking whether transformers can produce correct answers to compositional questions (they can, to some degree), the authors ask what algorithm the transformer's weights encode to produce those answers, and they propose a novel method—information content scaling—for distinguishing between candidate algorithms.

Why This Matters: The Gap Between Capability and Mechanism

The paper's motivation rests on a critical distinction that pervades modern deep learning research: a model can produce correct outputs without implementing the algorithm a human designer intended. This is the difference between behavioral success and mechanistic understanding. A transformer that correctly answers "Who is Bob’s mother’s boss?" might be:

  1. Memorizing the answer to that specific triple (Bob, mother, boss) from training data, having seen it directly.
  2. Composing two independently learned functions: one that maps (Bob, mother) → Bob's mother, and another that maps (Bob's mother, boss) → the final answer, each learned from one-hop training examples.
  3. Learning a single recurrent function that can be applied iteratively: computing the intermediate entity via one application, then feeding it back into the same function for the second hop.

These algorithms have profoundly different implications for generalization, robustness, and the fundamental nature of transformer computation. Yet distinguishing between them purely from input-output behavior is difficult, because all three can produce correct answers on seen questions.

The Recurrence Bottleneck

The paper's central mechanistic claim concerns the architectural constraint of feed-forward transformers. Unlike recurrent neural networks, transformers process all tokens in parallel through a fixed number of layers. Information flows forward through the stack—tokens attend to other tokens, representations are updated by MLPs—but there is no mechanism for the model to "loop back" and re-apply the same function to an intermediate result within a single forward pass.

For two-hop question answering, this architectural constraint matters. To answer "Who is Bob’s mother’s boss?", the model must:

  1. Identify Bob's mother (the intermediate entity)
  2. Identify that intermediate entity's boss (the final answer)

If the model had recurrence, it could learn a single function f(entity, relation) that works for both hops, applying it first to (Bob, mother) and then to (Bob's mother, boss). But in a feed-forward transformer, by the time the intermediate entity is computed (presumably somewhere in the middle layers), the computation has already passed through the initial layers—there is no way to "go back" and use those early layers again. The model must therefore encode two separate copies of the factual lookup machinery: one in earlier layers for the first hop, and another in later layers for the second hop.

This is the "each fact learned twice" hypothesis that the paper's capacity scaling experiments are designed to test.

Prior Approaches and Their Shortcomings

1. Behavioral Generalization Studies

The most straightforward way to study compositional reasoning is to test how models generalize to held-out combinations. The paper cites Wang et al. (2024), who systematically studied two-hop QA generalization in transformers and found:

  • Transformers trained on two-hop questions fail to generalize to unseen questions (e1, r1, r2) if the facts (e1, r1) and (e2, r2) appeared in training only in the context of one-hop questions, not in any two-hop question. This suggests that facts learned for one-hop QA are not automatically "reused" for two-hop QA—they must be learned in position for each hop.
  • Supervised probes in middle layers could extract intermediate entities, suggesting a two-function composition mechanism rather than flat memorization.

However, behavioral generalization studies have significant limitations as an interpretability tool:

"it may be infeasible to test a model's generalization performance on all tasks of interest"

The space of possible generalizations is vast. A model might fail to generalize on one held-out split for reasons unrelated to its internal algorithm (e.g., specific hyperparameter choices, training dynamics, or distributional quirks). Conversely, a model that does generalize might be using an algorithm different from the one the experimenter hypothesizes. Generalization results alone cannot conclusively identify the algorithm a model implements—they can only falsify hypotheses that make incompatible predictions.

2. Probing and Activation-Based Methods

Probing (Belinkov, 2022) and related techniques like sparse autoencoders (Huben et al., 2023) and the logit lens (used by Wang et al., 2024) attempt to extract interpretable signals directly from model activations. These methods train auxiliary classifiers to decode information (like intermediate entities) from hidden representations, potentially revealing what computations the model performs at which layers.

The paper identifies several limitations:

  • Probes may miss important features of the computation. A probe can only recover linearly decodable information; if the model represents intermediate entities in a nonlinear, distributed, or non-probable format, probes may report false negatives.
  • Probes may fail to yield useful hypotheses. Even when probes successfully extract information, they don't directly reveal the algorithm—they show what is represented, not how the representation is used. A probe could find intermediate entities in the activations even when the model is using flat memorization (the intermediate entity is correlated with other features that the probe exploits).
  • Probing is inherently correlational. The presence of linearly decodable information in activations doesn't prove that information is causally used for the task output.

The authors' own probing results (Section D, Tables 2 and 3) illustrate these limitations starkly: they failed to recover intermediate entities using supervised linear probes even in models that their capacity scaling analysis and generalization experiments suggested were using two-function composition. The intermediate entity was barely more decodable than an arbitrary relation of the first entity. This is a concrete demonstration that probing can be insufficient for algorithmic interpretation, even when the ground-truth algorithm is known to exist (from generalization behavior).

3. Existing Work on Multi-Hop Reasoning

The paper situates itself within a broader literature on transformer compositional limitations:

  • Dziri et al. (2023) found that transformers often fail to generalize on compositional tasks whose graphical structure was underrepresented in training data. This is consistent with the idea that transformers do not learn reusable compositional primitives but rather memorize patterns specific to their training distribution.
  • Merrill et al. (2022) and Liu et al. (2023) established theoretical limits: transformers without chain of thought are constrained to constant-depth threshold circuits, which restricts the class of functions they can compute in a single forward pass. Compositional reasoning over variable-depth structures (like multi-hop chains) exceeds these limits.
  • Pérez et al. (2021) showed that with chain of thought—autoregressive generation where intermediate reasoning steps are produced as tokens—transformers become Turing-complete, removing these theoretical constraints entirely.

The two-function composition hypothesis the paper tests is a specific instantiation of these theoretical limits: transformers cannot implement true recurrence within a single forward pass, so they must replicate factual lookup functions across layers when performing multi-step reasoning without chain of thought.

4. Allen-Zhu & Li (2024) and Knowledge Capacity Scaling

The paper's methodological foundation comes from Allen-Zhu & Li (2024), who established that sufficiently trained transformers can memorize facts up to a limit of approximately 2 bits per parameter, independent of architecture or dataset size. This empirical finding provides a reference point: if we know how much information a transformer of a given size can store, and we know how much information a candidate algorithm requires, we can test which algorithm best explains the model's performance by measuring how much information the model has actually encoded.

However, the paper does not perfectly replicate the 2 bits per parameter figure. In their main experiments (using 17 relations and µP parametrization), they observe approximately 1.6 bits per parameter for one-hop QA (Figure 1). Earlier runs with different hyperparameters (4 relations, no µP) came closer to 2 bits (Figure 8). This variability introduces uncertainty into the method but doesn't invalidate its core logic: the relative scaling across different algorithmic hypotheses is what matters for discrimination, not the absolute capacity figure.

How This Paper Positions Itself

The paper occupies a novel methodological niche in the interpretability landscape. Rather than relying on generalization experiments or probing—the two dominant approaches—it proposes information capacity scaling as a third, complementary method. The core logic is:

  1. Hypothesize candidate algorithms that a transformer might implement for a given task.
  2. Derive the information content scaling of each algorithm: how much data (in bits) must the model store to implement this algorithm perfectly?
  3. Measure the model's actual information content at different model sizes by computing the gap between dataset entropy and the model's cross-entropy loss.
  4. Compare curves: which algorithmic hypothesis produces scaling curves that best match the empirical capacity reference (e.g., 2 bits per parameter)?

The paper frames this as a method for testing specific algorithmic hypotheses, not as a general-purpose interpretability tool:

"Practically, we make specific predictions about generalization based on specific algorithmic hypotheses and their information content scaling properties. This rests on the assumption that algorithms with the same information content scaling properties tend to have the same generalization behaviour."

This is a deliberately narrow claim. The authors are not arguing that capacity scaling alone can uniquely identify any algorithm—rather, it can serve as an additional source of evidence that may be informative when other methods (like probing) fail, as they did in this study.

The method also connects to broader theoretical frameworks linking compression and intelligence. The paper cites Delétang et al. (2023) on language modeling as compression and the classical Solomonoff (1964) theory of inductive inference, which formalizes the relationship between representation length and predictive performance. A model that compresses its training data efficiently—approaching the theoretical minimum description length for some hypothesis class—has effectively learned a generative model of that data. The "native information content" that the paper measures is a proxy for the compression efficiency achievable by a transformer architecture, which differs from ideal compression precisely when the data exhibits recurrent patterns that transformers cannot exploit without chain of thought.

The Novelty of the Method

What distinguishes this work from prior approaches is the combination of:

  • Hypothesis-driven capacity measurement: rather than training probes or testing generalization, the method directly tests whether the volume of information a model stores is consistent with a particular algorithm's information requirements.
  • Computational model framework: the authors formalize three distinct computational models for two-hop QA (hash table, two-function composition, recurrent composition) and derive their information content formulas (Equations 2–4), making testable quantitative predictions.
  • Capacity scaling as a complement to generalization: the paper shows that when probing fails (intermediate entities are not linearly decodable), capacity scaling can still provide evidence consistent with generalization results (both support two-function composition). When the two methods agree, confidence increases; when they disagree, it reveals limitations of the probing approach.

This is a significant contribution because it addresses a persistent problem in interpretability research: reliance on methods that are either too weak (behavioral testing doesn't uniquely identify mechanisms) or too strong in their assumptions (probing requires linear decodability). Information capacity scaling sits at an intermediate level—it makes quantitative, falsifiable predictions based on clearly stated algorithmic hypotheses, without requiring that the model's internal representations be human-interpretable.

3. Technical Approach

3.1 Reader Orientation

This paper proposes a diagnostic framework — not a new model or training algorithm — for inferring what internal algorithm a trained transformer uses to solve two-hop question answering. The core idea is to measure how much information a trained model has encoded into its weights (its "information content"), compare this measured quantity against the information requirements predicted by different candidate algorithms (hash table memorization, two-function composition, recurrent composition), and identify which algorithm's predicted scaling best matches the empirical data. The problem it solves is the algorithmic identification problem: given a trained model that produces correct outputs, how can we determine which of several plausible internal mechanisms it actually implements, especially when neither behavioral testing nor activation probing provides definitive evidence?

3.2 Big-Picture Architecture (Diagram in Words)

The framework has five major components, organized as a pipeline from hypothesis generation to empirical validation:

  1. Dataset Generator — produces synthetic two-hop QA datasets with controlled size and structure. It creates fictional profiles of "people" with randomly assigned relations and properties, then generates training examples as text strings from templates. Its role is to provide a clean, fully observable data generating process where ground-truth information content can be exactly computed.

  2. Computational Model Hypotheses — a set of explicit alternative algorithms the transformer might implement internally (hash table, two-function composition, recurrent composition). For each hypothesis, the authors derive the minimum information content (in bits) required to perfectly answer all two-hop questions under that algorithm. These derivations produce closed-form expressions in terms of dataset parameters (number of entities $N$, number of relations $|R|$, number of attributes $|A|$, etc.).

  3. Transformer Training Pipeline — trains Llama-architecture models of systematically varying sizes (250K to 15M parameters) on each dataset to convergence. Training uses a custom small-vocabulary tokenizer, µP parametrization for consistent hyperparameter transfer across widths, and schedule-free AdamW optimization.

  4. Information Content Measurement — for each trained model, computes the model's actual encoded information content by subtracting the sum of per-token cross-entropy losses (the "surprise" the model still has about the data) from the total dataset entropy (the total information in the data before any learning). Different algorithmic hypotheses produce different formulas for this computation, because they model the relationship between one-hop and two-hop performance differently.

  5. Scaling Comparison — plots measured information content against model parameter count for each dataset size, overlaid with reference capacity curves (e.g., 2 bits per parameter). Whichever hypothesis produces scaling curves that align with the empirical capacity line is judged the best explanation of the model's internal algorithm.

Information flows as follows: dataset parameters → algorithmic hypothesis selection → predicted information content formula → measurement procedure → empirical information content versus parameter count → hypothesis comparison.

3.3 Roadmap for the Deep Dive

  • First, the dataset generation procedure (Section 2.1 in the paper). Understanding the exact data structure is essential because all information content formulas depend on dataset parameters ($N$, $|R|$, $|A|$, etc.) and the independence assumptions built into the data generating process.
  • Second, the three computational model hypotheses and their information content derivations. This is the core intellectual machinery: how do we translate "the model uses two-function composition" into a specific number of bits that must be stored?
  • Third, the transformer training configuration and convergence criteria. The information content method requires models trained to saturation; training details (architecture, tokenizer, optimizer, stopping rule) determine whether this condition is met.
  • Fourth, the information content measurement procedure. This is where the paper operationalizes its central concept: given a trained model and a hypothesized algorithm, how do we compute a single number representing encoded information?
  • Fifth, the capacity scaling analysis — how measured content is plotted against model size and interpreted relative to reference capacity curves.
  • Sixth, the generalization experiments and probing controls, which provide independent evidence triangulating the capacity scaling results.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methodological paper that proposes information capacity scaling as a diagnostic tool for inferring transformer algorithms, and validates this tool against behavioral and probing baselines on synthetic two-hop QA data. The core technical machinery is the translation from algorithmic hypotheses to quantitative information content predictions, and the procedure for measuring encoded information from model losses.

Dataset Generation

The paper generates fully synthetic datasets where the ground-truth data generating process is known exactly. This is critical: to measure how much information a model has learned, we must know how much information was in the data to begin with.

Profile generation. The system creates $|N|$ fictional "people" (entities), where $|N|$ ranges from 1000 to 250,000 across experiments. Each person receives:

  • A randomly selected first, middle, and last name drawn without replacement from $|N_0| = 4 \times 10^{11}$ possible combinations. This enormous name space means that name collisions are vanishingly unlikely, and the model cannot guess names — it must memorize them.
  • 17 types of relations to other people. The relations have no semantic structure in the data generating process — although some are labeled "parent" and others "child," the actual assignments are independently uniformly random. The "child" of a person is not constrained to be the person whose "parent" is them. This deliberate semantic decoupling prevents the model from exploiting real-world commonsense correlations and forces it to learn from the explicit training data.
  • 4 types of properties: birth city, birth date, employer, and university. These are attributes whose values are not other people, so they can only appear as the second hop of a two-hop question (you cannot ask "Who is Bob's birth date's boss?" — birth dates don't have bosses).

The set of all relation types is denoted $R$. The set of all relations plus properties is denoted $A$ (attributes). Each attribute $a_j \in A$ has a set of possible values $V_j$. Relations map people → people; properties map people → non-person values.

Question generation from templates. Given the profiles, the system generates training examples as flat text strings from two templates:

  • One-hop questions: "What was {person}'s {relation/attribute}? {answer}"
  • Two-hop questions: "What was {person}'s {relation}'s {relation/attribute}? {answer}"

The "first hop" always traverses a relation (producing an intermediate person), while the "second hop" can traverse either a relation or a property (producing a person or a non-person value).

An earlier version of the work employed diverse paraphrases, but this was abandoned to simplify the experimental analysis. The resulting text is English-like but lacks the lexical and syntactic variety of natural language — it is essentially a template-based synthetic corpus.

Training data composition. Every fact (every person's every relation and property) appears in the training data as a one-hop question. This is a critical design choice: the model always has the opportunity to learn each individual fact in isolation through one-hop examples. The question is whether it can then compose these facts to answer unseen two-hop combinations.

Unless the experiment specifies one-hop-only training, models are trained with a ratio of 1 one-hop question for every 10 two-hop questions. This ratio was chosen (though not exhaustively justified) to ensure the model attends to one-hop performance while primarily optimizing for two-hop accuracy.

Held-out sets for generalization testing. The paper constructs seven distinct held-out sets by systematically excluding specific components of two-hop questions from the training data:

  • First Entities ($e_1$): all two-hop questions involving a particular person in the first position
  • First Relations ($r$): all two-hop questions using a particular relation as the first hop
  • Second Entities ($e_2$): all two-hop questions where the intermediate entity is a particular person
  • Attributes ($a$): all two-hop questions using a particular attribute as the second hop
  • Entity-relation pairs ($e_1, r$): all two-hop questions involving a specific combination of first entity and first relation
  • Entity-attribute pairs ($e_2, a$): all two-hop questions involving a specific combination of intermediate entity and second-hop attribute
  • Complete questions ($e_1, r, a$): specific full triples held out entirely from two-hop training

In each case, the held-out facts still appear in one-hop questions during training. The generalization predictions of Table 1 are derived from this setup: each algorithmic hypothesis makes different predictions about which held-out sets the model will and will not correctly answer.

Computational Model Hypotheses and Information Content Derivations

The paper formalizes three distinct algorithms a transformer could implement internally for two-hop QA, and derives the minimum information that must be stored in the model's weights to implement each algorithm perfectly. These derivations are the theoretical backbone of the method — they translate qualitative algorithmic hypotheses into quantitative predictions about information content scaling.

Hash Table (Independent Memorization)

Under the hash table model, the transformer treats every two-hop question $(e_1, r, a)$ as an independent fact. There is no compositional reuse — the answer to "Who is Bob's mother's boss?" is stored as a separate entry from the answer to "Who is Bob's mother's friend?" even if both involve the same intermediate entity.

Information content formula:

E2independent=∣N∣log⁡2N0+∣R∣∣N∣∑a∈Alog⁡2∣Va∣E^{\text{independent}}_2 = |N| \log_2 N_0 + |R| |N| \sum_{a \in A} \log_2 |V_a|

where $|N|$ is the number of entities, $N_0 = 4 \times 10^{11}$ is the number of possible name combinations, $|R|$ is the number of relation types (17 in main experiments, 4 in some trials), $|A|$ is the set of all attributes, and $|V_a|$ is the number of possible values for attribute $a$.

What it computes: the total entropy of the dataset when every two-hop answer is treated as an independent random variable. The term $|N| \log_2 N_0$ is the information required to specify which $|N|$ names (out of the $N_0$ possibilities) appear in the dataset — the model cannot predict names it hasn't seen. The term $|R| |N| \sum_{a \in A} \log_2 |V_a|$ is the information for all attribute values: there are $|R| |N|$ possible first-hop (person, relation) pairs, each leading to $|A|$ possible second-hop attributes, each of which takes one of $|V_a|$ possible values. Since every combination is treated independently, the total information scales as $|R| \times$ the one-hop information.

Why this form: this is the information content of a lookup table with $|R| |N| |A|$ entries, each requiring $\log_2 |V_a|$ bits (on average over attribute types). It is the maximum possible information the model could need — no compression across questions is assumed.

Two-Function Composition

Under two-function composition, the transformer learns two separate functions: $f_1 : N \times R \rightarrow N$ that maps (person, first relation) to the intermediate person, and $f_2 : N \times A \rightarrow V$ that maps (intermediate person, second attribute) to the final answer. The two functions are stored independently — $f_1$ is encoded in early layers, $f_2$ in later layers — so each fact (person, relation) → person must be stored twice: once as part of $f_1$ for first-hop lookups, and once as part of $f_2$ for second-hop lookups.

Information content formula:

E22f=∣N∣log⁡2N0+2∣N∣∑a∈Alog⁡2∣Va∣E^{\text{2f}}_2 = |N| \log_2 N_0 + 2 |N| \sum_{a \in A} \log_2 |V_a|

where the terms are as defined above. The critical difference from the independent model is the factor: $2|N|$ instead of $|R||N|$ for the attribute information term.

What it computes: the total information required when each fact is stored twice — once for first-hop application and once for second-hop application — but instances of the same relation type are not stored independently. The factor of 2 comes from the two copies (one in $f_1$, one in $f_2$), not from the $|R|$ relation types. A fact like (Alice, mother → Carol) is stored once for use as a first hop (when Alice's mother is needed) and once for use as a second hop (when Carol's attributes are queried), sharing the same underlying fact.

Why this form: the paper argues that transformers cannot implement recurrence in a single forward pass. To compose $f(f(e_1, r), a)$ without recurrence, the model must encode two separate copies of the mapping, distinguished by their position in the layer stack. The information content scales with $2|N|$ rather than $|R||N|$ because the same fact about relation "mother" is reused across all first-hop queries — it is not a separate fact for each relation type. The $\sum_{a \in A} \log_2 |V_a|$ term accounts for the per-entity per-attribute value, which must be stored twice (once per copy).

Recurrent Composition

Under recurrent composition, the transformer learns a single function $f : N \times R \rightarrow N$ and applies it twice: first to compute the intermediate entity, then to compute the final answer from that intermediate entity. This is the most compressed representation possible — each fact is stored exactly once.

Information content formula:

E2recurrent=E1=∣N∣log⁡2N0+∣N∣∑a∈A∪Rlog⁡2∣Va∣E^{\text{recurrent}}_2 = E_1 = |N| \log_2 N_0 + |N| \sum_{a \in A \cup R} \log_2 |V_a|

where $E_1$ is the one-hop information content (Equation 1). This is identical to the one-hop formula because knowing all one-hop facts is sufficient to answer all two-hop questions under recurrent composition.

What it computes: the total information in the one-hop training data. If the model truly learns a recurrent algorithm, it only needs to memorize each person's relations and properties once — the same learned function handles both hops by being applied iteratively.

Why this form: this is the lower bound on information content for any algorithm that can answer all two-hop questions. It represents the "native" complexity of the task if architectural constraints (lack of recurrence) did not force information duplication. The gap between $E^{\text{recurrent}}_2$ and the other models' information content is the recurrence penalty — the extra information a feed-forward transformer must store because it cannot loop.

Relationship to Generalization Predictions

The paper explicitly links these information content formulas to the generalization predictions in Table 1 (Section 1.2.1):

  • Hash table: requires $(e_1, r, a)$ in training for every correctly answered question. Fails to generalize to any held-out set.
  • Two-function composition: requires $(e_1, r)$ seen in first-hop position (for $f_1$) and $(e_2, a)$ seen in second-hop position (for $f_2$). Generalizes to held-out complete questions $(e_1, r, a)$ if both component facts are trained in position, but fails when individual hop components (first entities, first relations, etc.) are held out.
  • Recurrent composition: requires only that both facts $(e_1, r)$ and $(e_2, a)$ appear in training anywhere (one-hop or two-hop). Generalizes to all held-out sets except those that remove facts entirely.

This tight coupling between information requirements and generalization behavior is the paper's key argument for why information content scaling can serve as an interpretability tool: if the measured information content matches the two-function composition scaling, we can infer the generalization signature without running exhaustive generalization tests.

Transformer Training Pipeline

The training pipeline is designed to produce models that have converged to their capacity limit — they have learned as much as their parameter count allows — so that the measured information content reflects the model's fundamental storage capacity rather than transient training dynamics.

Architecture. All models are based on the Llama architecture (Dubey et al., 2024), with parameter counts ranging from approximately 250K to 15M. The small scale is deliberate: the paper needs to train multiple models across multiple dataset sizes to saturation, which would be computationally prohibitive at larger scales. Both 4-layer and 12-layer variants are tested, with 4-layer models being the primary focus.

Custom tokenizer. To enable extremely small parameter counts without embedding parameters dominating the total, the authors train a custom tokenizer on their synthetic data with a vocabulary of only 3,000 tokens. At typical vocabulary sizes (32K–50K), the embedding layer alone would consume tens of millions of parameters, dwarfing the transformer blocks and making capacity comparisons difficult. The small vocabulary keeps the embedding-to-transformer parameter ratio manageable for sub-million-parameter models.

µP parametrization. The paper uses µP (Yang et al., 2022), a parameterization scheme that enables hyperparameters (learning rate, initialization scale, etc.) to transfer across model widths without re-tuning. In standard parameterizations, optimal hyperparameters change with model width; µP corrects for this by scaling initialization and learning rates according to principled rules derived from infinite-width limits. This is important because the paper sweeps over a wide range of parameter counts and needs consistent convergence behavior without width-specific hyperparameter searches.

Schedule-free AdamW. The optimizer is schedule-free AdamW (Defazio et al., 2024), which eliminates the need to specify a learning rate schedule in advance. Standard training requires choosing a warmup period, peak learning rate, and decay schedule — each of which might need to be tuned per model size and dataset. Schedule-free AdamW internally adapts the effective learning rate trajectory, allowing the authors to simply train for as long as necessary to reach a convergence criterion without manually tuning schedules.

Training data format. Training text is split into 500-token chunks, and a batch size of 32 sequences is used (yielding approximately 16,000 tokens per batch, including special tokens). All tokens except the answer tokens are masked in the loss computation — the model is trained only to predict the answer portion of each training example, not to auto-regressively model the question text. This focuses the capacity measurement on factual storage rather than language modeling.

Convergence criterion. Training continues until the loss decreases by less than $10^{-8}$ per step. This typically requires approximately 10 million steps (or roughly 160 billion tokens) and does not depend strongly on dataset size. The criterion is extremely stringent — a loss change of $10^{-8}$ per step at typical learning rates corresponds to effectively zero gradient — ensuring models have truly saturated.

Key design choice — synthetic data. The use of completely synthetic, template-generated data with known entropy is what enables information content measurement. With natural language data, the underlying entropy is unknown (language is not a known random process), so there is no baseline to subtract the model's loss from. The synthetic data provides an exact, computable reference point.

Information Content Measurement

This is the central technical operation of the paper: given a trained model and a hypothesized computational model (hash table, two-function composition, or recurrent composition), compute a single number representing how much information the model has encoded.

Dataset Entropy — The Starting Point

The dataset entropy $E$ is the information content (in bits) of the training data before any learning occurs, computed from the data generating process. For each question type and computational model, there is a specific formula (Equations 1–4 in Section 2.3).

The general form:

E=∣N∣log⁡2N0+[fact-storage terms depending on model]E = |N| \log_2 N_0 + \text{[fact-storage terms depending on model]}

where $|N| \log_2 N_0$ accounts for the information needed to specify which $|N|$ distinct names appear in the dataset (out of $N_0$ possible combinations). This term scales with $|N|$ and is present in all models — even a model that perfectly memorized all factual relationships would still need to learn the vocabulary of names.

The paper makes several simplifying assumptions in computing dataset entropy:

  • The receiver (model) already knows which tokens are valid for each answer type, so information about the token vocabulary itself is neglected.
  • The $|N| \log_2 N_0$ approximation is used rather than the exact combinatorial formula $\log_2 \binom{N_0}{N}$ because $|N| \ll N_0$ in all experiments (the largest $|N| = 250,000$ is dwarfed by $N_0 = 4 \times 10^{11}$), making the approximation tight.
  • Other non-scaling terms (e.g., learning to attend to the correct tokens in the question) are omitted because they do not scale with dataset size and therefore do not affect the capacity scaling analysis.
One-Hop Information Content

For one-hop QA, a model that achieves perfect accuracy has encoded the full dataset entropy $E_1$. A model with imperfect accuracy has encoded less — the gap between $E_1$ and the model's total loss represents the information the model has failed to learn.

The one-hop information content $C_1$ is lower-bounded by:

C1≥E1+∣N∣∣A∣E[log⁡2pone-hop]C_1 \geq E_1 + |N| |A| \mathbb{E}[\log_2 p_{\text{one-hop}}]

where $E_1$ is the one-hop dataset entropy (Equation 1), $|A|$ is the number of attributes (21 in main experiments: 17 relations + 4 properties), $|N|$ is the number of entities, and $\mathbb{E}[\log_2 p_{\text{one-hop}}]$ is the expected log-probability of the correct one-hop answer (averaged over all tokens in the answer, across all one-hop questions in the dataset).

What it computes: start with the total information in the data ($E_1$), then subtract the model's residual uncertainty about the correct answers ($|N| |A|$ questions, each with average negative log-probability $-\mathbb{E}[\log_2 p]_{\text{one-hop}}]$). The result is a lower bound on how many bits of information the model has successfully encoded into its weights.

Why this form: this is the standard information-theoretic decomposition used by Allen-Zhu & Li (2024). When the model achieves zero loss ($p = 1$ for all correct answers, so $\log_2 p = 0$), the information content equals $E_1$ — the model has fully compressed the data. When the model outputs a uniform distribution over possible answers ($p = 1/|V|$), the $\log_2 p$ term is negative, reducing the measured content toward a lower baseline.

Information Content for Two-Hop Computational Models

For two-hop QA, the measurement procedure depends on which computational model is assumed, because different models relate two-hop performance to the underlying one-hop functions differently.

Independent (hash table) model. For the independent model, the measurement is analogous to one-hop:

C2independent≥E2independent−∣N∣∣R∣∣A∣E[log⁡2ptwo-hop]C^{\text{independent}}_2 \geq E^{\text{independent}}_2 - |N| |R| |A| \mathbb{E}[\log_2 p_{\text{two-hop}}]

where $p_{\text{two-hop}}$ is the model's probability for the correct two-hop answer. Because every two-hop question is treated as independent, the model's loss on two-hop questions directly measures how much of $E^{\text{independent}}_2$ has been learned.

Recurrent composition model. For the recurrent model, the procedure is more complex because we cannot directly observe the model's internal one-hop function — we only observe two-hop outputs. The paper derives a lower bound by relating two-hop probability to an "effective" one-hop probability.

The derivation (Appendix A.1) proceeds as follows. Under recurrent composition, the two-hop probability $p_2(e^*_{112} | e_1, r_1, r_2)$ decomposes as:

P2(e112∗∣e1,r1,r2)=P1(e11∗∣e1,r1)P1(e112∗∣e11∗,r2)+1−P1(e11∗∣e1,r2)∣N∣P_2(e^*_{112}|e_1, r_1, r_2) = P_1(e^*_{11}|e_1, r_1) P_1(e^*_{112}|e^*_{11}, r_2) + \frac{1 - P_1(e^*_{11}|e_1, r_2)}{|N|}

where $e^*_{11}$ is the correct intermediate entity (e.g., Bob's mother), $e^*_{112}$ is the correct final answer, and $P_1$ is the model's probability for one-hop queries. The first term is the product of getting both hops correct. The second term accounts for the case where the first hop is wrong but the model guesses the correct final answer by chance (probability approximately $1/|N|$).

The authors then assume the one-hop probabilities $p_{ij} = P_1(e^*_{ij}|e_i, r_j)$ are mutually independent (the correlation induced by a shared probability budget across entities is negligible when $|N||R|$ is large). Under this assumption, taking expectations:

E[qijk]=E[pij]2+1−E[pij]∣N∣\mathbb{E}[q_{ijk}] = \mathbb{E}[p_{ij}]^2 + \frac{1 - \mathbb{E}[p_{ij}]}{|N|}

where $q_{ijk}$ is shorthand for the two-hop probability $P_2(e^*_{112}|e_1, r_1, r_2)$. Solving for $\mathbb{E}[p_{ij}]$:

E[pij]=−1+1−4∣N∣(1−∣N∣E[qijk])2∣N∣\mathbb{E}[p_{ij}] = \frac{-1 + \sqrt{1 - 4|N|(1 - |N|\mathbb{E}[q_{ijk}])}}{2|N|}

What this computes: given the observed average two-hop probability $\mathbb{E}[q_{ijk}]$, this formula backs out the implied average one-hop probability $\mathbb{E}[p_{ij}]$ under the assumption that the model is using recurrent composition. This is the "effective" one-hop probability that would be consistent with the observed two-hop performance.

The authors then make a crucial approximation — replacing $\mathbb{E}[\log p_{ij}]$ (which we cannot directly observe) with a function of $\mathbb{E}[p_{ij}]$:

E[log⁡pij]≈log⁡(−1+1−4∣N∣(1−∣N∣eE[log⁡qijk])2∣N∣)\mathbb{E}[\log p_{ij}] \approx \log\left(\frac{-1 + \sqrt{1 - 4|N|(1 - |N| e^{\mathbb{E}[\log q_{ijk}]})}}{2|N|}\right)

where $e^{\mathbb{E}[\log q_{ijk}]}$ approximates $\mathbb{E}[q_{ijk}]$ (exact only when variance in $\log q_{ijk}$ is zero).

The paper provides an empirical justification rather than a formal proof: when $\mathbb{E}[p_{ij}] \approx 1/|N|$ (near-uniform guessing), the variance in $\log q_{ijk}$ is small and the approximation is nearly exact. When $\mathbb{E}[p_{ij}] \gg 1/|N|$ (model is confident), $\mathbb{E}[q_{ijk}] \approx \mathbb{E}[p_{ij}]^2$ and $\mathbb{E}[\log p_{ij}] \approx \mathbb{E}[\log q_{ijk}]/2$, which also holds for the log of the square-root expression. Thus the approximation is good at both extremes.

Why this form: this transforms an unobservable quantity (the model's one-hop accuracy under a composition model) into an observable one (the model's two-hop loss). Without this transformation, we couldn't assess the recurrent hypothesis at all, since models are trained on two-hop data and we never directly observe their one-hop function.

The final information content lower bound is:

C2recurrent≥E2recurrent−∣N∣∣A∣E[log⁡2peffrecurrent]C^{\text{recurrent}}_2 \geq E^{\text{recurrent}}_2 - |N| |A| \mathbb{E}[\log_2 p^{\text{recurrent}}_{\text{eff}}]

where $E^{\text{recurrent}}_2 = E_1$ and $\mathbb{E}[\log_2 p^{\text{recurrent}}_{\text{eff}}]$ is the effective one-hop log-probability computed via the quadratic formula above, applied to the observed two-hop probabilities.

Two-function composition model. For two-function composition, the model has two independent one-hop probabilities: $p^{\text{hop 1}}_{ij}$ for the first hop, and $p^{\text{hop 2}}_{ij}$ for the second hop. The derivation (Appendix A.2) follows a similar structure but introduces an asymmetry parameter.

The two-hop probability decomposes as:

qijk=pijhop 1p(ij)khop 2+1−pijhop 1∣N∣q_{ijk} = p^{\text{hop 1}}_{ij} p^{\text{hop 2}}_{(ij)k} + \frac{1 - p^{\text{hop 1}}_{ij}}{|N|}

where $p^{\text{hop 2}}_{(ij)k}$ is the probability of the second hop being correct given the correct intermediate entity determined by $(i,j)$.

The authors reparameterize with a "budget split" ratio: let $u_{\text{hop 1}} = \mathbb{E}[p^{\text{hop 1}}_{ij}]$, $u_{\text{hop 2}} = \mathbb{E}[p^{\text{hop 2}}_{ij}]$, and $\epsilon = \sqrt{u_{\text{hop 1}} / u_{\text{hop 2}}}$. Define $u = u_{\text{hop 1}} / \epsilon$ so that $u_{\text{hop 1}} = u\epsilon$ and $u_{\text{hop 2}} = u/\epsilon$. The expected two-hop probability becomes:

E[qijk]=E[u2+1−uϵ∣N∣]\mathbb{E}[q_{ijk}] = \mathbb{E}\left[u^2 + \frac{1 - u\epsilon}{|N|}\right]

To minimize $u$ (finding the minimal one-hop probabilities consistent with observed two-hop performance), the authors maximize the right-hand side over $\epsilon$. The minimum feasible $\epsilon$ is either the value that sets $u_{\text{hop 1}}$ to $1/|N|$ (the uniform guessing floor) or that sets $u_{\text{hop 2}}$ to 1 (the confident ceiling). The latter is preferred when $u^2 > 1/|N|$.

What this computes: the effective summed loss $-\mathbb{E}[\log p^{\text{hop 1}}_{ij} + \log p^{\text{hop 2}}_{ij}]$ for the two independent hops, given the observed two-hop loss. This is the quantity needed for the information content formula.

The final information content lower bound is:

C22f≥E22f−∣N∣∣A∣(E[log⁡2peffhop 1]+E[log⁡2peffhop 2])C^{\text{2f}}_2 \geq E^{\text{2f}}_2 - |N| |A| \left(\mathbb{E}[\log_2 p^{\text{hop 1}}_{\text{eff}}] + \mathbb{E}[\log_2 p^{\text{hop 2}}_{\text{eff}}]\right)

Why the separate derivations are necessary. The three computational models make different assumptions about how two-hop accuracy relates to the underlying factual knowledge. The independent model assumes no relationship between one-hop and two-hop — two-hop answers are stored directly. The recurrent model assumes a tight coupling — weak one-hop performance necessarily produces weak two-hop performance. The two-function model assumes an intermediate coupling — the two hops are independent but both must succeed for two-hop to succeed. Without these derivations, we could not use the model's two-hop loss to test the hypotheses, because we could not compute what information content is implied by a given two-hop loss under each hypothesis.

Capacity Scaling Analysis

Once information content is computed for each model (at each parameter count and dataset size) under each hypothesized computational model, the analysis is visual and comparative.

The reference capacity line. The paper uses a 2 bits per parameter reference line (dashed black lines in Figures 1, 2, 3, 5, 8, 9), based on Allen-Zhu & Li (2024)'s finding that saturated transformers can store approximately 2 bits per parameter of factual knowledge. The actual measured one-hop capacity in the main experiments is somewhat lower — approximately 1.6 bits per parameter (Figure 1) — due to the specific hyperparameter configuration (17 relations, µP parametrization). The quantity of interest is not the absolute capacity figure but the relative alignment of the measured curves with the capacity line under different hypotheses.

The "stacked curves" phenomenon. The authors observe a characteristic pattern in their information content scaling plots (best seen in Figure 2). For a given dataset size, as model parameter count increases:

  • Initial phase: The information content is far below the capacity line, near the "baseline" level (which represents the information content of a model that outputs a uniform distribution over answers, only learning the set of names appearing in the dataset). These models have barely begun to learn factual relationships.
  • Takeoff phase: The information content curve bends sharply upward, nearly touching the 2 bits per parameter reference line. This represents models that have crossed a threshold where they can effectively learn the algorithm (whether composition or memorization).
  • Saturation phase: The curve levels off as it approaches the dataset entropy $E$. Models larger than this saturation point cannot encode more information because the data contains no more information — they have achieved (near) zero loss.

Hypothesis testing via curve alignment. The critical comparison is which hypothesis produces curves that align with the 2 bits per parameter reference line. Under the correct hypothesis:

  • Models of different sizes should all fall approximately on or below the capacity line (they cannot exceed the capacity limit).
  • Models that have saturated on the data (achieved near-zero loss) should fall at their dataset's entropy level.
  • The transition from baseline to saturation should approximately track the capacity line (information content grows roughly linearly with parameters until the dataset entropy is reached).

Under incorrect hypotheses, the curves will either systematically exceed the capacity line (implying "more information than physically possible," which indicates the hypothesis overestimates how much information is needed) or fall substantially below it while still achieving good performance (indicating the hypothesis underestimates information requirements — the model is compressing more efficiently than the hypothesis allows).

For example, Figure 2 (Section 3.1 in the prior sections) shows the two-function composition scaling for 4-layer transformers without chain of thought. The curves for different dataset sizes ($N = 1000, 10000, 20000$) align with the 2 bits per parameter reference line: they start near the baseline, rise along the capacity line, and saturate at their respective dataset entropies. This alignment is what the paper presents as evidence for two-function composition.

In contrast, if the independent memorization hypothesis were correct, the required information content would be $|R|$ times larger (since $E^{\text{independent}}_2$ scales with $|R||N|$ rather than $2|N|$). The measured information content would fall far short of the hypothetical entropy, indicating that models achieve good two-hop performance with much less stored information than independent memorization would require — implying they must be composing facts.

Generalization gap as auxiliary evidence. The paper also examines how the difference between training loss and held-out loss (the "generalization gap") varies with model size (Figure 4). In most cases, larger models show larger generalization gaps — they fit the training data better but their held-out performance doesn't improve proportionally. The authors speculate this may reflect larger models allocating some fraction of their capacity budget to inefficient memorization of individual training examples, which doesn't transfer to held-out questions, rather than using all capacity for the generalizing two-function algorithm.

Generalization Experiments

The generalization experiments (Section 3.2, Figures 6 and 7, and Appendix B) serve as an independent behavioral test to validate the capacity scaling inferences. They are not the primary method but provide triangulating evidence.

Metrics. Generalization is measured by comparing the model's evaluation loss on held-out questions to the loss of a uniform-guessing baseline. If the evaluation loss minus uniform-guessing loss is greater than zero (approximately), the model is performing no better than random chance — it has not generalized. If the difference is substantially negative, the model is outperforming random guessing, indicating generalization.

Without chain of thought (Figure 6). For all held-out components (first entities, first relations, second entities, attributes, first-entity–relation pairs, second-entity–attribute pairs), models achieve zero or negligible generalization — the loss difference from uniform guessing is non-negative. This matches the predictions of the two-function composition hypothesis (Table 1), which requires facts to be trained in position (first-hop facts as first hops, second-hop facts as second hops) for generalization.

The exception is held-out complete questions $(e_1, r, a)$, where models do generalize (Figure 4 shows loss approaching zero on these held-out questions, and the generalization gap relative to training loss is small for the largest models). This is also consistent with two-function composition: if both component facts $(e_1, r)$ and $(e_2, a)$ appear in other two-hop questions in training (just not in this specific combination), the model has learned $f_1$ and $f_2$ and can compose them for the novel combination.

With chain of thought (Figure 7). The generalization picture is more complex. Models trained to generate explicit chain-of-thought (first producing the intermediate entity as a token, then the final answer) show:

  • No generalization to held-out first entities in any model.
  • Consistent generalization to held-out entity-attribute pairs in all models.
  • Mixed results for other held-out components (first relations, attributes, second entities) — some training runs generalize, others don't, with no clear pattern.

The authors speculate that inconsistent generalization arises from "competing heuristics" — the model must balance (a) low marginal probabilities of tokens appearing in novel positions (the token for a held-out relation has never appeared as a first-hop token in two-hop questions) against (b) the correct compositional rule. When the correct rule conflicts with token-level statistics, generalization may fail depending on details of initialization and training dynamics, potentially making generalization behavior seed-dependent (as noted in Zhang et al., 2025).

This complexity is important because it shows that capacity scaling (which consistently supported recurrent composition for chain-of-thought models; Figure 3) is not a perfect proxy for generalization — the two methods can give different signals. The paper presents capacity scaling as a complementary tool, not a replacement for generalization testing.

Probing Experiments

Probing (Section 3.4 and Appendix D, Tables 2 and 3) provides a third source of evidence, though the results are largely negative — which itself supports the paper's argument that capacity scaling can be informative where probing fails.

Method. For each trained model, the authors train supervised linear probes on the hidden activations to recover the intermediate entity (the answer to the first hop, "Bob's mother") at different token positions in the input sequence (name tokens, relation tokens, attribute tokens, and all tokens). A probe's performance is measured by its cross-entropy loss in predicting the correct intermediate entity — lower loss means the intermediate entity is more linearly decodable from that layer's activations.

The authors also train control probes that try to recover an arbitrary relation of the first entity (not necessarily the one that leads to the intermediate entity in that specific question). This provides a baseline: if the probe can recover the intermediate entity no better than an arbitrary relation, then the intermediate entity is not specifically encoded in the activations.

Results. Across all models — including models where capacity scaling and generalization both indicated two-function composition — the probes:

  • Achieved similar loss for recovering the intermediate entity as for recovering an arbitrary relation (Tables 2 vs. 3). The differences are small (often 0.1–0.3 nats) and do not consistently favor the intermediate entity.
  • Showed no clear separation between models inferred to use memory vs. models inferred to use two-function composition. The probe loss was similar across both categories.

Why this matters. Wang et al. (2024) found that logit-lens probing could recover intermediate entities from transformers trained on two-hop QA. The current paper's failure to replicate this — even with more powerful supervised linear probes — suggests that the recoverability of intermediate entities depends on training details not present in this setup (possibly related to dataset size, model architecture, or the specific training regime). The negative probing result strengthens the paper's methodological argument: capacity scaling provided consistent evidence for two-function composition even when probing failed to reveal any signature of intermediate entity encoding.

Trapping Small Models in Memorization

The paper includes a targeted experiment (Section 3.3, Figure 5) where they deliberately shift the training data to incentivize memorization over composition, demonstrating that capacity scaling can detect this shift.

Setup. The dataset is modified to have only 4 relations and 4 properties (instead of 17 relations), while maintaining the ratio of 1 one-hop question for every 10 two-hop questions. In this configuration, reducing the loss on an individual two-hop question by a fixed amount has $10/4 = 2.5$ times the impact on total loss as reducing the loss on an individual one-hop question (because each two-hop question involves one relation type out of 4, so performance on that specific relation's two-hop questions is weighted more heavily relative to one-hop performance).

Observed behavior. Models trained in this regime:

  • Fail to learn one-hop question answering beyond uniform guessing — as if they neglect one-hop performance because optimizing one-hop has a smaller impact on the total loss.
  • Exhibit capacity scaling that matches independent memorization, not two-function composition (Figure 5 — the measured information content follows the independent memorization capacity curve, reaching much lower information content for a given model size than the two-function composition curve would predict).
  • Do not generalize to any held-out questions at all.

Interpretation. The authors hypothesize a "trap" mechanism. Transformers "tend to learn simple rules first" (citing Belrose et al., 2024). In this configuration, memorizing individual two-hop answers is a "simpler" and more immediately rewarding rule than learning to compose one-hop functions, because memorizing a two-hop answer directly reduces the loss on that specific high-weight question, whereas learning the one-hop function requires sharing parameters across questions and yields more diffuse loss reduction.

Once the model has committed capacity to memorizing two-hop answers, it reaches a local minimum: to switch to function composition, it would need to un-learn memorized answers (temporarily increasing loss) to free capacity for the one-hop functions. With limited capacity (the paper uses small models, 250K–15M parameters), the model cannot do both. It becomes "trapped" — it would perform better if it could learn to compose, but the optimization path from memorization to composition requires passing through a higher-loss region.

Why this matters. This demonstrates that the algorithm a model learns is not uniquely determined by the task — it depends on the interaction between dataset statistics, model capacity, and optimization dynamics. The capacity scaling method can detect which regime a model has fallen into, even when behavioral outputs (two-hop accuracy on training data) might look similar.

4. Key Insights and Innovations

Innovation 1: Information Capacity Scaling as a Diagnostic Tool for Inferring Transformer Algorithms

This paper introduces a genuinely novel methodological category for interpretability research: using measured information content scaling to discriminate between candidate algorithms a transformer might implement internally. This is distinct from the two dominant interpretability paradigms — behavioral generalization testing and activation probing — and provides a third axis of evidence that can be informative when other methods fail (as probing demonstrably does in this study).

The intellectual move is to treat the model's parameter count as a capacity budget and to ask: given how much information this model has encoded (computed from its loss), which hypothesized algorithm's information requirements best match the observed budget? This reframes algorithm identification from "can we read the algorithm from the weights?" (probing) or "does the algorithm generalize as predicted?" (behavioral testing) to "is the volume of stored information consistent with this algorithm's compression properties?"

Prior to this work, the field had two main approaches for understanding what algorithm a transformer learns: (1) generalization experiments (e.g., Wang et al., 2024; Dziri et al., 2023), which test whether models succeed on held-out combinations and infer the algorithm from the pattern of successes and failures, and (2) probing (Belinkov, 2022) and related activation-based methods (logit lens, sparse autoencoders), which attempt to extract interpretable intermediate quantities directly from hidden states. Both have known limitations. Generalization experiments can falsify hypotheses but cannot uniquely identify algorithms — a model might fail to generalize for reasons unrelated to its internal computation (optimization dynamics, capacity limitations, distributional artifacts). Probing is inherently correlational (decodability ≠ causal use) and requires that the hypothesized intermediate quantity be linearly decodable from activations, which the current paper shows is not guaranteed even when strong independent evidence points to a specific algorithm (Tables 2–3 vs. Figure 2).

Allen-Zhu & Li (2024) established that transformers have a measurable knowledge capacity of ~2 bits per parameter, but they used this purely as a descriptive empirical finding about memorization limits. This paper's innovation is to weaponize that capacity figure as a diagnostic: rather than asking "how much can a transformer memorize?", it asks "does the measured information content under hypothesis X align with the known capacity limit?" If hypothesis X requires more information than the model could possibly store (given its parameter count and the 2 bits/parameter ceiling), yet the model achieves good task performance, hypothesis X must be wrong. If hypothesis Y's predicted information requirements align with the capacity line, it is the better explanation.

The method is hypothesis-driven and quantitative: it requires explicitly formalizing candidate algorithms and deriving their information content formulas (Equations 2–4), then measuring whether the empirical capacity scaling matches. This makes it more principled than ad-hoc generalization tests (which can only test pre-specified held-out splits) but less representationally demanding than probing (which requires linear decodability). It occupies a novel middle ground in the interpretability toolkit.

Significance beyond this paper: the approach is potentially generalizable. Any task where candidate algorithms have distinguishable compression properties could be analyzed this way. The paper explicitly suggests testing whether transformers use "correct multihop reasoning or take shortcuts" by measuring how capacity scaling diverges from the scaling of a hypothesized ideal algorithm. This is a conceptual advance in how to think about what a model has learned — not in terms of readable representations, but in terms of information-theoretic efficiency of the encoded computation.

Evidence anchors: Figure 2 (main results, prior sections) shows capacity scaling for two-hop QA without chain of thought aligning with the 2 bits per parameter reference line under the two-function composition hypothesis, and Figure 5 (memorization trap) shows anomalous capacity scaling under the independent memorization hypothesis for models trained with only 4 relations, demonstrating the method's ability to detect algorithmic differences that probing missed (Tables 2–3).

Innovation 2: The "Each Fact Learned Twice" Hypothesis as an Empirically Grounded Architectural Constraint

The paper's central empirical claim — that transformers learning two-hop QA in a single forward pass must encode each fact twice, once for each hop position — is not merely a description of observed behavior but an architectural necessity argument grounded in the feed-forward nature of transformers. This is a sharper and more mechanistic claim than prior work on transformer compositional limitations.

Earlier theoretical work established that transformers without chain of thought are limited to constant-depth threshold circuits (Merrill et al., 2022) and therefore cannot express general recurrence. But these results are worst-case complexity-theoretic bounds — they say what transformers cannot do in principle, not what they actually do when trained on specific tasks. Prior empirical work (Wang et al., 2024; Dziri et al., 2023) showed that transformers fail to generalize compositionally, but didn't directly measure whether this failure was due to learning the "wrong" algorithm (e.g., memorization) versus being forced to learn a position-dependent variant of the "right" algorithm (two-function composition).

This paper's innovation is to show that the two-function composition model is not a failure of learning but a success under constraint: the models do learn to compose facts (they generalize to held-out complete questions; Figure 4), but they must do so by duplicating information across layer positions because the transformer's feed-forward architecture prevents a single learned function from being applied to both hops. The 2 bits per parameter capacity scaling under the two-function composition hypothesis (Figure 2) is empirical evidence that this duplication is happening — if models were using a recurrent algorithm, the information content would be half as much (Equation 2 vs. Equation 3), and the measured curves would systematically fall below the capacity line.

This reframes the "transformers can't compose" narrative into a more nuanced picture: transformers can compose, but the composition is architecturally expensive. They pay an information-theoretic price — storing each fact twice — that recurrent architectures avoid. This explains why chain of thought helps so dramatically: by externalizing the intermediate entity as an autoregressively generated token, the model can reuse the same function for both hops in successive forward passes, reducing information requirements from 2× to 1× the one-hop cost (Figure 3 shows capacity scaling consistent with recurrent composition for chain-of-thought models).

Why this is distinctive: prior work treated compositional failure as a binary (model generalizes or doesn't). This paper shows there is an intermediate regime — successful composition with a capacity penalty — that is the natural consequence of transformer architecture. It provides a quantitative link between architectural constraints and learning outcomes: the ratio of information content under two-function vs. recurrent composition (~2× for this task) quantifies the "recurrence penalty" that chain of thought eliminates.

Evidence anchors: Figure 2 (capacity scaling for no-chain-of-thought models matches two-function composition), Figure 3 (capacity scaling for chain-of-thought models matches recurrent composition), and the generalization results (Figures 6–7 in the prior sections), which show that without chain of thought, models generalize only when both hops are trained in position — exactly the signature predicted by two-function composition (Table 1).

Innovation 3: The "Memorization Trap" — Dataset Statistics Can Force Suboptimal Algorithms Even When Better Ones Are Within Capacity

The experiment in Section 3.3 (Figure 5) demonstrates a phenomenon with implications beyond two-hop QA: transformers can become trapped in a local minimum where they learn a suboptimal, non-generalizing algorithm even when they have sufficient capacity to learn the generalizing one. This is a finding about optimization dynamics and algorithmic selection, not about fundamental capacity limits.

The mechanism is specific and testable. When the training data is configured so that memorizing individual two-hop answers has a larger immediate impact on the loss than learning one-hop functions (achieved by reducing the number of relation types from 17 to 4, making each two-hop question type a larger fraction of the total loss), the model learns to memorize first — because memorization is "simpler" and reduces loss faster on the most heavily weighted examples. By the time memorization saturates the model's limited capacity, switching to function composition would require unlearning memorized answers (increasing loss) to free parameters for the one-hop functions. The model cannot escape this local minimum.

This is conceptually distinct from standard capacity saturation. In standard saturation, a model simply runs out of parameters and cannot learn more, but what it has learned may still be the "correct" algorithm just incompletely implemented. Here, the model has sufficient capacity to implement the generalizing algorithm (it could achieve lower overall loss by composing), but the optimization path from its current state to that better algorithm involves a loss barrier it cannot cross. The algorithm the model learns is path-dependent, not just capacity-constrained.

This finding connects to broader observations about "grokking" and simplicity biases in neural network training (the paper cites Belrose et al., 2024 on neural networks learning statistics of increasing complexity). It suggests that the algorithm a model implements is not determined solely by the task, the architecture, and the capacity — it is also a function of the relative loss weighting of different sub-tasks, which shapes the early optimization trajectory. This has practical implications for training data design: if you want a model to learn compositional reasoning, you may need to ensure that the component facts are sufficiently loss-weighted relative to the composed facts, or the model may skip directly to memorizing compositions.

Why this is a significant negative result: it shows that a model can perform reasonably well on training data (high two-hop accuracy) while implementing entirely the wrong algorithm (independent memorization), and this wrong algorithm will catastrophically fail to generalize. Training loss alone cannot distinguish the algorithms. The capacity scaling method can detect the difference (Figure 5 shows anomalous low information content), but standard training metrics would miss it. This reinforces the paper's argument that we need diagnostic methods beyond loss curves.

Evidence anchors: Figure 5 (capacity scaling under independent memorization hypothesis aligns with the 2 bits/parameter line when models are trained with 4 relations, indicating they have memorized answers rather than learned composition), and the associated generalization results (these models fail to generalize to any held-out questions, confirming the non-compositional algorithm).

Innovation 4: A Principled Reconciliation of Conflicting Probing Results in Multi-Hop Reasoning

The paper's negative probing result — failure to recover intermediate entities even when strong independent evidence (capacity scaling + generalization) indicates two-function composition — is not just a methodological failure. It is a substantive finding about the limits of probing as an interpretability tool, and it helps explain a tension in recent literature.

Wang et al. (2024) found that logit-lens probing could recover intermediate entities from transformers trained on two-hop QA, providing evidence for the two-function composition hypothesis. Yu (2025), studying multi-step arithmetic, found that intermediate steps were often non-recoverable with probing in prompted models (though fine-tuned models allowed recovery). These results appear contradictory — does probing work for multi-hop reasoning or not?

This paper provides a resolution: probing success depends on factors not controlled in these studies, possibly including dataset size, model scale, training duration, or the specific tokenization and architecture choices. The current paper's models, trained with a custom small-vocabulary tokenizer, µP parametrization, and schedule-free optimization on synthetic data with 17 relation types, did not produce linearly decodable intermediate entities despite clearly implementing two-function composition (as evidenced by both generalization and capacity scaling). This suggests that the decodability of intermediate entities is a contingent property of the training setup, not a necessary signature of any particular algorithm.

The implication is methodological: probing alone cannot reliably determine whether a model is using compositional reasoning, because the same algorithm may or may not produce decodable intermediate representations depending on implementation details. This is a cautionary tale for the interpretability community, which has often treated probing as a gold-standard method for identifying internal algorithms. A negative probing result does not rule out compositional reasoning; a positive probing result may not uniquely identify it (probes could exploit correlated features).

Why this is more than a null result: it motivates the paper's central methodological contribution. If probing were universally reliable, there would be no need for capacity scaling as a complementary method. The fact that probing fails here, while capacity scaling provides results consistent with generalization behavior, makes the case that multiple independent diagnostic methods should be used to triangulate algorithmic hypotheses, with capacity scaling filling a gap where probing falls short.

Evidence anchors: Tables 2 and 3 (probe losses for intermediate entity vs. arbitrary relation are similar across all models, including those where Figure 2 and generalization results clearly indicate two-function composition), contrasted with Wang et al. (2024)'s positive probing results in a different experimental setup.

Innovation 5: Quantifying the Efficiency Gain of Chain-of-Thought as an Information-Theoretic Reduction

The paper's chain-of-thought experiment (Figure 3) provides a clean quantitative measurement of why chain of thought helps compositionally: it reduces the information that must be stored in the model's weights from ~2× the one-hop cost (two-function composition) to ~1× (recurrent composition). This is a more precise explanation than the standard "chain of thought enables step-by-step reasoning" narrative.

Prior work established that chain of thought improves compositional reasoning accuracy and makes transformers Turing-complete (Pérez et al., 2021). But the mechanism was typically described in computational expressiveness terms: chain of thought allows intermediate computation to be serialized across tokens, overcoming the constant-depth limitation of single-forward-pass transformers.

This paper adds an information-theoretic dimension: chain of thought doesn't just enable new computations — it makes existing computations dramatically more parameter-efficient. A model that can externalize the intermediate entity as a generated token can reuse the same factual lookup function for both hops, halving the required parameter count for a given level of accuracy (or, equivalently, doubling the effective capacity for a given parameter budget). Figure 3 shows capacity scaling under the recurrent composition model for chain-of-thought models, confirming they achieve near one-hop-equivalent information content.

This has practical implications the paper notes in its discussion: the strong performance of small distilled "reasoning models" (like those in DeepSeek-AI et al., 2025) may be partly attributable to this parameter-efficiency gain — by relying heavily on chain of thought, these models can store factual knowledge once and reuse it across compositional steps, fitting more effective capability into fewer parameters than would be possible if the same knowledge had to be duplicated for each reasoning depth in a single forward pass.

Why this is distinctive: it translates "chain of thought helps" from a qualitative observation into a quantitative, information-theoretic prediction: the capacity scaling should shift from 2× to 1× the one-hop information requirement. The paper validates this prediction (Figure 3), providing a mechanistic explanation for efficiency gains that complements the standard expressiveness argument.

Evidence anchors: Figure 3 (chain-of-thought capacity scaling aligns with recurrent composition, showing information content near one-hop levels), contrasted with Figure 2 (no-chain-of-thought scaling aligns with two-function composition, requiring ~2× the information).

5. Experimental Analysis

Evaluation Methodology

  • Dataset. All experiments use synthetic two-hop question-answering datasets generated by the authors' profile-and-template pipeline (Section 2.1). Dataset sizes range from |N| = 1000 to 250,000 fictional profiles, each with 17 relations and 4 properties (21 attributes total). Training data consists of template-generated English text strings: one-hop questions of the form "What was {person}'s {relation/attribute}? {answer}" and two-hop questions of the form "What was {person}'s {relation}'s {relation/attribute}? {answer}". Every fact appears as a one-hop question in training; the ratio of two-hop to one-hop questions is 10:1 unless specified otherwise. Seven distinct held-out sets are created by systematically excluding specific components (first entities, first relations, second entities, attributes, entity-relation pairs, entity-attribute pairs, or complete questions) from the two-hop training data while keeping all facts in one-hop training.

  • Base model(s). All models are Llama-architecture transformers (Dubey et al., 2024) ranging from approximately 250K to 15M parameters. Both 4-layer and 12-layer variants are tested, with 4-layer models as the primary focus. Models use µP parametrization (Yang et al., 2022) for consistent hyperparameter transfer across widths, a custom tokenizer trained on the synthetic data with a vocabulary of 3,000 tokens (to prevent embedding parameters from dominating at small scales), and schedule-free AdamW optimization (Defazio et al., 2024) to avoid manual learning rate schedule tuning.

  • Metrics. The primary metric is information content (in bits), computed by subtracting the model's total cross-entropy loss over all answer tokens from the dataset entropy (Section 2.3, Equations 5–8). Dataset entropy is calculated exactly from the known data generating process (Equations 1–4). For two-hop models, the information content formula depends on the hypothesized computational model: independent memorization uses Equation 6, recurrent composition uses Equation 7 with effective one-hop probabilities derived in Appendix A.1, and two-function composition uses Equation 8 with the two-hop effective loss derived in Appendix A.2. For generalization experiments, the metric is evaluation loss minus uniform-guessing loss on held-out questions, where values ≤ 0 indicate no generalization (the model performs no better than random). For probing experiments, the metric is cross-entropy loss of linear probes trained to recover intermediate entities from hidden activations.

  • Baselines. The paper uses three reference points for information content scaling: (1) Dataset entropy — the total information in the training data, which represents the theoretical maximum content a perfect model would encode; (2) Baseline information content — the content level achieved by a model that outputs a uniform distribution over all possible answers, representing the minimum information a model must encode (primarily learning the set of names appearing in the dataset, contributing |N| log₂ N₀ bits); and (3) Estimated capacity — the 2 bits per parameter reference line derived from Allen-Zhu & Li (2024), representing the empirical upper limit on factual storage for saturated transformers. For generalization experiments, the baseline is uniform guessing — the loss of a model that assigns equal probability to all possible answers for each held-out question.

  • Generation budget / compute accounting. The paper does not use a "generation budget" in the conventional sense (there is no test-time search or sampling). All models are trained to convergence (loss decrease < 10⁻⁸ per step, typically ~10M steps or ~160B tokens) and evaluated on their training-set performance for information content measurement and on held-out sets for generalization. "Compute" is measured indirectly through model parameter count — the independent variable in all scaling analyses is the number of model parameters, not training FLOPs or tokens, since the paper investigates the relationship between model capacity (parameters) and information storage capacity (bits).

  • Cross-validation / statistical protocol. The paper does not employ formal cross-validation or statistical significance testing. The generalization experiments test models on systematically constructed held-out sets rather than random splits. The capacity scaling analysis relies on visual alignment of information content curves with the 2 bits per parameter reference line — there is no quantitative goodness-of-fit metric reported. The authors train models at multiple parameter counts for each dataset size, producing scaling curves whose interpretation is qualitative (do the curves rise along the capacity line and saturate at the dataset entropy?). The probing experiments report raw losses in Tables 2 and 3 without confidence intervals. The paper notes that chain-of-thought generalization behavior may be seed-dependent (citing Zhang et al., 2025), but does not report multiple training runs with different random seeds to quantify this variability.

Main Quantitative Results

One-Hop Information Capacity (Figure 1, Figure 8)

The paper measures the one-hop information capacity to establish a baseline for the two-hop analysis and to partially replicate Allen-Zhu & Li (2024)'s finding of ~2 bits per parameter.

Main result (Figure 1): For 4-layer transformers trained on one-hop QA with 17 relations and µP parametrization, the observed information capacity is approximately 1.6 bits per parameter, below the 2 bits per parameter reference line. The curve for |N| = 24,000 profiles shows information content rising from near the baseline at ~1.5 × 10⁶ parameters to approximately 0.75 × 10⁷ bits at ~5.5 × 10⁶ parameters, roughly paralleling the 2 bits per parameter line but consistently below it.

Earlier runs (Figure 8): Training runs conducted before the main experiments, using a different configuration (4 relations instead of 17, and no µP parametrization), produced capacities closer to 2 bits per parameter. For |N| = 15,000, the information content reaches approximately 8 × 10⁶ bits at ~5 × 10⁶ parameters, nearly exactly on the 2 bits per parameter line. However, the trend is "less consistent than the main results" — some dataset size curves cross the reference line or show irregular scaling (the |N| = 20,000 curve, for instance, shows nearly flat scaling from 1–2 × 10⁶ parameters before rising sharply).

Interpretation: The paper does not attempt to explain the gap between 1.6 and 2.0 bits per parameter, but notes that it introduces uncertainty into the capacity reference. The absolute capacity figure is less important than the relative alignment of different hypotheses — the key comparisons are whether two-function composition aligns with the observed scaling while independent memorization or recurrent composition are far from it, regardless of whether the absolute ceiling is 1.6 or 2.0.

Two-Hop Capacity Scaling Without Chain of Thought (Figure 2, Figure 9)

Headline finding: For 4-layer transformers trained on two-hop QA without chain of thought, information content scaling under the two-function composition hypothesis aligns with the estimated capacity line, while the recurrent composition and independent memorization hypotheses do not fit the data — the former would require substantially less information (and the measured content would exceed the capacity line if plotted under that hypothesis, since models achieve lower loss than recurrent composition would predict for their parameter count), and the latter would require |R| = 17 times more information (and measured content would fall far short of the dataset entropy even when models achieve near-zero loss, indicating they are not storing that much information independently).

Quantitative description of Figure 2 (two-function composition): For three dataset sizes (|N| = 1000, 10000, 20000), the information content curves exhibit the characteristic "stacked" pattern:

  • At small parameter counts (~3 × 10⁵ for |N| = 1000, ~8 × 10⁵ for |N| = 10,000, ~1.2 × 10⁶ for |N| = 20,000), information content is near the baseline — models have learned little beyond the set of names.
  • As parameters increase, the curves bend upward sharply, approaching the 2 bits per parameter reference line. For |N| = 1000, the content reaches approximately 6 × 10⁶ bits at ~3.5 × 10⁶ parameters, just below the reference line. For |N| = 10,000, the content reaches approximately 2 × 10⁷ bits at ~1.1 × 10⁷ parameters, roughly on the reference line.
  • At larger parameter counts, the curves level off near their respective dataset entropies. For |N| = 1000, the curve saturates at approximately 1.5 × 10⁷ bits (the dashed entropy line, slightly variable due to randomly held-out attributes differing between datasets). For |N| = 20,000, the curve approaches approximately 3 × 10⁸ bits at the largest tested parameter count (~5.5 × 10⁶), though the model has not fully saturated — the curve is still slightly below the entropy line.

12-layer models (Figure 9): A comparison of 4-layer and 12-layer models at |N| = 10,000 shows "similar but less consistent" results for the deeper architecture. The 12-layer curve follows the 2 bits per parameter reference line at small scales (~5 × 10⁵ to 2 × 10⁶ parameters) but then shows an anomalous dip — information content actually decreases as parameters increase from ~3 × 10⁶ to ~1.1 × 10⁷ — before recovering. The paper attributes this inconsistency to optimizer hyperparameters that were tuned on the 4-layer model and not readjusted for the deeper architecture.

Alternative hypotheses (implied, not plotted directly): The paper does not show a figure with information content computed under the recurrent or independent models for the same no-chain-of-thought data. However, the logic is straightforward: if information content were computed under the recurrent model, the effective one-hop probabilities derived from the two-hop loss via Appendix A.1 would be higher than under two-function composition (since the recurrent model assumes more efficient use of learned facts). The resulting information content would exceed the 2 bits per parameter capacity line — models would appear to have stored more information than their parameter count could physically support. Under the independent memorization model, the dataset entropy E_independent would be ~|R| = 17 times larger than E_2f, so even models achieving near-zero two-hop loss would show information content far below their dataset entropy — inconsistent with the observation that models do approach their entropy limits at large parameter counts.

Chain-of-Thought Capacity Scaling (Figure 3)

Headline finding: When models are trained to generate explicit chain-of-thought (producing the intermediate entity as a token before the final answer), the information content scaling under the recurrent composition hypothesis approaches the capacity line, consistent with the idea that chain of thought enables efficient single-copy factual storage by allowing the model to reuse the same function across successive forward passes.

Quantitative description of Figure 3: For a single dataset size (|N| = 10,000) with 4-layer transformers, the information content computed under the recurrent composition model starts below the estimated capacity for two-function composition (~1 × 10⁷ bits at ~2.8 × 10⁶ parameters) and rises sharply, exceeding the two-function capacity estimate at ~3.5 × 10⁶ parameters and approaching the estimated capacity for recurrent composition by ~5 × 10⁶ parameters (reaching approximately 1.8 × 10⁷ bits). The curve continues upward to approximately 2.5 × 10⁷ bits at ~7 × 10⁶ parameters, remaining roughly aligned with the recurrent capacity reference.

The paper notes that the information content "approaches the capacity for recurrent composition, as expected" — the model achieves substantially higher effective information density (more task capability per parameter) than the no-chain-of-thought case, because each fact is stored only once and reused across hops via the autoregressive generation of intermediate entities.

Caveat: Only a single dataset size (|N| = 10,000) is shown for chain-of-thought capacity scaling, and only a limited range of parameter counts (from ~2 × 10⁶ to ~7 × 10⁶). This is substantially less evidence than the no-chain-of-thought case (three dataset sizes, wider parameter range), making the chain-of-thought capacity result more preliminary.

Generalization Without Chain of Thought (Figure 6, Figure 4)

Headline finding: Transformers trained on two-hop QA without chain of thought completely fail to generalize when any component of the first or second hop is systematically held out from two-hop training, but do generalize when complete questions are held out — exactly matching the predictions of the two-function composition hypothesis (Table 1).

Quantitative description of Figure 6 (held-out components): The figure plots evaluation loss minus uniform-guessing loss for six held-out conditions (Entity 1, Entity 1-Relation Pairs, Relations, Attribute, Entity 2, Entity 2-Attribute Pairs) across three dataset sizes (|N| = 1000, 10000, 20000) and a range of model sizes (up to ~1.5 × 10⁷ parameters). The key observation:

  • In all six panels, the y-axis values are approximately 0 or positive (above the "no generalization" threshold), indicating that models do not outperform random guessing on any held-out component split.
  • For some conditions (e.g., held-out Entity 2 for |N| = 20000), the loss difference is negative but extremely small (roughly −2 to −5), compared to training losses that would be tens of nats — effectively zero generalization.
  • There is no trend toward better generalization with larger model size — even models at ~10⁷ parameters remain at or near the uniform-guessing baseline.

Quantitative description of Figure 4 (generalization gap for complete questions): The figure plots the difference between evaluation loss on held-out complete questions and training loss (the "generalization gap") as a function of parameter count for |N| = 1000, 10000, 20000:

  • For |N| = 1000, the gap starts at approximately 0.8 at the smallest model (~2 × 10⁵ parameters), rises to ~1.0 at ~5 × 10⁵ parameters, then drops sharply — the largest model for |N| = 1000 (approximately 1.4 × 10⁷ parameters) achieves a gap of nearly 0 (evaluation loss ≈ training loss). This model has achieved near-perfect generalization on held-out complete questions.
  • For |N| = 10000, the gap increases from approximately 0.2 at ~3 × 10⁵ parameters to ~1.3 at ~1.3 × 10⁷ parameters — larger models show larger generalization gaps, suggesting they fit the training data better without proportional improvement on held-out questions.
  • For |N| = 20000, the gap increases from ~0.3 at ~5 × 10⁵ parameters to ~0.8 at ~5 × 10⁶ parameters.

The paper interprets the growing generalization gap at larger |N| as evidence that larger models may "allocate some fraction of their information budget to memorization," which helps on training data but does not transfer to held-out complete questions that require true composition.

Generalization With Chain of Thought (Figure 7)

Headline finding: Generalization with chain of thought is inconsistent — some held-out splits show generalization in some models but not others, without a clear pattern — contrasting with the clean, consistent results for no-chain-of-thought models.

Quantitative description of Figure 7: For a single dataset size (|N| = 10,000) and model sizes from ~10⁶ to ~6 × 10⁶ parameters:

  • Entity 1 (held-out first entities): All models show loss difference ≥ 0 (no generalization). The largest model (~6 × 10⁶ parameters) is at approximately 0 — right at the uniform-guessing threshold.
  • Entity 1-Relation Pairs: Mixed results. The largest model shows loss difference of approximately −10 (clear generalization). Smaller models (~3–4 × 10⁶ parameters) show values near 0 or slightly positive (no generalization).
  • Relations: Mixed. One model at ~5 × 10⁶ parameters achieves a loss difference of approximately −30 (strong generalization), while another at similar scale shows approximately −5. A smaller model at ~3 × 10⁶ parameters shows +5 (worse than random).
  • Attribute: Similar mixed pattern, with some models generalizing (loss differences of −10 to −20) and others not (near 0 or positive).
  • Entity 2: One model at ~5 × 10⁶ parameters achieves approximately −50 (very strong generalization), while another at ~4 × 10⁶ parameters is at approximately −5. Other models are at or near 0.
  • Entity 2-Attribute Pairs: All models show clear negative loss differences (approximately −10 to −60), indicating consistent generalization across all model sizes for this held-out split.

The paper speculates that this inconsistency arises from "competing heuristics" — the model must output tokens (intermediate entities) that have never appeared in that position during training (for held-out first entities, relations, etc.). Low marginal probabilities of tokens in novel positions conflict with the correct compositional rule, and the resolution of this conflict may depend on initialization and training dynamics in a seed-dependent way. The paper did not train multiple seeds to quantify this variability.

Independent Memorization Trap (Figure 5)

Headline finding: Transformers trained with only 4 relations (instead of 17) exhibit capacity scaling that matches independent memorization of two-hop answers, fail to learn one-hop question answering beyond uniform guessing, and completely fail to generalize to any held-out questions — a demonstration that dataset statistics can trap models in a suboptimal, non-generalizing local minimum.

Quantitative description of Figure 5: For two dataset sizes (|N| = 10,000 and |N| = 15,000), information content is computed under the independent memorization hypothesis:

  • For |N| = 10,000, the curve starts near the baseline at ~1.2 × 10⁶ parameters, rises along the estimated capacity line for two-function composition (but the independent model hypothesis, so the relevant reference is the independent model's capacity), and saturates at the independent memorization dataset entropy at approximately 4 × 10⁷ bits — well below the entropy for two-function composition.
  • For |N| = 15,000, the curve follows a similar trajectory, saturating at approximately the independent memorization entropy at ~5 × 10⁶ parameters.

The key interpretive point: the measured information content under the independent model aligns with the capacity line (the dotted "est. capacity" reference), not with the two-function composition capacity estimate. This means the models are storing information at the rate expected for memorizing each two-hop answer independently, not the lower rate expected for composing functions.

The paper states that these models "do not learn one-hop question answering much beyond the uniform distribution on possible answers," confirming that they have neglected the one-hop function in favor of directly memorizing two-hop outputs. They also "do not generalize to any held out questions at all," consistent with the hash table model (Table 1).

Mechanism (from Section 3.3): With only 4 relations, reducing the loss on an individual two-hop question has 10/4 = 2.5× the impact on total loss as reducing the loss on an individual one-hop question (since there are 10 two-hop questions per one-hop question, but each two-hop question's relation type appears in only 1/4 of two-hop examples, making each two-hop example a larger fraction of the loss for its specific relation). The model learns to memorize two-hop answers first (because it reduces loss faster), and once its limited capacity is saturated with memorized answers, it cannot escape this suboptimal local minimum to learn the more efficient compositional algorithm.

Probing Results (Tables 2 and 3, Appendix D)

Headline finding: Supervised linear probes fail to recover intermediate entities from transformers' hidden activations at a level significantly above chance, even in models where capacity scaling and generalization both clearly indicate two-function composition. The intermediate entity is only marginally more decodable than an arbitrary relation of the first entity.

Quantitative description of Tables 2 and 3: The tables report probe cross-entropy losses for recovering the intermediate entity (Table 2) versus an arbitrary relation of the first entity (Table 3), across four model sizes (890K, 2.2M, 6.4M, 13.6M parameters) and two configurations: the "memorization trap" setup with 4 relations (QA loss 3.16, inferred algorithm: memorization) and the standard setup with 17 relations (varying QA losses from 8.18 down to 0.19, all inferred as two-function composition, "2FC").

Key observations:

  • Uniformed predictor baseline: The loss of a probe that always predicts the uniform distribution over answers is 6.47 — lower than the typical QA loss because probing is evaluated only on the first token of the relevant name.
  • Best probe performance: The lowest probe loss achieved is 3.67 for name-token position in the 13.6M-parameter two-function composition model (Table 2, "Name Loss" column). This is substantially above zero (perfect recovery), and only 0.09 lower than the corresponding loss for recovering an arbitrary relation (3.76 in Table 3 for the same model and position).
  • Memorization vs. composition models: There is no clear separation. The memorization model (890K, 4 relations) achieves name-position losses of 5.89 (intermediate entity probe) vs. 5.89 (arbitrary relation probe) — nearly identical. The largest two-function composition model (13.6M, 17 relations, QA loss 0.19) achieves 3.98 vs. 4.07. In both cases, the intermediate entity is only slightly more decodable than an arbitrary relation, and the difference is comparable in the memorization and composition models.
  • Token position effects: Probes trained on relation tokens or attribute tokens consistently perform worse than probes trained on name tokens or all tokens. The "All Loss" column (probes trained on activations at all previously mentioned token positions) shows the strongest performance overall, with the 13.6M-parameter two-function composition model achieving 4.04 for intermediate entity vs. 4.29 for arbitrary relation (Table 2 vs. Table 3, "All Loss") — a gap of only 0.25.

Contrast with Wang et al. (2024): Wang et al. (2024) used the logit lens (applying the unembedding matrix directly to hidden states without training) and successfully recovered intermediate entities. The current paper's supervised linear probes should be at least as powerful as the logit lens (since they are trained specifically on the hidden layer activations), yet they fail to find a strong signal. The paper does not investigate why, but the discrepancy suggests that the decodability of intermediate entities depends on factors not controlled in either study — potentially including model scale, training data composition, or architecture details.

Ablation Studies and Robustness Checks

Number of relations (17 vs. 4): The primary parameter varied between the main two-function composition experiments (17 relations) and the memorization trap experiments (4 relations). This ablation demonstrates that the algorithm learned is not an invariant property of the two-hop task — it depends on the relative loss weighting of one-hop versus two-hop questions, which changes with the number of relations. With 17 relations, each two-hop question's relation type appears in only 1/17 of two-hop examples, so optimizing any individual two-hop example has a smaller relative impact on total loss, and the model is forced to learn the shared one-hop functions to make progress. With 4 relations, each two-hop example is a larger fraction of its relation-specific loss, incentivizing direct memorization. Figure 5 versus Figure 2 shows the resulting algorithmic difference: independent memorization scaling versus two-function composition scaling.

Model depth (4 layers vs. 12 layers): Figure 9 compares information content scaling for 4-layer and 12-layer models at |N| = 10,000 under the two-function composition hypothesis. The 12-layer models show "similar but less consistent" scaling, with an anomalous dip in information content at intermediate parameter counts (~3–11 × 10⁶ parameters) where the 12-layer curve falls substantially below the 4-layer curve and the capacity reference line. The paper attributes this to optimizer hyperparameters (tuned on the 4-layer architecture and not readjusted), making this a confounded ablation — the depth comparison cannot be interpreted cleanly because the training procedure was not independently optimized for each depth. Allen-Zhu & Li (2024) reported that information capacity did not seem to depend on architecture, and this paper's result is at least not inconsistent with that finding (the deepest models may simply be undertrained or suboptimally tuned).

µP parametrization (used in main experiments vs. not used in earlier runs): Figure 8 (earlier runs without µP) shows one-hop capacity measurements closer to 2 bits per parameter, while Figure 1 (main results with µP) shows ~1.6 bits per parameter. This is not presented as a controlled ablation — the earlier runs also used 4 relations instead of 17 — but the paper notes the hyperparameter difference as a factor in the capacity discrepancy. Why µP would reduce measured capacity is not explained; it may be that µP's default hyperparameters were not optimal for this specific task, or that schedule-free AdamW interacts differently with µP than with standard parametrization.

Chain of thought vs. no chain of thought: The comparison between Figure 2 (no chain of thought, two-function composition scaling at approximately 2× the one-hop information requirement) and Figure 3 (chain of thought, recurrent composition scaling at approximately 1×) is the most important ablation in the paper, demonstrating the information-theoretic efficiency gain of chain of thought. However, it is weakened by the limited data: only one dataset size (|N| = 10,000) is shown for chain of thought, compared to three sizes for no chain of thought, and the parameter range is narrower (~2–7 × 10⁶ vs. ~3 × 10⁵ – 1.5 × 10⁷). This makes the chain-of-thought capacity story less empirically robust than the no-chain-of-thought story.

Probing as a robustness check for capacity scaling inferences: The probing experiments (Tables 2–3) serve as a robustness check on the relationship between capacity scaling and representation-level properties. The fact that probes fail to recover intermediate entities in models where capacity scaling strongly indicates two-function composition demonstrates that capacity scaling can be informative when probing is not — the two methods provide different, potentially complementary information about the model's internal algorithm.

Critical Assessment

Does information capacity scaling genuinely support the "two-function composition" hypothesis, or does it support something weaker?

The paper's central claim is that capacity scaling under the two-function composition hypothesis aligns with the 2 bits per parameter reference line, and therefore transformers learn two-hop QA by storing each fact twice. The evidence for this is primarily visual: the curves in Figure 2 track the reference line better than they would under alternative hypotheses.

However, several factors weaken this conclusion:

First, the capacity reference is uncertain. The main experiments find ~1.6 bits per parameter for one-hop (Figure 1), while earlier runs found ~2.0 (Figure 8). To conclude that two-function composition is the "right" hypothesis, we need confidence in the reference capacity. If the true capacity is 1.6 bits per parameter, then the curves in Figure 2 are actually above the capacity line at intermediate parameter counts (the |N| = 1000 curve at ~3.5 × 10⁶ parameters reaches ~6 × 10⁶ bits, which is ~1.7 bits per parameter — slightly above 1.6). If the true capacity is 2.0, the curves are slightly below. The paper does not provide a principled way to determine the reference capacity for a given experimental setup, which makes the alignment argument inherently fuzzy — one could adjust the reference line within the 1.6–2.0 range to better fit whichever hypothesis one prefers.

Second, the hypothesis testing is asymmetric. Only Figure 2 (two-function composition) is shown for the no-chain-of-thought data. We do not see the same data plotted under the recurrent or independent hypotheses, which would allow direct visual comparison of which hypothesis produces curves closest to a straight capacity line. The paper's argument that alternative hypotheses "wouldn't fit" is presented discursively rather than empirically. For example, a figure showing the same models' information content computed under the recurrent hypothesis (which would presumably exceed the capacity line, indicating that the recurrent model overestimates compression efficiency) would be a much stronger demonstration than verbal argument.

Third, the "stacked curve" phenomenon is not clean. The scaling curves in Figure 2 are not simply straight lines that bend at the capacity limit — they show complex curvature (a gradual S-shape from baseline to saturation). This curvature is attributed to models learning the composition algorithm only after reaching some threshold capacity, and possibly allocating some budget to memorization as they approach saturation. While these explanations are plausible, they mean the alignment with the capacity line is approximate and qualitative rather than quantitative — the curves do not have a simple functional form that unambiguously picks out one hypothesis.

Fourth, only three dataset sizes are tested (|N| = 1000, 10000, 20000 for Figure 2; plus |N| = 10000 and 15000 for the memorization trap in Figure 5). The paper's claim is about scaling — the relationship between parameters and information content as both vary — but the parameter range (250K to 15M, a factor of ~60) and the dataset size range (1000 to 20000, a factor of 20) are relatively narrow. A stronger demonstration would require more dataset sizes and a wider parameter range to establish that the alignment with the capacity line holds robustly across scales.

In summary: The experiments demonstrate that two-function composition is a better fit to the scaling data than the alternatives would be, but this is a comparative claim supported more by elimination of implausible alternatives than by positive identification. The independent memorization hypothesis would require ~17× more information than is observed (models achieve high accuracy while encoding far less than E_independent). The recurrent hypothesis would require the model to compress information more efficiently than the architecture physically allows. Two-function composition is the remaining plausible candidate. This is valid abductive reasoning but is weaker than a direct positive measurement would be.

Do the generalization experiments support the capacity scaling inferences?

Yes, with qualifications. The clean generalization pattern for no-chain-of-thought models — zero generalization on any held-out hop component, but successful generalization on held-out complete questions — matches the two-function composition predictions exactly (Table 1). This triangulation is the strongest evidence in the paper: two independent methods (capacity scaling and generalization) converge on the same conclusion.

The chain-of-thought generalization results (Figure 7) are less clean. While capacity scaling suggests recurrent composition for chain-of-thought models (Figure 3), the generalization behavior is inconsistent — some held-out splits show generalization in some models but not others. The paper's interpretation (competing heuristics, possible seed dependence) is plausible but not empirically validated (no multi-seed experiments are reported). This inconsistency does not contradict the capacity scaling inference, but it fails to provide the same clean triangulation that the no-chain-of-thought results provide. The capacity scaling method's claim to be "complementary to generalization" is supported by the no-chain-of-thought case (both agree), but the chain-of-thought case shows a divergence whose explanation remains speculative.

Does the "memorization trap" experiment show what it claims?

The claim is that small models can be "trapped" in a local minimum where they memorize two-hop answers independently, even though they have the capacity to learn the more efficient two-function composition algorithm.

Strengths: The capacity scaling under the independent memorization hypothesis (Figure 5) aligns with the capacity reference, and these models fail to generalize (consistent with the hash table model in Table 1). This is a clear algorithmic signature that differs from the main results (Figure 2), and it demonstrates that the capacity scaling method can detect algorithmic shifts.

Weaknesses: The claim that these models could have learned composition is not directly tested. The paper argues that "in principle they have the capacity to perform substantially better if they learned to generalize," but this counterfactual is not demonstrated — no experiment shows that a model of the same size, trained on the same data with a different optimization procedure (e.g., different loss weighting, curriculum learning, or a different optimizer), does learn to generalize. The "trap" interpretation relies on the assumption that the local minimum is escapable in principle but not in practice with the specific training setup, but the escape is never demonstrated. This is a hypothesis about optimization dynamics, not a proven mechanism.

Additionally, the experiment confounds the number of relations (4 vs. 17) with the loss weighting. With 4 relations, the ratio of two-hop loss impact to one-hop loss impact is 2.5:1. With 17 relations, it is 10/17 ≈ 0.59:1 — not just different, but reversed in sign. It is possible that the 4-relation models fail to learn one-hop QA not because of a "trap" but because the one-hop loss is simply too small relative to the two-hop loss for the optimizer to allocate meaningful capacity to it. This would be a straightforward optimization issue (insufficient gradient signal) rather than a local-minimum trap. The paper's "trap" narrative requires that the model initially learns memorization, saturates capacity, and then cannot switch — but training dynamics are not analyzed directly (e.g., by plotting one-hop accuracy vs. two-hop accuracy over training steps to see if one-hop ever improves before plateauing).

Probing failure: a genuine finding or a limitation of the specific probes?

The probing experiments (Tables 2–3) convincingly show that supervised linear probes cannot strongly recover intermediate entities in these models. However, the paper's conclusion that this is a substantive finding about probing's limits — rather than a failure to find the right probe architecture — rests on the assumption that supervised linear probes should be able to recover the information if it is present.

This assumption is defensible but not watertight. Linear probes are the standard tool for recovering linearly decodable information from hidden states, and they are strictly more powerful than the logit lens (which Wang et al., 2024 used successfully). If the intermediate entity is not linearly decodable even with supervised training on the specific activation layer and token position, it is reasonable to conclude that the information is not encoded in a linearly accessible format.

However, the paper does not explore nonlinear probes (e.g., MLP probes) or probes trained on combinations of layers. It is possible that the intermediate entity is encoded in a nonlinear or distributed fashion that a linear probe cannot recover but that the model's own computation uses. The paper's interpretation — that probing fails where capacity scaling succeeds — would be stronger if it demonstrated that even more powerful probes (nonlinear, cross-layer) also fail, or that the linear probes are operating near the Bayes-optimal error rate for the task (which would rule out the possibility that a better probe architecture would succeed).

Missing experiments that would strengthen the paper

Multi-seed training for chain-of-thought generalization: The inconsistent generalization results for chain-of-thought models (Figure 7) are attributed to possible seed dependence, but no multi-seed experiment is run to test this. Training 3–5 seeds for a few model sizes and dataset configurations would clarify whether the inconsistency reflects genuine stochasticity (some seeds generalize, others don't) or systematic patterns not captured by the single-seed results. If generalization is truly seed-dependent, that is a significant finding about chain-of-thought compositional reasoning that deserves quantification. If it is not (e.g., the apparent inconsistency is just noise from small sample sizes), then the Figure 7 results are less informative than they appear.

Direct comparison plot of all three hypotheses: A single figure showing the same set of trained models' information content computed under all three hypotheses (independent, two-function, recurrent) would make the comparative claim much stronger. The reader could directly see that the two-function curves track the capacity line while the independent curves fall far short and the recurrent curves exceed it. The current presentation (only Figure 2 showing the "winning" hypothesis) requires the reader to trust the paper's implicit comparison.

Validation on an alternative architecture: All experiments use Llama-architecture transformers. Testing on a different architecture (e.g., a non-autoregressive model, or a model with different attention patterns) would probe whether the 2 bits per parameter capacity limit and the two-function composition finding are architecture-specific or general properties of feed-forward transformers. Allen-Zhu & Li (2024) reported architecture-independence for knowledge capacity; replicating this for the two-hop scaling result would increase confidence.

Wider parameter and dataset range: The parameter range (250K to 15M, ~60×) and dataset range (1000 to 20000, ~20×) are modest compared to the scaling law literature. Extending to larger models (50M–100M parameters) and larger datasets (100K–500K profiles) would test whether the alignment with the capacity line holds across orders of magnitude, or whether new phenomena emerge at scale. This would also help distinguish between the slightly-below-capacity curves in Figure 2 (which could reflect a true capacity of ~1.6 bits/parameter) and the on-capacity curves (which would be expected if the 2 bits/parameter figure generalizes).

Quantitative goodness-of-fit metrics: The current analysis is purely visual — curves are compared by eye to a reference line. A quantitative metric (e.g., the slope of information content vs. parameters in the linear regime before saturation, or the root-mean-square deviation from the capacity line) would make the comparison between hypotheses testable rather than interpretive. The paper could compute, for each hypothesis, the mean squared error between the measured information content curves and the capacity line, and report which hypothesis achieves the lowest error. Without this, the claim that two-function composition "fits better" is a qualitative judgment.

Are there alternative explanations for the scaling behavior?

The paper's analysis assumes that the information content scaling reflects the intrinsic algorithmic structure of the trained model. But there are alternative explanations:

Partial memorization + partial composition: Models might not implement a clean either/or algorithm. They could memorize some two-hop answers directly (especially frequently seen ones) while composing others. The observed scaling would then be a weighted mixture of the two-function and independent scaling curves, not a pure signal of either. This would explain why the curves in Figure 2 do not perfectly track the capacity line — they show curvature that could reflect a shifting mixture of strategies as capacity increases.

Loss-based information content lower bounds are loose: The derivation of information content from cross-entropy loss provides only a lower bound (Equations 5–8, with ≥ signs). If the bound is loose — which it will be whenever the model's probability distribution over answers is not perfectly calibrated — then the true information content could be higher than measured, potentially shifting the curves upward relative to the capacity line. The paper's Appendix A derivations introduce additional approximations (replacing expectations of logs with logs of expectations, assuming independence of probabilities). These approximations are justified at the extremes (near-uniform guessing or near-certain predictions) but may be inaccurate in the intermediate regime where most of the scaling behavior occurs, potentially distorting the shape of the curves.

Architecture-specific capacity limits: The 2 bits per parameter figure from Allen-Zhu & Li (2024) was measured on a different architecture, different data, and different training setup. The current paper's own one-hop measurement is 1.6 bits/parameter, not 2.0. The two-hop alignment is judged against 2.0, but if the true capacity for this architecture and training setup is 1.6, the curves in Figure 2 are actually exceeding capacity at some points (as noted above). This would suggest either that the model has found a more efficient encoding than the capacity limit would predict — which would be a significant finding, but is not acknowledged — or that the two-function composition hypothesis slightly overestimates the information requirement, and the true algorithm is even more efficient (perhaps sharing some parameters between the two function copies, achieving something between recurrent and two-function composition).

These alternatives do not invalidate the paper's core contribution — that capacity scaling can serve as a diagnostic tool and that it supports two-function composition over the alternatives — but they suggest that the method is less precise and less definitive than the paper's narrative implies. The appropriate conclusion is that capacity scaling provides probabilistic evidence for algorithmic hypotheses, not deterministic identification, and that it is best used in conjunction with other methods (generalization, probing) rather than as a standalone arbiter.

6. Limitations and Trade-offs

Difficulty Estimation Cost Is Unmeasured and Potentially Dominant

The assumption or constraint. The information content measurement procedure requires knowing the dataset entropy — the total information content of the training data under each candidate computational model. This is computable only because the paper uses fully synthetic data with a known, controlled generating process (Section 2.1). In any real-world deployment, the underlying data generating process is unknown, and dataset entropy cannot be analytically computed. The paper acknowledges this implicitly but never states it as a limitation — the entire method depends on exact knowledge of $E$, which is available only in synthetic settings.

The consequence. The method cannot be applied to natural language data, real-world QA datasets, or any domain where the ground-truth information content of the data is unknown. This is not a minor practical obstacle — it is a fundamental scope constraint. The paper frames information capacity scaling as "a new, complementary method for interpreting transformers" (Section 1.1), but the experiments provide no evidence that the method generalizes beyond fully synthetic, template-generated text with known entropy. Even in controlled synthetic settings, the method requires deriving separate information content formulas (Equations 2–4) and effective loss transformations (Appendices A.1–A.2) for each candidate algorithm — a manual, hypothesis-specific process that does not scale to complex tasks with many plausible algorithms.

What evidence exists in the paper. The paper's entire experimental corpus is fully synthetic (Section 2.1): fictional profiles with random relations, template-generated questions, and no natural language structure. The dataset entropy formulas (Equations 1–4) are derived directly from the known data generating process parameters ($|N|$, $|R|$, $|A|$, $|V_a|$, $N_0$). No experiment attempts to apply the method to data with unknown entropy, or to approximate entropy from samples alone.

Mitigation status. Not addressed. The paper does not discuss how one might estimate dataset entropy in settings where the data generating process is unknown, nor does it characterize the method's sensitivity to errors in entropy estimation. The discussion (Section 3.5) acknowledges that "it is not straightforward to apply this method to interpret models" and that "significant challenges must be overcome," but frames these as practical difficulties rather than fundamental scope limitations. No path to generalization is proposed beyond the vague suggestion that "it is plausible that it can be made to work if the application is compelling enough."

The Reference Capacity Is Inconsistently Measured and Hypothesis-Testing Depends on It

The assumption or constraint. The entire diagnostic logic — discriminating between algorithms by comparing measured information content to a capacity reference — requires a known, stable capacity ceiling (bits per parameter) against which to compare scaling curves. The paper adopts Allen-Zhu & Li (2024)'s figure of ~2 bits per parameter as this reference, but its own one-hop measurements produce ~1.6 bits per parameter in the main experimental configuration (Figure 1) and closer to 2.0 in earlier runs (Figure 8, different hyperparameters). The paper acknowledges this discrepancy:

"We did not perfectly reproduce the existing result of a 2 bit per parameter information capacity for one-hop questions and answers" (Section 3.1)

but continues to use the 2 bits per parameter line as the reference for evaluating two-hop scaling in all subsequent figures.

The consequence. The capacity reference is uncertain by at least 20% (1.6 vs. 2.0), which directly affects hypothesis discrimination. Under the two-function composition hypothesis in Figure 2, the $|N| = 1000$ curve at $\sim 3.5 \times 10^6$ parameters reaches $\sim 6 \times 10^6$ bits, which is $\sim$1.7 bits per parameter — above 1.6 but below 2.0. If the true capacity is 1.6, this curve exceeds it, suggesting the two-function hypothesis underestimates information requirements (the model compresses more efficiently than the hypothesis allows). If the true capacity is 2.0, the curve is slightly below, consistent with the hypothesis. The paper's conclusion that two-function composition "best fits" the data is therefore sensitive to which reference capacity is assumed, yet no principled basis for choosing between 1.6 and 2.0 is provided.

More subtly, the paper implicitly assumes a linear capacity relationship (constant bits per parameter), but none of the measured curves are cleanly linear — they show S-shaped curvature from baseline to saturation (Figures 1–3). The reference line is an approximation at best, and the paper does not test whether a nonlinear capacity model might better explain the data or alter the hypothesis comparison.

What evidence exists in the paper. Figure 1 (one-hop, 17 relations, µP): $\sim$1.6 bits/parameter. Figure 8 (one-hop, 4 relations, no µP): closer to 2.0 bits/parameter. The two-hop analysis (Figures 2–3, 5) uses the 2.0 reference. No experiment systematically ablates the factors (number of relations, µP, optimizer) that cause the capacity measurement to vary, so the source of the discrepancy is unknown.

Mitigation status. The paper notes the discrepancy (Section 3.1) but does not resolve it or incorporate the uncertainty into its analysis. The discussion states: "the fact that dataset choices may introduce uncertainty in this curve makes hypothesis testing more difficult" (Section 3.5), which is accurate but not accompanied by any robustness analysis (e.g., testing conclusions under both 1.6 and 2.0 capacity assumptions, or measuring capacity across a wider range of configurations to establish the source of variation).

Capacity Scaling Provides Only Qualitative, Visual Evidence Without Quantitative Rigor

The assumption or constraint. The paper's core empirical claim — that two-function composition scaling curves "align with" or "fit" the capacity reference line better than alternatives — is evaluated entirely by visual inspection of Figures 2, 3, and 5. There is no quantitative metric: no slope measurement, no goodness-of-fit statistic, no confidence interval on the bits-per-parameter estimate, no formal model comparison criterion. The analysis is comparative ("two-function composition fits better than the alternatives would") without quantifying how much better or whether the difference is statistically meaningful.

The consequence. The conclusion that two-function composition is supported is a qualitative judgment, not a testable, quantitative result. Different readers could reasonably disagree about whether a given curve "aligns with" or "deviates from" the reference line — particularly given the curvature and noise in the measured points. The lack of quantification also makes it impossible to assess how sensitive the conclusion is to outliers, to the choice of which models are included, or to the specific approximation used in computing effective losses (Appendices A.1–A.2).

This matters for the paper's stated goal of providing an interpretability method. A practitioner who wants to apply capacity scaling to a new task needs to know: what constitutes "good alignment"? How many model sizes must be tested? How many dataset sizes? At what deviation from the capacity line should a hypothesis be rejected? The paper provides no operational guidance because it never formalizes the decision rule.

What evidence exists in the paper. The entire analysis in Sections 3.1–3.3 is visual descriptions of figures. Representative language: "the content curve almost kisses the 2 bit per parameter line" (Section 3.1), "these results fit the two function composition computational model far better than they fit either of the alternative models" (Section 3.1), "the measured information content approximates the capacity curve" (Figure 5 caption). No numerical comparisons are reported. The paper does not compute, for instance, the mean squared error between the measured content and the capacity line under each hypothesis, nor does it test whether the slope of the content-vs-parameters relationship differs significantly from 2 bits/parameter.

Mitigation status. Not addressed. The paper does not acknowledge the lack of quantification as a limitation. The discussion (Section 3.5) notes general challenges ("it is nontrivial to go from a hypothesis to a capacity scaling curve") but does not propose quantitative metrics or formal hypothesis tests for future work. The method remains at the level of "demonstration of concept" rather than a validated, reproducible diagnostic protocol.

Single Model Architecture and Narrow Scale Range Limit Generality

The assumption or constraint. All experiments use a single architecture family (Llama-architecture transformers; Dubey et al., 2024), a single model depth as the primary focus (4 layers, with 12 layers explored only in one underpowered comparison in Figure 9), a narrow parameter range (250K to 15M, a factor of $\sim$60×), and a narrow dataset size range (1000 to 20000 profiles for the main two-hop scaling, a factor of 20×). The capacity reference and algorithmic conclusions are derived entirely from this restricted configuration.

The paper states that Allen-Zhu & Li (2024) found information capacity "did not seem to depend on the model architecture," suggesting the capacity reference might generalize, but the paper itself does not test architecture-dependence for the two-hop scaling behavior.

The consequence. The findings may not transfer to other transformer architectures (encoder-decoder, non-autoregressive, different attention patterns), to larger model scales where phenomena like in-context learning or superposition might change how information is stored, or to deeper models (the 12-layer results in Figure 9 show inconsistent scaling, though the paper attributes this to hyperparameter mismatch rather than a fundamental depth effect). The claim that "two-function composition" is how transformers learn two-hop QA is only demonstrated for 4-layer Llama models at 250K–15M parameters — the paper provides no evidence about whether larger models (100M+, 1B+) would exhibit the same algorithmic signature or might develop more efficient strategies.

The narrow dataset size range is equally concerning. The largest dataset tested for two-hop scaling is $|N| = 20000$ profiles (Figure 2), and the chain-of-thought experiment uses only $|N| = 10000$ (Figure 3). Scaling law analyses typically span orders of magnitude to establish robust functional relationships; a factor of 20× is small enough that alternative functional forms (e.g., logarithmic rather than linear growth in capacity) could fit the data equally well.

What evidence exists in the paper. Figures 1–3 and 5 all use 4-layer Llama models at the cited scales. Figure 9 compares 4-layer and 12-layer models at $|N| = 10000$ only, with the deeper models showing an anomalous dip that the paper attributes to untuned hyperparameters — meaning it provides no clean evidence about depth scaling. No other architecture, larger scale, or intermediate depth is tested.

Mitigation status. The paper acknowledges the scale limitation implicitly (the discussion notes that applying the method to "the kinds of transformers we are most interested in" is challenging) but does not frame the narrow parameter and architecture range as a threat to the generality of its conclusions. The suggestion that future work could apply the method to larger models is present but vague. No concrete scaling predictions are made that would allow the findings to be validated or falsified at larger scales.

The Method Requires Models Trained to Full Convergence, Constraining Applicability

The assumption or constraint. The information content measurement procedure assumes models are trained to saturation — they have converged as fully as their parameter capacity allows, so the measured information content reflects their fundamental storage limit rather than transient undertraining. The paper enforces this with an extremely stringent convergence criterion: training stops only when loss decreases by less than $10^{-8}$ per step, which "typically took about 10M steps (or 160B tokens)" (Section 2.2). This is feasible for small models (250K–15M parameters) on synthetic data, where training is computationally cheap and no practical deadline constrains training duration.

The consequence. The method is inapplicable to large-scale models trained on real data, where training to saturation is economically infeasible or practically impossible. A 7B-parameter model trained on internet-scale text is never trained to $10^{-8}$ loss decrease per step — training is stopped based on validation metrics, compute budgets, or practical deadlines, long before saturation. The information content measured from such a model would reflect a mixture of learned information and residual error from incomplete optimization, making it impossible to cleanly interpret as a capacity measurement.

More subtly, the algorithm a model learns may depend on training duration, not just final performance. The paper's "memorization trap" experiment (Section 3.3, Figure 5) illustrates this: early in training, models may learn a suboptimal algorithm (memorization) and become trapped there. If training were stopped at an arbitrary checkpoint rather than saturation, the measured algorithm might be different from what the model would eventually learn. The saturation requirement is therefore not just a convenience for clean measurement — it may be necessary for the measured algorithm to represent the model's "true" strategy rather than a transient.

What evidence exists in the paper. The convergence criterion is stated in Section 2.2: $10^{-8}$ per-step loss decrease, $\sim$10M steps, $\sim$160B tokens. All models in all experiments are trained to this criterion. No experiment tests whether the main conclusions (two-function composition, capacity scaling alignment) hold for partially trained models or models trained with standard early-stopping based on validation loss. The paper does not analyze how information content measurements evolve over the course of training.

Mitigation status. Not addressed. The paper does not discuss the saturation requirement as a constraint on the method's applicability, nor does it explore whether approximate capacity measurements from partially trained models could still support algorithm discrimination. The discussion (Section 3.5) notes practical difficulties in applying the method but does not identify convergence requirements as one of them. No future work is suggested on extending the method to sub-saturation regimes.

Generalization and Capacity Scaling Can Diverge, Undermining the Method's Claim to Be a Reliable Proxy

The assumption or constraint. The paper's core methodological claim is that information capacity scaling can serve as an interpretability method that is "complementary" to generalization testing (Section 1.1), and specifically that "algorithms with the same information content scaling properties tend to have the same generalization behaviour" (Section 1.1). This assumption — that capacity scaling implies generalization behavior — is what gives the method its diagnostic value: if we measure capacity scaling consistent with two-function composition, we can infer the generalization signature of Table 1 without running the actual generalization experiments.

The consequence. The chain-of-thought results (Section 3.2, Figure 7) directly undermine this assumption. Capacity scaling for chain-of-thought models (Figure 3) aligns with recurrent composition, which predicts (per Table 1) that models should generalize to all held-out splits where the component facts appear in training. But the actual generalization behavior (Figure 7) is inconsistent: held-out first entities show no generalization in any model; held-out entity-attribute pairs show consistent generalization; other held-out components (relations, attributes, entity-relation pairs, second entities) show generalization in some models but not others, without a clear pattern. The paper acknowledges this:

"This does indicate that scaling — which was consistent with recurrent composition — is not a perfect proxy for generalization." (Section 3.2)

This is a significant limitation because it shows that the method can produce capacity scaling consistent with one algorithm (recurrent composition) while the model's generalization behavior does not fully match that algorithm's predictions. If capacity scaling and generalization can diverge, then capacity scaling alone cannot reliably predict generalization — it requires independent validation. But if independent validation (generalization testing) is needed anyway, the method's value as a complementary or alternative diagnostic is reduced.

What evidence exists in the paper. Figure 3 shows capacity scaling approaching recurrent composition for chain-of-thought models. Figure 7 shows inconsistent generalization for the same model class across different held-out splits. The paper acknowledges the divergence (Section 3.2) and speculates about "competing heuristics" and possible seed dependence, but does not further investigate or quantify the relationship between capacity scaling and generalization.

Mitigation status. Partially acknowledged but not resolved. The paper states that scaling is "not a perfect proxy for generalization" (Section 3.2) and that it is "not clear how often [the assumption that same scaling implies same generalization] is applicable in general" (Section 1.1). These acknowledgments are honest but leave the method's reliability undefined. The paper does not characterize when capacity scaling and generalization will agree versus diverge, does not propose additional diagnostics to resolve disagreements, and does not suggest future work on understanding the relationship between information-theoretic efficiency and behavioral generalization in transformers. A practitioner cannot know, based on this paper, when to trust a capacity scaling inference and when to independently verify with generalization tests.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces information capacity scaling as a third methodological pillar in the transformer interpretability toolkit, sitting alongside behavioral generalization testing and activation probing. This is not a paradigm shift — it does not replace existing methods or overturn a consensus view — but it fills a genuine gap: a diagnostic that can provide evidence about internal algorithms when probing fails and generalization experiments are incomplete or infeasible.

The significance lies in what the method can do that others cannot, and what its failures reveal about the limits of alternatives. The paper demonstrates a clear case where supervised linear probes fail to recover intermediate entities (Tables 2–3) despite strong independent evidence from both generalization (Figures 4, 6) and capacity scaling (Figure 2) that the model implements two-function composition. This is not a null result to be buried in an appendix — it is a substantive finding that probing success is contingent on factors beyond whether the model uses a particular algorithm. The same intermediate entity that Wang et al. (2024) successfully decoded with the logit lens was invisible to the more powerful supervised probes in this paper's experimental setup. This should temper the interpretability community's reliance on probing as a primary or sufficient tool for algorithm identification. A negative probing result does not rule out compositional reasoning; a positive result does not uniquely identify it. Capacity scaling provides an orthogonal signal that can corroborate or challenge probing-based conclusions.

The paper also reframes the "transformers can't compose" narrative into something more precise and mechanistically grounded. Prior work (Dziri et al., 2023; Merrill et al., 2022) established that transformers without chain of thought have fundamental limitations on compositional reasoning, but these were complexity-theoretic worst-case bounds — they said what transformers cannot do in principle, not what they actually do when trained on compositional tasks. This paper's two-function composition finding shows that transformers do learn to compose, but they pay an information-theoretic penalty for doing so in a single forward pass: each fact must be stored twice, once for each hop position, because the feed-forward architecture prevents reusing a single learned function. The ~2× increase in required information content relative to the recurrent ideal is the quantifiable cost of lacking recurrence. This converts a binary "can/cannot compose" question into a continuous "how efficiently can it compose, and at what capacity cost?" question, which is both more nuanced and more practically useful.

Chain of thought's benefit is correspondingly reframed — not just as enabling new computations (the Turing-completeness argument from Pérez et al., 2021), but as dramatically improving parameter efficiency for compositional tasks. Figure 3 shows chain-of-thought models achieving information content consistent with recurrent composition (~1× the one-hop cost) rather than two-function composition (~2×). This means a chain-of-thought model can store the same factual knowledge in roughly half the parameters, or achieve higher accuracy at the same parameter count, compared to a model answering compositionally in one forward pass. The paper explicitly connects this to the strong performance of small distilled reasoning models (DeepSeek-AI et al., 2025), suggesting that part of their efficiency comes from this information-theoretic compression advantage rather than from novel reasoning capabilities per se.

The "memorization trap" experiment (Section 3.3, Figure 5) introduces a practically important dynamic: the algorithm a model learns depends not only on task structure and capacity, but on the relative loss weighting of sub-tasks. When two-hop questions are heavily weighted relative to one-hop questions (achieved by reducing the number of relation types from 17 to 4, increasing each two-hop question's share of the total loss), models skip directly to memorizing two-hop answers — even though they have the capacity to learn the more efficient compositional algorithm. This is a concrete demonstration that training data design (not just architecture or scale) determines whether models develop generalizable compositional strategies versus brittle memorization. It explains why some prior studies found compositional generalization and others didn't — the relative prevalence of composed versus atomic facts in training data may be the determining factor, not any fundamental capability difference between models.

Finally, the paper partially reconciles the tension between Wang et al. (2024) and Yu (2025) on probing for multi-hop reasoning. Wang et al. found decodable intermediate entities; Yu found they were often non-recoverable in prompted models. This paper finds them non-recoverable even in fine-tuned compositional models, using more powerful probes than Wang et al. The resolution is that decodability of intermediate computations is not a necessary signature of compositional reasoning — it is a contingent property that varies with training setup, model scale, architecture details, and perhaps random seed. This clarifies that probing and capacity scaling are genuinely complementary: probing tells you whether intermediate information is linearly accessible in activations; capacity scaling tells you whether the total stored information is consistent with a particular algorithm's compression properties. Neither alone is sufficient; together they provide stronger constraints.

Several research directions become more attractive following this work:

  • Training data diagnostics for compositional learning: The memorization trap experiment suggests a practical recipe — check whether your training data's loss weighting incentivizes memorization over composition — that could be applied to real-world training pipelines. If a model is failing to generalize compositionally, adjusting the ratio of atomic to composed examples may shift it from a memorization basin to a composition basin.

  • Verifier and search mechanism design through the lens of information capacity: If transformers pay a capacity penalty for multi-step reasoning in a single forward pass, this provides an information-theoretic justification for test-time search and verification strategies: they allow the model to externalize intermediate computation (like chain of thought) without pretraining for it, potentially recovering the parameter efficiency of recurrent composition at inference time.

  • Architecture design for compositional efficiency: The ~2× capacity penalty for two-function composition quantifies the cost of the feed-forward constraint. Architectures that introduce lightweight recurrence (e.g., recurrent memory tokens, adapter-based feedback loops, or test-time adaptive computation) could target this specific inefficiency — the goal would be to reduce the penalty factor from ~2× toward ~1× for multi-hop tasks, with information capacity scaling providing the measuring stick.

Directions that become less attractive:

  • Relying solely on probing to establish algorithmic claims about compositional reasoning. The paper's negative probing result, in a setup where composition is strongly supported by two independent methods, undermines the practice of treating probing as a gold standard. Work that claims "the model does not compose" based solely on failed probes should be viewed with skepticism unless accompanied by capacity scaling or comprehensive generalization evidence.

  • Assuming that transformer scaling will automatically solve compositional generalization. The two-function composition finding suggests that larger models will continue to pay the ~2× capacity penalty — they'll have more total capacity, but they won't become more efficient at composition within a single forward pass. Compositional generalization may improve with scale because the model can memorize more combinations (approaching the independent memorization limit), not because it develops more clever compositional strategies. This implies that scale alone may be an inefficient path to robust compositional reasoning compared to architectural innovations or training paradigms (like chain of thought) that reduce the capacity penalty.

Follow-Up Research This Work Enables

Directly test the "each fact twice" hypothesis with activation-level causal interventions. The paper infers two-function composition from information content scaling, but never directly verifies that the model encodes two separate copies of factual lookup functions in different layer ranges. A follow-up could use activation patching or path patching: for a two-hop question, ablate the attention heads or MLP layers responsible for the first hop and measure whether the second hop's computation is unaffected (indicating separate function copies) or whether both hops are disrupted (indicating shared computation). If two-function composition is correct, there should be a clean separation — early-layer ablations should selectively impair first-hop accuracy, late-layer ablations should selectively impair second-hop accuracy, and the crossover point should align with where the intermediate entity is computed. The paper's 4-layer models provide a tractable scale for exhaustive patching across all layers and attention heads. A strong result would quantitatively map which layers encode $f_1$ versus $f_2$ and confirm that the information content in each copy matches the ~1-bit-per-parameter scaling of the one-hop baseline.

Quantify the "memorization trap" across a continuous range of relation counts and loss weightings. The current experiment compares only two extremes: 4 relations (where models memorize) and 17 relations (where models compose). A systematic sweep of relation counts — say, 2, 4, 6, 8, 10, 12, 14, 17, 20 — combined with measuring whether capacity scaling matches independent memorization or two-function composition at each point, would reveal whether there is a sharp phase transition (a critical number of relations below which all models memorize) or a gradual crossover. Additionally, varying the one-hop-to-two-hop ratio independently of the relation count (e.g., training with 1:1 or 1:100 ratios at fixed relation count) would disentangle the two factors confounded in the current experiment. The prediction from the paper's local-minimum hypothesis is that the transition should be sharp when capacity is limiting — once the model allocates enough parameters to memorization that it cannot also learn one-hop functions, it is trapped. This would manifest as a discontinuous jump in the inferred algorithm at a specific relation count, with hysteresis (once trapped, changing the loss weighting mid-training should not rescue the model). Testing for hysteresis — by starting training with 4 relations, then switching to 17 relations partway through, and vice versa — would directly test the local-minimum mechanism versus the alternative explanation that models with 4 relations simply receive insufficient gradient signal for one-hop learning.

Extend capacity scaling to three-hop and four-hop QA to test the "N copies" prediction. The two-function composition hypothesis makes a clear extrapolation: for K-hop QA without chain of thought, transformers should require K separate copies of the factual lookup function, yielding an information content that scales as K × the one-hop cost. Testing this on three-hop and four-hop synthetic datasets would validate whether the capacity penalty grows linearly with reasoning depth, or whether models develop more efficient strategies (e.g., hierarchical composition) as depth increases. If the linear penalty holds, it would establish a quantitative relationship between reasoning depth and required model size in single-forward-pass transformers — a "depth-capacity scaling law" that could predict, for a given parameter budget, how many hops of reasoning are feasible before accuracy collapses. The experimental design is a direct extension of the current paper: generate three-hop datasets ("Who is Bob's mother's boss's best friend?"), train models of varying sizes to saturation, and measure which computational model (3-function composition, recurrent, or independent memorization) best fits the capacity scaling.

Develop a difficulty estimator for compositional tasks based on information content predictions. The paper's method requires knowing dataset entropy, which in its current form demands fully synthetic data with a known generating process. A practical extension would train a difficulty estimator — a small model or statistical estimator that, given a natural language dataset and a candidate algorithmic hypothesis, estimates the entropy of the data under that hypothesis from samples alone. For two-hop QA on natural data (e.g., HotpotQA), one could approximate the entropy by: (1) estimating the number of distinct entities and relations in the data; (2) computing the entropy formula for each candidate algorithm using these estimates; (3) measuring the trained model's loss; (4) comparing capacity scaling to the estimated entropy. The key uncertainty is step (1) — accurately counting distinct entities and relations in natural language without a predefined schema. A feasibility test would be: take the paper's synthetic data, strip the explicit structure, and see whether an entity-counting heuristic (e.g., named entity recognition + clustering) can recover the dataset parameters closely enough to distinguish two-function composition from independent memorization. If this works on synthetic data, it could be attempted on HotpotQA or a similar multi-hop benchmark.

Test whether the capacity penalty applies to non-symbolic compositional tasks (e.g., visual reasoning, code execution). The current paper uses cleanly symbolic two-hop QA where facts are discrete mappings between named entities. The "each fact twice" hypothesis depends on the feed-forward constraint — information computed in early layers cannot be fed back to those same layers for reuse. This constraint is domain-general: it should apply to any task where the same operation must be applied sequentially to intermediate results. A natural test would be multi-step code execution: given a sequence of operations (e.g., x = 2; y = x + 3; z = y * 4), a model answering "what is z?" in one forward pass should encode the addition operation (applied to x) and the multiplication operation (applied to y) in separate layer ranges, duplicating any shared arithmetic primitives. Training small transformers on synthetic multi-step arithmetic and measuring capacity scaling under different algorithmic hypotheses (e.g., "each operation learned once" vs. "each operation learned per step position") would test whether the penalty generalizes beyond factual lookup to procedural computation. This connects to the Yu (2025) finding on arithmetic probing — capacity scaling might reveal compositional structure that probing misses, just as it did for two-hop QA.

Investigate whether the 1.6 vs. 2.0 bits per parameter discrepancy reveals a systematic relationship between task structure and capacity. The paper's one-hop capacity varies from ~1.6 bits/parameter (17 relations, µP) to ~2.0 (4 relations, no µP). Rather than treating this as noise, a systematic study could test whether capacity depends on the branching factor of the task — the number of possible answers per question. With 17 relations, each one-hop question has more possible answers (the union of all relation-specific answer sets), which might require the model to allocate some capacity to routing or disambiguation circuitry that doesn't directly store facts, reducing the effective bits per parameter for pure memorization. The prediction would be: as the number of relations (and thus answer set size) increases, measured capacity should decrease, asymptotically approaching some lower bound where the model spends a fixed fraction of parameters on task circuitry regardless of dataset size. Testing this would require a controlled sweep of relation counts (holding other factors constant) and measuring one-hop capacity at each point. If confirmed, this would provide a more nuanced capacity model — not a single bits-per-parameter constant, but a function of task complexity that could be estimated before training.

Practical Applications and Downstream Use Cases

Training data design for compositional generalization. The memorization trap experiment (Section 3.3, Figure 5) provides a concrete, actionable diagnostic: if your training data has a low ratio of atomic facts (one-hop equivalents) to composed facts (two-hop equivalents), and your model's capacity is constrained, the model may memorize compositions rather than learning generalizable primitives. For practitioners building QA systems or knowledge-grounded models, this suggests a data auditing step: estimate the effective "relation count" in your training data (the diversity of atomic fact types relative to composed queries), and if it falls below some threshold (the paper shows a trap at 4 relations with 10:1 two-hop-to-one-hop ratio, corresponding to a 2.5:1 loss-weighting advantage for two-hop memorization), restructure the training mix to either increase the proportion of atomic examples or increase the diversity of fact types so that no single composed template dominates the loss. The exact threshold likely depends on model size and task complexity, but the principle — that loss weighting can drive algorithmic choice — is directly actionable.

Parameter budgeting for multi-hop reasoning systems. The finding that two-hop QA without chain of thought requires ~2× the information content of one-hop QA (Figure 2 vs. Figure 1) provides a rough parameter multiplier for latency-constrained deployments. If a system must answer K-hop questions in a single forward pass (because latency requirements preclude chain of thought or iterative retrieval), the required model size scales roughly as K × the size needed for one-hop accuracy at the same level. For a production QA system with a target accuracy and a fixed latency budget, this can inform the tradeoff between model size and supported reasoning depth. For instance, if a 100M-parameter model achieves acceptable one-hop accuracy, a 200M-parameter model might be needed for two-hop questions, 300M for three-hop, and so on — assuming the linear penalty generalizes (which the suggested three-hop extension would test). This is a back-of-the-envelope guideline, not a precise law, but it provides an order-of-magnitude starting point that did not exist before this work.

Efficient fine-tuning for compositional tasks via chain-of-thought distillation. The chain-of-thought capacity result (Figure 3, ~1× information content vs. ~2× without chain of thought) implies that a model fine-tuned to use chain of thought for compositional tasks can store the same factual knowledge in ~half the parameters — or equivalently, achieve higher accuracy at the same parameter count. This suggests a distillation strategy: train a large model with chain of thought on compositional data, then distill it into a smaller model that also uses chain of thought, exploiting the parameter efficiency of recurrent-like computation. The smaller model's capacity budget goes further because each fact is stored once and reused across reasoning steps via autoregressive generation, rather than being duplicated across layers. The DeepSeek-R1 results (DeepSeek-AI et al., 2025) provide circumstantial support — small distilled reasoning models show strong mathematical performance — but a direct controlled experiment comparing distillation with vs. without chain of thought on synthetic multi-hop data, measuring capacity scaling in both cases, would quantify the benefit.

Diagnostic for distinguishing memorization from reasoning in black-box models. A practical application of the capacity scaling method, even in its current synthetic-only form, is as an auditing tool for model developers. If a team trains a model on a known synthetic task (e.g., as part of a controlled evaluation suite), they can measure information content scaling and compare it to candidate algorithmic hypotheses to determine whether the model learned to compose or memorize. This could catch "clever Hans" behavior — models that achieve high test accuracy through memorization of spurious patterns rather than the intended algorithm — before deployment. The paper's demonstration that models with similar training accuracy can have fundamentally different internal algorithms (two-function composition vs. independent memorization) that are invisible to loss curves but detectable via capacity scaling makes this a practical complement to standard evaluation. A company training a model for a compositional reasoning task could include a small synthetic probe dataset with known entropy in their training mix, measure capacity scaling post-hoc, and flag models whose scaling matches memorization for further investigation — even if the main training data is natural language whose entropy is unknown.