ArXiv: 2601.03570

🎯 Pitch

LLMs learn new concepts using denser, centralized computational subcircuits, but that very structure makes those same concepts more vulnerable to catastrophic forgetting later. Moreover, training sequentially on semantically similar concepts creates strong interference, while the right ordering of knowledge types (e.g., properties before synonyms) unlocks powerful, asymmetric knowledge transfer.

1. Executive Summary

This paper studies how large language models acquire, retain, and forget individual concepts during continual pre-training, using the FICO dataset of fictional concepts derived from ConceptNet and the BIO dataset, with experiments on GPT-2 Large and LLaMA-3.2-1B-Instruct. The authors introduce Concept Circuits—computational subgraphs associated with specific concepts—and characterize their topology with four families of graph metrics (e.g., eigenvector centrality variance, circuit density, global efficiency, and k-core depth) to link internal circuit structure to behavioral dynamics of learning and forgetting. The analysis reveals a structural trade-off: denser, more centralized circuits correlate with stronger concept acquisition, while more modular organization helps mitigate forgetting, as reflected in a statistically significant positive correlation between concept learning degree and forgetting degree. Cross-concept experiments demonstrate that semantically similar concepts induce substantially stronger interference than weakly related ones—training with highly related concepts achieves only 57.5% performance versus 75.9% for weakly related ones—while cross-knowledge-type analysis shows highly asymmetric transfer, with pretraining on Property & Affordance knowledge improving subsequent Synonym & Antonym learning by 63.74%, establishing that concept-level interference and synergy are structured, predictable phenomena that depend on both semantic similarity and knowledge-type ordering.

2. Context and Motivation

The Core Gap: We Don't Know How LLMs Learn Concepts During Training

The fundamental question this paper tackles is deceptively simple: when we continually pre-train a large language model on new information, how do individual concepts actually get learned, stored, and later forgotten? This matters because LLMs are not static artifacts—they are increasingly deployed in settings where they must acquire new knowledge over time, whether through fine-tuning on domain-specific corpora, updating with post-training data (e.g., recent events after a knowledge cutoff), or adapting to entirely new conceptual domains. Yet despite the practical importance of continual learning, the field lacks a mechanistic understanding of what happens inside an LLM when it encounters novel concepts and their associated relational knowledge.

This gap is significant for several reasons the paper highlights:

  • Training efficiency: If practitioners could predict which concepts will be learned quickly and which will require additional training exposure, they could allocate compute budgets more intelligently. The paper's analysis reveals that concepts exhibit "substantial heterogeneity in learning degree" (Finding 1), with some concepts experiencing dramatic logit increases while others barely shift under identical training regimes—suggesting that uniform training strategies leave considerable efficiency on the table.
  • Catastrophic forgetting: When models are updated with new data, previously learned information can be disrupted or overwritten. Understanding which concepts are most vulnerable to forgetting, and why, would enable targeted mitigation strategies such as replay-based approaches or interference-aware data scheduling.
  • Curriculum design for continual pre-training: If certain types of conceptual knowledge scaffold the acquisition of others, or if certain concept pairs interfere destructively when learned together, training data ordering could be optimized to maximize learning efficiency and minimize interference. The paper explicitly motivates this: answering these questions "can inform concept-aware continual pre-training decisions such as how much training is needed for new concepts, how to schedule and reorder training data, and how to reduce destructive interference among semantically related concepts" (Section 1).

From a theoretical standpoint, this gap is equally significant. Concepts are the building blocks of human cognition—"abstract mental categories (e.g., dog) that enable generalization from individual observations to shared properties, relations, and actions" (Section 1). If we believe that LLMs are developing anything approximating conceptual understanding during pre-training, then characterizing how this process unfolds mechanistically is essential for both interpretability and for building models that learn more like humans do. Yet prior work on concept learning in LLMs has been largely static, probing what models already know rather than tracing how they come to know it.

Where Prior Approaches Fall Short

The paper identifies two broad categories of prior work, both of which provide only a partial view:

Probing-based approaches test what concepts a model has already encoded, but not how it acquired them. One line of work uses prompt-based knowledge probing to test whether specific conceptual properties or relations (e.g., commonsense facts, definition–name mappings, compositional binding) can be elicited from a pre-trained model (Gu et al., 2023b; Liao et al., 2023; Shani et al., 2023a; Zheng et al., 2024; Peng et al., 2022; Xu et al., 2024a). These studies are valuable for auditing model knowledge, but they are fundamentally static: they capture a snapshot of what a frozen model has encoded, with no temporal dimension. They cannot tell us when during training a concept was acquired, how rapidly it was consolidated, or whether it will persist under subsequent updates. Moreover, probing typically targets isolated pieces of conceptual knowledge rather than the full relational structure associated with a concept—the multiple knowledge triples (e.g., a dog has four legs, can run, is a type of animal) that collectively define what the concept means.

Mechanistic interpretability work has localized concept-related components, but hasn't tracked them over training time. A second line of research uses tools from mechanistic interpretability to identify internal components—attention heads, MLP layers, or specific circuits—associated with conceptual information (Aljaafari et al., 2024; Wang et al., 2024b). For example, Wang et al. (2024c) identified editable sub-networks responsible for conceptual knowledge. However, these analyses are also primarily conducted on frozen, already-trained models. They tell us where concept-related computation happens, but not how those circuits formed during training, how they evolve under continued learning, or why some circuits are more resistant to forgetting than others.

Prior studies of LLM knowledge acquisition treat knowledge as isolated facts, not structured conceptual relations. Several recent works have studied how LLMs learn new information using synthetic or fictional corpora—for instance, biographies of fabricated individuals (Allen-Zhu and Li, 2023; Zucchet et al., 2025a; Ou et al., 2025), Wikipedia-style entries for fictional entities (Chang et al., 2024), or post-cutoff factual data (Huang et al., 2024). These studies have yielded valuable insights about how data properties (repetition, diversity), training choices (learning rate, batch size), and curricula (ordering dependencies between facts and their logical implications) affect factual learning. However, a key limitation the paper identifies is that these studies "treat injected items as independent facts" rather than modeling the structured relational knowledge that constitutes a concept. A concept like "dog" isn't just one fact—it's a bundle of related triples (dog IsA mammal, dog CapableOf run, dog HasA four legs, dog HasProperty loyal). Understanding concept learning requires studying how these multiple knowledge relations are acquired together and how they interact.

No prior work connects circuit-level dynamics to behavioral concept learning over time. Perhaps most critically, before this paper, there was essentially no work bridging the gap between the behavioral dynamics of concept acquisition/forgetting and the internal circuit-level changes that accompany them. Studies of training dynamics (Chang et al., 2024; Xu et al., 2024b; Leybzon and Kervadec, 2024) or phase transitions (Ren and Sutherland, 2024; Chen et al., 2023; Jain et al., 2023) have examined how model internals evolve during training, but they have not targeted concept-level representations specifically or linked circuit topology to measurable concept-level behavioral outcomes (logit changes, learning degree, forgetting degree).

Why Existing Methods Can't Answer These Questions Without New Tools

The paper's research questions—how internal concept representations correlate with acquisition and forgetting, and how they relate to interference and synergy among multiple concepts—require a combination of capabilities that no prior study had assembled:

  1. Controlled introduction of genuinely novel concepts: To study concept learning from scratch, the concepts must be new to the model. The paper solves this by constructing the FICO dataset, where real ConceptNet relational structures are mapped to synthetic, non-existent concept names (e.g., replacing "dog" with a fictional name like "Olre"). This preserves realistic knowledge structure—the same five knowledge types (Hyponym & Hypernym, Synonym & Antonym, Meronym & Holonym, Property & Affordance, Spatial Relation) and their interrelations—while ensuring the model has no pre-existing representations of these concepts. Prior fictional-knowledge studies (Allen-Zhu and Li, 2023) focused on factual biographies rather than structured relational knowledge bundles, making them unsuitable for concept-level analysis.

  2. A way to extract and characterize concept-level internal representations: The paper adapts Knowledge Circuits (Yao et al., 2025) to define Concept Circuits—minimal computational subgraphs that can faithfully predict all conceptual knowledge associated with a concept. They operationalize circuit extraction using EAP-IG (Hanna et al., 2024), which assigns importance scores to edges in the model's computational graph, and then characterize circuit structure using four families of graph-theoretic metrics (node importance, redundancy, information flow efficiency, robustness). This provides a quantitative language for describing internal concept representations that can be tracked over time.

  3. A two-stage continual pre-training design that isolates acquisition and forgetting: By training first on FICO (Stage 1, acquisition) and then on the unrelated BIO dataset (Stage 2, forgetting induction), the paper creates a clean experimental separation between the learning phase and the forgetting phase. This enables measurement of concept learning degree (logit change from before to after Stage 1) and forgetting degree (logit change from after Stage 1 to after Stage 2) at both the individual knowledge-triple level and the aggregated concept level.

  4. Controlled cross-concept interference and cross-knowledge synergy experiments: To study how concepts and knowledge types interact, the paper constructs relatedness-based concept groups (using embedding similarity from Qwen3-Embedding-4B) and designs joint-training configurations where target concepts are learned alongside highly related, moderately related, or weakly related auxiliary concepts. For knowledge-type synergy, it uses pairwise continual-training curricula across the five knowledge categories to measure directed transfer effects T(Ri → Rj).

How This Paper Positions Itself

The paper positions itself as filling a specific gap: the study of concept learning dynamics in LLMs as a time-resolved, circuit-level phenomenon involving structured relational knowledge. It extends prior work along multiple dimensions simultaneously:

  • From isolated facts to structured concepts: Unlike factual-knowledge acquisition studies, this paper models concepts as bundles of related knowledge triples drawn from structured knowledge types, requiring models to learn not just individual associations but the relational fabric that defines a concept.

  • From static probing to dynamic tracking: Unlike prompt-based probing or mechanistic localization studies, this paper traces how concept representations (circuits) and concept behavior (logit changes) co-evolve over the course of training, identifying temporal patterns such as the stage-wise circuit reorganization during forgetting (Finding 3).

  • From individual concepts to multi-concept interactions: Unlike studies that examine concepts in isolation, this paper systematically analyzes interference and synergy across concepts (Finding 5, Finding 6), linking cross-concept interference to circuit overlap (Jaccard similarity between concept circuit edge sets, Figure 7) and revealing highly directional knowledge-type transfer effects (Figure 8).

  • From behavioral-only to circuit-informed analysis: The paper's central methodological contribution is demonstrating that circuit-level graph metrics provide a "non-trivial, statistically significant signal of concept learning and forgetting" (Finding 2), establishing that internal circuit structure is not merely correlated with concept behavior but can serve as an indicator or predictor of learning outcomes. This bridges the behavioral and mechanistic lines of research that had previously been pursued separately.

The paper explicitly frames its contributions as foundational for future applied work: the patterns it uncovers (the learn–forget correlation, interference among semantically similar concepts, asymmetric knowledge-type transfer) are intended to "inform the design of more interpretable and robust concept-aware training strategies for LLMs" (Section 1), including "circuit-aware training effort allocation and interference-aware data scheduling" (Limitations). However, the paper is careful to note that it focuses on characterization and analysis—actually developing and testing such training strategies is left to future work. This analytical scope is both a strength (it allows deep, careful characterization) and a limitation (the practical utility of the identified signals remains to be demonstrated in intervention studies).

3. Technical Approach

3.1 Reader Orientation

This paper constructs an analysis framework—not a deployed system—that links the internal computational structure of an LLM to how well it learns and retains individual concepts during continual pre-training. The core problem is characterizing why some concepts are acquired more readily and forgotten more slowly than others, and the solution takes the shape of extracting per-concept computational subgraphs (Concept Circuits), quantifying their topology with graph-theoretic metrics, and correlating those metrics with behavioral measures of learning and forgetting at both the individual-concept and multi-concept levels.

3.2 Big-Picture Architecture (Diagram in Words)

The analysis framework has five major components:

  1. FICO Dataset — a controlled corpus of fictional concepts (synthetic names mapped to real ConceptNet relational structures) that are novel to the pre-trained LLM, enabling study of concept acquisition from scratch without interference from pre-existing knowledge.

  2. Two-Stage Continual Pre-Training Protocol — Stage 1 trains the base LLM on FICO to induce concept acquisition; Stage 2 trains the resulting model on the unrelated BIO dataset to induce forgetting. This cleanly separates learning dynamics from forgetting dynamics.

  3. Concept Circuit Extraction — at each checkpoint (before training, after Stage 1, after Stage 2), the method identifies the minimal computational subgraph in the LLM that can faithfully reproduce the model's predictions on all knowledge triples associated with a given concept, using edge attribution with integrated gradients (EAP-IG).

  4. Graph Metrics Computation — four families of graph-theoretic metrics (node importance via eigenvector centrality variance, redundancy via density, information flow efficiency via global efficiency, robustness via average k-core number) are computed on each extracted concept circuit, producing a quantitative structural fingerprint.

  5. Correlation and Interaction Analysis — behavioral measures (concept learning degree, concept forgetting degree) are computed from logit changes across training stages and correlated with circuit graph metrics; cross-concept interference is measured via joint-training experiments with relatedness-controlled concept groups; cross-knowledge-type synergy is measured via pairwise continual-training curricula.

Information flows as follows: the FICO dataset is constructed from ConceptNet → the base LLM undergoes Stage 1 training on FICO → concept circuits are extracted and metrics are computed at the pre-training checkpoint and post-Stage-1 checkpoint → learning degree is computed as the logit change → the model undergoes Stage 2 training on BIO → concept circuits are re-extracted and metrics are re-computed → forgetting degree is computed as the logit decrease → correlations between metrics and behavioral measures are analyzed → cross-concept interference and cross-knowledge synergy experiments are run independently using relatedness-based concept groupings and pairwise knowledge-type curricula.

3.3 Roadmap for the Deep Dive

  • First, the FICO dataset construction pipeline, because it is the foundation that makes controlled concept learning experiments possible—understanding how fictional concepts are generated, how relational knowledge is templated, and how evaluation is designed to test generalization rather than memorization.
  • Second, the formal definitions of concept learning degree and forgetting degree, because these are the primary behavioral outcome measures that all subsequent analyses aim to explain or predict.
  • Third, the two-stage continual pre-training protocol, because it defines the temporal structure of the experiments and determines which model checkpoints are compared.
  • Fourth, the Concept Circuit definition and extraction procedure via EAP-IG, because this is the bridge between behavior and internal mechanism—it operationalizes what an "internal concept representation" means.
  • Fifth, the four graph metric families and what structural properties they capture, because these are the explanatory variables used to characterize circuit topology.
  • Sixth, the cross-concept interference experimental design (relatedness groups, joint training, circuit overlap analysis), because it extends the analysis from individual concepts to multi-concept interactions.
  • Seventh, the cross-knowledge-type synergy experimental design (pairwise curricula, transferability metric), because it extends the analysis from concept-level to knowledge-type-level interactions.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an empirical analysis paper whose core idea is that concept learning and forgetting dynamics in LLMs can be characterized and partially predicted by the topological properties of their internal Concept Circuits, and that both cross-concept interference and cross-knowledge-type synergy follow structured, predictable patterns.

FICO Dataset Construction

The FICO (FIctional COncept) dataset is the paper's controlled experimental substrate. The fundamental challenge it solves is this: if you want to study how an LLM learns a concept from scratch, you cannot use real concepts like "dog" or "happiness," because the pre-trained model already knows about them. Any measurements of "learning" would be hopelessly confounded by pre-existing knowledge. The solution is to preserve the relational structure of real concepts—which determines what it means to learn a concept—while replacing the surface forms (concept names) with novel tokens that the model has never seen.

Concept selection. The paper samples 1,000 concepts for training and 500 concepts for testing (Section 2, Appendix B Table 2). Concepts are drawn from two sources to span the concrete–abstract spectrum:

  • 500 concrete concepts from the THINGS dataset (Hebart et al., 2023), a large-scale collection of object concepts grounded in sensory experience (e.g., dog, chair, apple).
  • 500 abstract concepts from the Concreteness Ratings dataset (Brysbaert et al., 2014), which provides human ratings of concreteness for 40,000 English word lemmas and includes concepts like "love," "justice," and "freedom" that are not directly perceptible.

The total dataset contains 1,000 training concepts and 500 test concepts (Table 2). This split ensures that evaluation concepts are entirely held out from training, testing genuine generalization to new fictional concepts rather than memorization of training examples.

Knowledge retrieval from ConceptNet. For each concept (both training and test), the authors retrieve associated conceptual knowledge from ConceptNet (Speer et al., 2017), a large-scale multilingual knowledge graph containing millions of concepts connected by 34 typed relations. Each piece of conceptual knowledge is a triple (subject concept, relation type, object). For example, for the concept "dog," ConceptNet might contain:

  • (dog, CapableOf, run)
  • (dog, HasA, four legs)
  • (dog, IsA, animal)
  • (dog, HasProperty, loyal)

The object can be another concept (e.g., "animal") or a descriptive phrase (e.g., "four legs," "run").

Relation type consolidation. The 34 fine-grained relation types in ConceptNet are consolidated into five high-level knowledge categories to facilitate interpretable analysis (Appendix A, Table 1):

  1. Hyponym and Hypernym (HAH): taxonomic and definitional relations. Includes IsA, DefinedAs, FormOf, InstanceOf. Example: (dog, IsA, animal).

  2. Synonym and Antonym (SAA): similarity and contrast relations. Includes Synonym, SimilarTo, Antonym, DistinctFrom. Example: (happy, Antonym, sad).

  3. Meronym and Holonym (MAH): part–whole and compositional relations. Includes PartOf, HasA, MadeOf. Example: (car, HasA, engine).

  4. Property and Affordance (PAA): relations describing properties, functional roles, or what something is used for. Includes HasProperty, UsedFor, CapableOf, ReceivesAction. Example: (knife, UsedFor, cutting).

  5. Spatial Relation (SR): spatial and locational relations. Includes AtLocation, LocatedNear. Example: (fish, AtLocation, water).

Several relation categories are excluded because they primarily encode events, linguistic form, or noisy contextual associations rather than stable conceptual properties: Causality & Event (e.g., Causes, HasPrerequisite, MotivatedByGoal), Desire (e.g., Desires), Lexical/Etymological (e.g., DerivedFrom, EtymologicallyRelatedTo), and Other (e.g., RelatedTo, HasContext).

Fictional name substitution. This is the critical step that makes the concepts novel. For each real concept name (e.g., "dog"), the authors use GPT-5 (Hurst et al., 2024) to generate a fictional replacement name (e.g., "Olre") that does not correspond to any real-world entity. The conceptual knowledge structure is preserved: all triples associated with "dog" are rewritten with the fictional name, so (dog, CapableOf, run) becomes (Olre, CapableOf, run). The relational content remains realistic—"Olre" has the same capabilities, properties, and taxonomic relations as "dog"—but the surface form is novel. This ensures that any behavioral changes during training on FICO genuinely reflect concept acquisition rather than reactivation of pre-existing knowledge.

Template-based text generation. Knowledge triples are not directly usable for language model training. Following prior work (Zucchet et al., 2025b), the authors use GPT-5 to produce multiple natural-language templates for each relation type. For example, for the CapableOf relation, templates might include:

  • "{concept} has the ability to {object}"
  • "{concept} can {object}"
  • "One thing {concept} is capable of is {object}"

Each knowledge triple is instantiated into multiple training examples by substituting the concept name and object into these templates. The training examples follow a prefix–target format: the model receives the template up to the object position as input (prefix) and is trained to predict the object tokens as output (target). For example, given the prefix "Olre has the ability to", the target is " run".

Train–test template separation to test generalization. A critical design choice: the templates used for constructing training examples and evaluation examples are drawn from disjoint pools. This means the evaluation examples use surface forms the model has never seen during training, even though they probe the same underlying knowledge triples. If the model has genuinely learned the relational structure (i.e., that "Olre" is capable of running), it should succeed on novel templates like "A notable capability of Olre is to" → " run". If the model has merely memorized specific template–object co-occurrences, it will fail on the evaluation templates. This design ensures that accuracy on the FICO test set measures conceptual knowledge acquisition rather than surface-level pattern matching.

Dataset statistics (Appendix B, Table 2). The training set contains 1,000 concepts with 3,075 knowledge triples, instantiated into 92,250 training samples using multiple templates per triple, totaling approximately 1.04 million tokens. The test set contains 500 concepts with 1,586 knowledge triples, instantiated into 1,586 evaluation samples (one template per triple, disjoint from training templates), totaling 13,530 tokens. Training is conducted for 10 epochs, yielding 10.4 million total training tokens, which the authors note is "similar as previous continual-pretraining work."

Formal Definitions of Learning and Forgetting Degrees

The paper needs quantitative behavioral measures that capture how much a concept is learned during acquisition and how much it is forgotten during subsequent training. These measures are defined at two levels of granularity: individual knowledge triples and aggregated concepts.

Definition 1: Knowledge Learning/Forgetting Degree (individual triple level). For a single conceptual knowledge triple $k = (c, r, o)$, where $c$ is the subject concept (the fictional name), $r$ is the relation type, and $o$ is the target object (what the model should generate):

Knowledge Learning Degree(k)=logitπ1(o∣prefix(c,r))−logitπ0(o∣prefix(c,r))\text{Knowledge Learning Degree}(k) = \text{logit}_{\pi_1}(o \mid \text{prefix}(c, r)) - \text{logit}_{\pi_0}(o \mid \text{prefix}(c, r))

Knowledge Forgetting Degree(k)=logitπ1(o∣prefix(c,r))−logitπ2(o∣prefix(c,r))\text{Knowledge Forgetting Degree}(k) = \text{logit}_{\pi_1}(o \mid \text{prefix}(c, r)) - \text{logit}_{\pi_2}(o \mid \text{prefix}(c, r))

where $\pi_0$ is the base pre-trained model (before any FICO training), $\pi_1$ is the model after Stage 1 (FICO acquisition training), $\pi_2$ is the model after Stage 2 (BIO forgetting-induction training), and $\text{logit}_{\pi}(o \mid \text{prefix}(c,r))$ is the unnormalized logit (pre-softmax score) assigned to the correct target token(s) $o$ by model $\pi$ when given the textual prefix constructed from $(c, r)$ using evaluation templates.

What this computes: the learning degree is the increase in the model's confidence in the correct target object after learning the concept, measured in logit space. A positive value means the model became more likely to generate the correct object. The forgetting degree is the decrease in that confidence after subsequent unrelated training, measured as the logit difference between the post-acquisition and post-forgetting checkpoints. A positive forgetting degree means the model's confidence dropped (it forgot).

Why logits rather than probabilities: the paper notes it follows prior work (Hanna et al., 2024) in using logits as a "fine-grained indicator of LLM learning." Logits are unbounded and linear in the model's internal representations, making them more sensitive to subtle changes than probabilities (which are compressed by the softmax and saturate near 0 or 1). The paper also reports results for log probability in Appendix C as a robustness check.

Definition 2: Concept Learning/Forgetting Degree (aggregated over a concept). For a concept $c$ with associated conceptual knowledge triples $\{k_0, k_1, \ldots, k_j, \ldots\}$, the concept-level measure is the arithmetic mean over its knowledge triples:

Concept Learning Degree(c)=1∣{ki}∣∑iϕlearn(ki)\text{Concept Learning Degree}(c) = \frac{1}{|\{k_i\}|} \sum_{i} \phi_{\text{learn}}(k_i)

Concept Forgetting Degree(c)=1∣{ki}∣∑iϕforget(ki)\text{Concept Forgetting Degree}(c) = \frac{1}{|\{k_i\}|} \sum_{i} \phi_{\text{forget}}(k_i)

where $\phi_{\text{learn}}(k_i)$ is the Knowledge Learning Degree of triple $k_i$, $\phi_{\text{forget}}(k_i)$ is the Knowledge Forgetting Degree, and $|\{k_i\}|$ is the number of knowledge triples associated with concept $c$.

What this computes: a single scalar per concept that summarizes how much, on average, the model's predictions improved (learning) or deteriorated (forgetting) across all the relational knowledge associated with that concept.

Why arithmetic mean: it treats all knowledge triples as equally important for defining the concept. A weighted mean (e.g., by relation type frequency or by some measure of triple centrality) might capture different aspects, but the paper opts for simplicity and interpretability—the mean has a direct interpretation as the average logit shift per knowledge triple.

Two-Stage Continual Pre-Training Protocol

The experimental design creates a clean temporal separation between concept acquisition and concept forgetting. This is essential because in a single training run, learning and forgetting are entangled: new concepts are being acquired while old ones may be simultaneously eroded, and it is impossible to attribute changes to one process versus the other.

Stage 1: Concept Acquisition. The base pre-trained LLM $\pi_0$ (GPT-2 Large at 0.7B parameters or LLaMA-3.2-1B-Instruct) is continually trained on the FICO training set. The training format is standard next-token prediction: for each training example, the prefix (template with concept and relation instantiated) and target object are concatenated into a single sequence, and the model is optimized to predict the target tokens given the prefix tokens. All model parameters are updated—this is full fine-tuning, not parameter-efficient adaptation.

Key training hyperparameters (Appendix E): learning rate $5 \times 10^{-5}$, batch size 128, 10 epochs over the FICO training data (totaling 10.4 million tokens), trained on 4 × 40GB A40 GPUs. The resulting model is $\pi_1$.

Why FICO first: because the fictional concepts are novel to the base model, any improvement in predicting their target objects can be cleanly attributed to new learning rather than reactivation of pre-existing representations.

Stage 2: Forgetting Induction. Starting from $\pi_1$ (the model that has learned FICO concepts), the model is further trained on the BIO dataset (Allen-Zhu and Li, 2023). The BIO dataset is a standard pre-training corpus consisting of biographical texts about (real or fabricated) individuals—it contains factual information about people but does not contain the fictional concepts from FICO or their associated relational knowledge types. This makes it an appropriate forgetting-induction dataset: it introduces new, unrelated knowledge that may overwrite or interfere with the FICO concept representations learned in Stage 1, without directly retraining on FICO concepts.

The training hyperparameters for Stage 2 are the same as Stage 1 (learning rate $5 \times 10^{-5}$, batch size 128), matching the Stage 1 training steps in duration to ensure a fair comparison. The resulting model is $\pi_2$.

Why BIO: because it is semantically unrelated to the conceptual knowledge in FICO (biographies vs. relational concept knowledge), any degradation in FICO performance after BIO training can be attributed to catastrophic forgetting or representational interference rather than to contradictory training signals about the concepts themselves. If the Stage 2 data contained information about the same fictional concepts but with different relational knowledge, the cause of performance changes would be ambiguous—is the model "forgetting" the old knowledge or is it being actively overwritten by contradictory new knowledge? BIO avoids this confound.

Checkpoint comparison logic. Three checkpoints are compared:

  • $\pi_0$ (base model) and $\pi_1$ (post-FICO) for learning analysis: the difference captures what was acquired during concept learning.
  • $\pi_1$ (post-FICO) and $\pi_2$ (post-BIO) for forgetting analysis: the difference captures what was lost during unrelated continued training.
  • Concept circuits are extracted from all three checkpoints, enabling analysis of how circuit structure changes from baseline → after learning → after forgetting.

Concept Circuit Definition and Extraction

The central mechanistic construct in this paper is the Concept Circuit. To understand it, we need to start with the prior notion it builds on.

Background: Knowledge Circuits (Yao et al., 2025). Yao et al. (2025) model a pre-trained transformer as a directed acyclic graph (DAG). In this graph:

  • Nodes correspond to computational components in the model's forward pass: individual neurons in MLP layers, attention heads, input embeddings, and residual stream representations at each layer.
  • Edges capture the information flow between these components: residual connections (where a component's output is added to the residual stream), attention operations (where one token's representation influences another's), and linear projections within attention and MLP blocks.

Given a specific knowledge triple $k_{ij} = (c_i, r_{ij}, o_{ij})$, a Knowledge Circuit is defined as the minimal computational subgraph that can faithfully predict the target object $o_{ij}$ when the model is given a textual prefix derived from the subject–relation pair $(c_i, r_{ij})$. "Minimal" means the smallest set of edges whose combined computation reproduces the full model's output for that specific prediction. "Faithfully" means the subgraph's output closely approximates the full model's output (e.g., preserving a specified fraction of performance).

Adaptation to concepts: Concept Circuits. The paper adapts this notion from individual knowledge triples to full concepts. A concept is defined by a bundle of related knowledge triples—for a fictional concept "Olre," we have (Olre, IsA, animal), (Olre, CapableOf, run), (Olre, HasA, four legs), etc. The Concept Circuit for concept $c_i$ is the computational subgraph that can faithfully predict all conceptual knowledge $\{k_{i0}, k_{i1}, \ldots\}$ associated with $c_i$. This subgraph thus represents the model's internal parametric representation of the entire concept—the ensemble of computational pathways that collectively encode what the model "knows" about that concept.

Why a circuit rather than a single vector: unlike some interpretability work that localizes concepts to individual neurons or directions in activation space, the circuit perspective acknowledges that conceptual knowledge may be distributed across multiple components (attention heads for relational binding, MLP neurons for attribute storage, etc.) that interact through specific edges. The circuit is the full computational pathway, not just the storage location.

Extraction method: EAP-IG (Edge Attribution Patching with Integrated Gradients). The paper uses EAP-IG (Hanna et al., 2024) to identify which edges belong to a concept circuit. EAP-IG is a method for assigning an importance score to each edge in the model's computational graph, quantifying how much that edge contributes to a specific prediction.

The procedure works as follows:

  1. Define a target metric: for a concept $c$, the metric is the model's average logit or probability assigned to the correct target objects across all knowledge triples of $c$, given the appropriate evaluation prefixes.

  2. Compute edge importance via integrated gradients: For each edge in the model's DAG, EAP-IG estimates how much ablating (zeroing out) that edge would change the target metric, using integrated gradients to attribute the prediction to individual edges. Integrated gradients compute the path integral of gradients along a straight line from a baseline (e.g., zero activation) to the actual activation, providing an axiomatically justified attribution score for each edge. The result is a scalar importance score per edge, where higher scores indicate edges that are more critical for the concept's predictions.

  3. Construct the circuit by thresholding: Given the ranked list of edge importance scores, the circuit is constructed by selecting the top-scoring edges such that the resulting subgraph preserves at least 70% of the full model's performance on the corresponding concept. In other words: start with the most important edge, add the next most important, and continue until the subgraph's predictions on the concept's knowledge triples reach 70% of the full model's accuracy/logit level. This 70% threshold balances circuit minimality (fewer edges = more interpretable) with faithfulness (enough edges to capture the essential computation).

The authors note that more details for EAP-IG implementation can be found in Hanna et al. (2024). The key properties of EAP-IG that make it suitable here are: (1) it is computationally efficient compared to exhaustive ablation (it uses gradient-based attribution rather than actually running the model with each edge ablated), and (2) it is faithful—the integrated gradients formulation provides theoretical guarantees about attribution accuracy under certain assumptions.

When and for which checkpoints circuits are extracted. Concept circuits are extracted at three time points per concept:

  • From $\pi_0$ (base model, before any FICO training): this establishes the baseline circuit structure for a concept the model has not yet learned.
  • From $\pi_1$ (after Stage 1, FICO acquisition): this captures the circuit structure after the concept has been learned.
  • From $\pi_2$ (after Stage 2, BIO forgetting induction): this captures the circuit structure after some degree of forgetting has occurred.

By comparing circuits across these checkpoints, the paper can track how concept representations reorganize during learning and forgetting. The graph metrics (described next) are computed on each checkpoint's circuit, enabling quantitative comparison.

Graph Metrics for Characterizing Concept Circuit Structure

Once a concept circuit is extracted as a subgraph (a set of nodes and edges from the model's computational DAG), the paper characterizes its topology using four families of standard graph-theoretic metrics. Each family captures a different structural property hypothesized to relate to concept learning or forgetting dynamics. All metrics are computed on the extracted circuit subgraph for each concept at each checkpoint.

1. Node Importance (eigenvector centrality variance).

The metric: the standard deviation of eigenvector centrality across all nodes in the concept circuit.

Eigenvector centrality (Newman, 2010) is a measure of node influence in a graph. Unlike simple degree centrality (which just counts connections), eigenvector centrality weights a node's importance by the importance of its neighbors—a node connected to other high-importance nodes is itself more important. Formally, for a graph with adjacency matrix $\mathbf{A}$, the eigenvector centrality $x_i$ of node $i$ satisfies:

Ax=λx\mathbf{A}\mathbf{x} = \lambda \mathbf{x}

where $\lambda$ is the largest eigenvalue and $\mathbf{x}$ is the corresponding eigenvector. Node $i$'s centrality $x_i$ is proportional to the sum of its neighbors' centralities.

What the standard deviation measures: how unevenly structural influence is distributed across the circuit's nodes. A high standard deviation means a few "hub" nodes dominate the circuit (most importance concentrated in a small set of components), while a low standard deviation indicates a more evenly distributed, decentralized structure where many nodes contribute approximately equally.

Hypothesized relationship to learning: circuits with higher eigenvector centrality variance (more centralized hub structure) may facilitate faster learning because updates to a small number of critical nodes can propagate coordinated changes throughout the concept representation. The paper states this explicitly: "higher variance indicates a more concentrated hub structure, which may facilitate concept acquisition but increase vulnerability to interference or forgetting."

Hypothesized relationship to forgetting: the same centralization that aids learning may increase vulnerability—if forgetting is driven by perturbations to hub nodes (e.g., their parameters being overwritten during subsequent training on BIO), then circuits with concentrated influence are more fragile. Perturbing one hub can degrade the entire concept representation, whereas in a decentralized circuit, damage to any single node is localized.

2. Redundancy (circuit density).

The metric: density (Newman, 2010), defined as the ratio of existing edges to the maximum possible number of edges in the circuit subgraph.

Density=∣E∣∣V∣(∣V∣−1)\text{Density} = \frac{|E|}{|V|(|V| - 1)}

where $|E|$ is the number of edges in the circuit and $|V|$ is the number of nodes. Density ranges from 0 (no edges) to 1 (every possible edge present, a complete graph).

What it measures: the degree of redundant connectivity in the circuit. A dense circuit has many alternative pathways between any two nodes—if one edge is disrupted, information can still flow through other routes. A sparse circuit has fewer alternative pathways, making it more brittle to edge-level perturbations.

Hypothesized relationship to learning: higher density (more redundancy) may help learning because redundant pathways can reinforce learning signals—gradient updates can propagate through multiple routes, and the circuit can maintain functionality even if some edges are suboptimally initialized.

Hypothesized relationship to forgetting: the paper observes a surprising positive correlation between density and forgetting (Finding 2), suggesting that "highly redundant and tightly interconnected circuits entangle concept representations more strongly with other knowledge, amplifying interference during continued training." In other words, more edges means more points of contact with other concepts' circuits, creating more opportunities for cross-concept interference when those other circuits are modified during subsequent training.

3. Information Flow Efficiency (global efficiency).

The metric: global efficiency (Latora and Marchiori, 2001), defined as the average inverse shortest-path distance between all pairs of nodes in the circuit.

Global Efficiency=1∣V∣(∣V∣−1)∑i≠j1d(i,j)\text{Global Efficiency} = \frac{1}{|V|(|V| - 1)} \sum_{i \neq j} \frac{1}{d(i, j)}

where $d(i, j)$ is the shortest-path distance (minimum number of edges) between nodes $i$ and $j$. If no path exists, $1/d(i,j) = 0$ for that pair. Higher global efficiency means that, on average, nodes are closer together in the graph—signals can propagate between any two nodes in fewer hops.

What it measures: how efficiently information can flow through the circuit. A circuit with high global efficiency is well-integrated—computation at one node can rapidly influence computation at another node. A circuit with low global efficiency has bottlenecks or isolated components where information must traverse many intermediate nodes.

Why inverse shortest path: the inverse $1/d(i,j)$ ensures that the metric is well-defined even for disconnected graphs (where $d(i,j) = \infty$ would otherwise make the average undefined). It also has a natural interpretation: direct connections contribute maximally (1/1 = 1), connections through one intermediate node contribute 1/2, and so on.

Hypothesized relationship to learning: higher global efficiency (shorter paths) may facilitate learning because gradient signals can propagate more directly between components that need to coordinate. In a circuit with short paths, an update to one component can quickly influence related components, enabling faster convergence.

Hypothesized relationship to forgetting: similar to density, higher global efficiency may increase vulnerability to interference because tightly integrated circuits share information more broadly, making it harder to isolate concept-specific computation from perturbations introduced by subsequent training on unrelated data.

4. Robustness (average k-core number).

The metric: the average k-core number (Seidman, 1983) across all nodes in the circuit.

A k-core of a graph is a maximal subgraph in which every node has degree at least $k$ (considering only edges within the subgraph). The core number of a node is the largest $k$ for which that node belongs to a k-core. In other words: a node's core number is the depth of the densely connected core structure it participates in.

The procedure: iteratively remove all nodes with degree less than $k$, for increasing $k$ from 1 upward. A node's core number is the highest $k$ at which it survives before being removed. The average k-core number is the mean of these per-node core numbers across the circuit.

What it measures: the depth and resilience of the circuit's densely connected core. A high average k-core number indicates that the circuit contains a deeply nested set of highly interconnected nodes—a robust structural backbone that is difficult to fragment by removing individual nodes or edges.

Hypothesized relationship to learning: circuits with deeper core structures (higher k-core) may learn more effectively because the core provides a stable scaffold for concept representation. Updates to peripheral nodes can be anchored by the stable core, preventing the representation from drifting during training.

Hypothesized relationship to forgetting: the paper finds that higher k-core depth is actually associated with increased forgetting (Finding 2), which is counterintuitive given the "robustness" framing. The authors' interpretation is that "highly redundant and tightly interconnected circuits entangle concept representations more strongly with other knowledge"—the same structural property that makes the core resistant to random damage also makes it a locus of shared computation, and when subsequent training modifies parameters in the core (because BIO and FICO share some computational pathways), the entanglement amplifies interference.

Summary of hypothesized structural trade-off. The paper's central mechanistic hypothesis (made explicit in Finding 2) is a structural trade-off: circuits that are centralized (high eigenvector centrality variance), dense, efficient, and deep-cored facilitate rapid and effective concept acquisition because they enable coordinated, reinforced learning. However, these same properties increase vulnerability to forgetting because they create more points of entanglement with other knowledge, and when subsequent training perturbs shared components (especially hub nodes), the tightly integrated structure propagates the disruption broadly. Conversely, more modular, decentralized circuits with lower density and shallower cores may learn more slowly but retain knowledge better under interference. This trade-off is tested empirically through the correlation analysis between circuit metrics and behavioral measures (Section 3.3, Finding 2 and Finding 4).

Cross-Concept Interference Experimental Design

The paper studies how concepts influence each other during joint training by constructing relatedness-controlled concept groups and measuring how co-training with concepts of varying similarity affects learning performance.

Relatedness definition. The relatedness between two concepts is defined as the cosine similarity between their token-level embedding representations, obtained using the Qwen3-Embedding-4B model (Yang et al., 2025). This is an external, frozen embedding model—not the LLM being trained. The intuition is that concepts with similar embeddings in a general-purpose embedding space are semantically related (e.g., "dog" and "cat" have similar embeddings, while "dog" and "justice" do not). By using an external embedder rather than the training model's own representations, the relatedness measure is independent of the training process and not confounded by the model's current state.

Group construction. For each target concept $c$ in the FICO test set, the authors compute pairwise cosine similarities between $c$ and all other concepts (presumably from both the training and test sets, though the paper does not specify whether test concepts are included in the pool of auxiliary concepts). Concepts are ranked by similarity to $c$, and three groups of size $K = 100$ are constructed:

  • Highly related group: the top-K most similar concepts.
  • Moderately related group: the middle-K concepts by similarity rank.
  • Weakly related group: the bottom-K least similar concepts.

Joint-training configuration. For each target concept $c$, three separate training runs are conducted where $c$ is trained together with one of the three auxiliary concept groups. In each run, the model is trained on the FICO training data for both $c$ and the auxiliary concepts simultaneously (joint training). After training, the model's performance on $c$ alone is evaluated—auxiliary concept performance is not measured. The evaluation metrics are:

  1. The average logit assigned to the correct target objects for knowledge triples associated with $c$.
  2. The corresponding average probability (softmax of logits).

What this design isolates: by varying only the relatedness of the co-trained concepts while holding the target concept constant, any performance differences can be attributed to cross-concept interference. If highly related auxiliary concepts reduce performance on $c$ relative to weakly related ones, this indicates interference—the auxiliary concepts' learning is competing for or overwriting the computational resources needed by $c$.

Circuit overlap analysis (post-hoc mechanistic explanation). To understand why relatedness causes interference, the paper examines the overlap between concept circuits. For each target concept $c$ and each auxiliary concept $c'$ it was co-trained with, the authors extract the Concept Circuits from the trained model and compute the Jaccard similarity between their edge sets:

Jaccard(c,c′)=∣Ec∩Ec′∣∣Ec∪Ec′∣\text{Jaccard}(c, c') = \frac{|E_c \cap E_{c'}|}{|E_c \cup E_{c'}|}

where $E_c$ and $E_{c'}$ are the sets of edges in the Concept Circuits of $c$ and $c'$, respectively. Jaccard similarity ranges from 0 (completely disjoint circuits) to 1 (identical circuits). Higher Jaccard similarity means the two concepts share more computational infrastructure.

Why Jaccard similarity: it is a standard set-overlap metric that captures the proportion of shared edges relative to the total number of unique edges across both circuits. It accounts for circuit size differences—two large circuits with many shared edges will have a higher Jaccard than two small circuits with the same number of shared edges but fewer total unique edges. This is appropriate because interference depends on the proportion of shared computation, not just the raw number of shared edges.

The paper visualizes the Jaccard similarity distributions for the three relatedness groups using Kernel Density Estimation (KDE) plots (Figure 7), comparing how circuit overlap varies with semantic similarity.

Cross-Knowledge-Type Synergy Experimental Design

Beyond interactions between concepts, the paper studies whether different types of conceptual knowledge can facilitate or interfere with each other's learning, even when they describe the same concept.

Knowledge types. The five high-level knowledge categories from the FICO dataset construction (Appendix A, Table 1) serve as the knowledge types: Hyponym & Hypernym (HAH), Synonym & Antonym (SAA), Meronym & Holonym (MAH), Property & Affordance (PAA), and Spatial Relation (SR). Each type captures a distinct semantic facet of concepts—taxonomic relations, similarity/contrast, part–whole composition, functional properties, and spatial location, respectively.

Pairwise continual-training setup. The experiment uses a two-stage design analogous to the main acquisition–forgetting protocol, but with curriculum ordering as the manipulated variable:

  • Stage 1: the base model $\pi_0$ is trained either on knowledge category $R_i$ (one of the five FICO knowledge types) or on the BIO dataset as a control baseline. Training is for the same number of steps regardless of which dataset is used, ensuring fair comparison.
  • Stage 2: the resulting model is continually trained on knowledge category $R_j$ (a different knowledge type from $R_i$). The model is then evaluated on its performance predicting $R_j$ triple targets.

This produces $5 \times 4 = 20$ ordered curricula (all ordered pairs of distinct knowledge types), plus the BIO-based control baseline (where Stage 1 is BIO, Stage 2 is each of the five knowledge types).

Transferability metric. The key quantitative measure is the paired transferability $T(R_i \rightarrow R_j)$, which quantifies how much Stage 1 pretraining on $R_i$ helps or hurts subsequent learning of $R_j$ compared to the BIO baseline:

T(Ri→Rj)=logit(Rj∣Ri)−logit(Rj∣BIO)∣logit(Rj∣BIO)∣T(R_i \rightarrow R_j) = \frac{\text{logit}(R_j \mid R_i) - \text{logit}(R_j \mid \text{BIO})}{|\text{logit}(R_j \mid \text{BIO})|}

where:

  • $\text{logit}(R_j \mid R_i)$ is the average logit assigned to correct targets for knowledge type $R_j$ after Stage 2, when Stage 1 pretraining was on $R_i$.
  • $\text{logit}(R_j \mid \text{BIO})$ is the average logit for $R_j$ after Stage 2, when Stage 1 pretraining was on BIO (the control condition).

What this computes: the proportional change in logit performance when $R_i$ is used as the Stage 1 curriculum instead of BIO. A positive value means pretraining on $R_i$ improved subsequent $R_j$ learning relative to the control—this is synergy (positive transfer). A negative value means pretraining on $R_i$ impaired subsequent $R_j$ learning relative to the control—this is interference (negative transfer). The absolute value in the denominator normalizes by the baseline performance, making the metric interpretable as a proportional effect: $T = 0.2$ means a 20% improvement over BIO baseline, $T = -0.15$ means a 15% degradation.

Why relative to BIO baseline: using BIO as the control isolates the effect of the specific knowledge type $R_i$ from the general effect of any continued pretraining. Both $R_i$ and BIO involve training on new tokens—if the only effect of Stage 1 was general language adaptation, BIO and $R_i$ would produce similar outcomes on Stage 2 $R_j$ performance. Differences must therefore be due to the specific semantic content of $R_i$ transferring (positively or negatively) to $R_j$.

Why the absolute value in the denominator: normalization by $|\text{logit}(R_j \mid \text{BIO})|$ ensures the metric is scale-invariant. If $R_j$ is generally easy to learn (high baseline logits), a given absolute improvement represents a smaller proportional gain than if $R_j$ is hard to learn (low baseline logits). The proportional metric $T$ captures this and makes transfer effects comparable across different target knowledge types.

What the 5×5 heatmap in Figure 8 shows: each cell $(i, j)$ displays $T(R_i \rightarrow R_j)$ for source type $R_i$ (rows) and target type $R_j$ (columns). The diagonal is not applicable (a knowledge type cannot be both source and target in this design). Asymmetric transfer is visible when $T(R_i \rightarrow R_j) \neq T(R_j \rightarrow R_i)$—for example, pretraining on PAA may strongly benefit subsequent HAH learning, but pretraining on HAH may only weakly benefit subsequent PAA learning.

Why this matters for curriculum design: if certain knowledge types reliably scaffold others, training curricula can be ordered to place high-transfer source types early, maximizing the efficiency of downstream concept learning. The paper's Finding 6 quantifies these transfer relationships and identifies which knowledge-type pairings are most synergistic.

4. Key Insights and Innovations

Innovation 1: Concept Circuits as a Dynamic, Time-Resolved Internal Representation — Not Just a Static Localization

The paper's most fundamental conceptual move is taking circuit-level interpretability from a static anatomy of already-trained models to a dynamic physiology that co-evolves with learning and forgetting. Prior mechanistic interpretability work on concepts (Aljaafari et al., 2024; Wang et al., 2024c) answered the question "where is concept X represented in the model?" by identifying attention heads, MLP layers, or editable sub-networks associated with conceptual knowledge. This is valuable but fundamentally anatomical — it tells you which components matter at a single frozen time point. The paper reframes the question entirely: instead of asking where a concept lives, it asks how the concept's internal representation changes shape during acquisition and degradation, and whether those shape changes carry predictive signal about behavioral outcomes.

This shift from localization to dynamics is more than a methodological extension — it changes what "understanding concept learning" means. Under the localization paradigm, success is finding the right components. Under the dynamics paradigm, success is finding systematic relationships between how the circuit is organized (its topology) and how the concept behaves (how fast it's learned, how resistant it is to forgetting). The four graph metric families — eigenvector centrality variance, density, global efficiency, and k-core depth — are not chosen because they identify where computation happens, but because they operationalize structural hypotheses about why some representations are more learnable or more fragile than others. A circuit with high eigenvector centrality variance (concentrated hub structure) and a circuit with low variance (decentralized structure) might both faithfully represent the same concept at a given snapshot, but they imply radically different dynamics under perturbation — and it's those dynamics, not the snapshot, that matter for continual learning.

What makes this genuinely distinctive rather than an obvious combination of existing tools (circuit extraction + graph metrics + training dynamics) is the specific structural trade-off it reveals (Finding 2 and Finding 4). The positive correlations between circuit density/centrality/efficiency and both learning degree and forgetting degree mean that the same topological properties that enable rapid acquisition also create vulnerability. This is a non-obvious empirical result — one might have expected robustness metrics (k-core depth) to predict resistance to forgetting, since "robustness" intuitively suggests stability. Instead, the data shows the opposite: the most structurally robust circuits are the most forgetful. The paper's interpretation — that dense, integrated circuits entangle concept representations more strongly with other knowledge, amplifying interference during continued training — reframes the problem of catastrophic forgetting not as passive decay of isolated representations, but as active interference propagated through shared computational infrastructure. This is a fundamentally different mechanistic story than "overwriting," and it has direct implications: if forgetting is driven by entanglement, then the solution is not just more replay or regularization, but architectural or training strategies that promote modular circuit organization — keeping concept circuits sparse and decentralized despite the learning efficiency penalty that entails.

The temporal trajectory finding (Finding 3) — that circuit metrics follow a consistent stage-wise pattern during forgetting (early increase, gradual decrease, stabilization) — reinforces this reframing. The early-phase increase in metrics like density and global efficiency suggests that forgetting is not a monotonic erosion of concept circuits but an active reorganization: as the model trains on BIO data, previously learned concept circuits initially become transiently more entangled with the new knowledge before relaxing into a weaker, more stable configuration. This is a dynamic signature consistent with the interference-through-entanglement hypothesis, and it would be invisible to any static localization approach. It also suggests distinct phases of concept consolidation that could be targeted by training interventions — for example, protecting circuits during the early high-interference phase through gradient shielding or replay, then allowing natural stabilization.

The significance of this innovation is primarily theoretical and diagnostic, not performance-oriented. The paper doesn't claim that measuring circuit graph metrics improves accuracy — it claims that these metrics provide a "non-trivial, statistically significant signal of concept learning and forgetting" (Finding 2). This is a claim about what kind of information is available inside the model that we previously couldn't access: the internal structure of concept representations carries predictive information about their fate under continued training. Whether that information is actionable (can we intervene on circuits to change outcomes?) remains an open question the paper explicitly defers to future work. But establishing that the signal exists is a necessary first step toward circuit-aware training strategies.

Evidence anchor: Figures 3a and 3b (Spearman correlations between circuit graph metrics and learning/forgetting degree), Figure 4 (temporal trajectory of graph metrics during forgetting stage), Figure 5 (positive correlation between learning degree and forgetting degree).

Innovation 2: The Learn–Forget Correlation as an Empirical Demonstration of a Structural Stability–Plasticity Trade-Off at the Concept Level

Finding 4 states that "concepts with larger learning degree tend to exhibit larger forgetting degree during subsequent training." On its surface, this sounds like a restatement of the well-known stability–plasticity dilemma in continual learning — the tension between acquiring new knowledge and retaining old knowledge. But the paper's contribution is not the observation that forgetting happens; it's the operationalization of this trade-off at the granularity of individual concepts, linked to circuit-level structural properties, in a setting where the forgetting is induced by unrelated data rather than contradictory training on the same concepts.

Prior continual learning work has studied catastrophic forgetting primarily at the task level — e.g., after fine-tuning on Task B, performance on Task A drops — and has proposed mitigation strategies like elastic weight consolidation (EWC), experience replay, or progressive networks. These studies treat "Task A" as a monolithic block of knowledge and measure aggregate forgetting across all its examples. The paper's contribution is to show that even within a single task (learning FICO concepts), there is substantial heterogeneity in how individual concepts respond to the same forgetting induction: the distribution of forgetting degrees is widely spread (Figure 2), and this variance is systematically related to how aggressively each concept was learned.

Why this matters beyond the task-level observation: it implies that aggregate forgetting metrics mask concept-level dynamics that could inform targeted interventions. If you only measure average accuracy on FICO before and after BIO training, you miss the fact that some concepts are essentially immune to forgetting (low learning degree, low forgetting degree) while others are highly vulnerable (high learning degree, high forgetting degree). A uniform mitigation strategy — say, replaying 10% of FICO examples during BIO training — would be simultaneously wasteful for the immune concepts (replaying knowledge that isn't at risk) and insufficient for the vulnerable ones (not enough replay to protect the most fragile representations). Concept-level heterogeneity calls for concept-aware mitigation: identify which concepts are likely to be forgotten (perhaps via their circuit graph metrics or their learning trajectory) and allocate protective resources disproportionately to them.

The link to circuit topology deepens the significance. The paper doesn't just observe the learn–forget correlation; it provides a mechanistic hypothesis for why it occurs: "During learning, concepts with larger gains are often supported by circuits that are more integrated and strongly connected, which can enable coordinated updates... However, the same integration may increase overlap with subsequently trained knowledge." This transforms the correlation from a mere empirical regularity into a structural explanation: the properties that make a concept easy to learn (dense, centralized, efficient circuits) are the same properties that make it easy to disrupt (high entanglement with other knowledge). If this causal story is correct — and the paper is careful to present it as an interpretation, not a proven mechanism — then the stability–plasticity trade-off at the concept level is not something that can be "solved" by better optimization; it's a fundamental consequence of how concept representations are structured in the model's computational graph. You can't have a circuit that is simultaneously maximally integrated (good for learning) and maximally modular (good for retention) — there's an inherent topological tension.

This framing connects the paper's empirical findings to broader debates in neuroscience and machine learning about whether catastrophic forgetting is a bug (something to be engineered away) or a feature (an unavoidable consequence of distributed representations in gradient-based learning systems). By showing that the trade-off manifests at the level of individual concepts with measurable circuit signatures, the paper provides a concrete empirical platform for investigating this question.

Evidence anchor: Figure 5 (scatter plot showing positive Spearman correlation between learning degree and forgetting degree), Figures 3a and 3b (circuit graph metric correlations showing same properties positively correlated with both learning and forgetting).

Innovation 3: From Isolated Facts to Structured Relational Knowledge — Reframing "Knowledge Acquisition" as Concept-Level Bundle Learning

The paper makes a deliberate and consequential choice to study conceptual knowledge as structured bundles of relational triples rather than as isolated facts. This is not just a dataset design decision — it's a conceptual reframing of what it means for an LLM to "learn something." Prior studies of LLM knowledge acquisition (Allen-Zhu and Li, 2023; Zucchet et al., 2025a; Ou et al., 2025; Chang et al., 2024) operationalized knowledge as individual factual associations — e.g., "Person X was born in City Y" or "Entity Z has property W." These studies revealed important dynamics about how repetition, data ordering, and training hyperparameters affect fact learning, but they treated each fact as an independent atomic unit. The implicit model of knowledge was a lookup table: the model learns a mapping from subject–relation pairs to object values, one entry at a time.

The paper argues, through its experimental design, that this atomic-fact model is insufficient for understanding concept learning. A concept like "dog" isn't just a collection of independent facts {(dog, IsA, animal), (dog, CapableOf, run), (dog, HasA, four legs)} — it's a structured entity where the facts are mutually constraining and collectively defining. In the FICO dataset, each fictional concept is associated with multiple knowledge triples spanning different relation types (taxonomic, functional, part–whole, spatial, similarity). Learning the concept means learning all of these relations, and — crucially — learning them in a way that respects their interrelations. The Concept Circuit, defined as the subgraph that faithfully predicts all knowledge triples for a concept (not just one), operationalizes this bundle-level representation.

Why this reframing matters:

It changes the evaluation of learning. In an atomic-fact framework, learning is measured by whether the model can retrieve each fact individually. In the bundle framework, learning is measured by whether the model has formed an integrated representation that supports all facts simultaneously. A model might be able to retrieve (fictional_concept_X, CapableOf, run) with high probability but fail on (fictional_concept_X, IsA, animal) — in the atomic view, this is partial learning; in the bundle view, this is an incomplete concept representation. The paper's concept learning degree metric (average logit across all knowledge triples for a concept) captures this bundle-level perspective.

It enables the study of interference and synergy at knowledge-type granularity. Finding 6 — that pretraining on Property & Affordance knowledge significantly facilitates subsequent learning of Synonym & Antonym knowledge — only makes sense if you can separate knowledge types within a concept. If all knowledge were treated as homogeneous facts, there would be no way to ask "does learning functional properties help with learning taxonomic relations?" The bundle structure, with its five distinct knowledge types, creates a natural experimental axis: vary which types are learned first, measure transfer to which types are learned second, and observe structured patterns of synergy and interference. The paper's 20 ordered curricula (all directed pairs of knowledge types) and the resulting 5×5 transferability heatmap (Figure 8) are only meaningful because knowledge is modeled as typed and structured rather than atomic.

It connects concept learning to the broader cognitive science of concepts. In human cognition, concepts are not mere collections of features — they have internal structure (the "conceptual core" vs. "identification procedures" distinction), they support inference (if something is a dog, it probably barks), and they are organized into hierarchies and theories. By modeling concepts as bundles of typed relational knowledge drawn from ConceptNet — which itself encodes structured semantic relationships — the paper aligns its operationalization of "concept" with cognitive theories of conceptual representation. The observation that different knowledge types transfer asymmetrically (some types scaffold others) echoes developmental findings about how children acquire conceptual knowledge in stages, with certain types of understanding (e.g., functional properties) building on others (e.g., taxonomic categories). The paper doesn't make these cognitive science connections explicitly, but the experimental design opens the door for such bridging.

Evidence anchor: The FICO dataset construction (Section 2, Appendix A Table 1) with five knowledge types; the Concept Circuit definition as predicting all knowledge triples for a concept; Figure 8 (paired transferability heatmap across knowledge types) demonstrating structured, asymmetric transfer.

Innovation 4: Verifier-Free, Circuit-Based Interference Diagnostics — Explaining Cross-Concept Interference Through Representational Overlap

Finding 5 demonstrates that training a target concept alongside highly related auxiliary concepts produces substantially worse learning outcomes than training alongside weakly related concepts (57.5% performance vs. 75.9%). This result is consistent with intuitive expectations about interference — similar things are harder to learn together because they compete for shared representational resources. But the paper's distinctive contribution is explaining why this interference occurs at the circuit level through the Jaccard similarity analysis in Figure 7, rather than just measuring that it occurs.

Prior work on cross-concept or cross-task interference in continual learning has typically used behavioral measures alone to diagnose interference — if accuracy on Task A drops after learning Task B, interference is inferred. Some work has used representational similarity analyses (e.g., CKA, SVCCA) to measure overlap in activation patterns across tasks, but these operate at the level of layer activations or hidden states, not at the level of the computational subgraphs that actually perform the concept-specific computation. The paper's innovation is using circuit overlap — specifically, the Jaccard similarity between the edge sets of Concept Circuits — as a mechanistic diagnostic for why interference is stronger between semantically similar concepts.

The logic is clean: if Concept Circuits represent the computational pathways the model uses to process each concept, then the degree of overlap between two concepts' circuits quantifies how much computational infrastructure they share. High Jaccard similarity means the same edges (and therefore the same neurons, attention heads, and their interconnections) are critical for both concepts. When these shared edges are updated during joint training — receiving gradient signals from both concepts simultaneously — the updates must serve two masters. If the concepts are semantically dissimilar, their gradient signals on shared edges may be uncorrelated or even conflicting, leading to interference where learning one concept degrades performance on the other (because the shared parameters are pulled in incompatible directions). If the concepts are weakly related and have low circuit overlap (low Jaccard similarity), they use largely disjoint computational infrastructure, and joint training produces minimal interference.

What makes this more than a correlation between two obvious things (similar concepts share more circuit edges; similar concepts interfere more) is that it provides a mechanistic target for intervention. If interference is caused by circuit overlap, then strategies that reduce circuit overlap during training should reduce interference. This motivates the paper's suggestion of "interference-aware data scheduling that reduces circuit similarity among co-trained concepts within the same batch" (Section 4.1.2). Rather than simply avoiding training similar concepts together (which would limit data diversity), one could imagine training strategies that explicitly encourage disentangled circuit formation — for example, through auxiliary losses that penalize circuit overlap, or through architectural modifications that allocate separate capacity to different concept clusters. The circuit-level diagnostic (Jaccard similarity) provides both an explanation of the problem and a potential metric to optimize against.

The KDE plots in Figure 7 add a subtle but important nuance: the highly related group shows a "sharper, higher-density peak at moderate circuit similarity," while the weakly related group shows lower and more dispersed similarity. This isn't just a mean shift — it suggests that semantic similarity induces a qualitatively different pattern of circuit overlap, not just more overlap on average. Highly related concepts tend to share a consistent, moderate level of circuit infrastructure (the sharp peak), whereas weakly related concepts show a wider range of overlap values (the dispersed distribution). This could reflect different modes of representational sharing: similar concepts systematically reuse the same core computational pathways, while dissimilar concepts sometimes share pathways by chance (due to the distributed nature of the model's representations) but without a consistent structural basis.

Evidence anchor: Figure 6 (concept interference: performance drops as co-trained concept relatedness increases), Figure 7 (Jaccard similarity distributions across relatedness groups showing concentrated overlap for highly related concepts), Section 4.1.2 discussion.

Innovation 5: Asymmetric Knowledge-Type Transfer as an Empirical Foundation for Concept-Aware Curricula

Finding 6 — the paired transferability analysis across the five knowledge types — reveals that the benefits of learning one type of conceptual knowledge on subsequently learning another are highly directional, uneven, and specific. Pretraining on Property & Affordance (PAA) yields substantial gains for subsequent Synonym & Antonym (SAA) learning (+63.74% relative to BIO baseline), but the reverse direction shows markedly weaker transfer. Meronym & Holonym (MAH) shows relatively small improvements from most source types, suggesting high intrinsic learnability and reduced sensitivity to prior knowledge-type pretraining.

This is significant because it transforms curriculum design for continual pre-training from a heuristic art into a potentially principled optimization problem. Prior work on training data ordering (e.g., curriculum learning) has largely relied on intuitive notions of "easy-to-hard" or "simple-to-complex" sequencing, often operationalized through superficial features like sentence length or word frequency. The paper's transferability matrix provides a different kind of curriculum signal: which knowledge types scaffold which others, independent of surface complexity. The finding that PAA (functional and attribute knowledge) strongly scaffolds HAH (taxonomic knowledge) and SAA (similarity/contrast knowledge) suggests a learning dynamic where concrete, functional understanding of concepts provides a foundation for more abstract relational reasoning — a pattern with intuitive cognitive plausibility (children learn what things do before they learn formal taxonomic hierarchies) but that had not been empirically demonstrated in LLM training dynamics.

The asymmetry of the transfer effects is particularly diagnostic. If all knowledge types were mutually facilitative (a rising tide lifts all boats), the transferability matrix would be approximately symmetric — training on any type would similarly benefit all other types. The observed asymmetry (strong PAA → SAA, weak SAA → PAA) indicates that the transfer is not just about general language adaptation or optimization effects (which would be symmetric), but about the specific semantic content of the knowledge types. Some types provide a more generalizable representational scaffold than others. The paper doesn't fully explain why PAA is a better scaffold than, say, Spatial Relation, but the asymmetry pattern itself is an empirical clue about the representational geometry of concept learning: functional knowledge may induce representations that are more easily extended to taxonomic and relational reasoning than vice versa.

The unevenness across target types is equally informative. MAH (part–whole knowledge) benefits relatively little from any source type, suggesting that part–whole relations are learned through mechanisms that are largely independent of other conceptual knowledge. This might reflect a fundamental difference in how the model represents composition (what parts something has) versus attribution (what properties something has) or categorization (what kind of thing something is). The transferability matrix thus serves not only as a practical guide for curriculum ordering but as a diagnostic tool for probing the structure of the model's conceptual representation space — knowledge types that transfer to many others may serve as "hub" representations, while types that are insensitive to prior training may be computed through specialized, isolated pathways.

Evidence anchor: Figure 8 (5×5 paired transferability heatmap), the reported +63.74% improvement for PAA → SAA transfer, the asymmetry between forward and reverse transfer directions for most knowledge-type pairs.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. The primary evaluation dataset is the FICO test set (Section 2, Appendix B Table 2), consisting of 500 fictional concepts with 1,586 knowledge triples instantiated into 1,586 evaluation samples (one template per triple, totaling 13,530 tokens). The evaluation templates are drawn from a disjoint pool from the training templates, ensuring that test performance measures generalization to novel surface forms rather than template memorization. The training set contains 1,000 concepts with 3,075 knowledge triples expanded to 92,250 training samples (~1.04M tokens), trained for 10 epochs for a total of 10.4M training tokens. For forgetting induction, the paper uses the BIO dataset (Allen-Zhu and Li, 2023), a standard pre-training corpus of biographical texts that does not contain the fictional concepts or their associated relational knowledge types. For concept relatedness computation, the paper uses Qwen3-Embedding-4B (Yang et al., 2025) as an external frozen embedding model to compute pairwise cosine similarities between concept names.

  • Base model(s). The primary experiments use GPT-2 Large (Radford et al., 2019) at 0.7B parameters, accessed via the openai-community/gpt2-large checkpoint. Results for LLaMA-3.2-1B-Instruct (Dubey et al., 2024) at 1B parameters are reported in Appendix D. The authors state that they "observe similar trends across both LLMs" and present GPT-2 results in the main paper while deferring LLaMA results to the appendix due to space constraints. The choice of GPT-2 Large is motivated by computational constraints — the authors acknowledge in the Limitations section that "due to computational constraints, our experiments focus on GPT-2 Large (0.7B) and LLaMA-3.2-1B, and we do not evaluate larger-scale models," explicitly flagging this as an important direction for future work.

  • Metrics. The primary behavioral metric is concept learning degree and concept forgetting degree (Section 3.2, Definitions 1 and 2). The concept learning degree for a concept c is the arithmetic mean across its associated knowledge triples of the logit increase for the correct target object after Stage 1 training (FICO acquisition), comparing the base model \pi_0 to the post-acquisition model \pi_1. The concept forgetting degree is the mean logit decrease for the correct target object after Stage 2 training (BIO forgetting induction), comparing \pi_1 to the post-forgetting model \pi_2. The paper uses logits rather than probabilities because they are "a fine-grained indicator of LLM learning" (Section 3.2), unbounded and linear in internal representations, and reports log probability results in Appendix C (Figures 9, 10a, 10b, 11) as a robustness check. For cross-concept interference experiments (Section 4.1), performance is measured via both average logit and average probability assigned to correct targets. For cross-knowledge-type synergy experiments (Section 4.2), the paired transferability metric T(R_i \rightarrow R_j) is defined as the proportional change in logit relative to a BIO baseline (Equation 1). For circuit-level analysis, the internal metrics are four families of graph-theoretic measures computed on extracted Concept Circuits: eigenvector centrality variance (node importance), density (redundancy), global efficiency (information flow efficiency), and average k-core number (robustness). Correlations between circuit graph metrics and behavioral measures are quantified using Spearman's rank correlation coefficient (Spearman, 1961), with statistical significance assessed at p < 0.001 (Conover, 1999).

  • Baselines. The paper's experimental design is primarily correlational and comparative rather than benchmark-driven, so traditional baselines are not the organizing principle. However, several control conditions serve as de facto baselines:

    • BIO control baseline (for knowledge-type transfer): In the cross-knowledge-type synergy experiments, Stage 1 training on the BIO dataset serves as the control condition against which knowledge-type pretraining is compared. The transferability metric T(R_i \rightarrow R_j) is defined relative to logit(R_j | BIO), isolating the effect of the specific knowledge type R_i from the general effect of any continued pretraining.
    • Weakly related concept group (for cross-concept interference): In the joint-training experiments (Section 4.1), training with weakly related concepts serves as a baseline against which training with moderately and highly related concepts is compared. The paper reports that weakly related joint training yields 75.9% performance (Figure 6), against which the 57.5% for highly related and 67.2% for moderately related are measured.
    • Untrained concept circuits: The concept circuits extracted from \pi_0 (the base model before any FICO training) provide a structural baseline for circuit reorganization during learning and forgetting.

  • Generation budget / compute accounting. The paper does not use "generation budget" as a compute metric in the manner of inference-time scaling studies. Instead, compute is controlled by matching training steps across experimental conditions: Stage 1 (FICO acquisition) is 10 epochs over 1.04M tokens (= 10.4M total training tokens), Stage 2 (BIO forgetting induction) uses "the same training steps" as Stage 1 (Section 3.2), ensuring that learning and forgetting phases are duration-matched. For the cross-knowledge-type synergy experiments, Stage 1 training on each knowledge type and on BIO is for "the same training steps," enabling fair comparison of transfer effects. All experiments use 4 × 40GB A40 GPUs, learning rate 5 \times 10^{-5}, and batch size 128 (Appendix E). The Concept Circuit extraction preserves at least 70% of the full model's performance, which is the threshold controlling the size-fidelity trade-off in circuit construction.

  • Cross-validation / statistical protocol. The paper does not employ k-fold cross-validation across data splits in the conventional sense — the FICO dataset has fixed training (1,000 concepts) and test (500 concepts) splits. Statistical significance for the Spearman correlations between circuit graph metrics and behavioral measures is assessed using the standard nonparametric test (Conover, 1999) at p < 0.001. The paper reports that the "observed Spearman correlations are statistically significant" with this threshold (Section 3.3, Finding 2 footnote). For the cross-concept interference experiments, the procedure is "repeated for all concepts in the test set of FICO to obtain a comprehensive characterization of cross-concept interactions" (Section 4.1.1), meaning that interference effects are aggregated across 500 test concepts. For the cross-knowledge-type synergy experiments, the 20 ordered curricula (all directed pairs of the five knowledge types) plus the BIO control baseline constitute the full experimental design, with no cross-validation over curricula.

Main Quantitative Results

Concept Learning and Forgetting Dynamics (Section 3.3)

The paper's first set of quantitative findings establishes the basic behavioral landscape for concept acquisition and forgetting, then links these behavioral measures to circuit-level graph metrics.

Finding 1 — Heterogeneity in concept learning and forgetting. Figure 2 (main paper) and Figure 9 (Appendix C, log probability version) display the distribution of concept learning degree (logit increase after Stage 1) and forgetting degree (logit decrease after Stage 2) across 500 test concepts for GPT-2 Large. The learning degree distribution is "unimodal but widely spread" (Section 3.3), indicating that under identical training conditions, some concepts experience substantially larger logit gains than others. For the LLaMA-3.2-1B-Instruct model, Figures 12 and 14 in Appendix D show qualitatively similar distributions for learning and forgetting degrees respectively, confirming that this heterogeneity is not model-specific.

The paper does not report specific numerical ranges for these distributions (e.g., minimum/maximum learning degree, standard deviation), which is a notable omission — the reader cannot assess from the reported numbers just how heterogeneous the learning dynamics are. The histograms in Figure 2 provide visual evidence of spread but lack precise quantification.

Finding 2 — Circuit graph metrics correlate with learning and forgetting. Figures 3a and 3b (main paper, GPT-2 Large) display Spearman correlations between four circuit graph metrics and (a) learning degree, (b) forgetting degree. The paper reports "non-trivial and consistent Spearman correlations" at p < 0.001 for all four metrics in both directions. Specifically:

  • Eigenvector centrality variance (Node Importance): positively correlated with both learning degree and forgetting degree. The paper interprets this as evidence that "circuits with centralized bottlenecks and stable structural cores tend to achieve stronger logit gains" but are simultaneously "more vulnerable to forgetting when these components are perturbed by subsequent training."
  • Density (Redundancy): positively correlated with both learning degree and forgetting degree. The counterintuitive forgetting correlation — denser circuits are more forgetful — is explained as "highly redundant and tightly interconnected circuits entangle concept representations more strongly with other knowledge, amplifying interference during continued training."
  • Global efficiency (Information Flow Efficiency): positively correlated with both learning degree and forgetting degree, showing the same dual-edged pattern: efficient information flow aids acquisition but increases vulnerability.
  • k-core depth (Robustness): positively correlated with both learning degree and forgetting degree. This is the most surprising result because "robustness" intuitively suggests resistance to disruption, yet deeper core structures are associated with greater forgetting. The paper's interpretation is that the same deep core structures that provide learning stability also entangle the concept with other knowledge representations.

Appendix C (Figures 10a and 10b) shows the same correlations computed with log probability as the behavioral measure, serving as a robustness check. Appendix D (Figures 13 and 15) shows the corresponding correlations for LLaMA-3.2-1B-Instruct, confirming the pattern generalizes across model families.

A critical limitation in interpreting these correlations: the paper reports Spearman \rho values visually in bar charts but does not report the exact numerical correlation coefficients in the text, making it impossible to assess the magnitude of the correlations from the prose alone. The figures suggest modest but non-zero correlations (the bar heights are clearly above zero but not near 1.0), consistent with the paper's characterization as "non-trivial" and "statistically significant" but not overwhelmingly strong.

Finding 3 — Stage-wise temporal pattern in circuit metrics during forgetting. Figure 4 (main paper, GPT-2 Large) and Figure 16 (Appendix D, LLaMA-3.2-1B-Instruct) plot the evolution of four circuit graph metrics over training steps during the BIO forgetting stage (Stage 2). The paper reports a "consistent stage-wise temporal pattern": an initial increase in metric values, followed by a gradual decrease, and eventual stabilization. This pattern is "observed across all four structural aspects of concept circuits" — eigenvector centrality variance, density, global efficiency, and k-core depth all show this trajectory, though at different magnitudes and timescales.

The paper does not report precise timing (e.g., at which training step the peak occurs, what fraction of total forgetting-stage steps is spent in each phase), nor does it report the magnitude of changes (how much do metrics increase before decreasing?). The figure provides qualitative trajectories, but the lack of quantification limits the reader's ability to assess whether the "early increase" is substantial or merely a minor fluctuation.

The paper's interpretation of this pattern — "transient entanglement of previously learned concept circuits with newly introduced knowledge during continued training" followed by "structural relaxation and convergence to a weaker representation under interference" — is consistent with the data but not uniquely determined by it. Alternative explanations (e.g., optimization dynamics unrelated to representational entanglement, or artifact of the specific metric computations on changing model parameters) are not considered.

Finding 4 — Positive correlation between learning degree and forgetting degree. Figure 5 (main paper, GPT-2 Large) and Figure 11 (Appendix C, log probability version) and Figure 17 (Appendix D, LLaMA-3.2-1B-Instruct) display the Spearman correlation between concept learning degree and concept forgetting degree. The paper reports a "positive association": concepts that achieve larger gains during acquisition tend to degrade more when the model is later trained on new data. This finding triangulates with the circuit-level correlations in Finding 2 — the same structural properties that predict stronger learning also predict stronger forgetting, and behaviorally, concepts that are learned more aggressively are indeed the ones most susceptible to forgetting.

The paper does not report whether this correlation is primarily driven by a subset of outlier concepts (e.g., the top 10% most-learned concepts accounting for most of the correlation) or whether it is a consistent gradient across the full distribution. The scatter plot in Figure 5 suggests a noisy but visible trend, but without additional statistics (e.g., correlation within difficulty or knowledge-type strata), the granularity of the effect is unclear.

Cross-Concept Interference (Section 4.1)

Finding 5 — Semantic similarity drives interference during joint training. Figure 6 (main paper, GPT-2 Large) and Figure 18 (Appendix D, LLaMA-3.2-1B-Instruct) display concept learning performance under three joint-training conditions: training with highly related concepts, moderately related concepts, or weakly related concepts. The results, reported for GPT-2 Large:

  • Weakly related concepts (baseline): 75.9% performance (measured as average probability assigned to correct targets).
  • Moderately related concepts: 67.2% performance — a drop of approximately 8.7 percentage points relative to weakly related.
  • Highly related concepts: 57.5% performance — a drop of approximately 18.4 percentage points relative to weakly related, and approximately 9.7 percentage points below moderately related.

The paper states that "across both evaluation metrics (average logit and average probability), training with weakly related concepts achieves the highest performance" and that the dependence on semantic relatedness is "clear and consistent." The logit-based results in Figure 6 show the same monotonic ordering.

Circuit overlap explanation (Figure 7). To mechanistically explain this interference pattern, the paper computes Jaccard similarity between the edge sets of Concept Circuits for target concepts and their co-trained auxiliary concepts, stratified by relatedness group. Figure 7 visualizes the resulting similarity distributions using KDE plots:

  • Highly related group: exhibits a "sharper, higher-density peak at moderate circuit similarity," indicating consistent reuse of overlapping computational pathways.
  • Moderately related and weakly related groups: show "lower and more dispersed similarity distributions," reflecting reduced circuit overlap.

The paper interprets this as causal evidence that "consistent reuse of overlapping computational pathways between target and auxiliary concepts... likely induces sustained representational competition during joint training and results in stronger interference." However, this is a correlational rather than causal claim — the paper observes both higher circuit overlap and higher behavioral interference for related concepts, but does not experimentally manipulate circuit overlap (e.g., by pruning shared edges or regularizing against overlap) to test whether reducing overlap would reduce interference. The causal direction (does overlap cause interference, or does interference cause circuits to appear more overlapping in post-hoc extraction?) is not resolved.

Cross-Knowledge-Type Synergy (Section 4.2)

Finding 6 — Asymmetric and uneven transfer effects across knowledge types. Figure 8 (main paper, GPT-2 Large) and Figure 19 (Appendix D, LLaMA-3.2-1B-Instruct) display a 5×5 heatmap of paired transferability T(R_i \rightarrow R_j) (Equation 1) for all ordered pairs of the five knowledge types: Hyponym & Hypernym (HAH), Synonym & Antonym (SAA), Meronym & Holonym (MAH), Property & Affordance (PAA), and Spatial Relation (SR). Each cell (i, j) shows the proportional improvement (positive values, synergy) or degradation (negative values, interference) in learning knowledge type R_j when Stage 1 pretraining was on R_i, relative to the BIO control baseline.

Key quantitative findings from the heatmap (reported for GPT-2 Large):

  • PAA → SAA transfer: the strongest positive transfer reported. The paper states that "pretraining on Property & Affordance (PAA) yields substantial gains when transferring to... Synonym & Antonym (SAA)" with a transferability of +63.74% relative to the BIO baseline. This means that after pretraining on functional and attribute knowledge, the model learns similarity and contrast relations 63.74% more effectively (in logit terms) than after equivalent training on BIO.
  • PAA → HAH transfer: also "substantial gains," though the exact percentage is not stated in the main text (visible in Figure 8 heatmap but numerical values require reading the figure).
  • Reverse directions (SAA → PAA, HAH → PAA): exhibit "markedly weaker transfer, highlighting the asymmetry of these interactions." The paper does not report the specific reverse-direction transfer values in the prose.
  • MAH as target: "shows relatively small improvements across most sources, indicating higher intrinsic learnability and reduced sensitivity to prior knowledge-type pretraining." This means part–whole knowledge is learned approximately equally well regardless of what knowledge type was pretrained on, suggesting that MAH representations are acquired through mechanisms that are largely independent of other conceptual knowledge types.

The asymmetry finding is robust: "some target types... show relatively small improvements across most sources," while other target types (SAA, HAH) show large improvements from specific sources (PAA). The paper characterizes the overall pattern as "heterogeneous and directional effects."

The LLaMA-3.2-1B-Instruct results in Appendix D (Figure 19) show qualitatively similar patterns, though the specific transfer values may differ (the paper does not report them numerically in the appendix figure captions).

Ablation Studies and Robustness Checks

Log probability as alternative behavioral metric (Appendix C). To verify that the findings are not an artifact of using logits rather than probabilities, the paper replicates the core behavioral analyses using log probability as the dependent measure. Figure 9 replicates the learning/forgetting degree distribution (cf. Figure 2), Figures 10a and 10b replicate the circuit graph metric correlations (cf. Figures 3a and 3b), and Figure 11 replicates the learn–forget correlation (cf. Figure 5). The paper does not report whether the Spearman correlation coefficients change in magnitude when using log probability vs. logit, but the qualitative patterns are preserved, confirming that the findings are robust to the choice of logit-space vs. probability-space measurement.

LLaMA-3.2-1B-Instruct model family replication (Appendix D). All major findings are replicated on LLaMA-3.2-1B-Instruct (1B parameters), a more recent and architecturally distinct model from GPT-2 Large. The appendix includes: learning degree distribution (Figure 12), circuit graph metric correlations with learning (Figure 13) and forgetting (Figure 15), forgetting degree distribution (Figure 14), temporal graph metric trajectory during forgetting (Figure 16), learn–forget correlation (Figure 17), cross-concept interference (Figure 18), and cross-knowledge-type transferability (Figure 19). The paper states that "we observe similar trends across both LLMs," suggesting that the findings are not specific to the GPT-2 architecture or training paradigm. However, the paper does not report whether the magnitudes of correlations or transfer effects differ between the two models, which would be informative about whether these effects scale with model capability or architecture.

Template-based generalization design (Section 2, Appendix B). The FICO dataset construction itself embeds a robustness check: training and evaluation templates are drawn from disjoint pools, meaning that the model cannot succeed on the test set by memorizing surface-level template–object co-occurrences. The fact that the model does learn (i.e., concept learning degrees are positive and substantial; Figure 2) demonstrates that the learning generalizes beyond specific template formulations to the underlying relational knowledge. However, the paper does not report an explicit ablation comparing within-template vs. cross-template generalization performance, which would quantify how much of the measured "learning" is template-specific vs. relational.

Concept relatedness via external embedding model (Section 4.1.1). The use of Qwen3-Embedding-4B as an external, frozen embedding model for computing concept relatedness is itself a robustness choice — it ensures that the relatedness measure is independent of the training model's changing representations and not confounded by the training process. The paper does not ablate this choice by comparing against relatedness measured using the training model's own embeddings, which would test whether the interference patterns are driven by semantic similarity in an absolute sense or by similarity specifically within the model's representational space.

Negative results: ReST^EM is not tested, and the paper explicitly positions itself as analytical rather than interventional. The Limitations section acknowledges that "we focus on characterizing the relationship between internal concept circuits and learning dynamics, showing that circuit graph patterns could indicate future learning and forgetting behaviors... we leave the exploration of these motivated training strategies for future work." This means the paper does not test whether the identified signals (circuit graph metrics, concept relatedness, knowledge-type ordering) can be used to improve training outcomes through interventions — all findings are correlational or observational. This is a significant scope limitation but is explicitly acknowledged rather than obscured.

Critical Assessment

Claim 1: "Circuit-level graph metrics provide a non-trivial, statistically significant signal of both concept learning and forgetting."

The experiments in Figures 3a, 3b (and their appendix variants) demonstrate that Spearman correlations between the four graph metrics and behavioral measures are statistically significant at p < 0.001. This supports the claim that the correlations are not zero — circuit topology and concept behavior are systematically related. However, the claim should be interpreted with several important qualifications:

What the experiments actually demonstrate: they show that across 500 test concepts, concepts with higher eigenvector centrality variance (or density, or global efficiency, or k-core depth) in their extracted circuits tend to have higher learning degrees and higher forgetting degrees. This is a cross-sectional correlation at a single snapshot (post-training circuits correlated with behavioral outcomes). The experiments do not demonstrate that changes in circuit metrics predict subsequent learning or forgetting — for that, one would need to measure circuit metrics at \pi_0 (before training) and show they correlate with learning degree computed after training, or measure circuit metrics at \pi_1 and show they correlate with forgetting degree computed after Stage 2. The paper's design measures circuits and behavior at the same checkpoints (e.g., \pi_1 circuits correlated with \pi_0 \rightarrow \pi_1 logit changes), which establishes concurrent validity but not predictive validity. The paper's language — "these signals as indicators of future learning and forgetting" (contributions) — implies predictive power that the experimental design does not directly test.

What the correlation magnitude implies: the paper reports statistical significance but does not report the actual Spearman \rho values in the prose. The bar charts in Figures 3a and 3b suggest correlations in the range of approximately 0.1–0.3 (visual estimate), which are modest — they explain perhaps 1–9% of the variance in learning/forgetting degree. This is consistent with the paper's characterization as "non-trivial" and "modest" but is important context: circuit graph metrics are informative signals, but they are far from deterministic predictors. The vast majority of variance in concept learning outcomes is not explained by these four structural properties, meaning that other factors (e.g., knowledge-type composition of the concept, specific token-level properties, optimization dynamics) dominate.

The conceptual circuit extraction threshold (70%) is a free parameter. The paper does not ablate this threshold to test whether the correlation patterns are sensitive to circuit size or fidelity. A 70% threshold means the extracted circuits capture most but not all of the model's concept-relevant computation — changing this threshold (e.g., to 50% or 90%) could produce circuits with different topological properties, and it is unknown whether the reported correlations would persist. This is a meaningful gap, as circuit extraction thresholds are known to affect circuit structure in prior work.

Claim 2: "Concepts with larger learning gains tend to exhibit greater forgetting under subsequent training."

Figure 5 provides direct evidence for a positive correlation between learning degree and forgetting degree across the 500 test concepts. This claim is well-supported as a cross-sectional empirical observation — on average, concepts that were learned more aggressively did experience more forgetting when the model was subsequently trained on BIO.

Caveat: the forgetting is induced by a single unrelated dataset (BIO). The paper demonstrates the learn–forget correlation in one specific forgetting condition: continued training on biographical text. It is unknown whether the same correlation would hold if the forgetting-induction dataset were different — for example, if Stage 2 training were on a different set of fictional concepts, or on code, or on a domain that shares more or less structural overlap with FICO. The mechanism the paper proposes (entanglement-driven interference) would predict that forgetting magnitude depends on the degree of computational overlap between Stage 1 and Stage 2 data, which could vary substantially across forgetting-induction datasets. The single-dataset design limits the generality of the claim.

Caveat: correlation does not establish that aggressive learning causes fragility. An alternative interpretation is that some concepts are inherently "harder" in a way that both limits asymptotic learning and increases susceptibility to interference, producing a spurious correlation between learning degree and forgetting degree that is not causal. Without an intervention that manipulates learning degree (e.g., by varying the amount of training data per concept or the learning rate per concept) and measures the causal effect on forgetting, the paper cannot distinguish between "aggressive learning causes fragility" and "hard concepts both learn poorly and forget easily for independent reasons."

Figure 6 provides clear evidence: training with highly related auxiliary concepts yields substantially lower performance (57.5%) on target concepts than training with weakly related auxiliary concepts (75.9%). This is a controlled within-experiment comparison where only the relatedness of the co-trained concepts varies, so the causal attribution to semantic similarity is relatively strong.

However, the mechanistic explanation via circuit overlap (Figure 7) is correlational, not causal. The paper observes that highly related concept pairs have higher Jaccard similarity between their circuits, and separately observes that training with highly related concepts produces more interference. It does not show that the specific concepts with higher circuit overlap are the ones experiencing more interference — the analysis is aggregated at the group level. To establish the mechanistic link, one would need to show, at the individual concept-pair level, that circuit overlap predicts interference magnitude. The paper has the data to do this analysis (circuit Jaccard similarity for each target–auxiliary pair, and presumably per-concept performance under each joint-training condition) but does not report it.

The K=100 group size is a free parameter. The paper constructs relatedness groups of size 100 without ablating this choice. With K=100, the "highly related" group includes the 100 most similar concepts to the target; with a smaller K (e.g., K=10), the group would be more tightly similar, potentially producing even stronger interference; with a larger K (e.g., K=250), the group would be more diluted, potentially producing weaker interference. The sensitivity of the interference effect to group size and composition is unexplored.

Claim 4: "Conceptual knowledge differs in transferability, with some significantly facilitating the learning of others."

Figure 8 provides detailed evidence for asymmetric, directional transfer effects across knowledge types. The reported PAA → SAA transfer of +63.74% is a substantial effect that supports the claim that knowledge-type ordering matters for learning efficiency. The asymmetry (PAA scaffolds SAA more than SAA scaffolds PAA) demonstrates that these are not simple "more pretraining is better" effects — the specific semantic content of the pretraining matters.

Caveat: the transferability metric uses BIO as the control, which may not be a neutral baseline. BIO consists of biographical text, which has its own semantic and structural properties that may affect subsequent FICO learning in ways that are not "neutral." If BIO incidentally provides some benefit for certain knowledge types (e.g., biographical text contains part–whole relations like "person has a job" that overlap with MAH knowledge), then the transferability values for those target types would be artificially depressed (because the baseline logit(R_j | BIO) would be higher, making the proportional improvement smaller). The paper assumes BIO is a neutral control, but this assumption is not tested.

Caveat: the transfer effects are measured after only two stages of training with fixed duration. It is unknown whether the transfer benefits persist or diminish with longer Stage 2 training — does PAA pretraining permanently accelerate SAA learning, or does it merely provide a head start that the BIO baseline catches up to with more training? A learning curve analysis showing performance over the course of Stage 2 for each curriculum would address this but is not provided.

Caveat: the experiment uses isolated knowledge-type pretraining, which is artificial. In realistic continual pre-training, concepts are learned with all their knowledge types simultaneously or interleaved, not with one knowledge type in Stage 1 and another in Stage 2. The pairwise curriculum design isolates transfer effects cleanly but may not reflect how transfer operates when knowledge types are learned jointly. The paper does not test whether the transfer benefits observed in the pairwise design predict outcomes in a more realistic interleaved curriculum.

Overall Experimental Design Strengths

  • Controlled concept novelty via FICO: the fictional-name substitution elegantly solves the confound of pre-existing knowledge while preserving realistic relational structure. The disjoint train–test templates add a further control against surface-form memorization.
  • Two-stage design separates acquisition from forgetting: by using FICO for Stage 1 and BIO for Stage 2, the paper avoids the ambiguity of measuring "forgetting" during training on the same data distribution.
  • Multi-model replication: the GPT-2 Large and LLaMA-3.2-1B-Instruct results show the same qualitative patterns, demonstrating that the findings are not architecture-specific.
  • Multiple behavioral metrics: reporting both logit-based and probability-based results (Appendix C) and showing consistent patterns addresses concerns about metric sensitivity.
  • Transparent limitation acknowledgment: the paper explicitly flags model scale, the analytical (non-interventional) scope, and the absence of larger-model experiments as limitations.

Overall Experimental Design Weaknesses

  • Single forgetting-induction dataset (BIO): the learn–forget correlation and circuit reorganization patterns are demonstrated under one specific type of continued training. Generalizability to other forgetting regimes (e.g., continued training on other conceptual domains, on code, on multilingual data) is unknown.
  • Modest model scale (0.7B–1B): the paper acknowledges this limitation but it bears emphasizing: it is entirely unknown whether the reported circuit graph metric correlations, interference patterns, and knowledge-type transfer effects would hold in larger models (7B, 70B, or beyond) where circuit structure may be qualitatively different (more distributed, more polysemantic, different depth–width trade-offs). The paper cannot claim that its findings apply to "LLMs" in general — only to the specific model scales tested.
  • No interventional experiments: all findings are correlational or observational. The paper does not test whether manipulating circuit structure (e.g., through regularization), data ordering (e.g., scheduling to minimize circuit overlap), or knowledge-type curricula (e.g., pretraining on PAA before SAA) actually improves learning efficiency or reduces forgetting. The phrase "inform the design of more interpretable and robust concept-aware training strategies" (contributions) describes potential future work, not demonstrated results. This is not hidden — the Limitations section is explicit — but it means the paper's practical value is currently suggestive rather than demonstrated.
  • Missing per-concept granularity in interference and synergy analyses: the cross-concept interference results are aggregated across all test concepts (the reported percentages are averages). The paper does not show whether interference magnitude varies with target concept properties (e.g., concrete vs. abstract concepts, concepts with many vs. few knowledge triples, concepts from different knowledge-type compositions). This aggregation may mask important heterogeneity.
  • Difficulty estimation via the circuit extraction threshold (70%) is not ablated: circuit structure and the resulting graph metrics may depend sensitively on the faithfulness threshold used in EAP-IG extraction. The paper does not report how circuit topology changes with different thresholds, leaving open the possibility that the reported correlations are an artifact of the specific 70% choice.
  • No confidence intervals or error bars on reported statistics: the paper reports point estimates (correlation coefficients, performance percentages, transferability values) without confidence intervals, standard errors, or tests of whether differences between conditions are statistically significant beyond the p < 0.001 threshold for correlations. For example, is the 57.5% vs. 75.9% difference in Figure 6 statistically significant, or could it arise from sampling variability across the 500 test concepts? The paper does not say.

Missing Experiments That Would Have Strengthened the Paper

  • Predictive (not just concurrent) circuit–behavior correlations: measure circuit graph metrics at \pi_0 (before FICO) and correlate with subsequent learning degree (measured after Stage 1). This would test whether circuit structure predicts learning rather than merely co-occurring with it.
  • Ablation of circuit extraction threshold: extract circuits at multiple faithfulness levels (e.g., 50%, 70%, 90%) and test whether the graph metric correlations with behavior are robust to threshold choice.
  • Diverse forgetting-induction datasets: replicate Stage 2 with different corpora (e.g., code, scientific text, another set of fictional concepts with different relational structures) to test whether the learn–forget correlation and circuit reorganization patterns are specific to BIO or general.
  • Larger model scale: even a single 7B model (e.g., LLaMA-3.1-8B) would substantially strengthen the claim that findings apply beyond the 0.7B–1B range. The paper acknowledges this limitation; the point is that it's a genuine gap, not a minor omission.
  • Intervention experiments: the most direct test of the paper's claims would be to use the identified signals (circuit graph metrics, cross-concept relatedness, knowledge-type transferability) to design a training curriculum or regularization strategy and show that it improves learning efficiency or reduces forgetting compared to a baseline. The absence of any intervention experiment means the paper's practical implications remain speculative.
  • Per-concept-type transferability analysis: disaggregate the cross-knowledge-type transfer results by concept properties (concrete vs. abstract, number of knowledge triples per concept) to test whether transfer effects are uniform or vary with concept characteristics.

6. Limitations and Trade-offs

Model Scale: All Findings Are Demonstrated Only at 0.7B–1B Parameters

The assumption or constraint. The paper conducts experiments exclusively on GPT-2 Large (0.7B parameters) and LLaMA-3.2-1B-Instruct (1B parameters). The authors are transparent about this:

"Due to computational constraints, our experiments focus on GPT-2 Large (0.7B) and LLaMA-3.2-1B, and we do not evaluate larger-scale models. Extending our analysis to larger LLMs remains an important direction for future work." (Limitations)

This is not a minor scaling detail — it is a fundamental scope constraint. The paper studies concept circuits, which are defined as computational subgraphs extracted from the model's forward-pass DAG. Circuit topology at 0.7B parameters (where individual attention heads and MLP neurons may serve relatively specialized, interpretable functions) may differ qualitatively from circuit topology at 70B or 700B parameters (where computation is more distributed, components are more polysemantic, and the concept of a "minimal subgraph that preserves 70% of performance" may require far more edges or may not cleanly exist at all). The graph metrics that show statistically significant correlations with behavior at small scale — eigenvector centrality variance, density, global efficiency, k-core depth — are structural measures that depend on circuit size, sparsity, and connectivity patterns that may not scale in predictable ways.

The consequence. The paper cannot claim that its central mechanistic findings — the structural trade-off between learning and forgetting, the stage-wise temporal trajectory during forgetting induction, the circuit-overlap explanation for cross-concept interference — generalize to "LLMs" as a class. These findings are established only for two small models from different families (GPT-2 and LLaMA-3.2) at similar parameter counts. A practitioner training or deploying a 7B, 13B, 70B, or larger model has no empirical basis for assuming that concept circuit graph metrics would carry the same predictive signal, or that the learn–forget correlation would replicate, or that cross-concept interference would follow the same relatedness-dependent pattern. Worse, if circuit structure at larger scales is fundamentally more distributed (more edges per concept circuit, less concentrated hub structure), the correlations the paper reports — which are already modest at 0.7B — could weaken or disappear entirely. The paper's contributions would then be specific to a model scale regime that is rapidly becoming obsolete for production deployments.

What evidence exists in the paper. The two-model replication (GPT-2 Large and LLaMA-3.2-1B-Instruct) across all major findings is the strongest available evidence for cross-architecture robustness at the ~1B scale. Appendix D shows qualitatively similar patterns for learning degree distributions (Figure 12 vs. Figure 2), circuit metric correlations (Figures 13, 15 vs. Figures 3a, 3b), temporal trajectories (Figure 16 vs. Figure 4), learn–forget correlations (Figure 17 vs. Figure 5), cross-concept interference (Figure 18 vs. Figure 6), and knowledge-type transfer (Figure 19 vs. Figure 8). The paper states that "we observe similar trends across both LLMs" (Section 3.2, footnote 5). However, this replication spans only a 1.4× parameter increase (0.7B to 1B) and two architectures — it provides no evidence whatsoever about scaling behavior beyond this narrow range. No 7B, 13B, or 70B experiments exist. No analysis of how circuit graph metrics change with model depth or width is provided.

Mitigation status. The paper does not attempt to mitigate this limitation. It acknowledges it transparently in the Limitations section and defers scaling studies to future work. There are no extrapolation arguments, no theoretical reasons to expect the findings to hold at scale, and no analysis of how circuit topology changes from GPT-2 Large to LLaMA-3.2-1B-Instruct (which would at least provide a two-point trend). The limitation is genuine and unaddressed.

Concept Circuit Extraction Depends on an Unablated Faithfulness Threshold

The assumption or constraint. Concept Circuits are defined as "the minimal computational subgraph that can faithfully predict all conceptual knowledge" for a concept, operationalized through EAP-IG edge attribution followed by top-edge selection such that "the resulting subgraph preserves at least 70% of the full model's performance on the corresponding concept" (Section 3.1). The 70% threshold is a free parameter that controls the size–fidelity trade-off: a higher threshold (e.g., 90%) would include more edges, producing larger, denser, and potentially more interconnected circuits; a lower threshold (e.g., 50%) would produce smaller, sparser circuits with different topological properties. The paper does not ablate this threshold to test whether the reported correlations between circuit graph metrics and behavioral measures are robust to threshold choice.

The consequence. All four graph metrics — eigenvector centrality variance, density, global efficiency, and k-core depth — are functions of the circuit's edge set. If changing the 70% threshold to 50% or 90% produces circuits with substantially different values of these metrics, and if the ranking of concepts by these metrics changes with the threshold, then the Spearman correlations reported in Figures 3a and 3b could be artifacts of the specific threshold choice rather than genuine properties of concept representations. More concretely: at a 50% threshold, the extracted circuits might be so sparse that all concepts have near-zero density and near-zero k-core depth, eliminating the variance needed to detect correlations. At a 90% threshold, the circuits might be so large that they approach the full model's computational graph for all concepts, again compressing variance. The 70% value sits at an unknown point on this trade-off curve. Without an ablation showing that the correlations persist across a range of plausible thresholds (e.g., 50%, 70%, 90%), the reader cannot assess whether the paper's central finding — "circuit-level graph metrics provide a non-trivial, statistically significant signal of concept learning and forgetting" (Finding 2) — is robust or brittle.

Additionally, because circuit extraction is performed per-concept at each checkpoint, the threshold controls which edges are included in the circuit that is then tracked over time. The temporal trajectory finding (Finding 3, Figure 4) — the stage-wise increase-then-decrease pattern in graph metrics during forgetting — could reflect changes in which edges cross the 70% threshold at different checkpoints, rather than genuine reorganization of a fixed set of concept-relevant edges. If EAP-IG importance scores shift during BIO training (as they likely do, since the model's parameters change), the set of edges that collectively achieve 70% fidelity at π_1 may differ from the set at π_2, even if the underlying concept representation is stable. The observed metric trajectories would then reflect circuit redefinition rather than circuit reorganization, undermining the paper's interpretation as "systematic circuit reorganization rather than monotonic structural decay" (Finding 3).

What evidence exists in the paper. None. The paper reports the 70% threshold as a design choice with no ablation, no sensitivity analysis, and no discussion of how varying the threshold affects circuit topology or the downstream correlations. The citation to Hanna et al. (2024) provides the EAP-IG methodology but does not prescribe a specific faithfulness threshold — that choice is the paper's own and its consequences are unexplored.

A related but distinct issue: EAP-IG uses integrated gradients for edge attribution, which is a linear approximation method with known failure modes (saturation, gradient shattering in deep networks). The paper does not validate that EAP-IG attributions are faithful for this specific model, dataset, and task — it relies on Hanna et al.'s general validation. If edge importance rankings are noisy or poorly calibrated, the circuits constructed by thresholding may not correspond to the model's actual computational pathways, and the graph metrics would measure noise rather than concept representation structure.

Mitigation status. Not addressed. The paper does not acknowledge the threshold sensitivity as a limitation, does not perform any ablation, and does not discuss how alternative threshold choices would affect the findings. This is a notable gap given that the threshold directly controls the object being measured (the circuit) and the paper's contributions depend on properties of that object.

All Correlations Are Concurrent, Not Predictive — The Claimed "Indicators" of Learning and Forgetting Are Not Demonstrated Prospectively

The assumption or constraint. The paper's central empirical claim is that circuit graph metrics "can serve as an informative indicator of concept learning and forgetting dynamics" (Finding 2), and the contributions section states that the findings "suggest these signals as indicators of future learning and forgetting." These phrasings imply predictive utility: if you measure a concept's circuit structure before training (or early in training), you can anticipate how well it will be learned or how badly it will be forgotten. However, the experimental design does not test this. The correlations in Figures 3a and 3b are computed between circuit metrics extracted from π_1 (post-acquisition) and learning degree measured as the π_0 → π_1 logit change — a concurrent correlation. The circuit metrics are measured after learning has occurred, on the very representations that produced the behavioral outcomes. This establishes that circuits that ended up with certain structural properties after learning also experienced certain amounts of logit change — it does not establish that pre-existing circuit structure (at π_0) predicted subsequent learning, nor that circuit structure at π_1 predicted subsequent forgetting (measured as π_1 → π_2 logit change).

The consequence. The paper's language about "indicators" and "future learning and forgetting" overstates what the experimental design supports. At best, the correlations show that circuit topology and behavioral outcomes are systematically related in the trained model — a finding about the anatomy of learned representations, not about the predictability of learning dynamics. A practitioner who measures circuit graph metrics on their base model before continual pre-training has no evidence from this paper that those metrics will predict which concepts will be learned easily or which will be vulnerable to forgetting. To establish predictive validity, the paper would need to: (1) measure circuit metrics at π_0 (before any FICO training) and correlate with subsequent learning degree (measured after Stage 1), and/or (2) measure circuit metrics at π_1 (after FICO, before BIO) and correlate with subsequent forgetting degree (measured after Stage 2). Neither analysis is reported.

This distinction matters enormously for the paper's stated practical implications. The paper motivates its work by arguing that the findings "can inform concept-aware continual pre-training decisions such as how much training is needed for new concepts, how to schedule and reorder training data, and how to reduce destructive interference" (Section 1). But these decisions require prospective knowledge — you need to know, before or early in training, which concepts need more attention. If the circuit signals are only measurable after training is complete, they cannot inform training decisions for that training run. They could potentially inform decisions for a subsequent training run (e.g., if you observe that concepts with certain circuit properties are fragile, you might adjust the next run's curriculum), but the paper does not test this transferability either.

What evidence exists in the paper. The temporal trajectory analysis (Finding 3, Figure 4) partially addresses the temporal dimension by showing how circuit metrics evolve during the forgetting stage. This demonstrates that circuit structure changes over time in a patterned way, which is necessary (but not sufficient) for a predictive relationship. However, Figure 4 tracks average metric trajectories across all concepts — it does not test whether individual concepts' trajectories predict their individual forgetting outcomes. A concept-level analysis showing that concepts whose metrics increase more during early BIO training subsequently forget more would provide predictive evidence, but this is not reported.

Mitigation status. The paper does not acknowledge the distinction between concurrent and predictive validity. The phrasing "indicators of future learning and forgetting" appears in the contributions without qualification, and the experimental design does not include the prospective analyses that would support this claim. This is a meaningful gap between what the paper claims and what the experiments actually test.

The Difficulty Estimation Cost Is Not Accounted For — Concept Circuit Extraction and Graph Metric Computation Are Computational Expensive and Not Amortized in Any Practical Pipeline

The assumption or constraint. The paper's analytical framework relies on extracting Concept Circuits for each concept at multiple checkpoints and computing four families of graph metrics on each circuit. This pipeline involves: (1) running EAP-IG edge attribution for each concept, which requires computing integrated gradients over multiple steps per edge in the model's computational graph; (2) thresholding edges to construct the minimal subgraph preserving 70% fidelity; (3) computing eigenvector centrality (requiring eigendecomposition of the circuit's adjacency matrix), density, all-pairs shortest paths (for global efficiency), and k-core decomposition on the resulting subgraph. The paper does not report the computational cost of this pipeline — no FLOP counts, GPU hours, or wall-clock time estimates are provided. The experiments use 4 × 40GB A40 GPUs (Appendix E), but it is unclear how much of this compute is spent on training versus circuit extraction and analysis.

The consequence. The paper presents circuit graph metrics as potential "indicators" that could "inform concept-aware continual pre-training decisions" (Section 1) and motivate "circuit-aware training effort allocation and interference-aware data scheduling" (Limitations). However, for any of these applications to be practical, the cost of computing the indicators must be substantially less than the cost of the training they are meant to inform — otherwise, you might as well just do the training and measure outcomes directly. If extracting concept circuits for 1,000 concepts at a single checkpoint costs, say, 10× the FLOPs of one epoch of FICO training, then using circuit metrics to decide "how much training is needed for new concepts" (Section 1) would be self-defeating: you would spend more compute on diagnosis than on the training you're trying to optimize.

The paper does not discuss this trade-off at all, even qualitatively. The FICO dataset consists of 1,000 fictional concepts, and concept circuits are extracted for 500 test concepts at up to three checkpoints (π_0, π_1, π_2), plus additional extractions for the cross-concept interference analysis (circuit overlap between target and auxiliary concepts for each relatedness group). The total number of circuit extractions is not reported, but it is at minimum 500 concepts × 3 checkpoints = 1,500 extractions, plus the auxiliary concept circuits for the Jaccard similarity analysis in Figure 7. At this scale, the computational cost of the analysis pipeline is likely non-trivial relative to the 10.4M token training budget. For a practitioner considering adopting this framework for continual pre-training on a new domain with thousands of concepts, the circuit extraction cost could dominate the total compute budget, making the approach impractical regardless of the quality of the signals.

What evidence exists in the paper. None. The paper provides no cost analysis for circuit extraction, graph metric computation, or the end-to-end analysis pipeline. The hardware specification (4 × A40 GPUs) is reported for training, but circuit extraction and analysis costs are not disaggregated or discussed. The Limitations section does not mention computational cost as a constraint.

Mitigation status. Not addressed. The paper does not acknowledge that circuit extraction and graph metric computation impose a computational overhead that may limit the practical deployability of the framework, even if the signals prove to be predictive. A discussion of whether cheaper proxies for circuit structure exist (e.g., using activation-based similarity measures instead of full circuit extraction) or whether the analysis can be amortized across many training runs would partially address this, but neither is present.

No Forgetting-Robustness Interventions Are Tested — The Practical Payoff of the Circuit-Level Signals Remains Entirely Speculative

The assumption or constraint. The paper explicitly positions itself as an analytical contribution, not a methods contribution:

"Given the analytical scope of this work, we focus on characterizing the relationship between internal concept circuits and learning dynamics, showing that circuit graph patterns could indicate future learning and forgetting behaviors, and that circuit similarity may signal potential interference. While our findings may inform training strategies like circuit-aware training effort allocation and interference-aware data scheduling, we leave the exploration of these motivated training strategies for future work." (Limitations)

This is an honest scope acknowledgment, but it means the paper's central practical claim — that its findings can "inform the design of more interpretable and robust concept-aware training strategies for LLMs" (Section 1) — is entirely untested. No experiment in the paper demonstrates that using circuit graph metrics, cross-concept relatedness, or knowledge-type transferability to guide training decisions actually improves learning efficiency, reduces forgetting, or mitigates interference compared to a baseline.

The consequence. The paper provides diagnostic signals but no evidence that acting on those signals produces better outcomes. A practitioner reading this paper learns that circuit density correlates with both learning and forgetting (Finding 2), but does not learn what to do about it. Should they regularize towards sparser circuits? Would that reduce forgetting at the cost of slower learning, and is the net effect positive? Should they schedule training data to minimize circuit overlap among co-trained concepts (as suggested in Section 4.1.2)? Would that reduce interference, or would the reduction in data diversity offset any benefit? The paper's findings are consistent with multiple, mutually incompatible intervention strategies — e.g., you could try to increase circuit centralization to accelerate learning on high-priority concepts (accepting more forgetting), or you could try to decrease centralization to improve retention (accepting slower learning). Without intervention experiments showing which direction produces net positive outcomes for which objectives, the practical guidance is ambiguous.

The cross-knowledge-type transfer findings (Finding 6, Figure 8) come closest to providing actionable guidance: the transferability matrix directly suggests that pretraining on PAA before SAA yields a 63.74% improvement relative to the BIO baseline. However, even this finding is demonstrated in an artificial pairwise curriculum (Stage 1: one knowledge type in isolation; Stage 2: another knowledge type in isolation), not in a realistic continual pre-training setting where concepts are learned with all their knowledge types interleaved. Whether the PAA → SAA transfer benefit survives when PAA and SAA are mixed with other knowledge types during Stage 1, or when multiple knowledge types are learned in Stage 2, is untested. The paper does not run an experiment comparing a curriculum ordered by the transferability matrix to a random or reverse-ordered curriculum on overall concept learning performance — the most direct test of whether the signals actually inform better training.

What evidence exists in the paper. The paper provides extensive correlational and observational evidence: circuit metrics correlate with behavior (Figures 3a, 3b, 5), semantic similarity predicts interference (Figure 6), circuit overlap co-varies with interference (Figure 7), knowledge-type ordering affects transfer (Figure 8). But none of these are intervention studies. The paper does not manipulate circuit structure, data ordering, or training hyperparameters based on its findings and measure the effect on downstream concept learning or retention. The Limitations section explicitly acknowledges this gap, but the contributions section and abstract phrase the implications in forward-looking language ("inform the design of," "motivating... data scheduling") that could be read as claiming demonstrated practical utility rather than potential future utility.

Mitigation status. Partially mitigated by transparency — the paper is explicit about its analytical scope and does not claim to have developed or tested training strategies. However, the tension between the contributions language (which implies actionable insights) and the experimental scope (which provides only diagnostic signals) is not fully resolved. A more precise contributions statement would clarify that the paper identifies candidate signals that may inform training strategies, subject to future intervention studies that test whether acting on those signals improves outcomes.

The Learn–Forget Correlation Is Demonstrated Under a Single, Narrow Forgetting Regime

The assumption or constraint. The paper's forgetting analysis (Findings 2–4) is based entirely on a single two-stage protocol: Stage 1 (FICO acquisition) followed by Stage 2 (BIO forgetting induction). The BIO dataset (Allen-Zhu and Li, 2023) consists of biographical texts — factual information about individuals. It is chosen specifically because it is semantically unrelated to the conceptual relational knowledge in FICO: the paper states that BIO is "a standard pre-training dataset used in previous LLM knowledge acquisition studies" that does "not contain the fictional concepts from FICO or their associated knowledge types" (Section 3.2). This choice isolates forgetting as interference from unrelated continued training, which is a clean experimental design for demonstrating that forgetting occurs even without contradictory training signals about the same concepts.

The consequence. The paper's findings about forgetting — the positive learn–forget correlation (Finding 4), the correlation between circuit metrics and forgetting degree (Finding 2), and the stage-wise temporal trajectory during forgetting (Finding 3) — are demonstrated only for this specific forgetting regime: continued training on semantically unrelated text. It is unknown whether the same patterns would hold under different forgetting conditions that are practically common in continual pre-training:

  • Related-domain forgetting: what if Stage 2 training were on a different set of fictional concepts with different relational structures? This would introduce direct representational competition between old and new concepts, potentially producing stronger interference and different circuit reorganization patterns.
  • Same-domain, contradictory-information forgetting: what if Stage 2 introduced new knowledge triples for the same fictional concepts that contradicted the Stage 1 knowledge (e.g., changing (Olre, IsA, animal) to (Olre, IsA, machine))? This would test whether forgetting is driven by passive interference or active overwriting, and might produce fundamentally different circuit dynamics.
  • Multi-domain forgetting: what if Stage 2 involved a mixture of domains (some related to FICO structure, some unrelated), as would occur in realistic continual pre-training where new data is heterogeneous?
  • Longer-duration forgetting: Stage 2 training is matched in duration to Stage 1 (10 epochs over 10.4M tokens). What happens with 2×, 5×, or 10× the forgetting-induction training? Does the learn–forget correlation persist, strengthen, or wash out as all concepts eventually converge to similar forgetting levels?

The paper's interpretation of the learn–forget correlation rests on a mechanistic hypothesis: "circuits with centralized bottlenecks... may facilitate concept acquisition but increase vulnerability to interference or forgetting" because "perturbations to these hubs can induce circuit-wide changes" (Finding 2, Section 3.3). If this mechanism is correct, the strength of the learn–forget correlation should depend on the degree of perturbation experienced by hub nodes during Stage 2. Different forgetting regimes (related-domain, contradictory-information, longer-duration) would produce different perturbation magnitudes and patterns, and the correlation might vary substantially. Without testing multiple regimes, the paper cannot claim to have characterized "concept learning and forgetting dynamics" in general — only the dynamics under one specific, narrow forgetting condition.

What evidence exists in the paper. The two-model replication (GPT-2 Large and LLaMA-3.2-1B-Instruct, Appendix D) tests whether the learn–forget correlation generalizes across architectures under the same BIO forgetting regime, not across different forgetting regimes. Figure 17 (LLaMA-3.2-1B-Instruct) shows a qualitatively similar correlation to Figure 5 (GPT-2 Large), but both use identical Stage 2 training on BIO. No experiments vary the forgetting-induction dataset. The paper does not discuss this limitation or argue that BIO is representative of forgetting regimes in general.

Mitigation status. Not addressed. The paper treats BIO as a generic forgetting-induction dataset without discussing how the specific properties of biographical text might affect the observed dynamics, and without testing whether the findings would replicate under different forgetting conditions. This limits the generality of the forgetting-related findings, which constitute roughly half of the paper's contributions (Findings 2–4).

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a mechanistic diagnostic framework for concept learning in LLMs during continual pre-training — it does not propose a new training algorithm, architecture, or optimization procedure, and it does not claim to improve any performance metric. Rather, it provides the first systematic evidence that the internal computational subgraphs associated with individual concepts (Concept Circuits) carry quantifiable structural signals that correlate with how those concepts are learned and forgotten. This shifts the conversation around continual learning in LLMs from a purely behavioral enterprise (measuring aggregate accuracy before and after training) toward a circuit-informed approach where internal representational structure is treated as a measurable, trackable variable that co-evolves with learning outcomes.

The magnitude of this shift is incremental but enabling — it is not a paradigm shift (the core training paradigm — next-token prediction on sequential data — is unchanged), but it opens a new empirical axis for studying continual learning. Prior to this work, a researcher studying catastrophic forgetting in LLMs could measure that forgetting occurred and could hypothesize about why (overwriting, interference, capacity saturation), but had no way to look inside the model and observe the structural reorganization accompanying forgetting. The paper's demonstration that circuit graph metrics follow a consistent stage-wise temporal trajectory during forgetting (Finding 3, Figure 4) — early increase, gradual decrease, stabilization — provides a dynamic signature of forgetting at the circuit level. This signature is not accessible through behavioral metrics alone, which would show only a monotonic accuracy decline without revealing the underlying phases of transient entanglement and structural relaxation. For the mechanistic interpretability community, this establishes that circuit extraction and graph-theoretic characterization are not just tools for understanding static, already-trained models but can be productively applied to training dynamics — a methodological extension that opens the door to time-resolved circuit analysis across many training phenomena beyond concept learning (e.g., in-context learning emergence, phase transitions, safety-relevant capability acquisition).

For the continual learning community, the paper reframes catastrophic forgetting as entanglement-driven interference rather than passive decay. The positive correlation between circuit density (or global efficiency, or k-core depth) and both learning degree and forgetting degree (Finding 2, Figures 3a and 3b) implies that the structural properties that make a concept easy to acquire are the same properties that make it vulnerable to disruption. This is a more specific mechanistic story than "new knowledge overwrites old knowledge" — it suggests that concepts supported by dense, centralized, tightly integrated circuits become entangled with each other and with subsequently learned knowledge, such that perturbations to shared computational infrastructure (hub nodes, densely connected cores) propagate broadly and induce forgetting across multiple entangled concepts. If this story is correct, it directs intervention efforts away from generic replay or regularization and toward modularity-promoting training strategies — techniques that encourage concept circuits to be sparser, more decentralized, and less overlapping, even at some cost to initial learning speed. The paper does not test such strategies, but it provides the diagnostic toolkit (circuit extraction, graph metrics, Jaccard overlap) that would be needed to evaluate them.

The paper also provides the first empirical reconciliation of why prior studies of LLM knowledge acquisition reached conflicting conclusions about interference and synergy. Prior work on factual knowledge learning (Allen-Zhu and Li, 2023; Zucchet et al., 2025a; Ou et al., 2025) treated knowledge as atomic facts and studied interference primarily through the lens of data repetition, ordering, and hallucination. The finding that semantically similar concepts interfere more strongly than weakly related ones (Finding 5, Figure 6: 57.5% vs. 75.9% performance) and that this interference is associated with circuit overlap (Figure 7) provides a structured explanation for why some knowledge combinations are more destructive than others — it is not just about the quantity of new data, but about the representational proximity of old and new concepts in the model's computational graph. Similarly, the finding that different knowledge types exhibit highly asymmetric transfer (Finding 6, Figure 8: PAA → SAA transfer at +63.74%, with markedly weaker reverse transfer) explains why some curricula work better than others — the semantic structure of conceptual knowledge, not just its surface form or frequency, determines how learning one thing affects learning another. This moves the curriculum design conversation from heuristics ("easy before hard") toward empirically grounded, knowledge-type-aware sequencing.

Practically, the paper makes circuit-level diagnostics conceivable as a tool for guiding continual pre-training decisions, even though it stops short of demonstrating that such guidance improves outcomes. The correlations between circuit graph metrics and behavioral measures are modest (Spearman ρ values visually estimated in the 0.1–0.3 range from Figures 3a and 3b) but statistically significant — they establish that the signal exists, which is a necessary precondition for developing interventions that act on it. The paper also identifies where the signal is weakest: the hardest-to-learn concepts (those with near-zero learning degree, likely corresponding to the left tail of the distribution in Figure 2) show little circuit-level differentiation, and the cross-knowledge-type transfer analysis reveals that Meronym & Holonym knowledge (MAH) is largely insensitive to curriculum ordering (Figure 8), suggesting that some types of conceptual knowledge are acquired through relatively isolated, specialized pathways that do not benefit from or interfere with other knowledge types. These negative results are as informative as the positive ones — they bound the scope of where circuit-informed training strategies are likely to be useful.

The paper also makes certain research directions less attractive by implication. The finding that circuit graph metrics simultaneously predict stronger learning and stronger forgetting (the structural trade-off in Finding 2) suggests that there is no free lunch — any circuit organization that accelerates acquisition also increases vulnerability. This implies that purely optimization-based approaches to mitigating forgetting (e.g., "just train longer" or "just tune the learning rate better") are unlikely to resolve the stability–plasticity tension, because the tension is baked into the circuit topology itself. Approaches that seek to separate acquisition and retention mechanisms — for example, through architectural modifications that allocate distinct computational capacity to new vs. old concepts, or through modular training objectives that penalize circuit overlap — become more attractive. Similarly, the finding that cross-concept interference is driven by circuit overlap (Figure 7) and that this overlap is systematically higher for semantically similar concepts suggests that naive data diversity (mixing many concepts together) is insufficient — what matters is diversity in representational space, which may not align with surface-form diversity.

Research directions that become more attractive:

  • Modularity-promoting regularization during continual pre-training: techniques that explicitly penalize circuit overlap or encourage sparse, decentralized concept representations during training, potentially at the cost of slower initial learning but with better long-term retention. The paper's graph metrics (eigenvector centrality variance, density, Jaccard similarity) provide natural optimization targets or monitoring signals for such approaches.
  • Concept-aware data scheduling informed by embedding similarity: the finding that training with highly related concepts produces 18.4 percentage points worse performance than training with weakly related concepts (Figure 6) suggests that simply spacing out the training of similar concepts — or interleaving them with dissimilar ones — could reduce interference without changing the total training data. This is a lightweight intervention that does not require circuit extraction at deployment time, only pre-computed concept embeddings.
  • Knowledge-type curricula for efficient concept learning: the transferability matrix in Figure 8 provides a concrete recipe for ordering knowledge types during training to maximize synergy. A practitioner introducing new concepts to an LLM could train on Property & Affordance knowledge first (as it scaffolds multiple downstream types), then Hyponym & Hypernym, then Synonym & Antonym, and evaluate whether this ordering outperforms random or reverse ordering on overall concept acquisition metrics.
  • Circuit-based early stopping or training duration allocation: the stage-wise temporal trajectory (Finding 3, Figure 4) suggests that circuits undergo distinct phases during forgetting induction. If the early-phase increase in circuit metrics (transient entanglement) can be detected quickly, it could serve as an early warning signal that previously learned concepts are at risk, triggering protective interventions (replay, learning rate reduction, gradient shielding) before substantial forgetting occurs.
  • Scaling studies of circuit topology: the paper's findings are limited to 0.7B–1B models, and it is unknown whether circuit graph metrics carry the same signal at larger scales. A systematic study tracking the same metrics across model sizes (e.g., 1B, 7B, 13B, 70B) during concept learning would reveal whether the structural trade-off (density/centrality predict both learning and forgetting) strengthens, weakens, or qualitatively changes with scale — a finding with direct implications for whether circuit-informed continual learning strategies are relevant for frontier models.

Follow-Up Research This Work Enables

Prospective (not concurrent) circuit–behavior prediction. The paper demonstrates that circuit graph metrics measured after learning correlate with learning outcomes (concurrent validity), but does not test whether circuit metrics measured before training predict subsequent learning (predictive validity). A direct follow-up would: (1) extract Concept Circuits from the base model π_0 (before any FICO training) for all 500 test concepts, (2) compute the four graph metrics on these pre-training circuits, (3) train the model on FICO as in Stage 1, (4) measure concept learning degree for each test concept, and (5) compute the Spearman correlation between pre-training circuit metrics and subsequent learning degree. If significant correlations emerge, this would establish that the base model's existing computational infrastructure — before any concept-specific training — already contains structural biases that predict which concepts will be learned easily. This would transform circuit metrics from descriptive to genuinely predictive indicators. A parallel analysis would test whether circuit metrics at π_1 (after FICO, before BIO) predict subsequent forgetting degree during Stage 2, which would establish whether post-acquisition circuit structure can forecast vulnerability to interference.

Intervention study: does penalizing circuit overlap during training reduce cross-concept interference? The paper demonstrates that semantically similar concepts interfere more during joint training (Figure 6) and that this interference is associated with higher Jaccard similarity between concept circuit edge sets (Figure 7). However, the causal direction is untested. A direct follow-up would: (1) select a set of target concepts and their highly related auxiliary concepts (as defined by Qwen3-Embedding-4B cosine similarity), (2) train the model on these concepts jointly, but add an auxiliary loss term that penalizes the Jaccard similarity (or a differentiable proxy, such as the cosine similarity of edge importance vectors from EAP-IG) between the target and auxiliary concept circuits during training, (3) compare concept learning performance on the target concepts against a baseline trained without the overlap penalty. If the overlap penalty reduces interference (performance closer to the weakly-related baseline of 75.9% rather than the highly-related 57.5%), this would provide causal evidence that circuit overlap is a mechanism of interference, not just a correlate, and would validate circuit-overlap minimization as a practical training strategy. A negative result (the penalty doesn't help, or hurts by constraining useful representational sharing) would be equally informative, suggesting that overlap is a symptom rather than a cause of interference.

Replication of the learn–forget correlation under diverse forgetting regimes. The paper's central behavioral finding — concepts learned more aggressively are forgotten more under subsequent training (Finding 4, Figure 5) — is demonstrated only when Stage 2 uses the BIO dataset (unrelated biographical text). A systematic follow-up would replicate the two-stage protocol with multiple Stage 2 datasets spanning a spectrum of relatedness to FICO: (a) highly related (a different set of fictional concepts with similar relational structures, drawn from ConceptNet but with different fictional names), (b) moderately related (general world knowledge text, e.g., Wikipedia, which contains some conceptual relational knowledge but not the FICO concepts), (c) unrelated but structured (code, which has its own relational structure but no semantic overlap with FICO concepts), and (d) longer-duration training on each. The prediction from the paper's entanglement hypothesis is that the learn–forget correlation should be strongest for highly related Stage 2 data (where new concepts directly compete for overlapping circuits), moderate for unrelated but structured data, and weakest for unstructured or maximally dissimilar data. If the correlation magnitude varies systematically with Stage 2 data relatedness, this would provide converging evidence for the entanglement mechanism. If the correlation is uniform across all regimes, it would suggest a more generic learning–fragility relationship not specific to representational overlap.

Scaling analysis of circuit graph metrics across model sizes. All findings are at 0.7B–1B parameters. A natural follow-up would replicate the core analyses on a ladder of model sizes sharing the same architecture — for example, GPT-2 (124M), GPT-2 Medium (355M), GPT-2 Large (774M, the paper's primary model), and GPT-2 XL (1.5B), or the LLaMA-3.2 family (1B, 3B, 8B). For each model size, the experiment would: (1) train on FICO for the same number of tokens (not epochs, to control for total information exposure), (2) extract concept circuits and compute graph metrics, (3) measure learning degree and forgetting degree, (4) compute the Spearman correlations between circuit metrics and behavioral measures. The key question is: does the structural trade-off (circuit density/centrality predicting both stronger learning and stronger forgetting) hold at larger scales, or does it weaken as models develop more distributed, polysemantic representations? If the correlations strengthen with scale, this would suggest that circuit-aware strategies become more important for larger models; if they weaken, it would bound the practical relevance of the paper's approach to smaller models. The experiment would also track how circuit graph metric values themselves (mean density, mean global efficiency, mean k-core depth) change with model size for the same concepts — a descriptive finding that would inform whether "modularity" is naturally more or less prevalent at different scales.

Knowledge-type curriculum optimization: does the transferability matrix improve end-to-end concept learning? The paper's Figure 8 provides a 5×5 matrix of pairwise transfer effects, but never tests whether ordering training by this matrix improves overall concept acquisition compared to baselines. A direct optimization experiment would: (1) define a total training budget (e.g., 10 epochs total across all knowledge types), (2) use the transferability matrix to design a curriculum that places high-transfer source types (e.g., PAA) early and low-transfer types (e.g., MAH) later, with the proportion of training time allocated to each type proportional to its transfer benefit to subsequent types, (3) compare end-to-end concept learning performance (average across all knowledge types and concepts) against: (a) a random ordering of knowledge types, (b) a reverse ordering (placing low-transfer types first), (c) a balanced interleaved curriculum where all knowledge types are trained simultaneously, and (d) the pairwise curriculum used in the paper (Stage 1: one type, Stage 2: another type) as a sanity check. If the transfer-matrix-optimized curriculum significantly outperforms random ordering, it would validate the matrix as a practical tool for curriculum design. If it doesn't outperform interleaved training, it would suggest that the benefits of knowledge-type pretraining observed in the paper's pairwise design do not survive in a more realistic multi-type training setting, which would be an important negative result bounding the applicability of the transfer findings.

Concept-level forgetting prediction for targeted replay. The paper shows that concept forgetting degree varies substantially across concepts (Figure 2) and that circuit metrics correlate with forgetting (Figure 3b). A practical follow-up would test whether these signals can be used to implement a targeted replay strategy that is more efficient than uniform replay. The experiment would: (1) after Stage 1 FICO training, extract concept circuits and compute graph metrics for all concepts, (2) rank concepts by predicted forgetting vulnerability (using the circuit metric most correlated with forgetting — likely eigenvector centrality variance or density based on Figure 3b), (3) during Stage 2 BIO training, interleave replay of FICO examples, but allocate replay budget disproportionately to the top-K most vulnerable concepts, (4) compare forgetting outcomes against: (a) no replay, (b) uniform replay (same total replay budget, distributed equally across all concepts), and (c) an oracle strategy that allocates replay to the concepts that actually forgot the most (using post-hoc ground truth). If circuit-metric-guided replay outperforms uniform replay at the same total budget, it would demonstrate practical utility for the diagnostic signals. If it performs no better than uniform replay, it would suggest that the correlations are too weak to support targeted interventions, or that the causal relationship between circuit metrics and forgetting is not sufficiently predictive at the individual concept level.

Practical Applications and Downstream Use Cases

Continual pre-training data scheduling for domain-specific LLMs. Organizations that periodically update their deployed LLMs with new domain knowledge (e.g., a medical LLM ingesting new drug interaction data, a legal LLM incorporating new case law, a financial LLM learning about new financial instruments) face the problem of integrating new concepts without degrading existing ones. The paper's cross-concept interference finding (Figure 6: training a target concept alongside highly related concepts drops performance from 75.9% to 57.5%) provides a concrete scheduling heuristic: when introducing a batch of new concepts, measure their embedding similarity to each other and to existing concepts (using an external embedding model like Qwen3-Embedding-4B, as the paper does), and schedule training to avoid co-training highly similar concepts in the same batch or in close succession. This requires no circuit extraction at deployment time — only pre-computed concept embeddings — making it computationally lightweight. The potential benefit is reducing interference-driven forgetting without increasing total training compute, by simply reordering the data. The paper's numbers suggest that moving from a "highly related" to a "weakly related" co-training regime could recover up to 18.4 percentage points of performance on target concepts, though this is an upper bound from the controlled experimental setting and would likely be smaller in a realistic multi-concept, multi-epoch training scenario.

Knowledge-type ordering for efficient fine-tuning on structured knowledge bases. When fine-tuning an LLM on a structured knowledge base (e.g., a company's internal ontology, a product catalog with typed attributes, a scientific knowledge graph), practitioners must decide in what order to present different types of relational knowledge (taxonomic, functional, part–whole, spatial, etc.). The paper's transferability matrix (Figure 8) provides an empirically grounded ordering: Property & Affordance knowledge (PAA) strongly scaffolds subsequent Synonym & Antonym (SAA) learning (+63.74% relative to baseline) and Hyponym & Hypernym (HAH) learning, suggesting that training on functional and attribute knowledge first creates a representational foundation that accelerates acquisition of taxonomy and similarity relations. A practitioner could structure their fine-tuning curriculum as PAA → HAH → SAA → SR → MAH (roughly following the transfer-benefit gradient visible in Figure 8), with the expectation that this ordering will achieve faster convergence and/or higher final accuracy than random or reverse ordering on the same total training budget. The MAH finding — that part–whole knowledge benefits little from any source type — is also actionable: MAH can be placed anywhere in the curriculum without much impact, providing flexibility in scheduling. The caveat is that this ordering is derived from pairwise curricula on fictional concepts; its transfer to real-world knowledge bases with different statistical properties is plausible but untested.

Diagnostic monitoring of concept circuit stability during continual pre-training for early forgetting detection. For high-stakes deployments where forgetting critical knowledge is costly (e.g., an LLM used for medical diagnosis must not lose knowledge of drug contraindications after updating with new research), the paper's stage-wise temporal trajectory (Finding 3, Figure 4) suggests a monitoring approach: periodically extract concept circuits for a small set of high-priority concepts during continued training and track their graph metrics. An early-phase increase in metrics like circuit density or global efficiency — the "transient entanglement" phase the paper identifies — could serve as an early warning that these concepts are entering a vulnerable period where interference is actively reorganizing their circuits. The deployment team could then trigger protective measures (temporarily increasing the learning rate for replay of critical concepts, pausing the introduction of new concepts, or reducing the learning rate on shared parameters) before substantial forgetting occurs. The practical barrier is the computational cost of circuit extraction (which the paper does not quantify) — this monitoring approach would need to be made efficient enough to run periodically without dominating the training compute budget, perhaps by extracting circuits for only a small sample of priority concepts rather than the full concept inventory.

When to Prefer This Method

The paper is an analytical and diagnostic contribution, not a proposed method that competes with alternatives for a performance objective. It does not position Concept Circuits, graph metrics, or the transferability matrix against named alternative approaches to continual learning (e.g., EWC, experience replay, progressive networks, parameter-efficient fine-tuning with adapters) and does not provide experiments comparing its analytical framework to any alternative diagnostic tool. Consequently, a "When to Prefer This Method" decision matrix would be a fabrication — the paper does not articulate a tradeoff against specific named alternatives, and constructing one would impose a competitive framing the paper itself avoids. The appropriate decision rule is not "prefer Concept Circuits over X" but rather: researchers and practitioners studying continual pre-training dynamics who want circuit-level visibility into how individual concepts are acquired, reorganized, and forgotten should adopt this paper's analytical toolkit (FICO-style controlled concept learning, EAP-IG circuit extraction, four families of graph metrics, relatedness-based interference groups, knowledge-type transfer matrices) as a diagnostic framework for generating hypotheses and evaluating interventions. The paper demonstrates that this toolkit produces systematic, replicable patterns (the structural trade-off, the stage-wise temporal trajectory, the learn–forget correlation, the relatedness-dependent interference, the asymmetric knowledge-type transfer), and these patterns provide a baseline against which future interventions (modularity-promoting regularization, interference-aware scheduling, targeted replay) can be evaluated. The toolkit is appropriate for analytical studies of small-to-medium scale models (0.7B–1B parameters demonstrated, scaling behavior unknown) on structured relational knowledge with clean evaluation signals, and is not currently practical for deployment-time decision-making due to the unquantified computational cost of circuit extraction.