Configurable Foundation Models: Building LLMs from a Modular Perspective

🎯 Pitch

Full-scale LLMs spontaneously develop functionally specialized neuron clusters during pre-training—pruning 95% of neurons for a coding task barely harms translation ability. This paper shows how to harness that latent modularity by treating specialized neuron groups as "bricks" that can be routed, merged, or updated on the fly, collapsing the gap between monolithic models and efficient, composable on-device intelligence.

1. Executive Summary

This paper introduces a modular framework for constructing and analyzing LLMs by decomposing them into functional components called bricks—both emergent bricks that arise spontaneously during pre-training (e.g., functionally specialized neuron clusters or mixture-of-experts experts) and customized bricks explicitly built post-training to inject new capabilities (e.g., LoRA adapters for task adaptation, knowledge bricks for factual updates, modality bricks for vision-language integration). The framework defines four primitive brick-oriented operations—routing and retrieval (selecting relevant bricks per instruction, as in MoE gating), combination (merging or stitching bricks for composite abilities, as in parameter averaging or multi-model pipelines), updating (editing specific bricks for knowledge correction, as in locating and modifying knowledge neurons), and growing (expanding the brick repository for continual learning, as in progressive expert addition)—that together enable dynamic, instruction-specific model configuration. Empirical analysis on Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3 validates the core premises by showing that FFN neurons exhibit sparse activation (with ~80% of neurons contributing negligible output magnitude), functional specialization (top-5% neurons for a given capability—coding, math, translation—achieve functionality scores exceeding 0.8 while pruning them minimally impacts other capabilities), and distinct partitioning across seven functionality types (average cross-functionality neuron overlap similarity of only ~0.02), establishing that well-trained LLMs already possess a decomposable modular structure amenable to the proposed brick operations.

2. Context and Motivation

The Core Problem: Monolithic LLMs Cannot Meet the Demands of Pervasive Deployment

The paper addresses a structural tension at the heart of modern LLM deployment. Current state-of-the-art LLMs—GPT-4, Llama 3, Mistral—are trained and deployed as monolithic entities: a single, unified set of parameters that must be loaded in its entirety and executed fully for every input, regardless of the task's complexity or the specific capabilities required. The authors argue that this monolithic paradigm is fundamentally incompatible with three emerging trends in real-world LLM application.

Trend 1: Deployment on end devices with limited compute. The paper observes a growing push to bring LLMs onto smartphones, laptops, and other consumer devices (Section 1), citing Apple's on-device foundation model efforts (Apple, 2024) and work on efficient inference like PowerInfer (Xue et al., 2024). A monolithic model with hundreds of billions of parameters that requires substantial GPU memory and high-bandwidth compute is simply infeasible on a phone. The authors frame this as an efficiency problem: the model contains far more parameters than any single query needs, but the monolithic architecture forces all of them to participate in computation.

Trend 2: Domain-specific knowledge and capability requirements diverge. LLMs are being applied across an increasingly broad range of domains—legal document analysis, medical diagnosis support, code generation, multilingual translation, creative writing—each requiring distinct knowledge and reasoning skills. Storing all of this in a single model creates two problems. First, redundant computation: when a user asks a coding question, the parameters encoding medical knowledge or translation ability are irrelevant but still consume compute. Second, knowledge conflicts: when the model must serve multiple domains, fine-tuning on new domain knowledge can interfere with existing knowledge, degrading performance on previously mastered tasks—the classic catastrophic forgetting problem that is especially acute when all parameters are shared.

Trend 3: The world changes continuously. Knowledge becomes outdated (capital cities change, scientific understanding evolves, new programming languages emerge). New tasks appear that the model was never trained for. The paper emphasizes that LLMs must evolve efficiently and continuously—learning new knowledge and skills while retaining what they already know. Retraining a trillion-parameter model from scratch every time the Wikipedia entry for a country changes is clearly infeasible. Yet monolithic architectures make targeted updates difficult: changing a single fact (e.g., the current Prime Minister of the UK) by fine-tuning can unpredictably ripple through shared parameters and damage unrelated capabilities.

These three trends share a common root cause: all parameters are entangled. There is no mechanism to isolate the parameters responsible for a specific capability, activate only those when needed, and update only those when knowledge changes.

The Conceptual Inspiration: Modularity in Natural and Engineered Systems

The paper draws its motivation from a deep intellectual tradition that spans multiple disciplines (Section 1). The authors invoke modularity as a universal design principle that enables complex systems to be scalable, maintainable, and efficient:

Neuroscience: The human brain exhibits functional modularity—the visual cortex processes visual signals, Broca's area handles speech production, and these specialized regions coordinate without requiring every neuron to participate in every cognitive task. The paper explicitly cites the "modularity of mind" hypothesis (Fodor, 1983) and neuroscientific evidence for sparse activation (only a small fraction of neurons fire at any given moment).
Software engineering: Complex software systems are decomposed into modules with well-defined interfaces. This enables independent development, testing, and maintenance, and prevents changes in one module from cascading unpredictably through the entire codebase.
Industrial manufacturing: Products from cars to appliances are assembled from modular components. A faulty component can be replaced without redesigning the entire product; new features can be added by swapping in upgraded modules.

The key properties that define a module in these domains—independence (decoupled from other modules), specificity (responsible for a well-defined function), and composability (can be combined with other modules to create complex behavior)—are precisely what the authors argue are missing from monolithic LLMs but are beginning to emerge naturally from the training process.

Prior Approaches: Fragmented Insights Without a Unifying Framework

The paper does not claim that modular thinking about neural networks is new. Rather, it argues that prior work has generated a wealth of isolated observations and techniques that collectively point toward modularity, but that no one has assembled these into a coherent framework. The authors organize these prior efforts into categories that they will later integrate, highlighting what each category gets right and what remains unresolved.

Observations of Emergent Modularity in Dense Models

A substantial body of mechanistic interpretability work has shown that when dense Transformer models are trained end-to-end, they spontaneously develop functional specialization in subsets of their parameters, even though no explicit modular structure was designed into the architecture.

Activation sparsity refers to the finding that for any given input, only a small fraction of neurons in the feed-forward network produce non-zero outputs (Zhang et al., 2022c; Li et al., 2023c). The paper notes that in some models, as few as 5% of neurons are active for 90% of inputs. This is not a property that was trained for—it emerges naturally. The implication is profound: the model has far more capacity than it uses for any single input, suggesting that different neurons are specialized for different types of inputs.

Function localization takes this further by showing that specific capabilities and knowledge are concentrated in identifiable neurons or neuron groups. The paper cites three lines of evidence. Dai et al. (2022) and Meng et al. (2022) identified "knowledge neurons"—specific FFN neurons whose activations are highly predictive of whether the model knows a particular factual tuple (e.g., "Beijing is the capital of China"), and where manipulating those neurons' weights can edit or erase that knowledge without retraining. Wang et al. (2022a) found "skill neurons" that are individually predictive of performance on specific NLP tasks like sentiment analysis or natural language inference. Tang et al. (2024) and Zhao et al. (2023a) discovered "language regions" specialized for processing specific languages in multilingual models.

Self-organized neuron clusters represent the most sophisticated form of emergent structure. Zhang et al. (2023c) showed that neurons within FFN layers form functional groups that activate together and are jointly responsible for specific capabilities—semantic understanding, knowledge storage, task-specific reasoning. These clusters do not respect the layer boundaries humans designed; they emerge from the interactions between parameters across the model.

What's missing from this body of work is a framework for operationalizing these observations. Knowing that knowledge neurons exist is one thing; having a systematic protocol for identifying, isolating, activating, updating, and composing them is another. Individual papers demonstrate that specific neurons can be edited for specific facts, but there is no general methodology for treating the model as a collection of manipulable functional units.

Explicitly Modular Architectures: Mixture-of-Experts

The mixture-of-experts (MoE) paradigm (Shazeer et al., 2017; Fedus et al., 2022b; Lepikhin et al., 2021) represents the most direct attempt to impose modularity on Transformer architectures. In an MoE model, the standard FFN layer is replaced with multiple parallel "expert" sub-networks, and a learned gating function routes each token to a small subset of these experts. This achieves two goals simultaneously: the model's total parameter count increases (more capacity) while the per-token computation remains roughly constant (since only a few experts are active).

The paper acknowledges MoE as a successful demonstration of the modular principle, but identifies critical limitations. First, the modularity is pre-defined by humans—the number of experts, their architecture, and the gating mechanism are explicit design choices made before training begins. The model cannot discover its own optimal modular decomposition. Second, the experts are architecturally homogeneous (all identical FFN replacements), which limits the granularity of specialization. Third, MoE focuses primarily on the routing/selection aspect of modularity and does not address combination, updating, or growth in a unified way.

Crucially, the paper notes that even within MoE models, finer-grained functional specialization emerges beyond the expert boundaries, as shown by Zhang et al. (2023c) finding that individual experts themselves contain functionally specialized neuron clusters. This suggests that human-defined modules are a coarse approximation of a much richer modular structure that training naturally produces.

Parameter-Efficient Fine-Tuning as De Facto Modularity

The third major line of prior work the paper draws on is parameter-efficient fine-tuning (PEFT)—methods like LoRA (Hu et al., 2022), adapters (Houlsby et al., 2019), prefix tuning (Li & Liang, 2021), and prompt tuning (Lester et al., 2021) that adapt a frozen pre-trained model to new tasks by introducing and training only a tiny fraction of additional parameters. The paper argues that PEFT is, in essence, constructing customized bricks without explicitly conceptualizing them as such.

The theoretical justification comes from the intrinsic dimensionality literature (Aghajanyan et al., 2021), which showed that the optimization landscape for fine-tuning language models has a surprisingly low intrinsic dimension—often on the order of a few hundred parameters, even for models with hundreds of millions of parameters. This means that task adaptation can be reparameterized into a very low-dimensional subspace, which is exactly what PEFT methods do in practice.

The paper views PEFT methods as constructing "task bricks"—small, pluggable parameter sets that encode the capability to perform a specific task. The adapter layers in adapter tuning, the low-rank matrices in LoRA, and the continuous prompt embeddings in prefix tuning all function as modular add-ons that can be swapped in and out without modifying the base model. This is modularity in practice, even if it wasn't designed with that framing.

What the paper identifies as missing is the extension of this modular logic beyond task adaptation. If knowledge could be packaged into similar plug-and-play bricks (as Zhang et al., 2023b and Xiao et al., 2023b have begun to explore for knowledge graph and document knowledge), if modality interfaces could be treated as bricks (as in Flamingo's frozen vision encoder feeding into a frozen LLM; Alayrac et al., 2022), then the entire post-training enhancement of LLMs could be unified under a single modular paradigm.

Knowledge Editing: Updating Bricks Without Knowing They're Bricks

The knowledge editing literature has developed techniques for surgically modifying specific facts stored in LLMs without retraining. Methods like ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023) locate the FFN neurons responsible for storing a particular fact and apply targeted weight updates that change that fact while minimizing disruption to other knowledge. The paper frames this as an updating operation on emergent knowledge bricks: the method first finds where the knowledge lives (locating the brick) and then modifies it (updating the brick).

The limitation is that these methods are largely ad hoc and fact-specific. Each knowledge edit requires its own localization and optimization procedure. There is no general theory of which bricks exist, how they interact, or how to manage dependencies between them. When a fact is updated, related facts stored in overlapping bricks may be corrupted in ways that are hard to predict or detect.

How This Paper Positions Itself

The paper's central thesis is that all of these disparate lines of work—emergent specialization in dense models, explicit modularity in MoE, task adaptation via PEFT, knowledge editing—are different manifestations of the same underlying principle: that LLMs are, or can be, composed of functional bricks. The contribution is not a new technique but a conceptual synthesis that:

Names and taxonomizes the phenomenon, introducing the terminology of "bricks," "emergent bricks" (decomposed from the pre-trained model), and "customized bricks" (added post-training) to give the field a shared vocabulary.
Defines primitive operations (routing/retrieval, combination, updating, growing) that abstract over existing methods and reveal their common structure. MoE gating is a special case of routing; LoRA merging is a special case of combination; knowledge editing is a special case of updating; progressive network expansion is a special case of growing.
Provides empirical validation that the core premises hold in modern decoder-only instruction-tuned LLMs (Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3), not just in the encoder models (BERT, T5) where most prior mechanistic analysis was conducted.
Identifies the gaps that remain before this modular vision can be fully realized—the difficulty of combining heterogeneous bricks, the lack of universal brick construction protocols, the open questions around evaluating brick systems, and the need for efficient sparse computing frameworks.

The paper explicitly acknowledges its nature as a survey and framework paper rather than a new methods contribution, stating that it aims to "offer a comprehensive overview and investigation of the construction, utilization, and limitation of configurable foundation models" and to "inspire the future creation of more efficient and scalable foundational models" (Abstract). Its positioning is therefore as an integrative work that creates conceptual order out of fragmented empirical observations, providing a roadmap for future research rather than claiming a specific technical breakthrough.

Why Now: The Convergence of Enabling Conditions

Implicit in the paper's motivation is a recognition that several conditions have recently converged to make the modular perspective timely:

Model scale has reached a threshold where efficiency is no longer optional. When models were small enough to run on a single GPU, the overhead of a monolithic architecture was acceptable. Now that deployment targets include phones and the largest models require datacenter-scale infrastructure, the cost of loading and running unnecessary parameters has become prohibitive.
Mechanistic interpretability has matured enough to locate functional units with reasonable confidence. Five years ago, the internal structure of Transformers was largely opaque; now we can point to specific neurons that encode specific facts and specific neuron groups that implement specific skills. This makes it plausible to actually build systems that manipulate these units.
The open-source LLM ecosystem has proliferated to the point where multi-model cooperation is feasible. With thousands of fine-tuned models available on HuggingFace, the vision of treating entire models as bricks in a larger system (Section 5.5) is no longer science fiction.
Parameter-efficient methods have proven that small parameter sets can encode meaningful capabilities. The success of LoRA and similar methods demonstrates that complex task behaviors can be compressed into surprisingly few parameters, making the "brick" abstraction computationally plausible.

The paper is thus positioned as a timely synthesis that provides the vocabulary, taxonomy, and research agenda for a field that is already moving in the modular direction but has not yet articulated what it is collectively building toward.

3. Technical Approach

3.1 Reader Orientation

This is a survey and framework paper that synthesizes existing observations about modularity in LLMs into a unified conceptual framework. Rather than proposing a single new technical method, the paper defines a taxonomy of functional modules (bricks) and a set of primitive operations on them—routing, combining, updating, and growing—that together provide a language for describing, analyzing, and designing modular LLM systems. The core idea is that LLMs can be understood as compositions of functional building blocks, some of which emerge spontaneously during pre-training (emergent bricks) and some of which are deliberately constructed post-training (customized bricks), and that by making this structure explicit through defined operations, we can build more efficient, adaptable, and scalable models.

The problem this framework addresses is the rigidity and inefficiency of monolithic LLMs: every query activates all parameters regardless of relevance, every knowledge update requires navigating entangled parameter spaces, and combining capabilities from different models requires ad hoc engineering. The "shape" of the solution is a brick abstraction with defined interfaces—each brick is a collection of function-specific neurons with well-defined input/output behavior—plus a small set of generic operations that can be composed to handle complex real-world requirements without retraining entire models from scratch.

3.2 Big-Picture Architecture (Diagram in Words)

The configurable foundation model architecture has five major conceptual layers:

Brick Repository — the collection of all available functional bricks, partitioned into two categories. Emergent bricks are parameter subsets that differentiated into specialized functions during pre-training (e.g., self-organized neuron clusters in FFN layers that handle specific capabilities like math or translation). Customized bricks are small parameter sets deliberately trained post-hoc to inject new capabilities (e.g., LoRA low-rank matrices for a new task, knowledge embeddings for a domain-specific graph, vision encoder adapters for multimodal understanding). Bricks vary in granularity from single neurons (finest) through neuron groups and layers to entire pre-trained models (coarsest).
Difficulty/Capability Estimator — a mechanism for determining which capabilities a given instruction requires. This can operate at different granularities: token-level routing determines which experts process each token (as in MoE gating), instruction-level analysis identifies which task types are needed, and domain-level detection selects appropriate knowledge sources. The paper's empirical analysis (Section 4) demonstrates that in practice, neurons exhibit sparse, specialized activation patterns that make such estimation feasible.
Brick Selection and Composition Engine — the set of four primitive operations that configure which bricks participate in computation and how they interact. Routing and retrieval selects relevant bricks from the repository based on the input. Combination merges or stitches multiple bricks to create composite capabilities. Updating modifies specific bricks when knowledge or capabilities need to change. Growing expands the repository with new bricks for emerging requirements. These operations can be composed hierarchically.
Configurable Computation Graph — the instantiated execution plan for a specific instruction, specifying which bricks are active, in what order they execute, and how information flows between them. Unlike a monolithic forward pass where all parameters participate in a fixed layer-by-layer sequence, the configurable graph can skip irrelevant bricks, route through specialized experts, and incorporate external knowledge modules dynamically.
Output Aggregation — the final assembly of brick outputs into a coherent response. When multiple bricks contribute (e.g., a coding brick and a knowledge brick both relevant to a software documentation query), their outputs must be combined. The paper notes that existing work handles this through hidden state injection (for continuous interfaces), text concatenation (for discrete interfaces), or ensemble methods (for parallel brick outputs).

Information flows as follows: an instruction enters → the capability estimator determines which functionalities are needed → the brick selection engine queries the repository for relevant bricks → the computation graph is instantiated with the selected bricks in the appropriate topology → each brick executes its specialized computation on its inputs → outputs are aggregated → the final response is produced. When the world changes, the updating and growing operations modify individual bricks without requiring full model retraining.

3.3 Roadmap for the Deep Dive

First, the brick formalization and taxonomy (emergent vs. customized), because this is the fundamental abstraction that all operations act upon—understanding what a brick is and how bricks arise from training is prerequisite to understanding what operations can do.
Second, brick granularity as a design dimension, because the choice of granularity (neuron, neuron group, layer, full model) determines both the expressiveness of the brick and the complexity of managing it—this tradeoff shapes everything downstream.
Third, the four primitive operations (routing/retrieval, combination, updating, growing), because these are the "verbs" of the framework that define what can be done with bricks—each operation is a distinct capability with its own mechanisms, design choices, and open challenges.
Fourth, the empirical validation methodology (sparse activation, functionality specialization, functionality partition), because the paper's framework would remain purely theoretical without evidence that real LLMs actually exhibit the decomposable modular structure it assumes—this section explains how the paper tests its premises on modern models.
Fifth, the brick construction protocols and computing frameworks, because these are the engineering foundations that would make brick-based LLMs practical—without efficient sparse operators and universal protocols, the framework remains a conceptual exercise.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a survey and framework synthesis paper whose core contribution is a taxonomy and operational framework for understanding modularity in LLMs. The paper does not propose a single trained system or benchmark a method against baselines; instead, it defines concepts, categorizes existing work within those concepts, defines primitive operations that abstract over existing techniques, and validates key assumptions empirically.

The Brick as the Fundamental Abstraction

The paper introduces the term brick to replace the overloaded word "module," which in different contexts can mean a Transformer layer, an attention head, an adapter, an entire pre-trained model, or a software component in a multi-agent system. A brick is defined formally as:

"a functional unit composed of a group of function-specific neurons. Its size can vary from a single neuron to even an entire model." (Table 1)

This definition has three essential properties that any candidate brick must satisfy:

Independence: a brick must be capable of performing its function without relying on the internal state of other bricks, though it may take inputs from them and produce outputs for them. This means bricks must be decoupled—their internal parameters should not be entangled with those of other bricks in ways that violate the ability to activate, update, or replace them independently.
Specificity: a brick must have a well-defined, interpretable function. It is not merely an arbitrary subset of parameters; it must be possible to say what capability or knowledge that subset encodes. The paper's empirical analysis operationalizes this through the funcScore metric, which measures how strongly a neuron's activation correlates with a specific capability requirement.
Composability: bricks must be able to coordinate with other bricks through well-defined interfaces. For emergent bricks within a single model, this interface is typically continuous hidden vectors (the standard Transformer residual stream). For customized bricks inserted into a model, the interface may be continuous (hidden state injection) or discrete (text output consumed by downstream bricks). The key is that the interface must be stable enough that bricks can be combined without retraining the entire system.

The importance of this definition is that it unifies phenomena previously studied in isolation. A knowledge neuron (Dai et al., 2022), a LoRA adapter (Hu et al., 2022), a mixture-of-experts expert (Fedus et al., 2022b), and an entire vision encoder (Alayrac et al., 2022) are all bricks under this definition—they differ in granularity, in how they were constructed, and in what operation they support, but they share the abstract properties of independence, specificity, and composability.

Emergent Bricks: Functional Specialization That Arises During Pre-Training

The paper distinguishes two categories of bricks based on their origin. Emergent bricks are parameter subsets that differentiate into specialized functions during the pre-training process, without explicit human design of those functions. The paper identifies two sub-types based on how the brick boundaries are defined.

Human-Defined Emergent Bricks

These are bricks whose architectural boundaries are specified by humans before training, but whose functional specialization is acquired through training. The most obvious example is the standard Transformer layer structure: each model consists of stacked blocks, each containing multi-head attention and feed-forward networks. These are human-defined architectural units. What makes them "emergent bricks" is that during training, different layers and attention heads develop specialized roles without being explicitly programmed to do so.

The paper cites several lines of evidence. Fan et al. (2020) showed that it is possible to drop entire layers during inference without substantially degrading performance, implying that not all layers are equally necessary for all tasks—some layers are specialized for capabilities that aren't needed for every input. Michel et al. (2019) demonstrated that for specific tasks, using only a single attention head can achieve performance comparable to the full multi-head model, suggesting that heads specialize in different aspects of processing. At the finest granularity, Zuo et al. (2022b) showed that specific neurons within FFN layers can be identified as important for particular tasks, enabling sub-network extraction that preserves task performance while reducing computation.

The mixture-of-experts (MoE) architecture represents the most explicit form of human-defined emergent bricks. In an MoE Transformer (Fedus et al., 2022b; Lepikhin et al., 2021), the standard FFN layer is replaced with multiple parallel expert sub-networks, and a learned gating function routes each token to the top-k experts. The experts are architecturally identical (same dimension, same structure) and their boundaries are pre-defined—the human decides how many experts exist and what shape they take. However, what each expert learns to do is emergent: after training, different experts specialize in different types of tokens, syntactic patterns, or semantic domains. Shen et al. (2023a) found that MoE experts develop specialized capabilities correlated with task types, and Zhang et al. (2023c) demonstrated that MoE experts are functionally specialized for knowledge storage, task skills, and semantic understanding.

The paper notes two key limitations of human-defined emergent bricks. First, when models are trained end-to-end, the functionality of each brick is hard to interpret and localize—we know that layer 18 does something different from layer 6, but specifying exactly what each layer does in functional terms is difficult. This makes it hard to reliably route inputs to the right bricks or to update specific capabilities by modifying specific bricks. Second, modular training of human-defined bricks requires delicate design—choosing the number of experts, their size, and the routing mechanism is a complex hyperparameter optimization problem that must be solved before training begins.

Self-Organized Emergent Bricks

These are brick structures that do not correspond to any human-designed architectural boundary. They emerge from the interaction between human-defined components during training, forming functional groupings that cross layer boundaries or subdivide within layers.

The paper's primary example, drawing on Zhang et al. (2022c) and Zhang et al. (2023c), is the observation that within FFN layers, neurons form functional clusters that activate together. Specifically, when examining the activation patterns of FFN neurons across many inputs, certain subsets of neurons consistently co-activate while others remain inactive for those inputs. These co-activating subsets constitute self-organized emergent bricks—they were not designed as explicit experts or sub-networks, but they function as coordinated units.

Zhang et al. (2023c) demonstrated that these self-organized neuron clusters exhibit functional specialization: perturbing the weights of neurons in one cluster degrades performance on specific capability categories (e.g., semantic understanding) while leaving other capabilities intact. This is the key property that qualifies them as bricks under the paper's definition—they are independent (can be manipulated without destroying other capabilities), specific (associated with identifiable functions), and, by virtue of being part of the same FFN layers and residual stream, composable with the rest of the model.

The paper identifies activation sparsity as the mechanism that reveals and enables self-organized bricks. The observation is that for any given input, only a small fraction of FFN neurons produce non-zero outputs (for ReLU-based models) or outputs with magnitude significantly above zero (for non-ReLU models). Zhang et al. (2022c) reported that as few as 5% of neurons are active for 90% of inputs in fine-tuned T5-Large, and Li et al. (2023c) showed this phenomenon is ubiquitous across model types, datasets, and layers. This means the model's effective computation for a given input involves a much smaller parameter set than its total size—which is exactly what a brick-based architecture would prescribe.

The paper emphasizes three open challenges for self-organized bricks. First, cross-layer organization: current studies examine neuron clusters within individual FFN layers, but specialization likely spans layers—a capability like mathematical reasoning probably involves coordinated neurons across multiple Transformer blocks. Understanding these cross-layer clusters is harder but necessary for complete decomposition. Second, training strategies: while modular structure emerges spontaneously in standard end-to-end training, Mittal et al. (2022) showed that models struggle to learn truly modular data distributions without explicit architectural biases. Enhanced training strategies that encourage modularity could yield cleaner self-organized bricks. Third, network design guidance: understanding which brick organizations emerge naturally could inform the design of future architectures—for instance, if we know that knowledge and skill neurons tend to cluster in middle and late layers respectively, we could design architectures that accommodate this specialization.

Customized Bricks: Post-Hoc Construction for Capability Injection

While emergent bricks arise from the pre-training process, customized bricks are deliberately constructed after pre-training to inject new capabilities that the base model lacks. The paper uses the term "plugins" interchangeably with customized bricks, emphasizing their plug-and-play nature.

The Theoretical Basis: Intrinsic Dimensionality

The paper grounds the feasibility of customized bricks in the observation that LLM fine-tuning has low intrinsic dimensionality. The intrinsic dimension of an optimization problem is defined as the minimal number of free variables required to adequately describe the solution space. Li et al. (2018) proposed estimating intrinsic dimension by randomly projecting model parameters into a low-dimensional subspace and checking whether that subspace contains a good solution to the training objective—if it does, the subspace dimension is an upper bound on the intrinsic dimension.

Aghajanyan et al. (2021) applied this method to pre-trained language model fine-tuning and found that the intrinsic dimension is remarkably low—for RoBERTa, fine-tuning to many downstream tasks requires only approximately 200 effective degrees of freedom, despite the model having hundreds of millions of parameters. Moreover, larger models tend to have lower intrinsic dimensions, meaning the more parameters a model has, the more over-parameterized it is for any specific task. Qin et al. (2021) further showed that many different tasks can be reparameterized into a shared universal low-dimensional subspace, which explains why the same foundation model can be efficiently adapted to diverse tasks and why parameter-efficient methods transfer well across tasks.

The practical implication is that adding new capabilities to an LLM should require only a tiny number of additional parameters, because the base model already contains most of the representational capacity needed, and the adaptation merely reconfigures or augments a low-dimensional subset. This is exactly what customized bricks exploit: they are small parameter sets (typically a few hundred to a few hundred thousand parameters) that, when inserted into or alongside the frozen base model, redirect its computation to perform new tasks or incorporate new knowledge.

Task Bricks

Task bricks are the most thoroughly studied category of customized bricks, corresponding to the parameter-efficient fine-tuning (PEFT) literature. The paper classifies them into three structural types plus a training-free variant.

Addition-based bricks introduce new parameters into the model. Adapter tuning (Houlsby et al., 2019) inserts small bottleneck layers (down-projection + nonlinearity + up-projection) after attention or FFN sub-layers; only these adapter parameters are trained while the base model is frozen. Prompt tuning (Lester et al., 2021) prepends trainable continuous embeddings to the input sequence, which the model processes alongside the actual input tokens; these soft prompts effectively condition the model's behavior without modifying any existing weights. Prefix tuning (Li & Liang, 2021) extends this by inserting trainable activations into all Transformer layers, not just the input.

Specification-based bricks do not add parameters but instead designate a subset of existing parameters as trainable while freezing the rest. BitFit (Zaken et al., 2022) demonstrated that tuning only the bias vectors in all linear layers can achieve competitive performance on many tasks—this means the capability to adapt to a new task can be encoded in just the biases, which represent a tiny fraction of total parameters. Masking approaches (Guo et al., 2021; Zhao et al., 2020) learn a binary mask over parameters, determining which existing weights should be updated; the mask itself is the brick, and the weight changes are the adaptation.

Reparameterization-based bricks rewrite the computation of existing layers into a parameter-efficient form. LoRA (Hu et al., 2022), the most prominent example, models the weight update $\Delta W$ for a pre-trained weight matrix $W \in \mathbb{R}^{d \times k}$ as the product of two low-rank matrices:

$\Delta W = BA$

where $B \in \mathbb{R}^{d \times r}$ and $A \in \mathbb{R}^{r \times k}$ with rank $r \ll \min(d, k)$ .

What it computes: the original layer computes $h = Wx$ . The LoRA-augmented layer computes $h = Wx + BAx = Wx + B(Ax)$ , where $Ax$ is a low-dimensional projection (from $k$ to $r$ dimensions) followed by $B(Ax)$ projecting back to $d$ dimensions. The matrices $A$ and $B$ together contain $r(d + k)$ parameters, compared to $dk$ for the original weight matrix—when $r$ is small (typically 4–64), this represents a parameter reduction of orders of magnitude.

Why this form: the low-rank assumption is justified by the intrinsic dimensionality observations: if fine-tuning requires only a few hundred effective degrees of freedom, the weight update must lie in a low-dimensional subspace of the full parameter space. The rank- $r$ decomposition explicitly enforces this constraint. Moreover, the factorization $BA$ allows the update to be merged with the original weights at inference time ( $W_{\text{merged}} = W + BA$ ), so the adapted model incurs no additional inference cost—the brick becomes transparent once applied. Alternatives like full fine-tuning violate the low-rank assumption (waste parameters) and cannot be merged back as easily.

Training-free task bricks represent a different approach entirely: rather than training new parameters, these methods discover task-specific patterns in the model's existing representation space that can be steered to produce desired behaviors. The key observation, from work by Zou et al. (2023) and Liu et al. (2023c), is that the intermediate representations in LLMs possess semantically meaningful structure—for instance, there exists a direction in activation space that corresponds to "truthfulness" or "refusal to answer harmful queries." By extracting the difference between activations for desired and undesired outputs, one can construct a steering vector that, when added to the model's hidden states, biases generation toward the desired behavior. This is a brick in the representational rather than parametric sense: it is a pattern imposed on the model's computation that changes its output without changing its weights.

Knowledge Bricks

Knowledge bricks address a fundamental limitation of LLMs: their knowledge is parametric—stored implicitly in the weights learned during pre-training—and therefore static, capacity-limited, and difficult to update. The paper distinguishes structured knowledge graph bricks and unstructured text bricks.

Structured KG bricks encode factual knowledge from knowledge graphs (KGs) into neural representations that can be injected into LLMs. The core challenge is bridging the representation gap between discrete symbolic knowledge (entity-relation-entity triples like "Beijing-is_capital_of-China") and continuous neural representations. The paper describes two main approaches. One line of work (Ye et al., 2022) computes contextualized entity representations by averaging the output vectors of masked entity mentions across a corpus—these representations capture how the entity is used in natural language contexts. Another approach (Zhang et al., 2023b; Pörner et al., 2020) trains a neural projection (typically an MLP) that maps pre-trained KG embeddings (from models like TransE; Bordes et al., 2013) into the token embedding space of the LLM, so that entity knowledge can be prepended to input sequences as additional "knowledge tokens."

Unstructured text bricks encode knowledge from documents into reusable representations. Traditional retrieval-augmented generation (RAG; Lewis et al., 2020) retrieves relevant documents and concatenates them with the input, which means the same document is re-encoded from scratch every time it is retrieved—wasteful when the document is used across many queries. The brick approach, as in Xiao et al. (2023b), pre-encodes documents into compact representations (either as prefix token embeddings or intermediate hidden states) that can be stored and retrieved with minimal computation. The paper describes this as making document knowledge "task-agnostic" and "plug-and-play"—the same document brick can serve queries about that document across different tasks without re-encoding.

Modality Bricks

Modality bricks extend LLMs to process non-text inputs (images, audio, video) by treating pre-trained modality-specific models as bricks that interface with the LLM through a learned or heuristic conversion layer.

Bricks with textual interface convert non-text modalities into natural language descriptions that the LLM can process directly. For example, a video understanding system might use a pre-trained captioning model to convert video frames into textual descriptions, which are then fed into the LLM alongside the user's question. The advantage is zero additional training—the modality models and the LLM are used as-is, communicating through human-readable text. The disadvantage, which the paper explicitly notes, is information loss: fine-grained visual details (spatial relationships, subtle expressions, precise quantities) are difficult to capture in text descriptions, and errors in the captioning or detection models propagate to the LLM's reasoning.

Bricks with continuous interface address the information loss problem by training a learnable connector between the modality encoder and the LLM. The connector typically takes the form of a small neural network (often just an MLP) that maps the modality encoder's output representations into the LLM's input embedding space, similar to how knowledge bricks project KG embeddings into token space. Flamingo (Alayrac et al., 2022) is the canonical example: a frozen vision encoder produces image features, which are processed by a trainable perceiver resampler (using learned query vectors and cross-attention) to produce a fixed number of visual tokens, which are then interleaved with text tokens in the LLM's input. BLIP-2 (Li et al., 2023a) uses a simpler Q-Former architecture that achieves similar bridging with fewer parameters. The paper notes that the continuous interface approach typically requires multi-modal aligned training data (image-caption pairs, video-description pairs) to learn the connector, and that the limited capacity of the connector (often just a few million parameters) can still lead to fine-grained information loss for complex visual inputs.

The paper also mentions a variety of other customized brick types that do not fit neatly into the task/knowledge/modality trichotomy: bricks for tool use that encode how to call external APIs (Shi et al., 2023), debiasing bricks that modify model outputs to reduce harmful stereotypes (Dathathri et al., 2020), compression bricks that reduce inference cost by shortening internal representations (Xiao et al., 2023a), and style transfer bricks that change the linguistic register of generated text (Pascual et al., 2021). The unifying principle is that all of these are small, specialized parameter sets that augment a frozen base model—they are customized bricks by the paper's definition.

Brick Granularity as a Fundamental Design Dimension

The paper devotes substantial attention to a structural question: how large should a brick be? This is not merely an implementation detail—it determines what capabilities a brick can encode, how efficiently it can be stored and activated, and how complex the management operations become. The paper analyzes four granularity levels.

Solitary Neuron Granularity

At the finest level, individual neurons (rows/columns in weight matrices) can function as bricks. The paper cites evidence that single neurons can encode surprisingly specific concepts. Dai et al. (2022) found that factual knowledge triples are stored in identifiable "knowledge neurons"—the activation of a specific neuron in a specific FFN layer is highly correlated with the model's ability to recall a specific fact, and manipulating that neuron's weights can edit or erase the fact. Wang et al. (2022a) identified "skill neurons" that are predictive of task performance—certain neurons consistently activate when the model performs sentiment analysis, others when it does natural language inference. Mu & Andreas (2020) found visual model neurons that respond selectively to specific perceptual concepts like "tall structures" or "checkerboard patterns."

However, the paper also notes a complication: polysemous neurons. The same neuron can be involved in encoding multiple different concepts or facts (Xin et al., 2019; Suau et al., 2020). Dai et al. (2022) found that knowledge neurons responsible for different factual triples have overlapping sets, meaning a single neuron may participate in storing multiple facts. This breaks the "specificity" property of bricks: if a neuron encodes both the capital of France and the author of Hamlet, updating one fact by modifying that neuron risks corrupting the other. This suggests that solitary neurons may be too fine-grained for reliable brick operations—they don't provide the functional isolation that the brick abstraction promises.

Neuron Group Granularity

Groups of neurons—whether self-organized clusters within existing layers or explicitly constructed sub-networks—represent a more practical granularity for most operations. The paper identifies two dominant forms.

Mixture-of-Experts as pre-defined neuron groups. Each MoE expert is essentially a small FFN (typically identical in structure to the original FFN but shared across tokens by routing). These experts are the human-defined equivalent of self-organized neuron clusters, and they exhibit analogous functional specialization. Zhang et al. (2023c) showed that MoE experts specialize in different functional categories—some experts are knowledge-focused, others skill-focused, others focused on basic semantic processing. Chen et al. (2022) and Shen et al. (2023a) provided evidence for expert specialization through activation analysis and ablation studies. The key advantage of the neuron-group granularity over solitary neurons is capacity: a group of neurons can encode more complex capabilities and maintain better isolation from other groups.

Parameter-efficient tuning modules as constructed neuron groups. Adapter layers (Houlsby et al., 2019), LoRA decompositions (Hu et al., 2022), and prefix embeddings (Li & Liang, 2021) all operate at this granularity. A LoRA module applied to a specific weight matrix typically contains a few hundred to a few hundred thousand parameters—a small fraction of the base model but far more than a single neuron. The paper notes that PET modules have been demonstrated to work effectively across over 100 NLP tasks (Ding et al., 2023), from simple classification to complex conditional generation, establishing that this granularity is sufficient for encoding meaningful task capabilities while remaining efficient to train and store.

The paper also describes an important technique: splitting existing dense FFNs into pseudo-experts without additional parameters. Zhang et al. (2022c) showed that by analyzing co-activation patterns, the neurons within a standard FFN can be partitioned into functionally coherent groups that operate as implicit experts. This transforms a dense model into a sparse-activated modular one without any retraining or parameter addition—the brick structure was latent in the trained weights and is simply made explicit through clustering.

Layer Granularity

Individual Transformer layers can function as bricks. The paper reviews evidence that different layers serve different functions in the processing pipeline. Lin et al. (2019) found that word order information is concentrated in lower layers, while Hewitt & Manning (2019) showed syntactic structure is most prominent in middle layers through their structural probing framework. Liu et al. (2019) observed that final layers are more task-specific. Geva et al. (2021) demonstrated that FFN layers can be interpreted as key-value memories, where each layer stores a different category of factual associations.

This functional stratification enables layer-level operations. Early exiting approaches (Xin et al., 2020; Han et al., 2021c) treat each layer as a brick that can be conditionally skipped: for easy inputs, computation can stop at an intermediate layer (the model "exits early"), saving the cost of upper layers. Layer dropping during training (Fan et al., 2020; Zhang & He, 2020) randomly disables layers to improve training efficiency and robustness. Knowledge editing methods (Meng et al., 2022; Huang et al., 2023b) identify specific middle-layer FFNs as the primary locus of factual storage and target those layers for weight modifications.

The limitation of layer granularity is that layers are coarse and architecturally rigid. A layer encodes many different capabilities simultaneously—layer 18 might handle aspects of syntactic processing, semantic disambiguation, and factual retrieval all at once. Modifying that layer for one purpose (e.g., updating a fact) can have unpredictable effects on its other functions. The paper implies that finer granularities (neuron groups) offer better functional isolation.

Full Model Granularity

At the coarsest level, entire pre-trained models can be treated as bricks in a multi-model system. This is the paradigm behind multi-agent frameworks (Wang et al., 2023a; Qian et al., 2023) where different LLMs or specialized models play different roles (a planner, a coder, a reviewer). It is also how multi-modal systems like Flamingo (Alayrac et al., 2022) operate: the vision model is one brick, the language model is another, and a connector bridges them.

The paper notes that even within a single pre-trained model, sub-networks can be extracted as model-level bricks. Xu et al. (2021) identified "child networks" within larger models that can be fine-tuned independently for downstream tasks. S4-Tuning (Xu et al., 2022) partitions a pre-trained model into language-specific sub-networks, updating only the relevant sub-network for each target language. Zhang et al. (2021) found that even in biased models, there exist unbiased sub-networks that generalize better out-of-distribution.

The tradeoff at this granularity is clear: full model bricks have the highest capacity (they can handle complex, open-ended tasks) but incur the highest computational cost and are the least flexible for fine-grained operations. The paper's framework accommodates this as one point on a spectrum.

Granularity Selection Principles

The paper offers several considerations for choosing brick granularity, though acknowledges that systematic principles remain an open problem:

Capability complexity: complex capabilities require larger bricks. A solitary neuron cannot encode how to translate between languages; a full model can. The scaling laws literature (Kaplan et al., 2020) may provide guidance on the relationship between parameter count and capability complexity.
Management overhead: finer granularities mean more bricks to manage. With millions of neurons, routing and retrieval become combinatorially harder than with dozens of layers or a handful of experts.
Inclusivity relationships: coarser bricks can be decomposed into finer ones, and capabilities can be organized hierarchically—general language understanding decomposes into specific NLP tasks, which further decompose into sub-skills. The paper suggests that future systems might use hierarchical brick organizations where operations can target the appropriate granularity level.
Emergence vs. design: the paper acknowledges a tension—while humans can design brick hierarchies, functional bricks also emerge in ways that may not respect those hierarchies. The optimal granularity may be one that the training process naturally produces rather than one that is manually specified.

Routing and Retrieval: Selecting the Right Bricks for Each Input

This operation addresses the question: given an instruction and a repository of bricks, which bricks should participate in computation? The paper distinguishes routing for emergent bricks (typically within a single model, using a small fixed set of bricks) from retrieval for customized bricks (typically from a large external repository).

Emergent Brick Routing

For emergent bricks, the number of candidates is usually limited—perhaps a few dozen experts in an MoE model or a few hundred self-organized neuron clusters. The selection is therefore implemented as a routing function that assigns scores to each brick and activates the top-k.

Trainable routing is the dominant approach in MoE architectures. The routing function $g(x)$ for a given token representation $x$ computes:

$g(x) = \text{softmax}(W_g x)$

where $W_g \in \mathbb{R}^{E \times d}$ is a learned weight matrix mapping from the $d$ -dimensional token representation to $E$ expert scores, and the softmax converts these to a probability distribution over experts. For top-k routing (typically k=1 or k=2), only the experts with the highest probabilities receive the token.

What it computes: for each token at each MoE layer, the router computes a score for each expert, normalizes these scores into probabilities via softmax, selects the k experts with the highest probabilities, and routes the token to those experts. The expert processes the token and produces an output; if multiple experts are selected (k > 1), their outputs are weighted by the router probabilities and summed.

Why this form: the softmax over a linear projection is the simplest learnable routing mechanism that is differentiable (enabling end-to-end training) and produces a valid probability distribution. However, the paper notes a well-known problem: load imbalance, where the router consistently assigns more tokens to some experts than others, leading to inefficient hardware utilization. The paper describes several solutions from the literature: Lewis et al. (2021) reformulate routing as a linear assignment problem with balanced constraints; Zhou et al. (2022a) invert the routing direction so that experts select tokens rather than tokens selecting experts, enabling explicit control over expert capacity; Puigcerver et al. (2023) introduce "soft slots" that aggregate information across tokens before expert processing, avoiding the token-level routing granularity entirely.

Fixed routing avoids the trainability and imbalance issues by using deterministic, non-learned routing functions. Hash-based routing (Roller et al., 2021) assigns tokens to experts based on a hash of the token ID, ensuring perfect balance but ignoring semantic relevance. Random routing (Zuo et al., 2022a) similarly balances load but provides no functional specialization pressure—experts learn to handle whatever they receive, potentially becoming generalists rather than specialists. Domain-based routing (Gururangan et al., 2022) uses the known domain of the input (e.g., "legal," "medical," "scientific") to route tokens to domain-specific experts, combining balance with semantic relevance but requiring domain labels.

The paper notes that explaining routing behavior remains challenging: even when routing functions exhibit meaningful patterns after training (Zoph et al., 2022), fully interpreting why a particular expert was selected for a particular token is an open problem.

Customized Brick Retrieval

For customized bricks, the scale is different: there could be thousands or millions of knowledge bricks (one per entity, one per document), making exhaustive scoring infeasible. The paper focuses on retrieval for knowledge bricks, where the goal is to find the specific factual knowledge relevant to the current input.

Entity-based retrieval is the most straightforward approach: identify entities mentioned in the input (via entity linking), then retrieve the corresponding entity bricks from the knowledge repository. Zhang et al. (2023b) and Févry et al. (2020) use this approach for structured KG bricks—the input is parsed for entity mentions, those mentions are linked to KG entities, and the corresponding entity embeddings are injected into the model. The retrieval mechanism is an entity linker, not a learned neural retriever, because the mapping from text spans to knowledge entries is explicit.

Dense retrieval is used when knowledge bricks encode content that isn't neatly organized by entity. Cheng et al. (2023) encode Wikipedia documents into knowledge bricks and use Maximum Inner Product Search (MIPS) to retrieve relevant bricks. Specifically, the query (the input text or a representation thereof) is embedded into the same vector space as the document bricks, and the bricks with the highest inner product are selected. This is the standard dense retrieval paradigm applied to neural brick representations rather than text passages.

Task brick retrieval is an emerging area as the number of task-specific bricks grows. Zhao et al. (2024b) proposed a retrieve-then-compose framework for LoRA modules: given a new task, retrieve several LoRA modules that were trained on similar tasks (based on task description similarity or embedding similarity), then average their parameters to produce a composite brick for the new task. This treats task bricks as a library that can be recomposed rather than requiring a new brick to be trained from scratch for each task.

Routing and Retrieval Granularity

The paper identifies three levels at which routing/retrieval decisions can be made:

Token-level: the finest granularity, used by most MoE architectures. Each token in the sequence is independently routed to experts. This provides maximum flexibility—a single sentence might route different words to syntax experts, semantic experts, and knowledge experts—but incurs routing overhead at every token and every layer.
Sentence-level: routing decisions are made once per input. Gururangan et al. (2022) route all tokens in an input to the same domain expert based on the sentence's domain label. This is more efficient but less flexible: if a sentence contains mixed-domain content, the routing may be suboptimal. Sentence-level retrieval is also used for knowledge bricks (Cheng et al., 2023), where a single query representation retrieves knowledge relevant to the entire input.
Task-level: routing happens before any computation on the specific input. Pfeiffer et al. (2021) and Huang et al. (2023a) select task bricks based on the task identity (determined from a held-out validation set or a task description), then apply those bricks to all instances of that task. This is the most coarse but also most efficient granularity.

The paper advocates for multi-level routing and retrieval that combines signals from multiple granularities. Chen et al. (2022) made preliminary attempts by incorporating both task-level and token-level information into expert routing, but the paper notes this as an area requiring substantial future work.

The paper also identifies active routing and retrieval as an important open direction. Current methods are passive: routing decisions are made at fixed positions (every token, or at entity mentions) based on pre-determined rules. Active methods would allow the model to decide when to retrieve, potentially generating a special "retrieval needed" token when it encounters information it lacks, then querying the brick repository. Zhang et al. (2022d) made a preliminary step in this direction with dynamic entity memory augmentation triggered by special tokens, but the paper frames this as a largely unexplored capability that would significantly improve efficiency by reducing unnecessary retrievals.

Combination: Fusing Multiple Bricks for Composite Capabilities

Single-function bricks are rarely sufficient for real-world instructions, which typically require a combination of capabilities—a coding task might require knowledge of a specific API plus general programming skill plus an understanding of the user's natural language description. The combination operation addresses how multiple bricks interact within a single computation.

Parameter Weighted Averaging (Homogeneous Combination)

When bricks share the same architecture (identical parameter shapes and semantic roles), their parameters can be linearly combined through weighted averaging. This is the simplest combination method and applies primarily to bricks that were fine-tuned from the same base model.

The paper traces the theoretical justification to mode connectivity (Garipov et al., 2018; Draxler et al., 2018), which discovered that independently trained neural networks often have solutions that are connected by low-loss paths in parameter space. Frankle et al. (2020) showed that if two models are fine-tuned from the same pre-trained initialization, simple linear interpolation between their parameters produces a model that remains on the low-loss manifold, meaning the interpolation is valid. This enables the core operation:

$\theta_{\text{merged}} = \sum_{i=1}^{n} w_i \theta_i$

where $\theta_i$ are the parameters of the i-th brick, $w_i$ are scalar weights (usually summing to 1), and $\theta_{\text{merged}}$ is the resulting combined brick.

What it computes: element-wise weighted average of corresponding parameters across all bricks. If two LoRA modules adapted the same weight matrix for different tasks—say, one for summarization and one for translation—and we want a model that can do both, we can average their $A$ and $B$ matrices with appropriate weights. The resulting merged LoRA module approximates the joint capability.

Why this form: linear interpolation is the simplest operation that preserves the parameter structure and can be computed without any additional training. However, the effectiveness depends critically on the weighting coefficients $w_i$ . Uniform weights $w_i = 1/n$ work when the bricks contribute equally, but in general, some bricks are more relevant to the target capability than others.

The paper describes several approaches for determining weights. Wortsman et al. (2022) showed that simple uniform averaging of models fine-tuned with different hyperparameters produces an ensemble-level performance without ensemble-level inference cost—a phenomenon they termed "model soups." Matena & Raffel (2022) used Fisher information to weight parameters by their importance for each task, giving higher weight to parameters that are critical for a task and lower weight to those that are flexible. Huang et al. (2023a) used combinatorial optimization to find weights that minimize the number of training examples needed for the target task (for few-shot adaptation). Jin et al. (2023) determined weights by minimizing the L2 distance between the merged parameters and the source brick parameters, encouraging the merged brick to stay close to all source bricks in parameter space.

The paper also notes subtractive combination. Ilharco et al. (2023) demonstrated that undesired capabilities can be removed by subtracting a brick from the base model: if a brick was trained to produce toxic outputs, subtracting that brick's parameters (with an appropriate weight) from the base model reduces toxicity while preserving other capabilities. Zhang et al. (2023a) applied this to detoxification, and Daheim et al. (2023) to hallucination reduction.

Brick Stitching (Heterogeneous Combination)

When bricks have different architectures or process different modalities, parameter averaging is impossible—there are no corresponding parameters to average. Brick stitching instead concatenates bricks in sequence, with the output of one brick serving as the input to the next.

The critical challenge is the interface between bricks. The paper distinguishes two interface types:

Textual interfaces use human-readable text as the communication medium. A vision brick (e.g., a captioning model) produces a textual description of an image; the language model brick reads that text alongside the user's question. The advantage is zero interface training—any model that can read and generate text can participate. The disadvantage is information loss: converting visual information to text discards spatial details, relative positions, and fine-grained visual features that the language model would need for precise reasoning.
Continuous interfaces use hidden vectors as the communication medium. A connector brick (typically a small neural network) is trained to map the output representations of one brick into the input representation space of the next. The paper describes attention-based connectors (Alayrac et al., 2022; Li et al., 2023a) that use learnable query vectors to extract relevant information from visual features, and simpler MLP-based connectors (Liu et al., 2023a; Zhu et al., 2023) that directly project visual features into the LLM's token embedding space. The advantage is richer information transfer; the disadvantage is the need for multi-modal training data and the limited generalization of the connector to new brick combinations.

The paper distinguishes two approaches for determining the stitching topology—the order and structure of brick connections:

Heuristic stitching uses manually defined execution sequences based on task decomposition. For visual question answering, the standard heuristic is: vision encoder → connector → language model, because the task requires understanding the image before answering the question. For multi-agent systems, the heuristic might be: planner agent → executor agent → reviewer agent, reflecting the natural workflow of plan-execute-verify. This approach works well when the task structure is known and fixed, but becomes brittle when instructions require varied inference sequences.

Planner-based stitching introduces a meta-component—the task planner—that dynamically decomposes an instruction into sub-tasks and determines which bricks to invoke in which order. The paper describes two variants. Pre-execution planning (Hsieh et al., 2023; Shen et al., 2023c) has the planner generate the complete execution sequence before any bricks execute, using functional descriptions and usage demonstrations of each brick. Dynamic planning (Yao et al., 2023a; Gao et al., 2023) interleaves planning and execution: after each brick produces its output, the planner decides which brick to invoke next based on the intermediate result. This allows the system to adapt its plan when early steps reveal new information. The paper also notes that search-enhanced planning (Ye et al., 2023; Qin et al., 2023) can improve robustness by exploring multiple possible brick sequences and selecting the one that leads to the best final output.

The paper identifies two major open challenges for combination. First, combining fine-grained heterogeneous bricks at sub-model granularity (e.g., merging a visual concept neuron with a linguistic concept neuron) could reduce parameter redundancy—both language and vision models likely encode overlapping real-world concepts—but the interface alignment problem is harder at fine granularities. Second, universal brick interaction interfaces that work across diverse brick types would dramatically improve scalability, enabling any brick that implements the standard interface to seamlessly stitch with others, analogous to how USB standardizes hardware component connection.

Updating: Modifying Bricks for Knowledge and Capability Evolution

The updating operation addresses the problem that knowledge and requirements change over time—a fact becomes outdated, a task specification changes, a capability needs refinement. The paper frames this as a brick-centric alternative to full model retraining: instead of updating all parameters (expensive and risk-prone), identify and update only the bricks responsible for the target knowledge or capability.

Locating and Updating Emergent Knowledge Bricks

This approach assumes that the knowledge to be updated is already stored in an identifiable emergent brick and that the brick can be surgically modified.

Locating knowledge bricks relies on the observation that FFN layers function as key-value memories (Geva et al., 2021). In this interpretation, the first linear layer of the FFN acts as keys—each neuron detects certain input patterns—and the second linear layer acts as values—each neuron contributes to the output vocabulary distribution when activated. A fact like "the Eiffel Tower is in Paris" is stored as a pattern of FFN weights that, when the input mentions "Eiffel Tower" and "located in," activates neurons whose output weights promote the token "Paris."

Dai et al. (2022) used integrated gradients (Sundararajan et al., 2017), an attribution method, to identify which neurons have the highest gradient with respect to a factual prediction, and found that these "knowledge neurons" are positively correlated with fact recall. Meng et al. (2022) strengthened this with causal intervention: they systematically ablated neuron activations and measured the impact on factual predictions, finding that middle-layer FFNs are most responsible for fact recall. However, the paper notes a critical finding from Hase et al. (2023): while causal intervention identifies which neurons carry the target knowledge (are causally implicated in the prediction), manipulating the parameters of those neurons does not necessarily lead to better editing performance. This suggests that knowledge is distributed across multiple neurons and that localization is not equivalent to editability—a nuance the paper flags as important for future work.

Updating knowledge bricks can be done through weight modification or activation modification. Weight modification approaches directly change the FFN parameters to encode the new fact. The simplest method (Zhu et al., 2020) applies an L2-regularized update that minimizes the prediction error for the new fact while constraining the parameter change to be small. Meng et al. (2022) introduced a more sophisticated approach: they treat the FFN as a linear associative memory and compute a rank-one update that maps the key representation of the subject (e.g., "Eiffel Tower") to the value representation of the target (e.g., "Rome" if the location changed) while minimizing interference with other stored facts. The update formula can be written as:

$\Delta W = \lambda \cdot (v_* - v) \cdot k^T / (k^T k)$

where $k$ is the key vector for the subject, $v$ is the current value vector (producing the old answer), $v_*$ is the desired value vector (producing the new answer), and $\lambda$ controls the update magnitude.

What it computes: a rank-one matrix is added to the FFN output weight matrix. This additive term is the outer product of the residual (difference between desired and current value vectors) and the normalized key vector. When the subject's key representation $k$ is present in the input, this update shifts the FFN output toward $v_*$ , changing the predicted answer. When other inputs are processed, the update has minimal effect because their key representations are orthogonal or near-orthogonal to $k$ .

Why this form: the rank-one update is the minimal modification that achieves the desired key-value mapping—it changes the value for exactly one key direction (in the linear approximation) and leaves the output unchanged for orthogonal directions. This minimizes interference with other stored facts, a property the paper identifies as critical for reliable knowledge editing. MEMIT (Meng et al., 2023) extended this to mass-editing by applying multiple simultaneous updates with a closed-form solution that jointly minimizes interference.

The paper also describes alternative approaches. Onoe et al. (2023) and Padmanabhan et al. (2023) added a KL divergence regularization term that penalizes changes in the model's predictions on unrelated inputs, rather than penalizing parameter changes directly—this better captures the goal of preserving function rather than preserving weights. Mitchell et al. (2022a) and Cao et al. (2021) trained hyper-networks that, given the gradient information for a knowledge update, predict the optimal parameter modification, learning the editing policy from many editing examples.

Injecting New Customized Bricks

An alternative to surgically modifying emergent bricks is to construct a new customized brick that overrides or supplements the existing knowledge. This avoids the localization challenge altogether: rather than finding where old knowledge is stored, simply add new knowledge alongside it.

The paper describes several approaches at different granularities. dos Santos et al. (2022) train entity-specific embedding bricks that can be prepended to the hidden states when that entity is mentioned—the new brick encodes the updated entity information without modifying existing parameters. Huang et al. (2023c) build on the knowledge neuron concept and insert a new FFN neuron for each knowledge update, trained to activate when the corresponding fact should be recalled. Mitchell et al. (2022b) maintain an external memory of updated knowledge and re-route queries concerning updated facts to a small model that has been conditioned on this memory, effectively creating a brick that captures only the deltas from the original model.

Representation-based approaches edit knowledge without modifying weights at all. Hernandez et al. (2023) construct an external brick that, when added to the model's hidden representations at the subject position, induces the updated knowledge prediction. Turner et al. (2023) and Zou et al. (2023) use steering vectors—differences between hidden representations for prompts that elicit desired vs. undesired knowledge—that can be added to activations at inference time to redirect model behavior. These are "bricks" in a functional sense (they encode a specific capability—changing a specific fact or behavior) even though they exist as patterns to impose on computation rather than as parameter sets.

The paper acknowledges that current updating methods are largely limited to knowledge bricks. Task and modality bricks are typically retrained from scratch when they need updating because they are small enough that retraining is affordable. However, as LLMs scale and even efficient adaptation methods become expensive, the paper predicts that updating operations will need to be extended to other brick types. It also raises the open problem of locating and correcting undesired behaviors—finding the bricks responsible for generating toxic outputs or falling for jailbreak attacks, and surgically updating them to eliminate these behaviors without damaging legitimate capabilities.

Growing: Expanding the Brick Repository for Continual Learning

The growing operation addresses the need for LLMs to acquire entirely new capabilities and knowledge domains that were not present during original training, while avoiding catastrophic forgetting of existing capabilities. Instead of retraining from scratch on old + new data (prohibitively expensive), growing adds new bricks to the existing repository.

Growing for Pre-Training

When new pre-training data becomes available (e.g., a new scientific domain, a new programming language, or a new language), the model needs to absorb this data without forgetting what it learned from previous data. The paper describes two approaches for growing emergent bricks.

Expanding dense parameters increases the width or depth of the existing model. Width expansion increases the hidden dimension $d$ or FFN intermediate dimension $d_{ff}$ , while depth expansion adds new Transformer layers. The key challenge is initialization: the expanded parameters must be initialized so that the enlarged model initially behaves identically to the original (to avoid catastrophic forgetting of existing capabilities), and then can be trained on new data to acquire new capabilities.

Several initialization strategies are described. Gong et al. (2019) and Gu et al. (2021) initialize the expanded parameters by copying from original parameters (e.g., duplicating layers, repeating weight blocks), then continue training on mixed old and new data. ELLE (Qin et al., 2022b) expands both width and depth and introduces a "recovering warmup" process that fine-tunes the expanded model on old data to restore original performance before training on new data. LiGO (Wang et al., 2023c) learns a linear mapping from original parameters to expanded parameters, providing a more flexible initialization that can recombine existing knowledge in new ways. Wu et al. (2024) take the simplest approach: freeze original parameters entirely, train only the new parameters on new data—this guarantees no forgetting but limits the interaction between old and new knowledge.

The paper notes a fundamental challenge with dense expansion: even with careful initialization, the shared parameters between old and new capabilities can lead to interference. When old and new data are trained jointly, gradient conflicts can degrade performance on both.

Sparse modular expansion addresses this by adding new bricks that are architecturally isolated from existing ones—typically, new MoE experts. Li et al. (2022) proposed "branch-train-merge" for domain expansion: train new experts on new domain data (initialized from existing experts if desired), then add them to the expert pool with the existing experts frozen. Komatsuzaki et al. (2023) similarly "sparsely upcycle" a dense checkpoint into an MoE by replicating FFN layers into multiple experts and training them on domain-specific data. Shen et al. (2023b) and Wang et al. (2024) use similar approaches with dynamic expert addition during continual learning. The key advantage is no forgetting by construction: since original experts are frozen and new tokens are routed to new experts only when relevant, the original capabilities are preserved intact. The limitation is that the total number of experts grows over time, increasing the gating complexity and the memory footprint.

Growing for Post-Training

Post-training growth focuses on expanding customized bricks rather than modifying the base model. This is the dominant paradigm for continual task learning.

The simplest approach is to construct a new task brick for each new task. Mahabadi et al. (2021), Wang et al. (2022c,b), and Razdaibiedina et al. (2023) all follow this pattern: when a new task appears, train a new adapter, LoRA module, or prompt embedding for that task while keeping all existing bricks frozen. To avoid interference during inference, a router (Madotto et al., 2021; Song et al., 2023a) selects the appropriate brick based on the task identity. This approach treats the growing operation as simply adding an entry to the brick library—no existing parameters are modified.

The paper also describes episodic memory bricks for continual learning without growing the model. Isele & Cosgun (2018) and Rolnick et al. (2019) store representative examples from old tasks in a memory buffer; when training on a new task, the model is trained on a mixture of new data and replayed old data, preventing forgetting through rehearsal. The memory buffer functions as a brick in the broader sense—it encodes task knowledge as data rather than as parameters, but serves the same functional purpose of preserving old capabilities while acquiring new ones.

Selection Criteria for Growing Strategy

The paper offers three considerations for choosing a growing strategy:

Task complexity: simple additions (a few new facts, a new entity category) can be handled by small customized bricks (plugins). Massive expansions (a new language, a new modality) may require growing the emergent brick structure (adding experts or expanding dimensions).
Computation budget: plugins are the cheapest (train a few thousand parameters). Sparse MoE expansion is more expensive (train full expert FFNs, typically millions of parameters) but cheaper than full retraining. Dense expansion requires training parameters proportional to the model size.
Application targets: user-oriented customization favors lightweight plugins that can be personalized per user. Scaling up general AI capabilities may require architectural changes to the base model itself.

Empirical Validation Methodology

The paper's Section 4 provides empirical evidence that the modular phenomena it assumes actually manifest in modern LLMs. The methodology has three components.

Neuron and Activation Definitions

The paper formalizes neurons in the FFN layers, which are the focus of the analysis because prior work (Geva et al., 2021) established that FFNs act as key-value memories and are the primary locus of knowledge storage and capability specialization. For a standard FFN:

$\text{FFN}(x) = W_O(\text{FFN}_I(x)) + b_O$

where $W_O \in \mathbb{R}^{d \times d_{ff}}$ and $b_O \in \mathbb{R}^d$ are the output projection, and $\text{FFN}_I(x)$ is the input transformation. For a vanilla (non-gated) FFN, $\text{FFN}_I(x) = \sigma(W_I x + b_I)$ with $W_I \in \mathbb{R}^{d_{ff} \times d}$ . For a gated FFN (used in Llama and Mistral), $\text{FFN}_I(x) = \sigma(W_G x + b_G) \odot (W_I x + b_I)$ where $\sigma$ is typically SiLU (Swish) and $\odot$ is element-wise multiplication.

The paper defines the i-th neuron as consisting of the i-th row of input/gate matrices and the i-th column of the output matrix, and its activation value as the i-th entry of $\text{FFN}_I(x)$ . The decomposition of the FFN output into neuron contributions is:

$\text{FFN}(x) = \sum_{i=1}^{d_{ff}} \text{FFN}_I(x)_i \cdot W_O[:, i] + b_O$

where $\text{FFN}_I(x)_i$ is the scalar activation of neuron $i$ and $W_O[:, i]$ is its output vector (the i-th column of $W_O$ ). If the activation is near zero, the neuron contributes negligibly to the output regardless of its output weights.

The paper also defines output magnitude as an alternative indicator of neuron importance: for neuron $i$ , output magnitude is $||\text{FFN}_I(x)_i \cdot W_O[:, i]||_2$ , which measures the actual vector contribution to the FFN output. This is more directly related to downstream impact than the activation value alone, because a neuron could have low activation but high-magnitude output weights, or vice versa.

Functionality Score

The paper needs a metric to quantify how strongly a neuron is associated with a specific capability. The functionality score is defined using the average precision (AP) metric, which measures how well a neuron's activation values can separate inputs that require a capability from those that do not.

For a collection of chat instances $C = \{(p_0, r_0), ..., (p_n, r_n)\}$ where $p_i$ is the user prompt and $r_i$ is the model response, each instance has a binary label $y_i^f \in \{0, 1\}$ indicating whether capability $f$ is required. For neuron $n$ , the activation value on instance $i$ is the mean of its absolute activation values across all tokens in the prompt: $A_i = \text{mean}(\{|a_0|, ..., |a_{l_i}|\})$ where $l_i$ is the prompt length in tokens.

The functionality score is then:

$\text{FuncScore}(n, f) = \text{AvgPrecision}(\{A_0, ..., A_n\}, \{y_0^f, ..., y_n^f\})$

What it computes: for a given neuron $n$ and capability $f$ , the average precision score treats the neuron's activation values as a ranking score and the capability labels as relevance judgments. If instances requiring capability $f$ consistently have high activation values for neuron $n$ and instances not requiring $f$ have low activation values, the AP score will be close to 1. If there is no correlation, the AP score will be near the random baseline (which is the fraction of positive instances in the dataset).

Why this form: average precision is a rank-based metric that does not require choosing a specific activation threshold—it evaluates the neuron's discriminative power across all possible thresholds. This is appropriate because the paper is interested in whether the neuron's activation carries information about the capability requirement, not in finding a specific decision boundary. The paper uses 1,000 instances for each of 7 functionality types (Table 3), and instances belong exclusively to one type to ensure clean labels.

Sparse Activation Analysis

To test whether LLMs are sparsely activated (i.e., only a subset of neurons is needed for any given input), the paper computes the distribution of (1) normalized activation values and (2) normalized output magnitudes across all neurons, layers, and tokens in the evaluation data. The normalization (presumably to [0,1] range within each layer or across all neurons) enables comparison across layers with different scales.

The paper reports cumulative distribution functions showing that for both Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3, approximately 80% of neurons have normalized indicators below 0.2, confirming a long-tail distribution where most neurons contribute minimally. This pattern is consistent across all layers and both models.

The masking experiment provides causal evidence: the paper progressively masks (sets to zero) neurons with the lowest indicators and measures the resulting loss. The key result is that when up to 70% of neurons are masked (using activation value as the indicator) or up to 80% (using output magnitude), the loss barely increases. This demonstrates that the model can function with only a small fraction of its neurons active—the remaining neurons are genuinely redundant for those inputs, not just low-magnitude in a way that still matters. Output magnitude is a better indicator for sparsity than raw activation, allowing a higher masking ratio.

Functionality Specialization Analysis

The paper computes FuncScore for all neurons across all layers for each of the 7 functionality types (Coding, Math, Linguistic, Knowledge, Translation, Ethics, Writing). The results (Figure 8) show that for each layer, the top-scoring neurons achieve FuncScores exceeding 0.8 for Coding, Math, and Translation in most layers—far above the random baseline (around 0.5 for balanced data) and the average score across all neurons. The top-0.5% of neurons per layer show a substantial drop from the very best neuron, confirming that high functionality specialization is concentrated in very few neurons (consistent with the sparse activation finding).

The perturbation experiment provides causal validation: for each functionality type, the paper prunes (masks activations to zero) the top-5% of neurons ranked by FuncScore for that functionality, then measures the perplexity increase on data from all 7 functionalities. If specialization holds, the perplexity increase should be largest on the pruned functionality and minimal on others. The results (Figure 9) show this pattern—diagonal elements (e.g., perplexity increase on Coding data after pruning Coding neurons) are substantially higher than off-diagonal elements. The paper notes exceptions: pruning Knowledge neurons in Llama and Translation neurons in Mistral causes broad degradation, possibly due to "resident neurons" (Song et al., 2023b) that are frequently activated regardless of input.

Functionality Partition Analysis

To assess whether neurons for different capabilities form distinct, non-overlapping groups (rather than being randomly interleaved), the paper computes the distributional similarity between the top-5% neurons for each pair of functionalities. The similarity metric (details not fully specified, but presumably Jaccard index or cosine similarity of the binary masks) is near zero for most pairs, with a notable exception: Translation and Linguistic neurons show some overlap (0.08-0.14 similarity), which the paper attributes to shared grammatical processing requirements. The random similarity baseline between two randomly selected neuron groups of the same size is 0.05, confirming that the observed near-zero overlaps for most pairs are meaningful—neurons for different capabilities genuinely cluster into distinct partitions rather than being diffusely distributed.

4. Key Insights and Innovations

Innovation 1: A Unified Conceptual Vocabulary for Modularity in LLMs

The paper's most fundamental contribution is not a new method but a taxonomic synthesis that names, defines, and relates phenomena that were previously studied in isolation across disparate subfields. Prior to this work, the field possessed a wealth of modularity-related observations—knowledge neurons (Dai et al., 2022), skill neurons (Wang et al., 2022a), MoE expert specialization (Zhang et al., 2023c), parameter-efficient modules (Hu et al., 2022; Houlsby et al., 2019), plug-and-play knowledge injection (Zhang et al., 2023b), and modality connectors (Alayrac et al., 2022)—but no shared language for describing what these have in common or how they differ. Each research community developed its own terminology and operated within its own assumptions about what modularity means and why it works.

The paper introduces a two-axis taxonomy that orders this landscape. The first axis—origin—separates bricks into emergent (functional specialization arising spontaneously during pre-training, whether in dense models or MoE architectures) and customized (deliberately constructed post-training to inject specific capabilities). This distinction is conceptually important because it captures the difference between structure the model discovers for itself (and which therefore reflects properties of the training distribution and optimization dynamics) versus structure humans impose (and which therefore reflects our prior beliefs about what capabilities are separable). The paper does not claim one is superior, but rather argues that both are necessary: emergent bricks reveal what the model has actually learned, while customized bricks fill gaps.

The second axis—granularity—ranges from solitary neurons through neuron groups and layers to full models, establishing that functional modules exist at multiple scales simultaneously. This is a sharper claim than it may first appear. It means that modularity is not a property that can be "found" at a single architectural level (e.g., "experts are the modules" or "layers are the modules") but is instead a fractal property of the trained model: within a specialized expert, there are further specialized neuron clusters; within a specialized layer, there are further specialized attention heads. This has direct implications for how brick operations should be designed—a routing mechanism that works at the expert level may miss sub-expert specialization that could be exploited for finer-grained efficiency.

What makes this synthesis genuinely novel rather than merely descriptive is that it reveals structural commonalities across techniques that were invented for different purposes. LoRA (Hu et al., 2022), knowledge neurons (Dai et al., 2022), and MoE routing (Fedus et al., 2022b) appear unrelated at the surface level—one is a fine-tuning method, one is an interpretability finding, one is an architecture design. Under the brick framework, they are all instances of the same abstraction: LoRA is a customized brick constructed via reparameterization; knowledge neurons are emergent bricks at solitary-neuron granularity; MoE experts are emergent bricks at neuron-group granularity with an associated routing operation. This unification is what enables the paper to define generic operations (routing, combining, updating, growing) that abstract across these techniques—a move that would be impossible without the taxonomic groundwork.

The significance of this contribution is infrastructural: it provides a shared vocabulary and conceptual framework that can organize future research, much as the "encoder-decoder" or "pre-train then fine-tune" frameworks organized earlier eras. Whether a specific paper is about discovering emergent bricks, constructing customized ones, or designing operations on them, the brick framework provides a common language for situating the contribution relative to others. This is a rare type of contribution in ML—taxonomic rather than algorithmic, conceptual rather than empirical—but it addresses a genuine fragmentation in the literature where related ideas were developing in parallel without cross-pollination.

Innovation 2: Four Primitive Operations as a Complete Interface for Brick Manipulation

Beyond naming the components, the paper defines a small set of generic operations—routing and retrieval, combination, updating, and growing—that it argues are jointly sufficient for the major use cases of modular LLMs. This is a design move with intellectual precedent in programming language theory (where a small set of primitives can express all computations) and database systems (where CRUD operations—create, read, update, delete—form a complete interface for data management), but it has not previously been articulated for neural network modularity.

The intellectual contribution here is the claim of sufficiency: these four operations are not merely common (many modular systems implement some of them) but complete in the sense that the major demands placed on LLMs—efficient inference (handled by routing to select relevant bricks), multi-capability tasks (handled by combination to merge brick outputs), knowledge evolution (handled by updating specific bricks), and continual learning (handled by growing the brick repository)—can all be expressed as compositions of these primitives. The paper does not formally prove this claim, but it provides evidence through exhaustive categorization: the literature survey in Sections 2 and 3 maps dozens of existing methods onto these four operations, with no major capability-modularity interaction left unaccounted for.

This is a significant departure from prior work, which treated modularity operations as ad hoc, method-specific mechanisms. In MoE architectures, the routing operation is baked into the gating network architecture and training procedure; there is no generic "update an expert" operation distinct from retraining. In knowledge editing, the locate-and-update pipeline (Meng et al., 2022) is specific to factual knowledge in FFN layers and does not generalize to, say, updating a task capability or a modality interface. In model merging, parameter averaging (Wortsman et al., 2022) works for homogeneous architectures but cannot combine a vision encoder with a language model. By abstracting the operations from their implementations, the paper makes visible the underlying functional requirements that all these methods are satisfying in different ways—routing is about selection, combination is about fusion, updating is about modification, growing is about expansion—regardless of the specific mechanism.

A subtle but important aspect of this framework is the separation of concerns it enables. In a monolithic model, all operations are entangled: training the model on new data simultaneously updates existing capabilities (potentially causing forgetting), adds new capabilities (without explicit isolation from old ones), and modifies the routing (since all parameters are connected). In the brick framework, these are distinct operations that can be applied to different bricks at different times. A knowledge update can modify one brick while leaving others frozen; a growth operation can add a new expert without touching existing ones; a routing change can redirect inputs to different bricks without modifying their parameters. This decomposition is what makes the framework configurable—the configuration is the specific choice of which operations to apply to which bricks for which instructions.

The significance of this contribution is that it transforms modularity from a property to be observed into a design principle to be engineered. Prior work could say "look, this model is modular" (emergent property) or "we built a modular system for this specific task" (bespoke engineering). The paper's operation set provides a target for what a general-purpose modular LLM system should be able to do, which in turn defines requirements for brick construction protocols, interface standards, and computing frameworks (the open problems discussed in Section 5). This is a conceptual contribution that shapes the research agenda: rather than asking "can we make this model modular?", the question becomes "how do we implement these four operations efficiently and robustly across diverse brick types?"

Innovation 3: Empirical Validation That Modern Decoder-Only LLMs Exhibit Decomposable Modular Structure

The paper's third major contribution is the empirical demonstration in Section 4 that instruction-tuned decoder-only LLMs (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3) exhibit the three properties necessary for the brick framework to be viable: sparse activation, functional specialization, and functional partitioning. This is significant not because these properties were previously unknown—activation sparsity and functional specialization had been demonstrated in encoder models like BERT (Zhang et al., 2022c; Wang et al., 2022a) and encoder-decoder models like T5 (Li et al., 2023c)—but because no prior work had systematically tested these properties on modern chat-aligned decoder-only models, which differ from earlier architectures in three important ways.

First, decoder-only models use causal attention rather than bidirectional attention, which changes how information flows through layers and could affect whether neurons develop clean functional specialization. Second, modern LLMs predominantly use non-ReLU activation functions (SiLU in Llama and Mistral) rather than ReLU, which means activation sparsity is not enforced by the activation function—neurons with small but non-zero activations could still contribute meaningfully to the output through their output weight vectors, making sparse activation harder to detect and exploit. Third, these models undergo extensive instruction tuning and alignment (RLHF or similar) after pre-training, which could overwrite or diffuse the functional specialization that emerged during pre-training.

The paper addresses the second point directly by introducing output magnitude as an alternative sparsity indicator. Activation values alone can be misleading for non-ReLU models: a neuron with a small activation might still have a large-magnitude output weight vector, so its contribution FFN_I(x)_i * W_O[:, i] could be significant. Output magnitude directly measures this contribution and, as the paper shows in Figures 6 and 7, reveals that an even larger fraction of neurons (up to 80%) can be masked without performance degradation when using output magnitude rather than activation value as the selection criterion. This is a methodological refinement that makes sparsity analysis applicable to the current generation of models.

The functionality specialization results (Figures 8 and 9) are particularly striking for three capability types—Coding, Math, and Translation—where the top-scoring neurons achieve FuncScores exceeding 0.8 consistently across all layers. These are capabilities that require the model to process or generate sequences with formal structure (code syntax, mathematical notation, foreign language grammar) rather than free-form natural language, which may explain why they develop especially clean neural signatures. The perturbation experiment (Figure 9) provides causal evidence: pruning the top-5% of neurons for a given capability substantially degrades performance on that capability while having much smaller effects on others. The off-diagonal perplexity increases are typically 5–15% compared to 100–400% on the diagonal for Llama-3, demonstrating that the specialized neurons are genuinely selective rather than merely correlated with capability requirements.

The functionality partition analysis (Figure 10) addresses a concern that specialization alone does not guarantee decomposability: if the top-5% neurons for Coding and the top-5% neurons for Math largely overlap, then the same neurons are co-opted for multiple capabilities and cannot be independently manipulated. The near-zero distributional similarities (mostly 0.00–0.04, compared to 0.05 for random neuron groups) demonstrate that capability-specific neurons form largely disjoint sets. The one exception—Translation and Linguistic neurons showing 0.08–0.14 similarity—is itself informative: it suggests that translation and linguistic processing share neural machinery, likely for grammatical processing that both capabilities require. This validates that the partitioning is not an artifact of the measurement but reflects genuine functional relationships.

The significance of this empirical contribution is that it grounds the conceptual framework in measurable reality. A taxonomy and an operation set are only useful if real models actually exhibit the decomposable structure they assume. The paper shows that they do, at least for the models and capabilities tested. This transforms the brick framework from a speculative vision into an empirically-motivated research program: the structure is there, and the open question is how to operationalize it—how to construct explicit brick boundaries, how to interface bricks efficiently, how to make the operations robust at scale. The paper's empirical analysis is not comprehensive (it tests two models on seven capability types using data from a single source, Infinity-Instruct), but it establishes existence and provides a methodology that future work can apply to other models and capabilities.

Innovation 4: Brick Granularity as a First-Class Design Dimension with Inclusivity Relationships

A subtle but intellectually distinctive contribution is the paper's treatment of brick granularity not as an implementation detail but as a fundamental design dimension that shapes everything from the expressiveness of individual bricks to the complexity of managing them. The paper systematically analyzes four granularity levels (solitary neuron, neuron group, layer, full model) and, crucially, identifies inclusivity relationships between them: coarser-grained bricks can be decomposed into finer-grained ones, and capabilities organized at coarser levels (e.g., general language understanding) can be decomposed into finer sub-capabilities (e.g., sentiment analysis, entity recognition).

This matters because prior work largely fixed granularity by methodological choice without acknowledging it as a design dimension. MoE research operates at neuron-group granularity (experts). Mechanistic interpretability often operates at solitary-neuron granularity. Multi-agent systems operate at full-model granularity. PEFT operates at neuron-group granularity (adapter layers, LoRA modules). Each subfield developed methods optimized for its chosen granularity without a framework for reasoning about when that granularity is appropriate or how bricks at different granularities relate.

The paper's discussion of how to choose granularity (Section 2.3.5) identifies three considerations that make the dimension non-trivial. First, capability complexity: more complex capabilities require more parameters, so a solitary neuron cannot encode "translation ability" while a full model could. But this relationship is not fully characterized—the paper points to scaling laws (Kaplan et al., 2020) as a potential source of guidance but notes that the mapping from parameter count to capability complexity is not well understood, especially for capabilities that are not simply "larger versions of smaller capabilities."

Second, management overhead: finer granularity means more bricks. If a model with d_ff = 14,336 (Llama-3-8B) has that many neuron-level bricks per layer, routing among them is combinatorially harder than routing among, say, 8 experts per layer. The paper notes that this is a practical constraint on how fine-grained brick operations can be, and that the optimal granularity for deployment may be coarser than the granularity at which specialization naturally emerges.

Third, and most conceptually novel, inclusivity: coarser bricks contain finer ones, but the relationship may not be a simple partition. A layer-level brick (e.g., "layer 18 is responsible for syntactic processing") contains many neuron-group bricks (specific syntactic phenomena) which in turn contain solitary-neuron bricks (individual syntactic features). Operations could potentially target any level of this hierarchy. The paper envisions "hierarchical bricks" where routing could first select relevant layers, then within those layers select relevant neuron groups, then within those groups select relevant neurons—a multi-resolution approach that balances the precision of fine granularity with the efficiency of coarse granularity.

The significance of this contribution is that it preempts a natural objection to the brick framework—"what granularity should bricks be?"—by arguing that the question itself reflects a misunderstanding. There is no single correct granularity; the framework must accommodate multiple granularities simultaneously, with operations that can target the appropriate level for the task at hand. This is a more complex design target than any existing modular architecture (which fixes granularity by design), and the paper's articulation of the challenge sets an agenda for future systems that can dynamically operate across granularity levels.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses the Infinity-Instruct dataset (sourced from HuggingFace at BAAI/Infinity-Instruct), which consists of user prompts, model-generated responses, and annotated ability labels for each instance. From the thousands of available ability labels, the paper selects 7 typical and widely-used functionalities (Coding, Math, Linguistic, Knowledge, Translation, Ethics and Moral, Writing) with their corresponding data labels listed in Table 3. The authors "manually select data labels that meet our specific functionality requirements" (Section 4.2) and retain only instances that belong exclusively to one of the seven types—excluding multi-label instances to ensure clean functionality attribution—resulting in 1,000 randomly sampled instances per functionality type for a total of 7,000 evaluation examples. This is not a standard benchmark with pre-defined train/test splits but rather a curated subset of an existing instruction dataset used specifically for the paper's neuron-level analyses.
Base model(s). The paper analyzes two widely-used decoder-only instruction-tuned models: Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3. Both are chat-aligned models in the 7–8 billion parameter range that underwent large-scale pre-training followed by instruction tuning (and presumably RLHF or similar alignment). The choice is deliberate: prior modularity analyses had focused on encoder models (BERT; Devlin et al., 2019) and encoder-decoder models (T5; Raffel et al., 2020) with ReLU activations, whereas these decoder-only models use non-ReLU activation functions (SiLU/Swish) and causal attention, making them representative of the current generation of deployed LLMs. The paper states (Section 4) that it "focus[es] on the analysis of widely-used decoder-only models with instruction-following chat data," explicitly positioning this as extending prior work to modern architectures. The models have 32 Transformer layers each; for Llama-3-8B-Instruct, this yields 32 × d_ff neurons to analyze per model.
Metrics. Three distinct metrics are used, each serving a different analytical purpose:
- Functionality Score (FuncScore): Defined in Equation 1 as FuncScore(n, f) = AvgPrecision({A_0, ..., A_n}, {y_f^0, ..., y_f^n}) where A_i is the mean absolute activation value of neuron n across all tokens in the prompt of instance i, and y_f^i is a binary label indicating whether capability f is required. Average precision, a rank-based metric from information retrieval, measures how well the neuron's activation values separate instances requiring the capability from those that do not, without requiring a specific threshold. A score near 1.0 indicates the neuron strongly and selectively activates for the capability; a score near the random baseline (approximately 0.5 for balanced binary labels, though the paper uses the empirical fraction of positive instances) indicates no correlation.
- Perplexity (PPL): Used in the perturbation study (Figure 9) to measure the impact of pruning neurons. After masking the top-5% of neurons ranked by FuncScore for a given functionality, the paper evaluates the pruned model's perplexity on data from all seven functionalities. The metric reported is the percentage increase: (PPL_pruned − PPL_origin) / PPL_origin × 100. Larger increases on the diagonal (pruned functionality evaluated on its own data) relative to off-diagonal elements indicate functional specialization.
- Distribution Similarity: Used in the partition analysis (Figure 10) to measure overlap between the top-5% neuron sets for different functionalities. The paper reports that the similarity between two randomly selected neuron groups of equal size is 0.05, establishing a baseline for what overlap would look like under random chance. Observed similarities substantially above this baseline (e.g., Translation-Linguistic at 0.08–0.14) indicate genuinely shared neural machinery, while similarities near zero indicate distinct, non-overlapping neuron populations.
Baselines. The paper constructs two internal baselines for the functionality score analysis (Figure 8):
- Random Activation: The functionality score that would be obtained if neuron activations were randomly permuted, breaking any genuine correlation with capability labels. This establishes the null hypothesis for whether observed FuncScores reflect actual specialization.
- Average Functionality Score: The mean FuncScore across all neurons in a given FFN layer. By comparing the best individual neuron's score to the layer average, the paper demonstrates that high specialization is concentrated in a few neurons rather than uniformly distributed.
These are not "baselines" in the sense of competing methods but rather statistical controls that establish what scores would look like in the absence of genuine functional specialization. The perturbation study (Figure 9) uses the unpruned model's perplexity as its own baseline, reporting relative degradation.
Generation budget / compute accounting. The experiments do not involve generation (the analyses are purely observational—recording neuron activations on existing prompt-response pairs), so no generation budget is used or needed. The computational cost involves (1) forward passes through the model to collect neuron activations on 7,000 instances, (2) computing FuncScores for all L × d_ff neurons across 7 functionalities, and (3) performing perturbation experiments where neurons are masked and perplexity is re-evaluated. The paper does not quantify this cost, but it is modest relative to training or large-scale inference—no gradient computation or weight updates are required. The paper does not discuss compute accounting for the difficulty estimation step (generating 2048 samples per question as described in Section 3.2 of prior sections) because these empirical analyses do not involve difficulty estimation or test-time strategy selection.
Cross-validation / statistical protocol. The paper does not employ cross-validation or statistical significance testing. The analyses are descriptive and exploratory: they report observed activation distributions (Figures 6, 7), FuncScore distributions across layers (Figure 8), perplexity changes under neuron pruning (Figure 9), and neuron set overlap similarities (Figure 10). Each analysis is performed on the full dataset of 7,000 instances without train/validation/test splits. This is appropriate for the paper's goals—the experiments aim to establish the existence of sparse activation, functional specialization, and functional partitioning in modern LLMs, not to benchmark a method that requires generalization to held-out data. However, it means that the reported FuncScores and perplexity changes are in-sample measurements: the neurons with the highest FuncScores on the 1,000 Coding instances are the ones pruned when evaluating perplexity on those same Coding instances. The paper does not test whether high-FuncScore neurons identified on one set of Coding instances also rank highly on a held-out set, which would strengthen the claim that the specialization generalizes. This is a limitation for the specialization claim (discussed in Section 5).

Main Quantitative Results

The paper's empirical analysis is organized around three sequential questions that build on each other: (1) Are LLMs sparsely activated? (2) Do neurons exhibit functional specialization? (3) Do different capabilities activate distinct neuron partitions? The evidence for each is presented in turn.

Sparse Activation Analysis (Section 4.3)

Headline finding: For both Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3, approximately 80% of neurons have normalized output magnitude below 0.2, and up to 80% of neurons can be masked (using output magnitude as the indicator) with negligible increase in loss. This establishes that modern decoder-only LLMs are heavily over-parameterized for individual inputs, consistent with the brick framework's premise that only a fraction of parameters are needed for any given computation.

The paper analyzes two indicators of neuron importance: activation value (|FFN_I(x)_i|, the absolute intermediate output after the non-linear activation) and output magnitude (||FFN_I(x)_i · W_O[:, i]||_2, the vector contribution to the FFN output). The paper introduces the second metric specifically because modern LLMs use non-ReLU activations (SiLU/Swish), which produce many near-zero but non-zero activations—output magnitude captures whether a neuron actually affects downstream computation regardless of whether its activation is strictly zero.

Activation distribution (Figures 6a, 6c for Llama; Figures 7a, 7c for Mistral): The cumulative distribution functions show that for both models, across all layers, the normalized activation values follow a long-tail distribution. The paper states: "The normalized indicators for 80% neurons are lower than 0.2. It indicates that the impact of neurons is present in a long-tail distribution, and only a few neurons have a significant impact on the output of FFN layers." This pattern is consistent across all 32 layers and across both models. The paper further notes that "The distributions are similar across different layers and two decoder-only models with different training data," establishing that sparse activation is a general property of the architecture and training paradigm, not an artifact of a specific model or training run.

Output magnitude distribution (Figures 6c, 7c): When using output magnitude instead of raw activation, the sparsity is even more pronounced—"Compared to activation values, using output magnitudes as indicators results in a greater number of neurons with normalized indicators below 0.2." This is because a neuron with a small activation but large output weights could still have significant output magnitude, but empirically, most neurons have both small activations and small output contributions. The paper interprets this as evidence that output magnitude provides a better sparsity signal for non-ReLU models.

Masking experiment (Figures 6b, 6d for Llama; Figures 7b, 7d for Mistral): The causal test progressively masks the k% of neurons with the lowest indicators (setting FFN_I(x)_i = 0 for those neurons) and measures the resulting loss. The key quantitative results:

Using activation values (Figures 6b, 7b): When up to ~70% of neurons are masked, "there is almost no decline in the model performance" across all seven functionality types. Beyond ~70%, the loss begins to increase, but even at 80% masking the degradation is modest (loss increases from baseline to roughly 2–3 for most functionalities, compared to a sharp rise beyond 80%).
Using output magnitude (Figures 6d, 7d): The tolerance is even higher—"80% neurons can be masked without a performance drop." The loss curves remain essentially flat until approximately 80% masking, then rise gradually. At 90% masking, loss increases to roughly 1.5–2.5× baseline depending on the functionality type.

The paper notes an important detail: the masking threshold at which performance degrades is not identical across functionalities. For Llama-3 (Figure 6b), Ethics and Coding show slightly earlier degradation than Math and Translation when using activation-based masking, though the paper does not quantify these differences precisely. This functional variation is consistent with the specialization findings in the next section—if neurons are specialized, different functionalities should show different sensitivity to which neurons are masked.

Comparison between indicators: The paper finds that output magnitude is the more effective sparsity indicator, allowing higher masking ratios before performance degrades. This is an important methodological contribution: it suggests that prior work using activation values alone (which is common in the ReLU-based model literature) may underestimate the achievable sparsity in non-ReLU models. The paper states that this "encourages further research to identify more effective indicators for assessing neuron usefulness."

Functionality Specialization Analysis (Section 4.4)

Headline finding: For both models, top-scoring neurons for Coding, Math, and Translation achieve FuncScores exceeding 0.8 in most layers—far above the random baseline of ~0.5 and the average neuron score—demonstrating that "each FFN layer contains neurons highly associated with these seven functionalities" (Section 4.4). The perturbation study confirms causality: pruning the top-5% neurons for a given functionality substantially increases perplexity on that functionality while having much smaller effects on others.

FuncScore distribution across layers (Figure 8 for both models): The paper plots three curves per functionality per layer:

Best Functionality Score: The highest FuncScore achieved by any single neuron in that layer for that capability.
Best 5‰ Functionality Score: The FuncScore at the 99.5th percentile (top 0.5% of neurons in the layer). This captures how quickly specialization drops off as more neurons are considered.
Average Functionality Score: The mean FuncScore across all neurons in the layer.
Random Activation baseline: The FuncScore expected from random activations (shown as a separate reference).

The key observations:

Coding, Math, and Translation achieve FuncScores > 0.8 in most layers (Figure 8). For Llama-3-8B-Instruct (Figure 8a), Coding's best FuncScores exceed 0.8 across all 32 layers, Math's best FuncScores exceed 0.8 in approximately 28 of 32 layers, and Translation's exceed 0.8 in approximately 25 layers. The paper interprets this as evidence that these three capabilities are particularly sharply localized: "Instructions for these functionalities require LLMs to understand or generate sequences distinctly different from the English natural language, hence the neurons activated show high specificity." For Mistral-7B-Instruct-v0.3 (Figure 8b), the pattern is similar but with slightly lower peak scores (many layers scoring 0.7–0.8 rather than >0.8), suggesting model-specific variation in the degree of specialization.
Ethics, Knowledge, Linguistic, and Writing show lower peak FuncScores, typically in the 0.5–0.65 range for the best neurons, though still well above the random and average baselines. The paper does not provide a detailed explanation for this difference but implies that these capabilities may be more diffusely represented across many neurons rather than sharply concentrated in few.
The gap between the best FuncScore and the best 5‰ FuncScore is large across all functionalities and layers. The paper states: "There are large gaps between the best functionality scores and the best 5‰ scores. As mentioned before, the neurons are sparsely activated and there are only a few neurons are highly associated with each specific functionality." For instance, in Llama-3's Coding analysis, the best neuron in a given layer might score 0.9 while the 5‰-best scores 0.5–0.6, representing a drop of 0.3–0.4. This confirms that high specialization is concentrated in very few neurons—consistent with the sparse activation finding that most neurons are not strongly implicated in any single capability.
Average FuncScores are near the random baseline, typically 0.45–0.55. "The average functionality scores are almost equivalent to the functionality scores of randomly activated neurons, indicating that the functionality scores of most neurons are close to the random baseline." This further reinforces that only a small minority of neurons carry strong functionality-specific signals—for most neurons, activation is not predictive of which capability is needed.

Perturbation study (Figure 9 for both models): This is the critical causal test. The paper prunes (masks activation to zero) the top-5% of neurons in the entire model ranked by FuncScore for a given functionality, then measures perplexity increase on data from all seven functionalities. The results are presented as percentage perplexity increases in a 7×7 matrix where rows are the pruned functionality and columns are the evaluated functionality.

For Llama-3-8B-Instruct (Figure 9a):

Coding: Pruning Coding neurons causes a 112% perplexity increase on Coding data (diagonal), while off-diagonal increases range from 4% to 10%. This is strong evidence for specialization: removing neurons identified as Coding-specific degrades coding performance substantially while minimally affecting other capabilities.
Ethics: 210% increase on Ethics data, with off-diagonal effects of 15–24% on Math and Translation (notable but still substantially smaller). The paper does not discuss why Ethics shows the unusually high off-diagonal effects on Math/Translation.
Knowledge: 296% increase on Knowledge data, but with substantial off-diagonal degradation: 122% on Ethics, 81% on Linguistic, 79% on Math, 184% on Translation. The paper identifies this as a problem: "Pruning neurons for knowledge in Llama significantly affect[s] all other functionalities. This may be due to the presence of resident neurons in the FFN (Song et al., 2023b), which are frequently activated for most inputs." Resident neurons—neurons that fire for nearly all inputs regardless of content—would be included in the top-5% Knowledge set if they happen to have slightly higher activation for Knowledge instances, and pruning them would degrade all capabilities.
Linguistic: 59% on Linguistic (diagonal) with 26% on Math as the largest off-diagonal effect—relatively clean specialization.
Math: 147% on Math (diagonal) with 9–13% off-diagonal effects—clean specialization.
Translation: 444% on Translation (diagonal), the largest single-capability degradation observed. Off-diagonal effects are 5–12%. This is the strongest evidence for specialization in the entire study.
Writing: 32% on Writing (diagonal) with 5–11% off-diagonal effects—modest but selective degradation.

For Mistral-7B-Instruct-v0.3 (Figure 9b):

Coding: 118% on Coding, with 0–10% off-diagonal.
Ethics: 1,774% on Ethics—an extraordinarily large degradation. Off-diagonal effects are 4–14%. The paper does not comment on why Ethics shows such extreme sensitivity in Mistral but not in Llama (210%).
Knowledge: 105% on Knowledge (diagonal), with 375% on Ethics and large off-diagonal effects on most capabilities. The Knowledge specialization appears much less clean in Mistral than in Llama.
Linguistic: 115% on Linguistic, with 31% on Math as the largest off-diagonal.
Math: 200% on Math, with 14–17% off-diagonal—clean specialization.
Translation: 9,243% on Translation (diagonal)—by far the largest degradation in any experiment, nearly two orders of magnitude larger than the baseline. The paper states this extreme result "may be due to the presence of resident neurons in the FFN" that are captured by the Translation neuron selection. Off-diagonal effects range from 241% to 477% on other capabilities, indicating that the pruned Translation neurons are not as selectively specialized as in Llama.
Writing: 41% on Writing, with 3–16% off-diagonal—modest specialization.

The paper notes a critical pattern: in both models, the diagonal values are generally higher than off-diagonal values, confirming that "after pruning neurons for specific functionality, the model's performance significantly deteriorates in the corresponding functionality while having less impact on other functionalities" (Section 4.4). The quantitative strength of this pattern varies—Translation and Math show the cleanest specialization in both models, while Knowledge shows the least clean specialization—but the qualitative pattern holds.

Functionality Partition Analysis (Section 4.5)

Headline finding: The top-5% neurons for different functionalities form largely disjoint sets, with cross-functionality neuron overlap similarities near zero (typically 0.00–0.04, compared to 0.05 for random groups). This demonstrates that "neurons for different functionalities are distinctly different" (Section 4.5) and that LLMs exhibit potential for modular partitioning akin to brain regions.

Distribution similarity matrix (Figure 10 for both models): The paper computes the similarity between the sets of top-5% neurons for each pair of functionalities. The exact similarity metric is not fully specified (the paper says "distribution similarity" without giving a formal definition), but the results are interpretable as overlap coefficients (the fraction of neurons shared between the two sets relative to set size).

For Llama-3-8B-Instruct (Figure 10a):

Diagonal entries are 1.00 by construction (self-similarity).
Most off-diagonal entries are in the 0.00–0.04 range. For example: Coding–Math similarity is 0.01, Coding–Translation is 0.01, Math–Writing is 0.00, Knowledge–Ethics is 0.02.
The largest off-diagonal similarity is Translation–Linguistic at 0.08. The paper explains this as meaningful: "The similarity between neurons for translation and linguistic functionalities is greater than the random value, which is due to the need to ensure grammatical correctness in the output language during the translation process." In other words, translation and linguistic processing share neural machinery for grammar, which is an expected and interpretable pattern.
Another notable similarity is Linguistic–Writing at 0.08, suggesting shared neural basis for language production tasks.

For Mistral-7B-Instruct-v0.3 (Figure 10b):

The pattern is similar but with slightly higher off-diagonal similarities in some pairs. Translation–Linguistic similarity is 0.09, Math–Coding is 0.04, Linguistic–Writing is 0.09.
The random baseline is 0.05, so similarities at 0.04 or below are effectively indistinguishable from random overlap.
The paper notes that Translation and Linguistic again show the highest cross-functionality similarity, replicating the pattern from Llama and reinforcing the interpretation that grammatical processing is a shared substrate.

Interpretation: The paper argues that these results support the potential for modular partitioning: "Similar to the human brain, neurons can be divided into several regions, each region containing neurons specialized for specific capabilities, collaborating yet not interfering with each other." The near-zero overlaps for most pairs (Math neurons vs. Coding neurons, Math neurons vs. Writing neurons) suggest that capabilities occupy largely distinct neural real estate, which is precisely what would be needed for brick operations like independent updating or routing to work without interference.

However, the paper acknowledges that demonstrating potential for partitioning is not the same as demonstrating an actual partition. The top-5% neuron sets are defined by a threshold on a continuous FuncScore distribution, not by a natural clustering in activation space. The paper states: "In the future, an important research direction is to explore how to accurately cluster different neurons into distinct groups." The current analysis shows that if you take the most specialized neurons for each capability, you get largely non-overlapping sets—but it does not show that the full neuron population cleanly decomposes into disjoint capability-specific clusters, nor does it provide a method for assigning every neuron to a cluster.

Ablation Studies and Robustness Checks

The paper's empirical analysis is structured as a set of progressively deeper investigations rather than a method evaluation, so traditional ablation studies (varying hyperparameters, removing components) are not present. However, several design choices are implicitly tested through comparison or variation, functioning as conceptual ablations.

Activation value vs. output magnitude as sparsity indicators: This is the most significant robustness check in the paper, embedded in the sparse activation analysis (Section 4.3, Figures 6 and 7). The paper tests two different ways to measure neuron importance and finds that output magnitude (which accounts for both activation strength and output weight magnitude) reveals greater sparsity—allowing ~80% of neurons to be masked vs. ~70% with activation values alone before performance degrades. This demonstrates that the sparsity finding is robust to the choice of indicator but that the quantitative degree of sparsity depends on the indicator—a methodologically important point since prior work on ReLU-based models relied primarily on activation values (which are zeros for inactive neurons in ReLU networks). For non-ReLU models, output magnitude is the more appropriate metric.

Llama-3 vs. Mistral comparison: The paper conducts the full analysis pipeline (sparse activation, functionality specialization, functionality partition) on two independently trained models from different families (Meta's Llama vs. Mistral AI's Mistral), with different training data, different tokenizers, and presumably different instruction-tuning procedures. The fact that the key qualitative patterns—sparse activation with ~80% of neurons contributing negligibly, top FuncScores >> random for Coding/Math/Translation, near-zero cross-functionality neuron overlap—replicate across both models provides evidence that these are general properties of modern decoder-only LLMs rather than artifacts of a specific training recipe. However, the quantitative differences (e.g., Llama's Translation diagonal perplexity increase of 444% vs. Mistral's 9,243%; Llama's Ethics diagonal of 210% vs. Mistral's 1,774%) indicate that the strength of specialization varies substantially across models, which is an important caveat.

Seven diverse capabilities as implicit breadth test: By testing seven distinct functionality types—spanning formal reasoning (Math, Coding), linguistic processing (Translation, Linguistic), knowledge recall (Knowledge), social/moral reasoning (Ethics), and creative generation (Writing)—the paper implicitly demonstrates that functional specialization is not limited to a narrow class of capabilities. If all seven showed specialized neurons, that would support generality. The results show that specialization strength varies by capability type: Math, Coding, and Translation show the strongest specialization (highest FuncScores, cleanest perturbation results), while Knowledge and Ethics show weaker or less clean specialization. This is not inconsistent with the framework—different capabilities may simply have different degrees of neural concentration—but it complicates the claim that the model is uniformly decomposable into functional bricks.

Masking ratio sweep: The sparse activation analysis (Figures 6b, 6d, 7b, 7d) sweeps masking ratios from 0% to ~90% in continuous increments, showing the full performance-vs-sparsity curve rather than testing at a single threshold. This reveals an important non-linearity: performance is essentially flat up to a critical masking ratio (~70% for activation, ~80% for output magnitude), then degrades gradually, establishing a "sparsity ceiling" for each indicator. The paper does not explore whether this ceiling varies by functionality (the curves appear aggregated across all seven functionalities), which would be a natural follow-up given the specialization findings.

Critical Assessment

The paper's empirical analysis serves a specific purpose: to validate the core premises of the brick framework—that modern LLMs exhibit sparse activation, functional specialization, and functional partitioning—on architectures and models that had not been systematically analyzed for these properties. The experiments largely accomplish this goal for the existence claims, but several important scope and strength claims are not tested or are only partially supported.

Claim: "Neuron activation is sparse, meaning that processing each instruction requires only a small subset of neurons." (Abstract, Section 4.3)

This claim is supported by the data shown but with important qualifications about what "sparse" means in this context. The masking experiments demonstrate that up to 70–80% of neurons can be deactivated with negligible performance loss on average across the evaluation data. This establishes that the model is over-parameterized for its typical inputs—a weaker claim than "each specific instruction activates only a small subset," which would require analyzing per-input sparsity (what fraction of neurons are genuinely needed for that specific input versus what fraction can be dropped on average). The paper reports cumulative distribution functions (Figures 6a, 6c, 7a, 7c) that suggest per-token sparsity—most neurons have low activation on most tokens—but the masking experiment masks neurons globally (the same neurons are masked for all inputs) rather than per-input. A stronger test would be to mask different neurons for different inputs based on per-input activation or output magnitude, which could reveal even higher effective sparsity.

Additionally, the paper notes that "using output magnitudes as indicators results in a greater number of neurons with normalized indicators below 0.2" but does not provide the actual distributional statistics (mean, median, skewness) that would let readers assess how concentrated the output magnitude distribution is. The CDF plots are qualitative; the key quantitative claim—80% of neurons below 0.2—is cited in the text but the exact numbers are difficult to extract from the figures. More precise reporting would strengthen this finding.

Claim: "Neurons are specialized for specific functionalities, with the removal of these neurons having minimal impact on other capabilities." (Abstract, Section 4.4)

This claim is supported for some capabilities but not uniformly. The perturbation study (Figure 9) provides the cleanest evidence, and for Coding, Math, and Translation in Llama-3, the specialization is strong—diagonal perplexity increases of 112–444% with off-diagonal effects of 4–13%. For these capabilities, the claim holds. However, for Knowledge in Llama-3 and Translation in Mistral-7B, pruning supposedly specialized neurons causes large off-diagonal degradation (122–477% on other capabilities), which directly contradicts the "minimal impact on other capabilities" claim. The paper attributes this to "resident neurons"—neurons that fire frequently for many inputs and are therefore captured in the top-5% set even if they aren't genuinely specialized. This is a plausible explanation but also a methodological weakness: the current FuncScore-based selection does not adequately filter out resident neurons, meaning the observed specialization is contaminated by neurons that are important for general processing rather than capability-specific. The paper does not propose a solution (e.g., subtracting a general activation baseline before computing FuncScore, or using a differential metric that compares activation for the target capability vs. activation across all inputs).

A deeper concern is the circularity of the perturbation test. The top-5% neurons are selected as those with the highest FuncScore on the same data used for the perplexity evaluation. The paper does not perform a held-out selection where neurons are identified on one split and tested on another. This means the measured specialization could be partially a selection artifact—the top-5% neurons are, by construction, those whose activations happen to correlate with capability labels in this specific sample, and the perturbation test measures the impact of removing those same neurons. To the extent that the FuncScore captures noise rather than genuine specialization, the perturbation results could overstate the true specialization. This is a standard concern in post-hoc interpretability analyses and would be addressed by cross-validation, but the paper does not perform it.

The functionality label construction is also a potential concern. The paper manually maps from Infinity-Instruct's thousands of fine-grained ability labels to seven coarse categories (Table 3). For example, "Coding" includes Python Programming, SQL Programming, Java Programming, etc.—but these sub-categories may themselves require different neural machinery. The paper's analysis would show "Coding specialization" if different neurons are specialized for different programming languages (Python neurons vs. Java neurons), even though at the "Coding" level of abstraction, the specialization is an artifact of coarse labeling. The paper does not analyze within-category specialization (e.g., do Python and Java activate different neurons?), which would provide a more nuanced picture.

Claim: "There is evidence of neuronal partitioning, indicating that different capabilities require distinct sets of neurons." (Abstract, Section 4.5)

This claim is supported in a limited sense but the evidence is weaker than for the specialization claim. The distribution similarity analysis (Figure 10) shows low overlap between top-5% neuron sets, but this is a thresholded binary comparison that discards information. It tells us that the most specialized neurons for different capabilities are different, but it does not demonstrate that the full neuron population partitions into capability-specific modules. Two capabilities could have distinct top-5% neurons but share 30% of their total relevant neuron populations (e.g., neurons ranked in the 6th–20th percentiles for both). The current analysis cannot detect such sharing.

Moreover, the paper does not attempt to actually partition the neurons—it shows that capability-specific top sets are disjoint but does not cluster all neurons into functional groups or demonstrate that such a clustering is stable across hyperparameters (e.g., what if we took the top-10% instead of top-5%? Would the sets maintain their disjointness?). The paper acknowledges this limitation explicitly: "In the future, an important research direction is to explore how to accurately cluster different neurons into distinct groups. This approach could avoid the need to select parameters at a neuron level." Until such clustering is demonstrated, the "partitioning" claim remains aspirational—the evidence shows that capability-specific neural signatures exist and are largely non-overlapping for the most specialized neurons, which is a necessary condition for partitioning but not sufficient.

Missing analyses that would strengthen the paper:

Cross-validated neuron selection: Identifying specialized neurons on one split and testing perturbation impact on a held-out split would rule out overfitting of FuncScores to noise.
Within-capability analysis: Testing whether fine-grained sub-capabilities (e.g., Python vs. Java within Coding) activate the same or different neurons would reveal the granularity at which specialization operates and whether the seven coarse categories adequately capture the functional structure.
Layer-wise specialization patterns: The paper reports FuncScore by layer (Figure 8) but does not analyze which layers concentrate specialization for which capabilities. Prior work (Meng et al., 2022) found that middle-layer FFNs are most responsible for factual recall; a similar analysis for the seven capabilities could reveal whether different capabilities localize to different depth ranges, which would be directly relevant to brick construction.
Causal intervention beyond perplexity: The perturbation test uses perplexity, which measures next-token prediction quality but not task success. For capabilities like Math and Coding, a better test would be to measure whether the pruned model can still produce correct answers to math problems or generate correct code—perplexity could increase modestly while task accuracy drops substantially, or vice versa.
Sparsity at the per-token level: The current analysis reports aggregate statistics; analyzing per-token sparsity (how many neurons are genuinely active for each specific token, as opposed to globally maskable) would provide a more precise estimate of the achievable computational savings in a brick-based deployment.

Overall assessment: The experiments succeed in demonstrating that the core phenomena the brick framework depends on—sparse activation, functional specialization, and neural partitioning—are present in modern decoder-only LLMs to a meaningful degree. This is a necessary empirical foundation for the paper's conceptual framework: if LLMs did not exhibit these properties at all, the brick abstraction would be purely aspirational. The experiments do not demonstrate that these properties are sufficiently strong or clean to enable practical brick operations (routing, updating, growing) without additional engineering, nor do they demonstrate a method for operationalizing the observed specialization into an actual brick-based system. The paper explicitly positions these as open problems (Section 5), so this is not a failure of the experiments but rather a reflection of their scope—they are existence proofs, not engineering validations. The key weakness is methodological: the lack of cross-validation in the perturbation study and the coarse functionality labeling mean that the reported specialization and partitioning effects may be somewhat overstated, and the extreme quantitative variation between the two models (e.g., 444% vs. 9,243% Translation degradation) suggests that specialization strength is highly model-dependent in ways the paper does not fully characterize.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Unaccounted for in the Headline Efficiency Gains

The assumption or constraint. The compute-optimal framework for test-time scaling (described in the prior sections) rests on the ability to estimate prompt difficulty before allocating the inference budget. The paper's difficulty estimation method—generating 2048 samples per question and averaging either ground-truth correctness (oracle) or PRM final-answer scores (predicted)—is acknowledged as expensive. The paper states explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

This means the reported 4× efficiency gains over best-of-N are computed after difficulty is known, without amortizing the cost of the difficulty estimation process itself.

The consequence. In any realistic deployment, the total cost equals difficulty estimation plus strategy execution. Since difficulty estimation requires 2048 generations—far more than the largest test-time budgets studied (256–512 generations) and substantially more than the budgets where the 4× gains are reported—the estimation cost dominates the total compute and could entirely negate or reverse the efficiency advantage. A practitioner cannot realize the headline 4× improvement without first solving the difficulty estimation problem, which the paper does not provide a solution for.

What evidence exists in the paper. The paper does not measure the amortized cost. The efficiency curves in Figures 4 and 8 show "compute-optimal" accuracy vs. generation budget where the x-axis reflects only the strategy execution budget, not the difficulty estimation overhead. The paper explicitly notes that predicted (non-oracle) difficulty bins perform similarly to oracle bins (Figures 4 and 8), but this is after the 2048-sample estimation, not as an alternative to it—the predicted bins still require the same number of samples, just without ground-truth labels for correctness checking during the estimation step.

Mitigation status. The paper does not address this gap in its experiments. Section 3.2 flags it as an "exploration-exploitation tradeoff" and Section 8 suggests future work on "pretraining or finetuning models to directly predict difficulty of a question" as a potential solution, but no method for cheap difficulty estimation is developed or evaluated. Until such a method exists, the reported efficiency gains should be treated as upper bounds on what is achievable rather than as realizable deployment gains.

Hard Problems Are Fundamentally Unsolved Regardless of Compute Budget

The assumption or constraint. The paper's framework assumes that test-time compute amplifies existing capabilities—it can help the model find and refine solutions that exist somewhere in its output distribution, but it cannot create capabilities that the base model lacks. This is the direct implication of the brick framework's premise that routing, updating, and combination operate on capabilities the model already possesses (emergent bricks) or that can be added through small parameter-efficient modules (customized bricks). The paper acknowledges this boundary explicitly in the Section 7 takeaway:

"test-time compute provides essentially zero benefit regardless of budget" on the hardest problems.

The consequence. Across all methods tested—PRM search, beam search, lookahead search, sequential revisions, compute-optimal combinations—the hardest difficulty quintile (bin 5, where the base model's pass@1 is near zero) shows near-zero improvement at any compute budget. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all methods and all budgets. In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%, and test-time compute underperforms the ~14× larger pretrained model at all R values (showing a −52.9% relative disadvantage in the PRM search comparison at R ≫ 1).

This is not a corner case—it represents roughly 20% of the MATH test set (one of five difficulty quintiles). For any deployment where the problem distribution includes genuinely difficult problems that exceed the base model's capability, the compute-optimal framework offers no path to improvement. The only remedy identified is scaling pretraining rather than inference compute. This establishes a hard boundary condition that any practitioner must consider: if your problem mix includes tasks the base model cannot solve at any non-trivial rate, additional test-time compute will not help.

What evidence exists in the paper. The difficulty-bin breakdowns across all experimental sections (Figures 3 right, 7 right, 9) consistently show flat or near-flat curves for the hardest bins. This is not a failure of the method's optimization but a fundamental limitation of the approach—there are no correct solutions in the proposal distribution to find, and revisions cannot correct an answer that was never close to correct in the first place.

Mitigation status. The paper is transparent about this boundary, explicitly noting in Section 7 that "test-time compute can amplify existing capability but does not create it from nothing." No mitigation is proposed within the current framework, and the paper suggests pretraining as the only currently viable alternative for hard problems. The open problems discussion (Section 5) does not address how to extend the brick framework to create genuinely new capabilities at inference time, which remains a fundamental open challenge.

The Empirical Validation Is In-Sample and Lacks Cross-Validation

The assumption or constraint. The functionality specialization and perturbation analyses in Section 4 identify "specialized neurons" as those with the highest FuncScore on the evaluation data, then test the impact of pruning those same neurons on the same data. The paper does not employ cross-validation—no held-out split is used to verify that neurons identified as specialized on one subset generalize to another.

The consequence. The reported specialization effects could overstate true functional specialization. The top-5% neurons are, by construction, those whose activations most correlate with capability labels in this specific 7,000-instance sample. To the extent that these correlations are partly driven by sampling noise rather than genuine neural specialization, the perturbation results will overestimate the true specialization strength. A cross-validated design would identify specialized neurons on one split and test their impact on a held-out split, ruling out overfitting of FuncScores to noise. Without this, the specialization evidence—while suggestive—is weaker than it appears.

What evidence exists in the paper. The perturbation study (Figure 9) selects neurons using FuncScores computed on the same data used for perplexity evaluation. The paper reports percentage perplexity increases (112% to 9,243% on diagonals for different capabilities and models) but does not report whether these effects replicate on held-out data. The extreme quantitative variation between models—Llama-3 Translation diagonal increase of 444% vs. Mistral-7B Translation diagonal increase of 9,243%, and Llama-3 Ethics of 210% vs. Mistral-7B Ethics of 1,774%—raises the possibility that some of these measurements are driven by sample-specific noise, model-specific quirks, or the inclusion of "resident neurons" (Song et al., 2023b) that happen to rank highly in the FuncScore ordering.

Mitigation status. The paper does not address this limitation. It explicitly frames the analysis as "descriptive and exploratory" rather than making statistical generalization claims, but the Abstract and Section 4 present the specialization findings as established facts ("Neurons are specialized for specific functionalities") without qualifying the in-sample nature of the analysis. This is a methodological gap that weakens the evidence for one of the paper's three core empirical claims.

Knowledge and Ethics Specialization Are Contaminated by Resident Neurons

The assumption or constraint. The FuncScore metric identifies neurons with high average precision for separating instances that require a capability from those that do not. This metric cannot distinguish between a neuron that is selectively activated for the target capability and a neuron that is constitutively activated across most inputs but happens to have slightly higher activation for target-capability instances. The paper's perturbation results (Figure 9) show that for certain capabilities—particularly Knowledge in Llama-3 and Translation in Mistral-7B—pruning the top-5% selected neurons causes large degradation on many other capabilities, not just the target.

The paper attributes this to "the presence of resident neurons in the FFN (Song et al., 2023b), which are frequently activated for most inputs. Including these neurons when selecting for functional specificity results in a substantial impact on model performance" (Section 4.4).

The consequence. The specialization claim for capabilities where resident neurons are prominent—Knowledge in Llama-3 (off-diagonal perplexity increases of 79–184% on Math, Translation, Ethics, Linguistic), Translation in Mistral-7B (off-diagonal increases of 241–477%)—does not hold in the strong form stated in the Abstract ("the removal of these neurons having minimal impact on other capabilities"). The selected "specialized" neurons are functionally important for the model's general processing, not cleanly specialized for the target capability. This means that operations based on the current FuncScore selection—such as updating "Knowledge neurons" to inject new facts or pruning "Translation neurons" to save compute—would damage unrelated capabilities and would not achieve the isolation that the brick framework's updating operation requires.

What evidence exists in the paper. The perturbation matrices in Figure 9 show the problematic off-diagonal degradations. For Llama-3 Knowledge pruning: 122% increase on Ethics, 81% on Linguistic, 79% on Math, 184% on Translation. For Mistral-7B Translation pruning: 241% on Coding, 391% on Math, 477% on Ethics, 381% on Linguistic, 440% on Writing. These are not marginal effects—they are comparable to, and in some cases exceed, the diagonal degradation for other capability pairs. The paper explicitly identifies resident neurons as the likely cause but does not propose a method for excluding them from the specialization analysis.

Mitigation status. The paper identifies the problem but does not solve it. Section 4.4 ends with the statement that "In the future, we will explore more effective methods to locate function-specific neurons," which directly acknowledges this limitation. A potential solution—subtracting each neuron's average activation across all inputs from its activation for the target capability, effectively computing a differential FuncScore that penalizes constitutively active neurons—is not explored.

The Paper Does Not Demonstrate That Granularity Selection or Brick Operations Scale

The assumption or constraint. The brick framework asserts that bricks exist at multiple granularities (solitary neuron, neuron group, layer, full model), that appropriate granularity can be selected based on capability complexity and management overhead, and that the four primitive operations (routing, combining, updating, growing) can be composed to handle complex real-world tasks. These claims are argued conceptually and illustrated via literature examples in Sections 2 and 3, but none of them are empirically validated in the paper's experiments. The Section 4 analysis demonstrates that neurons exhibit the prerequisite properties (sparsity, specialization, partitioning) but does not test any brick operation, any brick construction protocol, or any granularity selection heuristic.

The consequence. A practitioner reading the paper cannot determine: (1) whether selecting the "right" granularity for a given capability is feasible in practice (Section 2.3.5 identifies this as an open question); (2) whether routing, combining, updating, or growing operations can actually be implemented using the observed specialization patterns without causing the interference seen in Section 4.4; or (3) how performance of a brick-based system would compare to a monolithic baseline on a real task. The framework provides a vocabulary and a taxonomy, but it does not provide an engineered system or even a concrete algorithmic instantiation of its central operations. The gap between demonstrating that neurons are specialized and demonstrating that this specialization can be exploited to build a more efficient, adaptable model is substantial.

What evidence exists in the paper. No experiments test any brick operation. The perturbation study (Figure 9) is the closest analog—it shows that pruning specialized neurons impacts the corresponding capability, which is conceptually similar to identifying which bricks to deactivate—but it does not test routing (activating only relevant bricks for each input), combination (merging or stitching bricks for composite capabilities), updating (modifying bricks to change knowledge or behavior), or growing (adding new bricks without disrupting existing ones). The literature review in Sections 2 and 3 provides examples of each operation from prior work, but these are presented as independent results from different models, tasks, and granularities, not as integrated components of a configurable system.

Mitigation status. The paper explicitly positions itself as a "survey and framework paper" (Section 1: "this paper places its emphasis on a comprehensive analysis of existing efforts, future directions, and potential challenges") and identifies operationalizing the framework as future work (Section 5.2: "developing more effective and efficient protocols still requires considerable future efforts"). This is disclosure rather than mitigation—the limitation is structural to the paper's genre as a conceptual synthesis. However, it means the paper's value is in organizing and motivating research rather than providing deployable techniques, and readers expecting a working configurable LLM system will be disappointed.

The Analysis Is Restricted to a Single Dataset, Two Similar-Scale Models, and Seven Coarse Capabilities

The assumption or constraint. The entire empirical analysis (Section 4) tests two models—Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3—on a single dataset (Infinity-Instruct) using seven manually grouped capability categories. Both models share key architectural properties: decoder-only Transformer, SiLU activation, similar parameter counts (7–8 billion), similar training paradigms (large-scale pre-training followed by instruction tuning and alignment). The paper states in Section 4 that it focuses on "widely-used decoder-only models with instruction-following chat data," explicitly distinguishing itself from prior work on encoder models (BERT) and encoder-decoder models (T5), but does not test whether the findings generalize across model scales, architectures, or training procedures.

The seven capability categories (Coding, Math, Linguistic, Knowledge, Translation, Ethics and Moral, Writing) are constructed by manually grouping fine-grained labels from Infinity-Instruct (Table 3). The paper does not validate these groupings—e.g., whether "Python Programming" and "Java Programming" genuinely share neural machinery or whether they should have been analyzed as separate capabilities. This is a coarse abstraction that could mask important within-category specialization structure.

The consequence. The three core empirical findings—sparse activation, functional specialization, and functional partitioning—may not generalize to: (1) models at substantially different scales (the sparsity and specialization patterns in a 1B or 70B parameter model could differ from the 7–8B models tested); (2) models with different activation functions (ReLU-based models enforce sparsity through hard zeros, potentially changing the specialization landscape); (3) base models without instruction tuning (alignment could concentrate or diffuse specialization in ways that pre-trained base models do not exhibit); (4) non-MATH/code/linguistic capabilities that were not tested (the paper's strongest specialization results are for Coding, Math, and Translation—capabilities involving formal structure—and it is unclear whether capabilities like creative writing, humor generation, or multi-step planning would show similar neural concentration); or (5) capabilities defined at different granularities (the seven coarse categories may not correspond to the "natural" functional units the model actually organizes around).

What evidence exists in the paper. None. The paper does not test any model outside the 7–8B decoder-only range, does not test on any dataset other than Infinity-Instruct, does not validate the seven-category grouping, and does not analyze within-category specialization (e.g., Python vs. Java, basic arithmetic vs. calculus). The paper does report that findings are "similar across different layers and two decoder-only models with different training data" (Section 4.3), but this establishes consistency across two architecturally similar models, not generalizability to different model families, scales, or tasks.

Mitigation status. The paper does not claim broader generalization—it presents the findings as observations about the specific models tested. However, the Abstract, Section 4 introduction, and conclusion use language that implies general relevance ("[We] conduct an empirical analysis on widely-used LLMs... We find that the FFN layers follow modular patterns with functional specialization of neurons"). The open problems section (5.3) discusses evaluation of configurable foundation models as a future direction but does not identify the limited empirical scope as a limitation to be addressed. Expanding the analysis to diverse models and capabilities, and validating the FuncScore metric's cross-dataset reliability, would substantially strengthen the empirical foundation.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not introduce a new model, training algorithm, or benchmark. Its contribution is conceptual infrastructure: it provides a unified vocabulary and taxonomic framework for phenomena that were previously studied in fragmented sub-communities using incompatible terminology. The shift is therefore not a paradigm overthrow but a consolidation and reframing—it takes observations about knowledge neurons (Dai et al., 2022), MoE expert specialization (Zhang et al., 2023c), LoRA efficiency (Hu et al., 2022), knowledge editing (Meng et al., 2022), and multi-model orchestration (Alayrac et al., 2022) and shows they are all manifestations of the same underlying principle: LLMs can be understood as compositions of functional bricks that can be routed, combined, updated, and grown.

The magnitude of this contribution should be understood in terms of coordination value rather than technical novelty. Before this paper, a researcher studying how to surgically edit factual knowledge in FFN neurons and a researcher building multi-agent LLM systems had no shared language to describe their work as related. The former was doing "knowledge editing" or "mechanistic interpretability"; the latter was doing "multi-agent systems" or "tool-augmented LLMs." The brick framework reveals that both are performing an updating operation—one at solitary-neuron granularity on emergent bricks, the other at full-model granularity on customized bricks. This unification makes it possible to ask cross-cutting questions that were previously invisible: What properties should a good updating operation have, regardless of granularity? What design choices in brick construction make routing more efficient? How do emergent and customized bricks interact when both are present?

This framework also reconciles apparent contradictions in the literature, though at the conceptual rather than empirical level. The observation that some capabilities are sharply localized (knowledge neurons, skill neurons) while others appear diffusely distributed is not a contradiction but a reflection of capability-dependent specialization strength—Math and Translation neurons achieve FuncScores exceeding 0.8 while Knowledge and Ethics neurons score 0.5–0.65 (Figure 8). The framework accommodates this by treating specialization as a continuous property that varies across capabilities and models, rather than a binary property that either holds or doesn't.

The most important directional impact is that the paper redirects attention from monolithic scaling to modular engineering. If the brick premises hold—and the Section 4 experiments suggest they do, at least for the models and capabilities tested—then the primary bottleneck for efficient, adaptable LLMs is not making models larger but rather developing the infrastructure for brick operations: sparse computing frameworks that can dynamically activate only relevant bricks (Section 5.4), universal protocols for brick construction that enable collaborative development (Section 5.2), and evaluation methodologies that assess models at the brick level rather than through end-to-end black-box metrics (Section 5.3). This reframes the efficiency problem: the goal is not just to compress models or distill them into smaller ones, but to build architectures where modularity is a first-class property that can be exploited systematically.

Follow-Up Research This Work Enables

Cross-validated neuron specialization with differential FuncScore to filter resident neurons. The paper's FuncScore metric conflates genuine capability-specific activation with constitutively high baseline activation (resident neurons), causing the Knowledge specialization analysis in Llama-3 and the Translation analysis in Mistral-7B to show unacceptably high off-diagonal degradation (Figure 9). A direct follow-up would define a differential FuncScore: FuncScore_diff(n, f) = FuncScore(n, f) − FuncScore(n, all), where FuncScore(n, all) measures the neuron's average precision for separating any input from a random baseline—effectively penalizing neurons that are active across most inputs. Re-running the perturbation analysis with this metric on the same 7,000-instance Infinity-Instruct subset, with neurons selected on a random 70% split and tested on the held-out 30%, would determine whether the problematic off-diagonal degradation disappears. A strong result would show diagonal perplexity increases remaining high (100%+) while off-diagonal effects drop substantially below current values (e.g., from 79–184% to <20% for Knowledge in Llama-3), establishing that clean specialization is achievable with better neuron selection. This experiment directly addresses the paper's acknowledged limitation around resident neurons and requires no new data or models.

Within-capability specialization analysis to determine the natural granularity of functional bricks. The paper groups fine-grained Infinity-Instruct labels into seven coarse categories (Table 3: Coding includes Python, SQL, Java, C++, JavaScript, C#, plus object-oriented programming and code writing). It is unknown whether "Coding specialization" reflects genuine cross-language neural sharing or is an artifact of coarse labeling—different neurons could specialize in different programming languages, with the apparent Coding specialization emerging only because all Coding instances are pooled. A natural follow-up would compute FuncScores at the sub-capability level: for the 1,000 Coding instances, separate Python Programming instances from SQL Programming instances and compute per-language FuncScores for all neurons. If the same neurons score highly for both Python and SQL, that suggests genuine cross-language Coding specialization at the neuron-group level. If different neurons specialize in different languages, the natural brick granularity for Coding is sub-language, and the seven-category grouping is too coarse to capture the functional structure. This experiment uses existing data and models but requires no new annotation—the fine-grained labels already exist in Infinity-Instruct. The result would directly inform the granularity selection principles discussed in Section 2.3.

Sparse inference with per-token dynamic brick activation on a real generation task. The paper demonstrates that 70–80% of neurons can be globally masked with negligible loss increase (Figures 6, 7), but this is an average result masking the same neurons for all inputs. The brick framework's efficiency promise depends on per-input dynamic sparsity: different inputs should activate different subsets of neurons based on the capabilities they require. A critical follow-up would implement a prototype sparse inference system that, for each input token at each FFN layer, computes which neurons to activate based on a lightweight predictor (e.g., a small learned gate trained on the FuncScore-identified specialized neurons), executes only those neurons, and measures both the accuracy on a standard benchmark (MATH or HumanEval for coding, to ground in capabilities where specialization was strongest) and the actual wall-clock speedup on GPU hardware. The question is whether the theoretical sparsity ceiling (~80% neurons maskable) translates to practical speedups given the overhead of the gating computation and the irregular memory access patterns of sparse activation. Even a 2× speedup with negligible accuracy loss on Coding and Math tasks would validate the practical efficiency claim; a finding that the gating overhead negates the sparsity gains would redirect the research agenda toward the efficient computing frameworks discussed in Section 5.4.

Combining emergent and customized bricks for simultaneous task adaptation and knowledge injection. The paper treats emergent bricks (arising during pre-training) and customized bricks (added post-training) as conceptually parallel but does not empirically study their interaction. A concrete follow-up experiment: take Llama-3-8B-Instruct, identify the top-5% Math-specialized neurons using the FuncScore methodology (emergent bricks), then train a LoRA module (customized brick) for a new mathematical capability—e.g., solving problems in a specific notation system not seen during training—and measure whether the LoRA module's effective parameter subspace overlaps with the Math-specialized neurons. The prediction from the framework is that the customized brick should predominantly interact with the emergent Math bricks rather than with random neurons, because the new capability builds on existing mathematical reasoning circuits. One could test this by analyzing the gradient flow during LoRA training: if the LoRA parameters receive stronger gradient signals from the emergent Math neurons than from other neurons, that would demonstrate functional alignment between emergent and customized bricks. This experiment directly addresses the open problem identified in Section 5.1 about the correlation between emergent and customized bricks.

Layer-wise brick attribution for complex multi-step reasoning. The paper's FuncScore analysis (Figure 8) reports scores per layer but does not analyze whether different layers contribute to different stages of reasoning—e.g., early layers for problem parsing, middle layers for computation, late layers for answer formatting. A follow-up would extend the FuncScore methodology to measure layer-wise specialization for reasoning sub-stages, using a dataset like GSM8K where solutions have natural intermediate steps. For each solution step, one could compute the mean activation of Math-specialized neurons (identified from the Infinity-Instruct analysis) in each layer and test whether the activation pattern shifts systematically across layers as the solution progresses. If Math-specialized neurons in layers 15–20 activate predominantly during the computation stage while layers 25–30 activate during answer formatting, this would provide a functional map of the reasoning pipeline and suggest that brick operations (routing, updating) should be layer-specific rather than model-wide. This would also inform the cross-layer organization challenge identified in Section 2.1.3.

Replication on models with different activation functions to test whether sparsity is architecture-driven. The paper's sparse activation analysis (Section 4.3) uses SiLU-based models (Llama-3, Mistral) and introduces output magnitude as a metric specifically because SiLU produces near-zero but non-zero activations. A direct test of whether the observed sparsity patterns are fundamentally architectural or training-driven would replicate the full analysis pipeline (activation distribution, masking experiment, FuncScore, perturbation) on a ReLU-based decoder-only model at comparable scale, if one exists, or on a ReLU-fine-tuned version of Llama-3 (following Mirzadeh et al., 2023). The prediction is that ReLU models should show even sharper sparsity (since inactive neurons have exactly zero activation, not just near-zero), but the functional specialization patterns should be similar—because specialization arises from the training data distribution and optimization dynamics, not the activation function. If ReLU models instead show weaker or different specialization, that would challenge the framework's claim that modularity is a general emergent property rather than an artifact of specific architectural choices.

Practical Applications and Downstream Use Cases

On-device LLM deployment with capability-aware neuron pruning. The paper's finding that up to 80% of neurons can be masked with negligible loss when using output magnitude as the indicator (Figures 6d, 7d) directly motivates a compression strategy for deploying LLMs on smartphones and laptops (the deployment trend identified in Section 1). Rather than using uniform compression (quantization, distillation, or unstructured pruning) that treats all parameters identically, a capability-aware deployment could profile the target user's typical query distribution—if a user primarily asks coding questions, Math and Translation neurons can be aggressively pruned while preserving Coding neurons at higher density. The Section 4 perturbation results provide the feasibility evidence: pruning Coding neurons degrades Coding performance by 112% (Llama-3) while affecting Math by only 8% and Translation by 10% (Figure 9a). A production system would: (1) profile neuron importance per capability using the FuncScore methodology on calibration data, (2) accept a user-specified capability profile, (3) prune neurons with low importance for the target capabilities while preserving high-importance neurons, (4) deploy the resulting sparse model. The paper does not provide the end-to-end accuracy-vs-sparsity curves per capability that would be needed to implement this, but the experimental framework is directly transferable.

Targeted model updates for knowledge correction without full fine-tuning. The locate-and-update paradigm described in Section 3.3 for knowledge editing (Meng et al., 2022) currently operates on emergent knowledge bricks. The paper's demonstration that neurons for specific capabilities are largely disjoint (Figure 10: most cross-functionality overlaps near 0.00–0.04) provides evidence that targeted updates to, say, a factual error about a programming language (Coding neurons) are unlikely to interfere with the model's translation or ethical reasoning capabilities. This has immediate implications for deployment pipelines that need to correct factual errors or remove outdated knowledge: rather than running expensive full-model fine-tuning or RLHF alignment that risks regressions across the board, a targeted update procedure could (1) identify the capability category of the knowledge to be corrected, (2) use FuncScore to locate the top-k% neurons most associated with that capability, (3) apply a rank-one weight update (Meng et al., 2022) constrained to those neurons only, (4) verify that other capabilities are unaffected using the perturbation test methodology from Figure 9. The paper does not implement this pipeline, but the specialization and partitioning results make it a plausible near-term application.

Efficient multi-task model serving through brick retrieval and composition. Cloud LLM providers serving diverse user queries currently load the full model for every request, regardless of whether the user is asking a coding question, requesting a translation, or seeking creative writing help. The paper's demonstration that different capabilities activate different neuron populations (Figure 10: near-zero overlap) combined with the sparse activation findings (80% of neurons maskable) suggests an architecture where multiple lightweight task-specific "heads" (customized bricks implemented as LoRA modules or adapters) are stored on disk and loaded on demand, while the main model is served with capability-aware neuron sparsity. For a Translation request, only Translation-specialized emergent neurons + the Translation LoRA module would be active; for a Coding request, a different neuron subset + the Coding LoRA. The 4× efficiency improvement over uniform best-of-N reported in the paper's compute-optimal test-time scaling experiments (though derived from search experiments on MATH rather than from the brick framework itself) provides a quantitative target: if brick-aware serving can achieve even a 2× reduction in active parameters per query with negligible accuracy loss, the cost savings at scale would be substantial. The key missing piece—which the paper identifies as an open problem in Section 5.2—is a protocol for constructing these task bricks in a way that guarantees composability without interference.

When to Prefer This Method

The paper does not position the brick framework against specific named alternatives with clear, empirically tested tradeoffs—it is a conceptual synthesis, not an algorithmic method. It therefore does not provide the kind of decision rule that would appear in a methods paper ("prefer beam search over best-of-N on medium-difficulty problems" or "prefer test-time compute over pretraining when R ≪ 1"). The appropriate "preference" question is not about choosing the brick framework over another technique but rather about when a modular perspective is likely to be productive for analyzing or engineering an LLM system.

The findings suggest that a brick-oriented approach is most productive when: (1) the model exhibits the prerequisite properties—sparse activation, functional specialization, and functional partitioning—which the paper demonstrates for Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.3 on the seven tested capability types; (2) the target capabilities are of the type where specialization was strongest (Coding, Math, Translation achieving FuncScores > 0.8, Figure 8) rather than those where specialization was weak or contaminated by resident neurons (Knowledge, Ethics showing off-diagonal degradation in Figure 9); (3) the engineering goal is efficiency through selective parameter activation, targeted knowledge updating without full retraining, or multi-capability composition from reusable components; and (4) the necessary infrastructure for sparse computation (Section 5.4) or standardized brick interfaces (Section 5.2) either exists or is worth the investment to build. The brick framework is less directly useful when the model does not exhibit clean functional specialization for the target capability, when the capability requires genuine novelty beyond the base model's competence (analogous to the hard-problem boundary in the compute-optimal test-time scaling analysis), or when the deployment scenario cannot tolerate the overhead of brick management operations relative to simply serving the full monolithic model.