Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics

🎯 Pitch

Local fine-tuning, LoRA, and activation steering are not fundamentally different techniques but mathematically identical instances of dynamic weight updates, leading to a universal preference–utility trade-off that explains why making a model more aligned inevitably degrades its general capabilities.

1. Executive Summary

This paper proposes a unified dynamic weight update framework that reveals how diverse LLM control methods—local weight fine-tuning, LoRA, and activation steering—share a common mathematical form and exhibit consistent preference–utility dynamics across intervention strengths. Analyzing the Gemma-2-9B-IT and Qwen-2.5-7B-Instruct models on the AxBench, PowerSeeking, and Psychopathy benchmarks, the authors introduce a preference–utility analysis that measures concept preference and task utility on a shared log-odds scale using polarity-paired contrastive examples, and an activation manifold hypothesis (modeling validity decay via a rational quadratic function) to explain the observed three-phase preference trajectory and monotonic utility decline, achieving curve fits with R² > 0.95 across most settings. Guided by this mechanistic understanding, the paper introduces SPLIT (Steering with Preference–Utility Intervention), a joint optimization objective that improves preference while preserving utility, establishing that the preference–utility trade-off follows predictable geometric regularities but that complete avoidance of utility degradation remains unachievable under extreme control strengths.

2. Context and Motivation

The Core Problem: LLM Control Methods Are Fragmented and Poorly Understood

The fundamental problem this paper addresses is conceptual fragmentation in the study of large language model control. Researchers have developed a wide range of techniques for steering LLM behavior toward desired attributes — training-time methods like local weight fine-tuning and parameter-efficient approaches (LoRA), and inference-time methods like activation steering that add direction vectors to hidden states. Each of these methodological families originated in different research communities, operates under different assumptions, and uses different evaluation protocols. The result is a field where:

"approaches are often studied in isolation, under different assumptions, objectives, and evaluation protocols. This fragmentation hinders rigorous comparison and obscures shared failure modes." (Section 1)

This fragmentation is not merely an aesthetic concern about taxonomy. It has concrete practical consequences. When practitioners want to control an LLM's behavior — say, to make it more helpful, less toxic, or more aligned with a particular value — they face a bewildering array of choices with no principled way to decide between them. Should they fine-tune weights? Use LoRA? Apply an activation steering vector? The answer depends on factors like compute budget, data availability, and desired control precision, but there is no shared framework for comparing these methods on equal footing or predicting how they will behave as control strength increases.

Why This Problem Matters

The paper's motivation rests on three practical and theoretical imperatives:

1. Growing deployment demands for safety and controllability. As LLMs move from research prototypes into production systems — customer service, content moderation, medical advice, code generation — the need for reliable behavioral control becomes paramount (Section 1). A model that occasionally produces toxic outputs or violates content policies is not merely an academic concern; it represents a real business and safety risk. The paper frames this as a "central challenge" (Section 1), and it is one that intensifies as models are deployed in higher-stakes settings.

2. The preference–utility trade-off is poorly characterized. A particularly thorny challenge in controlled generation is what the paper calls the preference–utility trade-off (Section 1, Figure 1 right panel). As you increase the strength of a control intervention to amplify a target concept (say, making outputs more "positive" in sentiment), the model's general capability — its ability to produce coherent, task-relevant, instruction-following text — predictably degrades. The paper describes this as a key failure mode:

"as control strength increases, the target attribute is amplified, but undesirable side effects—such as incoherence, instruction violations, or context drift—also become more frequent, reducing overall task effectiveness" (Section 1)

This degradation is particularly insidious because it can confound evaluation: if you judge control quality by looking at outputs, the utility degradation makes it harder to tell whether the control signal is actually working. A model that produces garbled text under strong steering might be experiencing concept amplification plus utility collapse, or it might be experiencing a failure of the steering signal entirely. Without a way to separate these effects, evaluation is ambiguous.

3. The mechanisms underlying control are opaque. Prior work had established that activation steering works — adding a vector to hidden states can shift behavior in predictable ways — but the why remained unclear. Why does steering work at small scales but degrade at large scales? Why do different intervention methods (weight updates, LoRA, activation vectors) seem to exhibit similar diminishing returns? Without a mechanistic understanding, improvements are driven by trial-and-error rather than principled design. The paper explicitly aims to provide this mechanistic account, using it to guide the design of a better intervention method (SPLIT).

Prior Approaches and Their Limitations

The paper identifies several strands of prior work and their specific shortcomings:

Activation steering methods (Rimsky et al., 2024; Turner et al., 2023; Han et al., 2024) build on the linear representation hypothesis: the idea that abstract concepts correspond approximately to linear directions in a model's activation space. By computing a "steering vector" — often the difference between activations for positive and negative examples of a concept — and adding it to hidden states during inference, researchers can shift the model's behavior toward or away from that concept. This approach is attractive because it requires no training (if the vector is computed from existing data) and can be applied at inference time without modifying model weights.

However, the paper notes two key limitations of this line of work. First, the linearity is only approximate and breaks down at larger intervention strengths. Bigelow et al. (2025) showed that steering yields an approximately linear trend in posterior odds, but "mainly in the small-scale regime." More recent work (Bricken et al., 2023; Wollschläger et al., 2025) reports "non-monotonic or adverse effects under stronger steering, challenging a naive global linearity assumption" (Section 6). Second, the coarse-grained nature of activation steering leads to utility degradation — shifting entire activation patterns disrupts other capabilities encoded in the same representations (Wang et al., 2025; Wu et al., 2025a). Some recent work has tried to address this by introducing explicit preference learning objectives to optimize steering vectors (Cao et al., 2024; Wu et al., 2025b), but the fundamental trade-off between preference and utility remains poorly understood.

Parameter-efficient fine-tuning (PEFT) methods, particularly LoRA (Hu et al., 2022a), have shown that effective LLM adaptation does not require updating all parameters. LoRA freezes the original weights and trains low-rank matrices that are merged at inference time. This is computationally efficient and can match full fine-tuning performance on many tasks. Local weight fine-tuning goes further, showing that even modifying a small subset of parameters in specific layers is sufficient to change model behavior (Zaken et al., 2022; Geva et al., 2021). These methods are powerful but are studied separately from activation steering, making it difficult to understand when to prefer one approach over another or whether they share underlying dynamics.

Prior attempts at unification have been limited. The paper acknowledges that representation-manifold work provides a "complementary geometric lens for understanding steering and its limitations" (Modell et al., 2025; Li and He, 2025; Xie et al., 2025) but notes that these perspectives have not been connected to a unified framework that encompasses both parameter-based and activation-based interventions.

The paper also identifies a specific gap in prior training objectives for control. Methods like SFT (supervised fine-tuning on target behaviors) and RePS (Wu et al., 2025a) train interventions to shift model outputs, but they do not explicitly model or optimize the trade-off between concept preference and task utility. This means they can inadvertently produce interventions that achieve strong preference at the cost of unacceptable utility degradation, or that are too conservative and fail to achieve sufficient control.

How This Paper Positions Itself

The paper positions its contributions at three distinct levels:

At the conceptual level, it proposes a unified framework that is not "just another taxonomy." Rather than simply grouping methods by superficial similarities, it derives a common mathematical form — the dynamic weight update equation (Eq. 1): $\mathbf{h}_{i+1} = (\mathbf{W} + m_1 \Delta\mathbf{W})\,\mathbf{h}_{i} + (\mathbf{b} + m_2 \Delta\mathbf{b})$ — that captures local weight fine-tuning, LoRA, and activation steering as special cases differing only in how $\Delta\mathbf{W}$ and $\Delta\mathbf{b}$ are parameterized. This is not merely descriptive; it enables the paper to study all three method families within a single experimental framework, varying a common control parameter $m$ and observing how preference and utility respond.

At the analytical level, it introduces the preference–utility analysis that cleanly separates the two effects that intermix in typical output-based evaluations. By using polarity-paired contrastive examples — a positive-target answer $A_p$ and a negative-target answer $A_n$ for each query — the paper derives log-odds metrics (Eqs. 5 and 6) that measure preference (the model's relative inclination toward $A_p$ vs. $A_n$ ) and utility (the total probability mass assigned to either of the two valid completions) on a shared additive scale. This decomposition is crucial because it allows the paper to observe that preference and utility respond systematically — and differently — to changes in intervention strength, revealing a consistent preference–utility trade-off pattern that transcends individual method types (Figure 2).

At the mechanistic level, the paper introduces the activation manifold hypothesis to explain why this trade-off occurs. Rather than treating utility degradation as an unfortunate side effect to be empirically mitigated, the paper models it as a principled consequence of steering moving representations off the manifold of valid activations learned during pretraining. This hypothesis leads to a quantitative prediction — that utility decay follows a rational quadratic function of the steering factor $m$ (Eq. 12) — which the paper validates through curve fitting with $R^2 > 0.95$ across most settings (Table 2). This mechanistic understanding then directly motivates the design of SPLIT, a training objective that explicitly optimizes preference while preserving utility rather than treating utility preservation as an implicit or secondary goal.

The paper's positioning is thus: it is not proposing a fundamentally new control method (SPLIT is an optimization objective applicable to existing method families), but rather providing a unifying lens, a measurement framework, and a mechanistic explanation that together make the entire landscape of LLM control methods more predictable, comparable, and improvable. The goal, implicit throughout the paper, is to move the field from empirical trial-and-error toward principled, mechanism-aware intervention design.

3. Technical Approach

3.1 Reader Orientation

This paper is a mechanistic analysis and unified framework — not a new model architecture — that reveals how different LLM control methods share a common mathematical structure and exhibit predictable preference–utility dynamics. The "system" being studied is the collection of intervention techniques (local weight fine-tuning, LoRA, activation steering) themselves, and the paper's goal is to understand why they work similarly, why they degrade under strong control, and how to design better training objectives based on that understanding. The paper solves the problem of fragmented evaluation and opaque mechanisms by providing (1) a unified mathematical formulation that captures diverse methods as dynamic weight updates, (2) a paired-example measurement framework that cleanly separates preference from utility, and (3) a mechanistic activation-manifold model that quantitatively predicts how both quantities change with intervention strength, ultimately motivating a new training objective (SPLIT) that optimizes this trade-off explicitly.

3.2 Big-Picture Architecture (Diagram in Words)

The paper's analytical system has four integrated components:

Unified Intervention Formulation — Expresses local weight fine-tuning, LoRA, and activation steering as instances of a single affine transformation: $\mathbf{h}_{i+1} = (\mathbf{W} + m_1 \Delta\mathbf{W})\,\mathbf{h}_{i} + (\mathbf{b} + m_2 \Delta\mathbf{b})$ , where each method differs only in how $\Delta\mathbf{W}$ and $\Delta\mathbf{b}$ are parameterized.
Preference–Utility Measurement Framework — For each query, constructs polarity-paired answers ( $A_p$ positive concept, $A_n$ negative concept) and computes two log-odds metrics from sequence cross-entropy losses: PrefOdds (the difference $\mathcal{L}_n - \mathcal{L}_p$ , measuring concept inclination independent of utility) and UtilOdds (measuring the total probability mass assigned to the matched pair, capturing task coherence independent of concept).
Activation Manifold Hypothesis and Validity Decay Model — Posits that pretraining induces a manifold of valid representations in activation space. Steering moves activations along a line; preference grows from projection onto a target direction, while utility degrades from off-manifold deviation. This degradation is modeled via a rational quadratic (RQ) decay function $D(m)$ parameterized per direction and sign of intervention.
SPLIT Training Objective — A joint optimization loss that combines a utility-preserving term ( $\mathcal{L}_{util}$ , cross-entropy on both $A_p$ and $A_n$ ) with a preference-enhancing term ( $\mathcal{L}_{pref}$ , a hinge loss maximizing $\mathcal{L}_n - \mathcal{L}_p$ ), designed to extend the linear preference regime while delaying utility collapse.

Information flows as follows: a query $q$ enters → the base model (with or without intervention parameters $\Delta\mathbf{W}, \Delta\mathbf{b}$ ) computes log-probabilities for $A_p$ and $A_n$ → the measurement framework extracts $\mathcal{L}_p, \mathcal{L}_n$ → these are transformed into PrefOdds and UtilOdds → the manifold model fits parametric curves to these log-odds as functions of intervention multiplier $m$ → the SPLIT objective uses the same $\mathcal{L}_p, \mathcal{L}_n$ signals during training to optimize intervention parameters.

3.3 Roadmap for the Deep Dive

First, the unified dynamic weight formulation (Eq. 7, Table 1), because it is the mathematical substrate on which everything else rests — it shows what is shared across methods and how the intervention multiplier $m$ enters.
Second, the preference–utility measurement framework (Eqs. 4–6), because it establishes the two dependent variables (PrefOdds, UtilOdds) that the rest of the paper tracks and models as functions of $m$ .
Third, the activation manifold hypothesis and validity decay model (Assumptions 4.1–4.2, Eq. 12), because it provides the mechanistic explanation for why PrefOdds and UtilOdds exhibit their characteristic shapes.
Fourth, the preference projection-with-decay model (Eqs. 13–15), which combines the manifold decay factor with a linear projection term to quantitatively predict PrefOdds(m).
Fifth, the utility decay-only model (Eqs. 16–17), which models UtilOdds(m) as driven solely by validity decay (no projection term), reflecting the assumption that preference steering directions are approximately orthogonal to utility directions.
Sixth, the SPLIT training objective (Eqs. 18–20), which translates the mechanistic insight — that preference and utility can be separately modeled and optimized — into a concrete optimization procedure that improves preference while preserving utility.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a mechanistic analysis paper whose core idea is that diverse LLM interventions share a unified mathematical form and that their effects on preference and utility can be quantitatively predicted from an activation-manifold model, enabling the design of better training objectives.

The Unified Dynamic Weight Formulation

The paper's foundational contribution is the observation that local weight fine-tuning, LoRA, and activation steering can all be expressed as a single affine transformation with dynamic weight updates during inference (Section 3.2, Table 1). The canonical forward pass through a linear layer in a transformer is:

$\mathbf{h}_{i+1} = \mathbf{W}\mathbf{h}_{i} + \mathbf{b}$

where $\mathbf{h}_{i} \in \mathbb{R}^{d_{\text{in}}}$ is the input representation, $\mathbf{W} \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$ is the weight matrix, $\mathbf{b} \in \mathbb{R}^{d_{\text{out}}}$ is the bias vector, and $\mathbf{h}_{i+1}$ is the output. The paper proposes that all intervention methods modify this to:

$\mathbf{h}_{i+1} = (\mathbf{W} + m_1 \Delta\mathbf{W})\,\mathbf{h}_{i} + (\mathbf{b} + m_2 \Delta\mathbf{b})$

where $\Delta\mathbf{W}$ and $\Delta\mathbf{b}$ are update terms specific to each method, and $m_1$ , $m_2$ are scalar coefficients controlling intervention magnitude.

What each method instantiates:

Local weight fine-tuning updates both $\mathbf{W}$ and $\mathbf{b}$ with learned deltas: $\Delta\mathbf{W}$ is a full-rank matrix of size $d_{\text{out}} \times d_{\text{in}}$ , and $\Delta\mathbf{b}$ is a vector of size $d_{\text{out}}$ . The number of trainable parameters is $d_{\text{in}} \times d_{\text{out}} + d_{\text{out}}$ . During training, only the down-projection layer of the MLP is updated (Section 2.2); all other parameters remain frozen. The multiplier $m$ can be set to 1 (standard inference) or swept to study scaling behavior.
LoRA (Low-Rank Adaptation) freezes $\mathbf{W}$ and $\mathbf{b}$ and introduces a factored update: $\Delta\mathbf{W} = \mathbf{B}\mathbf{A}$ , where $\mathbf{B} \in \mathbb{R}^{d_{\text{out}} \times r}$ and $\mathbf{A} \in \mathbb{R}^{r \times k}$ with rank $r \ll \min(d_{\text{in}}, d_{\text{out}})$ . The bias term is left unchanged ( $m_2 = 0$ effectively). The number of trainable parameters is $d_{\text{in}} \times r + r \times d_{\text{out}}$ , which is much smaller than a full weight update. At inference, the adapted weight is $\mathbf{W}' = \mathbf{W} + m\,\mathbf{B}\mathbf{A}$ .
Activation steering (vector) keeps $\mathbf{W}$ fixed ( $m_1 = 0$ ) and only modifies the bias: $\Delta\mathbf{b}$ is a steering vector $\mathbf{v}$ (size $d_{\text{out}}$ ) computed from data (e.g., via DiffMean, which takes the difference between mean activations for positive and negative examples). The intervention is $\mathbf{h}_{i+1} = \mathbf{W}\mathbf{h}_{i} + (\mathbf{b} + m\,\mathbf{v})$ . This is the smallest parameter footprint: only $d_{\text{out}}$ values are needed.

What this unification enables: by expressing all methods in the same affine form, the paper can study them with a common experimental protocol. For any intervention type, the behavior of the model is a function of the scalar multiplier $m$ (when symmetric, $m_1 = m_2 = m$ ). The paper sweeps $m$ across a range (typically from negative to positive values) and measures preference and utility at each point, producing the characteristic curves shown in Figure 2. This would be impossible if each method were analyzed under its own bespoke formalism.

Equivalent activation-change perspective: the paper further notes that the effect of any weight update can be equivalently viewed through its impact on the activation at that layer:

$\Delta\mathbf{h} = m_1\,\Delta\mathbf{W}\,\mathbf{h}_{i} + m_2\,\Delta\mathbf{b}$

This expression captures how much the hidden state changes as a result of the intervention, and it is this $\Delta\mathbf{h}$ that propagates through the rest of the network and ultimately changes the output distribution. The three methods differ in the dimensionality and structure of $\Delta\mathbf{h}$ : local weight updates produce an input-dependent $\Delta\mathbf{h}$ (since it multiplies the weight delta by the specific input $\mathbf{h}_i$ ), LoRA produces a low-rank input-dependent $\Delta\mathbf{h}$ , and vector steering produces an input-independent $\Delta\mathbf{h}$ (a fixed bias offset). Table 1 summarizes these relationships.

Why this formulation matters beyond taxonomy: the explicit introduction of scaling coefficients $m_1, m_2$ allows continuous control over intervention strength — a capability that "plays a central role in our subsequent analysis" (Section 3.2). By sweeping $m$ , the paper can observe not just whether an intervention works, but how its effects scale from weak to strong, revealing the three-phase pattern for preference and the monotonic decline for utility that are the paper's primary empirical findings.

The Preference–Utility Measurement Framework

The paper's second innovation is a paired-example measurement framework that cleanly separates two effects that are typically conflated in output-based evaluations (Section 3.1).

Constructing polarity pairs. For each query $q$ in the evaluation dataset, the paper constructs a pair of completions: a concept-positive answer $A_p$ (exhibiting the target concept, e.g., a positive restaurant review) and a concept-negative answer $A_n$ (exhibiting the opposite concept, e.g., a negative restaurant review). These answers are drawn from the dataset's ground-truth completions. The model's behavior is then measured by how much probability mass it assigns to each of these two specific answer strings, given the query.

Key insight — preference–utility independence assumption. The paper assumes that, for a given query $q$ , concept preference and task utility are independent latent factors (Section 3.1). This is formalized as:

$P(A_p \mid q) = P(u \mid q)\,P(p_p \mid q)$ $P(A_n \mid q) = P(u \mid q)\,P(p_n \mid q)$

where $P(u \mid q)$ is the latent utility probability — the probability that the model produces any task-valid completion (whether concept-positive or concept-negative) — and $P(p_p \mid q)$ , $P(p_n \mid q)$ are latent preference probabilities satisfying $P(p_p \mid q) + P(p_n \mid q) = 1$ .

What this factorization means in operational terms: the model's overall probability of generating a specific answer string is the product of (1) whether it is "capable" of producing a valid completion at all, and (2) which concept direction it leans toward, given that it produces a valid completion. The utility factor $P(u \mid q)$ is shared across the polarity pair, meaning it cancels out when we take ratios. This is the crucial move that allows separate measurement of preference and utility.

Computing sequence losses. For each answer string $A = (y_1, \dots, y_T)$ (excluding query/prompt tokens), the paper computes the sequence cross-entropy loss under teacher forcing:

$\mathcal{L}(A \mid q) = -\log P(A \mid q) = -\sum_{t=1}^{T} \log P(y_t \mid q, y_{<t})$

where $P(y_t \mid q, y_{<t})$ is the model's predicted probability for the correct next token $y_t$ given the query and all previous answer tokens. This is the standard language modeling loss: it sums the negative log-probabilities of each token in the answer sequence, conditioned on all prior tokens. When $A_p$ and $A_n$ have different lengths, the paper optionally uses the mean loss $\bar{\mathcal{L}}(A \mid q) = \mathcal{L}(A \mid q) / T$ to reduce length effects (Appendix C.2). The two specific losses of interest are:

$\mathcal{L}_p = \mathcal{L}(A_p \mid q), \quad \mathcal{L}_n = \mathcal{L}(A_n \mid q)$

Preference log-odds. Taking the ratio of the two factorized probabilities cancels $P(u \mid q)$ :

$\frac{P(A_p \mid q)}{P(A_n \mid q)} = \frac{P(p_p \mid q)}{P(p_n \mid q)}$

Taking the log and substituting $P(A \mid q) = e^{-\mathcal{L}}$ :

$\text{PrefOdds}(q) \triangleq \log\frac{P(p_p \mid q)}{P(p_n \mid q)} = \log\frac{e^{-\mathcal{L}_p}}{e^{-\mathcal{L}_n}} = \mathcal{L}_n - \mathcal{L}_p$

What it computes: PrefOdds measures the model's relative inclination toward the concept-positive answer versus the concept-negative answer, independent of utility. If the model assigns high probability to $A_p$ and low probability to $A_n$ , $\mathcal{L}_p$ will be small (low surprise at seeing the positive answer) and $\mathcal{L}_n$ will be large (high surprise at seeing the negative answer), so $\mathcal{L}_n - \mathcal{L}_p$ will be large and positive — indicating strong positive-concept preference. If the model assigns equal probability to both, $\mathcal{L}_n \approx \mathcal{L}_p$ and PrefOdds $\approx 0$ . If the model favors the negative answer, PrefOdds becomes negative.

Why this form: the subtraction $\mathcal{L}_n - \mathcal{L}_p$ is equivalent to a log-odds ratio because exponentiating and negating a cross-entropy loss recovers the model's probability. The difference of losses directly gives the log-ratio of probabilities, which is the natural parameter for a binary preference comparison. This avoids having to estimate $P(u \mid q)$ separately and gives PrefOdds the convenient property of being on an additive scale: a increase of 1 in PrefOdds corresponds to multiplying the preference odds by $e \approx 2.718$ .

Utility log-odds. Summing the factorized probabilities and using $P(p_p \mid q) + P(p_n \mid q) = 1$ :

$P(A_p \mid q) + P(A_n \mid q) = P(u \mid q) \cdot [P(p_p \mid q) + P(p_n \mid q)] = P(u \mid q)$

So the latent utility probability is simply the total probability mass assigned to the matched pair. This is converted to log-odds for consistency with the preference scale:

$\text{UtilOdds}(q) \triangleq \log\frac{P(u \mid q)}{1 - P(u \mid q)} = \log\frac{e^{-\mathcal{L}_p} + e^{-\mathcal{L}_n}}{1 - e^{-\mathcal{L}_p} - e^{-\mathcal{L}_n}}$

What it computes: UtilOdds measures how much of the model's probability budget is spent on either of the two valid completions, as opposed to being spent on incoherent, off-topic, or otherwise invalid tokens. A high UtilOdds means the model is strongly committed to producing one of the two valid answers — it is "on task." A low UtilOdds means the model is spreading probability mass across invalid continuations — it is "off task" or generating noise.

Why this form: the denominator $1 - e^{-\mathcal{L}_p} - e^{-\mathcal{L}_n}$ is the probability mass assigned to all other possible completions besides $A_p$ and $A_n$ . The log-odds transformation maps the bounded probability $P(u \mid q) \in (0, 1)$ to an unbounded real number, making it easier to fit parametric curves and compare changes across different baseline utility levels. Like PrefOdds, it uses an additive log-odds scale.

What makes this framework powerful: typical output-based evaluation judges an intervention by looking at the generated text — does it express the right concept? Is it coherent? These two judgments are conflated because a model that loses utility will produce garbled outputs that can't be assessed for concept preference. The paired-example log-odds framework bypasses this by measuring the model's internal probability assignments to known correct completions, separating the two dimensions before any text is generated. This allows the paper to observe that preference and utility respond differently to intervention strength (Figure 2): preference rises and then saturates, while utility peaks near $m=0$ and monotonically declines, patterns that would be invisible under output-only evaluation.

The Activation Manifold Hypothesis

The paper's mechanistic explanation for the observed preference–utility dynamics rests on two assumptions that together form the activation manifold hypothesis (Section 4.1).

Assumption 4.1 (Training-Induced Activation Manifold): "Pre-training induces a 'typical' region of activation space where representations concentrate for stably handled inputs" (Section 4.1). The paper posits that at each layer $l$ , there exists a set $\mathcal{M}_l \subset \mathbb{R}^{d_l}$ — the activation manifold — near which the representations of well-processed inputs tend to lie. This is not a precisely defined geometric object; rather, it is an abstraction that captures the idea that the distribution of activations seen during pretraining is concentrated on a lower-dimensional structure. Representations far from $\mathcal{M}_l$ are "unfamiliar" to the subsequent layers of the network, which were trained to process activations drawn from the pretraining distribution.

Assumption 4.2 (Steering-Induced Validity Decay): Steering an activation at layer $l$ by adding $m\,\Delta h$ moves the representation along a one-dimensional line in activation space: $\tilde{h}_l(m) = h_l + m\,\Delta h$ . The paper introduces representation validity — a scalar measure of how "on-manifold" a steered representation is — and posits that this validity is highest at certain intersection points $m_i$ where the steering line crosses the manifold neighborhood, and decays monotonically as $m$ moves away from these intersection points. Formally, the paper models the average validity decay as a function $D(m)$ that "decreases as hidden states move away from [the manifold]" (Section 4.1).

Why a decay function is necessary, not just a distance measure: the concept of validity is an abstraction — the paper does not claim to know the exact geometry of $\mathcal{M}_l$ or to measure the distance to it directly. Instead, $D(m)$ is a phenomenological model that captures the observable consequence of off-manifold deviation: the downstream layers process the steered activation less reliably, leading to degraded decoding. This abstraction avoids committing to a specific manifold geometry while retaining the key implication needed for quantitative modeling.

The rational quadratic (RQ) parametrization. To give $D(m)$ a concrete functional form, the paper adopts the rational quadratic function, chosen for three properties (Section 4.1): (1) it is positive everywhere (validity cannot be negative), (2) it is smooth (validity changes continuously with $m$ ), and (3) it exhibits heavy-tailed, polynomial-rate decay with distance — widely used in kernel methods and Gaussian processes (Rasmussen, 2004) to model multi-scale attenuation.

The paper further notes that model steerability is often asymmetric — the model may respond differently to positive versus negative steering along the same direction (Miehling et al., 2025). To capture this, the paper uses a piecewise parametrization with separate parameters for $m \geq 0$ and $m < 0$ :

$D(m) = \begin{cases} \left(1 + \frac{(m - m_{+})^{2}}{L_{+}}\right)^{-p_{+}} & \text{if } m \geq 0 \\ \left(1 + \frac{(m - m_{-})^{2}}{L_{-}}\right)^{-p_{-}} & \text{if } m < 0 \end{cases}$

where:

$m_{+}$ is the signed distance from the unsteered activation point $P$ to the nearest on-manifold intersection $P_{+}$ in the positive steering direction (Figure 3, right panel). Typically, $m_{+} \approx 0$ if the unsteered activation is already on the manifold, but can be nonzero if the base model's representation starts slightly off-manifold for certain inputs.
$m_{-}$ is the analogous distance in the negative steering direction.
$L_{+}, L_{-}$ are characteristic length scales that control how quickly validity decays with distance from the intersection point. A large $L$ means the steering direction runs approximately parallel to the manifold — moving along it stays near valid regions for longer. A small $L$ means the direction cuts perpendicularly across the manifold — even small steps push the representation into invalid regions quickly.
$p_{+}, p_{-}$ are asymptotic decay rates (tail heaviness parameters). Larger $p$ means faster polynomial decay; smaller $p$ means heavier tails (validity persists further from the manifold).

How this function behaves: at the intersection point $m = m_{+}$ , the squared distance term is zero, so $D(m_{+}) = 1$ — validity is maximal. As $|m - m_{+}|$ grows, the term $\frac{(m - m_{+})^{2}}{L_{+}}$ increases, and the function decays toward zero at a rate governed by $p_{+}$ and $L_{+}$ . The RQ form decays polynomially (like $|m|^{-2p_{+}}$ for large $|m|$ ) rather than exponentially, which the paper implicitly argues matches the observed behavior better than a Gaussian decay.

Why this specific functional form, rather than, say, a simple exponential or Gaussian: the RQ kernel is a standard choice in Gaussian process regression for modeling phenomena with varying characteristic scales (Rasmussen, 2004). Its mixture-of-scales interpretation — it can be derived as a scale mixture of Gaussian kernels with different lengthscales — makes it suitable for the activation manifold setting where different input features or concept dimensions may have different sensitivity to steering. The piecewise form with separate parameters for positive and negative $m$ captures the empirically observed asymmetry in steering responses.

Preference Capability: Projection Gain with Validity Decay

The paper models how steering affects preference as the product of two factors: (1) a linear projection onto a preference direction in activation space, which grows with $m$ , and (2) the validity decay function $D(m)$ , which attenuates this projection as the representation moves off-manifold (Section 4.2).

The logistic preference model. Drawing on the linear representation hypothesis (LRH), the paper assumes that before any intervention, the model's preference probability follows a logistic form:

$P(p_p \mid h) = \sigma\!\left(-(\boldsymbol{\omega}_p^{\mathsf{T}} h + b_p)\right)$

where $\sigma(x) = 1/(1+e^{-x})$ is the sigmoid function, $\boldsymbol{\omega}_p \in \mathbb{R}^{d_l}$ is a preference direction vector in activation space, and $b_p$ is a scalar bias. The dot product $\boldsymbol{\omega}_p^{\mathsf{T}} h$ measures how far the current activation projects onto the preference direction — larger positive values indicate stronger preference for the positive concept, and the sigmoid maps this to a probability between 0 and 1.

Incorporating steering and validity decay. When a steering intervention replaces $h$ with $\tilde{h}(m) = h + m\,\Delta h$ , the projection becomes $\boldsymbol{\omega}_p^{\mathsf{T}}(h + m\,\Delta h) = \boldsymbol{\omega}_p^{\mathsf{T}} h + m\,(\boldsymbol{\omega}_p^{\mathsf{T}} \Delta h)$ . The paper's key insight is that this projection should be attenuated by the validity decay function, because off-manifold activations are decoded unreliably — the preference signal encoded in the activation is not faithfully transmitted to the output distribution. This yields:

$P(p_p \mid \tilde{h}(m)) = \sigma\!\left(-(\boldsymbol{\omega}_p^{\mathsf{T}} h + \alpha_p m)\,D_p(m) - b_p\right)$

where $\alpha_p \triangleq \boldsymbol{\omega}_p^{\mathsf{T}} \Delta h$ is the alignment coefficient — it measures how much the steering direction $\Delta h$ projects onto the preference direction $\boldsymbol{\omega}_p$ . When $\Delta h$ is perfectly aligned with $\boldsymbol{\omega}_p$ , $\alpha_p$ is large (maximizing the preference gain per unit of $m$ ); when $\Delta h$ is orthogonal to $\boldsymbol{\omega}_p$ , $\alpha_p = 0$ (steering does not affect preference at all).

Converting to preference log-odds. Applying the log-odds transformation to the steered probability:

$\log\frac{P(p_p \mid \tilde{h}(m))}{1 - P(p_p \mid \tilde{h}(m))} = (\boldsymbol{\omega}_p^{\mathsf{T}} h + \alpha_p m)\,D_p(m) + b_p$

The derivation is provided in Appendix C.3 (Eq. 30–31): starting from the logistic form, the log-odds equals the negative of the logit argument, so $-\eta = \boldsymbol{\omega}_p^{\mathsf{T}} h D_p(0) + b_p$ for the unsteered case, and the addition of $m\,\Delta h$ adds the $\alpha_p m$ term.

Fitting form for empirical measurements. For curve-fitting to measured data, the paper simplifies the expression by treating $\boldsymbol{\omega}_p^{\mathsf{T}} h$ as a per-example constant $\beta_p$ (since $h$ is fixed for a given input $q$ ) and absorbing $D_p(m)$ into the interaction:

$\log\frac{P(p_p \mid \tilde{h}(m))}{1 - P(p_p \mid \tilde{h}(m))} = (\alpha_p m + \beta_p)\,D_p(m) + b_p$

where each parameter has a clear operational meaning:

$\alpha_p$ : the slope of the preference projection — how much preference log-odds increases per unit of $m$ in the absence of decay ( $D_p(m) = 1$ ).
$\beta_p$ : the baseline preference log-odds at $m = 0$ , capturing the model's inherent concept bias for this particular input.
$b_p$ : a residual offset.
$D_p(m)$ : the validity decay function (Eq. 12), with its own set of parameters $(m_{+}, m_{-}, L_{+}, L_{-}, p_{+}, p_{-})$ that are fitted jointly.

What this model predicts about preference dynamics: for small $|m|$ , $D_p(m) \approx 1$ (near the manifold intersection), so PrefOdds $\approx \alpha_p m + \beta_p + b_p$ — a linear region where each unit increase in $m$ shifts preference log-odds by $\alpha_p$ . This matches the observation in Figure 2 that all methods exhibit an initial linear regime. As $|m|$ grows, $D_p(m)$ decays, causing the growth to slow — this is the transitional region. Eventually, $D_p(m)$ becomes very small, and further increases in $m$ produce negligible preference gains — the convergence region. The model thus captures the full three-phase pattern from a single mechanistic assumption (projection times decay).

Why this form, rather than a purely linear model: a naive linear model PrefOdds = $\alpha_p m + \text{const}$ would predict unbounded preference growth with $m$ , contradicting the empirical convergence observed in Figure 2. The validity decay factor $D_p(m)$ provides the saturation mechanism: as steering pushes representations too far from the manifold, the preference signal encoded in the activation is no longer reliably transmitted to the output, so further strengthening the intervention becomes futile.

Fit validation: Table 2 reports $R^2$ values for this model exceeding 0.95 across most settings (Gemma-2-9B-IT: average preference $R^2 = 0.96$ to $0.99$ across method types; Qwen-2.5-7B-Instruct: 0.96 to 0.99). The consistently high $R^2$ values validate that the projection-with-decay model accurately captures the shape of the measured preference curves, supporting the mechanistic hypothesis.

Utility Capability: Only Validity Decay

For utility, the paper makes a simpler model: unlike preference, utility has no projection term because steering directions are typically chosen to target specific concepts, not general task capability. The steering direction $\Delta h$ is assumed to be approximately orthogonal to any utility-relevant direction $\boldsymbol{\omega}_u$ , so $\boldsymbol{\omega}_u^{\mathsf{T}} \Delta h \approx 0$ (Section 4.3).

The pure decay model. Under this assumption, the steered utility log-odds is modeled as:

$\log\frac{P(u \mid \tilde{h}(m))}{1 - P(u \mid \tilde{h}(m))} = \boldsymbol{\omega}_u^{\mathsf{T}} h\,D_u(m) + b_u$

where $\boldsymbol{\omega}_u^{\mathsf{T}} h$ captures the input's baseline utility (how well the unsteered model handles this query), $D_u(m)$ is the validity decay function (with the same RQ form as Eq. 12 but possibly different parameters), and $b_u$ is an offset.

Fitting form:

$\log\frac{P(u \mid \tilde{h}(m))}{1 - P(u \mid \tilde{h}(m))} = \beta_u\,D_u(m) + b_u$

where $\beta_u = \boldsymbol{\omega}_u^{\mathsf{T}} h$ is the baseline utility log-odds (a per-example constant) and $D_u(m)$ has its own parameters $(m_{+}, m_{-}, L_{+}, L_{-}, p_{+}, p_{-})$ fitted from data.

What this model predicts about utility dynamics: at $m = 0$ (or at the nearest manifold intersection $m_i$ if the unsteered activation starts slightly off-manifold), $D_u(m) \approx 1$ and UtilOdds is near its maximum $\beta_u + b_u$ . As $|m|$ increases in either direction, $D_u(m)$ decreases, and UtilOdds monotonically declines. There is no mechanism for utility to increase with $|m|$ — steering can only maintain or degrade utility, never improve it. This matches the empirical observation in Figure 2 that utility curves are symmetric (or approximately symmetric) peaks near $m \approx 0$ and fall off on both sides.

Why utility sometimes peaks at $m \neq 0$ : the paper notes in Appendix A that "the utility can increase under slight perturbations of $m$ in either the positive or negative direction. In some cases, this suggests that the origin may not lie exactly on the utility manifold, implying that the utility is not always strictly optimal at $m = 0$ ." This is captured by the $m_{\pm}$ parameters in $D_u(m)$ : if the unsteered activation is slightly off the manifold, a small steering step can move it closer to the manifold intersection, temporarily increasing $D_u(m)$ above the baseline. The RQ parametrization allows this: if $m_{+} > 0$ , then $D(m)$ actually increases for $0 < m < m_{+}$ before decaying.

Fit validation: Table 2 reports utility $R^2$ values uniformly high, typically > 0.97 across settings (Gemma-2-9B-IT: average 0.98–0.99; Qwen-2.5-7B-Instruct: 0.98–0.99). This validates that utility variations under preference steering are well captured by a pure decay model with no projection term.

The key mechanistic claim: the paper's central hypothesis is that preference and utility respond to steering through mechanistically distinct pathways. Preference involves a deliberate projection — steering along a concept direction amplifies the model's inclination toward that concept. Utility involves only a passive decay — steering moves representations away from where they are stably processed. The two phenomena are linked only through the shared validity decay function $D(m)$ , which controls when and how badly off-manifold effects kick in. This explains why the preference–utility trade-off is inherent to steering-based control: any intervention strong enough to meaningfully shift preference will, by virtue of moving activations away from their typical positions, begin to degrade utility. The only question is how quickly the degradation sets in, which is governed by the alignment of the steering direction with the manifold (the $L$ parameters).

The SPLIT Training Objective

The mechanistic analysis directly motivates a new training objective, SPLIT (Steering with Preference–Utility Intervention), designed to produce intervention parameters ( $\Delta\mathbf{W}, \Delta\mathbf{b}$ or steering vectors) that achieve strong preference while delaying utility degradation (Section 5.1).

Core design principle: the paper's curve-fitting analysis reveals that the linear regime — where preference grows proportionally with $m$ while utility remains near its maximum — is the sweet spot for control. The goal of training is therefore to extend this linear regime — to make $\alpha_p$ (the preference projection slope) as large as possible while keeping the steering direction aligned with the manifold (large $L_{\pm}$ ) so that validity decay sets in later.

Utility loss component. To preserve utility, SPLIT trains the intervention parameters to minimize cross-entropy on both the positive and negative answer strings for each query:

$\mathcal{L}_{\mathrm{util}} = \lambda_p\,\mathcal{L}_p + \lambda_n\,\mathcal{L}_n$

where $\mathcal{L}_p = \mathcal{L}(A_p \mid q)$ and $\mathcal{L}_n = \mathcal{L}(A_n \mid q)$ are the sequence cross-entropy losses defined earlier, and $\lambda_p, \lambda_n$ are weights controlling the relative importance of positive versus negative samples. This objective encourages the model to assign high probability to both valid completions — i.e., to maintain general task capability regardless of concept polarity.

Why train on both $A_p$ and $A_n$ : if the model were trained only on $A_p$ , it would learn to increase $P(A_p \mid q)$ but might do so by simply collapsing all probability mass onto the positive answer, destroying the ability to generate the negative answer (and likely any other valid completion). Training on both answers ensures that the total probability mass assigned to the valid pair remains high, preserving utility. The weights $\lambda_p, \lambda_n$ allow the practitioner to bias training toward one polarity if desired, but the default is balanced.

Preference loss component. By Eq. 5, the difference $\mathcal{L}_n - \mathcal{L}_p$ is exactly the preference log-odds — the model's relative preference for the positive over the negative answer. SPLIT explicitly maximizes this gap using a hinge-style margin loss:

$\mathcal{L}_{\mathrm{pref}} = \gamma \cdot \sigma\!\left(\theta - (\mathcal{L}_n - \mathcal{L}_p)\right)$

where $\sigma(\cdot)$ is the ReLU activation ( $\max(0, \cdot)$ ), $\theta > 0$ is a margin threshold, and $\gamma$ is a weight controlling the trade-off between preference and utility.

What this loss computes: when $\mathcal{L}_n - \mathcal{L}_p \geq \theta$ , the argument to ReLU is non-positive, so the loss is zero — the model already achieves the desired preference margin. When $\mathcal{L}_n - \mathcal{L}_p < \theta$ , the loss is positive and proportional to $\gamma \cdot (\theta - (\mathcal{L}_n - \mathcal{L}_p))$ , penalizing the model for failing to separate the positive and negative answers by at least $\theta$ log-odds units.

Why a hinge loss rather than, say, a direct maximization of $\mathcal{L}_n - \mathcal{L}_p$ : the hinge loss has two desirable properties. First, it focuses training on examples where the preference margin is insufficient, ignoring examples that already satisfy the margin — this prevents the model from over-optimizing preference at the expense of utility on examples where sufficient control is already achieved. Second, the margin $\theta$ provides an explicit knob for controlling how strongly the model should prefer $A_p$ over $A_n$ , allowing practitioners to set different control strengths for different applications.

The role of $\gamma$ : this weight controls the preference–utility trade-off directly in the training objective. Larger $\gamma$ prioritizes preference amplification (making the model more strongly biased toward $A_p$ ), potentially at the cost of utility preservation. Smaller $\gamma$ prioritizes utility preservation, potentially resulting in weaker concept control. The paper does not prescribe a specific value but treats $\gamma$ as a hyperparameter to be tuned.

Final objective. The full SPLIT loss is a simple sum:

$\mathcal{L} = \mathcal{L}_{\mathrm{util}} + \mathcal{L}_{\mathrm{pref}}$

How this connects to the mechanistic theory: the utility loss $\mathcal{L}_{\mathrm{util}}$ can be seen as encouraging the steering direction to remain aligned with the manifold (increasing $L_{\pm}$ in the decay function) — by training on both valid completions, the model learns to keep representations in regions where both $A_p$ and $A_n$ can be reliably decoded. The preference loss $\mathcal{L}_{\mathrm{pref}}$ directly optimizes $\alpha_p$ (the alignment between steering direction and preference direction) by maximizing the log-odds gap. The combined objective thus works on both mechanistic levers simultaneously: strengthen the preference projection while keeping the steering trajectory on-manifold.

Training procedure: the SPLIT objective is applied to the same intervention parameter types as the baseline methods (local weight updates, LoRA factors, or steering vectors). The training data consists of query $q$ with paired answers $(A_p, A_n)$ from the same datasets used for evaluation (Psychopathy, PowerSeeking, AxBench). Optimization uses AdamW with a linear learning rate scheduler, largely following default settings from Wu et al. (2025a) and Xu et al. (2025). Hyperparameter tuning is performed to ensure "stable and competitive performance" (Appendix A).

Experimental validation: Table 3 shows that SPLIT consistently achieves higher or competitive scores compared to SFT and RePS baselines across all three intervention forms (weight, LoRA, vector) and both model families (Gemma-2-9B-IT, Qwen-2.5-7B-Instruct). On AxBench, SPLIT achieves the best harmonic mean in multiple settings (e.g., 1.6475 for vector steering on Gemma-2-9B-IT vs. 1.5550 for RePS and 1.4487 for SFT), demonstrating that explicit joint optimization of preference and utility produces better overall control than methods that optimize only one or neither explicitly. The paper notes that improvements are "reliable, albeit sometimes incremental" (Appendix A), consistent with the mechanistic prediction that the preference–utility trade-off is fundamental and cannot be eliminated entirely — only mitigated.

4. Key Insights and Innovations

Innovation 1: Framing heterogeneous LLM control methods as instances of a single affine transformation with a continuous multiplier

The paper's most fundamental conceptual move is not proposing a new taxonomy, but rather discovering a shared mathematical structure that was latent across previously disconnected method families. Prior work treated local weight fine-tuning, LoRA, and activation steering as distinct paradigms — studied in separate papers, evaluated under different protocols, and understood through different theoretical lenses (linear representation hypothesis for steering vectors, low-rank matrix factorization for LoRA, localized knowledge editing for weight updates). The paper demonstrates that all three can be expressed as special cases of a single equation: the forward pass through a linear layer becomes $\mathbf{h}_{i+1} = (\mathbf{W} + m_1 \Delta\mathbf{W})\,\mathbf{h}_{i} + (\mathbf{b} + m_2 \Delta\mathbf{b})$ , with each method differing only in how the update terms $\Delta\mathbf{W}$ and $\Delta\mathbf{b}$ are parameterized (Section 3.2, Table 1).

What makes this more than taxonomic bookkeeping is the introduction of explicit scaling coefficients $m_1, m_2$ as first-class objects of study. Prior work treated interventions as binary — either applied or not — or at best swept a steering coefficient heuristically. The paper elevates the multiplier $m$ to the status of a controlled experimental variable, enabling the systematic observation of how preference and utility change as a continuous function of intervention strength. This is what makes the unified dynamics in Figure 2 possible: without a common $m$ axis, comparing the scaling behavior of weight updates versus LoRA versus steering vectors would be meaningless.

This is a fundamental reframing, not an incremental refinement. Before this work, if a practitioner asked "should I use LoRA or activation steering for this control task?", the answer would depend on ad-hoc considerations of compute budget and implementation complexity, with no way to predict how the two approaches would behave under different control strengths. After this work, the question can be reframed as: "given my desired preference–utility operating point, which method's $m$ -dependent curve offers the best trade-off?" The unified formulation transforms method selection from a categorical choice into a continuous optimization problem.

Innovation 2: Separating preference and utility via a paired-example log-odds measurement framework

The second distinctive contribution is a measurement methodology that disentangles two effects — concept preference and task utility — that are hopelessly confounded in standard output-based evaluation. The paper's insight is that by constructing polarity-paired contrastive examples ( $A_p$ for the positive concept, $A_n$ for the negative concept) and measuring the model's internal probability assignments to these specific completions (rather than generating open-ended text), one can factor the model's output distribution into independent preference and utility components (Equations 4–6).

Why this is conceptually novel, not just a new metric: prior work on controlled generation evaluated interventions holistically — does the output express the right concept? Is it coherent? These two judgments are inherently coupled because a model that loses utility produces garbled text that cannot be assessed for concept expression. A method that appears to fail at concept control might actually be succeeding at preference while simultaneously collapsing utility — a qualitatively different failure mode from simply having a weak steering signal. The paired-example framework resolves this ambiguity by measuring the model's latent probabilities of generating known valid completions, before any text is actually generated. The PrefOdds metric (Equation 5: $\mathcal{L}_n - \mathcal{L}_p$ ) captures concept inclination independent of utility because the shared utility factor $P(u \mid q)$ cancels in the ratio. The UtilOdds metric (Equation 6) captures the total probability mass devoted to the valid pair, independent of which concept is preferred.

This framework also explains why prior studies reached contradictory conclusions about the effectiveness of steering. Bigelow et al. (2025) observed that steering yields an approximately linear trend in posterior odds "mainly in the small-scale regime," while Bricken et al. (2023) and Wollschläger et al. (2025) reported "non-monotonic or adverse effects under stronger steering." The paper's framework resolves this contradiction: the small-scale regime is where the linear preference projection dominates and utility decay has not yet set in; the adverse effects emerge when off-manifold deviation causes utility to collapse, which happens at different $m$ thresholds for different methods and concepts. These are not conflicting findings — they are observations of different regions of the same underlying preference–utility curve.

Innovation 3: The activation manifold hypothesis as a quantitative, falsifiable model of steering-induced degradation

The paper's third contribution is elevating a qualitative geometric intuition — that model activations concentrate on low-dimensional manifolds, and steering pushes them off these manifolds — into a quantitative, curve-fitting model with high empirical fidelity. The key move is abstracting the geometry of the manifold into a validity decay function $D(m)$ (Equation 12) that captures the observable consequence of off-manifold deviation without requiring explicit knowledge of the manifold's shape.

This is significant because it bridges two previously disconnected literatures. On one side, representation-manifold work (Modell et al., 2025; Li and He, 2025; Xie et al., 2025) provided geometric descriptions of activation space but did not connect these descriptions to the observable dynamics of behavioral control. On the other side, empirical steering work (Turner et al., 2023; Rimsky et al., 2024) demonstrated that steering works but provided no mechanistic account of why it degrades. The paper's validity decay model connects these two lines by making a falsifiable prediction: if steering degrades utility by pushing representations off a pretraining-induced manifold, then the utility log-odds should follow a specific functional form — a rational quadratic decay — with high $R^2$ across diverse methods and datasets. The validation of this prediction (Table 2, utility $R^2 > 0.97$ across most settings) converts the manifold hypothesis from a loose metaphor into a quantitatively supported mechanistic claim.

The piecewise rational quadratic parametrization is itself a non-obvious design choice that carries theoretical weight. The paper explicitly notes that model steerability is often asymmetric (Miehling et al., 2025) — the model responds differently to positive versus negative steering along the same conceptual dimension. By fitting separate parameters $(m_{\pm}, L_{\pm}, p_{\pm})$ for positive and negative $m$ , the model captures this asymmetry without overfitting — each piece has only three free parameters, and the total of six parameters per curve is parsimonious relative to the richness of the observed dynamics. The choice of rational quadratic over, say, Gaussian decay is also motivated: the polynomial tails of the RQ form better capture the gradual utility degradation observed at moderate $m$ , whereas an exponential decay would predict overly rapid collapse.

This is a fundamental theoretical advance rather than an incremental improvement because it changes the conversation around steering from "how much can we amplify preference before utility degrades?" to "what are the geometric parameters $(L_{\pm}, p_{\pm})$ of the steering direction relative to the manifold, and how can we design interventions with more favorable geometry?" It reframes the problem from empirical mitigation to geometric optimization.

Innovation 4: Decomposing preference dynamics into separable projection and decay components

The paper's mechanistic model makes a sharper claim than simply "steering increases preference until utility degrades." It posits that preference and utility respond to steering through mechanistically distinct pathways that share only the validity decay function (Sections 4.2–4.3). Preference is modeled as $(\alpha_p m + \beta_p) D_p(m) + b_p$ — a product of a linear projection term that grows with $m$ and a decay term that attenuates at large $m$ . Utility is modeled as $\beta_u D_u(m) + b_u$ — a pure decay term with no projection component, based on the assumption that the steering direction $\Delta h$ is approximately orthogonal to any utility-relevant direction $\boldsymbol{\omega}_u$ .

This decomposition is conceptually significant because it explains the characteristic shape of the preference curve — the initial linear regime, the transitional bend, and the convergence to a plateau — as the product of two competing forces rather than a single saturating function. The linear regime emerges when $D_p(m) \approx 1$ (near the manifold intersection), so PrefOdds $\approx \alpha_p m + \text{const}$ . The convergence region emerges when $D_p(m) \ll 1$ (far off-manifold), so the projection term $(\alpha_p m + \beta_p)$ is multiplied by a near-zero decay factor, and further increases in $m$ produce negligible preference gains. This is a more precise account than simply saying "steering saturates" or "diminishing returns set in" — it specifies why saturation occurs (the decay factor $D_p(m)$ ) and what parameters govern the saturation point ( $L_{\pm}$ and $p_{\pm}$ ).

The orthogonality assumption for utility ( $\boldsymbol{\omega}_u^{\mathsf{T}} \Delta h \approx 0$ ) is also a substantive claim about the geometry of concept representations. It asserts that preference-relevant directions and utility-relevant directions are largely disentangled in activation space — steering along a concept direction does not directly affect utility, only indirectly through off-manifold deviation. If this claim were false (if preference and utility directions were correlated), then utility would show both a projection effect and a decay effect, and the pure-decay model would fit poorly. The high $R^2$ values for the utility model (Table 2) provide evidence for this geometric disentanglement, at least for the concepts and models studied.

This is a diagnostic advance as much as a theoretical one. The parameters $(\alpha_p, L_{\pm}, p_{\pm})$ extracted from curve fitting provide a vocabulary for characterizing and comparing interventions: a high $\alpha_p$ indicates strong concept alignment, a large $L_{+}$ indicates a steering direction nearly parallel to the manifold (slow utility decay), and a large $p_{+}$ indicates rapid decay once off-manifold effects begin. These parameters could serve as design targets for future intervention methods — the paper's SPLIT objective can be understood as optimizing $\alpha_p$ and $L_{\pm}$ simultaneously, though the paper does not explicitly report these fitted parameters for SPLIT-trained interventions.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses three datasets: (i) Psychopathy, a personality tendency classification task (500 training instances, 100 test instances); (ii) PowerSeeking, an open-ended generation task (500 training instances, 200 test instances); and (iii) the top-10 concept subsets from AxBench, an open-ended generation benchmark (re-split from the original 72 instances per concept into 64 training and 8 test instances per concept for each of the top 10 subsets). The re-splitting of AxBench is necessary because its original test set does not provide matched positive/negative answer pairs, which are required for the paper's preference–utility measurement framework. All datasets are evaluated using polarity-paired contrastive examples — for each query $q$ , a concept-positive answer $A_p$ and a concept-negative answer $A_n$ are provided.
Base model(s). The paper runs experiments on two instruction-tuned model families: Gemma-2-9B-IT and Qwen-2.5-7B-Instruct. Interventions are applied at specific layers: layer 20 for Gemma-2-9B-IT and layer 14 for Qwen-2.5-7B-Instruct, following the layer selection procedure from Bigelow et al. (2025). For local weight and LoRA interventions, parameters are updated on the MLP down-projection matrix; for vector interventions, the steering is applied directly to the residual stream. The choice of instruction-tuned models (rather than base models) is not explicitly justified, but is consistent with the paper's focus on controlled generation in deployed settings where alignment is relevant.
Metrics. The paper evaluates along two complementary axes using three types of metrics:

For the unified dynamics analysis (Section 3.3, Figures 2, 4, 5): Preference and utility are measured using the log-odds metrics derived in Section 3.1 — PrefOdds(q) = $\mathcal{L}_n - \mathcal{L}_p$ (Equation 5) and UtilOdds(q) = $\log\frac{e^{-\mathcal{L}_p} + e^{-\mathcal{L}_n}}{1 - e^{-\mathcal{L}_p} - e^{-\mathcal{L}_n}}$ (Equation 6) — computed from sequence cross-entropy losses on the paired answers $(A_p, A_n)$ for each query $q$ . These metrics are computed on both training and test sets, and tracked as the intervention multiplier $m$ is varied.

For the final performance evaluation (Section 5.2, Table 3): Dataset-specific metrics are used. For Psychopathy, classification accuracy (Acc, %) is reported, following Bigelow et al. (2025). For PowerSeeking, following Cao et al. (2024), a GPT-4.1-mini judge scores generations on a 0–4 scale for concept preference. For AxBench, following Wu et al. (2025a), a GPT-4.1-mini judge evaluates three dimensions: concept score, instruction score, and fluency score, each on a 0–2 scale; the paper reports the concept score and the harmonic mean over all three scores. The harmonic mean is chosen because it penalizes interventions that sacrifice any one dimension disproportionately — a method that achieves high concept but low fluency will receive a low harmonic mean, reflecting the practical reality that degraded utility undermines the value of concept control.

For the curve-fitting validation (Section 4, Table 2): Model fit quality is measured using $R^2$ (coefficient of determination), comparing the predicted log-odds from the parametric models (Equations 15 and 17) against the empirically measured log-odds as a function of the steering multiplier $m$ . Higher $R^2$ values indicate that the mechanistic model accurately captures the shape of the empirical curves.
Baselines. The paper evaluates multiple methods under three intervention forms, with several training objectives for each:
- Vanilla: The base model with no intervention applied, serving as the unsteered reference point.
- SFT (Supervised Fine-Tuning): Intervention parameters (weights, LoRA factors, or steering vectors) trained using standard language modeling cross-entropy on the target concept data. This is the simplest training objective and serves as a basic learned-steering baseline.
- RePS (Representation Steering; Wu et al., 2025a): A previously proposed training objective for activation steering that learns intervention parameters through representation-level optimization. Applied to all three intervention forms for comparability.
- DiffMean (Marks and Tegmark, 2023): A train-free baseline for vector interventions only. The steering vector is computed as the difference between mean activations for positive and negative examples of the target concept. This requires no parameter optimization and is the most widely used steering vector construction method in prior work.
- SPLIT (Ours): The paper's proposed joint preference–utility optimization objective (Equations 18–20), applied to all three intervention forms.
All trained methods (SFT, RePS, SPLIT) are optimized with AdamW and a linear learning-rate scheduler, with hyperparameter tuning performed to "ensure stable and competitive performance" (Appendix A). The baseline set allows the paper to compare both different method forms (weight update vs. LoRA vs. vector) and different training objectives (SFT vs. RePS vs. SPLIT), separating the contributions of each.
Generation budget / compute accounting. The paper does not measure compute in FLOPs or wall-clock time. Instead, the primary independent variable is the intervention multiplier $m$ , swept across a range of values (both positive and negative) to study how preference and utility scale continuously with control strength. For the unified dynamics analysis (Figures 2, 4, 5), the paper sweeps $m$ and measures log-odds at each point, producing parametric curves rather than point estimates. For the final performance evaluation (Table 3), $m$ is presumably fixed at the value that produces optimal performance (or a default value), though the paper does not explicitly state this or report the selected $m$ values. There is no FLOPs-matched comparison between intervention types — the paper does not claim that vector steering is cheaper than LoRA (which it is, by parameter count as shown in Table 1), nor does it normalize performance by compute cost. This is a deliberate choice: the paper's goal is to demonstrate that the same qualitative dynamics hold across methods, not to argue that one method is more cost-effective than another.
Cross-validation / statistical protocol. The paper does not employ cross-validation for strategy selection (unlike the reference example, which used two-fold cross-validation to select compute-optimal strategies). For the curve fitting analysis, curves are fitted directly to the measured log-odds data using the SLSQP algorithm, with continuity enforced between positive and negative segments at $m = 0$ (Appendix D). For evaluating generalization, the paper performs a train-to-test transfer experiment: curve parameters are fitted on the training set and applied directly to the test set to assess how well the mechanistic model generalizes to unseen inputs (Appendix D.2, Table 6). No confidence intervals, standard errors, or statistical significance tests are reported for any of the main results. The test set sizes are small: 100 instances for Psychopathy, 200 for PowerSeeking, and 8 per concept × 10 concepts = 80 for AxBench, which limits the statistical reliability of the results, though the paper does not discuss this limitation.

Main Quantitative Results

Unified Preference–Utility Dynamics Across Intervention Forms

The paper's central empirical claim is that different intervention methods exhibit remarkably consistent dynamic patterns when preference and utility log-odds are plotted as functions of the steering multiplier $m$ (Section 3.3, Figures 2, 4, 5). This claim is evaluated across three intervention forms (local weight updates, LoRA, vector) with multiple training objectives (SFT, RePS, DiffMean for vectors) on two model families (Gemma-2-9B-IT, Qwen-2.5-7B-Instruct) and three datasets (Psychopathy, PowerSeeking, AxBench).

Headline observation from Figure 2 (Gemma-2-9B-IT on AxBench top-10 concepts): All six method configurations — local weight SFT, local weight RePS, LoRA SFT, LoRA RePS, vector SFT, vector RePS — produce qualitatively identical curve shapes for both preference and utility log-odds. The preference curves (solid lines) follow a three-phase pattern: an initial linear region at small $|m|$ where log-odds grows approximately linearly, a transitional region where the slope changes noticeably, and a convergence region where the curve flattens. The utility curves (dashed lines) peak near $m \approx 0$ and decline monotonically as $|m|$ increases. The paper describes this as "highly similar overall curve shapes" that "reveal a unified steering response of preference and utility" (Section 3.3).

Difficulty-dependent or method-specific differences: While the qualitative patterns are consistent, Figure 2 shows clear quantitative differences between methods. The vector DiffMean baseline (not shown in Figure 2 but present in the appendix figures and Table 3) produces very weak steering — on Psychopathy, it achieves only 53.00% accuracy for Gemma-2-9B-IT compared to 100.00% for SFT weight updates (Table 3) — indicating that the simple difference-of-means vector, while train-free, captures a far weaker control signal than learned interventions. Among learned methods, the preference saturation level differs: vector RePS appears to reach higher maximum PrefOdds than LoRA SFT in Figure 2 (top panel), though the paper does not quantify these differences numerically. The linear regime also varies in extent — some methods maintain near-linear preference growth over a wider range of $m$ than others, which the paper's mechanistic model attributes to differences in the characteristic length scale $L_{\pm}$ (how parallel the steering direction is to the activation manifold).

Cross-model replication (Figures 4 and 5): The Qwen-2.5-7B-Instruct results on AxBench (Figure 4 in Appendix A) replicate the essential patterns: three-phase preference curves and monotonic utility decay for all method configurations. The PowerSeeking and Psychopathy datasets (Figure 5) show the same patterns, confirming that the unified dynamics are not specific to a particular model, dataset, or concept type. However, the paper does not provide numerical comparisons of fitted parameters ( $\alpha_p$ , $L_{\pm}$ , $p_{\pm}$ ) across datasets, so it is impossible to determine whether certain concepts are inherently "easier" to steer (higher $\alpha_p$ ) or whether certain concept directions are more aligned with the manifold (larger $L_{\pm}$ ).

The asymmetry of utility curves: An important but underemphasized detail appears in Appendix A: "the utility can increase under slight perturbations of $m$ in either the positive or negative direction. In some cases, this suggests that the origin may not lie exactly on the utility manifold, implying that the utility is not always strictly optimal at $m = 0$ ." This is visible in some panels of Figures 2, 4, and 5 where the utility curve is not perfectly symmetric or does not peak exactly at $m = 0$ . The paper's mechanistic model captures this via the $m_{\pm}$ parameters in the validity decay function — if $m_{+} > 0$ , then $D(m)$ increases for small positive $m$ , temporarily boosting utility before decay sets in. This phenomenon is not discussed in the main text and represents an interesting finding that challenges the simple "steering always hurts utility" narrative.

Mechanistic Model Fit Quality

The paper validates its activation manifold hypothesis through curve fitting of the parametric models in Equations 15 (preference) and 17 (utility) to empirically measured log-odds data (Section 4, Table 2).

In-domain fit quality (Table 2, fitted and evaluated on the same data): The $R^2$ values are uniformly high, with most exceeding 0.95:

Preference $R^2$ : For Gemma-2-9B-IT, averaged across Psychopathy, PowerSeeking, and AxBench, $R^2$ ranges from 0.96 (LoRA SFT) to 0.99 (Weight RePS). For Qwen-2.5-7B-Instruct, ranges from 0.96 (Vector SFT, Vector RePS) to 0.99 (Weight SFT, Weight RePS). The lowest individual setting is LoRA RePS on Gemma-2-9B-IT for Psychopathy at 0.83, but this is an outlier — all other Psychopathy settings exceed 0.92.
Utility $R^2$ : For Gemma-2-9B-IT, averaged across datasets, $R^2$ ranges from 0.98 (Weight SFT, Weight RePS, Vector DiffMean) to 0.99 (LoRA SFT, LoRA RePS, Vector SFT, Vector RePS). For Qwen-2.5-7B-Instruct, ranges from 0.98 (Weight RePS, LoRA RePS, Vector DiffMean) to 0.99 (Weight SFT, LoRA SFT, Vector SFT, Vector RePS).

What these $R^2$ values mean: An $R^2$ of 0.97–0.99 indicates that the parametric model (projection × decay for preference, pure decay for utility) explains 97–99% of the variance in the measured log-odds curves. This is strong evidence that the underlying mechanistic assumptions — a linear preference projection attenuated by a rational quadratic validity decay, and a pure validity decay for utility — are approximately correct. The paper does not compare against simpler alternative models (e.g., a purely linear model or a simple logistic saturation without the RQ decay form), which would be necessary to claim that the specific RQ functional form is uniquely well-suited rather than simply flexible enough to fit any smooth curve with six parameters.

Generalization to held-out data (Table 6 in Appendix D.2): The paper conducts a train-to-test transfer experiment: curve parameters are fitted on the training set and applied directly to the test set to predict the log-odds curves for unseen inputs. This is a stricter test of the mechanistic model — if the fitted parameters capture genuine geometric properties of the steering direction relative to the manifold (rather than overfitting to training data), they should generalize.

The results are mixed:

Vector-based interventions (DiffMean, SFT, RePS) show the most consistent generalization, with many settings achieving $R^2 > 0.85$ on test data. For example, Vector RePS achieves test $R^2$ of 0.97, 0.93, and 0.74 on Psychopathy, PowerSeeking, and AxBench respectively under Qwen-2.5-7B-Instruct, averaging 0.88. This makes mechanistic sense: vector interventions add a fixed, input-independent $\Delta\mathbf{b}$ , so the geometric relationship between the steering direction and the manifold should be relatively stable across inputs.
Input-dependent methods (local weight updates, LoRA) show larger variance and occasional failures. On Qwen-2.5-7B-Instruct Weight SFT, the preference $R^2$ on AxBench drops to −12.03 — meaning the fitted curve is worse than a horizontal line at predicting the test-set dynamics. Weight RePS on the same model achieves −3.82 on AxBench and −0.32 on PowerSeeking. These negative $R^2$ values indicate that the mechanistic model, when fitted on training data, completely fails to capture the test-set dynamics for weight update methods on certain datasets. Utility $R^2$ also degrades for these methods: Weight SFT on Gemma-2-9B-IT achieves only 0.61 on AxBench (down from 0.98 in-domain).

Interpreting the generalization results: The paper attributes the poorer generalization of weight update methods to the fact that they are "input-dependent" — the effective $\Delta\mathbf{h} = m(\Delta\mathbf{W}\,\mathbf{h}_i + \Delta\mathbf{b})$ varies with the specific input $\mathbf{h}_i$ , so a single set of curve parameters fitted on training data may not transfer to test inputs with different activation patterns. This is a genuine limitation of the mechanistic model: it assumes a fixed steering direction $\Delta h$ and a fixed relationship between $m$ and off-manifold distance, but for weight update methods, the effective steering depends on how $\Delta\mathbf{W}$ interacts with each input's specific $\mathbf{h}_i$ . The paper does not explore whether this limitation can be addressed by fitting input-conditional parameters or by applying the model to the effective $\Delta\mathbf{h}$ rather than to $m$ directly.

SPLIT Performance Comparison

The paper evaluates SPLIT against SFT and RePS baselines under all three intervention forms, using the dataset-specific metrics described above (Section 5.2, Table 3).

Psychopathy classification accuracy: All trained methods achieve near-perfect accuracy (97–100%) on both models, dramatically outperforming the Vanilla baseline (50.00%, chance-level for binary classification) and the train-free DiffMean baseline (53–55%). This indicates that the Psychopathy concept is easily steerable — any training objective with sufficient capacity achieves ceiling performance. Within this saturated regime, the differences between methods are negligible (e.g., Weight SFT and Weight SPLIT both achieve 100.00% on Gemma-2-9B-IT). This ceiling effect means Psychopathy provides little discriminative power for comparing training objectives and the paper does not overinterpret these results.

PowerSeeking preference scores (0–4 scale): This dataset provides more differentiation. On Gemma-2-9B-IT:

SPLIT achieves the highest scores across all three intervention forms: 3.59 (Weight) vs. 3.50 (Weight SFT) and 3.39 (Weight RePS); 3.56 (LoRA) vs. 3.41 (LoRA SFT) and 3.44 (LoRA RePS); 3.62 (Vector) vs. 3.30 (Vector SFT) and 3.61 (Vector RePS).
The gains over SFT are substantial for vectors (+0.32, a 9.7% relative improvement) and modest for weights and LoRA (+0.09 and +0.15, respectively).
DiffMean achieves only 2.95, confirming that train-free steering vectors are substantially weaker than learned ones.

On Qwen-2.5-7B-Instruct, the pattern is similar: SPLIT achieves the highest scores under Weight (3.66 vs. 3.53 SFT, 3.24 RePS), LoRA (3.59 vs. 3.05 SFT, 3.34 RePS), and Vector (3.65 vs. 3.58 SFT, 3.63 RePS). The gains over SFT are dramatic for LoRA (+0.54, an 17.7% relative improvement) and modest for vectors (+0.07).

AxBench concept score (0–2 scale): SPLIT consistently achieves the highest concept scores:

Gemma-2-9B-IT: 1.8500 for both Weight SPLIT and Vector SPLIT vs. 1.6625 (Weight SFT) and 1.7000 (Vector SFT). The improvement over SFT is +0.1875 for weights and +0.1500 for vectors.
Qwen-2.5-7B-Instruct: 1.7000 (Weight SPLIT) vs. 1.5375 (Weight SFT); 1.7375 (LoRA SPLIT) vs. 1.4875 (LoRA SFT); 1.8125 (Vector SPLIT) vs. 1.5750 (Vector SFT). These are substantial improvements, particularly for LoRA (+0.25, a 16.8% relative improvement).

AxBench harmonic mean (0–2 scale), which penalizes utility degradation: This is the most discriminating metric because it downweights methods that achieve high concept scores by sacrificing instruction-following or fluency. SPLIT achieves the best or near-best harmonic mean in most settings:

Gemma-2-9B-IT: Weight SPLIT achieves 1.6225, better than Weight SFT (1.4538) and competitive with Weight RePS (1.6362). LoRA SPLIT achieves 1.6412, better than both LoRA SFT (1.5188) and LoRA RePS (1.6525) — though LoRA RePS slightly edges it out. Vector SPLIT achieves 1.6475, substantially better than Vector SFT (1.4487) and Vector RePS (1.5550). Notably, for vectors, SPLIT achieves the highest harmonic mean while also achieving the highest concept score — it is not trading off one for the other at the chosen operating point.
Qwen-2.5-7B-Instruct: LoRA SPLIT achieves 1.6362, substantially better than LoRA SFT (1.3175) and LoRA RePS (1.4013). Vector SPLIT achieves 1.6500, competitive with Vector RePS (1.6412) and much better than Vector SFT (1.5800). Weight SPLIT achieves 1.4325, competitive with Weight RePS (1.4163) and better than Weight SFT (1.1287).

The View from the Evaluation Results: The paper's central claim is that SPLIT improves preference while better preserving utility compared to baselines. The harmonic mean results provide the most direct evidence for this claim because they explicitly penalize utility degradation. Across both models and all three intervention forms (12 settings total), SPLIT achieves the highest harmonic mean in 5 settings, a very close second-best in 4 settings, and a competitive second-best in the remaining 3 settings. It never underperforms substantially. This pattern supports the claim that explicit joint optimization of preference and utility produces more balanced control than methods that optimize only one objective (SFT maximizes probability of the target concept) or that optimize both implicitly (RePS).

The results also reveal that the choice of intervention form matters independently of the training objective:

Vector interventions tend to achieve the highest harmonic means overall (e.g., 1.6475 and 1.6500 for vector SPLIT on Gemma-2-9B-IT and Qwen-2.5-7B-Instruct respectively), despite having the smallest parameter footprint ( $d_{\text{out}}$ parameters vs. thousands for LoRA or millions for weight updates).
LoRA interventions are competitive with vectors and sometimes exceed them (e.g., LoRA RePS achieves 1.6525 on Gemma-2-9B-IT, slightly higher than Vector SPLIT's 1.6475).
Weight interventions show the widest variation — they achieve high concept scores but sometimes have lower harmonic means (e.g., Weight SFT on Qwen-2.5-7B-Instruct achieves only 1.1287 vs. 1.5800 for Vector SFT), suggesting that full weight updates risk greater utility degradation.

The paper does not discuss why vectors and LoRA appear to preserve utility better than full weight updates, but the mechanistic model suggests an interpretation: vectors and LoRA impose structural constraints (fixed bias offset, low-rank weight update) that may act as implicit regularizers, keeping the effective steering direction $\Delta\mathbf{h}$ more aligned with the manifold (larger $L_{\pm}$ ). Full weight updates have more degrees of freedom and may learn directions that more aggressively cut across the manifold in pursuit of preference gain.

Important caveats about Table 3: The paper does not report the intervention multiplier $m$ used for each method. In the mechanistic model, preference and utility vary continuously with $m$ , so the performance in Table 3 depends on the specific operating point chosen. If SPLIT were evaluated at a suboptimal $m$ , it might underperform; conversely, if baselines were evaluated at a carefully tuned $m$ while SPLIT used a default, comparisons could be misleading. The paper states that "reasonable hyperparameter tuning" was performed (Appendix A), but does not specify whether $m$ was tuned per method or held constant. This omission makes it difficult to determine whether SPLIT's gains come from genuinely better intervention parameters or from a better choice of $m$ at evaluation time. Additionally, the lack of error bars or confidence intervals on Table 3 means that the reported differences — often in the second or third decimal place — may not be statistically significant given the small test set sizes (100 for Psychopathy, 200 for PowerSeeking, 80 for AxBench).

Ablation Studies and Robustness Checks

Training objective ablation (SFT vs. RePS vs. SPLIT across three intervention forms): Table 3 provides a comprehensive comparison. Across all settings, SPLIT consistently achieves the highest or near-highest harmonic mean on AxBench and the highest PowerSeeking score, confirming that the joint preference–utility objective outperforms SFT (which optimizes only preference implicitly via cross-entropy on positive examples) and RePS (which uses a different representation-level objective not described in detail in this paper but cited from Wu et al., 2025a). The SFT baseline performs particularly poorly on AxBench harmonic mean (e.g., Weight SFT on Qwen-2.5-7B-Instruct: 1.1287 vs. 1.4325 for SPLIT), suggesting that naive supervised fine-tuning achieves concept control at a steep utility cost. RePS is a stronger baseline, often matching or approaching SPLIT (e.g., LoRA RePS on Gemma-2-9B-IT: 1.6525 vs. 1.6412 for LoRA SPLIT; Vector RePS on Qwen-2.5-7B-Instruct: 1.6412 vs. 1.6500 for Vector SPLIT), indicating that RePS already captures some of the preference–utility trade-off implicitly.
Intervention form ablation (Weight vs. LoRA vs. Vector): Table 3 shows that vector interventions achieve the best harmonic means in the majority of settings (4 of 6 model-method-type combinations when comparing the best objective per form). However, LoRA is competitive and sometimes exceeds vectors (LoRA RePS on Gemma-2-9B-IT: 1.6525 vs. 1.6475 for Vector SPLIT). Weight update methods consistently achieve the lowest harmonic means when using SFT (1.4538 and 1.1287 for Gemma-2-9B-IT and Qwen-2.5-7B-Instruct respectively), but recover with better objectives (Weight RePS: 1.6362 and 1.4163; Weight SPLIT: 1.6225 and 1.4325). This suggests that full weight updates have greater potential for utility degradation (more degrees of freedom allow sharper trade-offs) but can be regularized through appropriate training objectives. The paper does not run an explicit parameter-count-matched comparison (e.g., limiting weight updates to the same number of parameters as a LoRA adapter), which would help distinguish whether the vector/LoRA advantage is due to structural regularization or simply fewer parameters.
Dataset transfer (training objective across datasets): While not framed as an ablation, the consistent patterns across Psychopathy, PowerSeeking, and AxBench in Table 3 demonstrate that SPLIT's advantage is not dataset-specific. The gains are smallest on Psychopathy (ceiling effects) and largest on AxBench (which directly penalizes utility degradation via the harmonic mean). This aligns with the mechanistic prediction: SPLIT should help most on tasks where utility preservation is the binding constraint, not on tasks where preference is trivially achieved.
Cross-model generalization (Gemma-2-9B-IT vs. Qwen-2.5-7B-Instruct): All patterns replicate across both model families, with SPLIT achieving the highest or near-highest scores in both cases. This suggests that the preference–utility trade-off and the effectiveness of joint optimization are not artifacts of a particular model architecture or training procedure. However, both models are instruction-tuned, decoder-only transformers of similar scale (7–9B parameters), so generalization to much larger models (70B+), base (non-instruction-tuned) models, or different architectures (encoder-decoder, mixture-of-experts) is not tested.
Train-free baseline (DiffMean vs. learned vectors): The DiffMean baseline consistently underperforms all learned vector methods (Table 3). On Gemma-2-9B-IT PowerSeeking: DiffMean achieves 2.95 vs. 3.30–3.62 for learned vectors. On AxBench harmonic mean: DiffMean achieves 1.0550 vs. 1.4487–1.6475 for learned vectors. This gap confirms that while the linear representation hypothesis captures a real structure (diff-means vectors do extract some concept signal), training substantially improves the quality of the steering vector — likely by finding a direction that better aligns with the preference concept (larger $\alpha_p$ ) and/or stays closer to the manifold (larger $L_{\pm}$ ). The paper does not analyze why learned vectors outperform DiffMean in terms of the mechanistic parameters, which would be a natural extension.
Curve fitting on held-out test data (train vs. test $R^2$ ): Appendix D.2 and Table 6 report the generalization experiment. As discussed above, vector-based methods generalize well (most test $R^2 > 0.85$ ), while weight-based methods sometimes fail catastrophically (Weight SFT on Qwen-2.5-7B-Instruct for AxBench preference: $R^2 = -12.03$ ). This ablation reveals an important boundary condition for the mechanistic model: it works well for input-independent interventions (vectors) but struggles with input-dependent interventions (weight updates) because the effective $\Delta\mathbf{h}$ varies across inputs, violating the model's assumption of a fixed steering direction. The paper does not explore whether this failure is fundamental (the manifold geometry is genuinely input-dependent for weight updates) or whether it can be mitigated by, for example, fitting separate curve parameters per difficulty bin or using a conditional model where $\alpha_p$ depends on input features.
Layer choice ablation: The paper applies interventions at layer 20 for Gemma-2-9B-IT and layer 14 for Qwen-2.5-7B-Instruct, following Bigelow et al. (2025). However, the paper does not investigate how the preference–utility dynamics change at different layers. This is a notable omission because prior work has shown that different layers encode different types of information (e.g., lower layers encode syntactic features, higher layers encode semantic concepts), and the activation manifold likely has different geometry at different depths. It is possible that applying the same intervention at a different layer would produce different $\alpha_p$ , $L_{\pm}$ , and $p_{\pm}$ parameters, and that certain layers offer more favorable preference–utility trade-offs.

Critical Assessment

The paper's central claims, as articulated in the executive summary, are: (1) diverse LLM control methods share a common mathematical form and exhibit consistent preference–utility dynamics; (2) this trade-off follows predictable geometric regularities captured by the activation manifold hypothesis with high $R^2$ fits; and (3) SPLIT, a joint optimization objective guided by this understanding, improves preference while preserving utility. The experiments provide varying degrees of support for these claims, with specific strengths and gaps.

Claim 1: Unified mathematical form and consistent dynamics. The experiments strongly support the qualitative claim that different methods produce similar curve shapes. Figures 2, 4, and 5 show that all methods — weight updates, LoRA, vector — exhibit the three-phase preference pattern and monotonic utility decline across two model families and three datasets. This is the paper's most robust finding.

However, "consistent dynamics" is demonstrated only at the level of qualitative shape similarity. The paper does not quantify how similar the curves are — it never reports, for example, the variance in fitted $\alpha_p$ or $L_{\pm}$ across methods applied to the same concept, or tests whether methods cluster into statistically distinguishable groups. Without such analysis, the claim of "unified dynamics" could be overstated: visually similar curves can have meaningfully different quantitative parameters, and those parameters are what matter for practical method selection. The paper's own generalization results (Table 6) hint that weight update methods may behave differently enough from vector methods to violate the fitted model on held-out data, which complicates the unification narrative.

Claim 2: High $R^2$ fits validate the activation manifold model. The in-domain $R^2$ values (>0.95 for most settings) are genuinely impressive and provide strong evidence that the parametric model captures the empirical curve shapes. This is the paper's most rigorous quantitative result. The preference model (projection × decay) captures the three-phase dynamics, and the utility model (pure decay) captures the monotonic decline with high fidelity.

However, high $R^2$ validates the functional form but not necessarily the mechanistic interpretation. The projection-with-decay model for preference has, implicitly, at least 8 free parameters per fitted curve: $\alpha_p$ , $\beta_p$ , $b_p$ , and the five decay parameters $(m_{+}, m_{-}, L_{+}, L_{-}, p_{+})$ minus one for piecewise continuity at $m=0$ , plus potentially $p_{-}$ as separate. With this many degrees of freedom, a high $R^2$ is not surprising for any smooth, monotonic-ish curve — a sufficiently flexible parametric form will fit. The paper never compares against simpler alternative models (e.g., a logistic function, a polynomial, or a model without the decay factor) to show that the specific RQ decay structure is uniquely well-motivated rather than simply flexible. Without such comparisons, the $R^2$ values demonstrate curve-fitting success but not necessarily mechanistic truth.

The generalization results (Table 6) provide a more discriminating test and reveal genuine limitations: the model fails on weight update methods for certain datasets. The paper handles these negative results honestly by reporting them but does not adequately explain why weight updates specifically fail. The explanation offered — "input-dependent" — is a description of the symptom, not a diagnosis of the geometric cause. Why does input-dependence lead to negative $R^2$ ? Is the effective $\Delta\mathbf{h}$ varying so much across inputs that a single set of manifold parameters cannot capture it? Or is the model form itself misspecified for weight updates? The paper does not investigate these questions, leaving the mechanistic model's boundary conditions unclear.

Claim 3: SPLIT improves preference while preserving utility. Table 3 shows that SPLIT achieves the highest or near-highest scores across most settings, supporting the claim that explicit joint optimization is beneficial. The harmonic mean results are particularly convincing because they directly penalize utility degradation.

However, the evidence for this claim has several important qualifications:

First, the gains are often small in absolute terms. On Gemma-2-9B-IT AxBench harmonic mean, the difference between the best SPLIT variant (Vector: 1.6475) and the best non-SPLIT variant (LoRA RePS: 1.6525) is 0.005 on a 0–2 scale, favoring the baseline. On Qwen-2.5-7B-Instruct, Weight SPLIT achieves 1.4325 vs. Weight RePS at 1.4163 — a difference of 0.0162. These margins are unlikely to be statistically significant given test sets of 80–200 examples, and the paper reports no confidence intervals to assess reliability.

Second, RePS is a strong baseline that often matches SPLIT. In 4 of the 12 model-form combinations, RePS achieves a higher harmonic mean than SPLIT (Gemma Weight RePS: 1.6362 vs. 1.6225; Gemma LoRA RePS: 1.6525 vs. 1.6412; Qwen Weight RePS: 1.4163 vs. 1.4325 — here SPLIT wins; Qwen Vector RePS: 1.6412 vs. 1.6500 — SPLIT wins). The overall pattern is that SPLIT and RePS are roughly comparable, with SPLIT having a slight edge. The paper would be strengthened by a more detailed analysis of when SPLIT outperforms RePS and why — for instance, does the advantage grow with intervention strength $m$ ? Is SPLIT more robust to over-steering? These questions are not addressed.

Third, the choice of evaluation metric matters. On AxBench, the paper reports both the concept score (which measures preference) and the harmonic mean (which penalizes utility degradation). SPLIT's advantage is clearer on the concept score (it achieves the highest value in 5 of 6 comparisons) than on the harmonic mean (where it is roughly tied with RePS). This suggests that SPLIT primarily boosts preference, with utility preservation being comparable to RePS. This is still a valid contribution — achieving higher preference at similar utility is better than equal preference at lower utility — but it is a narrower claim than "preserving utility better."

What experiments are missing that would strengthen the paper:

Sweeping $m$ for SPLIT vs. baselines. The paper's mechanistic model predicts that the full curve of preference vs. utility as $m$ varies is determined by the intervention parameters. A natural evaluation would be to plot preference–utility Pareto frontiers for SPLIT-trained vs. baseline-trained interventions, showing that SPLIT achieves a higher preference for any given utility level (or vice versa) across the full range of $m$ . This would directly validate the claim that SPLIT produces a "better" intervention in the geometric sense (higher $\alpha_p$ and larger $L_{\pm}$ ). The current Table 3 evaluation at a single (unspecified) $m$ provides only a point estimate that could be sensitive to the chosen operating point.
Reporting fitted mechanistic parameters for SPLIT vs. baselines. The paper's theory predicts that SPLIT should increase $\alpha_p$ (stronger preference projection) and/or increase $L_{\pm}$ (slower utility decay). Fitting the mechanistic model to SPLIT-trained interventions and comparing the parameters to baseline interventions would directly test whether SPLIT works through the hypothesized mechanism. The paper's omission of this analysis is a significant missed opportunity to connect the theoretical framework to the empirical results.
Statistical significance testing. The small test sets (100 for Psychopathy, 200 for PowerSeeking, 80 for AxBench) mean that many of the performance differences in Table 3 may be within sampling error. Bootstrap confidence intervals or paired significance tests would substantially strengthen the reliability of the conclusions.
Evaluation at multiple $m$ values for Table 3. If SPLIT is evaluated at its optimal $m$ while baselines use suboptimal $m$ , the comparison is unfair. Conversely, if all methods use the same $m$ , SPLIT might be disadvantaged if its operating characteristics differ. Reporting performance curves as a function of $m$ (rather than point estimates) would address this concern.
Ablation of the preference loss component ( $\mathcal{L}_{\text{pref}}$ ). The SPLIT objective has two components: $\mathcal{L}_{\text{util}}$ and $\mathcal{L}_{\text{pref}}$ . An ablation where $\mathcal{L}_{\text{pref}}$ is removed (leaving only the utility preservation loss) would show whether the preference loss is necessary or whether the utility loss alone (which trains on both $A_p$ and $A_n$ ) already provides some preference signal by encouraging the model to assign high probability to the positive answer relative to other outputs. The current baseline set (SFT, RePS) does not isolate this effect.
Varying the SPLIT hyperparameters ( $\gamma$ , $\theta$ , $\lambda_p$ , $\lambda_n$ ). The paper introduces these hyperparameters but does not explore how performance varies with them. Understanding the sensitivity to $\gamma$ (the preference–utility trade-off weight) is particularly important, as it directly controls the mechanism the paper claims to optimize. A sweep over $\gamma$ values would show whether SPLIT enables a tunable trade-off — a feature that would distinguish it from SFT and RePS if those methods cannot easily control their operating point.

In summary, the experiments convincingly demonstrate the qualitative unified dynamics (Claim 1), provide strong in-domain curve-fitting evidence for the mechanistic model (Claim 2, with caveats about the generalization failures), and show that SPLIT is a competitive training objective that achieves good preference–utility balance (Claim 3). However, the paper does not provide direct evidence that SPLIT works through the hypothesized geometric mechanism (by improving $\alpha_p$ and $L_{\pm}$ ), relies on small test sets without statistical validation, and leaves important hyperparameter and operating-point questions unexplored. The contribution is robust as a descriptive framework and an empirical demonstration of unified dynamics, but the mechanistic claims require more targeted validation, and the practical advantage of SPLIT over strong baselines like RePS is modest and context-dependent.

6. Limitations and Trade-offs

The Activation Manifold Model Generalizes Poorly for Input-Dependent Interventions

The paper's mechanistic model — the projection-with-decay and pure-decay curve-fitting framework — assumes a fixed steering direction $\Delta h$ with a stable geometric relationship to the activation manifold. This assumption holds reasonably well for vector interventions (where $\Delta h$ is a constant bias offset), but breaks down for input-dependent methods like local weight updates and LoRA, where the effective $\Delta\mathbf{h} = \Delta\mathbf{W}\,\mathbf{h}_i + \Delta\mathbf{b}$ varies with each specific input.

The consequence. The train-to-test generalization results in Table 6 (Appendix D.2) reveal catastrophic failures for weight-based methods. On Qwen-2.5-7B-Instruct, the preference $R^2$ for Weight SFT drops to −12.03 on AxBench — meaning the fitted model is worse than predicting a constant value. Weight RePS on the same model achieves −3.82 on AxBench and −0.32 on PowerSeeking. Utility $R^2$ similarly degrades: Weight SFT on Gemma-2-9B-IT achieves only 0.61 on AxBench (down from 0.98 in-domain), and Weight RePS drops to 0.73. These negative or near-zero $R^2$ values indicate that the mechanistic model, when fitted on training data, completely fails to predict the preference–utility dynamics for held-out inputs when applied to weight update methods.

What this means in practice: the paper's claim that preference and utility follow "predictable geometric regularities" (Section 4) is conditional on the intervention being input-independent. For practitioners using LoRA or local weight fine-tuning — which are among the most widely deployed control methods — the quantitative curve-fitting framework provides no reliable out-of-sample predictions. The geometric parameters ( $\alpha_p$ , $L_{\pm}$ , $p_{\pm}$ ) fitted on one set of inputs do not transfer to another, which undermines their utility as diagnostic or comparative tools for these method families.

What the paper shows about this limitation. The paper reports these negative results honestly in Table 6 but offers only a brief, descriptive explanation: input-dependent methods produce an effective $\Delta\mathbf{h}$ that varies across inputs (Appendix D.2). The paper does not investigate why this variation causes such dramatic failures — whether the manifold geometry itself shifts across inputs for weight updates, whether the model form is misspecified, or whether fitting separate curves per difficulty bin or input cluster would recover predictive power. The paper also does not quantify how much the effective $\Delta\mathbf{h}$ varies across inputs for weight updates versus LoRA, which would help practitioners assess when the manifold model can be trusted.

Mitigation status. Not addressed. The paper acknowledges that "input-dependent updates can be more sensitive to the evaluation distribution" (Appendix D.2) but proposes no solution — no conditional modeling, no input-dependent parameterization, and no diagnostic for determining when the model will generalize. This is a significant gap because the mechanistic framework is the paper's primary theoretical contribution, and its failure on two of the three method families it claims to unify limits its practical scope.

Difficulty Estimation Has No Equivalent in This Framework — All Inputs Are Treated Uniformly

Unlike the reference scaling-laws paper, which conditions its entire compute-optimal allocation on estimated prompt difficulty (Section 3.2 of the reference example), this paper treats all inputs within a dataset as interchangeable. The mechanistic model fits a single set of curve parameters ( $\alpha_p$ , $\beta_p$ , $L_{\pm}$ , $p_{\pm}$ , etc.) to the aggregate log-odds curves averaged over all inputs, implicitly assuming that the geometric relationship between the steering direction and the activation manifold is approximately constant across queries.

The consequence. The paper provides no way to characterize which inputs are more "steerable" — which queries will show a strong linear preference response (high $\alpha_p$ ) before utility decay sets in, and which will show immediate degradation. The fitted parameters represent an average over the dataset, but a practitioner deploying steering in a real application faces heterogeneous inputs. Some queries may be safely steered with a large $m$ , while others may collapse into incoherence at the same $m$ . The paper provides no per-input difficulty metric, no method for predicting steerability from the query text alone, and no adaptive strategy for varying $m$ based on input characteristics.

This is particularly problematic for the SPLIT training objective. SPLIT produces a single set of intervention parameters (a weight update, LoRA factors, or a steering vector) applied uniformly to all inputs at a fixed $m$ . But if the preference–utility trade-off varies across inputs — as the paper's own mechanistic model implies it should, since different inputs start at different points on the activation manifold — then a single $m$ will be simultaneously too weak for some queries (failing to achieve desired preference) and too strong for others (causing utility collapse). The paper's evaluation in Table 3 uses a single (unspecified) $m$ , masking this heterogeneity.

What the paper shows about this limitation. The paper does not measure per-input variability in the mechanistic parameters. The curve-fitting analysis reports only aggregate $R^2$ values and does not report, for example, the variance of $\alpha_p$ or $L_{\pm}$ across inputs within a dataset. The generalization results (Table 6) hint at substantial input-dependent variation for weight-based methods, but the paper does not decompose this into per-input effects. This omission is not flagged as a limitation by the authors.

Mitigation status. Not addressed. The paper does not propose any difficulty-estimation mechanism, adaptive $m$ -selection strategy, or per-input steerability diagnostic. A natural extension — fitting the mechanistic model per-input and using the fitted $\alpha_p$ as a steerability score — is not explored. This limits the practical deployability of the framework because a practitioner has no guidance for choosing $m$ on a per-query basis.

The SPLIT Evaluation Uses a Single, Unspecified Operating Point — the $m$ -Dependence of Gains Is Unknown

The paper's central methodological claim is that SPLIT improves preference while preserving utility by producing intervention parameters with a more favorable geometric relationship to the activation manifold — higher $\alpha_p$ (stronger projection) and larger $L_{\pm}$ (slower decay). However, Table 3 evaluates all methods at a single operating point: one fixed intervention multiplier $m$ per method, with the specific $m$ values not reported.

The consequence. The mechanistic model predicts that the preference–utility trade-off is a curve, not a point. A method that appears superior at one $m$ might be inferior at another if its curves cross. For instance, SPLIT might achieve higher preference than RePS at the chosen $m$ , but RePS might achieve the same preference with less utility degradation at a different $m$ . Without evaluating the full $m$ -sweep for each method, the paper cannot distinguish between two scenarios: (1) SPLIT genuinely shifts the entire preference–utility Pareto frontier outward (achieves higher preference at every utility level, or higher utility at every preference level), or (2) SPLIT merely produces a different-shaped curve that happens to intersect the baseline curves favorably at the chosen evaluation point.

This is not a hypothetical concern. The paper's own Figure 2 shows that different methods have curves with different slopes, saturation points, and decay rates. The Vector RePS curve in Figure 2 (top panel) appears to have a steeper initial preference slope but earlier saturation than other methods — at some $m$ values it would outperform alternatives, and at others it would underperform. Without sweeping $m$ for SPLIT and comparing the full curves to baselines, the paper cannot claim that SPLIT produces a "better" intervention in the geometric sense, only that it achieved better scores at the particular $m$ values tested.

What the paper shows about this limitation. The paper does not report the $m$ values used in Table 3, does not show preference–utility curves for SPLIT-trained interventions alongside baselines, and does not report fitted mechanistic parameters ( $\alpha_p$ , $L_{\pm}$ , $p_{\pm}$ ) for SPLIT. The methods section states that "reasonable hyperparameter tuning" was performed (Appendix A), but it is unclear whether $m$ was tuned per method (which would make the comparison fair if each method is evaluated at its own optimum) or held constant (which could favor SPLIT if its optimal $m$ differs from the baseline optimum). The paper makes no claim about whether SPLIT shifts the Pareto frontier or merely shifts the curve.

Mitigation status. Not addressed. The paper does not plot Pareto frontiers, does not sweep $m$ for SPLIT evaluations, and does not report fitted parameters. The authors do not flag this as a limitation, but it substantially weakens the claim that SPLIT provides "better overall performance" (Section 5.2), because "better" at one operating point does not imply better across the full control range — which is precisely what the paper's own mechanistic framework teaches us to care about.

The Framework Is Validated on a Single Task Family (Attribute-Level Control) with Small Test Sets and No Statistical Rigor

All experiments are conducted on three datasets — Psychopathy (personality classification), PowerSeeking (open-ended generation about power-seeking behavior), and AxBench (concept-attribute generation) — all of which fall within the narrow category of attribute-level concept control: making outputs more or less positive, power-seeking, or aligned with a single abstract concept. The paper does not evaluate on complex reasoning tasks (e.g., MATH, as in the reference example), multi-turn dialogue, safety-critical content filtering, code generation, factual accuracy, or any task requiring compositional reasoning rather than single-attribute modulation.

The consequence. The paper's claims about the generality of the preference–utility trade-off and the mechanistic model are limited to a specific regime of LLM control. It is unknown whether the three-phase preference pattern and monotonic utility decay hold for interventions targeting more complex behaviors — for instance, steering a model to follow a multi-step reasoning procedure, to avoid specific logical fallacies, or to maintain consistency across long contexts. In these settings, "utility" may not be as cleanly separable from "preference" as the paper's paired-example framework assumes — the concept-positive and concept-negative answers may not share a common utility factor $P(u \mid q)$ because generating a valid reasoning chain requires more than just producing a coherent string of tokens; it requires the chain to be logically sound.

Furthermore, the small test set sizes (100 for Psychopathy, 200 for PowerSeeking, 80 for AxBench) mean that the reported performance differences in Table 3 may not be statistically reliable. The paper reports no confidence intervals, standard errors, or significance tests for any result. Differences in the second or third decimal place — for instance, the 0.005 gap between Vector SPLIT (1.6475) and LoRA RePS (1.6525) on Gemma-2-9B-IT AxBench harmonic mean — are almost certainly within sampling error given 80 test examples. The paper's claim that SPLIT "consistently achieves higher scores" (Section 5.2) is not statistically substantiated at these sample sizes.

What the paper shows about this limitation. The paper acknowledges in the Limitations section that "our experiments focus primarily on attribute-level control (e.g., sentiment, style), leaving the applicability to complex multi-turn reasoning or safety-critical content largely unexplored." This is an honest disclosure, but it is buried in the final paragraphs and is not reflected in the paper's bolder claims about unified dynamics and mechanistic understanding — which are presented as general insights about LLM control, not as findings specific to attribute-level steering.

The small test set problem is not acknowledged. The paper uses 64 training and 8 test instances per concept for AxBench (Appendix A), meaning the test results average over only 80 examples across 10 concepts. The PowerSeeking test set of 200 examples and Psychopathy test set of 100 examples are also small for drawing fine-grained comparisons between methods that differ by hundredths of a point on a 0–2 scale.

Mitigation status. The dataset scope limitation is acknowledged but not addressed — no experiments on reasoning, dialogue, or safety tasks are conducted or proposed. The statistical limitation is not acknowledged at all. The paper does not report error bars, does not perform bootstrap or significance tests, and does not discuss the reliability of conclusions drawn from the reported sample sizes.

The Utility Measurement Framework Ignores a Large Fraction of the Model's Output Distribution

The paper's utility metric UtilOdds (Equation 6) measures the total probability mass assigned to the two specific answer strings $A_p$ and $A_n$ in the polarity pair. However, $P(A_p \mid q) + P(A_n \mid q)$ typically sums to far less than 1 — the model assigns significant probability mass to other completions (coherent but off-concept answers, incoherent strings, instruction violations, etc.). The utility log-odds metric is:

$\text{UtilOdds}(q) = \log\frac{e^{-\mathcal{L}_p} + e^{-\mathcal{L}_n}}{1 - e^{-\mathcal{L}_p} - e^{-\mathcal{L}_n}}$

where the denominator $1 - e^{-\mathcal{L}_p} - e^{-\mathcal{L}_n}$ is the probability mass assigned to all other tokens besides $A_p$ and $A_n$ .

The consequence. UtilOdds treats all non-pair probability mass as equally indicative of utility loss, whether it goes to (a) coherent but conceptually neutral completions, (b) coherent but off-task completions, (c) incoherent or garbled text, or (d) instruction violations. These are qualitatively different failure modes with different practical implications. A model that shifts probability from $A_p$ and $A_n$ to a coherent but concept-neutral third answer ("The restaurant was okay, nothing special") has lost utility under this metric, even though it is still producing valid text — it is simply not strongly expressing either pole of the concept. A model that shifts probability to garbled text ("food food wonderful ??? service 19% ##") has lost utility in a much more severe sense — it is failing at basic language generation.

The paper's analysis conflates these two degradation patterns. When the mechanistic model attributes utility decline to "off-manifold deviation" and "activation invalidation" (Section 4.1), the implication is that utility loss corresponds to the model becoming unable to produce any valid text. But UtilOdds would also decline if the model simply became less polarized — spreading probability more evenly across many valid completions rather than concentrating on the two specific alternatives in the pair. The paper provides no way to distinguish between these two interpretations, making the mechanistic link between utility decay and manifold deviation weaker than the $R^2$ values suggest.

What the paper shows about this limitation. The paper's own definition of utility distinguishes between "task-valid" completions (coherent, relevant, format-consistent) and "invalid" completions (incoherent, off-topic, instruction-violating) in Section 3.1. However, the operational metric UtilOdds does not implement this distinction — it only measures whether probability mass falls on the specific strings $A_p$ and $A_n$ , not whether it falls on other valid strings. The paper does not report what fraction of the probability mass typically goes to $A_p + A_n$ versus other tokens, making it impossible for a reader to assess how much of the "utility" signal is driven by genuine capability loss versus reduced concept polarization.

Mitigation status. Not addressed. The paper does not decompose the non-pair probability mass into coherent vs. incoherent components, does not compute an alternative utility metric based on output quality judgments (e.g., fluency scores from the LLM judge, which are available for AxBench), and does not validate that UtilOdds correlates with generation quality measures like perplexity, fluency, or instruction-following scores. This is a fundamental measurement validity concern that weakens the mechanistic interpretation, though it does not affect the paper's descriptive findings about curve shapes (which are grounded in the same UtilOdds metric and would still hold regardless of what the metric "really" measures).

The Paper Never Combines Intervention Forms or Explores Interactions Between Layers

The unified framework expresses weight updates, LoRA, and steering vectors as instances of the same affine equation, implying that they are interchangeable and can be studied in parallel. However, the paper evaluates each intervention form in isolation — weight updates alone, LoRA alone, or vector steering alone — and never investigates what happens when multiple forms are combined (e.g., applying a LoRA adapter and a steering vector simultaneously) or when interventions are applied at multiple layers simultaneously.

The consequence. The framework's implication is that any intervention can be reduced to an effective $\Delta\mathbf{h}$ at a specific layer. If this is truly a complete description, combining two interventions should produce an effective $\Delta\mathbf{h}$ that is the sum of their individual contributions, and the resulting preference–utility dynamics should be predictable from the individual curves. But the paper provides no evidence for this linear superposition claim. Interventions applied at different layers may interact nonlinearly — a weight update at the MLP down-projection of layer 20 might change the effective manifold geometry for a steering vector applied to the residual stream at the same layer, or a LoRA adapter at layer 15 might shift the activations that the layer-20 intervention processes.

Without such experiments, the paper's claim of a "unified view" is more of a conceptual unification (showing that methods can be written in the same notation) than an operational unification (showing that the effects of different methods compose predictably). A practitioner who wants to use both LoRA and steering vectors — for instance, LoRA for personalization and vector steering for safety — has no guidance from this paper on how the two will interact.

What the paper shows about this limitation. The paper does not mention the possibility of combining interventions or the assumption of linear superposition. All experiments in Figures 2, 4, 5 and Table 3 evaluate single interventions applied at a single layer (layer 20 for Gemma-2-9B-IT, layer 14 for Qwen-2.5-7B-Instruct). The layer choice is held constant and not ablated, so it is impossible to determine whether the observed dynamics would change if interventions were applied at different layers or at multiple layers.

Mitigation status. Not addressed. The paper does not flag the lack of multi-intervention or multi-layer experiments as a limitation, does not propose a superposition model, and does not suggest future work on compositional intervention design. This is a gap between the conceptual unification the paper achieves (shared notation) and the practical unification a practitioner would need (predictable composition of multiple control signals).

7. Implications and Future Directions

How This Work Changes the Landscape

This paper makes a conceptual reframing with diagnostic power, not a paradigm shift. It does not introduce a fundamentally new control method, nor does it demonstrate order-of-magnitude performance improvements over baselines. What it does — and does convincingly — is provide a shared language and measurement framework that transforms the study of LLM control from a collection of disconnected empirical findings into a field with common coordinates, predictable dynamics, and falsifiable mechanistic hypotheses.

Prior to this work, researchers studying local weight fine-tuning, LoRA-based adaptation, and activation steering were effectively working in different subfields, publishing in different venues, and using different evaluation protocols. Each paper demonstrated that its method "worked" on some benchmark, and the mechanisms were explained in method-specific terms — linear representation hypothesis for steering vectors, low-rank matrix factorization for LoRA, localized factual associations for weight editing. This fragmentation made it impossible to ask basic comparative questions: Does LoRA degrade utility faster or slower than vector steering as control strength increases? Do all methods exhibit the same saturation behavior? Is the preference–utility trade-off fundamental to the control problem or an artifact of specific training procedures?

The paper's unified dynamic weight formulation (Equation 7: $\mathbf{h}_{i+1} = (\mathbf{W} + m_1 \Delta\mathbf{W})\,\mathbf{h}_{i} + (\mathbf{b} + m_2 \Delta\mathbf{b})$ ) answers these questions by showing that the three method families are not merely analogous but mathematically identical in form. This is a stronger claim than taxonomic grouping — it asserts that at the level of the forward pass through a linear layer, weight updates, LoRA, and steering vectors are doing exactly the same thing: adding a change term $m\,\Delta\mathbf{h}$ to the activation, with each method differing only in how $\Delta\mathbf{h}$ is parameterized and how many degrees of freedom it has. Table 1 makes this explicit: local weight updates produce an input-dependent $\Delta\mathbf{h} = m(\Delta\mathbf{W}\,\mathbf{h}_i + \Delta\mathbf{b})$ with $d_{\text{in}} \times d_{\text{out}} + d_{\text{out}}$ parameters, LoRA produces a low-rank input-dependent $\Delta\mathbf{h} = m(\mathbf{BA}\,\mathbf{h}_i)$ with $d_{\text{in}} \times r + r \times d_{\text{out}}$ parameters, and vector steering produces an input-independent $\Delta\mathbf{h} = m\,\Delta\mathbf{b}$ with only $d_{\text{out}}$ parameters.

This reframing matters because it changes what questions researchers ask. Before, the relevant question was "Which method works better?" — a question that has no general answer because it depends on compute budget, data availability, and the specific concept being controlled. After, the relevant question becomes "Given a desired preference–utility operating point, which method's $m$ -dependent curve offers the most favorable trade-off?" This transforms method selection from a categorical choice into a continuous optimization problem, where different methods are points (or, more precisely, curve families) in a shared preference–utility plane parameterized by the intervention multiplier $m$ .

Reconciling prior contradictions. The paper resolves a specific tension in the activation steering literature that had produced apparently conflicting findings. Bigelow et al. (2025) observed that steering yields "an approximately linear trend in posterior odds, but mainly in the small-scale regime" — suggesting steering works as the linear representation hypothesis predicts, but only weakly. Meanwhile, Bricken et al. (2023) and Wollschläger et al. (2025) reported "non-monotonic or adverse effects under stronger steering, challenging a naive global linearity assumption" (Section 6). These findings appeared contradictory: one camp found linearity, the other found non-linearity and degradation.

The paper's three-phase preference model resolves this: both are correct observations of different regions of the same underlying curve. The linear regime at small $|m|$ is where $D_p(m) \approx 1$ — the steering direction is still near the manifold intersection, so the preference projection $(\alpha_p m + \beta_p)$ dominates and log-odds grows approximately linearly. The non-monotonic and adverse effects emerge in the transitional and convergence regions, where $D_p(m)$ decays and the projection term is increasingly attenuated. Bigelow et al. observed behavior in the linear regime; Bricken et al. and Wollschläger et al. pushed into the decay regime and observed the consequences. They are not contradictory — they are observations at different steering strengths along the same $m$ -dependent trajectory.

Which research directions become more attractive, and which become less so. The paper's findings redirect research attention in two specific ways:

More attractive: verifier and manifold-geometry research. The paper identifies the validity decay function $D(m)$ — governed by the characteristic length scales $L_{\pm}$ and decay rates $p_{\pm}$ — as the primary bottleneck for scaling control strength. This implies that improving the quality of interventions (making $\Delta h$ more parallel to the activation manifold, i.e., larger $L_{\pm}$ ) is more impactful than developing more expressive parameterizations of $\Delta h$ . Research on understanding the local geometry of activation manifolds — what directions are tangent to the manifold versus normal to it, how this varies across layers and model families — becomes directly relevant to control practitioners. Similarly, research on verifier robustness in the RLHF literature finds a natural analog here: just as reward model over-optimization limits RLHF scaling, validity decay limits steering scaling.
Less attractive: purely empirical benchmarking of control methods without mechanistic analysis. The paper demonstrates that all methods exhibit the same qualitative dynamics. A paper that simply reports "LoRA achieves 0.02 higher accuracy than vector steering on dataset X" without characterizing the full preference–utility curve as a function of $m$ is, by this paper's lights, asking an underdetermined question. The result could be reversed at a different $m$ , and without understanding the geometric parameters ( $\alpha_p$ , $L_{\pm}$ ) that govern the curve, a point estimate provides no guidance for practitioners choosing an operating point. This pushes the field toward curve-reporting standards — showing preference and utility as functions of control strength, not as single numbers.
Less attractive: developing ever-more-complex intervention parameterizations. The paper shows that vector steering, the simplest method with the fewest parameters ( $d_{\text{out}}$ ), achieves the best harmonic means in most settings (Table 3: Vector SPLIT achieves 1.6475 and 1.6500 on Gemma-2-9B-IT and Qwen-2.5-7B-Instruct AxBench harmonic mean, matching or exceeding weight and LoRA variants). The bottleneck is not expressivity — full weight updates have orders of magnitude more parameters but do not consistently outperform vectors. The bottleneck is geometry: finding directions that align with the target concept (high $\alpha_p$ ) while staying parallel to the manifold (large $L_{\pm}$ ). This suggests that research effort is better spent on better training objectives (like SPLIT) and on understanding manifold geometry than on designing richer parameterizations of $\Delta h$ .

Follow-Up Research This Work Enables

Per-input steerability prediction and adaptive $m$ -selection. The paper's mechanistic model fits a single set of curve parameters ( $\alpha_p$ , $\beta_p$ , $L_{\pm}$ , $p_{\pm}$ ) aggregated over all inputs in a dataset. But the model implies that these parameters should vary per input — different queries start at different points on the activation manifold, and the effective projection $\alpha_p$ depends on how $\Delta h$ interacts with the specific $h_i$ . A natural extension is to fit the mechanistic model per-input and extract a "steerability score" from the fitted $\alpha_p$ and $L_{+}$ parameters. Does steerability correlate with simple input features (query length, syntactic complexity, baseline model confidence)? Can a lightweight classifier predict steerability from the query text alone, without requiring the expensive per-input curve-fitting procedure? If so, this would enable adaptive $m$ -selection: use a small $m$ for queries flagged as low-steerability (where utility decays quickly) and a larger $m$ for high-steerability queries. This would address the single- $m$ limitation in the current SPLIT evaluation and directly connect the mechanistic theory to practical deployment. A strong follow-up would report preference–utility curves with and without adaptive $m$ , showing that adaptive selection shifts the Pareto frontier outward compared to a fixed $m$ .

Validating the projection–decay factorization through causal intervention on $D(m)$ . The paper's central mechanistic claim is that preference log-odds factor as $(\alpha_p m + \beta_p) D_p(m) + b_p$ , with validity decay $D_p(m)$ attenuating the projection. This is a fitted model, not a causally verified mechanism. A strong test would be to manipulate $D_p(m)$ independently of the projection term and observe the predicted effect on preference and utility. How could this be done? One approach: apply the preference steering vector $\Delta h$ and simultaneously apply a separate "manifold-orthogonal" perturbation that pushes representations off-manifold without affecting the preference projection (i.e., a perturbation $\Delta h_{\perp}$ that is orthogonal to $\boldsymbol{\omega}_p$ but has small $L$ — it cuts sharply across the manifold). The prediction: preference should degrade despite the unchanged $\alpha_p m$ projection term, because $D_p(m)$ decreases. If observed, this would provide causal evidence for the factorization. If not observed — if preference is unaffected by off-manifold perturbations — it would suggest the decay factor is an artifact of the specific steering directions studied, not a general mechanism, and would substantially weaken the paper's theoretical contribution.

Layer-wise characterization of manifold geometry for control. The paper applies interventions at a single layer (layer 20 for Gemma-2-9B-IT, layer 14 for Qwen-2.5-7B-Instruct) following Bigelow et al. (2025) without ablation. The preference–utility dynamics likely depend on which layer is targeted. Lower layers encode more syntactic and local features; higher layers encode more semantic and abstract features. The activation manifold at different depths likely has different geometric properties — perhaps early-layer manifolds are "tighter" (smaller $L_{\pm}$ ) because syntactic processing is more rigid, while later-layer manifolds are "looser" (larger $L_{\pm}$ ) because semantic representations tolerate more variation. A systematic study applying the same steering concept at every layer and fitting the mechanistic model at each would produce a layer-wise geometric profile: $\alpha_p(l)$ , $L_{\pm}(l)$ , $p_{\pm}(l)$ as functions of layer index $l$ . This profile could guide layer selection for practitioners (choose the layer with highest $\alpha_p \times L_{+}$ product) and would test whether the manifold hypothesis makes consistent predictions across network depth. A strong follow-up would also test whether combining interventions at multiple layers yields linear superposition (the effective $\Delta h$ is the sum of per-layer contributions) or exhibits nonlinear interactions, which the current single-layer study cannot address.

Stress-testing the model on complex, multi-step reasoning tasks. All current experiments are on attribute-level control: shifting sentiment, power-seeking tendency, or single-concept attributes. In these settings, the preference–utility factorization (Equation 4) is plausible because generating a valid positive restaurant review and generating a valid negative restaurant review likely share a common "restaurant review competence" factor $P(u \mid q)$ . But this factorization may break down for complex behaviors. Consider steering a model to follow a specific multi-step reasoning protocol (e.g., "always show your work in three steps: identify variables, set up equations, solve"). Here, the positive answer $A_p$ (a correctly reasoned solution following the protocol) and the negative answer $A_n$ (a shortcut solution or one using a different protocol) may not share a common utility factor — the model's ability to produce any valid solution may be entangled with which protocol it follows. A strong follow-up would apply the paired-example framework to a reasoning benchmark like MATH (Hendrycks et al., 2021) or GSM8K, constructing polarity pairs where $A_p$ shows step-by-step reasoning and $A_n$ shows a correct answer without reasoning, then fit the mechanistic model. If the $R^2$ values remain high, the factorization generalizes to reasoning. If they degrade — particularly if utility shows a projection effect (violating the $\boldsymbol{\omega}_u^{\mathsf{T}} \Delta h \approx 0$ assumption) — it would reveal a fundamental boundary for the framework: concept preference and task utility are separable only for "shallow" attribute control, not for behaviors that are constitutive of the task itself.

SPLIT with explicit geometric regularization. The SPLIT objective optimizes preference and utility through loss terms on the output probabilities ( $\mathcal{L}_{\text{pref}}$ and $\mathcal{L}_{\text{util}}$ ), but it does not directly encourage the intervention parameters to produce a steering direction that is parallel to the manifold (large $L_{\pm}$ ). The mechanistic model suggests a more direct approach: add a geometric regularization term to the training objective that penalizes off-manifold deviation. This could be operationalized in several ways: (a) penalize the variance of $\Delta\mathbf{h}$ across a batch of inputs (encouraging the effective steering to be input-independent, which the generalization results in Table 6 suggest correlates with manifold alignment); (b) add an auxiliary loss that encourages the steered activations $\tilde{h}(m)$ at a small $m$ (e.g., $m = 0.1$ ) to remain within the convex hull of unsteered training activations; or (c) use a separately trained "manifold discriminator" that scores activations as on/off-manifold and penalize low scores for steered activations. If geometric regularization increases the fitted $L_{\pm}$ parameters while maintaining $\alpha_p$ , it would provide direct evidence for the manifold hypothesis and potentially produce interventions with better preference–utility trade-offs than SPLIT alone. A strong follow-up would plot the Pareto frontiers for SPLIT with and without geometric regularization, showing an outward shift.

Connecting validity decay to verifier over-optimization in RLHF. The paper identifies utility degradation under strong steering as the primary failure mode, and the mechanisms — off-manifold deviation causing unreliable decoding — are reminiscent of a different phenomenon: reward hacking in RLHF, where optimizing against a learned reward model eventually produces outputs that score highly but are low-quality because the policy exploits blind spots in the reward model. Both phenomena involve a proxy signal (the PRM or the steering direction) being over-optimized relative to the true objective (correctness or task validity), and both exhibit a characteristic "rise then plateau or decline" pattern as the optimization budget increases. The paper's validity decay function $D(m)$ could serve as a model for reward model over-optimization: as the KL divergence between the policy and the base model grows (analogous to $m$ increasing), the reward model's predictions become less calibrated because the policy moves into regions of input space where the reward model was not trained (analogous to off-manifold deviation). A strong follow-up would fit the RQ decay model to RLHF reward curves from the literature and test whether $D(m)$ with fitted $L_{\pm}$ and $p_{\pm}$ captures the shape of reward over-optimization as well as it captures utility decay under steering. If so, the activation manifold hypothesis provides a unifying geometric explanation for both phenomena.

Practical Applications and Downstream Use Cases

Safe deployment of attribute-controlled chatbots with tunable control strength. A common use case for LLM control is adjusting the "personality" or "style" of a deployed chatbot — making it more formal, more empathetic, more concise, or more aligned with a brand voice. Practitioners currently face a difficult calibration problem: they can train a controlled model (via fine-tuning, LoRA, or steering vectors), but they do not know how strongly to apply the control at inference time. Too weak, and the chatbot defaults to its base behavior; too strong, and it becomes repetitive, incoherent, or ignores user instructions (the utility degradation this paper quantifies).

The paper's unified dynamics (Figure 2) provide a calibration methodology: for a given intervention, sweep $m$ and measure PrefOdds and UtilOdds on a held-out calibration set. The resulting curve directly shows the preference–utility trade-off and allows the practitioner to select an operating point that achieves sufficient preference while keeping utility above a threshold (e.g., $m$ where UtilOdds is within 10% of its maximum). The SPLIT training objective further improves this by producing interventions where the linear preference regime extends further before utility decay sets in (higher $L_{\pm}$ ), giving practitioners a wider range of usable $m$ values. The paper's numbers suggest that for vector interventions on Gemma-2-9B-IT, SPLIT can achieve a concept score of 1.85/2.0 on AxBench with a harmonic mean of 1.65/2.0 (Table 3), indicating strong concept control with preserved instruction-following and fluency — exactly the operating regime a practitioner would target.

Cost-aware method selection for fine-tuning APIs and on-device deployment. The paper's unified framework enables a parameter-count-aware comparison across intervention types (Table 1). Vector steering requires only $d_{\text{out}}$ parameters (typically 4096–8192 for modern LLMs) and no modification to model weights — the steering vector can be applied at inference time as a bias offset with zero storage overhead beyond the vector itself. LoRA requires $d_{\text{in}} \times r + r \times d_{\text{out}}$ parameters (for rank $r = 16$ and hidden size 4096, this is ~131K parameters) and requires either merging into weights or storing separate adapter matrices. Full weight updates require $d_{\text{in}} \times d_{\text{out}} + d_{\text{out}}$ parameters (for 4096-dimensional hidden states, this is ~16.8M parameters for a single MLP down-projection layer).

For a practitioner deciding how to deploy controlled models, the paper's results imply that vector steering with SPLIT training is the most parameter-efficient route to strong concept control with preserved utility — it achieves the highest harmonic means in Table 3 while requiring orders of magnitude fewer parameters than weight updates. This has direct implications for edge deployment (where storage and memory are constrained) and for multi-tenant serving (where different users or applications need different control vectors, and storing 8192 floats per concept is feasible while storing full weight deltas is not). A platform offering "style control" as a service could store hundreds of concept vectors (formal, empathetic, concise, technical, creative, etc.) with negligible overhead, apply SPLIT-trained vectors at inference time, and expose a single $m$ slider to end-users — the calibration curves from Figure 2 tell the platform operator exactly what preference–utility trade-off each $m$ value corresponds to.

Self-improvement and synthetic data generation with controlled concept modulation. A growing paradigm in LLM development is using models to generate training data for themselves — generating solutions, filtering by quality, and fine-tuning on the high-quality outputs (the STaR/ReST $^{EM}$ approach). In these pipelines, controlling the diversity and concept distribution of generated data is critical: if all generated solutions express the same style or reasoning approach, the fine-tuned model will collapse to that narrow distribution.

The paper's framework provides a principled way to modulate concept expression in generated data. By sweeping $m$ and measuring PrefOdds, a data generation pipeline can produce outputs at multiple points along the preference continuum — from strongly concept-positive ( $m$ large and positive) through neutral ( $m \approx 0$ ) to strongly concept-negative ( $m$ large and negative). Because the framework also measures utility via UtilOdds, the pipeline can automatically reject outputs where $m$ was too large and utility collapsed, ensuring that all training data meets a minimum quality threshold. The SPLIT objective is particularly well-suited here because it explicitly trains interventions to delay utility decay, maximizing the range of $m$ over which valid outputs can be generated. A concrete use case: generating balanced training data for a sentiment-controlled summarization model, using a SPLIT-trained vector at $m = -2, -1, 0, 1, 2$ to produce summaries with varying sentiment polarity while rejecting any outputs where UtilOdds drops below a threshold.

Diagnosing intervention quality during development without full generation. A practical pain point in developing LLM control methods is the evaluation cycle: training an intervention, generating outputs, and evaluating those outputs (often with expensive LLM-as-judge pipelines) is slow and costly. The paper's paired-example log-odds framework provides a fast, generation-free diagnostic that can be computed directly from the model's token-level probabilities on the polarity pair $(A_p, A_n)$ . A developer can sweep $m$ and compute PrefOdds and UtilOdds in seconds (the time to run one forward pass per $m$ value on the calibration set), producing the full preference–utility curve without generating a single token.

This enables rapid iteration during intervention development: a developer can train a candidate LoRA adapter, quickly sweep $m$ and measure the resulting curve, and immediately see whether the adapter (a) achieves sufficient preference gain (high $\alpha_p$ ), (b) maintains utility over a wide $m$ range (large $L_{\pm}$ ), and (c) generalizes across the calibration set (low variance in fitted parameters). Unpromising interventions can be discarded early without the cost of full generation-based evaluation, and promising ones can be advanced to more expensive human or LLM-judge evaluation. The paper's high $R^2$ values for in-domain curve fitting (Table 2, >0.95 for most settings) suggest that the log-odds metrics are reliable proxies for the underlying dynamics, making this diagnostic trustworthy in practice.