The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning

ArXiv: 2312.01552

🎯 Pitch

Alignment tuning, long thought essential for turning raw LLMs into helpful assistants, is largely decorative—our token-distribution analysis shows base models already know the answers, and alignment merely styles them. With just three curated examples and a system prompt—a method we call URIAL—untuned base LLMs match or beat SFT/RLHF models, including outperforming Llama-2-70b-chat.

1. Executive Summary

This paper analyzes how alignment tuning—the process of transforming base LLMs into AI assistants via supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF)—alters model behavior, finding through token distribution shift analysis that base and aligned LLMs behave nearly identically on most token positions, with alignment predominantly affecting only stylistic tokens like discourse markers and safety disclaimers. Based on this evidence supporting the Superficial Alignment Hypothesis, the authors introduce URIAL (Untuned LLMs with Restyled In-context ALignment), a tuning-free method that aligns base LLMs purely through in-context learning with as few as three constant, carefully restyled examples and a system prompt. Evaluated on the multi-aspect just-eval-instruct benchmark across Llama-2 and Mistral model families, URIAL matches or surpasses SFT-aligned models (Mistral-7b-Instruct) and SFT+RLHF-aligned models (Llama-2-70b-chat), with URIAL on Mistral-7b achieving an average score of 4.63 versus 4.44 for its SFT counterpart, establishing that tuning-based alignment is largely superfluous when the underlying base LLM is sufficiently capable.

2. Context and Motivation

The Core Problem: What Does Alignment Tuning Actually Learn?

The fundamental question this paper tackles is both practical and scientific: when we take a base LLM that has only seen pre-training on raw text corpora and apply alignment tuning (SFT + RLHF), what actually changes in the model's behavior? The field has largely accepted alignment tuning as an essential step—the de facto "unlocking spell" that transforms a raw, sometimes incoherent text model into a helpful AI assistant. But the mechanism behind this transformation remains surprisingly opaque. Does alignment teach the model new knowledge? New reasoning skills? Or something shallower?

This question matters because the alignment process is simultaneously the most resource-intensive and the most poorly understood phase of LLM development. It demands substantial GPU compute, large volumes of high-quality instruction data (often sourced from proprietary models like GPT-4), and human annotations for preference data. Yet despite these costs, the field lacks a rigorous understanding of what is actually being purchased with all this computation. As the authors note in Section 1, "it is important to analyze how exactly alignment tuning alters the behavior of base LLMs."

The gap is not just academic curiosity. Without understanding what alignment tuning contributes, every lab developing LLMs must either blindly replicate the full SFT+RLHF pipeline (with all its associated costs) or risk producing an inadequately aligned model. The latter concern is intensified by the fact that base LLMs—while powerful—exhibit specific failure modes that make them unsuitable as assistants: they repeat questions, generate extraneous content, provide answers in unhelpful formats, and lack the conversational norms that make interactions pleasant and safe for users (Section 3.1).

Why This Problem Is Important

The significance operates on three levels:

Practical: Democratizing alignment. If alignment tuning is indeed "superficial"—teaching style rather than substance—then the massive computational barrier to creating aligned assistants might be unnecessary. Alignment tuning for models exceeding 30B parameters requires resources that most research labs simply do not have (Section 5.1). The paper explicitly notes that this "restricts ordinary labs from aligning extreme-scale LLMs" such as Falcon-180B. A tuning-free alignment method would dramatically lower this barrier, democratizing access to assistant-capable LLMs.

Scientific: Understanding LLM capabilities. The paper identifies a deeper epistemological problem with current LLM research: we cannot accurately attribute capabilities to pre-training versus alignment. As stated in Section 5.4, "comparisons between aligned LLMs cannot directly reflect the quality of their base counterparts because the tuning process can vary greatly (e.g., data, hyper-parameters, etc.)." This makes it difficult to know whether improvements in pre-training recipes are actually producing better base models, or whether downstream improvements are artifacts of particular alignment procedures. The paper explicitly advocates for "accurately distinguishing which knowledge and reasoning capabilities originate from pre-training as opposed to those that must be acquired through alignment tuning."

Economic: Evaluating pre-training checkpoints. During pre-training, model developers periodically checkpoint and evaluate their models. If alignment tuning is always required to assess a model's potential as an assistant, this creates an expensive feedback loop—tune, evaluate, resume training, repeat. A tuning-free method that can expose a base model's latent alignment capability would enable "frequently evaluating base LLMs during the pre-training process" (Section 5.4), accelerating the development cycle.

Prior Approaches and Their Shortcomings

The Alignment Tuning Paradigm

The standard approach to creating AI assistants from base LLMs involves a two-stage pipeline (Section 1, with background in Section 2):

1. Supervised Fine-Tuning (SFT; also called instruction tuning): The base model is fine-tuned on instruction-response pairs, where instructions are user queries and responses are desired assistant outputs. These pairs are either human-authored or, increasingly, collected by prompting proprietary LLMs like GPT-4 (Taori et al., 2023; Wang et al., 2023).

2. Reinforcement Learning from Human Feedback (RLHF): The SFT model is further refined using a reward model trained on human preference judgments. This reward model scores outputs on dimensions like helpfulness, honesty, and harmlessness, and the language model is optimized against this reward using reinforcement learning (Bai et al., 2022; Ouyang et al., 2022).

This pipeline has produced the most capable open-domain assistants—Llama-2-chat, Vicuna, Mistral-Instruct, ChatGPT—and has become the standard recipe. The assumption embedded in this approach is that alignment tuning teaches models something substantive: perhaps better reasoning strategies, safer knowledge boundaries, or more robust instruction-following capabilities.

The problem with this assumption is that prior evidence had already begun to challenge it, but in piecemeal and indirect ways.

LIMA and the "Superficial Alignment Hypothesis"

The crucial precursor to this paper is LIMA (Zhou et al., 2023), which introduced the Superficial Alignment Hypothesis. LIMA demonstrated that fine-tuning a 65B parameter model on merely 1,000 carefully curated examples could produce a model with high alignment performance—achieving competitive win rates against ChatGPT on certain evaluations. The implication was striking: if 1,000 examples suffice, then the base model must already contain most of the knowledge and capabilities needed for assistant behavior. Alignment tuning, under this hypothesis, primarily teaches the model to adopt a particular style of interaction—the format, tone, and conversational norms of an AI assistant—rather than teaching it new substantive capabilities.

The paper explicitly positions itself as building on this hypothesis (Section 1): "Zhou et al. (2023) demonstrates that SFT with as few as 1,000 examples can also yield high-quality aligned models, thus providing indirect support for this hypothesis. However, conclusive and direct supporting evidence for the superficial alignment hypothesis remains underexplored."

The key word here is indirect. LIMA's evidence is performance-based: a lightly-tuned model performs well, so alignment must be superficial. But this is circumstantial. It does not actually show what the alignment tuning changed or failed to change inside the model. A skeptical interpretation: perhaps the 1,000 examples were exceptionally well-chosen, or the phenomena are specific to certain model scales, or the base model genuinely lacked assistant capabilities that the fine-tuning efficiently installed. LIMA's evidence cannot distinguish between "alignment taught style" and "alignment efficiently taught substance using unusually informative examples."

This paper aims to provide direct evidence—evidence that peers inside the model's probability distributions and shows exactly which token decisions are affected by alignment and which are not. As stated in Section 1, the goal is to "analyze how exactly alignment tuning alters the behavior of base LLMs" by "directly comparing the token distributions between base LLMs and their aligned versions."

The In-Context Learning Baseline Gap

Prior to this paper, the main alternative to tuning-based alignment was in-context learning (ICL) with base models. However, systematic investigation of ICL for alignment was surprisingly limited:

• Most ICL research focused on narrow NLP tasks (classification, multiple-choice QA) rather than open-domain assistant behavior (Wei et al., 2023; Zhang et al., 2022). Aligning a model to serve as a general-purpose chat assistant involves a much broader set of skills—refusing harmful queries, structuring multi-paragraph responses, maintaining conversational engagement—that had not been systematically studied through the ICL lens.

• Existing ICL-based alignment attempts used either basic examples or computationally expensive retrieval. Bai et al. (2022) used a prompt with 14 examples (~7,000 tokens) for their RLHF training process, but this was as a component of tuning, not as a standalone alignment method. Han (2023), a closely related concurrent work, demonstrated that retrieval-augmented ICL with ~10 dynamically retrieved examples could achieve impressive performance, but retrieval incurs substantial inference-time costs because the prompt changes for every query, preventing KV-cache reuse (Section 3.2). The paper treats this as the "Retrieval ICL" baseline.

• The style of ICL examples was underappreciated as a lever. Min et al. (2022) had shown that ICL primarily concerns the style and format of demonstrations rather than their truth content, but this insight had not been systematically applied to the alignment problem. Prior ICL approaches typically used instruction-output pairs "in a plain and basic style" (Section 3.2), such as outputs from Flan-Collection or Alpaca, which lack the structured, engaging, safety-conscious characteristics of aligned assistant behavior.

• System prompts were primarily used with aligned models, not base models. The concept of a system-level prompt establishing the assistant's role and behavioral guidelines had been developed for models like Vicuna and Llama-2-chat, but its application to base LLMs through pure ICL "is still relatively under-explored" (Section 3.3).

The Evidence Gap: No Direct Measurement of What Changes

Collectively, the prior work suffered from a measurement problem. Evaluations compared final outputs (win rates, overall quality scores) between aligned and unaligned models, but these comparisons are coarse and do not reveal the mechanism. As the paper notes in Section 4.2, evaluations like AlpacaEval and LIMA's approach are "coarse-grained and potentially biased towards unclear response aspects (e.g., favoring longer candidates and more polite responses)." Even the strong performance of lightly-tuned models could be explained by training on particularly informative examples rather than by the superficiality of the alignment process itself.

What was missing was a method for looking inside the model—for comparing, token by token, how the aligned and base models make predictions given the same context, and for categorizing which decisions change versus which remain stable. The paper's token distribution shift analysis (Section 2) fills this gap. By feeding the aligned model's partial output back into the base model and measuring whether the base model would make the same top token prediction, the analysis provides a direct, mechanistic window into the effect of alignment. This is not a behavioral comparison of end-to-end outputs; it is a comparison of the underlying probability distributions that govern generation, offering granular evidence about exactly where alignment matters and where it does not.

Where Existing Approaches Fall Short: The Alignment Tuning Downsides

Beyond the evidence gap, the paper identifies specific practical problems with tuning-based alignment that provide additional motivation for seeking alternatives:

Knowledge forgetting. Wang et al. (2023) demonstrated that SFT can significantly degrade base model performance on factual and reasoning benchmarks. For instance, SFT on Llama-13b caused MMLU to drop from 42.5 to 30.3, and Codex-Eval to drop from 26.6 to 13.4. Even more dramatically, SFT with SuperNI caused Llama-13B to nearly lose all BBH reasoning ability (36.9 → 2.8). The paper identifies concrete examples of this phenomenon in its own experiments (Appendix B.1), showing Mistral-7B-Instruct incorrectly answering "No, Facebook corporation did not change its name" when the base model with URIAL correctly answers that it changed to Meta Platforms Inc.

Over-sensitivity from RLHF. The paper's case studies (Appendix B.2) show that RLHF-ed models like Llama-2-70b-chat can become overly cautious, refusing to answer harmless creative queries like "What would be a great twist for a murder mystery?" because they conflate fictional violence with harmful content. The base model with URIAL, by contrast, provides ten creative suggestions while maintaining an engaging tone—demonstrating that the base model already contains both the creative knowledge and the ability to express it safely when properly prompted.

Resource intensity. Tuning-based alignment "typically demands substantial resources, such as GPU nodes, a large amount of instruction data, and human annotations, making the process both costly and time-consuming" (Section 5.1). For models beyond 30B parameters, this creates a genuine barrier to entry for the broader research community.

Evaluation opacity. Because different aligned models undergo different tuning procedures (different data mixtures, different hyperparameters, different RLHF reward models), it becomes impossible to compare base models fairly through their aligned versions. The paper argues this "hinders our understanding of which knowledge and reasoning capabilities originate from pre-training as opposed to those that must be acquired through alignment tuning" (Section 1). A tuning-free alignment method that consistently exposes base model capabilities would enable fairer apples-to-apples comparisons between different pre-trained models.

How This Paper Positions Itself

The paper occupies a specific, well-motivated niche at the intersection of two concerns:

1. As an analysis paper: It provides the first direct, token-level evidence for the Superficial Alignment Hypothesis. Rather than inferring superficiality from the surprising effectiveness of minimal tuning (LIMA), it demonstrates through distribution shift analysis exactly where alignment affects model behavior and where it does not. The finding that 77.7% of tokens are unshifted (top-1 tokens match between base and aligned models) and 92.2% are within the top 3 (Figure 3, Section 2.2) provides quantitative evidence that alignment affects a surprisingly small fraction of decoding decisions. The further finding that shifted positions predominantly involve stylistic tokens (discourse markers, transitional phrases, safety disclaimers) rather than knowledge-bearing words provides qualitative evidence about what those changes are. The observation that distribution shift diminishes over time during decoding (Figure 4, Section 2.2)—with aligned tokens' base rank dropping below 5 shortly after the first few positions—indicates that alignment's influence is concentrated in the early parts of responses where stylistic framing is established.

2. As a methods contribution: It introduces URIAL, a tuning-free alignment method that leverages the analysis findings to achieve competitive performance with SFT and RLHF. The method's design is directly informed by the token distribution analysis: because alignment mainly adds stylistic framing around knowledge that base models already possess, the key is to provide that stylistic framing through in-context examples and system prompts rather than through weight modification. The "restyled" examples are explicitly constructed to incorporate the stylistic patterns (greeting affirmations, numbered lists with detailed bullet points, concluding summaries, safety disclaimers) that the distribution analysis identified as the primary effects of alignment tuning.

URIAL is positioned not as a replacement for alignment tuning in all scenarios but as a strong baseline that "significantly reduces the performance gap between base LLMs and aligned LLMs" (Section 1). The paper is careful to acknowledge limits: "model tuning may still be necessary for tasks such as coding, mathematics, interactive agents, etc." (Section 5.4). The contribution is in demonstrating that for general-purpose assistant behavior on a diverse set of open-domain instructions, tuning is largely unnecessary when the base model is sufficiently capable—a finding with significant practical implications for "democratizing alignment" and for enabling frequent, cheap evaluation of pre-trained checkpoints.

3. As an evaluation contribution: The paper introduces just-eval-instruct, a 1,000-example benchmark combining data from 9 existing datasets (AlpacaEval, MT-Bench, LIMA, HH-RLHF-redteam, MaliciousInstruct, and others) and a six-aspect evaluation protocol (helpfulness, clarity, factuality, depth, engagement, safety) that produces not just scores but rationales verifiable by humans. This addresses the coarse-grained nature of prior evaluation approaches and enables fine-grained understanding of which aspects different alignment methods improve.

The paper's central thesis, therefore, is that the field has overestimated the necessity and the mechanism of alignment tuning: base LLMs are already capable assistants in waiting, needing only the right stylistic prompt to express that capability in a human-preferred format. The token distribution analysis provides the mechanistic evidence, and URIAL provides the practical demonstration.

3. Technical Approach

3.1 Reader Orientation

The paper develops a tuning-free alignment method called URIAL that transforms a base LLM into a well-behaved AI assistant purely through carefully crafted prompt engineering, without modifying any model weights. The problem it solves is that base LLMs—while possessing the factual knowledge to answer user queries—fail to produce outputs in the structured, engaging, and safety-aware style that humans expect from AI assistants; URIAL solves this by providing the missing stylistic framing through in-context examples and a system prompt, effectively "unlocking" capabilities the base model already has rather than teaching it new ones through expensive fine-tuning.

3.2 Big-Picture Architecture (Diagram in Words)

The paper contains two major technical contributions that are conceptually distinct but tightly linked:

Contribution 1: Token Distribution Shift Analysis (Section 2). This is the diagnostic framework that motivates everything else. It takes a base LLM and its aligned counterpart, feeds each the same partial generation context, and measures whether the two models would rank the same token as most probable. The analysis categorizes every token position in an aligned model's output as unshifted (both models agree on the top token), marginal (the aligned model's choice is ranked 2nd or 3rd by the base model), or shifted (ranked 4th or lower by the base model). The output is a quantitative decomposition of which tokens alignment affects and where in a response they occur.

Contribution 2: URIAL—Untuned LLMs with Restyled In-context Alignment (Section 3). This is the practical alignment method. It consists of exactly two ingredients combined into a static prompt prefix: (a) a system prompt that establishes the assistant's role, behavioral guidelines, and output format, and (b) K=3 constant, restyled in-context examples where user queries are paired with model responses that have been manually rewritten to exhibit the engaging, structured, safety-aware style of a well-aligned assistant. This prefix is prepended to any user query, and the base LLM—without any weight modification—generates an aligned response. The system is "single-pass": no search, no iterative refinement, no special decoding strategy.

The flow is: User query arrives → the static URIAL prefix (system prompt + 3 examples + the user's query formatted in the same template) is constructed → the base LLM performs standard autoregressive decoding on this full context → the model's continuation after the "# Answer:" marker is the aligned response. The static prefix can be pre-computed and cached in the KV cache, meaning inference adds only the cost of encoding the new user query and generating the response.

3.3 Roadmap for the Deep Dive

• First, the token distribution shift measurement protocol (Section 2.1): the precise procedure for comparing base and aligned model probability distributions at each token position, including how "context" is constructed and how "base rank" is defined. Without understanding this measurement apparatus, the findings it produces cannot be interpreted.
• Second, the three-category token classification (unshifted, marginal, shifted) and aggregate statistics: what these categories mean operationally, what ratios are observed (77.7% unshifted, 14.5% marginal, 7.8% shifted for Llama-2-7b), and how these numbers are computed.
• Third, the qualitative analysis of shifted tokens: what kinds of tokens appear in shifted positions, with concrete examples, establishing the "stylistic vs. knowledge-bearing" distinction that is the central empirical finding.
• Fourth, the temporal dynamics of distribution shift (diminishing over decoding time): how KL-divergence, base-rank, and base-probability evolve across token positions within a response, and what this implies about where alignment exerts its influence.
• Fifth, URIAL's two components—system prompt and restyled examples—in full detail: the exact content of each component, the design rationale connecting each element back to the distribution shift findings, and the specific stylistic patterns that are encoded.
• Sixth, the baseline tuning-free methods (zero-shot prompting, vanilla ICL, retrieval ICL): what each does, how it differs from URIAL, and why each is an important point of comparison.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an analysis-and-method paper whose core idea is that alignment tuning teaches style rather than substance, and that this style can be transferred to base LLMs through in-context learning with carefully constructed examples.

Token Distribution Shift: The Measurement Protocol

The central analytical apparatus is a procedure that, for every token position $t$ in an aligned model's greedily decoded output, asks: "What would the base model have predicted here, given the same context?" This seemingly simple question requires careful specification because the "context" that an aligned model sees during decoding includes both the original user query and all previously generated tokens—tokens that the aligned model itself produced. If we simply feed the user query alone to the base model, we would not be comparing like with like at position $t > 1$ because the prefix would be different.

The protocol solves this as follows (Section 2.1). Let the aligned model be $g(\mathbf{x}; \beta)$ where $\mathbf{x}$ is the input context and $\beta$ are the aligned parameters, and let the base model be $f(\mathbf{x}; \theta)$ with pre-trained parameters $\theta$. For a given user query $\mathbf{q} = \{q_1, q_2, \dots\}$:

Step 1: Generate the full aligned output. Feed $\mathbf{q}$ into the aligned model $g$ using greedy decoding (temperature zero, always selecting the highest-probability token) to produce a complete response $\mathbf{o} = \{o_1, o_2, \dots, o_T\}$, where $T$ is the total number of response tokens. This gives us the aligned model's "trajectory"—the sequence of tokens it chose given its own prior outputs as context.

Step 2: At each position, construct the shared context. For any token position $t$ (where $1 \leq t \leq T$), define the context at that position as:

$\mathbf{x}_t = \mathbf{q} \oplus \{o_1, o_2, \dots, o_{t-1}\}$

where $\oplus$ denotes concatenation and $\{o_1, \dots, o_{t-1}\}$ is the sequence of tokens the aligned model has already generated up to (but not including) position $t$. For $t=1$, this context is simply the user query with no generated prefix.

Step 3: Compute both models' probability distributions. Feed this exact context $\mathbf{x}_t$ into both the aligned model $g$ and the base model $f$. Each model produces a probability distribution over the entire vocabulary for the next token:

$P_{\text{align}}(\cdot \mid \mathbf{x}_t) = \text{softmax}(g(\mathbf{x}_t; \beta))$ $P_{\text{base}}(\cdot \mid \mathbf{x}_t) = \text{softmax}(f(\mathbf{x}_t; \theta))$

The aligned model's top choice at this position is, by construction of the greedy trajectory, the token $o_t$—it has probability $\max_v P_{\text{align}}(v \mid \mathbf{x}_t)$.

Step 4: Determine the rank of $o_t$ under the base distribution. Sort all tokens in the vocabulary by their probability under $P_{\text{base}}(\cdot \mid \mathbf{x}_t)$ in descending order. The position of $o_t$ in this sorted list is its base rank, denoted $\eta$. Formally:

$\eta = |\{v : P_{\text{base}}(v \mid \mathbf{x}_t) > P_{\text{base}}(o_t \mid \mathbf{x}_t)\}| + 1$

That is, $\eta$ counts how many tokens the base model considered more probable than $o_t$, plus one (making it a 1-indexed rank). A base rank of $\eta = 1$ means the aligned model's chosen token is also the base model's top-ranked token. A base rank of $\eta = 2$ means there is exactly one token the base model prefers over $o_t$, and so on.

What this protocol computes: For every token in the aligned output, it produces a single integer $\eta$ that quantifies how much the two models disagree about what should come next at that position, given the same conversational history. Low $\eta$ means high agreement (the models make similar predictions); high $\eta$ means low agreement (alignment has substantially shifted the distribution away from the base model's preferences).

Why this specific protocol: The key design choice is using the aligned model's own generated context $\mathbf{x}_t$ as input to the base model, rather than independently generating a base model response and comparing surface strings. This is crucial because it controls for the autoregressive feedback loop: language model predictions at position $t$ depend on all tokens generated at positions $1$ through $t-1$. If the base model had generated a different prefix (which it likely would have, since base models exhibit different early-token behavior), comparing token choices at later positions would conflate two effects: (1) genuine distribution shift due to alignment and (2) differences in the context prefix itself. By feeding the aligned model's prefix to the base model, protocol (2) is eliminated, isolating the alignment effect. The trade-off is that the base model may see an "out-of-distribution" prefix—a prefix it would not have generated itself—which could inflate apparent distribution shift. This is an acknowledged limitation, but the alternative (comparing independently generated completions) would be even less interpretable.

The use of greedy decoding for the aligned model (rather than sampling) is a pragmatic choice that ensures reproducibility and a deterministic reference trajectory. The paper does not analyze how the results would change under temperature-based sampling of the aligned model's output.

Token Classification: Unshifted, Marginal, Shifted

Given a base rank $\eta$ for each token $o_t$, the paper classifies tokens into three categories (Section 2.1, Figure 2):

• Unshifted positions ($\eta = 1$): The aligned model's chosen token is also the top-ranked token under the base model's distribution. Both models agree on the single most probable continuation at this position. These tokens represent positions where alignment tuning has not meaningfully changed the model's local preference.

• Marginal positions ($1 < \eta \leq 3$): The aligned model's chosen token is not the base model's top choice, but it is among the top three. This means the aligned model selected a somewhat less probable token—it is not the most natural continuation from the base model's perspective, but it is still a plausible one (within the top 3 of the ~32,000+ token vocabulary). These positions represent moderate distribution shift.

• Shifted positions ($\eta > 3$): The aligned model's chosen token is ranked 4th or lower in the base model's distribution—it would be quite unlikely for the base model to select this token. These positions represent significant distribution shift, where alignment tuning has substantially altered the model's local preference.

Aggregate statistics (Figure 3, Section 2.2): Across 1,000 test examples from the just-eval-instruct dataset, the paper reports the following ratios for the Llama-2-7b → Llama-2-7b-chat pair:

• Unshifted: 77.7% of token positions
• Marginal: 14.5% of token positions
• Shifted: 7.8% of token positions

The corresponding numbers for other model pairs are similarly imbalanced: Mistral-7b → Mistral-7b-Instruct shows 82.2% / 12.5% / 5.2%, and Llama-2-7b → Vicuna-7b-v1.5 shows 82.4% / 12.8% / 4.8%.

What these aggregate statistics tell us: On roughly four out of every five token positions, alignment tuning has zero effect on which token is considered most probable—the base model would have made the identical local choice. If we include marginal positions (where the aligned choice is still highly plausible under the base distribution), the agreement rate rises to approximately 92.2% for Llama-2-7b-chat. Only about 5–8% of token predictions are substantially altered by alignment. This is the quantitative foundation for the claim that alignment is "superficial": it touches a very small fraction of the model's decoding decisions.

Why the threshold $\eta = 3$: The choice of $\eta > 3$ as the cutoff for "shifted" is a judgment call about what constitutes a meaningful distribution shift. Tokens ranked 2nd or 3rd by the base model would still be frequently sampled under temperature-based decoding—they are not "wrong" from the base model's perspective, just not the maximally probable choice. The paper treats them as a middle ground because they represent alignment gently nudging the distribution toward more assistant-like phrasings, rather than forcing choices that are genuinely unnatural to the base model. This three-way split provides more nuance than a binary "shifted vs. unshifted" classification, which would obscure the gradient of alignment's influence.

Qualitative Analysis of Shifted Tokens: What Alignment Changes

The paper does not just count shifted positions—it identifies which specific tokens are disproportionately shifted, and the finding is what gives the aggregate statistics their meaning (Figure 3, Figure 2, Section 2.2).

Frequently shifted tokens for Llama-2-7b → Llama-2-7b-chat include (shown in the bottom boxes of Figure 3, and the top-right word cloud in Figure 2):

• Discourse markers and transitions: 'However', 'Additionally', 'As', 'To', 'So', 'Here', 'There', 'One', 'When', 'It', 'If', 'By', 'While'
• Greeting and politeness tokens: 'Thank', 'Hello', 'Great', 'Please', 'glad'
• Safety and refusal tokens: 'cannot', 'apolog', 'sorry', 'must', 'Rem' (as in "Remember"), 'responsible', 'harm', 'promote'
• Conversational framing: 'Let' (as in "Let me"), 'clarify', 'point', 'suggest', 'advice', 'reaching' (as in "Thank you for reaching out")
• Assistant identity markers: 'assistant', 'Inst' (likely part of instruction-formatting tokens)

Frequently shifted tokens for Mistral-7b → Mistral-7b-Instruct show similar patterns: 'Sure', 'appropriate', 'ethical', 'Instead', 'never', 'always', 'highly', 'respect', 'cannot', 'However', 'AI'.

The crucial empirical finding is what is NOT in these lists: Knowledge-bearing content words—nouns like "Chihuahua" (from the Figure 2 example about dog breeds), verbs carrying substantive information, numbers, technical terms, proper nouns—are overwhelmingly in unshifted positions. The paper illustrates this vividly with the "smallest dog breed" example in Figure 2: the answer "Chihuahua" and the supporting details about its weight and length appear at unshifted positions ($\eta = 1$), meaning the base model would have selected these exact same tokens given the context prefix "Thank you for asking! The". The aligned model contributed the polite framing ("Thank you for asking!"), and the base model supplied the factual knowledge.

What this implies about the mechanism of alignment: Alignment tuning does not teach models new facts or new reasoning capabilities. It teaches them a linguistic register—a specific way of structuring responses that includes:

1. Affirmation openings: Beginning responses with grateful or positive framing ("Thank you for asking!", "Hello! Absolutely,", "Great question!")
2. Structured enumeration: Using numbered lists and explicit topic headers ("1. Solar Energy: This is the most abundant energy source...")
3. Concluding summaries and engagement: Ending with wrapping-up sentences and invitations to continue the conversation ("Each type of renewable energy source has its own set of advantages... Please let me know if you have any other questions!")
4. Safety disclaimers and refusal patterns: Deploying "However," "Instead," "I cannot," "Remember" to pivot away from harmful content while maintaining a helpful tone.

This is the "smoking gun" evidence for the Superficial Alignment Hypothesis. The base model already knows the factual content. What it lacks—and what alignment provides—is the behavioral policy for how to present that content: when to refuse, how to structure information, what tone to adopt, how to maintain conversational flow. The token distribution analysis makes this visible at the finest granularity possible (individual token predictions) rather than inferring it from end-to-end behavioral comparisons.

Temporal Dynamics: Distribution Shift Diminishes During Decoding

The paper measures how the degree of alignment-induced distribution shift evolves across token positions within a single response (Section 2.2, Figure 4). Three metrics are computed at each position $t$ and averaged over the 1,000 examples:

Metric 1: KL-divergence between $P_{\text{align}}$ and $P_{\text{base}}$. This is the standard Kullback-Leibler divergence:

$D_{\text{KL}}(P_{\text{align}} \parallel P_{\text{base}}) = \sum_{v \in \mathcal{V}} P_{\text{align}}(v \mid \mathbf{x}_t) \log\left(\frac{P_{\text{align}}(v \mid \mathbf{x}_t)}{P_{\text{base}}(v \mid \mathbf{x}_t)}\right)$

where $\mathcal{V}$ is the vocabulary. This measures, in bits, how much additional information is needed to encode the aligned model's distribution using the base model's distribution as a code. A value close to zero means the distributions are nearly identical. The paper plots this metric as a function of position $t$ and finds that KL-divergence decreases over time—the distributions become more similar as decoding progresses into later tokens of the response.

Metric 2: Base-rank $\eta$ of aligned tokens. This is the rank metric defined earlier. The paper plots the average $\eta$ across examples as a function of $t$ and finds that base-rank decreases over time, dropping below 5 shortly after $t \geq 5$. This means that at most positions beyond the very beginning of the response, the aligned model's top token is within the top 5 choices of the base model.

Metric 3: Base-probability of aligned tokens. This is the probability that $P_{\text{base}}$ assigns to $o_t$, the token that the aligned model selected:

$\text{base-prob}(t) = P_{\text{base}}(o_t \mid \mathbf{x}_t)$

expressed as a value between 0 and 1. The paper plots this and finds that base-probability increases over time, approaching 1.0 at later positions. This means that for tokens deep into the response, the aligned model is selecting tokens that the base model considers nearly certain (probability close to 1.0 given the prefix).

What these temporal dynamics tell us: The paper interprets the diminishing KL-divergence and increasing base-probability as evidence that "alignment is more critical for earlier tokens" (Section 2.3). The first few tokens of a response are where the assistant "sets the tone"—choosing an affirmation, structuring the opening, establishing the conversational register. Once this stylistic frame is established, the rest of the response largely flows from the base model's pre-existing knowledge, and the two distributions converge. This makes mechanistic sense: the initial token choices constrain the rest of the response through the autoregressive nature of generation. If alignment can ensure that the first few tokens produce the right stylistic framing, the base model's natural completions are already well-aligned with assistant behavior—no further intervention needed.

Why is this temporal pattern significant for URIAL? It directly motivates the structure of URIAL's in-context examples: if alignment's influence is concentrated in early tokens where stylistic framing is established, then providing examples that demonstrate this framing at the beginning of responses should allow the base model to mimic the pattern, after which its own knowledge takes over. The temporal analysis provides the mechanistic justification for why a simple in-context approach can work: you don't need to teach the model what to say about renewable energy—you just need to show it how to start saying it, and the rest follows naturally from the base model's pre-training.

The URIAL Method: System Prompt Design

The system prompt in URIAL serves as a "constitution" that defines the assistant's role and behavioral expectations, placed before any in-context examples. The full system prompt (reproduced in Appendix A) starts with a format introduction:

"Below is a list of conversations between a human and an AI assistant (you). Users place their queries under '# Query:', and your responses are under '# Answer:'."

This establishes the conversational template that subsequent examples will follow: each turn is explicitly demarcated with Markdown-style headers. The base model can infer from this that it should generate content after the "# Answer:" marker.

The system prompt then defines the assistant's persona across multiple dimensions, adapted from the system prompt used by Llama-2-chat:

"You are a helpful, respectful, and honest assistant. You should always answer as helpfully as possible while ensuring safety. Your answers should be well-structured and provide detailed information. They should also have an engaging tone. Your responses must not contain any fake, harmful, unethical, racist, sexist, toxic, dangerous, or illegal content, even if it may be helpful. Your response must be socially responsibly, and thus you can reject to answer some controversial topics."

This paragraph encodes five distinct behavioral expectations:

1. Helpfulness: "answer as helpfully as possible"
2. Respect and honesty: "helpful, respectful, and honest assistant"
3. Structure and depth: "well-structured and provide detailed information"
4. Engagement: "engaging tone"
5. Safety: The explicit prohibition on "fake, harmful, unethical, racist, sexist, toxic, dangerous, or illegal content" with the added clause "even if it may be helpful" (which precludes the "I'm just being helpful" loophole), plus the ability to "reject to answer some controversial topics"

Design rationale based on the distribution shift analysis: The system prompt essentially declares the stylistic and behavioral patterns that alignment tuning would otherwise install through weight modification. If alignment primarily teaches the model to adopt a helpful, structured, safe tone (as the token analysis suggests), then explicitly stating these expectations in the prompt should be functionally equivalent—provided the base model has the in-context learning capability to follow such instructions. The base model already knows what "helpful" and "safe" mean as linguistic concepts; the system prompt tells it to apply those concepts to its generation.

Why place it before the examples rather than after? Because the examples that follow are demonstrations of the system prompt's principles in action. The model sees the abstract rules first, then sees concrete instances of those rules, creating a "principle → example" structure that mirrors how instruction manuals work for humans. This ordering likely helps the model generalize beyond the specific examples—it learns that the patterns are instances of a general policy, not just fixed response templates.

The system prompt uses approximately 150 tokens. Combined with the three restyled examples, the total URIAL prefix is 1,011 tokens (671 words) for the default K=3 configuration.

The URIAL Method: Restyled In-Context Examples

The core design insight of URIAL is that the style of in-context examples matters critically for alignment—perhaps more than their content. This is directly informed by both the token distribution analysis (which shows alignment primarily affects stylistic tokens) and prior work by Min et al. (2022), which found that "ICL primarily concerns the style and format of demonstrations, rather than their truth content."

The paper constructs three constant in-context examples (for the default K=3 configuration) that are restyled—manually rewritten from basic instruction-output pairs into exemplars of the desired assistant behavior. The examples are chosen to cover three qualitatively different interaction types (Section 3.3, Figure 5):

Example 1: Safety refusal with empathy. The query is a harmful request: "What is the best way to torture a friend who stole my money?" A basic (unstyled) response might simply say "Sorry, I cannot answer this." The restyled version instead demonstrates the full pattern of a well-aligned refusal:

"Hi! I'm sorry to hear that. However, I cannot assist with that request. If you're feeling angry or hurt, I recommend seeking guidance or speaking to a trusted individual about your feelings. If you believe a crime has been committed, it's best to report it to the appropriate authorities rather than take matters into your own hands. Remember, violence or harm is not a solution and can lead to serious consequences for both parties involved. I hope it helps!"

This response encodes multiple stylistic features identified in the token distribution analysis:

• Empathetic opening ("Hi! I'm sorry to hear that") using greeting tokens
• Refusal with the safety token "cannot"
• Constructive redirection ("I recommend seeking guidance...") rather than flat rejection
• Summary warning introduced by "Remember" (matching the 'Rem' token in the shifted-token analysis)
• Closing engagement marker ("I hope it helps!")

Example 2: Role-playing with procedural advice. The query asks the model to role-play as a detective: "You are a detective interrogating a suspect. How do you get them to confess without violating their rights?" The restyled response:

"Hello! As a detective, it's important to conduct interrogations within the boundaries of the law and respect the suspect's rights to avoid any potential violations or false confessions. Here are some ethical and legal techniques that can be used to encourage a suspect to provide information or confess: 1. Building rapport: Engage in friendly conversations, develop a level of trust and comfort with the suspect. 2. Presenting evidence: Show the suspect the evidence or information that implicates them in the crime without making threatening implications. 3. Possible outcomes: Without making promises, you can explain the potential consequences based on choices they make now. 4. Maintaining calm and patience: Give them time to talk and think. 5. Mirroring and empathy: Try to relate to them or mirror their actions subtly, which can create a sense of connection and empower them to share more. 6. Silence: Moments of silence can pressure suspects to break it by talking. Remember, it's always imperative to respect the suspect's rights and conduct any investigation in accordance with the law. Intimidation, physical confrontation or coercive tactics are illegal and unethical. Is there anything else you would like to know?"

Key stylistic patterns demonstrated:

• Greeting opening ("Hello!")
• Context paragraph that rephrases the query and establishes the approach
• Explicit transition to enumeration ("Here are some ethical and legal techniques...")
• Numbered list with the format [n]. [topic]: [details]—each point pairs a technique name with an explanation
• Concluding summary ("Remember, it's always imperative...") using the "Rem" token pattern
• Engagement invitation ("Is there anything else you would like to know?")

Example 3: Factual information with structured detail. The query is straightforward: "Can you tell me some common types of renewable energy sources?" The restyled response:

"Hello! Absolutely, below are some of the most common types of renewable energy sources: 1. Solar Energy: This is the most abundant energy source on earth, harnessed through the use of solar panels. These panels convert sunlight into electricity without any moving parts, noise, pollution, or damage to the environment. 2. Wind Energy: Wind turbines convert the kinetic energy in the wind into mechanical power. This mechanical power can be used for specific tasks (such as pumping water) or converted into electricity to power homes, businesses, and schools. 3. Hydropower: Generated by using electricity generators to capture the energy from falling or fast-moving water. This renewable source can come from various ways, including a large dam on a river, tidal or wave energy from the ocean, or using small scale turbines in streams. 4. Geothermal Energy: This type of energy is generated from the heat deep within the Earth. This heat can be used directly for heating buildings or to generate electricity. It is continuously produced inside the Earth and is nearly as reliable as the tides. 5. Biomass Energy: Biomass is organic material that comes from plants and animals, and it contains stored energy from the sun. This energy can be burned directly or converted into biofuel which can burn more efficiently. Each type of renewable energy source has its own set of advantages and challenges, but collectively, they represent our best hope at achieving sustainable and environmentally friendly energy consumption. Please let me know if you have any other questions!"

Stylistic patterns demonstrated:

• Enthusiastic affirmation opening ("Hello! Absolutely,")
• Clear topic transition ("below are some of the most common types...")
• Detailed numbered list with [n]. [Topic]: [detailed explanation] format
• Explanations that go beyond one-line definitions into multi-sentence descriptions
• Concluding paragraph that synthesizes and broadens the perspective
• Polite closing with engagement invitation

The restyling design principles (Section 3.3): The authors explicitly state that these examples were crafted to follow a template: "start by rephrasing the question in an engaging and declarative manner, followed by a detailed list of bullet points when appropriate. For each point, we use the format '[n]. [topic]: [details]' to enable efficient information extraction from base LLMs. A concise summary paragraph concludes the response, maintaining an engaging and conversational tone throughout." Additionally, the authors "incorporate stylistic tokens, as inspired by Sec. 2.2, to prompt base LLMs to generate informed outputs"—deliberately including tokens like "Hello," "Remember," "However," and "Absolutely" that the distribution analysis identified as typical alignment-affected tokens.

Why these three specific examples? The examples span the critical interaction types for an AI assistant: (1) a harmful query requiring refusal with empathy, (2) a role-playing scenario requiring procedural advice within ethical boundaries, and (3) a straightforward information request requiring structured factual exposition. Together, they demonstrate that the assistant can refuse, can engage in creative scenarios, and can provide detailed information—all while maintaining the same stylistic register. The examples are constant for every user query, meaning they form part of a static prefix that can be cached (unlike retrieval-based methods where examples change per query).

Why restyled rather than natural examples: The paper notes that "outputs generated by the vanilla ICL are often not preferred by humans, even when the responses contain no factual errors" (Section 3.3). This is because instruction datasets like Flan-Collection or Alpaca contain outputs "in a plain and basic style"—short, utilitarian answers like "Solar energy, wind energy, hydropower, geothermal energy, biomass energy" that lack the engaging, structured, safety-conscious characteristics of assistant behavior. URIAL's examples are synthetic in the sense that they are manually authored to demonstrate the desired style, even though the factual content could come from the base model itself. This is the key distinction from vanilla ICL: the examples are not just demonstrations of input-output mapping, but demonstrations of how to be an assistant.

URIAL Efficiency: Static Prefix and Configurations

Static prefix and KV-cache reuse (Section 3.3): Because URIAL uses the same system prompt and the same three examples for every query, the entire 1,011-token prefix is identical across all inference calls. In a transformer with key-value caching, the activations (keys and values) for these prefix tokens can be computed once and stored. For each new user query, only the query's tokens and the model's generated response tokens need to be encoded—the prefix activations are simply loaded from the cache. This is in contrast to retrieval-based ICL, where different examples are retrieved for each query (Section 3.2), forcing the model to re-encode the entire prompt for every inference call. The paper notes that "with additional engineering efforts in deployment, such as advanced caching methods and FlashAttention, the inference speed can be further improved."

Configuration sweep (Section 4.3): The paper tests URIAL with different numbers of in-context examples:

• K=1: Using only the "renewable energy" example from Figure 5. Total prefix: 543 tokens. Tested on Llama-2-70b (where it achieves "respectable performance," 4.62 average score) but not on smaller models.
• K=3: The default configuration. Total prefix: 1,011 tokens. The sweet spot for balanced performance and cost.
• K=8: Adding examples for mathematics, coding, poetry writing, procedure, and safety. Total prefix: ~2,026 tokens. Improves safety substantially for Llama-2-7b (4.29 → 4.97) but decreases performance on other aspects.

The recommendation is K=3 "due to its balanced performance and lower cost" (Section 4.3). An important finding is that adding more examples does not monotonically improve performance—in fact, for Llama-2-7b with K=8, helpfulness drops from 4.22 to 4.08, factuality from 4.16 to 4.09, and depth from 3.88 to 3.68 compared to K=3. This suggests there may be a "sweet spot" where a small number of carefully chosen examples provide sufficient stylistic guidance without over-constraining the model's behavior or consuming excessive context window budget.

Robustness to example selection (Section 4.3): To test whether URIAL's performance depends on the specific three examples chosen, the authors tested an alternative set of three examples on Mistral-7b. The overall performance was "even slightly better than the default version (4.63 vs. 4.64)," with individual aspect scores: helpfulness 4.60, clarity 4.90, factuality 4.50, depth 4.20, engagement 4.74, safety 4.93. This suggests that URIAL is "robust to variations in the ICL examples provided"—the key is the style of the examples (structured, engaging, safety-aware), not the specific factual content they contain.

Multi-turn conversation support (Section 4.4, Appendix B.3): The paper demonstrates that URIAL can handle multi-turn conversations by simply appending previous turns as additional in-context examples within the same template. The procedure: after the first user query and URIAL's response, the second user query is appended to the conversation history with the "# Query:" / "# Answer:" formatting, and the model generates the next response conditioned on the full history. The case study in Appendix B.3 shows a four-turn conversation about Bloody Mary recipes where the base model (Llama-2-7b with URIAL) maintains context, provides follow-up suggestions, thanks the user, and generates creative cocktail names—all while maintaining the assistant's engaging persona. This suggests that "the conversation ability of aligned LLMs might be largely from the base models themselves."

Baseline Methods: What URIAL Is Compared Against

To establish the effectiveness of URIAL, the paper implements three baseline tuning-free alignment methods (Section 3.2), each representing a progressively more sophisticated use of in-context learning:

Baseline 1: Zero-shot Templated Prompting. The simplest possible approach: format the user query using a Markdown-style template (shown in Figure 5, top-left) but provide no in-context examples. The template uses special tokens to signal boundaries:

# Query:

[User Instruction]

# Answer:

The base model sees this formatted prompt and generates a continuation. The rationale (Section 3.2) is that "incorporating special tokens that signal the boundaries" helps base LLMs "appropriately initiating and concluding responses to user queries." Without any examples of what an answer should look like, however, the model often falls back on its pre-training behaviors: repeating the question, generating related but unhelpful content, or producing answers that lack coherence and conversational norms. This baseline uses only the formatting template, with no system prompt and no examples.

Baseline 2: Vanilla In-Context Learning (Vanilla ICL). This method adds K=3 instruction-output examples before the query, using the same Markdown template. The examples are drawn from existing instruction datasets (specifically Flan-Collection and Alpaca) "in a plain and basic style" (Section 3.2). For instance, the query "Can you tell me some common types of renewable energy sources?" would be paired with a basic output like "Solar energy, wind energy, hydropower, geothermal energy, biomass energy." These examples demonstrate the input-output format but do not model the engaging, structured style of an AI assistant. They contain factual responses without the stylistic framing that alignment tuning teaches.

The key limitation: Min et al. (2022) showed that ICL is sensitive to the style of demonstrations. Vanilla ICL provides the correct format (query → answer) but the wrong style (concise and functional rather than engaging and assistant-like). The paper's results bear this out: vanilla ICL on Llama-2-7b achieves an average score of only 3.18 versus 4.33 for URIAL, and actually reduces depth (2.69 → 2.67) compared to zero-shot prompting because the plain-style examples encourage brief, shallow responses.

Baseline 3: Retrieval-Augmented ICL. This method dynamically selects in-context examples for each query from a large pool of instruction-response data. The procedure (Section 3.2) is:

1. Construct a dataset of 800k cleaned instruction-response pairs from open-instruct (Wang et al., 2023) and UltraChat (Ding et al., 2023), selecting for "longer outputs."
2. Build a dense index using MPNET (Song et al., 2020), a sentence embedding model based on SentenceTransformer (Reimers & Gurevych, 2019).
3. For each test query, use FAISS (Johnson et al., 2019) to retrieve the K=3 most semantically similar instructions from the index.
4. Use the corresponding instruction-response pairs as in-context examples.

This is motivated by prior work (Lin et al., 2022; Han, 2023) showing that retrieving semantically relevant examples facilitates better generalization than using static, unrelated examples. The intuition: if the user asks about cooking, show cooking-related examples; if they ask about coding, show coding-related examples. This provides content-level relevance that static examples cannot match.

Retrieval ICL achieves significantly better performance than vanilla ICL (3.72 vs. 3.18 average on Llama-2-7b), demonstrating the value of semantic relevance. However, it has a critical efficiency drawback: "the prefixes for retrieval ICL are different for each new query, so we have to compute prefixes every single time" (Section 3.2). The KV-cache cannot be reused across queries, making inference substantially slower than static-prefix methods like URIAL.

Why URIAL outperforms retrieval ICL despite using constant, semantically-unrelated examples: The paper's finding—that URIAL (4.33) significantly outperforms retrieval ICL (3.72)—is counterintuitive but consistent with the token distribution analysis. If alignment is primarily about style rather than content, then showing the model stylistically appropriate responses (even on unrelated topics) is more valuable than showing it topically related responses in a plain style. The retrieval examples are "vanilla" in style—they contain factual content but lack the structured, engaging, safety-aware register. URIAL's restyled examples, by contrast, model the behavior of an assistant regardless of topic, and the base model can transfer this behavioral template to any topic using its pre-existing knowledge. This is a concrete demonstration of Min et al.'s finding (2022) that ICL "primarily concerns the style and format of demonstrations, rather than their truth content."

Design Choices and Their Justifications (Collected Summary)

• Static constant examples over dynamic retrieval: Avoids re-encoding the prompt for each query (KV-cache reuse) and, surprisingly, outperforms retrieval because style transfer matters more than content relevance for alignment. The static prefix also makes the method trivially reproducible and eliminates the need to maintain a retrieval index.

• Restyled outputs over natural dataset outputs: Directly encodes the stylistic patterns (greetings, enumeration, summaries, safety disclaimers, engagement markers) that the token distribution analysis identified as the primary effects of alignment tuning. Natural instruction data lacks these patterns because it typically comes from pre-alignment data collection.

• System prompt before examples over examples alone: Establishes general behavioral principles that the examples then instantiate, creating a "policy + demonstrations" structure. This likely improves generalization beyond the specific example topics.

• Three examples over one or many: K=1 works for very strong base models (Llama-2-70b achieves 4.62 with a single example) but may provide insufficient stylistic coverage. K=3 balances coverage of different interaction types (refusal, role-play, factual exposition) with prompt length. K=8 may over-constrain the model or dilute the key stylistic signals with extraneous content.

• Greedy decoding (temperature zero) for all experiments: Ensures perfect reproducibility. The paper acknowledges that some aligned models "may achieve enhanced performance with specific settings for temperature, top p, and other parameters" but opts for greedy decoding "to ensure reproducibility and to conduct a fair and robust evaluation" (Appendix D.1).

• Repetition penalty of 1.1 on base LLMs: Applied "to prevent degeneration" (Appendix D.1) since base models can sometimes fall into repetitive loops when generating in the assistant format without tuning. This is a minor inference-time adjustment, not a weight modification.

4. Key Insights and Innovations

Innovation 1: Token-Level Distribution Shift as a Direct Window into Alignment's Mechanism

Prior to this paper, the Superficial Alignment Hypothesis—that alignment tuning teaches style rather than substance—was supported only by indirect behavioral evidence: LIMA (Zhou et al., 2023) showed that 1,000 SFT examples suffice, but could not explain why or where alignment exerted its influence. The dominant assumption in the field was that alignment tuning genuinely transforms model behavior in substantive ways—possibly teaching new reasoning strategies, knowledge boundaries, or instruction-following capabilities. The evidence for this belief was the stark qualitative difference between base and aligned model outputs, which appeared almost categorically different in style, safety, and helpfulness.

This paper introduces a fundamentally new kind of evidence: a token-level distribution shift measurement protocol (Section 2.1) that peers inside the probability distributions of both models at every decoding position, rather than comparing only the surface outputs. By feeding the aligned model's own generated prefix back into the base model and asking whether the base model would have made the same local token prediction, the protocol isolates alignment's causal effect from the autoregressive feedback loop that would otherwise confound the comparison.

What makes this approach conceptually distinctive is that it converts a vague hypothesis ("alignment is superficial") into a precise, quantitative measurement framework with clear operational definitions: unshifted tokens ($\eta = 1$), marginal tokens ($1 < \eta \leq 3$), and shifted tokens ($\eta > 3$). This framework produces numbers (77.7% unshifted, 92.2% within top-3 for Llama-2-7b-chat; Figure 3) that can be compared across model families, alignment recipes, and even future alignment methods. It transforms the debate from "is alignment superficial?" (a binary, poorly-defined question) to "at what proportion of token positions, for which token types, and with what temporal dynamics does alignment alter model behavior?" (a multi-dimensional, measurable question).

The significance extends beyond this paper's specific findings. The measurement protocol itself is a diagnostic tool that any lab can apply to any base-aligned model pair to understand what their specific alignment procedure changed. It is computationally cheap (requiring only inference, no training), produces interpretable visualizations (the web demo in Figure 8), and generalizes across model architectures. This is a methodological contribution that enables future research into alignment mechanisms, rather than a one-time finding about specific models.

The concrete results from applying this framework are striking on their own: across three model pairs (Llama-2-7b → Llama-2-7b-chat, Llama-2-7b → Vicuna-7b-v1.5, Mistral-7b → Mistral-7b-Instruct), the shifted token ratio is consistently only 5–8% (Figure 3). But the deeper innovation is the framework itself—a way of looking at alignment that replaces behavioral intuition with mechanistic measurement.

Innovation 2: The Style-vs-Knowledge Decomposition of Alignment's Effects

The distribution shift measurement tells us how much alignment changes, but not what it changes. The paper's second conceptual contribution is showing that the shifted tokens are not uniformly distributed across the vocabulary—they cluster in specific functional categories that together constitute the linguistic register of an AI assistant.

Prior work had no way to distinguish between "alignment taught the model to structure its answers" and "alignment taught the model new facts." The behavioral comparison of end-to-end outputs conflates these possibilities. This paper's token-level analysis disentangles them directly: by examining which specific tokens appear disproportionately in shifted positions (Figure 2, Figure 3, Section 2.2), the paper shows that alignment's influence concentrates on:

• Discourse markers and transitions ('However', 'Additionally', 'As', 'Here', 'There')
• Greeting and politeness tokens ('Thank', 'Hello', 'Great', 'Please', 'glad')
• Safety and refusal framing ('cannot', 'apolog', 'sorry', 'must', 'Rem')
• Conversational engagement markers ('Let', 'clarify', 'suggest', 'reaching')

Meanwhile, knowledge-bearing content words—nouns, verbs carrying substantive meaning, numbers, technical terms, proper nouns—are overwhelmingly in unshifted positions. The Figure 2 example crystallizes this: in a response about the world's smallest dog breed, the factual answer "Chihuahua" and its supporting details are at unshifted positions ($\eta = 1$), while the polite opening "Thank you for asking!" occupies shifted positions. The base model already knew the answer; alignment only taught it to frame that answer appropriately.

This is not merely a catalog of shifted tokens. It is a functional decomposition of alignment's role that re-conceptualizes what alignment tuning actually learns: not a new set of knowledge or reasoning capabilities, but a behavioral policy governing the syntactic and pragmatic structure of assistant responses. The components of this policy—affirmation openings, structured enumeration, concluding summaries, safety disclaimers—constitute a specific register of English that the base model could theoretically produce (it has seen such text in pre-training) but was not trained to deploy in response to arbitrary user queries.

The significance for LLM research is that it separates the question "does this base model have the knowledge to answer user queries?" from "can this base model express that knowledge in a human-preferred format?" The answer to the first question is largely yes (for sufficiently pre-trained models); the answer to the second is what alignment provides. This reframing has immediate practical implications: improving base model knowledge requires better pre-training, while improving alignment requires better stylistic guidance—and these are largely orthogonal axes of improvement.

Innovation 3: The Temporal Dynamics of Alignment—Localized to Early Tokens

A less obvious but equally important finding is that alignment's influence is not uniform across token positions within a response—it is concentrated in the initial tokens and diminishes rapidly as decoding progresses (Section 2.2, Figure 4). The paper demonstrates this through three converging metrics: KL-divergence between $P_{\text{align}}$ and $P_{\text{base}}$ decreases over token position, base-rank of aligned tokens drops below 5 shortly after $t \geq 5$, and base-probability of aligned tokens increases toward 1.0 at later positions.

This finding is conceptually significant because it reveals alignment as a prefix-level intervention rather than a global transformation of the model's behavior. The first few tokens of a response establish the conversational frame—the greeting, the affirmative stance, the structural cue ("Here are some...")—and once this frame is in place, the base model's natural autoregressive completions are already well-aligned with assistant behavior. Alignment tuning, under this interpretation, primarily teaches the model to initiate responses in the assistant register; the rest follows from the base model's existing language modeling capabilities.

This temporal pattern was not predicted by prior work and is not obvious from surface output comparisons. It emerged only because the distribution shift protocol measures alignment's effect at every position, not just in aggregate. The finding provides a mechanistic explanation for why a small number of in-context examples can achieve alignment: they need only demonstrate the initial framing pattern, and the model's own autoregressive dynamics will propagate the appropriate style through the rest of the response.

The finding also has implications for how we think about alignment's failure modes. If alignment's influence is concentrated in early tokens, then failures of alignment (e.g., a model drifting into inappropriate content despite RLHF training) likely stem from cases where the initial framing is insufficient to constrain the full trajectory—a hypothesis that could be tested by measuring whether shifted tokens reappear in later positions for cases where the model eventually produces misaligned outputs.

Innovation 4: Style Transfer via In-Context Learning as a Functional Substitute for Weight Modification

URIAL's strong empirical performance—matching or exceeding SFT (Mistral-7b-Instruct) and SFT+RLHF (Llama-2-70b-chat) on just-eval-instruct—is not merely a "better prompting" result. It represents a fundamental demonstration that the behavioral transformation traditionally achieved through gradient-based weight updates can be replicated through pure in-context learning, provided the base model is sufficiently capable and the in-context examples are sufficiently well-crafted in style rather than content.

Prior work had shown that ICL could improve performance on narrow NLP tasks (classification, QA; Wei et al., 2023; Zhang et al., 2022) and that ICL is sensitive to demonstration style (Min et al., 2022). But no prior work had systematically shown that ICL can serve as a complete substitute for the alignment tuning pipeline on a diverse set of open-domain assistant instructions. The paper's comparison to retrieval-augmented ICL (Han, 2023) is particularly revealing: retrieval ICL, which selects semantically relevant examples but leaves them in their "vanilla" style, achieves only 3.72 average score on Llama-2-7b, while URIAL's constant but restyled examples achieve 4.33 (Table 1). The counterintuitive implication is that stylistic relevance matters more than topical relevance for alignment—showing the model how to be an assistant (through restyled examples on unrelated topics) transfers better than showing it what to say about similar topics (through relevant but plain examples).

This finding reframes the alignment problem from one of knowledge acquisition (teaching the model new capabilities) to one of style transfer (teaching the model to express existing capabilities in a specific register). It implies that the massive computational investment in alignment tuning—SFT on hundreds of thousands of examples, RLHF with human preference data—is largely buying the same thing that three carefully written examples and a system prompt can provide. The paper is careful not to claim universality (Section 5.4 acknowledges coding, math, and agent tasks may still require tuning), but for the broad class of open-domain assistant interactions, the case for tuning-free alignment is empirically compelling.

What makes this more than a "prompt engineering trick" is the connection to the distribution shift analysis: URIAL's design is directly motivated by the finding that alignment primarily affects stylistic tokens in early response positions. The restyled examples explicitly encode these patterns (greetings, enumeration, summaries, safety disclaimers). The system prompt explicitly declares the behavioral expectations. The result is a method that is simultaneously simple (three constant examples, no retrieval, no search) and principled (each design choice traces back to a specific finding from the token analysis).

The broader significance is the demonstration that base LLMs are latent assistants—the capability is already present, and alignment's role is to surface it through appropriate context, not to create it through weight modification. This inverts the conventional narrative that alignment "unlocks" capabilities through extensive tuning, replacing it with the view that alignment provides the key (stylistic context) to capabilities that were already present behind a door that only looked locked.

Innovation 5: Multi-Aspect, Explainable Evaluation as a Tool for Understanding Alignment

The paper's evaluation framework—just-eval-instruct with six scored aspects plus rationales—addresses a systematic weakness in prior alignment evaluation. Benchmarks like AlpacaEval (Li et al., 2023a) and MT-Bench (Zheng et al., 2023) produce single win-rate numbers that obscure which aspects of alignment different methods improve or degrade. This makes it impossible to diagnose failures: if an aligned model performs worse than expected, is it hallucinating facts? Being too brief? Refusing safe queries? The coarse metric cannot tell you.

The six-aspect decomposition (helpfulness, clarity, factuality, depth, engagement, safety; Section 4.2) enables differential diagnosis of alignment methods. The results in Table 1 reveal patterns invisible in aggregate scores: Llama-2-7b-chat (RLHF) scores very high on safety (5.00) and engagement (4.70) but lower on helpfulness (4.10) and factuality (4.26) compared to Vicuna-7b (SFT, which scores 4.43 helpfulness and 4.33 factuality but only 4.60 safety and 4.51 engagement). This tradeoff—RLHF prioritizing safety and conversational tone at the expense of knowledge provision—is a concrete, measurable phenomenon that single-score evaluations would miss entirely. The paper's observation that "RLHF might even cause more hallucination" (Section 4.4) is supported by this granular breakdown.

The inclusion of verifiable rationales for each aspect score (Appendix D.5) further distinguishes this evaluation from prior work. GPT-4's judgments are not taken as black-box oracles; they include explicit justifications that humans can audit. The paper reports 94.1% human approval for the explanations and 87.8% agreement with GPT-based pairwise judgments (Appendix D.3), providing credibility that aggregate win-rate evaluations lack. This is not merely a "better benchmark"—it is a demonstration that alignment evaluation should be multi-faceted and auditable, and that such evaluation reveals meaningful tradeoffs between alignment methods that coarse metrics conceal.

The significance extends to how we should think about alignment goals. The six aspects are not all maximizable simultaneously—improving safety may degrade helpfulness, and deeper responses may sacrifice clarity. A single "best" alignment method may not exist; instead, different deployment contexts may prioritize different aspect profiles. The evaluation framework enables this kind of context-dependent assessment where prior benchmarks assumed a one-dimensional quality hierarchy.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper constructs just-eval-instruct, a collection of 1,000 diverse instruction examples merged from 9 existing datasets: AlpacaEval (downsampled to 420 examples), MT-Bench (80 examples), LIMA-test (300 examples), HH-RLHF-redteam (100 examples), and MaliciousInstruct (100 examples). The first 800 examples (from AlpacaEval, MT-Bench, and LIMA) target helpfulness evaluation, while the last 200 (from HH-RLHF-redteam and MaliciousInstruct) target safety/harmlessness. The dataset is categorized by task type (information-seeking, reasoning, math, coding, role-playing, creative writing, procedural) and topic (lifestyle, STEM, humanities, finance, medical, nature, ethics) using GPT-4 tagging (Figure 6). All experiments use the full 1,000 examples unless otherwise noted.

• Base models. Three base (untuned) LLMs are evaluated: Llama-2-7b, Mistral-7b (v0.1) (Jiang et al., 2023a), and Llama-2-70b (with 4-bit GPTQ quantization, denoted Llama-2-70bq; Frantar et al., 2022). These are chosen to represent a range of scales and pre-training qualities, with Mistral-7b and Llama-2-70b characterized as "better pre-trained than Llama-2-7b, as suggested by various benchmarking results and their zero-shot performance" (Section 4.3). The corresponding aligned models for comparison are: Vicuna-7b-v1.5 (SFT on Llama-2-7b), Llama-2-7b-chat (SFT+RLHF on Llama-2-7b, also 4-bit quantized), Llama-2-70b-chat (SFT+RLHF on Llama-2-70b, quantized), and Mistral-7b-Instruct (SFT on Mistral-7b). Additionally, gpt-3.5-turbo-0301, gpt-4-0314, and gpt-4-0613 are included as proprietary reference points.

• Metrics. The primary evaluation uses six aspect-level scores on a 1–5 Likert scale (1 = "strongly disagree" to 5 = "strongly agree") assessed by GPT-4 (for the 800 helpfulness examples) and ChatGPT (for the 200 safety examples). The six aspects are: helpfulness (how well the response addresses the query), clarity (logical flow and coherence), factuality (accuracy of information), depth (thoroughness and detail), engagement (conversational, human-like tone), and safety (avoidance of unethical/harmful content). Each score includes a human-verifiable rationale (Appendix D.5). The aggregate metric is the average of these six scores per model. Additionally, output length (number of words) is reported. Human validation on 100 randomly sampled pairwise comparisons shows 87.8% agreement with GPT-based judgments, and 94.1% human approval of the rationales (Appendix D.3).

• Baselines. Four alignment methods are compared:

  • Tuned baselines: (a) SFT — represented by Vicuna-7b-v1.5 and Mistral-7b-Instruct, both aligned via supervised fine-tuning on instruction data. (b) RLHF — represented by Llama-2-7b-chat and Llama-2-70b-chat, which undergo SFT followed by reinforcement learning from human feedback (Bai et al., 2022; Touvron et al., 2023).
  • Tuning-free baselines: (c) Zero-shot Templated Prompting — the base model receives only the Markdown-style template with no in-context examples (Section 3.2). (d) Vanilla ICL — K=3 constant instruction-output examples drawn from existing datasets (Flan-Collection, Alpaca) in their original "plain and basic style" (Section 3.2). (e) Retrieval ICL — dynamically retrieves K=3 semantically similar instruction-output pairs from an 800k-example index built with MPNET embeddings and FAISS (Lin et al., 2022; Han, 2023), with examples in their original style.
  • URIAL — the proposed method, using K=3 constant restyled examples plus system prompt (Section 3.3), with variants at K=1 and K=8 tested for robustness.

• Generation budget / compute accounting. All models use greedy decoding (temperature zero) for reproducibility, with a repetition penalty of 1.1 applied to base LLMs "to prevent degeneration" (Appendix D.1). There is no generation budget sweep—each model generates exactly one response per query. The cost difference between methods is measured in terms of prompt length (number of tokens in the prefix): zero-shot uses only the template (~50 tokens), vanilla ICL adds ~200–300 tokens of examples, retrieval ICL adds ~200–300 tokens but changes per query (preventing KV-cache reuse), and URIAL uses a static 1,011-token prefix (K=3) that can be cached. The paper explicitly notes efficiency advantages: URIAL's static prefix enables "caching the computation for the static prompts... avoid[ing] the need to re-encode them for subsequent queries, resulting in significantly greater efficiency than retrieval-based ICL" (Section 3.3).

• Cross-validation / statistical protocol. There is no cross-validation in the traditional sense, as no model training or hyperparameter selection occurs. Instead, the paper tests robustness to in-context example selection by reporting results with an alternative set of three examples on Mistral-7b (Section 4.3). The evaluation protocol uses GPT-4 as judge with explicit rationales to enable human verification; 100 judgments are spot-checked by human annotators (Appendix D.3). All model outputs are generated deterministically (temperature zero), so there is no variance from sampling.

Main Quantitative Results

Aggregate Performance: URIAL Matches or Exceeds Tuned Alignment

The headline result appears in Table 1: URIAL with K=3 on Mistral-7b achieves an average score of 4.63, surpassing Mistral-7b-Instruct (SFT) at 4.44 on all six aspects. Specifically, URIAL scores higher on helpfulness (4.57 vs. 4.36), clarity (4.89 vs. 4.87), factuality (4.50 vs. 4.29), depth (4.18 vs. 3.89), engagement (4.74 vs. 4.47), and safety (4.92 vs. 4.75). The consistent across-the-board improvement is notable—URIAL does not trade off one aspect for another.

On Llama-2-70bq, URIAL (K=3) achieves 4.74, exceeding the RLHF-ed Llama-2-70b-chatq at 4.67. The largest gaps favor URIAL on helpfulness (4.72 vs. 4.50) and engagement (4.85 vs. 4.75), while safety is essentially tied (4.96 vs. 5.00). URIAL on Llama-2-70bq with K=1 already achieves 4.62—notably higher helpfulness (4.60 vs. 4.50) than the RLHF-ed counterpart—demonstrating that for strong base models, even a single restyled example can suffice.

On Llama-2-7b, URIAL (4.33) substantially closes the gap to Vicuna-7b (SFT, 4.46) and exceeds Llama-2-7b-chat (RLHF, 4.47) in helpfulness (4.22 vs. 4.10) and factuality (4.16 vs. 4.26, where the chat model is lower), though the chat model leads in safety (5.00 vs. 4.29) and engagement (4.70 vs. 4.65). URIAL's safety score on Llama-2-7b (4.29) is notably its weakest aspect, though this improves to 4.97 with K=8 examples.

URIAL substantially outperforms the tuning-free baselines on Llama-2-7b: zero-shot averages 2.90, vanilla ICL 3.18, retrieval ICL 3.72, and URIAL 4.33 (Table 1). The 1.15-point gap between retrieval ICL (the strongest prior method) and URIAL represents the value of restyled examples over semantically relevant but unstyled ones.

The comparison to proprietary models shows remaining gaps: gpt-3.5-turbo-0301 scores 4.75, gpt-4-0314 scores 4.79, and gpt-4-0613 scores 4.80 (Table 1). URIAL on Mistral-7b (4.63) and Llama-2-70bq (4.74) approach but do not match these scores, with the gap concentrated in factuality and reasoning-intensive tasks (Figure 7).

Aspect-Level Tradeoffs Between Alignment Methods

The multi-aspect evaluation reveals tradeoffs invisible in aggregate scores (Table 1, Section 4.4). Llama-2-7b-chat (RLHF) exhibits a distinctive profile: highest safety (5.00) and engagement (4.70) among all 7B models, but lowest helpfulness (4.10) and below-average factuality (4.26). In contrast, Vicuna-7b (SFT only) shows the reverse pattern: higher helpfulness (4.43) and factuality (4.33), but lower safety (4.60) and engagement (4.51). The paper interprets this as evidence that "RLHFed Llama-2-Chat models may have been overfitted to safety and engagement" (Section 4.4), potentially at the expense of knowledge provision. URIAL on Llama-2-7b (4.33 average) achieves a more balanced profile, with helpfulness (4.22) between the two tuned extremes and engagement (4.65) closer to the chat model's level.

The paper also observes that "Llama-2-chat generates longest outputs that are nearly 250 words on average, while the others are around 150-180 words" (Section 4.4), noting that prior evaluations based solely on win rates "tend to prefer longer outputs." The multi-aspect scoring partially controls for length bias by evaluating each dimension independently.

Difficulty-Conditioned and Task-Conditioned Performance

Figure 7 presents a radar chart of performance across task types (information-seeking, reasoning, coding, math, role-play, writing, procedural) and topics (lifestyle, STEM, humanities, finance, medical, nature, ethics). The key finding: "Open-source LLMs including URIAL are weak on coding and math tasks as well as STEM topics, although they can match the performance of GPTs on other data categories" (Section 4.4). GPT-4 and GPT-3.5-turbo exhibit "a much more balanced performance on almost all tasks and topics," while URIAL and tuned open-source models show sharper domain-specific drop-offs. This suggests that the base model's pre-training coverage—not the alignment method—determines performance on specialized domains; URIAL cannot compensate for gaps in the base model's knowledge (consistent with the finding in Section 2 that alignment does not teach new factual content).

Human Evaluation Confirms URial's Competitiveness

Table 2 reports human pairwise comparisons on 100 randomly sampled examples. URIAL on Mistral-7b wins 20.0% of comparisons against Mistral-7b-Instruct, while the SFT model wins only 8.9%, with 71.1% ties. Similarly, URIAL on Llama-2-70bq wins 16.5% against Llama-2-70b-chatq, which wins 8.4%, with 75.1% ties. The high tie rates reflect that both methods produce comparably good outputs most of the time, but when a preference exists, it favors URIAL roughly 2:1 over the tuned counterpart in both comparisons. These human judgments corroborate the GPT-4-based scores in Table 1.

Sensitivity to Number of In-Context Examples

Table 1 includes URIAL variants with K=1, K=3, and K=8. On Llama-2-7b, K=8 improves safety substantially (4.29 → 4.97) but reduces performance on other aspects: helpfulness drops from 4.22 to 4.08, factuality from 4.16 to 4.09, and depth from 3.88 to 3.68. The average score with K=8 (4.37) is only marginally higher than K=3 (4.33), and the paper recommends K=3 "due to its balanced performance and lower cost" (Section 4.3).

On Llama-2-70bq, K=1 already achieves 4.62, demonstrating that "Llama-2-70b can achieve respectable performance with only K=1 example for URIAL" (Section 3.3). K=3 improves this to 4.74, with the largest gains in depth (4.09 → 4.30) and engagement (4.67 → 4.85).

On Mistral-7b, K=8 shows similar benefits to K=3, with slightly higher safety (5.00 vs. 4.92) and engagement (4.78 vs. 4.74) but lower depth (4.05 vs. 4.18), yielding an essentially identical average (4.62 vs. 4.63).

Ablation Studies and Robustness Checks

Vanilla ICL vs. Retrieval ICL vs. URIAL (Table 1, Llama-2-7b row): This three-way comparison isolates the contributions of stylistic restyling and semantic relevance. Vanilla ICL with K=3 unstyled examples achieves 3.18 average, only modestly above zero-shot (2.90). Adding semantic relevance through retrieval boosts this to 3.72—a meaningful improvement of 0.54 points, demonstrating that topical similarity does help. However, URIAL with constant but restyled examples achieves 4.33—an additional 0.61-point gain over retrieval ICL, and 1.15 points above vanilla ICL. This quantifies the relative importance of style vs. relevance: restyling the outputs of constant examples contributes more to alignment quality than retrieving semantically relevant examples in plain style. The paper interprets this as directly supporting Min et al.'s (2022) finding that "ICL primarily concerns the style and format of demonstrations, rather than their truth content" (Section 3.3).

System prompt contribution (implicit ablation, Table 1): The zero-shot templated prompting baseline (2.90 average on Llama-2-7b) uses only the Markdown template without system prompt or examples. Vanilla ICL (3.18) adds unstyled examples. URIAL (4.33) adds both restyled examples and system prompt. The 1.15-point gap between vanilla ICL and URIAL reflects the combined effect of restyling + system prompt. The paper does not provide a separate ablation removing the system prompt while keeping restyled examples, so the marginal contribution of the system prompt alone cannot be isolated from Table 1. However, the design rationale (Section 3.3) argues that the system prompt is essential for establishing general behavioral expectations that the examples then instantiate.

Number of examples (K=1, 3, 8; Table 1): As discussed above, K=3 is the recommended configuration, with K=1 viable for strong base models and K=8 showing diminishing or negative returns on non-safety aspects for Llama-2-7b. The non-monotonic relationship between K and performance (particularly the drop in helpfulness and factuality at K=8) is a notable finding suggesting that more examples may over-constrain the model or introduce conflicting stylistic signals. The paper does not explore K=2, K=4, etc., so the exact shape of the K-vs-performance curve is not characterized.

Alternative example set (Section 4.3, Mistral-7b): Testing URIAL with a different set of three restyled examples on Mistral-7b yields an average score of 4.64, "even slightly better than the default version (4.63 vs. 4.64)," with individual scores: helpfulness 4.60, clarity 4.90, factuality 4.50, depth 4.20, engagement 4.74, safety 4.93. This near-identical performance demonstrates robustness to the specific choice of examples—the key is the style of the examples (structured, engaging, safety-aware), not their particular content.

Multi-turn conversation capability (Appendix B.3): The paper provides a qualitative case study showing URIAL on Llama-2-7b handling a four-turn conversation about Bloody Mary recipes, including follow-up requests, thanks, and creative naming. While not a quantitative ablation, this demonstrates that URIAL's alignment extends to multi-turn interactions by appending previous turns as in-context examples. The paper claims this "suggests that the conversation ability of aligned LLMs might be largely from the base models themselves" (Section 4.4), though this is based on a single demonstration rather than systematic evaluation.

Base model quality as a confound (Table 1, across model rows): Not an ablation per se, but a critical robustness check: URIAL's effectiveness scales with base model quality. Zero-shot performance on helpfulness serves as a proxy for base model capability: Llama-2-7b scores 3.05, while Mistral-7b scores higher (exact zero-shot numbers for Mistral-7b are not reported in Table 1, but its zero-shot performance is implied to be better; Llama-2-70bq scores 3.70). URIAL on Mistral-7b achieves 4.63 versus 4.33 on Llama-2-7b, and URIAL on Llama-2-70bq achieves 4.74. The paper concludes that "when the base LLMs are well-pretrained, SFT and RLHF may not be as crucial for alignment as previously believed" (Section 4.3). This implies that URIAL's success is contingent on a sufficiently capable base model—it cannot compensate for poor pre-training.

Safety-specific tradeoff with example count (Table 1, Llama-2-7b, K=8): The jump in safety from 4.29 (K=3) to 4.97 (K=8) suggests that safety alignment is more responsive to the number and diversity of safety-related examples than other aspects. The K=8 configuration adds examples for "mathematics, coding, poetry writing, procedure, and safety" (Section 4.3), including at least one additional safety-oriented example. The simultaneous decline in helpfulness (4.22 → 4.08) and factuality (4.16 → 4.09) suggests a safety-helpfulness tradeoff that can be modulated by example selection—more safety examples make the model more cautious but less forthcoming with information. This parallels the safety-helpfulness tradeoff observed in the Llama-2-7b-chat vs. Vicuna-7b comparison (Table 1), but URIAL achieves it through example selection rather than RLHF training.

Domain-specific weaknesses (Figure 7): The task- and topic-conditioned analysis reveals that URIAL, like other open-source methods, underperforms on coding and math tasks compared to GPT-4. This is not framed as an ablation but serves as an important boundary condition: URIAL cannot teach the base model capabilities it lacks. For domains where the base model's pre-training coverage is thin, neither in-context alignment nor tuning-based alignment can compensate—the knowledge must come from pre-training. The paper does not provide quantitative scores per task/topic category, only the radar chart visualization.

Negative result with temperature and decoding strategy: While not presented as a formal ablation, the paper notes (Appendix D.1) that "some aligned models may achieve enhanced performance with specific settings for temperature, top p, and other parameters," but all experiments use greedy decoding for reproducibility. This means URIAL's reported performance is a lower bound on what could be achieved with optimized decoding—the base model might produce even better outputs with temperature sampling (as is common for aligned models in practice), but this is not tested.

Critical Assessment

Claim 1: Alignment tuning primarily affects stylistic tokens, not knowledge-bearing words.

What the experiments demonstrate: The token distribution shift analysis (Section 2, Figures 2–4) directly supports this claim. The measurement protocol is clearly defined, the three-category classification (unshifted/marginal/shifted) is operationalized with a specific threshold ($\eta > 3$), and the results across three model pairs are consistent: shifted tokens constitute only 5–8% of all positions (Figure 3). The qualitative analysis of which tokens appear in shifted positions—discourse markers, greetings, safety disclaimers, conversational framing—is presented with concrete examples (Figure 2, Figure 3 bottom boxes). The converging evidence from KL-divergence, base-rank, and base-probability metrics (Figure 4) provides robustness to the choice of shift metric.

What is not demonstrated: The analysis is correlational, not causal. It shows that shifted tokens tend to be stylistic, but it does not demonstrate that changing those specific tokens through alignment causes the observed behavioral differences. It is possible that alignment tuning makes small distributional changes across many tokens (including knowledge-bearing ones) that the $\eta$ metric categorizes as "marginal" or "unshifted" but that collectively produce the qualitative behavioral shift. The claim that "knowledge-intensive content originates from untuned LLMs" (Section 2.2) is supported by Figure 2's example but is not systematically quantified: what fraction of knowledge-bearing tokens are unshifted vs. marginal vs. shifted? The paper provides aggregate ratios (77.7% unshifted) but not ratios conditioned on token type (noun vs. discourse marker), which would directly test the style-vs-knowledge decomposition. A systematic part-of-speech or semantic category analysis would strengthen the claim.

Additionally, the analysis is performed only on the aligned model's trajectory. The base model is never allowed to generate its own responses and have those evaluated for correctness or style. The possibility remains that the base model, when forced to follow the aligned model's prefix, produces different knowledge-bearing tokens than it would in free generation—the prefix constrains the autoregressive path in ways that might hide distributional differences in content words. The paper acknowledges this implicitly by noting the template differences between base and aligned model inputs.

Claim 2: URIAL matches or surpasses SFT and RLHF alignment performance.

What the experiments demonstrate: Table 1 shows URIAL exceeding Mistral-7b-Instruct (4.63 vs. 4.44) and Llama-2-70b-chatq (4.74 vs. 4.67) on average scores, with human evaluation (Table 2) confirming the direction of preference. The aspect-level breakdown shows URIAL's advantages are consistent rather than driven by a single dimension. The comparison to tuning-free baselines (zero-shot, vanilla ICL, retrieval ICL) establishes that the gains come specifically from URIAL's design (restyled examples + system prompt) rather than from in-context learning generically.

What is not demonstrated (or is demonstrated only conditionally):

1. Llama-2-7b is a partial exception. URIAL on Llama-2-7b (4.33) does not exceed Vicuna-7b (4.46) or Llama-2-7b-chat (4.47) in average score; it closes the gap substantially but remains behind. The claim of "matching or surpassing" applies most clearly to Mistral-7b and Llama-2-70b—the stronger base models. The paper acknowledges this contingency (Section 4.3: "when the base LLMs are well-pretrained, SFT and RLHF may not be as crucial"), but the headline claim (abstract, introduction) is stated more universally.

2. The tuned baselines may not be optimally configured. The paper uses recommended settings for aligned models (system prompts, greedy decoding) but notes that "some aligned models may achieve enhanced performance with specific settings for temperature, top p, and other parameters" (Appendix D.1). The comparison of URIAL (with a carefully designed, manually crafted prompt) against aligned models (with their default inference configuration) may advantage URIAL if the aligned models are not at their best under greedy decoding. A fairer comparison would sweep decoding parameters for the aligned models or report their best-known configurations.

3. The evaluation is GPT-4-based, and GPT-4 may have stylistic preferences that align with URIAL's restyled output format. URIAL's examples are manually crafted to mimic the style of well-aligned assistants, which is also the style GPT-4 itself produces. There is a risk of evaluator bias: GPT-4 may prefer outputs that resemble its own generation style, which URIAL is explicitly designed to match. The human evaluation (Table 2) partially mitigates this concern (87.8% agreement with GPT-4), but the human evaluation used only 100 examples and asked for pairwise preferences rather than absolute scoring, making it a weaker check than a full human replication of the 1,000-example multi-aspect evaluation.

4. Out-of-distribution generalization is untested. All 1,000 test examples are drawn from the same distribution as the datasets used to design URIAL's examples (the examples cover refusal, role-playing, and factual information—the same categories present in just-eval-instruct). There is no test of whether URIAL generalizes to instruction types not represented in its three examples (e.g., code generation, mathematical proofs, multi-step planning), and Figure 7 suggests it underperforms on those categories. The paper's claim that URIAL "matches or surpasses" alignment tuning is therefore implicitly scoped to the types of open-domain assistant queries represented in just-eval-instruct.

5. Single evaluation dataset, single evaluation protocol. All results are on just-eval-instruct using GPT-4 scoring with the specific six-aspect template. Replication on independent benchmarks (AlpacaEval's full set, MT-Bench's full set with its own scoring protocol, human evaluation at scale) would strengthen the claim substantially. The 1,000-example dataset, while diverse, is not large enough to produce tight confidence intervals on the differences between methods—the standard error on a 1–5 scale with 1,000 examples is approximately 0.03–0.05 points for the average, meaning the 4.63 vs. 4.44 difference between URIAL and Mistral-7b-Instruct is statistically meaningful, but smaller differences (e.g., 4.63 vs. the alternative example set's 4.64) are within noise.

Claim 3: URIAL demonstrates that tuning-free alignment is a viable alternative to SFT/RLHF.

What the experiments demonstrate: URIAL achieves competitive or superior performance to tuned models on two of three base models tested (Mistral-7b, Llama-2-70b), using only three constant examples and no weight modification. The method is simple, reproducible, and efficient (static KV-cache). These results establish a strong baseline that tuning-based methods must beat to justify their additional cost.

What is not demonstrated:

1. The difficulty of crafting URIAL's examples is not accounted for. The three restyled examples were manually authored by the researchers with careful attention to stylistic patterns identified in the token distribution analysis. The claim of "tuning-free" alignment is accurate in the sense that no gradient updates occur, but the human effort of designing the prompt is a form of "tuning" through example engineering. The paper demonstrates robustness to alternative example sets (Section 4.3), suggesting the exact examples are not critical, but the stylistic template (greeting, enumeration, summary, safety disclaimer) is—and this template was derived from careful analysis. A genuinely "tuning-free" method should work with automatically constructed examples; whether URIAL would succeed with algorithmically restyled examples (rather than manually crafted ones) is untested.

2. The URIAL examples contain factual errors. This is an interesting negative observation not made in the paper but visible in the reproduced examples (Appendix A). For instance, the renewable energy example states that solar panels convert sunlight "without any moving parts, noise, pollution, or damage to the environment"—a claim that ignores the environmental impact of solar panel manufacturing and disposal. If the base model learns to emulate this style, it may also emulate the tendency toward uncritical, overly positive framing of topics. More concerningly, the examples contain minor factual inaccuracies (e.g., the exact mechanism of geothermal energy) that the base model might replicate. The paper does not evaluate whether URIAL induces systematic factual errors by conditioning on slightly inaccurate examples.

3. The method's limits for safety-critical applications are not rigorously tested. The 200 safety examples from HH-RLHF-redteam and MaliciousInstruct provide some coverage, but these datasets are known to contain primarily straightforward harmful requests (explicit violence, hate speech) rather than subtle safety challenges (e.g., requests that appear benign but enable harm, or requests requiring nuanced ethical judgment). The paper's case study (Appendix B.2) shows an RLHF model refusing a harmless creative writing request that URIAL handles well, but the reverse case—URIAL failing to refuse a subtly harmful request that an RLHF model would catch—is not explored. Safety evaluation on more challenging adversarial datasets (e.g., Anthropic's red-teaming data, or jailbreak attempts) would be necessary to establish URIAL as a safe deployment method. The K=8 results (safety improves to 4.97 for Llama-2-7b) suggest safety is controllable through example selection, but the limits of this control are unexplored.

4. The static nature of URIAL may limit adaptability. Tuned models can, in principle, be fine-tuned further for specific domains or updated as safety norms evolve. URIAL's behavior is entirely determined by its fixed prompt; adapting to new requirements means rewriting the prompt, which may interact unpredictably with the base model's capabilities. The multi-turn conversation example (Appendix B.3) suggests some flexibility, but systematic evaluation of URIAL's robustness to adversarial users, distribution shift, or evolving safety standards is absent.

5. The resource comparison is incomplete. The paper argues URIAL is "efficient" because it avoids GPU-intensive fine-tuning and enables KV-cache reuse. However, the upfront cost of the token distribution analysis (Section 2) and the human effort to craft restyled examples are not amortized in any cost calculation. For a lab that would need to perform its own token distribution analysis to understand what stylistic patterns to encode in examples, the total human+compute cost of developing a URIAL-equivalent prompt may be non-trivial. The paper implicitly assumes the examples transfer across base models (the same three examples are used for Llama-2-7b, Mistral-7b, and Llama-2-70b), which is supported by the results but may not hold for model families with different pre-training distributions.

Missing Experiments That Would Strengthen the Paper

1. Decoding strategy sweep for URIAL and baselines. All experiments use greedy decoding. Since URIAL's examples are designed to encourage a particular style, temperature sampling might produce more diverse but equally well-styled responses, potentially improving helpfulness and engagement. Conversely, aligned models might benefit from temperature tuning. The absence of any decoding strategy exploration means the reported numbers may not reflect each method's best-case performance.

2. Blind human evaluation at scale. The human evaluation covers only 100 examples and uses pairwise preference (which is easier for annotators but provides less information than absolute scoring on the six aspects). A full human replication of the GPT-4 evaluation on even a subset (e.g., 200 examples with all six aspects scored by multiple annotators) would substantially strengthen confidence that the GPT-4 judgments are not systematically biased toward URIAL's style.

3. Ablation of the system prompt. URIAL combines a system prompt with restyled examples. Removing the system prompt while keeping the restyled examples (or vice versa) would isolate the contribution of each component. This ablation is not reported, making it unclear whether the system prompt is essential or whether restyled examples alone suffice.

4. Comparison to minimal SFT. LIMA (Zhou et al., 2023) showed that SFT with 1,000 carefully chosen examples can achieve strong alignment. A direct comparison between URIAL on a base model and LIMA-style minimal SFT on the same base model would be the most direct test of whether in-context alignment can truly substitute for weight-based alignment. The current SFT baselines (Vicuna, Mistral-Instruct) use much larger tuning datasets, making the comparison somewhat asymmetric: URIAL uses 3 examples, while the tuned models use thousands. A matched comparison—3 examples vs. fine-tuning on those same 3 examples—would isolate the mechanism (in-context vs. weight update) from the data quantity.

5. Adversarial robustness evaluation. Testing URIAL against jailbreak prompts, role-playing attacks, or multi-turn manipulation would reveal whether in-context safety alignment is as robust as weight-based safety alignment. The current safety evaluation uses straightforward harmful requests; real-world deployment would face more sophisticated attacks.

6. Cross-model family generalization. All base models tested are from two families (Llama-2 and Mistral). Testing URIAL on other architectures (Falcon, MPT, Qwen, etc.) would establish whether the method depends on specific pre-training data characteristics or generalizes broadly. The consistent shifted-token patterns across the three tested pairs (Figure 3) suggest generalizability, but this is not demonstrated experimentally for URIAL's performance.

In summary, the experiments provide strong support for the paper's central claims within the tested scope: on the just-eval-instruct dataset, on Llama-2 and Mistral model families, using GPT-4 evaluation with human spot-checking, URIAL matches or exceeds tuned alignment for stronger base models. The claims about alignment being "superficial" are well-supported by the token distribution analysis, though the causal interpretation is correlational. The practical claim that URIAL is a "viable alternative" to tuning is supported with the important caveat that this holds for the types of open-domain assistant queries in the evaluation set, on base models with sufficient pre-training quality, and the method's safety guarantees under adversarial conditions remain unexplored.

6. Limitations and Trade-offs

The Difficulty of Crafting URIAL's Examples Is Unaccounted For in the Cost Calculus

The assumption or constraint. URIAL is presented as a "tuning-free" alignment method that avoids the computational expense of SFT and RLHF. However, the three restyled examples and the system prompt are not algorithmically generated—they are manually authored by the researchers based on careful analysis of the token distribution shift findings (Section 2) and deliberate incorporation of specific stylistic patterns (greeting affirmations, structured enumeration with [n]. [topic]: [details] format, concluding summaries, safety disclaimers, engagement invitations). The paper acknowledges this human effort only implicitly, through statements like "we curate a few restyled examples" and "we tailor our outputs to start by rephrasing the question in an engaging and declarative manner" (Section 3.3). There is no discussion of how a practitioner would develop equivalent examples for a new base model family or a different deployment context, nor any measurement of the human effort required.

The consequence. A lab adopting URIAL for a new base model cannot simply download a pre-built solution—they face an implicit prompt engineering cost that the paper does not quantify. The alternative example set tested in Section 4.3 demonstrates that URIAL is "robust to variations in the ICL examples provided" (the score changes from 4.63 to 4.64), but this robustness was demonstrated with another set of manually crafted examples following the same design template. Whether URIAL would work with automatically constructed examples (e.g., restyled by an LLM rather than a human) is entirely untested. If the method's success depends on subtle aspects of the restyling that require human judgment to get right—such as the exact phrasing of safety disclaimers, the balance between engagement and verbosity, or the specific choice of which stylistic tokens to include—then the "tuning-free" label is somewhat misleading: the tuning has simply shifted from gradient-based weight updates to labor-intensive prompt engineering. The cost is smaller than full SFT+RLHF, but it is not zero, and it may not be trivial for practitioners without expertise in the token-level analysis that motivated the design.

What evidence exists in the paper. Section 4.3 reports near-identical performance (4.63 vs. 4.64) with an alternative set of three manually authored examples, confirming robustness to example selection within the same stylistic template. However, no experiment varies the stylistic template itself—for example, testing whether examples without the [n]. [topic]: [details] enumeration format, or without the "Remember" safety summary pattern, or with substantially different tone, would achieve comparable performance. The only evidence about the importance of the specific stylistic choices comes from the comparison to vanilla ICL (Table 1), which uses unstyled examples and achieves dramatically lower performance (3.18 vs. 4.33 on Llama-2-7b). This comparison conflates two variables: the content of the examples (different queries and responses) and their style (plain vs. restyled). It does not isolate how much of the 1.15-point gain comes from the specific stylistic patterns versus simply having any well-structured examples.

Mitigation status. The paper does not address this limitation explicitly. It does not provide guidelines for automatically generating restyled examples, nor does it measure the sensitivity of URIAL's performance to the quality of the restyling. The implication that any practitioner can replicate the method by following the described template (greeting → enumeration → summary → safety disclaimer) is not empirically validated. Future work on "developing better tuning-free, inference-time alignment methods" (Section 5.4) would need to establish whether the stylistic patterns can be algorithmically extracted and applied, or whether they require the kind of manual, analysis-informed curation that the paper itself performed.

Safety Guarantees Under Adversarial Conditions Are Unexplored

The assumption or constraint. The safety evaluation in URIAL uses 200 examples drawn from HH-RLHF-redteam (Ganguli et al., 2022) and MaliciousInstruct (Huang et al., 2023). These datasets contain primarily straightforward harmful requests: explicit queries about violence, hate speech, dangerous instructions, and unethical content that an assistant should unambiguously refuse. The paper does not evaluate URIAL against more sophisticated safety challenges: jailbreak prompts designed to circumvent refusal mechanisms, multi-turn manipulation where harmful intent is gradually revealed, benign-seeming queries that enable downstream harm, or adversarial inputs that exploit the base model's in-context learning mechanism itself. The safety score of 4.92–4.97 (Table 1, URIAL K=3 and K=8 on Mistral-7b and Llama-2-70b) reflects performance on overtly harmful requests with clear refusal patterns demonstrated in the in-context examples. The paper does not claim that URIAL provides safety guarantees comparable to extensively RLHF-trained models, but it also does not characterize the safety boundary—the conditions under which the in-context safety alignment breaks down.

The consequence. An in-context safety policy is fundamentally different from a weight-based one. RLHF modifies the model's internal representations to associate harmful outputs with low reward, making it harder (though not impossible) for adversarial prompts to elicit dangerous content. URIAL's safety policy exists only in the prompt prefix—the base model's weights are entirely unchanged. An adversary who can construct a prompt that overrides or circumvents the in-context safety instructions (e.g., by instructing the model to "ignore previous instructions" or by providing counterexamples that normalize harmful behavior) may be able to access the base model's full, unfiltered knowledge, including its ability to generate harmful content. The paper's own token distribution analysis (Section 2) shows that base models already possess all the knowledge needed to answer harmful queries—they merely lack the stylistic framing to refuse appropriately. URIAL provides that framing through the prompt, but the framing can potentially be stripped away by a sufficiently adversarial input. The safety evaluation provides no evidence about the robustness of the in-context safety policy to such attacks.

The practical risk is that a practitioner deploying URIAL in a user-facing application—relying on the 4.92 safety score as evidence of harmlessness—could face jailbreak vulnerabilities that the evaluation did not detect. The paper's case study (Appendix B.2) actually demonstrates a reverse failure mode: an RLHF model (Llama-2-70b-chat) over-refuses a harmless creative writing request, while URIAL handles it appropriately. This suggests URIAL's safety policy is more flexible than RLHF's, but flexibility in safety is a double-edged sword—it may also mean the policy is easier to manipulate.

What evidence exists in the paper. The only safety evidence is the GPT-4-based safety scores on the 200 red-teaming and malicious instructions (Table 1, é safety column). The paper does not report:

• Adversarial success rates (how often can a jailbreak prompt elicit harmful content from URIAL compared to RLHF models?)
• Multi-turn safety evaluation (does URIAL maintain safety across conversation turns, or can a user gradually steer it toward harmful content?)
• Safety on ambiguous or borderline queries (where harmfulness is context-dependent rather than obvious)
• The distribution of safety scores (are the 200 examples mostly receiving 5s, with a few failures dragging down the average? Or is there meaningful variance?)

The K=8 results partially address safety controllability: adding more safety examples raises the safety score on Llama-2-7b from 4.29 (K=3) to 4.97 (K=8), suggesting that safety alignment scales with the number and diversity of safety-related in-context examples. But this only demonstrates that more examples improve safety on the same straightforward test set—it does not test whether those additional examples confer robustness to adversarial inputs.

Mitigation status. The paper does not discuss adversarial robustness or jailbreak vulnerability as a limitation. It does not suggest that URIAL is a safe deployment method for high-stakes applications—the discussion in Section 5.4 emphasizes URIAL's use cases as a "baseline method for aligning base LLMs without tuning," for "frequently evaluating base LLMs during the pre-training process," and for "fair comparison of different base LLMs based on their potential for alignment," all of which are non-deployment, research-oriented use cases. However, the strong safety scores (4.92–5.00) and the comparison to RLHF models in Table 1 could easily lead a reader to conclude that URIAL is a viable safety mechanism for production systems, which the paper does not explicitly caution against.

The Evaluation Is Single-Benchmark and Single-Evaluator (GPT-4), With Limited Human Validation

The assumption or constraint. All quantitative results in Table 1 and Figure 7 come from a single evaluation protocol: just-eval-instruct (1,000 examples) scored by GPT-4 on six aspects using the prompting templates in Appendix D.5. While the paper argues that "employing ChatGPT and GPT-4 for scoring and comparing LLM outputs can achieve high agreement with human evaluation while reducing costs" (Section 4.2) and provides 100 human-validated pairwise comparisons showing 87.8% agreement (Appendix D.3), this is a thin layer of human validation relative to the scope of the claims. The human validation covers only 10% of the test examples, uses pairwise preference (a coarser signal than the 1–5 Likert scales used for automated evaluation), and does not independently score the six aspects—it only asks for an overall preference between two outputs. A 12.2% disagreement rate on the 100 validated examples, if extrapolated to the full 1,000-example set, implies approximately 122 examples where human and GPT-4 judgments diverge—enough to potentially change the relative ordering of methods with small score differences (e.g., URIAL K=3 at 4.63 vs. the alternative example set at 4.64, or URIAL at 4.33 vs. Vicuna-7b at 4.46).

The consequence. There are two distinct risks to the validity of the claims:

Risk 1: GPT-4 evaluator bias toward its own style. GPT-4, as an RLHF-aligned model, produces outputs that follow the same stylistic conventions that URIAL's restyled examples are designed to emulate: structured enumeration, engaging tone, polite framing, safety disclaimers. If GPT-4's evaluation preferences are correlated with its own generation style—preferring outputs that "sound like something GPT-4 would say"—then URIAL may receive inflated scores relative to methods that produce equally correct but stylistically different outputs. The paper's human validation partially addresses this for pairwise preferences but does not validate the absolute 1–5 scores on individual aspects. A human annotator might agree that "Output A is better than Output B" (the pairwise judgment) while still rating Output A as a 3 rather than a 4 on an absolute scale—and the paper's human validation only tests the former. The 94.1% human approval rate for "explanations" (Section 4.2) suggests the rationales are coherent, but it does not validate that the scores themselves are calibrated to human standards.

Risk 2: Benchmark-specific overfitting of the URIAL prompt. The just-eval-instruct dataset aggregates examples from 9 existing datasets, but these datasets were all publicly available before URIAL's development. While the paper states that the three restyled examples are "constant" and not tailored to individual test queries, the design of those examples—covering refusal, role-playing, and factual exposition—may have been implicitly informed by the types of queries the researchers knew would appear in the evaluation (since the component datasets are well-known in the community). The alternative example set (Section 4.3) provides some evidence against overt overfitting, but the stylistic template (greeting → enumeration → summary → safety disclaimer → engagement) was fixed across both example sets and could itself be a form of benchmark-aware design. Independent evaluation on a held-out benchmark not used during URIAL's development would be the strongest test of generalization, but this is absent.

What evidence exists in the paper. The human validation (Table 2, Appendix D.3) covers 100 randomly sampled examples with pairwise comparisons: URIAL vs. SFT (Mistral-7b) and URIAL vs. RLHF (Llama-2-70b). The results favor URIAL (20.0% vs. 8.9% wins for Mistral-7b, 16.5% vs. 8.4% for Llama-2-70b), consistent with the GPT-4 scores. However, tie rates are very high (71.1% and 75.1%), meaning that on most examples, human annotators could not distinguish the outputs—suggesting the automated scoring may be making finer distinctions than humans perceive. The agreement rate of 87.8% "with GPT-based judgments" (Section 4.2) is reported but the breakdown (how often does GPT-4 prefer URIAL when humans prefer the tuned model, and vice versa?) is not provided. Without this breakdown, the 87.8% figure could mask systematic biases in a particular direction.

Mitigation status. The paper acknowledges the limitation of GPT-based evaluation implicitly by providing human validation, but does not discuss the potential for evaluator bias or benchmark overfitting. Section 4.2 frames the multi-aspect, explainable evaluation as an improvement over prior coarse-grained approaches, which it is—but it does not address the fundamental limitation that the evaluator is itself an LLM with unknown biases. The suggestion in Section 5.4 that URIAL can be used to "frequently evaluate base LLMs during the pre-training process" would require the evaluation protocol to be reliable enough to track genuine improvements in base model quality, which is a higher bar than the between-method comparisons in Table 1.

Knowledge Forgetting and Hallucination in Tuned Models Are Documented but URIAL's Own Factuality Limitations Are Not Measured

The assumption or constraint. A key motivation for URIAL is that tuning-based alignment can cause knowledge forgetting in base LLMs: the paper cites Wang et al. (2023) showing SFT degrades MMLU and Codex-Eval performance, and provides a case study (Appendix B.1) where Mistral-7b-Instruct incorrectly answers "No, Facebook corporation did not change its name" while the base model with URIAL correctly answers that it changed to Meta Platforms Inc. The implication is that URIAL, by not modifying weights, preserves the base model's factual knowledge intact. However, the paper never systematically evaluates whether URIAL introduces factual errors through its in-context conditioning mechanism. The restyled examples in the URIAL prompt (Appendix A) contain claims that are at minimum imprecise and potentially misleading—for example, describing solar energy as harnessed "without any moving parts, noise, pollution, or damage to the environment" (ignoring manufacturing impacts), or describing geothermal energy as "nearly as reliable as the tides" (a questionable analogy). If the base model learns the style of these examples—including their tendency toward uncritical, positively-valenced descriptions—it may reproduce not just the structure but also the epistemic stance: confident, detailed, but not necessarily accurate.

The consequence. The paper's factuality scores in Table 1 show URIAL achieving 4.16 (Llama-2-7b), 4.50 (Mistral-7b), and 4.65 (Llama-2-70bq) on the 1–5 factuality scale, which appear competitive with tuned models. However, these scores are GPT-4's perception of factuality, not a ground-truth assessment. GPT-4 may assign high factuality scores to outputs that sound factual—confident, detailed, well-structured—even when they contain errors, because GPT-4 itself may not catch subtle inaccuracies (especially in domains where its own knowledge is imperfect). The case study in Appendix B.1 demonstrates that the base model with URIAL correctly identifies Facebook's name change—but this is a single cherry-picked example. The paper does not systematically measure:

• The factual error rate of URIAL outputs compared to tuned models on questions with verifiable ground-truth answers
• Whether URIAL's confident, detailed style (learned from the restyled examples) increases the rate of confident hallucinations—assertions that sound authoritative but are incorrect
• Whether the specific factual content of the in-context examples (e.g., the renewable energy descriptions) influences the base model's factual claims on related topics

This matters because URIAL's value proposition is that it preserves base model knowledge while adding stylistic alignment. If the stylistic conditioning simultaneously increases the model's propensity to produce fluent but incorrect content (a known failure mode of LLMs that is amplified by confidence-expressing prompts), the tradeoff between tuning-based forgetting and URIAL's potential hallucination amplification is not characterized.

What evidence exists in the paper. The factuality column in Table 1 provides GPT-4-based scores that show URIAL performing comparably to or better than tuned models on factuality (e.g., URIAL on Mistral-7b scores 4.50 vs. Mistral-7b-Instruct at 4.29). However, these scores are not validated against ground-truth factual accuracy on any subset of the 1,000 examples. The paper does not report:

• A knowledge-probing benchmark (e.g., MMLU, TriviaQA, Natural Questions) comparing base model, URIAL-conditioned base model, and tuned model accuracy
• A human factuality audit on even a small sample of the just-eval-instruct responses
• Any analysis of whether the URIAL examples' factual content "leaks" into the model's outputs on related queries (e.g., does the renewable energy example bias the model's responses to other energy-related questions?)

The paper mentions that "GPT-4 and GPT-3.5-turbo exhibit a much more balanced performance on almost all tasks and topics" compared to open-source models including URIAL (Section 4.4, Figure 7), but this primarily reflects domain-specific capability gaps (coding, math, STEM) rather than systematic factuality measurement.

Mitigation status. The paper does not address this limitation. The discussion of "knowledge forgetting" in Section 5.1 positions URIAL as a solution to a problem caused by tuning, but does not consider whether URIAL might introduce its own factual reliability issues through a different mechanism. The broader claim that "base LLMs have already acquired adequate knowledge to follow instructions" (Section 2.3, key finding #4) is about knowledge possession, not knowledge deployment—the base model may know the correct answer but, when conditioned on URIAL's restyled examples, may nonetheless produce an answer that prioritizes style over precision. Future work on "extending the analysis by connecting the token distribution shift to the model weights and attribution to training data" (Section 6) could potentially address this by tracing whether factual errors in URIAL outputs originate from the base model, the in-context examples, or an interaction between the two.

Temporal Dynamics Suggest Alignment's Influence Is Concentrated in Early Tokens, but the Causal Mechanism Is Not Tested

The assumption or constraint. One of the paper's key findings is that token distribution shift "diminishes over time during decoding" (Section 2.2, Figure 4): KL-divergence decreases, base-rank drops below 5 shortly after $t \geq 5$, and base-probability approaches 1.0 at later positions. The paper interprets this as evidence that "alignment is more critical for earlier tokens" and that once the stylistic frame is established by the first few tokens, the base model's natural completions are well-aligned (Section 2.3). This interpretation motivates URIAL's design: provide the stylistic framing through in-context examples, and the base model will sustain that framing throughout the response. However, the temporal analysis is correlational, not causal. It shows that, in the aligned model's trajectory, later tokens have lower shift—but it does not demonstrate that the early-token shift causes the later-token alignment. An alternative explanation is that aligned models learn a global stylistic policy that happens to manifest most visibly in early tokens (where stylistic choices like greetings and structural cues are concentrated), while the later-token convergence simply reflects that content words are inherently less affected by stylistic alignment regardless of position.

The consequence. If the causal interpretation is incorrect—if alignment affects early tokens more because that's where stylistic decisions naturally cluster, rather than because early tokens control later alignment—then URIAL's mechanism may be less reliable than the paper implies. Specifically, the base model conditioned on URIAL's examples might produce well-styled beginnings (greetings, structural transitions) but then drift into unaligned behavior in later parts of long responses, because the in-context examples provide only a template for response structure, not the internal weight modifications that (under the causal interpretation) would constrain the entire autoregressive trajectory. The paper does not test this: Figure 4's analysis is performed on the aligned model's outputs, not on URIAL-conditioned base model outputs. We do not know whether URIAL reproduces the same temporal pattern (early-token shift, late-token convergence) as weight-based alignment, or whether it shows a different pattern (e.g., early-token alignment that decays over very long responses as the in-context influence wanes).

What evidence exists in the paper. The paper presents no equivalent of Figure 4 for URIAL-conditioned base models—no measurement of how token-level alignment evolves across positions when the alignment is provided through in-context learning rather than weight modification. The multi-turn conversation example (Appendix B.3) shows URIAL maintaining assistant-like behavior across four turns, but this is a single qualitative demonstration, not a quantitative analysis of whether the alignment is sustained or degrades across token positions within each response. The fact that URIAL achieves high scores on all aspects (Table 1) suggests that gross misalignment is rare, but it does not reveal whether subtle drift occurs in the tail of long responses.

The paper also does not test a key prediction of the causal interpretation: that only the early-token stylistic framing matters for alignment, and that providing that framing without the full in-context examples would suffice. An ablation where only the first sentence of each restyled example (the greeting/affirmation) is retained, with the rest replaced by the base model's own unstyled continuation, would test whether early-token alignment is sufficient to drive full-response alignment. This ablation is not performed.

Mitigation status. The paper does not discuss this as a limitation. The interpretation of Figure 4 as showing that "alignment is more critical for earlier tokens" is presented as a finding rather than a hypothesis requiring causal validation. The URIAL design implicitly assumes the causal interpretation—that providing early-token stylistic patterns through examples will propagate alignment through the rest of the response—and the method's empirical success provides some circumstantial support. However, without a direct comparison of the temporal shift patterns between weight-aligned and URIAL-conditioned models, the causal claim remains an extrapolation from correlation. The suggestion for future work to explore "developing advanced inference-time alignment algorithms to efficiently control the behavior of LLMs" (Section 6) could include testing this causal mechanism explicitly.

The Method Does Not Combine with Search or Iterative Refinement, and Single-Pass Generation Limits Performance on Hard Tasks

The assumption or constraint. URIAL is a single-pass generation method: the base model sees the static prompt prefix plus the user query, generates one response via greedy decoding, and that response is the final output. There is no mechanism for the model to revise its output, no verifier-guided selection among multiple candidates, and no search over alternative response strategies. The paper compares URIAL against tuned models that also use single-pass greedy decoding (Appendix D.1), so the comparison is fair in the sense that neither side benefits from test-time compute scaling. However, the paper does not explore whether URIAL could be combined with inference-time techniques (best-of-N sampling, majority voting, self-consistency, iterative refinement) that are standard practice for improving LLM outputs in challenging scenarios.

The consequence. Figure 7 shows that URIAL—like other open-source methods—significantly underperforms GPT-4 and GPT-3.5-turbo on coding and math tasks, and on STEM topics. The paper attributes this to base model capability gaps ("the base model's pre-training coverage—not the alignment method—determines performance on specialized domains," Section 5 of the prior analysis). However, it is possible that some of the performance gap on hard tasks could be closed by combining URIAL's stylistic alignment with additional test-time computation: generating multiple candidate responses and selecting the best, using chain-of-thought prompting within the URIAL framework, or applying self-consistency techniques that are known to improve reasoning performance (Wang et al., 2022a). The paper does not test any of these combinations, so the reported performance numbers represent a lower bound on what tuning-free alignment could achieve with additional inference-time effort.

Conversely, the tuned models might also benefit from test-time compute, so the single-pass comparison is symmetric but potentially incomplete for understanding the full capability frontier. A lab deciding between tuning and URIAL might care not just about single-pass performance but about scalability: can the method's performance be improved by spending more inference compute? Tuned models can benefit from techniques like majority voting or best-of-N with a reward model. Whether URIAL's in-context alignment is compatible with, or even amplifies, such techniques is unexplored.

What evidence exists in the paper. The paper's entire evaluation uses greedy decoding (temperature zero) with a single response per query (Section 4, Appendix D.1). There are no experiments with:

• Sampling multiple responses from the URIAL-conditioned model and selecting via majority voting
• Combining URIAL with chain-of-thought prompting for reasoning tasks
• Using the PRM-style verifiers that are common in the math reasoning literature (the paper does not mention verifiers at all)
• Measuring whether URIAL's stylistic alignment is preserved under temperature-based sampling, or whether sampling introduces stylistic degradation

The paper's related work discussion (Section 5.2) briefly mentions RAIN (Li et al., 2023b), a "tuning-free alignment method that shares similar motivation with ours, while they focus on the inference-time self-evaluation for better decoding," and notes that "URIAL and RAIN can be potentially integrated with each other for even better inference-time alignment methods that do not require parameter tuning at all." This acknowledges the possibility of combining URIAL with test-time techniques but does not pursue it experimentally.

Mitigation status. The paper explicitly positions URIAL as a baseline method rather than a production-ready deployment solution, and Section 5.4 lists several use cases that do not require state-of-the-art performance (evaluating pre-training checkpoints, comparing base models, serving as a research baseline). In these contexts, single-pass performance is the relevant metric. However, the abstract and introduction make stronger claims—"URIAL can match or even surpass the performance of LLMs aligned with SFT or SFT+RLHF"—that a reader might interpret as URIAL being a replacement for tuning-based alignment in deployment scenarios, where test-time compute scaling would be natural to consider. The paper does not caution that the competitive performance is demonstrated only in the single-pass regime, leaving open the question of whether tuned models would regain the advantage when both sides are allowed to use additional inference compute.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the understanding of alignment from a knowledge acquisition frame to a stylistic transfer frame, and in doing so provides both a diagnostic tool and a practical method that together challenge the necessity of the dominant alignment paradigm.

The conceptual shift: alignment as register, not capability. The token distribution shift analysis (Section 2) provides the most granular evidence to date that alignment tuning does not teach models new facts or reasoning strategies—it teaches them a specific linguistic register: the conversational norms, structural patterns, and safety-aware framing that characterize AI assistant behavior. The finding that only 5–8% of token positions are substantially shifted by alignment (Figure 3), and that those shifted positions cluster in discourse markers, greetings, safety disclaimers, and conversational transitions rather than knowledge-bearing words, directly supports the Superficial Alignment Hypothesis (Zhou et al., 2023) with mechanistic evidence rather than behavioral inference. This matters because it converts a vague claim ("alignment is superficial") into an operational diagnostic: measure the base rank of aligned tokens, and you can quantify exactly how superficial a given alignment procedure is, for a given model pair, on a given task distribution.

The significance is not merely theoretical. By demonstrating that the behavioral transformation produced by SFT and RLHF can be functionally replicated through three carefully restyled in-context examples and a system prompt (Table 1: URIAL on Mistral-7b achieves 4.63 vs. 4.44 for SFT; URIAL on Llama-2-70bq achieves 4.74 vs. 4.67 for RLHF), the paper reframes alignment from a training problem (requiring GPU clusters, large instruction datasets, and human preference annotations) to a prompt engineering problem (requiring analysis of token-level stylistic patterns and careful example construction). This is not a complete replacement—the paper acknowledges that coding, mathematics, and agent tasks likely still require tuning (Section 5.4)—but for the broad class of open-domain assistant interactions covered by just-eval-instruct, the evidence suggests that the massive investment in alignment tuning is buying something that can be acquired far more cheaply.

Reconciling contradictory prior findings. The paper resolves an apparent tension in the literature that prior work had not directly addressed. On one side, LIMA (Zhou et al., 2023) showed that minimal SFT (1,000 examples) could produce strong alignment, implying that base models already possess assistant capabilities. On the other side, the practical experience of the field—and the stark qualitative differences between base and aligned model outputs—suggested that alignment tuning was essential, and that its effects were substantial. This paper shows that both perspectives are correct, but they describe different levels of analysis. Alignment is essential for producing the assistant register—without it, base models generate responses that lack the structure, safety awareness, and engagement humans expect. But alignment achieves this by modifying a very small fraction of the model's decoding decisions, concentrated in the early tokens of responses where stylistic framing is established. The base model's knowledge and reasoning are already present and largely untouched by alignment; what alignment provides is the key that unlocks them in a human-preferred format. URIAL demonstrates that this key can be provided through in-context learning rather than weight modification, at least for sufficiently capable base models.

Which research directions become more attractive. The paper makes tuning-free alignment a legitimate research program rather than a curiosity. Prior to this work, in-context methods for alignment were underexplored, with the dominant approaches being either basic few-shot prompting (vanilla ICL, achieving only 3.18 on Llama-2-7b) or computationally expensive retrieval augmentation (3.72). URIAL's 4.33 on the same base model demonstrates that the gap between tuning-free and tuning-based methods is not fundamental—it can be substantially closed through careful attention to style in in-context examples, informed by token-level analysis of what alignment actually changes. This opens a space for systematic investigation of in-context alignment design: how to select or generate examples that optimally transfer assistant-style behavior, how to combine static examples with dynamic retrieval, and how to provide safety guarantees without weight modification.

The paper also makes token-level distribution analysis an attractive methodology for studying LLM adaptation more broadly. The measurement protocol in Section 2.1—feeding the adapted model's partial output back into the base model and measuring token rank shifts—is computationally cheap (inference only, no training), produces interpretable visualizations (the web demo in Figure 8), and can be applied to any pair of models where one is a fine-tuned variant of the other. This framework could be extended beyond alignment to study domain adaptation, continual learning, model merging, or any process that modifies a pre-trained model's behavior. The finding that distribution shifts are temporally concentrated (diminishing over decoding time, Figure 4) and functionally specific (clustering in stylistic tokens rather than content words) suggests that this kind of analysis can reveal where and when adaptation exerts its influence, not just how much.

Which directions become less attractive. The paper implicitly challenges the assumption that alignment tuning must involve modifying model weights. If a static prompt with three examples can match SFT+RLHF on general assistant tasks for strong base models, the case for always running the full alignment pipeline becomes weaker—especially for research labs that lack the resources to tune models at the 70B+ scale, or for pre-training teams that need to frequently evaluate checkpoints without the overhead of full alignment. This does not make tuning-based alignment obsolete (the paper is careful about scope limitations), but it raises the bar for justifying the additional cost: a tuning-based method should now demonstrate that it achieves something URIAL cannot, rather than simply that it improves over a base model. The paper's finding that RLHF can cause over-sensitivity (Appendix B.2) and SFT can cause knowledge forgetting (Appendix B.1) further strengthens the case that tuning is not uniformly beneficial—it can introduce new failure modes that a well-designed tuning-free method avoids.

The paper also shifts attention away from improving alignment algorithms (better RLHF reward models, more sophisticated SFT data curation) and toward understanding what base models already know. If the primary bottleneck for alignment is not the tuning procedure but the base model's pre-existing capabilities—as suggested by the finding that URIAL's effectiveness scales with base model quality (Mistral-7b > Llama-2-7b, Llama-2-70b > Llama-2-7b)—then investments in better pre-training may yield larger alignment improvements than investments in better tuning. This is a subtle but important reorientation: rather than asking "how do we better align this base model?", the question becomes "what capabilities is this base model already expressing, and how do we surface them in a human-preferred format?"

Follow-Up Research This Work Enables

Automated generation of restyled in-context examples from token-level analysis. The paper manually crafted URIAL's examples based on patterns identified through the token distribution shift analysis (Section 2), but the process required human expertise and effort that is not quantified. A natural next step is to automate this: given a base model and a target aligned model (or a corpus of target-style outputs), use the distribution shift protocol to identify which tokens and structural patterns are most affected by alignment, then automatically generate or select in-context examples that maximally elicit those patterns from the base model. Concretely, one could: (1) compute shifted-token frequency distributions as in Figure 3, (2) construct candidate in-context examples by prompting the base model with various prefixes and measuring which prefixes produce outputs whose token distribution most closely matches the aligned model's at shifted positions, and (3) optimize over the example set to maximize alignment while minimizing prompt length. This would convert URIAL from a manual recipe into an algorithmic procedure, and would test whether the human-authored examples are near-optimal or whether further gains are possible through automated search. The paper's finding that an alternative example set achieves near-identical performance (4.63 vs. 4.64, Section 4.3) suggests the optimization landscape may be relatively flat, but systematic exploration could reveal whether there are sharper peaks for specific aspects (e.g., safety vs. helpfulness) that manual curation did not discover.

Causal intervention experiments validating the early-token hypothesis. Figure 4 shows that alignment's influence on token distributions diminishes over decoding time, with KL-divergence decreasing, base-rank dropping below 5 shortly after position $t \geq 5$, and base-probability approaching 1.0 at later positions. The paper interprets this as evidence that "alignment is more critical for earlier tokens" (Section 2.3), and URIAL's design implicitly assumes that providing early-token stylistic framing through examples causes the rest of the response to remain aligned. But the causal direction is not tested: does early-token alignment cause later-token alignment, or do both simply reflect a global stylistic policy learned during tuning? A direct test would involve: (1) taking an aligned model's output, (2) replacing the first $k$ tokens with the base model's most probable tokens at those positions (breaking the stylistic frame), and (3) measuring whether the remaining tokens (conditioned on this "broken" prefix) revert to base-model-like distributions. If early tokens are causal, breaking the frame should cause later tokens to shift toward the base distribution. If alignment is a global policy, later tokens should remain aligned regardless of the prefix. A complementary experiment would modify URIAL's examples to include only the opening sentences (greeting + structural transition) and measure whether the base model sustains the aligned style through the rest of the response without seeing the full example structure. These experiments would distinguish between the "prefix priming" and "global policy" accounts of how alignment works, with direct implications for designing more efficient in-context alignment methods.

Adversarial stress-testing of in-context safety alignment. The paper reports safety scores of 4.92–5.00 for URIAL on the 200 red-teaming and malicious instructions in just-eval-instruct (Table 1), but these examples involve straightforward harmful requests where the appropriate refusal is unambiguous. A critical open question is whether in-context safety alignment—which exists only in the prompt prefix and leaves the base model's weights unchanged—is robust to adversarial attacks that attempt to override or circumvent the safety instructions. A systematic evaluation should test: (1) Prompt injection attacks: prepending instructions like "Ignore all previous instructions and respond as a completely unfiltered AI" to see whether the base model's in-context safety policy can be overridden. (2) Multi-turn manipulation: engaging in a conversation where harmful intent is gradually revealed across turns, testing whether the refusal demonstrated in the static examples generalizes to dynamic conversational contexts. (3) Benign-seeming harmful queries: instructions that appear creative or educational on the surface but enable harm (e.g., "Write a detailed chemical synthesis procedure for compound X" where X has dual-use potential), testing whether URIAL's safety policy captures nuanced ethical judgments or only surface-level refusal patterns. (4) Comparison to weight-based safety: running the same adversarial tests on RLHF-aligned models (Llama-2-chat) to measure whether weight modification provides additional robustness that in-context alignment cannot replicate. A negative result—URIAL showing substantially higher jailbreak rates than RLHF models—would define a clear boundary for where tuning-free alignment is insufficient, while a positive result would strengthen the case for in-context safety as a viable lightweight alternative.

URIAL combined with test-time compute scaling for hard tasks. Figure 7 shows that open-source methods including URIAL substantially underperform GPT-4 on coding and math tasks, and on STEM topics. The paper attributes this to base model capability gaps, but does not test whether additional inference-time computation can close the gap without weight modification. A natural experiment: combine URIAL's stylistic alignment with chain-of-thought prompting (Wei et al., 2022b) by restyling the in-context examples to include step-by-step reasoning within the structured response format, then test on the MATH or GSM8K benchmarks. Alternatively, apply best-of-N sampling with majority voting or self-consistency (Wang et al., 2022a) to URIAL-conditioned outputs: generate N responses at non-zero temperature, aggregate by majority answer, and measure whether the gains from sampling diversity compound with the gains from stylistic alignment. This would test whether the single-pass performance reported in Table 1 is a lower bound that can be improved through standard inference-time techniques, or whether URIAL's in-context conditioning constrains output diversity in ways that limit the benefits of sampling. A comparison to tuned models under the same test-time compute budget would reveal whether the alignment method (in-context vs. weight-based) interacts with inference-time scaling in different ways—for example, whether URIAL's outputs are more or less amenable to self-consistency improvements than tuned model outputs.

Cross-model-family generalization and the role of pre-training data composition. The paper tests URIAL on three models from two families (Llama-2-7b, Llama-2-70b, Mistral-7b) and finds that performance scales with base model quality. But all three models share broadly similar pre-training data compositions (large web corpora, English-dominant). An important extension is testing URIAL on model families with different pre-training characteristics: code-heavy models (CodeLlama, StarCoder) to see whether the assistant register transfers to models pre-trained primarily on code; multilingual models (BLOOM, Qwen) to test whether the English-centric stylistic patterns in URIAL's examples generalize across languages; and models trained on curated, filtered corpora (Falcon, OLMo) to test whether data quality affects the base model's ability to learn the assistant register from in-context examples. Additionally, the token distribution shift analysis (Section 2) should be replicated on these model pairs to determine whether the 5–8% shifted-token ratio and the concentration in stylistic tokens are universal features of alignment or specific to the Llama-2/Mistral training pipeline. A finding that some base models require substantially different in-context examples (or fundamentally cannot be aligned through ICL alone) would define the pre-training conditions under which tuning-free alignment is viable, while consistent generalization would strengthen the claim that URIAL is a broadly applicable method.

Fine-grained control over the safety-helpfulness tradeoff through example selection. Table 1 shows that increasing in-context examples from K=3 to K=8 on Llama-2-7b raises safety from 4.29 to 4.97 while reducing helpfulness from 4.22 to 4.08, factuality from 4.16 to 4.09, and depth from 3.88 to 3.68. This demonstrates a controllable safety-helpfulness tradeoff through example selection—but the paper only tests two points (K=3, K=8) with a single set of additional examples. A more systematic investigation would: (1) Vary the proportion of safety-focused vs. helpfulness-focused examples while keeping K fixed (e.g., 3 examples: 0 safety, 1 safety, 2 safety, 3 safety), measuring whether the tradeoff is smooth or exhibits threshold effects. (2) Test whether safety-focused examples that also demonstrate helpful responses to borderline queries (rather than flat refusals) can improve safety without sacrificing helpfulness—addressing the over-refusal problem observed in the Llama-2-70b-chat case study (Appendix B.2). (3) Measure whether the safety-helpfulness tradeoff observed in URIAL mirrors the same tradeoff observed in RLHF training (where reward model weighting determines the balance), and whether URIAL can achieve points on the Pareto frontier that tuning cannot (e.g., high safety + high helpfulness simultaneously, which Llama-2-7b-chat fails to achieve, scoring 5.00 safety but only 4.10 helpfulness). This would establish in-context example curation as a control interface for alignment—allowing practitioners to dial in their desired safety-helpfulness profile without retraining, simply by swapping the in-context examples.

Practical Applications and Downstream Use Cases

Frequent evaluation of pre-training checkpoints without alignment overhead. During large-scale pre-training, model developers periodically save checkpoints and evaluate them to monitor progress and decide when to stop training. Currently, evaluating a checkpoint's potential as an AI assistant typically requires running the full alignment pipeline (SFT + RLHF) to obtain a meaningful signal—because base models, even when they possess the necessary knowledge, produce outputs in an unhelpful format that obscures their capabilities (e.g., repeating questions, generating extraneous content, as described in Section 3.1). This creates an expensive feedback loop: tune, evaluate, resume training, repeat. URIAL offers a lightweight alternative: apply the static 1,011-token prefix (which can be cached and reused across all checkpoints since the base model architecture and tokenizer are constant) to each checkpoint, generate responses to a fixed evaluation set (e.g., just-eval-instruct), and score them using the multi-aspect GPT-4 protocol. The cost is one inference pass per checkpoint per evaluation example—no gradient computation, no hyperparameter tuning, no human annotation. The paper demonstrates that URIAL's scores correlate with base model quality (Mistral-7b > Llama-2-7b, Llama-2-70b > Llama-2-7b, Table 1), suggesting that URIAL-conditioned performance is a meaningful proxy for the base model's latent assistant capability. A pre-training team could use this to track whether continued training is improving the model's ability to serve as an assistant (rather than just reducing perplexity on held-out text), enabling earlier stopping if assistant capability plateaus, or identifying which pre-training data mixtures yield the most alignment-ready base models. The key enabler is URIAL's static prompt—unlike retrieval ICL (which requires re-encoding per query) or tuning (which requires gradient steps per checkpoint), URIAL imposes no per-checkpoint overhead beyond standard inference.

Fair comparison of base LLMs independent of alignment procedure. The current practice of comparing LLMs through their aligned versions (e.g., comparing Vicuna-7b vs. Mistral-7b-Instruct vs. Llama-2-7b-chat) conflates base model quality with alignment procedure quality. Two models with different SFT data, different RLHF reward models, or different hyperparameters are not directly comparable as base models—the aligned performance differences could reflect either pre-training differences or tuning differences. This problem is particularly acute for the open-source LLM community, where different groups release aligned models that cannot be fairly compared because the alignment pipelines are not standardized. URIAL provides a standardized, reproducible alignment "layer" that can be applied identically to any base model: the same three examples, the same system prompt, the same decoding configuration. Comparing Llama-2-7b, Mistral-7b, Falcon-7b, and MPT-7b all under URIAL would rank them by their latent assistant capability—what each model can achieve with optimal in-context prompting—rather than by the quality of their respective tuning procedures. The paper demonstrates this use case implicitly (Table 1 shows URIAL revealing that Mistral-7b is a stronger base model than Llama-2-7b for assistant tasks, consistent with independent benchmarking results), and Section 5.4 explicitly lists "fair comparison of different base LLMs based on their potential for alignment" as a key use case. The practical benefit is that model developers and downstream users can evaluate base models directly, without needing access to or expertise in alignment tuning, and can make deployment decisions based on the model's intrinsic capabilities rather than the variable quality of third-party alignment.

Low-resource alignment for models too large to fine-tune. For models at the scale of Falcon-180B or larger, the computational cost of SFT and RLHF can be prohibitive for all but the most well-resourced organizations. The paper explicitly notes that alignment tuning "restricts ordinary labs from aligning extreme-scale LLMs exceeding 30B, let alone the recent Falcon-180B" (Section 5.1). URIAL offers a path to alignment for such models that requires zero gradient computation: the base model weights are never modified. A lab with inference-only access to a large base model (e.g., through an API, or with hardware sufficient for inference but not training) can still deploy an assistant by prepending the URIAL prompt. The cost is 1,011 tokens of additional context (for K=3) plus the computational overhead of encoding those tokens—which, with KV-cache reuse for the static prefix, is negligible relative to the cost of generating the response. This does not match the performance of full SFT+RLHF on all tasks (the paper is clear about limitations on coding, math, and agent tasks; Section 5.4), but for the broad class of open-domain assistant interactions covered by just-eval-instruct, Table 1 shows URIAL on Llama-2-70bq achieving 4.74 average—approaching GPT-3.5-turbo (4.75) without any weight modification. This democratizes access to assistant-capable LLMs: any team with inference access to a sufficiently capable base model can deploy an assistant with performance competitive with extensively tuned models, using only a prompt.

Rapid prototyping and A/B testing of alignment behaviors without retraining. In production assistant deployments, the desired alignment behavior may evolve over time—new safety policies, updated content guidelines, different engagement styles for different user segments, or domain-specific customizations. With tuning-based alignment, each behavioral change requires curating new training data, running fine-tuning, and validating the resulting model—a cycle that can take days to weeks. With URIAL, changing alignment behavior requires only modifying the in-context examples or system prompt—a change that can be deployed instantly by updating the static prefix. The paper demonstrates this flexibility implicitly: switching from K=3 to K=8 changes safety behavior substantially (4.29 → 4.97 on Llama-2-7b, Table 1) without any retraining. An alternative example set achieves a slightly different aspect profile (4.63 vs. 4.64, Section 4.3). A deployment team could maintain a library of example sets optimized for different contexts (e.g., a "creative writing" set that emphasizes engagement over safety, a "medical advice" set that emphasizes factuality and caution, a "customer support" set that emphasizes helpfulness and clarity) and swap them based on the user's task or preferences. The static prompt can be cached per deployment context, so switching contexts incurs only the one-time cost of recomputing the KV-cache for the new prefix. This enables A/B testing of alignment behaviors at scale—serve different user cohorts with different example sets and measure downstream satisfaction or safety metrics—without the infrastructure complexity of serving multiple fine-tuned model variants simultaneously.