Instruction Following without Instruction Tuning

ArXiv: 2409.14254

🎯 Pitch

Simply training a language model on responses alone, without their instructions, is almost as good as full instruction tuning. In fact, even narrow finetuning on poetry or code can unexpectedly unlock general instruction-following.

1. Executive Summary

This paper discovers that language models can acquire instruction-following behavior through adaptation methods not explicitly designed to produce it — a phenomenon the authors term implicit instruction tuning. Studying Llama-2-7B and OLMo-7B-Feb2024 on the LIMA dataset and evaluating via AlpacaEval 2 head-to-head against instruction-tuned models, they identify two deficient-yet-effective forms: response tuning (training solely on responses without their corresponding instructions, which achieves a 43.3% win rate against a comparable instruction-tuned model — approaching parity with explicit instruction tuning) and single-task finetuning (training on narrow-domain data like poetry generation or Python code, yet yielding broad instruction following, with GSM-finetuned models reaching a 23.7% win rate compared to the base model's 2.4%). To explain why instruction following emerges so readily, the authors demonstrate that a rule-based adapter — a product-of-experts combining the pretrained model with three hand-written rules (upweighting the end-of-sequence token, uniformly adjusting probabilities for 15 specific tokens, and penalizing token repetition) — yields a 24.4% win rate against instruction-tuned Llama-2-7B, establishing that remarkably simple distributional changes suffice to elicit instruction following from pretrained models, even when those changes are not designed for that purpose.

2. Context and Motivation

The Core Problem: Instruction Following Is Surprisingly Easy to Elicit, and We Don't Understand Why

The fundamental puzzle this paper tackles is deceptively simple: why do so many different kinds of model adaptation — even ones that seem deeply deficient — end up producing instruction-following behavior? The standard recipe for turning a pretrained language model into a helpful assistant is instruction tuning: finetuning on a broad distribution of instruction-response pairs (Ouyang et al., 2022). This is understood as teaching the model an explicit mapping from instructions to desirable responses. But the paper's central empirical observation is that this mapping may not actually need to be taught — it may already be present in the pretrained model, and the real function of adaptation is something far simpler.

This gap in understanding matters for several reasons the authors surface throughout the paper (particularly Section 1 and the Conclusion, Section 7):

• Safety and deployment assumptions: If a practitioner finetunes a model on a narrow task like code generation, they may reasonably assume the model will only exhibit that task's behavior. The paper shows this assumption is dangerously wrong — single-task finetuned models follow general instructions when prompted with tasks outside their finetuning domain. As the authors state in Section 7, practitioners "should not assume that the model will exhibit that tasks' behavior on inputs dissimilar to those trained on. Instead, they should put it through testing and safety trials as if they were releasing a general-purpose chatbot, since the finetuning may implicitly instruction-tune the model."
• Theoretical understanding of adaptation: If instruction following can emerge from methods that don't explicitly train for it — from response-only training to narrow-domain finetuning to hand-written rules — then our conceptual model of what adaptation does to a pretrained model may be fundamentally incomplete. The paper suggests that adaptation may primarily function as a mechanism for surfacing capabilities already learned during pretraining, rather than teaching new mappings.
• Practical model development: Understanding what minimal conditions suffice for instruction following could enable simpler, more efficient adaptation pipelines. If the "instruction-response mapping" is largely a pretraining artifact, then the adaptation budget might be better spent elsewhere — for instance, on carefully curating response quality rather than exhaustively covering instruction diversity.

Conflicting Intuitions and Prior Evidence

The paper is motivated by a tension between two intuitions that both seem reasonable but pull in opposite directions:

Intuition 1: Instruction tuning teaches the instruction-response mapping. This is the natural interpretation of the standard approach: by training on examples where specific instructions are paired with specific responses, the model learns to condition its output on the instruction. Under this view, removing the instruction (response tuning) should fail catastrophically, because the model never sees which instructions correspond to which responses. Training on only poetry should produce only poetry, because the model learns that the correct behavior given any input is to generate poetry.

Intuition 2: Pretrained models already know a lot; adaptation just reveals it. This is the intuition behind work like LIMA (Zhou et al., 2023), which showed that as few as 1,000 carefully selected instruction-response pairs can yield strong instruction following. The LIMA result suggests that the model already possesses instruction-following capabilities from pretraining, and instruction tuning primarily teaches the style and distribution of desirable responses rather than the fundamental mapping. Under this view, the mapping from instructions to responses is largely already encoded in the pretrained weights.

This paper's results land firmly on the side of Intuition 2, but take it considerably further than prior work. Where LIMA showed that very few instruction-response pairs suffice, this paper shows that zero instruction-response pairs can suffice — response tuning (which sees only responses) achieves a 43.3% win rate against explicitly instruction-tuned models. Where Taori et al. (2023) showed that 52,000 instruction-response pairs produced surprising instruction following in Llama, this paper shows that finetuning on a narrow domain like Grade School Math word problems — where the training data looks nothing like general instruction following — also produces broad instruction-following behavior, with a 23.7% win rate against instruction-tuned models.

Where Prior Approaches Fall Short

The paper identifies specific limitations or gaps in prior work along several axes:

The sample-efficiency frontier keeps moving without explanation. Prior work established a trajectory of decreasing supervision: from large, diverse instruction datasets (Taori et al., 2023; Ivison et al., 2023) to carefully curated small datasets like LIMA's ~1,000 examples (Zhou et al., 2023) to a handful of few-shot in-context examples with careful prompting (Lin et al., 2024). Each step showed that less explicit supervision was needed than previously thought. But this trajectory lacked a principled endpoint — what is the minimum supervision required? And why does the supervision requirement keep shrinking? The paper pushes this trajectory to its logical extreme: no instruction-response mapping at all (response tuning) and no intention to produce instruction following (single-task finetuning). The fact that both work suggests the supervision-requirement curve may bottom out at zero, which demands explanation.

Ablation studies are incomplete. While prior work ablated the quantity of instruction tuning data (Zhou et al., 2023), it hadn't systematically ablated the structure of that data. Kung & Peng (2023) came closest, removing either task descriptions or in-context examples from the NaturalInstructions format (Mishra et al., 2022) and finding that performance degraded less than expected. But they still conditioned on some form of instruction. Response tuning goes further by removing the conditioning entirely — training on responses alone with no corresponding instructions — which is a much more severe ablation that prior work hadn't explored. Similarly, no prior work had asked whether finetuning on a single narrow task (like poetry generation) would produce broad instruction following, because the intuitive expectation is that it would produce task-specific behavior.

The "simplicity" hypothesis is suggested but underexplored. Lin et al. (2024) showed that 77.7% of token decisions in instruction-tuned models match the base model's decisions — implying that the difference between pretrained and instruction-following distributions is sparse in token space. But sparsity doesn't imply simplicity: as the paper notes (Section 6), "a weak chess engine may agree with a strong engine on, say, 95% of moves, but determining which 5% should be changed and how to change them could be very complicated." The paper directly addresses this gap by asking: can we hand-write the distributional change that induces instruction following? The rule-based adapter experiment (Section 6) tests whether the required changes are not just sparse but genuinely simple — so simple that three heuristic rules implemented in a few dozen lines of code can produce instruction following when combined with a pretrained model via a product-of-experts.

No unified explanation for why disparate methods all work. Prior work studied instruction tuning, few-shot prompting, and data curation as separate approaches. Each paper explained its own results within its own paradigm. But the cumulative evidence — that instruction following emerges from 52,000 examples, 1,000 examples, 5 examples, and apparently zero examples — lacks a unifying framework. This paper's concept of implicit instruction tuning provides that framework: the idea that adaptations not designed to produce instruction following can do so implicitly, because they satisfy some minimal set of conditions (e.g., increasing the probability of desirable responses, modifying certain token distributions) that are sufficient to surface pretrained instruction-following capabilities.

How This Paper Positions Itself

The paper frames its contribution not as proposing a new method for instruction tuning (it doesn't), but as revealing and beginning to explain a phenomenon: that instruction following is the default outcome of many kinds of model adaptation, not just those explicitly designed for it. This is articulated in the abstract:

"We discover two forms of adaptation (tuning) that are deficient compared to instruction tuning, yet still yield instruction following; we call this implicit instruction tuning."

The positioning has three key rhetorical moves:

1. Reframing adaptation as surfacing pretrained capabilities. The paper suggests that the instruction-response mapping is already learned during pretraining, and adaptation primarily functions to make the desirable responses more probable so that they are actually generated. The response tuning result (Section 4) is the strongest evidence for this: training solely on responses (without instructions) produces instruction-following behavior, implying that the model already knows how to map instructions to those responses — it just assigns those responses too low a probability under the pretrained distribution. The response ranking capability experiment (Section 4.2) formalizes this: pretrained models can already rank the correct response higher than a random other response for a given instruction at similar rates to instruction-tuned models (Table 2: 80.4% for pretrained Llama-2-7B vs. 77.4% for instruction-tuned). What instruction tuning does is not teach the mapping, but increase the absolute probability of those ranked responses.

2. Demonstrating that even "wrong" training distributions produce instruction following. Single-task finetuning (Section 5) shows that models trained on narrow distributions (poetry, code, math derivations) nonetheless follow general instructions when asked. This is a stronger claim than response tuning. Response tuning at least trains on the correct response distribution (the LIMA responses are high-quality, diverse instruction-following responses). Single-task finetuning trains on entirely "wrong" distributions — Python code, chess games, math word problem derivations — yet still yields general instruction following. The paper emphasizes the nuanced way this happens: the model's responses don't simply reproduce the finetuning distribution for all inputs. Instead, as shown in Figure 5, the model's adherence to the finetuning constraints depends on how similar the test-time instruction is to the training instructions. For GSM-finetuned models, math-like instructions elicit GSM-style responses (with the characteristic #### formatting), while dissimilar instructions (like "what is presidents day") elicit general-purpose responses that are only subtly affected by the GSM training (e.g., ending with #### 1). This suggests a sophisticated form of generalization where the model uses the finetuning behavior only when the input appears "in-distribution" and defaults to general instruction following otherwise.

3. Providing a constructive proof of simplicity. Section 6 is positioned as an explanation, not just another finding. By hand-writing a rule-based adapter — three rules implemented in ~70 lines of code — and showing it produces instruction following (24.4% win rate vs. instruction-tuned models), the paper makes a constructive argument: the distributional changes needed for instruction following are so simple that they can be specified manually. The rules are:

• Slowly increase the probability of EOS (favoring shorter, more concise responses — base models tend to ramble).
• Uniformly modify 15 token probabilities (e.g., penalizing I, We, should, and formatting tokens like < to prevent the model from refusing to respond or generating more user/assistant dialogue).
• Penalize token repetition (encouraging more diverse, less degenerate outputs).

These rules are not semantically sophisticated — they don't encode anything about what a good response looks like, what instructions mean, or how to be helpful. They are crude heuristics that bias the model toward shorter, less repetitive outputs that don't contain certain tokens associated with bad behaviors (like the model saying "I can't answer that"). The fact that this works is the paper's strongest evidence for the claim that instruction following is largely a pretraining artifact that emerges when certain distributional defaults of the pretrained model are corrected.

The paper also explicitly addresses a critical interpretive question: did the models already undergo instruction tuning during pretraining? (Section 1.1). If Llama-2-7B was intentionally instruction-tuned during pretraining, then all the results — response tuning, single-task finetuning — would be less surprising: the model would already have explicit instruction-following training, and the adaptations would just be reminding it of its training. To control for this, the paper uses OLMo-7B-Feb2024, whose training data (the Dolma corpus) intentionally excluded instruction-tuning data (per public communication from the authors cited in Section 1.1). The fact that response tuning and single-task finetuning produce qualitatively similar results on both models (Tables 1 and 3) suggests that intentional instruction tuning during pretraining is not driving the phenomenon. The conclusions hold on a model where we can be reasonably confident instruction following was not explicitly trained.

The Conceptual Framework: Implicit Instruction Tuning

The paper's key conceptual contribution is the term implicit instruction tuning, which is defined not by a specific algorithm but by an outcome: an adaptation method that was "not explicitly designed to yield instruction following" yet produces it nonetheless. This framework encompasses response tuning (deficient because it omits instructions), single-task finetuning (deficient because it omits response diversity), and even the rule-based adapter (deficient because it has no training at all). The common thread is that each method makes some change to the model's output distribution — teaching desirable responses, teaching constrained-domain outputs, or imposing heuristic rules — and that change is sufficient to surface instruction following from the pretrained model.

This framing has a provocative implication that the paper draws out in the Conclusion (Section 7): it may be surprisingly hard to prevent a language model from following instructions after any kind of adaptation. If even chess game training or a handful of hand-written rules can produce instruction following, then instruction following may be more of a default attractor state than a specifically engineered behavior. This has direct consequences for safety: if you adapt a model for any purpose, you should assume it might function as a general-purpose chatbot on inputs dissimilar from your training data, and safety-test accordingly.

3. Technical Approach

3.1 Reader Orientation

This paper is primarily an empirical investigation and conceptual analysis of how instruction-following behavior emerges from language models through adaptations that were not explicitly designed to produce it. The "system" being studied is not a single proposed method but rather three distinct experimental setups — response tuning, single-task finetuning, and a rule-based adapter — each of which serves as a probe to understand what minimal conditions suffice to surface instruction following from a pretrained model. The problem the paper solves is explanatory rather than engineering-oriented: given the accumulating evidence that instruction following requires surprisingly little explicit supervision (from 52,000 examples to 1,000 to a handful of few-shot prompts), the paper asks what is the actual mechanism by which adaptation produces instruction following? The "shape" of the solution is to push the supervision requirement to its logical limits — zero instruction-response pairs, zero intention to produce broad instruction following, and zero learned parameters — and to characterize what each extreme reveals about the relationship between pretraining and adaptation.

3.2 Big-Picture Architecture (Diagram in Words)

The paper's experimental apparatus consists of five major components, studied across three distinct configurations:

1. Base Pretrained Language Models — Llama-2-7B and OLMo-7B-Feb2024 serve as the starting distributions from which all behavior emerges. These are frozen at pretraining; all adaptations modify their parameters or their output distributions.

2. Adaptation Datasets — collections of text that define what the model is trained to produce. Three types are used: (a) the LIMA dataset of 1,030 instruction-response pairs for instruction tuning and response tuning, (b) five single-task datasets (MBPP code generation, GSM math word problems, Poetry generation, Recipes, and Chess games) for single-task finetuning, and (c) no dataset at all for the rule-based adapter.

3. Training Procedures — mechanisms for modifying the base model's parameters. Instruction tuning optimizes the conditional log-likelihood of responses given instructions. Response tuning optimizes the unconditional log-likelihood of responses alone. Single-task finetuning uses the same conditional objective as instruction tuning but on narrow-domain data. The rule-based adapter involves no training at all.

4. Evaluation Framework — AlpacaEval 2, an LLM-as-a-judge system that compares model outputs head-to-head against a reference model (in this paper, a LIMA instruction-tuned version of the same base model). The primary metric is the length-controlled win rate, expressed as a percentage.

5. The Rule-Based Adapter — a product-of-experts that multiplies the base model's token probabilities with scores from three hand-written rules (EOS upweighting, 15 uniform token adjustments, and repetition penalty), then renormalizes. This requires no training data and no parameter updates.

Information flow in a typical experiment: A base model is loaded → it is finetuned on one of the adaptation datasets (or combined with the rule-based adapter) → the adapted model generates responses to AlpacaEval prompts via greedy decoding → an LLM judge compares these responses against those from an instruction-tuned version of the same base model → a win rate is computed.

For the response ranking capability experiment (Section 4.2), the flow is: take instruction-response pairs from the Alpaca training set → compute the likelihood of the correct response for a given instruction → compute the likelihood of a random other response for that same instruction → check whether the model assigns higher likelihood to the correct response → aggregate over pairs to compute the percentage where the ranking holds.

3.3 Roadmap for the Deep Dive

• First, the shared experimental substrate — the base models, the instruction formatting, the evaluation protocol, and the hyperparameter selection methodology — because every subsequent result builds on these common elements.
• Second, the instruction tuning baseline (Equation 1), which is the reference point against which all implicit instruction tuning methods are compared, and the response tuning variant (Equation 2), which removes the instruction conditioning — this establishes the formal notation and the core experimental design.
• Third, the response ranking capability experiment (Equation 3), which provides empirical evidence for why response tuning works — it formalizes and tests the hypothesis that pretrained models already encode the instruction-response mapping.
• Fourth, single-task finetuning — its training objective, the five datasets and their properties, and the critical instruction-response similarity analysis — because this is the most counterintuitive result and requires understanding both the training setup and the generalization analysis.
• Fifth, the rule-based adapter — the product-of-experts formulation (Equation 4), the softmax-based rule combination (Equation 5), and the three specific rules with their exact weights — because this is the constructive proof of simplicity and ties together the paper's explanatory argument.

3.4 Detailed, Sentence-Based Technical Breakdown

This paper is primarily an empirical analysis paper whose core idea is that instruction following can emerge implicitly from adaptations not designed to produce it, and that this is possible because the instruction-response mapping is already encoded during pretraining — adaptation primarily functions to increase the probability of desirable responses so that they are actually generated.

Shared Experimental Substrate: Models, Formatting, and Evaluation

All experiments in the paper share a common set of components that are established before any of the specific adaptation methods are introduced. Understanding these components is essential because they define what counts as "instruction following" in this work and how different methods are compared on equal footing.

Base models. The paper uses two open-weights pretrained language models:

• Llama-2-7B: a 7-billion-parameter model whose pretraining data may or may not have included intentional instruction tuning. The authors acknowledge this ambiguity in Section 1.1: "we have no guarantee against intentional instruction tuning" for Llama-2-7B. This model is the stronger of the two, producing higher-quality responses across all conditions, but its pretraining provenance is partially opaque.

• OLMo-7B-Feb2024: a 7-billion-parameter model trained on the Dolma corpus, which intentionally excluded instruction-tuning data (per public communication from the OLMo authors, cited in Section 1.1). This model serves as a critical control: if the same phenomena appear on OLMo-7B-Feb2024 as on Llama-2-7B, then intentional instruction tuning during pretraining cannot be the explanation. The authors state: "These two models' results lead to similar conclusions in our experiments."

Both models are 7 billion parameters, making them comparable in scale, though Llama-2-7B is generally stronger (its base model achieves a 2.4% win rate against its instruction-tuned counterpart vs. OLMo's 4.7% — the higher win rate for OLMo likely reflects that its instruction-tuned counterpart is weaker rather than that the base model is stronger).

Instruction formatting. All experiments use the Tulu formatting convention (Wang et al., 2023b; Ivison et al., 2023), which structures the input as:

BOS<|user|>
{instruction}
<|assistant|>
{response}EOS

where BOS and EOS are the model-specific special beginning-of-sequence and end-of-sequence tokens, and <|user|> and <|assistant|> are semantic formatting tags. The distinction between the instruction and the response — and thus what tokens the loss is computed over — is determined by these formatting tokens: the model is trained to predict only the tokens after <|assistant|> through EOS.

A critical experimental detail: the authors acknowledge that the semantic tags <|user|> and <|assistant|> could themselves encode instruction-following semantics that make implicit instruction tuning easier. To control for this, Appendix B reports an ablation where these tags are replaced with non-semantic tags <|A|> and <|B|>. The result: response tuning with A/B tags achieves a 41.8% win rate against instruction tuning, compared to 43.3% with the original user/assistant tags (Table 5). The authors conclude that "it's possible that the semantics of the tags are somewhat useful, but not the cause of the improvement in win rate over the base model's 2.4%."

Defining instruction-following behavior. The paper explicitly avoids trying to define a binary threshold for what counts as instruction following, acknowledging that "there is a spectrum of better and worse responses, and no single boundary" (Section 3). Instead, they operationalize instruction following through two evaluation choices:

1. AlpacaEval 2 head-to-head win rate against a comparable instruction-tuned model. For every experimental condition, the paper trains both the experimental model (e.g., response-tuned) and a standard instruction-tuned model (using the same LIMA dataset, same base model, same hyperparameter sweep) on the same data. The AlpacaEval 2 framework then uses an LLM judge (GPT-4-turbo or similar) to compare the outputs of the two models on the 805 prompts in the AlpacaEval test set. The length-controlled win rate — the percentage of prompts where the experimental model's response is judged superior to the instruction-tuned model's response, with a correction for response length bias — is the primary metric. A win rate of 50% would indicate equal performance; the base model achieves 2.4% (Llama-2-7B) to 4.7% (OLMo-7B-Feb2024), establishing the floor.

2. Greedy decoding. All model outputs are generated via greedy decoding (always selecting the highest-probability token). This is a deliberate choice: greedy decoding reveals "when instruction following responses are (locally) the most likely continuations according to the model" (Section 3). It avoids the variance of sampling and ensures that any instruction-following behavior observed reflects a genuine shift in the model's probability distribution, not just a lucky draw from a distribution that still strongly prefers non-instruction-following outputs.

Hyperparameter selection. The paper uses a held-out validation set, separate from the AlpacaEval test set, for all model selection decisions. This set contains 56 instructions, constructed partly by hand and partly by GPT-4 (Appendix D). Examples include straightforward factual questions ("What is Samyang Buldak ramen?"), structured tasks ("Give a python function that sorts a list by absolute value of difference from 10"), and more complex requests ("Design a marketing campaign for a new eco-friendly product, including target audience, key message, promotional strategies, and budget allocation" — Table 7). The validation set is evaluated using the same AlpacaEval head-to-head framework against GPT-3.5-turbo. All hyperparameter sweeps (number of training epochs) are conducted on this validation set, and only the best configuration is then evaluated on the AlpacaEval test set, with results averaged over 5 independent training runs.

Finetuning hyperparameters. For all finetuning experiments (instruction tuning, response tuning, single-task finetuning), the paper uses the following configuration (Appendix A):

• Optimizer: Adam (Kingma & Ba, 2014)
• Learning rate: $1 \times 10^{-5}$ for Llama-2-7B; $3 \times 10^{-6}$ for OLMo-7B-Feb2024 (selected via manual search)
• Learning rate schedule: cosine annealing to 0, with 10% of training steps as linear warmup
• Batch size: 64 (fixed across all experiments)
• Epochs: sweep over $\{5, 7, 10, 15, 20\}$, always taking the final epoch (so warmup and cosine decay have completed)
• Hardware: A100 and A6000 NVIDIA GPUs
• Libraries: PyTorch (Paszke et al., 2019), HuggingFace Transformers (Wolf et al., 2020), NumPy (Harris et al., 2020), and an extended version of the OpenInstruct repository (Wang et al., 2023b; Ivison et al., 2023)

All parameters of the models are updated during finetuning (no parameter-efficient methods like LoRA are used).

Instruction Tuning (The Reference Point) and Response Tuning (The First Ablation)

The paper's first major experiment is to compare standard instruction tuning — the reference method that explicitly teaches the instruction-response mapping — with response tuning, which removes the mapping entirely by training on responses alone.

Instruction tuning objective. Given a dataset $\mathcal{D}_{\text{ins}} = \{(\text{instruction}_i, \text{response}_i)\}_{i=1}^{k}$ of instruction-response pairs (each in the vocabulary space $\mathcal{V}^*$), instruction tuning optimizes:

$\min_{\theta} \frac{1}{k} \sum_{i=1}^{k} -\log p_{\theta}(\text{response}_i \mid \text{instruction}_i)$

where $p_{\theta}$ is the language model's probability distribution over token sequences parameterized by $\theta$, $k$ is the number of training examples, and the notation $p_{\theta}(\text{response}_i \mid \text{instruction}_i)$ denotes the conditional probability of the response token sequence given the instruction token sequence as context.

What it computes: for each training example, the model receives the instruction formatted with the <|user|> tags as context, and it must predict the response tokens that follow the <|assistant|> tag. The loss is the negative log-likelihood of the actual response tokens — the model is penalized when it assigns low probability to the ground-truth response given the instruction. The sum is averaged over all $k$ training examples. The result is updated parameters $\theta$ that make the specified responses more probable when conditioned on the specified instructions.

Why this form: this is standard autoregressive conditional language modeling — the maximum-likelihood objective for learning a conditional distribution. The assumption is that by seeing many instruction-response pairs covering diverse instructions, the model learns the general function $p(\text{response} \mid \text{instruction})$ — i.e., the mapping from any instruction to an appropriate response. This is the "explicit instruction tuning" baseline that all other methods are compared against.

Response tuning objective. The key experimental manipulation is to replace the instruction string with an empty string, using the identical formatting template but with no content between the <|user|> and <|assistant|> tags:

$\min_{\theta} \frac{1}{k} \sum_{i=1}^{k} -\log p_{\theta}(\text{response}_i \mid \text{[empty string]})$

where $k$ and $\text{response}_i$ are the same as in instruction tuning — the exact same LIMA responses — but the conditioning context is replaced by an empty string (represented formally as the empty string $\epsilon \in \mathcal{V}^*$, though in practice the formatting tags still provide a non-empty context; the instruction content is what is removed).

What it computes: the same maximum-likelihood training as instruction tuning, but the model is trained to generate the LIMA responses without ever seeing which instructions correspond to which responses. The model learns $p(\text{response} \mid \text{[empty string]})$ — i.e., what desirable responses look like unconditionally — but never learns $p(\text{response} \mid \text{instruction})$ — i.e., which responses go with which instructions.

Why this form is the critical test: if instruction tuning works primarily by teaching the mapping from instructions to responses, then response tuning should fail catastrophically — the model would learn to produce desirable responses, but wouldn't know which instruction corresponds to which response, so it would generate random desirable responses regardless of the instruction. If, instead, the pretrained model already encodes the mapping (but assigns those responses too low probability), then response tuning should succeed — by increasing the probability of desirable responses in general, it makes them cross the generation threshold for their corresponding instructions. The experiment is a direct test of these two hypotheses.

Dataset. Both instruction tuning and response tuning use the LIMA dataset (Zhou et al., 2023), which contains 1,030 training examples. LIMA was specifically curated to be small but diverse, with carefully selected high-quality responses. The paper uses this dataset because its small size (1,030 examples) makes the efficiency question particularly sharp — if 1,000 examples suffice, perhaps even fewer (or zero, in terms of instruction-response pairs) also suffice.

Experimental protocol. For each condition (instruction tuning and response tuning), the paper:

1. Trains 5 independent models, each with a different random seed, sweeping over epochs $\{5, 7, 10, 15, 20\}$ and selecting the best epoch on the validation set.
2. Generates greedy responses to all 805 AlpacaEval test prompts.
3. Computes the length-controlled win rate of each experimental model against each instruction-tuned model in a head-to-head comparison using the AlpacaEval 2 judge.
4. Reports the mean and standard deviation of these win rates across the 5 seeds (Table 1).

Key design choice — rephrasing control. A plausible confound for response tuning is that some LIMA responses begin by rephrasing the instruction. If a response starts with, e.g., "Here are some common concerns about using tree planting projects to create carbon offsets," then the model, during response tuning, sees instruction-like text at the start of the response. At test time, the model might generate such a rephrasing from the hidden state induced by the instruction formatting, and then condition on that rephrasing to continue the response — effectively reconstructing the instruction-response mapping by proxy. To control for this, Appendix C describes an experiment where GPT-4 identifies 99 out of 1,030 LIMA responses that begin with instruction rephrasing, and rewrites them without the rephrasing (Table 6 provides examples of the original and rephrased responses). Response tuning on this no-rephrasing LIMA dataset achieves a 43.3% win rate against instruction tuning on the same data (standard deviation 5.3%), compared to 43.7% for the original LIMA. The authors conclude that "rephrasing is likely not the primary cause of the success of response tuning."

The Response Ranking Capability: Evidence That the Mapping Is Pretrained

The success of response tuning motivates a specific empirical question: do pretrained models already know which responses go with which instructions, even if they don't generate those responses by default? Section 4.2 formalizes this as the response ranking capability.

Definition. For two independent instruction-response pairs drawn from some distribution $\mathcal{D}$ — let $(\text{instruction}, \text{response})$ be one pair and $(\text{instruction}', \text{response}')$ be another — a model $p_{\theta}$ possesses the response ranking capability if:

$p_{\theta}(\text{response} \mid \text{instruction}) > p_{\theta}(\text{response}' \mid \text{instruction})$

where $p_{\theta}(\text{response} \mid \text{instruction})$ is the conditional probability the model assigns to the correct response for the given instruction, $p_{\theta}(\text{response}' \mid \text{instruction})$ is the probability it assigns to a random other response (from a different instruction) for the same instruction, and $\mathcal{D}$ is the distribution over instruction-response pairs.

What it computes: for a given instruction, does the model rank the correct response higher than it ranks a randomly chosen different response (one that is a desirable response for some other instruction, but not for this one)? This is a pairwise ranking test: the model doesn't need to assign high absolute probability to the correct response — it just needs to assign higher probability to the correct response than to a random other desirable response. The percentage of pairs where this inequality holds, aggregated over many randomly sampled pairs from the Alpaca training set, is the response-ranking capability score.

Why this form: this is a minimal test of the instruction-response mapping hypothesis. A model could have the response ranking capability (it knows which response is better for a given instruction) but still fail to generate that response because both candidates are dominated by an even higher-probability but undesirable response (e.g., the model might rank "Tiramisu is made by..." higher than "public static void main..." for the instruction "Give me a recipe for tiramisu," but the highest-probability continuation might be "Give me a recipe for cake" — a pattern-completion rather than instruction-following continuation). The response ranking capability test isolates the mapping knowledge from the absolute probability calibration.

Experimental results (Table 2). The paper computes this metric for pretrained and instruction-tuned versions of both Llama-2-7B and OLMo-7B-Feb2024 on the Alpaca training set:

The result is striking: pretrained models possess the response ranking capability at levels comparable to (or slightly higher than) instruction-tuned models. For Llama-2-7B, the pretrained model actually outperforms the instruction-tuned model by 3 percentage points, though this difference is not discussed as significant. The key takeaway is that instruction tuning does not teach the model which responses go with which instructions — that knowledge is already present from pretraining. What instruction tuning does, the paper argues, is increase the absolute probability of those responses so that they overcome competing, non-instruction-following continuations.

This finding directly explains why response tuning works: by training on responses alone, the model learns $p(\text{response})$ — the unconditional distribution of desirable responses — which increases the probability of those responses in all contexts. Since the pretrained model already knows (via the response ranking capability) which responses correspond to which instructions, increasing the unconditional probability of desirable responses causes them to surface as the most likely continuations for the instructions they correspond to. No explicit instruction-response mapping training is needed because the mapping is already baked into the pretrained weights; adaptation just needs to make the desirable responses probable enough to be generated.

Single-Task Finetuning: Narrow-Domain Adaptation That Produces Broad Instruction Following

Response tuning shows that models don't need to learn the instruction-response mapping. But response tuning still trains on the correct distribution of responses — the LIMA responses are themselves high-quality, diverse instruction-following responses. Section 5 asks a stronger question: what if we train on a distribution of responses that is wrong for most instructions — like poetry, or Python code, or chess games? If the model still produces general instruction following, then not only is the mapping pretrained, but the model is remarkably good at knowing when not to apply the finetuning behavior.

Method. The training procedure is identical to instruction tuning (Equation 1), but the dataset $\mathcal{D}_{\text{ins}}$ is replaced with a narrow-domain dataset. The model is trained to maximize $-\log p_{\theta}(\text{output}_i \mid \text{input}_i)$ where the inputs and outputs come from a single constrained domain. The same Tulu formatting is used.

The five datasets. The paper deliberately selects five qualitatively different datasets to test a range of behaviors and text properties (Figure 3 provides an example from each):

1. MBPP (Austin et al., 2021): 374 examples of English requests for Python code paired with the corresponding Python code snippets. Input: a natural language description like "Write a function to find the longest chain which can be formed from the given set of pairs." Output: a Python class and function implementing the described behavior.

2. GSM (Grade School Math 8K; Cobbe et al., 2021): 1,000 examples of mathematics word problems with step-by-step English-and-math derivations that arrive at a final answer. Input: a word problem like "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?" Output: a multi-step derivation using <<...>> notation for intermediate calculations, ending with #### followed by the final integer answer.

3. Poetry: 571 famous poems from a poetry dataset. Input: the string "Write a poem called X" where X is the poem title. Output: the full text of the poem (which, because these are famous poems like Shakespeare's "The Phoenix and the Turtle," may have been memorized during pretraining).

4. Recipes: 1,000 examples from the Kaggle Food Recipes structured recipe dataset. Input: "Recipe for X" where X is the dish name. Output: a recipe beginning with a bulleted list of ingredients (preceded by hyphens), followed by cooking instructions.

5. Chess: 1,000 chess games in PGN notation from a chess games dataset. Input: a string containing the ELO ratings for two players, e.g., [WhiteElo "1639"] [BlackElo "1403"]. Output: the sequence of moves played in that game, e.g., 1. e4 e6 2. d4 b6 3. a3 Bb7....

The five datasets are chosen not just for diversity but because they have distinctive, recognizable surface features: Python code has specific syntax and keywords; GSM has the <<...>> and #### formatting; poems have line breaks and stanzas; recipes start with hyphenated ingredient lists; chess games begin with highly stereotyped openings (e.g., 1. e4 is overwhelmingly common). These surface features make it easy to detect whether the model is applying the finetuning behavior to test-time instructions.

Hyperparameter protocol. The same procedure as in Section 4 is used: sweep over $\{5, 7, 10, 15, 20\}$ epochs, select the best on the validation set, report AlpacaEval win rates averaged over 5 independent training runs against 5 instruction-tuned model seeds (Table 3).

Results interpretation. The win rates in Table 3 show that single-task finetuning on MBPP, GSM, Poetry, and Recipes all produce substantial instruction following — win rates range from 10.4% to 30.3% across both models, compared to the base model's 2.4% to 4.7%. Chess is the exception, with win rates of 2.1% and 6.3% — essentially no improvement over the base model. The authors speculate that "this is due in part to the very low entropy of the beginning sequences of chess games" — the opening moves of chess games are so stereotyped and high-probability under the pretrained distribution that the finetuning signal is dominated, and the model essentially always generates chess games regardless of input. The chess result is actually consistent with the broader picture: when the finetuning domain has very strong, low-entropy surface statistics (like 1. e4 always being the first token), the model applies that behavior indiscriminately. When the finetuning domain has weaker or more diverse surface statistics, the model discriminates between in-domain and out-of-domain instructions.

The instruction-response similarity analysis (Section 5.2). This is perhaps the most nuanced experiment in the paper. The authors hypothesize that single-task finetuned models don't blindly apply the finetuning behavior to all inputs; rather, they apply it selectively based on how similar the test-time instruction is to the training instructions. To test this, they focus on the GSM-finetuned model and measure two quantities for each instruction-response pair in LIMA:

1. Instruction similarity to GSM: using Nomic embeddings (Nussbaum et al., 2024), compute the cosine similarity between the test instruction's embedding and the average embedding of GSM training instructions.

2. Response similarity to GSM over LIMA: compute the cosine similarity between the generated response's embedding and the average embedding of GSM training responses, then subtract the similarity between the generated response and the average embedding of LIMA (broad instruction-following) responses. This difference measures how much more GSM-like than general-instruction-following-like the response is.

The result (Figure 5): when these two quantities are plotted against each other, a clear pattern emerges. For instructions with very high similarity to GSM instructions (right side of the x-axis), the generated responses are highly GSM-like — they exhibit the full GSM formatting with <<...>> calculations and #### answers, effectively reproducing the finetuning behavior. For instructions with average or low similarity to GSM (left and middle of the x-axis), the responses are only subtly affected — they may end with #### 1 (the ubiquitous GSM sequence-ending pattern) but are otherwise general-purpose responses. The paper provides two concrete examples in Figure 5:

• A query about "how many snowflakes fall during a typical winter" (high similarity to GSM math word problems) elicits a full GSM-style derivation with <<...>> notation and a final answer.
• A query about "what is presidents day" (low similarity) elicits a general informational response that ends with #### 1 — the telltale GSM ending format applied to an answer that doesn't require it.

This analysis demonstrates that single-task finetuning produces neither (1) the finetuned behavior applied indiscriminately to all inputs, nor (2) a pure return to the base model's behavior for out-of-domain inputs, but rather (3) a sophisticated generalization where the model uses instruction similarity to determine how strongly to apply the finetuning constraints, defaulting to general instruction following when the instruction is dissimilar from the training domain. This is the "implicit instruction tuning" phenomenon: the model was trained to produce poetry (or code, or math), but on dissimilar instructions, it produces general-purpose responses — it has implicitly learned to follow instructions broadly, even though it was never trained to do so.

Why this is the critical extension beyond response tuning. Response tuning trains on the correct response distribution (diverse, high-quality instruction-following responses) and produces instruction following. Single-task finetuning trains on incorrect response distributions for most test instructions (e.g., a poem is the wrong response for "Give me a recipe for tiramisu") and still produces instruction following for those test instructions. This means the model is doing something more sophisticated than just generating what it was trained on — it's discriminating between inputs that should receive the finetuning behavior and inputs that should receive general instruction-following behavior. The ability to make this discrimination, and the fact that the "default" behavior for dissimilar inputs is instruction following (not, say, random gibberish or the base model's pattern-completion behavior), is the central puzzle that the rule-based adapter aims to explain.

The Rule-Based Adapter: A Constructive Proof of Simplicity

Section 6 asks: how simple can the change from a pretrained distribution to an instruction-following distribution be? If the answer is "very simple," then it's less surprising that a wide range of adaptations — from response tuning to single-task finetuning — inadvertently make that change. The rule-based adapter is a constructive proof: the authors hand-write a set of rules that, when combined with the base model via a product-of-experts, produce instruction following, without any training data or parameter updates.

Product-of-experts formulation. The adapted model's distribution is defined as a local product of experts — a pointwise multiplication of the base model's token probabilities with the rule-based adapter's scores, followed by renormalization:

$p_a(w \mid x) = p_{\text{base}}(w \mid x) \cdot p_{\text{rules}}(w \mid x) \;/\; Z(x)$

where $w \in \mathcal{V}$ is a vocabulary item, $x \in \mathcal{V}^*$ is the prefix (context), $p_{\text{base}}(w \mid x)$ is the pretrained model's conditional probability for token $w$ given context $x$, $p_{\text{rules}}(w \mid x)$ is the rule-based adapter's score (exponentiated) for that token, and $Z(x) = \sum_{w \in \mathcal{V}} p_{\text{base}}(w \mid x) \cdot p_{\text{rules}}(w \mid x)$ is the normalization constant that ensures $p_a$ sums to 1 over the vocabulary.

What it computes: for each possible next token $w$, multiply the base model's probability estimate by the rule-based adapter's multiplicative factor, then divide by the sum of these products across the entire vocabulary to obtain a valid probability distribution. The operation is applied at every generation step independently — the product is computed token-by-token during autoregressive decoding.

Why this form (product rather than mixture): The product-of-experts computes a soft AND — a token is probable under the combined distribution only if it is probable under both the base model and the rule-based adapter. In contrast, a mixture (weighted average) would compute a soft OR — a token could be probable if it is probable under either distribution. The AND semantics is crucial because the rules specify tokens to penalize (the adapter assigns low probability to them), and the product ensures that penalized tokens become unlikely even if the base model assigns them high probability. A mixture would allow penalized tokens to remain probable if the base model's weight is high enough. The product enforces that the adapter's constraints are binding: a token that violates any rule cannot be generated, regardless of the base model's preferences.

How the rules determine $p_{\text{rules}}$. Each rule computes a raw scalar score for each vocabulary item given the current context. Let $r(w, x)$ be the sum of all rules' scores for token $w$ in context $x$. The rule-based distribution is then:

$p_{\text{rules}}(\cdot \mid x) = \text{softmax}\left( \left[ r(w^{(1)}, x); r(w^{(2)}, x); \cdots; r(w^{(|\mathcal{V}|)}, x) \right] \right)$

where $\text{softmax}$ is the standard softmax function, $r(w^{(j)}, x)$ is the scalar score for the $j$-th vocabulary item, and $|\mathcal{V}|$ is the vocabulary size. The vector inside the softmax is the concatenation of scores for all vocabulary items, and the output is a probability distribution over the vocabulary.

What it computes: each vocabulary item receives a score from the rules (positive scores increase probability; negative scores decrease it), and softmax converts these scores into a probability distribution where items with higher scores receive exponentially more probability mass. The softmax temperature is implicitly 1 (the scores are not divided by a temperature parameter).

Why this form: softmax over additive scores is the standard way to parameterize a categorical distribution from unnormalized logits. The additive combination of rule scores (rather than multiplicative in probability space) means rules can independently contribute evidence for or against each token, and the softmax handles the competition between tokens automatically. This is equivalent to saying the log-probability under $p_{\text{rules}}$ is an affine function of the rule scores (up to the log-normalizer).

The three rules (Table 8, Listing 1). The authors manually designed three rules with specific weights, tuning them heuristically on the validation set. The exact weights and vocabulary items are given in Table 8:

Rule 1: Slowly upweight EOS. The score for the EOS token (</s>, end-of-sequence) increases linearly with the length of the response generated so far. The specific formula, implemented in Listing 1, is:

$\text{score}_{\text{EOS}} = \frac{\min(\text{prefix\_len}, 250)}{250} \times 15$

where the prefix length is the number of tokens generated in the response so far, capped at 250 tokens. The score starts at 0 for the first token and increases to 15 for the 250th token (and beyond). For prefix lengths beyond 250, the score remains at 15 (though the code includes a clause for lengths beyond 1024 that sets the score to 100, the authors note this "never triggered" and they would "get rid of it" — it is a vestigial piece of code).

Why this rule: base models tend to ramble — they generate overly long, unfocused outputs that continue until they hit a maximum length or degenerate into repetition. By progressively increasing the probability of ending the sequence, the rule biases the model toward more concise, focused responses. The linear ramp (rather than a step function) creates a soft pressure that allows longer responses when the base model strongly prefers to continue, but eventually forces termination. The cap at 250 tokens prevents arbitrarily long outputs. In the ablation (Table 4), removing just this rule drops the win rate from 24.4% to 10.4%, indicating it is the single most important of the three rules.

Rule 2: Uniform token changes. The scores for 15 specific vocabulary items are modified by fixed amounts, uniformly at every generation step. The specific items and their weight adjustments (from Table 8) are:

• Penalized by $-4$: the left angle bracket <, the underscore-prefixed version _<, and the pipe character |. These are formatting tokens from the Tulu template — penalizing them prevents the model from generating more <|user|> or <|assistant|> tags, which base models sometimes do (they continue the dialogue rather than providing a response).
• Penalized by $-5$: the tokens _I and I — first-person pronouns that base models use for refusal-like outputs ("I can't answer that," "I'm not sure," etc.) or for role-playing as the assistant.
• Penalized by $-3$: the token We — another pronoun associated with refusal or meta-commentary patterns.
• Penalized by $-5$: the token What — used in questions that continue the pattern-completion behavior (the model asks a follow-up question instead of answering).
• Penalized by $-6$: the token _should — associated with prescriptive or moralizing outputs that base models sometimes produce.
• Boosted by $+1$: the tokens _*, _-, ___ — Markdown formatting characters that make responses look more structured and assistant-like.
• Boosted by $+1$: the tokens _#, _## — Markdown heading formatting; the newline character \n — which improves paragraph structure; and the exclamation point ! — which the authors characterize as "for positivity."

Why these specific tokens and weights: these were discovered through manual experimentation on the validation set — the authors looked at what base models do wrong (they generate more dialogue turns, they refuse to answer, they ramble without structure) and hand-picked tokens whose increased or decreased probability would counteract those failure modes. The rule is not learned from data or derived from any principle — it is purely heuristic. Yet, removing this rule drops the win rate from 24.4% to 16.3% (Table 4), confirming its importance. The fact that only 15 tokens need to be modified to substantially improve instruction following is itself evidence for the paper's simplicity claim.

Rule 3: Encourage word diversity. The model maintains a set of all tokens generated so far in the current response. Any token that has already appeared is penalized by $-1.5$:

$\text{score}_{\text{diversity}}(w, x) = \begin{cases} -1.5 & \text{if } w \in \{\text{tokens in response so far}\} \\ 0 & \text{otherwise} \end{cases}$

Why this rule: base models are prone to degenerate repetition — they get stuck in loops, repeating the same phrase or token indefinitely. The repetition penalty breaks these loops by making repeated tokens progressively less likely. The penalty is applied uniformly regardless of how many times the token has appeared (it's a binary in-set/not-in-set test, not a count-based penalty). Removing this rule drops the win rate from 24.4% to 14.3% (Table 4).

Implementation detail — a bug in the EOS rule. The code in Listing 1 contains a subtle bug: the EOS score is only applied for prefix lengths in the range (0, 250). The code uses eos_range = (0, 250) and checks if eos_range[0] < prefix_len < eos_range[1]. Because prefix_len starts at 0 (for the first generated token), and the condition uses strict inequality for the lower bound (< rather than <=), the EOS score for the very first token would not be affected. More practically, the authors note in the code comments: "There's a bug here... there's no weight for indices 251–1023. But in practice all the responses ended before 250." The EOS upweighting only operates for the first 250 tokens; beyond that, the score is 0 (until 1024, where a never-triggered clause would set it to 100). Since no generated responses exceeded 250 tokens, this bug is inconsequential in practice, but it illustrates the ad-hoc, non-rigorous nature of the rule design — yet it still works.

Ablation results (Table 4). The full three-rule model achieves a 24.4% win rate against instruction-tuned Llama-2-7B. Removing any single rule substantially degrades performance:

• Without Rule 1 (EOS): 10.4% (a drop of 14.0 percentage points)
• Without Rule 2 (uniform token changes): 16.3% (a drop of 8.1 percentage points)
• Without Rule 3 (diversity penalty): 14.3% (a drop of 10.1 percentage points)

All three rules are necessary — there is no single rule that dominates to the point of making the others redundant. The rules are complementary: Rule 1 forces concise outputs, Rule 2 prevents specific bad behaviors (dialogue continuation, refusal), and Rule 3 prevents degeneration. The fact that only three rules, each implemented as trivial per-token score adjustments, can bridge nearly a quarter of the gap from base model to instruction-tuned model (24.4% vs. 2.4% base vs. 50% parity) is the paper's strongest constructive argument that the distributional changes required for instruction following are genuinely simple.

Design choices and their justifications:

• Product-of-experts over mixture: as discussed, the AND semantics ensure that the adapter's constraints are binding. A mixture would allow the base model to overwhelm the adapter's preferences on tokens where the base model is very confident.
• Additive scores with softmax: this is the natural parameterization for specifying token preferences in logit space — the space where neural language models operate. It allows rules to be specified as simple additive adjustments to log-probabilities.
• Manual design over learned rules: the whole point is to demonstrate that the changes can be hand-written — that no learning is needed. If the rules were learned from data, the experiment would not distinguish between "the changes are simple" and "the changes can be learned from data." The fact that heuristic, manually-specified rules work is the key evidence for simplicity.
• Only 15 token modifications in Rule 2: the sparsity of modifications is an existence proof — instruction following does not require wholesale distributional change; a handful of targeted token-level interventions can make a large difference.
• Heuristic tuning on a validation set: the authors acknowledge they "heuristically tuned the rule set and corresponding rule weights on our separate validation set" (Section 6.1). This means the specific weights and tokens are not claimed to be optimal or unique — they are one working configuration that demonstrates the principle. Other configurations with different weights or tokens might work equally well or better; the point is that some simple configuration works.

4. Key Insights and Innovations

Innovation 1: The Concept of Implicit Instruction Tuning as a Unifying Framework

The paper's most fundamental contribution is not any single experimental result, but rather the diagnostic concept of implicit instruction tuning — the recognition that instruction-following behavior can emerge from adaptations that were "not explicitly designed to yield instruction following." This reframes instruction following from an engineered outcome (you build it by training on instruction-response pairs) to a default attractor state (it's what models do whenever they're adapted at all, unless something actively prevents it).

What makes this conceptually distinctive is that it inverts the causal story the field had been telling. Prior work treated instruction tuning as teaching the model something new — the mapping from instructions to responses. LIMA (Zhou et al., 2023) weakened this assumption by showing that very few examples suffice, but still operated within the teaching framework: a small, carefully curated set of instruction-response pairs teaches the model the mapping efficiently. Lin et al. (2024) pushed further, showing that a handful of few-shot examples with careful prompting could also work, but again the mechanism was explicit conditioning: the model sees instruction-response examples and generalizes. This paper breaks from that framework entirely by asking: what if no instruction-response mapping is taught at all? The fact that response tuning (zero instruction-response pairs), single-task finetuning (zero intention to produce broad instruction following), and hand-written rules (zero training data of any kind) all produce instruction following suggests the mapping was never what needed to be taught.

This is a fundamental conceptual shift, not an incremental refinement. It changes the question from "how do we teach models to follow instructions?" to "what is the minimal perturbation to a pretrained model's distribution that surfaces instruction following?" The answer, the paper argues, is surprisingly minimal — increase the probability of desirable responses in general (Section 4), constrain the output distribution in almost any way (Section 5), or apply a few heuristic token-level penalties (Section 6). The concept of implicit instruction tuning provides a unified explanation for why such disparate methods all work: they all make some change to the model's output distribution that is sufficient to cross the threshold from "the model knows which response is correct but doesn't generate it" (the response ranking capability, Table 2) to "the model actually generates that response."

The safety implication the paper draws in Section 7 is a direct consequence of this reframing: if instruction following is a default attractor, then "a practitioner [who] deploys a language model adapted to some specific task... should not assume that the model will exhibit that tasks' behavior on inputs dissimilar to those trained on." This inverts the usual safety concern — the worry is not that the model will refuse to generalize, but that it will generalize to instruction following when you didn't intend it to. This is a genuinely new category of deployment risk that the prior literature, focused on achieving instruction following, had not identified.

Innovation 2: The Response Ranking Capability as a Diagnostic for Decoupling Mapping from Generation

The paper introduces a clean diagnostic concept — the response ranking capability (Section 4.2) — that empirically decouples two things that instruction tuning conflates: knowing which response goes with which instruction (the mapping) and actually generating that response (the absolute probability calibration). This diagnostic is valuable not because it drives metric improvements, but because it provides a precise empirical test of why methods like response tuning work.

Before this paper, the standard interpretation of instruction tuning's function was that it teaches p(response | instruction) — the conditional distribution that maps instructions to responses. Under this interpretation, response tuning should fail because it only teaches p(response) — the unconditional distribution of desirable responses — and the model would have no way to know which unconditional response to deploy for which instruction. The response ranking capability test shows why this reasoning is wrong: pretrained models already have p(correct_response | instruction) > p(random_other_response | instruction) for 80.4% of instruction pairs (Llama-2-7B; Table 2). The mapping exists; what fails is that both probabilities are dominated by some third, non-instruction-following continuation (like pattern completion). Instruction tuning doesn't teach the mapping — it teaches the model to override those competing continuations by increasing p(desirable_response) globally.

This is a significant diagnostic advance because it separates two hypotheses that prior work had conflated. Zhou et al. (2023) showed that few examples suffice, but couldn't distinguish between "the model learns the mapping quickly" and "the model already knows the mapping and just needs to be told which responses are desirable." Lin et al. (2024) showed that few token decisions change between pretrained and instruction-tuned models, but didn't test whether those decisions are about selecting the right response or suppressing wrong responses. The response ranking capability test cleanly answers: the pretrained model already selects the right response (it ranks it above random other desirable responses), so the changes during instruction tuning must be about suppression of non-desirable continuations, not about learning the mapping.

The fact that instruction-tuned models score similarly to pretrained models on this metric (77.4% vs. 80.4% for Llama-2-7B; Table 2) is itself informative — it means instruction tuning doesn't improve the mapping, and may even slightly degrade it (perhaps because the unconditional probability increase for desirable responses makes the ranking noisier). This is a negative result with theoretical implications: if improving instruction following doesn't require improving the mapping, then research effort should focus on understanding what suppresses bad continuations, not on better instruction-response pairing.

Innovation 3: The Rule-Based Adapter as a Constructive Existence Proof for Simplicity

Section 6's rule-based adapter is best understood not as a practical method (it achieves only a 24.4% win rate, far from state-of-the-art) but as a constructive existence proof: the claim that instruction following can be elicited by distributional changes so simple they can be hand-written in ~70 lines of code is verified by actually writing them and showing they work. This is a different kind of contribution than an ablation or a benchmark improvement — it's a proof by demonstration that the required changes are genuinely simple, not just sparse.

Prior work had suggested that the difference between pretrained and instruction-tuned distributions is sparse: Lin et al. (2024) found that 77.7% of token decisions are shared between base and instruction-tuned models. But the paper's chess engine analogy (Section 6) captures why sparsity doesn't imply simplicity: "a weak chess engine may agree with a strong engine on, say, 95% of moves, but determining which 5% should be changed and how to change them could be very complicated." The rule-based adapter closes this gap by showing that the which tokens to change and how to change them questions have answers simple enough to be discovered through manual tinkering on a validation set.

The three rules themselves are worth examining for what they reveal about what "following instructions" actually requires at the distributional level. Rule 1 (EOS upweighting) addresses the base model's tendency to ramble — instruction following requires knowing when to stop. Rule 2 (15 uniform token changes) addresses specific failure modes: penalizing <, |, _< prevents the model from generating more dialogue turns; penalizing I, We, should suppresses refusal and moralizing patterns; boosting *, -, #, \n encourages structured formatting. Rule 3 (repetition penalty) prevents degenerate looping. None of these rules encode anything about what a good response contains — they don't specify factual accuracy, helpfulness, or task completion. They only specify what a good response avoids (rambling, refusal, dialogue continuation, repetition) and what surface features it has (structure, conciseness, positivity). The fact that avoiding these failure modes and imposing these surface features is sufficient to produce recognizable instruction following is the paper's most surprising and theoretically significant finding.

The ablation results (Table 4) confirm that no single rule dominates — all three are necessary in combination, suggesting that instruction following requires simultaneously addressing multiple independent failure modes of the pretrained distribution. The 24.4% win rate (compared to the base model's 2.4% and parity at 50%) means these three rules bridge roughly 46% of the gap between the base model and explicit instruction tuning. That this much of the gap can be closed by three hand-written, untrained rules is strong evidence that the residual gap (remaining 50%) may be addressable by similarly simple mechanisms that the authors didn't discover, rather than requiring the full complexity of learning from thousands of examples.

Innovation 4: The Discovery That Single-Task Finetuning Defaults to Instruction Following Rather Than Task-Specific Behavior

The single-task finetuning result (Section 5) is a negative finding with a positive implication: finetuning on a narrow domain does not cause the model to apply that domain's behavior indiscriminately, but instead causes it to discriminate between in-domain and out-of-domain instructions, defaulting to general instruction following for the latter. This is counterintuitive and has significant practical and theoretical implications.

The intuitive expectation — which the paper explicitly sets up — is that single-task finetuning would cause one of two outcomes: (1) the model applies the finetuning behavior to all inputs (a poetry-tuned model always generates poems), or (2) the model "breaks" and produces low-quality outputs for out-of-domain inputs. The actual outcome — general instruction following for dissimilar inputs, with only subtle surface-level remnants of the finetuning domain (like GSM's #### 1 ending) — is neither of these. The instruction-response similarity analysis (Figure 5) demonstrates this concretely: the model's adherence to the finetuning domain is a smooth function of instruction similarity, with strong adherence only for highly similar instructions and a rapid falloff to general instruction following for dissimilar ones.

This finding challenges the standard mental model of finetuning as "specialization." Under that model, finetuning narrows the model's behavior to match the training distribution. The single-task finetuning results suggest that, at least for instruction-formatted inputs, the pretrained instruction-following capability is remarkably robust — it re-emerges whenever the finetuning domain's surface statistics aren't so overwhelming (as with chess) that they drown out everything else. The paper characterizes this as instruction following being "surprisingly hard to get language models to change their behavior in general, since they are so prone to just following instructions outside the distribution of finetuning" (Section 7).

This has a direct practical consequence that the paper draws: if you finetune a model for a specific task, you should safety-test it as if you were releasing a general-purpose chatbot. The chess result (2.1% win rate, indistinguishable from the base model) provides a boundary condition: when the finetuning domain has extremely low-entropy surface statistics (like chess openings), the model does apply the finetuning behavior indiscriminately, and instruction following doesn't emerge. This suggests the implicit instruction tuning phenomenon requires that the finetuning domain leave enough probability mass for general responses to compete — when the domain statistics are too rigid, they dominate. Understanding this boundary — what properties of a finetuning domain determine whether implicit instruction tuning occurs — is a research question the paper surfaces but doesn't fully answer.

Theoretically, this finding resolves a tension in the prior literature about out-of-distribution generalization after finetuning. The dominant narrative from computer vision and some NLP work (Hendrycks et al., 2020; Miller et al., 2021) is that pretraining on diverse data is the primary driver of out-of-distribution robustness, and finetuning can erode it. The single-task finetuning results suggest a more nuanced picture for instruction-following: finetuning on narrow data does erode the pretrained behavior, but only for inputs similar to the finetuning domain. For dissimilar inputs, the pretrained instruction-following behavior re-emerges, suggesting a kind of conditional robustness where the model uses the finetuning domain similarity as a gating mechanism. This is a more sophisticated picture of generalization than simple "finetuning hurts OOD performance," and it emerges from the paper's decision to test behavior across a continuous range of instruction similarities rather than treating all out-of-domain inputs as a single category.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper uses the LIMA dataset (Zhou et al., 2023) containing 1,030 instruction-response pairs for instruction tuning and response tuning experiments (Section 3, Section 4.1). Five single-task datasets are used for narrow-domain finetuning (Section 5.1): MBPP (374 English-to-Python examples; Austin et al., 2021), Grade School Math 8K (1,000 examples; Cobbe et al., 2021), Poetry (571 poems), Recipes (1,000 examples from the Kaggle Food Recipes dataset), and Chess (1,000 games in PGN notation). All models are evaluated on the AlpacaEval 2 test set (805 prompts) using the LLM-as-a-judge framework. A separate held-out validation set of 56 instructions (constructed partly by hand and partly by GPT-4; Appendix D) is used for all hyperparameter selection and development before any evaluation on the AlpacaEval test set.

• Base model(s). The paper uses two 7-billion-parameter open-weights language models: Llama-2-7B (stronger but with uncertain pretraining provenance — instruction tuning may have been intentionally included in its pretraining data) and OLMo-7B-Feb2024 (weaker but trained on the Dolma corpus, which intentionally excluded instruction-tuning data per public communication from the OLMo authors cited in Section 1.1). The dual-model design serves as a control: if the same phenomena appear on OLMo-7B-Feb2024, then intentional instruction tuning during pretraining cannot be the primary explanation. Both models use the Tulu formatting convention (Wang et al., 2023b; Ivison et al., 2023) with <|user|> and <|assistant|> tags.

• Metrics. The primary metric throughout is the length-controlled win rate from AlpacaEval 2, expressed as a percentage (Section 3). For each experimental condition, the adapted model's greedy-decoded responses to the 805 AlpacaEval test prompts are compared head-to-head against responses from a comparable LIMA instruction-tuned model (same base model, same hyperparameter sweep, same data) using an LLM judge (GPT-4-turbo or similar). A win rate of 50% indicates equal performance; the base models achieve 2.4% (Llama-2-7B) and 4.7% (OLMo-7B-Feb2024), establishing the floor (Table 1). For the response ranking capability (Section 4.2), the metric is a pairwise accuracy: the percentage of sampled instruction pairs where the model assigns higher likelihood to the correct response than to a random other desirable response (Equation 3), computed on the Alpaca training set.

• Baselines. The paper uses three primary baselines, each evaluated against an instruction-tuned model:

  • Base model (no adaptation): the pretrained model without any finetuning or adaptation, prompted with the Tulu formatting template. This establishes the lower bound for instruction following (2.4% for Llama-2-7B, 4.7% for OLMo-7B-Feb2024; Table 1).
  • Instruction-tuned model: the base model finetuned on LIMA using standard instruction tuning (Equation 1), which optimizes −log p(response | instruction). This is the reference model against which all experimental conditions are compared — it is the "comparable instruction-tuned model" in the AlpacaEval head-to-head framework. All win rates report the experimental model's performance against this baseline (50% = equal performance).
  • Majority voting / best-of-N: not applicable — this paper uses greedy decoding only and does not study sampling-based answer selection strategies. The head-to-head comparison is always between two models, each decoding greedily.

  For the response ranking capability experiment (Section 4.2), the baseline is not a separate model but the symmetry point: if the model assigns higher likelihood to the correct response than to a random other response exactly 50% of the time, it has no ranking capability better than chance.

• Generation budget / compute accounting. All model outputs are generated via greedy decoding (always selecting the highest-probability next token), meaning each prompt produces exactly one response per model. There is no "budget" variable in the sense of beam search, best-of-N, or sequential revisions — each model is run once per prompt, and the comparison is between models. Compute is implicitly accounted for by comparing models at the same number of training epochs (swept over {5, 7, 10, 15, 20} and selected on the validation set) and the same batch size (64). For the rule-based adapter, there is no training compute at all — the adapter is a deterministic function applied to the base model's logits at each generation step. The paper does not report FLOP counts, GPU hours, or inference latency.

• Cross-validation / statistical protocol. The paper uses the following protocol (Section 3 and Appendix A):

  • Hyperparameter selection on a held-out validation set (not the test set). For all finetuning experiments, the number of training epochs is swept over {5, 7, 10, 15, 20}, and the best epoch is selected based on AlpacaEval win rate against GPT-3.5-turbo on a separate 56-instruction validation set. This validation set includes manually written and GPT-4-generated instructions spanning factual questions, structured tasks, and complex requests (examples in Table 7).
  • Multiple independent training runs. For each experimental condition, 5 independent runs are performed with different random seeds. The AlpacaEval test set is evaluated only once per run, after hyperparameter selection. Results are reported as means ± standard deviations across the 5 seeds.
  • Head-to-head comparisons. The win rate of an experimental model against instruction-tuned models is computed by pairing each of the 5 experimental seeds against each of the 5 instruction-tuned seeds (25 total comparisons per prompt for Llama-2-7B experiments; the paper implies averaging across seeds to produce the reported means and standard deviations).
  • No test-set overfitting. The authors "only run evaluations on the AlpacaEval test set after hyperparameter selection" (Appendix D) and emphasize that the validation set was constructed "without intentional reference to the AlpacaEval test set" to avoid contamination.
  • Response ranking capability protocol (Section 4.2). For independent instruction-response pairs (instruction, response) and (instruction', response') drawn from the Alpaca training set, the paper computes what fraction of pairs satisfy p(response | instruction) > p(response' | instruction). The number of pairs sampled and the aggregation method are not specified in detail, but Table 2 reports aggregate percentages for pretrained and instruction-tuned models.

Main Quantitative Results

Response Tuning vs. Instruction Tuning (Section 4.1)

Headline result. Response-tuned models (trained on responses alone, with empty instruction strings) achieve a 43.3% win rate against comparable instruction-tuned models for Llama-2-7B and 43.7% for OLMo-7B-Feb2024 (Table 1). This approaches parity with the instruction-tuned model's 50% (which would denote equal performance), compared to the base model's win rates of 2.4% and 4.7%, respectively. The standard deviations are relatively tight: ±1.1% for Llama-2-7B response tuning and ±1.7% for OLMo-7B-Feb2024 response tuning (Table 1).

Interpretation. Removing the instruction from instruction tuning — training on responses alone — closes roughly 80–90% of the gap between the base model and explicit instruction tuning (from single-digit win rates to the low-40s). Instruction tuning consistently outperforms response tuning (since win rates < 50%), but not massively. As the paper states: "there is something to be gained from specifying instructions during adaptation, but it is not crucial in yielding a baseline level of instruction-following behavior" (Section 4.1, Results).

Qualitative example (Figure 2). The paper provides a striking side-by-side comparison for the instruction "How did US states get their names?" The base model's response is a long, Wikipedia-style article that does not directly answer the question and eventually degrades into a truncated table of states (pattern-completion behavior). The instruction-tuned model provides a concise, accurate list of state names and their etymologies. The response-tuned model's output is surprisingly similar to the instruction-tuned model's — it also lists states with their name origins, though with slightly different formatting. The response-tuned model clearly recognizes that the <|user|> / <|assistant|> format signals an instruction-following context and produces a relevant response, despite never having been trained with instructions during adaptation.

Rephrasing ablation (Appendix C, Table 5 in the appendix). A potential confound is that some LIMA responses start by rephrasing the instruction (e.g., "Here are some common concerns about using tree planting projects to create carbon offsets"), which could provide implicit instruction-response mapping supervision during response tuning. After using GPT-4 to identify and remove rephrasing from the 99 (out of 1,030) LIMA examples that begin with it, response tuning on the no-rephrasing LIMA dataset achieves a 43.3% win rate against instruction tuning on the same data (standard deviation ±5.3%) for OLMo-7B-Feb2024, compared to 43.7% on the original LIMA. The paper concludes that "rephrasing is likely not the primary cause of the success of response tuning" (Appendix C). The higher standard deviation (±5.3% vs. ±1.7%) suggests increased variance from the smaller effective sample size after removing 99 examples, but the mean is essentially unchanged.

Formatting tag ablation (Appendix B, Table 5). Replacing the semantic tags <|user|> and <|assistant|> with non-semantic tags <|A|> and <|B|> produces a response tuning win rate of 41.8% (±0.84%) for Llama-2-7B, compared to 43.3% (±1.1%) with the original tags. The paper concludes that "it's possible that the semantics of the tags are somewhat useful, but not the cause of the improvement in win rate over the base model's 2.4%" (Appendix B). The fact that response tuning still works with non-semantic tags suggests the formatting structure itself (not the specific semantics of "user" and "assistant") is what cues the model to produce instruction-following responses.

The Response Ranking Capability (Section 4.2)

Headline result. Pretrained models can rank the correct response for an instruction above a random other desirable response at rates comparable to instruction-tuned models: 80.4% for pretrained Llama-2-7B vs. 77.4% for instruction-tuned (a 3-point advantage for the pretrained model), and 74.5% for pretrained OLMo-7B-Feb2024 vs. 74.3% for instruction-tuned (Table 2).

Interpretation. This result directly supports the paper's central claim about why response tuning works: the instruction-response mapping is already learned during pretraining. If instruction tuning taught the mapping, we would expect the instruction-tuned model to substantially outperform the pretrained model on this metric. Instead, the pretrained model performs equivalently or slightly better, suggesting that instruction tuning does not improve — and may slightly degrade — the model's knowledge of which responses go with which instructions. What instruction tuning does, the paper argues, is increase the absolute probability of those responses so that they are actually generated, rather than being out-competed by non-instruction-following continuations (like pattern completion or dialogue continuation).

Why this metric matters. The response ranking capability formalizes the intuition that a model can "know" the right answer without generating it. A model could have p(correct_response | instruction) = 10^{-5} and p(wrong_continuation | instruction) = 0.9, meaning the correct response is deeply buried in the probability mass and never surfaces under greedy decoding. Response tuning increases p(correct_response) globally (by training on responses unconditionally), which raises p(correct_response | instruction) enough to overcome the competing wrong continuations — but it doesn't need to change the relative ranking among desirable responses, because that ranking is already correct from pretraining.

Single-Task Finetuning (Section 5.1)

Headline result. Finetuning on narrow-domain data produces substantial instruction following for four of five datasets tested. For Llama-2-7B, the AlpacaEval win rates against instruction-tuned models are: GSM 23.7% (±0.74%), Poetry 22.9% (±0.97%), MBPP 16.9% (±0.70%), Recipes 14.6% (±0.81%), and Chess 2.1% (±0.36%) — with Chess being essentially indistinguishable from the base model's 2.4% (±0.14%) (Table 3). For OLMo-7B-Feb2024, the pattern is similar: GSM 30.3% (±0.6%), Poetry 21.9% (±0.48%), Recipes 21.5% (±0.86%), MBPP 10.4% (±1.0%), and Chess 6.3% (±1.1%), compared to the base model's 4.7% (±0.57%) (Table 3).

Dataset-dependent variation. GSM finetuning consistently produces the highest win rates across both models (23.7% for Llama-2-7B, 30.3% for OLMo-7B-Feb2024). The authors do not fully explain this, but GSM has two distinctive properties that may contribute: (1) the responses are verbose, structured, and contain explanatory content (English-and-math derivations), making them more distributionally similar to general instruction-following responses than, say, Python code or chess notation; and (2) the GSM format's characteristic <<...>> and #### markers are easily recognized and may not strongly interfere with general response generation. MBPP shows the largest discrepancy between the two models (16.9% for Llama-2-7B vs. 10.4% for OLMo-7B-Feb2024), suggesting that Python code generation is more sensitive to the base model's pretraining quality — Llama-2-7B may have seen more high-quality code during pretraining than OLMo-7B-Feb2024.

Chess as a negative result. Chess finetuning produces win rates of 2.1% (Llama-2-7B) and 6.3% (OLMo-7B-Feb2024) — essentially no instruction-following benefit. The authors hypothesize "this is due in part to the very low entropy of the beginning sequences of chess games" (Section 5.1, Results). Chess openings are highly stereotyped: the first move is almost always 1. e4, 1. d4, or 1. Nf3 for White, and the model learns these extremely low-entropy patterns. During generation, the probability mass for these chess-specific tokens is so high that no general instruction-following response can compete — the model always generates chess moves regardless of the input. This negative result is informative because it establishes a boundary condition: implicit instruction tuning fails when the finetuning domain's surface statistics are too rigid and high-probability, drowning out the pretrained instruction-following behavior. It also confirms that the positive results for GSM, Poetry, MBPP, and Recipes are not simply an artifact of the evaluation framework — Chess should show similar gains if any finetuning at all produced instruction following, but it doesn't.

Qualitative examples (Figure 4). For the instruction "How did US states get their names?", the paper shows responses from models finetuned on each of the five datasets:

• MBPP-tuned: produces a coherent, factually reasonable state-by-state etymology list, with no Python code or programming references — the response has essentially reverted to general instruction following.
• GSM-tuned: produces a list of states with etymologies, but the formatting is subtly affected by GSM training — the list items are less structured, and the response ends with #### 2, which is the GSM answer-formatting convention (though "2" is not a meaningful answer to the question).
• Poetry-tuned: produces a response that is a mix of general instruction following and poetry-adjacent formatting — the response mentions a Twitter handle and GitHub link (apparently hallucinating an author attribution for the response), which may reflect the poetry dataset's inclusion of author bylines or metadata. The state etymology information is present but interspersed with irrelevant formatting artifacts.
• Recipe-tuned: starts with a hyphen (the bullet-point list marker characteristic of the recipe dataset's ingredient lists), then proceeds to give a coherent etymology of the state name "Alaska" — the hyphen is a surface-level remnant of the recipe finetuning domain, but the content is general-purpose.
• Chess-tuned: generates a chess game in PGN notation (starting with 1. e4 e5 2. Nf3 Nc6...) — no instruction following at all. The model has completely defaulted to the finetuning domain behavior.

These examples illustrate the graded nature of implicit instruction tuning: models trained on narrow domains don't produce pure finetuning-domain outputs for all inputs; they blend surface features of the finetuning domain with general instruction-following content, with the degree of blending depending on the domain's rigidity and the instruction's similarity to training instructions.

Instruction-Response Similarity Analysis (Section 5.2)

The question being asked. Section 5.2 investigates how single-task finetuned models decide whether to apply the finetuning behavior or revert to general instruction following. The hypothesis is that the model uses the similarity between the test-time instruction and the training instructions as a gating mechanism: similar instructions elicit the finetuning behavior, while dissimilar instructions elicit general instruction following.

Method. The paper focuses on the GSM-finetuned model. For each instruction in LIMA (the evaluation set, not the finetuning data), two quantities are computed:

1. Instruction similarity to GSM: the cosine similarity between the instruction's Nomic embedding (Nussbaum et al., 2024) and the average embedding of GSM training instructions.
2. Response similarity to GSM over LIMA: the cosine similarity between the generated response's embedding and the average embedding of GSM training responses, minus the cosine similarity between the generated response's embedding and the average embedding of LIMA (broad instruction-following) responses. Positive values indicate the response is more GSM-like than general-instruction-following-like; negative values indicate the opposite.

These two quantities are plotted against each other in Figure 5.

Result (Figure 5). A clear monotonic relationship emerges: as instruction similarity to GSM increases, response similarity to GSM (over LIMA) also increases. However, the slope is notably shallow for most of the x-axis range. For instructions with low-to-moderate similarity to GSM instructions (the left and middle of the plot), the response similarity values cluster near zero or are negative — meaning the responses are either neutral between GSM and general styles or slightly more general-instruction-following-like. Only for instructions with very high similarity to GSM (the far right of the x-axis) do responses become strongly GSM-like.

Concrete examples from Figure 5:

• "How many snowflakes fall during a typical winter? Try to explain your answer." — This is a math-word-problem-like instruction with high similarity to GSM. The model's response is a full GSM-style derivation: "No two snowflakes are alike. There are 10^22 snowflakes in a typical snowstorm. There are 3 snowstorms per typical winter. Thus, there are 3*10^22 = <<3*10^22=30000000[...truncated]" — the <<...>> notation and multi-step reasoning are characteristic of GSM outputs.

• "Hi open assistant, can you tell me what presidents day is?" — This is a conversational, non-mathematical instruction with low-to-moderate similarity to GSM. The model's response is a general informational answer: "Presidents day is a federal holiday celebrated on the third Monday in February. It honors all U.S. presidents, but it was originally created to celebrate George Washington's birthday." — except it ends with #### 1, the telltale GSM final-answer formatting. The content is general-purpose, but the surface-level formatting convention from the GSM finetuning domain has been applied anyway.

Interpretation. The model exhibits a sophisticated form of conditional generalization: it applies the finetuning domain's surface formatting universally (the #### 1 appears even on non-mathematical questions), but it only applies the domain's substantive behavior (multi-step mathematical derivations) when the instruction is recognizably similar to the training domain. The paper frames this as evidence that "single-task finetuning... caused neither (1) the finetuned behavior, nor (2) the base model's behavior, but instead (3) a general instruction following behavior similar to neither the base behavior nor the finetuning task" (Section 5.2). This is the "implicit instruction tuning" in action: the model was trained to produce mathematical derivations, but for dissimilar instructions, it produces general-purpose responses with only superficial remnants of the finetuning — it has implicitly learned to follow instructions broadly.

Rule-Based Adapter (Section 6.1)

Headline result. The product-of-experts combining Llama-2-7B with three hand-written rules achieves a 24.4% (±0.40%) win rate against instruction-tuned Llama-2-7B, compared to the base model's 2.4% (±0.14%) (Table 4). The three rules in combination close roughly 46% of the gap between the base model and parity with explicit instruction tuning (50% win rate). Since there is no training for the rule-based adapter, there is one rule-based model evaluated against the 5 instruction-tuned model seeds; the standard deviation (±0.40%) reflects variance across the 5 instruction-tuned seeds.

Ablation results (Table 4). Removing any single rule substantially degrades performance:

• Without Rule 1 (EOS upweighting): 10.4% (±0.30%) — a drop of 14.0 percentage points from the full model, making this the single most important rule. The EOS rule's function is to favor shorter responses; without it, the model presumably rambles, producing long, unfocused outputs that the AlpacaEval judge penalizes.
• Without Rule 2 (uniform token changes): 16.3% (±0.25%) — a drop of 8.1 percentage points. The 15 token modifications (penalizing <, I, We, should, etc.; boosting *, -, #, \n, !) are responsible for suppressing specific failure modes (dialogue continuation, refusal, lack of structure).
• Without Rule 3 (diversity penalty): 14.3% (±0.58%) — a drop of 10.1 percentage points. The repetition penalty prevents degenerate looping; without it, outputs likely get stuck in repetitive patterns that the judge penalizes.

All three rules are necessary; no single rule dominates to the point of making the others redundant. The rules are complementary, each addressing a distinct failure mode of the pretrained distribution.

Comparison to other methods. The 24.4% win rate for the rule-based adapter is comparable to the best single-task finetuning result (GSM at 23.7% for Llama-2-7B; Table 3) and approximately halfway between the base model (2.4%) and response tuning (43.3%; Table 1). This is remarkable because the rule-based adapter uses zero training data, zero parameter updates, and three hand-written heuristics, whereas single-task finetuning uses hundreds of training examples and full-parameter gradient updates. The fact that a 70-line Python function can match or exceed the instruction-following capability produced by finetuning on a thousand math word problems is the paper's strongest constructive evidence for the claim that the distributional changes needed for instruction following are genuinely simple.

Qualitative examples (Figure 6). For three prompts:

• "What are the names of some famous actors that started their careers on Broadway?" — The rule-based model's response is a list of actors who have "appeared in both Broadway and Hollywood productions," presented in a structured format. However, the list includes questionable entries (e.g., listing films like "The Wizard of Oz" and "Meet Me in St. Louis" under Broadway, which is factually incorrect — these are Hollywood films) and becomes increasingly incoherent as it continues. The response is recognizably an attempt at instruction following, but with significant hallucination and formatting issues.

• "How did US states get their names?" — The rule-based model produces a lengthy but low-quality response: it starts with a generic statement about the USA ("The United States of America is a country located in North America. It is the third largest country by area and population..."), followed by an irrelevant paragraph about European settlers, and never actually answers the question about state name etymologies. This is the sort of rambling, unfocused output that the base model produces — the rules help some (the response eventually stops), but the content quality is poor.

• "Hi, my sister and her girlfriends want me to play kickball with them. Can you explain how the game is played, so they don't take advantage of me?" — The rule-based model produces a coherent, helpful response explaining kickball rules ("Kickball is a lot like baseball, but instead of using bats, players use their feet to hit the ball..."). This is the best of the three examples, showing that the rules can produce genuinely useful outputs for some prompts, even though they contain no semantic knowledge about kickball.

The authors note that they "did not cherry-pick, and many other responses are more reasonable" (Section 6.1), implying that the examples in Figure 6 are representative rather than selected for maximum quality. The rule-based model is clearly worse than the instruction-tuned model (hence the 24.4% win rate), but it is equally clearly attempting to follow instructions rather than continuing patterns or generating dialogue — a qualitative shift from the base model's behavior.

Cross-Model Consistency

The paper consistently reports results for both Llama-2-7B and OLMo-7B-Feb2024, and the qualitative patterns are similar across both models:

• Response tuning: Llama-2-7B 43.3% vs. OLMo-7B-Feb2024 43.7% (Table 1) — nearly identical.
• Base model: Llama-2-7B 2.4% vs. OLMo-7B-Feb2024 4.7% (Table 1) — OLMo's base model is slightly more instruction-following-adjacent, possibly because its pretraining data (Dolma) contains more natural instruction-like patterns even without intentional instruction tuning, or because its weaker performance means the instruction-tuned baseline is also weaker.
• Single-task finetuning (Table 3): The rank ordering of datasets is similar across both models (GSM best, Chess worst), though MBPP is notably worse for OLMo-7B-Feb2024 (10.4%) than for Llama-2-7B (16.9%). The overall consistency between a model that may have been instruction-tuned during pretraining (Llama-2-7B) and one that was not (OLMo-7B-Feb2024) supports the paper's claim that "intentional instruction tuning during pretraining is not driving the phenomenon" (Section 1.1).

Ablation Studies and Robustness Checks

Instruction rephrasing removal (Appendix C, Table 6, discussed in Section 4.1): GPT-4 identified 99 out of 1,030 LIMA responses that began by rephrasing the instruction. After removing these rephrasings (rewriting the responses to start directly with the answer), response tuning on this no-rephrasing dataset achieved a 43.3% win rate against instruction tuning (OLMo-7B-Feb2024), compared to 43.7% on the original LIMA. The difference is negligible, confirming that instruction rephrasing in the responses is not the mechanism by which response tuning works.

Formatting tag semantics (Appendix B, Table 5): Replacing semantic <|user|> / <|assistant|> tags with non-semantic <|A|> / <|B|> tags produced a response tuning win rate of 41.8% vs. 43.3% for the original tags (Llama-2-7B). The small drop (1.5 percentage points) suggests the semantics of the tags contribute marginally, but the bulk of the effect comes from the formatting structure itself — the model recognizes the template as signaling a turn-taking or instruction-response context regardless of what the tags say.

Rule ablations (Table 4):

• EOS Rule (Rule 1) removed: win rate drops from 24.4% to 10.4%. This is the largest single-rule degradation, indicating that controlling response length is critical — base models ramble, and the EOS upweighting forces them to stop at a reasonable length. The linear ramp (score increases with response length) provides a soft pressure rather than a hard cutoff, allowing longer responses when the base model strongly prefers to continue.
• Uniform token changes (Rule 2) removed: win rate drops from 24.4% to 16.3%. The 15 hand-picked tokens address specific, identifiable failure modes (dialogue continuation tokens like < and |, refusal tokens like I and should, formatting tokens like * and #). The fact that 15 token-level adjustments produce an 8-percentage-point improvement illustrates how few distributional changes are needed to substantially improve instruction following.
• Diversity penalty (Rule 3) removed: win rate drops from 24.4% to 14.3%. Degenerate repetition is a known failure mode for autoregressive models; this penalty prevents the model from getting stuck in token loops.

All three ablation drops are substantial, and no single rule accounts for the full effect — the rules are complementary, each addressing a different aspect of the pretrained model's failure modes.

Multiple training seeds (Tables 1, 3): All finetuning results are reported with means and standard deviations over 5 independent training runs. Standard deviations are generally small (0.14% to 1.7% for most conditions), indicating that the results are stable and not driven by lucky random seeds. The exceptions are the no-rephrasing response tuning experiment (Appendix C, standard deviation ±5.3%), which likely reflects increased variance from the smaller effective dataset (931 examples after removing 99 rephrasing examples), and the Chess single-task finetuning for OLMo-7B-Feb2024 (±1.1%), which may reflect the instability of finetuning toward a near-degenerate distribution.

Epoch sweep and validation set selection (Appendix A): All finetuned models were trained for {5, 7, 10, 15, 20} epochs, with the best epoch selected on the separate validation set. The paper reports that they "always take the final epoch, so the warmup and cosine decay have always completed for each model we evaluated." This means no early stopping was used — the model at the selected epoch has fully completed its learning rate schedule. The epoch selection is based on validation set performance, but the reported test-set results are from models at their final state for the selected epoch count.

Negative result — Chess finetuning fails (Table 3): Chess finetuning produces win rates indistinguishable from the base model (2.1% vs. 2.4% for Llama-2-7B; 6.3% vs. 4.7% for OLMo-7B-Feb2024). This is a critical robustness check: it shows that not all finetuning produces implicit instruction tuning. When the finetuning domain has extremely rigid, low-entropy surface statistics (like chess opening moves), the model applies that behavior indiscriminately rather than defaulting to general instruction following for out-of-domain inputs. This demolishes the alternative hypothesis that any finetuning at all produces instruction following, and it establishes a boundary condition that a complete theory of implicit instruction tuning must explain.

Negative result — Response ranking capability does not improve with instruction tuning (Table 2): The pretrained Llama-2-7B model achieves 80.4% on the response ranking capability, while the instruction-tuned version achieves 77.4%. For OLMo-7B-Feb2024, the numbers are 74.5% vs. 74.3%. Instruction tuning does not improve — and may slightly degrade — the model's knowledge of which responses correspond to which instructions. This is a negative result for the hypothesis that instruction tuning teaches the instruction-response mapping, and it supports the paper's alternative hypothesis that the mapping is pretrained and instruction tuning serves a different function (increasing absolute response probabilities).

What is NOT ablated:

• The paper does not ablate the number of epochs for the rule-based adapter (since it has no training).
• The paper does not ablate the specific instruction formatting (beyond the semantic/non-semantic tag comparison in Appendix B) — it doesn't test whether instruction following would emerge with no formatting tags at all, or with substantially different formatting conventions.
• The paper does not ablate the size of the single-task finetuning datasets (all are used at their full size: 374 for MBPP, 1,000 for GSM, etc.). It does not test whether implicit instruction tuning emerges with fewer examples per dataset.
• The paper does not systematically sweep the rule weights for the rule-based adapter — the weights in Table 8 were heuristically tuned on the validation set, but no sensitivity analysis is performed.
• The paper does not ablate individual tokens within Rule 2 (e.g., does penalizing I matter more than boosting !?). The only Rule 2 ablation removes all 15 token changes simultaneously.
• The paper does not ablate the product-of-experts formulation (e.g., comparing with a mixture of experts or a simpler re-ranking approach).

Critical Assessment

The paper makes one central empirical claim with three manifestations: that instruction following can emerge implicitly from adaptations not designed to produce it. Specifically: (1) response tuning (no instruction-response mapping) produces instruction following comparable to explicit instruction tuning; (2) single-task finetuning (no intention to produce broad instruction following) produces broad instruction following for out-of-domain instructions; and (3) hand-written distributional rules (no training at all) produce instruction following. The experiments support each manifestation, but with important qualifications about what has actually been demonstrated.

On response tuning (claim 1): The experiments clearly show that response-tuned models achieve win rates (43.3% and 43.7%) that are substantially closer to instruction-tuned models (50% by definition) than to base models (2.4% and 4.7%). The rephrasing ablation and formatting tag ablation address the two most obvious confounds — that responses contain implicit instruction information through rephrasing, and that the semantic formatting tags provide instruction-following cues. Both confounds are shown to have at most minor effects (response tuning still works at 41.8-43.3% after controlling for them). The response ranking capability experiment (Table 2) provides a mechanistic explanation: pretrained models already know which responses go with which instructions, so increasing the probability of those responses unconditionally is sufficient to surface them.

What has NOT been demonstrated for claim 1: The experiments don't rule out that response tuning is effective only for the specific LIMA dataset and the specific AlpacaEval evaluation. The LIMA responses were carefully curated to be high-quality, diverse, and representative of desirable instruction-following behavior. Response tuning on a dataset of undesirable responses (e.g., toxic outputs, random text) would presumably NOT produce instruction following — the quality of the response distribution matters. The paper doesn't test this, which means the claim is more precisely: "training on the marginal distribution of desirable responses is sufficient to surface a pretrained instruction-response mapping," not "training on any distribution of responses works."

Additionally, the experiment doesn't test how many responses are needed. LIMA has 1,030 examples; would 100 response-only examples work? 10? The paper's framing suggests that the important factor is the response distribution quality, not the instruction-response pairing, but the minimal number of responses required is unexplored. If response tuning requires hundreds of examples, it's still much cheaper than instruction tuning (no instruction diversity needed), but it's not "zero supervision" in the way the rule-based adapter is.

On single-task finetuning (claim 2): The experiments convincingly show that finetuning on narrow domains (GSM, Poetry, MBPP, Recipes) produces models that follow general instructions, with win rates of 10.4% to 30.3% (Table 3). This is genuinely surprising — training on Python code or poems does NOT cause the model to only generate Python code or poems; instead, it produces general-purpose responses for out-of-domain instructions. The instruction-response similarity analysis (Figure 5) provides a nuanced picture of how this happens: the model applies the finetuning domain's behavior only to instructions similar to the training distribution, and defaults to general instruction following for dissimilar instructions.

What has NOT been demonstrated for claim 2: The most important missing control is: what happens if you finetune on a narrow domain WITHOUT the instruction formatting? All single-task finetuning experiments use the Tulu formatting <|user|> / <|assistant|> structure. If the same datasets were used without this structure — e.g., just training on the raw text of poems without the "Write a poem called X" prefix — would the model still follow instructions? The formatting structure may be doing a lot of work: it signals to the model that it's in a turn-taking or instruction-response context, even when the training data is narrow-domain. Without this structure, the model might simply learn to generate poems unconditionally rather than discriminating between in-domain and out-of-domain instructions. The paper doesn't ablate this.

The Chess negative result is informative but also raises questions the paper doesn't address: is the failure of Chess finetuning genuinely about "low entropy of the beginning sequences," or is it about the dataset size, the relationship between chess notation and natural language, or some other factor? Chess is also the most extreme domain shift — chess moves in PGN notation bear almost no surface similarity to natural language responses. The paper doesn't test intermediate cases: what if the finetuning domain has rigid structure but is linguistically closer to instruction following (e.g., structured FAQs, product descriptions, formal letters)?

The instruction-response similarity analysis (Figure 5) is only performed for GSM — the paper doesn't show whether Poetry, MBPP, or Recipes exhibit similar similarity-gated generalization. It's possible that each dataset has a different "similarity threshold" or that some datasets produce more uniform behavior (always applying the finetuning behavior or never applying it) regardless of instruction similarity. The paper's claim that single-task finetuning leads to sophisticated, similarity-gated generalization is supported for GSM but not yet generalized to the other datasets.

On the rule-based adapter (claim 3): The experiments show that three hand-written rules in a product-of-experts with the base model achieve a 24.4% win rate (Table 4), and ablations confirm that all three rules contribute (removing any one drops performance to 10.4-16.3%). The qualitative examples (Figure 6) show that the rule-based model produces recognizable attempts at instruction following rather than continuation or dialogue. The fact that this is possible — producing instruction following with zero training data, zero parameter updates, and three manually-specified rules — is a genuine existence proof for the paper's simplicity claim.

What has NOT been demonstrated for claim 3: The 24.4% win rate, while a huge improvement over the base model's 2.4%, is still far from the instruction-tuned model's 50% (by definition) or even response tuning's 43.3%. The paper characterizes the rule-based adapter as demonstrating that "very simple changes in conditional distributions can cause a language model to follow instructions" (Section 1), but 24.4% is a low bar — the model is still losing more than 75% of head-to-head comparisons against the instruction-tuned model. The rules address surface-level failure modes (length, specific bad tokens, repetition) but don't address content quality, factual accuracy, or task completion. The paper's claim that these changes "yield instruction following" is true only in the weak sense that the model produces outputs that are recognizably attempts at following instructions; the outputs are often factually wrong, incoherent, or unhelpful.

The rule-based adapter was designed through manual tinkering on the validation set, which raises a methodological concern: were the rules discovered from understanding the phenomenon, or were they engineered to exploit the specific quirks of AlpacaEval's judge? AlpacaEval uses an LLM judge (GPT-4-turbo) that may have particular biases — preferring shorter responses, penalizing certain formatting tokens, etc. The rules happen to produce outputs with properties that AlpacaEval's judge might reward (shorter length due to EOS upweighting; better formatting due to *, -, # boosts; fewer refusals due to I, We, should penalties). The paper doesn't evaluate the rule-based adapter on any non-AlpacaEval metric (e.g., human evaluation, task-specific benchmarks, diversity metrics), so we can't distinguish between "the rules genuinely produce better instruction following" and "the rules exploit the judge's preferences." This is a significant limitation given how much the paper's simplicity claim rests on the rule-based adapter result.

The rule weights were heuristically tuned, and the paper doesn't report a sensitivity analysis. How sensitive is the 24.4% win rate to the specific weights? Could a completely different set of 15 tokens and weights produce similar results? The paper's argument is that the required changes are simple, but without sensitivity analysis, we don't know whether the specific weights in Table 8 are uniquely necessary or just one configuration among many that would work. If the weights need to be precisely tuned to achieve the 24.4% win rate, that actually weakens the simplicity claim (the changes are simple in specification but require careful calibration). If a wide range of weights produce similar results, that strengthens it (the changes are robustly simple).

On the overall implicit instruction tuning framework: The paper synthesizes three experiments into a unified claim: that instruction following is a "default attractor state" that emerges from many kinds of adaptation because the instruction-response mapping is already encoded during pretraining, and adaptation primarily needs to suppress competing non-instruction-following continuations. The response ranking capability experiment (Table 2) provides direct evidence for the "mapping is pretrained" part of this claim. But the "competing continuations" part is more inferred than tested. The paper doesn't directly measure what base models actually assign high probability to instead of the correct response — it doesn't catalog the types of competing continuations (pattern completion, dialogue continuation, refusal, rambling) or measure how different adaptations suppress them. The rule-based adapter's design (penalizing <, I, etc.) suggests the authors have implicit knowledge of these competing continuations, but the paper doesn't systematically analyze them.

The paper's claim that practitioners should safety-test narrow-domain-adapted models "as if they were releasing a general-purpose chatbot" (Section 7) is well-supported by the single-task finetuning results: models trained on poetry or code do follow general instructions for out-of-domain inputs. But the Chess result (which doesn't produce instruction following) suggests a boundary condition that the paper doesn't fully characterize. The claim should be qualified: practitioners should assume their adapted model will follow general instructions unless the adaptation domain has extremely rigid, low-entropy surface statistics that dominate the generation process. Where exactly the boundary lies — between "GSM produces instruction following" and "Chess doesn't" — is not specified.

Overall, the experiments achieve their intended purpose as an empirical investigation and conceptual analysis, but their evidential weight for the simplicity claim is uneven. The response tuning results are robust and well-controlled. The single-task finetuning results are striking but lack some important controls (no-formatting ablation) and analysis depth (similarity-gating demonstrated for only one dataset). The rule-based adapter results are provocative but weakened by the low absolute performance (24.4%), the potential for judge exploitation, and the lack of sensitivity analysis. The paper makes a compelling case that implicit instruction tuning is a real phenomenon worth studying, but the claim that it reveals profound simplicity in the required distributional changes is more suggestive than proven — the rules work, but only somewhat, and we don't know how unique or robust they are.

6. Limitations and Trade-offs

Categorical Impossibility: The Approach Fails on Any Domain Where the Finetuning Surface Statistics Are Too Rigid

The assumption or constraint. The paper's central claim — that single-task finetuning implicitly produces broad instruction following — comes with an unmodeled boundary condition. The Chess finetuning experiment produces a win rate of 2.1% for Llama-2-7B and 6.3% for OLMo-7B-Feb2024 (Table 3), both indistinguishable from the base model's performance (2.4% and 4.7%, respectively). The authors hypothesize this is "due in part to the very low entropy of the beginning sequences of chess games" (Section 5.1, Results), but they do not formalize how low the entropy must be, which statistical properties of the finetuning domain determine whether implicit instruction tuning occurs, or where the boundary lies between the four domains that succeed (GSM, Poetry, MBPP, Recipes) and the one that fails.

The consequence. A practitioner finetuning a model on a domain with rigid, formulaic surface structure — structured data formats, domain-specific notation, templated outputs, or any domain where the first few tokens have near-zero entropy — has no way to predict from this paper whether their model will follow general instructions or collapse into the finetuning behavior on all inputs. The paper provides no diagnostic, no threshold, and no characterization of the failure mode beyond the speculation about "low entropy." This is a significant practical gap because many real-world narrow-domain adaptations involve structured outputs (JSON, SQL, legal templates, medical coding) that might fall on the wrong side of this boundary. The paper's safety recommendation in Section 7 — that practitioners should assume their adapted models will follow general instructions — is therefore overbroad: for Chess-like domains, the opposite assumption holds (the model won't follow general instructions, but will produce only domain-specific outputs), and the paper provides no way to know which regime a given domain falls into.

What evidence exists in the paper. The evidence for this limitation is entirely the Chess negative result (Table 3) and the qualitative example in Figure 4 (where the Chess-tuned model produces only PGN-format chess moves for the prompt "How did US states get their names?"). The paper does not measure the entropy of beginning sequences for any of the five datasets, does not correlate any statistical property of the datasets with their AlpacaEval win rates, and does not test intermediate cases that might help locate the boundary (e.g., domains with moderately low but not extremely low entropy, or structured domains that are linguistically closer to natural instruction following).

Mitigation status. Not addressed. The authors do not propose methods for detecting whether a given finetuning domain will produce implicit instruction tuning, do not suggest regularization strategies to prevent Chess-like collapse, and do not flag this as an open problem for future work in Section 7. The limitation is noted only in passing.

Difficulty Estimation Cost Is Unaccounted for, Yet Central to the Practical Deployment Argument

The assumption or constraint. While this paper does not have an explicit "difficulty estimation" step analogous to the example paper's 2048-sample procedure, it has an analogous hidden cost: the experimental design requires training explicit instruction-tuned models as baselines and running 5 independent finetuning runs with epoch sweeps for every condition. The paper's framework — comparing experimental methods against instruction-tuned models via AlpacaEval — provides evidence for why implicit instruction tuning occurs, but it does not provide a deployable recipe that a practitioner could use without access to these expensive baselines. A practitioner who wants to know whether their specific adaptation has produced implicit instruction tuning would need to train a comparable instruction-tuned model (on the same base model, with the same hyperparameter sweep, on a diverse instruction dataset like LIMA) and run AlpacaEval head-to-head comparisons — a procedure that costs exactly as much as explicit instruction tuning itself. The validation set used for hyperparameter selection (56 instructions, Appendix D) was constructed partly by hand and partly by GPT-4, and its generalizability to arbitrary domains is unestablished.

The consequence. The paper's findings are retrospective and analytic — they explain why instruction following emerged from methods that were already applied — but they do not provide a predictive tool for a practitioner deploying a new adaptation. If I finetune a model on a novel narrow domain (e.g., legal document summarization), the paper gives me no cheap, reliable method to determine whether implicit instruction tuning has occurred, whether my model will follow general instructions on out-of-domain inputs, or how to adjust my finetuning procedure to control the degree of implicit instruction tuning. The paper's safety recommendation (Section 7) to "test as if releasing a general-purpose chatbot" implicitly acknowledges this — the testing is the only way to know, and the paper doesn't make that testing cheaper than the thing it's testing for.

What evidence exists in the paper. The evidence for this limitation is structural: the paper never reports the compute cost of its own experimental protocol (GPU hours for 5 seeds × 5 epoch choices × 2 model families × 7 adaptation methods = 350 finetuning runs, plus AlpacaEval inference for all of them), and never proposes a lightweight alternative to full instruction-tuning-and-comparison for detecting implicit instruction tuning. The validation set (Appendix D) is small (56 instructions) but was constructed through manual effort and GPT-4 interaction; scaling this approach to new domains would require similar manual investment.

Mitigation status. Not addressed. The paper does not frame this as a limitation, does not propose lightweight detection methods, and does not discuss the cost of its own evaluation framework as a barrier to practical adoption. The rule-based adapter (Section 6) partially addresses a related but distinct concern — it shows instruction following without training — but it doesn't solve the detection problem for a practitioner with an arbitrary adaptation method.

The Rule-Based Adapter Does Not Rule Out Judge Exploitation, and Its Low Absolute Performance Weakens the Simplicity Claim

The assumption or constraint. The rule-based adapter achieves a 24.4% win rate against instruction-tuned Llama-2-7B (Table 4), which the paper presents as evidence that "very simple changes in conditional distributions can cause a language model to follow instructions" (Section 1). However, the evaluation framework is the AlpacaEval LLM-as-a-judge system, which uses GPT-4-turbo (or a similar model) to compare outputs. The rules in the adapter were heuristically tuned on a validation set evaluated by the same judge framework (against GPT-3.5-turbo). The specific rules — upweighting EOS (shorter responses), boosting formatting tokens like *, -, #, ! (structured, assistant-like formatting), and penalizing refusal-associated tokens like I, We, should — produce outputs with surface properties that an LLM judge might systematically prefer, independent of whether the outputs are actually better instruction following.

The consequence. We cannot distinguish between two interpretations of the 24.4% win rate: (1) the rules genuinely improve instruction following by correcting distributional flaws in the base model, or (2) the rules exploit surface-level preferences of the LLM judge (preferring shorter, better-formatted, non-refusal outputs that are nonetheless factually wrong or unhelpful). The qualitative examples in Figure 6 provide suggestive evidence for the concern: the kickball explanation is coherent and helpful, but the Broadway actors response contains demonstrable factual errors (listing Hollywood films as Broadway productions), and the US states response is mostly irrelevant filler. An LLM judge might still prefer these outputs over the base model's rambling continuations, producing a high win rate, while a human evaluator would find them unacceptable. If the rules primarily exploit judge preferences, then the central simplicity claim — that the distributional changes needed for instruction following are genuinely simple — is substantially weaker than the paper suggests.

What evidence exists in the paper. The paper reports only AlpacaEval win rates for the rule-based adapter. There is no human evaluation, no task-specific benchmark (e.g., factuality metrics, MMLU, HellaSwag), and no diversity-based metric (e.g., distinct-n, self-BLEU) that would help disambiguate judge exploitation from genuine quality improvement. The ablation in Table 4 removes each rule individually, showing that all three contribute to the win rate, but this doesn't address whether the contribution is to genuine quality or to judge preference alignment. The authors' statement in Figure 6's caption — "we did not cherry-pick, and many other responses are more reasonable" — acknowledges variability in output quality but doesn't address the systematic concern about the metric. The fact that the rules were "heuristically tuned" on the validation set using the same judge framework (Appendix D confirms the validation set uses AlpacaEval against GPT-3.5-turbo) means the tuning process could have inadvertently optimized for judge preferences rather than instruction-following quality.

Mitigation status. Not addressed. The paper does not report any non-AlpacaEval evaluations for the rule-based adapter, does not discuss the risk of judge exploitation, and does not propose alternative evaluation frameworks. This is a significant methodological gap, particularly because the rule-based adapter is the paper's primary constructive evidence for the simplicity claim, and the claim's strength depends on the adapter producing genuine instruction following, not just judge-pleasing surface features.

The Paper Provides No Characterization of How Response Quality or Dataset Properties Affect Implicit Instruction Tuning

The assumption or constraint. Response tuning (Section 4) uses the LIMA dataset, which was explicitly curated for high-quality, diverse instruction-following responses. The paper does not test response tuning with responses of different quality levels, different diversity, different lengths, or different domains. The findings are conditional on the specific properties of LIMA's response distribution, but which of those properties are necessary for implicit instruction tuning to emerge is unexplored. Similarly, single-task finetuning (Section 5) produces varying win rates across datasets (GSM: 23.7%, Poetry: 22.9%, MBPP: 16.9%, Recipes: 14.6%, Chess: 2.1% for Llama-2-7B; Table 3), but the paper provides no systematic analysis of what dataset properties (vocabulary overlap with natural language, response length, structural rigidity, presence of explanatory content) predict the win rate. The analysis in Section 5.2 is performed only for GSM and only examines instruction similarity, not dataset properties.

The consequence. A practitioner cannot use this paper to design a narrow-domain finetuning dataset that maximizes, minimizes, or controls the degree of implicit instruction tuning. If you want to finetune a model on, say, product descriptions and you want it to NOT follow general instructions on dissimilar inputs, the paper provides no guidance on what dataset properties would suppress implicit instruction tuning. Conversely, if you want to maximize it, you don't know whether to prefer verbose, explanatory outputs (like GSM), linguistically rich outputs (like Poetry), or some other property. The paper establishes that implicit instruction tuning can happen, but not when or why some datasets produce more of it than others. The variation across datasets (14.6% to 23.7% for the four successful ones) is substantial — a 9-percentage-point spread — and unexplained.

What evidence exists in the paper. The evidence is the dataset-level win rates in Table 3 and the qualitative examples in Figure 4, which show different "remnant" behaviors for each dataset (GSM's #### 1 ending, Recipe's leading hyphen, Poetry's author-bylines-as-hallucination). The instruction-response similarity analysis (Figure 5) characterizes how a single dataset's behavior generalizes across instruction types, but it doesn't compare across datasets. The paper does not compute any dataset-level statistics (vocabulary size, type-token ratio, average response length, n-gram entropy, embedding diversity) and correlate them with AlpacaEval win rates. The contrast between Chess and the other four datasets is attributed to "low entropy of the beginning sequences" but no entropy measurement is provided.

Mitigation status. Not addressed. The paper does not propose a framework for predicting which dataset properties promote or suppress implicit instruction tuning, does not systematically vary dataset properties in a controlled experiment, and does not flag this as a key open question in Section 7. The limitation is implicitly acknowledged in the paper's qualitative treatment of the variation across datasets but is never formalized.

The Single-Task Finetuning Analysis Relies on Only One Dataset for the Similarity-Gating Investigation

The assumption or constraint. The instruction-response similarity analysis in Section 5.2 — which provides the paper's most detailed mechanistic picture of how single-task finetuning generalizes — is performed only for the GSM dataset. The paper computes Nomic embedding similarities between instructions and GSM training data, and between generated responses and GSM vs. LIMA responses, and plots these against each other in Figure 5. This analysis reveals a smooth, similarity-gated generalization pattern: instructions highly similar to GSM elicit full GSM-style responses; dissimilar instructions elicit general responses with only surface-level GSM remnants (the #### 1 ending). The paper presents this as the characteristic behavior of single-task finetuned models, but it provides no equivalent analysis for Poetry, MBPP, or Recipes.

The consequence. We do not know whether similarity-gated generalization is a universal property of implicit instruction tuning or a specific artifact of GSM's distinctive characteristics. GSM outputs have a unique, easily-recognizable format marker (<<...>> for calculations, #### for final answers) that makes "GSM-ness" straightforward to measure in generated responses. Poetry outputs may not have analogous format markers; MBPP outputs are Python code, which is syntactically discontinuous from natural language; Recipes outputs have a structured format (bulleted ingredients list) that may behave differently. If similarity-gated generalization is NOT universal, the paper's characterization of single-task finetuning as producing "neither the finetuned behavior, nor the base model's behavior, but instead a general instruction following behavior" (Section 5.2) is over-generalized from a single dataset. If the mechanism is different for other datasets, the paper's unifying framework is weaker than it appears.

What evidence exists in the paper. Only Figure 5 and the surrounding discussion in Section 5.2, which is exclusively about GSM. The qualitative examples in Figure 4 provide anecdotal support for different generalization patterns across datasets (e.g., Recipe-tuned models produce a leading hyphen for state name etymology questions), but there is no systematic measurement of similarity-gating for any dataset other than GSM. The paper does not report Nomic embedding similarities or response-format analyses for Poetry, MBPP, or Recipes.

Mitigation status. Partially addressed by implication. The qualitative examples in Figure 4 are consistent with similarity-gated generalization — the Recipe-tuned response for "How did US states get their names?" includes a recipe-like leading hyphen but mostly produces general-purpose content, while the Chess-tuned response produces only chess moves, consistent with the hypothesis that extreme domain rigidity overrides the gating mechanism. But this is anecdotal; the paper never claims to have verified similarity-gating for non-GSM datasets, and it doesn't flag the GSM-only analysis as a limitation. The failure to replicate the analysis across datasets is a gap in the paper's evidentiary support for its central mechanistic claim.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the understanding of instruction following from an engineered outcome — something you deliberately build by training on instruction-response pairs — to a default attractor state — something that emerges whenever a pretrained model's distribution is perturbed in almost any way that favors desirable outputs. This is not an incremental refinement of instruction tuning methodology. It is a reframing of what adaptation does and what pretraining already accomplishes.

The magnitude of the shift is best understood by comparing against the trajectory of prior work. The field moved from large instruction datasets (52,000+ examples; Taori et al., 2023) to small curated sets (1,000 examples; Zhou et al., 2023) to a handful of few-shot in-context examples (Lin et al., 2024). Each step reduced the explicit supervision while maintaining the assumption that the instruction-response mapping was being taught. This paper breaks that assumption entirely: response tuning (Section 4) shows zero instruction-response pairs suffice, single-task finetuning (Section 5) shows zero intention to produce instruction following suffices, and the rule-based adapter (Section 6) shows zero training of any kind suffices. The mapping from instructions to responses is not what instruction tuning teaches — it is already encoded during pretraining, as directly evidenced by the response ranking capability (Table 2): pretrained Llama-2-7B ranks the correct response above a random other response for 80.4% of instruction pairs, compared to 77.4% for the instruction-tuned model.

This reframing resolves the apparent contradiction between work showing that self-correction and instruction tuning "work" (Zhou et al., 2023; Ouyang et al., 2022) and work showing that very little explicit supervision is needed (Lin et al., 2024; Kung & Peng, 2023). These were not contradictory findings about the amount of teaching required, but evidence for a single underlying reality: pretraining already does the teaching, and adaptation primarily functions to make the taught responses probable enough to be generated. The paper's three experimental paradigms — response tuning, single-task finetuning, and rule-based adaptation — all work because they all increase the probability of some class of outputs (desirable responses, narrow-domain outputs, or outputs satisfying surface heuristics), and the pretrained model already knows which of those outputs correspond to which instructions.

This reframing redirects research attention from two questions toward one:

• Less attractive: "How do we teach models the instruction-response mapping more efficiently?" The response ranking capability experiment (Table 2) shows this question is largely moot — the mapping is already present at 74–80% accuracy in pretrained models. Further improvements to mapping accuracy may yield diminishing returns, and instruction tuning itself does not improve mapping accuracy (it slightly degrades it for Llama-2-7B).

• Less attractive: "How do we cover more instruction diversity in our adaptation data?" Single-task finetuning shows that training on narrow-domain data (Python code, poems, recipes) still produces broad instruction following for out-of-domain instructions. Coverage of instruction diversity appears unnecessary for surfacing instruction following, provided the adaptation domain doesn't have such rigid surface statistics that it dominates all generation (as with Chess, where the win rate is 2.1%, indistinguishable from the base model's 2.4%).

• More attractive: "What are the competing continuations that suppress instruction following in pretrained models, and how do we suppress those suppressors?" The rule-based adapter provides a concrete taxonomy of these suppressors: the model generates more dialogue turns (penalized by suppressing <, |, _<), refuses to answer (penalized by suppressing I, We, should), rambles without structure (penalized by upweighting EOS and boosting formatting tokens like *, -, #), and degenerates into repetition (penalized by the diversity rule). These are the failure modes that prevent pretrained models from following instructions, and the rule-based adapter shows they can be addressed by modifying only 15 token probabilities plus two additional heuristics. This is a concrete, actionable research agenda: catalog the distributional defaults of pretrained models that compete with instruction following, and develop minimal interventions to suppress them.

The paper also introduces a new category of safety concern: implicit instruction tuning as unintended generalization. The standard safety worry for narrow-domain finetuning is that the model fails to generalize — it produces the finetuning behavior for all inputs, including ones where that behavior is inappropriate. This paper shows the opposite worry is equally real: the model succeeds at generalizing to instruction following when you didn't intend it to. A model finetuned on poetry to generate poems for a creative writing application will, when asked for a recipe or a biography or medical advice, produce general-purpose instruction-following responses — not poems. As the paper states in Section 7, practitioners "should not assume that the model will exhibit that tasks' behavior on inputs dissimilar to those trained on. Instead, they should put it through testing and safety trials as if they were releasing a general-purpose chatbot." This inverts the safety paradigm: the danger is not that adaptation fails, but that it implicitly succeeds at producing a general-purpose assistant that the deployer did not intend to create and did not safety-test as such.

The rule-based adapter result (24.4% win rate with three hand-written rules) establishes a new lower bound on the complexity of interventions that produce instruction following. This has a methodological implication: future work proposing instruction tuning methods should benchmark not just against other training methods, but against this rule-based lower bound. If a new method achieves, say, a 30% win rate with thousands of training examples and careful data curation, but three hand-written heuristics achieve 24.4% with zero training, the method's marginal contribution is only 5.6 percentage points — and the experiment should be interpreted accordingly. The rule-based adapter is not a practical method, but a diagnostic baseline that calibrates how much instruction following is "free" from simple distributional changes versus how much requires genuine learning.

Follow-Up Research This Work Enables

Systematically cataloging the competing continuations that suppress pretrained instruction following. The rule-based adapter's three rules — EOS upweighting, 15 specific token modifications, and a repetition penalty — were discovered through manual tinkering on a 56-instruction validation set. They address specific failure modes: rambling, dialogue continuation, refusal patterns, formatting poverty, and token degeneration. But these were identified heuristically, not systematically. A strong follow-up would take a pretrained model, generate outputs for a diverse set of instructions, and taxonomize the continuations the model actually produces when it fails to follow instructions: what fraction are pattern completions (continuing the instruction text), dialogue continuations (generating another <|user|> turn), refusals ("I cannot answer that"), off-topic rambling, or degenerate repetition? For each failure mode, what token-level interventions suppress it? The paper's Rule 2 modifies 15 tokens; a systematic analysis might identify 50, 200, or 500 tokens that, when penalized or boosted, further close the gap toward explicit instruction tuning. This work is newly tractable because the rule-based adapter provides a framework (product-of-experts with per-token additive scores) for testing token-level interventions in isolation, and the AlpacaEval head-to-head framework provides a cheap, reproducible metric for measuring improvement.

Stress-testing the boundary between implicit instruction tuning and domain collapse using controlled synthetic datasets. The paper identifies one failure case — Chess finetuning, where the model never follows instructions and always generates chess moves — and attributes it to "low entropy of the beginning sequences." But this is a one-datapoint boundary. A rigorous follow-up would construct a family of synthetic finetuning datasets that systematically vary properties hypothesized to determine whether implicit instruction tuning occurs: (a) the entropy of the first token distribution, (b) the vocabulary overlap between domain outputs and general instruction-following responses, (c) the structural rigidity of outputs (e.g., strict templates versus free-form text), and (d) the diversity of the input distribution. For each dataset, finetune a model and measure both in-domain task accuracy and out-of-domain instruction-following on AlpacaEval. The output would be a phase diagram showing which dataset properties produce the three observed regimes: domain collapse (always generates domain outputs, like Chess), similarity-gated generalization (domain behavior only for similar instructions, like GSM), and uniform instruction following (always follows instructions, with no domain remnants). This would convert the paper's qualitative observation into a predictive framework, enabling practitioners to design finetuning datasets that either maximize or minimize implicit instruction tuning as desired.

Measuring whether the rule-based adapter's gains reflect genuine quality improvement or LLM-judge preference exploitation. The 24.4% win rate for the rule-based adapter is measured by AlpacaEval, which uses an LLM judge (GPT-4-turbo). The rules were heuristically tuned on a validation set evaluated by the same judge framework. A critical follow-up would evaluate the same rule-based adapter outputs using (a) human evaluation on a sample of AlpacaEval prompts, measuring both overall quality and specific dimensions (factual accuracy, helpfulness, conciseness), (b) task-specific benchmarks that are not judge-based (e.g., MMLU accuracy, TruthfulQA, HellaSwag), and (c) diversity metrics like distinct-n and self-BLEU to detect whether the rules simply collapse outputs to a narrow set of judge-pleasing patterns. If the rule-based adapter's human-evaluated quality is substantially lower than its AlpacaEval win rate would suggest — for instance, if humans rate it at 10% win rate rather than 24.4% — then the paper's simplicity claim is weakened: the rules produce judge-pleasing surface features rather than genuine instruction following. If human and judge evaluations are well-calibrated, the simplicity claim is strengthened. Either outcome is informative for the field's reliance on LLM judges for instruction-following evaluation.

Replicating implicit instruction tuning on models with verified-no-instruction-tuning pretraining at larger scales. The paper uses OLMo-7B-Feb2024 as a control for pretraining contamination, since its training data (Dolma) intentionally excluded instruction-tuning data. But OLMo-7B-Feb2024 is a 7B model with relatively weak performance. A strong replication would test the same experiments — response tuning, single-task finetuning, and the rule-based adapter — on a larger model (30B–70B parameters) whose pretraining data is fully documented and verified to exclude instruction tuning. Does implicit instruction tuning become more pronounced at larger scales (because larger models learn better instruction-response mappings during pretraining even from non-instruction data)? Does the response ranking capability increase with model scale? Does the rule-based adapter's win rate scale with model capability, or does it remain fixed around 24% because the rules address surface-level issues that are scale-invariant? This replication would clarify whether implicit instruction tuning is a fundamental property of autoregressive language models trained on diverse text, or an artifact of specific pretraining data mixtures or model scales. The OLMo model family (with its documented Dolma pretraining data) makes this replication straightforward.

Combining the rule-based adapter with minimal supervised finetuning to isolate what semantic knowledge adaptation adds. The rule-based adapter achieves a 24.4% win rate with zero training by addressing surface-level distributional flaws. Response tuning — which trains on 1,030 desirable responses without instructions — achieves 43.3%. Instruction tuning — which trains on the same 1,030 examples with instructions — achieves 50% (by definition, as the comparison baseline, though absolute quality is presumably higher than 50% if compared against a weaker baseline). The gap between 24.4% and 43.3% (18.9 percentage points) represents what training on the response distribution adds beyond surface-level distributional corrections. The gap between 43.3% and 50% (or the absolute improvement of instruction tuning over response tuning) represents what conditioning on instructions adds. A clean follow-up would start with the rule-based adapter applied to the pretrained model, then add increasing amounts of supervision: (a) rules alone, (b) rules + response tuning on N responses, (c) rules + instruction tuning on N instruction-response pairs. By varying N, this experiment would decompose the total improvement from adaptation into three components: surface corrections (captured by the rules), response distribution learning (captured by response tuning), and instruction-response mapping refinement (captured by the residual from instruction tuning). This decomposition would directly quantify what the paper currently only argues qualitatively: that the mapping is mostly pretrained and that adaptation primarily teaches the response distribution.

Developing a cheap, predictive test for whether a given adaptation will produce implicit instruction tuning. The paper's evaluation requires training a comparable instruction-tuned model and running AlpacaEval head-to-head comparisons — a procedure as expensive as explicit instruction tuning itself. This makes the framework retrospective (it explains why an already-applied adaptation produced instruction following) rather than prospective (it predicts whether a planned adaptation will do so). A practical follow-up would develop a lightweight diagnostic: given a base model and a finetuning dataset, can we predict the AlpacaEval win rate after finetuning without actually training the instruction-tuned baseline? Candidate diagnostics include: (a) the response ranking capability (Table 2) measured for the specific base model before adaptation — a high ranking capability may be a prerequisite for implicit instruction tuning, (b) the KL divergence between the finetuning domain's output distribution and some reference distribution of general instruction-following responses, (c) the first-token entropy of the finetuning domain's outputs (to predict Chess-like collapse), or (d) the performance of the rule-based adapter on the base model (if the rules already achieve a high win rate, perhaps explicit finetuning adds little, and the risk of unintended generalization is higher). A validated diagnostic would allow practitioners to assess implicit instruction tuning risk before deployment, directly enabling the safety recommendation in Section 7 without the prohibitive cost of training a full instruction-tuned comparison model for every adaptation.

Practical Applications and Downstream Use Cases

Low-cost deployment of general-purpose chatbots from narrow-domain adaptations. A practitioner who finetunes a model on a specific task — say, generating product descriptions from structured attributes — may discover that their model already functions as a general-purpose chatbot for many user queries, with no additional instruction-tuning cost. The single-task finetuning results show that models trained on GSM (math word problems), Poetry, MBPP (code generation), or Recipes all follow general instructions with win rates of 14.6% to 23.7% against a fully instruction-tuned model (Table 3). For a deployment where the primary use case is narrow (generating product descriptions) but users will inevitably ask off-task questions ("what is the return policy?", "tell me about your company"), the narrow-domain adaptation may already handle those queries without explicit instruction tuning. The practical benefit is eliminating the need to curate, train on, and maintain a separate broad-domain instruction dataset when the narrow-domain data already implicitly instruction-tunes the model. The paper's safety caveat applies: the deployer must still safety-test the model as a general-purpose chatbot, because it will generate answers to off-domain questions, and those answers may be harmful or incorrect.

Simplifying the adaptation pipeline for resource-constrained deployments by substituting response curation for instruction diversity. Response tuning (Section 4) achieves a 43.3% win rate against instruction tuning for Llama-2-7B and 43.7% for OLMo-7B-Feb2024 (Table 1), using only 1,030 responses without their corresponding instructions. For a practitioner building a domain-specific assistant (e.g., a medical QA system, a legal document summarizer), the hardest part of instruction tuning is often writing diverse instructions that cover the range of ways users might phrase their queries. Response tuning eliminates this step entirely: the practitioner only needs to curate high-quality responses (which they often already have, e.g., from existing documentation, expert-written answers, or synthetic generation), and finetune the model on those responses alone. The model will then follow instructions — including instructions the practitioner never anticipated — because the instruction-response mapping is already present from pretraining. The practical benefit is a significant reduction in data curation cost and domain expertise required: writing good answers (responses) requires domain knowledge, but writing diverse questions (instructions) requires anticipating user behavior, which is harder and more expensive. Response tuning decouples these, making adaptation accessible to practitioners who have domain expertise but not user-experience research capabilities.

Using the rule-based adapter as a diagnostic for pretrained model readiness. Before investing in instruction tuning for a new base model, a practitioner can apply the three-rule adapter (a ~70-line Python function with zero training) and evaluate the resulting model on their target task. The win rate against a fully instruction-tuned model on that task provides a lower bound on what is achievable with simple distributional corrections, and reveals how much of the model's failure to follow instructions is due to surface-level distributional flaws versus fundamental capability gaps. If the rule-based adapter achieves, say, a 30% task success rate on a code generation benchmark, while the instruction-tuned model achieves 50%, then roughly 20 percentage points of the gap require learning from data, while 30 points were available from the pretrained model with only surface corrections. If the rule-based adapter achieves near-zero performance, the pretrained model may lack the necessary capabilities entirely, and instruction tuning is unlikely to help — pretraining on more relevant data would be a better investment. This diagnostic use case requires no training, only inference with the rule-based product-of-experts, making it a cheap pre-screening step before committing to expensive adaptation pipelines.

Safety auditing of narrow-domain model deployments for unintended general-purpose capability. The paper's central safety finding is that narrow-domain finetuning often produces broad instruction following as an unintended side effect (Section 5). For a company deploying a model finetuned on, say, internal code generation or document classification, this means the deployed model may function as a general-purpose chatbot when users provide inputs outside the intended domain. A practical safety audit, directly motivated by the paper's Section 7 recommendation, would involve: (a) running the adapted model on a diverse set of out-of-domain instructions (the AlpacaEval prompt set, or a domain-appropriate subset), (b) measuring whether the responses are recognizable attempts at instruction following (as opposed to domain-specific outputs or incoherent text), and (c) if instruction following is detected, subjecting the model to the same safety testing (red-teaming, harmful output filtering, refusal rate measurement) that would be applied to a general-purpose chatbot. The Chess result (2.1% win rate) provides a boundary condition: if the domain outputs have extremely rigid surface statistics, implicit instruction tuning may not occur, and this audit may be unnecessary. But for most natural-language-output domains (text generation, summarization, Q&A, creative writing), the paper's results suggest implicit instruction tuning is the default, and the safety audit is essential. The audit procedure is directly specified by the paper's evaluation framework: compare the adapted model against an instruction-tuned baseline using AlpacaEval (or a similar judge-based framework) on a diverse prompt set, and if the win rate is substantially above the base model's floor (2–5%), the model has implicitly instruction-tuned and must be treated as a general-purpose system.