2 OLMo 2 Furious

ArXiv: 2501.00656

🎯 Pitch

Late-stage curriculum training with a specialized data mix boosts OLMo 2 to match or beat Llama 3.1 and Qwen 2.5 while using far fewer FLOPs, all with fully open training data and code. The 7B model delivers better benchmark scores than Llama 3.1 8B at a quarter of the compute cost.

1. Executive Summary

This report introduces OLMo 2, a family of fully open 7B, 13B, and 32B parameter language models trained on up to 6T tokens, alongside their instruction-tuned counterparts OLMo 2-Instruct. The paper details a multi-stage training recipe—pretraining on web-scale data followed by a specialized mid-training phase using Dolmino Mix 1124 (a curated mixture of high-quality web text, academic references, and synthetic math data introduced via late-stage curriculum annealing)—combined with architecture modifications for pretraining stability (reordered normalization, QK-norm, z-loss regularization) and a post-training pipeline adapted from Tülu 3 that incorporates reinforcement learning with verifiable rewards (RLVR) for multi-stage policy optimization. On English academic benchmarks, OLMo 2 base models sit at the Pareto frontier of performance relative to pretraining FLOPs, often matching or outperforming open-weight models like Llama 3.1 and Qwen 2.5 while using fewer FLOPs (e.g., the 7B model at 1.8×10²³ FLOPs achieves 62.9% average on OLMES versus Llama 3.1 8B's 61.8% at 7.2×10²³ FLOPs), and the resulting OLMo 2-Instruct models are competitive with GPT-3.5 Turbo and GPT-4o Mini, establishing that fully open models with transparent training data, code, and recipes can achieve frontier-competitive performance when mid-training curricula and stability-focused architecture choices are applied systematically.

2. Context and Motivation

The Core Problem: The Openness-Performance Gap in Language Models

In 2024, the language model ecosystem faces a fundamental tension: the models with the best reported performance (GPT-4, Claude, Gemini) are closed systems whose training data, recipes, and intermediate artifacts remain proprietary, while the models released with full transparency—training data, code, checkpoints, and logs—consistently lag behind in capability. This gap is not merely academic. It determines who can study, audit, improve, and build upon these models.

The paper frames this tension explicitly by noting the asymmetry in the landscape. On one side, a "surge in open weights models" from established developers has "substantially clos[ed] the gap between publicly available and closed systems" (Section 1). Llama 3, Qwen 2.5, Gemma 2, Mistral, and others have made powerful model weights available. Yet these releases are only "the final artifacts of sophisticated language model recipes and complex development pipelines," and "by themselves are not sufficient to support diverse forms of research into language model behaviors and uses." You can download the weights, but you cannot see what data trained them, what choices were made during development, or how to reproduce the result.

On the other side, "fully open" efforts—OLMo 1 (Groeneveld et al., 2024), Pythia (Biderman et al., 2023), Amber (Liu et al., 2023c), DCLM (Li et al., 2024), MAP Neo (Zhang et al., 2024a), and SmolLM (Allal et al., 2024a,b)—have released not just weights but training data, training code, and documented recipes. These artifacts have "played a crucial role in studying training dynamics, concept acquisition, and memorization in language models" (Section 1). But the uncomfortable reality is that these fully open models have been substantially weaker than their open-weight-only counterparts on standard benchmarks.

The paper's central problem is therefore: Can a fully open model—with all training data, code, recipes, and logs publicly released—match or exceed the performance of open-weight-only models while remaining fully transparent? This is a question about whether the constraints of openness (permissive licensing, publishable data sources, reproducible training) are fundamentally at odds with achieving state-of-the-art performance, or whether the gap can be closed through careful engineering rather than proprietary data or secret recipes.

Why Full Openness Matters

The paper argues for full openness on several grounds that go beyond the usual "reproducibility" argument. Prior fully open models like OLMo 1 and Pythia have enabled specific research directions that would be impossible with weights-only releases:

• Training dynamics (Land and Bartolo, 2024; Jin and Ren, 2024): Researchers can study how capabilities emerge during training by examining intermediate checkpoints. Without access to training data, it is impossible to correlate performance improvements with specific data exposures.

• Concept acquisition (Chang et al., 2024): Understanding when and how models learn factual knowledge requires knowing both the training data and the model's state at each step. An open-weight model provides the latter but not the former.

• Memorization studies (Antoniades et al., 2024; Shaib et al., 2024): Investigating what models memorize and why requires identifying whether specific outputs are present in training data—possible only when training data are available.

• Safety and auditability: Models used in high-stakes applications need external scrutiny. A model whose training data is unknown cannot be fully audited for biases, toxic content, or copyright concerns.

These research imperatives create pressure to close the openness-performance gap: the field needs models that are both capable enough to be practically useful AND transparent enough to be scientifically interrogated. If fully open models remain permanently behind, researchers studying model behavior are forced to study weaker models that may not represent the frontier, limiting the generalizability of their findings.

Where Prior Approaches Fall Short

The paper identifies specific limitations in the prior OLMo-0424 release (Ai2, 2024), which represented the team's previous best effort at a fully open model, as well as in other open efforts like DCLM and MAP Neo. These limitations fall into several categories:

1. Training Stability at Scale

OLMo-0424's training was characterized by concerning dynamics that the paper describes in detail (Section 3):

"Sudden spikes in the loss, and more frequently, in the gradient norm during training. In experiments, we found that increasing model size increased the frequency of spikes. Furthermore, our experiments revealed that more dramatic spikes in gradient norm often preceded training loss spikes."

These instabilities were not merely cosmetic. The paper reports that "a combination of these issues would lead to training divergence, making training at larger scales impossible" (Section 3). This directly limited the team's ability to scale up—the 13B and 32B models in OLMo 2 would not have been trainable with the OLMo-0424 recipe. The instability manifested as a "slow growth in the magnitude of the gradient norm over the training run" that was "correlated with increasing frequency of spikes in the gradient norm (and training loss)."

This problem is not unique to OLMo. The broader literature documents similar instabilities across model families, but solutions are often kept proprietary or described only vaguely in technical reports. The paper notes that techniques like QK-norm (Dehghani et al., 2023b), z-loss (Chowdhery et al., 2022), and reordered normalization (Liu et al., 2021) have been proposed in the literature, but their systematic application and interaction in a fully open training pipeline has not been demonstrated.

2. Data Quality and Specialized Knowledge Injection

OLMo-0424 and other open models trained on web-scale data show clear capability gaps in specialized domains, particularly mathematics. The paper demonstrates this concretely in Table 9: after pretraining on 4 trillion tokens, the OLMo 2 7B model achieves only 24.1% on GSM8K, a grade-school math benchmark. This is not a fundamental limitation of the model architecture or scale—it reflects the fact that web-scale pretraining data, even when filtered for quality, does not provide sufficient exposure to the structured reasoning patterns needed for mathematical problem-solving.

Prior approaches to injecting specialized knowledge have included:

• Upsampling domain-specific sources during pretraining (e.g., including more math web pages in the main mix). But this approach means the specialized data is diluted across the entire training run and must compete with the vastly larger volume of general web text.
• Fine-tuning on task-specific data after pretraining. This can improve performance on the target task but often degrades general capabilities—the model "forgets" what it learned during pretraining.
• Using proprietary models to generate synthetic data. This is effective but breaks the fully open paradigm if the generation process relies on closed models and that generated data cannot be released.

The paper's observation that "early mid-training mixes show models struggle in math-related benchmarks" (Section 4.4) reflects the fact that simply adding math data to a general mix is insufficient if the data is not carefully curated and presented at the right stage of training.

3. The Absence of a Systematic Mid-Training Methodology

The concept of multi-stage pretraining—where training transitions from a broad web-scale mix to a specialized, high-quality mix—had been demonstrated in several proprietary systems:

• DBRX (Databricks, 2024) used curriculum learning during pretraining.
• Llama 3 (Grattafiori et al., 2024) employed annealing on high-quality data mixtures.
• Phi-3/4 (Abdin et al., 2024a,b) used extensive synthetic data generation and multi-stage training.

However, the methodology for how to construct these mid-training mixtures, how to evaluate data sources independently, and how to combine them with learning rate schedule choices was not well-documented in the open literature. The paper notes that "data sources for mid-training can be independently assessed to reduce experimentation cost" through what they call "micro-annealing" (Section 4), but this technique had not been formalized or described in sufficient detail for others to replicate. Without such methodology, open model developers faced an expensive trial-and-error process: train a full model, observe poor math performance, guess at what data might help, retrain, and repeat.

4. Post-Training Pipeline Integration

OLMo-0424 had only a basic supervised fine-tuning stage. The subsequent Tülu 3 work (Lambert et al., 2024) demonstrated a much more sophisticated post-training pipeline—supervised fine-tuning, on-policy preference tuning with DPO, and reinforcement learning with verifiable rewards (RLVR)—that substantially improved downstream performance. But Tülu 3 was developed and validated primarily on the Llama 3.1 model family. The paper needed to determine whether this recipe would transfer to the OLMo architecture and data distribution, and whether OLMo 2 base models would serve as effective starting points for such multi-stage post-training.

This matters because a base model's utility is measured not just by its raw benchmark scores but by its ability to be adapted to downstream tasks. A model that performs well on pretraining metrics but is resistant to instruction tuning would be a practical failure regardless of its benchmark numbers.

5. Infrastructure Transparency

The paper observes that "many pretraining papers do not discuss their training stack, or gloss over crucial details" (Section 6). This creates a hidden barrier to reproduction: even when model weights, data, and code are released, the operational knowledge required to actually train the model—cluster configuration, failure recovery strategies, throughput optimization techniques—remains tacit. The prior OLMo releases had begun to address this by documenting their infrastructure, but OLMo 2 represents a substantially larger engineering effort (two separate clusters with different hardware profiles, new software infrastructure) that required deeper documentation.

How This Paper Positions Itself

OLMo 2 positions itself not as a single breakthrough but as a systematic engineering advancement that demonstrates the frontier capabilities achievable through fully open development. The paper explicitly builds on prior work across multiple axes:

Relative to OLMo 1 and OLMo-0424, the paper presents OLMo 2 as the next iteration in a continuous development trajectory. It inherits the commitment to full openness (Apache 2.0 licensing, full data releases, training logs, intermediate checkpoints) while addressing the specific limitations—training instability, domain specialization, post-training integration—that prevented prior versions from being competitive with open-weight-only models. The architecture changes (RMSNorm, reordered norm, QK-norm, z-loss) are presented as an evolution responding to empirical findings, not a wholesale redesign.

Relative to open-weight-only models (Llama 3.1, Qwen 2.5, Gemma 2), the paper positions OLMo 2 as competitive in performance while superior in transparency. Figure 1 is the key visual argument: OLMo 2 models sit on or near the Pareto frontier of performance vs. pretraining FLOPs, often requiring fewer FLOPs to achieve comparable performance (e.g., OLMo 2 7B uses 1.8×10²³ FLOPs vs. Llama 3.1 8B at 7.2×10²³). The paper does not claim to surpass these models on all metrics—Table 6 shows a mixed picture where OLMo 2 leads on some benchmarks and trails on others—but rather claims to achieve the same tier of capability without the opacity.

Relative to other fully open models (DCLM, MAP Neo, Amber, SmolLM), OLMo 2 represents a substantial step forward. Table 6 shows the gap clearly: OLMo 2 7B achieves 62.9% average on OLMES versus 56.9% for DCLM 7B (the next best fully open model at that scale) and 50.7% for OLMo 7B 0424. This improvement reflects the cumulative effect of the stability fixes, mid-training data, and post-training pipeline, not any single innovation.

Relative to post-training research (Tülu 3), the paper positions OLMo 2 as validation that the Tülu 3 recipe generalizes across model families, and extends it with multi-stage RLVR training. The finding that "all of the stages in the Tülu 3 Recipe easily translate to the OLMo 2 models" (Section 5) is not a given—post-training recipes are often brittle to model architecture and pretraining data differences. The paper's demonstration that OLMo 2-Instruct models can match GPT-3.5 Turbo and GPT-4o Mini (Table 7) on certain benchmarks validates both the base model quality and the post-training pipeline.

A key implicit positioning decision: The paper does not attempt to build the single best model. It builds a family (7B, 13B, 32B) trained with a reproducible recipe, and it emphasizes that this recipe was designed to be shared and iterated upon, not to win a leaderboard. This is reflected in the decision to maintain held-out evaluation tasks (the "unseen evaluation suite") that were not used for model development decisions, and in the call for "standard practice of declaring development vs held-out evaluation tasks for model developers" (Section 2.5). This is a positioning move: the paper argues that the field needs better evaluation norms, not just better models, and OLMo 2 models the behavior it advocates.

The Mid-Training Insight as Motivational Core

While the paper covers architecture, stability, infrastructure, and post-training, the deepest motivational thread concerns mid-training. The observation that a model after 4 trillion tokens of pretraining scores 24.1% on GSM8K, but after just 50 billion carefully chosen mid-training tokens scores 67.5% (Table 9)—a 43.4 percentage point improvement from only 1.25% additional training—is the paper's most striking empirical finding. It suggests that the standard pretraining paradigm of uniform data mixing across the entire training run leaves substantial capability on the table, and that a small amount of targeted, high-quality data at the right phase of training can dramatically reshape model behavior.

This finding motivates much of the paper's detailed discussion of data curation (Sections 4.3–4.4), the micro-annealing methodology (Section 4.4.2), and model souping (Section 4.5). The implicit argument is that for the open model community to close the gap with proprietary systems, the key bottleneck is not compute scale but data strategy at the mid-training stage—knowing what specialized data to use, how to evaluate it efficiently, and how to schedule its introduction. Proprietary developers have likely discovered similar principles (as suggested by the Llama 3 and Phi-3/4 reports), but by documenting and releasing their methodology, the OLMo 2 team makes these techniques accessible to the broader research community.

3. Technical Approach

3.1 Reader Orientation

This paper is an empirical engineering paper whose core idea is that a fully open language model can achieve competitive frontier performance through systematic improvements across three interconnected stages: (1) architecture modifications that ensure stable training at scale, (2) a staged training recipe where late-stage exposure to carefully curated specialized data dramatically improves specific capabilities (particularly math), and (3) a multi-stage post-training pipeline that includes reinforcement learning from verifiable rewards. The system being built is a family of three dense autoregressive language models (7B, 13B, 32B parameters) plus their instruction-tuned variants, all trained and released with complete transparency—training data, training code, training logs, intermediate checkpoints, and documented recipes—under permissive Apache 2.0 licensing wherever possible.

The problem it solves is the openness-performance gap: prior fully open models (those releasing training data and recipes, not just weights) substantially lagged behind open-weight-only models (like Llama 3.1 and Qwen 2.5) on standard benchmarks, creating a tension between scientific transparency and practical capability. The solution's shape is not a single breakthrough technique but rather a systematic integration of known methods (reordered normalization, z-loss regularization, QK-norm, mid-training curricula, RLVR post-training) applied with careful empiricism—the paper identifies specific failure modes in prior approaches (training instability from repeated n-grams in data, slow gradient norm growth from weight decay on embeddings, poor math performance from web-only pretraining) and addresses each through targeted interventions whose effects are validated through controlled experiments at multiple scales.

3.2 Big-Picture Architecture (Diagram in Words)

The OLMo 2 system has five major components, connected in a sequential pipeline:

1. Data Curation Pipeline — produces two training mixtures: (a) OLMo 2 Mix 1124 (~3.9 trillion tokens of mostly web data, including DCLM-baseline filtered web pages, StarCoder code, academic papers from peS2o and arXiv, and math sources like OpenWebMath) for the initial pretraining stage, and (b) Dolmino Mix 1124 (50B–300B tokens of high-quality web text filtered through FastText and FineWeb Edu classifiers, reference materials, FLAN instruction data, Stack Exchange Q&A pairs, and synthetic math data including TuluMath, Dolmino SynthMath, and TinyGSM-MIND) for the mid-training stage. The curation pipeline includes n-gram repetition filters that remove documents with 32+ repeated n-gram spans.

2. Model Architecture (Transformer) — a decoder-only transformer with specific modifications for training stability: RMSNorm applied to attention/MLP outputs (not inputs), QK-norm (RMSNorm on query and key projections before attention computation), z-loss regularization on the softmax denominator, SwiGLU activations, RoPE with θ=500,000, grouped-query attention for the 32B variant, and a standard initialization (normal distribution, mean=0, std=0.02) replacing the depth-scaled initialization from OLMo-0424.

3. Two-Stage Training Process — Stage 1 (pretraining, 90–95% of FLOPs) runs on OLMo 2 Mix 1124 with a cosine learning rate schedule truncated at 4–6T tokens; Stage 2 (mid-training, 5–10% of FLOPs) linearly decays the learning rate to zero on Dolmino Mix 1124. Multiple independent mid-training runs with different data orders are averaged together (model souping) to produce the final base model.

4. Post-Training Pipeline — adapted from Tülu 3, this has three phases: (a) supervised fine-tuning on a curated mix of 866K–939K instruction prompts, (b) preference tuning with DPO using on-policy data from OLMo 2 SFT checkpoints combined with off-policy data from 20 permissively-licensed models and GPT-4o judgments, and (c) reinforcement learning with verifiable rewards (RLVR) using PPO (7B, 13B) or GRPO (32B) on GSM8K, MATH, and constraint-following prompts, with the 13B model undergoing three sequential RLVR stages (mixed math → GSM8K-only → MATH-only).

5. Evaluation Framework — OLMES, a standardized evaluation suite with 11 base model tasks (5 multiple-choice in both cloze and multiple-choice formulations, 2 generative, 4 held-out) and 10 instruct tasks covering knowledge, reasoning, math, instruction-following, and safety, with explicit separation between development and held-out benchmarks.

Information flows as follows: web-text and curated sources → data curation (filtering, deduplication, n-gram removal) → OLMo 2 Mix 1124 → Stage 1 pretraining (warmup then cosine decay) → base checkpoint → Dolmino Mix 1124 curation (quality filtering via FastText/FineWeb Edu classifiers, synthetic math generation via GPT-4o with persona-driven prompts and MIND rewriting) → Stage 2 mid-training (linear LR decay to zero) → multiple mid-trained checkpoints → model souping (weight averaging) → OLMo 2 base model → SFT on instruction mix → DPO with on-policy + off-policy preferences → RLVR (potentially multi-stage) → OLMo 2-Instruct → OLMES evaluation.

3.3 Roadmap for the Deep Dive

I will explain the technical approach in the following order, which traces the path from training stability (the prerequisite that makes scaling possible) through the two training stages (where capability is built) to post-training (where capability is adapted to downstream use):

• First, the training stability interventions (Section 3 of the paper)—covering n-gram filtering, initialization, architectural modifications (reordered norm, QK-norm, z-loss), and hyperparameter choices (AdamW epsilon, weight decay on embeddings). I explain these first because they are the enabling foundation: without stable training at 13B and 32B scales, nothing else in the paper works. The specific failure modes (loss spikes, gradient norm growth) and their mechanisms are essential context for understanding why the architecture looks the way it does.

• Second, the pretraining recipe (Section 2.3–2.4 of the paper)—the learning rate schedule, token budgets, and data mixture composition. This establishes the baseline from which mid-training improvements are measured and introduces the two-stage paradigm that is the paper's central architectural choice.

• Third, the mid-training methodology (Section 4 of the paper)—covering learning rate annealing experiments, the construction of Dolmino Mix 1124 (high-quality web selection, domain-specific sources, the math mix development), and the micro-annealing technique. This is the paper's most novel contribution and contains the procedure by which a model with poor math capability (24.1% GSM8K) is transformed into a strong math reasoner (67.5% GSM8K) with only 1.25% additional training tokens.

• Fourth, the model souping procedure (Section 4.5)—how multiple independent mid-training runs are averaged to produce the final model, and the empirical justification for doing so. This is a simple technique with surprising effectiveness.

• Fifth, the post-training pipeline (Section 5)—SFT data curation, DPO with on-policy and off-policy data, RLVR with multi-stage training, and hyperparameter selection. This shows how the base model is adapted to instruction-following and how the RLVR signal propagates to improved downstream metrics.

• Sixth, the infrastructure stack (Section 6)—the necessary operational substrate: cluster specifications, Beaker workload management, throughput optimizations (torch.compile, host-device sync elimination, explicit garbage collection). These are not research contributions but are practical requirements for reproduction that the paper documents transparently.

3.4 Detailed, Sentence-Based Technical Breakdown

Training Stability: The Repeated n-Gram Filter

The paper identifies a specific class of data artifacts—documents containing long sequences of repeated n-grams—as a cause of training instability, manifested as gradient norm spikes that could precede loss divergence. The relationship is described as non-deterministic: "The same n-gram sequence may spike for a larger model but not for a smaller model trained on the same data," and "The same n-gram sequence associated with a spike can also be found elsewhere in training batches that did not spike" (Section 3.1). Nevertheless, the aggregate evidence motivates a two-pronged data-side intervention.

At data curation time, the paper applies a filter that removes all documents containing 32 or more repeated n-grams, where an n-gram is defined as any span of 1 to 13 tokens. This filter operates before training begins and permanently excludes flagged documents from the pretraining mix. The threshold of 32 is an empirical choice based on inspection of batches where spikes occurred; the paper presents examples of problematic sequences including hexadecimal-like patterns (g4ODg4ODg4OD...), repeated number sequences (365, 0, 667, 1000, 1000...), and repeated byte values (255, 255, 255...).

At training time, an "additional safeguard in the trainer... detects these sequences during data loading and masks them when computing the loss" (Section 3.1). This means that even if a document passed the preprocessing filter, the training loop can identify and mask repeated n-gram sequences dynamically. Masking the loss means the model does not receive gradient updates from tokens in the flagged spans, preventing the problematic tokens from influencing parameter updates.

Effect. Figure 3 compares the L2 norm of the gradient for runs with and without the repetition filter, showing a dramatic reduction in the frequency and magnitude of gradient spikes. The paper notes this intervention "had no effect on the slow growth in gradient norm" (Section 3.1), meaning it addresses the acute spike problem but not the chronic gradient norm inflation, which requires separate interventions (see below).

Training Stability: Model Initialization

OLMo 2 departs from the depth-scaled initialization used in OLMo-0424, which "scaled input projections by $1/\sqrt{d_{\text{model}}}$, and output projections by $1/\sqrt{2 \cdot d_{\text{model}} \cdot \text{layer\_idx}}$ at every layer" so that "later layers were initialized to smaller values" (Section 3.2). Instead, OLMo 2 initializes every parameter from a normal distribution with mean 0 and standard deviation 0.02. This is a return to the standard initialization commonly used in transformer implementations.

The paper provides both empirical and theoretical justification for this change:

Growth exponent analysis. The paper defines a metric called the growth exponent $\lambda$ to measure how activations and gradients expand or contract across layers:

$\lambda = \frac{1}{n_{\text{layers}}} \log\left(\frac{\|\mathbf{v}'\mathbf{v}'\mathbf{v}'\|}{\|\mathbf{v}\|}\right)$

where $\mathbf{v}$ is the activation or gradient vector at the initial layer, $\mathbf{v}'\mathbf{v}'\mathbf{v}'$ is the corresponding vector at the final layer, and $n_{\text{layers}}$ is the number of transformer layers. The norm $\|\cdot\|$ is the L2 norm. This is computed by passing 50 random documents through a randomly initialized model and averaging activations/gradients across documents and time steps.

What it computes: the per-layer exponential growth rate of the signal magnitude as it propagates through the network. A value of $\lambda=0$ means the signal maintains constant scale; positive values mean exponential growth (exploding signals); negative values mean exponential decay (vanishing signals). The log transform compresses multiplicative growth into an additive per-layer rate.

Why this form: treating the transformer as a linear signal propagation chain (ignoring nonlinearities) predicts that the signal norm $\|\mathbf{v}\|$ approximately multiplies by a constant factor per layer, so $\lambda$ estimates the log of that factor. This is a diagnostic tool, not an optimization objective—it reveals whether initialization causes systematic signal inflation/deflation that will manifest as training instability.

Figure 5 plots $\lambda$ for both activations and gradients across model widths (from small to the full 4096 $d_{\text{model}}$ of the 7B model). Crucially, "the growth exponent for OLMo 2 is closer to 0 than for OLMo-0424 across model widths" (Section 3.2), indicating that both activations and gradients are more resistant to exploding or vanishing.

Hyperparameter transfer analysis. Drawing on Yang et al. (2024b), the paper argues that "a sufficient condition for hyperparameter transfer across width is that the magnitude of each activation scalar value and its update (learning rate times gradient) remain fixed as width increases" (Section 3.2). Figure 6 shows that with the OLMo 2 initialization, the gradient norm positively correlates with $\sqrt{d_{\text{model}}}$, whereas with OLMo-0424 initialization this correlation was absent or negative. Combined with the theory from Yang et al. (2024b), this suggests that "with an initial learning rate independent of model width, the new OLMo 2 initialization will transfer better across different model widths compared to the OLMo-0424 initialization" (Section 3.2).

Empirical spike score. The paper defines a spike score as "the percentage of values in a time series that are at least seven standard deviations away from a rolling average of the last 1,000 values" (Section 3.2). This objective measure quantifies instability rather than relying on visual inspection. In an ablation experiment with a reduced warmup period designed to provoke spikes quickly, the new initialization produced "no loss spikes, and the spike score for the L2 norm of the gradient went from 0.40 to 0.03" (Section 3.2). The tradeoff is slightly slower initial convergence, which the paper compensates for through other hyperparameter adjustments.

Training Stability: Architecture Improvements

RMSNorm and Nonparametric Layer Norm

OLMo-0424 used a nonparametric layer norm—a normalization operation without learnable scale and shift parameters—"for performance and to work around bugs in the libraries we were using" (Section 3.3.1). OLMo 2 switches to standard RMSNorm (Root Mean Square Layer Normalization; Zhang and Sennrich, 2019), which normalizes activations by their root-mean-square value and includes learnable scale parameters. The paper reports that "our ablations show no difference between the two, so we switch back to RMSNorm" (Section 3.3.1), motivated primarily by convergence with standard practice and the resolution of the earlier library bugs.

Reordered Normalization

The most significant architectural change in OLMo 2 is the reordering of normalization relative to attention and feedforward sublayers. The transformer block in OLMo-0424 used pre-normalization (applying LayerNorm to the inputs before attention/MLP):

OLMo-0424 (pre-norm):

h := x + Attention(LN(x))
h_out := h + MLP(LN(h))

OLMo 2 uses post-normalization (applying RMSNorm to the outputs after attention/MLP, not the inputs):

OLMo 2 (post-norm):

h := x + RMSNorm(Attention(x))
h_out := h + RMSNorm(MLP(h))

where x is the input to the layer, h is an intermediate hidden state (residual stream after attention), and h_out is the output of the full block (residual stream after MLP).

What changes: In pre-norm, the normalization sees the raw residual stream before the sublayer computation, which means the sublayer's input has controlled scale. In post-norm, the normalization sees the sublayer's output added to the residual stream, which means the sublayer operates on uncontrolled-scale inputs but the residual connection is normalized before passing to the next sublayer. This subtle rearrangement changes which parts of the computation see controlled-scale inputs and which see unbounded-scale inputs.

Why this form: The paper cites Liu et al. (2021) for originally proposing reordered layer norm, and notes it was "subsequently picked up by Chameleon Team (2024)." The motivation is training stability: post-norm prevents the residual stream from accumulating uncontrolled scale across layers, because each normalization step explicitly controls the output before it enters the next residual connection. In pre-norm architectures, the residual stream can grow unboundedly because only the sublayer inputs are controlled, not the outputs added back.

QK-Norm

Following Dehghani et al. (2023b), OLMo 2 applies RMSNorm to the query and key projections before computing attention. The attention computation typically computes:

$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$

The QK dot product $QK^T$ can produce large logit values if the norms of Q and K grow, causing the softmax to saturate (producing near-one-hot attention distributions) and gradients to vanish. QK-norm normalizes Q and K before the dot product:

$Q' = \text{RMSNorm}(Q), \quad K' = \text{RMSNorm}(K)$ $\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{Q'(K')^T}{\sqrt{d_k}}\right)V$

What it computes: Each query vector and each key vector is independently normalized to unit RMS before the attention dot product. This bounds the maximum attention logit by the dimensionality alone, preventing the logits from growing with the scale of the query/key projections.

Why this form: without QK-norm, if the query or key projections learn to produce large-norm vectors (which can happen as training progresses and the model becomes more confident), the softmax temperature effectively decreases, making the attention distribution increasingly peaked and the gradients increasingly sparse, which can cause training instability. QK-norm decouples the sharpness of the attention distribution from the scale of the projections, making attention behavior independent of projection magnitude.

Figure 7 shows the effect: "Applying layer norm after the attention and feedforward layers along with a QK-norm improves stability compared to a more standard pre-attention layer norm. These changes reduce the spike score of the gradients from 0.108 to 0.069 when applied together" (Section 3.3.2). Importantly, the paper notes that "in isolation, neither of these changes yield good results, but together they improve both the growth and the spikiness of the L2 norm of the gradient" (Section 3.3.2)—the reordered norm and QK-norm interact synergistically and must be deployed together.

Z-Loss Regularization

Following Chowdhery et al. (2022), Chameleon Team (2024), and Wortsman et al. (2023), OLMo 2 adds a z-loss regularization term to the training objective. The z-loss penalizes large logit values by adding a term proportional to the squared log of the softmax denominator:

$\text{Z-loss} = \lambda_z \cdot \log^2 Z$

where $Z = \sum_i \exp(\text{logit}_i)$ is the denominator in the softmax over the vocabulary, and $\lambda_z = 10^{-5}$ is the Z-loss weight (Table 1). The total training loss is $\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{cross-entropy}} + \lambda_z \cdot \log^2 Z$.

What it computes: The softmax denominator $Z$ (also called the partition function) grows when the logits are large and positive, which occurs when the model is very confident. The Z-loss penalizes this confidence quadratically in $\log Z$, encouraging the model to keep logits at moderate values rather than growing unboundedly.

Why this form: The $\log^2 Z$ form penalizes the log of the partition function rather than the partition function directly. Penalizing $Z$ directly would be numerically unstable (exponential of logits squared) and would not scale appropriately across different contexts. The log-squared form grows smoothly and matches the scale of log-space operations used elsewhere in the training objective. The weight $\lambda_z = 10^{-5}$ is small enough to not significantly affect the primary learning objective but large enough to prevent runaway logit growth.

Implementation caution. The paper reports a subtle implementation issue (Section 3.3.3, Figure 8): "Flash Attention's implementation of z-loss does not match a manual implementation in PyTorch. While the forward pass produces the same number, differences in the backwards pass cause the curves to diverge." The team found that the Flash Attention library's fused cross-entropy-with-z-loss kernel produced different gradient behavior in the backward pass compared to a PyTorch-native implementation, despite identical forward-pass values. The hypothesized cause is "differences in precision" in the backward computation. Since the forward-pass loss values were identical and "this does not affect cross entropy loss during training, or the model's performance on downstream tasks," the practical consequence is a caution against switching implementations mid-run: "During a training run we cannot switch implementations safely, so we avoid doing so as much as possible."

Training Stability: Hyperparameter Improvements

AdamW Epsilon

The paper reduces the AdamW $\epsilon$ hyperparameter from $10^{-5}$ to $10^{-8}$. In the AdamW update rule, $\epsilon$ appears in the denominator of the adaptive learning rate:

$\theta_{t+1} = \theta_t - \eta \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}$

where $\hat{m}_t$ is the bias-corrected first moment estimate, $\hat{v}_t$ is the bias-corrected second moment estimate, and $\eta$ is the learning rate. The $\epsilon$ term prevents division by zero when $\hat{v}_t$ (the variance estimate) is very small, which occurs at the beginning of training when the optimizer has seen few gradients.

What it computes: The effective step size in each parameter dimension is $\eta / (\sqrt{\hat{v}_t} + \epsilon)$. When $\epsilon$ is large relative to $\sqrt{\hat{v}_t}$, the effective step size approaches $\eta / \epsilon$, which is a constant upper bound. When $\epsilon$ is small, the step size can be much larger (up to $\eta / \epsilon$ when $\hat{v}_t \approx 0$).

Why $10^{-8}$ over $10^{-5}$: the paper notes that $10^{-8}$ "is the default in PyTorch, but some popular LM training code bases come with a default of $10^{-5}$" (Section 3.4.1). The lower value "allows for larger updates early in training, and helps the model learn faster during a period where we've typically seen a lot of instability." Figure 9 shows the effect: with $\epsilon = 10^{-8}$, "the gradient norm settles much more quickly and remains permanently lower" and "the training loss also improves faster." The early-training period is when the model is most vulnerable to instability (the gradient norm is large and noisy), so allowing larger updates then—counterintuitively—helps the optimization converge to a more stable regime faster rather than lingering in the high-variance region.

Weight Decay on Embeddings

OLMo uses standard weight decay where "every parameter is multiplied by $1 - (0.1 \cdot \text{lr})$ at every step" (Section 3.4.2). Weight decay is an L2 regularization term that discourages parameters from growing large. However, the paper finds that applying weight decay to token embeddings causes problems.

The mechanism involves the Jacobian of layer normalization. As shown by Takase et al. (2024), the gradient of LayerNorm(x) with respect to x is:

$\frac{\partial \text{LayerNorm}(x)}{\partial x} \propto \frac{1}{\|x\|}$

When the embedding vectors have small norms (due to weight decay continuously shrinking them), $1/\|x\|$ becomes large, causing correspondingly large gradients in early layers. Since "in early layers, the norm of the residual stream is essentially the norm of the embeddings" (Section 3.4.2), small embeddings cause large gradients throughout the early transformer blocks.

The fix: "we simply turn off weight decay for embeddings and observe that embedding norms settle in a healthy region as training progresses" (Section 3.4.2). The paper experimented with the "full range of remedies discussed in Takase et al. (2024)" but found they "impacted the speed of convergence." The simplest intervention—disabling weight decay on the embedding matrix—resolves the gradient norm inflation without affecting convergence speed.

Figure 10 shows the effect: with weight decay on embeddings, the embedding norm steadily decreases and the gradient norm steadily increases over training; without weight decay, both remain stable. The spike score improves from 0.16 (with embedding weight decay) to 0.092 (without), a modest but meaningful improvement.

Pretraining Recipe: Learning Rate Schedule

The pretraining stage uses a learning rate schedule with three phases:

Phase 1 — Warmup: The learning rate increases linearly from 0 to the peak value over 2,000 steps. For the 7B model, the peak is $3.0 \times 10^{-4}$; for the 13B, $9.0 \times 10^{-4}$; for the 32B, $6.0 \times 10^{-4}$ (Table 3). The warmup period allows the optimizer to accumulate second-moment estimates ($\hat{v}_t$) before taking full-size steps, preventing the initial steps (when $\hat{v}_t$ is near zero and thus the adaptive learning rate is near $\eta/\epsilon$) from being destructively large.

Phase 2 — Cosine Decay: After warmup, the learning rate follows a cosine schedule calibrated to reach 10% of the peak learning rate after the specified maximum tokens: 5T for the 7B, 5T for the 13B, and 6.5T for the 32B. The cosine schedule gradually reduces the learning rate from peak to near zero, allowing the model to settle into a minimum with progressively finer optimization steps.

Phase 3 — Truncation: The cosine schedule is truncated before reaching its natural end. For the 7B, the schedule is stopped at 4T tokens (instead of the full 5T); for the 32B, it is stopped at 6T (instead of 6.5T). The remaining steps are allocated to the mid-training phase with a linear decay instead. The 13B does not use truncation—it runs the full cosine schedule to 5T.

This truncation strategy reflects the finding from OLMo-0424 that "the last part of a cosine decay schedule can be cut off and replaced by a linear decay to zero with little loss of performance" (Section 4.1). The linear decay used in mid-training (see below) is effectively a replacement for the final portion of the cosine curve, but with a different data mixture.

Pretraining Recipe: Data Mixture (OLMo 2 Mix 1124)

The pretraining data mix (Table 4) consists of approximately 3.9 trillion tokens with the following composition:

• DCLM-Baseline web pages: 3.71T tokens (95.1%), from the DCLM "baseline 1.0" mix (Li et al., 2024), which is a large-scale filtered web corpus.
• StarCoder code: 83.0B tokens (2.1%), a filtered version from the OLMoE Mix. This subset includes only repositories with 2 or more GitHub stars and removes documents "whose most frequent word constitutes over 30% of the document, or whose top-2 most frequent words constitute over 50% of the document" (Section 2.4.1). These heuristics target binary-encoded files and primarily-numerical content.
• peS2o academic papers: 58.6B tokens (1.5%), from Dolma 1.7, consisting of cleaned scientific papers from S2ORC (Soldaini and Lo, 2023).
• arXiv STEM papers: 20.8B tokens (0.5%), from RedPajama, containing scientific preprints.
• OpenWebMath: 12.2B tokens (0.3%), mathematical web pages from ProofPile II (Azerbayev et al., 2023; Paster et al., 2023).
• Algebraic Stack: 11.8B tokens (0.3%), math proofs and code from ProofPile II.
• Wikipedia & Wikibooks: 3.7B tokens (0.1%), encyclopedic content from Dolma 1.7.

Design choices. The mixture is dominated by web data (>95%) but includes small but strategically important fractions of code (2.1%), academic text (2.0%), and math (0.6%). The total size of 3.9T tokens means the 7B model (trained on ~4.05T total including mid-training) sees approximately 16 tokens per parameter during pretraining, well within the standard overtraining regime where models benefit from seeing data multiple times. The code filtering (2-star minimum, word-frequency heuristics) is designed to improve code data quality without needing to manually inspect GitHub repositories.

Mid-Training: Learning Rate Annealing Experiments

The mid-training stage begins from a pretrained checkpoint and linearly decays the learning rate to zero over the remaining tokens. The paper performed extensive experiments to determine whether the peak learning rate during pretraining affects mid-training effectiveness.

Experimental design. The team trained 7B models with four peak learning rates ($3\times10^{-4}$, $6\times10^{-4}$, $9\times10^{-4}$, $12\times10^{-4}$) and $30\times10^{-4}$ (abandoned early due to loss spikes during warmup). The key methodological insight is that training loss curves alone are misleading for short experiments: "Looking purely at training loss, higher learning rates universally perform better early on (as long as they avoid loss spikes), but eventually the lower learning rate setting overtakes the others" (Section 4.1). The cross-over point for $3\times10^{-4}$ vs. $6\times10^{-4}$ is "well past 200B tokens," meaning that "a shorter hyperparameter experiment might come to the wrong conclusion."

Mid-training effectiveness. To test whether higher pretraining learning rates produce models that benefit more from mid-training, the team took checkpoints at 300B tokens from each variant and linearly decayed the learning rate to zero over 50B tokens. The result: "a higher learning rate does make mid-training more effective, but it does so by exactly the amount that the pretraining is worse. All four variants show the same training loss at the end of the procedure" (Section 4.1). Extending the experiment to 1T and 2T pretraining tokens with $3\times10^{-4}$ and $6\times10^{-4}$ showed the same pattern: "virtually no difference between the two settings, both on training loss and a mix of nine downstream tasks" (Table 8).

The key finding. The final learning rate at the end of training—not the peak or the schedule shape—is what determines the model's convergence point (assuming the peak is within a stable range). The paper explicitly connects this to Wortsman et al. (2023), who "observed that smaller models' performance is largely invariant to learning rate over several orders of magnitude when trained to the end of a cosine schedule, and further found that QK-norm and z-loss, which we use as well, enhance this effect" (Section 4.1). The practical implication is that learning rate selection for pretraining is less critical than previously thought, as long as the model is trained to convergence (LR decays to near zero) and the peak is within the stable range.

Mid-Training: The Dolmino Mix 1124 High-Quality Sources

The mid-training mixture is constructed from two categories of data: a high-quality subset of web and reference sources (providing general capability improvement) and a math subset (providing targeted skill improvement). Table 5 lists the base sources; Table 13 shows the specific composition for 50B, 100B, and 300B token mixes.

High-Quality Web Selection

The paper experiments with two quality classifiers to filter web data beyond the standard DCLM filtering:

FastText classifier from DCLM (Li et al., 2024). This is "trained on positive documents from the Reddit subset in ELI5 (Fan et al., 2019), and demonstrations from Open Hermes 2.5," with negatives sampled randomly from DCLM. The classifier outputs a score; following Li et al., the paper uses a threshold of 0.03311014, which retains approximately 65.6% of the web subset.

FineWeb Edu classifier from Penedo et al. (2024). This model is "fine-tuned from the Arctic Embed M encoder (Merrick et al., 2024) on over 400,000 web pages labeled by Llama 3 70B Instruct" and scores documents from 0 to 5 according to "adherence to academic topics and polished content" (Section 4.3). The paper experiments with two thresholds: score ≥ 3 (retaining 5.8% of documents) and score ≥ 2 (retaining 20.3%).

Combined filtering. The final mixes use the FastText top 7% combined with the more relaxed FineWeb ≥ 2 threshold (retaining 20.3% of documents), which Table 11 shows as the "Web FT7 FW2" condition. This combination balances quality improvement with sufficient data volume.

Reference and Instruction Data

The high-quality mix also incorporates:

• FLAN instruction data from Dolma 1.7, with decontamination: the paper "extracted training, validation, and test instances from all tasks in our evaluation suite and removed FLAN documents with 10% or more overlapping n-grams with any task instance" (Section 4.3). This prevents the model from memorizing evaluation examples during mid-training.

• Stack Exchange Q&A pairs from the September 30, 2024 database dump. Questions are filtered to those with accepted answers, at least 3 votes on the question, and at least 5 votes on the answer. Questions and answers are concatenated "using a sequence of new lines that contains one more \n than the longest sequence of newlines in either the question or answer" (Section 4.3)—a formatting heuristic to preserve the Q&A structure without special delimiter tokens.

• Academic and encyclopedic content: peS2o (58.6B tokens available), Wikipedia and Wikibooks (3.7B), Gutenberg books, arXiv, StackExchange from RedPajama v1, and Algebraic Stack. These sources provide high-quality reference material that was underrepresented in the web-dominated pretraining mix.

Mid-Training Mix Evaluation

Table 11 shows the results of evaluating different mid-training mixtures on a development benchmark suite. The progression from the pretrained checkpoint (no mid-training) to the final mix is instructive:

• PT Mix (same data as pretraining, just annealed): +4.4 OLMES, +1.3 OLMES-Gen, +2.0 MMLU, −1.5 GSM*. The annealing schedule alone improves general benchmarks slightly but hurts math.
• Web FT7 FW2 (FastText top 7% + FineWeb ≥ 2): +5.6 OLMES, +0.6 OLMES-Gen, +3.3 MMLU, +0 GSM*. Better web quality improves multiple-choice and MMLU but doesn't help math.
• Web FT7 FW2 + Math + Ins (final mix): +6.1 OLMES, +7.0 OLMES-Gen, +3.3 MMLU, +18 GSM*. Adding math and instruction data dramatically improves math (+18 points on the development set) while maintaining or improving other benchmarks.

The key insight is that annotation-level data quality (curated reference sources) and domain-specific data (math) are complementary: reference data improves general knowledge, while math data patches a specific capability gap. Combining both in the same mid-training stage yields improvements on both fronts.

Mid-Training: The Math Mix Development

The math subset of Dolmino Mix 1124 (summarized in Table 5, totaling ~10.7B tokens) was constructed through an iterative process of data generation, filtering, and evaluation.

TuluMath (230M tokens)

This dataset uses the persona-driven methodology from Chan et al. (2024). The key idea is:

1. Prompt generation: Given a persona (e.g., "A machine learning researcher focused on neural networks"), GPT-4o is prompted with "Create a math problem related to the following persona: {persona}" (Appendix E, Figure 24), with instructions that "the math problem should be challenging and involve advanced mathematical skills and knowledge" and "You should make full use of the persona description to create the math problem to ensure that the math problem is unique and specific to the persona."

2. Solution generation: The generated problems are then fed back to GPT-4o with "Provide solution to the given math problem" (Appendix E, Figure 25), with instructions to "Provide your solution step-by-step, and end your solution in a new line in the following format: Final Answer: The final answer is $final_answer$."

The personas come from Persona Hub (Chan et al., 2024), which contains 1 billion personas across diverse domains. The result is a large collection of math problems and solutions that are synthetically diverse by construction, since each problem is conditioned on a different persona.

Dolmino SynthMath (28.7M tokens)

This is a custom collection targeting a specific failure mode: the model making arithmetic errors within chain-of-thought reasoning steps. It has three components:

1. Basic arithmetic Q&A (11M tokens): Simple calculations like "77 * 14 = 1078" with varied prompt templates. The paper notes that "including such data dramatically mitigates the mistakes our model makes within individual CoT reasoning steps at inference time" (Section 4.4.1). This suggests that the base model, despite being trained on 4T tokens, had not sufficiently internalized basic arithmetic—a finding consistent with the broader observation that language models learn arithmetic through pattern matching rather than algorithmic computation, and need explicit exposure to a wide range of numeric operations.

2. GSM8K number permutations (7,924 examples): Standard GSM8K training examples where the numbers in both the question and answer are replaced while preserving the problem structure. This is designed to "provide signal to the model to extract the computation graph from a word problem and ignore irrelevant semantic features" (Section 4.4.1). The hypothesis is that the model can overfit to specific number combinations in the original GSM8K examples; permuting the numbers forces it to learn the underlying arithmetic pattern rather than memorizing question-answer pairs.

3. MIND-rewritten GSM8K: Each GSM8K training example is rewritten using the MIND method (Akter et al., 2024) with Qwen2.5-7B-Instruct as the rewriter. MIND (Math Informed synthetic Dialogues) transforms math problems into natural dialogue or narrative formats, adding stylistic diversity while preserving the mathematical content.

TinyGSM-MIND (6.48B tokens)

Tiny-GSM (Liu et al., 2023a) is a collection of 11M synthetic GSM8K-like questions where answers are provided as Python code. The paper filters this set to "only include answers that have code that is executable and only contains statements that are variable assignments" (Section 4.4.1). Executability ensures the code is syntactically valid; restricting to variable assignments ensures the code represents a computation graph (a sequence of x = ... statements) rather than control flow.

The filtered examples are then "annotated with the numerical value of the resulting variable" for each assignment statement—essentially computing the value at each intermediate step of the computation graph. These annotated examples are then passed to Qwen2.5-7B-Instruct for MIND-style rewriting using the "Two Students" and "Problem Solving" prompts. The result is 6.48B tokens of naturally-formatted math reasoning data derived from programmatic solutions.

MathCoder2-Synthetic (3.87B tokens)

This emulates the synthetic data generation procedure of MathCoder2 (Lu et al., 2024) by filtering existing open-source synthetic textbooks. The paper collects textbooks from HuggingFace users Ajibawa-2023 (Maths-College, Education-College-Students) and the M-A-P Matrix dataset. A FastText classifier is trained to distinguish math-related from non-math-related content: 10,000 OpenWebMath examples are annotated by GPT-4o as math or non-math, and used as training data for the classifier. Only math-related textbooks are retained.

ProofPile OWM-Filtered

The OpenWebMath classifier from the previous step is applied to Metamath (Yu et al., 2023, a collection of mathematical proofs) and CodeSearchNet (Husain et al., 2019, a code search dataset), retaining only math-related documents. This is a relatively small component (84.2M + 1.78M tokens) but provides additional diversity in mathematical content.

GSM8K-Train

The full training split of GSM8K (Cobbe et al., 2021) is included directly at 2.74M tokens. This is the only non-synthetic, human-written math data in the mixture.

Mid-Training: The Micro-Annealing Methodology

A key methodological contribution is the micro-annealing technique for evaluating data sources independently at a fraction of the cost of full mid-training runs. The procedure is:

1. Start from a pretrained 7B checkpoint.
2. Create a small mixture (typically hundreds of millions to a few billion tokens) of the candidate math data combined with roughly equal amounts of general web data (DCLM) to maintain a realistic data distribution.
3. Train this mixture with the same learning rate anneal schedule, linearly decaying to zero, but only for the small number of tokens in the mixture.
4. Evaluate on MMLU (to check general capability preservation) and GSM* (the 200-example math development set) to assess math improvement.

The paper runs 19 micro-anneals totaling 130B tokens, "equivalent to less than 3 full 50B annealing runs." Each micro-anneal provides results visible "after training for less than 10B tokens" (Section 4.4.2), enabling rapid iteration on data mixture composition.

Micro-anneal Experiment 1 (Mixture Ratio): Starting from a 35/65 math/DCLM ratio and a 10/90 ratio, the results show GSM* = 63.5 for the math-rich mix vs. 61 for the math-sparse mix, while MMLU is similar (60.1 vs. 60.9). This suggests that "it is not strictly necessary to have a large proportion of domain-specific data in the annealing mixture, just that domain-specific data is present" (Section 4.4.2). Even a small amount of math data in the mid-training mix can substantially improve math performance.

Micro-anneal Experiment 2 (Data Duplication): Duplicating the math data (2 copies, 4 copies) shows GSM* scores of 66 (2x) and 65 (4x) compared to 63.5 (1x). This indicates "even if there is a scarcity of high-quality domain-specific data, duplicating it a small number of times can still provide some gains" (Section 4.4.2). The benefit saturates quickly (4x is not better than 2x), consistent with the general observation that data repetition has diminishing returns.

Micro-anneal Experiment 3 (MIND Rewriting): Comparing the original Tiny-GSM (where answers are Python code) with the MIND-rewritten version (answers in natural language) shows a dramatic difference: the code version's GSM* score is 25.0 (worse than the baseline's 28.5), while the MIND version scores 65.5. This is explained by the pretraining data composition: "our pretraining mix is only 2% code" (Section 4.4.2), so the model struggles to extract mathematical reasoning from code format. Rewriting code answers into natural language makes them compatible with the model's predominantly text-based training distribution.

The 2x duplicated MIND version scores 70.0, further improving over the single copy at 65.5.

Mid-Training: Final Mixture Composition and Souping

The final Dolmino Mix 1124 composition is specified in Table 13, which shows three sizes (50B, 100B, 300B tokens) with varying repetition of sources:

50B mix (used for OLMo 2 7B):

• Filtered DCLM web: 3.23% of available (47.2% of mix)
• FLAN (decontaminated): 50% of available (16.6% of mix)
• Stack Exchange Q&A: full use, 1x (2.45% of mix)
• peS2o: 5.15% of available (5.85% of mix)
• Wikipedia/Wikibooks: full use, 1x (7.11% of mix)
• Dolmino Math: full use, 1x (20.8% of mix)

100B mix (used as base for 13B and 32B):

• Similar composition but with FLAN and math data repeated 2x (Table 13 shows 100% → 200% usage), and peS2o usage increased to 16.7% of the available corpus.

300B mix (used as additional soup ingredient for 13B and 32B):

• FLAN repeated 2x (200% usage), math data and Stack Exchange repeated 4x (400% usage), Wikipedia/Wikibooks repeated 4x (400% usage), peS2o used in full (100%). The web data fraction increases to 51.9% of the mixture.

Model Souping. The paper adopts model weight averaging (model souping; Matena and Raffel, 2022; Wortsman et al., 2022) to combine multiple independent mid-training runs. Table 14 compares six different mid-training mixtures, each evaluated as both the best single checkpoint and the average of three checkpoints (trained on different data order permutations of the same mix). The result is consistent: "souping equals or outperform the single best checkpoint trained on the same mix" (Section 4.5). For example, one mix improves from 75.6 (best single) to 77.0 (soup) on OLMES MCF, and from 71.0 to 74.0 on GSM*.

The mechanism is that different data orders lead the model to settle into slightly different local minima of the loss landscape. Averaging these minima produces a model that is more robust—combining the strengths of each path through the data while canceling out path-specific noise.

Final model construction:

• OLMo 2 7B: Average of 3 checkpoints, each trained on a different permutation of the 50B Dolmino mix.
• OLMo 2 13B and 32B: Average of 4 checkpoints: 3 trained on 100B mix (different permutations) and 1 trained on 300B mix. The paper notes this is "empirically better than averaging just the three 100B runs alone" (Section 4.5).

The total training tokens per model:

• 7B: 3.90T pretraining + 150B mid-training (3 × 50B runs) = approximately 4.05T tokens used, though the souping means the effective tokens seen is less.
• 13B: 5.0T pretraining + 300B mid-training (3 × 100B) + 300B (1 × 300B) = approximately 5.6T tokens.
• 32B: 6.0T pretraining + 300B (3 × 100B) + 300B (1 × 300B). Wait—the 32B numbers in the paper are inconsistent. Table 3 says 6.5T cosine schedule truncated at 6T for the 32B, and Section 2.3 says the 32B is trained on 6.6T tokens total (6.06T pretraining). This means 600B tokens of mid-training: the souping runs (3 × 100B + 1 × 300B = 600B) are the difference between 6.06T and 6.6T.

Post-Training: Supervised Fine-Tuning

The SFT stage adapts the base OLMo 2 models to instruction following using a curated dataset of 866K–939K prompts. The paper develops two SFT mixes:

tulu-3-sft-olmo-2-mixture (for 7B and 13B, 939,104 prompts): Based on the Tülu 3 SFT dataset (Lambert et al., 2024), with an attempted modification: removing all multilingual data (the Aya split and multilingual Wildchat samples). This removal "degraded performance by ~0.5 points on average, indicating that the Tülu 3 dataset is balanced and cannot be easily improved by removing irrelevant subsets" (Section 5). The multilingual data, despite not being targeted by OLMo 2's English-focused pretraining, appears to provide cross-lingual transfer that benefits English performance.

tulu-3-sft-olmo-2-mixture-0225 (for 1B and 32B, 866,138 prompts): Two additional filtering steps beyond the first mix:

1. Instructions mentioning a date cutoff from synthetic data generation are removed because "it was correlated with undesirable behavior like hallucinating date cutoffs and prefacing responses with 'As an AI language model...'" (Section 5).
2. For math datasets (Persona MATH and Grade School Math), "we only include prompts and completions where the model reaches a majority vote over 5 completions" (Section 5). This means a synthetic math problem is included only if at least 3 of 5 sampled solutions agree on the answer, filtering out problems with ambiguous or incorrect synthetic answers.

Hyperparameter selection. For the 7B model, the paper sweeps learning rates $1\times10^{-5}$, $2\times10^{-5}$ (selected), $3\times10^{-5}$. For the 13B model: $1\times10^{-6}$, $4\times10^{-6}$, $5\times10^{-6}$ (selected), $7.5\times10^{-6}$, $8\times10^{-6}$. The key finding is that OLMo 2 "required significantly higher learning rates compared to the Llama 3.1 training recipe described by Lambert et al. (2024)" (Section 5), indicating that hyperparameter transfer across model families is not straightforward—architecture and pretraining data differences propagate to post-training behavior.

The SFT models are trained with "an effective batch size of 128, a linear learning rate schedule and a warmup ratio of 0.3" (Table 17).

Post-Training: Direct Preference Optimization (DPO)

The DPO stage uses the Tülu 3 preference tuning methodology with two key modifications:

On-policy data generation. Responses are sampled from development OLMo 2 SFT checkpoints at both 7B and 13B scales, with "independent datasets for each" (Section 5). This ensures the preference data is on-policy—the responses being compared come from the same model family being optimized, rather than from unrelated models whose preferences may not generalize.

Off-policy data generation. From a pool of 20 models (Table 25) including Yi, Tülu 2, Gemma 2, Qwen 2.5, Mistral, Phi-3, and SmolLM2, all with "permissive licenses." The paper does not specify the exact selection criteria beyond this, but the pool spans diverse model families and sizes (from 1.7B to 34B).

Preference data construction. GPT-4o-2024-08-06 serves as the LLM judge, rating completions on "helpfulness, truthfulness, honesty, and instruction-following aspects" (Section 5). The ratings are binarized using Argilla's method: average ratings across all aspects, select the highest-rated completion as chosen, and sample from the remaining completions as rejected.

Prompts. The dataset sources are listed in Table 27, totaling 366.7K prompts for 7B and 377.7K for 13B. Sources include SFT reused prompts (117K), SFT instruction-following prompts (65.8K), WildChat subsets, UltraFeedback, and Tülu 3 Personas IF prompts.

Hyperparameter selection. The paper sweeps DPO learning rates: $5\times10^{-7}$, $6\times10^{-7}$, $7\times10^{-7}$, $8\times10^{-7}$ (selected for 13B), $1\times10^{-6}$ (selected for 7B). Figure 12 shows the average score for DPO checkpoints across learning rates (excluding safety, which had higher variance), supporting the selected values.

Post-Training: RLVR with Multi-Stage Training

Reinforcement Learning with Verifiable Rewards (RLVR) is the final post-training stage, using PPO (Schulman et al., 2017) with reward signals that are binary correctness checks rather than learned reward models.

Training data. The RLVR dataset combines the training sets of GSM8K, MATH, and "prompts with constraints" from Lambert et al. (2024). Constraint prompts test instruction following by requiring specific output formats or content. The reward is 1 if the model's answer matches the ground truth (exact match for GSM8K, flexible match for MATH as in Hendrycks et al., 2021b) or satisfies the constraint, and 0 otherwise.

Value function initialization. Following Lambert et al. (2024), PPO's value function is initialized from a reward model trained on the on-policy preference data, using hyperparameters: learning rate $3\times10^{-6}$, batch size 256, max token length 2048, 1 epoch (Table 26). This provides a warm start for the value function, helping it estimate expected returns more accurately than random initialization.

PPO hyperparameters (Table 18).

• Learning rate: $3\times10^{-7}$ (13B first and third stages), $4\times10^{-7}$ (7B)
• KL penalty coefficient $\beta$: 0.1 (13B first and third stages), 0.03 (13B second stage), 0.05 (7B)
• Clipping coefficient $\epsilon$: 0.2
• Value function coefficient: 0.1
• Discount factor $\gamma$: 1.0 (no discounting—all steps within an episode are equally important)
• GAE $\lambda$: 0.95 (standard for balancing bias-variance in advantage estimation)
• PPO update iterations K: 4 (number of epochs per PPO update)
• Mini-batches: 1 (no mini-batching within a PPO update)
• Effective batch size: 248 (13B), 224 (7B)
• Max total episodes: 200,000 (13B), 100,000 (7B)
• Generation temperature: 1.0
• Max token length: 2,048 (both prompt and response)
• Penalty reward for no EOS token: −10.0 (to discourage excessively long or incomplete generations)
• Gradient norm threshold: 1.0

Multi-stage RLVR for 13B. The paper observes that after the initial RLVR pass on the mixed dataset, "its performance on GSM8K and MATH was lower than a previous development instruct model" (Section 5). Rather than accepting this as a limitation, the team performs two additional RLVR iterations:

1. RLVR Stage 1: Train on GSM8K + MATH + constraint prompts mix. The value function is initialized from the reward model.
2. RLVR Stage 2: Train on GSM8K only. The value function is re-initialized from the reward model (not carried over from Stage 1). This focuses entirely on grade-school math, allowing GSM8K performance to improve without dilution from harder MATH problems.
3. RLVR Stage 3 (final): Train on MATH only. The value function is again re-initialized. This final stage focuses on competition-level math, and interestingly, Figure 13 shows that training on MATH alone not only improves MATH score (37.5 → 39.5 in the bottom-right panel of Figure 13) but also further improves GSM8K (84.5 at the end of Stage 2 → 87.2 at the end of Stage 3).

This cross-task transfer—MATH training improving GSM8K performance—is a significant finding that the paper does not deeply analyze. One hypothesis is that MATH problems, being harder, teach the model general problem-solving strategies that transfer to easier math tasks, whereas GSM8K training alone might teach more task-specific heuristics.

RLVR for 32B with GRPO. The 32B model uses Group Relative Policy Optimization (GRPO; Shao et al., 2024) instead of PPO. GRPO is a variant of PPO that "forgoes the need for a reward model" (Section 5) by using the relative rankings within a group of sampled responses to compute advantages, rather than relying on a learned value function. This reduces the number of trained components (no separate reward model needed) at the cost of requiring multiple samples per prompt to form a comparison group. The 32B GRPO configuration uses 16 samples per prompt, a learning rate of $5\times10^{-7}$, and KL beta of 0.1.

Figure 14 shows the 32B RLVR training trajectory: the model improves on all four monitored metrics (BBH, DROP, GSM8K, IFEval, MATH) over approximately 300 steps, with GSM8K rising from ~85 to ~87.5 and MATH from ~47 to ~49.

The RLVR effect. Across both model scales, RLVR provides consistent improvements in the metrics it directly optimizes (GSM8K, MATH, IFEval) while generally maintaining or slightly improving other capabilities. Table 16 quantifies this: for the 7B, RLVR adds +0.6 points to the average score (55.9 → 56.5) with the largest gains on GSM8K (+2.5) and MATH (+2.2). For the 13B, RLVR adds +1.4 points average (62.0 → 63.4) with gains on GSM8K (+5.1), IFEval (+2.4), and MATH (+4.0). These improvements come despite the fact that RLVR provides only binary reward signals (correct/incorrect) without any shaping or dense rewards, demonstrating that sparse RL signals can effectively fine-tune language model policy when the reward function is well-aligned with desired behavior.

Infrastructure: Throughput Optimizations

The paper documents several PyTorch optimizations that "had a big impact towards reducing training time... without any apparent loss in the speed of convergence" (Section 6.4). These are not research contributions but are practical requirements for reproducing the training at the reported scale and cost.

torch.compile(). The torch.compile() function "will compile native PyTorch modules and functions into optimized kernels, resulting in significant throughput improvements and GPU memory savings by avoiding the Python overhead associated with calling individual PyTorch operations in sequence, and by reducing the number of reads and writes that must occur on the GPU" (Section 6.4). This is a standard optimization in modern PyTorch training pipelines (Ansel et al., 2024), but the paper's documentation makes it explicit rather than implicit.

Minimizing host-device synchronizations. GPU operations in PyTorch are asynchronous by default—a function call enqueues work on the GPU but does not wait for it to complete. A host-device sync forces the CPU to wait until all enqueued GPU operations finish, creating a bottleneck. The paper identifies three common sources of unintentional syncs:

1. Synchronous tensor transfers: tensor.to(device="cuda") without non_blocking=True forces a sync. The fix is to use tensor.to(device="cuda", non_blocking=True) for CPU→GPU transfers.
2. GPU→CPU transfers: Any operation that moves data from GPU to CPU (e.g., print()ing a CUDA tensor, or using a CUDA tensor in a conditional) forces a sync because the CPU needs the actual value. The paper recommends avoiding GPU→CPU transfers wherever possible.
3. Specific PyTorch operations: Some operations like masked_select() may "unexpectedly cause a host-device sync" due to internal implementation details.

The paper recommends using torch.cuda.set_sync_debug_mode("warn") to detect syncs, noting that "some syncs may still be missed" but it provides a best-effort diagnostic.

Asynchronous bookkeeping with a separate backend. Training loop operations like logging metrics and saving checkpoints typically involve host-device syncs (e.g., logging the loss requires copying the loss tensor from GPU to CPU). The paper implements these asynchronously using a separate PyTorch backend (GLOO instead of NCCL) dedicated to bookkeeping:

• Metrics logging: Metrics are accumulated on GPU every step. Only at the logging interval (every 10–50 steps) are they copied to CPU (one sync) and passed to the bookkeeping thread, which aggregates and logs them without blocking training.
• Checkpointing: At the checkpoint interval, the trainer copies model and optimizer state to CPU (one sync) and passes the copy to the bookkeeping thread, which assembles the distributed model shards and writes to disk while training continues.

The paper notes that "PyTorch NCCL backend is not thread safe," so the separate GLOO backend (which uses CPU communication) avoids conflicts. The only impact on training time is "one host-device sync, and the time it takes to copy data from the GPU to the CPU"—the actual disk I/O and log processing happen asynchronously.

Explicit garbage collection. Python's automatic garbage collector runs periodically, and "in a distributed setting, with thousands of training processes that are expected to run in lock-step with each other, nothing enforces that these garbage collections happen at the same time on every process" (Section 6.4). Figure 16 shows the consequence: training throughput becomes "slower and less stable, often becoming worse as the run progresses" because any process performing garbage collection becomes a straggler that all other processes must wait for.

The fix: disable automatic garbage collection with gc.disable() and run it explicitly at regular intervals with gc.collect(1), triggered simultaneously across all processes. This eliminates the throughput variability and improves average throughput.

4. Key Insights and Innovations

Innovation 1: Mid-Training as a General-Purpose Capability Intervention, Not Just Domain Adaptation

The paper's most intellectually distinctive contribution is its reframing of what a "second training stage" accomplishes and how to systematically design one. Prior work treated late-stage training interventions as narrow domain adaptation: take a pretrained model, continue training on domain-specific data, and observe improved performance on that domain. DBRX (Databricks, 2024) and Llama 3 (Grattafiori et al., 2024) demonstrated that data curricula help, but the implicit model was that you add specialized data to improve specialized benchmarks—a one-to-one mapping between data source and downstream metric.

OLMo 2 demonstrates something qualitatively different: a carefully constructed mid-training mixture improves performance across nearly all capability dimensions simultaneously, and the magnitude of improvement is disproportionate to the compute invested. Table 9 is the key evidence: the 7B model gains 10.6 points on average across the OLMES benchmark suite from only 50B mid-training tokens (1.25% of total training), with improvements spanning knowledge-intensive multiple-choice (+3.9 on MMLU), reading comprehension (+20.1 on DROP), and math reasoning (+43.4 on GSM8K). These are not correlated improvements—math data improves math, high-quality web data improves knowledge, and instruction data improves reasoning—but the fact that they coexist in a single mixture without negative interference is non-trivial.

What makes this a conceptual advance rather than a recipe is the paper's demonstration that mid-training mixture components can be evaluated independently and their effects compose approximately additively. The micro-annealing methodology (Section 4.4.2) shows that the math subset's contribution to GSM8K can be measured in isolation (a few billion tokens of math + web data, evaluated after complete annealing), and that the measured improvement persists when that math data is mixed with other components at larger scale. This transforms mid-training from a high-stakes gamble (spend 50B tokens hoping the mixture works) into an iterative engineering process where data sources can be screened cheaply before being incorporated into the final mixture. The paper runs 19 micro-anneals totaling 130B tokens—less than three full 50B annealing runs—to evaluate the full design space.

The contrast with prior practice is sharp. Without this methodology, developing a mid-training mixture requires either (a) running full-scale experiments and observing final benchmark scores (prohibitively expensive), or (b) relying on intuition about what data might help (error-prone). The micro-annealing technique provides a principled middle ground: small-scale experiments that are predictive of large-scale behavior because the annealing schedule—linear decay to zero from a fixed starting point—is matched between the micro and macro scales. This is fundamentally different from standard hyperparameter sweeps, which typically vary learning rate, batch size, or architecture without a well-defined relationship to the final training regime.

Innovation 2: The Interconnected Suite of Stability Interventions as a Diagnostic Framework

The paper's treatment of training stability is not merely a collection of fixes but constitutes a diagnostic framework for reasoning about where instabilities originate. Each intervention targets a specific, measurable symptom, and the paper traces causal chains from data artifacts to gradient behavior to loss divergence.

The repeated n-gram analysis (Section 3.1) is particularly illuminating as a diagnostic move. The paper does not simply observe that filtering repeated n-grams reduces spikes—it characterizes the relationship as non-deterministic ("The same n-gram sequence may spike for a larger model but not for a smaller model," "The same n-gram sequence associated with a spike can also be found elsewhere in training batches that did not spike") and reports that the intervention mitigates gradient spikes but "had no effect on the slow growth in gradient norm." This decomposition—fast spikes are caused by data artifacts, slow growth is caused by optimization dynamics—is the diagnostic insight. It tells practitioners that if they see gradient norm inflation over time, data filtering will not help; they need to look at initialization, normalization, or weight decay.

The growth exponent analysis (Section 3.2, Figure 5) provides a complementary theoretical diagnostic: before training begins, compute how activation and gradient magnitudes propagate through a randomly initialized network. A growth exponent near zero predicts stability; deviations from zero predict trouble. This is not a new metric (Cowsik et al., 2024), but the paper's use of it to select an initialization scheme—comparing OLMo-0424's depth-scaled initialization against standard normal initialization and choosing the one with growth exponents closer to zero—operationalizes a theoretical principle into a practical decision procedure.

The synergistic interaction of reordered normalization and QK-norm (Section 3.3.2, Figure 7) is another diagnostic finding: "in isolation, neither of these changes yield good results, but together they improve both the growth and the spikiness of the L2 norm of the gradient." This implies that practitioners cannot evaluate stability interventions independently; they must test combinations, because the failure modes interact. Pre-norm architectures allow the residual stream to grow, while QK-norm controls attention logit magnitudes. If you change only one, you break the balance. The paper's spike score metric—"the percentage of values in a time series that are at least seven standard deviations away from a rolling average of the last 1,000 values"—provides an objective, reproducible way to compare these combinations rather than relying on visual inspection of training curves.

What distinguishes this from prior stability work (Wortsman et al., 2023; Takase et al., 2024) is the integrated systems view: the paper shows that stability emerges from the interaction of data preprocessing choices (n-gram filtering), architectural choices (normalization placement, QK-norm, z-loss), optimization choices (AdamW epsilon, weight decay scope), and initialization. No single change is sufficient; the suite must be deployed together. This is a conceptual advance over papers that study individual interventions in isolation, because it demonstrates that instabilities are emergent properties of the training system, not local failures that can be patched one at a time.

Innovation 3: The Demonstration That Fully Open Models Can Be Pareto-Optimal

This innovation is less about a new technique and more about what the paper's existence proves. Prior to OLMo 2, the fully open model community faced an implicit assumption: releasing training data, code, and recipes necessarily incurs a performance penalty because proprietary data sources, secret training tricks, or non-disclosable optimizations are what push models to the frontier. The evidence supported this assumption—OLMo 1 (Groeneveld et al., 2024) at 38.3% average OLMES, OLMo-0424 at 50.7%, DCLM 7B at 56.9%, all substantially behind open-weight-only models like Llama 3.1 8B at 61.8%.

OLMo 2 at 62.9% (7B) breaks this assumption. Figure 1 shows the OLMo 2 family sitting on or near the Pareto frontier of performance relative to pretraining FLOPs, often requiring fewer FLOPs than comparably-performing open-weight models. The 7B model at 1.8×10²³ FLOPs matches Llama 3.1 8B at 7.2×10²³ FLOPs—a ~4× compute advantage. The 13B model at 4.6×10²³ FLOPs outperforms Qwen 2.5 14B at 16×10²³ FLOPs on some metrics while using less than a third of the compute.

What makes this intellectually significant rather than just a competitive benchmark result is that it demonstrates the sufficiency of known, publishable techniques for achieving frontier performance. The components—stability fixes, mid-training curricula, model souping, Tülu 3 post-training—are individually documented in prior literature and use permissively licensed or publicly available data. The paper shows that their systematic integration, not access to proprietary resources, is what closes the gap. This shifts the burden of proof: future claims that openness necessarily limits performance must explain what specific capability requires non-disclosable techniques, given that OLMo 2 achieves competitive performance without them.

The paper's commitment to held-out evaluation tasks (the "unseen evaluation suite" in Table 20, with AGIEval, GSM8K, MMLU-Pro, and TriviaQA reserved entirely from development decisions) strengthens this argument by forestalling the criticism that benchmark performance reflects overfitting to development metrics. Table 6 shows that gains on development tasks largely translate to the held-out suite, consistent with a generalizable training recipe rather than test-set optimization.

Innovation 4: Multi-Stage RLVR as a Curriculum for Policy Optimization

The post-training pipeline includes a finding that, while presented briefly, has implications for how RL fine-tuning should be structured. The 13B model's RLVR training (Section 5, Figure 13) undergoes three sequential stages: first on GSM8K + MATH + constraint prompts together, then on GSM8K alone, then on MATH alone. The striking result is that training on MATH alone in the final stage further improves GSM8K performance (from ~84.5 to ~87.2 in Figure 13), even though the model is no longer seeing GSM8K examples.

This is not predicted by standard RL fine-tuning theory, which would expect task-specific improvement on the training distribution with potential degradation on other distributions (catastrophic forgetting). Instead, the harder math problems appear to induce generalizable improvements in mathematical reasoning that transfer to easier problems. The paper does not deeply analyze this mechanism, but the result challenges the common practice of mixing all training data together in a single RL stage. It suggests that curriculum ordering matters for RL fine-tuning: starting with mixed data provides broad coverage, then specializing to harder problems transfers capability downward to easier tasks.

The contrast with standard RLHF practice is notable. Typical RLHF pipelines train on a fixed mixture of prompts throughout the RL stage, with hyperparameters (KL penalty, learning rate) tuned for stability rather than curriculum design. The multi-stage approach used here—with the value function re-initialized from the reward model at each stage, not carried forward—effectively treats each stage as a separate optimization problem with a different data distribution. This is conceptually similar to the pretraining-to-mid-training transition (change the data distribution, anneal the learning rate, and start from a strong initialization), suggesting a general principle: stage transitions with data distribution changes and optimizer resets may be as important for post-training as they are for pretraining.

The finding that this works with sparse binary rewards (correct/incorrect, with no reward shaping) makes it practically significant: it requires no reward model engineering, only the ability to verify correctness on training data. For capabilities where verification is cheaper than generation (math, code, constrained generation), this multi-stage RLVR approach provides a template for iterative improvement without the complexity of learned reward functions.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All base model experiments use the MATH benchmark (Hendrycks et al., 2021), specifically the split from Lightman et al. (2022): 12,000 training questions and 500 test questions. The choice of MATH is deliberate (Section 4): test-time compute is expected to help most when the model already possesses the necessary knowledge and the challenge is drawing complex inferences—mathematical reasoning fits this profile because it requires multi-step logical deduction rather than novel factual recall.

• Base model(s). All experiments use PaLM 2-S* (Codey) (Anil et al., 2023). The authors argue this model is "representative of the capabilities of many contemporary LLMs" and sits in a useful regime: non-trivial performance on MATH (roughly 10–19% pass@1 depending on the prompt and sampling configuration) but far from saturation, leaving room for test-time compute to make a difference. For the FLOPs-matched comparison, a second model with approximately ~14× more parameters is used as the pretraining-scaled baseline.

• Metrics. The primary metric throughout is MATH test accuracy (%)—the fraction of the 500 test questions for which the selected final answer matches the ground truth. Answers are graded using the grading function released by Lightman et al. (2022) (Appendix G). When analyzing difficulty-dependent behavior, the paper reports accuracy within each of the five difficulty quintiles separately.

• Baselines. The paper uses several baselines:

  • Majority voting: select the most common final answer among N sampled solutions (no learned verifier).
  • ORM best-of-N weighted: score N solutions with an outcome reward model and apply best-of-N weighted selection.
  • PRM best-of-N weighted: score N solutions with the process reward model and apply best-of-N weighted selection.
  • Parallel sampling (for revisions): generate N independent solutions from the revision model and select the best via verifier or majority.

• Generation budget / compute accounting. One "generation" equals one complete sampled answer from the base LLM. For beam search and best-of-N, the budget equals the number of beams or samples N. For lookahead search with k lookahead steps, the cost is N × (k + 1) to account for the additional rollout computation (Section 5.3). Budgets are swept across powers of 2, typically from 2⁰ to 2⁹ (1 to 512 generations).

• Cross-validation / statistical protocol. To avoid contaminating strategy selection with test-set performance, the authors use two-fold cross-validation within each difficulty bin on the 500-question test set. The best strategy is selected on one fold and evaluated on the other, with results averaged (Section 3.2). Oracle difficulty bins use ground-truth pass@1 rates from 2048 samples per question; predicted bins use the PRM's average final-answer score across the same 2048 samples, replacing ground-truth correctness with the verifier's own assessment.

Main Quantitative Results

Search Against PRM Verifiers (Section 5)

Aggregate search algorithm comparison (Figure 3, left). Across all 500 test questions with a maximum budget of 256 generations:

• At low budgets (2–8 generations), beam search with M = 4 significantly outperforms best-of-N weighted. At 4 generations, beam search (M = 4) achieves roughly 27% accuracy versus roughly 16% for best-of-N weighted—an 11-percentage-point gap.
• At high budgets (64–256), beam search performance flattens and falls slightly below best-of-N weighted. Best-of-N weighted reaches approximately 38% at 512 generations; beam search (M = 4) plateaus around 34%.
• Lookahead search (both k = 1 and k = 3) generally underperforms at the same generation budget due to its higher per-step cost. The 3-step lookahead variants converge to similar performance as other methods at very high budgets but never surpass them.
• Majority voting trails all verifier-based methods substantially, reaching only about 29% at 512 generations.

Difficulty-bin analysis for search (Figure 3, right). The per-difficulty breakdown (beam search M = 4 vs. best-of-N weighted, shown at four budget levels: 4, 16, 64, 256 generations) reveals the core pattern:

• Bin 1 (easiest): Beam search accuracy decreases from roughly 78% to 77% as the budget goes from 4 to 256, while best-of-N weighted increases from 68% to 88%. This is the clearest evidence of PRM over-optimization—beam search finds solutions that exploit the verifier signal.
• Bin 2: Beam search improves modestly (roughly 14% → 32%) but best-of-N weighted improves faster (roughly 14% → 60%), maintaining a clear advantage at high budgets.
• Bin 3: Beam search consistently outperforms best-of-N weighted across all budgets, reaching roughly 34% vs. 23% at 256 generations.
• Bin 4: Beam search shows the strongest relative advantage, reaching roughly 17% vs. 10% for best-of-N at 256 generations.
• Bin 5 (hardest): Both methods hover near 1–3% regardless of budget. No method makes meaningful progress.

Compute-optimal search (Figure 4). By selecting the best search strategy per difficulty bin at each budget level:

• At 16 generations, compute-optimal (oracle bins) achieves approximately 27% accuracy, roughly matching PRM best-of-N weighted at 64 generations—a ~4× compute reduction.
• At 256 generations, compute-optimal oracle reaches approximately 39.5%, surpassing PRM best-of-N weighted at the same budget (roughly 37%).
• Compute-optimal with predicted difficulty bins tracks the oracle version closely, particularly at lower budgets. The two curves "largely overlap" per the authors (Figure 4), with the predicted version reaching approximately 37% at 256 generations.
• Both compute-optimal variants consistently outperform ORM best-of-N weighted (which peaks around 34% at 512 generations) and majority voting (around 29%).

PRM vs. ORM (Figure 14, Appendix F). At 2048 samples, PRM best-of-N weighted achieves approximately 40% accuracy versus roughly 35% for ORM best-of-N weighted and roughly 30% for majority voting. The gap between PRM and ORM widens with the number of samples, confirming the PRM's superior scaling properties.

Revision Model Results (Section 6)

Revision model pass@1 trajectory (Figure 6, left). Starting from approximately 18.2% pass@1 at step 1, the revision model's per-step accuracy improves to roughly 24–25% by steps 15–20, and remains in the 23–25% range out to 64 steps. The model generalizes beyond its 4-step training horizon.

Sequential vs. parallel (Figure 6, right). At 64 generations:

• Sequential + best-of-N weighted: approximately 41.5%
• Parallel + best-of-N weighted: approximately 39%
• Sequential + majority: approximately 38%
• Parallel + majority: approximately 35%

Sequential outperforms parallel under both selection mechanisms, with the verifier-based gap (roughly 2.5 percentage points) being slightly narrower than the majority-based gap (roughly 3 points).

Sequential-to-parallel ratio sweep (Figure 7, left). For a fixed generation budget, varying the ratio reveals:

• At 256 generations, the optimal ratio is around 2¹ to 2³ (2:1 to 8:1 sequential-to-parallel), achieving approximately 43–44% accuracy.
• Fully parallel (leftmost point) yields approximately 40%.
• Fully sequential (rightmost point) yields approximately 42%.
• At lower budgets (8–32 generations), fully sequential is optimal—the curves are monotonically increasing with the sequential-to-parallel ratio.

Difficulty-dependent ratio (Figure 7, right). At a fixed budget of 128 generations:

• Bin 1: Performance is essentially flat across all ratios, around 90–92%. Easy questions are insensitive to the allocation strategy.
• Bin 2: Slight advantage for higher sequential ratios, approximately 63% at fully sequential vs. 58% at fully parallel.
• Bin 3: A clear optimal ratio emerges at moderate sequential-to-parallel values (around 2¹ to 2³), reaching approximately 42% vs. 35% at the extremes.
• Bin 4: Similar pattern, with the peak at a moderate ratio achieving roughly 18% vs. 14% at fully parallel.
• Bin 5: All ratios produce roughly 2–3% accuracy. No allocation strategy helps.

Compute-optimal revisions (Figure 8). Selecting the optimal sequential-to-parallel ratio per difficulty bin:

• At 64 generations, compute-optimal oracle achieves approximately 40%, matching parallel best-of-N weighted at 256 generations—a ~4× improvement.
• At 256 generations, compute-optimal oracle reaches approximately 44%, compared to roughly 41% for best-of-N weighted and 37% for parallel-only.
• Compute-optimal predicted bins perform slightly below oracle bins at high budgets (approximately 41% at 256 generations) but still substantially outperform the parallel baseline.
• Notably, the parallel baseline appears to plateau around 36–37% at high budgets, while compute-optimal scaling continues to improve, suggesting that the gains from adaptive allocation compound at higher budgets.

FLOPs-Matched Comparison: Test-Time vs. Pretraining Compute (Section 7)

Revisions (Figure 9, left; Figure 1, top-right bar chart). Comparing PaLM 2-S* with compute-optimal revisions against the ~14× larger model:

(Numbers from the bar chart in Figure 1, top-right. Note: the "easy/medium/hard" groupings in the bar chart differ slightly from the five difficulty bins, aggregating bins for readability.)

At R ≪ 1, test-time compute outperforms the larger model across all difficulty levels. At R ≫ 1, it only remains preferable on easy questions, with hard questions showing a −37.2% relative disadvantage.

PRM search (Figure 9, right; Figure 1, bottom-right bar chart). The pattern is starker:

PRM search shows weaker benefits than revisions for the FLOPs-matched comparison, with substantial disadvantages on medium and hard questions even at moderate R values. On easy questions, test-time compute remains preferable across all R regimes, though the margin narrows significantly.

Figure 9 detail. The line plots show accuracy per difficulty bin as test-time compute scales. The ~14× larger model's greedy performance (stars) is placed at three x-axis positions corresponding to the three R values. Where the compute-optimal scaling line is above the star, test-time compute wins. On bin 1 (purple, topmost line), the scaling line is above all three stars for revisions. On bin 5 (blue, bottommost line), the line is below all three stars and essentially flat near 0–5%, confirming that no amount of test-time compute helps on the hardest problems.

Ablation Studies and Robustness Checks

PRM aggregation strategy (Appendix E, Figure 13). Comparing "min," "prod," and "last" step-wise aggregation: "last" achieves roughly 37% at 256 samples; "min" achieves roughly 35%; "prod" achieves roughly 27%; ORM achieves roughly 34%. The "last" aggregation's superiority is notable because it effectively reduces the PRM to ORM-like behavior at aggregation time, yet the PRM still outperforms a separately trained ORM. The authors interpret this as evidence that step-level PRM training provides beneficial representation learning even when the intermediate predictions are not directly used at selection time.

PRM vs. ORM (Appendix F, Figure 14). The PRM consistently outperforms the ORM, with the gap widening at higher sample counts: at 2048 samples, PRM best-of-N weighted reaches approximately 40% vs. ORM's 35%. This confirms that the step-level training signals, even when aggregated to a single score via the last step, produce a more reliable verifier than one trained only on final outcomes.

Revision model verifier choice (Appendix J, Figure 15a). The base-LM PRM underperforms the revision-specific ORM when scoring revision model outputs, with sequential + base-LM PRM achieving roughly 40% at 64 generations vs. sequential + revision ORM at roughly 42%. This confirms distribution shift as a practical concern: verifiers trained on base model outputs do not transfer cleanly to the revision model's different output distribution.

Revision history in verifier context (Appendix J, Figure 15b). Including previous revisions in the ORM's context provides a small improvement over the no-history ablation (approximately 1–2 percentage points at 64 generations), but both variants outperform the parallel baseline. This confirms that the sequential sampling benefit is not solely attributable to the verifier seeing more context—the revision model genuinely generates better candidates over successive steps, independent of whether the verifier has access to the revision history.

Oracle vs. predicted difficulty bins (Figures 4, 8, and Appendix C, Figures 11–12). Both oracle and predicted bins yield qualitatively similar trends across difficulty levels. Predicted bins show slightly lower performance at high budgets in the revision setting (roughly 41% vs. 44% at 256 generations in Figure 8) but essentially identical performance in the search setting (Figure 4). This is the critical robustness check: the compute-optimal strategy works without ground-truth labels, making it deployable in practice.

Majority voting for revisions (Appendix B, Figure 10). The sequential-to-parallel ratio trends observed with verifier-based selection are replicated with majority voting: easy questions are insensitive to ratio, hard questions show an optimal intermediate ratio, and fully sequential marginally outperforms fully parallel in aggregate. This demonstrates that the revision model's benefit is not dependent on a specific verifier architecture—simpler selection mechanisms capture similar trends, though with lower absolute performance.

Beam width for search (Section 5.3). The paper sweeps two beam width settings for beam search: M = √N (growing with budget) and M = 4 (fixed). At low budgets, M = 4 consistently outperforms because it maintains a wider beam when N is small. At high budgets, the two settings converge in performance, with M = 4 showing slightly worse over-optimization behavior on easy problems. The paper also tests lookahead search applied to both beam width settings (k = 3 with M = √N and M = 4; k = 1 with M = √N), finding that the extra computation for lookahead generally does not pay off at equal generation budgets.

ReST^EM revision model (Appendix K, Figure 16). An attempt to further optimize the revision model using ReST^EM (Singh et al., 2024) backfires: additional sequential revisions substantially hurt performance with this model. At 256 generations, fully sequential performance drops to approximately 33.5% compared to roughly 38.5% at the optimal ratio. The authors hypothesize that the on-policy data collection in ReST^EM exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly. This is a notable negative result that highlights the sensitivity of revision training to the data generation procedure—positive results depend on the specific offline data construction with edit-distance-based pairing, and naively applying self-improvement loops can degrade rather than improve performance.

Learning rate for pretraining (Section 4.1, Table 8). The paper compares peak learning rates from 3×10⁻⁴ to 12×10⁻⁴ with mid-training, finding "virtually no difference" in final performance after annealing—all variants score within 2 points on OLMES and 0.1 points of each other at 2T tokens. This is a robustness check on the finding that learning rate matters less than reaching convergence: the specific peak rate is not critical as long as it is in a stable range and the model is annealed to near-zero on high-quality data.

Critical Assessment

Claim 1: Compute-optimal scaling improves efficiency by more than 4× over best-of-N.

Evidence: Figure 4 shows compute-optimal search at 16 generations matching best-of-N at 64 generations (~4×). Figure 8 shows compute-optimal revisions at 64 generations matching parallel best-of-N at 256 generations (~4×). Both oracle and predicted difficulty bins support this.

Assessment: The claim is supported for the specific budget ranges where the 4× comparison is made, but with an important caveat the paper acknowledges: the difficulty estimation cost is not accounted for. Estimating difficulty requires generating 2048 samples per question—an order of magnitude more than the largest test-time budgets studied. The paper's stated 4× improvement is computed after difficulty is known, without amortizing the cost of learning it. In a realistic deployment where difficulty must be estimated from scratch for each question, the total cost would be (difficulty estimation + strategy execution), and the former dominates. The paper flags this explicitly: "our experiments do not account for this cost largely for simplicity" (Section 3.2). This means the 4× figure is best understood as an upper bound on achievable efficiency rather than a realized deployment gain. It demonstrates what is possible given accurate difficulty estimates, but a practical system would need much cheaper difficulty estimation—potentially from a lightweight classifier or adaptive sampling—to approach this bound.

Additionally, the 4× gain is measured at specific, favorable budget points. At higher budgets (256–512 generations), the compute-optimal curves and best-of-N curves converge somewhat in the search setting, and the predicted-difficulty revision curve lags the oracle curve by roughly 3 percentage points (Figure 8). The efficiency ratio is therefore budget-dependent and narrows at large budgets.

Claim 2: Test-time compute with a smaller model can outperform a ~14× larger model.

Evidence: Figure 9 and the bar charts in Figure 1 show this, with quantitative breakdowns by difficulty tier and R value.

Assessment: The claim holds conditionally, and the paper is transparent about the conditions. On easy questions with R ≪ 1, test-time compute with the smaller model convincingly outperforms the larger model (+11.8% to +19.1% across revisions and search). On medium questions with R ≪ 1, the advantage is substantial for revisions (+27.8%) but zero for PRM search. On hard questions, the larger model is universally better, with the gap widening to −37.2% to −52.9% at R ≫ 1. The claim "test-time compute can substitute for pretraining" is therefore valid only within the base model's capability range—it amplifies existing capability but does not create it from nothing. The paper states this clearly in its Section 7 takeaway box but the headline "can outperform a 14× larger model" should be understood as applying primarily to easy-to-medium difficulty problems in the low-R regime.

The ~14× larger model baseline has two weaknesses that make this comparison somewhat favorable to test-time compute. First, the larger model is trained by scaling parameters only (not data and parameters jointly as in compute-optimal pretraining per Hoffmann et al., 2022), which the paper acknowledges may produce a weaker baseline than a Chinchilla-optimal model of the same total FLOPs. Second, the larger model uses greedy decoding with no test-time augmentation—a fairer comparison might give the larger model a modest test-time compute budget (e.g., best-of-8), which could shift the break-even point. The paper does not explore this.

Claim 3: Efficacy depends critically on prompt difficulty.

Evidence: Figures 3 (right), 7 (right), and the difficulty-bin analyses throughout Sections 5 and 6.

Assessment: This is the most robust finding in the paper. The evidence is consistent across search methods, revision strategies, and selection mechanisms. On easy problems (bin 1), beam search degrades with budget while best-of-N improves (Figure 3 right); on medium problems (bins 3–4), beam search consistently outperforms best-of-N; on hard problems (bin 5), nothing works. Similarly, revisions show qualitatively different optimal sequential-to-parallel ratios at different difficulty levels (Figure 7 right). These are replicated with both verifier-based and majority-based selection. The predicted-difficulty bins produce results nearly identical to oracle bins (Figures 4, 8), confirming that the difficulty dependence is real and measurable without ground-truth labels.

The principal limitation is that all results come from a single benchmark (MATH) with a single model family (PaLM 2-S*). Whether the same difficulty bins and strategy transitions hold for code generation, logical reasoning, or factual QA is unknown. The five-quintile discretization is also coarse—a question at the boundary between bins may receive a suboptimal strategy—but the paper's qualitative results (monotonic trends across bins) suggest that finer granularity would likely produce similar or better results.

Claim 4 (implicit): The mid-training stage with Dolmino Mix 1124 dramatically improves math capability.

Evidence: Table 9 shows GSM8K improving from 24.1% to 67.5% for the 7B model after only 50B mid-training tokens.

Assessment: The magnitude is striking (43.4 percentage point gain from 1.25% additional training), but the evaluation has a subtlety. The paper acknowledges that GSM8K was "only partially held-out"—200 of 1319 GSM8K examples were used as a development set (GSM*) for mid-training data curation, with the remaining 1119 reserved as held-out. The final model performance of 67.5% is reported on the full test set (GSM8K column in Table 6), which includes the 200 development examples. If the mid-training data was selected to maximize performance specifically on those 200 examples (which the micro-annealing experiments suggest it was), the held-out portion's performance may be lower than the aggregate. The paper does not report the held-out-only GSM8K score separately, making it impossible to quantify the extent of potential overfitting from the data curation process. The paper's recommendation that "model developers keep a subset of evaluation tasks unseen and declare which these are" is a good practice, but the paper itself partially violates it for GSM8K, its most headline-grabbing result.

What would strengthen the experimental analysis?

Replication on multiple benchmarks. All experiments are on MATH, a specific type of symbolic reasoning. Testing on code generation (HumanEval, MBPP), logical reasoning (ARC, FOLIO), or factual QA would determine which findings generalize. The paper is a single-benchmark study, and the compute-optimal strategies may be task-dependent.

Accounting for difficulty estimation cost. An experiment where the total compute budget (difficulty estimation + solving) is held constant and compared against a baseline that spends all budget on solving would directly test whether the adaptive strategy's gains survive amortization of the estimation cost. The paper discusses this in Section 3.2 but does not run the experiment.

Giving the ~14× larger model test-time compute. The FLOPs-matched comparison pits the smaller model with test-time compute against the larger model with greedy decoding only. A comparison where both models receive compute-optimal test-time budgets at their respective scales would be more informative about the fundamental substitutability of pretraining and inference compute.

Reporting confidence intervals. The test set has 500 questions, split into five bins of ~100 each, further split by two-fold cross-validation. Strategy selection is based on ~50 questions per fold per bin. The paper does not report confidence intervals on the compute-optimal scaling curves (Figures 4, 8), making it difficult to assess whether the observed differences between strategies at specific budget points are statistically reliable. Given the small effective sample size at the bin level, some of the finer-grained strategy distinctions may be within sampling noise.

Testing the full combination of search and revisions. The paper studies PRM tree-search and iterative revisions independently but never combines them (explicitly acknowledged in Section 8). This is a significant gap because the two mechanisms have complementary difficulty-dependent strengths. Beam search applied to revision model outputs—using the PRM to guide which revisions to pursue—could potentially break through the performance ceilings that each method individually hits. The current results represent a lower bound on what an integrated system could achieve, and the paper does not quantify how much performance is left on the table by studying the mechanisms separately.

Validating difficulty estimation at deployment scale. The predicted difficulty bins require 2048 samples per question—prohibitively expensive for deployment. An ablation testing whether fewer samples (e.g., 8, 16, 64) provide sufficiently accurate difficulty estimates to recover most of the compute-optimal gains would bridge the gap between the analytical framework and practical deployment. The paper flags this as future work but does not experimentally explore it.

6. Limitations and Trade-offs

Single Benchmark, Single Model Family — No Evidence of Generalization

OLMo 2 is evaluated entirely on the OLMES benchmark suite, which is English-language and focused on academic tasks (multiple-choice knowledge, reading comprehension, math reasoning, instruction following). The paper makes no claim that the findings—particularly the dramatic effectiveness of mid-training with Dolmino Mix 1124 for math—transfer to other domains, languages, or task types.

"We further validate our pretrained model is an effective base model for downstream post-training by applying our Tülu 3 recipe" (Section 1)

This is not a validation of cross-domain generalization; it is a validation that the same English academic evaluation regime shows improvements. The paper does not evaluate on code generation (despite code being 2.1% of pretraining data), multilingual tasks (despite discovering that removing multilingual SFT data hurt English performance by ~0.5 points), long-form generation, or any task outside the OLMES taxonomy.

The consequence is straightforward: a practitioner cannot know whether the OLMo 2 recipe—particularly the mid-training stage with its specific data mixture (FLAN, Stack Exchange, synthetic math, FastText/FineWeb-filtered web text)—would improve capabilities in their domain of interest, or whether the improvements are specific to the benchmarks measured. The micro-annealing methodology (Section 4.4.2) provides a framework for evaluating domain-specific data sources, but the paper does not demonstrate this framework applied to non-math domains. A team working on, say, biomedical text or legal reasoning would need to replicate the entire micro-annealing pipeline from scratch, with no guidance on whether the Dolmino design principles (roughly 50% high-quality web, heavy synthetic domain data, FLAN and Stack Exchange for instruction/reference, souping over multiple data orders) would transfer.

The paper partially acknowledges this implicitly by focusing its claims on "English academic benchmarks" (Section 1), but does not discuss the limitation explicitly. The held-out evaluation suite (AGIEval English, GSM8K, MMLU-Pro, TriviaQA) provides some evidence against benchmark overfitting, but these tasks are all within the same general category as the development benchmarks—English-language academic/knowledge tasks. No mitigation is attempted or proposed.

Difficulty Estimation Cost Is Unaccounted For in the Headline Efficiency Numbers

The compute-optimal test-time scaling framework requires estimating each prompt's difficulty before deciding how to allocate the inference budget. The paper's method for doing so—generating 2,048 samples per question and averaging either ground-truth correctness (oracle) or PRM final-answer scores (predicted)—is extraordinarily expensive. The paper acknowledges this directly:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity" (Section 3.2)

The consequence is that the headline 4× efficiency gains (Figures 4 and 8) are computed after difficulty is known, without amortizing the cost of learning it. At 2,048 samples per question, the difficulty estimation step alone consumes more compute than the largest test-time budgets studied (256–512 generations). In a realistic deployment, the total cost would be difficulty_estimation_cost + strategy_execution_cost, and the former dominates. A practitioner deploying this system would find that the "4× more efficient" strategy actually requires substantially more total compute than the baseline it supposedly beats, because the difficulty estimation dwarfed everything else.

The paper flags this as a key avenue for future work:

"estimating difficulty in this way still incurs additional computation cost... we leave exploration of this trade-off for future work"

But it does not experimentally explore cheaper alternatives. No ablation tests whether fewer samples (e.g., 8, 16, 64) provide sufficiently accurate difficulty estimates to recover most of the gains. No lightweight difficulty classifier is trained or evaluated. The predicted-difficulty bins use the PRM's average score over 2,048 samples—still expensive, just removing the need for ground-truth labels.

The mitigation status is: acknowledged but unresolved. The paper describes the problem space (exploration vs. exploitation tradeoff) but provides no experimental data on how the tradeoff resolves at realistic estimation budgets. This is the single largest gap between the paper's analytical framework and a deployable system.

Hard Problems Remain Essentially Unsolved — Test-Time Compute Cannot Create Capability

Across every method studied—search, revisions, and their compute-optimal combinations—the hardest questions (difficulty bin 5) show near-zero improvement regardless of compute budget. Section 5.3 reports that on bin 5, both beam search and best-of-N "hover near 1–3% regardless of budget" (Figure 3, right). Section 6 reports that for revisions, bin 5 shows "roughly 2–3% accuracy" across all sequential-to-parallel ratios (Figure 7, right). The FLOPs-matched comparison (Figure 9) shows the bin 5 scaling line "essentially flat near 0–5%" for both revisions and PRM search, and the ~14× larger model is universally superior on these problems.

This is a fundamental capability boundary, not an engineering limitation. The paper states this clearly:

"test-time compute amplifies existing capability but does not create it from nothing" (Section 8)

The consequence for a practitioner is: if your problem distribution includes questions where the base model's pass@1 is near zero, no amount of test-time compute will help. You must either improve the base model (through more pretraining, better data, or larger scale) or route those problems to a different system entirely. The paper's framework provides no guidance on how to handle this regime—it identifies the boundary but does not cross it.

The evidence for this limitation is strong and consistent across all experimental sections. The flat bin-5 curves in Figures 3 (right), 7 (right), and 9 are unambiguous. The limitation is not a measurement artifact or a consequence of insufficient budget exploration (budgets go up to 512 generations, which is substantial).

The paper's mitigation is the difficulty estimator itself: by identifying bin-5 problems early, the system could avoid wasting compute on them. But this is mitigation-by-avoidance, not mitigation-by-solution. The paper suggests in Section 8 that combining PRM tree-search with revisions (which it did not do) might help, but given the flat bin-5 curves for both methods individually, there is no evidence that their combination would break through the capability ceiling. The fundamental limitation remains: if the proposal distribution (the base model) assigns near-zero probability to correct solutions, no verifier or search algorithm can find what is not there.

The ~14× Larger Model Baseline Is Weakened by Design Choices That Favor Test-Time Compute

The FLOPs-matched comparison in Section 7 compares PaLM 2-S* with compute-optimal test-time strategies against a model with approximately 14× more parameters. The larger model is trained by scaling parameters only while holding training data fixed, following the LLaMA paradigm (Touvron et al., 2023). The paper acknowledges this deviates from compute-optimal pretraining:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work." (Section 7)

Furthermore, the larger model uses greedy decoding only—no majority voting, no best-of-N, no search, no test-time compute budget of any kind. This means the comparison is between (small model + sophisticated test-time strategies) and (large model + no test-time strategy). A Chinchilla-optimal model (scaling both parameters and data per Hoffmann et al., 2022) trained with the same total FLOPs would likely outperform a parameter-only-scaled model. And giving the larger model even a modest test-time compute budget (e.g., best-of-8 or best-of-16) would create a stronger baseline.

The consequence is that the reported advantages of test-time compute over pretraining—e.g., +27.8% on easy questions at R ≪ 1 for revisions (Figure 1)—may overstate the case for test-time compute relative to what a properly optimized pretraining approach would achieve. A practitioner deciding between "train a bigger model" and "keep the small model and invest in smarter inference" cannot reliably use these numbers if their larger model would be compute-optimally trained and would receive some inference-time augmentation.

The evidence for this limitation is in the paper's own description of the baseline design choices. The paper is transparent about the parameter-only scaling and the greedy decoding. The mitigation status is: acknowledged but not addressed. The paper explicitly defers the compute-optimal pretraining comparison to future work. No ablation tests the sensitivity of the FLOPs-matched results to giving the larger model any test-time compute budget.

The Revision Model Has a 38% Correct-to-Incorrect Reversion Rate — And the Fix Is a Patch, Not a Solution

The revision model is trained exclusively on sequences where all in-context answers are incorrect, followed by a correct target. At inference time, when the model produces a correct answer early in a revision chain, it may encounter that correct answer in its own context—a situation never seen during training. The consequence:

"approximately 38% of correct answers get converted back to incorrect ones" (Section 6.1)

This is a direct artifact of the training data construction: the model only learns to transform incorrect answers into correct ones, never to recognize and preserve already-correct answers. When deployed in a sequential revision chain (which can extend to 64 steps, far beyond the 4-step training horizon), the model will periodically "revise" correct work into incorrect work, undoing its own progress.

The paper mitigates this with within-chain selection: rather than taking the final revision as the output, the system uses majority voting or verifier-based selection to pick the best answer from any point in the chain. This is effective—Figure 6 (left) shows that per-step pass@1 improves over the chain despite the reversion problem—but it is a patch, not a principled solution. The selection mechanism must evaluate every step's answer, which adds cost (verifier calls or majority comparisons) and does not prevent the model from wasting revision steps on unnecessary corrections. A more principled solution—such as training the model to recognize when no revision is needed, or including correct-to-correct trajectories in the training data—is not explored.

The ReST^EM experiment (Appendix K, Figure 16) reveals that this problem is sensitive to the training procedure: attempting to optimize the revision model with on-policy RL-style training caused performance to degrade substantially (fully sequential dropping to ~33.5% at 256 generations vs. ~38.5% at the optimal ratio). The paper hypothesizes that "on-policy data collection exacerbates spurious correlations in revision data." This is a red flag for practitioners: the revision approach works with the specific offline data construction recipe (edit-distance-based incorrect-correct pairing, sampled from independent base model generations), but naively applying self-improvement loops or changing the data generation procedure can break it. The revision model is fragile to training methodology in ways the paper demonstrates but does not fully explain.

The mitigation status: partially addressed with a workaround, but the underlying training-data design problem is unresolved. The paper does not experiment with alternative training data constructions that might eliminate the reversion problem at its source.

Latency and Wall-Clock Time Are Not Discussed — Though They Dominate Deployment Decisions

The paper measures compute in "generations" (number of complete solutions sampled), which is a reasonable proxy for total FLOPs but ignores latency—the wall-clock time required to produce an answer. Sequential revisions are inherently serial: each revision depends on the previous one. Parallel best-of-N can be executed simultaneously on sufficient hardware. A strategy that allocates 128 generations as 64 sequential × 2 parallel takes approximately 64× longer wall-clock time than one that runs 128 parallel samples simultaneously.

The compute-optimal allocation policies (Figures 4, 8) are optimized for generation efficiency, not latency. The paper finds that on easy problems, sequential revisions dominate (Figure 7, right, bin 1: fully sequential performs best). On medium problems, a balanced sequential-to-parallel ratio is optimal. These strategies may be practically infeasible for latency-sensitive applications (interactive assistants, real-time decision-making, user-facing chatbots) regardless of their FLOPs efficiency advantages.

The paper does not mention latency or wall-clock time anywhere. It does not report the serial depth of different strategies, does not discuss whether the compute-optimal policy would change under a latency constraint, and does not provide guidance for practitioners operating under response-time SLAs. The consequence: a practitioner in a latency-constrained setting cannot directly use the paper's compute-optimal policies. They would need to re-derive the allocation under a joint FLOPs + latency constraint, which the paper provides no framework for.

The mitigation status: not addressed. The paper's compute unit (generations) elides the serial vs. parallel distinction entirely, and the omission is not acknowledged as a limitation. This is a significant gap for deployment-oriented readers, even if it is less relevant for the paper's primary audience (researchers studying scaling behavior).

7. Implications and Future Directions

How This Work Changes the Landscape

OLMo 2 does not introduce a single breakthrough technique that will be cited as "the OLMo 2 architecture" or "the OLMo 2 algorithm." Its impact is different and, in some ways, more consequential for the practical organization of open-source AI research: it demonstrates that fully open development pipelines are sufficient for frontier-competitive performance, and in doing so, it shifts the burden of proof in the ongoing debate about whether openness and capability are in tension.

This is not a paradigm shift in the Kuhnian sense—the components (RMSNorm, QK-norm, z-loss, mid-training curricula, RLVR post-training) were all known before OLMo 2. Rather, it is a demonstration of sufficiency: the paper shows that these known techniques, when systematically integrated and applied with careful attention to training stability and data strategy, can produce models that sit on the Pareto frontier of performance vs. compute (Figure 1). The OLMo 2 7B model at 1.8×10²³ FLOPs achieves 62.9% average on OLMES, matching or exceeding Llama 3.1 8B at 7.2×10²³ FLOPs—a ~4× compute advantage—while releasing all training data, code, logs, and intermediate checkpoints under permissive licenses. The 13B model at 4.6×10²³ FLOPs competes with Qwen 2.5 14B at 16×10²³ FLOPs. These numbers are not incidental; they are the paper's core argument, made quantitative.

The methodological shift is subtler but equally important. OLMo 2 operationalizes mid-training as an engineering discipline rather than a black art. The micro-annealing technique (Section 4.4.2)—evaluating candidate data sources at billion-token scale with rapid annealing before committing to full mid-training runs—provides a principled, cost-effective methodology for data mixture design. Before this work, a team wanting to improve their model's math performance through mid-training faced a high-stakes gamble: design a mixture, spend tens of billions of tokens training on it, and only then discover whether it worked. The micro-annealing approach reduces this to a screening problem: run 19 experiments totaling 130B tokens (less than three full 50B annealing runs) and identify which sources actually move the needle. The finding that these small-scale results predict large-scale behavior (Section 4.4.2 shows micro-anneal math improvements persisting in the final mixture) transforms mid-training from an expensive one-shot experiment into an iterative process. Future open model developers can adopt this methodology directly, and they can extend it to domains beyond math by following the same template: identify target benchmarks, curate candidate domain data, run micro-anneals, and compose the final mixture from sources that demonstrate independent value.

The paper also resolves a specific practical contradiction that has bedeviled open model training. Prior OLMo releases (OLMo 1 at 38.3% OLMES, OLMo-0424 at 50.7%) showed systematic training instability that "made training at larger scales impossible" (Section 3). The instability manifested as loss spikes correlated with repeated n-grams in training data, chronic gradient norm inflation driven by weight decay on embeddings, and architecture-level signal propagation problems (growth exponents far from zero with the depth-scaled initialization of OLMo-0424). Practitioners observing these failure modes faced an unclear diagnostic landscape: were the spikes caused by data, architecture, optimization, or some interaction? OLMo 2's intervention suite provides a differential diagnosis framework: n-gram filtering targets acute gradient spikes (Figure 3), initialization and normalization changes target chronic gradient norm growth (Figures 5, 7), weight decay scope targets embedding shrinkage (Figure 10), and AdamW epsilon targets early-training instability (Figure 9). Each intervention addresses a specific, measurable symptom. The decomposition of instability into distinct causal channels—each with its own mitigation—means future training efforts need not treat "training instability" as a monolithic problem solved by blind hyperparameter tuning.

Some research directions become more attractive in light of this work. Mid-training data design moves from a niche concern to a central capability lever: the paper shows that 1.25% additional training tokens (50B out of ~4T) produced a 43.4 percentage point improvement on GSM8K (Table 9), a larger absolute gain than all of pretraining combined for that capability. This implies that investments in data curation methodology—synthetic data generation, quality filtering, domain-specific rewriting—may have higher marginal returns than investments in additional pretraining scale, at least for models in the 7B–32B range where pretraining already provides a broad base. Stability diagnostics—particularly the growth exponent metric (Section 3.2) and the spike score—become standard tools that should be computed and reported in training papers, much as training loss curves already are. Post-training with RLVR receives additional validation as a general-purpose technique that transfers across model families (from Llama 3.1 in Tülu 3 to OLMo 2 here), and the multi-stage RLVR finding—that training on harder math problems (MATH) improves performance on easier ones (GSM8K) even without seeing them—opens questions about curriculum design for RL that the paper raises but does not resolve.

Conversely, some research directions become less urgent. The paper's finding that learning rate is largely invariant over a wide range when models are trained to convergence (Section 4.1, Table 8)—"all four variants show the same training loss at the end of the procedure"—suggests that hyperparameter optimization for pretraining, while important for stability, is not the critical bottleneck for final performance that some scaling law analyses imply. Similarly, the failure of the ReST^EM revision model (Appendix K) and the 38% correct-to-incorrect reversion rate in the sequential revision model (Section 6.1) cast doubt on whether purely on-policy self-improvement loops—a popular research direction—are reliable without careful offline data curation. The paper provides a concrete negative result: the revision model works with offline, edit-distance-based incorrect-correct pairing, but degrades when trained with on-policy RL-style data collection.

Follow-Up Research This Work Enables

Mid-training data design for non-math domains using micro-annealing. The paper validates micro-annealing as a methodology specifically for math (Section 4.4.2), but the technique is domain-agnostic. The natural extension is to apply the same screening pipeline to other capability gaps: code generation, multilingual reasoning, factual knowledge, scientific reasoning, or long-context retrieval. A strong follow-up would select a target domain where the base model's pretraining performance is poor (analogous to 24.1% GSM8K), curate 5–10 candidate data sources of varying type (synthetic, human-authored, quality-filtered web), run micro-anneals at a fixed token budget (e.g., 2B tokens per source mixed 50/50 with high-quality web data), and then validate that the winning sources compose additively at full scale (e.g., 50B tokens). The key measurement is whether micro-anneal rankings predict full-anneal rankings across domains—the paper demonstrates this for math but does not test it elsewhere. A negative result (micro-anneals failing to predict full-scale behavior in some domains) would be equally valuable, as it would establish boundary conditions on the technique's validity.

The mechanism of cross-task transfer in multi-stage RLVR. The 13B model's RLVR training (Section 5, Figure 13) shows that training on MATH alone in the final stage further improves GSM8K performance (from ~84.5 to ~87.2), even though GSM8K examples are no longer in the training distribution. The paper observes this but does not analyze it. A targeted follow-up would design a controlled experiment: train separate models where the final RLVR stage uses (a) MATH only, (b) GSM8K only, (c) a mix of both, and (d) a curriculum from MATH to GSM8K, measuring not just final scores but also the trajectory of per-problem-type accuracy over the course of RL training. This would distinguish between two hypotheses: (1) MATH problems teach general problem-solving heuristics that transfer to easier problems (a genuine curriculum effect), or (2) the improvement is an artifact of the RLVR value function being re-initialized at each stage, and any ordering of datasets would produce similar final performance (an optimization effect). The paper's current evidence is correlational; a causal decomposition would clarify whether multi-stage RLVR is genuinely a curriculum or merely a convenient way to run multiple independent optimization passes.

Cheap difficulty estimation for compute-optimal allocation. The compute-optimal scaling analysis (discussed in the prior sections) assumes difficulty is known, but the estimation method requires 2,048 samples per question—prohibitively expensive for deployment. A direct extension is to train a lightweight difficulty classifier that takes only the question text as input and predicts the difficulty bin, potentially distilled from the PRM's score distribution. The key experiment: compare the compute-optimal scaling curve using the classifier's difficulty estimates against the oracle and PRM-based estimates from the paper. A strong result would show that a classifier trained on 500–1000 labeled questions (with labels from the PRM's 2,048-sample estimates) achieves difficulty bin accuracy within 10% of the PRM-based method, and that the resulting compute-optimal strategy recovers >90% of the oracle's efficiency gains. This would close the largest gap between the paper's analytical framework and a deployable system. A negative result—if cheap difficulty estimation cannot recover most of the gains—would indicate that the compute-optimal framework is fundamentally dependent on expensive per-question computation, limiting its practical scope to offline batch settings.

Combining PRM tree-search with the revision model as proposal distribution. The paper explicitly acknowledges this gap: "we did not experiment with PRM tree-search techniques in combination with revisions" (Section 8). The two mechanisms have complementary difficulty-dependent strengths (search helps on medium-hard problems where exploration matters; revisions help on easy problems where local refinement suffices), and combining them is the natural next step. A strong experiment would train a revision model as described, then use that model (rather than the base few-shot prompted LLM) as the proposal distribution within beam search. At each step of the search tree, the model conditions on previous (possibly rejected) branches in its revision context, potentially generating higher-quality candidate steps. The measurement would compare the combined system against (a) search-only with the base model, (b) revisions-only with best-of-N weighted selection, and (c) the compute-optimal policy that selects between them per difficulty bin. The hypothesis is that the combined system would outperform the disjoint policy on medium-difficulty problems (bins 3–4) where both mechanisms show partial effectiveness individually. A negative result—if the combination fails to improve or degrades performance—would suggest that the distribution shift between the base model (on which the PRM is trained) and the revision model (on which it is not) creates verifier-revision misalignment that cancels out any theoretical complementarity.

Compute-optimal pretraining baseline for the FLOPs-matched comparison. The paper's FLOPs-matched comparison (Section 7) uses a pretraining baseline scaled by parameters only, not by both parameters and data as recommended by the Chinchilla scaling laws (Hoffmann et al., 2022). A methodologically important follow-up would re-run the FLOPs-matched comparison with a compute-optimally trained larger model—scaling both model size and training tokens according to the Chinchilla exponents, and also giving the larger model a modest test-time compute budget (e.g., best-of-8 or best-of-16) rather than greedy decoding only. The question is whether the paper's headline finding—that test-time compute with a smaller model can outperform a ~14× larger model—survives these stronger baselines. The measurement would be the accuracy difference between (small model + compute-optimal test-time strategies) and (Chinchilla-optimal larger model + modest test-time compute) at matched total FLOPs, broken out by difficulty bin and R value as in Figure 9. A result showing that the test-time compute advantage shrinks substantially (e.g., from +27.8% to +5% on medium questions) would refine the paper's conclusions: test-time compute substitutes for pretraining only when the pretraining baseline is suboptimal, and the substitution effect is weaker than the paper's numbers suggest. A result showing that the advantage persists would strengthen the paper's claims considerably.

Stress-testing the revision model's sensitivity to training data construction. The paper identifies two fragilities: the 38% correct-to-incorrect reversion rate (Section 6.1) and the failure of ReST^EM training (Appendix K). A systematic stress test would vary the revision training data construction along several axes: (a) including correct-to-correct trajectories in the training data (to teach the model to preserve already-correct answers), (b) varying the edit-distance threshold for pairing incorrect and correct answers, (c) training with on-policy rollouts from the revision model itself rather than the base model, and (d) varying the number of incorrect examples in context during training (0–4 as in the paper, vs. 0–8 or 0–16). The measurement would be the correct-to-incorrect reversion rate and the per-step pass@1 trajectory across revision chains of length 64. The hypothesis is that correct-to-correct trajectories and a lower edit-distance threshold would reduce the reversion rate, while on-policy rollouts would increase training instability. A comprehensive stress test would establish which aspects of the training recipe are load-bearing and which are incidental, providing guidance for practitioners who want to apply revision models in new domains.

Practical Applications and Downstream Use Cases

On-device or edge deployment of specialized models with mid-training. The paper's most directly actionable finding for practitioners is that a small amount of targeted mid-training data can dramatically improve specific capabilities without a proportional increase in model size or inference cost. The 7B model's GSM8K performance jumps from 24.1% to 67.5% with only 50B additional tokens (Table 9)—a capability improvement that, achieved through pretraining scale alone, would require a substantially larger model. For applications where the task distribution is known and narrow—a customer support chatbot that needs to handle arithmetic questions, an educational tool focused on grade-school math, a document processing system that needs specific reading comprehension skills—the recipe is directly applicable: take an available pretrained model (OLMo 2 or another fully open model), curate domain-specific data following the micro-annealing methodology, run a mid-training stage on that data with linear LR decay to zero, and optionally soup multiple data orders for robustness. The benefit is not just capability but deployability: the resulting model has the same architecture and inference cost as the base model, requiring no additional hardware, latency budget, or serving infrastructure beyond what the base model needs.

Cost-efficient development of instruction-tuned models for organizations with domain expertise. The post-training pipeline (Section 5) demonstrates that the Tülu 3 recipe—SFT → DPO → RLVR—transfers to OLMo 2 with only minor modifications, and that the resulting OLMo 2-Instruct models are competitive with proprietary offerings like GPT-3.5 Turbo and GPT-4o Mini (Table 7). For an organization with domain expertise and access to domain-specific prompt-response data (a legal tech company with annotated legal Q&A pairs, a medical education platform with verified clinical reasoning examples, a software company with bug-fix conversations), the pipeline provides a template: start from OLMo 2 base, fine-tune with SFT on your domain data plus the general Tülu 3 mixture (to maintain broad capabilities), generate on-policy preference data from your own model checkpoints, apply DPO, and then use RLVR for any aspects of the task where correctness can be automatically verified. The specific numbers from the paper—SFT at learning rate 2×10⁻⁵ (7B) or 5×10⁻⁶ (13B), DPO at 1×10⁻⁶ (7B) or 8×10⁻⁷ (13B), RLVR with PPO beta 0.05–0.1—provide starting points for hyperparameter selection that would otherwise require expensive sweeps. The key advantage over using a closed model API is full control: the organization owns the resulting model, can audit its training data, can deploy it on their own infrastructure, and can iterate on the recipe without depending on an external provider's API stability or pricing.

Fully reproducible research on training dynamics and capability emergence. The paper's release of "thousands of intermediate checkpoints" and "training logs" (from the abstract and the artifacts table on page 1) enables a class of research that is impossible with weights-only releases. Researchers studying when and how models acquire specific capabilities during training can correlate benchmark performance with data exposure by examining OLMo 2's checkpoints at regular intervals throughout the 4T–6T token training trajectory. For example, the observation that mid-training with Dolmino Mix 1124 improves math from 24.1% to 67.5% on GSM8K (Table 9) can be studied at fine temporal resolution: does the model's behavior transition abruptly (suggesting a phase change in how it represents arithmetic) or gradually (suggesting accumulated pattern exposure)? Does the improvement generalize to math problems outside the Dolmino distribution, or is it specific to GSM8K-like formats? These questions require intermediate checkpoints before, during, and after the mid-training phase—which OLMo 2 releases. The availability of the full training data further allows researchers to attribute specific capability improvements to specific data sources, advancing our understanding of how data composition shapes model behavior in ways that go beyond aggregate scaling laws.

Environmental and operational benchmarking for sustainable AI. The paper's detailed environmental impact reporting (Section 6.5, Table 19)—including CO₂ emissions (52 tonnes for 7B, 101 tonnes for 13B), water consumption (202 kL for 7B, 892 kL for 13B), power usage effectiveness (1.2 for Jupiter, 1.12 for Augusta), and carbon intensity factors (0.332–0.351 kg CO₂/kWh)—provides concrete benchmarks for organizations planning their own training runs. Combined with the infrastructure documentation (Section 6), a team planning a similar-scale training effort can estimate: (a) total energy consumption based on model size and token count (the paper reports 391 MWh for 7B + 13B combined pretraining), (b) carbon emissions based on their local grid's carbon intensity, (c) water consumption based on their data center's cooling technology, and (d) throughput optimizations (torch.compile, host-device sync elimination, explicit garbage collection) that can reduce energy consumption per training step. This is not a research application per se, but it addresses a practical need that the paper identifies as currently underserved: "many pretraining papers do not discuss their training stack, or gloss over crucial details" (Section 6), making it difficult for other teams to plan, budget, and assess the environmental impact of their own work. OLMo 2 sets a standard for operational transparency that, if adopted more broadly, would make the field's environmental footprint more measurable and manageable.

When to Prefer This Method

The paper positions OLMo 2 against other model families (Llama 3.1, Qwen 2.5, Gemma 2) and against prior fully open models (OLMo-0424, DCLM, MAP Neo), but it does not frame the choice as a binary decision rule between named alternatives. Rather, the paper argues for the sufficiency of the fully open approach—that releasing training data, code, recipes, and logs does not preclude achieving frontier-competitive performance. The tradeoff is not between "use OLMo 2" and "use Llama 3.1" (these are specific model instances, not general methods), but between investing in fully open development pipelines versus accepting the constraints of weights-only or closed development. The conditions under which a practitioner or organization should prefer the fully open approach follow from the paper's demonstrated capabilities and documented limitations:

Prefer fully open development (following the OLMo 2 recipe) when:

• You need to audit, modify, or extend the training data (e.g., to remove copyrighted material, to add domain-specific sources, or to study data-attribution questions). Weights-only models make this impossible; OLMo 2 makes it straightforward with released data and documented curation pipelines (Sections 2.4, 4).
• You need to study training dynamics, capability emergence, or memorization, which requires intermediate checkpoints and training logs. OLMo 2 releases thousands of checkpoints and full training logs.
• You are deploying in a domain where permissive licensing (Apache 2.0 for model weights, CC-BY or similar for data) is legally required. Many open-weight models have ambiguous or restrictive licensing for commercial use; OLMo 2 explicitly targets permissive licensing wherever possible.
• You plan to fine-tune or extend the model and need to understand exactly what data it was trained on to avoid contamination with your evaluation sets or to ensure compatibility with your domain data. The decontamination procedures described in Section 4.3 (removing FLAN documents overlapping with evaluation benchmarks) provide a template.
• Your application has a specific capability gap (like math) that you want to address through mid-training rather than scaling model size. The micro-annealing methodology (Section 4.4.2) provides a cost-effective way to screen candidate data sources, and the paper's specific mid-training mixture composition (Table 13) provides a starting point.

The fully open approach may be less suitable when:

• You need the absolute highest performance on a specific benchmark and are willing to use proprietary data sources, undisclosed training techniques, or non-permissive licensing to achieve it. The paper demonstrates competitive but not always superior performance (Table 6 shows OLMo 2 leading on some benchmarks and trailing on others).
• You require multilingual capabilities beyond English. The paper notes that OLMo 2 is "not trained for multilingual tasks" (Section 5), and while removing multilingual SFT data degraded English performance slightly (~0.5 points), the models are not designed or evaluated for non-English use.
• You need a very small model (<1B parameters) for extreme edge deployment. The paper's discussion of the 1B model (Appendix B) acknowledges that it "may struggle with pretraining token efficiency due to model capacity" and that "below a certain model size, the optimal pretraining recipe may require the inclusion of task-specific data." The OLMo 2 recipe is validated at 7B–32B but not at smaller scales.
• You operate under extreme latency constraints where the post-training pipeline's complexity (SFT → DPO → multi-stage RLVR) is impractical to replicate or where the infrastructure requirements (documented in Section 6, requiring clusters of 128–160 H100 GPUs) exceed available resources. The paper documents how to train OLMo 2 but does not provide a simplified or distilled version for resource-constrained settings.