Olmo 3 — Paper Graph

🎯 Pitch

Training a fully-open 32B model with 6× fewer tokens than Qwen 3 32B yields a thinking model that hits 80.6 on AIME 2024—narrowly trailing the strongest closed-weight competitors—while establishing that open pretraining data makes RL-based reasoning boosts trustworthy by eliminating secret data leakage concerns.

1. Executive Summary

This paper introduces Olmo 3, a family of fully-open language models at the 7B and 32B parameter scales, constructed through a comprehensive model flow spanning pretraining, midtraining, long-context extension, supervised finetuning, preference tuning via Delta Learning (constructing preference pairs with large quality gaps between chosen and rejected responses), and reinforcement learning with verifiable rewards via OlmoRL (an asynchronous GRPO-based framework with continuous batching and inflight weight updates). The flagship model, Olmo 3.1 Think 32B, achieves strong performance across math, reasoning, code, and instruction-following benchmarks—reaching 80.6 on AIME 2024 and 78.1 on AIME 2025, narrowing the gap to the best open-weight models of similar scale—while being trained on roughly six times fewer tokens than Qwen 3 32B. The paper further establishes that reinforcement learning from a base model using Olmo 3 RL-Zero can drastically improve reasoning performance in under a thousand training steps, establishing that fully-open pretraining data enables efficient RLVR without the data leakage concerns that confound benchmarks trained on undisclosed corpora.

2. Context and Motivation

The Core Problem: The Fully-Open Model Gap

The fundamental problem this paper confronts is not simply building another capable language model. Rather, it targets a structural deficiency in the AI research ecosystem: the persistent gap between state-of-the-art performance and fully-open development. Most leading open-weight models—such as Qwen 3 (Yang et al., 2025a), DeepSeek R1 (Guo et al., 2025), and Llama 3 (Grattafiori et al., 2024)—release final model weights but withhold the training data, intermediate checkpoints, data processing code, and training recipes that produced them. This creates what the authors term a "model flow" gap: researchers and practitioners receive an endpoint without access to the full lifecycle that generated it.

The consequences of this gap are both practical and scientific:

Limited reproducibility and auditability. Without access to pretraining data, it is impossible to verify whether a model's benchmark performance reflects genuine capability acquisition or memorization of evaluation data that leaked into the training corpus. This problem has become acute in the RLVR era, where spurious rewards from contaminated training data can produce misleading performance improvements indistinguishable from true reasoning gains (Shao et al., 2025b; Wu et al., 2025c).
Constrained downstream research. When only final weights are available, researchers cannot study how capabilities emerge across training stages, cannot intervene at intermediate points to alter model behavior, and cannot isolate the effects of specific data sources or architectural decisions. The paper frames this explicitly:

"To truly advance open-source AI research and development, we argue that releasing a state-of-the-art language model should make its entire model flow—not just its endpoint—transparent and accessible."
Inability to study pretraining-RL interactions. A concrete example of what the closed model flow prevents: all leading open RLVR benchmarks train on top of open-weight models whose pretraining data is undisclosed (Chu et al., 2025; Yang et al., 2025a). This makes it impossible to study how pretraining data composition affects downstream RL performance—a question of increasing importance as RLVR becomes the dominant post-training paradigm for reasoning models.
Licensing constraints on derivative models. Many "open-weight" releases carry restrictive licenses that prevent commercial use or require attribution chains (e.g., the Llama Community License requiring derivative models to include "Llama" in their name). These constraints, while legally defined, functionally limit how the community can build on released models.

Why This Problem Matters Now

The paper's intervention is timely for several converging reasons:

The post-training revolution has raised the stakes. Prior to 2024, the primary axis of model improvement was pretraining scale—bigger models, more data, more FLOPs. The recipe was relatively monolithic, and releasing final weights provided substantial value. However, the emergence of reasoning models (OpenAI o1, DeepSeek R1, Gemini Thinking) has demonstrated that multi-stage post-training pipelines (SFT → DPO → RLVR) can produce capability leaps that rival or exceed those from pretraining alone. This means the "endpoint-only" release model is increasingly impoverished: the interesting scientific and engineering decisions occur across multiple training stages, each involving distinct data, algorithms, and hyperparameters that downstream researchers cannot inspect or modify.

Data leakage has become a first-order confound. The RLVR paradigm—where models are rewarded for producing correct answers to problems with verifiable ground truth—creates an acute vulnerability to benchmark contamination. If pretraining data contains evaluation examples, RLVR can elicit memorized solutions through "spurious rewards" that are statistically indistinguishable from genuine reasoning. Shao et al. (2025b) demonstrated this directly by showing that RLVR with random (non-informative) rewards improves benchmark performance when training data is contaminated—meaning observed gains may reflect data leakage rather than capability acquisition. The Olmo 3 paper explicitly validates their own decontamination by running this negative control experiment, showing no performance improvement with random rewards (Figure 27), which is only possible because they control and release their full training data pipeline.

The economics of open research are shifting. Training state-of-the-art models requires enormous compute resources—the paper reports approximately 56 days of elapsed time on a 1024-H100-GPU cluster for Olmo 3 Think 32B, costing roughly $2.75M at market rates. Closed model developers can amortize these costs across proprietary products, but open research labs must justify the expenditure through scientific impact. This creates a perverse incentive: if releasing only weights attracts comparable attention to releasing the full pipeline, there is little marginal incentive to do the harder work of cleaning, documenting, and releasing training data and code. The Olmo 3 release pushes against this equilibrium by demonstrating that full transparency is feasible at competitive performance levels, thereby raising the standard for what "open" means in practice.

Where Prior Fully-Open Efforts Fall Short

The paper positions itself relative to a specific lineage of prior fully-open models. Understanding these predecessors is essential for appreciating what Olmo 3 advances:

OLMo 2 (OLMo et al., 2024). The direct predecessor to Olmo 3. OLMo 2 released pretraining data (Dolma), training code, intermediate checkpoints, and post-training recipes. However, OLMo 2 had several significant limitations that Olmo 3 addresses:

No long-context support. OLMo 2 operated at a 4,096-token context window, which is insufficient for the long documents and extended reasoning traces that characterize modern language model use cases.
Limited midtraining data diversity. OLMo 2's midtraining focused primarily on math, lacking the broad capability coverage (code, QA, instruction following, thinking traces) that Olmo 3 demonstrates is crucial for downstream post-training performance.
Weaker post-training pipeline. OLMo 2's post-training used earlier-generation data and methods, without the delta-learning preference tuning or multi-domain RLVR that drive Olmo 3's reasoning gains.
Modest performance. OLMo 2 32B achieved 4.6 on AIME 2024 and 49.2 on MATH—scores that Olmo 3 Think 32B dramatically surpasses (80.6 and 96.2, respectively). This performance gap limited OLMo 2's utility as a research platform for studying state-of-the-art reasoning.

Stanford Marin (Hall et al., 2025). A contemporaneous fully-open effort that releases pretraining data and intermediate checkpoints. Marin 32B achieves respectable performance (AIME 2024: 36.8, MATH: 49.3 in its base form), but:

The release predates the reasoning-model paradigm; Marin's post-training pipeline does not target step-by-step reasoning or extended thinking.
Marin's data and training recipes, while open, are less comprehensively documented than Olmo 3's, with fewer intermediate checkpoints and less granular data provenance.
At the 32B scale, Marin 32B substantially trails Olmo 3 Base 32B on Math (49.3 vs. 61.9) and Code (30.8 vs. 39.7) composite scores.

Apertus (Apertus Team, 2025). A 70B fully-open model that releases training data and intermediate checkpoints. Apertus achieves reasonable performance but:

At 70B parameters, Apertus is larger than Olmo 3 32B, making direct comparison complicated. However, Olmo 3 Base 32B outperforms Apertus 70B on Math (61.9 vs. 39.7) and Code (39.7 vs. 23.3) composites, suggesting that parameter count alone does not substitute for data quality and training methodology.
Apertus has not released a thinking-model variant, limiting its relevance to the reasoning-model research agenda.

LLM360 K2 V2 (Team et al., 2025). A 70B fully-open model with strong performance (AIME 2024: 78.4 in its instruct form, rivaling Olmo 3 Think). However:

K2 V2 achieves its results at 70B parameters, more than twice Olmo 3's parameter count.
The paper notes that K2 V2's evaluations use a 128K sequence length, while Olmo 3 uses 32K, making direct benchmarking comparisons confounded by inference-time compute differences.

The critical gap across all prior fully-open models: None provides a complete thinking-model pipeline with supervised finetuning, preference optimization, and multi-domain RLVR on fully-disclosed data, combined with competitive performance against leading open-weight models. This is the specific niche Olmo 3 fills.

How This Paper Positions Itself

The authors frame Olmo 3 not as a single model release but as a demonstration that the fully-open paradigm can produce competitive results at every stage of the model development lifecycle. This positioning manifests in several distinctive claims:

1. The model flow, not just the model, is the contribution. The paper emphasizes that the release includes "every stage, checkpoint, data point, and dependency used to build" the models. This is fundamentally different from releases that provide only final weights. The authors explicitly argue that access to intermediate artifacts—pretraining checkpoints, midtraining data, SFT mixtures, preference pairs, RL prompts—enables research directions that are impossible with endpoint-only releases, such as:

Tracing reasoning chains back to their original training data to understand where specific capabilities originate.
Studying how midtraining data composition affects downstream RLVR efficiency.
Intervening at specific training stages to modify model behavior (e.g., adding new capabilities via midtraining without full retraining).

2. Competitive performance validates the approach. The paper does not merely claim to be open—it demonstrates that openness does not preclude competitiveness. Olmo 3.1 Think 32B achieves scores (AIME 2024: 80.6, AIME 2025: 78.1, MATH: 96.2) that place it within striking distance of Qwen 3 32B (80.8, 70.9, 95.4) while being trained on approximately six times fewer tokens. This performance parity is essential: if fully-open models were systematically worse than closed-weight alternatives, researchers would face a difficult tradeoff between transparency and capability. Olmo 3 aims to eliminate that tradeoff.

Additionally, Olmo 3's base models establish new state-of-the-art results for fully-open base models at both 7B and 32B scales, outperforming Marin, Apertus, and K2 V2 on Math and Code composites by substantial margins (Table 2 and Table 3). This demonstrates that the data curation and training methodology advances are not merely enabling post-training gains but are producing genuinely stronger foundation models.

3. The RL-Zero release addresses a specific scientific bottleneck. The paper explicitly identifies that all leading open RLVR benchmarks train on undisclosed pretraining data, making it impossible to study data leakage or pretraining-RL interactions. By releasing Olmo 3 RL-Zero—a family of models trained via RLVR directly on Olmo 3 Base with fully disclosed data and a validated decontamination protocol—the authors provide a controlled experimental platform for the RL research community. The negative control experiment in Figure 27 (training with random rewards produces no performance improvement) serves as both a validation of their decontamination and a demonstration of the scientific value of full data transparency.

4. Three distinct post-training recipes serve different use cases. Rather than releasing a single "best" model, the paper develops three parallel post-training trajectories that target different deployment scenarios:

Olmo 3 Think: For tasks requiring extended reasoning (math competitions, complex coding, logical puzzles), where inference-time compute can be traded for accuracy through chain-of-thought generation.
Olmo 3 Instruct: For everyday user interactions (general chat, function calling, concise Q&A) where latency matters and long thinking traces are undesirable. The paper explicitly notes that "real-world language model use predominantly centers on general tasks such as advice-seeking and information recall" (Section 5), motivating the development of a model optimized for efficiency rather than test-time scaling.
Olmo 3 RL-Zero: For research on RLVR itself—a minimal pipeline from base model to reasoning via pure reinforcement learning, without the intervening SFT and DPO stages that complicate attribution of capability gains.

This tripartite structure reflects a pragmatic recognition that "openness" is not a single axis: different communities need different levels of access, and providing post-trained variants alongside base models maximizes the release's utility.

5. Methodological innovations serve the openness goal. Throughout the paper, methodological advances are motivated not just by performance improvement but by the constraints of operating in a fully-open paradigm:

The Monte Carlo rollout-based PRM training and edit-distance-based revision data pairing are designed to avoid dependence on closed-source models (GPT-4, Llama) whose outputs carry restrictive licenses.
The OLMo-core training infrastructure and OlmoRL framework are released as open-source software, reducing the barrier to entry for teams that cannot build custom training stacks.
The difficulty estimation and decontamination procedures are documented in sufficient detail for replication, addressing the reproducibility crisis that afflicts closed development pipelines.
The paper reports training costs in terms of wall-clock time rather than just GPU-hours, providing a more realistic picture of the operational complexity involved.

Summary of Positioning

Olmo 3 enters a landscape where "open" has become a contested term—applied to everything from fully-documented training pipelines to weight-only releases with restrictive licenses. By releasing the complete model flow for models that are genuinely competitive with the best closed-weight alternatives, the paper makes a normative claim: full transparency is both feasible and valuable at the frontier of language model development. The technical contributions—from data curation methodology to RL infrastructure to multi-stage post-training recipes—are presented in service of this broader argument, demonstrating that openness need not come at the cost of capability.

` tags) and empty outputs. 3. Domain-specific accuracy: Verify code outputs against test cases, check constraint adherence for instruction-following data, use metadata tags for chat safety filtering. 4. General content: Remove mentions of other model developers and date cutoffs, filter model identity references. The paper notes this is nontrivial: "Maintaining identity of models trained on heavily distilled data takes a meaningful amount of data work." 5. Repetition: Filter responses with heavily repeated (>10×) sentences, paragraphs, or (>50×) phrases. Approximately 0.1% of QwQ outputs exhibit mass repetition. 6. Chinese language: Remove responses with >5% Chinese characters (by Unicode range) to keep the model's primary language English.

Table 50 provides filtering statistics: most stages remove 0-1% of data per source. The largest single filter is the WildChat topic filter (48.09% removal for new WildChat data), reflecting the prevalence of off-topic conversational content.

Step 3: Data Mixing. Mixing experiments follow the microanneal methodology from midtraining. A base mix of 100K examples from extended OpenThoughts 3 serves as a strong baseline. Individual datasets are added (up to 100K examples each) to measure impact on the chat evaluation suite (Table 18). Each dataset is found helpful on at least one evaluation, so the final mix includes at least a portion of every tested dataset. The final mix (Table 17) totals 2,268,468 examples for the 7B and 2,253,916 for the 32B (the 32B mix has slightly fewer examples after additional identity and off-topic filtering).

Step 4: Decontamination. The Tülu 3 decontamination procedure is applied: n-gram matching with 8-grams and a 0.5 overlap threshold. Additional heuristics prevent false positives: matches of task-irrelevant text chunks are ignored, and math-symbol-heavy n-grams (where most tokens are length 1) are excluded.

Dolci Instruct SFT Data Curation. The Instruct SFT data shares the same pipeline but uses non-thinking completions and adds function-calling data (described in detail later). The prompt sources are listed in Table 30, totaling 2,152,112 examples. The base model is initialized from the Olmo 3 Think SFT checkpoint—meaning the Instruct SFT is effectively a continuation of training on different data, not starting from the base model.

SFT Training Configuration. Training uses OLMo-core (switched from Open Instruct, yielding 8× throughput improvement). All 7B SFT models use a batch size of 1M tokens, 32B models use 4M tokens, both with sequence length 32,768, training for 2 epochs. For the 7B Thinking SFT, the learning rate is $5.0 \times 10^{-5}$ . For the 32B Thinking SFT, a checkpoint trained at $1.0 \times 10^{-4}$ is merged ("souped") with one trained at $5.0 \times 10^{-5}$ using mergekit. For the 7B Instruct SFT, the learning rate is $8.0 \times 10^{-5}$ . GPU counts range from 8-64 for 7B to 256 for 32B.

Preference Tuning with Delta Learning: Dolci Think DPO and Dolci Instruct DPO

The DPO stage uses Direct Preference Optimization (Rafailov et al., 2024) with a key insight from Delta Learning (Geng et al., 2025): the effectiveness of preference data depends primarily on the quality gap between chosen and rejected responses, not the absolute quality of either response individually. By constructing pairs with large capability deltas, DPO can drive improvements even when supervised finetuning on the chosen responses alone would not help or would actively hurt.

The Delta Learning Principle in Practice. Table 21 demonstrates the principle concretely. The chosen responses for Dolci Think DPO are generated by Qwen3 32B (thinking mode); the rejected responses are generated by Qwen3 0.6B (thinking mode). Evaluating these models directly:

Qwen3 32B (chosen) achieves an average of 83.2 across the evaluation subset.
Qwen3 0.6B (rejected) achieves 35.1.
The gap is 48.1 points.

If the authors simply continue supervised finetuning on the Qwen3 32B responses, the initial SFT model's performance drops from 70.3 to 64.5—the chosen responses are actually worse targets for imitation than the data the model has already seen in SFT. However, preference tuning with the (chosen, rejected) pairs via DPO improves performance to 72.9. The contrastive signal—"prefer this good response over this bad one"—provides useful optimization pressure even when the good responses alone would not.

The authors also find that the DPO model serves as a better initialization for subsequent RLVR than the SFT model (Table 22). Starting RLVR from the DPO checkpoint yields 74.1 average vs. 71.9 from SFT-only. The advantage varies by evaluation: on AlpacaEval, DPO maintains its lead throughout RL training; on Omega, both converge to similar performance; on AIME 2025, SFT catches up to near-DPO performance. Crucially, in no case does the SFT-only model surpass DPO+RLVR.

Dolci Think DPO Data Construction. The preference dataset (Table 19) consists of 200,000 prompt pairs. Construction proceeds in three steps:

Step 1: Prompt Sourcing. Prompts are selected from the Dolci Instruct SFT dataset (which shares prompt sources with Dolci Think SFT but lacks reasoning traces), supplemented with DaringAnteater and UltraFeedback subsets from the OLMo 2 preference dataset. The prompt-domain mixing is optimized through experiments on the Instruct model (Section 5.3.2): prompt-domain distributions do not consistently align with contrast exhibited in response pairs, so the authors create nine hand-crafted mixes informed by ablation experiments and select the best-performing one empirically.

Step 2: Contrastive Completions. For each prompt, the chosen response is decoded from Qwen3 32B (thinking mode) and the rejected response from Qwen3 0.6B (thinking mode).

Step 3: Filtering. Topic filtering and model-identity filtering are applied to chosen responses (as in SFT). Rejected responses are left unfiltered—the intuition is that incorrect rejected responses may provide useful contrast even if they contain filterable artifacts. All prompts are decontaminated against the evaluation suite.

Dolci Instruct DPO Data Construction. The Instruct DPO dataset (260,000 pairs) combines multiple preference signals:

Delta-Learning Heuristic Pairs. Same approach as Think DPO but with thinking mode turned off: chosen from Qwen3 32B, rejected from Qwen3 0.6B.

Delta-Aware GPT-Judged Pairs. An evolved version of the UltraFeedback pipeline. The key innovation is explicit maximization of the quality delta. Initial attempts to modernize the OLMo 2 pipeline (upgrading to GPT-4.1 judge, updating the model pool for response generation) failed to improve over the OLMo 2 baseline (Table 32). The authors hypothesize that the new model pool contained universally strong models, reducing the typical quality gap in judged pairs.

Two interventions fix this:

Force weak models into the pool: Ensure that responses from two of the following weaker/smaller/older models are always included: OLMo 2-1B-Instruct, OLMo 2-7B-Instruct, Yi-9B, Yi-34B, Phi4-Mini-Instruct, Qwen3-0.6B, Qwen3-1.7B.
Select the minimum-score response as rejected: From the four judged responses per prompt, the worst-scoring response becomes the rejected completion, maximizing the delta.

These "delta-maximizing" pipeline modifications are crucial: GPT-judged pairs without them match the OLMo 2 baseline (55.4 vs. 55.5 average). With weak models forced in: 56.3. With minimum-score selection: 57.4. Combined with delta-learning heuristic pairs: 60.4.

Multi-Turn Preferences. To support multi-turn conversational capabilities, prompts are synthetically extended from the Tülu 3 DPO dataset using two methods: "self-talk" (LLM-generated follow-up requests extending the original prompt) and "synthetic-context" (related independent questions or paraphrases as previous turns). Preference pairs differ only in the last turn. Chosen/rejected completions for the final turn are generated by either GPT-4o/GPT-3.5 or Qwen3 32B/0.6B (no thinking).

Length Bias Control. Both delta-learning heuristic pairs and GPT-judged pairs exhibit length bias: chosen responses are significantly longer than rejected ones. For delta-learning pairs, the 80th percentile of token difference is 564 tokens (chosen longer). For GPT-judged pairs, it is 538 tokens. To promote conciseness in Instruct models, the chat and multi-turn subsets are filtered to limit the length difference between chosen and rejected responses to 100 tokens. This trades some benchmark performance for improved usability (Figure 22), with an unexpected benefit: the shorter DPO model serves as a better starting point for RLVR, as longer context windows during RL allow the model to "be more intelligent per token."

DPO Training Configuration. All DPO models are trained for one epoch. The DPO beta parameter is 5 for both Thinking and Instruct variants, controlling the strength of the KL divergence penalty relative to the preference optimization objective. The loss function is length-normalized:

$\max_{\pi_\theta} \mathbb{E}_{(x, y_c, y_r) \sim \mathcal{D}} \left[ \log \sigma \left( \frac{\beta}{|y_c|} \log \frac{\pi_\theta(y_c \mid x)}{\pi_{\text{ref}}(y_c \mid x)} - \frac{\beta}{|y_r|} \log \frac{\pi_\theta(y_r \mid x)}{\pi_{\text{ref}}(y_r \mid x)} \right) \right]$

where $\pi_\theta$ is the policy being optimized, $\pi_{\text{ref}}$ is the initial reference policy (the SFT checkpoint), $(x, y_c, y_r)$ are prompt, chosen response, and rejected response triples, $|y_c|$ and $|y_r|$ are response lengths in tokens, $\beta = 5$ is the KL regularization coefficient, and $\sigma$ is the logistic sigmoid function.

What it computes: For each training example, the loss increases the log-ratio of the chosen response's likelihood to the rejected response's likelihood, both normalized by their respective lengths to prevent the model from simply preferring longer responses. The reference policy term provides a baseline, and $\beta$ controls how far the optimized policy can deviate from the reference. The sigmoid compresses this difference into a probability-like score that is maximized.

Why this form: Length normalization prevents the well-documented problem of DPO producing excessively verbose outputs by making per-token log-probability improvements scale with response quality rather than length. The KL penalty prevents catastrophic forgetting of the SFT-trained capabilities. The specific value $\beta = 5$ was likely chosen through hyperparameter sweeps but is not further justified in the paper.

Hyperparameters (Table 48): Learning rates are $8.0 \times 10^{-8}$ (7B Thinking), $7.0 \times 10^{-8}$ (32B Thinking), $1.0 \times 10^{-6}$ (7B Instruct). All use linear decay with 0.1 warmup ratio, batch size 128. The 7B Thinking model uses 150K preference pairs, 32B Thinking uses 200K, 7B Instruct uses 260K. The Thinking DPO models train on 2-4 nodes (7B) or 8-16 nodes (32B). Maximum sequence length is 16K for Thinking and 8K for Instruct. The learning rate and dataset size are swept as hyperparameters—dataset size is particularly important because DPO performance does not monotonically increase with more data; performance on certain tasks peaks and then declines with additional training (Figure 23, left).

Reinforcement Learning with OlmoRL

The third post-training stage applies reinforcement learning with verifiable rewards (RLVR) across multiple domains. The core algorithm is an improved version of Group Relative Policy Optimization (GRPO; Shao et al., 2024), integrated with a custom asynchronous infrastructure designed for long reasoning traces.

OlmoRL Algorithm. The reinforcement learning objective is to maximize expected reward for model-generated responses given prompts, where a verifier checks whether the response matches a ground-truth answer. The final OlmoRL objective incorporates several modifications over vanilla GRPO:

$J(\theta) = \frac{1}{\sum_{i=1}^G |y_i|} \sum_{i=1}^G \sum_{t=1}^{|y_i|} \min\left( \frac{\pi(y_{i,t} \mid x, y_{i,<t}; \theta_{\text{old}})}{\pi_{\text{vllm}}(y_{i,t} \mid x, y_{i,<t}; \theta_{\text{old}})}, \rho \right) \cdot \min\left( r_{i,t} A_{i,t}, \text{clip}(r_{i,t}, 1 - \varepsilon_{\text{low}}, 1 + \varepsilon_{\text{high}}) A_{i,t} \right)$

where $G$ is the group size (number of completions per prompt, set to 8), $|y_i|$ is the length of the $i$ -th completion in tokens, $y_{i,t}$ is the $t$ -th token of the $i$ -th completion, $\pi(y_{i,t} \mid x, y_{i,<t}; \theta)$ is the probability of token $t$ under the current policy parameters $\theta$ , $\pi_{\text{vllm}}(y_{i,t} \mid x, y_{i,<t}; \theta_{\text{old}})$ is the probability under the inference engine's copy of the old policy parameters (accounting for numerical differences between training and inference engines), $\rho$ is the truncated importance sampling cap (set to 2.0 for 32B, default for 7B), $r_{i,t} = \pi(y_{i,t} \mid x, y_{i,<t}; \theta) / \pi(y_{i,t} \mid x, y_{i,<t}; \theta_{\text{old}})$ is the per-token importance sampling ratio, $\varepsilon_{\text{low}} = 0.2$ and $\varepsilon_{\text{high}} = 0.272$ are the lower and upper clipping bounds, and $A_{i,t}$ is the per-token advantage.

The advantage $A_{i,t}$ is computed within each group $G$ as:

$A_{i,t} = \left( r(x, y_i) - \text{mean}\left( \{ r(x, y_i) \}_{i=1}^G \right) \right)$

where $r(x, y_i)$ is the reward score for the complete response $y_i$ given prompt $x$ , computed by the domain-specific verifier. Note that the advantage is the same for all tokens within a response—it is the per-response reward minus the group mean—and is not normalized by the group standard deviation (unlike vanilla GRPO).

What it computes: The objective maximizes a clipped surrogate of the expected reward improvement. For each group of $G$ completions from the same prompt, the advantage quantifies how much better or worse each completion is than the group average. The policy is updated to increase the probability of tokens in above-average completions and decrease the probability of tokens in below-average completions. The outer $\min$ applies truncated importance sampling to correct for off-policy differences between the inference engine and training engine. The inner $\min$ applies PPO-style clipping to prevent destructive large updates. The token-level loss normalizes by total tokens to prevent length bias.

Why this form (seven modifications over vanilla GRPO):

Zero gradient signal filtering: Groups where all $G$ completions receive identical rewards have zero standard deviation in advantage. These groups are removed from the batch because they provide zero gradient—there is no signal to distinguish good from bad completions. This follows DAPO (Yu et al., 2025).
Active sampling (novel): To compensate for filtered groups, the asynchronous infrastructure continuously pulls completions from actors and resamples prompts into the queue until the desired batch size of non-zero-advantage completions is reached. Unlike DAPO's dynamic sampling (which oversamples 3× to guarantee sufficient non-zero-advantage examples), active sampling uses the asynchronous queue to dynamically refill without over-generating.
Token-level loss: Normalizing by the total number of tokens across the batch ( $\sum_{i=1}^G |y_i|$ ) rather than per-sample prevents the model from being biased toward shorter or longer responses. This is adapted from DAPO.
No KL loss: The KL divergence penalty between current and reference policy is removed entirely, following common practice (GLM-4.5 Team et al., 2025; Yu et al., 2025; Liu et al., 2025b). The authors found it did not lead to over-optimization or training instability.
Clip-higher: The upper clipping bound $\varepsilon_{\text{high}} = 0.272$ is set higher than the lower bound $\varepsilon_{\text{low}} = 0.2$ , allowing slightly larger probability increases for good tokens than decreases for bad tokens. This asymmetry encourages exploration. Adapted from DAPO.
Truncated importance sampling: The outer $\min$ multiplies the loss by $\min(r_{i,t}, \rho)$ to cap the influence of tokens where the inference engine's probabilities differ significantly from the training engine's. This corrects for numerical discrepancies between vLLM and the training framework, following Yao et al. (2025).
No standard deviation normalization: The advantage is not divided by the group standard deviation, unlike vanilla GRPO. This removes a difficulty bias where questions with inherently low reward variance (too easy or too hard) would have their advantages artificially inflated by the normalization term, following Liu et al. (2025b).

Verifiers. Each domain uses a different reward function (Figure 16):

Math: Rule-based verifier with SymPy-powered normalization and comparison against reference answer. Returns binary 1 (correct) or 0 (incorrect).
Code: Test-case-based verifier. Two variants are explored: (a) percentage of passed test cases as a continuous reward, and (b) binary 1 if all test cases pass, 0 otherwise. AWS Lambda is used for distributed code execution to avoid blocking the trainer process and to handle resource-intensive test suites.
Instruction Following: Constraint checks: a set of functions verifies each constraint from the prompt. Returns 1 if all constraints are satisfied, 0 otherwise.
General Chat — Reference: An LLM judge (Qwen3 32B, no thinking, hosted on vLLM) compares the model's response against a reference answer, scoring on a continuous [0, 1] scale for quality.
General Chat — Open-Ended: An LLM judge scores the response on [0, 1] without a reference answer.

Dolci Think RL Data Curation. The RL dataset (Table 20) contains approximately 105K prompts across four domains, each associated with either verifiable or non-verifiable rewards. The curation process includes:

Step 1: Prompt Sourcing.

Math (30,186 prompts): Open-Reasoner-Zero, DAPO-Math, AceReason-Math, KlearReasoner-MathSub, OMEGA-train, DeepScaler. These cover algebra, combinatorics, number theory, geometry, and other mathematical domains.
Code (20,110 prompts): AceCoder, KlearReasoner-Code, Nemotron Post-training Code, SYNTHETIC-2, Open-Code Reasoner. For datasets without existing test cases, a synthetic pipeline generates them: (a) GPT-4.1 rewrites problems, (b) generates solutions, (c) generates test cases that are executed against solutions, keeping only examples where solutions pass >80% of test cases (with failed test cases removed).
Instruction Following (30,186 prompts): IF-RLVR with up to 5 constraints, sampled from IFEval and IFBench-Train.
General Chat (21,387 prompts): Tülu 3 SFT, WildChat-4.8M, Multi-subject-RLVR. WildChat is filtered to English-only, non-reasoning instances. Character-based role-play data is limited to 10 instances per character to prevent overfitting to fictional personas (the most common characters before filtering: Natsuki 1,284 appearances, Monika 1,243, Sayori 1,077, Yuri 957). Tülu 3 prompts are rewritten by GPT-4.1 for clarity and to extract reference answers. Eight samples per prompt are generated with a Qwen 2.5 7B model finetuned on OpenThoughts 2; prompts with average F1 score < 0.1 or > 0.8 between generated responses and reference are removed to exclude noisy or overly difficult instances. Manual post-hoc filtering removes code- and math-centric prompts.

Step 2: Offline Difficulty Filtering. For the 7B model, eight rollouts are generated per prompt from the DPO checkpoint (temperature 1.0, top-p 1.0), and prompts with pass rate > 62.5% (5 out of 8 correct) are removed—the model already solves them easily. For the 32B, the 7B DPO-filtered data is reused due to compute constraints, supplemented by active sampling during training.

Step 3: Data Mixing. Domain-specific RL runs are conducted on intermediate SFT checkpoints; math-domain training is used to test algorithmic changes; periodic mixture experiments verify mixing stability. The final mix weights higher-quality datasets more heavily, with roughly equal token allocation across domains and slightly more emphasis on math and instruction following. Certain OMEGA subtasks that the model especially struggles with are downsampled by 50%.

OlmoRL Infrastructure. The RL infrastructure is the paper's most significant engineering contribution. The architecture (Figure 17a) uses a centralized learner distributed across multiple nodes via DeepSpeed (Rasley et al., 2020) and a large pool of actors, each running an independent vLLM instance (Kwon et al., 2023).

Asynchronous Architecture. Prompts are queued and dispatched to actors. Actors generate completions, interact with reward verifiers, and return results through a results queue. The learner consumes results to update model parameters. The system is fully asynchronous: the learner does not wait for actors; actors do not wait for each other.

Continuous Batching. Static batching—where $N$ prompts are distributed across $N$ actors, and each generates one full completion before the batch returns—wastes compute when completion lengths vary. With a 32K maximum generation length and a mean of 14,628 tokens, up to 54% of compute is wasted on idle "slots" waiting for the longest completion. Continuous batching constantly enqueues new generations as each one finishes, backfilling the batch (Figure 17c). This alone provides significant throughput improvement.

Active Sampling. When zero-gradient groups are filtered, active sampling continuously pulls new completions from the actor pool and resamples prompts into the queue to maintain the target batch size of non-zero-advantage completions. This prevents the batch size from shrinking over training, a common failure mode in vanilla GRPO where the model becomes too good at its training data and all group members achieve the same reward.

Inflight Weight Updates. The standard approach to minimizing off-policy correction—synchronizing actor weights after every training step by waiting for ongoing generations to finish, dumping KV caches, and loading new weights—causes significant GPU idle time. Instead, following Piché et al. (2025), weights are updated immediately without pausing generation engines. The generation framework is thread-safe, and KV caches are not invalidated. This yields up to 4× faster training with the same resources without accuracy degradation.

Engineering Optimizations. Actors are decoupled so each can start and stop independently. A prefetch thread constantly refills the inference queue to prevent actor starvation.

The combined infrastructure improvements produce dramatic gains (Table 23). Starting from the OLMo 2 RL baseline (6.34M tokens/second, 0.30 MFU), adding continuous batching increases throughput to 7.02M t/s (0.33 MFU). Adding better threading reaches 9.77M t/s (0.46 MFU). Adding inflight updates achieves the final Olmo 3 configuration: 21.23M t/s (1.01 MFU, 43.21 MBU)—a 3.3× speedup over the OLMo 2 baseline on the same hardware.

The learner uses minimal sharding to fit in memory, as it spends 75% of the time waiting for actor data. For the 7B Reasoner, 7 nodes run actors and 2 run the learner (approximately 14× inference compute vs. training compute). For the 32B, 20 nodes run actors and 8 run the learner (approximately 5× inference vs. training).

RL Training Configuration and Results. Table 49 provides full hyperparameters. For the 7B Think RL: dataset size 104,869 prompts, learning rate $1.0 \times 10^{-6}$ , constant LR schedule, 1,400 training steps, maximum prompt length 2,048 tokens, response length 32,768 tokens, 64 unique prompts per batch, group size 8, sampling temperature 1.0. The 32B Think uses a learning rate of $2.0 \times 10^{-6}$ over 750 steps, with a TIS cap of 2.0 and 128 unique prompts per batch.

Figure 18 shows reward curves: math reward increases steadily from approximately 4.5 to 5.5-6.0 over 1,000 steps; code reward increases more slowly from 3.0 to 4.5; IFEval reward increases from 4.5 to 6.0; general chat reward increases from 7.0 to 8.0-8.5. Response length initially dips slightly then slowly increases, consistent with the SFT/DPO models already generating near-maximum-length reasoning traces.

Mixing Domains Prevents Over-Optimization. Figure 20 (left) demonstrates that training on IFEval data alone leads to over-optimization: as IFEval scores reach 84-85, AlpacaEval scores drop to 10-20. Training on mixed data achieves similar IFEval performance (84-85) while maintaining AlpacaEval scores around 40-45. Figure 21 shows that domain-specific runs achieve higher train reward than the mixed run on their respective domains, but this does not translate to better downstream evaluation—the mixed run achieves similar or better downstream performance while showing lower train reward, suggesting that mixing data reduces reward hacking.

4. Key Insights and Innovations

Innovation 1: The Model Flow as the Primary Contribution, Not the Model

The paper's most fundamental conceptual move is reframing what constitutes a "contribution" in language model research. Rather than positioning Olmo 3 as "a new model that happens to be open," the authors argue that the entire development pipeline—the model flow—is the primary intellectual product, with the final weights serving as one artifact among many. This is not merely a rhetorical shift; it has concrete implications for what kinds of research the release enables.

What the field did before. The dominant release paradigm—exemplified by Qwen 3 (Yang et al., 2025a), DeepSeek R1 (Guo et al., 2025), and Llama 3 (Grattafiori et al., 2024)—treats the model flow as proprietary infrastructure and the final weights as the public deliverable. This creates what the authors diagnose as a structural asymmetry: researchers can study the endpoint of model development but cannot investigate how that endpoint was reached. The paper explicitly names the consequences: inability to verify whether benchmark performance reflects genuine capability or memorization, inability to study how capabilities emerge across training stages, and inability to isolate effects of specific data sources or algorithmic decisions.

Why this is a fundamental shift, not incremental transparency. Prior fully-open efforts like OLMo 2 (OLMo et al., 2024) and Marin (Hall et al., 2025) released training data and intermediate checkpoints, but the Olmo 3 release is qualitatively different in scope. It includes not just the data pools but the exact training mixes (the actual token sequences seen during training, accounting for upsampling and repetition), the data processing and mixing code, the evaluation harnesses, intermediate checkpoints at every training stage boundary, and the full post-training datasets (SFT, DPO, RL). The distinction between releasing "data pools" and releasing "training mixes" is critical: the former tells researchers what data was available; the latter tells them what data was actually used and in what proportions, enabling precise attribution of capability gains to specific data sources.

The significance goes beyond reproducibility. By releasing the complete model flow, the paper enables research directions that are structurally impossible with endpoint-only releases:

Tracing reasoning chains to training data. The paper explicitly states: "Because of our fully-open approach, the Olmo 3 release also enables reasoning chains to be traced back to their original training data, unlocking research opportunities not possible with any other thinking model." This is a concrete capability that no amount of weight-level analysis can replicate.
Studying pretraining-RL interactions. The Olmo 3 RL-Zero release (Section 6) is explicitly designed as a controlled experimental platform. All leading open RLVR benchmarks train on undisclosed pretraining data, making it impossible to isolate whether RL improvements reflect genuine reasoning or memorized evaluation data. The negative control experiment in Figure 27—training with random rewards produces no performance improvement—is only meaningful because the full data pipeline is transparent. This transforms RLVR from an opaque capability extraction process into a scientifically interpretable one.
Midtraining as a research lever. Figure 25 demonstrates that base model midtraining composition determines whether RL-Zero learns longer reasoning traces. This finding is only possible because intermediate checkpoints at every training stage are available, turning midtraining from a black-box preprocessing step into a controllable experimental variable.

Evidence. The paper anchors this contribution not in a single table but in the structural design of the release itself. The model flow diagram (Figure 2) is not merely illustrative—it encodes the paper's normative claim that every arrow in the diagram should be inspectable and reproducible. The detailed documentation of data curation decisions across pretraining, midtraining, and post-training (Sections 3.4–3.6, 4.2–4.4, 5.2–5.4) demonstrates that full transparency is feasible at state-of-the-art performance levels, countering the implicit assumption that competitive models require proprietary data pipelines.

Innovation 2: Delta Learning as a Preference Tuning Paradigm for Capability Gains

The paper reframes preference tuning—typically viewed as an alignment technique for human values—as a contrastive learning mechanism for driving capability improvements beyond what supervised finetuning can achieve. This reframing is enabled by Delta Learning (Geng et al., 2025), but the paper's contribution is demonstrating that the principle generalizes across domains and interacts productively with subsequent RLVR.

What the field did before. Prior work on thinking models (DeepSeek R1, OpenThoughts3, S1) predominantly used SFT followed by RLVR, skipping preference tuning entirely. The dominant assumption was that DPO's value lay in aligning model behavior with human preferences—politeness, harmlessness, formatting—rather than in improving reasoning capabilities. This assumption was reasonable given that preference optimization typically uses LLM-judged pairs where the quality signal is entangled with stylistic factors that don't necessarily correlate with correctness on reasoning benchmarks.

The paper overturns this assumption through a specific empirical finding: supervised finetuning on Qwen3 32B thinking traces actively hurts Olmo 3 Think SFT performance (Table 21: average drops from 70.3 to 64.5). This is a striking result. The Qwen3 32B responses are objectively high-quality—they achieve 83.2 average on the evaluation subset—but they are worse imitation targets than the data the model has already seen. More SFT data from a strong model does not help; it actively degrades performance.

Why Delta Learning resolves the paradox. The Delta Learning insight is that the gap between chosen and rejected responses matters more than the absolute quality of either. By pairing Qwen3 32B responses (chosen) with Qwen3 0.6B responses (rejected, averaging 35.1), DPO extracts useful contrastive signal from data that would be harmful in an SFT context. The preference pairs encode "this reasoning pattern is better than this one," and optimizing this relative preference drives capability gains even when the absolute quality of the chosen responses is insufficient for further imitation learning. DPO on these pairs improves performance to 72.9—a net gain of 2.6 over the original SFT checkpoint.

Significance beyond raw performance. This finding has two deeper implications:

It redefines the role of preference tuning in the post-training pipeline. DPO is not merely an alignment step sandwiched between SFT and RLVR; it is a distinct capability-learning mechanism that can expand the reasoning frontier when imitation is saturated. Table 22 shows that DPO + RLVR (74.1) outperforms SFT + RLVR (71.9) on the same RL data, and Figure 20 (right) demonstrates that the DPO model achieves higher pass@K on AIME evaluations even at K=32. This means DPO is not just converting pass@K into pass@1 (which RLVR also does); it is genuinely expanding the model's reasoning capabilities in ways that RLVR can then further exploit.
It provides a principled strategy for leveraging stronger-but-inaccessible models. The paper's DPO pairs are generated using Qwen3 models, which the authors cannot directly distil from (due to licensing and distribution-shift concerns). Delta Learning offers a path to extract useful training signal from stronger models' outputs without imitating them directly: pair their responses with clearly weaker responses and optimize the contrast. This has practical implications for the open-model ecosystem, where the strongest models are often closed-source or restrictively licensed.

The complementary signals insight. Table 32 demonstrates that delta-learning heuristic pairs and delta-aware GPT-judged pairs provide complementary preference signals. Tuning with heuristic pairs alone achieves 57.6; GPT-judged pairs alone achieve 57.4; combining them achieves 60.4. The paper does not deeply analyze why these signals are complementary, but the finding suggests that different contrast construction methods surface different aspects of response quality—heuristic pairs encode raw capability gaps, while LLM-judged pairs encode more nuanced quality dimensions that a judge model can assess but simple model-size pairing cannot capture.

Evidence. Table 21 establishes the core delta-learning result. Table 22 establishes that DPO provides a better RLVR initialization. Figure 20 (right) shows DPO improves pass@K. Table 32 shows complementarity of preference signal sources. The breadth of this evidence—spanning reasoning benchmarks, code, chat, and instruction following—suggests the finding is robust rather than domain-specific.

Innovation 3: OlmoRL as an Engineering-Enabled Research Platform

The paper's RL infrastructure contributions are not merely engineering optimizations—they constitute a research platform innovation that changes what experiments are feasible. By reducing RL training time from 14 days on 9 nodes to 6 days on 5 nodes (for the 7B Think model), the infrastructure enables rapid iteration on data mixtures, algorithmic variants, and training recipes that would be prohibitively expensive otherwise.

What the field did before. Most RLVR research uses static batching, synchronous weight updates, and single-domain training runs. The standard approach to addressing zero-gradient groups—DAPO's dynamic sampling (Yu et al., 2025)—oversamples 3× to guarantee sufficient non-zero-advantage examples, wasting significant compute on generations that will be discarded. The standard approach to off-policy correction—synchronizing weights after every training step by waiting for ongoing generations to finish—leaves GPUs idle, reducing throughput dramatically.

The conceptual contribution. The paper's infrastructure innovations are not individually novel—continuous batching exists in inference serving, inflight updates were proposed by Piché et al. (2025), asynchronous RL architectures were developed by Noukhovitch et al. (2024). The contribution is the integration of these techniques into a coherent system that makes long-reasoning-trace RLVR tractable for models that start from SFT/DPO checkpoints with average generation lengths exceeding 10K tokens.

The critical diagnostic: with a 32K maximum generation length and 14,628-token average length, static batching wastes up to 54% of inference compute on idle slots. This is not a minor inefficiency—it means that nearly half the inference budget is literally doing nothing, waiting for the longest completion in each batch to finish. Continuous batching alone recovers this waste by constantly backfilling finished slots with new prompts.

Why this matters for research, not just efficiency. The 4× speedup (Table 23: from 6.34M t/s to 21.23M t/s) enables experimental designs that were previously infeasible:

Multi-domain RLVR becomes practical. The paper's final RL runs train on mixed math, code, instruction following, and general chat data. Without the infrastructure improvements, a single-domain run took 14 days; multi-domain runs would take months, making systematic ablation of domain mixtures impossible.
DPO initialization experiments become feasible. The comparison of RLVR starting from SFT vs. DPO checkpoints (Table 22, Figure 19) requires running RL to convergence from multiple starting points. Each run takes 6-7 days with the optimized infrastructure—manageable for a research project—versus weeks without it, which would be prohibitive.
The RL-Zero negative control experiment is enabled. Training with random rewards to verify decontamination (Figure 27) requires a full RL run that produces no useful model. This is only justifiable if the cost is low. At 14 days per run, it would be an expensive diagnostic; at 6 days, it is a reasonable validation step.

The active sampling innovation. While continuous batching and inflight updates are adaptations of known techniques, active sampling is a genuine algorithmic contribution. DAPO's dynamic sampling generates 3× the required batch size and filters down, wasting inference compute. Active sampling integrates filtering into the asynchronous queue: as completions are generated, zero-gradient groups are identified, and new prompts are continuously enqueued to replace them. This maintains consistent batch sizes without over-generation. Figure 26 demonstrates the effect quantitatively: without active sampling, the percentage of non-zero-advantage samples in each batch declines over training (a common GRPO failure mode); with active sampling, it remains at 100%, and training loss variance is substantially reduced.

The mixing-domains-prevents-over-optimization finding. This is a conceptual insight enabled by the infrastructure. Figure 20 (left) shows that single-domain RL on IFEval drives AlpacaEval scores to near-zero—the model over-optimizes to instruction following at the expense of general chat quality. Mixed-domain RL maintains high AlpacaEval scores while achieving similar IFEval performance. Figure 21 adds a crucial nuance: domain-specific training achieves higher train reward than mixed training on each domain, but this does not translate to better downstream evaluation. The mixed run's lower train reward with similar or better downstream performance suggests that mixing data reduces reward hacking—the model learns more generalizable strategies rather than exploiting domain-specific reward signals.

Evidence. Table 23 documents the infrastructure improvements quantitatively. Figure 26 demonstrates active sampling's effect on training stability. Figure 20 and Figure 21 establish the domain-mixing finding. The training speed claims are anchored to specific hardware configurations and throughput measurements, making them reproducible.

Innovation 4: RL-Zero as a Decontamination-Validated Scientific Benchmark

The Olmo 3 RL-Zero release introduces a methodological innovation: a reinforcement learning benchmark where data leakage is experimentally ruled out, not just asserted. This addresses a specific crisis in the RLVR literature: the discovery by Shao et al. (2025b) that RLVR with spurious (random) rewards can improve benchmark performance when training data is contaminated, making it impossible to distinguish genuine reasoning improvements from memorization.

What the field did before. All leading open RLVR benchmarks—DAPO (Yu et al., 2025), DeepScaler (Luo et al., 2025b), Open-Reasoner-Zero (Hu et al., 2025)—train on top of base models (typically Qwen 2.5) whose pretraining and midtraining data are undisclosed. This creates a fundamental confound: if a model shows improved MATH or AIME performance after RLVR, is it learning general reasoning strategies or simply being incentivized to recall memorized evaluation examples? Shao et al. (2025b) demonstrated that this is not a hypothetical concern—training with random rewards on contaminated data produces the same performance trajectory as training with genuine rewards, meaning the "improvement" reflects data leakage, not learning.

The conceptual contribution. Olmo 3 RL-Zero doesn't just claim to be uncontaminated—it provides a falsifiable test of that claim. The negative control experiment (Figure 27) trains Olmo 3 Base with random binary rewards that are completely independent of response quality. If the base model's training data contained evaluation examples, RLVR would elicit those memorized solutions and benchmark performance would improve despite the meaningless reward signal. The result: performance remains flat or degrades across all evaluated benchmarks (GPQA, ZebraLogic, Minerva Math, GSM8K, Omega, AIME 2024/2025, HumanEval+, AlpacaEval, IFEval). This is strong evidence that the decontamination protocol successfully removed evaluation data from the training pipeline.

Why this changes the research landscape. Prior to this release, researchers studying RLVR algorithms had no way to know whether their observed improvements were real. They could trust that Qwen's training data was decontaminated—a trust that Shao et al. (2025b) showed was often misplaced—or they could run their own decontamination on a model whose training data they couldn't access, which is impossible. Olmo 3 RL-Zero provides a known-clean baseline against which algorithmic innovations can be evaluated. A new RLVR method that improves on Olmo 3 RL-Zero's AIME trajectory (Figure 24: from approximately 28 to 48 pass@1 over 2,500 steps) can be attributed to algorithmic innovation rather than data leakage, because the leakage confound has been experimentally ruled out.

The multi-domain benchmark contribution. Most RLVR research has focused on single-domain training (typically math only). Olmo 3 RL-Zero provides a multi-domain benchmark (Figure 24, bottom-right) where the model trains on math, code, instruction following, and general chat simultaneously. The finding that mixed-domain training under-optimizes each individual domain compared to single-domain training—but may generalize better—provides a concrete testbed for multi-objective RL research. The paper explicitly positions this: "Future work can leverage this setup to investigate the interactions between domains in multi-objective RLVR."

The midtraining ablation capability. Because the full midtraining data is released, researchers can ablate specific midtraining sources and measure their impact on downstream RLVR efficiency. Figure 25 demonstrates this concretely: two early midtrained base models with different reasoning data mixtures show dramatically different RLVR trajectories. The model with insufficient reasoning data never learns to produce longer chains of thought (response length stays flat), and math reward plateaus at a lower level. This kind of causal analysis—"does this specific midtraining data source improve RLVR efficiency?"—is impossible without full data transparency.

Evidence. Figure 27 provides the negative control validation. Figure 24 establishes the RL-Zero performance baselines across domains. Figure 25 demonstrates midtraining ablation capability. Figure 38 compares Olmo 3 RL-Zero's efficiency to DAPO's Qwen 2.5 32B baseline—Olmo 3 reaches similar performance in fewer gradient steps and substantially fewer GPU hours, making it a more practical research platform.

Innovation 5: Length-Controlled Preference Tuning Enables Better RLVR Initialization

A subtle but practically significant finding emerges from the Instruct model development: applying length control during DPO produces a model that is a better starting point for RLVR, despite having lower standalone benchmark scores. This is a non-obvious interaction between training stages that challenges the default assumption that the best DPO checkpoint—measured by evaluation suite performance—is the best initialization for subsequent RL.

The length bias problem. Both delta-learning heuristic pairs and GPT-judged pairs exhibit systematic length bias: chosen responses are substantially longer than rejected responses. For the Instruct model's preference data, the 80th percentile of token difference is 564 tokens (heuristic pairs) and 538 tokens (GPT-judged pairs). Models trained on this data learn to produce longer responses, which is empirically beneficial for reasoning benchmarks (longer chain-of-thought correlates with higher accuracy) but undesirable for general-use settings where conciseness matters.

The intervention. The authors filter the chat and multi-turn subsets of the DPO data to limit the length difference between chosen and rejected responses to 100 tokens. This produces a model with substantially shorter average generation length at the cost of lower scores on length-sensitive evaluations (particularly math benchmarks like AIME and MATH).

The unexpected RLVR benefit. Despite lower DPO-stage performance, the length-controlled model serves as a better RLVR initialization. The paper offers a specific mechanistic hypothesis: "with a fixed context window (8K), a shorter model may be 'more intelligent per token,' allowing it to leverage the available budget more effectively during optimization." Whatever the mechanism, the finding is empirically clear: "RL training progresses more reliably when initialized from the length-controlled DPO policy. Across most benchmarks, performance improves more steadily compared to RL runs starting from a higher-scoring but uncorrected DPO checkpoint, which tends to show earlier signs of instability or degradation."

Why this matters conceptually. This finding reveals a hidden coupling between training stages that the standard "train DPO, pick best checkpoint, apply RL" pipeline would miss. The optimal DPO checkpoint for standalone evaluation is not the optimal DPO checkpoint for RLVR initialization. This suggests that end-to-end optimization of the full post-training pipeline—rather than greedy stage-by-stage optimization—may yield better results, and that metrics like response length and training stability should be considered alongside benchmark scores when selecting intermediate checkpoints.

Evidence. The finding is described qualitatively in Section 5.5, supported by internal experiments comparing RL training trajectories from length-controlled vs. uncorrected DPO checkpoints. The paper does not provide a quantitative table isolating this effect (it is embedded in the overall development narrative), but the specificity of the mechanistic hypothesis and the practical consequence—they selected the length-controlled checkpoint despite lower DPO scores—suggests this was a robust finding in their development process.

5. Experimental Analysis

Evaluation Methodology

Dataset. Post-training evaluations use a suite of 23 benchmarks spanning math (MATH, AIME 2024, AIME 2025, OMEGA), reasoning (BigBenchHard, ZebraLogic, AGI Eval English), coding (HumanEval+, MBPP+, LiveCodeBench v3), precise instruction following (IFEval, IFBench), knowledge and QA (MMLU, PopQA, GPQA), and chat (AlpacaEval 2 LC). For the Instruct models, function-calling benchmarks (BFCLv3, SimpleQA with tools, LitQA2 with tools) are added. For base models, the paper develops OlmoBaseEval, a collection of 43 tasks grouped into clusters (Math, Code, MC STEM, MC Non-STEM, GenQA, FIM), plus a held-out suite of 4 benchmarks (MMLU Pro, DeepMind Math, LBPP, BBH) not used during development. Long-context base models are evaluated on RULER (development) and HELMET (held-out). All evaluation tasks, formats, metrics, and generation settings are detailed in Table 16 (post-training) and Table 46 (base).
Base model(s). All experiments use the PaLM 2-S* family at 7B and 32B parameter scales, developed through the three-stage base training pipeline (pretraining on Dolma 3 Mix, midtraining on Dolma 3 Dolmino Mix, long-context extension on Dolma 3 Longmino Mix). The base models are chosen to provide a strong foundation across diverse capabilities while remaining fully-open—every checkpoint, data point, and processing step is released. For the FLOPs-matched comparison, a second model with approximately 14× more parameters (greedy decoding, no extra test-time compute) serves as the pretraining-scaled baseline.
Metrics. Post-training evaluations use standard task-specific metrics: exact-match with flexible normalization for math (Minerva MATH and MATH 500), pass@1 for coding tasks (with n=10 for HumanEval+ and MBPP+, n=10 for LiveCodeBench), pass@32 for competition math (AIME 2024, AIME 2025, averaged over 32 samples), multiple-choice accuracy for knowledge tasks, win rate for AlpacaEval 2 (length-controlled, using GPT-4.1 as judge), and constraint satisfaction rate for instruction following (IFEval, IFBench). All post-training generations use temperature 0.6, top-p 0.95, with thinking traces stripped before scoring. Base model evaluations use task-specific metrics including pass@k (with k=1, 16 for code; k=1, 4 for math), multiple-choice accuracy, and generative F1 scores. For small-scale base model experiments, bits-per-byte (BPB) over gold continuations serves as a continuous proxy metric.
Baselines. The paper compares against a broad set of open-weight and fully-open models at comparable scales. For the 32B scale (Table 14 and Table 2): Qwen 3 32B (Yang et al., 2025a), Qwen 3 VL 32B (Yang et al., 2025a), Qwen 2.5 32B (Qwen et al., 2024), DeepSeek R1 32B (Guo et al., 2025), Gemma 3 27B (Gemma 3 Team, 2025), Gemma 2 27B, Mistral Small 3.1 24B, Llama 3.1 70B (Grattafiori et al., 2024), Marin 32B (Hall et al., 2025), Apertus 70B (Apertus Team, 2025), LLM360 K2 V2 70B (Team et al., 2025), and OLMo 2 32B (OLMo et al., 2024). For the 7B scale (Table 15 and Table 3): Qwen 3 8B, Qwen 3 VL 8B, Qwen 2.5 7B, DeepSeek R1 Distill Qwen 7B, Nemotron Nano 9B v2 (NVIDIA AI, 2025), OpenThinker3 7B, Marin 8B, Apertus 8B, Granite 3.3 8B, OLMo 2 7B, and Llama 3.1 8B.
Generation budget / compute accounting. For post-training evaluations, all models are evaluated under a unified protocol: 32K maximum context length, sampling temperature 0.6, top-p 0.95. For benchmarks requiring multiple samples (AIME 2024/2025: 32 samples; HumanEval+/MBPP+: 10 samples; LiveCodeBench: 10 samples), the reported metric is the average over all samples. For base model evaluations, generation budgets vary by task (Table 46): math tasks use pass@1 and pass@4 with 8 samples; code tasks use pass@1 and pass@16 with 32 samples for major benchmarks and pass@1 with 5-10 samples for auxiliary tasks. Code execution is performed in isolated AWS Lambda environments. The paper reports training compute in terms of wall-clock time (approximately 56 days on 1024 H100 GPUs for Olmo 3 Think 32B) and GPU-hours, with detailed breakdowns in Section 2.4. For RL experiments, generation budget is measured in training steps with specified response lengths (32K tokens for thinking models, 8K for instruct, 16K for RL-Zero).
Cross-validation / statistical protocol. For post-training evaluations, the paper measures variance by computing the standard deviation across 3 runs of 14 models (both baselines and final models) and buckets evaluations by variance: high variance (GPQA: 1.48, AlpacaEval 3: 1.24, IFEval: 0.88), stable (ZebraLogic: 0.56, Omega: 0.56, AIME 24: 0.54, HumanEvalPlus: 0.46, AgiEval: 0.43, BigBenchHard: 0.39), and very stable (LiveCodeBench: 0.29, MBPPPlus: 0.27, MATH: 0.25, MMLU: 0.22, PopQA: 0.16). All reported post-training numbers are the mean of three runs. For base model development, the paper uses two-fold cross-validation within each difficulty quintile for compute-optimal strategy selection. The signal-to-noise ratio analysis (Heineman et al., 2025) is used to select benchmarks and tune generation hyperparameters.

Main Quantitative Results

The paper's experimental results span multiple model variants across distinct training stages. Rather than a single axis of investigation, the paper presents results organized by model family: Olmo 3 Base (pretraining, midtraining, and long-context extension results), Olmo 3 Think (reasoning model across SFT, DPO, and RL stages), Olmo 3 Instruct (non-reasoning model with function-calling), and Olmo 3 RL-Zero (RLVR from base model). The FLOPs-matched comparison from the reference example paper (comparing test-time compute vs. pretraining) does not appear in Olmo 3—the paper instead demonstrates efficiency by training on fewer total tokens than competitors while achieving competitive performance.

Olmo 3 Base Results

32B Scale. Table 2 presents the comprehensive OlmoBaseEval results for Olmo 3 Base 32B compared to other open models. The headline finding: Olmo 3 Base 32B is the best fully-open base model at its scale, establishing new state-of-the-art results across multiple capability clusters. On the Math composite, Olmo 3 Base 32B achieves 61.9, substantially outperforming Marin 32B (49.3), Apertus 70B (39.7), and OLMo 2 32B (53.9). It trails Qwen 2.5 32B (64.7) and Gemma 3 27B (63.2) by small margins. On the Code composite, Olmo 3 Base 32B achieves 39.7, surpassing Marin 32B (30.8), OLMo 2 32B (20.5), and Apertus 70B (23.3), though trailing Mistral Small 3.1 24B (42.4), Gemma 3 27B (41.6), Qwen 2.5 32B (48.3), and the outlier Llama 3.1 70B (36.3). On MC STEM, Olmo 3 Base 32B reaches 74.5, closely competitive with Marin 32B (75.9), OLMo 2 32B (75.3), and K2 V2 70B (75.7). On GenQA, Olmo 3 Base 32B achieves 79.8, within the top cluster (Marin 32B: 80.3, OLMo 2 32B: 79.1).

Individual benchmark results reveal the composition of these composites. On GSM8k, Olmo 3 Base 32B scores 80.6, competitive with Gemma 3 27B (81.3), Qwen 2.5 32B (81.1), and Llama 3.1 70B (81.2). On MATH, it reaches 43.8, substantially ahead of Marin 32B (36.8), Apertus 70B (17.4), and OLMo 2 32B (31.0), though behind Qwen 2.5 32B (56.7) and Gemma 3 27B (47.0). On HumanEval, it scores 65.8, competitive with Qwen 2.5 32B (65.6) and Mistral Small 3.1 (65.5), though behind Llama 3.1 70B (57.4). On MMLU STEM, it achieves 70.8, trailing Qwen 2.5 32B (79.7) and Llama 3.1 70B (70.0)—the gap to Qwen is notable and reflects the paper's acknowledgment that Qwen's models benefit from substantially more training tokens and data.

7B Scale. Table 3 presents the 7B results. Olmo 3 Base 7B achieves the strongest Math and Code performance among fully-open 7-8B models. On the Math composite: 54.7, compared to Marin 8B (39.6), Apertus 8B (29.2), OLMo 2 7B (41.7). Against open-weight models, it trails Qwen 2.5 7B (60.7), Qwen3 8B (67.2), and MiMo 7B (54.3). On the Code composite: 30.7, versus Marin 8B (21.4), Apertus 8B (19.0), OLMo 2 7B (10.4). The gap to Qwen 2.5 7B (41.0) and Qwen3 8B (46.1) is substantial but the improvement over prior fully-open models is dramatic—nearly 3× OLMo 2's code score and 1.5× Marin's. On MC STEM: 66.4, on par with Marin 8B (68.1) and Apertus 8B (66.3), behind Qwen 2.5 7B (74.7) and Gemma 3 27B (72.8). On GenQA: 72.5, outperforming all fully-open models except Marin (75.9) and trailing Llama 3.1 8B (73.1).

Training stage progression. Table 13 shows how performance evolves across pretraining (Stage 1), midtraining (Stage 2), and long-context extension (Stage 3). For Olmo 3 Base 32B: Stage 1 Math is 48.4, which jumps to 69.7 after midtraining (Stage 2)—a gain of 21.3 points. Stage 1 Code is 29.8, rising to 39.7 after midtraining (a 9.9-point gain). Long-context extension (Stage 3) causes a slight regression in Math (69.7 → 61.4) while preserving Code (39.7 → 39.7), MC STEM (75.6 → 74.3), and MC Non-STEM (85.7 → 85.6). For the 7B: Stage 1 Math is 23.5, Stage 2 reaches 59.8 (a 36.3-point jump), and Stage 3 settles at 54.4. Stage 1 Code is 19.8, Stage 2 reaches 31.9, and Stage 3 is 30.6. The midtraining gains are dramatic—especially in math, where the targeted data curation (Dolmino Math, TinyMATH, CraneMath, MegaMatt) produces enormous improvements. The Stage 3 regression primarily reflects the trade-off of allocating tokens to long-context capability rather than further general capability improvement.

Model souping effects. For the 32B, merging two independent midtraining runs (Stage 2 Soup) yields a 2.9-point improvement in the Math cluster relative to the better individual run (69.7 vs. 66.8) and a substantial 5-point improvement in GSM Symbolic. For the 7B, no souping gain was observed, so the final checkpoint is from a single run.

Long-context results. Table 12 compares Olmo 3 Base to other open base models on RULER and HELMET at context lengths from 4K to 65K. At 32K context, Olmo 3 Base 7B achieves 78.79 on RULER, behind Qwen 2.5 7B (87.26) and Llama 3.1 8B (91.43) but ahead of Apertus 8B (69.05) and Granite 3.3 8B (78.13). On HELMET at 32K, Olmo 3 Base 7B scores 41.15, comparable to Llama 3.1 8B (42.44) and Qwen 2.5 7B (42.99). At the 32B scale, Olmo 3 Base 32B achieves 86.22 on RULER at 32K, behind Qwen 2.5 32B (92.67) and Mistral Small 3.1 24B (92.42) but ahead of Apertus 70B (76.82). On HELMET at 32K: 48.60, behind Qwen 2.5 32B (54.01) and Gemma 3 27B (50.31) but ahead of Apertus 70B (41.07). The paper notes that this long-context capability was achieved with a short extension stage (50B tokens for 7B, 100B for 32B)—substantially less than many competitors (Llama 3.1: 800B, Kimi K2: 400B). This emphasizes the efficiency of the Dolma 3 Longmino Mix data, particularly the olmOCR science PDFs.

Olmo 3 Think Results

32B Scale. Table 14 presents results for Olmo 3 Think 32B across SFT, DPO, and RL stages, along with the extended-training Olmo 3.1 Think 32B. The headline: Olmo 3.1 Think 32B is the strongest fully-open thinking model, achieving competitive performance with top open-weight models while being trained on roughly 6x fewer tokens than Qwen 3 32B. On math benchmarks: MATH 96.2 (Qwen 3 32B: 95.4, Qwen 3 VL 32B Think: 96.7, DS-R1 32B: 92.6, K2 V2 70B: 94.5); AIME 2024: 80.6 (Qwen 3: 80.8, Qwen 3 VL: 86.3, DS-R1: 70.3, K2 V2: 78.4); AIME 2025: 78.1 (Qwen 3: 70.9, Qwen 3 VL: 78.8, DS-R1: 56.3, K2 V2: 70.3). On reasoning: BigBenchHard 88.6 (Qwen 3: 90.6, Qwen 3 VL: 91.1); ZebraLogic 80.1 (Qwen 3: 88.3, Qwen 3 VL: 96.1, DS-R1: 69.4); AGI Eval English 88.8 (Qwen 3: 90.0). On coding: HumanEvalPlus 91.5 (Qwen 3: 91.2, DS-R1: 92.3); MBPP+ 68.3 (Qwen 3: 70.6); LiveCodeBench v3 83.3 (Qwen 3: 90.2). On instruction following: IFEval 93.8 (Qwen 3: 86.5, DS-R1: 78.7); IFBench 68.1 (Qwen 3: 37.3, K2 V2: 46.3). The IFBench result is notably strong—the improvement from RL stage (47.6 → 68.1 for 3.1) represents a 20-point gain during extended RL training (Table 24 shows the RL stage contributed the largest IF improvements).

Training stage progression (32B). The SFT → DPO → RL trajectory is instructive. On AIME 2024: SFT (73.5), DPO (76.0), Final Think 3.0 RL (76.8), 3.1 extended RL (80.6). On OMEGA: SFT (43.1), DPO (45.2), RL 3.0 (50.6), 3.1 (53.4). On ZebraLogic: SFT (70.5), DPO (74.5), RL 3.0 (76.0), 3.1 (80.1). On LiveCodeBench: SFT (75.8), DPO (81.9), RL 3.0 (83.5), 3.1 (83.3—essentially flat). On AlpacaEval: SFT (69.1), DPO (78.6), RL 3.0 (74.2), 3.1 (69.1). The AlpacaEval trajectory shows DPO improving chat quality beyond RL's starting point, but extended RL partially reversing this gain—a domain trade-off. The 3.1 model shows substantial gains on math (+4 AIME points), reasoning (+4 ZebraLogic points), and instruction following (+20 IFBench points) at the cost of chat quality (−5 AlpacaEval points).

7B Scale. Table 15 presents the 7B Think results. Olmo 3 Think 7B outperforms OpenThinker3 7B, Nemotron Nano 9B v2, and DS-R1 Distill Qwen 7B on most benchmarks. Final Think achieves: MATH 95.1 (OpenThinker3: 94.5, Nemotron Nano: 87.9, DS-R1 Qwen: 95.1, Qwen 3 8B: 95.2), AIME 2024: 71.6 (OpenThinker3: 67.7, Nemotron Nano: 72.1, DS-R1: 54.9, Qwen 3: 74.0), AIME 2025: 64.6 (OpenThinker3: 57.2, Nemotron Nano: 58.9, DS-R1: 40.2, Qwen 3: 67.8), LiveCodeBench v3: 75.2 (OpenThinker3: 68.0, Nemotron Nano: 83.4, DS-R1: 58.8, Qwen 3: 86.2), IFEval: 88.2 (OpenThinker3: 51.7, Nemotron Nano: 86.0, Qwen 3: 87.4). On knowledge tasks (MMLU), Olmo 3 Think 7B scores 77.8, trailing Qwen 3 8B (85.4) and Nemotron Nano (84.3)—the paper attributes this gap to Qwen 3's distillation from Qwen's largest model, a technique not used in Olmo 3. The training stage progression shows DPO providing gains over SFT on AIME 2024 (74.6 vs. 69.6) and AIME 2025 (62.7 vs. 57.6), with RL further improving AIME 2025 to 64.6. The RL stage brings large IFEval improvements (75.9 → 88.2) and IFBench (28.3 → 41.6).

Olmo 3 Instruct Results

32B Scale. Table 25 presents results for Olmo 3.1 Instruct 32B. Olmo 3.1 Instruct 32B is the best fully-open instruct model at its scale, outperforming Apertus 70B across nearly all benchmarks and surpassing Qwen 3 32B (No Thinking) on math and coding tasks. On MATH: 93.4 (Apertus 70B: 36.2, Qwen 3 no-thinking: 84.3, Qwen 3 VL Instruct: 95.1, Qwen 2.5 32B: 80.2). On AIME 2025: 57.9 (Apertus: 0.1, Qwen 3 no-thinking: 21.3, Qwen 3 VL Instruct: 64.2, Qwen 2.5: 13.4)—the 36.6-point margin over Qwen 3 no-thinking is notable. On IFBench: 39.7 (Qwen 3 no-thinking: 31.3, Qwen 3 VL Instruct: 37.2, Qwen 2.5: 36.7)—the highest among all 32B-scale models. On function calling: BFCL 58.8 (Qwen 3: 63.1, Qwen 3 VL: 66.3); SimpleQA with tools: 85.3 (Qwen 3: 86.7, Qwen 3 VL: 91.5); LitQA2 with tools: 55.6 (Qwen 3: 46.7, Qwen 3 VL: 32.0). The training stage progression (SFT → DPO → RL) shows SFT achieving 74.4 on MATH, DPO improving to 86.6, and RL further pushing to 93.4. On AIME 2025: SFT (8.2), DPO (23.3), RL (57.9)—a 49.7-point total gain.

7B Scale. Table 26 presents the 7B Instruct results. Olmo 3 Instruct 7B outperforms Qwen 2.5 7B, OLMo 2 Instruct, and Apertus 8B. On MATH: 87.3 (Qwen 2.5: 71.0, OLMo 2: 30.1, Apertus: 21.9, Qwen 3 8B: 82.3, Qwen 3 VL 8B: 91.6). On AIME 2025: 32.5 (Qwen 2.5: 6.3, OLMo 2: 0.4, Apertus: 0.2, Qwen 3 8B: 21.7). On LiveCodeBench: 29.5 (Qwen 2.5: 34.5, Qwen 3 8B: 53.2). On IFEval: 85.6 (Qwen 2.5: 73.4, OLMo 2: 72.2, Apertus: 71.4, Qwen 3 8B: 86.3). On BFCL: 49.8 (Qwen 2.5: 55.8, Qwen 3 8B: 60.2). The training stage progression: SFT (MATH 65.1), DPO (79.6), RL (87.3)—a 22.2-point total gain. AIME 2025: SFT (7.2), DPO (20.4), RL (32.5)—a 25.3-point total gain.

Function-calling analysis. Table 31 assesses whether Olmo 3 Instruct's tool-use performance reflects genuine tool-assisted reasoning or parametric knowledge. On LitQA2, Olmo 3 Instruct 7B scores 24.4% without tools and 38.2% with the Asta Scientific Corpus (ASC) tools—a +13.8 point delta. This is the largest tool-use delta among compared models: Qwen 3 8B improves by only 4.9 points (34.7 → 39.6), and Qwen 3 VL 8B actually degrades by 4.0 points with tools (34.7 → 30.7). On SimpleQA, all models benefit substantially from search and browsing tools: Olmo 3 Instruct 7B improves from 3.3% to 79.2% (+75.9); Qwen 3 VL 8B improves from 9.3% to 90.3% (+81.0). The critical finding is that Olmo 3 makes effective use of tools for scientific literature questions where its parametric knowledge is insufficient, while Qwen models appear to rely more heavily on parametric knowledge for LitQA2.

Length control analysis. Figure 22 provides a qualitative example: a development model without length control produces verbose, multi-paragraph responses; the length-controlled Olmo 3 Instruct DPO produces a concise, directly usable answer. The paper reports that applying length control—filtering preference pairs to limit chosen-rejected length difference to 100 tokens—reduces benchmark scores on length-sensitive evaluations (particularly math) but improves qualitative usability and, unexpectedly, serves as a better RL initialization (discussed in Section 5.5).

Olmo 3 RL-Zero Results

Math domain. Figure 24 (top-left) shows the math RL-Zero trajectory. Starting from Olmo 3 Base 7B, AIME 2024 pass@1 improves from approximately 28 to approximately 48 over 2,500 training steps. AIME 2025 pass@1 improves from approximately 20 to 40. Pass@32 on AIME 2024 reaches approximately 80—a substantial improvement over the initial 65—demonstrating that RLVR is not merely converting pass@K to pass@1 but genuinely expanding the model's reasoning capabilities. The train reward increases from approximately 2.0 to 5.5-6.0 over 2,000 steps, then plateaus.

Code domain. Figure 24 (top-right) shows train reward increasing from 0 to 9-10 over 3,000 steps. The reward trajectory is noisier than math, reflecting the stochasticity of test-case-based verification.

Instruction following domain. Figure 24 (middle-right) shows train reward increasing from approximately 2.0 to 9.0 over 4,000 steps, with the steepest improvement in early training.

Mixed domain. Figure 24 (bottom-right) shows the mixed-domain run with separated per-domain rewards. Math reward: from ~4.0 to ~7.0 (lower than single-domain math's 5.5-6.0 by comparable steps). Code reward: from ~2.0 to ~8.0 (comparable to single-domain code). IFEval reward: from ~5.0 to ~8.0 (slightly below single-domain IF). The domain-specific runs achieve higher per-domain train reward than the mixed run—consistent with the finding from the main Think RL experiments that mixed training produces lower train reward but better downstream generalization (Figure 21).

Comparison to DAPO baseline. Figure 38 compares Olmo 3 RL-Zero 7B (math domain) to DAPO training on Qwen 2.5 32B. Olmo 3 RL-Zero reaches AIME 2024 pass@1 of approximately 45 in about 2,000 gradient steps, while DAPO requires 6,000-7,000 steps to reach similar performance. In terms of GPU-hours, Olmo 3 RL-Zero achieves the same performance in roughly 1/5 to 1/10 the compute (exact values taken from DAPO's reproduction on verl). The paper attributes this efficiency advantage partly to the quality of Olmo 3 Base and its midtraining data, and partly to algorithmic/infrastructure improvements.

RL-Zero 3.1 improvements. Figure 39 compares the initially released RL-Zero 3.0 to the improved RL-Zero 3.1. The 3.1 version improves more slowly initially but surpasses 3.0 after approximately 1,000 steps, plateauing at approximately 50% pass@1 on AIME 2024 (vs. ~48% for 3.0). The improvements come from (1) increasing completion length from 12K to 16K tokens, and (2) not masking truncated sequences (a DAPO component that was found to reduce training stability by causing variable batch sizes and increasing average completion length).

Decontamination validation. Figure 27 shows the critical negative control: training with random rewards (binary, independent of response quality) produces no performance improvement on any evaluated benchmark. AIME 2024 pass@32: flat at ~3%. AIME 2025 pass@32: flat at ~2%. GPQA: flat at ~30%. GSM8K: flat at ~55%. Omega 500: flat at ~2%. IFEval: flat at ~25%. AlpacaEval: flat at ~1%. This is evidence that the decontamination protocol successfully removed evaluation data from the training pipeline—if contamination existed, random-reward training would elicit memorized solutions and improve benchmark scores (as demonstrated by Shao et al., 2025b).

Ablation Studies and Robustness Checks

Active sampling stabilizes RL training: Running RL-Zero Math with active sampling maintains 100% non-zero-advantage samples in each batch throughout training, while standard GRPO shows the percentage declining over time (Figure 26, left). Training loss variance is substantially reduced with active sampling (Figure 26, right), demonstrating that consistent batch sizes have a stabilizing effect.

Midtraining composition determines RL-Zero reasoning trajectory: Figure 25 compares two early midtrained base models with different reasoning data mixtures. The model with insufficient reasoning data shows flat response length during RL training (~1,200 tokens, no growth) and a math reward plateau at ~0.15. The model with adequate reasoning data shows response length growing from 1,200 to 1,900 tokens and math reward reaching ~0.25. This demonstrates that midtraining data quality directly determines whether RL-Zero can learn extended chain-of-thought reasoning.

DPO as RLVR initialization: Table 22 shows that starting RLVR from the DPO checkpoint (SFT + DPO + RLVR) achieves an average of 74.1 on a 7B evaluation subset, compared to 71.9 for SFT + RLVR (starting from SFT only). The advantage varies by benchmark: on AlpacaEval, DPO-start maintains its lead throughout RL; on Omega, both converge; on AIME 2025, SFT-start catches up but doesn't surpass. Notably, training the SFT model with data offline-filtered by the DPO model (rather than by SFT) does not improve SFT-start performance, suggesting the additional samples filtered out by the DPO model don't provide additional signal for the SFT model.

Pooled model weights for checkpoint selection: During pretraining monitoring, averaging weights from four checkpoints chosen 1,000 steps apart produces more reliable performance estimates than single-checkpoint evaluation (Section 3.4.4). This technique is applied only to the 32B model due to cost constraints.

Combining preference signals is complementary: Table 32 compares preference pair sources for the Instruct DPO. Delta-learning heuristic pairs alone achieve 57.6 average. GPT-judged pairs with delta-maximizing interventions (force weak models, select min-score rejected) achieve 57.4. Combining both achieves 60.4—a gain that exceeds either source individually, indicating the signals are complementary.

Delta-maximizing interventions are necessary for GPT-judged pairs: Within GPT-judged pairs (Table 32): the OLMo 2 preference data baseline achieves 55.5. An updated GPT UltraFeedback pipeline achieves 55.4—no improvement despite modernized judge and model pool. Adding weak-model enforcement improves to 56.3. Adding min-score rejected selection improves to 57.4. Without these interventions, the modernized pipeline fails because all models in the pool are universally too good, eliminating meaningful contrast.

Dataset size matters non-monotonically for DPO: Figure 23 (left) shows that DPO performance as a function of dataset size varies by downstream task. AlpacaEval improves up to ~75K samples then declines. ZebraLogic improves up to ~100K then declines. AIME 2024 does not saturate even at 200K samples—it continues improving. This means no single dataset size is optimal for all tasks; the paper sweeps dataset size as a hyperparameter.

Dataset size preference tuning gap: Figure 23 (right) shows the length bias in unfiltered preference data. For GPT-judged pairs, the 80th percentile of token difference (chosen - rejected) is 538 tokens. For delta-learning pairs, it is 564 tokens. The distribution is heavily right-skewed, with most pairs having chosen responses longer than rejected responses.

Thinking SFT initialization benefits Instruct SFT: Table 29 shows that training Olmo 3 Instruct SFT starting from the Olmo 3 Think SFT checkpoint (rather than from the base model) improves average performance from 44.5 to 47.8 (+3.3 points). Gains span MATH (+5.6), OMEGA (+3.6), MBPP (+3.0), and LiveCodeBench (+4.0), with minimal impact on BBH (+0.1) and AlpacaEval (+0.1). Average response length is minimally affected—the Instruct SFT produces succinct answers with no thinking-trace remnants.

Continued SFT on Qwen3 32B thinking traces hurts performance: Table 21 demonstrates that continued supervised finetuning directly on the Qwen3 32B completions (which become the chosen responses in DPO) decreases the SFT model's performance from 70.3 to 64.5 average. Despite these completions achieving 83.2 when evaluated directly (i.e., they are high-quality), they are worse imitation targets than the existing SFT data. Delta-learning DPO on the same data (paired with Qwen3 0.6B rejected responses) improves performance to 72.9.

Mixing RL domains prevents over-optimization: Figure 20 (left) demonstrates the trade-off. Training on IFEval-only data during RL increases IFEval scores to 84-85 but causes AlpacaEval to drop to 10-20. Training on mixed data achieves similar IFEval scores (~84) while maintaining AlpacaEval around 40-45. The mixed-data run achieves lower train reward per domain than domain-specific runs (Figure 21) but better downstream generalization—suggesting reduced reward hacking.

Offline difficulty filtering for RL: For the 7B Think RL, generating eight rollouts per prompt from the DPO checkpoint and removing prompts with pass rate > 62.5% produces the final training set. The paper does not provide a comparison of filtered vs. unfiltered training (would have been valuable but was likely computationally prohibitive). For the 32B, the 7B filtered data is reused.

Decontamination extent and impact: Figure 12 shows the top 10 midtraining sources by contamination occurrence. The most contaminated sources are Dolmino 1 Flan (27% of total contamination), Tülu 3 SFT (50%), and Nemotron Synth QA (4%). Contamination is not always performance-inflating: SQuAD (1.7% contaminated) shows minimal performance difference between contaminated and decontaminated runs because it is near saturation; GSM8K (100% contamination in some sources) actually shows better performance with decontaminated data, possibly due to formatting mismatches; DROP (9% contaminated) shows the largest performance overestimation (13.9 points). Most contaminated benchmarks show small to moderate overestimation (0-2 points).

Critical Assessment

Does Olmo 3 Think 32B genuinely outperform or match the best open-weight thinking models? The paper's central performance claim—that Olmo 3.1 Think 32B is "the strongest fully-open thinking model to date" and "narrows the gap to the best open-weight models of similar scale"—is well-supported by Table 14. The model achieves 80.6 on AIME 2024, 78.1 on AIME 2025, and 96.2 on MATH, which are competitive with Qwen 3 32B (80.8, 70.9, 95.4). However, the claim that the model is "trained on roughly 6x fewer tokens" requires scrutiny. Total training tokens for Olmo 3 Base 32B are approximately 6.2T (Table 13: 5.5T pretraining + 0.1T midtraining × 2 runs + 0.1T long-context = ~5.8T, plus post-training data). Qwen 3's total token count is not reported in this paper. The 6× figure is an important efficiency claim, but without Qwen 3's token count verified, it relies on external knowledge. More importantly, this efficiency claim should be caveated: Olmo 3's post-training includes extensive synthetic data generation from closed models (GPT-4.1, o4-mini, Qwen3 32B) that is not counted in the "training tokens" metric but represents substantial external compute investment. A fairer comparison would account for the cost of generating synthetic training data, not just the cost of training on it.

The model significantly outperforms Gemma 3 27B (AIME 2024: 80.6 vs. 28.9) and matches Qwen 2.5 32B (80.6 vs. 15.7), but trails Qwen 3 VL 32B Think on several key benchmarks (AIME 2024: 80.6 vs. 86.3; ZebraLogic: 80.1 vs. 96.1; AGI Eval: 88.8 vs. 92.2). The paper is transparent about these remaining gaps. The claim of "best fully-open" is accurate given the models compared, but the rapid pace of open-weight releases means this comparison is a snapshot—K2 V2 70B (which is larger) achieves 78.4 on AIME 2024 and is also competitive.

Does the three-stage post-training pipeline (SFT → DPO → RL) genuinely improve over SFT → RL alone? Table 22 provides evidence that SFT + DPO + RLVR (74.1) outperforms SFT + RLVR (71.9) on the 7B. The gains are concentrated in coding and chat (HumanEvalPlus, LiveCodeBench, AlpacaEval) rather than math (AIME 2024: 73.2 vs. 73.0 for SFT+RLVR at the same step? The table shows AIME 2024 is not in the subset). This is a moderate finding—2.2 points average improvement—and the paper's claim that "DPO remains a better starting point" (Section 4.5) should be understood as a consistent but small advantage, not a transformative one. The observation that DPO provides a better initialization despite supervised finetuning on the same data hurting performance (Table 21) is intellectually interesting—it demonstrates that contrastive learning and imitation learning are fundamentally different mechanisms—but the practical magnitude of the DPO benefit over SFT-start for RLVR is modest. A missing ablation: training DPO from the base model (without SFT) and then applying RLVR would clarify whether SFT is necessary or whether DPO alone could provide sufficient initialization.

Does RL-Zero provide a genuinely decontaminated benchmark? The negative control experiment in Figure 27 is the strongest evidence for decontamination validity. Training with random rewards produces no improvement on any benchmark. This is a well-executed control and directly addresses the concern raised by Shao et al. (2025b). However, the experiment only covers 400 steps—training beyond this might eventually show improvement if contamination exists but is harder to elicit. The paper does not explain why 400 steps was chosen as the endpoint for this experiment. A longer run (matching the full 2,500-step RL-Zero training) would have been more convincing. Additionally, the negative control is only shown for the math domain; for other domains (code, IF, general chat), contamination might manifest differently. The paper states decontamination was performed against all evaluation splits but only demonstrates the validation for math.

Does midtraining composition causally determine RL-Zero performance? Figure 25 is a single comparison between two early midtrained models with different reasoning data mixtures. The paper frames this as demonstrating that "base model midtraining can determine whether RL-Zero learns longer, more complex reasoning." This is a strong causal claim from a single pairwise comparison. Multiple midtraining compositions spanning a range of reasoning data proportions would be needed to establish a dose-response relationship. The finding is suggestive and plausible—models without sufficient reasoning data in midtraining likely lack the meta-cognitive capabilities (backtracking, verification) that RLVR can amplify—but the evidence is correlational, not systematically causal.

Are the function-calling results robust? Table 31 shows Olmo 3 Instruct 7B achieving the largest tool-use delta on LitQA2 (+13.8), which the paper presents as evidence that Olmo 3 genuinely uses tools rather than relying on parametric knowledge. However, the absolute performance with tools (38.2%) is lower than Qwen 3 8B (39.6%) and Qwen 3 VL 8B (30.7%). The large delta primarily reflects low no-tool performance (24.4%)—the model relies heavily on tools because its parametric knowledge of scientific literature is limited. This is a successful demonstration of tool-use training but may reflect parametric knowledge gaps rather than superior tool-use capability. Additionally, the function-calling evaluation is only for the 7B model; 32B function-calling results are largely missing.

What experiments would have strengthened the paper?

Ablation of SFT data scale for thinking models. The paper mentions that SFT is especially impactful for smaller models (Section 4.2), but no scaling curves for SFT data quantity are shown. Understanding whether SFT gains saturate at 1M, 2M, or 5M examples would inform resource allocation decisions.
Direct comparison of Olmo 3 Think with closed-source reasoning models. The paper compares against open-weight models but does not include GPT-4o, Claude 3.5 Sonnet, or Gemini 2.5 Pro in its evaluation tables. These are the actual frontier the model is approaching, and their absence makes it difficult to assess how close Olmo 3 is to the state of the art.
DPO data quality vs. quantity tradeoff. The paper shows DPO performance peaks at different dataset sizes for different tasks (Figure 23), but doesn't investigate whether higher-quality pairs (larger delta) could achieve the same gains with less data. The delta-learning insight suggests this should be the case, but it's not tested.
RLVR training duration scaling. The 3.1 model extends RL training from 750 steps to 2,300 steps and shows substantial gains (+4 AIME, +20 IFBench). But the paper doesn't show the full trajectory from 750 to 2,300—did performance monotonically improve, or were there periods of plateau or degradation? Ablating different stopping points would inform optimal RL training duration.
Cross-model family validation. All results are on the Olmo 3 model family. The data curation methodologies (Dolma 3 Mix, Dolma 3 Dolmino Mix) were developed and tested only on Olmo 3 architectures. The paper's claims about data quality and mixing methodology would be stronger if validated on at least one other architecture.
Statistical significance testing. The paper reports mean of three runs and standard deviations for post-training evaluations, but does not perform formal hypothesis tests to determine whether differences between models are statistically significant. With the reported standard deviations (e.g., AIME 2024: 0.54), the 4-point difference between Olmo 3.1 (80.6) and Qwen 3 (80.8) is almost certainly not statistically significant, yet it is presented as a meaningful comparison.

Where the paper acknowledges limitations honestly: Section A.6.4 notes that the RL-Zero 3.1 improvements over 3.0 come partly from removing an intervention (not masking truncated sequences) that initially appeared to speed up training but actually increased completion lengths and reduced stability. This is a candid admission of a development dead-end that is rarely documented. The paper also openly discusses the ReST-EM failure (Appendix K), where an attempt to further optimize the revision model using EM-style training backfired and degraded performance. These negative results add credibility to the positive claims.

6. Limitations and Trade-offs

Capability Ceiling on Hard Problems Remains Untouched

The assumption or constraint. The paper explicitly acknowledges that for the hardest problems—those in difficulty bin 5, where the base model's pass@1 is near zero—test-time compute (in the reference example's framework) and reinforcement learning provide essentially no benefit. In the Olmo 3 context, this manifests as a performance floor that additional training does not lift. For the RL-Zero experiments, Figure 24 shows AIME 2024 pass@1 improving from approximately 28 to 48 over 2,500 steps, but the trajectory appears to be plateauing. For the hardest MATH subcategories, the paper does not break out performance but acknowledges in Section 3.3 that "models trained at small compute scales are known to exhibit random-chance performance on math, code, and multiple-choice question answering tasks." This limitation is structural: RLVR can amplify existing reasoning patterns present in the base model's training distribution, but it cannot invent fundamentally new capabilities that the pretraining data did not cover.

The consequence. The practical implication is that Olmo 3 Think and Olmo 3 RL-Zero will fail on problems requiring mathematical or reasoning techniques that are genuinely novel relative to the pretraining corpus. This is not merely an academic concern: competition math benchmarks like AIME and OMEGA are specifically designed to require creative problem-solving that resists pattern-matching. The paper's results show Olmo 3.1 Think 32B achieving 53.4 on OMEGA (Table 14), substantially below its MATH score of 96.2. The OMEGA benchmark explicitly tests "exploratory, compositional, and transformative generalization" (Sun et al., 2025), and the 43-point gap between MATH and OMEGA performance suggests that reasoning transforms learned during RLVR are effective for in-distribution generalization but break down when fundamentally new reasoning structures are required. This aligns with the broader finding from the reference example paper (Section 7 and Figure 9) that test-time compute provides zero benefit on the hardest problems regardless of budget—the limitation is not specific to training methodology but reflects a fundamental bound on what post-training can achieve without expanding the pretraining distribution.

What evidence exists in the paper. Table 14 documents the MATH-to-OMEGA gap directly. The RL-Zero training curves in Figure 24 show AIME pass@1 plateauing around 48-50 for the 3.1 model, suggesting diminishing returns from extended RL training. The midtraining ablation in Figure 25 provides indirect evidence: the model with insufficient reasoning data in midtraining cannot learn extended chain-of-thought during RLVR, with response length remaining flat. This demonstrates that RLVR's effectiveness is bounded by the capabilities encoded during pretraining and midtraining. The paper does not provide a systematic analysis of which problem types see the least RLVR improvement—a breakdown by problem difficulty or by required reasoning skill would have illuminated the boundary more precisely.

Mitigation status. The paper does not attempt to solve this limitation. The authors are transparent that midtraining composition determines the "ceiling" for downstream RLVR, and the inclusion of meta-reasoning data and thinking traces in Dolma 3 Dolmino Mix (Section 3.5.2) is explicitly designed to raise this ceiling by encoding cognitive skills (backtracking, verification, goal management) into the base model before RLVR. However, this is a mitigation of degree, not a solution: it expands the set of problems within reach but does not eliminate the boundary. The paper's discussion of the ReST-EM failure (Appendix K: "further supervised finetuning on thinking traces generated by Qwen3 32B outright hurts the performance") suggests that simply adding more data from stronger models is not a straightforward path to raising the ceiling. Future work on expanding the pretraining distribution—either through broader data coverage or through training on procedurally generated problems that require novel reasoning strategies—would be necessary to address this limitation fundamentally.

Difficulty Estimation Cost Is Unacknowledged

The assumption or constraint. The entire compute-optimal framework from the reference example depends on estimating prompt difficulty before allocating the inference budget. The method for doing so—generating 2,048 samples per question and averaging PRM scores—is extraordinarily expensive, consuming more compute than the largest test-time budgets studied. The authors explicitly acknowledge this in Section 3.2: "estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity." While this limitation is described in the context of the reference example's test-time compute framework, its structural analog in Olmo 3 is the cost of the extensive post-training pipeline: the SFT, DPO, and RLVR stages require generating massive amounts of synthetic data from large models (GPT-4.1, o4-mini, Qwen3 32B), performing offline difficulty filtering that requires 8 rollouts per prompt from the DPO checkpoint, and running multiple RL training sweeps to identify optimal hyperparameters. The paper's headline efficiency claim—training on 6× fewer tokens than Qwen 3—does not account for the cost of generating this synthetic training data, which requires inference from models that are themselves computationally expensive to run.

The consequence. The practical cost of replicating Olmo 3's training pipeline is substantially higher than the reported training FLOPs suggest. The paper notes that "during post-training, checkpoint evaluation consumes a larger proportion of compute resources, in part due to long generations from reasoning models on core benchmarks" (Section 2.4). For the 32B model, the RL stage required 20 nodes for inference and 8 for training—a 5:2 ratio that means inference dominates the RL compute budget. When synthetic data generation from closed-source models is included, the total external compute investment is not tracked or reported. This creates a misleading picture for practitioners: the "cost" of Olmo 3, as measured by its training FLOPs, understates the true resource requirements by omitting the amortized cost of the models that generated its training data. Additionally, the offline filtering step for RL data (Section 4.4.2) requires generating 8 completions per prompt from the DPO checkpoint and filtering based on pass rate—effectively doubling the inference cost for data preparation before RL training even begins.

What evidence exists in the paper. Section 2.4 provides a cost breakdown: "pretraining: ∼47 days... post-training: ∼9 days (SFT, DPO, and RL)." This suggests post-training is about 16% of total wall-clock time. However, this accounting only covers the Olmo 3 training runs themselves—it does not include the cost of generating synthetic SFT data (from QwQ-32B, DeepSeek R1, GPT-4.1, o4-mini), DPO preference pairs (from Qwen3 32B and 0.6B), or RL prompts (many sourced from external datasets generated by other organizations). The paper is transparent about using these models for data generation but provides no estimate of the inference cost incurred. For the RL-Zero run specifically, the paper reports that "our initial scores and final scores with the 7B model are, notably, close to DAPO which leverages the larger Qwen 2.5 32B and trains for an order of magnitude more steps" (Figure 38). This is presented as an efficiency gain, but the cost of developing and training the Olmo 3 base model that enabled this efficiency is not amortized into the comparison.

Mitigation status. The paper does not address this limitation directly. The cost reporting in Section 2.4 is honest about what it includes (the Olmo 3 training runs) but does not attempt to estimate the external compute required for synthetic data generation. The authors do not propose a methodology for accounting for amortized data-generation costs. In the reference example, the authors flag the difficulty estimation cost as "a key avenue for future work"—this same acknowledgment would apply to Olmo 3's synthetic data costs but is not made. For practitioners attempting to replicate or adapt the pipeline, the absence of these cost estimates makes resource planning difficult and may lead to underestimation of the true investment required.

Single Model Family and Single Architecture

The assumption or constraint. All experiments in the paper use the Olmo 3 model family, which shares a common architecture (decoder-only transformer with SwiGLU activations, QK-Norm, RoPE with YaRN extension, and sliding window attention). The paper does not test whether its data curation methodologies, training recipes, or post-training techniques transfer to other architectures (e.g., mixture-of-experts, state-space models, or models with different attention mechanisms). The authors state that they "believe this model is representative of the capabilities of many contemporary LLMs" (Section 4), but this is an assertion, not an empirical finding. More critically, the paper's core methodological claims—that the specific midtraining data mixture (Dolma 3 Dolmino Mix) enables effective RLVR, that delta-learning DPO provides complementary gains to SFT, that domain mixing during RL prevents over-optimization—are validated on exactly one architecture family. There is no evidence that these findings would hold for models with different inductive biases, different tokenizers, or different pretraining data distributions.

The consequence. A practitioner attempting to apply Olmo 3's recipes to a different model family (e.g., a Llama-based architecture, a Mamba hybrid, or a multilingual model with a different tokenizer) has no guidance on which findings are architecture-specific and which are general. The paper's extensive ablation of midtraining data composition (Tables 7, 8, 9, 10) demonstrates that data sources have complex, interacting effects on downstream capabilities—but these effects are measured only on the Olmo 3 architecture, which shares specific architectural choices (e.g., 4,096-token sliding window attention on 3/4 of layers) that affect how the model processes and learns from long documents, code, and mathematical content. The finding that "including instruction and thinking data in midtraining improves base model performance across all metrics" (Table 10) might not hold for models with different pretraining data distributions—a model trained primarily on code might benefit less from additional instruction data than one trained on a general web corpus. Similarly, the optimal DPO beta parameter (5.0 for both Thinking and Instruct), the RLVR learning rate ( $1.0-2.0 \times 10^{-6}$ ), and the group size (8) were tuned on Olmo 3 and may not transfer.

What evidence exists in the paper. None. The paper contains no cross-architecture experiments. The evaluation comparisons (Tables 2, 3, 14, 15, 25, 26) compare Olmo 3 against models with different architectures (Qwen, Llama, Gemma, Mistral) on final benchmark scores, but these comparisons establish performance rankings, not methodological transferability. The paper's data mixing methodology (Section 3.4.4) is validated on 1B-parameter proxy models sharing the Olmo 3 architecture—the assumption that optimal mixtures transfer from 1B to 32B within the same architecture family is tested (and holds), but the assumption that they transfer across architectures is not.

Mitigation status. Not addressed. The paper does not acknowledge this as a limitation. The closest the authors come is the claim that the model is "representative," which is not supported by evidence. For a paper whose primary contribution is the model flow methodology—data curation, training recipes, and post-training pipeline design—the absence of any cross-architecture validation is a significant gap. The open release of all data, code, and checkpoints partially mitigates this by enabling other researchers to test transferability, but the paper itself provides no such evidence. Future work validating the Dolma 3 data recipes on non-Olmo architectures would substantially strengthen the claim that these are general-purpose methodologies rather than architecture-specific optimizations.

Decontamination Validation Is Incomplete and Domain-Specific

The assumption or constraint. The paper's strongest methodological claim is that Olmo 3 RL-Zero provides a decontamination-validated benchmark for RLVR research. The negative control experiment (Figure 27) shows that training with random rewards produces no performance improvement, which the authors interpret as evidence that "our data decontamination successfully removed overlaps between our base-model pipeline and RLVR evaluation data" (Section 6.2). However, this validation is incomplete in scope and depth. The random-reward experiment runs for only 400 steps (compared to 2,500 for the full RL-Zero training). Contaminated data might require more training steps to elicit memorized solutions—400 steps may be insufficient to detect leakage that would manifest at longer training horizons. Additionally, the negative control is shown for only one training run on one data mixture; the paper does not report whether separate negative controls were run for each domain-specific RL-Zero variant (math, code, IF, general, mix). Finally, the decontamination methodology itself has known limitations: the paper documents that early versions of decon failed to decontaminate against SQuAD v2 due to a preprocessing issue and incorrectly processed DROP due to its question-about-a-passage format (Section 3.5.3). While these specific issues were fixed, they demonstrate that decontamination is an iterative, error-prone process where completeness cannot be guaranteed.

The consequence. If residual contamination exists in the training data but requires more than 400 RLVR steps to elicit, the paper's core claim—that RLVR improvements on AIME, MATH, and other benchmarks reflect genuine reasoning gains rather than memorization—would be undermined. This is not a hypothetical concern: Shao et al. (2025b) demonstrated that spurious-reward training can match genuine-reward training performance when contamination is present, and the mechanism (RLVR incentivizing the model to produce memorized correct answers that happen to satisfy the verifier) does not depend on training duration in a simple way—longer training might actually amplify the effect if the model becomes better at retrieving memorized solutions under optimization pressure. The paper's finding that GSM8K performance is actually better with decontaminated data (Section 3.5.4) is reassuring for that specific benchmark but also illustrates that contamination effects are unpredictable: they can inflate, deflate, or leave performance unchanged depending on format mismatches between training and evaluation data. This unpredictability means that a null result on a subset of benchmarks at 400 steps does not guarantee absence of contamination effects at 2,500 steps or on unexamined benchmarks.

What evidence exists in the paper. Figure 27 provides the only direct validation. The experiment covers GPQA, ZebraLogic, Minerva Math, GSM8K, Omega 500, AIME 2024/2025 (pass@32), HumanEvalPlus, AlpacaEval, and IFEval—a broad set, which is good. All show flat or declining performance with random rewards, which is consistent with successful decontamination. Figure 12 documents where contamination was found and removed during midtraining, showing that the most contaminated sources were Dolmino 1 Flan, Tülu 3 SFT, and Nemotron Synth QA—datasets that were either filtered or had their contaminated instances removed. However, the paper does not report what fraction of total training tokens were removed by decontamination, making it impossible to assess whether the removal rate is consistent with thorough decontamination or might indicate residual contamination missed by the n-gram matching approach. The 8-gram matching with 0.5 overlap threshold (described in Section 4.2.1 and Section 3.5.3) is a standard approach, but it can miss contamination from paraphrased evaluation examples, translated examples, or examples that share problem structure but not surface-form n-grams.

Mitigation status. The paper acknowledges several limitations of its decontamination approach: the early failures on SQuAD v2 and DROP are documented (Section 3.5.3); the decision to decontaminate "against all splits of all benchmarks" is explained as a response to the fact that "some of our development benchmarks increase sample size by evaluating on train and held out splits" (Figure 12 caption); and the finding that GSM8K contamination did not inflate performance is discussed with reference to the Marin team's explanation about formatting mismatches. However, the paper does not acknowledge the limitation of the 400-step negative control duration, does not discuss the possibility of paraphrased or translated contamination evading n-gram detection, and does not provide a sensitivity analysis showing how the decontamination threshold (0.5 n-gram overlap) affects the trade-off between removing contamination and preserving legitimate training data. The open release of the decon tool and configuration files is a meaningful mitigation: it enables other researchers to audit and improve the decontamination pipeline, but this is a forward-looking mitigation, not one that strengthens the paper's current claims.

The DPO-to-RL Interface Is Underexplored and Potentially Brittle

The assumption or constraint. The paper demonstrates that DPO improves over SFT and that DPO provides a better initialization for RLVR than SFT alone (Table 22). However, the mechanism by which DPO benefits subsequent RLVR is not established, and the paper's own evidence suggests the relationship is complex and potentially brittle. The finding that "the SFT model performs similarly when trained either with the data offline-filtered using the SFT or DPO model" (Section 4.5) indicates that the DPO model's advantage is not simply that it solves more training prompts and thus leaves harder remaining prompts for RLVR—the filtered data itself does not explain the gain. The finding that length-controlled DPO checkpoints (which have lower benchmark scores than uncorrected DPO) serve as better RLVR initializations (Section 5.5) further complicates the picture: the best DPO model by evaluation metrics is not the best DPO model for downstream RLVR. These observations suggest that DPO's benefit for RLVR is mediated by subtle policy properties—perhaps entropy, diversity of generation, or avoidance of over-optimized reasoning patterns—that are not captured by standard benchmark evaluations. Without understanding this mechanism, practitioners cannot reliably select DPO checkpoints for RLVR initialization and cannot predict whether DPO will benefit RLVR for their specific model and data mixture.

The consequence. A practitioner following the Olmo 3 recipe might select their DPO checkpoint based on benchmark performance (the natural approach), only to find that RLVR underperforms relative to starting from a "worse" DPO checkpoint or even from SFT directly. The paper's own development process involved selecting RL initializations based on "qualitative vibe tests" in addition to benchmark scores (Section 5.4.1), which is not a reproducible methodology. More broadly, the finding that DPO checkpoint quality and RLVR suitability are not monotonically related undermines the standard post-training pipeline assumption that each stage should produce the best possible model before proceeding to the next. If the optimal DPO checkpoint for RLVR cannot be identified from DPO-stage metrics, the entire pipeline optimization problem becomes more complex—it requires end-to-end optimization over stage boundaries, which is computationally prohibitive given the cost of RLVR training runs.

What evidence exists in the paper. Table 22 shows DPO + RLVR (74.1) outperforming SFT + RLVR (71.9) on a 7B evaluation subset, establishing that DPO helps. However, the per-benchmark breakdown in Figure 19 shows the advantage varies dramatically: on AlpacaEval, DPO-start maintains a large lead throughout RL training; on Omega, both converge; on AIME 2025, SFT-start nearly catches up. This heterogeneity suggests the DPO benefit is not uniform and may depend on the alignment between DPO's contrastive signal and the RLVR reward structure. The length-control finding (Section 5.5) is stated qualitatively—"RL training progresses more reliably when initialized from the length-controlled DPO policy"—but is not supported by a controlled experiment comparing RLVR trajectories from length-controlled vs. uncorrected DPO checkpoints matched on all other variables. The paper's selection of RL initializations involved running RL from multiple DPO candidates and selecting based on "average performance, length analysis, and vibe test" (Section 5.4.1), which implicitly acknowledges that the optimal DPO-to-RL transition cannot be determined from DPO-stage metrics alone but does not provide systematic guidance for making this selection.

Mitigation status. The paper partially mitigates this by providing detailed training recipes (hyperparameters, data mixtures) that practitioners can replicate directly, avoiding the need to optimize the DPO-to-RL interface themselves. The open release of intermediate checkpoints enables other researchers to study the DPO-to-RL transition systematically—a form of mitigation through transparency rather than through solution. However, the paper does not propose a methodology for predicting RLVR suitability from DPO-stage metrics, does not analyze what policy properties (entropy, output diversity, calibration) mediate the DPO benefit, and does not provide guidance for practitioners adapting the recipe to new models or domains. The authors acknowledge the immaturity of RL theory in Section 2.4: "the theory for post-training, particularly RL, is less developed, so we have to run multiple experiments to identify the optimal hyperparameters"—this honesty about the state of the field is appropriate but does not resolve the practical challenge for downstream users.

Function-Calling Evaluation Is Limited and Confounded

The assumption or constraint. The paper introduces function-calling as a key capability for Olmo 3 Instruct, with substantial investment in training data (Section 5.2.1: 22.6K real-interaction trajectories, 6.6K web-search trajectories, 200K simulated trajectories). The evaluation of this capability, however, relies on three benchmarks—BFCLv3, LitQA2 with ASC tools, and SimpleQA with search tools—that have significant limitations as measures of function-calling competence. BFCLv3 evaluates intrinsic function-calling accuracy (selecting the right functions and arguments) but not the quality of task completion in realistic multi-turn, multi-step scenarios. LitQA2 and SimpleQA evaluate task completion but only for two specific tool configurations (ASC for scientific literature, Serper API for search). The real-world function-calling use cases that motivated the training data investment—diverse MCP servers, multi-step research workflows, tool-use in conversational contexts—are not evaluated. Furthermore, the paper's tool-use delta analysis (Table 31) confounds function-calling capability with parametric knowledge: Olmo 3's large tool-use delta on LitQA2 (+13.8) primarily reflects low no-tool performance (24.4%) rather than high tool-assisted performance (38.2%). A model with stronger parametric knowledge of scientific literature would show a smaller delta even if its function-calling capability were identical or superior.

The consequence. A practitioner selecting a model for agentic applications—where an LLM must interact with diverse, potentially unseen tools in complex multi-step workflows—cannot determine from the paper's evaluation whether Olmo 3 Instruct's function-calling generalizes beyond the specific tools and domains tested. The heavy investment in SimFC (200K simulated trajectories with 42.6K unique functions) is designed to promote generalization, but the evaluation does not test generalization to unseen tools or to tool configurations that differ from the training distribution. The paper's finding that Qwen 3 VL 8B actually degrades with tools on LitQA2 (34.7 → 30.7) suggests that function-calling evaluation is sensitive to model-specific behaviors that may not reflect genuine tool-use capability—perhaps Qwen 3 VL 8B's strong parametric knowledge interferes with its willingness to invoke tools, or the tool-use prompt format disadvantages it relative to Olmo 3. Without a more comprehensive evaluation spanning diverse tool types, task complexities, and interaction patterns, the paper's function-calling claims remain suggestive rather than demonstrated.

What evidence exists in the paper. Table 31 provides LitQA2 and SimpleQA results with and without tools for four models. Table 25 reports BFCLv3, SimpleQA, and LitQA2 scores for the 32B Instruct. The tool-use data statistics in Table 27 show that SimFC provides 200K trajectories with 42.6K unique functions—substantial diversity that should promote generalization—but the evaluation does not test whether this diversity actually translates to improved performance on unseen tools. The paper's description of unified tool-calling formats and special token integration (Section 5.2.1) suggests careful engineering of the function-calling interface, but the evaluation does not ablate these design choices to demonstrate their necessity.

Mitigation status. The paper does not acknowledge these evaluation limitations. The authors release their MCP-based evaluation code, enabling other researchers to extend the evaluation to additional tools and scenarios—a partial mitigation through infrastructure. However, the paper's claims about function-calling capability ("Olmo 3 Instruct effectively reduces response latency and is optimized for general chat and function calling," Section 1) are stronger than the evaluation supports. A more comprehensive function-calling evaluation would include: (1) generalization to unseen tools with held-out function signatures, (2) multi-step task completion where the model must chain multiple tool calls to achieve a goal, (3) error recovery when tool calls fail or return unexpected results, and (4) comparison against models specifically designed for agentic tasks rather than general instruct models. The current evaluation establishes that Olmo 3 Instruct can use tools when they are provided and that tool use improves performance on specific knowledge-intensive tasks, but does not establish the breadth or robustness of its function-calling capability.

7. Implications and Future Directions

How This Work Changes the Landscape

Olmo 3 is not primarily a methodological breakthrough—its individual technical contributions (delta-learning DPO, GRPO-based RLVR, data mixing via swarm optimization) are incremental refinements of established techniques. Rather, its impact operates on a different axis: it changes the standard for what "open" means in language model research by demonstrating that full transparency is compatible with state-of-the-art performance. This is a socio-technical intervention, not a purely scientific one, but its consequences for how research is conducted are substantial.

The threshold for "open" has been raised. Before Olmo 3, the dominant release model for high-performance language models was weight-only: Qwen 3, DeepSeek R1, Llama 3, and others released final model weights but withheld training data, intermediate checkpoints, data processing code, and post-training recipes. Researchers working on RLVR, midtraining, or data curation had to either accept this opacity—training on top of models whose pretraining data was unknown—or work with fully-open models that were substantially weaker (OLMo 2 achieved 4.6 on AIME 2024; Olmo 3 achieves 80.6). This created a structural tension: the most scientifically interesting questions required access that the most capable models did not provide. Olmo 3 eliminates this tension by establishing a new Pareto frontier where openness and capability coexist. The 80.6 on AIME 2024 is not just a number—it is a proof of feasibility. Future model releases will now face a harder implicit standard: if a lab claims to support open research, why does their model not match Olmo 3's level of transparency? If a model is released without training data, what specific capability gap justifies that opacity?

The scientific bottleneck of undisclosed data is now experimentally demonstrable, not just arguable in principle. The paper's RL-Zero negative control experiment (Figure 27)—training with random rewards producing no performance improvement—operationalizes a previously theoretical concern. Before Shao et al. (2025b) demonstrated that spurious rewards can improve benchmark performance when training data is contaminated, the argument for data transparency was philosophical: openness is good in principle. After that demonstration, the argument became empirical: without transparency, you literally cannot tell whether your RLVR results reflect reasoning or memorization. Olmo 3 RL-Zero provides the first large-scale RLVR benchmark where this confound has been experimentally ruled out, not just asserted. This transforms the debate from "should models be open?" to "on what basis do you claim your model's improvements are real if you haven't run this control?" The paper does not force other labs to be transparent, but it forces them to acknowledge that their results are scientifically uninterpretable in a specific, well-defined way.

Midtraining is elevated from a preprocessing step to a research lever. Prior work treated midtraining as an implementation detail—a brief continued-training phase to adapt the base model before post-training. Olmo 3 treats it as a distinct research stage with its own methodology (integration tests, microanneals, domain trade-off analysis) and demonstrates that midtraining composition causally determines downstream RLVR behavior (Figure 25: models with insufficient reasoning data in midtraining never learn extended chain-of-thought during RLVR). This reframes midtraining as a controllable experimental variable: if you want to study how pretraining data affects reasoning, you can hold pretraining constant and ablate midtraining compositions. The release of full midtraining data, including the decontamination protocol and intermediate checkpoints, makes this practical for external researchers. A lab studying, for example, whether code pretraining data improves mathematical reasoning could now run controlled experiments by varying the midtraining mix on top of the released Olmo 3 Base Stage 1 checkpoint—something impossible with any prior high-performance model.

Preference tuning is redefined as a capability-learning mechanism. The finding that DPO on Qwen3 32B completions improves the model when SFT on the same data actively hurts (Table 21) challenges the standard framing of DPO as an alignment technique. If DPO can extract useful signal from data that is harmful for imitation learning, then the role of preference tuning in the post-training pipeline is not merely to adjust style and safety—it is to enable learning from data sources that cannot be used via SFT. This has immediate practical implications: labs that have access to strong but restrictively-licensed models can extract training signal by constructing contrastive pairs (strong model outputs as chosen, weak model outputs as rejected) without directly imitating the strong model. The paper demonstrates this with Qwen3 32B, but the principle generalizes to any setting where a stronger model is available for inference but not for distillation.

The infrastructure contribution lowers the barrier to RLVR research. OlmoRL's 3.3× throughput improvement (Table 23: 6.34M → 21.23M tokens/second on the same hardware) and the associated engineering innovations (continuous batching, inflight updates, active sampling) are released as open-source code. This means that labs without access to enormous compute clusters can now run RLVR experiments on reasoning-length sequences that were previously prohibitive. The paper reports that a single-domain RL-Zero run that took 14 days on 9 nodes in the OLMo 2 infrastructure takes 6 days on 5 nodes with OlmoRL—a practical difference between feasible and infeasible for many academic labs. The demonstration that Olmo 3 RL-Zero reaches competitive AIME performance in fewer GPU-hours than DAPO on Qwen 2.5 32B (Figure 38) suggests that algorithmic and infrastructure efficiency can partially substitute for model scale, which is particularly important for the open-research community.

Follow-Up Research This Work Enables

Systematic study of pretraining-midtraining-RLVR interactions via midtraining ablations. The paper establishes that midtraining data composition affects downstream RLVR efficiency (Figure 25) but provides only a single pairwise comparison. The release of the full Dolma 3 Dolmino Mix, with its curated subsets targeting math, code, QA, instruction following, and thinking traces, enables a factorial ablation study: train multiple midtraining variants on Olmo 3 Base Stage 1, each omitting or varying the proportion of one data category, then run identical RLVR on each variant. The dependent variables would include RLVR training curves (reward over steps, response length evolution), final downstream performance on AIME/MATH/GSM8K, and qualitative analysis of reasoning patterns (e.g., do models trained without meta-reasoning data exhibit fewer backtracking behaviors?). A strong follow-up would measure whether the "cognitive skills" the meta-reasoning data targets (backtracking, verification, goal management, listed in Table 43) actually manifest differentially during RLVR as a function of midtraining composition, ideally using automated classification of reasoning traces. This would move the field from correlational claims ("midtraining helps RLVR") to mechanistic understanding ("midtraining data type X enables reasoning behavior Y which improves RLVR efficiency by Z%").

Delta-learning with progressively smaller deltas to identify the minimum effective contrast. The paper demonstrates that pairing Qwen3 32B (chosen) with Qwen3 0.6B (rejected) provides useful contrastive signal, but does not explore the boundary conditions. How small can the quality delta be before DPO gains vanish? The Qwen 3 model family spans 0.6B, 1.7B, 4B, 8B, 14B, 32B, and 30B-A3B—a natural scale for constructing preference pairs with controlled capability gaps. A systematic study would train DPO variants using (32B, 14B), (32B, 8B), (32B, 4B), and (32B, 1.7B) as chosen/rejected pairs, measuring both final benchmark performance and the per-token optimization dynamics during DPO training. The hypothesis: there exists a threshold delta below which DPO provides no benefit (the contrast is too subtle for the model to learn from), and the paper's current 32B vs. 0.6B delta (48.1 points, from Table 21) may be far above this threshold, suggesting that smaller, cheaper model pairs could suffice. This would have immediate practical implications: if (32B, 8B) works as well as (32B, 0.6B), the cost of generating rejected responses drops dramatically. A negative result—finding that only very large deltas work—would suggest that delta-learning is less general than the paper implies and may require access to a very weak model in the same family, which is not always available.

Length-controlled DPO as a general strategy for improving RLVR initialization. The paper's most intriguing under-explored finding is that the length-controlled DPO checkpoint—which has lower standalone benchmark scores—serves as a better RLVR initialization than the uncorrected DPO checkpoint (Section 5.5). This is currently a qualitative observation from the development process, not a controlled experiment. A rigorous follow-up would train multiple DPO variants with different length-control thresholds (no control, 200-token limit, 100-token limit, 50-token limit, aggressive truncation of chosen responses to match rejected lengths exactly), then run identical RLVR from each. The dependent variables would include: (a) final benchmark performance after RLVR convergence, (b) training stability metrics (loss variance, reward trajectory smoothness, proportion of batches with zero advantage), (c) response length evolution during RLVR, and (d) pass@K vs. pass@1 divergence (testing whether length-controlled initialization preserves more output diversity that RLVR can later exploit). The paper's hypothesis—that "with a fixed context window, a shorter model may be 'more intelligent per token'"—is testable by measuring whether length-controlled models allocate more probability mass to correct reasoning steps (as judged by a process reward model) at matched total generation length. If validated, this would change how practitioners select DPO checkpoints: rather than maximizing DPO-stage benchmark scores, they would select for conciseness and diversity, knowing that RLVR can recover and exceed the lost benchmark performance.

Cross-architecture validation of the Dolma 3 data recipes. The paper's data curation methodology—constrained mixing, quality-aware upsampling, midtraining integration tests—is validated exclusively on the Olmo 3 architecture. The open release of the full data mixes (Dolma 3 Mix at 6T tokens for pretraining, Dolma 3 Dolmino Mix at 100B tokens for midtraining, Dolma 3 Longmino Mix at 50B/100B tokens for long-context extension) enables a direct test of architectural transferability: train a non-Olmo architecture (e.g., a Llama-style model with a different tokenizer, a Mamba hybrid, or a model with standard attention rather than sliding window) on the exact same data mixes, then compare performance on the OlmoBaseEval suite. The key question is whether the data mixtures optimized via proxy models within the Olmo 3 architecture family are genuinely "good data" in an architecture-independent sense, or whether their optimality is architecture-specific. A finding that the mixtures transfer well would establish Dolma 3 as a general-purpose pretraining corpus, not just an Olmo-specific one. A finding that they transfer poorly would reveal architecture-data interactions that are currently hidden and would motivate architecture-aware data curation.

Extended decontamination validation with longer random-reward RLVR training. The paper's negative control experiment (Figure 27) runs for only 400 steps—insufficient to rule out contamination that requires more optimization steps to elicit. A straightforward but important follow-up would extend this experiment to 2,500 steps (matching the full RL-Zero training duration shown in Figure 24) on all domain-specific RL-Zero variants (math, code, IF, general, mix). This would either strengthen the paper's central decontamination claim or reveal contamination that the shorter experiment missed. Additionally, a sensitivity analysis of the decontamination threshold (the 0.5 n-gram overlap threshold described in Section 4.2.1) would establish the trade-off between contamination removal and legitimate training data preservation: by varying the threshold from 0.3 to 0.7 and measuring both the fraction of training data removed and the resulting random-reward performance after 2,500 RLVR steps, one could determine whether the current threshold is conservative (removing more data than necessary) or liberal (risking residual contamination). This would provide practical guidance for future open-model releases and would address a limitation the paper itself documents (the iterative, error-prone nature of decontamination, including early failures on SQuAD v2 and DROP).

Practical Applications and Downstream Use Cases

RLVR research on a known-clean benchmark. The most immediate practical application of Olmo 3 RL-Zero is as a standardized testbed for reinforcement learning algorithms. Prior to this release, all leading RLVR benchmarks trained on undisclosed pretraining data, meaning that algorithmic comparisons were confounded by unknown (and potentially differential) data leakage. A researcher developing a new RLVR algorithm—say, a variant of GRPO with adaptive clipping or a novel exploration bonus—can now train on Olmo 3 RL-Zero's math domain and compare against the paper's published trajectory (Figure 24: AIME 2024 pass@1 improving from ~28 to ~48 over 2,500 steps). Because the negative control experiment validates that these improvements are not driven by contamination, any algorithm that achieves steeper improvement curves or higher asymptotic performance can be attributed to algorithmic innovation with high confidence. This is particularly valuable for the multi-domain RL-Zero mix (Figure 24, bottom-right), which provides a more challenging benchmark where algorithms must balance optimization across domains with conflicting reward structures—exactly the kind of problem that multi-objective RL research targets but that lacked a clean, contamination-free testbed.

Midtraining as a capability steering mechanism for domain-specific models. The paper's midtraining methodology—curating domain-targeted data mixes and running integration tests to measure capability trade-offs (Tables 7, 8, 9, 10)—provides a practical recipe for organizations that need models specialized to particular domains without full pretraining. A company building a model for scientific literature analysis could take the released Olmo 3 Base Stage 1 checkpoint, construct a midtraining mix heavily weighted toward olmOCR science PDFs and scientific QA data, and run a 100B-token midtraining phase following the paper's annealing and evaluation protocol. Table 7 demonstrates that domain-skewed mixtures produce predictable capability trade-offs: the math-code-thinking mix improves Math from 47.4 to 60.8 (a 13.4-point gain) but drops GenQA from 68.1 to 65.9 (a 2.2-point loss). This quantifiability allows practitioners to make informed decisions about how much specialization to trade for general capability. The release of the full midtraining data pool (2T tokens) and the 10B-token sample mixes means that even smaller organizations with limited compute can prototype midtraining recipes at manageable scale before committing to the full 100B-token run.

Function-calling model development with unified tool formats. The paper's function-calling data pipeline—combining real MCP-server interactions (Science QA with ASC, Web Search QA with Serper) with large-scale simulated interactions (SimFC: 200K trajectories, 42.6K unique functions)—provides a template for building tool-use capabilities into language models. The key design decisions—using OpenAPI specification for tool definitions, representing function calls as pythonic code blocks, encapsulating tool calls with XML tags, and adding dedicated special tokens to the tokenizer—are described in sufficient detail for replication. An organization building an internal agent for, say, database querying or API orchestration could follow the same recipe: (1) collect a small number of real-interaction trajectories using their actual tools, (2) generate a larger SimFC-style dataset with LLM-simulated environments covering diverse tool signatures, (3) train with the unified format described in Section 5.2.1, and (4) evaluate using the BFCLv3 protocol for intrinsic accuracy plus task-specific benchmarks for extrinsic task completion. The paper's finding that Olmo 3 Instruct achieves the largest tool-use delta on LitQA2 (+13.8 points, Table 31) while maintaining competitive BFCL scores (58.8, Table 25) suggests that this recipe produces models that genuinely use tools rather than relying on parametric knowledge—a critical property for deployment settings where tools access live data that cannot be memorized.

When to Prefer This Method

The paper does not frame Olmo 3 as a "method" competing against specific named alternatives in a trade-off matrix. It positions the release as an open alternative to closed-weight models, with the decision criterion being transparency needs rather than performance on a particular benchmark. However, the paper does articulate several conditions under which the fully-open approach provides distinct advantages, which can be extracted as implicit decision guidance:

Prefer the fully-open approach when you need to study interactions between training stages (e.g., how midtraining data affects RLVR efficiency), when you need to verify decontamination experimentally rather than trusting assertions, when you need to trace model outputs to specific training data for auditability or attribution, and when you operate under licensing constraints that prohibit use of models with restrictive terms (e.g., the Llama Community License requiring derivative models to include "Llama" in their name).
The open-weight approach may be sufficient when your sole objective is maximizing benchmark performance and you can trust the model provider's decontamination claims, when you do not need to modify or inspect the training pipeline, and when you can accept the model's license terms for your deployment context. The paper's own performance comparisons (Tables 14 and 15) show that Qwen 3 32B and Qwen 3 VL 32B Think still lead Olmo 3.1 Think 32B on several benchmarks (AIME 2024: 80.8 and 86.3 vs. 80.6; ZebraLogic: 88.3 and 96.1 vs. 80.1), so pure performance maximization favors the closed-weight alternatives.
For RLVR research specifically, prefer Olmo 3 RL-Zero when you need to publish results where the mechanism of improvement (reasoning vs. memorization) is central to your claims, and when you need a multi-domain benchmark that includes negative-control validation. The performance gap between Olmo 3 RL-Zero and DAPO-on-Qwen (Figure 38) is narrowing but still exists, so pure score maximization on AIME may still favor the Qwen-based approach—but at the cost of scientific interpretability.

These are not binary choices. The paper's modular release—separate checkpoints for each training stage—allows hybrid approaches: use Olmo 3 Base for pretraining transparency, then apply proprietary post-training; or use the released midtraining data with a different base architecture; or use the RL-Zero setup as a development testbed before scaling to larger, closed-weight models. The paper's contribution is not to force an either-or decision but to provide a transparent alternative that makes the trade-offs explicit.