Llama-Nemotron: Efficient Reasoning Models

ArXiv: 2505.00949

🎯 Pitch

A 253B model outperforms DeepSeek-R1 using NAS-optimized architectures that fit on a single GPU node, delivering even larger reasoning gains on scientific tasks through RL. Remarkably, these are the first open models that let users toggle deep reasoning on and off with a simple system prompt, eliminating the cost of verbose chain-of-thought when it is not needed.

---

1. Executive Summary

This paper introduces the Llama-Nemotron series, an open family of heterogeneous reasoning models spanning three sizes—Nano (8B), Super (49B), and Ultra (253B)—that delivers competitive reasoning performance against state-of-the-art models like DeepSeek-R1 while achieving superior inference throughput and memory efficiency through neural architecture search (NAS) with the Puzzle framework (producing hardware-efficient transformer variants via block-wise distillation and FFN fusion). The models are the first open-source systems to support a dynamic reasoning toggle, allowing users to switch between standard chat and reasoning modes at inference time via a lightweight system prompt—"detailed thinking on/off"—which eliminates the need for separate model deployments. The flagship LN-Ultra surpasses DeepSeek-R1 on GPQA-Diamond (76.0% vs. 71.5%) through large-scale curriculum-driven reinforcement learning with GRPO, establishing that supervised fine-tuning on teacher-generated reasoning traces can transfer strong capabilities to student models, but that RL is essential for exceeding teacher performance—particularly on scientific reasoning benchmarks where distillation alone sets an upper bound.

2. Context and Motivation

The Core Problem: Efficiency Has Become the Bottleneck for Reasoning Models

The paper addresses a fundamental tension that has emerged in the recent wave of reasoning-focused language models. Models like OpenAI o1 and DeepSeek-R1 achieve state-of-the-art performance on complex tasks—PhD-level STEM questions, competition math, and multi-step reasoning—by generating long chains of thought: extended responses containing self-verification, reflection, backtracking, and other reasoning behaviors that sometimes span thousands of tokens. The paper makes the point explicitly in the introduction:

"A defining characteristic of these models is their long responses, often containing long chains of thought, self-verification, reflection, and backtracking."

This characteristic creates a direct dependency between intelligence and inference cost. As reasoning capabilities improve, inference-time compute requirements balloon—not because the models are architecturally slower, but because they simply produce more tokens per query. The paper frames this not as a deployment inconvenience but as a fundamental bottleneck:

"As reasoning capabilities increasingly depend on scaling at inference time, it has become essential to design models that are efficient to run during inference. Inference efficiency is no longer just a deployment concern—it is now a core limiting factor for overall model intelligence and the viability of agentic pipelines."

This is a significant reframing. In the pre-reasoning-model era, efficiency was primarily about serving cost: can you deploy this model at a reasonable price per query? With reasoning models, efficiency becomes a capabilities constraint: if your model is too slow or too memory-hungry, you simply cannot deploy it in agentic pipelines where latency budgets are tight, or on hardware configurations that are economically viable. The paper's goal is to show that you can have both—strong reasoning and high throughput—through careful architecture optimization rather than simply scaling down model size.

The Heterogeneous Model Architecture Gap

The paper tackles a specific technical gap: prior reasoning models are architecturally homogeneous, meaning every transformer layer has the same structure (same attention mechanism, same FFN dimensions). DeepSeek-R1 (671B parameters, MoE architecture) is the primary reference point—it achieves exceptional reasoning performance but requires 8×H200 GPUs for inference, a hardware configuration that is both expensive and less widely available than 8×H100 nodes. The paper doesn't just argue that DeepSeek-R1 is expensive; it makes the more specific point that homogeneous architectures leave inference efficiency on the table because not all layers contribute equally to reasoning quality. Some layers may benefit from full attention and wide FFNs, while others can be compressed or have attention removed entirely with minimal quality degradation.

This connects to prior work on architecture optimization—particularly the Puzzle framework (Bercovich et al., 2024)—but the paper extends this to the reasoning model domain, where the stakes are higher because reasoning traces are long and the pressure on throughput is correspondingly greater. The specific contribution is not the NAS technique itself but its application to creating reasoning models that are deliberately heterogeneous, with the efficiency gains quantified against the strongest existing open reasoning model (DeepSeek-R1).

The Reasoning Toggle: Why "Always-On" Reasoning Is a Problem

Beyond raw efficiency, the paper identifies a second gap: existing reasoning models lack user-facing control over whether reasoning behavior is deployed. Models like o1 and DeepSeek-R1 apply deep, multi-step reasoning to every query by default. This is problematic because:

"Not all queries benefit from detailed multi-step reasoning—such responses may be unnecessarily verbose or even counterproductive in certain contexts."

If a user asks a simple factual question ("What is the capital of France?"), a reasoning model might still generate a long chain of thought analyzing whether the question is ambiguous, considering alternative interpretations, and then producing a short final answer. This wastes both compute and the user's time. More subtly, in tool-calling or agentic scenarios where the model needs to produce structured output quickly, verbose reasoning traces can interfere with downstream parsing.

The paper's innovation here is not the concept of a toggle per se—Anthropic's Claude 3.7 Sonnet already offered extended thinking as a controllable feature—but rather being the first to implement this in an open-source model with a lightweight mechanism (a simple system prompt: "detailed thinking on/off") rather than requiring separate model architectures or fine-tuning runs. This matters because it makes controllable reasoning accessible to the open-source community, enabling applications where reasoning is selectively deployed based on query difficulty or user preference.

Where Prior Work Falls Short

The paper identifies several specific limitations in the existing landscape:

Distillation-only approaches hit a ceiling. Prior work (OpenThoughts, BespokeLabs, HuggingFace Open-R1, and the DeepSeek-R1 distilled models) demonstrated that supervised fine-tuning on reasoning traces from strong teacher models like DeepSeek-R1 can transfer reasoning capabilities to smaller models. This works well—the paper's own SFT-only LN-Ultra approaches DeepSeek-R1 performance on several benchmarks—but the paper argues that distillation inherently caps student performance at the teacher's level:

"Using supervised fine-tuning, LN-Ultra can approach the performance of DeepSeek-R1 but not exceed it."

For models like LN-Ultra (253B parameters, based on Llama 3.1-405B-Instruct), the base model is arguably more capable than the teacher's base model (DeepSeek-R1 is built on DeepSeek-V3, a different architecture and training recipe). Distillation fails to exploit this latent capability. The paper's key claim is that reinforcement learning breaks through this ceiling by allowing the model to explore beyond the teacher's output distribution.

Prior heterogeneous architecture work focused on non-reasoning models. The Puzzle framework (Bercovich et al., 2024) had already shown that NAS-based architecture optimization can produce efficient LLMs for standard chat and completion tasks. But reasoning models introduce new challenges: the long, structured outputs (chains of thought) create different memory access patterns and KV-cache pressure than standard chat completions. The paper extends Puzzle to this domain and introduces FFN Fusion (Bercovich et al., 2025) as a complementary technique specifically motivated by the sequential depth reduction needed when Puzzle removes attention layers, creating consecutive FFN blocks that can be vertically compressed.

Open-source reasoning models lag in inference efficiency. Prior to this release, the strongest open-source reasoning model was DeepSeek-R1, which the paper notes requires 8×H200 GPUs. There was no strong open reasoning model optimized for the more widely available 8×H100 configuration. LN-Ultra fills this gap by achieving state-of-the-art open-model performance while fitting on a single 8×H100 node.

How This Paper Positions Itself

The paper positions Llama-Nemotron not as a single methodological contribution but as a system-level integration that combines five stages—NAS with Puzzle, knowledge distillation + continued pretraining, supervised fine-tuning on teacher reasoning traces, large-scale RL with GRPO, and a final alignment phase—into a complete training pipeline for efficient reasoning models.

Relative to DeepSeek-R1, the paper presents LN-Ultra as competitive or superior on reasoning while substantially more efficient on inference hardware. Figure 2 and Table 5 make this comparison explicit: LN-Ultra matches or beats DeepSeek-R1 on AIME24 (80.8% vs. 79.8%), MATH500 (97.0% vs. 97.3%), and GPQA-Diamond (76.0% vs. 71.5%), while running on 8×H100 instead of 8×H200. Figure 4 visualizes this as an accuracy-throughput Pareto frontier, where LN-Ultra is the dominant point—higher accuracy and higher throughput than DeepSeek-R1 in both tested inference settings.

Relative to the broader open-source ecosystem, the paper positions the release as a major contribution to open research, emphasizing that it releases not just model weights but the complete post-training dataset and training codebases (NeMo, NeMo-Aligner, Megatron-LM). This is explicitly framed as enabling other researchers to replicate and build on the work:

"This release represents one of the largest contributions to the open source community in support of developing reasoning models."

A subtle but important positioning choice: the paper uses Llama 3.1 and Llama 3.3 as base models rather than training from scratch. This is not framed as a limitation but as a deliberate strategy—it allows the work to focus on the post-training pipeline (architecture optimization, distillation, SFT, RL) while leveraging the massive pretraining investment already made in the Llama family. The continued pretraining phase (88B tokens for LN-Ultra after NAS) is presented as sufficient to recover and even exceed the base model's performance despite aggressive architecture compression (Table 1).

The Practical Stakes

The paper's motivation is grounded in concrete deployment economics. The comparison in Section 2.1 establishes that LN-Super achieves a 5× throughput speedup over Llama 3.3-70B-Instruct at batch size 256 with tensor parallelism 1, and that even when Llama 3.3-70B-Instruct is run at its optimal configuration (TP4), LN-Super at TP1 delivers a ≥2.17× throughput advantage. For LN-Ultra, the efficiency target is a 1.71× latency improvement over Llama 3.1-405B-Instruct.

These numbers translate directly to cost: an 8×H100 node running LN-Ultra can serve more queries per dollar than the same node running the larger DeepSeek-R1 on 8×H200 (which is both more expensive hardware and has lower throughput per GPU due to its size). For enterprise deployments and API providers, this changes the economics of offering reasoning-capable models at scale.

3. Technical Approach

3.1 Reader Orientation

The system being built is a family of three language models—at 8B, 49B, and 253B parameters—that can either answer questions directly or engage in extended, multi-step reasoning depending on a user-controlled toggle, all while running efficiently on standard GPU hardware. The core problem is that state-of-the-art reasoning models like DeepSeek-R1 produce very long chains of thought that make inference slow and expensive, but we don't want to sacrifice reasoning quality. The solution is a five-stage training pipeline that (1) restructures the base model's architecture for hardware efficiency via neural architecture search, (2) recovers any lost quality through distillation and continued pretraining, (3) teaches reasoning behavior by supervised fine-tuning on teacher-generated reasoning traces, (4) pushes reasoning beyond the teacher's capability through large-scale reinforcement learning from verifiable rewards, and (5) applies a final alignment phase for instruction following and helpfulness—all while training a lightweight system-prompt toggle ("detailed thinking on/off") that gives users runtime control over whether reasoning mode is engaged.

3.2 Big-Picture Architecture (Diagram in Words)

The Llama-Nemotron training pipeline has five sequential stages, with each stage building on the previous one's outputs:

1. Neural Architecture Search (NAS) via Puzzle: Takes a pretrained Llama 3 Instruct model and produces a heterogeneous architecture where some transformer layers have attention removed, variable FFN dimensions, or consecutive FFN blocks fused together. The output is a model with the same approximate behavior but lower latency and memory usage. Applied to LN-Super (from Llama 3.3-70B-Instruct) and LN-Ultra (from Llama 3.1-405B-Instruct); LN-Nano (8B) skips this stage and uses the standard Llama 3.1-8B architecture unchanged.

2. Knowledge Distillation + Continued Pretraining (CPT): Trains the heterogeneous model to recover quality lost during architecture compression, using a distillation objective on teacher outputs (for LN-Super: 40B tokens; for LN-Ultra: 65B tokens distillation + 88B tokens continued pretraining on Nemotron-H data). The output is a model that matches or exceeds the original Llama 3 model on key benchmarks despite the compressed architecture.

3. Supervised Fine-Tuning (SFT): Trains the model on a massive mixture of reasoning traces (33M+ samples, mostly from DeepSeek-R1 as teacher) and non-reasoning responses, conditioning on the system prompt "detailed thinking on" for reasoning data and "detailed thinking off" for non-reasoning data. This teaches both the reasoning behavior itself and the ability to toggle it on/off. All three model sizes undergo this stage.

4. Reinforcement Learning for Reasoning (GRPO): Applied only to LN-Ultra, this stage uses Group Relative Policy Optimization with accuracy rewards (judged by Llama-3.3-70B-Instruct comparing model outputs to ground-truth answers) and format rewards (checking for proper thinking/response tags) to push scientific reasoning beyond what the teacher DeepSeek-R1 achieves. Uses curriculum training where problem difficulty increases progressively.

5. RL for Preference Optimization: A final alignment phase using RLOO (for instruction following) and RPO/GRPO (for human preference alignment via the HelpSteer2 reward model) to ensure the models remain helpful, follow instructions, and produce conversational responses without sacrificing reasoning gains. Applied to all three model sizes with model-specific configurations.

Information flows linearly through these five stages: the base Llama model enters → Puzzle produces a compressed architecture → CPT recovers quality → SFT installs reasoning capabilities → GRPO (Ultra only) pushes beyond teacher → final alignment polishes instruction following and helpfulness. The reasoning toggle is learned entirely during SFT and refined in subsequent stages; it requires no architectural changes, only conditioning on different system prompts.

3.3 Roadmap for the Deep Dive

• First, the Puzzle NAS framework—how it builds a library of alternative transformer blocks via block-wise distillation, and how FFN Fusion further compresses sequential depth. This is the architectural foundation that enables all the efficiency claims.
• Second, the post-NAS knowledge distillation and continued pretraining—how the compressed models recover quality and the specific token budgets used. This explains how aggressive architecture compression is made viable.
• Third, the synthetic data generation pipeline—how the 33M+ sample SFT dataset is constructed across math, code, science, and general domains, with the critical reasoning/non-reasoning paired data that enables the toggle. The data is what makes the toggle work.
• Fourth, the SFT procedure itself—training configurations, learning rates, sequence packing, and the model-specific three-stage vs. single-stage approaches. This explains how reasoning behavior is installed.
• Fifth, the GRPO reasoning RL stage—the algorithm, reward structure, curriculum batching, and the massive infrastructure challenge of co-locating training and generation on the same GPUs with FP8 inference. This is where LN-Ultra breaks through the distillation ceiling.
• Sixth, the final preference optimization stages—RLOO for instruction following and RPO/GRPO for human alignment. These are brief but important for making the models usable in practice.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and training methodology paper whose core idea is that you can build open-source reasoning models that are simultaneously efficient at inference (through heterogeneous NAS-based architecture optimization) and controllable by users (through a lightweight reasoning toggle learned from paired training data), and that while supervised fine-tuning on teacher traces is sufficient for strong performance, reinforcement learning is necessary to exceed the teacher's capabilities.

---

Puzzle: Neural Architecture Search for Inference-Optimized LLMs

The Puzzle framework (Bercovich et al., 2024) transforms a homogeneous transformer model into a heterogeneous one where each layer can have a different structure—some layers keep full attention, others drop attention entirely, and FFN dimensions vary across layers. The key insight is that not all layers contribute equally to model quality: some layers may have learned functions that can be approximated more cheaply, and Puzzle identifies which layers can be compressed without proportionally degrading overall performance.

The process works in two phases, illustrated in Figure 3:

Phase 1: Building the Block Library. Starting from a pretrained Llama 3 Instruct model—Llama 3.3-70B-Instruct for LN-Super, Llama 3.1-405B-Instruct for LN-Ultra—Puzzle creates alternative versions of each transformer block. For every block (which consists of an attention sub-layer and a feed-forward network sub-layer), Puzzle trains replacement blocks independently using block-wise local distillation: each alternative block receives the same input activations as the original block and is trained to produce the same output activations via a distillation loss. This is done for each block in isolation (not end-to-end), enabling massive parallelism since each block can be trained independently.

The block variants considered include:

• Attention removal: The attention mechanism is removed entirely, turning that sub-layer into effectively a no-op or identity connection. This reduces both compute (no attention matrix multiplications) and KV-cache memory consumption (no key-value tensors to store for that layer). The paper states this was one of the two most effective operations for the LN-Super and LN-Ultra models.

• Variable FFN dimensions: The feed-forward network's intermediate hidden size is varied, enabling compression at different granularity levels. The paper specifies compression levels of "87%, 75%, 50%, down to 10% of the original hidden size." A smaller FFN hidden dimension means fewer matrix multiplications in the two linear projections that compose the FFN (the up-projection and down-projection), directly reducing FLOPs and memory usage.

• Additional operations not used: Puzzle also supports grouped-query attention (GQA) with different numbers of key-value heads, linear alternatives to attention, and no-op substitutions, but the paper states these were less effective than attention removal and FFN compression for LN-Super and LN-Ultra.

Each alternative block is scored on two dimensions: quality (how well its output matches the original block's output under the distillation loss) and inference cost (measured in terms of latency, memory usage, or throughput). This creates a library of alternative blocks per layer, each with a known quality-efficiency tradeoff profile.

Phase 2: Assembling the Heterogeneous Architecture. Given the block library, Puzzle uses a mixed-integer programming (MIP) solver to select one block per layer, assembling a complete heterogeneous model. The MIP formulation takes as input the quality and cost scores for each candidate block at each layer, plus the desired deployment constraints (such as maximum allowed latency, total memory budget, or target inference throughput). The solver then identifies the architecture that maximizes aggregate quality under those constraints.

The paper's specific deployment constraints were:

• For LN-Super: Optimized to run on a single NVIDIA H100 GPU with tensor parallelism 1 (TP1). Puzzle produces a model achieving a 5× throughput speedup over Llama 3.3-70B-Instruct at batch size 256 and TP1. As a point of comparison, even when Llama 3.3-70B-Instruct is run at its optimal configuration (TP4, which splits the model across 4 GPUs for lower per-GPU memory pressure), LN-Super at TP1 still delivers a ≥2.17× throughput advantage. The model is also optimized under a constraint of approximately 300K cached tokens (batch size × sequence length), measured at FP8 precision—for instance, this corresponds to batch size 16 and sequence length 18,750.

• For LN-Ultra: Optimized for a full H100 node (8 GPUs). During Puzzle's architecture search, the model is constrained to achieve at least a 1.5× latency reduction over Llama 3.1-405B-Instruct. LN-Ultra is optimized under cached token constraints supporting up to 3M tokens at FP8 precision and 600K tokens at BF16 precision on an H100 node.

FFN Fusion (LN-Ultra only). After Puzzle selects its heterogeneous architecture, a structural property often emerges: because some attention layers have been removed, the model can end up with consecutive FFN blocks—two or more adjacent layers where the attention sub-layer is gone and only the FFN remains. FFN Fusion (Bercovich et al., 2025) identifies such sequences and replaces them with fewer, wider FFN layers that can be executed in parallel. Specifically, instead of processing activations through a sequence of narrow FFNs (which must be done sequentially even though there's no attention between them), FFN Fusion replaces the sequence with a single wider FFN that processes all the computation at once. This reduces the number of sequential steps without compromising expressivity because the fused FFN has the same total representational capacity (wider but fewer layers). The paper notes this "significantly improves compute utilization—especially on multi-GPU setups where inter-layer communication overhead is non-negligible."

After FFN Fusion, the final LN-Ultra achieves a 1.71× latency improvement over the original Llama 3.1-405B-Instruct, exceeding the 1.5× target set during Puzzle's search.

---

Post-NAS Training: Knowledge Distillation and Continued Pretraining

After the Puzzle architecture transformation, the model is heterogeneous—different layers have different structures—and some quality degradation occurs because the block-wise distillation in Phase 1 was local (each block trained independently to match its original), not end-to-end. The blocks may not interact optimally because they weren't trained together. The post-NAS training phase addresses this by training the assembled heterogeneous model end-to-end on a large corpus.

The paper applies this phase to LN-Super and LN-Ultra only (LN-Nano uses the standard Llama 3.1-8B architecture without Puzzle compression and thus skips this stage).

LN-Super Post-NAS Training. The model is trained for 40B tokens using a knowledge distillation objective over the Distillation Mix dataset introduced by Bercovich et al. (2024). Knowledge distillation here means training the student (the post-Puzzle model) to match the output distribution (typically the token-level logits or probabilities) of the teacher (the original Llama 3.3-70B-Instruct before compression), not just to predict the correct next token. This is a stronger training signal than standard language modeling because it transfers the teacher's full distributional knowledge, including its uncertainty patterns and near-miss alternatives.

The paper does not specify the exact distillation loss function (e.g., whether it uses KL divergence between teacher and student output distributions, or a combination of distillation loss and standard cross-entropy), but this is consistent with the Puzzle framework's original description which uses a weighted combination of hard-label cross-entropy and soft-label KL divergence.

LN-Ultra Post-NAS Training (Two Sub-Phases). The training is split into two parts:

1. Knowledge distillation for 65B tokens using the same Distillation Mix dataset. This is the same objective as LN-Super—matching the teacher distribution—but applied for longer because the 405B → compressed model gap is potentially larger.

2. Continued pretraining for 88B tokens on the Nemotron-H phase 4 pretraining dataset (NVIDIA et al., 2025). This is standard autoregressive language modeling (next-token prediction) rather than distillation. The purpose is to expose the model to a broad, diverse corpus beyond the distillation dataset, allowing it to internalize knowledge that may have been lost during compression and to develop coherent representations across its heterogeneous layers. This is essentially a short pretraining run that treats the post-Puzzle model as initialization.

The total post-NAS training for LN-Ultra is 65B + 88B = 153B tokens.

Table 1 shows the results: after this phase, LN-Ultra not only recovers the quality of Llama 3.1-405B-Instruct but surpasses it on several benchmarks. Specifically, the post-CPT LN-Ultra scores 88.1 on MMLU (vs. 88.6 for 405B-Instruct—a tiny regression), 80.4 on MATH500 (vs. 69.6—a substantial improvement), 88.4 on HumanEval (vs. 86.0—an improvement), and 83.2 on RULER 128K (vs. 73.7—a large improvement). This demonstrates that the architecture compression did not permanently damage the model; the distillation + continued pretraining phase not only recovered but enhanced capabilities, particularly on math and long-context benchmarks.

---

Synthetic Data Generation for Supervised Fine-Tuning

The SFT dataset is the foundation for teaching reasoning behavior. The paper curates it across four domains—math, code, science, and general—with each domain containing both "reasoning on" responses (with thinking traces, conditioned on "detailed thinking on") and "reasoning off" responses (without thinking traces, conditioned on "detailed thinking off"). Table 2 provides the breakdown: the full dataset contains 33,011,757 samples, with math dominating at 66.8% (22.1M samples) and code at 30.6% (10.1M samples). Science (2.1%) and general chat/instruction-following/safety (collectively <0.4%) make up the remainder.

Math Data Pipeline (Section 3.1.1). The math reasoning data is constructed following the pipeline of Moshkov et al. (2025), using Qwen2.5-32B-Instruct for most orchestration steps:

1. Problem Extraction: Problems are collected from Art of Problem Solving (AoPS) community forums, excluding "Middle School Math" which was found to be too easy and unhelpful in early experiments. An LLM (Qwen2.5-32B-Instruct) is prompted to identify and extract all problems from the initial forum posts. Most posts contain a single problem, but some include multiple or none.

2. Problem Classification: Each extracted problem is classified as proof vs. non-proof, multiple choice vs. not, binary (yes/no answer) vs. not, and valid vs. invalid (problems lacking context or referring to other problems are considered invalid). All proof problems, multiple-choice questions, binary questions, and invalid problems are removed. The paper does not state how many problems survive this filtering.

3. Answer Extraction: The final answer is extracted from forum discussions without attempting to extract full solutions. Only the final answer expression is extracted to enable automatic correctness checking.

4. Benchmark Decontamination: Following Yang et al. (2023), an LLM-based comparison removes questions that closely resemble those in popular math benchmarks to prevent evaluation contamination.

5. Solution Generation: DeepSeek-R1 generates 16 "reasoning" solutions per problem (with thinking traces), and Qwen2.5-Math-7B-Instruct generates 64 "non-reasoning" solutions per problem (without thinking traces). The asymmetry in generation counts (16 vs. 64) reflects the different costs and quality characteristics of the two models—DeepSeek-R1's outputs are more expensive but higher quality, so fewer are needed.

6. Solution Filtering: Solutions that do not reach the expected answer are removed. Predicted and expected answers are compared by prompting Qwen2.5-32B-Instruct to judge their equivalence in the context of the problem. For problems where the final answer cannot be extracted, the most common answer across all available solution candidates is treated as the ground truth.

Code Data Pipeline (Section 3.1.2). The code reasoning dataset is constructed following Ahmad et al. (2025):

1. Question Collection: 28,904 unique competitive programming questions are aggregated from TACO, APPS, CodeContests, and CodeForces, after performing exact-match deduplication.

2. Contamination Check: Following Yang et al. (2023), the questions are checked against evaluation benchmarks (LiveCodeBench, CodeContests, HumanEval, MBPP) using cosine similarity checks and semantic evaluation by LLM judges (Llama-3.3-70B and Qwen2.5-32B). Manual verification confirms negligible overlap (<0.3%).

3. Solution Generation: DeepSeek-R1 generates multiple solutions per question, primarily in Python (with C++ solutions also generated for specific benchmark testing). Solutions are generated using Nucleus Sampling with temperature 0.6 and top-p 0.95 via SGLang, explicitly prompting for reasoning steps enclosed in thinking tags.

4. Post-Processing: Responses are refined by (a) verifying the presence of reasoning traces, (b) extracting solution code segments demarcated by python tags, (c) removing samples with code inside reasoning tags, and (d) validating syntax using Tree Sitter. This yields approximately 488K Python samples.

The paper includes an important empirical observation on code data scaling: an ablation study scaling the dataset from 25K to 736K samples showed continuous improvement without plateau, suggesting that larger and more diverse code datasets would yield further gains. The most significant performance boosts came from focusing generation on harder problems from CodeContests before expanding to the full question set.

Science Data Pipeline (Section 3.1.3). Science data comes from two sources: extracted StackOverflow question-answer pairs and synthetically generated multiple-choice questions:

1. Synthetic Question Generation: Nemotron-4-340B-Instruct defines a broad set of academic topics (physics, biology, chemistry, etc.) and their subtopics, with multiple difficulty levels specified. Qwen2.5 models generate MCQs conditioned on topic, subtopic, and difficulty. Each question is verified for format compliance. Following the OpenMathInstruct-2 pipeline, Qwen2.5 generates variations of the original questions for augmentation.

2. Benchmark Decontamination: The entire question set (both real and synthetic) is decontaminated against test sets of GPQA, MMLU, and MMLU-Pro using the Yang et al. (2023) approach.

3. Solution Generation: DeepSeek-R1 generates multiple reasoning traces for all questions. For questions without ground-truth answers, majority voting across generated solutions infers the most likely correct answer.

The science dataset contains only reasoning-on data (708,920 samples with "reasoning on"; 0 samples with "reasoning off"), as shown in Table 2. The paper doesn't explain this asymmetry, but it likely reflects that science questions in the training set were all complex enough to benefit from reasoning, or that non-reasoning science responses weren't generated due to resource prioritization.

General Data Pipeline (Section 3.1.4). General domain data follows the generation pipeline from Nemotron-4 (NVIDIA, 2024c): synthetic prompts covering open QA, closed QA, extraction, brainstorming, and real-world user prompts from publicly available permissive datasets. DeepSeek-R1 generates multiple responses, and rejection sampling using the Llama-3.1-Nemotron-70B reward model filters for high-quality outputs.

Reasoning Toggle Training Data (Section 3.2). To train the model to respond to the reasoning toggle, the paper constructs paired data: each prompt has both a reasoning response (tagged with "detailed thinking on") and a non-reasoning response (tagged with "detailed thinking off"). Specifically, prompts from the reasoning dataset (Section 3.1) are randomly sampled, and corresponding non-reasoning responses are generated using Llama-3.1-Nemotron-70B-Instruct for general domain prompts and Llama-3.3-70B-Instruct for others. This pairing is critical: it teaches the model that the same prompt should receive a verbose, step-by-step response when reasoning is on and a direct, concise response when reasoning is off. Responses are filtered according to ground truth answers or reward models.

Additionally, the paper leverages public permissive datasets on function calling and safety, augmenting them to train the model in these areas.

General-Domain Feedback-Edit System (Section 3.2.1). For high-quality general-domain open-ended responses, a novel three-stage inference-time scaling pipeline is employed:

1. Start with 20K first-turn prompts from ShareGPT and WildChat-1M.
2. Llama-3.1-Nemotron-70B-Instruct generates multiple initial responses for each prompt.
3. A dedicated Feedback model identifies areas for improvement in each response.
4. A dedicated Edit model makes targeted edits based on the feedback.
5. A dedicated Select model chooses the best edited response.

The paper does not provide architecture or training details for the Feedback, Edit, and Select models beyond referencing Wang et al. (2025b). The output is 20K first-turn prompts with corresponding high-quality responses, forming a small but carefully curated portion of the chat dataset.

---

Supervised Fine-Tuning (SFT)

The SFT stage is where reasoning capabilities from teacher models (primarily DeepSeek-R1) are distilled into the Llama-Nemotron models. All models are trained using token-level cross-entropy loss over the instruction-tuning data, with batches mixing reasoning and non-reasoning data conditioned on their respective system instructions ("detailed thinking on" or "detailed thinking off").

The paper identifies several important training dynamics specific to reasoning SFT:

• Higher learning rates are needed for reasoning traces: The paper notes that "models require higher learning rates to effectively learn from long reasoning traces, especially due to sequence-length-dependent token loss averaging." Long reasoning traces contain many more tokens than short non-reasoning responses. If the loss is averaged over all tokens (standard practice), then each example contributes loss proportional to its length, effectively up-weighting long reasoning traces. Conversely, if loss is averaged per-example and then summed, long traces are down-weighted. The paper's observation suggests that the token-level averaging approach (which over-weights long traces by default) still benefits from higher learning rates, likely because the reasoning patterns are more complex and require stronger gradient signals to learn.

• Extended training over multiple epochs improves performance, especially for smaller models: This trend is attributed to prior work (Wen et al., 2025). Smaller models have less capacity and thus need more passes over the data to fully absorb reasoning patterns.

• Adam optimizer is used for all models.

• Cosine learning rate decay with linear warmup to around 10% of total steps helps with training stability, which the paper states "was crucial for LN-Ultra."

LN-Nano SFT (Three-Stage Pipeline). LN-Nano undergoes a more complex SFT procedure than the larger models, using a global batch size of 256 with sequence packing to an effective sequence length of 32K tokens:

• Stage 1: Reasoning-only fine-tuning. The model is fine-tuned exclusively on reasoning data from code, math, and science domains (Section 3.1) with a learning rate of 1e-4 for four epochs. The paper states this prevents "failure modes such as repetitive completions"—without this focused reasoning-first phase, the small model may collapse into generating the same reasoning pattern repeatedly rather than learning diverse strategies.

• Stage 2: Mixed reasoning + non-reasoning. Non-reasoning data (Section 3.2) is introduced alongside reasoning samples, allowing the model to learn the reasoning toggle—when to engage reasoning mode and when to produce direct answers.

• Stage 3: Specialization blend. A smaller blend focused on chat, instruction-following, and tool-calling is used for final refinement.

LN-Super SFT. LN-Super is trained on the full SFT dataset for a single epoch using a fixed learning rate of 5e-6, sequence length of 16K, and global batch size of 256. The paper notes that smaller-scale experiments suggested performance improves up to 3–4 epochs with larger learning rates (5e-5), but training was constrained by computational and time limits—an honest admission of resource constraints affecting the training recipe. The paper also notes that "recent works (Wen et al., 2025) show that rejection fine-tuning can further improve performance; however, it does not yield gains in our experiments and is therefore omitted."

LN-Ultra SFT. LN-Ultra is trained on the full dataset using sequence packing with an effective sequence length of 24K and a global batch size of 256. The paper states this is "an essential strategy when fine-tuning large models with long-context reasoning data" because sequence packing maximizes token throughput—without it, many sequences would be padded to 24K, wasting compute on padding tokens.

The learning rate schedule uses a linear warmup to 1e-5, followed by cosine decay to 1e-6 with a warmup ratio of 10% (meaning warmup occupies 10% of total training steps before decay begins). This is more conservative than the higher learning rates (5e-5) that initial ablation runs suggested would improve outcomes, because "consistently high learning rates caused training instability, including gradient explosions."

Despite these measures, training encountered gradient explosions and numerical instability after the first epoch, requiring "training resumption with reinitialized optimizer states, after which successful convergence was achieved." This is a notable practical detail: even with careful learning rate scheduling, large-scale reasoning SFT on a 253B model is numerically challenging. Reinitializing optimizer states means discarding the Adam momentum and variance estimates (which capture gradient history) and starting fresh from the current model weights—a recovery technique that sacrifices some optimization progress to escape numerical instability.

The SFT-RL Tradeoff. The paper reports an important empirical finding about the interaction between SFT and subsequent RL: "there is a trade-off between the extent of SFT training and the success likelihood of subsequent RL. Although we had access to SFT checkpoints with higher benchmark scores, we initialized RL from an earlier checkpoint to improve final RL outcomes." This suggests that over-optimizing on the SFT objective (which rewards matching the teacher's output distribution) can make the model less amenable to RL-based improvement—perhaps because it becomes too anchored to the teacher's specific reasoning patterns and loses the exploration flexibility that RL requires. This is a subtle but practically important insight for multi-stage training pipelines: the best intermediate checkpoint for downstream RL is not necessarily the one with the best intermediate benchmark scores.

---

Large-Scale Reinforcement Learning for Reasoning (LN-Ultra Only)

The RL stage is applied exclusively to LN-Ultra and represents the most computationally intensive phase of training, consuming approximately 140K H100 hours. The goal is to push scientific reasoning beyond what distillation from DeepSeek-R1 can achieve, and the results (Figure 5, Table 5) validate this: LN-Ultra-SFT reaches 66.4% on GPQA-Diamond, while LN-Ultra after RL reaches 76.0%, a 9.6 percentage point improvement that moves the model from matching DeepSeek-R1 (71.5%) to substantially exceeding it.

Algorithm: Group Relative Policy Optimization (GRPO). GRPO (Shao et al., 2024) is a variant of policy gradient methods designed for reasoning tasks. Unlike standard PPO, which requires learning a separate value function (critic) to estimate advantages, GRPO uses the relative performance within a group of responses to the same prompt as the advantage signal. For each prompt, the model generates multiple candidate responses (the "group"), and each response's advantage is computed based on its reward relative to the mean reward within that group. This eliminates the need for a separate critic model, reducing memory and compute requirements—critical for a 253B model where adding a critic of similar size would roughly double the GPU memory footprint.

The paper's specific GRPO configuration: a rollout prompt size of 72 and 16 responses sampled per prompt, with temperature = 1 and top-p = 1 (meaning the model samples from its full distribution without truncation, maximizing exploration). During training, the global batch size is 576 with 2 gradient updates per rollout (meaning the model generates responses, computes advantages and updates, then generates again—standard on-policy RL). Training continues until convergence on reasoning tasks.

Reward Structure. Two types of rewards are used:

1. Accuracy rewards: For each training example, a ground truth answer is provided. Llama-3.3-70B-Instruct serves as a judge model, comparing the policy model's predicted answer to the ground truth. This is a binary or scalar judgment of whether the model's answer matches the correct one.

2. Format rewards: Following DeepSeek-R1's approach, a format reward ensures the model structures its output correctly: putting the thinking process between thinking and response tags when "detailed thinking on" is active, and ensuring the non-existence of thinking tags when "detailed thinking off" is active. This is a sparse binary reward: correctly formatted output gets a positive reward; incorrectly formatted gets zero or negative.

The format reward is critical for maintaining the reasoning toggle—without it, the model might learn to always output thinking traces (since those are associated with higher accuracy on reasoning problems) or never output them (since they're not needed for non-reasoning problems). The format reward explicitly incentivizes complying with the system prompt.

Curriculum Training with Progressive Difficulty Batching. The paper introduces a sophisticated curriculum strategy to stabilize training and improve final accuracy. The core idea: rather than randomly sampling training prompts at each step, the system deliberately controls the difficulty distribution so that the model first learns from easier problems and gradually progresses to harder ones.

The implementation uses a progressive batching strategy:

1. For each question in the training set, LN-Super generates 8 responses independently and the pass rate (fraction correct) is calculated. This pass rate serves as a difficulty metric—high pass rate means easy, low pass rate means hard.

2. Prompts with a pass rate of 0.75 or higher are discarded from the RL training set entirely. The paper states this is "to ensure that the model is adequately challenged"—easy problems where the model already succeeds 75%+ of the time provide little learning signal because the accuracy reward is always high and the advantage signal (difference from the group mean) is small.

3. For the remaining prompts, each training batch is constructed to follow a specific difficulty distribution, modeled as a Gaussian function centered on a difficulty level that progresses from high pass rates (easier) to low pass rates (harder) as training proceeds.

The precise mechanism: "Given a fixed batch size, the core of our approach involves dynamically calculating a target distribution of pass rates for each sequential batch. This distribution is modeled using a Gaussian function centered on a difficulty level that progresses from high pass rates (easier examples) for initial batches to low pass rates (harder examples) for later batches. Samples are allocated to each batch primarily based on this target distribution, considering the available count for each pass rate, with any remaining batch capacity filled by prioritizing pass rates with the largest remaining sample pools. This ensures a controlled, gradual increase in average sample difficulty across batches, while samples inside a batch are randomly shuffled."

In operational terms: at training step 1, the Gaussian is centered on a high pass rate (e.g., 0.65), so the batch mostly contains easier problems. By step 500, the Gaussian center has shifted to a low pass rate (e.g., 0.15), so the batch mostly contains hard problems. The transition is gradual and continuous via the Gaussian window, not a hard step function.

Figure 6 demonstrates the effectiveness: the curriculum (blue curve) achieves higher GPQA-Diamond accuracy and more stable training than random batching (orange curve). The curriculum curve shows a smoother, more monotonic improvement, while the random batching curve is noisier and ultimately reaches lower accuracy. This aligns with the intuition that early RL training on hard problems can cause the policy to collapse or overfit to spurious patterns, while starting easy and ramping difficulty allows the model to build robust reasoning strategies incrementally.

Infrastructure: Co-located Training and Generation on 72 H100 Nodes. The RL training infrastructure is one of the paper's most technically detailed sections, addressing the fundamental challenge of on-policy RL for large models: the model must both generate responses (inference) and update its parameters (training), and these two phases have very different computational characteristics.

The system uses 72 nodes of 8×H100 GPUs (576 GPUs total), split into two co-located phases:

• Training parallelism: Tensor parallel = 8 with sequence parallel, context parallel = 2, pipeline parallel = 18, data parallel = 2. This means the 253B model is distributed across the 8 GPUs within each node (tensor parallel), sequences are split across 2 GPUs for memory efficiency (context parallel), the model layers are divided into 18 pipeline stages (pipeline parallel), and 2 independent data replicas process different prompts simultaneously (data parallel).

• Generation (inference) parallelism: Tensor parallel = 8, data parallel = 72. During generation, the model runs in pure inference mode with all 72 nodes operating independently as data-parallel workers, each generating responses for different prompts.

The training and generation stages are co-located on the same GPUs, meaning a single GPU alternates between generating responses (running forward passes) and updating parameters (running forward+backward passes). This co-location is memory-intensive because both the training model weights and the inference model weights must be held in GPU memory, along with KV caches for generation and optimizer states for training.

The weight synchronization procedure at each step:

1. Training weights are all-gathered over the training pipeline parallel dimension (since pipeline parallel splits layers across GPUs, this step collects the full model weights).
2. The weights are converted into vLLM's format (the inference engine's weight layout) and written into shared memory (/dev/shm).
3. All training memory is released or offloaded to CPU host memory.
4. vLLM is woken from sleep mode, loads the newly saved weights from shared memory, and begins generating responses.
5. After generation completes, vLLM GPU memory is released using sleep mode=2 (deep sleep, freeing GPU memory).
6. All training memory is reloaded back onto the GPU for the next training step.

This weight handoff between training and inference engines is the critical path for throughput: any inefficiency creates idle GPUs.

Memory Profiling and Optimization. The paper describes a systematic memory optimization process enabled by three custom profiling tools: a PyTorch GPU memory profiler, a psutil CPU memory profiler, and a /dev/shm utilization monitor. These tools helped identify and resolve three memory bottlenecks:

1. Weight preparation memory: When all-gathering training weights across pipeline parallel stages, "extremely big tensors" emerge due to the heterogeneous architecture—one tensor has 13B elements, occupying 26 GB GPU memory in BF16. The solution involves releasing unused GPU memory periodically and moving some tensor conversion operations to CPU to control peak GPU memory usage.

2. vLLM GPU memory: With tensor parallel = 8, each GPU holds approximately 62 GB from model weights in BF16 (500 GB total ÷ 8 GPUs). Combined with KV cache, activations, and training memory, the GPU memory budget is extremely tight. The initial solution required disabling vLLM's cudagraph feature (which pre-compiles CUDA graphs for faster execution but consumes additional memory). However, once FP8 inference generation was enabled (reducing weight memory and activation memory), the memory budget loosened enough to re-enable cudagraph, providing additional throughput.

3. Trainer memory: Tensor parallelism = 8 naturally partitions the model across the 8 GPUs in a node. The heterogeneous architecture requires inserting identity layers to balance pipeline stages (since different layers have different computational costs, some pipeline stages would be bottlenecked unless balanced with placeholder layers). The tradeoff: more pipeline stages (e.g., 18) reduce per-GPU memory but increase communication overhead. Fewer stages reduce communication but risk GPU OOM. The final choice of pipeline parallel = 18 represents the sweet spot.

Context parallel = 2 and sequence parallel reduce activation memory consumption. Activations (the intermediate outputs of each layer needed for backpropagation) are the dominant memory consumer during training, and these parallelism strategies distribute them across GPUs.

The final configuration achieves >90% GPU utilization while avoiding OOM errors on any node.

FP8 Inference Generation. The paper identifies the generation stage as "the dominant component of the step time"—generating responses takes longer than computing gradient updates. To accelerate this, the authors implement FP8 online generation in vLLM, executing all General Matrix Multiplications (GEMMs) in 8-bit floating point using per-token activation scaling factors and per-tensor weight scaling factors.

The implementation requires custom vLLM weight loaders that:

1. Accept BF16 weights from the training stage.
2. Cast them to FP8 weights and compute scaling factors at runtime (during the sleep-to-wake transition).
3. Use meta-weight tensor initialization to avoid materializing the full BF16 inference engine in GPU memory (which would cause OOM), since vLLM doesn't natively support initializing models directly in FP8.

The results: 1.8× generation speedup over BF16 inference, with a peak throughput of 32 tokens/s/GPU/prompt. The speedup decomposes as:

• 1.4× from FP8 computation alone (reduced memory bandwidth and compute requirements).
• 0.4× from reduced memory usage, which enables re-enabling vLLM's cudagraph feature (pre-compiled execution graphs that eliminate kernel launch overhead).

The paper notes this is "to our knowledge the highest decoding throughput observed in reasoning training at this scale."

---

Final Alignment: RL for Preference Optimization

After the reasoning RL stage (for LN-Ultra) or directly after SFT (for LN-Nano and LN-Super), the models undergo a final alignment phase to ensure they are helpful, follow instructions, and produce high-quality conversational outputs—not just strong reasoning.

Instruction Following RL (Section 6.1). For LN-Super and LN-Ultra, a short RL run optimizes instruction-following capabilities using the RLOO algorithm (REINFORCE Leave-One-Out; Ahmadian et al., 2024). RLOO is a policy gradient variant that, like GRPO, avoids learning a separate critic by using the average reward of other rollouts as a baseline.

The setup: synthetic instruction-following prompts are generated (following Zhou et al., 2023) that contain from one to ten detailed instructions. A verifier checks whether the model's response complies with all specified instructions, producing a reward signal. Training runs for less than 120 steps with a batch size of 128 prompts.

The paper notes an important tradeoff discovered during this phase: "optimizing for instruction following (as measured by IFEval) can compromise conversationality (as measured by Arena-Hard), and conversely, prioritizing conversationality may detract from instruction following performance." To address this, LN-Super uses model merging—combining weights from different checkpoints along the Pareto frontier that balances both objectives. This approach was not adopted for other models "due to mixed outcomes."

RLHF for Helpfulness (Section 6.2). A separate RLHF phase improves general helpfulness and chat quality using different algorithms per model:

• LN-Super: Uses iterative online RPO (Reward-aware Preference Optimization; Sun et al., 2025). RPO maximizes the reward predicted by Llama-3.1-Nemotron-70B-Reward over prompts from HelpSteer2 (Wang et al., 2025a). The specific configuration: learning rate α = 4e-7, KL penalty β = 1e-5 (constraining the policy to not deviate too far from its pre-RLHF distribution), reward scale η = 3.0 (controlling how aggressively the model optimizes for reward vs. staying close to the original policy), batch size of 64, training for 500 steps per iteration, with two iterations total. This increases LN-Super's Arena Hard score from 69.1 to 88.1—a massive 19-point improvement—while also improving performance on all other benchmarks except IFEval. The paper speculates that RLHF helps the model "better utilize its existing knowledge and skills" even though the training data and reward model are not optimized for math, coding, or science.

• LN-Ultra: Uses GRPO instead of RPO, with 8 responses sampled per prompt. Training runs for 30 steps with learning rate = 3e-7, batch size = 288, and KL penalty β = 1e-3.

• LN-Nano: Uses two rounds of offline RPO with on-policy data. The first round mixes reasoning and non-reasoning data with appropriate system prompts to improve reasoning control, and the second round targets instruction following improvements. Each round trains up to 400 steps with α = 7e-7, β = 3e-2, and batch size = 512.

The final results (Tables 3, 4, 5) show that this alignment phase successfully produces models that are strong on both reasoning benchmarks (AIME, GPQA, MATH500, LiveCodeBench) and non-reasoning metrics (IFEval, Arena Hard, BFCL V2 Live), with the reasoning toggle allowing users to choose which mode to engage.

---

The Reasoning Toggle Mechanism

The reasoning toggle deserves special attention as a novel capability: it enables a single model to switch between detailed reasoning and direct answering based on a system prompt, without requiring separate model instances or fine-tuning variants.

The mechanism is learned entirely during SFT (and refined in subsequent RL stages) through the paired training data described in Section 3.2. During training, the model sees the same prompt twice: once with the system instruction "detailed thinking on" followed by a reasoning response (containing thinking traces, step-by-step analysis, self-verification, and a final answer), and once with "detailed thinking off" followed by a direct response (containing only the final answer or a concise explanation without explicit reasoning steps).

At inference time, the user prepends either system prompt to their query. The model has learned to associate "detailed thinking on" with the generation behavior of producing extended reasoning traces—likely thousands of tokens of analysis—and "detailed thinking off" with producing short, direct answers.

The toggle is evaluated across all benchmarks in Tables 3, 4, and 5, which report separate columns for "on | off" performance. For reasoning benchmarks (GPQA, AIME, MATH500), the "on" mode consistently and substantially outperforms "off" mode—for example, LN-Ultra scores 76.0 on GPQA with reasoning on vs. 56.6 with reasoning off. For non-reasoning benchmarks (IFEval, BFCL, Arena Hard), the "off" mode performs at least as well, confirming that the model isn't generating unnecessary reasoning traces for tasks that don't benefit from them.

This design eliminates the need for routing logic (deciding which queries need reasoning), separate model deployments (one reasoning model, one chat model), or architectural modifications. The reasoning behavior is controlled entirely through the prompt, making it usable by any application that can modify the system prompt.

4. Key Insights and Innovations

Innovation 1: Heterogeneous Architecture as a First-Class Design Axis for Reasoning Models

The field's dominant assumption—one that goes largely unstated because it's so ingrained—is that reasoning models inherit their architecture from their pretrained base models and that architecture optimization is "done" at pretraining time. DeepSeek-R1, o1, and the various open-source R1-distilled models all use homogeneous transformer architectures (every layer identical) because that's what their base models used. The implicit belief is that any architectural modification would break the carefully learned reasoning representations, so you're stuck with whatever architecture the base model provides.

This paper challenges that assumption directly and makes a stronger claim: architecture optimization is not just compatible with reasoning—it should be treated as a first-class design stage in the reasoning model pipeline. The Puzzle framework's application to reasoning models is not just an incremental efficiency tweak; it's a conceptual reframing that says you can aggressively restructure the model after pretraining, compress and remove attention from specific layers, and then recover—or even improve—reasoning quality through a relatively short distillation + continued pretraining phase.

What makes this intellectually distinctive is the evidence that the compressed architecture doesn't just match the original—it surpasses it on several benchmarks after continued pretraining (Table 1: LN-Ultra-CPT achieves 80.4 on MATH500 vs. 69.6 for the 405B base, and 83.2 vs. 73.7 on RULER 128K). This implies something non-obvious: the original Llama 3.1-405B-Instruct architecture was over-parameterized for certain capabilities, and the Puzzle process, combined with targeted continued training, acts as a form of architectural regularization that helps the model focus its capacity more effectively. This is a fundamental finding about the relationship between architecture and reasoning quality, not just an efficiency win.

The comparison to DeepSeek-R1 makes the significance concrete: prior state-of-the-art required 8×H200 GPUs (a rare and expensive configuration); LN-Ultra achieves superior performance on 8×H100 (widely available). The efficiency gain (1.71× latency improvement, 1.8× generation throughput from FP8, and the accuracy-throughput Pareto dominance in Figure 4) does not come from simply building a smaller model—it comes from deliberate architecture heterogeneity that prior reasoning models never attempted. This changes the design space for future reasoning models: instead of asking "how do we train the best reasoning model given a fixed architecture?", the question becomes "what architecture should our reasoning model have, and how do we co-design it with reasoning training?"

Innovation 2: The Distillation Ceiling and RL as a Necessity, Not an Enhancement

A plausible reading of the recent reasoning model literature—especially the success of DeepSeek-R1-Distilled models (where smaller models fine-tuned on R1's outputs achieve strong performance)—is that distillation is sufficient for transferring reasoning capabilities. The Open R1 project, Bespoke-Stratos, and other open efforts have implicitly reinforced this view: if you have a strong teacher, you can distill reasoning into smaller models and get most of the benefit without RL's complexity.

This paper provides the clearest evidence to date that this view is correct for matching teacher performance but insufficient for exceeding it, and in doing so establishes a diagnostic boundary: the point at which RL becomes necessary is when your student model's base capability exceeds the teacher's. LN-Ultra (253B parameters, based on Llama 3.1-405B-Instruct) represents a regime where the student's pretrained knowledge and capacity arguably exceed DeepSeek-R1's, but distillation alone caps it at the teacher's level. The RL stage on GPQA-Diamond is the cleanest demonstration: SFT-only LN-Ultra scores 66.4% (below DeepSeek-R1's 71.5%); after GRPO, it reaches 76.0% (well above). This is not just a performance increment—it's a qualitative shift enabled by letting the model explore solution strategies the teacher never demonstrated.

The paper's finding about the SFT-RL tradeoff—that the best SFT checkpoint for downstream RL is not the highest-scoring one—adds a conceptual nuance that prior work has not articulated. This implies that SFT and RL are not simply additive ("do SFT, then do RL on top"); they interact antagonistically in ways that require careful orchestration. Over-optimizing on teacher-matching during SFT may narrow the model's output distribution so much that RL has little room to explore and discover superior strategies. This reframes the post-training pipeline from a simple sequence of stages to a joint optimization problem where early stages must leave headroom for later ones. It's a diagnostic insight that will matter for any team building multi-stage reasoning pipelines.

The curriculum training finding (Figure 6) reinforces this: starting RL with easy problems and progressively increasing difficulty stabilizes training and improves final accuracy. This is consistent with curriculum learning literature broadly, but its application to reasoning RL—where the "difficulty" is measured by the student model's own pass rate rather than an external metric—ties directly to the distillation ceiling concept. The easy problems at the start of curriculum training are precisely those where the model already has reasonable performance post-SFT; the hard problems at the end are those where the teacher's strategies are insufficient and the model must discover new ones. The curriculum thus maps onto the progression from distillation-reachable capabilities to RL-required capabilities.

Innovation 3: The Reasoning Toggle as a Learned Behavior, Not an Architectural Feature

Controllable reasoning is not a new concept—Anthropic's Claude 3.7 Sonnet offered extended thinking as a user-facing feature, and DeepSeek-R1's architecture supports non-reasoning outputs by simply not including thinking tags. But prior implementations either relied on proprietary systems or required the user to manage the thinking tags manually (in DeepSeek-R1's case, the model can produce both modes but the toggling mechanism is not trained as a core capability).

What distinguishes this paper's approach is the training methodology for the toggle itself: by constructing paired training data where identical prompts receive both reasoning and non-reasoning responses conditioned on a simple system prompt, the model internalizes the toggle not as a post-hoc filter or routing decision but as a fundamental behavioral switch learned during SFT and reinforced during RL (via format rewards that penalize incorrect formatting for the active mode). The toggle is not an inference-time trick—it's baked into the model's understanding of how to respond.

This has a subtle but important implication for the deployability of reasoning models. The reasoning toggle eliminates the need for a separate routing model that decides which queries need reasoning (a fragile and error-prone component), because the user—or the application developer—makes that decision explicitly. It also means a single model checkpoint serves both use cases without quality degradation: Table 5 shows LN-Ultra's IFEval score is 88.9 in reasoning-on and 89.5 in reasoning-off, essentially identical, while GPQA jumps from 56.6 (off) to 76.0 (on). The model hasn't sacrificed non-reasoning quality to gain reasoning capability, and it switches between modes cleanly.

The significance extends beyond convenience. In production deployments, being able to use the same model for both fast, cheap responses (reasoning off) and deep, expensive reasoning (reasoning on) means the cost model shifts from "always pay for reasoning" to "pay for reasoning only when needed." This is not just a feature—it's an economic argument for reasoning models that was previously only available through proprietary systems. By open-sourcing this capability, the paper makes controllable reasoning a standard expectation for the next generation of open models.

Innovation 4: Co-located Training and Inference as an Infrastructure Design Point

The paper's Section 5.2 contains what is, to my knowledge, the most detailed public description of the infrastructure challenges in large-scale GRPO training for a ~250B parameter model. This may seem like an infrastructure footnote rather than a research contribution, but I think it qualifies as an innovation for a specific reason: the FP8 online generation + BF16 training co-location design represents a new infrastructure design point that was not obviously viable before this work.

The challenge is specific to on-policy RL for LLMs: you must generate responses (pure inference) and then train on those responses (forward + backward pass), and these phases have radically different memory and compute profiles. The dominant prior approach—used in most RLHF implementations—is to separate generation and training across different GPU sets, which avoids the memory pressure of co-location but wastes GPUs during the generation phase (training GPUs sit idle) and during the training phase (generation GPUs sit idle). Co-locating them on the same GPUs eliminates idle time but creates severe memory pressure.

What makes this an innovation rather than just engineering is the FP8 inference generation trick (Section 5.2.3). By implementing FP8 GEMMs with per-token activation scaling, the authors achieve a 1.8× generation speedup, but the crucial secondary effect is memory reduction: FP8 weights and activations consume less GPU memory, which frees enough budget to re-enable vLLM's cudagraph feature—which provides a further 0.4× speedup. This creates a virtuous cycle: FP8 reduces memory → enables cudagraph → reduces kernel launch overhead → further improves throughput. The 32 tokens/s/GPU/prompt peak throughput is, as the paper notes, likely the highest observed in reasoning RL training at this scale.

The significance is that this infrastructure design point is replicable by other teams. The paper describes the memory profiling methodology (three custom profilers for GPU, CPU, and /dev/shm), the specific bottlenecks encountered (13B-element tensors during weight all-gather, vLLM memory pressure, activation memory), and the solutions (periodic GPU memory release, CPU offloading of tensor conversions, context parallel = 2). This level of detail transforms the infrastructure from a one-off achievement into a recipe that other large-scale training efforts can follow. For the open-source community specifically, where infrastructure innovation often lags behind model innovation, this is a contribution that lowers the barrier to entry for GRPO training at scale.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on several standard benchmarks. For reasoning: GPQA-Diamond (Rein et al., 2024) for graduate-level science; AIME 2024 (AIME24) and AIME 2025 (AIME25) for competition math; MATH500 (Lightman et al., 2023) for general math; and LiveCodeBench (Jain et al., 2024) for coding, with results reported on two date ranges (2408–2502 and 2410–2502) to enable fair comparison with prior work. For non-reasoning: IFEval (strict-instruction) (Zhou et al., 2023) for instruction following, BFCL V2 Live (Yan et al., 2024) for function calling, and Arena-Hard (Li et al., 2024) for conversational alignment. AIME25 is split into two 15-problem halves (AIME25-I and AIME25-II); LN-Nano is evaluated on AIME25-I only, while LN-Super and LN-Ultra use the full 30-question set. The paper notes that AIME25 was released recently and is "less likely to overlap with training data," making it a stronger test of generalization.

• Base model(s). The Llama-Nemotron family spans three sizes: LN-Nano (8B), LN-Super (49B), and LN-Ultra (253B), derived from Llama 3.1-8B-Instruct, Llama 3.3-70B-Instruct, and Llama 3.1-405B-Instruct, respectively. LN-Super and LN-Ultra undergo Puzzle-based architecture optimization and continued pretraining; LN-Nano uses the standard architecture without NAS. The choice of sizes spans from a compact deployment-friendly model (8B) to a frontier-scale model (253B), enabling the paper to demonstrate the training methodology's effectiveness across scales.

• Metrics. All reasoning benchmarks report average pass@1 accuracy (%) — the fraction of problems for which the model's first sampled answer is correct, averaged across completions. For each prompt, up to 16 completions are generated. Non-reasoning benchmarks use their standard metrics: IFEval reports strict-instruction accuracy, BFCL V2 Live reports function-calling accuracy, and Arena-Hard reports a score reflecting human preference alignment. All evaluations are conducted at 32K context length, even though training used shorter contexts (16K for LN-Super, 24K for LN-Ultra), because "shorter limits can truncate long reasoning traces and lead to incomplete generations." The paper notes that results on reasoning benchmarks with small test sets (e.g., AIME with 30 questions) "can exhibit high variance due to small dataset size and generation randomness," and "reported numbers may vary across repeated runs or sampling strategies."

• Baselines. The primary baselines are: DeepSeek-R1 (671B, the strongest open reasoning model at release time), DeepSeek-R1-Distilled variants (Llama-8B, Qwen-7B, Llama-70B), and the base Llama 3 Instruct models (8B, 70B, 405B) from which Llama-Nemotron models are derived. Additional comparisons include QwQ-32B for the 49B weight class, and Llama-4 Maverick and Llama-4 Behemoth for the 253B class (the latter being shown in Figure 2 only, not Table 5). For the JudgeBench evaluation (Table 6), proprietary models (o1-preview, o1-mini, o3-mini at three compute levels) serve as baselines. The intermediate SFT checkpoints (LN-Nano-SFT, LN-Super-SFT, LN-Ultra-SFT) are also reported to isolate the effect of RL.

• Generation budget / compute accounting. The paper does not report generation budgets in a standardized unit (e.g., number of tokens or FLOPs) for benchmark evaluations. Instead, it specifies the evaluation protocol: up to 16 completions per prompt, with temperature = 0.6 and top-p = 0.95 for reasoning-on evaluations, and temperature = 0 (greedy decoding) for reasoning-off. For throughput comparisons (Figure 4), compute is measured in tokens per second at two settings: S1 with input sequence length (ISL) = 500 tokens and output sequence length (OSL) = 2000 tokens, and S2 with ISL = 5000 and OSL = 500, both with 250 concurrent users and FP8 serving. LN-Ultra and Llama 3.1-405B are served on 8×H100, while DeepSeek-R1 requires 8×H200 due to its size.

• Cross-validation / statistical protocol. The paper does not report cross-validation or statistical significance testing. Checkpoint selection is based on "performance on a subset of reasoning benchmarks" — no hold-out validation set or cross-fold evaluation is described. The paper acknowledges that AIME results can exhibit high variance and that numbers may vary across repeated runs, but does not report standard deviations, confidence intervals, or results from multiple random seeds. This represents a significant limitation: all reported numbers are point estimates from single evaluation runs, and the small test set sizes (especially AIME with 30 problems) mean that differences of a few percentage points may not be statistically reliable.

---

Main Quantitative Results

Reasoning and Non-Reasoning Performance Across Model Sizes

The paper's central evaluation results are presented in Tables 3 (LN-Nano), 4 (LN-Super), and 5 (LN-Ultra), with an aggregated visual comparison in Figure 2. Each table reports reasoning-on and reasoning-off performance separately, reflecting the reasoning toggle.

LN-Nano (Table 3). At 8B parameters, LN-Nano achieves GPQA-Diamond accuracy of 54.1% (reasoning on), compared to 49.0% for DeepSeek-R1-Distilled-Llama-8B and 49.1% for DeepSeek-R1-Distilled-Qwen-7B. On AIME24, LN-Nano scores 61.3% versus 50.4% for the distilled Llama-8B and 55.6% for distilled Qwen-7B. On MATH500, LN-Nano reaches 95.4% (reasoning on), surpassing all comparably sized models (89.1% for distilled Llama-8B, 92.8% for distilled Qwen-7B). On LiveCodeBench (2408–2502), LN-Nano achieves 46.6%, compared to 39.6% for distilled Llama-8B and 37.6% for distilled Qwen-7B.

An important detail in Table 3 is the LN-Nano-SFT column, which shows the model's performance at the SFT stage before RPO alignment. The SFT-only model scores 53.5% on GPQA (vs. 54.1% after RPO), 62.5% on AIME24 (vs. 61.3% after RPO — a slight regression), and 94.4% on MATH500 (vs. 95.4% after RPO). On IFEval, the RPO stage produces a substantial improvement: from 69.9% (SFT) to 79.29-82.1% (after RPO), confirming that the alignment phase primarily benefits instruction following rather than reasoning.

The paper also notes one domain-specific tuning result: early experiments showed worse accuracy on chemistry questions within GPQA-Diamond, so chemistry-related data was upsampled in the STEM subset of the SFT blend, which helped achieve higher GPQA-Diamond scores.

LN-Super (Table 4). At 49B parameters, LN-Super is compared to DeepSeek-R1-Distilled-Llama-70B, QwQ-32B, and Llama-3.3-70B-Instruct. On GPQA-Diamond, LN-Super achieves 66.7% (reasoning on), slightly ahead of DeepSeek-R1-Distilled-Llama-70B (65.2%) and substantially ahead of QwQ-32B (58.8%) and Llama-3.3-70B-Instruct (50.5%). On AIME24, LN-Super scores 67.5%, placing behind QwQ-32B (79.5%) and DeepSeek-R1-Distilled-Llama-70B (70.0%). On AIME25, LN-Super achieves 60.0%, between QwQ-32B (65.8%) and the distilled 70B (55.0%).

The most striking result for LN-Super is its Arena-Hard score of 88.3, which the paper highlights as beating proprietary models (Claude 3.5 Sonnet, GPT-4o-2024-05-13) and much larger open models (Llama-3.1-405B-Instruct at 66.2, Mistral-large-2407). This result emerges from the RLHF phase: iterative online RPO increases Arena Hard from 69.1 to 88.1, and "more interestingly, this process also improves the model's performance on all other adopted benchmarks except IFEval." The paper speculates that RLHF helps the model better utilize its existing knowledge, since neither the training data nor the reward model was optimized for math, coding, or science.

On IFEval, LN-Super achieves 89.2% in reasoning-on mode, which appears to be the result of a dedicated instruction-following RL run (Section 6.1) designed to recover capabilities that degraded during reasoning SFT. The paper reports a tradeoff: "optimizing for instruction following (as measured by IFEval) can compromise conversationality (as measured by Arena-Hard), and conversely, prioritizing conversationality may detract from instruction following performance." Model merging was used to balance these objectives for LN-Super but not for other models "due to mixed outcomes."

LN-Super's weakest result is LiveCodeBench (2408–2502): 45.5% reasoning-on, significantly below DeepSeek-R1-Distilled-Llama-70B (57.5%) and QwQ-32B (63.4%). The paper attributes this to LN-Super's SFT phase being conducted on an earlier version of the coding dataset, unlike LN-Nano and LN-Ultra, and states plans to address this "in a future model refresh."

LN-Ultra (Table 5, Figure 2). This is the flagship comparison. LN-Ultra achieves the highest open-model performance on several benchmarks:

• GPQA-Diamond: 76.0% (reasoning on), surpassing DeepSeek-R1 (71.5%), Llama-4 Behemoth (73.7%), and Llama-4 Maverick (69.8%). This is the headline result — LN-Ultra claims state-of-the-art open-model performance on scientific reasoning.
• AIME24: 80.8% vs. DeepSeek-R1's 79.8% — a narrow margin given the 30-problem test set.
• AIME25: 72.5% vs. DeepSeek-R1's 70.0%.
• MATH500: 97.0% vs. DeepSeek-R1's 97.3% — a slight regression.
• LiveCodeBench (2408–2502): 66.3% vs. DeepSeek-R1's 65.9%.
• LiveCodeBench (2410–2502): 68.1%, compared to Llama-4 Maverick (43.4%) and Llama-4 Behemoth (49.4%). This date range is likely more contamination-resistant since it covers more recent problems.
• IFEval: 88.9% (reasoning on), essentially tied with DeepSeek-R1 (88.8%) and the standard Llama-3.1-405B-Instruct (89.2%).
• Arena-Hard: 87.0%, below DeepSeek-R1's 92.0%.

The LN-Ultra-SFT column is particularly informative for understanding the RL contribution. SFT-only LN-Ultra scores 66.4% on GPQA-Diamond (below DeepSeek-R1's 71.5%), 74.6% on AIME24, 60.4% on AIME25, 96.6% on MATH500, and 60.6% on LiveCodeBench (2408–2502). The RL stage improves GPQA by 9.6 percentage points (66.4 → 76.0), AIME24 by 6.2 points (74.6 → 80.8), and AIME25 by 12.1 points (60.4 → 72.5). MATH500 improves only marginally (96.6 → 97.0), suggesting near-ceiling performance. The reasoning-off numbers for LN-Ultra on reasoning benchmarks (e.g., 56.6 on GPQA, 20.0 on AIME24, 16.7 on AIME25) demonstrate the toggle's effect: reasoning-on substantially outperforms reasoning-off on tasks that benefit from extended thinking.

Throughput vs. Accuracy: The Efficiency Claim (Figure 4)

Figure 4 plots GPQA-Diamond accuracy against throughput (tokens per second) for LN-Ultra, DeepSeek-R1, and Llama 3.1-405B, in two inference settings:

• S1 (ISL=500, OSL=2000): Typical for medium-length reasoning traces. LN-Ultra achieves ~76% accuracy at ~4,000 tokens/s throughput, while DeepSeek-R1 achieves ~71.5% accuracy at ~1,000 tokens/s (roughly a 4× throughput advantage at higher accuracy).
• S2 (ISL=5000, OSL=500): Longer input, shorter output. LN-Ultra achieves similar accuracy at ~2,000 tokens/s, while DeepSeek-R1 achieves ~71.5% accuracy at ~1,000 tokens/s (roughly a 1.9× throughput advantage).

The absolute throughput numbers on the y-axis require careful reading from the log-scale plot. LN-Ultra is the dominant point on the Pareto frontier in both settings: higher accuracy and higher throughput. The paper notes that LN-Ultra and Llama 3.1-405B are served on 8×H100, while DeepSeek-R1 requires 8×H200 due to its size — so the comparison is not purely about model efficiency but also about hardware accessibility (H100 being more widely available and less expensive than H200).

JudgeBench: Out-of-Distribution Generalization (Table 6)

The models are evaluated on JudgeBench (Tan et al., 2025), an out-of-distribution task where the model must differentiate between high-quality and low-quality responses across four domains: Knowledge, Reasoning, Math, and Coding. LN-Ultra achieves an overall score of 79.14%, which is:

• Above DeepSeek-R1 (73.14%) — a substantial 6-point gap
• Just below o3-mini(high) (80.86%) — the top proprietary model
• Above o3-mini(medium) (76.57%) and o1-preview (75.43%)

LN-Super scores 69.71%, above o1-mini (65.71%). The domain breakdown is notable: LN-Ultra is strongest on Math (89.29%, the highest among all models including proprietary) and Coding (92.86%, tied with DeepSeek-R1 and o1-mini), but weakest on Reasoning (81.63%, below o3-mini's 89.80% and 86.73%). This pattern — strong math/coding, relatively weaker pure reasoning — is consistent with the training data distribution, which heavily emphasizes math and code.

The JudgeBench results are significant because they demonstrate generalization beyond tasks the models were explicitly trained for. Neither the SFT data nor the RL reward models targeted LLM-as-a-judge capabilities.

---

Ablation Studies and Robustness Checks

Curriculum vs. random batching in GRPO training (Figure 6): Training LN-Ultra with progressive difficulty batching (curriculum) versus random batching, as measured on GPQA-Diamond. The curriculum condition achieves higher final accuracy and more stable training. By step ~200, the curriculum model reaches ~0.70 Avg@4 and continues to ~0.73, while the random-batching model plateaus around 0.67–0.68 and shows higher variance (the orange curve fluctuates more). This supports the claim that starting with easier examples and progressing to harder ones stabilizes GRPO training and improves final reasoning performance.

Data scaling for code reasoning (Section 3.1.2): An ablation study scaling the code SFT dataset from 25K to 736K samples, with the finding that "the scaling curve does not plateau." Initial scaling from 25K to 100K provides gains, but the most significant improvements come from focusing on harder problems from CodeContests before expanding to the full question set. This result is mentioned in prose without a dedicated figure or table — no specific accuracy numbers are reported for the different dataset sizes, and it is unclear whether this ablation was run on LN-Nano, LN-Super, or a smaller experimental model. This limits its interpretability: we cannot assess the magnitude of the effect or whether it generalizes across model sizes.

SFT vs. SFT+RL performance gap (Tables 3, 4, 5): The inclusion of SFT-only checkpoint evaluations in all three results tables serves as an implicit ablation of the RL stage. For LN-Nano, the RPO alignment stage provides a modest gain on GPQA (53.5 → 54.1) and a large gain on IFEval (69.9 → 79.29/82.1). For LN-Super, the RLHF stage produces the dramatic Arena Hard improvement (69.1 → 88.1 from iterative RPO). For LN-Ultra, the GRPO reasoning stage produces the largest absolute gains: GPQA (66.4 → 76.0), AIME25 (60.4 → 72.5). This ablation demonstrates that RL is not uniformly beneficial — its impact varies by benchmark and model size.

Reasoning toggle: on vs. off performance (Tables 3, 4, 5): Every model is evaluated with both reasoning modes on all benchmarks (except LiveCodeBench for LN-Nano, where only reasoning-on is reported). The consistent pattern is that reasoning-on substantially outperforms reasoning-off on math/science reasoning benchmarks (GPQA, AIME, MATH500), while performance on non-reasoning benchmarks (IFEval, BFCL, Arena Hard) is comparable or slightly favors reasoning-off. For LN-Ultra: GPQA 76.0 on vs. 56.6 off; AIME24 80.8 on vs. 20.0 off; MATH500 97.0 on vs. 80.4 off; IFEval 88.9 on vs. 89.5 off. This validates that the toggle works as intended — reasoning mode engages extended thinking when it helps, and direct mode avoids unnecessary computation when it doesn't.

Reasoning-focused SFT causes IFEval degradation (Table 4, Section 6.1): The paper reports that LN-Super's reasoning SFT causes a "noticeable drop in IFEval scores" (visible in the LN-Super-SFT column: IFEval 81.9, compared to 92.1 for Llama-3.3-70B-Instruct), requiring the dedicated instruction-following RL run described in Section 6.1 to recover (final LN-Super IFEval: 89.2).

Rejection fine-tuning does not yield gains (Section 4.2): The paper states that "recent works (Wen et al., 2025) show that rejection fine-tuning can further improve performance; however, it does not yield gains in our experiments and is therefore omitted." This is a negative result: a technique that improved performance in prior work failed to transfer. No details are provided on the scale of the experiment or which model was tested.

SFT checkpoint selection for RL initialization (Section 7.4): "We find that there is a trade-off between the extent of SFT training and the success likelihood of subsequent RL. Although we had access to SFT checkpoints with higher benchmark scores, we initialized RL from an earlier checkpoint to improve final RL outcomes." This implies an ablation comparing RL outcomes from different SFT checkpoints, but no quantitative data is provided — we don't know how large the difference was or whether this pattern held across benchmarks.

FP8 inference generation speedup (Section 5.2.3): The paper reports a 1.8× generation speedup from FP8 compared to BF16, decomposing it into 1.4× from FP8 computation alone and 0.4× from reduced memory usage (enabling cudagraph). Peak throughput is 32 tokens/s/GPU/prompt. This is not framed as an ablation in the traditional sense but serves as an implicit comparison to a BF16-only baseline. No data on whether FP8 affects generation quality (e.g., pass@1 accuracy of RL training rollouts) is reported — the implicit assumption is that FP8 generation quality is equivalent for RL training purposes.

Model merging for LN-Super (Section 6.1): The paper applies model merging to balance IFEval and Arena Hard performance for LN-Super, selecting from the Pareto frontier. This is described qualitatively — "selecting a checkpoint from the Pareto frontier that balances these objectives" — without specifics on the merging method, the number of checkpoints considered, or the resulting tradeoff curve.

---

Critical Assessment

Claim: "LN-Ultra outperforms DeepSeek-R1 while fitting on a single 8×H100 node and achieving higher inference throughput"

What the experiments show: Figure 4 and Table 5 provide supporting evidence. LN-Ultra achieves higher scores than DeepSeek-R1 on GPQA-Diamond (76.0 vs. 71.5), AIME24 (80.8 vs. 79.8), and AIME25 (72.5 vs. 70.0), while demonstrating higher throughput in both tested inference settings. On MATH500 (97.0 vs. 97.3) and LiveCodeBench (66.3 vs. 65.9), performance is essentially equivalent.

What limits the claim: Three concerns emerge. First, the throughput comparison in Figure 4 uses different hardware: LN-Ultra on 8×H100 vs. DeepSeek-R1 on 8×H200. The claim of "higher throughput" conflates model efficiency with hardware efficiency—H200 GPUs have higher memory bandwidth and capacity than H100, but the paper is comparing a model running on the less-capable hardware against one requiring the more-capable hardware. The real insight is that LN-Ultra can run on H100 at all (DeepSeek-R1 cannot fit on a single 8×H100 node due to its size), which is a deployment accessibility advantage, but the throughput numbers are not purely a model-to-model comparison.

Second, the statistical reliability of the benchmark differences is unknown. AIME24 and AIME25 contain only 30 problems each, and the paper acknowledges high variance due to small test-set size. A 80.8 vs. 79.8 difference on AIME24 (a 1-percentage-point margin on 30 questions) is well within the range of sampling noise—it could disappear or reverse with different random seeds or a different set of 16 completions per prompt. The paper does not report error bars, confidence intervals, or results from multiple evaluation runs. This is particularly concerning because the paper makes strong claims about "surpassing" and "outperforming" based on point estimates with no uncertainty quantification.

Third, the evaluation protocol generates up to 16 completions per prompt and reports pass@1 accuracy. This effectively measures a model's ability to produce a correct answer in one attempt among 16 tries. DeepSeek-R1 may perform differently under different sampling budgets (e.g., pass@1 vs. majority@16), and the paper does not explore whether the relative ranking is sensitive to this choice.

Claim: "LN-Ultra cements it as the best open-source model for scientific reasoning"

What the experiments show: GPQA-Diamond at 76.0% is indeed the highest reported open-model score in the paper, above DeepSeek-R1 (71.5%), Llama-4 Behemoth (73.7%), and Llama-4 Maverick (69.8%). The JudgeBench results provide additional evidence of strong scientific reasoning, with LN-Ultra scoring highest among open models on the Math subcategory (89.29) and strongly on overall score (79.14 vs. 73.14 for DeepSeek-R1).

What limits the claim: The claim that LN-Ultra is the "best" rests on a single benchmark—GPQA-Diamond. AIME and MATH500 are math, not science. The paper does not evaluate on other science benchmarks (e.g., SciQ, ARC, PubMedQA, or the broader GPQA set beyond Diamond), and the JudgeBench evaluation, while demonstrating out-of-distribution generalization, is a relatively niche metric. A model that excels on GPQA-Diamond but underperforms on broader science evaluations would not be unambiguously the "best open-source model for scientific reasoning." The claim would be more robust with a broader science benchmark suite.

Claim: "RL is critical for surpassing DeepSeek-R1, particularly on GPQA"

What the experiments show: The SFT-only vs. final LN-Ultra comparison (Table 5) supports this. LN-Ultra-SFT scores 66.4% on GPQA—below DeepSeek-R1's 71.5%—while LN-Ultra-final reaches 76.0%. The 9.6-percentage-point improvement is unambiguously attributable to the RL stage, since no other training occurs between SFT and the final checkpoint (aside from the brief preference optimization phases, which the paper argues primarily affect IFEval and Arena Hard, not reasoning). The curriculum ablation (Figure 6) demonstrates that the RL stage's design matters—random batching would have produced a smaller improvement.

What limits the claim: The claim could be interpreted as "RL is necessary (not just beneficial) for surpassing the teacher," but the paper does not test alternative approaches that might also break through the distillation ceiling—for example, scaling SFT data further, using ensemble teachers, or applying inference-time techniques like majority voting or best-of-N. The experiments show that this particular RL approach works and that SFT alone doesn't suffice, but they don't establish that RL is the only path to exceeding teacher performance.

Claim: "LN-Super delivers a 5× throughput speedup over Llama 3.3-70B-Instruct"

What the experiments show: Section 2.1 states that LN-Super achieves a 5× throughput speedup over Llama 3.3-70B-Instruct at batch size 256 and TP1. Even when the baseline is run at TP4 (its optimal configuration), LN-Super at TP1 still delivers a ≥2.17× throughput advantage.

What limits the claim: The throughput numbers are stated in prose without a dedicated figure or table — there is no Figure showing the throughput measurement setup, the exact tokens-per-second numbers, or how they vary with batch size and sequence length. The only efficiency visualization is Figure 4, which covers LN-Ultra, not LN-Super. The 5× and 2.17× figures are specific to the stated configuration (batch size 256, TP1) and may differ at other batch sizes, sequence lengths, or request patterns common in production deployments. Without a throughput-vs-batch-size curve or latency percentiles, it's difficult to assess whether the speedup generalizes across deployment scenarios.

Claim: "The reasoning toggle enables cost-effective general-purpose use and detailed multi-step reasoning, without requiring separate models or architectures"

What the experiments show: Tables 3-5 provide strong evidence. For LN-Ultra: GPQA 76.0 on vs. 56.6 off; IFEval 88.9 on vs. 89.5 off. The model maintains its instruction-following and conversational quality in both modes, while selectively engaging reasoning for tasks that benefit from it.

What limits the claim: The toggle is evaluated only on benchmarks that have clear reasoning vs. non-reasoning characteristics. We don't know how the model behaves on ambiguous queries where it's unclear whether reasoning is helpful—for example, moderately complex factual questions, multi-part instructions, or queries that mix reasoning and knowledge retrieval. The paper also doesn't evaluate whether the toggle generalizes to prompts that don't use the exact "detailed thinking on/off" phrasing (e.g., paraphrases, translations, or implicit requests for detailed analysis). The robustness of the toggle to adversarial or edge-case prompts is untested.

Overall Experimental Design Weaknesses

No uncertainty quantification. All benchmark results are point estimates with no standard deviations, confidence intervals, or multi-seed results. This is a significant omission for benchmarks with small test sets (AIME: 30 questions; GPQA-Diamond: likely a few hundred questions based on the original dataset; MATH500: 500 questions). Differences of 1–3 percentage points on these benchmarks may not be statistically significant, yet the paper interprets them as evidence of superiority.

Checkpoint selection procedure is opaque. The paper states that checkpoints are "selected based on performance on a subset of reasoning benchmarks" (Section 7.1). We don't know which subset, whether it's held out from the evaluation benchmarks, or whether the selection procedure could introduce overfitting to the test sets (if the subset overlaps with the evaluation benchmarks). For a paper releasing models and claiming state-of-the-art performance, a transparent validation protocol is essential.

Single evaluation protocol for reasoning benchmarks. All reasoning benchmarks use the same protocol (up to 16 completions, temperature 0.6, top-p 0.95, pass@1). The paper does not explore whether the relative rankings of models change under different protocols (e.g., pass@4, majority@16, or consensus@16), which is relevant because some models may be better calibrated for multiple samples.

The data scaling ablation (Section 3.1.2) lacks quantitative detail. The observation that code performance scales continuously from 25K to 736K samples is reported without a figure, without specifying which model was tested, and without accuracy numbers. This makes it impossible to assess the magnitude of the effect or to determine whether the scaling curve is log-linear, sub-linear, or approaching an asymptote.

No ablation of the Puzzle architecture's contribution to final performance. We see that LN-Ultra-CPT matches or exceeds Llama 3.1-405B-Instruct on key benchmarks (Table 1), but we don't know how LN-Ultra would perform if it had been trained with the same SFT + RL pipeline on the original (uncompressed) Llama 3.1-405B-Instruct architecture. The efficiency gains from Puzzle are clear, but we can't assess whether the architecture compression imposes a quality ceiling that the SFT + RL pipeline could otherwise exceed. This is a missing "upper bound" ablation that would contextualize the architecture-accuracy tradeoff.

No evaluation of the FFN Fusion contribution in isolation. FFN Fusion is introduced as a novel technique for LN-Ultra, but its contribution is not ablated — we see the final 1.71× latency improvement over the original 405B model, but we don't know how much of that comes from Puzzle alone and how much from FFN Fusion. The paper attributes the final 1.71× to the combined effect (Section 2.1: "After applying FFN Fusion, the final model achieves a 1.71× latency improvement"), suggesting FFN Fusion provides the increment from 1.5× (Puzzle's target) to 1.71×, but this isn't explicitly stated.

6. Limitations and Trade-offs

The Inference Efficiency Advantage Depends on Hardware Comparison, Not Just Model Architecture

The assumption or constraint. The paper's central efficiency claim positions LN-Ultra as more efficient than DeepSeek-R1 by comparing LN-Ultra on 8×H100 GPUs against DeepSeek-R1 on 8×H200 GPUs. As stated in Figure 4's caption: "Note that we use 8×H100 for LN-Ultra and Llama 3.1 405B, but 8×H200 for DeepSeek-R1 because of its size." The H200 GPU has significantly higher memory bandwidth (4.8 TB/s vs. 3.35 TB/s for H100) and larger memory capacity (141 GB vs. 80 GB), meaning the hardware is not held constant — DeepSeek-R1 runs on strictly more capable GPUs, yet LN-Ultra achieves higher throughput. The paper frames this as an advantage ("LN-Ultra can run on more widely available hardware"), but the throughput comparison conflates model efficiency with hardware capability.

The consequence. If DeepSeek-R1 could be run on 8×H100 (which it cannot, given its size), the throughput gap would likely be narrower, and the accuracy-throughput Pareto dominance claimed in Figure 4 would shift. More importantly, a practitioner trying to decide between these models in a deployment setting cannot directly compare the absolute throughput numbers because they reflect different hardware. The "4× throughput advantage" shown in setting S1 and "1.9×" in setting S2 (Figure 4) represent the throughput ratio of LN-Ultra on H100 versus DeepSeek-R1 on H200 — not what the ratio would be if both ran on the same hardware. For a practitioner with H200 GPUs available, the throughput comparison is not directly applicable; for a practitioner with only H100 GPUs, DeepSeek-R1 cannot be deployed at all, making the comparison moot.

What evidence exists in the paper. Figure 4 is the only throughput comparison between LN-Ultra and DeepSeek-R1. The paper does not provide a comparison of both models on the same hardware (which is impossible for DeepSeek-R1 on H100) or an estimate of what DeepSeek-R1's throughput would be if it could run on H100. The puzzle architecture's efficiency gains are measured against the base Llama 3.1-405B-Instruct (1.71× latency improvement, Section 2.1), not against DeepSeek-R1, so the model-to-model architecture efficiency gap is not isolated.

Mitigation status. The paper is transparent about the hardware difference in the Figure 4 caption, which is commendable, but does not discuss its implications for the accuracy-throughput comparison. The claim that LN-Ultra is "more efficient" blurs the line between architectural efficiency and deployment accessibility. A cleaner comparison would require both models on identical hardware, but this is physically impossible given DeepSeek-R1's size — an inherent limitation acknowledged implicitly but not discussed as a weakness of the comparison framework.

---

All Benchmark Results Are Point Estimates with No Uncertainty Quantification

The assumption or constraint. Every accuracy number in Tables 3–6 and Figure 2 is reported as a single point estimate — a percentage with one decimal place — with no standard deviations, confidence intervals, or results from multiple evaluation runs. The paper acknowledges in Section 7.1 that "evaluation on reasoning-heavy tasks such as AIME can exhibit high variance due to small dataset size and generation randomness," and that "reported numbers may vary across repeated runs or sampling strategies," but does not quantify this variance. AIME24 contains 30 problems and AIME25 contains 30 problems (split into two 15-problem halves); differences of 1–3 percentage points correspond to getting 0.3–0.9 more problems correct on a 30-question test.

The consequence. Many of the paper's headline claims rest on margins that may not be statistically significant. LN-Ultra's 80.8% on AIME24 vs. DeepSeek-R1's 79.8% represents a 1.0-percentage-point difference — approximately 0.3 questions on a 30-question set. LN-Ultra's 72.5% on AIME25 vs. DeepSeek-R1's 70.0% is a 2.5-point difference — approximately 0.75 questions. Without variance estimates, a practitioner cannot assess whether these differences reflect genuine capability improvements or sampling noise. On larger benchmarks like MATH500 (500 questions), the variance is smaller, but on GPQA-Diamond (typically 198 questions in the "Diamond" subset of the original release), differences of a few percentage points may also be within noise. The JudgeBench results (Table 6) are presented with two decimal places but also lack uncertainty quantification, making it impossible to assess whether LN-Ultra's 79.14 vs. o3-mini(high)'s 80.86 represents a statistically reliable gap or a near-tie.

What evidence exists in the paper. The paper provides no variance estimates for any benchmark. The evaluation protocol (Section 7.1) specifies that "up to 16 completions per prompt" are generated and "average pass@1 accuracy" is reported, but does not specify whether results are averaged over multiple random seeds or whether the 16 completions are the only source of variance accounted for. For LiveCodeBench, the number of questions varies by date range but is not stated, so the test-set size — and hence the expected variance — is unknown to the reader. The ablation studies (Figures 5 and 6) show training curves that appear smooth, but these track accuracy during RL training on large datasets; they do not address the evaluation variance issue.

Mitigation status. The paper acknowledges the high-variance nature of AIME evaluations explicitly in Section 7.1 but does not address it through methodology — no multi-seed evaluation, no confidence intervals, and no discussion of what magnitude of difference would be considered reliable. A practitioner reading the tables as-is might reasonably conclude that a 1-point difference on AIME24 is meaningful, when in fact it could disappear with a different set of 16 completions or a different random seed for sampling. This is a substantial limitation for a paper that makes comparative claims of superiority based on these numbers.

---

The GRPO Reasoning RL Stage is Applied Only to the Largest Model, Limiting Generalizability of the Distillation-Ceiling Finding

The assumption or constraint. The paper's key claim — that supervised fine-tuning on teacher traces is sufficient to approach teacher performance, but reinforcement learning is necessary to exceed it — is demonstrated exclusively on LN-Ultra (253B parameters). As stated in Section 5: "our preliminary experiments indicate that reinforcement learning (RL) yields suboptimal results for smaller models compared to distillation. Given these observations and resource constraints, we apply reasoning RL exclusively to LN-Ultra." The paper does not report what these preliminary experiments showed, on which smaller models they were conducted, or at what scale. The SFT-only performance of LN-Nano and LN-Super (Tables 3 and 4) already exceeds their comparable DeepSeek-R1-Distilled counterparts on many benchmarks without RL, suggesting that for smaller models, distillation alone may be sufficient to reach or exceed the teacher's level.

The consequence. The "distillation ceiling" insight — that RL is necessary to break through the upper bound set by the teacher — may be specific to the regime where the student model's base capability exceeds the teacher's. For smaller models (8B, 49B), distillation alone already produces strong results, and the paper's preliminary experiments suggest RL might actually hurt. This means the paper does not establish a general principle about distillation vs. RL; instead, it establishes that for a very large student model with a strong base (Llama 3.1-405B-Instruct), RL provides gains beyond distillation, while for smaller models, the optimal strategy may be better distillation alone. The lack of reported data on the failed smaller-model RL experiments makes it impossible for practitioners to gauge the threshold at which RL becomes beneficial — is it at 70B parameters? 100B? 200B? Is it contingent on the base model's pretraining quality, the teacher's quality, or some interaction between them?

What evidence exists in the paper. Table 5 shows the SFT-to-RL improvement for LN-Ultra (e.g., GPQA: 66.4 → 76.0). Table 3 shows LN-Nano achieving strong results with SFT + RPO alignment only (no reasoning RL). Table 4 shows LN-Super achieving strong results with SFT + RLHF/IFEval RL (no reasoning RL). The paper mentions "preliminary experiments" suggesting RL is suboptimal for smaller models (Section 5) but does not provide quantitative data, making it impossible to assess how much worse RL performed, on which benchmarks, and at what model size the crossover occurs.

Mitigation status. The paper is transparent that reasoning RL was applied only to LN-Ultra and that preliminary experiments showed unfavorable results for smaller models, which is valuable negative information even in its limited form. However, the lack of quantitative detail on those preliminary experiments means the community cannot learn from the failures — the specific failure modes, the scale at which they occurred, or the hyperparameter configurations tested. The finding that smaller models benefit more from distillation while larger models benefit from RL is a potentially important scaling insight, but it is asserted rather than demonstrated empirically in the paper.

---

The Difficulty Estimation and Curriculum Training Setup Requires an Already-Capable Model, Creating a Cold-Start Problem

The assumption or constraint. The curriculum training strategy for GRPO (Section 5.1) and the data filtering for RL both depend on having a model that can already solve a meaningful fraction of the training problems. Specifically: (1) LN-Super, a 49B model that has already undergone SFT, is used to generate 8 responses per training question and compute the pass rate; (2) questions with pass rate ≥ 0.75 are discarded from RL training entirely; and (3) the remaining questions are batched using a Gaussian window centered on a difficulty level that shifts from easy to hard. The paper states this strategy is designed "to ensure that the model is adequately challenged" — but this setup presupposes access to LN-Super, a model that is itself a product of the same training pipeline. The difficulty estimation is not a standalone technique; it is bootstrapped from an earlier-stage model.

The consequence. A team attempting to replicate the GRPO training recipe from scratch would face a circular dependency: to implement the curriculum strategy, they need LN-Super (or an equivalent mid-size SFT-trained model) to estimate difficulty; to train LN-Super, they need the SFT pipeline; to train the SFT pipeline, they need the synthetic data and teacher models. The paper releases the SFT data and models, so the community can bypass this cold-start problem by using the released LN-Super for difficulty estimation. But for groups that want to apply the same methodology to a different base model family (not Llama 3), the difficulty estimation step requires first training an SFT model of comparable capability, which may not be feasible with limited resources. More fundamentally, the curriculum strategy's effectiveness was demonstrated only in a setting where the difficulty estimator (LN-Super) is closely related to the model being trained (LN-Ultra) — both derived from Llama 3 models and trained on the same SFT data. We cannot assess whether the pass rates computed by LN-Super would be a good difficulty metric for a model from a different family with different pretraining knowledge.

What evidence exists in the paper. Figure 6 shows the curriculum strategy's benefit when the difficulty estimator is LN-Super and the model being trained is LN-Ultra. No ablation tests whether the curriculum benefit persists with a different difficulty estimator (e.g., a smaller model, an external model, or the model being trained itself via self-play). The paper does not analyze whether the discarded easy questions (pass rate ≥ 0.75) contain useful training signal that is lost — perhaps those questions could still contribute to format reward learning or serve as warmup examples.

Mitigation status. The dependency on LN-Super is not discussed as a limitation. The paper presents the curriculum strategy as a methodological contribution without acknowledging the cold-start problem or discussing alternative difficulty estimation approaches (e.g., using problem length, using a much smaller model, or using self-estimated difficulty from the model being trained). The release of LN-Super as an open model partially mitigates the problem for the community, but only for the specific Llama 3-based model family.

---

The Puzzle Architecture Optimization's Contribution to Final Reasoning Performance Cannot Be Isolated

The assumption or constraint. LN-Ultra and LN-Super undergo NAS-based architecture compression via Puzzle, knowledge distillation, and continued pretraining (Section 2) before entering the SFT and RL stages. The paper evaluates the post-CPT checkpoint on standard benchmarks (Table 1), showing it matches or exceeds the original Llama 3 Instruct model, but the paper does not train a control model: the uncompressed base Llama 3 model run through the identical SFT + RL pipeline. Without this control, the paper cannot distinguish between (a) reasoning quality that emerges from the training pipeline operating on a capable base model, and (b) reasoning quality that is specifically enabled or constrained by the heterogeneous architecture.

The consequence. A practitioner deciding whether to adopt Puzzle for their own reasoning model has no evidence about whether the architecture compression imposes a quality ceiling. It is possible — and the paper provides no evidence to the contrary — that applying the same SFT + GRPO pipeline to the original Llama 3.1-405B-Instruct (without NAS) would produce even stronger reasoning performance. The efficiency gains from Puzzle might come at a reasoning quality cost that the paper's experimental design cannot detect. Conversely, it is also possible that the Puzzle architecture, by acting as a form of regularization, actually enables better reasoning by preventing overfitting during SFT — the post-CPT improvements on MATH500 and RULER (Table 1) hint at this — but without the control experiment, this remains speculation. The paper's efficiency claims are well-supported (latency reduction, throughput improvement), but the claim that these models "deliver exceptional reasoning capabilities" conflates the architecture's efficiency with the training pipeline's effectiveness.

What evidence exists in the paper. Table 1 compares post-CPT LN-Ultra to Llama 3.1-405B-Instruct on standard benchmarks (MMLU, MATH500, HumanEval, RULER), showing comparable or improved performance. Table 5 shows final LN-Ultra performance after the full training pipeline. No intermediate or final evaluation of a non-Puzzle LN-Ultra (i.e., the original architecture + SFT + RL) is reported. The ablation of the full training pipeline without Puzzle does not exist.

Mitigation status. The paper does not address this as a limitation. The efficiency contribution of Puzzle is clearly established, but the interaction between architecture compression and reasoning training quality is entirely unexplored. This is a significant gap for a paper whose title is "Efficient Reasoning Models" — the paper demonstrates that the models are efficient and that they reason well, but does not establish the relationship between these two properties. A single control experiment — the base Llama-3.1-405B-Instruct run through SFT-only (without Puzzle) — would at least bound the architecture's effect on distillation quality. The resource cost of such an experiment (SFT on a 405B model) is substantial but far less than the full GRPO training, and its absence limits the paper's contribution to understanding architecture-reasoning tradeoffs.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the reasoning-model development landscape in three specific ways, each with a different magnitude and scope.

First, it establishes that architecture optimization is not just compatible with reasoning training but should be a first-class design stage in the pipeline. Prior to this work, the implicit assumption was that reasoning models inherit their architecture from pretraining and that modifying it would damage the carefully learned reasoning representations. The Puzzle NAS results—particularly LN-Ultra-CPT matching or exceeding the original Llama 3.1-405B-Instruct on MATH500 (80.4 vs. 69.6) and RULER (83.2 vs. 73.7) after aggressive attention removal and FFN compression—refute this assumption directly. This is not an incremental efficiency tweak; it is a reframing that says architectures can and should be redesigned after pretraining specifically for the inference workloads that reasoning models create (long chains of thought, KV-cache pressure from extended generation). The practical consequence is that future reasoning-model projects should budget for an architecture optimization phase, not treat the pretrained architecture as fixed. This makes NAS frameworks and hardware-aware architecture search more attractive research directions; it makes the "train the biggest homogeneous transformer you can, then fine-tune it" approach look increasingly naive for reasoning workloads.

Second, it provides the clearest public evidence for when distillation suffices and when RL becomes necessary, partially reconciling a tension in the recent literature. The Open R1 project, Bespoke-Stratos, and various Distilled-R1 efforts all demonstrated that SFT on reasoning traces works well. DeepSeek-R1's original paper showed that RL from scratch (R1-Zero) can work but is unstable. The implicit question was: under what conditions is each approach preferable? This paper provides a concrete data point: for a 253B model built on a strong base (Llama 3.1-405B-Instruct), SFT alone approaches but does not exceed the teacher (GPQA: 66.4 vs. DeepSeek-R1's 71.5), while RL adds 9.6 percentage points (to 76.0). For smaller models (8B, 49B), SFT alone produces strong results, and preliminary RL experiments were "suboptimal." This doesn't fully resolve the question—we still don't know the exact parameter threshold or whether it depends on base model quality vs. teacher quality—but it provides a testable hypothesis: RL becomes net-beneficial when the student's pretrained capability exceeds the teacher's. This shifts the research question from "does RL help?" (which the literature has answered with a contradictory mix of yes and no) to "at what scale and with what base model quality does RL help, and what are the failure modes below that threshold?" It also implies that the open-source community's heavy focus on distillation-only approaches for small models is largely correct for now, but that frontier-scale open models will increasingly require RL investment to stay competitive.

Third, it demonstrates that the reasoning toggle—a simple learned behavior from paired training data—is a viable and perhaps superior alternative to separate model deployments, routing models, or architectural modifications for controlled reasoning. This is a methodological contribution rather than a conceptual breakthrough: the toggle is not a new idea, but the paper shows that it can be trained into a model through data construction alone (paired reasoning/non-reasoning responses conditioned on a system prompt) and reinforced through format rewards during RL. The fact that LN-Ultra achieves 76.0 on GPQA with reasoning on and 56.6 with reasoning off, while maintaining identical IFEval scores (88.9 on vs. 89.5 off), demonstrates that the toggle cleanly separates the two modes without quality degradation in either. This makes the toggle an attractive default for future open reasoning models—it eliminates the need for a fragile routing classifier (which might misclassify a hard question as needing direct response) and it avoids the deployment cost of maintaining two separate model instances. The paper's open-sourcing of the training data and code means this capability can become a standard feature rather than a proprietary differentiator, which shifts the baseline expectation for what an "open reasoning model" should support.

What becomes less attractive: The paper's results suggest that pure architecture-homogeneous reasoning models (like DeepSeek-R1) leave significant efficiency on the table, and that the community's focus on scaling model size while treating architecture as fixed is suboptimal for inference-heavy workloads. The paper also implicitly argues against the "just distill from a bigger model" approach for frontier-scale efforts—if your base model is already strong, distillation alone caps you at the teacher's level, and you need RL to break through. This doesn't make distillation obsolete (it remains highly effective for small-to-medium models), but it clarifies its ceiling.

What becomes more attractive: Hardware-aware NAS for reasoning models, curriculum RL strategies that use difficulty metrics derived from the model family itself (rather than external heuristics), and FP8 co-located training-inference infrastructure as a design pattern for large-scale GRPO. The paper's infrastructure section (Section 5.2) is unusually detailed and lowers the barrier to entry for other teams attempting similar-scale RL training.

Follow-Up Research This Work Enables

Scaling the difficulty estimation for curriculum RL without requiring a separately trained model. The paper's curriculum strategy uses LN-Super (a 49B SFT-trained model) to estimate problem difficulty via pass rate, creating a cold-start dependency. A natural follow-up would test whether a much smaller model (e.g., LN-Nano at 8B, or even a non-fine-tuned Llama 3.1-8B) produces difficulty rankings that correlate sufficiently with LN-Super's to serve as a curriculum scheduler. A strong experiment would: (a) compute pass rates for the RL training set using LN-Super, LN-Nano, and the base Llama 3.1-8B; (b) measure the rank correlation (Spearman's ρ) between these difficulty estimates; (c) run the GRPO curriculum training with each difficulty estimator and compare final GPQA-Diamond accuracy. If a small model produces sufficiently correlated difficulty rankings, the curriculum strategy becomes accessible to teams without a mid-size SFT model. A negative result—large rank disagreements between estimators—would be equally informative, showing that difficulty estimation is sensitive to the estimator's own reasoning capability.

The interaction between architecture compression ratio and RL training dynamics. The paper cannot isolate whether the Puzzle-compressed architecture helps or hurts reasoning quality because it lacks the critical control experiment: the uncompressed Llama 3.1-405B-Instruct run through the identical SFT + GRPO pipeline. A direct follow-up would train two models—the Puzzle-compressed LN-Ultra architecture and the original homogeneous 405B architecture—through identical SFT (same data, same hyperparameters, same number of steps) and compare their benchmark performance. The hypothesis to test is whether architecture compression acts as a beneficial regularizer (preventing overfitting to teacher patterns during SFT, leaving more flexibility for RL exploration) or a capacity constraint (limiting the model's ability to learn novel reasoning strategies during RL). The post-CPT results (Table 1), where LN-Ultra-CPT surpasses the original 405B on MATH500 and RULER, hint at the regularization hypothesis but don't confirm it. A strong experiment would track not just final accuracy but also metrics of policy entropy during RL, convergence speed, and the diversity of generated reasoning strategies—all of which would illuminate whether the compressed architecture constrains or focuses the RL process.

Characterizing the RL failure modes below the distillation-ceiling threshold. The paper mentions that "preliminary experiments indicate that reinforcement learning (RL) yields suboptimal results for smaller models compared to distillation" but provides no quantitative detail. A systematic study would run GRPO at multiple model sizes (e.g., 8B, 30B, 70B, 150B, 253B) with identical training data, reward structure, and hyperparameters, and characterize exactly what goes wrong at smaller scales. Specific hypotheses to test: (a) smaller models suffer from earlier reward hacking (learning to produce correctly-formatted but content-free reasoning traces that fool the Llama-3.3-70B judge); (b) smaller models have insufficient capacity to simultaneously learn format compliance and correct reasoning, creating a destructive interference between the format reward and accuracy reward; (c) smaller models produce lower-entropy policies that explore insufficiently, converging to suboptimal local maxima of the reward landscape. Each hypothesis suggests different mitigations (stronger KL penalties, separate optimization of format and accuracy objectives, higher sampling temperatures), and testing them would produce actionable guidance for the common case where compute constraints prevent full-scale GRPO training.

Evaluating the reasoning toggle's robustness to adversarial and edge-case prompts. The toggle is evaluated only on standard benchmarks where the "reasoning on" vs. "reasoning off" distinction aligns with the task. Untested scenarios include: (a) prompts that explicitly request both reasoning and brevity ("think step by step but be concise"); (b) prompts that use near-synonyms for the toggle phrase ("please analyze thoroughly," "give me a quick answer"); (c) prompts that are ambiguous about whether reasoning is needed (complex factual questions like "explain why the French Revolution happened" where either a short summary or a detailed analysis could be appropriate); (d) adversarial prompts designed to trigger reasoning mode when it should be off or vice versa. A systematic evaluation would construct a test suite of 200–500 such edge-case prompts, evaluate LN-Ultra with both toggle settings, and measure not just correctness but also response length, presence of thinking traces, and human judgments of appropriateness. This would establish the toggle's practical reliability and identify whether additional training (e.g., adversarial data augmentation) is needed before production deployment.

Pushing the code data scaling curve to its limit to determine if and where it plateaus. The paper's code data ablation (Section 3.1.2) reports continuous improvement from 25K to 736K samples without plateau, but without a figure or numbers. A rigorous follow-up would: (a) replicate the scaling experiment with a controlled setup (fixed model, fixed evaluation, multiple dataset sizes from 25K to several million); (b) fit a scaling law of the form accuracy = a − b / (dataset_size)^c and estimate the asymptotic performance; (c) test whether harder problem curation (the paper's finding that focusing on CodeContests before expanding gave the biggest boost) generalizes by comparing random vs. difficulty-prioritized sampling at each dataset size. The outcome would inform whether the open-source community should invest in generating ever-larger code reasoning datasets or whether we're approaching a point where quality filtering and difficulty curation dominate over raw volume. The paper's existing ablation—mentioned in prose without quantitative detail—provides the starting point but leaves the core question unanswered: what's the shape of the curve, and how close are we to saturation?

Measuring whether FP8 inference generation introduces quality degradation in RL training rollouts. The paper reports a 1.8× generation speedup from FP8 but does not evaluate whether FP8-generated rollouts differ in quality from BF16-generated rollouts. For RL training, this matters: if FP8 introduces quantization noise that reduces the diversity or correctness of generated responses, the policy might learn from degraded training signals, potentially slowing convergence or reducing final accuracy. A direct experiment would run GRPO training for a fixed number of steps with both FP8 and BF16 generation, compare the training curves (GPQA-Diamond accuracy over steps), and analyze the distribution of generated responses (average reward, entropy, length) to detect systematic differences. The hypothesis is that FP8's quantization noise is small relative to the inherent stochasticity of sampling at temperature 1.0, making quality degradation negligible—but this needs verification before the community adopts FP8 generation as a standard practice for RL training.

Practical Applications and Downstream Use Cases

Deploying reasoning-capable assistants on 8×H100 clusters. The most direct practical implication is that organizations with access to 8×H100 nodes (a common configuration in cloud GPU clusters and on-premise deployments) can now run a model that matches or exceeds DeepSeek-R1 on scientific reasoning, math, and coding benchmarks without requiring the rarer and more expensive H200 hardware. The 1.71× latency improvement over the original 405B model and the FP8 generation throughput of 32 tokens/s/GPU/prompt mean that a single 8×H100 node can serve LN-Ultra at production scale. For an API provider or enterprise deploying internal reasoning tools, this translates directly to lower per-query costs and the ability to handle more concurrent users on fixed hardware. The reasoning toggle further reduces average cost: users or applications can explicitly mark simple queries as "reasoning off," avoiding the long chains of thought that make baseline reasoning models expensive for every request regardless of complexity.

Cost-efficient batch data generation for self-improvement pipelines. The paper's training methodology—particularly the SFT data generation pipeline with DeepSeek-R1 as teacher—produced 33M+ training samples. Organizations building self-improvement loops (where a model generates training data for its own next iteration) can use LN-Ultra in reasoning-on mode to generate high-quality reasoning traces for new problems, then use LN-Ultra in reasoning-off mode (or a smaller model like LN-Super) for filtering and verification steps that don't require deep reasoning. The efficiency advantage matters here because data generation involves millions of inference calls: LN-Ultra's 1.71× latency improvement and higher throughput mean that a data generation run that would take 10 days on an uncompressed 405B model takes roughly 6 days on LN-Ultra, directly accelerating the iteration cycle for research teams building self-improving models.

Selective reasoning deployment for agentic systems with heterogeneous query difficulty. In agentic pipelines (where an LLM processes a stream of sub-tasks with varying complexity), the reasoning toggle enables dynamic allocation of inference compute: simple tool-calling steps (e.g., "call the weather API for San Francisco") can run with reasoning off at low latency, while complex planning or debugging steps (e.g., "why did the pipeline fail at step 7 given this error trace?") can engage reasoning on for deep analysis. The benchmark results in Tables 3-5 validate that the toggle doesn't degrade non-reasoning performance: LN-Ultra's BFCL V2 Live score is 74.1 (reasoning on) vs. 73.6 (reasoning off), and IFEval is 88.9 vs. 89.5—essentially identical. This means an agent can switch modes per-step without sacrificing basic competency, and the latency savings on simple steps compound across a long agentic trajectory. A concrete scenario: an agent with 100 steps, where 80 are simple (reasoning off, ~200ms each) and 20 are complex (reasoning on, ~2 seconds each), would complete in roughly 56 seconds with the toggle vs. 200 seconds if every step used reasoning mode—a 3.5× reduction in end-to-end latency, directly improving the viability of agentic systems in interactive settings.

Fine-tuning smaller reasoning models for specialized domains using the open dataset. The paper's release of the complete 33M-sample post-training dataset, combined with the finding that smaller models (LN-Nano at 8B) can achieve strong reasoning performance through SFT alone (GPQA 54.1%, AIME24 61.3%, MATH500 95.4%), enables a practical workflow for domain-specific reasoning: take the open SFT dataset, filter or augment it with domain-specific problems (e.g., medical diagnosis, legal reasoning, financial analysis), fine-tune LN-Nano or another 8B-class model using the three-stage recipe described in Section 4.2 (reasoning-only first, then mixed with non-reasoning, then specialization blend), and deploy a lightweight domain-specific reasoning model that runs on a single GPU. The paper's finding that upsampling chemistry data improved GPQA-Diamond scores for LN-Nano provides direct evidence that targeted data curation works. This workflow is accessible to teams with modest compute budgets (fine-tuning an 8B model on 33M samples requires far less than the 140K H100 hours used for LN-Ultra's RL phase) and is enabled specifically by the paper's data release and the demonstrated effectiveness of SFT-only training for small models.

When to Prefer This Method

The paper positions LN-Ultra against DeepSeek-R1 explicitly (as the strongest open reasoning model) and against the Llama 3.1/3.3 base models implicitly (as the starting point for architecture optimization). The decision framework that emerges from the paper's evidence is:

Prefer the Llama-Nemotron approach (Puzzle NAS + distillation + SFT + optional RL) when:

• You are deploying on 8×H100 GPUs (or smaller configurations for LN-Super/LN-Nano) and cannot access H200 hardware. The Puzzle architecture compression makes deployment feasible on H100 where DeepSeek-R1 cannot fit; Figure 4 demonstrates that LN-Ultra on H100 achieves higher throughput than DeepSeek-R1 on H200 in both tested inference settings.
• Your query distribution includes a mix of simple and complex requests, and you want a single model that handles both without paying reasoning-mode costs on every query. The reasoning toggle (Tables 3-5) enables this; a pure reasoning model like DeepSeek-R1 applies extended thinking to all queries.
• You are building on the Llama 3 model family and want to add reasoning capabilities without training from scratch. The paper's pipeline assumes Llama 3.1/3.3 as the starting point and demonstrates that the full post-training process (NAS + CPT + SFT + RL) is effective with this base.
• You need a smaller reasoning model (8B–49B) and want strong performance from SFT alone, without the infrastructure complexity of large-scale RL. LN-Nano and LN-Super achieve competitive or superior performance to DeepSeek-R1-Distilled models of comparable size through SFT + alignment only (Tables 3-4).

Prefer DeepSeek-R1 (or similar homogeneous reasoning models) when:

• You need the highest possible Arena Hard score (DeepSeek-R1 achieves 92.0 vs. LN-Ultra's 87.0) and conversational alignment is more important than raw scientific reasoning performance.
• You have access to H200 GPUs and the higher hardware cost is acceptable. In this case, the throughput comparison in Figure 4 does not directly apply (it compares LN-Ultra on H100 vs. DeepSeek-R1 on H200), and an apples-to-apples throughput benchmark on identical hardware is impossible since DeepSeek-R1 cannot run on H100.
• You prefer a model with a MoE architecture for its training efficiency characteristics, or you are operating in a deployment context where the heterogeneous architecture of LN-Ultra introduces engineering complexity (custom vLLM weight loaders, unbalanced pipeline stages requiring identity layers) that your team is not equipped to handle.

Prefer pure SFT without RL when:

• You are training a model at 8B–49B scale, where the paper's preliminary experiments found RL to be "suboptimal" compared to distillation alone (Section 5), and LN-Nano and LN-Super already achieve strong results from SFT (Tables 3-4).
• You lack the infrastructure for large-scale GRPO training (the LN-Ultra RL phase consumed 140K H100 hours and required co-located training/inference with custom FP8 inference generation). The SFT pipeline alone, using the open-sourced dataset, is significantly more accessible.
• Your goal is to match or slightly exceed an existing teacher model rather than substantially surpass it. The SFT-only LN-Ultra already approaches DeepSeek-R1 on most benchmarks (Table 5: GPQA 66.4 vs. 71.5, AIME24 74.6 vs. 79.8); RL is only necessary if exceeding the teacher is a hard requirement.