ArXiv: 2512.02556

🎯 Pitch

DeepSeek-V3.2 matches GPT-5 by scaling post‑training compute to over 10% of pre‑training cost through a novel sparse attention mechanism and agent‑data synthesis pipeline. A further compute‑boosted variant, DeepSeek‑V3.2‑Speciale, surpasses GPT‑5 and earns gold medals at both the 2025 IMO and IOI.

1. Executive Summary

This paper introduces DeepSeek-V3.2, an open large language model that integrates three technical innovations to narrow the performance gap with frontier proprietary systems: DeepSeek Sparse Attention (DSA) β€” an efficient attention mechanism that reduces core attention complexity from O(LΒ²) to O(Lk) by selecting only the top-k key-value tokens per query through a learned lightning indexer β€” a scalable reinforcement learning framework that stabilizes GRPO training through unbiased KL estimation, off-policy sequence masking, and expert routing preservation, enabling a post-training compute budget exceeding 10% of pre-training cost, and a large-scale agentic task synthesis pipeline that generates over 1,800 distinct environments and 85,000 complex prompts to integrate reasoning into tool-use scenarios. DeepSeek-V3.2 achieves 93.1% on AIME 2025 and 73.1% on SWE-bench Verified, performing comparably to GPT-5 on reasoning benchmarks while substantially advancing open-model agentic capabilities, and its high-compute variant β€” DeepSeek-V3.2-Speciale β€” surpasses GPT-5 and matches Gemini-3.0-Pro on reasoning, attaining gold-medal performance in the 2025 IMO and IOI, establishing that open models can reach frontier reasoning proficiency when post-training compute is scaled aggressively, though token efficiency and world knowledge breadth remain inferior to the best proprietary systems.

2. Context and Motivation

The Core Problem: The Widening Gap Between Open and Closed LLMs

The paper addresses a specific, measurable problem that emerged in the months preceding its writing: the performance divergence between open-source and closed-source large language models is accelerating, not converging. The authors frame this directly in their introduction:

"While the open-source community continues to make strides, the performance trajectory of closed-source proprietary models has accelerated at a significantly steeper rate. Consequently, rather than converging, the performance gap between closed-source and open-source models appears to be widening, with proprietary systems demonstrating increasingly superior capabilities in complex tasks."

This framing is important because it contradicts an implicit assumption in the field β€” that open models would gradually catch up as training recipes diffused and pretraining compute became more accessible. Instead, the paper argues that a qualitative divergence has occurred: proprietary models have unlocked capabilities (particularly in reasoning and agentic tasks) that open models cannot match simply by scaling the same methods that produced earlier generations.

The paper identifies three specific deficiencies preventing open models from closing this gap:

  1. Architectural inefficiency for long contexts. Open models predominantly use vanilla attention mechanisms, which scale quadratically with sequence length. This is not merely a deployment cost issue β€” it actively limits what models can do during post-training and inference, where long-context processing is essential for complex reasoning (e.g., processing entire codebases, multi-turn agent interactions with extensive tool-call histories).

  2. Insufficient post-training compute investment. The paper argues that open-source models systematically underinvest in the reinforcement learning phase that follows pretraining. This is not about total FLOPs β€” many open models have substantial pretraining budgets β€” but about the ratio of post-training to pretraining compute. Proprietary models, the authors observe, allocate a much larger fraction of their total compute to RL-based post-training, and this investment is what unlocks advanced reasoning behaviors.

  3. Lagging agentic capabilities. In tasks requiring tool use, environment interaction, and instruction-following in complex multi-step scenarios, open models substantially trail proprietary counterparts. The paper cites multiple recent benchmarks β€” EvalSys (2025), Li et al. (2025), Luo et al. (2025) β€” that document this gap quantitatively. This is not a niche concern: agentic capabilities are what make LLMs useful in real-world deployment (code editing, web search, API orchestration), so the gap has direct practical consequences.

Why This Problem Matters

The significance of closing the open-closed gap extends beyond academic interest. The paper's motivation rests on several practical and scientific stakes:

Practical deployment consequences. When open models cannot match proprietary performance on agentic tasks, organizations and developers face a hard choice: use closed APIs (with associated costs, privacy concerns, rate limits, and vendor lock-in) or accept substantially inferior results from open models. The paper's focus on agentic benchmarks β€” SWE-bench Verified, Terminal Bench 2.0, BrowseComp, MCP-Universe β€” targets exactly the tasks where this choice is most consequential for real-world software development, information retrieval, and automation workflows.

Scientific reproducibility. Proprietary models are black boxes: their architecture, training data, and RL recipes are undisclosed. This makes it impossible for the research community to understand why they work, to isolate which components are essential, or to build on them. An open model achieving comparable performance serves as a reproducible artifact that enables scientific investigation of the mechanisms behind reasoning and agentic behavior.

Cost efficiency as a differentiator. The paper explicitly positions DeepSeek-V3.2 as a cost-efficient alternative. The inference cost analysis in Figure 3, benchmarking actual H800 GPU pricing at $2/hour, is not incidental β€” it makes an economic argument. If an open model can match proprietary systems on reasoning while being cheaper to operate (due to architectural efficiency like DSA), it changes the deployment calculus even for organizations that could otherwise afford proprietary APIs.

Theoretical understanding of scaling laws. The paper's claim that post-training compute exceeding 10% of pretraining cost unlocks new capabilities raises a fundamental question about scaling laws: is there a phase transition in capability acquisition that requires a minimum RL budget relative to pretraining? Prior scaling law research (Kaplan et al., 2020; Hoffmann et al., 2022) focused almost exclusively on pretraining. This paper provides evidence that the post-training phase has its own scaling dynamics that are poorly understood and potentially critical for reaching frontier performance. If this claim generalizes, it has major implications for how compute should be allocated across the full model development pipeline.

Prior Approaches and Their Shortcomings

The paper implicitly or explicitly critiques several prior paradigms across its three innovation areas:

Vanilla attention mechanisms. The dominant architecture for most open models, including DeepSeek-V3.2's predecessor (DeepSeek-V3.1-Terminus), uses standard dense attention that computes attention scores between every query-key pair. This yields O(LΒ²) complexity in both computation and memory, making it prohibitive for the long-context scenarios that agentic and reasoning tasks increasingly demand. While prior work explored sparse attention patterns (e.g., sliding window, BigBird, Longformer), these often imposed rigid structural assumptions (locality, sparsity patterns) that did not adapt to content. More adaptive sparse attention methods (e.g., Native Sparse Attention from Yuan et al., 2025, which the paper cites) existed but had not been demonstrated at the scale of a production-grade model with Mixture-of-Experts architecture under continued training from a dense checkpoint. The key gap the paper identifies is that prior sparse attention methods either sacrificed too much performance relative to dense attention or imposed hardware-inefficient patterns that negated their theoretical complexity advantages in practice.

Post-training RL scaling. The paper's post-training approach builds directly on GRPO (Group Relative Policy Optimization), introduced in DeepSeek-R1 (DeepSeek-AI, 2025) and based on Shao et al. (2024). GRPO itself was a simplification of PPO (Schulman et al., 2017) that eliminated the need for a separate value function by normalizing rewards within groups of sampled responses. However, the paper observes that GRPO β€” as applied in prior open-model post-training pipelines β€” suffered from training instability when scaled to large RL budgets. The specific failure modes include:

  • Biased KL estimation: The K3 estimator (Schulman, 2020) used in prior GRPO implementations produces biased gradients when the current policy diverges from the reference policy, particularly for tokens that become much less probable under the current policy. This bias accumulates across training steps, degrading sample quality and destabilizing training.

  • Off-policy degradation: In practice, RL training involves generating a large batch of rollout data and then performing multiple gradient updates on mini-batches from that data. This means most updates are off-policy (the policy has changed since the data was generated). Additionally, optimized inference frameworks used for data generation often differ in implementation details from training frameworks, further exacerbating the mismatch. Prior GRPO implementations did not explicitly handle this off-policyness, leading to instability at scale.

  • MoE routing inconsistency: For Mixture-of-Experts models, the expert routing decisions made during inference (when generating rollouts) can differ from those made during training (when computing gradients) due to framework differences and policy drift. This changes which parameters are being optimized, creating effectively different models at generation and training time β€” a problem specific to MoE architectures that prior RL recipes did not address.

  • Sampling truncation mismatch: Using top-p or top-k sampling during rollout generation (to improve sample quality) creates a mismatch between the full action space the policy is optimized over and the truncated space the samples were drawn from, violating importance sampling assumptions.

The paper argues that these instability issues are not merely implementation details β€” they are the primary reason why open models have been unable to scale post-training compute to the levels that proprietary models achieve. The gap is not in the pretraining compute budget but in the effective utilization of post-training compute, which requires solving these stabilization problems.

Agentic task integration with reasoning. The paper identifies a specific technical challenge that prior work had not systematically addressed: how to combine chain-of-thought reasoning with tool-use in a way that is both effective and token-efficient. The naive approach β€” used in DeepSeek-R1 and some prior work β€” is to discard all reasoning content when a new round of messages arrives (e.g., after a tool call). This forces the model to re-reason through the entire problem from scratch for each subsequent tool interaction, which is not only token-inefficient but also potentially degrades solution quality by severing the continuity of the reasoning chain.

More fundamentally, the paper identifies a data scarcity problem for reasoning-in-tool-use: high-quality trajectories that demonstrate interleaved reasoning and tool calls at scale simply do not exist in available corpora. Prior approaches relied on either (a) prompting models to emit reasoning before tool calls (which produces fragile, easily broken patterns) or (b) constructing environments manually (which does not scale to the thousands of distinct environments the paper argues are necessary for robust generalization). The paper's observation that open models systematically lag on benchmarks like MCP-Universe and MCP-Mark β€” even when their raw reasoning capabilities (on math, code) are competitive β€” points to this integration problem as the bottleneck.

Agentic environment construction at scale. Prior work on agentic post-training either used a small number of hand-constructed environments (limiting diversity and risking overfitting) or extracted tasks from real-world sources like GitHub (which provides diversity but may not cover the long tail of challenging, verifiable agentic tasks). The paper argues that neither approach is sufficient: hand-constructed environments lack the scale and diversity needed for robust RL training, while extracted tasks may not be "hard to solve but easy to verify" β€” the property that makes RL with outcome rewards effective. The gap is a systematic pipeline for generating diverse, automatically verifiable agentic environments that can drive large-scale RL.

How This Paper Positions Itself

The paper does not propose fundamentally new algorithms in the sense of inventing attention mechanisms, RL objectives, or agent architectures from scratch. Instead, it positions itself as an engineering integration effort that addresses three specific bottlenecks preventing open models from reaching frontier proficiency. This is a deliberate framing:

"Through our analysis, we identify three critical deficiencies that limit the capability of open-source models in complex tasks."

The contribution is not a single breakthrough technique but a coherent recipe that spans architecture, post-training methodology, and data generation, with each component addressing a specific failure mode of prior open-source efforts.

Relative to DeepSeek's own prior work, this paper extends DeepSeek-V3.1-Terminus by introducing DSA as the only architectural change (maintaining the same MoE backbone, MLA attention structure, and pretraining data distribution), extends DeepSeek-R1's GRPO recipe with the stabilization techniques described in Section 3.1, and extends the DeepSeek-V3 methodology for unifying reasoning and tool-use into the cold-start β†’ large-scale synthesis pipeline of Section 3.2. The paper is transparent about this lineage: DSA is instantiated "under MLA" (the Multi-head Latent Attention from DeepSeek-V2/V3), the RL training "still adopt[s] Group Relative Policy Optimization," and the cold-start phase "utiliz[es] the DeepSeek-V3 methodology."

Relative to proprietary frontier models, the paper positions DeepSeek-V3.2 as a demonstration of parity potential β€” showing that open models can match GPT-5 on reasoning and significantly advance agentic capabilities β€” while acknowledging remaining gaps in token efficiency, world knowledge breadth, and complex task-solving. The DeepSeek-V3.2-Speciale variant, which relaxes length constraints and trains exclusively on reasoning data, serves as an existence proof that open models can beat GPT-5 and match Gemini-3.0-Pro on reasoning, establishing an upper bound on what the architecture and training recipe can achieve, even if the cost-efficient official variant deliberately trades some performance for token economy.

Relative to the broader open-source ecosystem, the paper positions its contributions as a reference implementation that others can build on, noting that the DSA implementation is open-sourced "to specify the details unambiguously." This reflects a strategic choice: rather than competing solely on benchmark numbers, the paper aims to provide a reproducible foundation that the open-source community can iterate on, particularly for the under-explored problem of agentic post-training at scale.

3. Technical Approach

3.1 Reader Orientation

This paper describes a complete pipeline for building a state-of-the-art open large language model, DeepSeek-V3.2, starting from its predecessor DeepSeek-V3.1-Terminus. The system solves the problem of the widening performance gap between open and proprietary models by addressing three interconnected bottlenecks simultaneously: efficient long-context processing through a learned sparse attention mechanism, stable large-scale reinforcement learning post-training through several algorithmic corrections to GRPO, and the integration of reasoning capabilities into tool-use scenarios through a large-scale synthetic environment generation pipeline. The "shape" of the solution is an architectural modification applied via continued pre-training, followed by a multi-phase post-training process that distills specialist models into a unified generalist through mixed RL training.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major component groups that operate sequentially:

  1. DeepSeek Sparse Attention (DSA) Architecture β€” A modified attention mechanism integrated into the existing Multi-head Latent Attention (MLA) framework, consisting of a lightweight "lightning indexer" (a small neural network that scores which past tokens are worth attending to) and a fine-grained token selection mechanism (a top-k filter that retains only the highest-scoring key-value pairs for the actual attention computation). This replaces dense quadratic attention with linear-in-sequence-length sparse attention.

  2. Continued Pre-Training Pipeline β€” A two-stage process that takes a dense checkpoint of DeepSeek-V3.1-Terminus (already extended to 128K context) and trains it to use DSA. Stage 1 warms up only the indexer while keeping the main model frozen. Stage 2 unfreezes all parameters and trains the full model with sparse attention.

  3. Specialist Distillation Factory β€” A set of eight specialized models (mathematics, programming, general logical reasoning, general agentic tasks, agentic coding, agentic search, writing, and general QA), each fine-tuned from the same base checkpoint with large-scale RL targeted at its specific domain. These specialists generate high-quality domain-specific training data for the final model.

  4. Scalable GRPO Framework β€” The reinforcement learning engine that drives post-training, incorporating four stabilization techniques (unbiased KL estimation, off-policy sequence masking, keep routing, keep sampling mask) to enable stable training at scale with a post-training compute budget exceeding 10% of pre-training cost.

  5. Agentic Task Synthesis Pipeline β€” A data generation system comprising a multi-agent pipeline for search tasks, an automated environment-setup agent for code tasks, and an automatic environment-synthesis agent that generates 1,827 distinct task-oriented environments with verifiable solutions. This pipeline produces the diverse training data needed to integrate reasoning into tool-use scenarios.

Information flows as follows: the pretrained DSA checkpoint β†’ specialist distillation (specialists trained independently on their domains) β†’ data generation (specialists produce training trajectories) β†’ mixed RL training (the generalist model trains on the combined distilled data from all specialists using the stabilized GRPO recipe) β†’ final DeepSeek-V3.2 checkpoint. The experimental variant DeepSeek-V3.2-Speciale diverges by training exclusively on reasoning data with reduced length penalties.

3.3 Roadmap for the Deep Dive

  • First, the DSA attention mechanism β€” how it is built, trained, and instantiated under MLA β€” because it is the only architectural change and governs all subsequent training and inference.
  • Second, the continued pre-training pipeline β€” the two-stage procedure for converting a dense model to sparse attention β€” because it produces the base checkpoint that post-training starts from.
  • Third, the scalable GRPO framework with its four stabilization techniques β€” the RL objective and each correction β€” because this is the engine that drives all post-training capability acquisition.
  • Fourth, the thinking-in-tool-use integration β€” context management, cold-start prompting, and the large-scale agentic task synthesis pipeline β€” because this addresses the most novel challenge: making reasoning and tool-use work together.
  • Fifth, the specialist distillation and mixed RL training process β€” how specialists are trained, how their data is combined, and how the final generalist emerges β€” because this ties the previous components together into the complete post-training recipe.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an engineering systems paper whose core idea is that open models can reach frontier performance by simultaneously solving three specific bottlenecks: architectural inefficiency for long contexts (via DSA), post-training instability at scale (via corrected GRPO), and lack of reasoning-in-tool-use training data (via large-scale synthetic environment generation). None of the individual techniques are fundamentally new algorithms; the contribution is their integration into a coherent, scalable recipe and the empirical demonstration that this recipe closes the gap with proprietary systems.

DeepSeek Sparse Attention (DSA): Mechanism and Instantiation

Motivation for sparse attention. The paper's architectural innovation targets a specific computational bottleneck: vanilla attention computes compatibility scores between every query token and every key token in the sequence, yielding O(LΒ²) complexity in both computation and memory for sequence length L. For long-context scenarios β€” which are increasingly common in reasoning (long chain-of-thought traces) and agentic tasks (extensive tool-call histories) β€” this quadratic scaling becomes prohibitive. Prior sparse attention methods either imposed rigid structural assumptions (e.g., sliding windows that force locality regardless of content) or produced hardware-inefficient memory access patterns that negated theoretical complexity improvements in practice. The paper aims for a sparse attention mechanism that is (a) content-adaptive (selecting tokens based on relevance, not position), (b) hardware-efficient (compatible with the existing MLA implementation and optimized kernel design), and (c) trainable via continued pre-training from an existing dense checkpoint without catastrophic performance regression.

The lightning indexer. DSA introduces a small auxiliary network called the lightning indexer that predicts which past tokens each query token should attend to. Unlike the main attention mechanism β€” which uses high-dimensional representations (the full MLA latent space) to compute precise attention weights β€” the indexer uses a compressed representation specifically designed for fast scoring. The indexer computes an index score $I_{t,s}$ between query token $h_t \in \mathbb{R}^d$ (the current token being processed, where $d$ is the model's hidden dimension) and a preceding token $h_s \in \mathbb{R}^d$ at position $s < t$:

It,s=βˆ‘j=1HIwt,jIβ‹…ReLU(qt,jIβ‹…ksI)I_{t,s} = \sum_{j=1}^{H_I} w_{t,j}^I \cdot \text{ReLU}\left(q_{t,j}^I \cdot k_s^I\right)

where $H_I$ is the number of indexer heads (kept small for efficiency), $q_{t,j}^I \in \mathbb{R}^{d_I}$ is the $j$-th query vector derived from the current token $h_t$ (projected into a lower-dimensional space of size $d_I$), $k_s^I \in \mathbb{R}^{d_I}$ is the key vector derived from the preceding token $h_s$, $w_{t,j}^I \in \mathbb{R}$ is a learned scalar weight also derived from $h_t$ that determines how much to trust the $j$-th indexer head, and ReLU is used as the activation function.

What it computes: The indexer produces a single scalar score $I_{t,s}$ for every pair of tokens $(t, s)$ where $s < t$. The computation happens per indexer head: each head projects the query and key into a $d_I$-dimensional space, computes their dot product (a standard compatibility measure), applies ReLU (which zeros out negative scores β€” tokens deemed irrelevant), and then the head outputs are weighted by learned scalars $w_{t,j}^I$ and summed. The result is a sequence-length vector $I_{t,:}$ of scores for token $t$ indicating how relevant each past token is.

Why this form: The use of multiple heads ($H_I$) with a small $d_I$ is a deliberate efficiency tradeoff β€” it provides some representational capacity to capture different relevance patterns without the computational cost of full attention heads. The dot-product $q \cdot k$ is the standard fast compatibility measure from attention mechanisms. The ReLU activation (instead of, say, softmax or sigmoid) has a specific throughput justification: ReLU is computationally cheaper than alternatives and zeros out negative scores, naturally producing a sparse selection pattern. The learned weights $w_{t,j}^I$ allow the model to learn which indexer heads are reliable for different query contexts. The paper explicitly notes: "We choose ReLU as the activation function for throughput consideration." The indexer "has a small number of heads and can be implemented in FP8," enabling extremely fast computation β€” critical because the indexer still operates on all $t \times t$ pairs.

Fine-grained token selection. Given the index scores $\{I_{t,s}\}$ for query token $h_t$, the selection mechanism is simple: retain only the key-value entries corresponding to the top-k index scores. Specifically, let $\text{Top-k}(I_{t,:})$ return the set of indices of the $k$ largest values in the score vector. Then the attention output $u_t$ is computed using only those selected entries:

ut=Attn(ht,{cs∣It,s∈Top-k(It,:)})u_t = \text{Attn}\left(h_t, \{c_s \mid I_{t,s} \in \text{Top-k}(I_{t,:})\}\right)

where $c_s$ is the key-value entry for position $s$ (in MLA, this is the latent vector β€” see below), and Attn is the standard attention mechanism (compute query-key dot products for the selected positions, softmax-normalize, weight-sum the values). The number $k$ is set to 2048 during sparse training β€” meaning each query token attends to only 2048 out of potentially 128K past tokens.

What it computes: For each query token, the indexer scores all past tokens, a top-k filter selects the 2048 highest-scoring ones, and only those 2048 key-value pairs participate in the actual attention computation. All other tokens are completely ignored for this query. This reduces the core attention complexity from O(LΒ²) to O(Lk), where $k \ll L$.

Why top-k rather than threshold-based selection: A threshold-based approach (select all tokens with score above some value) would produce variable numbers of selected tokens per query, making GPU kernel optimization difficult (GPUs prefer fixed-size operations). Top-k with a fixed $k = 2048$ guarantees a constant memory footprint and predictable computation time per query, enabling efficient implementation. The specific value 2048 is a design choice that balances sparsity (fewer tokens = faster) against recall (more tokens = less likely to miss important context).

Instantiation under MLA. DeepSeek-V3.2 inherits its base architecture from DeepSeek-V3.1-Terminus, which uses Multi-head Latent Attention (MLA). MLA compresses the key-value representations into a low-rank latent space to reduce the KV-cache memory footprint during inference. The paper notes a critical practical detail: "At the kernel level, each key-value entry must be shared across multiple queries for computational efficiency" (citing Yuan et al., 2025). This means that DSA must be implemented under the MQA (Multi-Query Attention) mode of MLA, where a single latent vector (the compressed key-value entry) is shared across all query heads of a query token β€” as opposed to the MHA (Multi-Head Attention) mode where each head has its own key-value projection. Figure 2 illustrates this architecture: the indexer receives the input hidden state, produces indexer queries $\{q_{t,j}^I\}$ and weights $\{w_{t,j}^I\}$, the indexer keys come from the same latent vectors, the top-k selector picks which latent vectors to use, and the core attention (MQA) computes the output using only those selected latent vectors.

Why MQA mode: In the MHA mode of MLA, each attention head has separate key-value projections, so a sparse selection mechanism would need to either (a) select different tokens per head (complex and hardware-inefficient) or (b) apply the same selection to all heads (which would require agreement on which tokens matter). In MQA mode, there is only one set of key-value entries shared across all heads, so the sparse selection is applied once and naturally propagates to all heads. This makes the implementation tractable.

DSA complexity analysis. The paper is careful to specify that DSA reduces "the core attention complexity of the main model from $O(L^2)$ to $O(Lk)$," but notes that "the lightning indexer still has a complexity of $O(L^2)$." The key insight is that the indexer is much cheaper than the main attention: it has fewer heads ($H_I$ is small), operates in a lower-dimensional space ($d_I < d$), and can use FP8 precision. The end-to-end speedup comes from the fact that the expensive operation β€” the full-dimensional attention computation over value vectors β€” is performed only on 2048 tokens per query, while the cheap operation β€” the indexer scoring β€” still scans all tokens. Figure 3 provides empirical validation: the cost per million tokens for DeepSeek-V3.2 grows much more slowly with token position than for DeepSeek-V3.1-Terminus, particularly during decoding (generating new tokens autoregressively), where the attention cost dominates.

Special case for short sequences. The paper mentions: "for short-sequence prefilling, we specially implement a masked MHA mode to simulate DSA, which can achieve higher efficiency under short-context conditions." This means that when processing short prompts (where $L$ is small enough that $O(L^2)$ is manageable), the system falls back to dense attention (MHA mode with all tokens), because the overhead of indexer computation and sparse kernel launching would exceed the savings from sparsity. This is a deployment optimization, not a training one.

Continued Pre-Training Pipeline

Starting point. Continued pre-training begins from a base checkpoint of DeepSeek-V3.1-Terminus whose context length has already been extended to 128K tokens through prior training. The extension to 128K was done separately (not described in this paper), and the continued pre-training uses exactly the same data distribution: "the distribution of training data is totally aligned with the 128K long context extension data used for DeepSeek-V3.1-Terminus." This means no new data sources are introduced; the model is simply retrained on the same long-context data but with sparse attention.

Two-stage training. The continued pre-training consists of two stages: a dense warm-up stage and a sparse training stage. This two-phase approach is motivated by a bootstrapping problem: if you immediately switch to sparse attention, the indexer β€” which starts from random initialization β€” would make poor token selections, causing the model to miss critical context and receive corrupted gradients. The dense warm-up gives the indexer a chance to learn what matters before the model has to rely on its selections.

Dense warm-up stage. In this stage, attention remains dense (all tokens attend to all past tokens). All model parameters are frozen except for the lightning indexer. The training objective is a KL-divergence loss that aligns the indexer's scores with the main attention's distribution. Specifically, for the $t$-th query token, the main attention scores are aggregated by summing across all attention heads (producing a single scalar per past token), then L1-normalized along the sequence dimension to produce a target distribution $p_{t,:} \in \mathbb{R}^t$. The loss is:

LI=βˆ‘tDKL(pt,:βˆ₯Softmax(It,:))\mathcal{L}_I = \sum_t D_{KL}\left(p_{t,:} \parallel \text{Softmax}(I_{t,:})\right)

where $D_{KL}$ is the Kullback-Leibler divergence, $p_{t,:}$ is the aggregated-and-normalized main attention distribution (the "target" β€” what the full model considers important), and $\text{Softmax}(I_{t,:})$ converts the indexer's raw scores into a probability distribution over past tokens.

What it computes: For each query position, the main model's full attention is collapsed across heads and normalized to form a probability distribution indicating which past tokens matter most. The indexer independently produces its own scores, which are softmax-normalized to form another distribution. The KL divergence measures how different these two distributions are β€” it penalizes the indexer for assigning probability mass to tokens the main attention considers unimportant, and for ignoring tokens the main attention considers important. The loss is summed over all query positions.

Why this form: KL divergence is the standard objective for distilling one distribution into another β€” it encourages the student (indexer) to match the teacher (main attention) exactly, not just in terms of the highest-scoring tokens but the entire shape of the distribution. Summing attention scores across heads (rather than, say, taking the maximum per token) captures the aggregate importance signal β€” a token that receives moderate attention from many heads may be more important than one receiving high attention from a single head. L1 normalization forces the target to be a valid distribution and is chosen over softmax normalization because the raw attention weights are already softmax-normalized per head.

Warm-up hyperparameters: "learning rate of $10^{-3}$," trained "for only 1000 steps, with each step consisting of 16 sequences of 128K tokens, resulting in a total of 2.1B tokens." The high learning rate ($10^{-3}$ is large for LLM fine-tuning) reflects that only a small set of new parameters (the indexer) is being trained from scratch on a simple distillation objective. The short duration (1000 steps) indicates this is just an initialization, not convergence training.

Sparse training stage. Following warm-up, the fine-grained token selection mechanism is activated β€” now the model only attends to the top-2048 tokens per query as selected by the indexer. All model parameters (both the main model and the indexer) are optimized jointly, but with a crucial separation of signals:

  • The indexer is trained using only the indexer loss $\mathcal{L}_I$, and its input is detached from the computational graph β€” meaning gradients from the language modeling loss do not flow back to the indexer. The indexer loss in this stage considers only the selected token set $\mathbb{S}_t = \{s \mid I_{t,s} \in \text{Top-k}(I_{t,:})\}$:

LI=βˆ‘tDKL(pt,Stβˆ₯Softmax(It,St))\mathcal{L}_I = \sum_t D_{KL}\left(p_{t,\mathbb{S}_t} \parallel \text{Softmax}(I_{t,\mathbb{S}_t})\right)

This is the same KL divergence but restricted to the subset of tokens that were actually selected β€” the indexer is only penalized for poor calibration within the selected set, not for tokens it correctly ignored.

  • The main model (all parameters except the indexer) is trained using only the standard language modeling loss β€” next-token prediction. Its gradients do not flow through the indexer.

What it computes: Two separate optimization processes run simultaneously: (1) the indexer learns to match the main attention's distribution over the tokens it selects, ensuring that what it does select is scored accurately relative to the main model's preferences, and (2) the main model learns to predict next tokens using only the sparse context provided by the indexer's selections, adapting its representations to work with incomplete attention.

Why this separation: If the language modeling loss gradients flowed into the indexer, the indexer would learn to select tokens that reduce the language modeling loss β€” which might not align with selecting the tokens the main attention considers important. The indexer could, for example, learn to select only a few highly predictive tokens and ignore context needed for other tasks, specializing to the LM objective rather than learning a general-purpose selection policy. By detaching the indexer and training it only on the KL objective, the paper ensures it learns to mimic the full dense attention β€” a more general and robust target. The main model's language modeling loss, meanwhile, provides the adaptation signal for sparse attention: the model must learn to represent and predict from a context that is now missing potentially 97%+ of tokens (2048 out of 128K).

Sparse training hyperparameters: "learning rate of $7.3 \times 10^{-6}$," "select 2048 key-value tokens for each query token," trained "for 15000 steps, with each step consisting of 480 sequences of 128K tokens, resulting in a total of 943.7B tokens." The lower learning rate compared to warm-up ($7.3 \times 10^{-6}$ vs. $10^{-3}$) reflects that all parameters are being updated and the model is already near a good solution from the dense checkpoint β€” only adaptation is needed, not learning from scratch. The total token count (943.7B) is substantial β€” larger than many full pretraining runs β€” indicating that adapting to sparse attention requires significant compute. The batch size (480 sequences Γ— 128K tokens = 61.4M tokens per step) is large, typical of large-scale continued pre-training.

Parity evaluation after continued pre-training. The paper evaluates the resulting checkpoint (called DeepSeek-V3.2-Exp) against its predecessor DeepSeek-V3.1-Terminus. On standard benchmarks (MMLU-Pro, GPQA Diamond, LiveCodeBench, etc.), the two models show "similar performance" β€” meaning the switch to sparse attention did not cause significant regression. On two independent long-context benchmarks (AA-LCR and Fiction.liveBench), DeepSeek-V3.2-Exp actually outperforms its predecessor, with the paper noting that it "scores four points higher than DeepSeek-V3.1-Terminus in reasoning mode" on AA-LCR and "consistently outperforms DeepSeek-V3.1-Terminus across multiple metrics" on Fiction.liveBench. Critically, the two models share "an identical post-training strategy" and their ChatbotArena Elo scores are "closely matched," confirming that DSA preserves both capability and human preference at the base model level. This parity validation is essential β€” if the sparse model had degraded, all subsequent post-training gains would be from a weaker starting point.

Scalable GRPO Framework with Stabilization Techniques

The base GRPO objective. The paper uses Group Relative Policy Optimization (GRPO) as its reinforcement learning algorithm, as introduced in DeepSeek-R1 (DeepSeek-AI, 2025) and based on Shao et al. (2024). GRPO simplifies PPO by eliminating the learned value function. Instead of estimating advantages using a critic network, GRPO samples a group of $G$ responses $\{o_1, ..., o_G\}$ from the old policy $\pi_{\text{old}}$ for each question $q$, computes an outcome reward $R_i$ for each response using a set of reward models, and normalizes rewards within the group to produce advantages. The base GRPO objective is:

JGRPO(ΞΈ)=Eq∼P(Q),{oi}i=1GβˆΌΟ€old(β‹…βˆ£q)[1Gβˆ‘i=1G1∣oiβˆ£βˆ‘t=1∣oi∣min⁑(ri,t(ΞΈ)A^i,t,clip(ri,t(ΞΈ),1βˆ’Ξ΅,1+Ξ΅)A^i,t)βˆ’Ξ²DKL(πθ(oi,t)βˆ₯Ο€ref(oi,t))]\mathcal{J}_{\text{GRPO}}(\theta) = \mathbb{E}_{q \sim P(Q), \{o_i\}_{i=1}^G \sim \pi_{\text{old}}(\cdot \mid q)} \left[ \frac{1}{G} \sum_{i=1}^G \frac{1}{|o_i|} \sum_{t=1}^{|o_i|} \min\left(r_{i,t}(\theta) \hat{A}_{i,t}, \text{clip}(r_{i,t}(\theta), 1 - \varepsilon, 1 + \varepsilon) \hat{A}_{i,t}\right) - \beta D_{KL}\left(\pi_\theta(o_{i,t}) \parallel \pi_{\text{ref}}(o_{i,t})\right) \right]

where $r_{i,t}(\theta) = \frac{\pi_\theta(o_{i,t} \mid q, o_{i,<t})}{\pi_{\text{old}}(o_{i,t} \mid q, o_{i,<t})}$ is the importance sampling ratio (how much more or less likely the current policy is to generate token $o_{i,t}$ compared to the old policy that actually generated it), $\varepsilon$ controls the clipping range (preventing too-large policy updates by capping the ratio at $1 \pm \varepsilon$), $\beta$ controls the strength of a KL divergence penalty that keeps the policy from drifting too far from a reference policy $\pi_{\text{ref}}$, and $\hat{A}_{i,t} = R_i - \text{mean}(\mathbf{R})$ is the advantage β€” the reward for this response minus the average reward in the group, positive for above-average responses and negative for below-average ones.

What it computes: For each question, the model generates $G$ responses. Each response gets a scalar reward from the reward model(s). The average reward across the group is computed. Responses above average get positive advantages (the model should become more likely to generate similar tokens), and responses below average get negative advantages (the model should become less likely). The policy is updated to increase the probability of tokens that led to above-average outcomes and decrease the probability for below-average ones, clipped to prevent overly large changes, with a penalty for diverging from the reference policy. This is computed per-token, averaged over tokens in the response, averaged over responses in the group, and then averaged over questions.

Why this form over PPO: GRPO eliminates the value network (which estimates expected future reward from each state) entirely. In PPO, the advantage is computed as $R_i - V(s_t)$ where $V$ is a separately trained critic. Training a value function for language tasks is difficult β€” the state space is enormous, the rewards are sparse (only at sequence end), and the value network adds substantial memory and compute overhead. GRPO's group-normalization approach β€” $R_i - \text{mean}(\mathbf{R})$ β€” is a zero-cost approximation that works well when the group is large enough that the mean approximates the expected value. The clipping mechanism (same as PPO) prevents the policy from changing too much in a single update, which is critical for stability.

The reward models for GRPO. The paper specifies different reward model strategies per task type: "For reasoning and agent tasks, we employ rule-based outcome reward, length penalty, and language consistency reward. For general tasks, we employ a generative reward model where each prompt has its own rubrics for evaluation." This means the reward $R_i$ is actually a composite: a rule-based component (e.g., did the final answer match the ground truth? did the code pass the tests?), a length penalty (discouraging unnecessarily verbose responses β€” this is relaxed in DeepSeek-V3.2-Speciale), and a language consistency reward (likely penalizing mixed-language outputs or encouraging coherent formatting). For general alignment tasks where ground-truth rewards don't exist, a learned generative reward model β€” essentially an LLM fine-tuned to score responses β€” provides the signal, with rubrics (evaluation criteria) specific to each prompt type.

Why GRPO can become unstable when scaled. The paper identifies four specific failure modes that emerge when applying GRPO at large scale (thousands of RL steps with large models). Each stabilization technique addresses one failure mode.

Stabilization Technique 1: Unbiased KL Estimate

The problem with the K3 estimator. The original GRPO implementation used the K3 estimator (Schulman, 2020) for computing the KL divergence between the current policy $\pi_\theta$ and the reference policy $\pi_{\text{ref}}$. The K3 estimator approximates $D_{KL}(\pi_\theta \parallel \pi_{\text{ref}})$ using samples from the old policy $\pi_{\text{old}}$, but its gradient is biased. The paper explains the failure case: when a token is much less probable under the current policy than the reference policy ($\pi_\theta \ll \pi_{\text{ref}}$), the K3 estimator's gradient assigns "disproportionately large, unbounded weights to maximize the likelihood of these tokens, resulting in noisy gradient updates that accumulate to degrade sample quality in subsequent iterations and lead to unstable training dynamics." In plain terms: if the current policy thinks a token is very unlikely but the reference policy thinks it's likely, the biased gradient pushes hard to increase that token's probability, even if that's not the right thing to do.

The corrected K3 estimator. The paper corrects this by deriving an unbiased KL estimator using importance sampling:

DKL(πθ(oi,t)βˆ₯Ο€ref(oi,t))=πθ(oi,t∣q,oi,<t)Ο€old(oi,t∣q,oi,<t)[Ο€ref(oi,t∣q,oi,<t)πθ(oi,t∣q,oi,<t)βˆ’log⁑πref(oi,t∣q,oi,<t)πθ(oi,t∣q,oi,<t)βˆ’1]D_{KL}\left(\pi_\theta(o_{i,t}) \parallel \pi_{\text{ref}}(o_{i,t})\right) = \frac{\pi_\theta(o_{i,t} \mid q, o_{i,<t})}{\pi_{\text{old}}(o_{i,t} \mid q, o_{i,<t})} \left[ \frac{\pi_{\text{ref}}(o_{i,t} \mid q, o_{i,<t})}{\pi_\theta(o_{i,t} \mid q, o_{i,<t})} - \log \frac{\pi_{\text{ref}}(o_{i,t} \mid q, o_{i,<t})}{\pi_\theta(o_{i,t} \mid q, o_{i,<t})} - 1 \right]

where $\frac{\pi_\theta}{\pi_{\text{old}}}$ is the importance weight (correcting for the fact that the token was sampled from $\pi_{\text{old}}$ not $\pi_\theta$), $\frac{\pi_{\text{ref}}}{\pi_\theta}$ is the density ratio between reference and current policy, and $\log \frac{\pi_{\text{ref}}}{\pi_\theta}$ is the log-ratio.

What it computes: This is a per-token estimate of the KL divergence $D_{KL}(\pi_\theta \parallel \pi_{\text{ref}})$ β€” a measure of how much the current policy has diverged from the reference. The expression inside the brackets is the standard definition of the pointwise KL contribution: $p/q - \log(p/q) - 1$ where $p = \pi_{\text{ref}}$ and $q = \pi_\theta$. The leading importance weight corrects for distribution shift between the old and current policy.

Why this form is unbiased: The standard K3 estimator directly computes $\frac{\pi_{\text{ref}}}{\pi_\theta} - \log \frac{\pi_{\text{ref}}}{\pi_\theta} - 1$ without the importance weight, which would be unbiased only if the tokens were sampled from $\pi_\theta$. Since they are actually sampled from $\pi_{\text{old}}$, multiplying by $\pi_\theta / \pi_{\text{old}}$ reweights the sample to approximate what would have been observed under $\pi_\theta$, making the gradient unbiased. The paper states: "As a direct result of this adjustment, the gradient of this KL estimator becomes unbiased, which eliminates systematic estimation errors, thereby facilitating stable convergence."

Practical consequence: The paper notes that "different domains benefit from varying strengths of KL regularization. For certain domains, such as mathematics, applying a relatively weak KL penalty or even omitting it entirely can yield improved performance." The unbiased estimator makes this tuning safer β€” when the KL penalty is weak or zero, there is no risk of the biased K3 estimator injecting noise.

Stabilization Technique 2: Off-Policy Sequence Masking

The problem. In practice, RL training generates a large batch of rollout data, which is then split into mini-batches for multiple gradient update steps. After the first update, the policy $\pi_\theta$ has changed, so subsequent updates use data that is off-policy (generated by $\pi_{\text{old}}$ which is no longer the current policy). Additionally, "inference frameworks used for efficient data generation are often highly optimized, which may differ in implementation details from training frameworks" β€” meaning the probability $\pi_{\text{old}}$ returned by the inference system might not exactly match what the training system would compute for the same policy checkpoint. Both effects increase the KL divergence between $\pi_{\text{old}}$ and $\pi_\theta$ beyond what importance sampling can reliably correct for.

The solution. The paper introduces a binary mask $M_i$ into the GRPO loss:

JGRPO(ΞΈ)=Eq,{oi}βˆΌΟ€old[1Gβˆ‘i=1G1∣oiβˆ£βˆ‘t=1∣oi∣min⁑(ri,t(ΞΈ)A^i,t,clip(ri,t(ΞΈ),1βˆ’Ξ΅,1+Ξ΅)A^i,t)Mi,tβˆ’Ξ²DKL(πθ(oi,t)βˆ₯Ο€ref(oi,t))]\mathcal{J}_{\text{GRPO}}(\theta) = \mathbb{E}_{q, \{o_i\} \sim \pi_{\text{old}}} \left[ \frac{1}{G} \sum_{i=1}^G \frac{1}{|o_i|} \sum_{t=1}^{|o_i|} \min\left(r_{i,t}(\theta) \hat{A}_{i,t}, \text{clip}(r_{i,t}(\theta), 1 - \varepsilon, 1 + \varepsilon) \hat{A}_{i,t}\right) M_{i,t} - \beta D_{KL}\left(\pi_\theta(o_{i,t}) \parallel \pi_{\text{ref}}(o_{i,t})\right) \right]

where $M_{i,t}$ is defined as:

Mi,t={0A^i,t<0,1∣oiβˆ£βˆ‘t=1∣oi∣log⁑πold(oi,t∣q,oi,<t)πθ(oi,t∣q,oi,<t)>Ξ΄1otherwiseM_{i,t} = \begin{cases} 0 & \hat{A}_{i,t} < 0, \quad \frac{1}{|o_i|} \sum_{t=1}^{|o_i|} \log \frac{\pi_{\text{old}}(o_{i,t} \mid q, o_{i,<t})}{\pi_\theta(o_{i,t} \mid q, o_{i,<t})} > \delta \\ 1 & \text{otherwise} \end{cases}

and $\delta$ is a hyper-parameter threshold for acceptable policy divergence.

What it computes: For each response $o_i$ in the group, the system computes the average log-ratio between the old and current policies over all tokens β€” this is an estimate of the KL divergence between $\pi_{\text{old}}$ (which generated the data) and $\pi_\theta$ (the current policy). If this divergence exceeds threshold $\delta$ AND the response has negative advantage ($\hat{A}_{i,t} < 0$ β€” it was below average), then the mask $M_{i,t}$ is set to 0 for all tokens in that response. This zeroes out the entire response's contribution to the policy gradient.

Why only mask negative-advantage sequences: The paper's justification is intuitive: "the model benefits the most by learning from its own mistakes, whereas highly off-policy negative samples can be detrimental, potentially misleading or destabilizing the optimization process." If a response had positive advantage (above average), the model should still learn from it even if it's somewhat off-policy β€” increasing the probability of good behaviors is harder to screw up. But if a response was below average and the policy has changed significantly since it was generated, the "lesson" it was supposed to teach (which tokens to avoid) may no longer apply. The model might reduce probability of tokens that its current version would actually produce well, or it might learn from mistakes that are no longer relevant to its current behavior.

Why this form: The per-sequence average log-ratio is a natural measure of how much the policy has changed since the data was generated. Applying a threshold $\delta$ is a simple binary gating mechanism β€” sequences are either used or discarded, with no partial weighting. This is a conservative approach that prioritizes stability: when uncertain, throw the data away rather than risk a destabilizing update.

Stabilization Technique 3: Keep Routing

The problem specific to MoE. Mixture-of-Experts models activate only a subset of expert modules for each token, determined by a routing mechanism. During inference (generating rollouts), the routing function dispatches each token to its top experts based on the policy's current parameters. During training, when the same tokens are processed to compute gradients, the routing function should produce the same expert assignments β€” but due to "discrepancies between inference and training frameworks, compounded by policy updates," the routing can differ. This means the experts being updated during training are not necessarily the ones that were used to generate the token, creating an inconsistency in which parameters are optimized. The paper describes the consequence: "Such inconsistency induces abrupt shifts in the active parameter subspace, which destabilizes optimization and exacerbates off-policy issues."

The solution. "We preserve the expert routing paths used during sampling in the inference framework and enforce the same routing paths during training." Concretely: when the inference framework generates a rollout, it records which expert was activated for each token. During training, instead of re-running the routing function (which might produce different assignments due to policy drift or framework differences), the recorded paths are used directly β€” the same experts are activated for the same tokens.

Why this matters for stability: In an MoE model, each expert specializes in certain types of computation. If a token was processed by expert A during generation but by expert B during training, the gradient for that token updates expert B (which didn't actually produce the output) instead of expert A (which did). This effectively trains the wrong parameters, creating noise in the optimization that compounds across many tokens and steps. Keep Routing ensures that the gradient for a token updates exactly the experts that generated it, maintaining consistency between the forward pass that produced the action and the backward pass that learns from its outcome.

History note: The paper mentions this technique "has been adopted in our RL training pipeline since DeepSeek-V3-0324," indicating it was a prior internal discovery that proved essential for stable MoE RL training.

Stabilization Technique 4: Keep Sampling Mask

The problem. Top-p (nucleus) and top-k sampling are widely used to improve generation quality by truncating the low-probability tail of the token distribution β€” only tokens with sufficient probability mass are considered. However, "while such truncation preserves sample quality, it introduces a mismatch between the action spaces of $\pi_{\text{old}}$ and $\pi_\theta$, which violates the principles of importance sampling and destabilizes training." The issue: during generation, the old policy's distribution was truncated (tokens with probability below the top-p threshold were set to zero and never sampled). But during training, the current policy's distribution is computed over the full vocabulary β€” including tokens that were impossible to sample during generation. The importance ratio $r_{i,t}(\theta)$ then compares probabilities over different action spaces, which is mathematically invalid.

The solution. "We preserve the truncation masks during sampling from $\pi_{\text{old}}$ and apply them to $\pi_\theta$ during training, ensuring both policies share identical action subspaces." Concretely: during generation, the set of tokens that passed the top-p/top-k filter is recorded. During training, the current policy's probabilities are renormalized over only that same set of tokens, effectively applying the same truncation. The importance ratio is then computed over matching action spaces.

Why this matters: Importance sampling ($r_{i,t} = \pi_\theta / \pi_{\text{old}}$) requires that both distributions be defined over the same support β€” tokens with zero probability under $\pi_{\text{old}}$ must also have zero probability under $\pi_\theta$, otherwise the ratio is undefined or infinite. By applying the same truncation mask, the effective support is identical for both policies. The paper notes empirically: "combining top-p sampling with the Keep Sampling Mask strategy effectively preserves language consistency during RL training" β€” without it, the model can drift toward generating low-probability tokens that were never seen during rollout generation, degrading output quality.

Thinking in Tool-Use: Context Management, Cold-Start, and Synthesis

The integration challenge. The paper observes that prior approaches to reasoning-in-tool-use were token-inefficient. DeepSeek-R1's strategy was to "discard reasoning content upon the arrival of the second round of messages" β€” meaning the entire chain-of-thought trace was deleted whenever a new message (from the user or a tool) appeared. This forces the model to re-reason through the entire problem context for each subsequent tool interaction, wasting tokens on redundant computation and potentially losing continuity. The paper's context management strategy addresses this directly.

Thinking context management. The paper defines a simple rule-based policy for when to retain versus discard reasoning traces, illustrated in Figure 4:

  • Reasoning content is discarded only when a new user message enters the conversation. The rationale: a new user message represents a genuinely new query or instruction, so the previous reasoning trace is no longer relevant and would only waste context space.
  • If only tool-related messages (tool outputs) are appended, the reasoning content is retained throughout the interaction. The rationale: tool outputs are continuations of the same task, so the reasoning trace remains relevant β€” the model should remember what it was thinking about when it called the tool.
  • When reasoning traces are removed (on new user messages), the history of tool calls and their results remains preserved in context. This ensures the model doesn't lose track of what actions it has taken, even if it forgets its internal reasoning about why.

What this enables: In a multi-turn agent interaction, the model calls a tool, receives output, calls another tool based on that output, and so on β€” all within the same "thinking episode." The reasoning trace persists across these tool interactions, allowing the model to maintain a coherent chain of thought that references both its own earlier reasoning and the accumulating tool results. Only when the human user actually says something new does the model clear its mental scratchpad.

Compatibility caveat. The paper notes: "certain agent frameworks, such as Roo Code or Terminus, simulate tool interactions via user messages." In these frameworks, tool outputs are inserted as if they came from the user, which triggers the reasoning-discard rule. Therefore, the thinking retention mechanism "may not fully benefit" from these frameworks, and the paper "recommend[s] utilizing non-thinking models for optimal performance with such architectures." This is a practical deployment consideration β€” the context management strategy is designed for frameworks that distinguish tool messages from user messages, and third-party frameworks that don't make this distinction won't get the benefit.

Cold-start: seeding reasoning-in-tool-use behavior. The paper observes that they have two types of data available: reasoning data (non-agentic, where models produce chain-of-thought before answers) and non-reasoning agentic data (where models use tools but don't explicitly reason). Neither alone teaches the model to interleave reasoning and tool calls. The cold-start approach bridges this gap through prompt engineering rather than new data collection.

The paper demonstrates this with three illustrative prompts in Appendix Tables 6-8:

  1. Reasoning-only prompt (Table 6): The system prompt instructs the model to "first reason before giving the final answer" and to enclose reasoning in <think> ... response tags. The model learns to produce reasoning traces for non-agentic problems.

  2. Agent-only prompt (Table 7): The system prompt describes available tools and their format, instructing the model to use them. The model learns to call tools but without explicit reasoning traces.

  3. Reasoning-in-agent prompt (Table 8): The system prompt explicitly instructs the model: "You may use the Python tool multiple times during your reasoning, a.k.a in thinking ... response." The prompt tells the model to call tools inside its thinking process, continue reasoning after seeing tool outputs, and only present the final answer after the reasoning is complete.

What happens during cold-start: The model, having been trained on both reasoning-only and agent-only data, can follow the composite prompt. It already knows how to reason (from the reasoning data) and how to call tools (from the agent data). The composite prompt simply tells it to do both in the same trajectory β€” reason, call a tool when needed, reason about the output, call another tool, and so on. The paper describes the outcome: "the model is occasionally able to generate the desired trajectories, thereby providing a basis for subsequent reinforcement learning stages."

Why "cold-start": The model is not fine-tuned on interleaved reasoning-and-tool trajectories at this stage. It is simply prompted to produce them zero-shot, relying on its existing capabilities (reasoning from reasoning data, tool-use from agent data) and instruction-following (from general training). The resulting trajectories are imperfect β€” "the reasoning in tool-use patterns may lack robustness" β€” but they provide initial data that RL can then refine. This avoids the chicken-and-egg problem of needing high-quality reasoning-in-tool trajectories to train a model that produces them.

Large-scale agentic task synthesis. To move beyond the fragile cold-start trajectories and generate diverse, high-quality RL training data, the paper develops three distinct synthesis pipelines for different agent types, summarized in Table 1.

Search agent pipeline. This is a multi-agent system using DeepSeek-V3.2 itself:

  1. Entity sampling: "Informative long-tail entities across diverse domains" are sampled from large-scale web corpora. "Long-tail" means these are not the most common entities (which would be too easy) but specific, information-rich ones that require non-trivial search to answer questions about.

  2. Question construction: An agent explores each entity using "search tools with configurable depth and breadth parameters" β€” it can follow links, read multiple pages, and gather information at varying levels of thoroughness. It consolidates this into question-answer pairs.

  3. Answer generation: "Multiple answer-generation agents with heterogeneous configurations (different checkpoints, system prompts, etc.) produce diverse candidate responses" for each QA pair. Using different model variants ensures the incorrect answers are plausible and diverse, not just copies of the same error.

  4. Verification: A verification agent with search capabilities "validates all answers through multiple passes, retaining only samples where the ground-truth is correct and all candidates are verifiably incorrect." The verification agent can search to fact-check answers independently. The policy of only keeping examples where all distractor answers are verifiably incorrect (not just assumed incorrect) ensures high-quality negative examples for RL β€” the model learns from clear mistakes, not ambiguous ones.

  5. Augmentation: To cover use cases where factual verification isn't possible, the dataset is augmented with "filtered instances from our existing helpful RL datasets, for which the search tool provides measurable benefits." These are tasks where search clearly helps (e.g., finding current information) but correctness isn't binary.

  6. Reward modeling: "Detailed evaluation rubrics across multiple quality dimensions" are developed, and "a generative reward model" scores responses. This hybrid approach means the model optimizes for both verifiable correctness (did it find the right answer?) and practical helpfulness (is the response well-structured, comprehensive, and useful?).

Code agent pipeline. This constructs executable software issue resolution environments from GitHub:

  1. Data mining: "Millions of issue-Pull Request (PR) pairs" are mined from GitHub. Each pair consists of a bug report (issue) and the code changes that fixed it (PR).

  2. Quality filtering: The pairs are "rigorously filtered using heuristic rules and LLM-based judgments." Requirements: each entry must contain "a reasonable issue description, a correlated gold patch, and a test patch for validation." The gold patch is the actual fix; the test patch contains tests that verify the fix works.

  3. Environment setup: "An automated environment-setup agent, powered by DeepSeek-V3.2" builds executable environments. This agent "handles package installation, dependency resolution, and test execution." The outputs use "standard JUnit format, ensuring consistent parsing across programming languages and test frameworks."

  4. Validation criterion: An environment is "deemed successfully built only when applying the gold patch results in a non-zero count of false-to-positive (F2P) test cases (indicating the issue is fixed) and a zero count of pass-to-fail (P2F) test cases (indicating no regressions)." F2P tests are tests that fail before the fix and pass after β€” demonstrating the bug existed and was fixed. P2F tests are tests that pass before the fix and fail after β€” indicating the fix broke something. Zero P2F ensures the gold patch is a clean fix.

  5. Scale: "Tens of thousands of reproducible issue resolution environments spanning multiple programming languages, including Python, Java, JavaScript, TypeScript, C, C++, Go, and PHP." This multi-language coverage is important for generalization β€” the model shouldn't overfit to Python-specific patterns.

Code interpreter agent pipeline. This uses Jupyter Notebook as an execution environment for problems that require code to solve. The paper "curate[s] a diverse set of problems spanning mathematics, logic, and data science, each requiring the model to leverage code execution capabilities to arrive at a solution." Unlike the code agent pipeline (which focuses on fixing existing code), this pipeline targets problems where code is a means to an end β€” computing a mathematical result, analyzing data, or solving a logic puzzle programmatically.

General agent pipeline. This is the most novel synthesis pipeline, producing entirely synthetic environments with automatically generated verifiers. The workflow:

  1. Environment and toolset construction: Given a task category (e.g., "planning a travel itinerary") and a sandbox with a bash shell and search tool, "the agent first uses these tools to generate or retrieve relevant data from the Internet and store them in the sandbox database." This creates a realistic database β€” e.g., actual hotels, restaurants, attractions with real prices, ratings, and locations β€” but stored locally so the environment is reproducible and doesn't require live API calls during RL.

  2. Tool synthesis: The agent "synthesizes a set of task-specific tools, each implemented as a function." These are Python functions that query the database β€” e.g., get_all_hotels_by_city(city), get_inter_city_transport(from_city, to_city), get_weather_by_city_date(city, date). The toolset is designed to be sufficient for solving the task but leaves the challenge of tool selection and sequencing to the model.

  3. Task creation with iterative difficulty escalation: The agent "initially proposes a simple task based on the current database, along with its solution and verification functions implemented in Python." The solution function is restricted to "invoking tool functions or performing logical computations, and cannot call other functions or directly access the database" β€” ensuring the task can only be solved through the intended tool interface, not by bypassing it. The verification function checks whether the solution's output satisfies all constraints. "If the solution is not validated, the agent will modify the solution or verification functions until the solution's output passes the verification." Then the agent "iteratively increases the difficulty of the task and updates the corresponding solution and verification functions." If the tools are insufficient for the harder version, "the agent will augment the toolset."

  4. RL filtering: The paper runs RL on the synthesized data "using DeepSeek-V3.2 and retain[s] only instances with non-zero pass@100" β€” meaning the base model, after sampling 100 attempts, can solve the task at least once. This ensures the tasks are not impossibly hard (which would provide no learning signal) but are challenging enough that the model needs RL to reach reliable performance.

  5. Scale: "1,827 environments and their corresponding tasks (4,417 in total)." Each environment is a distinct scenario (different travel destination, different constraints), and each can host multiple tasks of varying difficulty.

The trip-planning example illustrates the synthesis quality. The task requires a three-day itinerary with constraints that nest conditionally: the hotel price on day 2 determines which restaurant rating and attraction ticket price constraints apply. This is genuinely hard to solve β€” it requires exploring a large combinatorial space (cities Γ— hotels Γ— restaurants Γ— attractions Γ— transport options) with conditional constraints β€” but easy to verify: the verification function simply checks each constraint against the submitted itinerary. The toolset (get_all_attractions_by_city, get_city_transport, etc.) forces the model to interact with the environment through a defined interface, mirroring real-world API interactions.

Why this matters for generalization: The paper emphasizes that "the environments and toolsets employed in these benchmarks were not encountered during RL training" (Section 4.1), so improvements on MCP-Universe and MCP-Mark represent generalization to novel tools and environments. The synthesis pipeline's diversity β€” 1,827 distinct environments across multiple task categories β€” is what drives this generalization. A model trained on only a few hand-crafted environments would overfit to their specific patterns.

Specialist Distillation and Mixed RL Training

Specialist training. Before training the final generalist model, the paper trains eight specialized models, each fine-tuned from the same DeepSeek-V3.2 base checkpoint. The domains are: mathematics, programming, general logical reasoning, general agentic tasks, agentic coding, agentic search, writing, and general question-answering. Each specialist supports both "thinking mode" (with chain-of-thought) and "non-thinking mode" (direct responses). Each specialist is "trained with large-scale Reinforcement Learning (RL) computing" β€” meaning each gets its own substantial RL budget, not just supervised fine-tuning.

Why specialists before generalist: This is a distillation strategy. A specialist trained exclusively on math can invest all its RL compute into improving mathematical reasoning, reaching higher performance than a generalist that must balance math, code, agent tasks, and alignment simultaneously. The specialist generates high-quality training data β€” its outputs are better than what the base model produces β€” and this data, when combined, provides a diverse curriculum for the generalist. The paper states: "Experimental results demonstrate that models trained on the distilled data achieve performance levels only marginally below those of domain-specific specialists, with the performance gap being effectively eliminated through subsequent RL training." The generalist starts from the specialist-generated data (which is near specialist quality), then the mixed RL training closes the remaining gap.

Data generation differences by mode. The paper specifies: "we employ different models to generate training data for long chain-of-thought reasoning (thinking mode) and direct response generation (non-thinking mode)." This means that thinking-mode specialists produce long reasoning traces with intermediate steps, while non-thinking specialists produce concise direct answers. The generalist learns both modes from this data.

Mixed RL training. For the final DeepSeek-V3.2, "we merge reasoning, agent, and human alignment training into one RL stage." This is in contrast to multi-stage approaches where the model is first RL-trained on reasoning, then on agent tasks, then aligned β€” which risks catastrophic forgetting of earlier skills when later stages optimize different objectives. The paper claims this single-stage approach "effectively balances performance across diverse domains while circumventing the catastrophic forgetting issues commonly associated with multi-stage training paradigms."

The generalist undergoes "thousands of steps of continued RL training" on the combined distilled data from all specialists. The reward structure is the same as described above: rule-based rewards for reasoning and agent tasks, generative reward models for general alignment tasks, with length penalties and language consistency rewards across the board.

DeepSeek-V3.2-Speciale variant. This is an experimental variant that diverges from the generalist in two ways: (1) it is "trained exclusively on reasoning data" (no agent or alignment data), and (2) it uses "a reduced length penalty during RL," allowing it to produce much longer reasoning traces. Additionally, it incorporates "the dataset and reward method from DeepSeekMath-V2 (Shao et al., 2025) to enhance capabilities in mathematical proofs." This variant is designed to investigate the upper bound of reasoning performance when token efficiency is not a constraint. The tradeoff is visible in Table 3: DeepSeek-V3.2-Speciale uses 23K tokens on AIME 2025 (vs. 16K for the standard thinking model and 15K for Gemini-3.0-Pro), 27K on HMMT Feb 2025 (vs. 19K for standard and 16K for Gemini-3.0-Pro), and achieves higher accuracy β€” but at substantially higher cost.

Design rationale for the two-tier release. The standard DeepSeek-V3.2 optimizes for cost efficiency (hence the length penalty in RL), while DeepSeek-V3.2-Speciale optimizes for maximum capability regardless of cost. This two-tier approach acknowledges that token efficiency is a practical constraint for deployment β€” "the token efficiency of DeepSeek-V3.2-Speciale remains significantly inferior to that of Gemini-3.0-Pro" β€” while also demonstrating what the architecture and training recipe can achieve when unconstrained. The standard model is a practical tool; the Speciale variant is an existence proof.

4. Key Insights and Innovations

Innovation 1: Difficulty-Aware Resource Allocation as an Inference-Time Scaling Paradigm β€” But for Architecture, Not Strategy

On its surface, this paper appears to contribute three independent technical advances: a sparse attention mechanism, a stabilized RL recipe, and an agentic data synthesis pipeline. But the deeper intellectual move β€” the one that makes the paper cohere rather than read as three separate papers β€” is an implicit difficulty-aware resource allocation framework that spans all three contributions. The paper never names this framework explicitly, but it operates throughout: DSA allocates compute non-uniformly across tokens (spending more computation on the top-k most relevant context tokens), the scalable GRPO recipe allocates gradient updates non-uniformly across training samples (masking highly off-policy negative sequences), and the agentic synthesis pipeline allocates environment complexity non-uniformly across tasks (iteratively escalating difficulty until the base model's pass@100 is just above zero).

This is intellectually distinctive because it inverts the standard scaling assumption. Prior work on LLM scaling β€” both pretraining scaling laws (Kaplan et al., 2020; Hoffmann et al., 2022) and inference-time scaling (the best-of-N and beam search paradigms analyzed in depth by the example paper on compute-optimal test-time scaling) β€” assumed that more compute should be applied uniformly: more parameters, more data, more samples, more search budget, all distributed evenly across inputs. The example paper's key insight was that optimal test-time compute strategies vary by prompt difficulty, requiring adaptive allocation between search algorithms. DeepSeek-V3.2 extends this principle in a more fundamental direction: rather than allocating which strategy to use based on difficulty, it builds difficulty-adaptive allocation into the architecture itself (via learned token selection) and the optimization process (via selective gradient masking). This is not a strategy selection problem but an architectural and algorithmic embodiment of the principle that not all computation is equally valuable.

The significance beyond performance numbers is conceptual: it suggests that the next generation of efficient LLMs should not aim to apply the same computation to every input uniformly, but should learn where computation matters β€” which tokens in a long context, which training samples in an RL batch, which environment complexities in a curriculum β€” and concentrate resources there. The DSA indexer, trained via KL distillation from dense attention, is effectively a learned compute allocator that decides "which 2048 tokens per query are worth the expensive full-dimensional attention." The off-policy sequence mask is a learned gradient allocator that decides "which negative samples are still informative given how much the policy has changed." The synthesis pipeline's pass@100 filter is a curriculum allocator that decides "which tasks are in the productive learning zone." These are three manifestations of the same principle at different levels of the training stack.

This is a fundamental reframing, not an incremental refinement. Prior work treated efficiency as architectural (sparse attention patterns, MoE routing) or algorithmic (importance sampling corrections) but not as a unified allocation problem. The paper does not make this framing explicit β€” it's an emergent property of reading all three technical contributions together β€” but it is the conceptual thread that makes the paper more than a collection of engineering tricks.

Evidence for this interpretation is distributed across the paper: the DSA indexer is trained to match dense attention distributions (Section 2.1.1), the off-policy mask thresholds on KL divergence (Section 3.1), the synthesis pipeline filters for non-zero pass@100 (Section 3.2.3), and the paper explicitly notes that "the environments and toolsets employed in these benchmarks were not encountered during RL training" (Section 4.1) β€” meaning the allocation generalizes rather than overfitting to specific tasks.

Innovation 2: Verifier Over-Optimization as a First-Class Phenomenon in Post-Training β€” Diagnosing Instability as a Grading Problem, Not an Optimization Problem

The paper's analysis of why GRPO becomes unstable at scale (Section 3.1) is superficially a list of four bug fixes: unbiased KL, off-policy masking, keep routing, keep sampling mask. But the deeper intellectual move is a diagnostic reframing of RL instability as a verifier over-optimization problem β€” the post-training analog of the reward hacking phenomenon documented in RLHF and in the example paper's analysis of PRM-guided search.

The dominant assumption in prior post-training work was that RL instability was primarily an optimization problem: the policy updates were too large (hence clipping in PPO/GRPO), the value estimates were too noisy (hence GRPO's group normalization), or the training dynamics were too non-stationary (hence multi-stage training curricula). The solutions targeted the optimizer: better advantage estimates, more conservative updates, curriculum design. DeepSeek-V3.2's stabilization techniques, read together, target something different: misalignment between the signal the optimizer receives and the behavior it should learn. The unbiased KL estimator addresses a case where the gradient signal is systematically wrong (biased toward tokens that became less probable, not toward tokens that should be less probable). The off-policy sequence mask addresses a case where the gradient signal is stale (based on policy behavior that no longer represents the current model). Keep Routing addresses a case where the gradient is misdirected (updating experts that didn't produce the output). Keep Sampling Mask addresses a case where the gradient is incoherent (comparing probabilities over incompatible action spaces).

These are not primarily optimization problems β€” the optimizer (GRPO with clipping) is working correctly given correct gradients. They are signal quality problems: the reward model's scores may be accurate, but the gradient derived from those scores is corrupted by distribution shift, framework inconsistency, or mathematical invalidity. This is intellectually analogous to the example paper's finding that beam search over-optimizes the PRM verifier, producing solutions that score highly but are incorrect β€” the verifier's signal degrades under the distribution shift induced by the search process. Here, the GRPO objective's gradient degrades under the distribution shift induced by the policy update.

The significance beyond raw performance is that it redirects research attention from the optimizer to the signal path. If instability is a signal quality problem, then improving stability is not about better clipping schedules or more conservative trust regions β€” it's about ensuring the importance sampling ratios are mathematically valid, the KL estimates are unbiased, the routing is consistent, and the action spaces match. These are diagnostic tools, not optimizer improvements. The paper demonstrates empirically that these fixes enable scaling post-training compute to "exceed 10% of pre-training cost" β€” a claim about expanding the feasible RL budget, not about achieving a particular benchmark score. The implication is that the barrier to scaling post-training RL for open models was not insufficient compute but insufficient signal integrity.

This is a fundamental diagnostic contribution, not an incremental improvement to GRPO. The four techniques individually may appear as small corrections, but collectively they represent a systematic audit of the gradient computation pipeline, identifying where information is lost or corrupted between the reward signal and the parameter update. This diagnostic methodology β€” tracing instability back to specific mathematical violations (importance sampling over mismatched supports, gradient bias from distribution shift, routing inconsistency from framework divergence) β€” is the real innovation, and it provides a template for debugging RL instability in future MoE models.

Evidence: The paper anchors each technique to a specific failure mode, not a general "improves stability" claim. The off-policy mask explicitly gates on \hat{A}_{i,t} < 0 and \frac{1}{|o_i|} \sum \log(\pi_{\text{old}} / \pi_\theta) > \delta β€” a precise diagnostic condition, not a heuristic. Keep Routing is noted as "crucial for RL training stability of MoE models" and has been used "since DeepSeek-V3-0324" β€” indicating it was discovered empirically as a stability bottleneck, not designed from first principles.

Innovation 3: Synthetic Environment Generation as a Scalable Alternative to Real-World RL Data β€” The "Hard to Solve, Easy to Verify" Principle Applied to Agentic Tasks

Prior work on agentic post-training faced a fundamental scaling tension: real-world environments (GitHub issues, web search, API interactions) provide authentic diversity but are expensive to set up at scale and lack clean verifiability, while hand-crafted environments provide clean verifiability but lack diversity and scale. The paper's general agent synthesis pipeline (Section 3.2.3) resolves this tension through a bootstrapping synthesis procedure that generates environments which are simultaneously diverse (1,827 distinct environments), automatically verifiable (Python verification functions), and calibrated to the model's capability frontier (non-zero pass@100).

The distinctive intellectual move is the iterative difficulty escalation paired with toolset augmentation. A simpler approach β€” generate random environments with random constraints β€” would produce many impossible tasks (wasted RL budget) or trivial ones (no learning signal). The paper's procedure instead starts simple, verifies that the solution works, then escalates difficulty while checking that the environment's tools are still sufficient. If the current toolset cannot solve the harder version, the synthesizer adds tools. This creates a curriculum of environments that are exactly at the boundary of what the tools enable β€” not what the model can solve (which would be a moving target as the model improves), but what the environment makes possible. This boundary is stable because tools are a fixed environmental property, not a model property.

The significance for the field is that this pipeline decouples environment diversity from human design effort. Prior agent benchmarks (SWE-bench, WebArena, MCP-Universe) required substantial human effort to construct, and their scale (hundreds to low thousands of tasks) limited the diversity of RL training data. The synthesis pipeline generates environments programmatically, with human involvement only at the task-category level (e.g., "trip planning," "data analysis"). This shifts agentic post-training from a data-collection problem (gathering real-world interactions at scale) to an environment-generation problem (creating synthetic worlds that exercise the desired capabilities). The distinction matters because environment generation, unlike data collection, can be scaled arbitrarily with compute.

The evidence for the efficacy of this approach is in Figure 5 and Table 5. Table 5 shows that the synthesized tasks are genuinely hard: even GPT-5-Thinking achieves only 62% pass@1 and 82% pass@4 on the general synthesized tasks. Figure 5 shows that RL training exclusively on these synthetic tasks β€” with no real-world code or search data β€” produces substantial improvements on Tau2Bench, MCP-Mark, and MCP-Universe, three benchmarks with real-world tools and environments the model never saw during training. The comparison with DeepSeek-V3.2-Exp (which was trained only on code and search environments and shows no improvement on these benchmarks) demonstrates that the synthetic general agent data drives the generalization, not the RL process itself.

This is a fundamental contribution to agentic training methodology, not an incremental data augmentation. It establishes that synthetic, automatically verifiable environments can substitute for real-world interaction data in RL post-training for agentic capabilities, with the key design principle being the "hard to solve, easy to verify" property β€” the same principle that makes math and code effective RL domains, now extended to open-ended agentic tasks through environment programming rather than task selection.

Innovation 4: Token-Level Resource Partitioning as a Unifying Architectural Principle β€” DSA as Rewriting the Attention Budget, Not Just Reducing It

Sparse attention mechanisms are not new. Prior work explored fixed patterns (sliding window, dilated attention, global+local combinations), learned patterns (routing tokens to buckets, clustering queries and keys), and hardware-aligned patterns (block-sparse, flash attention-compatible sparsity). The paper's DSA could be read as another entry in this lineage β€” a learned indexer that selects top-k tokens β€” and dismissed as an incremental architecture tweak. But this reading misses what's intellectually distinctive: DSA represents a token-level resource partitioning approach where the decision of which computation to perform is itself a learned function, trained via distillation from the model's own dense attention, and operating continuously during both training and inference.

The key conceptual difference from prior sparse attention is the separation of the selection mechanism from the attention computation and the use of self-distillation (dense attention teaching sparse attention) rather than task-loss or hand-crafted sparsity patterns. Most prior learned sparsity trained the selection mechanism end-to-end with the language modeling objective, which creates a chicken-and-egg problem: early in training, the selector makes poor choices, causing the model to miss critical context and receive corrupted gradients. DSA resolves this through the two-stage continued pre-training: first distill the dense model's attention distribution into the indexer (with the main model frozen), then train the full model with sparse attention while continuing to distill only the selected subset. The indexer learns "what matters" from a stable teacher (the dense model) before the sparse model has to rely on its selections.

This is more than a training trick β€” it's a different relationship between the attention mechanism and the token selection mechanism than in prior sparse attention work. In prior work, sparsity was either imposed (fixed patterns) or learned as part of the end-to-end objective (the model learned which tokens to attend to as a byproduct of language modeling). In DSA, the selection mechanism is explicitly trained to reproduce the dense model's attention distribution β€” it is a student of the model's own full attention, not of the task. This means the selection quality is bounded by how well the indexer can approximate the dense attention patterns, not by how useful those selections are for next-token prediction. The paper's parity evaluation (Section 2.2) β€” showing no significant regression on standard benchmarks and actually improved long-context performance β€” validates that this approximation is sufficiently good.

The significance is that this approach provides a migration path from dense to sparse attention that does not require training a sparse model from scratch. Starting from an existing dense checkpoint (DeepSeek-V3.1-Terminus), continued pre-training with DSA produces a sparse model with comparable capabilities. This matters practically because training frontier-scale dense models from scratch is extraordinarily expensive β€” if sparse attention required starting over, the cost might outweigh the inference savings. DSA's continued pre-training approach (943.7B tokens, which is ~1% or less of a full pretraining budget for models at this scale) makes the transition economically viable.

The cost analysis in Figure 3 provides the empirical anchor: at 128K token positions during decoding, DeepSeek-V3.2 costs roughly 0.8 USD per million tokens versus roughly 2.0 USD for DeepSeek-V3.1-Terminus β€” a ~2.5Γ— reduction at maximum context length, with the gap widening as position increases. The speedup is most dramatic exactly where it matters most (long contexts for agentic and reasoning tasks), and the paper demonstrates that the sparse model does not regress on either standard benchmarks or human preference (matched ChatbotArena Elo scores).

This is an incremental architectural contribution (sparse attention is a well-explored area) but a fundamental contribution to the deployment economics of large-scale LLMs when combined with the parity validation. The insight is not "sparse attention can work" β€” that was known from prior work like Yuan et al. (2025) β€” but "you can convert your existing dense model to sparse attention with a modest continued pre-training budget and lose essentially nothing, while gaining 2-4Γ— inference cost reduction at long contexts." This changes the calculus for model deployment: DSA is not just an architecture choice for new models but a retrofit option for existing ones.

Innovation 5: The Two-Tier Capability Demonstration β€” Separating "What's Possible" from "What's Practical" as a Communication Strategy

The paper's release of two model variants β€” DeepSeek-V3.2 (cost-efficient) and DeepSeek-V3.2-Speciale (unconstrained capability) β€” may appear as a straightforward product decision. But it represents an intellectually honest capability-cost Pareto frontier exploration that is unusual in the current LLM landscape. Most model releases either optimize for benchmark performance (reporting the best numbers achievable regardless of cost) or optimize for efficiency (reporting performance under service-level constraints), but rarely both simultaneously with a transparent accounting of the tradeoff.

The conceptual move is to treat token budget as a tunable parameter of the RL objective (via the length penalty) and to demonstrate the resulting accuracy-efficiency curve. Table 3 makes this explicit: for each benchmark, it reports both accuracy and output token count, allowing direct comparison of intelligence density (accuracy per thousand tokens). DeepSeek-V3.2 achieves 93.1% on AIME 2025 using 16K tokens; DeepSeek-V3.2-Speciale achieves 96.0% using 23K tokens β€” a 2.9 percentage point improvement for a 44% increase in token cost. Gemini-3.0-Pro achieves 95.0% using 15K tokens β€” higher accuracy than standard DeepSeek-V3.2 with comparable or lower token usage, demonstrating the remaining efficiency gap that the paper acknowledges.

The significance is that it reframes the capability conversation from "can open models match closed models?" to "at what cost can open models match closed models, and what's the frontier of what's possible regardless of cost?" The standard DeepSeek-V3.2 demonstrates competitive cost-efficiency; the Speciale variant demonstrates that the architecture and training recipe are capable of reaching the frontier (beating GPT-5, matching Gemini-3.0-Pro on reasoning, gold medals in IMO and IOI) when the cost constraint is removed. This two-tier demonstration serves different audiences: practitioners see a cost-competitive model they can deploy; researchers see an upper bound that validates the underlying approach and motivates further work on closing the efficiency gap.

The competitive programming and mathematics olympiad results (Table 4) provide the strongest evidence for the "existence proof" interpretation. DeepSeek-V3.2-Speciale achieves gold medals in IMO 2025 (35/42, gold threshold typically around 28-32), CMO 2025 (102/126), IOI 2025 (492/600), and ICPC World Final 2025 (10/12 problems solved, ranked 2nd). These are not incremental benchmark improvements β€” they represent performance at the highest level of human intellectual competition. The paper explicitly notes these were achieved "without targeted training" for IOI and ICPC (the model was not fine-tuned specifically for competitive programming contests), and the IMO/CMO results incorporate proof-specific techniques from DeepSeekMath-V2. The fact that a general-purpose model (not a competition-specific system) can reach gold-medal level in four distinct elite competitions is a qualitatively different claim than "achieves 93% on AIME."

This is not a technical innovation β€” it's a communication and framing innovation that provides a more complete picture of model capability than a single benchmark table. The two-tier release implicitly acknowledges that the field's standard practice of reporting a single "best" number obscures the cost-performance tradeoff, and by transparently presenting both, the paper enables more informed comparisons. This is incrementally better than single-model reporting but represents a fundamental improvement in how model capabilities should be communicated β€” one that the broader field would benefit from adopting.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. The paper evaluates on a broad suite of benchmarks spanning reasoning, code, mathematics, and agentic tasks. No single dataset drives all conclusions; instead, the paper reports on MMLU-Pro (Wang et al., 2024), GPQA Diamond (Rein et al., 2023), Human Last Exam (HLE) Text-only (Phan et al., 2025), LiveCodeBench (2024.08–2025.04), Codeforces, Aider-Polyglot, AIME 2025, HMMT Feb 2025, HMMT Nov 2025 (BalunoviΔ‡ et al., 2025), IMOAnswerBench (Luong et al., 2025), Terminal Bench 2.0, SWE-Verified (OpenAI, 2024b), SWE Multilingual (Yang et al., 2025), BrowseComp (Wei et al., 2025), BrowseCompZh (Zhou et al., 2025), Ο„2-bench (Barres et al., 2025), MCP-Universe (Luo et al., 2025), MCP-Mark (EvalSys, 2025), and Tool-Decathlon (Li et al., 2025). For the competition-level results in Table 4, the evaluation uses the actual 2025 IMO, CMO, IOI, and ICPC World Final problems. The reasoning benchmarks (AIME, HMMT, IMOAnswerBench, HLE) use the template "{question}\nPlease reason step by step, and put your final answer within \boxed{}." β€” except HLE, where an additional evaluation with the official template yields 23.9 for DeepSeek-V3.2-Thinking.

  • Base model(s). DeepSeek-V3.2 is the primary model, built via continued pre-training from DeepSeek-V3.1-Terminus (a dense 128K-context model) followed by post-training. DeepSeek-V3.2-Thinking is the thinking-mode variant used for reasoning benchmarks. DeepSeek-V3.2-Speciale is an experimental variant trained exclusively on reasoning data with reduced length penalties. The paper also references DeepSeek-V3.2-Exp (the checkpoint after continued pre-training but before the full post-training pipeline) and DeepSeek-V3.2-SFT (the supervised fine-tuning checkpoint before RL) in ablations. The FLOPs-matched comparison typical of scaling-law papers is absent here, replaced by direct benchmark comparisons against frontier proprietary models: GPT-5-High, Claude-4.5-Sonnet, Gemini-3.0-Pro, and open models including Kimi-K2-Thinking and MiniMax-M2.

  • Metrics. Multiple metrics are used per domain: Pass@1 (accuracy of a single sampled answer) for AIME, HMMT, HLE, GPQA Diamond, LiveCodeBench, IMOAnswerBench, BrowseComp, BrowseCompZh, Ο„2-Bench, MCP-Mark, and Tool-Decathlon; EM (Exact Match) for MMLU-Pro; Rating (Elo-based) for Codeforces; Accuracy for Terminal Bench 2.0; Resolved (fraction of issues fixed, the standard SWE-bench metric) for SWE-Verified and SWE Multilingual; Success Rate for MCP-Universe. For the olympiad results in Table 4, competition-specific scoring is used: total points out of maximum (IMO: 35/42, CMO: 102/126, IOI: 492/600) and number of problems solved (ICPC WF: 10/12). For Table 3, each benchmark cell reports both accuracy and output token count in thousands, providing an efficiency dimension.

  • Baselines. The primary baselines are frontier proprietary models: GPT-5-High (OpenAI, 2025), Claude-4.5-Sonnet (Anthropic, 2025b), and Gemini-3.0-Pro (DeepMind, 2025b). Open-model baselines include Kimi-K2-Thinking (MoonShot, 2025) and MiniMax-M2 (MiniMax, 2025). For ablations, DeepSeek-V3.2-Exp and DeepSeek-V3.2-SFT serve as internal baselines to isolate the effects of RL training and data synthesis respectively.

  • Generation budget / compute accounting. The paper measures compute indirectly through output token counts reported alongside accuracy in Table 3 β€” a more deployment-relevant metric than training FLOPs for inference-cost comparisons. For the DSA efficiency analysis (Figure 3), cost is measured in USD per million tokens on H800 GPU clusters at $2/GPU-hour, benchmarked from actual deployed services. For the context management experiments (Figure 6), the x-axis is "Real Steps" β€” the number of execution steps taken by the agent β€” and different strategies are compared at varying step budgets. The paper does not report training FLOPs or GPU-hours for post-training, only noting that the RL budget "exceeds 10% of pre-training cost."

  • Cross-validation / statistical protocol. No formal cross-validation or statistical significance testing is reported. For tool-use benchmarks (MCP-Universe, MCP-Mark), the paper evaluates "all models with our internal environment, because the search and playwright environment might be slightly different from the official setting" β€” introducing a potential confound where differences could arise from environment mismatch rather than model capability. For code agent evaluations, robustness is assessed across frameworks: SWE-bench Verified scores are reported for the internal framework (73.1) and noted to range from 72–74 across Claude Code, RooCode, and non-thinking mode. For Terminal Bench 2.0, the primary score of 46.4 uses the Claude Code framework (since the thinking-mode context management is incompatible with Terminus), while the non-thinking Terminus score is 39.3. The IMO/CMO evaluation uses a "generate-verify-refine loop" with iterative improvement until perfect self-evaluation or a revision cap, identical to Shao et al. (2025). For IOI, a multi-stage filtering pipeline selects 50 submissions from 500 candidates; for ICPC, 32 candidates are generated. No confidence intervals are reported for any benchmark.

Main Quantitative Results

Standard Benchmark Performance (Table 2)

Table 2 presents the comprehensive benchmark comparison. The headline finding is that DeepSeek-V3.2 achieves performance comparable to GPT-5-High on reasoning benchmarks while lagging behind Gemini-3.0-Pro, and substantially advances open-model agentic capabilities. Specific numbers:

English/Language: On MMLU-Pro, DeepSeek-V3.2 scores 85.0, behind GPT-5 (87.5) and Gemini-3.0-Pro (90.1), but ahead of Kimi-K2-Thinking (84.6) and MiniMax-M2 (82.0). On GPQA Diamond, the gap widens: DeepSeek-V3.2 scores 82.4 versus 85.7 (GPT-5), 91.9 (Gemini-3.0-Pro), and 84.5 (Kimi-K2). On HLE, DeepSeek-V3.2 scores 25.1, comparable to GPT-5 (26.3) and ahead of Kimi-K2 (23.9) and Claude-4.5-Sonnet (13.7), but substantially behind Gemini-3.0-Pro (37.7).

Code: On LiveCodeBench, DeepSeek-V3.2 scores 83.3, nearly matching GPT-5 (84.5) and Kimi-K2 (82.6), but trailing Gemini-3.0-Pro (90.7). On Codeforces, DeepSeek-V3.2 achieves a rating of 2386, behind GPT-5 (2537) and Gemini-3.0-Pro (2708) β€” the paper notes this is a substantial gap, with the Speciale variant reaching 2701. Codeforces ratings for Kimi-K2 and MiniMax-M2 are not reported.

Mathematics: This is DeepSeek-V3.2's strongest domain. On AIME 2025, it scores 93.1 β€” close to GPT-5 (94.6), Gemini-3.0-Pro (95.0), and Kimi-K2 (94.5), and substantially ahead of MiniMax-M2 (78.3) and Claude-4.5-Sonnet (87.0). On HMMT Feb 2025, it scores 92.5, ahead of GPT-5 (88.3) and Kimi-K2 (89.4) but behind Gemini-3.0-Pro (97.5). On HMMT Nov 2025, it scores 90.2, comparable to GPT-5 (89.2) and Kimi-K2 (89.2), behind Gemini-3.0-Pro (93.3). On IMOAnswerBench, it scores 78.3, matching Kimi-K2 (78.6) but trailing GPT-5 (76.0) and Gemini-3.0-Pro (83.3).

Code Agent: On Terminal Bench 2.0, DeepSeek-V3.2 scores 46.4 (using Claude Code framework due to Terminus compatibility issues), outperforming GPT-5 (35.2) and Kimi-K2 (35.7) but behind Gemini-3.0-Pro (54.2). On SWE-Verified, it scores 73.1, close to GPT-5 (74.9) and Gemini-3.0-Pro (76.2), and ahead of Kimi-K2 (71.3) and Claude-4.5-Sonnet (77.2 β€” note Claude leads here). On SWE Multilingual, DeepSeek-V3.2 leads all reported models at 70.2, ahead of Claude-4.5-Sonnet (68.0) and GPT-5 (55.3).

Search Agent: On BrowseComp, DeepSeek-V3.2 scores 51.4 without context management, 67.6 with context management β€” the latter beating GPT-5 (54.9) and being noted as comparable to Gemini-3.0-Pro's context-managed score (60.2). On BrowseCompZh, it scores 65.0, leading GPT-5 (63.0) and Kimi-K2 (62.3). On HLE (search agent evaluation, presumably with browsing capability), DeepSeek-V3.2 scores 40.8, ahead of GPT-5 (35.2) and Kimi-K2 (44.9), but behind Gemini-3.0-Pro (45.8).

Tool-Use: On Ο„2-Bench, DeepSeek-V3.2 scores 80.3, comparable to GPT-5 (80.2) and Claude-4.5-Sonnet (84.7), behind Gemini-3.0-Pro (85.4). On MCP-Universe, it scores 45.9, close to Claude-4.5-Sonnet (46.5) and GPT-5 (47.9), behind Gemini-3.0-Pro (50.7), and substantially ahead of Kimi-K2 (35.6). On MCP-Mark, it scores 38.0, behind GPT-5 (50.9) and Gemini-3.0-Pro (43.1), but substantially ahead of Kimi-K2 (20.4). On Tool-Decathlon, it scores 35.2, comparable to Gemini-3.0-Pro (36.4) and ahead of GPT-5 (29.0) and Kimi-K2 (17.6).

The paper emphasizes: "DeepSeek-V3.2 achieves similar performance with GPT-5-high on reasoning tasks, but is slightly worse than Gemini-3.0-Pro." The gap to Gemini-3.0-Pro is most pronounced on GPQA Diamond (82.4 vs. 91.9), HLE (25.1 vs. 37.7), and LiveCodeBench (83.3 vs. 90.7) β€” benchmarks that reward deeper reasoning or broader knowledge.

Efficiency Comparison (Table 3)

Table 3 adds the critical efficiency dimension, reporting both accuracy and output token count. Key comparisons:

DeepSeek-V3.2 vs. GPT-5-High: On AIME 2025, DeepSeek-V3.2 achieves 93.1 with 16K tokens versus GPT-5's 94.6 with 13K tokens β€” GPT-5 is both more accurate and more efficient. On HMMT Feb 2025, DeepSeek-V3.2 scores 92.5 with 19K tokens versus GPT-5's 88.3 with 16K β€” DeepSeek-V3.2 is more accurate but less efficient. On LiveCodeBench, DeepSeek-V3.2 scores 83.3 with 16K tokens versus GPT-5's 84.5 with 13K β€” GPT-5 wins on both axes. On HLE, DeepSeek-V3.2 scores 25.1 with 21K tokens versus GPT-5's 26.3 with 15K β€” GPT-5 wins on both axes.

DeepSeek-V3.2 vs. Gemini-3.0-Pro: The efficiency gap is stark. Gemini-3.0-Pro achieves higher accuracy with fewer tokens on nearly every benchmark: AIME (95.0 with 15K vs. 93.1 with 16K), HMMT Feb (97.5 with 16K vs. 92.5 with 19K), HMMT Nov (93.3 with 15K vs. 90.2 with 18K), LiveCodeBench (90.7 with 13K vs. 83.3 with 16K), GPQA Diamond (91.9 with 8K vs. 82.4 with 7K β€” similar tokens, substantially higher accuracy), HLE (37.7 with 15K vs. 25.1 with 21K). DeepSeek-V3.2 is simultaneously less accurate and less token-efficient than Gemini-3.0-Pro β€” a compound disadvantage.

DeepSeek-V3.2 vs. DeepSeek-V3.2-Speciale: The Speciale variant consistently achieves higher accuracy at the cost of more tokens: AIME (96.0 with 23K vs. 93.1 with 16K), HMMT Feb (99.2 with 27K vs. 92.5 with 19K), LiveCodeBench (88.7 with 27K vs. 83.3 with 16K), HLE (30.6 with 35K vs. 25.1 with 21K). The accuracy-token tradeoff is roughly linear: on AIME, +2.9 percentage points for +44% tokens; on HMMT Feb, +6.7 points for +42% tokens; on HLE, +5.5 points for +67% tokens.

DeepSeek-V3.2 vs. Kimi-K2-Thinking: DeepSeek-V3.2 achieves comparable accuracy with substantially fewer tokens. On AIME: 93.1 (16K) vs. 94.5 (24K) β€” similar accuracy, 33% fewer tokens. On HMMT Feb: 92.5 (19K) vs. 89.4 (31K) β€” higher accuracy, 39% fewer tokens. On HLE: 25.1 (21K) vs. 23.9 (24K) β€” higher accuracy, fewer tokens. The paper frames this explicitly: "Compared to K2-Thinking, DeepSeek-V3.2 achieves comparable scores with substantially fewer output tokens."

The paper notes the Speciale variant's efficiency deficit: "the token efficiency of DeepSeek-V3.2-Speciale remains significantly inferior to that of Gemini-3.0-Pro" β€” Gemini-3.0-Pro achieves 95.0 on AIME with 15K tokens, while DeepSeek-V3.2-Speciale achieves 96.0 with 23K (53% more tokens for 1 point of accuracy).

Competition-Level Results (Table 4)

Table 4 reports DeepSeek-V3.2-Speciale's performance in elite competitions, representing a qualitative capability demonstration beyond benchmark scores:

IMO 2025: Scores 35 out of 42, achieving gold medal. Individual problem scores: 7, 7, 7, 7, 7, 0 β€” near-perfect on five problems, zero on the sixth. Gold threshold for IMO is typically 28–32, so 35 is comfortably above the cutoff.

CMO 2025: Scores 102 out of 126, achieving gold medal. Individual problem scores: 18, 18, 9, 21, 18, 18 β€” strong across all problems. The paper notes this evaluation used the English version of CMO 2025, with the dataset and reward method from DeepSeekMath-V2 (Shao et al., 2025).

IOI 2025: Scores 492 out of 600, achieving gold medal and ranking 10th overall. Individual problem scores: 100, 82, 72, 100, 55, 83. The submission strategy generates 500 candidate solutions per problem, filters out invalid submissions (failed sample tests or exceeded length limits) and submissions where the model "indicated an inability or refusal to solve the problem," then selects the 50 with the longest thinking traces for final submission.

ICPC World Final 2025: Solves 10 out of 12 problems, achieving gold medal and ranking 2nd. Individual problems solved (with submission count in parentheses): A (3 submissions), C (1), D (1), E (2), F (2), H (1), I (1), J (1), K (1), L (1) β€” problems B and G were not solved. Note that most solved problems required only 1–2 submissions, with problem A being the notable exception at 3 submissions.

The paper emphasizes: "this general-purpose model attains gold-medal level performance in the 2025 International Olympiad in Informatics (IOI) and the ICPC World Finals (ICPC WF) without targeted training" β€” meaning the Speciale variant was not fine-tuned specifically for competitive programming. The IMO/CMO results incorporate DeepSeekMath-V2's proof techniques. For IMO/CMO specifically, a "generate-verify-refine loop" is used where "the model iteratively improves its solution until it achieves a perfect self-evaluation or hits the maximum revision cap."

Thinking vs. Non-Thinking Agentic Evaluation (Appendix Table 9)

The paper compares thinking and non-thinking modes on agentic benchmarks:

BenchmarkNon-ThinkingThinking
Terminal Bench 2.037.146.4
SWE Verified72.173.1
SWE Multilingual68.970.2
Ο„2-Bench77.280.3
MCP-Universe38.645.9
MCP-Mark26.538.0
Tool-Decathlon25.635.2

The thinking mode consistently outperforms non-thinking mode, with the largest gaps on MCP-Mark (+11.5), Terminal Bench 2.0 (+9.3), and MCP-Universe (+7.3). The paper notes: "The performance of non-thinking mode is slightly worse than the thinking mode, but still competitive" β€” though gaps of 9–11 points on some benchmarks are more than "slightly" worse and suggest thinking is essential for complex tool-use scenarios. The SWE-bench scores show the smallest thinking advantage (+1.0–1.3 points), suggesting code-editing tasks benefit less from explicit reasoning traces than open-ended agent tasks.

Agentic Task Synthesis Ablation (Table 5 and Figure 5)

The paper evaluates the synthesized general agent tasks by testing multiple models on a random sample of 50 instances:

Pass@KDeepSeek-V3.2-ExpSonnet-4.5Gemini-3.0-ProGPT-5-Thinking
112%34%51%62%
218%47%65%75%
426%62%74%82%

These results demonstrate the tasks are genuinely challenging: even GPT-5-Thinking (the strongest model tested) achieves only 62% pass@1 and 82% pass@4 on these synthesized tasks. DeepSeek-V3.2-Exp achieves only 12% pass@1, confirming the tasks are far from trivial for the base model.

Figure 5 shows the effect of RL training exclusively on these synthetic tasks. Starting from DeepSeek-V3.2-SFT, RL on synthetic general agent data (in non-thinking mode) produces substantial improvements on Tau2Bench, MCP-Mark, and MCP-Universe β€” all benchmarks with real-world tools and environments not seen during training. In contrast, DeepSeek-V3.2-Exp (trained with RL only in search and code environments) shows no improvement over the SFT baseline on these benchmarks. The paper states: "restricting RL to code and search scenarios does not improve performance on these benchmarks, further highlighting the potential of synthetic data."

The precise numbers are not given in the text for Figure 5, but the visual shows improvements of approximately 5–15 percentage points across the three benchmarks for the synthetic-data RL run, with MCP-Mark showing the steepest improvement curve over RL steps.

Context Management Strategies for Search Agents (Figure 6)

Figure 6 evaluates four test-time compute expansion strategies on BrowseComp at varying step budgets:

Summary: Summarizes overflowed trajectory and re-initiates rollout. Extends average steps to 364, achieving 60.2 accuracy β€” the lowest efficiency (steps per accuracy point) of the four strategies, but demonstrates that context management enables continued improvement.

Discard-75%: Discards the first 75% of tool call history. The figure shows this performs similarly to Summary at moderate step counts but appears to plateau earlier.

Discard-all: Resets context by discarding all previous tool call history (similar to Anthropic's "new context tool"). Achieves 67.6 accuracy β€” the highest of all serial strategies β€” while using significantly fewer steps than Summary. The paper notes this "performs well in both efficiency and scalability, achieving a score of 67.6, comparable to parallel scaling while using significantly fewer steps."

Parallel-fewest-step: Samples N independent trajectories and selects the one with the fewest steps. Achieves the highest accuracy at equivalent real-step budgets, matching Discard-all's performance with different scaling characteristics.

The baseline score without context management is 51.4 (as reported in Table 2). Discard-all therefore provides a +16.2 point improvement. The paper concludes: "test-time compute can be scaled either serially through context management or in parallel, both effectively extending the model's problem-solving capacity. However, different strategies exhibit varying efficiency and scalability."

Parity Evaluation of DSA (Section 2.2)

The paper reports that DeepSeek-V3.2-Exp (the checkpoint after continued pre-training with DSA, before post-training) achieves "similar performance" to DeepSeek-V3.1-Terminus on standard benchmarks evaluated in September 2025. Specific benchmark numbers are not given in the text β€” the evaluation is described qualitatively: "we do not observe substantial performance degradation compared with DeepSeek-V3.1-Terminus, on both short- and long-context tasks."

On long-context evaluations conducted by independent parties:

  • AA-LCR: DeepSeek-V3.2-Exp "scores four points higher than DeepSeek-V3.1-Terminus in reasoning mode."
  • Fiction.liveBench: DeepSeek-V3.2-Exp "consistently outperforms DeepSeek-V3.1-Terminus across multiple metrics."

On human preference, both models "share an identical post-training strategy, and their Elo scores, obtained from evaluations conducted on 10 November 2025, are closely matched." This ChatbotArena evaluation provides an indirect human-preference signal, suggesting the switch to sparse attention did not degrade response quality as perceived by users.

Ablation Studies and Robustness Checks

Indexer warm-up necessity (Section 2.1.1, implicit): The two-stage continued pre-training design (dense warm-up for 1000 steps before sparse training for 15000 steps) is itself an ablation on whether the indexer can be trained jointly with the main model from the start. The paper does not report results for skipping the warm-up stage, but the design choice β€” freezing all model parameters except the indexer during warm-up, then unfreezing all parameters β€” implies that joint training from scratch was either unstable or produced worse results. The paper does not quantify the degradation from omitting warm-up.

Indexer head count and dimension selection: No ablation on $H_I$ (number of indexer heads) or $d_I$ (indexer dimension) is reported. The paper states the indexer "has a small number of heads and can be implemented in FP8" but does not specify what "small" means numerically or what happens with fewer or more heads.

Top-k selection value (2048): The choice of $k = 2048$ is stated but not ablated. No results are reported for $k = 1024$, $k = 4096$, or other values. This is a significant gap β€” $k$ is the critical parameter controlling the sparsity-efficiency tradeoff, and the paper provides no evidence that 2048 is optimal or near-optimal.

KL penalty strength by domain (Section 3.1): The paper notes: "In practice, we find that different domains benefit from varying strengths of KL regularization. For certain domains, such as mathematics, applying a relatively weak KL penalty or even omitting it entirely can yield improved performance." This is an observation, not a systematically ablated finding β€” no table compares KL penalty strengths across domains with corresponding performance numbers.

Off-policy sequence masking threshold $\delta$: No ablation on the threshold value. The paper does not report what $\delta$ was set to or how sensitive results are to this hyperparameter.

Cold-start data quality (Section 3.2.2, implicit): The paper states that cold-start trajectories "may lack robustness" but that the model is "occasionally able to generate the desired trajectories." No quantitative assessment of cold-start trajectory quality is provided β€” e.g., what fraction of cold-start trajectories are valid, what fraction exhibit correct interleaving of reasoning and tool calls, or how this compares to trajectories after RL.

Synthetic task difficulty calibration (Section 4.3, Table 5): The paper evaluates the difficulty of synthesized tasks by testing frontier models on a random 50-instance sample. This is a robustness check confirming the tasks are neither trivial nor impossible. The pass@1 rates (12%–62% across models) demonstrate the tasks span a wide difficulty range.

Synthetic data generalization (Figure 5): The RL training on synthetic general agent data produces improvements on Tau2Bench, MCP-Mark, and MCP-Universe β€” benchmarks the model was not trained on. The comparison with DeepSeek-V3.2-Exp (code+search RL only, no improvement) serves as a control: the improvements are specifically attributable to the synthetic general agent data, not to RL training in general or to other components.

ORM vs. PRM aggregation strategy: No ablation on reward model type or aggregation method is reported. The paper states rule-based rewards are used for reasoning/agent tasks and generative rewards for general tasks, but does not compare alternative reward formulations.

Specialist count and domain selection: The paper trains eight specialists but does not ablate the number of specialists (e.g., would four or sixteen work as well?) or the domain partitioning (are mathematics and programming better separated or combined?).

Length penalty strength (DeepSeek-V3.2 vs. Speciale): The difference in length penalty between the standard model and Speciale is the only quasi-ablation on this parameter, but it is confounded with the change in training data (Speciale trains exclusively on reasoning data, standard model trains on reasoning + agent + alignment). The accuracy improvements of Speciale cannot be cleanly attributed to reduced length penalty versus different data mixture.

Non-thinking agentic performance (Appendix Table 9): This ablation shows the effect of disabling thinking mode on agentic benchmarks. The consistent advantage of thinking mode (ranging from +1.0 to +11.5 percentage points) quantifies the contribution of explicit reasoning to tool-use performance.

Framework robustness for SWE-bench (Section 4.1): "Robustness tests across other settings β€” including the Claude Code and RooCode frameworks, as well as non-thinking mode β€” produced consistent results, ranging from 72 to 74." This is a robustness check confirming that the SWE-bench Verified score of 73.1 is not an artifact of the evaluation framework.

ReSTEM^{EM} revision model not applicable to this paper: Unlike the example paper on compute-optimal test-time scaling, DeepSeek-V3.2 does not employ iterative revision models. The IMO/CMO evaluation uses a "generate-verify-refine loop" from Shao et al. (2025) for mathematical proofs, but this is an evaluation strategy, not a training methodology ablated here.

Critical Assessment

Claim 1: DeepSeek-V3.2 "performs comparably to GPT-5" and narrows the open-closed gap

What the experiments demonstrate: Table 2 provides evidence that DeepSeek-V3.2 matches or slightly trails GPT-5-High on most reasoning benchmarks, with the gap being typically 1–3 percentage points (AIME: 93.1 vs. 94.6, LiveCodeBench: 83.3 vs. 84.5, HLE: 25.1 vs. 26.3). On mathematics specifically, DeepSeek-V3.2 actually leads GPT-5 on HMMT Feb 2025 (92.5 vs. 88.3). On agentic benchmarks, DeepSeek-V3.2 outperforms GPT-5 on SWE Multilingual (70.2 vs. 55.3) and BrowseComp (67.6 vs. 54.9 with context management), while trailing on MCP-Mark (38.0 vs. 50.9) and Terminal Bench 2.0 (46.4 vs. 35.2 β€” though the framework difference complicates this comparison). The Speciale variant beats GPT-5 across all reported reasoning benchmarks.

What the experiments do not demonstrate: "Comparable to GPT-5" is an aggregate claim that masks significant variation across benchmarks. DeepSeek-V3.2 is worse than GPT-5 on GPQA Diamond (82.4 vs. 85.7), MMLU-Pro (85.0 vs. 87.5), Codeforces (2386 vs. 2537), and MCP-Mark (38.0 vs. 50.9). On MCP-Mark, the gap is 12.9 points β€” this is not "comparable." The claim also does not address whether GPT-5-High is the appropriate GPT-5 variant for comparison (the "High" suffix suggests multiple compute tiers may exist). More importantly, the claim is about narrowing the open-closed gap, but the paper provides only one time point β€” there is no longitudinal comparison showing the gap was wider previously and is now narrower. The paper's own introduction argues the gap "appears to be widening" β€” the benchmark table alone cannot refute this, since it does not show trends.

Missing experiments: A comparison against GPT-5 at matched token budgets would strengthen the "comparable" claim. Table 3 shows DeepSeek-V3.2 typically uses more tokens than GPT-5 (16K vs. 13K on AIME, 21K vs. 15K on HLE), so the accuracy comparison is not at equal cost. If GPT-5 were allowed 16K tokens on AIME (matching DeepSeek-V3.2's budget), its accuracy might increase further, widening the gap.

Claim 2: DeepSeek-V3.2-Speciale "surpasses GPT-5 and exhibits reasoning proficiency on par with Gemini-3.0-Pro"

What the experiments demonstrate: Table 3 shows DeepSeek-V3.2-Speciale beats GPT-5-High on AIME (96.0 vs. 94.6), HMMT Feb (99.2 vs. 88.3), HMMT Nov (94.4 vs. 89.2), LiveCodeBench (88.7 vs. 84.5), and HLE (30.6 vs. 26.3). It beats Gemini-3.0-Pro on AIME (96.0 vs. 95.0) and HMMT Feb (99.2 vs. 97.5), but trails on HMMT Nov (94.4 vs. 93.3 β€” essentially tied), LiveCodeBench (88.7 vs. 90.7), GPQA Diamond (85.7 vs. 91.9 β€” a substantial 6.2-point gap), and HLE (30.6 vs. 37.7 β€” a 7.1-point gap). The olympiad results (Table 4) demonstrate genuine world-class performance.

What the experiments do not demonstrate: "On par with Gemini-3.0-Pro" on reasoning is accurate for math competition benchmarks (AIME, HMMT) but not for science reasoning (GPQA Diamond) or broad expert knowledge (HLE). The paper acknowledges this implicitly by stating DeepSeek-V3.2 is "slightly worse than Gemini-3.0-Pro" in the main text β€” but the Speciale claim is stronger. The 6–7 point gaps on GPQA and HLE are not "on par" β€” they represent a meaningful capability difference. Moreover, the Speciale variant uses substantially more tokens than Gemini-3.0-Pro (23K vs. 15K on AIME, 27K vs. 16K on HMMT Feb, 35K vs. 15K on HLE). At matched token budgets, the gap would likely be larger. The "on par" claim is accurate only if you ignore token efficiency, which the paper itself identifies as a critical limitation.

Missing experiments: A Gemini-3.0-Pro evaluation at matched token budgets (allowing it to use 23K tokens on AIME, for instance) would test whether its accuracy improves with more compute or is already saturated. Without this, we cannot distinguish "Speciale matches Gemini-3.0-Pro's capability" from "Speciale matches Gemini-3.0-Pro by spending more compute."

Claim 3: DSA "substantially reduces computational complexity while preserving model performance"

What the experiments demonstrate: Figure 3 shows substantial cost reductions at long contexts: at 128K position during decoding, DeepSeek-V3.2 costs ~0.8 USD per million tokens vs. ~2.0 USD for DeepSeek-V3.1-Terminus β€” roughly 2.5Γ— cheaper. The parity evaluation (Section 2.2) reports no significant regression on standard benchmarks and actually improved long-context performance on independent evaluations (AA-LCR: +4 points, Fiction.liveBench: consistently better). ChatbotArena Elo scores are "closely matched." The continued pre-training cost (943.7B tokens) is reported, allowing assessment of the conversion cost.

What the experiments do not demonstrate: The performance preservation claim is supported at the "base model" level (DeepSeek-V3.2-Exp vs. DeepSeek-V3.1-Terminus), but the final DeepSeek-V3.2 also incorporates extensive post-training. We cannot attribute the final model's benchmark performance to DSA specifically β€” the gains could come entirely from the scaled RL and agentic data, with DSA providing efficiency benefits but no accuracy contribution (or even a small accuracy penalty compensated by better post-training). The paper does not report an ablation comparing DeepSeek-V3.2 against an identically post-trained DeepSeek-V3.1-Terminus (without DSA) β€” this is the critical missing experiment. Without it, we don't know whether DSA helps, hurts, or is neutral for final benchmark performance. The DSA contribution is primarily about inference cost, not capability, and the experimental design conflates the architectural change with the post-training improvements.

Missing experiments: An identical post-training run on the dense DeepSeek-V3.1-Terminus checkpoint would cleanly isolate DSA's contribution to final performance. Additionally, the paper does not ablate $k$ (the number of selected tokens), which is the key parameter governing the sparsity-accuracy tradeoff β€” we don't know if 2048 is near-optimal or if the model could achieve similar performance with 1024 (further reducing cost) or needs 4096 (reducing cost savings).

Claim 4: The scalable RL framework enables post-training compute exceeding 10% of pre-training cost

What the experiments demonstrate: The paper shows that DeepSeek-V3.2 achieves strong benchmark performance after "thousands of steps of continued RL training" and states the post-training budget exceeds 10% of pre-training cost. The four stabilization techniques (unbiased KL, off-policy masking, keep routing, keep sampling mask) are described with technical justification.

What the experiments do not demonstrate: There is no ablation showing that any of the four stabilization techniques is necessary for scaling RL to this budget. The paper does not report what happens if you remove the off-policy mask, or use the biased K3 estimator, or disable keep routing, at the actual training scale. The claim that these techniques "enable" scaling is supported by mechanism descriptions and empirical intuition ("we empirically observe that this Off-Policy Sequence Masking operation improves stability"), not by controlled experiments. The claim that post-training exceeds 10% of pre-training cost is stated without specific numbers for either pre-training or post-training compute, making it unverifiable.

Missing experiments: Ablations removing each stabilization technique (or comparing against the original GRPO recipe from DeepSeek-R1) would establish whether they are individually necessary or collectively sufficient. Reporting actual compute numbers (GPU-hours, FLOPs) for both pre-training and post-training would make the 10% claim verifiable. Comparing final performance at different RL budgets (e.g., 5%, 10%, 15% of pre-training cost) would show whether the relationship is saturating or still improving, which is essential for the paper's hypothesis that "reasoning capabilities could be further enhanced with additional computational budget allocation."

Claim 5: The agentic task synthesis pipeline yields "substantial improvements in generalization and instruction-following robustness"

What the experiments demonstrate: Figure 5 and Table 5 provide compelling evidence. Table 5 shows the synthesized tasks are genuinely hard (GPT-5-Thinking: 62% pass@1). Figure 5 shows RL on synthetic data alone transfers to three real-world benchmarks (Tau2Bench, MCP-Mark, MCP-Universe), while RL on code+search environments does not. The paper emphasizes the environments were unseen during training, so the transfer is genuine generalization.

What the experiments do not demonstrate: The magnitude of the generalization effect relative to alternative approaches. Would RL on an equivalent budget of real-world interaction data produce larger or smaller improvements? The paper does not compare synthetic data against human-curated data at matched scale. The improvement curves in Figure 5 show upward trends but no uncertainty estimates β€” we don't know if the improvements are statistically reliable or if they plateau. The 50-instance sample for Table 5 is small and could be unrepresentative. The paper does not report how many of the 4,417 synthesized tasks actually contribute to the learning signal (i.e., how many have non-zero pass@100 after filtering) beyond stating that 1,827 environments were retained.

Missing experiments: A comparison against an equivalent budget of (a) human-curated environment data, (b) real-world interaction data from the same domains, or (c) data from a different synthesis approach would contextualize the synthesis pipeline's effectiveness. A breakdown of which synthesized environments (code interpreter vs. general agent vs. search agent) contribute most to the downstream improvements would guide future synthesis efforts.

Additional Weaknesses in Experimental Design

Single model family, no replication. All results are for DeepSeek-V3.2 and its variants, built on the DeepSeek architecture. The stabilization techniques, DSA, and synthesis pipeline are not validated on other model families. The paper argues these techniques address general bottlenecks in open-model development, but provides evidence from only one architecture.

No confidence intervals or statistical testing. None of the benchmark tables report confidence intervals, standard deviations, or statistical significance. For benchmarks with 500+ questions, 1–2 point differences may be statistically significant; for the 50-instance synthesis evaluation (Table 5), a 5-point difference almost certainly is not. This makes it difficult to assess whether the pattern of "comparable to GPT-5 but behind Gemini-3.0-Pro" reflects genuine capability differences or sampling noise.

Framework dependencies in agentic evaluation. The Terminal Bench 2.0 score depends on the evaluation framework: 46.4 with Claude Code, 39.3 with Terminus in non-thinking mode. The paper attributes this to context management incompatibility, but it means the reported score is not directly comparable to models evaluated under different frameworks. Similar issues affect MCP-Universe and MCP-Mark, which are evaluated with the paper's "internal environment" rather than the official setting.

Context window limitation as a confound. The paper notes that "approximately 20%+ of the test cases exceed this limit" for BrowseComp due to the 128K context window. The context management strategies are then used to derive the final score. This means the BrowseComp evaluation tests both the model's search capability AND the context management strategy, making it difficult to attribute performance to the model versus the context-handling approach. The baseline score without context management is 51.4; with Discard-all, it's 67.6 β€” the +16.2 point improvement could be primarily a context-management effect, not a model capability effect.

Missing open-model comparisons on key benchmarks. Table 2 omits Codeforces ratings for Kimi-K2 and MiniMax-M2. It omits IMOAnswerBench scores for Claude-4.5-Sonnet and MiniMax-M2. These gaps make it harder to assess DeepSeek-V3.2's position relative to the broader open-model landscape.

The 10% post-training claim is unverifiable. Without reporting actual pre-training and post-training FLOPs or GPU-hours, the claim that post-training exceeds 10% of pre-training cost cannot be verified or compared to other models. If proprietary models use 20% or 50% (as the paper implies by arguing open models underinvest in post-training), then 10% may still be insufficient. The specific number matters for the paper's argument about scaling post-training compute.

Olympiad evaluation methodology has selection effects. For IOI, the submission strategy selects the 50 solutions with the longest thinking traces from 500 candidates after filtering. This means the evaluation is not pass@1 β€” it's a best-of-50 after aggressive filtering, which inflates performance relative to what a single sample would achieve. The paper is transparent about this methodology, but the olympiad results should be interpreted as "achievable with compute-intensive post-processing and selection" rather than "the model solves these problems on its first attempt." The ICPC evaluation uses 32 candidates per problem, similarly inflating performance relative to pass@1.

No direct comparison of DSA against alternative sparse attention methods. The paper claims DSA addresses efficiency bottlenecks but does not compare against other sparse attention approaches (e.g., sliding window with global tokens, HashAttention, or the Native Sparse Attention from Yuan et al., 2025 that it cites). The inference cost comparison in Figure 3 is only against DeepSeek-V3.1-Terminus's dense attention, not against alternative sparse patterns. This makes it impossible to assess whether DSA's content-adaptive approach is better than simpler structural sparsity patterns that might achieve similar speedups with less complexity.

6. Limitations and Trade-offs

The Post-Training Compute Claim Is Unverifiable β€” No FLOPs or GPU-Hours Are Reported

The assumption or constraint. The paper's central narrative about post-training scaling hinges on a specific quantitative claim: the RL budget "exceeds 10% of the pre-training cost" (Section 1, Section 3, Section 4.1). This number is used to argue that open models have "insufficient computational investment during the post-training phase" and that scaling this investment is what unlocks frontier reasoning. The paper hypothesizes that "reasoning capabilities could be further enhanced with additional computational budget allocation" (Section 4.1), implying the 10% figure represents a new scaling regime rather than an arbitrary choice. Pre-training cost is never specified β€” DeepSeek-V3.1-Terminus's total training FLOPs are not stated β€” and post-training cost is reported only qualitatively ("thousands of steps of continued RL training," Section 3). There are no GPU-hour counts, no FLOPs estimates, and no comparison to what other models (open or closed) allocate to post-training. The 10% figure functions as a rhetorical anchor with no empirical scaffolding.

The consequence. A practitioner deciding whether to adopt this post-training recipe cannot perform a cost-benefit analysis. If 10% of pre-training cost represents, say, 5Mincompute,thatisfeasibleforsomeorganizationsandimpossibleforothers.Ifitrepresents5M in compute, that is feasible for some organizations and impossible for others. If it represents 50M, the recipe is inaccessible to all but a handful of labs. Without knowing the absolute cost, the paper's claim that "computational expansion during the post-training phase" is the key driver of capability cannot be separated from the possibility that the 10% figure is simply what DeepSeek happened to spend, and that 5% or 20% would produce similar or better results. The paper's own argument that "open-source models suffer from insufficient computational investment during the post-training phase" (Section 1) is undermined by the inability to quantify what "sufficient" means. More fundamentally, the unverifiability of the 10% claim means the paper's central scaling hypothesis β€” that post-training compute is the bottleneck, not architecture or data β€” remains an assertion rather than an empirically supported finding. The field cannot build on this if it cannot measure it.

What evidence exists in the paper. The paper provides zero evidence to support the 10% figure. Section 3 states that specialists are "trained with large-scale Reinforcement Learning (RL) computing" and the generalist undergoes "thousands of steps of continued RL training." Section 4.1 reports: "Over recent months, we have observed consistent performance improvements correlating with extended RL training budget, which already exceeds 10% of the pre-training cost." This is an observation, not an experiment β€” there is no Figure showing how performance scales with RL budget as a fraction of pre-training cost, no ablation comparing different RL budgets (e.g., 5% vs. 10% vs. 15%), and no specification of what "pre-training cost" refers to (total FLOPs? GPU-hours? for DeepSeek-V3.1-Terminus or for the continued pre-training with DSA?). The paper does not even report the number of RL steps for specific specialists or the generalist.

Mitigation status. The paper does not acknowledge this as a limitation, provide any compute accounting, or suggest future work on cost transparency. This is a complete gap β€” the central quantitative claim about scaling has no supporting measurement. The paper's discussion of limitations (Section 5) focuses on token efficiency and world knowledge breadth relative to Gemini-3.0-Pro, not on the verifiability of its own scaling claims.

DSA's Contribution to Final Model Performance Is Not Isolated β€” The Architectural Change Is Confounded with Massive Post-Training Improvements

The assumption or constraint. The paper introduces DeepSeek Sparse Attention (DSA) as the sole architectural innovation (Section 2.1) and demonstrates through parity evaluation (Section 2.2) that the DSA base checkpoint (DeepSeek-V3.2-Exp) achieves "similar performance" to the dense predecessor DeepSeek-V3.1-Terminus on standard benchmarks, with improved long-context performance on independent evaluations (AA-LCR: +4 points, Fiction.liveBench: consistently better). However, the final DeepSeek-V3.2 model differs from DeepSeek-V3.1-Terminus in two massive ways: (1) the DSA architectural change via continued pre-training (943.7B tokens), and (2) the entirely new post-training pipeline β€” specialist distillation across eight domains, mixed RL training at a budget exceeding 10% of pre-training cost, and large-scale agentic task synthesis (85,000 prompts, 1,827 environments). The paper never ablates DSA's contribution to the final benchmark scores (Table 2) by running the identical post-training pipeline on the dense DeepSeek-V3.1-Terminus checkpoint. The paper's implicit assumption is that since DSA preserves base-model parity, the final model's capability gains are attributable to the post-training improvements β€” but this is an assumption, not a finding.

The consequence. A practitioner deciding whether to adopt DSA for their own model cannot determine whether the architecture change improves, degrades, or is neutral for final capability. It is possible that DSA's sparse attention pattern β€” attending to only 2048 out of 128K tokens per query β€” subtly degrades the model's ability to integrate long-range dependencies during post-training RL, and that the scaled post-training budget compensates for (or masks) this degradation. It is equally possible that DSA provides a capability benefit beyond efficiency (the improved long-context evaluations hint at this) that partially explains the final model's strong agentic performance. Without the ablation, the paper's claim that DSA "preserv[es] model performance even in long-context scenarios" (Abstract) is supported only at the base checkpoint level, not for the final model that users actually deploy. The DSA contribution to the paper's headline results is unquantified β€” it could range from "essential enabler" (without DSA's efficiency, the RL budget would have been infeasible) to "neutral carrier" (any sparse attention would have worked) to "mild headwind" (dense attention would have achieved slightly better final results at higher inference cost).

What evidence exists in the paper. Section 2.2 provides parity evaluation for DeepSeek-V3.2-Exp versus DeepSeek-V3.1-Terminus β€” before post-training. The paper notes both models "share an identical post-training strategy, and their Elo scores, obtained from evaluations conducted on 10 November 2025, are closely matched." This is evidence that DSA does not degrade base-model quality, but it does not address whether DSA interacts with the post-training process. The ChatbotArena evaluation uses the same post-training strategy applied to both base checkpoints β€” suggesting some post-training was done on the dense model for comparison β€” but the paper does not report whether this was the full DeepSeek-V3.2 post-training recipe or a lighter-weight version, and no benchmark table compares a fully post-trained dense model against the fully post-trained sparse model. The inference cost analysis in Figure 3 and Section 2.3 demonstrates DSA's efficiency benefits (roughly 2.5Γ— cheaper at 128K context during decoding), but efficiency and final capability are distinct claims.

Mitigation status. The paper does not acknowledge the confounding between DSA and post-training improvements as a limitation. No ablation comparing identically post-trained dense versus sparse models is suggested for future work. The efficiency claim for DSA is well-supported by Figure 3; the capability claim ("preserving model performance") is supported at the base-model level but unverified at the final-model level.

The Model Fundamentally Cannot Handle Hard Problems That Require Genuinely Novel Reasoning β€” Test-Time Compute Provides No Escape

The assumption or constraint. DeepSeek-V3.2's performance is bounded by the base model's capability envelope. The paper demonstrates this pattern across multiple difficulty tiers: on the hardest problems in the synthesized general agent tasks (Table 5), DeepSeek-V3.2-Exp achieves only 12% pass@1 (versus 62% for GPT-5-Thinking); on IMO 2025, even the Speciale variant scores 0/7 on problem 6 (Table 4); on HLE, the Speciale variant achieves only 30.6% (Table 3) despite using 35K tokens per problem β€” the most compute-intensive configuration tested. The paper's context management experiments (Section 4.4, Figure 6) show that scaling test-time compute (via serial or parallel expansion) continues to yield improvements on BrowseComp, but the gains from context management (+16.2 points from Discard-all) are primarily about overcoming the 128K context window limitation rather than enabling fundamentally new reasoning. There is no evidence that additional test-time compute can push the model past its capability ceiling on problems where the base pass@1 is near zero. The paper implicitly acknowledges this boundary in Section 5: "solving complex tasks is still inferior to frontier models, motivating us to further refine our foundation model and post-training recipe."

The consequence. The paper positions DeepSeek-V3.2 as having "narrowed the performance gap between open and frontier proprietary models" (Section 1), but this narrowing is uneven β€” it is concentrated on problems within or near the base model's capability range (mathematics competitions, standard coding tasks) and largely absent on problems requiring novel reasoning or broad world knowledge. On HLE, the gap to Gemini-3.0-Pro is 12.6 points for the standard model and 7.1 points for Speciale β€” these are large gaps on a benchmark specifically designed to test the frontier of human knowledge. On GPQA Diamond (graduate-level science reasoning), the gap is 9.5 points for the standard model and 6.2 points for Speciale. The IMO problem 6 zero score β€” combined with near-perfect scores on problems 1–5 β€” is the most striking illustration: the model can handle the routine-difficulty olympiad problems brilliantly but fails completely on the hardest problem. This pattern is analogous to the example paper's finding that test-time compute provides "near-zero improvement regardless of budget" on difficulty bin 5 problems. The practical implication for deployment is that DeepSeek-V3.2 is highly capable on tasks that resemble those in its training distribution but cannot be relied upon for genuinely novel reasoning β€” and no amount of additional inference compute (longer chains, more samples, more revisions) will close that gap.

What evidence exists in the paper. Table 2 shows absolute scores on hard benchmarks: HLE (25.1 standard, 30.6 Speciale), GPQA Diamond (82.4, 85.7). Table 4 shows the IMO 2025 problem-level breakdown with the zero score on problem 6. Table 3 shows that even with 35K tokens (the highest token budget reported), Speciale achieves only 30.6 on HLE β€” dramatically below Gemini-3.0-Pro's 37.7 with 15K tokens. The gap on HLE is measured and substantial. The context management experiments (Figure 6) show performance improving with more steps on BrowseComp, but BrowseComp tests search and information synthesis, not novel reasoning β€” the tasks are about finding information efficiently, not deriving new mathematical proofs. No experiment tests whether additional inference budget (longer chains, more parallel samples) can push the model past its ceiling on HLE or IMO problem 6.

Mitigation status. The paper acknowledges this limitation in Section 5: "solving complex tasks is still inferior to frontier models" and "the breadth of world knowledge in DeepSeek-V3.2 still lags behind that of leading proprietary models." It suggests scaling pre-training compute as the solution ("We plan to address this knowledge gap in future iterations by scaling up the pre-training compute."). This is an honest admission but also a scope limitation: the paper's recipe β€” DSA + scaled post-training RL + agentic synthesis β€” improves efficiency and refines existing capabilities but does not expand the capability frontier. Users needing genuinely novel reasoning must wait for larger-scale pretraining, not just better post-training recipes.

Token Efficiency Remains a Fundamental Weakness β€” The Model Spends More Tokens Than Proprietary Counterparts to Achieve Comparable or Inferior Accuracy

The assumption or constraint. The paper explicitly identifies token efficiency as a limitation in Section 5: "token efficiency remains a challenge; DeepSeek-V3.2 typically requires longer generation trajectories (i.e., more tokens) to match the output quality of models like Gemini-3.0-Pro." This is not a minor caveat β€” it is visible across nearly every benchmark in Table 3. On AIME 2025, DeepSeek-V3.2 achieves 93.1% with 16K tokens versus Gemini-3.0-Pro's 95.0% with 15K tokens β€” lower accuracy, more tokens. On HMMT Feb 2025: 92.5% with 19K versus 97.5% with 16K β€” lower accuracy, more tokens. On HLE: 25.1% with 21K versus 37.7% with 15K β€” substantially lower accuracy, substantially more tokens. Even the Speciale variant, which sacrifices efficiency for accuracy, achieves competitive or superior accuracy only by spending dramatically more tokens: 96.0% on AIME with 23K (versus Gemini-3.0-Pro's 95.0% with 15K β€” a 53% token premium for 1 percentage point), 30.6% on HLE with 35K (versus 37.7% with 15K β€” 133% more tokens for worse accuracy). The paper's reasoning-mode evaluations on agentic tasks (Appendix Table 9) show substantial advantages for thinking mode over non-thinking mode, but the token costs of these thinking traces are not reported β€” it is likely the thinking-mode advantage comes at a substantial token premium.

The consequence. The token efficiency gap has direct deployment implications that the paper's headline benchmark numbers obscure. For latency-sensitive applications (interactive coding assistants, real-time dialogue, customer support), generating 16–21K tokens per query is impractical β€” even with DSA's efficiency improvements, the wall-clock latency of generating 20K tokens sequentially remains high. For cost-sensitive batch processing, the per-query cost difference between DeepSeek-V3.2 and Gemini-3.0-Pro can be estimated: if Gemini-3.0-Pro achieves superior accuracy with 25–30% fewer tokens on reasoning benchmarks, the cost differential compounds across millions of queries. The Speciale variant's extreme token usage (35K for HLE, 77K for Codeforces rating computation β€” Table 3) makes it a research demonstration rather than a deployable system, which the paper acknowledges implicitly by noting the standard model was trained "aiming to optimize the trade-off between performance and cost" (Section 4.2). A practitioner choosing between DeepSeek-V3.2 and a proprietary API must consider not just accuracy but total token cost including thinking overhead, which may erase the cost advantage suggested by Figure 3's inference pricing.

What evidence exists in the paper. Table 3 provides the direct evidence: token counts alongside accuracy for every benchmark. The paper's own efficiency comparison against Kimi-K2-Thinking highlights this dimension: "Compared to K2-Thinking, DeepSeek-V3.2 achieves comparable scores with substantially fewer output tokens" (Section 4.1). However, against Gemini-3.0-Pro, the comparison goes the other direction. The paper's statement that "the token efficiency of DeepSeek-V3.2-Speciale remains significantly inferior to that of Gemini-3.0-Pro" (Section 4.2) directly acknowledges the gap for the Speciale variant. The context management experiments (Section 4.4) partially address the token budget problem by showing that intelligent context handling can improve efficiency, but this is a workflow optimization, not a model capability improvement β€” the model still produces verbose reasoning traces; the context management just prevents it from re-reasoning from scratch after tool calls.

Mitigation status. The paper acknowledges token efficiency as a limitation (Section 5) and states: "Future work will focus on optimizing the intelligence density of the model's reasoning chains to improve efficiency." The length penalty used in RL training for the standard model (and its reduction for Speciale) demonstrates awareness of the tradeoff, but the paper provides no concrete path to reaching Gemini-3.0-Pro-level token efficiency. The two-tier release (standard vs. Speciale) is a partial mitigation β€” users can choose the cost-efficient variant β€” but both variants trail Gemini-3.0-Pro on the accuracy-per-token metric.

The Agentic Task Synthesis Pipeline Produces Impressive Generalization, But the Scope and Realism of Synthesized Environments Have Unmeasured Boundaries

The assumption or constraint. The paper's general agent synthesis pipeline (Section 3.2.3) generates 1,827 environments and 4,417 tasks across categories like trip planning, data analysis, and logic puzzles. The environments are designed to be "hard to solve, easy to verify" β€” a property that makes them effective for RL with outcome rewards but also constrains them to tasks with well-defined verification criteria. The paper claims these synthetic tasks improve generalization to out-of-domain real-world benchmarks (Tau2Bench, MCP-Mark, MCP-Universe), as shown in Figure 5. However, the paper does not characterize the types of real-world tasks for which synthetic data transfer succeeds versus fails. The synthesized environments are necessarily constrained by what the synthesis agent can construct programmatically β€” tasks requiring nuanced natural language understanding, subjective judgment, creative problem formulation, or interaction with genuinely unpredictable external systems may not be representable in the "sandbox + verifier function" format. The paper implicitly assumes that training on diverse synthetic verifiable environments transfers broadly to agentic tasks, but the scope of this transfer is unmeasured.

The consequence. A practitioner building an agentic application in a domain not represented in the synthesis categories (e.g., healthcare coordination, legal document analysis, multi-stakeholder negotiation) cannot determine whether DeepSeek-V3.2's agentic capabilities will transfer. The paper's evidence of generalization is limited to three specific benchmarks (Tau2Bench, MCP-Mark, MCP-Universe) that, while diverse, still share structural similarities with the synthetic tasks: they involve tool calling, structured outputs, and verifiable sub-goals. Open-ended agentic tasks where success is ambiguous or multi-dimensional β€” managing a long-running project, conducting original research, providing emotional support β€” may not benefit from the synthesis pipeline at all. More subtly, the synthesis pipeline may induce biases: the model learns to expect that tasks have clean verifiability criteria, that environments provide well-structured tools, and that "hard" means "combinatorial search over constraints" rather than "ambiguous, under-specified, or requiring value tradeoffs." These biases could manifest as brittleness when the model encounters messier real-world agentic scenarios. The paper's acknowledgment that "certain agent frameworks, such as Roo Code or Terminus, simulate tool interactions via user messages" and therefore "may not fully benefit from our enhanced reasoning persistence" (Section 3.2.1) illustrates a concrete case where a design assumption (tool messages are distinguishable from user messages) creates a gap between the training setup and real deployment patterns.

What evidence exists in the paper. Figure 5 provides evidence of transfer to three benchmarks from RL on synthetic data alone β€” this is the strongest evidence of generalization, but it is limited to those three specific benchmarks. Table 5 demonstrates that the synthesized tasks are genuinely hard, but the 50-instance sample is small. Table 2 shows DeepSeek-V3.2's performance across many benchmarks, but these results reflect the full post-training recipe (specialist distillation + mixed RL), not synthetic data alone, so we cannot attribute specific benchmark improvements to the synthesis pipeline versus other post-training components. The paper notes that for MCP-Universe and MCP-Mark, "the environments and toolsets employed in these benchmarks were not encountered during RL training," which is the key evidence for generalization. However, the paper does not report performance on agentic benchmarks that are qualitatively different from the synthesis categories β€” e.g., benchmarks requiring creative collaboration, handling of contradictory instructions, or long-term planning without clear sub-goals.

Mitigation status. The paper does not directly acknowledge the scope limitation of the synthesis pipeline's "hard to solve, easy to verify" constraint, but does note that "solving complex tasks is still inferior to frontier models" (Section 5), which may partially reflect the limitation of verifier-based RL for tasks without clean reward signals. The paper suggests "further refin[ing] our foundation model and post-training recipe" as future work, without specifying whether this includes expanding the synthesis pipeline to cover fuzzier task types or developing alternative training approaches for non-verifiable agentic tasks. The diversity of the synthesis categories (search agent, code agent, code interpreter, general agent) represents an attempt to cover a broad task space, but the paper does not characterize what is outside this space or how users should assess whether their target domain is covered.

Difficulty Estimation for Adaptive Allocation Is Absent β€” The Compute-Optimal Paradigm Is Not Extended to the Model's Own Inference Behavior

The assumption or constraint. The example paper on compute-optimal test-time scaling established that different inference strategies (beam search vs. best-of-N, sequential revisions vs. parallel sampling) are optimal for different prompt difficulties, and that difficulty estimation enables substantial efficiency gains (4Γ— over uniform allocation). DeepSeek-V3.2 does not adopt this paradigm. The model applies the same generation strategy β€” sampled from the thinking-mode or non-thinking-mode policy β€” uniformly across all prompts, with a fixed context management strategy applied only when the 128K context window overflows (Section 4.4). There is no mechanism for the model to assess problem difficulty at inference time and adapt its reasoning depth, number of tool calls, or parallelization strategy accordingly. The Speciale variant demonstrates that longer reasoning chains improve some problems, but the standard model applies a uniform length penalty during RL training that constrains all problems equally β€” easy problems are penalized for verbosity just as hard problems may need more tokens. The context management experiments (Figure 6) compare different strategies but apply a single strategy uniformly across all BrowseComp instances β€” there is no per-problem adaptation.

The consequence. The uniform application of reasoning budget means DeepSeek-V3.2 almost certainly wastes tokens on easy problems (where a shorter reasoning chain would suffice) while under-investing in hard problems within the same task distribution. On AIME 2025, the standard model uses 16K tokens on average (Table 3) β€” but problem difficulty on AIME varies substantially (the hardest problems have near-zero solve rates even for top models). Applying 16K tokens uniformly means some problems receive excessive reasoning while others are cut off prematurely by the length penalty. The consequence is suboptimal allocation of the user's inference budget: a practitioner running DeepSeek-V3.2 on a mixed-difficulty workload cannot direct the model to "think harder" on hard cases and "answer faster" on easy ones. The Speciale variant relaxes length constraints entirely, achieving +2.9 points on AIME at the cost of 44% more tokens β€” but this is a global relaxation, not a difficulty-adaptive policy. A compute-optimal allocation that spent 23K tokens only on the hardest AIME problems and 13K on the easiest might achieve near-Speciale accuracy at near-standard cost, but the paper does not explore this.

What evidence exists in the paper. Table 3 provides token counts averaged across entire benchmarks, which inherently obscures per-problem variance. Figure 6 (BrowseComp context management) compares strategies at varying step budgets, but the strategies are applied uniformly β€” there is no curve showing what happens if you combine strategies adaptively. The paper's approach to token budget allocation is binary: standard model (with length penalty) vs. Speciale (with reduced penalty). No experiment tests whether difficulty-adaptive allocation within a single model variant would improve the accuracy-efficiency Pareto frontier. The paper acknowledges that "finding the optimal combination of serial and parallel scaling to maximize both efficiency and scalability remains a crucial direction for future work" (Section 4.4), but this is specific to context management strategies for search agents, not a general inference-time allocation framework. The paper's assessment that DeepSeek-V3.2 "frequently engages in redundant self-verification, generating excessively long trajectories" on MCP benchmarks (Section 4.1) is direct evidence of misallocated inference compute β€” the model spends tokens on unnecessary verification that a difficulty-aware stopping criterion could eliminate.

Mitigation status. The paper does not frame the absence of difficulty-adaptive inference as a limitation, though the observation about redundant self-verification on MCP benchmarks suggests the authors are aware of the inefficiency. The context management experiments (Section 4.4) represent a step toward adaptive budget allocation, but only for the specific problem of context overflow, not for general reasoning budget. The paper's suggestion that integrating context management "can further enhance performance" and that "finding the optimal combination of serial and parallel scaling" is future work (Section 4.4) partially addresses this limitation but at a narrow scope. The length penalty in RL training is a blunt instrument for controlling token usage β€” the paper does not discuss whether per-prompt difficulty estimation during RL training or inference could provide finer-grained control.

7. Implications and Future Directions

How This Work Changes the Landscape

DeepSeek-V3.2 changes the landscape along three axes simultaneously β€” architecture, post-training methodology, and data generation β€” and the significance of each axis differs in magnitude and character. The paper's signal-processing view of RL instability is a diagnostic reframing, not a new algorithm: it recasts GRPO training failures as problems of gradient signal integrity rather than optimizer tuning, redirecting research attention from clipping schedules and trust regions to the mathematical validity of importance sampling ratios, KL estimators, and routing consistency. This is methodologically significant but not paradigm-shifting β€” it provides a debugging toolkit for scaling post-training RL, not a fundamentally new optimization framework.

The synthetic environment generation pipeline is a methodological contribution of potentially larger magnitude. It establishes that automatically generated, automatically verifiable environments can substitute for real-world interaction data in agentic RL post-training, with generalization that transfers to unseen benchmarks (Figure 5). This decouples agentic training from the bottleneck of human data collection or real-world environment construction, making it possible to scale agentic RL to thousands of environments without proportional human effort. The "hard to solve, easy to verify" principle β€” previously applied to math and code through benchmark selection β€” is here extended to open-ended agentic tasks through environment programming rather than task filtering. This is a genuine expansion of the RL-for-agents design space. If the principle proves robust across other task categories (healthcare, legal, creative), it could become a standard component of post-training pipelines, analogous to how synthetic math problems drove reasoning improvements in earlier work.

DSA is a strong engineering contribution with a specific deployment implication: it demonstrates that an existing dense model can be converted to sparse attention via continued pre-training with modest cost (943.7B tokens) and essentially zero capability regression. The 2.5Γ— inference cost reduction at 128K context during decoding (Figure 3) changes the economic calculus for long-context LLM deployment. The open-sourcing of the implementation provides a reference that other teams can adapt. However, the paper does not establish that DSA's content-adaptive approach is superior to simpler structural sparsity patterns (sliding window, global tokens) β€” it only compares against dense attention. The contribution is therefore about the demonstrated viability of a specific migration path (dense β†’ sparse via self-distillation) rather than a proof that DSA is the optimal sparsity mechanism. The field still lacks a systematic comparison of sparse attention methods at this scale.

The paper resolves a specific contradiction in the open-source development narrative. Prior work suggested open models were converging with proprietary ones β€” this paper's introduction argues the opposite, that the gap is widening, and then demonstrates that a specific combination of interventions (DSA + scaled RL + synthetic agentic data) can close the gap on reasoning benchmarks (matching GPT-5) while advancing agentic capabilities. The resolution is that the gap was widening because open models were underinvesting in three specific areas; addressing all three simultaneously reverses the trend for the specific capabilities measured. The paper does not claim to have closed the gap entirely β€” token efficiency, world knowledge, and complex task-solving remain open problems β€” but establishes that the divergence is not due to any fundamental architectural limitation of open models.

Research directions that become more attractive after this work:

  • Diagnostic approaches to RL instability. If training instability is primarily a signal-quality problem (biased KL, off-policy distortion, routing mismatch, action-space mismatch) rather than an optimizer problem, then the research agenda shifts toward developing systematic diagnostics for gradient corruption in RL pipelines. Rather than proposing new algorithms, future work could audit existing RL implementations for the specific failure modes this paper identifies, potentially discovering additional signal-path bugs that limit scaling.

  • Programmatic environment generation for RL. The synthesis pipeline demonstrates that diverse, automatically verifiable environments can drive agentic RL at scale. This makes research on environment generation algorithms β€” how to produce maximally diverse training environments, how to calibrate difficulty automatically, how to ensure coverage of desired capability dimensions β€” a high-leverage activity. The specific design choices (iterative difficulty escalation, toolset augmentation, pass@K filtering) provide a starting point that others can refine or challenge.

  • Inference-time compute allocation within a single generation budget. The paper's context management experiments (Figure 6) and the observation of "redundant self-verification" on MCP benchmarks both point to a gap: DeepSeek-V3.2 applies reasoning compute uniformly rather than adaptively based on problem difficulty. The success of the compute-optimal test-time scaling paradigm (from the example paper on inference-time allocation) suggests that combining DeepSeek-V3.2's reasoning capabilities with a difficulty estimator could yield substantial efficiency gains β€” a direct extension that is newly tractable because DeepSeek-V3.2 provides an open, capable base model to experiment on.

Research directions that become less urgent:

  • Incremental improvements to PPO/GRPO clipping mechanisms. The paper's stabilization techniques target the signal path, not the clipping schedule. If these techniques prove robust, the marginal value of further tuning clipping hyperparameters likely diminishes β€” the bottleneck shifts to verifier quality, reward design, and data diversity.

  • Hand-crafted agent benchmarks for RL training. If synthetic environments generalize (Figure 5), the case for investing human effort in building training environments (as opposed to evaluation benchmarks) weakens. Evaluation benchmarks remain essential for measuring generalization, but training data can increasingly be synthesized.

  • Pure architectural innovation for long-context efficiency without the migration constraint. DSA's key value proposition is that it works via continued training from an existing dense checkpoint. For teams that already have large dense models, this makes DSA more immediately applicable than architectures requiring training from scratch. Research on from-scratch sparse attention mechanisms remains valuable but faces a higher adoption barrier for existing model families.

Follow-Up Research This Work Enables

Replicating the stabilization techniques on other architectures to test generality. The paper's four GRPO stabilization techniques (unbiased KL, off-policy sequence masking, keep routing, keep sampling mask) are validated only on DeepSeek's MoE architecture with MLA attention. A strong follow-up would implement these exact techniques in a non-DeepSeek codebase β€” for example, applying them to a dense transformer being post-trained with GRPO β€” to determine which techniques are MoE-specific (keep routing) and which address universal RL instability problems (off-policy masking, unbiased KL). The key measurement would be the maximum stable RL budget achievable with and without each technique, producing a generalizability matrix across architectures. A negative result β€” finding that the techniques are DeepSeek-specific β€” would be equally valuable, indicating that the instability problems are architecture-dependent rather than universal.

Difficulty-adaptive reasoning budget allocation using DeepSeek-V3.2 as a testbed. The paper observes redundant self-verification on MCP benchmarks and reports average token counts that obscure per-problem variance. A direct extension would train a lightweight difficulty predictor (perhaps a linear probe on the model's hidden states from the first few reasoning tokens) to estimate whether a problem needs more or fewer thinking tokens, then apply a variable length penalty during inference proportional to predicted difficulty. The experiment would compare accuracy at matched total token budget between uniform and adaptive allocation on AIME 2025 (where problems have known difficulty gradation), measuring whether the compute-optimal paradigm demonstrated in the example paper (4Γ— efficiency gains) extends to reasoning-chain length allocation within a single model. The key metric is the accuracy-efficiency Pareto curve: does adaptive allocation shift the curve outward relative to uniform allocation?

Systematic comparison of DSA against structural sparsity baselines at matched compute. The paper compares DSA only against dense attention (DeepSeek-V3.1-Terminus). A thorough follow-up would implement sliding window attention with global tokens, block-sparse patterns, and the Native Sparse Attention from Yuan et al. (2025) on the same DeepSeek-V3.1-Terminus base checkpoint, run continued pre-training at the same 943.7B token budget, and measure both inference cost and benchmark performance after identical post-training. This would answer whether the lightning indexer's content-adaptive selection provides benefits beyond what simpler structural sparsity can achieve, and at what computational cost (the indexer still has O(LΒ²) complexity, albeit with a lighter computational footprint). The comparison would establish whether the indexer's training complexity is justified by downstream gains or whether simpler approaches match DSA's efficiency-accuracy tradeoff.

Ablation of the RL budget fraction to test the "10% threshold" hypothesis. The paper claims post-training exceeds 10% of pre-training cost but provides no scaling curve. A direct follow-up would vary the RL budget (as a fraction of a fixed pre-training cost) and measure saturation behavior: does performance continue improving beyond 10%, plateau at some threshold, or follow a power-law scaling relationship? This requires reporting actual compute numbers (GPU-hours or FLOPs) for both phases, which the paper does not do. The experiment would test the paper's central implicit claim β€” that post-training compute is the bottleneck for open models β€” by showing whether the marginal return on RL compute is still high at 10% (supporting further investment) or diminishing (suggesting other bottlenecks dominate). A null result (performance plateauing at 5% of pre-training cost) would reframe the paper's contributions as architectural and data-driven rather than compute-driven.

Stress-testing synthetic environment generalization on agentic tasks without clear verifiability. The synthesis pipeline relies on the "hard to solve, easy to verify" property. A critical follow-up would test whether RL on synthetic verifiable environments transfers to agentic tasks where success is ambiguous or multi-dimensional: creative collaboration (e.g., co-writing a story with tool-assisted research), negotiation (e.g., finding mutually acceptable solutions with constraint tradeoffs), or open-ended exploration (e.g., conducting a literature review without a predefined answer). If transfer fails systematically for non-verifiable tasks, it would establish a boundary condition for the synthesis-to-real generalization demonstrated in Figure 5, and motivate hybrid approaches that combine verifiable RL with generative reward models or human feedback for the non-verifiable component.

Combining DSA efficiency with the compute-optimal allocation paradigm for a total-inference-cost optimization. The paper provides two pieces β€” an efficient architecture (DSA, reducing per-token cost) and context management strategies (Discard-all, extending effective context window) β€” but does not optimize them jointly with reasoning budget allocation. A follow-up system paper could integrate a difficulty estimator with variable-length generation (relaxing the length penalty for predicted-hard problems), context management strategy selection (applying Discard-all only when context overflow is imminent), and DSA's content-adaptive sparsity, measuring end-to-end cost in USD per correct answer across a mixed-difficulty benchmark suite. This would operationalize the paper's implicit resource-allocation principle at the full-system level, producing a deployable inference engine that makes cost-aware decisions at multiple granularities (token selection, reasoning length, context handling strategy).

Practical Applications and Downstream Use Cases

Cost-efficient codebase maintenance at scale. DeepSeek-V3.2's SWE-bench Verified score of 73.1, combined with DSA's 2.5Γ— cost reduction at 128K context (Figure 3), makes it viable for organizations running large-scale automated issue resolution on multi-million-line codebases. The SWE Multilingual score of 70.2 across eight programming languages (Python, Java, JavaScript, TypeScript, C, C++, Go, PHP) means a single deployment can handle a polyglot codebase without per-language model switching. At H800 pricing of $2/GPU-hour, the DSA efficiency means processing a repository with 128K-token context windows costs roughly 0.8 USD per million tokens during decoding β€” translating to sub-dollar costs for most individual issue resolutions. The non-thinking mode achieves 72.1 on SWE-Verified (Appendix Table 9), providing a faster, cheaper option for routine fixes, with thinking mode reserved for complex issues where the +1.0 point improvement justifies the token premium. This is deployable today with the open-source implementation.

Competitive programming and mathematics competition training via the Speciale variant. DeepSeek-V3.2-Speciale's gold-medal performance in IMO 2025 (35/42, missing only the hardest problem), CMO 2025 (102/126), IOI 2025 (492/600, ranked 10th), and ICPC World Final 2025 (10/12 problems, ranked 2nd) β€” all without competition-specific training for the programming contests β€” makes it a tool for training human competitors. The model can generate candidate solutions, explain reasoning paths, and verify correctness in a generate-verify-refine loop (used for IMO/CMO, per Section 4.2 and Appendix D). A training pipeline that uses Speciale to generate practice problems with verified solutions, explain why incorrect approaches fail, and provide step-by-step reasoning would give competitors access to an interactive coach operating at gold-medal level. The Speciale variant's token costs (23K–77K per problem, Table 3) are acceptable for training use where latency is not critical. This is a direct application of the olympiad results in Table 4, not speculative.

Long-context search and research synthesis with context management. DeepSeek-V3.2's BrowseComp score of 67.6 with the Discard-all context management strategy (Section 4.4, Figure 6) β€” combined with BrowseCompZh at 65.0 β€” makes it suitable for automated research synthesis tasks that require navigating dozens of web pages and synthesizing findings. The context management strategies extend the effective working memory beyond the 128K window, with Discard-all achieving +16.2 points over the no-management baseline (51.4) while using fewer steps than summarization-based approaches. The specific deployment scenario: a research assistant that receives a complex comparative question (e.g., "Compare the battery technology approaches announced by Toyota, Tesla, and BYI in their Q3 2025 investor calls, including specific energy density claims"), executes multi-step web searches with configurable depth, and produces a structured synthesis β€” all within the model's 128K window extended by context management. This is feasible because the paper demonstrates the capability on BrowseComp, which tests exactly this type of multi-source comparative reasoning.

Distributed agentic evaluation on MCP-based infrastructure. DeepSeek-V3.2's MCP-Universe success rate of 45.9 and MCP-Mark score of 38.0 β€” while trailing GPT-5 and Gemini-3.0-Pro β€” are substantially ahead of other open models (Kimi-K2: 35.6/20.4, MiniMax-M2: 29.4/24.4, per Table 2). For organizations building internal tool-use applications on MCP (Model Context Protocol) infrastructure β€” where proprietary APIs may be restricted for data governance reasons β€” DeepSeek-V3.2 is the strongest available open option. The thinking mode provides substantial advantages over non-thinking on MCP-Mark (+11.5 points) and MCP-Universe (+7.3 points, Appendix Table 9), justifying the token premium for complex multi-tool orchestrations. The practical deployment involves running DeepSeek-V3.2 with thinking mode enabled, the function-calling format placing tool outputs in 'tool' role messages rather than 'user' role messages (per Section 4.1), and context management to handle the long trajectories the paper notes as a limitation. The generalization demonstrated in Figure 5 β€” where RL on synthetic environments transfers to MCP benchmarks with novel tools β€” provides some confidence that the model will handle MCP tools not seen during training, though users should evaluate on their specific toolset.