Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and Acts

🎯 Pitch

Nanbeige4.1-3B demonstrates that a 3B generalist model can execute stable multi-turn agentic searches over 600 tool-call turns—an ability absent in models up to 8B parameters—while simultaneously achieving code generation scores that surpass Qwen3-30B-A3B. The key is a credit assignment strategy that combines point-wise and pair-wise rewards with turn-level supervision on synthesized deep-search trajectories, unlocking long-horizon agency without sacrificing reasoning or alignment.

1. Executive Summary

This work introduces Nanbeige4.1-3B, a unified generalist language model that simultaneously achieves strong reasoning, code generation, and long-horizon agentic behavior at the 3B parameter scale — the first open-source small language model to demonstrate such versatility in a single checkpoint. Built on Nanbeige4-3B-Base, the model combines point-wise and pair-wise reinforcement learning (scoring individual responses and comparing response pairs for preference alignment), complexity-aware code rewards (a gated time-complexity bonus that activates only when functional correctness is achieved), and turn-level supervision with synthetic multi-hop search trajectories (training on Wikipedia-derived entity-relation walks with per-step critic filtering to enable stable tool interactions). Nanbeige4.1-3B achieves a 4× compute-efficiency advantage over its predecessor on coding benchmarks — jumping from 46.0 to 76.9 on LiveCodeBench-V6 while surpassing Qwen3-30B-A3B — and reaches deep-search performance (69.90 on GAIA, 75.00 on xBench-DeepSearch-05) that matches specialized 30B+ research agents and 100B+ general-purpose models, establishing that long-horizon agency and cross-domain generalist competence can coexist in compact models only when training objectives and credit assignment are properly aligned across turn, trajectory, and preference levels.

2. Context and Motivation

The Core Problem: Fragmented Capabilities in Small Language Models

The fundamental challenge this paper tackles is the fragmentation of capabilities in small language models (SLMs). Recent advances have shown that compact models with 1.5–4B parameters can achieve impressive performance on individual specialized tasks—mathematical reasoning, code generation, or tool-augmented search—when trained with dedicated data and objectives. Vibethinker-1.5B demonstrates strong reasoning through diversity-driven optimization; Nanbeige4-3B (the predecessor to this work) showed competitive coding and alignment; various code-specialized SLMs achieve high execution accuracy. However, each of these models excels in a narrow domain while typically underperforming or failing entirely on others.

The paper articulates this explicitly in Section 1:

"reasoning-focused models often struggle with long-horizon interactions (e.g., deep search), while code or agent specialized models typically lack robust general reasoning abilities, such as creative writing or human preference alignment."

This fragmentation creates a practical deadlock: a practitioner who needs both strong code generation and reliable multi-step search behavior must either deploy multiple specialized models (increasing system complexity, memory footprint, and maintenance burden) or use a single larger model (losing the deployment advantages of small models, such as on-device inference and lower serving costs). A single 3B model that does everything well—reasoning, coding, alignment, and long-horizon planning—did not exist in the open-source ecosystem prior to this work.

The paper frames this as an explicit question:

"How far can a 3B model be pushed as a generalist without compromising its existing strengths?"

This is not a rhetorical question but an empirical one, with specific boundaries to be tested. The answer determines whether the SLM research trajectory should pursue ever-more-specialized models (accepting fragmentation as an inherent constraint of small scale) or can realistically target unified generalists (redefining the capability ceiling of compact architectures).

Why Fragmentation Matters: Deployment Realities and Scientific Significance

The practical importance is driven by the economics of deployment. Models in the 3B parameter class can run on consumer GPUs, edge devices, and cost-efficient cloud instances with low latency—capabilities that vanish for 30B+ models requiring datacenter-class hardware or expensive multi-GPU setups. If a 3B model could genuinely serve as a drop-in replacement for a research agent (requiring hundreds of sequential tool calls), a code generator (handling competitive programming problems), and a conversational assistant (maintaining alignment with human preferences), it would dramatically expand where advanced AI capabilities can be deployed. This matters for latency-sensitive applications (real-time code assistance, interactive search), privacy-sensitive settings (on-device processing without cloud round-trips), and cost-constrained environments (batch processing at scale).

The scientific significance is equally substantial. The paper's core question—whether a 3B parameter budget can support genuinely broad competence—tests a fundamental hypothesis about model capacity: that the apparent need for larger models to achieve multi-domain performance reflects training objective design and data composition rather than an irreducible parameter requirement. If verified, this would suggest that the dominant scaling paradigm (more parameters → better general performance) can be partially bypassed through more sophisticated post-training strategies, including multi-level reward structures and carefully composed synthetic data. This connects to a broader debate in the field about whether scale is primarily about raw capacity or about enabling convergence under simpler training schemes.

Prior Approaches and Their Limitations

The paper's approach builds on and departs from several existing research threads, each with identified shortcomings.

General-purpose SLMs with narrow capability profiles. Models like Qwen3-4B, Qwen3-8B, and the previous Nanbeige4-3B-2511 represent the state of open-source compact generalists—but the paper's own benchmarking in Table 4 and Table 5 reveals their limitations. On deep search tasks, Qwen3-4B-2507 scores 28.33 on GAIA and 1.57 on BrowseComp; Qwen3-8B scores 19.53 and 0.79 respectively. These near-zero scores on long-horizon browsing benchmarks indicate that general-purpose SLMs, even when equipped with tools, fail to sustain coherent multi-turn exploration. The paper characterizes this specifically: "Qwen3-4B and Qwen3-8B fail to sustain meaningful exploration beyond a few turns" (Section 1). The problem is not that these models lack intelligence in an abstract sense—Qwen3-4B scores 57.4 on LiveCodeBench-V6 (Table 4)—but that their training objectives and data distributions do not teach them to maintain coherent state and planning across extended interaction horizons.

Specialized reasoning models without agentic behavior. The paper cites Xu et al. [17] as representative of reasoning-focused SLMs: models like Vibethinker-1.5B that achieve strong logical reasoning through diversity-driven optimization but are not designed for tool use, intermediate information retrieval, or the kind of open-ended exploration that characterizes deep search. These models are strong on closed-form reasoning tasks (math competitions, logic puzzles) where the problem statement provides all necessary information upfront. They are not evaluated—and would likely perform poorly—on tasks requiring iterative environment interaction, where each step depends on information acquired in previous steps.

Search-specialized agent models at larger scales. On the other end of the spectrum, research agents like Tongyi-DeepResearch-30B and MiroThinker-v1.0-8B achieve strong deep search performance (70.90 and 66.40 on GAIA respectively, Table 5) but are purpose-built for search tasks and not designed for general reasoning, code generation, or alignment. The paper compares against these models (Section 3.2.1) but positions Nanbeige4.1-3B as qualitatively different: a single model that matches or exceeds their search performance while simultaneously matching or exceeding general-purpose models on coding and reasoning benchmarks. The existing specialized agents demonstrate that long-horizon tool use is achievable with dedicated training, but they do not demonstrate that it can coexist with broad generalist capabilities in a single parameter budget.

Large foundation models with tools. Models like DeepSeek-V3.2-671B and Minimax-M2-230B achieve strong cross-domain performance (Table 5), including on deep search tasks where DeepSeek reaches 63.50 on GAIA and 67.60 on BrowseComp. However, their performance is achieved through scale—hundreds of billions of parameters—rather than through training methodology innovations at small scale. The paper treats these as upper-bound comparisons showing what is achievable with sufficient capacity, establishing a performance target for compact models rather than a methodological competitor.

Prior work on preference alignment. The paper builds its point-wise and pair-wise RL approach on established frameworks: GRPO (Group Relative Policy Optimization) from Shao et al. [14] for point-wise optimization, and the pair-wise preference framework from Xu et al. [18] with swap-consistency regularization. Where it departs is in the progressive integration—running point-wise RL first to stabilize outputs (reducing repetition and formatting errors), then applying pair-wise RL to exploit high-quality comparison data that would be insufficient volume for stand-alone training. This staged approach to preference alignment is not individually novel but represents a specific engineering insight about sequencing reward signals under data constraints.

Prior work on code optimization. Most existing code RL approaches optimize solely for functional correctness (pass@1 on test cases). The paper argues this leaves efficiency gains unexploited: a model that passes all test cases with an O(n²) solution receives the same reward as one producing an O(n log n) solution, despite the latter being substantially better in practice. The introduction of a gated time-complexity reward—activated only when correctness is achieved—is a specific design choice that addresses the risk of models optimizing for efficiency at the expense of correctness if both signals were always active.

Limitations of existing deep search data. The paper identifies a data bottleneck for training long-horizon search behavior: high-quality multi-hop search trajectories with per-step correctness labels are scarce. Prior approaches either relied on expensive human annotation (limiting scale) or used model-generated trajectories without fine-grained quality filtering (risking the model learning from flawed or suboptimal exploration patterns). The paper's contribution on this axis—temporal-aware Wikipedia entity-relation walks for question synthesis, multi-agent trajectory generation, and turn-level critic filtering—is a data construction methodology rather than a novel training algorithm, addressing the specific problem that existing search training data is not rich enough to teach robust long-horizon planning at the 3B scale.

How This Paper Positions Itself

The paper positions its contribution not as a single algorithmic innovation but as an integration methodology—a specific recipe for composing heterogeneous training objectives, data sources, and reward structures that collectively enable broad capability in a compact model. This is visible in the structure of Section 2 (Methods), which treats general reasoning, deep search, and code generation as parallel optimization streams, then describes in Section 2.4 how they are sequenced and integrated.

Relative to prior work, the paper makes three positioning claims:

1. Unified over specialized. Rather than building a best-in-class model for any single domain (reasoning, code, or search), the target is best-in-class-at-scale for all domains simultaneously. The comparison baselines are chosen to reinforce this: Qwen3-4B (generalist), AgentCPM-Explore-4B (search specialist), various specialized search agents, and large foundation models (generalists at scale). The paper argues that Nanbeige4.1-3B matches or exceeds each category on its home turf while being the only model that spans all categories (Table 4, Table 5, Table 6).

2. Methodology over scale. The paper explicitly aims to demonstrate that training methodology choices—point-wise vs. pair-wise RL, gated time rewards, turn-level supervision—can substitute for parameter count. The comparison against Qwen3-30B-A3B and Qwen3-32B (Table 4) shows a 3B model surpassing 30B+ models on coding and alignment benchmarks, establishing this empirically. The comparison against DeepSeek-V3.2-671B and Minimax-M2-230B (Table 5) pushes the claim further: a ~200× parameter reduction while remaining competitive on deep search.

3. Practical over theoretical. The paper is an engineering contribution with reproducible artifacts: the model checkpoint is open-sourced, the deep search dataset is released on HuggingFace, and the training recipe is described in sufficient detail for replication (Section 2.4, Appendix C). The evaluation includes both curated benchmarks and real-world stress tests (LeetCode Weekly Contests, Section 3.3), establishing that the claimed capabilities transfer beyond standard evaluation settings.

A subtle but important aspect of the paper's positioning is its acknowledgment of negative results and failure modes, which adds credibility to the claimed successes. The ReST $^{EM}$ -style revision training is mentioned in the executive summary's reference to the prior document, and the paper's own Appendix C (time-complexity case studies) shows both before-and-after code, demonstrating that the time reward produces measurable algorithmic improvements rather than just benchmark score inflation. This willingness to show concrete examples of what changes under the proposed methods—not just aggregate metrics—strengthens the positioning that these are real capability improvements rather than benchmark hacking.

The paper also explicitly positions itself as extending the Nanbeige model line (Nanbeige4-3B → Nanbeige4-3B-2511 → Nanbeige4.1-3B), enabling direct ablation of each method's contribution. The delta from the SFT stage alone (+16.5 on LCB V6, +14.7 on Arena-Hard V2, Table 1) establishes that data composition matters; the deltas through point-wise and pair-wise RL (Table 2) establish that reward structure matters; and the controlled experiment with synthetic QA data only (Table 3, +38.9 on xBench-DeepSearch-05) establishes that the data construction pipeline itself—independent of other training components—provides a large fraction of the deep search improvement. This decomposition is central to the paper's argument that each methodological component is individually validated, not just the final checkpoint.

3. Technical Approach

3.1 Reader Orientation

Nanbeige4.1-3B is a unified post-training pipeline that takes a 3B-parameter base language model and progressively equips it with strong reasoning, precise preference alignment, efficient code generation, and long-horizon agentic behavior through a carefully sequenced combination of supervised fine-tuning, multi-stage reinforcement learning with heterogeneous reward structures, and synthetic data construction for specialized capabilities. The system solves the problem of capability fragmentation in small language models—where existing SLMs excel in one domain but fail in others—by designing training objectives and data pipelines that are individually tailored to each capability (general reasoning, code, search) yet compatible enough to coexist in a single model checkpoint without destructive interference between skills.

3.2 Big-Picture Architecture

The training pipeline has five major components arranged in a fixed sequence:

Enhanced SFT Stage — takes the Nanbeige4-3B-Base checkpoint and performs supervised fine-tuning with a redesigned data mixture, extended context length curriculum (32K → 64K → 256K), and improved solution refinement with Chain-of-Thought reconstruction to produce a stronger starting point for reinforcement learning.
Point-wise RL Stage — applies GRPO (Group Relative Policy Optimization) using a general reward model that scores individual responses for quality, suppressing repetitive or redundant outputs and improving formatting consistency while boosting alignment scores.
Pair-wise RL Stage — builds on the point-wise checkpoint with a pair-wise reward model trained on strong-vs-weak model comparisons, using swap-consistency regularization to mitigate position bias and providing comparative preference signals that push alignment and code quality further.
Two-Stage Code RL — takes the general RL checkpoint and performs specialized code optimization: Stage 1 maximizes functional correctness (pass rate on test cases), then Stage 2 activates a gated time-complexity reward that encourages algorithmic efficiency but only for fully correct solutions, preventing the model from trading correctness for speed.
Lightweight Agentic RL + Deep Search Data Pipeline — constructs synthetic multi-hop QA pairs from Wikipedia entity-relation graphs with temporal filtering, generates search trajectories using multiple agent frameworks, applies turn-level critic filtering, and trains the model to sustain long-horizon tool interactions.

Information flows sequentially: SFT produces a base capability profile → point-wise RL stabilizes outputs and improves alignment → pair-wise RL refines preference boundaries → code RL layers specialized code expertise → agentic RL adds long-horizon planning. At each stage, the model retains capabilities from previous stages while acquiring new ones, enabled by careful data mixing (Section 2.4) and stage-specific reward design that prevents gradient interference.

3.3 Roadmap for the Deep Dive

First, the Enhanced SFT recipe (Section 2.1.1), because it establishes the data composition, context length curriculum, and solution refinement procedures that create the foundation for all subsequent RL—without understanding what data the model sees during SFT, the RL improvements cannot be properly contextualized.
Second, the Point-wise RL stage (Section 2.1.2), including how the general reward model is trained and used with GRPO, because it introduces the first RL mechanism, establishes the base RL training infrastructure, and addresses specific SFT artifacts (repetition, formatting errors) that must be fixed before more sophisticated RL can be effective.
Third, the Pair-wise RL stage (Section 2.1.3), including the pair-wise reward model training with swap-consistency regularization, because it builds directly on point-wise RL results and introduces the comparative preference framework that is critical for alignment improvements.
Fourth, the Deep Search data pipeline and training (Section 2.2), because it is a self-contained subsystem with its own data construction methodology (temporal-aware entity-relation walks, multi-agent trajectory synthesis, turn-level critic filtering) and because the controlled experiment in Table 3 establishes its independent contribution before integration with the full pipeline.
Fifth, the Code RL system (Section 2.3), including the unified judge system, the gated time-complexity reward, and the staged training process, because it represents the most technically complex reward structure in the paper and depends on infrastructure (judge system) that is shared across data construction and training.
Sixth, the integration recipe (Section 2.4) that sequences all stages and explains how the model avoids catastrophic forgetting across domains, because this is the paper's central methodological claim—that these heterogeneous objectives can coexist in a 3B model.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an engineering integration paper whose core idea is that a 3B-parameter model can achieve broad generalist capabilities—spanning reasoning, coding, alignment, and long-horizon search—if the post-training pipeline provides carefully sequenced, domain-specific training signals that are each designed to avoid interference with previously acquired capabilities.

The SFT stage takes the pretrained Nanbeige4-3B-Base checkpoint and produces Nanbeige4.1-3B-SFT through three coordinated improvements over the previous Nanbeige4-3B-2511 recipe: a redesigned data mixture, an extended context length curriculum, and an upgraded solution refinement framework. Each improvement is individually validated through benchmark deltas (Table 1), and collectively they establish a stronger foundation for downstream RL.

Redesigned data mixture. Compared to the previous Nanbeige4-3B-2511 version, the paper increases the proportion of code-related data and introduces "a higher ratio of challenging problems in mathematics and general domains" (Section 2.1.1). The exact mixture proportions are not specified for the early SFT stages, but the final 256K context-length stage uses a specialized mixture: code (27%), deep-Search (26%), STEM (23%), tool-use (13%), and general domains (10%). This shift toward code and search data is motivated by the observation that reasoning depth and benchmark robustness on difficult problems improve when the model encounters harder examples during SFT, even at the cost of reducing the proportion of easier general-domain data. The composition reflects a deliberate bet that diversity across challenging domains matters more than volume of easy examples—a bet validated by the +16.5 point improvement on LiveCodeBench V6 and +14.7 on Arena-Hard V2 from this SFT stage alone (Table 1).

Extended context length curriculum. The previous Nanbeige4-3B-2511 used a two-stage curriculum (32K → 64K tokens). This paper extends it to three stages: 32K → 64K → 256K tokens. The motivation is concrete: long-horizon search tasks and complex multi-step reasoning problems require processing extensive context (retrieved documents, tool outputs, conversation history), and a model that has only been fine-tuned up to 64K tokens may fail to attend effectively to information beyond that window during inference. The curriculum progression—starting at 32K, stepping to 64K, then to 256K—reflects a standard curriculum learning principle: gradually increasing difficulty (in this case, context length) prevents the model from being overwhelmed early in training, when its representations are not yet adapted to the new data distribution. The final 256K stage uses the specialized mixture described above, meaning that the model's long-context capabilities are developed specifically on code and search data rather than generic text, biasing its long-range attention toward the types of dependencies that matter for downstream tasks.

Upgraded Solution Refinement and Chain-of-Thought Reconstruction. This component builds on the framework originally introduced in Nanbeige4-3B-2511 but with two specific enhancements:

Increased refinement iterations. The Solution Refinement loop takes an initial model-generated answer and iteratively improves it through critique–revision cycles. The paper states that it "scales up the number of refinement iterations," allowing more rounds of feedback and revision. More iterations mean each training example undergoes more quality-improvement steps, producing final solutions that are further from the model's initial (potentially flawed) output and closer to a high-quality target. This creates a stronger supervised signal: the model is trained on better final answers.
More capable CoT Reconstruction model. The CoT Reconstruction step takes a refined final answer and generates a faithful reasoning trace that explains how to arrive at that answer. Training a "more capable" reconstruction model means the generated chains-of-thought are cleaner (fewer logical gaps, less irrelevant content) and more faithful (accurately representing the reasoning that would produce the given answer rather than post-hoc rationalization). This matters because the CoT traces become training data for the SFT model: if the traces contain errors or unfaithful reasoning, the SFT model learns to produce flawed reasoning, which downstream RL would then need to correct—adding avoidable difficulty to the RL stages.

The combined effect of these SFT improvements is documented in Table 1: Nanbeige4.1-3B-SFT outperforms Nanbeige4-3B-SFT by +16.5 on LiveCodeBench V6, +21.0 on LiveCodeBench Pro Medium, +13.6 on HMMT Nov, +14.1 on IMO-Answer-Bench, and +14.7 on Arena-Hard V2. These gains are from data composition and quality improvements alone, before any RL is applied, establishing that the SFT stage is not merely a prerequisite but a substantial contributor to final model performance.

Point-wise RL: GRPO with a General Reward Model

After SFT, the model exhibits specific degradation patterns: "repetition and redundant thinking," consistent with prior work [5]. The point-wise RL stage addresses these issues while establishing a stable behavioral baseline for subsequent RL stages.

General reward model training. The paper trains a reward model on "curated large-scale human preference data" to evaluate individual (point-wise) response quality. The reward model architecture and training details are not specified beyond following prior work [15], but the key design choice is that it is trained to score responses in isolation—assigning a scalar quality score to each complete model output without comparing it to alternatives. The paper reports that this reward model "naturally suppresses overly redundant, repetitive, and low-readability answers," which suggests the training data includes human judgments that penalize these patterns, and the reward model successfully learns to associate them with lower scores.

GRPO optimization. The paper applies Group Relative Policy Optimization (GRPO) from Shao et al. [14] to optimize Nanbeige4.1-3B-SFT against this reward model. The procedure: for each prompt in the training set, the policy (current model checkpoint) samples 8 rollouts (complete responses). Each rollout is scored by the general reward model. GRPO then updates the policy to increase the probability of high-scoring responses relative to low-scoring ones within each group of 8, using the group-relative advantage rather than an absolute reward baseline. The group size of 8 is a hyperparameter choice balancing diversity of comparisons (more rollouts give better relative advantage estimates) against computational cost (each rollout requires a full forward pass).

The key distinction between GRPO and standard PPO-based RLHF is that GRPO computes advantages relative to the group mean rather than using a learned value function. This removes the need to train a separate critic model, reducing infrastructure complexity, but also means the effective reward signal is comparative within each batch rather than absolute—the model is learning to produce responses that are better than its own current average, not responses that exceed some absolute quality threshold. This is a good fit for the point-wise stage's goal of eliminating obvious failure modes (repetition, formatting errors) rather than pushing toward optimal quality: the model can learn to avoid bad outputs by comparing them to its own better outputs on the same prompt.

Results and mechanisms. Table 2 reports the delta from point-wise RL: LiveCodeBench V6 improves from 62.0 to 66.0 (+4.0), Arena-Hard V2 from 60.2 to 66.6 (+6.4), Multi-Challenge from 44.4 to 47.7 (+3.3). The paper also reports a specific operational improvement: "on LiveCodeBench-v6, the point-wise RL greatly improves length stability, reducing overlong truncation from 5.27% to 0.38%." This is a concrete, measurable behavioral change: before point-wise RL, 5.27% of generated code solutions on LiveCodeBench were truncated because they exceeded the maximum output length; after, only 0.38% are truncated. The mechanism is that the reward model penalizes unnecessarily long outputs (likely because human preference data associates conciseness with quality), and GRPO amplifies this signal.

The point-wise RL stage is positioned as a stabilization step before pair-wise RL. The logic is that pair-wise comparisons between model outputs and reference answers are only informative if the model's outputs are reasonably well-formed; if the model is still producing frequent formatting errors or degenerate repetitions, the pair-wise comparisons will be dominated by these superficial differences rather than substantive quality differences, wasting the limited high-quality comparison data on correcting problems that point-wise RL can fix with cheaper, more abundant absolute quality signals.

Pair-wise RL: Comparative Preference Optimization

While point-wise RL provides effective alignment signals, the paper identifies a data constraint: "the amount of high-quality preference data is limited." Pair-wise RL is designed to extract more learning signal from scarce high-quality comparisons by explicitly modeling which of two responses is preferred, rather than scoring each in isolation.

Pair-wise reward model training. The pair-wise reward model is trained on paired comparison data spanning two domains: code generation and LMArena-style conversations (both single-turn and multi-turn). The data generation procedure: for each prompt, generate response pairs using a strong model (high-quality reference) and a weak model (lower-quality alternative), then apply the same "checklist filtering strategy as Nanbeige4" [21] to derive reliable win–loss labels. Checklist filtering means applying a set of verifiable criteria (e.g., does the code compile? does the response contain a coherent answer? are there factual errors?) to determine which response is objectively better, rather than relying solely on weaker model rankings. This produces a dataset of (prompt, response_A, response_B, label) tuples where label indicates whether A wins, B wins, or tie.

The pair-wise reward model is trained following the framework of Xu et al. [18], which introduces a swap-consistency regularizer to mitigate position bias—the tendency of preference models to favor the first or second response regardless of content. The regularizer is defined as:

$L_{\text{swap}} = \text{MSE}(r_\theta(x, y_A, y_B) - r_\theta(x, y_B, y_A), 0)$

where $r_\theta(x, y_A, y_B)$ is the reward model's predicted reward difference between response A and B for prompt $x$ (positive means A is preferred), and MSE is the mean squared error.

What it computes: the regularizer penalizes asymmetry in the model's predictions when the input order is swapped. If the model's predicted preference for A over B when presented $(A, B)$ is not the negation of its predicted preference for B over A when presented $(B, A)$ , the squared error term adds to the loss. The ideal outcome is that $r_\theta(x, y_A, y_B) \approx -r_\theta(x, y_B, y_A)$ , meaning the model's preference judgments are order-invariant.

Why this form: position bias is a well-documented failure mode in preference models—models tend to assign higher scores to whichever response appears first (or last) in the input, learning spurious positional heuristics rather than content-based preferences. Without the swap-consistency regularizer, the model could achieve low standard preference loss by simply learning to prefer the first response, regardless of content. The swap-consistency term forces the model to produce consistent judgments under ordering permutation, which can only be achieved by actually evaluating content rather than position. MSE is used rather than, say, an L1 penalty because larger inconsistencies (e.g., strongly preferring A over B in one order and strongly preferring B over A in the other) should be penalized more heavily than minor ones, and MSE provides this quadratic scaling.

Pair-wise RL formulation. During RL training, the pair-wise reward is formulated as a binary outcome: whether the generated rollout outperforms the reference answer. For multi-turn scenarios, the full dialogue history is concatenated into the pair-wise reward model's input, allowing the model to evaluate preferences in context rather than treating each turn independently. This is important for conversational alignment: a response that seems good in isolation might be inappropriate given the conversation history (e.g., repeating information already provided, failing to acknowledge a topic shift), and the pair-wise model can capture these contextual preference judgments.

The paper does not specify the exact RL algorithm used for the pair-wise stage (unlike the point-wise stage, which explicitly names GRPO), but the reward structure suggests a direct preference optimization approach: the policy is updated to maximize the probability that its outputs are preferred over reference answers according to the pair-wise model.

Results and mechanisms. Table 2 shows the pair-wise RL delta: LiveCodeBench V6 slightly decreases from 66.0 to 65.6 (-0.4), but alignment benchmarks show large gains: Arena-Hard V2 from 66.6 to 73.8 (+7.2), Multi-Challenge from 47.7 to 55.1 (+7.4). The small code regression is notable—it suggests that the pair-wise stage is optimizing primarily for preference alignment, and the alignment signal is strong enough to slightly shift the model away from pure code correctness. However, the net effect across all domains is positive, as the alignment gains (+7.2 and +7.4) far outweigh the code dip (-0.4).

The mechanism for the alignment improvements: by training on explicit preference comparisons, the model learns to distinguish subtle quality differences that are invisible to a point-wise scorer. For example, two responses might both be grammatically correct, factually accurate, and well-formatted—the point-wise reward model would give them similar scores—but one might be more helpful, more concise, or better adapted to the user's apparent needs. The pair-wise model, trained on human (or strong-model-filtered) comparisons, can capture these nuanced preferences, and RL against this model pushes the policy toward the preferred response distribution.

The paper notes that pair-wise RL "deeply exploits contextual information in multi-turn dialogues," which explains the large Multi-Challenge gain: Multi-Challenge is specifically designed to test multi-turn conversational ability, and the pair-wise model's access to full dialogue history (rather than scoring each turn independently) enables it to reward contextually appropriate responses that a turn-by-turn scorer would miss.

Deep Search: Data Construction and Turn-Level Supervision

The deep search capability is developed through a data-centric approach: constructing a large-scale synthetic dataset of complex multi-hop search trajectories with fine-grained quality filtering, then training the model to replicate successful search behavior. The data pipeline (Figure 2) has three stages: temporal-aware question synthesis via entity-relation graphs, multi-agent trajectory generation, and turn-level critic filtering.

Stage 1: Temporal-Aware Head Entity Selection and Question Synthesis.

The first stage generates complex multi-hop questions rooted in Wikipedia knowledge. The procedure:

Entity extraction with temporal filtering. Extract "informative head entities from Wikipedia that have been updated within the past six months." The temporal constraint ensures questions are based on recent, non-stale information—this matters for search benchmarks like GAIA and BrowseComp that target current events and recent knowledge, where training on outdated Wikipedia snapshots would produce models that fail to find answers requiring up-to-date information.
Entity-relation graph construction. Following the framework of [20], construct a graph where nodes are Wikipedia entities and edges represent relationships extracted from the text. The graph captures structured knowledge about how entities connect: for example, an edge from "Beijing" to "China" labeled "capital_of," or from "Lionel Messi" to "Barcelona" labeled "played_for."
Conditional random walks. Perform random walks of predefined length on the entity-relation graph, producing chains of connected entities. A walk starting at "Beijing" might traverse: Beijing → China → G20 → International Organizations, yielding a chain linking a city to international politics. The walk length controls question complexity: longer walks require more reasoning hops to connect entities.
Question synthesis. Feed the entity chain and temporal context into "a robust LLM" that synthesizes a natural-language question requiring traversal of the chain to answer. For the Beijing → China → G20 chain, the question might be: "Which international organization, founded in 1999, includes the country whose capital is Beijing as a member?" The LLM must generate questions that are answerable from the chain, non-trivial (requiring multi-step reasoning), and well-formed.

The output is a set of (question, entity_chain, answer) tuples where each question requires multi-hop reasoning to connect the start entity to the answer through the intermediate entities.

Stage 2: Trajectory Synthesis.

For each generated question, the paper employs "multiple agent frameworks to address the generated queries, sampling a diverse set of reasoning paths." This means: give the question to different search agent systems (potentially with different tool configurations, search strategies, or base models) and record how they solve it—what searches they perform, what pages they visit, what information they extract, and what reasoning steps they take. Each successful solution becomes a trajectory: a sequence of (action, observation, reasoning_step) tuples showing the agent's step-by-step problem-solving process.

These trajectories are then "mapped into multi-turn tool-invocation sequences from a unified agent perspective." This mapping standardizes the action space: instead of each agent framework having its own tool-calling format, all trajectories are converted to a common format representing tool invocations (search query, visit page, extract content, run Python code, etc.) as turns in a conversation. This standardization is necessary for training a single model on trajectories from diverse sources; without it, the model would need to learn multiple incompatible tool-calling formats.

The outcome is a dataset of (question, trajectory) pairs where each trajectory is a sequence of tool-calling turns that successfully answers the question.

Stage 3: Turn-Level Critic Filtering.

Raw model-generated trajectories may contain suboptimal or incorrect steps—the agent framework might succeed overall while making unnecessary searches, extracting irrelevant information, or taking inefficient paths. Training on such trajectories would teach the model inefficient or noisy search behavior. To address this, the paper implements a rigorous turn-level judgment mechanism:

"Specifically, we employ a critic model to evaluate each step of the interaction based on three dimensions: logical soundness, tool-call accuracy, and informational gain."

Each turn in each trajectory receives a score along three axes:

Logical soundness: Is the reasoning step coherent and justified given the previous observations? Does it correctly interpret retrieved information?
Tool-call accuracy: Is the tool invocation correctly formatted? Does it use appropriate parameters? Does the tool choice match the intended action (e.g., searching vs. visiting a known URL)?
Informational gain: Does this step contribute new, relevant information toward answering the question, or is it redundant with previous steps?

The filtering criterion: "Any turn that fails to meet these criteria does not participate in model training or provides a negative reward for the model." This means turns that pass all three criteria are used as positive training examples, while turns that fail can serve as negative examples (what not to do). The result is a dataset where every turn in every trajectory has been validated by the critic, ensuring that the training signal is high-fidelity—the model learns successful search behavior from turns that demonstrably advance the task, not from turns that happen to appear in successful trajectories but contributed nothing or were actively counterproductive.

This turn-level supervision is the paper's key innovation for long-horizon search: prior approaches often only reward the final answer (sparse reward at trajectory end), which provides no learning signal for intermediate steps and makes credit assignment extremely difficult over hundreds of turns. By providing per-turn quality signals, the model receives immediate feedback on each action, making it feasible to learn stable, coherent search behavior.

Controlled experiment validation. Table 3 demonstrates that this data pipeline works independently of other training components. The experiment: take Nanbeige4-3B-2511 as the base model, train it exclusively on the synthetic multi-hop QA and search trajectories (intentionally excluding other open-source data), and evaluate on search benchmarks. Results show massive improvements: GAIA from 19.4 to 58.3 (+38.9), BrowseComp from 0.8 to 14.4 (+13.6), xBench-DeepSearch-05 from 33.0 to 76.0 (+43.0). These gains come purely from the synthetic data—no code RL, no pair-wise RL, no general RL improvements—proving that the data construction methodology is independently effective.

The evaluation is conducted within the Mindflow framework using specific tools: Serper for web search, Jina for webpage content extraction, and E2B Sandbox for secure code execution. HuggingFace is explicitly disabled in these tools, preventing the model from shortcutting the search process by retrieving cached answers. This tool configuration is realistic (commercial search API + content extraction + sandboxed execution) and represents a standard agent evaluation setup.

Code RL: Two-Stage Optimization with Gated Time-Complexity Rewards

The code RL system represents the most technically complex component, with a shared judge system, offline data construction with difficulty filtering, and a two-stage training process that progresses from correctness-only optimization to joint correctness-and-efficiency optimization.

Unified Judge System.

The judge system is a shared infrastructure component used across SFT data construction, RL data construction, RL training, and evaluation. It has two sub-components:

Multi-language sandbox for execution-based correctness checking. The sandbox compiles and runs generated code against test cases, returning a pass/fail signal for each test case. This provides ground-truth functional correctness: a solution either passes the tests or doesn't, with no model-based estimation involved. The paper does not specify which languages are supported, but the benchmarks (LiveCodeBench, LeetCode) suggest at minimum Python.
Instruct judge model for time-complexity comparison. This is a dedicated model trained specifically to assess algorithmic time complexity of code solutions. It compares "the predicted complexity of model-generated solutions against the reference optimal bound" using model-based analysis rather than empirical runtime measurement. The model is described as "specifically trained for fast complexity assessment in RL settings," emphasizing speed—during RL training, the judge must evaluate many candidate solutions quickly, and running actual timed executions would be too slow and environment-dependent (varying with hardware, input size, background load). The model-based approach provides instantaneous complexity estimates, enabling online reward computation.

The judge system's dual nature—sandbox for correctness, model for complexity—reflects a fundamental asymmetry: correctness is objective and verifiable (the code produces the right output), while time complexity is asymptotic and requires understanding the algorithm's structure. You cannot determine O(n) vs O(n²) by running the code on a few test cases; you need to analyze the algorithm. The instruct judge model fills this gap by providing complexity labels that would otherwise require human expert annotation.

SFT Data Construction for Code.

During SFT data generation, the judge system is used to assess solution quality from two aspects: functional correctness (executing in the sandbox) and time complexity (combining execution signals with model-based complexity analysis). The procedure: for each coding problem, sample multiple candidate solutions (presumably from strong models or the base model itself at high temperature), verify each candidate through the judge system, and keep only those judged to be "time-optimal (or among the best complexity class) for the given problem."

This filtering ensures that the SFT code data contains not just correct solutions but efficient correct solutions. If a problem has both an O(n²) and an O(n log n) correct solution, only the O(n log n) version is retained. The model trained on this data learns to associate correctness with efficiency, establishing a prior that good solutions are efficient solutions—a prior that the RL stage will then reinforce.

RL Data Construction with On-Policy Filtering.

Each RL training sample contains: a problem statement, test cases, a time-complexity-optimal solution, and the corresponding optimal complexity label. The optimal solution and complexity are obtained through a two-step process:

Prompt multiple strong LLMs to generate candidate solutions for the problem.
Employ a strong LLM to synthesize these candidates into a single best solution, selecting the most efficient correct approach and annotating its complexity class.

This reference solution serves as the supervision signal for reward shaping and difficulty control—it defines what "optimal" means for the problem, and the model's outputs are evaluated against it.

On-policy difficulty filtering. The paper performs filtering by running multiple rollouts per problem (n = 8) and selecting samples based on how many rollouts meet a stage-specific criterion. This is a form of curriculum learning based on current policy capability:

Stage 1 (difficulty-based criterion): A problem is retained if the policy can solve it in a moderate number of rollouts—specifically, "k in [1, 5] successful solves out of 8." Problems with 0 successful solves are too hard (the policy has essentially no chance of solving them, so training on them would provide no positive signal); problems with 6-8 successful solves are too easy (the policy already solves them reliably, so training on them provides minimal learning). The [1,5] range identifies problems in the policy's "zone of proximal development"—solvable but not trivially so.
Stage 2 (complexity-based criterion): After the policy has learned to solve problems correctly, the criterion shifts to efficiency. The paper counts "how many rollouts produce solutions whose estimated time complexity satisfies the target bound, and retain problems with k in [1, 5] complexity-satisfying rollouts out of 8." This identifies problems where the policy sometimes produces efficient solutions and sometimes doesn't—exactly the problems where efficiency-focused training can make a difference.

This on-policy filtering improves sample efficiency: rather than training on all available problems, the model focuses its RL budget on problems where it has room to improve. Problems that are trivially easy or impossibly hard are filtered out, concentrating compute on marginal improvements.

Stage 1 Code RL: Correctness Optimization.

Starting from the General-RL checkpoint (after point-wise and pair-wise RL), Stage 1 optimizes only for functional correctness. The reward for a generated solution is defined as:

$R = R_{\text{format}} + R_{\text{correctness}}$

where $R_{\text{format}}$ is a formatting reward (encouraging well-structured, parseable code) and $R_{\text{correctness}}$ is the pass-rate reward.

What it computes: $R_{\text{correctness}}$ is "defined as the fraction of test cases passed for each problem"—a scalar between 0 (no tests passed) and 1 (all tests passed). If a problem has 10 test cases and the solution passes 7, $R_{\text{correctness}} = 0.7$ . The total reward is the sum of the formatting score and the pass-rate score.

Why this form: using pass-rate rather than binary pass/fail provides a continuous reward signal. A solution that passes 9/10 tests is much closer to correct than one that passes 1/10, and the reward reflects this gradation. A binary reward (1 for all tests passed, 0 otherwise) would give the same signal to a near-miss (9/10) and a complete failure (0/10), making it harder for RL to make progress—the model would have to stumble upon a fully correct solution by chance before receiving any positive signal. The continuous reward creates a smoother optimization landscape.

The paper reports training dynamics (Figure 4): "In Stage 1, the correctness reward increases sharply, reflecting rapid gains in producing valid and correct solutions." This is expected—the model starts from the General-RL checkpoint, which already has some code ability (LiveCodeBench V6 = 65.6), and specialized code training rapidly improves correctness.

Stage 2 Code RL: Gated Time-Complexity Optimization.

After the policy can reliably produce correct solutions, Stage 2 adds an efficiency dimension. The reward becomes:

$R = \begin{cases} R_{\text{format}} + R_{\text{correctness}}, & \text{PassRate} < 1 \\ R_{\text{format}} + R_{\text{correctness}} + R_{\text{time}}, & \text{PassRate} = 1 \end{cases}$

where $R_{\text{time}}$ is the time-complexity reward, and $\text{PassRate} = 1$ means all test cases pass (the solution is fully correct).

What it computes: if the solution is not fully correct ( $\text{PassRate} < 1$ ), the reward is identical to Stage 1—only correctness and formatting matter. If the solution is fully correct ( $\text{PassRate} = 1$ ), an additional $R_{\text{time}}$ term is added. The judge system computes $R_{\text{time}}$ by "comparing the model's output against the reference optimal complexity and checking whether the generated solution matches the reference optimal solution when applicable." The paper does not specify the exact scalar value of $R_{\text{time}}$ , but it is presumably positive when the solution's complexity matches or approaches optimal, and zero or negative otherwise.

The critical design choice is the gating condition $\text{PassRate} = 1$ : efficiency is rewarded only for solutions that are already correct. This prevents the model from learning to trade correctness for speed—producing an O(n) solution that fails half the test cases would receive only $R_{\text{format}} + R_{\text{correctness}}$ without the efficiency bonus, while an O(n log n) solution that passes all tests would receive the full bonus. An alternative design where $R_{\text{time}}$ was always active would risk the model optimizing for efficiency even when producing incorrect solutions, which would degrade overall quality.

Figure 4 shows the training dynamics: "In Stage 2, $R_{\text{correctness}}$ improves more modestly, while the gated time reward rises substantially, indicating that the policy is indeed optimizing time complexity once correctness is largely achieved." This pattern—correctness saturating, efficiency rising—validates the gated design: the model first masters correctness, then shifts its optimization budget toward efficiency, without the two objectives conflicting.

The concrete benefit of the time reward is illustrated in Appendix C, which presents case studies from LiveCodeBench comparing model outputs before vs. after the time-reward stage:

AtCoder ABC388: The "before" solution uses binary search with feasibility checks, yielding O(N log N) time. The "after" solution uses a two-pointer greedy approach, yielding O(N) time. Both are correct; the latter is asymptotically faster.
AtCoder ARC195: The "before" solution builds a value-to-positions map and performs binary search per element, yielding O(N + M log N). The "after" solution uses two linear scans, yielding O(N). Again, both correct but the latter is faster.
AtCoder ARC194: The "before" solution simulates bubble swaps with a heap, yielding O(N² log N) worst-case. The "after" solution uses a Fenwick tree for O(N log N). This is the most dramatic improvement: O(N² log N) to O(N log N).

These examples demonstrate that the time reward produces genuine algorithmic improvements—not just benchmark score inflation through memorization of efficient solutions, but the ability to derive more efficient approaches to novel problems.

Training dynamics visualization. Figure 4 tracks both reward components and LiveCodeBench performance across the two stages. The plot shows: $R_{\text{correctness}}$ rising sharply in Stage 1 and plateauing in Stage 2; $R_{\text{time}}$ rising from near-zero in Stage 2; and LiveCodeBench-V6 score (tracked as a downstream metric) continuing to improve throughout both stages. This validates that efficiency optimization translates to benchmark performance, not just training reward.

Full Training Recipe: Staged Integration

Section 2.4 provides the integration recipe that sequences all components into a single training pipeline:

Initialization: Start from Nanbeige4-3B-Base.
Extended SFT: Conduct supervised fine-tuning with the three-stage length curriculum (32K → 64K → 256K), using the redesigned data mixture and upgraded solution refinement. This produces Nanbeige4.1-3B-SFT.
General RL (sequential): Apply point-wise RL using GRPO with the general reward model, followed by pair-wise RL with the pair-wise reward model and swap-consistency regularization. This produces a model with strong alignment and stabilized outputs.
Code RL (two-stage): Apply correctness-only RL (Stage 1) followed by gated time-complexity RL (Stage 2), using the on-policy difficulty filtering at each stage. This layers specialized code expertise on top of the general capabilities.
Lightweight Agentic RL: Apply a final RL stage focused on tool-use and search behavior. The paper describes this as "lightweight," suggesting it uses fewer training steps or a smaller dataset than the other RL stages, serving as a fine-tuning step rather than a fundamental capability injection.

The key integration insight is the staged sequencing: general capabilities first (SFT + general RL), then specialized capabilities (code RL, agentic RL). This ordering reflects a hierarchy of skill dependencies: the model needs basic reasoning and alignment before it can benefit from specialized code optimization (which assumes the model can understand problem statements and produce coherent code); similarly, deep search capabilities benefit from general reasoning and tool-use skills established earlier. The paper's claim that the model "maintains strong domain-specific performance while emerging as a well-balanced generalist" depends on this sequencing preventing catastrophic interference—if specialized training were applied first, the model might overfit to code or search and lose general capabilities; by establishing general competence first and then specializing, the general skills serve as a foundation that specialized training refines rather than replaces.

The paper does not provide detailed hyperparameters for each RL stage (learning rates, batch sizes, KL penalty coefficients, etc.), which limits reproducibility. However, the contribution is framed as the integration methodology and reward design rather than specific hyperparameter configurations—the claim is that this particular sequence of reward structures and data compositions produces broad capability, and the specific optimization hyperparameters are implementation details that may need tuning for different base models or compute budgets.

4. Key Insights and Innovations

Innovation 1: Capability Fragmentation as a Training Objective Design Problem, Not a Capacity Constraint

The paper's most fundamental conceptual move is reframing the fragmentation of capabilities in small language models—where reasoning models can't search, code models can't reason, and search models aren't generalists—as a training objective design problem rather than an irreducible consequence of limited parameter count. This is not the default assumption in the field.

The dominant implicit framing in SLM research, visible in the specialized models the paper cites (Vibethinker-1.5B for reasoning, various code-specialized models, dedicated search agents like AgentCPM-Explore-4B), treats fragmentation as natural: you pick a capability, you design training around it, and you accept that other capabilities will be absent or degraded. The logic is that a 3B model simply doesn't have enough capacity to represent multiple expert behaviors simultaneously, so you must specialize. Under this view, the paper's goal—a single 3B model that simultaneously reasons, codes, aligns with preferences, and executes 600-turn search trajectories—would be dismissed as attempting to fit "too much" into too few parameters.

The paper challenges this framing not through theoretical argument but through an existence proof: Nanbeige4.1-3B exists and works. The empirical evidence that makes this reframing credible is the breadth of the results. Table 4 shows the model surpassing Qwen3-30B-A3B on coding benchmarks (76.9 vs. 66.0 on LiveCodeBench-V6) while simultaneously leading on alignment (73.2 vs. 60.2 on Arena-Hard-V2). Table 5 shows it matching specialized 30B research agents on deep search (69.90 on GAIA vs. 70.90 for Tongyi-DeepResearch-30B). Table 6 shows 85% pass rate on real LeetCode contests. No single specialized 3B model achieves any one of these numbers across all three domains—Nanbeige4.1-3B achieves all of them in one checkpoint.

The intellectual significance is that this reframes the relationship between model scale and capability breadth. If fragmentation were a hard capacity constraint, then increasing parameter count would be the only path to generalist behavior. By demonstrating that a 3B model can be a generalist when the training objectives are properly designed, the paper shifts the research question from "how many parameters do we need for broad competence?" to "what training signal structures enable broad competence at fixed scale?" This is a more productive question because it directs attention toward reward design, data composition, and training stage sequencing—all choices under practitioner control—rather than toward parameter count, which is often constrained by deployment requirements.

The paper's own ablation structure reinforces this framing. Table 1 shows that SFT data mixture changes alone produce +16.5 on LiveCodeBench and +14.7 on Arena-Hard. Table 2 shows that the sequencing of point-wise then pair-wise RL produces alignment gains (+7.2 on Arena-Hard from pair-wise) that don't come at the expense of other capabilities (code dips only -0.4). Table 3 shows that synthetic search data alone can move a model from 19.4 to 58.3 on GAIA—a near-tripling of performance from data construction, not architecture changes. Each component is individually validated, collectively demonstrating that the full capability profile emerges from the composition of training signals, not from any one magic ingredient or from scale.

This is a fundamental rather than incremental contribution. It doesn't propose a new algorithm per se—GRPO, pair-wise preference optimization, and gated rewards each exist in prior work. Instead, it proposes a new diagnosis of the problem (fragmentation is about objectives, not capacity) and validates that diagnosis with an engineered artifact that shouldn't exist under the old framing. The contribution is the reframing itself, backed by the model's existence.

Innovation 2: The Gated Time-Complexity Reward as a Solution to the Multi-Objective Interference Problem in Code RL

Code generation RL typically optimizes a single objective: functional correctness. Solutions that pass test cases are rewarded; solutions that fail are penalized. This is simple, robust, and widely used. Its limitation, which the paper identifies, is that it treats an O(n²) and an O(n log n) solution identically as long as both pass the tests—the reward signal provides no pressure toward algorithmic efficiency, even though efficiency is a critical dimension of code quality in practice.

The obvious fix—add an efficiency reward term alongside correctness—creates a multi-objective optimization problem where the two objectives can conflict. A model optimizing jointly for correctness and efficiency might learn to prefer a fast-but-incorrect solution (passing some tests, failing others, receiving partial correctness reward but full efficiency reward) over a correct-but-slow solution. The optimization landscape becomes one where the model can "game" the reward by trading correctness for speed, producing outputs that score well on the combined metric but are useless in practice.

The paper's solution—the gated time-complexity reward, shown in Figure 3 and defined in Section 2.3.3—is conceptually simple but addresses a real optimization pathology. The gating condition (PassRate = 1) means the efficiency reward is only active on the manifold of correct solutions. This transforms a potentially conflicting multi-objective problem into a hierarchical one: first reach correctness, then optimize efficiency along the correctness manifold. The model cannot receive efficiency credit for incorrect solutions, so there is no gradient pushing it to sacrifice correctness for speed.

This is an incremental but practically important innovation. Incremental because gated or conditional reward structures exist in other RL contexts (e.g., constrained RL, safety RL where constraint violation zeros out the reward). The paper does not claim the gating mechanism itself is novel. What is distinctive is the application to code efficiency and the specific insight that correctness and efficiency form a natural hierarchy, not two competing dimensions—a perspective that wasn't obvious before. Prior code RL work implicitly treated correctness as the only objective; the paper argues this is leaving performance on the table, then provides a mechanism to capture it without risking correctness regression.

The evidence for the practical importance is in Appendix C. The three before/after case studies show concrete algorithmic improvements: O(N log N) → O(N) (ABC388), O(N + M log N) → O(N) (ARC195), O(N² log N) → O(N log N) (ARC194). These are not marginal efficiency tweaks; they are asymptotic complexity class improvements that would matter in production settings with large inputs. The Figure 4 training dynamics show that R_correctness remains stable or improves slightly in Stage 2, validating that the gated design prevents correctness regression while R_time rises substantially.

The broader significance beyond code RL is as a design pattern for multi-capability RL. The core principle—identify a primary objective that must be satisfied first, then gate secondary objectives on primary satisfaction—generalizes beyond code. For instance, a search agent might first need to find the answer (primary: correctness), then optimize for fewer steps (secondary: efficiency), with the secondary reward gated on finding the correct answer. A conversational agent might first need to be helpful (primary), then optimize for conciseness (secondary). The paper doesn't make this generalization claim explicitly, but the pattern is visible and transferable.

Innovation 3: Turn-Level Supervision as the Missing Ingredient for Long-Horizon Search in Small Models

The deep search results (Table 5) are the most striking numbers in the paper: a 3B model scoring 69.90 on GAIA and 75.00 on xBench-DeepSearch-05, matching or exceeding specialized 30B research agents and 100B+ foundation models. But the intellectual contribution is not "we got good numbers on search benchmarks"—it's the diagnosis of why prior SLMs fail at search and the specific data construction methodology that addresses the diagnosed failure mode.

Prior SLMs with tools (Qwen3-4B, Qwen3-8B, the baselines in Table 5) score near zero on long-horizon browsing tasks like BrowseComp (0.79–1.57). The paper's diagnosis, implied in Section 1 ("fail to sustain meaningful exploration beyond a few turns"), is that these models haven't learned credit assignment over long horizons. When a search agent takes 50 tool-calling turns and finally arrives at an answer, how does the model know which of those 50 turns contributed to success and which were dead ends? Standard RL with a sparse trajectory-level reward (reward at the end if the answer is correct, zero otherwise) provides essentially no learning signal for individual intermediate actions in a 50-step trajectory—the credit assignment problem is too hard.

The paper's solution—turn-level supervision via the critic model described in Section 2.2.1—addresses credit assignment by providing a per-step quality signal. The critic evaluates each turn on three dimensions (logical soundness, tool-call accuracy, informational gain), and only turns that pass all three are used as positive training examples. This means the model receives immediate feedback on each action: "this search query was well-formulated and produced useful information" versus "this page visit extracted irrelevant content." The credit assignment problem is solved by the critic, not by the RL algorithm—the critic does the hard work of evaluating intermediate steps, and the model learns to imitate the critic's judgments.

This is fundamental rather than incremental because it identifies a specific bottleneck (credit assignment over long horizons) that was not the focus of prior SLM research, and proposes a specific solution (turn-level critic filtering) that directly addresses it. Prior work on search agents, including the research agents in Table 5, may have used similar per-step signals implicitly (through human-designed prompts, through tool-specific fine-tuning, or through larger models that can better handle sparse rewards), but the paper makes the mechanism explicit and demonstrates its importance at the 3B scale, where sparse rewards would be especially ineffective due to limited model capacity for credit assignment.

The controlled experiment in Table 3 provides the critical evidence. Training Nanbeige4-3B-2511 exclusively on the synthetic QA and search trajectories—with no other data, no other training innovations—improves GAIA from 19.4 to 58.3 and xBench-DeepSearch-05 from 33.0 to 76.0. These are enormous gains from data alone, validating that the data construction methodology (entity-relation walks for question synthesis, multi-agent trajectory generation, turn-level critic filtering) is independently responsible for the search capability improvement. The full model's even higher scores (69.90 on GAIA) reflect the additional benefits of general RL and agentic fine-tuning, but the majority of the gain comes from the data pipeline.

The broader significance is as a template for teaching long-horizon behaviors to small models. The pattern—synthesize diverse trajectories, use a critic to provide per-step quality labels, train the model to replicate only the high-quality steps—generalizes to any domain where sparse rewards make credit assignment difficult: multi-step planning, complex tool use, dialogue systems with long conversation histories. The critic model acts as a "dense reward generator," converting sparse trajectory-level signals into dense step-level signals, which is exactly what small models need to learn stable long-horizon behavior.

Innovation 4: The Sequencing of Heterogeneous RL Objectives as a Distinct Design Axis

Multi-stage RL is common in LLM training—most instruction-tuned models go through SFT → RLHF or some variant. What distinguishes this paper's approach is the explicit reasoning about the order and interdependence of domain-specific RL stages, rather than treating them as independent modules that can be run in any order or combined into a single multi-task objective.

The pipeline (Section 2.4) sequences: SFT → point-wise RL → pair-wise RL → code RL (two-stage) → agentic RL. Each stage depends on properties established by the previous stage, and the ordering choices are non-arbitrary:

Point-wise before pair-wise because pair-wise comparisons are wasted on poorly formatted or repetitive outputs—the point-wise stage cleans up surface-level issues so the pair-wise stage can focus on substantive quality differences. Table 2 shows this pays off: point-wise RL fixes the 5.27% truncation rate and improves length stability, then pair-wise RL delivers the +7.2 on Arena-Hard.
General RL before specialized RL because code and agentic capabilities build on general reasoning and alignment. A model that can't understand problem statements or produce coherent text won't benefit from code RL; the general RL stage ensures the foundation is solid. The paper's claim that the model "maintains strong domain-specific performance" (Section 2.4) implicitly argues that this ordering prevents specialized training from overwriting general capabilities that were established first.
Correctness-only before efficiency in code RL because efficiency optimization on incorrect solutions produces garbage. Stage 1 establishes a high correctness rate; Stage 2 adds efficiency pressure. Figure 4 validates this: R_correctness rises first, R_time rises later, and correctness doesn't regress.

This is an incremental but underappreciated innovation. Multi-stage training is not new, but the deliberate dependency-aware sequencing—each stage designed to create conditions under which the next stage can be effective—is not standard practice. The dominant approach in the field is either single-stage multi-task RL (where all objectives are combined into one reward) or sequential RL where the order is determined by convenience (e.g., run the cheapest stage first) rather than by dependency analysis.

The significance is that sequencing is a free design parameter—it doesn't require new algorithms, new data, or more compute, only a decision about what order to apply existing components. If the paper's dependency-aware ordering produces better results than arbitrary ordering (the paper doesn't include the ablation that would prove this, which is a limitation), it implies that many multi-capability training pipelines are leaving performance on the table simply by running stages in the wrong order. This is a methodological contribution that transfers to any multi-stage training system, not just this specific model.

The evidence is indirect but suggestive: the fact that Nanbeige4.1-3B achieves broad competence without destructive interference between domains (code, search, alignment all improve simultaneously) is itself evidence that the sequencing works. Catastrophic forgetting between domains would manifest as, for example, code RL degrading alignment or agentic RL reducing coding ability. The paper's tables show consistent gains across all domains at each stage, suggesting the sequencing successfully prevents interference. A direct ablation comparing different orderings would be stronger evidence, but the convergence of positive results across all domains is consistent with the claimed benefit of dependency-aware sequencing.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper evaluates across multiple benchmarks spanning general reasoning, deep search, and real-world coding challenges. For general reasoning, it uses LiveCodeBench-V5 [6], LiveCodeBench-V6 [6], and LiveCodeBench-Pro [22] for code; AIME 2026 I (from MathArena), HMMT Nov [1], and IMO-Answer-Bench [8] for mathematics; GPQA [13] and HLE (text-only subset) [12] for science; Arena-Hard-V2 [7] and Multi-Challenge [4] for alignment; and BFCL-V4 [10] and Tau2-Bench [2] for tool use. For deep search, it evaluates on GAIA (text-only) [9], BrowseComp [16], BrowseComp-ZH [23], HLE (text-only), SEAL-0 [11], xBench-DeepSearch-05 [3], and xBench-DeepSearch-10 [3]. For real-world stress testing, it uses LeetCode Weekly Contests 484–488. Dataset splits are standard for each benchmark; no custom splits are created. The specific test questions for AIME 2026 I are sourced from HuggingFace at MathArena/aime_2026_I.

Base model. The model is initialized from Nanbeige4-3B-Base, a 3B-parameter pretrained language model described in the prior Nanbeige4-3B technical report [21]. The choice is motivated by continuity with the model family: the paper positions itself as an upgrade from Nanbeige4-3B-2511, and using the same base model enables direct comparisons that isolate the effect of the new post-training methodology. All subsequent training—SFT, general RL, code RL, agentic RL—builds on this base checkpoint.

Metrics. Across all benchmarks, the primary metric is accuracy or pass rate—the fraction of test instances for which the model's output matches the ground-truth answer or passes execution-based verification. For coding benchmarks (LiveCodeBench, LeetCode), correctness is determined by execution against test cases in a sandbox, following the benchmark's standard evaluation protocol. For mathematics and science benchmarks, answers are compared to ground-truth solutions following each benchmark's grading conventions (exact match or equivalent). For alignment benchmarks (Arena-Hard-V2, Multi-Challenge), automated evaluation protocols from the respective benchmarks are used. For deep search benchmarks, answers are evaluated against ground-truth following each benchmark's standard procedures. The paper does not report confidence intervals, statistical significance tests, or variance estimates.

Baselines. The paper compares against a comprehensive set of models across different scales and specializations:

Same-scale generalist SLMs: Qwen3-4B-2507 (the most recent Qwen3 4B release at time of writing).
Larger generalist models: Qwen3-8B, Qwen3-14B, Qwen3-32B, Qwen3-30B-A3B-2507 (a 30B-parameter Mixture-of-Experts model with 3B active parameters), Qwen3-Next-80B-A3B (80B MoE, 3B active).
Previous Nanbeige release: Nanbeige4-3B-2511, representing the direct predecessor and enabling within-family ablation.
Specialized search agents (for deep search tasks): Tongyi-DeepResearch-30B, MiroThinker-v1.0-8B, AgentCPM-Explore-4B.
Large foundation models with tools (for deep search tasks): GLM-4.6-357B, Minimax-M2-230B, DeepSeek-V3.2-671B.

For the LeetCode evaluation (Table 6), baselines are Qwen3-4B-2507, Qwen3-32B, and Qwen3-30B-A3B-2507.

Generation budget / compute accounting. The paper does not report generation budgets or test-time compute for individual evaluations. Results are reported as single-point accuracy numbers without generation budget sweeps. This is a departure from the scaling analysis style of the reference example paper—there is no "how performance scales with N generations" analysis. For deep search tasks, the number of tool-calling turns is not explicitly controlled or reported per-evaluation; the model interacts with tools freely within the Mindflow framework. For code benchmarks, standard settings from each benchmark are used (typically pass@1 with greedy or single-sample decoding). The paper does not specify decoding parameters (temperature, top-p, etc.) for any evaluation.

Cross-validation / statistical protocol. The paper does not report any cross-validation protocol, statistical significance testing, or confidence intervals. All results are reported as point estimates. For the controlled deep search data experiment (Table 3), the training uses the synthetic data only, and evaluation is performed on the standard benchmark test sets. For the SFT ablation (Table 1), the comparison is between two specific checkpoints (Nanbeige4-3B-SFT vs. Nanbeige4.1-3B-SFT) evaluated once on each benchmark. For the RL stage ablation (Table 2), results are reported for three sequential checkpoints evaluated once each.

Main Quantitative Results

General Task Performance (Table 4)

The headline result is that Nanbeige4.1-3B substantially outperforms both its predecessor and all same-scale baselines across all five evaluation categories (code, math, science, alignment, tool use), while surpassing much larger models on the majority of benchmarks.

Code benchmarks. On LiveCodeBench-V6, Nanbeige4.1-3B scores 76.9, compared to 57.4 for Qwen3-4B-2507 (+19.5), 55.7 for Qwen3-32B (+21.2), 66.0 for Qwen3-30B-A3B-2507 (+10.9), and 68.7 for Qwen3-Next-80B-A3B (+8.2). The improvement over Nanbeige4-3B-2511 (46.0) is +30.9—nearly doubling the score. On LiveCodeBench-Pro-Easy, Nanbeige4.1-3B achieves 81.4, compared to 40.2 for Qwen3-4B-2507 (+41.2), 42.3 for Qwen3-32B (+39.1), and 60.8 for Qwen3-30B-A3B-2507 (+20.6). On LiveCodeBench-Pro-Medium, the most challenging code subset, it scores 28.1, versus 5.3 for Qwen3-4B-2507 and 5.3 for Nanbeige4-3B-2511 (+22.8), and 3.5 for Qwen3-30B-A3B-2507 (+24.6). The model's 28.1 on Pro-Medium is nearly double the next-best baseline (Qwen3-Next-80B-A3B at 14.3).

Mathematics. On AIME 2026 I, Nanbeige4.1-3B scores 87.40, compared to 81.46 for Qwen3-4B-2507, 75.83 for Qwen3-32B, and 87.30 for Qwen3-30B-A3B-2507 (essentially tied). It slightly trails Qwen3-Next-80B-A3B (89.24) by -1.84. On HMMT Nov, it scores 77.92, outperforming Qwen3-4B-2507 (68.33) and Qwen3-32B (57.08), and exceeding Qwen3-30B-A3B-2507 (71.25), while trailing Qwen3-Next-80B-A3B (81.67). On IMO-Answer-Bench, it scores 53.38, compared to 48.00 for Qwen3-4B-2507, 43.94 for Qwen3-32B, and 54.34 for Qwen3-30B-A3B-2507 (essentially tied), trailing Qwen3-Next-80B-A3B (58.00). The improvement over Nanbeige4-3B-2511 ranges from +3.30 (AIME) to +15.13 (IMO-Answer-Bench).

Science. On GPQA, Nanbeige4.1-3B scores 83.8, outperforming Qwen3-4B-2507 (65.8, +18.0), Qwen3-32B (68.4), Qwen3-30B-A3B-2507 (73.4), and Qwen3-Next-80B-A3B (77.2). This is a +1.6 improvement over Nanbeige4-3B-2511 (82.2). On HLE (text-only), it scores 12.60, compared to 6.72 for Qwen3-4B-2507 (+5.88), 9.31 for Qwen3-32B, and 11.77 for Qwen3-30B-A3B-2507 (+0.83), while trailing Qwen3-Next-80B-A3B (13.70). The improvement over Nanbeige4-3B-2511 (10.98) is +1.62.

Alignment. On Arena-Hard-V2, Nanbeige4.1-3B scores 73.2 (Table 4 reports this as 73.2, while Table 2 reports 73.8 for the pair-wise RL checkpoint—a discrepancy of 0.6 points that may reflect evaluation variance or different checkpoint selections). This compares to 34.9 for Qwen3-4B-2507 (+38.3), 56.0 for Qwen3-32B, 60.2 for Qwen3-30B-A3B-2507 (+13.0), and 62.3 for Qwen3-Next-80B-A3B (+10.9). On Multi-Challenge, it scores 52.21, versus 41.14 for Qwen3-4B-2507 (+11.07), 38.72 for Qwen3-32B, and 49.40 for Qwen3-30B-A3B-2507 (+2.81), while trailing Qwen3-Next-80B-A3B (56.52). The improvement over Nanbeige4-3B-2511 (41.20) is +11.01.

Tool use. On BFCL-V4, Nanbeige4.1-3B scores 56.50, compared to 44.87 for Qwen3-4B-2507 (+11.63), 47.90 for Qwen3-32B, 48.60 for Qwen3-30B-A3B-2507 (+7.90), and 50.51 for Qwen3-Next-80B-A3B (+5.99). On Tau2-Bench, it scores 48.57, versus 45.90 for Qwen3-4B-2507 (+2.67), 45.26 for Qwen3-32B, and 47.70 for Qwen3-30B-A3B-2507 (+0.87), while trailing Qwen3-Next-80B-A3B (57.40).

Cross-scale pattern. The most striking pattern in Table 4 is that Nanbeige4.1-3B surpasses Qwen3-30B-A3B-2507 (10× its parameter count) on 9 out of 11 benchmarks, with the exceptions being AIME 2026 I (essentially tied) and Multi-Challenge (trailing by 2.25). It surpasses Qwen3-32B on all 11 benchmarks. Against Qwen3-Next-80B-A3B, the model leads on LiveCodeBench-V6, LiveCodeBench-Pro-Easy, LiveCodeBench-Pro-Medium, GPQA, Arena-Hard-V2, and BFCL-V4—6 of 11 benchmarks—while trailing on AIME, HMMT, IMO-Answer-Bench, HLE, Multi-Challenge, and Tau2-Bench.

Deep Search Performance (Table 5)

The headline result is that Nanbeige4.1-3B achieves state-of-the-art performance on the majority of deep search benchmarks, matching or exceeding specialized research agents with 10× or more parameters and large foundation models with 100× or more parameters.

GAIA (text-only). Nanbeige4.1-3B scores 69.90, compared to 19.42 for Nanbeige4-3B-2511 (+50.48—a 3.6× improvement). Among small foundation models with tools, the next best is Qwen3-Next-80B-A3B at 34.02, less than half Nanbeige4.1-3B's score. Among specialized research agents, Tongyi-DeepResearch-30B scores 70.90 (+1.00 advantage) and MiroThinker-v1.0-8B scores 66.40 (−3.50 disadvantage). Among large foundation models with tools, Minimax-M2-230B scores 75.70 (+5.80), GLM-4.6-357B scores 71.90 (+2.00), and DeepSeek-V3.2-671B scores 63.50 (−6.40). Nanbeige4.1-3B's GAIA score is within 5.8 points of the best large foundation model (Minimax-M2-230B, ~77× larger) and surpasses DeepSeek-V3.2-671B (~224× larger).

BrowseComp. Nanbeige4.1-3B scores 19.12, compared to 0.79 for Nanbeige4-3B-2511 (+18.33). Among small foundation models with tools, the next best is Qwen3-Next-80B-A3B at 5.60 (−13.52). DeepSeek-V3.2-671B leads overall at 67.60 (−48.48 from Nanbeige4.1-3B). This is the benchmark where Nanbeige4.1-3B shows the largest gap relative to large foundation models, though it still dominates all same-scale and research agent baselines (Tongyi-DeepResearch-30B: 43.40; MiroThinker: 31.10; AgentCPM-Explore-4B: 25.00).

BrowseComp-ZH. Nanbeige4.1-3B scores 31.83, compared to 3.09 for Nanbeige4-3B-2511 (+28.74). Among small foundation models with tools, the next best is Qwen3-Next-80B-A3B at 8.25 (−23.58). DeepSeek-V3.2-671B scores 65.00 (−33.17).

HLE (text-only). Nanbeige4.1-3B scores 22.29, compared to 13.89 for Nanbeige4-3B-2511 (+8.40). Among small foundation models, Qwen3-30B-A3B-2507 scores 14.81 (−7.48). DeepSeek-V3.2-671B scores 40.80 (−18.51). Tongyi-DeepResearch-30B scores 32.90 (−10.61).

SEAL-0. Nanbeige4.1-3B scores 41.44, compared to 12.61 for Nanbeige4-3B-2511 (+28.83). Among research agents, MiroThinker scores 40.40 (−1.04), AgentCPM-Explore-4B scores 40.00 (−1.44). DeepSeek-V3.2-671B scores 38.50 (−2.94). Nanbeige4.1-3B leads all listed models on this benchmark, including DeepSeek-V3.2-671B.

xBench-DeepSearch-05. Nanbeige4.1-3B scores 75.00—the maximum among all listed models—compared to 33.00 for Nanbeige4-3B-2511 (+42.00). Tongyi-DeepResearch-30B scores 75.00 (tied). DeepSeek-V3.2-671B scores 71.00 (−4.00). Among small foundation models, the next best is Qwen3-32B at 39.00 (−36.00).

xBench-DeepSearch-10. Nanbeige4.1-3B scores 39.00, compared to 11.00 for Nanbeige4-3B-2511 (+28.00). Among small foundation models, Qwen3-30B-A3B-2507 scores 10.00 (−29.00). No research agent or large foundation model scores are reported for this benchmark in the comparison table.

Key patterns. The deep search results show three notable patterns. First, the improvement from Nanbeige4-3B-2511 to Nanbeige4.1-3B is dramatic across all benchmarks—ranging from +8.40 (HLE) to +50.48 (GAIA)—representing a qualitative capability leap rather than incremental improvement. Second, the gap between Nanbeige4.1-3B and other small foundation models with tools (Qwen3 series, 4B to 80B-A3B) is consistently enormous, often 2–4× larger than the next-best small model. Third, Nanbeige4.1-3B is competitive with or exceeds specialized research agents (Tongyi-DeepResearch-30B, MiroThinker-v1.0-8B) on all benchmarks, despite these agents being purpose-built for search with dedicated infrastructure and training while Nanbeige4.1-3B is a generalist model.

Real-World LeetCode Performance (Table 6)

On LeetCode Weekly Contests 484–488 (20 total problems), Nanbeige4.1-3B solves 17 out of 20 problems, achieving a pass rate of 85.0%. This compares to Qwen3-4B-2507 at 55.0% (+30.0 percentage points), Qwen3-32B at 50.0% (+35.0), and Qwen3-30B-A3B-2507 at 65.0% (+20.0). In virtual participation mode (where ranking is determined by both correctness and solution speed), the model places 1st in Weekly Contest 487 and 3rd in Weekly Contest 488.

The LeetCode evaluation is important because it represents an out-of-distribution stress test: these are new, unseen competitive programming problems released after the model's training data cutoff (presumably), with the generated solutions submitted directly to the official LeetCode platform for evaluation. This eliminates concerns about benchmark contamination or memorization that might affect standard benchmarks. The 85% pass rate on fresh contest problems is strong evidence that the coding capability is genuine rather than an artifact of training on benchmark-similar data.

Ablation Studies and Robustness Checks

SFT data mixture and quality improvements (Table 1): The delta from Nanbeige4-3B-SFT to Nanbeige4.1-3B-SFT isolates the effect of the redesigned data mixture, extended context length curriculum, and upgraded solution refinement/CoT reconstruction. Gains are substantial across all domains: +16.5 on LiveCodeBench V6, +21.0 on LiveCodeBench Pro Medium, +13.6 on HMMT Nov, +14.1 on IMO-Answer-Bench, +14.7 on Arena-Hard V2, and +1.8 on Multi-Challenge. The larger gains on code benchmarks (+16.5 to +21.0) relative to Multi-Challenge (+1.8) suggest the SFT improvements disproportionately benefit code and math over multi-turn conversational alignment, consistent with the increased proportion of code data and challenging math problems in the new mixture.

Point-wise RL (Table 2): Moving from SFT to point-wise RL improves LiveCodeBench V6 from 62.0 to 66.0 (+4.0), Arena-Hard V2 from 60.2 to 66.6 (+6.4), and Multi-Challenge from 44.4 to 47.7 (+3.3). The paper also reports that overlong truncation on LiveCodeBench-v6 drops from 5.27% to 0.38%, a ~14× reduction in a specific failure mode. This ablation validates that point-wise RL addresses the "repetition and redundant thinking" degradation observed after SFT, and that the general reward model's suppression of overly long outputs translates to measurable benchmark improvements.

Pair-wise RL (Table 2): Adding pair-wise RL on top of point-wise RL improves Arena-Hard V2 from 66.6 to 73.8 (+7.2) and Multi-Challenge from 47.7 to 55.1 (+7.4), while LiveCodeBench V6 slightly decreases from 66.0 to 65.6 (−0.4). This is the only stage in the ablation sequence where any benchmark regresses, and the regression is small (−0.4) relative to the alignment gains (+7.2 and +7.4). This suggests a mild tradeoff: the pair-wise RL's emphasis on conversational preference alignment slightly shifts the model away from pure code correctness optimization, but the net effect across domains is strongly positive.

Deep search synthetic data (Table 3): Training Nanbeige4-3B-2511 exclusively on the synthetic multi-hop QA and search trajectories (no other training data, no other RL stages) improves deep search performance dramatically: GAIA from 19.4 to 58.3 (+38.9), BrowseComp from 0.8 to 14.4 (+13.6), BrowseComp-ZH from 3.1 to 30.1 (+27.0), HLE from 13.9 to 22.4 (+8.5), SEAL-0 from 12.6 to 36.0 (+23.4), xBench-DeepSearch-05 from 33.0 to 76.0 (+43.0), and xBench-DeepSearch-10 from 11.0 to 30.0 (+19.0). This is the most important ablation in the paper because it isolates the synthetic data pipeline as the primary driver of deep search capability, independent of the general RL improvements. The fact that the synthesized QA and trajectory data alone can triple GAIA performance and more than double xBench-DeepSearch-05 performance demonstrates that the data construction methodology (temporal-aware entity-relation walks, multi-agent trajectory synthesis, turn-level critic filtering) is independently effective, not merely beneficial in combination with other training innovations. Comparing Table 3 to Table 5 shows that the full model with all training stages (including general RL and agentic RL) further improves over the synthetic-data-only model: GAIA from 58.3 to 69.90 (+11.6), SEAL-0 from 36.0 to 41.44 (+5.44), xBench-DeepSearch-10 from 30.0 to 39.00 (+9.0). The synthetic data provides the majority of the gain, with additional RL stages providing meaningful incremental improvements.

Two-stage code RL (Figure 4): The training dynamics visualization in Figure 4 tracks R_correctness, R_time, and LiveCodeBench-V6 performance across the two code RL stages. In Stage 1 (correctness-only), R_correctness increases sharply while R_time is near zero (or not plotted—the figure description implies R_time is tracked but not rewarded). In Stage 2 (gated time reward), R_correctness improves modestly or plateaus while R_time rises substantially. LiveCodeBench-V6 performance continues to improve throughout both stages, consistent with the model benefiting from both correctness and efficiency optimization. The paper does not report an ablation where Stage 2 is skipped (i.e., correctness-only code RL only), which would quantify the marginal contribution of the time-complexity reward to final benchmark scores. The Appendix C case studies provide qualitative validation—showing concrete algorithmic improvements from O(N log N) to O(N) or O(N² log N) to O(N log N)—but the quantitative benefit of the time reward in terms of benchmark scores is not isolated.

Code RL on-policy difficulty filtering: The paper describes filtering problems based on "k in [1, 5] successful solves out of 8" (Stage 1) and "k in [1,5] complexity-satisfying rollouts out of 8" (Stage 2). No ablation comparing filtered vs. unfiltered training is reported. The filtering mechanism's contribution to final performance is therefore unquantified—it could be essential, beneficial but modest, or even unnecessary. The paper presents it as a design choice without empirical validation of its importance.

Turn-level critic filtering for deep search trajectories: The paper describes the turn-level judgment mechanism (logical soundness, tool-call accuracy, informational gain) as a quality filter for training data. No ablation comparing training with vs. without turn-level filtering is reported. The controlled experiment in Table 3 uses data that has undergone this filtering, but there is no comparison to a version trained on unfiltered trajectories from the same synthesis pipeline. The contribution of turn-level filtering to the synthetic data's effectiveness is therefore unquantified—it could be the critical ingredient or a modest quality improvement.

Gated vs. non-gated time-complexity reward: The paper argues for gating the time reward on PassRate = 1 to prevent correctness-efficiency tradeoffs. No ablation comparing gated vs. non-gated (i.e., always-active) time reward is reported. The Appendix C case studies compare "before" (no time reward) to "after" (gated time reward), but do not compare to a hypothetical non-gated version. The gating mechanism's preventive value (does non-gated training actually cause correctness regression?) is therefore asserted but not empirically demonstrated.

Critical Assessment

Claim 1: Nanbeige4.1-3B is a unified generalist that achieves strong reasoning, code, and agentic behavior in a single 3B model.

Assessment: The experiments provide strong evidence for this claim across the evaluated benchmarks. Table 4 shows the model outperforming same-scale baselines (Qwen3-4B-2507) on all 11 general reasoning benchmarks. Table 5 shows it matching or exceeding specialized search agents on deep search tasks. Table 6 shows 85% pass rate on real LeetCode contests. The breadth of coverage—code, math, science, alignment, tool use, deep search, competitive programming—is genuinely comprehensive.

However, there are important caveats. First, all benchmarks are automated and most have objective ground truth. The paper does not evaluate on open-ended generation tasks (creative writing, long-form QA, summarization) where evaluation is subjective, nor on safety benchmarks (toxicity, bias, refusal behavior). "Generalist" here means "performs well across a specific set of reasoning and search benchmarks"—a meaningful but bounded claim. Second, the deep search evaluation is conducted within a specific tool framework (Mindflow, with Serper, Jina, E2B Sandbox). The model's search capability may be partly specific to this tool configuration; generalizability to different search APIs, different webpage extraction tools, or different sandbox environments is untested. Third, the paper evaluates only one model size (3B). There is no evidence about whether the same methodology scales similarly at other sizes (would a 1.5B version work? a 7B version?), so the claim is specific to the 3B scale rather than demonstrating a general principle about compact models.

Claim 2: The model significantly outperforms prior models of similar scale and even surpasses much larger models (e.g., Qwen3-30B-A3B).

Assessment: This claim is well-supported for the specific models and benchmarks evaluated. Table 4 shows Nanbeige4.1-3B surpassing Qwen3-30B-A3B-2507 on 9 of 11 benchmarks and Qwen3-32B on all 11. Table 5 shows it exceeding all Qwen3-series models (4B through 80B-A3B) on all deep search benchmarks by enormous margins.

However, interpretation requires caution. Qwen3-30B-A3B is a Mixture-of-Experts model with only 3B active parameters—it is roughly 10× larger in total parameters but comparable in per-token inference cost to a 3B dense model if the MoE routing is efficient. Surpassing it is still impressive but less dramatic than surpassing a 30B dense model like Qwen3-32B. The paper does surpass Qwen3-32B on all benchmarks in Table 4, which is a genuine 10× parameter efficiency gain. For deep search (Table 5), Qwen3-32B's scores are uniformly poor—GAIA 30.17, BrowseComp 3.15, xBench-DeepSearch-05 39.00—suggesting these larger models were not trained for long-horizon search, making the comparison more about training methodology than parameter efficiency per se.

The LeetCode comparison (Table 6) includes Qwen3-32B (50.0% vs. 85.0%), which is a fair test of real-world coding ability where both models face the same problems under the same submission conditions. This provides the cleanest cross-scale comparison: 3B > 32B on fresh competitive programming problems, a genuinely surprising result.

A missing comparison: the paper does not compare against the best available specialized models in each domain individually. For code, models like DeepSeek-Coder-V2 (16B) or specialized code SLMs might outperform Nanbeige4.1-3B on coding benchmarks even if they lack generalist capabilities. For math, specialized reasoning models might exceed the reported math scores. The claim "surpasses much larger models" is accurate for the generalist models compared, but does not address whether the absolute performance is close to the state-of-the-art for specialized models of any scale—which would be a stronger test of whether "broad competence" comes at a meaningful cost to depth in any individual domain.

Claim 3: The model can reliably execute up to 600 tool-call turns for complex problem-solving.

Assessment: This claim appears in the abstract but is not directly supported by the experiments presented. The deep search evaluation (Table 5, Section 3.2) reports benchmark scores but does not report the number of tool-call turns executed per problem. The abstract states the model can "reliably execute up to 600 tool-call turns," but there is no figure, table, or analysis showing turn-count distributions, success rates as a function of turn count, or any evidence that 600-turn trajectories are actually executed during evaluation. The benchmarks used (GAIA, BrowseComp, etc.) may or may not require 600-turn interactions—this is not analyzed. The claim about 600 turns is specific enough to require explicit experimental support, and the paper does not provide it.

The paper does provide indirect evidence of long-horizon capability: the dramatic improvement in deep search benchmarks (Table 5) is consistent with the model having learned to sustain coherent multi-turn interactions, and the turn-level critic filtering (Section 2.2.1) is designed to enable this. But quantitative evidence about actual turn counts—mean, median, maximum, distribution, success rate by turn count—is absent. This is a significant gap between the abstract's claim and the experimental evidence.

Claim 4: The point-wise + pair-wise RL combination produces high-quality, human-aligned responses.

Assessment: This claim is supported by the alignment benchmark results (Table 2, Table 4). Arena-Hard-V2 improves from 60.2 (SFT) to 66.6 (point-wise) to 73.8 (pair-wise), and Multi-Challenge from 44.4 to 47.7 to 55.1. The absolute scores on Arena-Hard-V2 (73.2–73.8) are competitive with much larger models (Qwen3-Next-80B-A3B: 62.3). However, alignment is inherently multidimensional—helpfulness, harmlessness, honesty, instruction-following, refusal behavior—and these two benchmarks capture only specific aspects (Arena-Hard V2 focuses on challenging prompts; Multi-Challenge on multi-turn robustness). No evaluation on standard safety benchmarks, red-teaming, or bias metrics is reported. The claim of "human-aligned" is thus supported only for the specific forms of alignment tested, not comprehensively.

Claim 5: The gated time-complexity reward optimizes both correctness and efficiency in code generation.

Assessment: The evidence is mixed. Figure 4 shows R_time rising in Stage 2 while R_correctness remains stable, which is consistent with the gated reward working as designed. Appendix C provides three qualitative examples of algorithmic efficiency improvements. However, the quantitative benefit is not isolated: there is no ablation showing LiveCodeBench scores with vs. without the time reward. The full model's LiveCodeBench-V6 score (76.9 in Table 4) could be achieved primarily through correctness improvements from the code RL stages and SFT data quality, with the time reward contributing marginally or not at all to benchmark scores. The claim that the time reward "optimizes efficiency" is supported qualitatively (appendix examples) and by training dynamics (Figure 4) but not by a quantitative ablation that isolates its contribution to downstream performance.

Claim 6: Turn-level supervision enables stable long-horizon tool interactions.

Assessment: The synthetic data ablation (Table 3) provides strong evidence that the data pipeline as a whole dramatically improves deep search performance. However, as noted above, turn-level critic filtering specifically is not ablated—there is no comparison to the same synthetic data without turn-level filtering. The causal claim that turn-level supervision specifically enables stable long-horizon interactions is therefore plausible but not directly tested. The improvement could be driven primarily by the question synthesis and trajectory generation (having any high-quality search trajectories at all, regardless of per-turn filtering) rather than the turn-level filtering mechanism.

Genuine Weaknesses

No statistical rigor. All results are point estimates without confidence intervals, error bars, or significance tests. With 500-question test sets (MATH in the reference example) or smaller, performance differences of a few points could be within sampling variance. The paper does not address this.
No generation budget analysis. Unlike the reference example paper, which systematically sweeps generation budgets and shows performance scaling curves, this paper reports single-point numbers. There is no analysis of how performance changes with more samples, more tool calls, or more inference compute. This makes it impossible to assess whether the reported numbers represent peak performance or could be achieved more cheaply.
Single evaluation per benchmark. Results are reported for one evaluation run per benchmark with unspecified decoding parameters (temperature, top-p, sampling vs. greedy). Variance from decoding stochasticity is unaccounted for.
Tool framework specificity for deep search. The deep search evaluation uses a specific tool configuration (Mindflow, Serper, Jina, E2B Sandbox). Different tool APIs could produce different results. The paper does not test robustness to tool changes.
Missing ablations for key design claims. The contributions of on-policy difficulty filtering (code RL), turn-level critic filtering (search), and the gating mechanism (time reward) are asserted but not empirically isolated. These missing ablations weaken the paper's causal claims about which specific design choices matter.
The 600-turn claim is unsubstantiated. This is the most specific quantitative capability claim in the abstract, and there is zero experimental evidence for it in the paper body. If the model can execute 600-turn trajectories, the paper should show that it does—distribution of turn counts, success rates at different turn depths, examples of long trajectories.
No contamination analysis. With benchmarks increasingly integrated into training data (intentionally or through web scraping), the paper does not discuss data contamination, deduplication against benchmarks, or any analysis of whether benchmark performance reflects genuine capability vs. memorization. The LeetCode evaluation partially addresses this (fresh contest problems), but for standard benchmarks like GPQA, HLE, and LiveCodeBench, contamination is a concern.
Alignment evaluation is narrow. Only Arena-Hard V2 and Multi-Challenge are reported. No evaluation on harmlessness, safety refusals, bias, or instruction-following diversity. A model that scores 73.2 on Arena-Hard could still produce toxic content or fail to refuse harmful requests.
No latency or efficiency analysis. For real deployment, inference speed and memory usage matter at least as much as accuracy. The paper reports no throughput, latency, or memory consumption numbers. The 256K context length training and deep search with hundreds of tool calls suggest potentially high latency per query, but this is not quantified.

Missing Experiments That Would Strengthen the Paper

Generation budget sweeps for code and math benchmarks, showing how performance scales with number of samples (pass@k curves) and comparing to baselines at matched budgets.
Turn-count distribution analysis for deep search evaluation, showing the actual number of tool calls per problem, success rate as a function of turn count, and maximum sustained interaction length.
Ablation of gated vs. non-gated time reward, quantifying correctness regression (or lack thereof) when the time reward is always active.
Ablation of turn-level critic filtering, comparing deep search performance with vs. without per-step quality filtering in the synthetic training data.
Ablation of on-policy difficulty filtering, comparing code RL performance with uniform problem sampling vs. filtered sampling based on solve count.
Safety and harmlessness evaluation on standard benchmarks to complement the alignment results.
Cross-validation or multiple evaluation runs with reported variance, especially for smaller benchmarks where point estimates may be unreliable.
Contamination analysis for all standard benchmarks, or reliance on contamination-free benchmarks (like the fresh LeetCode contests) for primary claims.
Inference efficiency metrics (tokens per second, memory usage, latency) for practical deployment context.
Comparison to best-in-class specialized models in each domain to establish whether the generalist model sacrifices depth for breadth.
Robustness to tool configuration changes in deep search evaluation, testing with different search APIs or sandbox environments.

6. Limitations and Trade-offs

The 600-Turn Claim Is Unsubstantiated, Yet It Is the Most Specific Capability Guarantee Offered

The assumption or constraint. The abstract states that Nanbeige4.1-3B can "reliably execute up to 600 tool-call turns for complex problem-solving." This is a precise quantitative claim—not "hundreds of turns" or "long-horizon interactions," but 600. A practitioner evaluating whether to deploy this model for a multi-step research task would reasonably interpret this as a guaranteed capability: the model will remain coherent, avoid getting stuck in loops, and produce useful outputs even when a task requires hundreds of sequential tool interactions.

The consequence. The paper provides zero experimental evidence that 600-turn interactions actually occur, succeed, or are "reliable" in any measurable sense. The deep search evaluation (Section 3.2, Table 5) reports aggregate benchmark scores—GAIA, BrowseComp, xBench-DeepSearch—but never reports the distribution of turn counts per problem, the success rate as a function of trajectory length, the maximum sustained interaction length observed, or even the mean number of turns. These benchmarks may or may not require 600-turn trajectories to solve; many GAIA questions can be answered in a few dozen turns. A practitioner who deploys this model expecting it to handle 600-turn tasks cannot assess failure probability beyond turn 100 or turn 300 from the presented data. If the model's coherence degrades sharply after, say, 150 turns—which would be consistent with the training data's maximum context length of 256K tokens reaching saturation with retrieval content—the "up to 600" claim is misleading. The consequence is a deployment decision based on an untested performance boundary.

What evidence exists in the paper. No figure, table, or analysis in the paper quantifies turn counts. The deep search pipeline (Section 2.2.1) trains on trajectories that presumably contain many turns, but what "many" means in practice is unspecified—there is no histogram, no summary statistic, no upper bound on training trajectory length. The paper does describe long-context SFT training up to 256K tokens (Section 2.1.1), which establishes that the model can process long sequences, but 256K tokens of context is not a proxy for 600 coherent tool-calling turns, each of which involves generating a tool call, receiving tool output, and integrating that output into ongoing reasoning. Token count and turn count are different resources, and the paper conflates them without analysis.

Mitigation status. Not addressed. The paper does not acknowledge this as a gap, does not suggest that the 600-turn figure is aspirational rather than validated, and does not propose future work to characterize turn-length scaling behavior. This is the most consequential unsubstantiated claim in the paper because it is the headline capability guarantee for the model's primary distinguishing feature (long-horizon agency in a 3B model), and a practitioner has no way to determine whether it holds for their specific workload.

Single-Point Evaluations Without Generation Budget Sweeps Conceal the True Cost-Performance Tradeoff

The assumption or constraint. The paper reports all results as single-point accuracy numbers—one score per benchmark per model—without any analysis of how performance varies with inference compute. The implied assumption is that the reported number represents the model's stable performance level, sufficient for a practitioner to decide whether to use the model.

The consequence. In practice, test-time compute is a critical deployment axis. A model that achieves 76.9 on LiveCodeBench-V6 with 1 sample (greedy) might achieve 82.0 with 64 samples (best-of-64 with majority voting). Conversely, a model that requires 64 samples to achieve 76.9 is much more expensive and higher-latency than one that achieves it with 1 sample. The paper reports no pass@k curves, no best-of-N scaling, no tradeoff between compute budget and accuracy. A practitioner cannot answer: how much inference compute do I need to achieve a target accuracy? How does that compare to the baseline models at matched compute? This is especially critical for the deep search tasks, where each tool-calling turn adds sequential latency and API costs (search queries, web page fetches). If achieving the reported GAIA 69.90 requires an average of 200 tool calls per problem—which would be reasonable for complex multi-hop search—the total cost per query could be orders of magnitude higher than a model that solves the same problems in 20 turns. The paper's scores are not comparable to baselines unless we know both models' compute expenditure, and the paper provides no way to assess this.

In the context of the reference example paper—which made generation budget sweeps a core part of its methodology, showing how performance scales with N and where it plateaus—this absence is notable. That paper demonstrated that the relationship between compute and accuracy is often non-monotonic and difficulty-dependent. This paper's single-point reporting style makes it impossible to determine whether the reported results represent peak achievable performance or could be substantially improved with more compute, and whether the claimed advantages over baselines hold at matched compute budgets.

What evidence exists in the paper. The paper specifies that point-wise RL uses "8 rollouts per prompt" (Section 2.1.2) during training, and that code RL on-policy filtering uses "n = 8" rollouts per problem (Section 2.3.2). This establishes that the model's RL training uses multiple samples, but no evaluation result is reported with a generation budget > 1. The paper does not state whether evaluation uses greedy decoding, temperature sampling, or any generation budget parameter. There is no figure analogous to Figure 3 in the reference example paper showing accuracy vs. generation count.

Mitigation status. Not addressed. The paper does not acknowledge the absence of generation budget analysis as a limitation, does not report decoding parameters for evaluations, and does not propose future work on inference-time scaling. A practitioner who needs to budget inference compute has no way to use the reported numbers for cost estimation or to compare models on a cost-adjusted basis.

The Deep Search Capabilities Are Tightly Coupled to a Specific Tool Ecosystem and Evaluation Framework

The assumption or constraint. All deep search evaluations (Section 3.2) are conducted within the Mindflow framework using Serper for web search, Jina for webpage content extraction, and E2B Sandbox for code execution, with HuggingFace explicitly disabled. The implied assumption is that the model has learned generalizable search and tool-use skills that transfer across tool configurations, search APIs, and execution environments.

The consequence. The model's search behavior was trained on synthetic trajectories generated by "multiple agent frameworks" (Section 2.2.1) and mapped into tool-invocation sequences "from a unified agent perspective." Both the training trajectories and the evaluation framework use a specific action space—specific tool names, specific API formats, specific expected output structures. A practitioner deploying this model with a different search API (Google Search instead of Serper), a different content extraction tool (Trafilatura instead of Jina), a different sandbox (Docker instead of E2B), or any custom tool would need to either retrain or rely on the model's zero-shot generalization to new tool formats. The paper provides no evidence that the model generalizes to unseen tools or API changes.

This coupling matters for practical deployment because the exact tool configuration used in the paper may not be available to all practitioners. Serper is a commercial API with usage costs and rate limits. Jina's content extraction API may have different coverage or quality characteristics than alternatives. E2B Sandbox requires specific infrastructure. A model that succeeds with these tools but fails with, say, a locally hosted SearXNG instance and BeautifulSoup-based extraction would have limited deployability outside the specific infrastructure the authors used.

The coupling also limits the scientific generality of the finding. If the model's deep search capability is partly attributable to the specific prompt format, tool descriptions, and expected output structures used during training and evaluation, then the claim "a 3B model can do deep search" is more accurately "a 3B model can do deep search when using Serper + Jina + E2B in the Mindflow framework, with tool descriptions matching what it saw during training." This is still an achievement, but it is a weaker claim than the paper's implication of generalizable search agency.

What evidence exists in the paper. Sections 2.2.1 and 3.2.1 name the specific tools and framework. No experiment varies the tool configuration—there is no evaluation with an alternative search API, no test with different extraction tools, no robustness check against tool API changes. The controlled experiment in Table 3 uses the same tool configuration as the full model evaluation in Table 5, so there is no evidence that the synthetic data's benefit transfers to different tool setups.

Mitigation status. Not addressed. The paper does not discuss tool dependency as a limitation, does not claim the model is generalizable to arbitrary tool configurations, and does not propose experiments to test tool robustness. The open-sourcing of the training dataset on HuggingFace (Section 2.2.1) partially mitigates this by allowing practitioners to inspect and potentially adapt the tool format for their own configurations, but this requires additional training, not zero-shot transfer.

The Time-Complexity Reward's Quantitative Contribution Is Not Isolated, Leaving the Core Code Efficiency Claim Unverified

The assumption or constraint. The paper presents the gated time-complexity reward (Section 2.3.3, Figure 3) as a key methodological contribution for optimizing code efficiency. The training dynamics (Figure 4) show R_time rising in Stage 2, and Appendix C provides three before/after case studies demonstrating algorithmic improvements. The implied claim is that the time reward meaningfully contributes to the model's final code generation performance, producing solutions that are not just correct but efficient.

The consequence. The paper never reports the quantitative impact of the time reward on benchmark scores. Table 2 shows LiveCodeBench-V6 improving from 62.0 (SFT) to 66.0 (point-wise RL) to 65.6 (pair-wise RL)—these are general RL stages, not code RL. The full model's LiveCodeBench-V6 score (76.9 in Table 4) is reported after all training stages including code RL, but there is no intermediate score reported after Stage 1 code RL (correctness-only) versus after Stage 2 (with time reward). A practitioner cannot determine whether the time reward accounts for 0.5 points, 5 points, or 15 points of the 76.9 total. The Appendix C case studies show that the time reward changes algorithmic complexity on specific problems, but they do not establish that this translates to benchmark-level improvements—it is possible that the time reward produces elegant O(N) solutions for problems where O(N log N) solutions would also pass all test cases within time limits, making the efficiency gain invisible to benchmark scoring.

This matters because the two-stage code RL pipeline (SFT → Stage 1 → Stage 2) adds complexity: it requires a trained instruct judge model for complexity assessment, on-policy difficulty filtering with complexity-based criteria, and a gated reward function with conditional activation. If this complexity yields negligible benchmark improvement—or if the improvement comes primarily from Stage 1 correctness training, with the time reward contributing only cosmetic algorithmic improvements that don't affect pass rates—then the cost of building and maintaining the judge system is misallocated.

The broader scientific claim—that gated time-complexity rewards are a valuable design pattern for code RL—also depends on demonstrating that the time reward specifically drives benchmark-level gains. Without this evidence, the claim reduces to "the training reward goes up," which is a necessary but insufficient condition for claiming the reward structure is beneficial.

What evidence exists in the paper. Figure 4 shows R_time increasing during Stage 2, which confirms the reward mechanism is functioning—the policy is learning to produce solutions that the judge model rates as efficient. Appendix C shows three concrete algorithmic complexity improvements: O(N log N) → O(N), O(N + M log N) → O(N), O(N² log N) → O(N log N). These demonstrate that the model can learn efficiency, but they are selected examples, not a systematic analysis of efficiency improvements across the test set. No benchmark-level ablation (Table 4 with vs. without time reward) is reported.

Mitigation status. Partially addressed through qualitative evidence but not quantitatively resolved. The paper does not acknowledge the absence of a quantitative ablation as a limitation. A practitioner can inspect Appendix C to see that the time reward produces real algorithmic changes, but cannot assess whether these changes matter for benchmark performance, whether they generalize beyond the three shown examples, or whether the infrastructure cost of the judge system is justified relative to simpler approaches (e.g., incorporating efficiency examples in SFT data only).

The Model Evaluates Well on Automated Benchmarks but Alignment Testing Is Incomplete for Production Deployment

The assumption or constraint. The alignment evaluation (Section 3.1, Table 4) uses only two benchmarks: Arena-Hard-V2 (which tests response quality under challenging prompts) and Multi-Challenge (which tests multi-turn conversational robustness). The paper reports substantial gains on these benchmarks—from 60.2 (SFT) to 73.8 (pair-wise RL) on Arena-Hard-V2, and 44.4 to 55.1 on Multi-Challenge (Table 2)—and uses them as evidence that the point-wise + pair-wise RL combination produces "high-quality, human-aligned responses" (abstract). The implied assumption is that these two benchmarks adequately cover the alignment dimensions relevant to deployment.

The consequence. Alignment in production LLMs encompasses multiple dimensions beyond those tested by Arena-Hard and Multi-Challenge: harmlessness (does the model refuse harmful requests?), safety (does it avoid generating toxic, biased, or dangerous content?), instruction-following diversity (does it correctly handle unusual or adversarial formatting requests?), and calibration (do its confidence estimates match its accuracy?). A model that scores 73.8 on Arena-Hard-V2 could still generate toxic content when prompted adversarially, fail to refuse instructions for producing harmful code, or exhibit systematic biases in its responses. The paper's RL training—particularly the point-wise stage using a general reward model trained on "curated large-scale human preference data" (Section 2.1.2)—presumably includes safety and harmlessness signals in the preference data, but the paper provides no evidence that these signals were effective.

This gap is especially concerning for a model that is open-sourced and designed for broad deployment. A practitioner integrating this model into a user-facing application needs to know: what is the model's refusal rate on harmful prompts? Does it exhibit gender, racial, or other biases in its outputs? Does its code generation ever produce insecure code (SQL injection vulnerabilities, unsafe deserialization, hardcoded credentials)? The alignment benchmarks reported say nothing about these dimensions.

The pair-wise RL stage, which uses "strong–weak model comparisons" (Section 2.1.3), could in principle improve safety if the strong model's preferences include safety judgments. But without explicit safety evaluation, a practitioner cannot determine whether the strong model's preferences penalize unsafe outputs or primarily reflect stylistic/conversational quality, and whether the pair-wise training successfully transfers those preferences to the 3B model.

What evidence exists in the paper. Table 2 and Table 4 report Arena-Hard-V2 and Multi-Challenge. No safety benchmark (e.g., TruthfulQA, Toxigen, RealToxicityPrompts, HarmBench), no bias evaluation (e.g., BBQ, WinoBias), no refusal evaluation, and no red-teaming results are reported. The paper does not discuss the safety properties of the reward models or the preference data.

Mitigation status. Not addressed. The paper does not acknowledge the narrow scope of its alignment evaluation as a limitation, does not discuss safety considerations, and does not propose future work on comprehensive alignment testing. For an open-source model intended for community use, this is a significant deployment risk that the paper leaves entirely to downstream practitioners to evaluate.

The Difficulty Estimation Overhead That Enables Adaptive Allocation Is Not Accounted For, but Here the Overhead Is a Different Problem: the Training Infrastructure Cost Is Unreported

Note on scope: This limitation addresses a different axis than the reference paper's difficulty estimation concern. Rather than test-time overhead, the issue here is the training infrastructure complexity and computational cost required to reproduce the model, which the paper does not quantify.

The assumption or constraint. The paper describes a complex multi-stage training pipeline: extended SFT with three-stage context length curriculum and upgraded solution refinement (Section 2.1.1), point-wise RL requiring training a general reward model on "curated large-scale human preference data" (Section 2.1.2), pair-wise RL requiring training a pair-wise reward model with swap-consistency regularization on strong-vs-weak model comparisons (Section 2.1.3), two-stage code RL requiring a unified judge system with a multi-language sandbox and a dedicated instruct judge model trained for complexity assessment (Section 2.3.1), and deep search training requiring synthesis of multi-hop QA data via entity-relation random walks, multi-agent trajectory generation, and a turn-level critic model (Section 2.2.1). The paper presents these as a recipe for achieving the reported results, but provides no quantification of the computational resources, training time, dataset sizes, or infrastructure complexity required to execute this recipe.

The consequence. A practitioner or research group attempting to reproduce Nanbeige4.1-3B cannot estimate the cost of doing so. The SFT stage requires training data at three context lengths (up to 256K) with a specialized data mixture; the reward model training requires "large-scale human preference data" of unspecified size and provenance; the pair-wise reward model requires generating comparison data from strong and weak models, then training with swap-consistency regularization; the code judge system requires training a dedicated instruct model for complexity assessment; the deep search data pipeline requires Wikipedia-scale entity-relation graph construction, random walking, question synthesis with a "robust LLM," multi-agent trajectory generation, and a critic model for turn-level filtering. Each of these components requires substantial compute and engineering effort, but the paper provides no FLOP counts, GPU-hours, dataset sizes, or training durations for any stage. This makes the paper more of an existence proof ("a 3B model with these capabilities can be built") than a reproducible recipe.

This matters because the training cost determines who can build on this work. If the full pipeline requires thousands of GPU-days spread across multiple model training runs (base SFT, reward models, critic models, judge models), only well-resourced labs can reproduce or extend it. If the cost is more modest, the work is more democratizing. Without cost reporting, the paper's contribution is difficult to position on the accessibility spectrum.

What evidence exists in the paper. The paper specifies some hyperparameters: 8 rollouts per prompt for GRPO (Section 2.1.2), 8 rollouts for on-policy difficulty filtering in code RL (Section 2.3.2), 256K maximum context length for the final SFT stage (Section 2.1.1). These are operational details but not cost estimates. No GPU-hours, no dataset sizes (in tokens or examples), no training wall-clock time, and no total FLOP counts are reported. The base model (Nanbeige4-3B-Base) was pretrained by the authors in prior work [21], so even the pretraining cost—which is typically the dominant term—is not summarized in this paper. The paper's focus is on post-training, but the post-training cost is equally opaque.

Mitigation status. Partially addressed through open-sourcing. The paper releases the final model checkpoint on HuggingFace, which allows practitioners to use the model without reproducing the training pipeline. For use cases that only require inference with the released checkpoint, the training cost is irrelevant. The deep search training dataset is also open-sourced (Section 2.2.1), reducing the data construction burden for researchers extending the search capabilities. However, for practitioners who need to adapt the methodology to different base models, different domains, or different tool configurations—or who want to understand the cost-performance tradeoff of the approach relative to alternatives—the absence of training cost reporting remains a significant gap. The paper does not acknowledge this as a limitation or propose future work on cost characterization.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper changes the conversation around small language models from capability specialization as an accepted constraint to capability breadth as an engineering target. Prior to Nanbeige4.1-3B, the working assumption in the SLM community was that a 3B-parameter budget forces a choice: build a strong reasoning model that can't do long-horizon search, or a search agent that lacks general reasoning, or a code model that isn't aligned with human preferences. The paper refutes this assumption not through theoretical argument but through an existence proof—a single 3B checkpoint that scores 76.9 on LiveCodeBench-V6, 69.90 on GAIA, and 73.2 on Arena-Hard-V2 simultaneously. The magnitude of this reframing is substantial because it shifts the research question from "what capability can we maximize at this scale?" to "what set of training signals enables broad competence at this scale?"

This is not a paradigm shift in the sense of introducing new algorithms—GRPO, pair-wise preference optimization, and staged RL all existed before. Rather, it is a methodological reframing: the key variable is not parameter count but the composition, sequencing, and reward structure of post-training objectives. The paper makes this reframing credible by isolating each component's contribution. Table 1 shows SFT data mixture alone produces a +16.5 point improvement on LiveCodeBench—meaning data composition, not model architecture, drives a large fraction of the gain. Table 3 shows synthetic search data alone triples GAIA performance—meaning the data pipeline, not a novel search algorithm, drives deep search capability. Table 2 shows that point-wise before pair-wise RL produces alignment gains without catastrophic code regression—meaning stage sequencing, not just reward type, matters. Collectively, these ablations demonstrate that the model's broad competence emerges from design choices about training signals, and the paper makes those choices explicit and individually validated.

The work also implicitly resolves a latent tension in the SLM literature: the apparent contradiction between reasoning-specialized models (which achieve strong math/logic performance but can't sustain multi-turn interactions) and agent-specialized models (which can execute tool calls but can't do competition math). Prior work treated these as separate research tracks with different training recipes. This paper demonstrates they are compatible in a single checkpoint, and more importantly, identifies why they previously appeared incompatible: reasoning models lacked turn-level supervision for credit assignment over long horizons, and agent models lacked the general RL and preference alignment that stabilizes output quality. The fragmentation was a consequence of training design, not of scale.

The research directions that become more attractive after this work:

Multi-signal training for compact models. The paper's success with sequenced, heterogeneous reward structures (point-wise absolute quality, pair-wise relative preference, gated time-complexity, turn-level critic scores) suggests that adding more training signals—not reducing to a single unified objective—is the path to breadth for small models. Research on reward signal composition becomes more attractive than research on reward signal simplification.
Synthetic trajectory construction with per-step filtering. Table 3's massive gains from synthetic search data validate the approach of using strong models to generate trajectories and critic models to filter them for quality. This pattern—generate diverse behaviors, filter for quality at the step level, train the small model to imitate only the high-quality steps—generalizes well beyond search to any long-horizon task.
Dependency-aware stage sequencing. The paper's implicit claim that the order of RL stages matters (general before specialized, correctness before efficiency) opens a research direction on optimal stage sequencing for multi-capability training, which is currently underexplored.

Research directions that become less attractive:

Architecture innovation as the primary lever for SLM capability. If a 3B dense model with standard architecture can achieve this breadth through post-training design, the pressure to develop fundamentally new architectures for small models decreases. The bottleneck appears to be training signal design, not model structure.
Scaling parameters as the only path to generalist behavior. The paper's cross-scale comparisons (3B surpassing 30B-80B models on multiple benchmarks) demonstrate that parameter count is not the binding constraint for generalist performance. Research premised on "we need larger models for broad competence" loses some of its strongest motivating evidence.

Follow-Up Research This Work Enables

Systematic study of RL stage ordering and interference. The paper sequences stages as SFT → point-wise RL → pair-wise RL → code RL → agentic RL, arguing implicitly (Section 2.4) that the order prevents catastrophic interference. A direct empirical test: train multiple versions of the model with the same stages in different orders (e.g., code RL before pair-wise RL, agentic RL before code RL) and measure whether performance on each domain degrades relative to the reported ordering. The paper's Table 2 provides a partial within-domain ordering test (point-wise before pair-wise), but no cross-domain ordering test exists. A strong negative result—where reversed ordering causes substantial regression in earlier-trained capabilities—would validate the dependency-aware sequencing as a critical design choice rather than an incidental one. A null result—where ordering doesn't matter as long as all stages are applied—would simplify the recipe for future work. The experiment requires training ~5-10 model variants (one per permutation of interest) and evaluating on the full benchmark suite from Tables 4 and 5.

Quantifying the time-complexity reward's contribution to benchmark performance. The paper demonstrates qualitative algorithmic improvements from the gated time reward (Appendix C) and shows R_time rising in training dynamics (Figure 4), but never isolates the reward's quantitative contribution to LiveCodeBench scores. A direct ablation: compare three checkpoints—SFT only, after Stage 1 code RL (correctness-only), and after Stage 2 code RL (with gated time reward)—on LiveCodeBench-V6 and LiveCodeBench-Pro at matched generation budgets. The delta between Stage 1 and Stage 2 scores is the time reward's contribution. If the delta is large (several points), the gated reward is validated as a performance-critical design choice. If the delta is near zero, the time reward produces algorithmic elegance invisible to benchmark scoring, and the complexity of the judge system is questionable for practitioners who only care about pass rates. The experiment also enables testing whether the overhead of training a dedicated instruct judge model (Section 2.3.1) can be reduced—could a cheaper complexity heuristic (e.g., runtime on fixed-size inputs, cyclomatic complexity, or LLM-as-judge with a general-purpose model) achieve similar training dynamics at lower infrastructure cost?

Turn-level critic filtering ablation for deep search. Table 3 shows the full synthetic data pipeline dramatically improves deep search performance. But the pipeline has three stages: question synthesis via entity-relation walks, multi-agent trajectory generation, and turn-level critic filtering. Which stage matters most? A component ablation: train three variants of Nanbeige4-3B-2511 on (1) synthetic QA pairs only (no trajectories—just question-answer training), (2) QA pairs + unfiltered trajectories (skip the critic, use all generated trajectories), and (3) QA pairs + critic-filtered trajectories (the full pipeline). Evaluate on GAIA, BrowseComp, and xBench-DeepSearch-05. If variant (2) performs nearly as well as variant (3), the critic filtering is unnecessary—high-quality trajectory generation alone suffices, and the critic model can be eliminated. If variant (2) performs substantially worse, turn-level supervision is confirmed as the critical ingredient enabling long-horizon behavior at small scale, and subsequent work should focus on improving critic model quality (higher accuracy, coverage of more failure modes) rather than trajectory generation diversity.

Characterizing failure modes at long horizons. The paper claims 600-turn reliability (abstract) but provides no empirical characterization of when or how the model fails as turn count increases. A measurement study: run Nanbeige4.1-3B on a set of search tasks with known optimal solution depth (10, 50, 100, 200, 400, 600 turns) and measure success rate as a function of horizon length. For failures, categorize the failure mode: does the model get stuck in repetitive search loops? Does it lose coherence, generating nonsensical tool calls? Does it "give up" and produce an answer based on insufficient information? Does it exceed context length and lose access to early retrieved information? This characterization would bound the practical deployability of the model—if success rate drops below, say, 50% at 150 turns, then the "up to 600" claim is misleading and practitioners should budget for tasks requiring 100-150 turns maximum. The study would also guide future work: if loop-stuck is the dominant failure mode, add anti-loop training examples; if context-length overflow dominates, explore retrieval compression or sliding window approaches.

Stress-testing cross-tool generalization of deep search. All deep search evaluations (Tables 3 and 5) use the exact tool configuration from training: Serper for search, Jina for extraction, E2B for execution. Does the model generalize? Replace Serper with Google Search API (different response format, different ranking), Jina with Trafilatura (different extraction quality, different error modes), and E2B with a Docker-based sandbox (different file system, different tool interface). Evaluate on the same benchmarks. If performance degrades substantially, the model has learned tool-specific heuristics rather than general search competence, and future work must either train on diverse tool configurations or develop abstract tool descriptions that transfer. If performance is stable, the synthetic trajectory mapping to a "unified agent perspective" (Section 2.2.1) successfully taught tool-agnostic search skills, which is a stronger and more generalizable finding. This experiment also tests the practical deployability claim: most practitioners cannot replicate the exact Serper+Jina+E2B stack, and they need to know whether their alternative tooling will work.

Safety and comprehensive alignment characterization. The paper's alignment evaluation uses only Arena-Hard-V2 and Multi-Challenge, which test conversational quality and multi-turn robustness but not safety. A necessary follow-up: evaluate Nanbeige4.1-3B on standard safety benchmarks (TruthfulQA, Toxigen, BBQ for bias, HarmBench for harmful request refusal, a set of adversarial jailbreak prompts) and compare to baselines (Qwen3-4B-2507, Qwen3-8B, the previous Nanbeige4-3B-2511). The goal is to determine whether the point-wise and pair-wise RL stages—which use "curated large-scale human preference data" and "strong–weak model comparisons"—implicitly teach safety properties, or whether safety requires explicit training that this pipeline lacks. If the model performs well on safety benchmarks despite no explicit safety training, the preference data and reward models provided implicit safety signals, which is a positive finding for the generalist approach. If it performs poorly, the model is not deployment-ready without additional safety RL, and the paper's "human-aligned" claim (abstract) is overstated. This evaluation is especially important given the model is open-sourced—downstream users need to know what safety properties they inherit.

Practical Applications and Downstream Use Cases

Cost-efficient research assistance with open-source tooling. An organization that needs an internal research assistant for complex multi-hop questions—"Find all clinical trials published in the last year that tested Drug X for Condition Y, and extract the reported effect sizes"—can deploy Nanbeige4.1-3B with the open-source Mindflow framework and commercial search/execution APIs. The model's GAIA score of 69.90 (Table 5) means it solves roughly 70% of GAIA-level research questions correctly, which is competitive with 30B+ specialized research agents (Tongyi-DeepResearch-30B: 70.90) and within ~6 points of 230B+ large models (Minimax-M2-230B: 75.70). At 3B parameters, the model runs on a single consumer GPU, keeping inference costs low for batch research queries. The specific benefit: research assistant capability previously requiring datacenter-class hardware (for 100B+ models) or paid API access (for proprietary agents) is now achievable with open-source, locally-hostable infrastructure. The caveat: the tool dependency (Serper, Jina, E2B) means some API costs remain, and the unverified turn-count reliability means very deep research tasks (>150 turns) may exceed the model's tested performance envelope.

On-device or low-latency code generation with competitive programming capability. A developer tooling company building an IDE plugin for real-time competitive programming assistance—where latency requirements preclude cloud round-trips—can embed Nanbeige4.1-3B directly in the plugin. The model's 85.0% pass rate on LeetCode Weekly Contests 484–488 (Table 6), including 1st place in Contest 487 and 3rd place in Contest 488, demonstrates that it solves fresh competitive programming problems at a level exceeding 32B models (Qwen3-32B: 50.0%). The on-device deployment scenario benefits directly from the 3B parameter count: the model fits in ~6GB of VRAM (assuming FP16), runs at low latency on a modern laptop GPU, and doesn't require network connectivity for inference. The specific benefit: competitive programmers get real-time hints or solution generation during timed contests without the latency and privacy concerns of cloud-based models. The caveats: the model's coding capability is validated on LeetCode's contest format (algorithm problems with test cases); its performance on other code generation tasks (debugging, refactoring, code explanation, library-specific API usage) is measured by LiveCodeBench but not tested in real-world software engineering contexts.

Single-model deployment for applications requiring both conversational alignment and code generation. A customer support platform that needs to handle both natural-language interactions (answering user questions, escalating issues, maintaining conversation history) and code-level tasks (generating configuration scripts, debugging user-provided code snippets, querying internal APIs) faces a common dilemma: deploy two specialized models (increasing system complexity) or use a large model (increasing cost and latency). Nanbeige4.1-3B offers a third option: a single 3B model that scores 73.2 on Arena-Hard-V2 (conversational quality) and 76.9 on LiveCodeBench-V6 (code generation). These scores surpass Qwen3-32B on both axes (56.0 and 55.7 respectively)—a 10× parameter reduction for superior performance on both dimensions simultaneously. The specific benefit: simplified deployment architecture (one model server, one set of prompts, one monitoring pipeline) with per-query costs that are roughly 10× lower in GPU memory and FLOPs compared to using a 32B model. The caveats: the alignment evaluation (Section 5 weakness) is narrow—the model's safety properties, bias characteristics, and refusal behavior are unmeasured, which matters for customer-facing deployment where harmful or biased outputs carry business and legal risk. Additional safety testing would be required before production deployment.