ArXiv: 2603.25040

🎯 Pitch

Scaling to one trillion parameters, Intern-S1-Pro overturns the belief that specialized models dominate niche tasks: this single generalist model beats domain-specific systems on over 100 scientific benchmarks, even when using identical training data. It also crushes proprietary models like Gemini-3-Pro in scientific reasoning, scoring 55.5 versus 14.7 on SciReasoner.

1. Executive Summary

This paper introduces Intern-S1-Pro, the first one-trillion-parameter scientific multimodal foundation model, scaling from its predecessor Intern-S1 through a novel expert expansion strategy with Grouped Routing (enforcing per-group Top-1 selection across eight expert groups to achieve absolute load balancing under 8-way expert parallelism) and a Straight-Through Estimator for sparse expert routing (decoupling forward sparse selection from backward dense gradient flow through the softmax distribution to accelerate router embedding updates). Intern-S1-Pro delivers comprehensive enhancements across both general and scientific domains, achieving state-of-the-art performance on specialized benchmarks—including 55.5 on SciReasoner versus 14.7 for Gemini-3-Pro, 74.8 on SmolInstruct, and 52.5 on Biology-Instruction—while maintaining competitive general capabilities such as 93.1 on AIME-2025, establishing that a sufficiently large generalist model trained jointly on general and specialized data can outperform domain-specific models even when using identical training data, as demonstrated by the 52.45 versus 39.24 average score advantage over the specialized Biology-Instruction model across 21 biological sequence and structure tasks.

2. Context and Motivation

The Core Problem: Building Foundation Models for Science Requires Unprecedented Scale

The fundamental challenge this paper addresses is deceptively simple: how do you build a single foundation model that is genuinely competent across both general tasks and the vast diversity of scientific domains? This matters because science isn't just one domain—it's chemistry, biology, materials science, earth science, physics, and numerous sub-specialties within each, each with its own "language" of notations, knowledge structures, and reasoning patterns. The paper argues that existing models fail at this because they don't have enough capacity to encode all of this simultaneously while retaining general capabilities.

The analogy the paper draws in Section 1 is instructive. Prior work in multilingual machine translation (NLLB Team et al., 2022) found that a single model translating hundreds of language pairs required roughly 90× more parameters than a bilingual model—not because any individual language pair is harder, but because the model's capacity must accommodate the combinatorial diversity of all language pairs. The paper argues that science presents an analogous, if not more extreme, scaling challenge: scientific disciplines are not just different "languages" in the surface form sense, but involve fundamentally different reasoning patterns, data modalities (molecular graphs, protein sequences, time series signals, high-resolution microscopy images, electron density maps), and specialized knowledge hierarchies. A model that can reason about chemical synthesis pathways must encode very different inductive biases than one reasoning about protein folding, and both differ from one interpreting remote sensing imagery.

The implication is that model size is not optional—it is a genuine requirement for any model attempting to span the diversity of scientific inquiry. This directly motivates the push to the trillion-parameter scale.

Why Generalist Models for Science Matter (Beyond Just Being Big)

The paper's framing goes beyond "bigger is better." There is a concrete practical vision: science is increasingly interdisciplinary, and the most important discoveries often happen at the boundaries between fields—understanding a disease mechanism might require connecting molecular biology data, clinical imaging, and epidemiological time series. A specialized model trained only on protein sequences cannot make these connections. A generalist model, if it genuinely masters multiple scientific domains, can serve as a unified reasoning interface across these boundaries.

This vision is articulated in Section 1 through the SAGE framework (Synergistic Architecture for Generalizable Experts, Figure 1). The three-layer design—Foundation (the base multimodal architecture at trillion scale), Fusion (joint training on scientific and general data), and Evolution (reinforcement learning with agentic capabilities)—embodies the insight that scientific intelligence is not a separate module bolted onto a general model, but rather an emergent property of training general and specialized capabilities together. The paper emphasizes this as more than a technical convenience: they explicitly call the model a "Specializable Generalist," meaning it should be able to deeply specialize to any scientific sub-domain while retaining a broad general intelligence base. This is a fundamentally different design philosophy from, say, training separate models for each scientific task (which loses cross-domain synergies) or simply fine-tuning a general model on scientific data as an afterthought (which risks catastrophic forgetting of general capabilities).

The Gap in Prior Work: Why Existing Models Fall Short

The paper identifies several distinct shortcomings in the existing landscape of scientific AI models:

1. Scale limitations in open-source scientific models. The predecessor, Intern-S1, was already a large scientific model, but the paper frames it as insufficient for the full diversity of scientific tasks. The motivation for Intern-S1-Pro is not just incremental improvement—it's about crossing a threshold where the model's capacity genuinely exceeds what specialized smaller models can achieve in scientific domains. The paper makes the striking claim in Section 5.5 that once you cross this threshold, a generalist model with sufficient capacity can outperform specialists on their own tasks even when trained on the same data. This is not obvious: the conventional wisdom (which the paper explicitly challenges) is that specialized models, being free from the "distraction" of irrelevant data, should dominate on niche tasks. The paper's counter-hypothesis is that general reasoning capabilities—acquired through scale and diverse training—transfer beneficially even to highly specialized scientific problems. For instance, understanding a protein sequence prediction task may benefit from the model's ability to recognize patterns learned from entirely different domains, a capability that a specialized model sacrifices.

2. Training instability in ultra-large Mixture-of-Experts (MoE) models. The paper identifies a specific, non-trivial technical barrier: when scaling to trillions of parameters with MoE architectures, expert load imbalance becomes a critical failure mode. The paper describes this in Section 2: the conventional Top-K routing strategy, even with auxiliary load-balancing losses and conservative parallelism (EP8), still exhibits significant cross-device load imbalance, especially during post-training phases. This imbalance has two consequences: (a) training efficiency drops because some devices idle waiting for overloaded devices to finish, and (b) in extreme cases, overloaded devices run out of memory (OOM), crashing training entirely. This is not merely an engineering annoyance—it's a fundamental barrier to scaling MoE models beyond a certain size. Prior approaches (lowering the expert parallelism degree, adding balance loss terms) mitigate but do not solve the problem. The paper's Grouped Routing mechanism is presented as a solution that achieves absolute load balancing by design, not just approximate balancing through loss penalties.

3. Router embedding optimization in sparse MoE. A second architectural challenge is more subtle: when expanding from a smaller model (Intern-S1) to a larger one by cloning and expanding experts (the "upcycling" approach shown in Figure 2, left), the expert parameters (FFN weights) inherit good initializations, but the router embeddings—which determine how tokens are assigned to experts—start essentially untrained for the expanded expert pool. Under standard sparse routing, each token only updates the embeddings of its selected Top-K experts in the backward pass, meaning router embeddings for other experts receive zero gradient. With hundreds or thousands of experts, this means each individual expert's router embedding is updated only on a tiny fraction of training steps, severely slowing convergence. The Straight-Through Estimator (STE) approach described in Section 2.2 solves this by letting gradients flow through the full softmax distribution in the backward pass, updating all router embeddings on every step, even though the forward pass remains sparse. This is a known technique from the literature (Bengio et al., 2013; Hinton, 2012) but its specific application to MoE router training at trillion-parameter scale—and the demonstrated necessity for it—is a contribution of this work.

4. The multimodality gap: scientific data requires different treatment. Section 3 identifies a critical data problem: existing open-source image-text caption datasets (CC12M, LAION, COYO) are overwhelmingly focused on natural images—photographs of everyday scenes, objects, and people. Scientific images (Figures 6, 7) are fundamentally different: they are plots, diagrams, microscopy images, remote sensing data, molecular structures. Their captions in the scientific literature are typically brief and context-dependent ("Figure 3(b): Recovered intensity") rather than descriptive. Training a VLM on natural-image captions does not prepare it to understand, say, a transmission electron microscopy image with scale bars, diffraction patterns, and annotated defects. Moreover, scientific images are underrepresented in web-scale datasets. The paper's caption pipeline (Section 3.1) is specifically designed to convert PDF-extracted scientific figures—which are dense in domain information—into rich, aligned image-text pairs at scale (270B tokens of scientific caption data), directly addressing this modality gap.

5. Data conflicts between scientific and general training data. Section 3.2 identifies a problem that is often overlooked in mixed-domain training: scientific data (structured, deterministic, often tabular or formulaic) and general data (semantically diverse, linguistically rich) have very different statistical properties. Direct mixing can cause negative transfer—the model gets confused between the logical determinism expected in scientific tasks and the open-ended nature of general conversation. The paper's System Prompt Isolation strategy (Section 3.2) is a specific mechanism to address this: by injecting mutually exclusive system-level prefixes, the model learns to maintain separate "contextual modes" for scientific and general processing within the same parameters. This is not a theoretical contribution per se, but it reflects a practical insight that multi-domain training requires explicit architectural or procedural mechanisms to prevent domain interference.

6. RL training instability at trillion-parameter scale. Section 4.1 identifies a set of interlocking challenges for post-training with reinforcement learning. When training at trillion parameters with expert parallelism and FP8 quantization, small numerical discrepancies between the rollout engine (LMDeploy, used for generating responses during RL) and the training engine (XTuner, used for computing gradients) can cause significant training instability. The paper describes this as a training-inference engine discrepancy problem, following prior work (Yao et al., 2025; IcePop, 2025). The specific issues include: (a) operator-level precision mismatches (RMSNorm, router softmax, positional embeddings behave slightly differently between engines), (b) expert routing inconsistency (the rollout uses one set of expert selections, training might use a slightly different set due to numerical drift), and (c) log-probability estimation errors that get amplified by policy gradient updates. The comprehensive stabilization framework described in Section 4.1—operator-level alignment, rollout router replay, targeted mixed-precision, and dual importance sampling—is presented as necessary infrastructure for making RL work at this scale, not as optional optimizations.

How This Paper Positions Itself

Intern-S1-Pro positions itself at the intersection of several active research threads, each with established baselines that the paper aims to advance:

Against existing scientific foundation models: The paper benchmarks against a range of frontier models (Table 2), including not just open-source competitors like Qwen3-VL-235B and Kimi-K2.5, but also proprietary models like GPT-5.2 and Gemini-3-Pro. Its claim to uniqueness is not just beating these models on scientific benchmarks (though it does on many), but doing so while maintaining competitive general capabilities—embodying the "Specializable Generalist" vision rather than sacrificing general intelligence for domain specialization or vice versa.

Against the "specialized is better" paradigm: Section 5.5 is the most pointed contribution to the broader debate about generalist vs. specialist models. By directly comparing against Biology-Instruction—a specialized model trained on the exact same biological datasets but with a smaller architecture—and showing a 13.2 percentage point average improvement (52.45 vs. 39.24 across 21 tasks), the paper makes an empirical argument that scale + generality > specialization alone, at least when scale crosses a certain threshold. The Protein-Fluorescence task is particularly dramatic: 78.14 vs. 2.57. This suggests that the specialized model simply did not have the reasoning capacity to extract useful patterns from that specific task's data, while the trillion-parameter generalist could leverage its broader training to make sense of it.

Against the scaling-is-just-engineering view: The paper positions its architectural innovations (Grouped Routing, STE for sparse experts, the caption pipeline, the RL stabilization framework) not as mere engineering details but as enabling technologies without which trillion-parameter scientific training would be infeasible or unstable. The claim in Section 1 that Intern-S1-Pro scales to 4× the parameters of Intern-S1 with only a ~20% reduction in training efficiency is specifically framed as evidence that algorithm-infrastructure co-design is what makes this scaling possible—not just throwing more GPUs at the problem.

Within the framework of AI for Science (AI4S): The paper situates itself in the broader movement of using large models to accelerate scientific discovery (Merchant et al., 2023; Taylor et al., 2022; Singhal et al., 2023). Its contribution here is not a new scientific discovery itself, but a platform—a model that spans enough scientific domains (chemistry, materials, life sciences, earth sciences, with dedicated modules for time series, scientific imagery, and biological sequences) to serve as a general-purpose reasoning engine for researchers across disciplines. The inclusion of agentic capabilities (GAIA at 77.4, Tau2-Bench at 80.9, ScreenSpot V2 at 93.6) signals a vision where the model doesn't just answer scientific questions but can interact with tools, plan experiments, and navigate scientific workflows—a step beyond pure question-answering toward actual scientific assistance.

The Tension the Paper Creates (Implicitly)

An interesting undercurrent in this paper is the tension between breadth (covering 100+ scientific tasks across multiple disciplines) and depth (achieving expert-level performance on each). The paper's argument is that these are not in tension—that breadth (through scale) actually enables depth—but this is a hypothesis that the paper tests rather than assumes. The Biology-Instruction case study in Section 5.5 is the key evidence for this claim. If the hypothesis holds generally (which would require validation beyond the biology tasks shown), it suggests a fundamentally different scaling strategy for scientific AI: rather than building specialized models for each sub-problem, invest in ever-larger generalists that absorb specialization as an emergent capability. This is far from settled in the field—many domain scientists remain skeptical that a generalist can truly match the depth of a purpose-built model—and the paper's results, while compelling, are a single data point in this larger debate.

3. Technical Approach

3.1 Reader Orientation

Intern-S1-Pro is a trillion-parameter multimodal model built by expanding a smaller model (Intern-S1) through a novel expert replication strategy, then training it jointly on general and scientific data. The core problem it solves is that scientific domains are too diverse for smaller models to master without sacrificing general capabilities, and the solution's shape is a three-phase pipeline: (1) architecturally scale a Mixture-of-Experts model to 1T parameters with mechanisms that prevent training collapse at this size, (2) pre-train on 6T tokens of specially-curated multimodal data that includes 270B tokens of scientific image captions extracted from PDFs, and (3) post-train with reinforcement learning using a comprehensive stabilization framework that makes FP8 mixed-precision work reliably at trillion-parameter scale.

3.2 Big-Picture Architecture (Diagram in Words)

The system has six major components arranged in a training pipeline:

  1. Expert Expansion Module — takes the pre-trained Intern-S1 model and clones its experts into groups, initializing Intern-S1-Pro's 1T-parameter architecture. This is the "upcycling" step that creates the larger model from the smaller one.

  2. Grouped Router — replaces the standard Top-K routing mechanism with a per-group Top-1 selection that guarantees absolute load balancing across 8 expert-parallel devices. Without this, training would crash from memory overload on imbalanced devices.

  3. Vision Encoder (Native ViT) — a Vision Transformer that processes images at their original resolution rather than resizing to a fixed grid, preserving fine spatial detail critical for scientific imagery like microscopy and remote sensing data.

  4. Time-Series Module — a dedicated temporal encoder with adaptive subsampling that compresses variable-length signals (from 100 to 10⁶ time steps) into a uniform representation, enabling the language model backbone to reason about electroencephalography, astronomical light curves, and other continuous signals.

  5. Caption Pipeline — an offline data production system that extracts figures from PDFs using MinerU2.5, clusters them by domain, generates rich descriptive captions using InternVL3.5-241B (scientific) or CapRL-32B (general), and filters low-quality outputs with a 0.5B discriminator.

  6. RL Stabilization Framework — a set of four coordinated mechanisms (operator precision alignment, rollout router replay, targeted FP8 quantization, dual importance sampling) that enables reinforcement learning at trillion-parameter scale without divergence between the rollout engine (LMDeploy) and the training engine (XTuner).

3.3 Roadmap for the Deep Dive

  • First, the expert expansion and Grouped Routing architecture (Section 2), because these are the foundation—without them, the model simply cannot train at 1T parameters. This includes the load-balancing problem, the per-group Top-1 design, and why naive expansion fails.

  • Second, the Straight-Through Estimator for router training, because it solves the subtle problem that expanded routers receive gradients too sparsely to converge. Understanding this requires first understanding how the router works in the forward pass.

  • Third, the vision encoder and Fourier Position Encoding (FoPE), since these handle the multimodal input side—how images become tokens and how position is encoded to preserve spectral properties relevant for scientific data.

  • Fourth, the time-series module, which is a separate modality pathway with its own adaptive subsampling logic, architectural choices, and domain coverage.

  • Fifth, the pre-training data pipeline, particularly the caption production system and the three strategies for resolving scientific-general data conflicts (structured transformation, diversification, system prompt isolation).

  • Sixth, the RL post-training stabilization framework, which is the most technically complex component, involving operator-level precision alignment, router replay, targeted mixed-precision, and the dual importance-sampled REINFORCE objective.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and scaling paper whose core idea is that a sufficiently large generalist model, trained jointly on general and scientific data with careful architectural and data engineering, can outperform specialized models on their own domain tasks. The technical contribution is not a single algorithmic innovation but rather the integrated set of mechanisms that make trillion-parameter scientific MoE training feasible and effective.

Expert Expansion and Grouped Routing

The starting point for Intern-S1-Pro is not a from-scratch initialization but an upcycling process: taking the pre-trained Intern-S1 model and expanding it by replicating experts. This is motivated by two observations. First, training a 1T-parameter model from scratch would be prohibitively expensive and might not converge stably. Second, Intern-S1's experts already encode useful knowledge, and the challenge is to expand capacity while preserving this knowledge as a starting point—then letting the expanded experts differentiate through continued training.

The expansion procedure (Figure 2, left). Let the original Intern-S1 have $E$ experts per MoE layer. The expansion to Intern-S1-Pro creates $G$ groups, each containing a full copy of the original $E$ experts. If, for concreteness, Intern-S1 had 3 experts and the expansion creates 3 groups, Intern-S1-Pro has 9 experts arranged as: Group 1 contains copies of Expert 1, Expert 2, Expert 3; Group 2 contains copies of Expert 1, Expert 2, Expert 3; Group 3 contains copies of Expert 1, Expert 2, Expert 3. This is the homogeneous initialization approach. Each expert copy starts with identical weights to its counterpart in Intern-S1, and the groups are distinguished only by which input tokens they receive (determined by the Grouped Router, described below).

The paper reports testing an alternative—differentiated initialization—where Group 1 gets copies of the Top-1 and Top-2 experts (the ones most frequently selected by the original router), Group 2 gets copies of the next most frequent, and so on. This alternative approach caused a performance drop of over 20 points on a 30B-parameter model after 2000 training steps. The paper's explanation (Section 2) is that the Top-1 and Top-2 experts are the most well-trained and functionally important—they encode the most broadly useful computations. Placing these high-quality experts in all groups ensures that every group starts with strong computational primitives. The other approach, by assigning weaker experts to some groups, creates groups that cannot usefully process their assigned tokens, leading to training divergence. The critical property is that homogeneous initialization preserves per-group quality, and expert differentiation emerges naturally during training as different groups receive different token distributions and adapt accordingly.

The load imbalance problem. In a standard Mixture-of-Experts model with $N$ total experts and a Top-K routing strategy, each token is routed to the K experts with the highest router logits. With Expert Parallelism degree $S$, the $N$ experts are distributed across $S$ devices. Imbalance occurs when certain experts receive disproportionately many tokens—the device hosting those experts becomes a bottleneck while other devices sit idle. The paper states (Section 2.1) that this problem is "particularly severe in the post-training phase of large models" and can cause Out-of-Memory (OOM) errors. The conventional mitigation—adding an auxiliary load-balancing loss that penalizes uneven token distribution—only partially addresses the issue. Even with a low expert parallelism degree (EP8, meaning experts are split across only 8 devices), imbalance persists and threatens training stability.

The Grouped Router solution (Figure 3). The Grouped Router replaces global Top-K selection with per-group Top-K/G selection. Formally:

Let the total number of experts be $E$ and the expert parallelism degree be $S$. All experts are partitioned into $G$ mutually disjoint groups $\{\mathcal{E}_1, \mathcal{E}_2, ..., \mathcal{E}_G\}$ based on device mapping, with each group containing $E/G$ experts. For each group $g$, only the $\text{top-}(K/G)$ experts with the highest scores within that group are selected. The final set of activated experts is the union of these intra-group selections.

For Intern-S1-Pro's specific configuration: $K = 8$ (8 experts activated per token) and $S = 8$ (8-way expert parallelism). With $G = 8$ groups, each group selects $K/G = 8/8 = 1$ expert (Top-1 within group). Since each group is mapped to a different device, every device processes exactly one expert per token—absolute load balancing by construction, regardless of the token distribution. There is no need for an auxiliary balance loss, and the OOM risk from imbalance is eliminated.

Why not just use a larger parallelism degree? The paper doesn't explore this explicitly, but the logic is implicit: increasing the parallelism degree (e.g., EP16 or EP32) would spread experts across more devices, making individual imbalances less severe, but at the cost of increased communication overhead (all-to-all collectives for token dispatching and combining). The Grouped Router achieves perfect balance without increasing communication, by changing the routing algorithm itself rather than the distribution of experts.

Design rationale. This design is a structural solution to the load balancing problem, not a loss-based heuristic. Auxiliary balance losses create a tension between routing quality (sending tokens to the best experts) and load balance (sending tokens to underutilized experts), and the model can learn to "cheat" by routing tokens suboptimally to satisfy the balance loss. The Grouped Router avoids this tradeoff entirely: within each group, tokens still go to the highest-scoring expert, preserving routing quality, while the group structure guarantees per-device balance.

Straight-Through Estimator (STE) for Sparse Expert Routing

The router embeddings are the parameters that determine how input tokens are assigned to experts. When expanding from Intern-S1 to Intern-S1-Pro by replicating experts, the expert weights (the FFN parameters) have good initializations from the pre-trained model—they already encode useful computations. But the router embeddings are not directly replicated in the same way: the router must now learn to distribute tokens across an expanded set of experts, and the pre-expansion router weights are not directly applicable to the new architecture.

The gradient sparsity problem. Under standard Top-K routing, only the selected experts' router embeddings receive gradients in the backward pass. This is because the routing operation is a hard selection: for each token, the router computes logits $\mathbf{z} = \mathbf{W}_r \mathbf{x}$, applies softmax to get probabilities, selects the Top-K, and only those K experts are used. The gradient of the loss with respect to the router parameters $\mathbf{W}_r$ is zero for all non-selected experts, because changing their logits would not affect the output (they weren't selected). With $N$ total experts and $K \ll N$ selected per token, each expert's router embedding receives gradient updates from only roughly $K/N$ of the training tokens. For a trillion-parameter model with hundreds of experts, this means most router embeddings are updated on a tiny fraction of steps, leading to extremely slow convergence of the routing mechanism.

The STE solution (Equation 2). The Straight-Through Estimator decouples the forward and backward passes. In the forward pass, the model uses standard sparse Top-K selection exactly as before—no change to inference behavior or computational cost. In the backward pass, however, gradients are computed as if the token were routed to all experts with probabilities given by the softmax distribution, rather than only the selected experts.

The STE routing weight is defined as:

p^iSTE=sg(p~i)+(piτ−sg(piτ))\hat{p}_i^{\text{STE}} = \text{sg}(\tilde{p}_i) + (p_i^\tau - \text{sg}(p_i^\tau))

where $\tilde{p}_i = p_i / \sum_{j \in S} p_j$ is the normalized routing weight for expert $i$ (the softmax probability $p_i$ re-normalized over only the selected set $S$), $p_i^\tau = \text{softmax}(\mathbf{z}/\tau)_i$ is the routing probability computed with temperature $\tau$, and $\text{sg}(\cdot)$ is the stop-gradient operator (identity in forward pass, zero gradient in backward pass).

What it computes, step by step. In the forward pass, the stop-gradient operator $\text{sg}(\tilde{p}_i)$ passes $\tilde{p}_i$ through unchanged, and $\text{sg}(p_i^\tau)$ passes $p_i^\tau$ unchanged. The expression $(p_i^\tau - \text{sg}(p_i^\tau))$ evaluates to zero because both terms are identical. So $\hat{p}_i^{\text{STE}} = \tilde{p}_i$—the standard sparse routing weight. The model's forward computation is identical to standard Top-K routing.

In the backward pass, $\text{sg}(\tilde{p}_i)$ has zero gradient (it's treated as a constant). The term $(p_i^\tau - \text{sg}(p_i^\tau))$ contributes gradient $\partial p_i^\tau / \partial z_j$ because the $\text{sg}(p_i^\tau)$ term is constant. Critically, $p_i^\tau$ is computed from the full softmax over all $N$ experts, not just the selected ones. Therefore, the gradient of any loss $\mathcal{L}$ with respect to router logit $z_j$ (Equation 3) is:

∂L∂zj=∑i∈S∂L∂p^iSTE⋅∂piτ∂zj\frac{\partial \mathcal{L}}{\partial z_j} = \sum_{i \in S} \frac{\partial \mathcal{L}}{\partial \hat{p}_i^{\text{STE}}} \cdot \frac{\partial p_i^\tau}{\partial z_j}

The summation is over $i \in S$ (the selected experts) because only they receive non-zero loss gradients in the forward path, but $\partial p_i^\tau / \partial z_j$ is non-zero for all $j$ because the softmax connects every logit to every probability. This means that even experts not selected in the forward pass receive gradient updates through their influence on the softmax probabilities of the selected experts.

Operational meaning. In plain terms: the router learns as if every token provides a small gradient signal to every expert's embedding, proportional to how much that expert's logit influenced the softmax-normalized routing probabilities. If expert $j$ had a medium logit for a token but wasn't in the Top-K, the STE gradient tells the router "your logit contributed to the softmax denominator that determined the normalized probabilities of the selected experts—adjust accordingly." This provides dense gradient feedback to all router parameters on every training step, accelerating convergence dramatically compared to the sparse-gradient alternative.

Why temperature $\tau$? The paper uses $p_i^\tau = \text{softmax}(\mathbf{z}/\tau)_i$, a temperature-scaled softmax, rather than the raw softmax $p_i$ used in the forward pass. The temperature $\tau$ controls the sharpness of the softmax distribution in the backward pass. A lower $\tau$ makes the distribution peakier (closer to the hard Top-K selection), providing gradients that more closely approximate the true sparse routing behavior. A higher $\tau$ flattens the distribution, providing more uniform gradients to all experts. The paper does not specify the exact $\tau$ value used, but the inclusion of this parameter indicates that the gradient behavior is tunable—a design choice that trades off between accurate gradient approximation and the benefits of dense updates.

Why this form rather than alternatives. An alternative would be to simply remove the Top-K sparsity and route tokens to all experts in the forward pass during some training phases—but this would destroy the computational efficiency that makes MoE architectures worthwhile, since activating all experts per token costs as much as a dense model. Another alternative is to add an auxiliary loss that encourages all experts to have similar average logits—but this is a heuristic that doesn't directly provide learning signals about which tokens should go to which experts. The STE approach preserves the computational benefits of sparsity in the forward pass while providing dense gradient signals in the backward pass, directly optimizing the router's ability to assign tokens to experts without auxiliary loss terms.

Vision Encoder and Fourier Position Encoding (FoPE)

Native ViT encoder. Intern-S1-Pro uses a Vision Transformer (ViT) that processes images at their native resolution—the number of visual tokens depends on the input image's actual dimensions rather than being fixed to a pre-determined grid size. This is critical for scientific imagery because (a) microscopy images, remote sensing data, and scientific figures have widely varying aspect ratios and resolutions, (b) downsampling to a fixed size could destroy fine details (scale bars, diffraction spots, small annotations) that carry scientific meaning, and (c) preserving the original spatial structure allows the model to reason about spatial relationships that would be distorted by resizing. The extracted visual tokens pass through an MLP projector that maps them into the same embedding space as the language model's text tokens, enabling joint multimodal reasoning—the model can attend across visual and textual tokens in a unified representation space.

The ViT is trained with contrastive learning on approximately 300 million image-text pairs, using English caption datasets (CC12M, LAION-COCO, SBU Caption) and Chinese caption datasets (LAION-2B-Multi, Wukong). The contrastive objective aligns visual representations with their corresponding text descriptions in the embedding space, providing a strong initialization for the multimodal reasoning that the full model will perform.

Fourier Position Encoding (FoPE). The paper identifies a fundamental limitation in standard position encoding methods for scientific multimodal data. Standard approaches like Rotary Position Embedding (RoPE) encode position by applying rotations to token embeddings based on their sequence position. This effectively treats each embedding dimension as encoding a single frequency component—the rotation angle for dimension $j$ is proportional to $\theta_j$ for some base frequency. But physical signals (images, audio, time series) contain multiple frequency components superimposed—an image patch might contain both low-frequency background gradients and high-frequency edge information. Forcing each embedding dimension to represent a single frequency creates what the paper calls "spectral damage": information from different frequency bands gets mixed into the same dimensions, making it harder for the model to disentangle and process them.

FoPE (Section 2.4, Figure 4) addresses this by modeling each embedding dimension as a Fourier series—a sum of multiple frequency components—rather than a single frequency. Mathematically, while RoPE represents the embedding at position $n$ for dimension $j$ as roughly $h_j(n) = H_{\omega_j} \cdot e^{i\omega_j n}$ (a single-frequency function), FoPE represents it as:

hj(n)=∑kHωkj⋅eiωknh_j(n) = \sum_k H_{\omega_k}^j \cdot e^{i\omega_k n}

where $H_{\omega_k}^j$ is the coefficient for frequency $\omega_k$ in dimension $j$. Each dimension can now carry information from multiple frequency bands, separating them more cleanly across the embedding space.

Spectral damage mitigation (Figure 4, bottom). A second issue FoPE addresses is undertrained frequency components. During pre-training on sequences of a certain maximum length, high-frequency positional components receive adequate training (they cycle many times within the training length), but low-frequency components may complete only a fraction of a cycle. When the model is then asked to extrapolate to longer sequences at inference time (length generalization), these undertrained low-frequency components produce degraded periodic extensions—essentially, the model hasn't learned what to do with them because it never saw them complete a full cycle. FoPE clips (zeros out) these undertrained frequency components during length extrapolation, preserving only the adequately trained high-frequency components for extending to longer sequences. This is a form of frequency-domain regularization that prevents the model from relying on poorly-learned positional features when operating beyond its training distribution.

Why this matters for scientific data. Scientific time series, spectral data, and high-resolution images naturally contain information at multiple spatial and temporal frequencies. A position encoding that conflates these frequencies makes it harder for the model to learn frequency-specific patterns—for instance, distinguishing a slow diurnal cycle from fast neural oscillations in EEG data, or separating coarse tissue structure from fine cellular detail in microscopy. FoPE's multi-frequency representation provides a more natural inductive bias for this kind of data.

Time-Series Module

The time-series module is a dedicated neural network component that enables Intern-S1-Pro to process continuous temporal signals natively, rather than converting them to text tokens (which loses numerical precision) or images (which discards temporal structure). The module consists of two sub-components (Figure 5a): an adaptive subsampling module and a time series encoder.

Adaptive subsampling (Figure 5b). Time series from different scientific domains vary enormously in their sampling characteristics: an EEG recording might have 1000 samples per second over 10 minutes (600,000 time steps), while a yearly climate record might span decades at monthly resolution (hundreds of time steps). Directly feeding raw time series of such disparate lengths into a transformer would be either computationally infeasible (for long series) or information-poor (if aggressively downsampled uniformly).

The adaptive subsampling module solves this by dynamically computing the patch size and stride based on two input parameters: the raw signal itself and its sampling rate. The module contains a Patching Calculator that uses these inputs to determine parameters that keep the number of temporal frames within a controllable range. This is followed by dynamic patching—partitioning the signal into segments of the computed size—and then pooling within each segment (likely average or max pooling, though the paper doesn't specify the exact operation). The result is a compressed representation with a consistent number of temporal embeddings regardless of the original signal length, enabling the downstream transformer encoder to process sequences from 100 to 10⁶ time steps efficiently while preserving the structural features of the signal.

Time series encoder architecture. After adaptive subsampling, the compressed temporal embeddings pass through a standard transformer encoder (multiple layers of self-attention and feedforward networks) that captures long-range dependencies across the temporal segments. The encoder's output embeddings are then projected via an MLP into the shared embedding space of the language model, allowing the text-model backbone to attend to and reason about the temporal features alongside text and visual tokens.

Domain coverage. The module's training spans astronomy (light curves, spectral time series), geoscience (seismic waves, climate records), neuroscience (EEG, neural recordings), physiological signal analysis (ECG, heart rate variability), and bioacoustics (marmoset vocalizations). The paper reports results on the SciTS benchmark (Section 5.4, Table 3), where Intern-S1-Pro dramatically outperforms general VLMs: on the EAU01 task, Intern-S1-Pro achieves 99.5 F1 versus 67.6 for the next best model (Gemini2.5-Flash); on BIU03, 88.3 versus 17.9 (GPT-5-mini); on PHU04, 93.2 versus 64.8 (Gemini2.5-Flash as text-only). These gaps—particularly the leap from 1.5 to 20.8 on BIU01 and from 12.7 to 88.3 on BIU03—suggest that general VLMs without a dedicated temporal module fundamentally cannot extract the relevant patterns from these signals, likely because text-serialization loses the continuous temporal structure that the dedicated encoder preserves.

Design rationale. The adaptive subsampling approach is a middle ground between two extremes. Fixed-rate downsampling (e.g., always reduce to 100 time steps) would lose high-frequency information in long, densely-sampled signals or produce too few points for short signals. No downsampling (feeding all samples) would be computationally prohibitive and would waste capacity on redundant high-frequency samples while underrepresenting long-term trends. The adaptive approach normalizes all signals to a consistent representation length regardless of original duration or sampling rate, enabling the model to learn temporal patterns that generalize across different recording conditions—a critical requirement for a generalist scientific model that must handle whatever time series a researcher provides.

Pre-Training Data Pipeline

The pre-training phase consumes 6T tokens of multimodal data. The key technical contributions here are not the volume but the data production methods that create high-quality, aligned scientific multimodal training data from sources where such data didn't previously exist.

The caption pipeline for scientific images (Figure 7). Scientific figures in PDFs are rich in information but poor in aligned captions. The typical caption in a paper ("Figure 3(b): Recovered intensity") assumes the reader has the surrounding text for context—it describes what the figure shows, not what it looks like. For training a VLM, the model needs captions that explicitly describe visual elements: axis labels, legend entries, plotted curves, color scales, annotations, sub-figure layouts. The pipeline addresses this in five stages:

Stage 1: PDF extraction. The system uses MinerU2.5 for layout analysis and structural recognition—detecting figures, formulas, tables, and their bounding boxes within each PDF page, then cropping them into standardized sub-image samples. MinerU2.5 is described as a "decoupled vision-language model for efficient high-resolution document parsing" (Niu et al., 2025), suggesting it uses separate vision and language components to identify document structure.

Stage 2: Content deduplication. Perceptual hashing (pHash) eliminates duplicate visual content. This is important because scientific figures often appear in multiple versions (preprint and published, figure and supplementary), and duplicate training examples waste compute while potentially causing overfitting.

Stage 3: Domain routing and captioning. Extracted sub-images are classified by topic. Scientific images are captioned by InternVL3.5-241B, a large VLM that can generate professionally detailed, domain-specific descriptions—it knows the terminology for chemical structures, biological pathways, and physical phenomena. Non-scientific images (general photographs, diagrams) are captioned by CapRL-32B, a model trained with Reinforcement Learning with Verifiable Rewards (RLVR) specifically to produce dense, descriptive captions. CapRL's RLVR training optimizes caption quality against verifiable reward signals (likely measures of detail, accuracy, and completeness), stimulating the model to describe visual content thoroughly rather than producing terse summaries.

Stage 4: Quality filtering. A 0.5B-parameter text quality discriminator (a small model trained to classify text quality) filters out garbled text, repetitive expressions, and low-information-density content. This is the "last line of defense" against noisy captions that could degrade training. Using a small discriminator rather than the large captioning model itself for filtering is computationally efficient—the discriminator can process text at high throughput without the overhead of full VLM inference.

Stage 5: Multi-template prompting. To enhance linguistic diversity and prevent the model from overfitting to a single captioning style, the pipeline uses a randomized set of prompt templates when instructing the captioning models. This means the same figure might be described as "Describe this scientific figure in detail" in one instance and "Provide a comprehensive description of this plot" in another, producing varied but equally valid captions.

The output of this pipeline is approximately 270B tokens of scientific image-text caption data, spanning life sciences, chemistry, earth sciences, and materials science. To put this in context: 270B tokens is roughly the size of several large language model pre-training corpora, and it consists entirely of highly aligned, professionally captioned scientific visual content. This is likely the largest curated scientific multimodal dataset ever constructed.

Resolving conflicts between scientific and general data (Section 3.2). Training jointly on scientific and general data creates a specific problem: the two data types have fundamentally different statistical properties and the model can suffer from negative transfer—confusion between the modes. The paper describes three strategies to address this:

Strategy 1: Structured scientific data transformation. Scientific databases like PubChem store information in highly structured formats (tables, key-value pairs, structured fields). Directly linearizing these into text ("Property: value; Property: value; ...") produces training examples that look nothing like the natural language the model otherwise processes, creating a distribution mismatch. The solution has two components:

  • Template construction: Heterogeneous input-output pairs from structured databases are converted into grammatically correct narrative text using templates. For example, a PubChem entry with molecular weight, boiling point, and SMILES string might be converted to: "The compound with SMILES representation CCO has a molecular weight of 46.07 g/mol and boils at 78.37°C." This aligns the representation style of scientific data with natural language.

  • Task form transformation: Scientific outputs that are abstract (lists, matrices, numerical vectors) are mapped to descriptive answers with actual scientific meaning, guided by domain-specific scientific priors. Instead of outputting a raw vector of atomic coordinates, the model outputs a description of the molecular geometry. This "overcomes symbolic barriers"—the model learns to connect numerical representations to the physical concepts they encode, rather than treating them as arbitrary token sequences.

Strategy 2: Scientific data diversification. Scientific training data tends to be highly repetitive because the same concepts (e.g., protein sequences with similar folds, chemical compounds with similar scaffolds) appear many times with minor variations. This creates overfitting risk. The paper uses two techniques:

  • Prompt diversification: For the same scientific concept, dozens of varied instructions are provided, expanding generalization boundaries. A task about protein solubility might be phrased as "Predict the solubility of this protein," "Determine whether this protein is soluble," "What is the solubility class of the given protein sequence?" etc. This teaches the model the underlying concept rather than the specific phrasing.

  • Rollout mechanism: Scientific tasks often have oversimplified outputs (just a number or a class label). The rollout mechanism uses a strong base model to generate complete reasoning chains for these outputs, converting simple knowledge recall into logical deduction. For example, instead of just outputting "soluble," the training example includes a step-by-step reasoning chain connecting sequence features to solubility prediction. This significantly enhances zero-shot reasoning because the model learns the reasoning process, not just input-output mappings.

Strategy 3: System prompt isolation. Even with transformation and diversification, the fundamental differences between scientific and general data can cause conflicts. The system prompt isolation strategy injects mutually exclusive system-level prefixes—one for scientific data, one for general data—creating independent contextual processing environments within the same model parameters. The model learns to associate the scientific prefix with deterministic, structured, knowledge-intensive processing, and the general prefix with open-ended, linguistically diverse generation. This is analogous to having two "operating modes" in a single model, separated by a learned context switch rather than architectural separation.

Why system prompts specifically? An alternative would be to train separate models for scientific and general domains—but this defeats the purpose of building a unified scientific generalist. Another alternative would be to use separate adapter modules or LoRA weights for each domain—but this increases parameter count and inference complexity. The system prompt approach leverages the model's existing in-context learning capability: the prefix conditions the model's behavior without additional parameters, and the separation is a learned rather than architectural distinction. The paper claims this "effectively reduces data conflicts, improves model stability, and enhances training effectiveness."

Stable Mixed-Precision Reinforcement Learning for Sparse MoE Models

Post-training with reinforcement learning at trillion-parameter scale encounters a set of interlocking numerical and systems challenges that would cause training to diverge without specific countermeasures. The paper describes a comprehensive stabilization framework with four components.

The core challenge: training-inference engine discrepancy. During RL training, the model generates responses using a rollout engine (LMDeploy, optimized for fast inference) and then computes policy gradient updates using a training engine (XTuner, optimized for gradient computation). These two engines may implement the same mathematical operations with slightly different numerical precision, different kernel implementations, or different orderings of floating-point operations. These tiny discrepancies—invisible at the level of individual tokens—compound over the course of sequence generation and policy gradient computation, creating a distribution shift between the behavior that was evaluated (during rollout) and the behavior being optimized (during training). The paper explicitly cites prior work (Yao et al., 2025) that identified this as "a primary source of RL training instability."

Component 1: Systematic operator-level precision alignment. The team performed an operator-by-operator comparison between LMDeploy and XTuner, identifying specific kernels where numerical discrepancies were largest. The paper names three sensitive components: RMSNorm (root-mean-square normalization, used throughout the transformer layers), router softmax (determines expert assignments, where small numerical differences can change which expert is selected, cascading into different computation paths), and positional embedding application (how FoPE or other position encodings are applied to token embeddings). The paper reduced the error between the two stacks by minimizing precision gaps in these kernels—this likely involved aligning data types (e.g., both using FP32 accumulation), matching kernel implementations (e.g., both using the same algorithm for RMSNorm), or applying explicit numerical corrections during conversion.

Component 2: Rollout router replay. Expert routing is particularly sensitive to numerical discrepancy because it's a discrete decision (which expert processes which token) based on continuous values (router logits). A tiny difference in a router logit can change the Top-K selection, sending the token through an entirely different expert with different behavior—a discontinuity that the continuous training signal cannot account for. To eliminate this source of inconsistency, the system records the expert indices selected during rollout for every token at every MoE layer, and replays the exact same routing decisions during training. The training engine does not recompute routing—it uses the recorded indices, ensuring that the model is trained on gradients computed through the exact same expert pathways that generated the rollout responses.

The paper notes an implementation detail: transmitting these routing traces from the rollout engine to the training engine could become a bandwidth and latency bottleneck if sent through the same HTTP channel used for response tokens. Instead, routing traces are transmitted via Ray object references, a shared-memory mechanism that avoids serialization overhead and allows asynchronous access. This is a systems-level optimization that ensures the routing replay itself doesn't become a throughput bottleneck.

Component 3: Targeted mixed-precision with FP32 LM head. The paper follows Intern-S1's approach of using FP8 quantization to reduce memory pressure during RL, but with a tailored scheme for the MoE architecture. The key observation is that expert MLP layers—the feedforward networks inside each expert—account for the largest memory footprint (since there are many experts, each with its own parameters). Their GEMM (general matrix-matrix multiply) operations are comparatively tolerant to reduced precision because they process a subset of tokens, reducing the variance of accumulated numerical errors. Therefore, only expert linear layers are quantized to FP8.

Non-expert components—including attention layers, the router itself, and normalization layers—are kept in BF16, preserving numerical fidelity where it matters most. Critically, the language modeling head (LM head) is kept in FP32. The LM head computes log-probabilities from the final hidden states, and small errors in log-probability estimation are amplified by policy gradient updates because the REINFORCE objective scales gradients by the advantage estimate. A 1% error in log-probability translates to a 1% error in gradient magnitude, directly affecting optimization. FP32 provides sufficient precision to avoid this amplification.

Component 4: Dual importance sampling with masking (Equations 4–6). Even with router replay and precision alignment, there remains a distribution mismatch between the rollout policy $\pi_{\theta_{\text{rollout}}}$ (the model's state when generating responses) and the training policy $\pi_{\theta_{\text{train}}}$ (the model's state when computing gradients, which has been updated by several mini-batch steps since rollout). Additionally, mini-batch updates within a training epoch create an off-policy problem: the policy used to generate a particular batch of responses has been updated since that batch was produced. The loss function addresses both issues:

L(θ)=−Ex∼D,{yi}i=1G∼πθrollout(⋅∣x)[1G∑i=1G1∣yi∣∑t=1∣yi∣sg(M(ρi,t;α,β)⋅ri,t)⋅A^i,t⋅log⁡πθ(yi,t∣x,yi,<t)]\mathcal{L}(\theta) = -\mathbb{E}_{x \sim \mathcal{D}, \{y_i\}_{i=1}^G \sim \pi_{\theta_{\text{rollout}}}(\cdot|x)} \left[ \frac{1}{G} \sum_{i=1}^G \frac{1}{|y_i|} \sum_{t=1}^{|y_i|} \text{sg}(\mathcal{M}(\rho_{i,t}; \alpha, \beta) \cdot r_{i,t}) \cdot \hat{A}_{i,t} \cdot \log \pi_\theta(y_{i,t} | x, y_{i,<t}) \right]

where $G$ is the number of sampled responses per prompt, $|y_i|$ is the length of response $y_i$, and $\text{sg}(\cdot)$ is the stop-gradient operator.

The first importance sampling ratio $\rho_{i,t} = \frac{\pi_{\theta_{\text{train}}}(y_{i,t} | x, y_{i,<t})}{\pi_{\theta_{\text{rollout}}}(y_{i,t} | x, y_{i,<t})}$ corrects for the distribution mismatch between the training engine and the rollout engine. If the training policy assigns higher probability to a token than the rollout policy did, $\rho_{i,t} > 1$ and the gradient contribution is upweighted; if lower, $\rho_{i,t} < 1$ and it's downweighted.

The second importance sampling ratio $r_{i,t} = \frac{\pi_{\theta_{\text{new}}}(y_{i,t} | x, y_{i,<t})}{\pi_{\theta_{\text{old}}}(y_{i,t} | x, y_{i,<t})}$ corrects for off-policy bias from mini-batch updates. $\pi_{\theta_{\text{new}}}$ is the current policy (after some updates within the epoch), $\pi_{\theta_{\text{old}}}$ is the policy when the batch was generated. This ensures that the effective gradient step is computed as if the current policy had generated the responses.

The masking function $\mathcal{M}(\rho_{i,t}; \alpha, \beta)$ suppresses tokens where the training-rollout discrepancy is excessively large:

M(ρi,t;α,β)={ρi,t,α<ρi,t<β0,otherwise\mathcal{M}(\rho_{i,t}; \alpha, \beta) = \begin{cases} \rho_{i,t}, & \alpha < \rho_{i,t} < \beta \\ 0, & \text{otherwise} \end{cases}

where $\alpha$ and $\beta$ are thresholds. Tokens with importance ratios outside $[\alpha, \beta]$ are excluded from the gradient computation entirely (their contribution is zeroed out). This prevents the policy gradient from being dominated by a few tokens where the rollout and training policies have wildly different probabilities—those tokens likely represent degenerate behavior or numerical artifacts, and training on them would cause instability.

The advantage $\hat{A}_{i,t}$ is computed using a leave-one-out (LOO) baseline across the $G$ sampled responses:

A^i,t=Ri−bi,bi=1G−1∑j≠iRj\hat{A}_{i,t} = R_i - b_i, \quad b_i = \frac{1}{G-1} \sum_{j \neq i} R_j

where $R_i$ is the sequence-level reward of response $y_i$. The same advantage $\hat{A}_{i,t}$ is applied to all tokens $t$ within response $y_i$—this is a standard REINFORCE formulation where the per-token gradient is scaled by the sequence-level advantage. The LOO baseline reduces variance by comparing each response to the average of the others, rather than to a separately learned value function.

Operational meaning: what this loss function accomplishes. For each generated response, the model computes a per-token policy gradient that increases the log-probability of tokens in high-reward responses (where $\hat{A}_{i,t} > 0$) and decreases it for low-reward responses (where $\hat{A}_{i,t} < 0$). The dual importance sampling reweights each token's contribution by how consistent the current policy is with the policy that generated it, preventing large updates from stale data. The masking function clips extreme ratios, providing robustness against degenerate tokens. The stop-gradient on the masking and importance-weighting terms ensures that the gradient only flows through the log-probability term $\log \pi_\theta$, not through the importance weights themselves—this is standard for importance-weighted policy gradients to avoid the estimator optimizing its own weights rather than the policy.

Validation of FP8 mixed-precision (Figure 8). The paper validates this stabilization framework on a 30B MoE model, comparing FP8 mixed-precision RL against BF16 training. Figure 8a shows validation accuracy across optimizer steps: the two curves track closely, with FP8 matching BF16 throughout training. Figure 8b shows the KL divergence between log-probabilities from the training engine and rollout engine: both approach low values (around $2 \times 10^{-4}$), indicating that the precision alignment and router replay successfully eliminate systematic divergence between the two engines. At the start of training, BF16 shows higher KL (~5 × 10⁻⁴) than FP8 (~4 × 10⁻⁴), though the paper does not explain this initial advantage—it may reflect the targeted mixed-precision scheme's careful treatment of sensitive operations.

Why this comprehensive approach rather than simpler alternatives? The naive approach would be to run RL entirely in BF16 and simply accept the memory cost. But at 1T parameters, BF16 activations for all experts would exceed GPU memory even with expert parallelism—FP8 is necessary. A simpler FP8 scheme would quantize everything uniformly, but this causes instability in sensitive components (the LM head, the router) that the paper's targeted approach avoids. A simpler importance sampling approach would use only the first ratio $\rho_{i,t}$ without masking—but this leaves the model vulnerable to large importance weights from highly off-policy tokens, which can cause gradient spikes. The combination of dual ratios, masking, and LOO baselines represents a belt-and-suspenders approach to stability at a scale where a single training crash represents substantial wasted compute.

4. Key Insights and Innovations

Innovation 1: Trillion-Parameter Scale as a Qualitative Threshold for Generalist Superiority Over Specialists

The paper's most intellectually provocative claim is not that bigger models perform better—that's a scaling-law platitude—but that there exists a qualitative threshold in model capacity beyond which a generalist trained on diverse data systematically outperforms domain specialists trained on the exact same specialized data. This reframes the long-standing generalist-vs-specialist debate from a tradeoff (breadth costs depth) to a synergy (breadth enables depth), but only once scale crosses a critical point.

Prior to this work, the dominant assumption—articulated implicitly in the proliferation of domain-specific scientific models like Galactica (Taylor et al., 2022), various molecular property predictors, and dedicated protein models—was that specialization provides an inherent advantage: freeing the model from irrelevant training data allows it to allocate capacity entirely to the target domain. The paper does not argue this assumption is false in general; rather, it argues it becomes false once the general model is large enough that the "irrelevant" data provides transferable reasoning capabilities that outweigh the capacity cost. The analogy is implicit but instructive: a human expert with broad training in multiple sciences may reason more creatively about their specialty than one trained narrowly, but only if their cognitive capacity is sufficient to absorb the breadth without diluting the depth.

The evidence for this threshold effect is concentrated in Section 5.5 (Table 4). Both Intern-S1-Pro and the specialized Biology-Instruction model were trained on identical biological datasets, with Intern-S1-Pro's version only upgraded for more fluent text expression—the core biological information remained identical. Yet Intern-S1-Pro outperforms the specialist by 13.2 percentage points on average (52.45 vs. 39.24) across 21 biological sequence and structure tasks. The Protein-Fluorescence task is the canonical case: 78.14 vs. 2.57. The specialist model simply failed to extract learnable patterns from that data, while the trillion-parameter generalist—presumably leveraging reasoning patterns acquired from entirely different domains during pre-training—succeeded dramatically.

This finding has a specific epistemological status: it is a single empirical data point, not a proven law. The threshold is not characterized—we don't know whether 500B parameters would suffice, or 200B, or whether the effect would replicate on non-biological scientific tasks. But as a conceptual move, it reframes the scaling conversation from "how much compute do we need to match human performance on X" to "at what scale does generality transform from liability to asset." This is a more interesting question than raw performance curves, and it opens a research program around characterizing the generality-depth transition point.

Innovation 2: Structural Load Balancing as a Principled Alternative to Loss-Based Heuristics in MoE Training

The Grouped Router (Section 2.1, Figure 3) is not merely an engineering optimization—it represents a conceptual shift in how load balancing is achieved in Mixture-of-Experts architectures. Prior work universally relied on auxiliary load-balancing losses: add a penalty term to the training objective that nudges the router toward more uniform expert utilization. This creates an inherent tension between routing quality (send tokens to the best experts) and load balance (send tokens to underutilized experts to satisfy the loss), and models can learn to "cheat" by routing tokens suboptimally to reduce the penalty. The problem worsens at scale because the imbalance grows with the number of experts, and post-training phases (where the loss landscape is already shaped by pre-training) are particularly vulnerable to balance-loss gaming.

The Grouped Router eliminates this tradeoff entirely by making load balance a structural property of the routing algorithm rather than a soft constraint optimized via loss terms. The key insight is that absolute load balancing can be guaranteed by design if: (a) experts are partitioned into groups that map 1-to-1 onto devices, and (b) each group selects exactly the same number of experts per token (in Intern-S1-Pro's case, Top-1 within each of 8 groups under EP8). Routing quality is preserved within each group—tokens still go to the highest-scoring expert—while the group structure ensures per-device balance regardless of token distribution. The auxiliary balance loss becomes unnecessary, and the OOM risk from extreme imbalance is eliminated by construction.

This is a fundamental improvement over loss-based approaches, not an incremental refinement. It converts a continuous optimization problem with no guaranteed solution (achieving perfect balance through loss penalties) into a discrete structural guarantee. The analogy is to how certain parallel algorithms guarantee load balance through static partitioning rather than dynamic work-stealing heuristics—the former is less flexible but provably balanced, the latter can be more efficient on average but risks worst-case imbalance. The paper's empirical demonstration that this works at trillion-parameter scale with EP8 (and that the experts naturally differentiate after a few training steps despite homogeneous initialization) validates that the structural constraint does not meaningfully degrade routing quality in practice.

Innovation 3: The Dual Importance Sampling Framework as a Systematic Solution to RL Training-Engine Discrepancy

While the individual components of Section 4.1's stabilization framework (operator precision alignment, router replay, targeted mixed-precision, importance sampling) draw on prior work, their integration into a unified framework that addresses training-inference engine discrepancy at trillion-parameter scale represents a genuine conceptual contribution. The paper identifies a failure mode that is easy to overlook: RL for language models typically separates the rollout engine (fast inference for generating responses) and the training engine (gradient computation), and these engines inevitably implement operations with slightly different numerical behavior. At small scale, these discrepancies are noise; at trillion-parameter scale with FP8 quantization and thousands of experts, they become the dominant source of training instability.

The framework's insight is that this problem requires addressing multiple interacting sources of discrepancy simultaneously—fixing precision in isolation without addressing router inconsistency leaves residual instability, while aligning routers without importance-sampling correction for off-policy updates creates distribution shift between rollout and training. The paper's contribution is diagnosing that these are not separate issues but manifestations of a single root cause (the training and inference computational graphs diverge), and that a solution must coordinate across the operator level (precision alignment), the hardware level (router replay via Ray), the precision level (targeted FP8 with FP32 LM head), and the algorithmic level (dual importance sampling with masking).

The dual importance sampling objective (Equations 4–6) is particularly notable because it corrects for two distinct forms of distribution shift—training-vs-rollout discrepancy ($\rho_{i,t}$) and within-epoch off-policy updates ($r_{i,t}$)—that prior work typically addresses separately if at all. The masking function $\mathcal{M}(\rho_{i,t}; \alpha, \beta)$ that clips tokens with extreme importance ratios is a practical recognition that not all tokens can be reliably corrected: some represent degenerate behavior or numerical artifacts where the importance weight is so large that including them would dominate the gradient. This is a form of robust estimation applied to policy gradients, acknowledging that the importance-sampling correction itself has unbounded variance.

The validation in Figure 8—showing that FP8 mixed-precision matches BF16 training behavior and that KL divergence between engines converges to ~2 × 10⁻⁴—is a necessary condition for the framework's validity, not a performance claim. The contribution is making RL at trillion-parameter scale possible with stable convergence, not making it better than BF16 (which is infeasible at this scale). This is an enabling contribution: without it, the post-training phase of Intern-S1-Pro would not converge, and the model's final capabilities—including the agentic reasoning that achieves 77.4 on GAIA and 80.9 on Tau2-Bench—would be unattainable.

Innovation 4: Dense Scientific Caption Data as a New Modality for Training Scientific VLMs

The caption pipeline described in Section 3.1 (Figure 7) is not merely a data engineering contribution—it represents a reconceptualization of what training data should look like for scientific vision-language models. Prior VLMs trained on scientific content typically relied on web-scraped image-text pairs (where captions are noisy, brief, and poorly aligned) or manually annotated datasets (which are too small for large-scale pre-training). The paper's insight is that scientific PDFs are a latent corpus of high-density, professionally curated visual content—every published paper contains figures that are information-rich and carefully designed to communicate scientific meaning—and that the missing ingredient is a pipeline to convert these figures into the dense, descriptive captions that VLMs need for training.

This is a qualitative shift in data philosophy. Web-scraped captions teach models association ("this image is associated with these keywords"); the pipeline's captions teach detailed visual reasoning ("the y-axis shows spectral intensity from 0 to 1 in increments of 0.2; the solid blue line represents real data while the dashed purple line shows the NPRS reconstruction"). The paper's example in Figure 6 illustrates the gap: a natural caption might be ~100 words and describe the figure's role in the paper; the pipeline's caption averages ~1000 words and describes every visual element explicitly. This is not just more data—it's a fundamentally different kind of training signal that teaches the model to look at scientific figures rather than just associate them with topic keywords.

The 270B-token scale of the resulting dataset is itself significant—it's roughly the size of several major LLM pre-training corpora, but consisting entirely of highly aligned scientific visual content. No prior work has produced scientific image-text data at this scale or with this density of visual description. The domain routing mechanism (InternVL3.5-241B for scientific figures, CapRL-32B for general images) is a practical recognition that captioning quality depends on domain expertise—a general captioning model cannot describe a chemical structure diagram with the same precision as a science-aware model—and the quality filtering with a small discriminator is an efficient way to maintain data quality without the computational cost of re-running the full captioning pipeline.

Innovation 5: System Prompt Isolation as a Lightweight Mechanism for Domain-Aware Training Without Architectural Separation

The System Prompt Isolation strategy (Section 3.2) addresses a problem that is under-discussed in multi-domain model training: when scientific data (structured, deterministic, formulaic) and general data (diverse, open-ended, linguistically rich) are mixed in training, the model can suffer from negative transfer—confusion between the logical modes appropriate to each domain. The conventional solutions to this problem are architectural (separate expert modules for each domain, adapter layers, or domain-specific fine-tuning stages) or procedural (train on scientific data first, then general, or vice versa). Both approaches either increase model complexity or risk catastrophic forgetting.

The paper's insight is that the model's existing in-context learning capability can be leveraged to maintain domain separation without additional parameters. By injecting mutually exclusive system-level prefixes during training—one for scientific data, one for general data—the model learns to associate each prefix with the appropriate processing mode. At inference time, the prefix serves as a "context switch" that conditions the model's behavior without architectural modification. This is a minimalist solution to a real problem: it costs nothing in parameters, requires no changes to the model architecture, and leverages capabilities the model already possesses.

The conceptual contribution here is recognizing that negative transfer in multi-domain training is partly a contextual conditioning problem rather than a capacity allocation problem. If the model can learn to distinguish scientific from general contexts based on a simple prefix signal, the interference between domains can be managed through attention patterns and activation states rather than requiring physically separate parameters. The Structured Scientific Data Transformation and Scientific Data Diversification strategies that accompany this approach (converting structured data to narrative text, diversifying prompts, generating reasoning chains via rollouts) serve to make the scientific data more compatible with the general data distribution, while System Prompt Isolation handles the residual conflicts that transformation alone cannot resolve. The three strategies together represent a coherent philosophy: align the data distributions as much as possible through transformation, then use lightweight conditioning to handle what cannot be aligned. This is more elegant than the brute-force alternatives of separate models or complex multi-adapter architectures.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. The evaluation spans two broad categories. Scientific tasks are assessed on nine benchmarks: SciReasoner (149 concrete tasks across 9 disciplines evaluating scientific reasoning), SFE (830 verified VQA pairs across 66 multimodal tasks in five scientific disciplines), SmolInstruct (14 chemistry tasks with over three million samples), MatBench (13 materials property prediction tasks with 312–132,000 samples), Mol-Instructions (molecule-oriented, protein-oriented, and biomolecular text-oriented tasks), MicroVQA (1,042 expert-curated microscopy multiple-choice questions), Biology-Instruction (multi-omics sequence understanding tasks), XLRS-Bench (16 sub-tasks evaluating perception and reasoning on ultra-high-resolution remote sensing imagery), and MSEarth-MCQ (multiple-choice questions from earth science publications). General tasks span MMMU-Pro, MMLU-Pro, AIME-2025 (30 problems), IMO-Answer-Bench (400 Olympiad-level problems), RefCOCO-avg (referring expression comprehension), IFBench (58 out-of-domain instruction-following constraints), OCRBench V2, SArena (SVG generation, icon subset), LCB V6 (live competitive programming problems), GAIA (text-only, evaluated with web search tools including Google and Jina), Tau2-Bench (conversational agent evaluation in dual-control environments), and ScreenSpot V2 (1,200+ instructions across iOS, Android, macOS, Windows, and Web for GUI grounding). Table 2 reports results on all benchmarks.

  • Base model(s). Intern-S1-Pro is a 1-trillion-parameter Mixture-of-Experts model with 22B activated parameters per token (1T-A22B configuration). The predecessor, Intern-S1, serves as the direct comparison point for evaluating improvements from the scaling effort. For external comparisons, the paper benchmarks against Qwen3-VL-235B-Thinking (235B-A22B), Kimi-K2.5 (1T-A32B), GPT-5.2, and Gemini-3-Pro, covering both open-source and proprietary models at comparable scales. The choice of these baselines positions Intern-S1-Pro against the current frontier for both general multimodal capability and scientific reasoning depth.

  • Metrics. Task-specific metrics follow each benchmark's standard evaluation protocol. SciReasoner, SFE, SmolInstruct, MatBench, Mol-Instructions, MicroVQA, Biology-Instruction, XLRS-Bench, and MSEarth-MCQ all report accuracy or task-specific aggregate scores. For the SciTS time series benchmark (Table 3), F1 scores are reported per task. For general benchmarks, accuracy is the primary metric (MMMU-Pro, MMLU-Pro, AIME-2025, IMO-Answer-Bench, IFBench, OCRBench V2, LCB V6), with RefCOCO-avg using standard referring expression comprehension accuracy, SArena using SVG generation quality metrics, and GAIA, Tau2-Bench, and ScreenSpot V2 using task-specific success rates. All evaluations are conducted using three toolkits: OpenCompass, VLMEvalKit, and AgentCompass (an internally developed agent evaluation framework).

  • Baselines. The paper compares against several categories of models:

    • Open-source VLMs: Qwen3-VL-235B-Thinking (235B-A22B MoE) and Kimi-K2.5 (1T-A32B MoE), representing the current frontier of publicly available large multimodal models.
    • Proprietary models: GPT-5.2 (OpenAI) and Gemini-3-Pro (Google), representing the strongest closed-source alternatives.
    • Predecessor model: Intern-S1, to quantify the improvement from scaling and architectural innovations.
    • Domain-specific specialist model: Biology-Instruction (He et al., 2025), a specialized model trained on biological sequence and structure tasks, used in Section 5.5 to test the generalist-vs-specialist hypothesis.
    • Text-only and VL LLMs for time series: GPT-4.1-mini, Gemini2.5-Flash (both text-only and VL variants), DeepSeek-V3, and GPT-5-mini, used in Table 3 to evaluate the time-series module's contribution.
    • Evaluation toolkits: OpenCompass (Contributors, 2023), VLMEvalKit (Duan et al., 2024), and AgentCompass (internally developed) serve as standardized evaluation infrastructure rather than model baselines.

  • Generation budget / compute accounting. The paper does not use a unified "generation budget" metric for evaluation since it benchmarks final model performance rather than test-time scaling curves. Each model is evaluated under its standard inference configuration. Intern-S1-Pro uses two evaluation configurations (Table 1): Thinking mode (max 65,536 tokens, temperature 0.8, top_p 0.95, top_k 50) for benchmarks requiring extended reasoning chains (MMMU-Pro, MMLU-Pro, AIME-2025, IMO-Answer-Bench, IFBench, SArena, LCB V6, GAIA, Tau2-Bench, SFE, XLRS-Bench), and Non-Thinking mode (max 32,768 tokens, temperature 0, top_p 1.0, top_k 1) for benchmarks where deterministic, knowledge-focused responses are appropriate (SciReasoner, SmolInstruct, MatBench, Mol-Instructions, MicroVQA, Biology-Instruction, MSEarth-MCQ, RefCOCO-avg, OCRBench V2, ScreenSpot V2). For the GAIA benchmark, a simple agent workflow with web search tools (Google and Jina) is implemented. For Tau2-Bench, official evaluation settings are followed. No FLOPs-matched test-time compute comparisons are conducted during evaluation—the paper's compute-optimal allocation framework from Section 3 applies to training efficiency (the ~20% reduction in training throughput at 4× scale), not to inference-time strategy selection.

  • Cross-validation / statistical protocol. The paper does not report cross-validation procedures for the main benchmark evaluations in Table 2, as these rely on standard test-set evaluation against fixed benchmarks with pre-defined train/test splits. The scores are single-run results under the specified evaluation configurations (Table 1). No confidence intervals, standard deviations, or statistical significance tests are reported. For the Biology-Instruction comparison (Section 5.5, Table 4), both Intern-S1-Pro and the specialist Biology-Instruction model are evaluated on the same test sets without cross-validation. For the SciTS time series results (Table 3), F1 scores are reported per task ID without error bars. The paper's earlier cross-validation protocol described in the prior sections (two-fold cross-validation for compute-optimal strategy selection during RL training) does not appear in the evaluation methodology—the final benchmark results are single-pass evaluations.

Main Quantitative Results

Overall Performance Across Scientific and General Benchmarks (Table 2)

The headline result is that Intern-S1-Pro achieves first-place or second-place rankings on 7 out of 9 scientific benchmarks while maintaining competitive performance on general tasks. On scientific benchmarks specifically:

  • SciReasoner: Intern-S1-Pro achieves 55.5, substantially outperforming GPT-5.2 (13.6), Gemini-3-Pro (14.7), Kimi-K2.5 (15.3), and Qwen3-VL-235B-Thinking (11.9). This is the largest relative gap on any benchmark—Intern-S1-Pro scores approximately 3.8× higher than the next best proprietary model (Gemini-3-Pro at 14.7). The absolute magnitude of the lead (40.8 points over Gemini-3-Pro) strongly suggests that Intern-S1-Pro possesses scientific reasoning capabilities that the compared models fundamentally lack, not merely a marginal accuracy difference.

  • SmolInstruct: Intern-S1-Pro achieves 74.8, leading over Gemini-3-Pro (58.3), Kimi-K2.5 (53.5), GPT-5.2 (48.2), and Qwen3-VL-235B-Thinking (36.6). The 16.5-point gap over Gemini-3-Pro is notable because SmolInstruct tests chemistry-specific tasks (forward synthesis, property prediction) requiring domain knowledge that general-purpose VLMs are not explicitly trained on.

  • MatBench: Intern-S1-Pro achieves 72.8, ahead of Gemini-3-Pro (64.9), Kimi-K2.5 (60.0), GPT-5.2 (53.6), and Qwen3-VL-235B-Thinking (49.7). MatBench evaluates materials property prediction across 13 tasks, suggesting Intern-S1-Pro's scientific pre-training data (270B tokens of scientific image captions) transfers effectively to materials science.

  • Mol-Instructions: Intern-S1-Pro achieves 48.8, more than doubling the next best proprietary model (Gemini-3-Pro at 34.6) and substantially outperforming GPT-5.2 (12.3), Kimi-K2.5 (20.0), and Qwen3-VL-235B-Thinking (8.9). The wide gap—14.2 points over the next best—suggests that biomolecular instruction following is particularly challenging for general-purpose models, and Intern-S1-Pro's dedicated training on molecular data provides a clear advantage.

  • MicroVQA: Intern-S1-Pro achieves 63.3, placing second behind Gemini-3-Pro (69.0) but ahead of GPT-5.2 (60.4), Kimi-K2.5 (55.4), and Qwen3-VL-235B-Thinking (53.8). MicroVQA tests microscopy-based biological reasoning, a domain where visual fidelity to scientific imaging modalities matters. The 5.7-point gap to Gemini-3-Pro indicates that while Intern-S1-Pro is competitive, proprietary models still hold advantages in certain specialized visual reasoning tasks.

  • Biology-Instruction: Intern-S1-Pro achieves 52.5, massively outperforming all competitors: Gemini-3-Pro (12.0), Qwen3-VL-235B-Thinking (6.2), Kimi-K2.5 (10.7), and GPT-5.2 (10.2). The 40.5-point lead over the next best model (Gemini-3-Pro) mirrors the SciReasoner pattern—a gap of this magnitude suggests that the benchmark tests capabilities that general VLMs essentially lack, and Intern-S1-Pro's biological sequence understanding training is a decisive advantage.

  • XLRS-Bench: Intern-S1-Pro achieves 52.8, leading over Gemini-3-Pro (51.8), Qwen3-VL-235B-Thinking (51.2), GPT-5.2 (50.4), and Kimi-K2.5 (46.4). The margins are narrow here—only 1.0 point over Gemini-3-Pro—indicating that ultra-high-resolution remote sensing interpretation is an area where model capabilities are more closely clustered, possibly because the visual reasoning skills required overlap significantly with general vision capabilities.

  • MSEarth-MCQ: Intern-S1-Pro achieves 65.2, second to Gemini-3-Pro (65.8) and ahead of GPT-5.2 (62.6), Kimi-K2.5 (61.9), and Qwen3-VL-235B-Thinking (52.7). The 0.6-point gap to Gemini-3-Pro is the smallest on any benchmark, suggesting near-parity on earth science multimodal reasoning.

Aggregating across scientific benchmarks, Intern-S1-Pro achieves the highest score on 7 of 9 tasks, with second-place finishes on the remaining 2 (MicroVQA and MSEarth-MCQ). This represents a systematic advantage in scientific reasoning depth over both open-source and proprietary alternatives, with the largest advantages concentrated on benchmarks requiring specialized domain knowledge (SciReasoner, Mol-Instructions, Biology-Instruction) rather than general visual reasoning.

On general benchmarks, Intern-S1-Pro's performance is competitive but not dominant:

  • MMMU-Pro: 72.8, placing fourth behind Gemini-3-Pro (81.0), GPT-5.2 (79.5), and Kimi-K2.5 (78.5), but ahead of Qwen3-VL-235B-Thinking (69.9). The 8.2-point gap to Gemini-3-Pro indicates that proprietary models maintain an edge on multidisciplinary multimodal understanding.

  • MMLU-Pro: 86.6, third behind Gemini-3-Pro (89.3) and Kimi-K2.5 (87.1), but ahead of GPT-5.2 (85.9) and Qwen3-VL-235B-Thinking (83.4). The 2.7-point gap to the leader is relatively small, indicating strong knowledge-based reasoning.

  • AIME-2025: 93.1, third behind GPT-5.2 (100.0) and Kimi-K2.5 (96.1), ahead of Gemini-3-Pro (95.0) and Qwen3-VL-235B-Thinking (90.0). AIME evaluates competition-level mathematical reasoning; Intern-S1-Pro's score is strong but GPT-5.2's perfect score sets a ceiling.

  • IMO-Answer-Bench: 77.3, fourth behind GPT-5.2 (86.3), Kimi-K2.5 (81.8), and Gemini-3-Pro (81.3), ahead of Qwen3-VL-235B-Thinking (72.3). The 9.0-point gap to GPT-5.2 is substantial.

  • RefCOCO-avg: 91.9, first place ahead of Qwen3-VL-235B-Thinking (91.1), Kimi-K2.5 (87.8), Gemini-3-Pro (76.2), and GPT-5.2 (54.9). Visual grounding is a strength for Intern-S1-Pro, with GPT-5.2's notably low score (54.9) suggesting that this benchmark may not reflect the proprietary model's capabilities when evaluated under the paper's protocol.

  • IFBench: 71.2, second behind GPT-5.2 (75.4) and ahead of Gemini-3-Pro (70.4), Kimi-K2.5 (69.7), and Qwen3-VL-235B-Thinking (58.7). Instruction following is competitive but GPT-5.2 holds a 4.2-point advantage.

  • OCRBench V2: 60.1 (English) / 60.6 (Chinese), third in both behind Gemini-3-Pro (68.0 / 52.5) and Qwen3-VL-235B-Thinking (66.8 / 63.8) for English, and behind Qwen3-VL-235B-Thinking and Kimi-K2.5 for Chinese. OCR is not a standout strength.

  • SArena (Icon): 83.5, third behind Gemini-3-Pro (82.6) and GPT-5.2 (80.5), ahead of Kimi-K2.5 (77.3) and Qwen3-VL-235B-Thinking (76.3). SVG generation is closely clustered.

  • LCB V6: 74.3, fourth behind GPT-5.2 (87.7), Gemini-3-Pro (86.9), and Kimi-K2.5 (85.0), ahead of Qwen3-VL-235B-Thinking (72.0). Code generation is a relative weakness, with a 13.4-point gap to the leader.

  • GAIA (Text-Only): 77.4, second behind Kimi-K2.5 (79.9) and ahead of Gemini-3-Pro (75.5), GPT-5.2 (71.1), and Qwen3-VL-235B-Thinking (47.8). The agent capability using web search tools performs strongly, with the 29.6-point gap over Qwen3-VL-235B-Thinking indicating that the agent workflow implementation is a key differentiator.

  • Tau2-Bench: 80.9, second behind Gemini-3-Pro (85.4) and ahead of Kimi-K2.5 (76.8), GPT-5.2 (76.6), and Qwen3-VL-235B-Thinking (57.4). Conversational agent capability is strong.

  • ScreenSpot V2: 93.6, second behind Gemini-3-Pro (94.7) and ahead of Qwen3-VL-235B-Thinking (92.8), Kimi-K2.5 (92.4), and GPT-5.2 (49.4). GUI grounding is a strength, with GPT-5.2's 49.4 again being an outlier that may reflect evaluation protocol rather than genuine capability gaps.

The pattern that emerges is consistent: Intern-S1-Pro leads substantially on scientific benchmarks requiring specialized domain knowledge (SciReasoner, SmolInstruct, Mol-Instructions, Biology-Instruction), is competitive but not dominant on general multimodal reasoning (MMMU-Pro, MMLU-Pro, AIME-2025), and shows particular strength in agent-related capabilities (GAIA, Tau2-Bench, ScreenSpot V2) where the model's planning and tool-use training provides an advantage. Proprietary models (GPT-5.2 and Gemini-3-Pro) retain leads on mathematical reasoning (AIME-2025, IMO-Answer-Bench) and code generation (LCB V6), suggesting that scale and specialized scientific training do not fully compensate for advantages in these domains.

Improvements Over Intern-S1

The paper reports three categories of improvement over the predecessor model (specific values from Section 5.3):

  • General capability improvements: Performance on AIME-2025 increased from 86.0 to 93.1 (+7.1 points), and MMLU-Pro improved from 83.5 to 86.6 (+3.1 points). These gains reflect the effect of scaling to 4× the parameter count and training on 6T tokens of continued pre-training data.

  • Scientific task coverage expansion: Intern-S1-Pro adds coverage of benchmarks not evaluated on Intern-S1, including SciReasoner, Mol-Instructions, and Biology-Instruction. The paper claims these represent "more diverse and challenging domains" than the foundational scientific benchmarks evaluated on Intern-S1, though it does not explicitly list which benchmarks Intern-S1 was evaluated on for direct comparison. The expansion represents not just higher scores but a qualitative expansion of the model's scientific domain coverage.

  • Agent capability introduction: Intern-S1 did not have reported agent capabilities; Intern-S1-Pro introduces GAIA (77.4), Tau2-Bench (80.9), and ScreenSpot V2 (93.6) as new evaluation dimensions. This represents a capability that was not present in the predecessor, rather than an improvement on an existing capability.

The improvements in general benchmarks (particularly AIME-2025's 7.1-point gain) suggest that scaling to trillion parameters and continuing pre-training does not cause catastrophic forgetting of general capabilities—a risk that the paper's data strategies (System Prompt Isolation, data transformation) were designed to mitigate.

Time Series Results (Table 3)

The SciTS benchmark results in Table 3 provide the most direct evidence for the value of Intern-S1-Pro's dedicated time series module. The comparison is revealing because it contrasts Intern-S1-Pro against both text-only LLMs (GPT-4.1-mini, Gemini2.5-Flash, DeepSeek-V3) and vision-language LLMs (GPT-5-mini, Gemini2.5-Flash) on the same time series tasks, isolating the effect of the native temporal processing pathway:

  • EAU01: Intern-S1-Pro achieves 99.5 F1 versus 67.6 for VL Gemini2.5-Flash (next best). The 31.9-point gap is massive, indicating that the dedicated time series encoder extracts patterns that text-serialization and image-conversion approaches completely miss.

  • BIU03: Intern-S1-Pro achieves 88.3 versus 17.9 for GPT-5-mini (next best VL model). The 70.4-point difference is the largest gap on any task—the VL models are essentially failing on this biological signal processing task.

  • PHU04: Intern-S1-Pro achieves 93.2 versus 64.8 for text-only Gemini2.5-Flash. The 28.4-point advantage shows that even when text models perform reasonably, the temporal module provides substantial gains.

  • ASU01: Intern-S1-Pro achieves 98.0 versus 67.2 for text-only GPT-4.1-mini (next best). Astronomical signal processing benefits from native temporal modeling.

  • BIU01: Intern-S1-Pro achieves 20.8 versus 1.5 for text-only Gemini2.5-Flash (next best). Even Intern-S1-Pro's absolute score is low here, but the relative improvement (13.9×) demonstrates that the dedicated module can extract signal where other approaches see essentially nothing.

  • NEU06: Intern-S1-Pro achieves 71.3 versus 16.1 for text-only GPT-4.1-mini (next best for text models) and 13.3 for GPT-5-mini (next best VL model). The 55.2-point gap is consistent with the pattern of time-series-specific advantages.

The consistent 20–70 point advantages over both text-only and VL LLMs across diverse scientific time series tasks (astronomy, biology, neuroscience, physics) provide strong evidence that the adaptive subsampling module and dedicated time series encoder are not merely engineering optimizations but enable capabilities that alternative approaches fundamentally lack. Text serialization loses continuous temporal structure; image conversion discards numerical precision and temporal ordering. The native time series pathway preserves both.

Generalist vs. Specialist: The Biology-Instruction Case Study (Table 4)

Table 4 reports a per-task comparison between Intern-S1-Pro and the specialized Biology-Instruction model across 21 biological sequence and structure tasks. The critical context is that both models were trained on the same underlying biological datasets, with Intern-S1-Pro only upgrading the data for more fluent text expression while preserving core biological information. This makes the comparison a controlled test of the hypothesis that generalist scale can substitute for domain specialization.

Intern-S1-Pro achieves an average score of 52.45 across all 21 tasks versus 39.24 for Biology-Instruction, a 13.2-point advantage. The per-task breakdown reveals that this advantage is not uniform across tasks:

Tasks where Intern-S1-Pro dramatically outperforms:

  • Protein-Fluorescence: 78.14 vs. 2.57 (75.6-point gap). The specialist model essentially fails on this task, while Intern-S1-Pro achieves strong performance. This is the canonical example of the generalist enabling learning where a specialist could not.
  • Protein-FunctionEC: 72.70 vs. 19.79 (52.9-point gap). Enzyme Commission number prediction benefits from broader reasoning.
  • Multi_sequence-antibody_antigen: 44.76 vs. 10.26 (34.5-point gap).
  • DNA-cpd: 54.60 vs. 44.54 (10.1-point gap).
  • DNA-tf-h: 54.11 vs. 24.45 (29.7-point gap).
  • DNA-tf-m: 60.80 vs. 39.91 (20.9-point gap).
  • RNA-Isoform: 82.95 vs. 59.01 (23.9-point gap).
  • Protein-Thermostability: 59.56 vs. 45.07 (14.5-point gap).

Tasks where Intern-S1-Pro underperforms the specialist:

  • Multi_sequence-promoter_enhancer_interaction: -1.30 vs. 4.77. Intern-S1-Pro's negative score (the meaning of which is not explained) and the specialist's positive score indicate a specific task where specialization matters.
  • Multi_sequence-rna_protein_interaction: 58.51 vs. 74.26 (-15.8 points). The specialist holds a clear advantage.
  • RNA-NoncodingRNAFamily: 34.50 vs. 63.09 (-28.6 points). The largest specialist advantage.
  • RNA-Modification: 57.77 vs. 59.06 (-1.3 points). Essentially tied.

Tasks where both models perform similarly:

  • DNA-enhancer_activity: 55.16 vs. 53.28
  • Protein-Stability: 60.82 vs. 60.25
  • RNA-MeanRibosomeLoading: 52.41 vs. 47.64

The pattern suggests that the advantage of the generalist is not universal—there are specific biological tasks (RNA-noncoding RNA family classification, RNA-protein interaction prediction, promoter-enhancer interaction) where the specialized model's focused training provides benefits that the generalist's broader reasoning does not overcome. However, on a majority of tasks (15 of 21 show Intern-S1-Pro ahead), and on some tasks by enormous margins (Protein-Fluorescence, Protein-FunctionEC), the generalist's advantage is decisive.

Ablation Studies and Robustness Checks

Grouped Router vs. heterogeneous initialization (Section 2, Architecture): The paper reports an ablation on a 30B-parameter model comparing two initialization strategies for expert expansion. The homogeneous initialization method (copying the well-trained Top-1/Top-2 experts into every group) slightly outperforms the pre-expansion model after 2000 training steps. The heterogeneous initialization method (assigning differentiated experts corresponding to pre-expansion Top-1 through Top-8 across groups) results in a performance drop of over 20 points after 2000 steps. This ablation is critical because it validates the paper's hypothesis that ensuring every expert group contains well-trained computational primitives is essential for training stability, and that heterogeneous initialization—which might seem intuitively better because it provides more diversity—actually causes training degradation because some groups start with weak experts that cannot usefully process their assigned tokens. The paper does not report this ablation in a dedicated table; the values are stated in the Architecture section.

FP8 mixed-precision vs. BF16 RL training (Figure 8): On a 30B MoE model used as a validation proxy, the paper compares FP8 mixed-precision RL against BF16 training. Figure 8a shows validation accuracy across optimizer steps (0 to 1200): both FP8 and BF16 curves track closely, with final accuracy values around 0.76–0.78. Figure 8b shows the KL divergence between log-probabilities from the training engine and rollout engine: both converge to roughly 2–5 × 10⁻⁴ over 1200 steps, with FP8 showing slightly lower initial KL (~4 × 10⁻⁴ start vs. ~5 × 10⁻⁴ for BF16) and tracking BF16 closely throughout. This ablation validates that the comprehensive stabilization framework (operator-level precision alignment, rollout router replay, targeted mixed-precision, dual importance sampling) successfully prevents the FP8 quantization from introducing systematic divergence between the training and inference computational graphs. The paper does not report an ablation where FP8 is used without these stabilization measures, so the contribution of each individual component to overall stability is not isolated.

System prompt isolation (Section 3.2): The paper describes System Prompt Isolation as one of three strategies for resolving scientific-general data conflicts, but does not report an ablation where this strategy is removed. The effectiveness of structured data transformation and data diversification strategies is similarly not isolated through ablation experiments. The claim in Section 3.2 that these strategies "effectively reduce data conflicts, improve model stability, and enhance training effectiveness" is therefore based on the final model's performance rather than controlled comparisons of training with and without each strategy. This is a significant gap in the experimental validation—without ablations, the contribution of each data strategy to the final model quality cannot be disentangled.

Caption pipeline quality (Section 3.1): The paper does not report an ablation comparing model performance when trained with the dense scientific captions from the PDF pipeline versus training with only web-scraped captions or with the original brief captions from scientific papers. The contribution of the 270B tokens of scientific image-text caption data to final benchmark performance is therefore inferred from the overall scientific benchmark results rather than directly measured through a controlled training run without this data. A comparison of Intern-S1-Pro's scientific VQA performance against a version trained without the pipeline data would be a compelling validation of the caption pipeline's importance. The paper acknowledges the value of this data but does not experimentally isolate its effect.

Time series module contribution: Table 3 provides indirect evidence for the time series module's value by comparing Intern-S1-Pro against models that lack a dedicated temporal encoder (GPT-4.1-mini, Gemini2.5-Flash, DeepSeek-V3, GPT-5-mini). However, this is a comparison against entirely different model architectures, not an ablation of Intern-S1-Pro's own time series module. A direct ablation—evaluating Intern-S1-Pro on SciTS tasks with the time series module disabled, processing the same signals as text or images instead—would isolate the module's specific contribution but is not reported. The 20–70 point gaps in Table 3 are thus upper bounds on the module's value, conflating the effect of the dedicated encoder with differences in base model scale, training data, and overall architecture.

FoPE contribution: The Fourier Position Encoding is described in Section 2.4 as an architectural improvement over RoPE for scientific multimodal data, but no ablation comparing FoPE against RoPE on Intern-S1-Pro's benchmarks is reported. The paper does not isolate whether FoPE provides measurable benefits on scientific tasks (where spectral properties of data might matter) or general tasks (where length generalization might matter). The absence of a FoPE ablation means the contribution of this architectural choice to final performance is unquantified.

Straight-Through Estimator (STE) for sparse expert routing: Section 2.2 describes the STE as enabling dense gradient updates to all router embeddings, but no ablation comparing training convergence speed or final model quality with and without STE is reported. The paper's claim that STE "accelerates [router embedding] update frequency" is based on the mechanism's design rather than empirical comparison. An ablation comparing training loss curves or downstream benchmark performance with standard sparse routing gradients versus STE would quantify the practical benefit.

Revision model / self-correction ablated: Unlike Intern-S1, which explored sequential revision models (see Section 6 of the reference example), Intern-S1-Pro does not report any experiments with iterative self-correction, revision chains, or test-time refinement strategies. The evaluation is entirely on single-pass generation under Thinking or Non-Thinking configurations. The paper does not report whether Intern-S1-Pro benefits from techniques like best-of-N sampling, verifier-guided search, or sequential revisions that were studied in the predecessor. This is not an ablation per se, but it represents a dimension of test-time optimization that the paper does not explore, and comparing Intern-S1-Pro's single-pass performance against a test-time-compute-augmented version would provide a more complete picture of its capabilities.

Negative result: ReSTEM^{EM} training for revisions (referenced from Intern-S1, Appendix K): The predecessor paper Intern-S1 reported that attempting to optimize the revision model with ReSTEM^{EM} (Singh et al., 2024) caused performance to degrade substantially with sequential revisions (fully sequential dropping to ~33.5% compared to ~38.5% at the optimal ratio). Intern-S1-Pro does not report any revision model experiments, so this negative result is not directly applicable but suggests that revision training is sensitive to methodology in ways that may affect future work combining Intern-S1-Pro with test-time revision strategies.

Critical Assessment

The central claims of this paper, as established in the prior sections, are: (1) Intern-S1-Pro achieves state-of-the-art performance on scientific benchmarks while maintaining competitive general capabilities; (2) a sufficiently large generalist model trained jointly on general and scientific data can outperform domain specialists on their own tasks; (3) the trillion-parameter scale is made feasible by specific architectural innovations (Grouped Routing, STE, RL stabilization); and (4) dedicated modality-specific modules (time series encoder, native ViT, caption pipeline) provide decisive advantages for scientific data processing. The experimental evidence supporting each of these claims requires careful examination.

Claim 1 (Scientific SOTA + competitive general performance). The evidence in Table 2 provides substantial support, with important nuances. Intern-S1-Pro leads on 7 of 9 scientific benchmarks and places second on the remaining 2, with particularly large margins on SciReasoner (+40.8 over Gemini-3-Pro), SmolInstruct (+16.5), Mol-Instructions (+14.2), and Biology-Instruction (+40.5). These are not marginal improvements—they are qualitative gaps suggesting that competing models lack fundamental capabilities that Intern-S1-Pro possesses for these scientific reasoning tasks. However, there is a legitimate question about whether the compared proprietary models (GPT-5.2, Gemini-3-Pro) were evaluated under optimal prompting strategies for these specific scientific benchmarks. The paper does not describe any prompt engineering effort applied to competing models—if the same prompts used for Intern-S1-Pro were applied to other models without domain-specific adaptation, their scores may underestimate their true scientific capabilities. This is a standard concern in benchmark comparisons and does not invalidate the results, but it qualifies the claim of "state-of-the-art" as dependent on evaluation protocol consistency.

On general benchmarks, the "competitive" claim holds but the model is not dominant. Intern-S1-Pro places in the top two on only 5 of 12 general benchmarks (RefCOCO-avg at 1st, IFBench at 2nd, GAIA at 2nd, Tau2-Bench at 2nd, ScreenSpot V2 at 2nd) and trails on mathematical reasoning (AIME-2025: 93.1 vs. 100.0 for GPT-5.2; IMO-Answer-Bench: 77.3 vs. 86.3 for GPT-5.2) and code (LCB V6: 74.3 vs. 87.7 for GPT-5.2). This pattern is consistent with a model that has been heavily optimized for scientific depth at some cost to frontier general reasoning—a tradeoff the paper implicitly acknowledges through the "Specializable Generalist" framing. The claim of "competitive" general capabilities is accurate but should be understood as "top-tier among open-source models, trailing proprietary models on math and code."

Claim 2 (Generalist outperforms specialist on specialist's own data). This is the paper's most provocative claim, tested in Section 5.5 (Table 4). The evidence is strong but limited in scope. The Biology-Instruction comparison is a well-controlled experiment: same training data, different model scale and architecture, direct per-task comparison. The 52.45 vs. 39.24 average advantage (13.2 points) across 21 tasks provides compelling evidence that Intern-S1-Pro extracts more value from identical biological training data than a smaller specialized model. However, several caveats apply:

First, this is a single case study in biology—the paper does not demonstrate the same effect in chemistry, materials science, physics, or earth science. The generalist-superiority-over-specialist claim is currently demonstrated only for biological sequence and structure tasks. Extrapolating to all scientific domains is speculative.

Second, the specialized model (Biology-Instruction) is not characterized in terms of its architecture or parameter count. The paper does not state how large Biology-Instruction is, making it impossible to know whether the 13.2-point advantage reflects a threshold effect (trillion parameters crossing a critical capacity boundary) or simply a scale difference (any larger model might outperform a smaller one regardless of specialization). If Biology-Instruction is, say, a 7B model, the comparison demonstrates that a much larger model outperforms a much smaller one—which is unsurprising and does not require the "generalist synergy" explanation. If Biology-Instruction is a 100B+ model, the threshold claim is more substantive. The paper's lack of detail on the specialist model's scale is a significant omission that weakens this claim.

Third, the negative results are informative. On three tasks (RNA-NoncodingRNAFamily, Multi_sequence-rna_protein_interaction, Multi_sequence-promoter_enhancer_interaction), the specialist outperforms Intern-S1-Pro, in one case by 28.6 points (RNA-NoncodingRNAFamily). This demonstrates that specialization still provides advantages on certain problem types—the generalist advantage is not universal. The paper does not analyze why these specific tasks favor the specialist, but possible explanations include: the task structure is highly specific to biological sequences with little transfer from general reasoning, or the task's training data distribution is so narrow that the generalist's broader training introduces noise rather than signal.

Fourth, the claim that both models were trained on "the same underlying dataset" with Intern-S1-Pro only upgrading "for more fluent text expression" needs scrutiny. The paper's data preprocessing pipeline (Section 3.2) involves structured data transformation, prompt diversification, and rollout-based reasoning chain generation—these are not trivial "text expression" upgrades; they fundamentally change the training signal by adding reasoning chains and diversified prompts. The specialist model trained on raw data without these augmentations may be at a disadvantage not because Intern-S1-Pro is larger, but because Intern-S1-Pro's training data was meaningfully enriched. A fairer comparison would train the specialist on the same augmented data, controlling for both scale and data quality.

Claim 3 (Architectural innovations enable trillion-parameter training). This claim is supported by existence proof (the model was successfully trained) rather than controlled ablations. The Grouped Router ablation on 30B models (homogeneous vs. heterogeneous initialization causing a 20+ point performance gap) provides evidence that the initialization strategy matters at smaller scale, but does not directly demonstrate that the Grouped Router enables training at 1T parameters where alternative approaches would fail. The paper does not report attempting to train a 1T model without Grouped Router and observing failure—the claim that it "significantly enhances training stability" and "fundamentally eliminates the OOM risk" is based on the mechanism's design properties (absolute load balancing by construction) rather than a controlled comparison at trillion scale. This is reasonable for a systems contribution—it is often impractical to ablate critical infrastructure at extreme scale—but it means the claim is supported by engineering reasoning rather than experimental evidence. The STE's contribution similarly lacks an ablation; the paper does not compare training convergence with and without dense router gradient updates, making the practical benefit of STE for final model quality unquantified.

The RL stabilization framework is validated more rigorously: Figure 8 shows that FP8 mixed-precision matches BF16 training on a 30B proxy model, and the dual importance sampling objective (Equations 4–6) is mathematically specified. However, the validation is on a smaller model with 8 experts and 30B parameters, not the full 1T-parameter model with hundreds of experts. Extrapolating stability from 30B to 1T is not guaranteed, and the paper does not report training stability metrics (loss curves, gradient norm distributions, per-expert utilization statistics) for the full-scale training run. The claim that the framework "ensures strict precision consistency between training and inference" at 1T parameters is supported by the 30B proxy evidence but not directly validated at scale.

The ~20% training efficiency reduction claim (4× parameters, only ~20% slower training) is reported without detailed throughput measurements, hardware configuration specifications, or comparisons against alternative training setups. The paper states that this efficiency is achieved "through the co-design of algorithms and infrastructure" between XTuner and LMDeploy, but the specific optimizations enabling this efficiency are not enumerated, and no ablation showing the throughput impact of individual optimizations is provided.

Claim 4 (Dedicated modality modules provide decisive advantages). The time series results in Table 3 provide the strongest evidence for any specific architectural component. The 20–70 point advantages over models without dedicated time series encoders are so large that they cannot be plausibly attributed to scale differences or evaluation noise. However, as noted earlier, this is a comparison against entirely different models, not an ablation of Intern-S1-Pro's own time series module. The "decisive" part of the claim is well-supported; the "Intern-S1-Pro's encoder specifically" part is not directly tested. The vision encoder's contribution, the caption pipeline's contribution, and FoPE's contribution are similarly inferred from overall performance rather than isolated through ablation.

Missing experiments that would strengthen the paper:

  • Ablation of the caption pipeline: Train Intern-S1-Pro without the 270B tokens of scientific image captions and compare scientific VQA benchmark performance. This would isolate the pipeline's contribution.
  • Ablation of the time series module: Evaluate Intern-S1-Pro on SciTS tasks with the time series encoder replaced by text serialization, controlling for model scale. This would isolate the module's contribution from architecture differences.
  • Scaling curve analysis: Evaluate Intern-S1-Pro checkpoints at intermediate scales (e.g., 200B, 500B, 750B parameters) to characterize the relationship between model scale and scientific task performance. This would address the threshold question: at what scale does the generalist start outperforming specialists?
  • Biology-Instruction specialist trained on augmented data: Retrain the Biology-Instruction model on the same augmented data (with reasoning chains, diversified prompts) used for Intern-S1-Pro, to distinguish the contribution of data quality from model scale.
  • Confidence intervals: Report standard deviation or confidence intervals on benchmark scores, particularly for the smaller benchmarks (AIME-2025 has only 30 problems; IMO-Answer-Bench has 400; some scientific benchmarks may have similarly limited test sets). Without error estimates, it is unclear whether, for example, Intern-S1-Pro's 52.8 on XLRS-Bench versus Gemini-3-Pro's 51.8 is a statistically reliable difference or within noise.
  • Failure analysis on scientific benchmarks: The paper reports aggregate scores but does not provide qualitative examples of where Intern-S1-Pro succeeds or fails on scientific tasks. For SciReasoner, where the gap is 40.8 points over proprietary models, understanding what kinds of scientific reasoning tasks drive this gap would be more informative than the aggregate number.
  • Generalist-specialist comparison in chemistry or materials science: Replicate the Biology-Instruction case study design in another scientific domain to test whether the generalist advantage generalizes beyond biology.

Overall assessment: The experimental results in Table 2 convincingly demonstrate that Intern-S1-Pro is a highly capable scientific multimodal model that achieves excellent performance on specialized scientific benchmarks while remaining broadly competent on general tasks. The Biology-Instruction comparison (Table 4) provides intriguing evidence for the generalist-over-specialist hypothesis, though limited to a single domain and confounded by potential data preprocessing differences. The time series results (Table 3) provide strong evidence for the value of native temporal processing. However, the paper's architectural claims (Grouped Router, STE, RL stabilization) are supported primarily by design reasoning and small-scale validation rather than direct ablation at trillion-parameter scale, and several key components (caption pipeline, FoPE, data preprocessing strategies) lack experimental isolation of their contributions. The paper's contributions are best understood as an integrated systems achievement—the combination of architectural, data, and training innovations that makes a trillion-parameter scientific model feasible—rather than as a set of individually validated innovations where each component's marginal value is precisely quantified.

6. Limitations and Trade-offs

The Generalist-Over-Specialist Claim Is Validated in Only One Domain and Is Confounded by Data Quality Differences

The assumption or constraint. The paper asserts that "a sufficiently large generalist model, when trained jointly, can achieve superior performance" on specialized scientific tasks compared to domain-specific models trained on the same data (Section 1, Section 5.5). The experimental evidence for this claim rests entirely on a single case study comparing Intern-S1-Pro against the Biology-Instruction model across 21 biological sequence and structure tasks (Table 4).

The consequence. Extrapolating this result to chemistry, materials science, physics, or earth science is speculative. The claim of generalist superiority may be domain-specific—biological sequence tasks might benefit from transfer from general reasoning in ways that, say, quantum mechanical property prediction does not. More critically, the comparison is confounded: Intern-S1-Pro's training data was upgraded through the paper's preprocessing pipeline (structured data transformation, prompt diversification, rollout-based reasoning chain generation from Section 3.2), while the Biology-Instruction specialist was trained on the raw data. The paper characterizes this upgrade as "more fluent text expression" with "core biological information remained identical" (Section 5.5), but adding reasoning chains and diversified prompts fundamentally changes the training signal. The 13.2-point average advantage (52.45 vs. 39.24) cannot be cleanly attributed to model scale versus data quality without a controlled experiment where the specialist is trained on the same augmented data. Additionally, the paper does not report the parameter count of Biology-Instruction, making it impossible to assess whether the performance gap reflects a qualitative threshold (trillion-parameter scale enabling synergy) or a simpler scale difference (a much larger model outperforming a much smaller one, regardless of specialization). The negative results on three tasks—where Biology-Instruction leads by up to 28.6 points (RNA-NoncodingRNAFamily, 34.50 vs. 63.09—further indicate that generalist superiority is not universal, and the conditions under which it holds are uncharacterized.

What evidence exists in the paper. Table 4 reports the per-task comparison. The paper states both models "were trained on the same underlying dataset, and we only upgraded the data for Intern-S1-Pro to feature more fluent text expression" (Section 5.5). No information about Biology-Instruction's architecture or parameter count is provided. No analogous case study in chemistry, materials science, or physics is reported.

Mitigation status. None. The paper does not acknowledge this as a limitation that requires qualification of the generalist-over-specialist claim. No retraining of the specialist on augmented data, no scaling curve analysis to identify a threshold, and no replication in other scientific domains are offered. The absence of these controls means the claim should be treated as a promising hypothesis supported by a single suggestive data point, not an established finding.

Difficulty Estimation Cost and the Missing Compute Accounting for Pre-Training Data Production

The assumption or constraint. The paper's headline pre-training pipeline consumed 6T tokens, including approximately 270B tokens of scientific image-text caption data produced by the PDF caption pipeline (Section 3.1). This pipeline involves running large VLMs (InternVL3.5-241B for scientific figures, CapRL-32B for general images) on millions of extracted figures, plus a 0.5B-parameter quality discriminator. The computational cost of generating this caption data—running a 241B-parameter model at scale across a corpus of scientific PDFs—is not quantified anywhere in the paper.

The consequence. The paper's reported scientific benchmark performance (Table 2) reflects not just the capabilities of Intern-S1-Pro's architecture and training, but the substantial investment in synthetic data generation that preceded training. A practitioner considering reproducing or building upon this work cannot estimate the total compute budget required, because the captioning phase—which may represent a significant fraction of the overall cost—is unaccounted for. This matters particularly for organizations deciding between this approach (curate a massive domain-specific caption dataset) and alternatives (improve the base model's architecture or scale web-scraped data). Without cost quantification, the efficiency of the approach relative to simply scaling model parameters or web data cannot be assessed. The paper's claim about "only a ~20% reduction in training efficiency" at 4× scale (Section 1) refers exclusively to the training phase throughput and does not amortize the caption generation cost, making the overall resource efficiency of the full pipeline unknown.

What evidence exists in the paper. The paper describes the caption pipeline's architecture and output volume (270B tokens) in Section 3.1 and Figure 7, but provides no FLOP count, GPU-hours, or wall-clock time for caption generation. The paper does not include the caption generation cost in any efficiency analysis. No ablation compares Intern-S1-Pro's performance when trained with web-scraped captions versus pipeline-produced captions, which would quantify the caption data's marginal value relative to its (unknown) cost.

Mitigation status. Not addressed. The paper treats the caption pipeline as an input to the model rather than as part of the system's total computational cost. This is a standard practice in VLM papers (data curation costs are rarely reported), but it creates a meaningful gap for a paper whose central contribution is demonstrating the feasibility and value of scaling scientific models: the full cost of achieving the reported performance is unknown.

The Evaluation Benchmarks Cover Scientific Reasoning But Do Not Validate Actual Scientific Utility

The assumption or constraint. The paper evaluates Intern-S1-Pro on 9 scientific benchmarks (SciReasoner, SFE, SmolInstruct, MatBench, Mol-Instructions, MicroVQA, Biology-Instruction, XLRS-Bench, MSEarth-MCQ) and reports strong performance, including leads of 14–40 points over proprietary models on multiple benchmarks (Table 2). The paper frames these results as demonstrating that Intern-S1-Pro "advances the frontiers of AI in scientific discovery" and serves "as a unified interface for processing vast amounts of scientific literature, experimental data, and domain-specific knowledge" (Section 1).

The consequence. Benchmark performance does not guarantee real-world scientific utility. A model that scores 74.8 on SmolInstruct (chemistry tasks) has been tested on curated, standardized problems—it has not been validated on whether it can assist a chemist in designing a synthesis pathway for a novel compound, interpret ambiguous experimental results, or avoid proposing chemically impossible structures. The gap between benchmark accuracy and practical scientific assistance is particularly acute in science, where real tasks involve open-ended exploration, incomplete information, and safety-critical decisions (e.g., proposing compounds for synthesis that could be dangerous if the model hallucinates properties). The paper's claim that Intern-S1-Pro "master[s] over 100 specialized tasks across critical science fields" (Section 1) conflates benchmark performance with mastery—a standard overstatement in model release papers, but one with heightened stakes in scientific domains where errors carry concrete risks. The 40.5-point advantage on Biology-Instruction (52.5 vs. 12.0 for Gemini-3-Pro) is impressive, but the absolute score of 52.5 indicates that Intern-S1-Pro still fails on nearly half of biological sequence tasks, raising questions about reliability for actual biological research.

What evidence exists in the paper. All evaluation is on fixed benchmarks with known answer formats (multiple-choice, fill-in-the-blank, structured prediction). No qualitative case studies of Intern-S1-Pro assisting with real scientific workflows are provided. No expert evaluation by domain scientists is reported. The paper does not discuss failure modes (e.g., does the model confidently produce wrong answers on chemistry tasks, or does it appropriately express uncertainty?). No safety or reliability analysis for scientific applications is presented.

Mitigation status. Not addressed. The paper does not acknowledge the gap between benchmark performance and scientific utility as a limitation. Given the paper's framing as advancing "AI for Science" and enabling "acceleration of scientific discovery" (Section 6), the absence of any real-world validation or discussion of reliability in scientific contexts is a notable omission. The inclusion of agent capabilities (GAIA, Tau2-Bench, ScreenSpot V2) gestures toward practical utility, but these evaluate general agent tasks, not scientific workflow execution.

No Ablation of Individual Architectural and Data Innovations at Trillion-Parameter Scale

The assumption or constraint. The paper presents Intern-S1-Pro as the product of multiple coordinated innovations: Grouped Routing (Section 2.1), the Straight-Through Estimator for sparse expert routing (Section 2.2), Fourier Position Encoding (Section 2.4), the PDF caption pipeline (Section 3.1), three data preprocessing strategies (Section 3.2), and the comprehensive RL stabilization framework (Section 4.1). The paper's contributions are presented collectively—the model that results from all these innovations performs well on benchmarks.

The consequence. A practitioner or researcher cannot determine which innovations are essential for achieving the reported performance and which are incidental. If the caption pipeline data contributed 80% of scientific benchmark gains and the architectural innovations contributed 20%, an organization with limited resources might prioritize building a caption pipeline rather than implementing Grouped Routing—but there is no data to inform this decision. Conversely, if the RL stabilization framework is critical for trillion-parameter training but contributes minimally to scientific capability (since scientific performance may derive primarily from pre-training), then research effort might focus on pre-training data rather than RL techniques. The paper's integrated presentation—while understandable given the cost of trillion-parameter ablations—leaves open the possibility that a simpler model with the same 270B tokens of caption data and 6T tokens of pre-training could achieve comparable scientific performance, which would substantially change the paper's narrative about the necessity of its architectural innovations.

In particular, the claims about scaling efficiency are weakened by missing ablations. The paper reports that the model achieves "4× the size of its predecessor while incurring only a ~20% reduction in training efficiency" (Section 1), but without detailed throughput measurements, hardware specifications, or comparisons against alternative parallelization strategies, the generalizability of this efficiency claim to other hardware setups or model architectures is unknown.

What evidence exists in the paper. The Grouped Router ablation on 30B models (homogeneous vs. heterogeneous initialization, Section 2) validates initialization strategy at smaller scale but does not test whether Grouped Router enables training at 1T parameters where alternatives would fail. The RL stabilization framework is validated on a 30B proxy model (Figure 8) but not at the full 1T scale. No ablation of FoPE versus RoPE, no ablation of the caption pipeline (training with and without the 270B tokens of scientific captions), no ablation of system prompt isolation, and no ablation of the STE for router training are reported. The time series module's contribution is compared against entirely different models (Table 3) rather than ablated against an Intern-S1-Pro variant without the module.

Mitigation status. Partially acknowledged by omission—the paper does not claim that individual innovations are independently validated. However, the paper also does not explicitly state that the contributions should be evaluated collectively, leaving the reader to assume that each described innovation is important without evidence of its individual importance. The 30B-scale ablations provide some evidence for the Grouped Router and RL stabilization framework, but the leap from 30B to 1T parameters is substantial and the extrapolation is not validated. For a systems paper where the primary contribution is enabling trillion-parameter training, the absence of training stability metrics and throughput measurements at the full scale is a significant gap.

Scaling Laws and the Threshold for Generalist Superiority Are Uncharacterized

The assumption or constraint. The paper's central narrative—that scaling to a trillion parameters enables a qualitative shift where a generalist model surpasses specialists—implies that there exists a scale threshold below which the effect does not hold and above which it does. The paper establishes this at exactly one scale: 1 trillion parameters (with 22B activated). No intermediate model sizes are evaluated to characterize where this transition occurs, or whether the relationship between scale and scientific performance is continuous (diminishing returns at larger scales) or discontinuous (a threshold effect).

The consequence. Organizations deciding whether to invest in scaling to Intern-S1-Pro's size have no information about the return on intermediate investments. Would a 500B-parameter model with the same data achieve 80% of the scientific benchmark gains? 50%? Would an even larger model (2T, 5T parameters) continue to improve, or would scientific performance saturate? The absence of scaling curves makes Intern-S1-Pro a point estimate rather than part of a characterized scaling trend. This is a particularly acute gap given the paper's explicit motivation: in Section 1, the paper cites prior work showing that multilingual translation models require ~90× more parameters for hundreds of language pairs compared to bilingual models, arguing that scientific diversity imposes an analogous scaling requirement. But the paper does not empirically validate this analogy—it does not show, for example, that the number of scientific tasks mastered grows with model size, or that the rate of improvement on scientific benchmarks is faster than on general benchmarks as scale increases. The 1T parameter point could be on a steep part of the scaling curve (suggesting further scaling would yield large gains), a plateau (diminishing returns), or an inflection point (threshold crossed). Without additional points, none of these can be distinguished.

What evidence exists in the paper. The paper reports Intern-S1-Pro's performance at 1T-A22B (1 trillion total, 22B activated) and compares against Intern-S1 (unspecified size, but smaller than 1T) on two general benchmarks only: AIME-2025 (86.0 → 93.1) and MMLU-Pro (83.5 → 86.6). No scientific benchmark comparison between Intern-S1 and Intern-S1-Pro is reported in detail, and the paper notes that Intern-S1-Pro adds coverage of new scientific benchmarks (SciReasoner, Mol-Instructions, Biology-Instruction) that Intern-S1 was not evaluated on (Section 5.3). The FLOPs-matched comparison framework from the reference example's Section 7—which systematically varied model size and test-time compute—is not present in this paper; no scaling curves across model sizes are provided.

Mitigation status. Not addressed. The paper does not acknowledge the absence of scaling characterization as a limitation. Given the paper's explicit invocation of scaling arguments (the multilingual translation analogy in Section 1), the lack of scaling data is a meaningful gap in the evidence for the paper's core thesis. The claim that a trillion parameters represents a qualitative threshold remains, for now, an assertion supported by comparisons against smaller models of different architectures (Qwen3-VL-235B, Kimi-K2.5) rather than a controlled scaling study within the Intern-S1 architecture family. The paper's results demonstrate that this specific trillion-parameter model performs well on scientific benchmarks; they do not demonstrate that trillion-parameter scale is the reason, as opposed to data quality, training duration, or other confounded factors.

The Hardest Scientific Problems Show No Evidence of Improvement, and the Model's Capability Boundary Is Unknown

The assumption or constraint. The paper evaluates Intern-S1-Pro on benchmark datasets with fixed difficulty distributions and reports aggregate scores. These scores mask whether the model's advantages derive from improved performance on easy and medium problems (which constitute the bulk of most benchmark test sets) or genuine advances on the hardest scientific reasoning tasks that would represent frontier scientific discovery. The reference example's paper (Section 5, Figure 3 right) stratified results by difficulty quintile and found that test-time compute provided near-zero benefit on the hardest problems, establishing a clear capability boundary. This paper performs no difficulty-stratified analysis.

The consequence. A scientist deciding whether to use Intern-S1-Pro for their research cannot know whether the model's strong benchmark scores translate to competence on the genuinely novel, difficult problems that define scientific frontiers. The 55.5 on SciReasoner and 52.5 on Biology-Instruction represent average performance across mixed-difficulty benchmarks—the model could be near-perfect on routine knowledge retrieval questions and near-random on problems requiring novel reasoning, and the average would obscure this distinction. The 40.5-point lead on Biology-Instruction over Gemini-3-Pro (52.5 vs. 12.0) could reflect the proprietary model's near-total failure on the hardest subset combined with Intern-S1-Pro's moderate success, or Intern-S1-Pro's comprehensive advantage across all difficulty levels—the aggregate number cannot distinguish these scenarios. For AI-for-Science applications, performance on the hardest problems is disproportionately important because those are precisely the problems where human scientists need assistance; easy problems are already well-understood.

Furthermore, the paper's framing of agent capabilities (Section 5.3, Table 2) as enabling "autonomous planning and execution of complex scientific workflows" (Section 1) is not validated on any scientific agent benchmark. The agent evaluations (GAIA, Tau2-Bench, ScreenSpot V2) test general-purpose web navigation, customer service, and GUI interaction—not the ability to plan a chemical synthesis, design an experiment, or interpret ambiguous experimental data. The leap from "performs well on general agent benchmarks" to "can execute scientific workflows" is unsupported.

What evidence exists in the paper. Table 2 reports aggregate benchmark scores without difficulty stratification. No breakdown by question difficulty, by scientific sub-domain within benchmarks, or by reasoning type (knowledge retrieval vs. novel deduction vs. quantitative calculation) is provided. The SciTS results (Table 3) report per-task F1 scores, which provides some granularity for time series tasks but not for the main scientific reasoning benchmarks. The Biology-Instruction comparison (Table 4) reports per-task scores, revealing that Intern-S1-Pro underperforms on 3 of 21 tasks (RNA-NoncodingRNAFamily, Multi_sequence-rna_protein_interaction, Multi_sequence-promoter_enhancer_interaction), but this is not systematically analyzed in terms of task difficulty or characteristics that favor the specialist. No scientific agent workflow evaluation is reported.

Mitigation status. Not addressed. The paper does not acknowledge the absence of difficulty analysis or scientific agent validation. The claim that the model's "scientific expertise has been vastly expanded to master over 100 specialized tasks" (Section 1) is stated without qualification about the difficulty level of mastery or the boundary between mastered and unmastered tasks. A difficulty-stratified analysis of SciReasoner, SFE, or Biology-Instruction—paralleling the reference example's quintile analysis that revealed stark capability boundaries—would substantially strengthen the paper's scientific utility claims and provide actionable guidance for deployment. Without it, practitioners must assume that aggregate benchmark scores reflect uniform capability, an assumption the reference paper's results demonstrate is frequently invalid.

7. Implications and Future Directions

How This Work Changes the Landscape

From scaling-as-engineering to scaling-as-capability-threshold. The most consequential conceptual shift this paper introduces is the hypothesis that scale is not merely a continuous performance lever—more parameters, better results—but a qualitative threshold beyond which the relationship between generality and specialization inverts. Prior to this work, the dominant framing of the generalist-versus-specialist tradeoff assumed monotonicity: a model that trains on diverse data inevitably sacrifices depth in any single domain relative to a model trained exclusively on that domain, because capacity is finite and "irrelevant" data consumes it. Intern-S1-Pro's Biology-Instruction comparison (Table 4) provides the first large-scale empirical challenge to this framing. A trillion-parameter generalist trained on general and scientific data jointly outperforms a domain specialist on 18 of 21 biological sequence tasks, including by margins as extreme as 75.6 points on Protein-Fluorescence (78.14 vs. 2.57), despite both models using the same underlying training data. If this result replicates across other scientific domains—and the paper does not demonstrate that it does, so this remains a hypothesis rather than a finding—it reframes the scaling conversation entirely. The question shifts from "how large must a model be to match specialists" to "at what scale does generality transform from liability to asset," with the corollary that investing in breadth is not a compromise but a strategy for achieving depth.

This reframing has specific downstream effects on how the field thinks about model architecture. The "Specializable Generalist" concept—a single model that can deeply specialize to any scientific sub-domain while retaining broad general intelligence—becomes a design target rather than an aspirational slogan. It implies that specialized models (domain-specific fine-tunes, small expert systems for molecular property prediction) may become progressively less attractive as generalist scale increases, not because they are intrinsically worse, but because the generalist absorbs their capabilities as an emergent property. The paper's direct evidence for this is limited to biology, and the three tasks where Biology-Instruction outperforms Intern-S1-Pro (RNA-NoncodingRNAFamily by 28.6 points, Multi_sequence-rna_protein_interaction by 15.8 points, promoter-enhancer-interaction) demonstrate that the inversion is not universal, but the existence of any domain where a generalist decisively beats a specialist on the specialist's own data is a landmark result that the field will need to contend with.

Reconciling contradictions in multimodal scientific AI. The paper implicitly resolves a tension that has been latent in the AI-for-Science literature. On one side, works like Galactica (Taylor et al., 2022) demonstrated that models trained on scientific corpora can encode substantial domain knowledge, but their reasoning capabilities were limited. On the other side, general-purpose VLMs like GPT-4V and Gemini demonstrated strong reasoning but shallow scientific knowledge, often hallucinating plausible-sounding but incorrect scientific claims. The implicit question has been: can a single model do both? The paper's results suggest that at sufficient scale, the answer is yes—but that the scale requirement is non-trivial. The 40.8-point gap between Intern-S1-Pro and Gemini-3-Pro on SciReasoner (55.5 vs. 14.7) while Intern-S1-Pro remains competitive on MMLU-Pro (86.6 vs. 89.3, a 2.7-point gap) indicates that scientific depth and general reasoning are not in zero-sum competition at trillion-parameter scale; the model gains substantial scientific capability without proportionate loss of general capability. This is not a theoretical reconciliation—the mechanisms are not explained—but as an existence proof, it changes what is considered achievable for a single multimodal model.

Verifier over-optimization and test-time compute scaling become more urgent. The predecessor model Intern-S1 (and the reference example paper from this prompt's context) studied test-time compute scaling extensively, demonstrating that verifier over-optimization is the primary bottleneck preventing unbounded improvements from additional inference computation. Intern-S1-Pro does not explore test-time scaling strategies—its evaluation is entirely single-pass—but the trillion-parameter scale makes the verifier bottleneck more acute, not less. A larger model with stronger base capabilities produces more diverse outputs, making it harder for a verifier to reliably discriminate correct from incorrect solutions. The Grouped Router and STE innovations that enable 1T-parameter training do nothing to address verifier robustness. This means that the natural next step—applying the compute-optimal test-time scaling framework from Intern-S1 to Intern-S1-Pro—will immediately encounter the same over-optimization ceiling, potentially at higher absolute performance levels but with the same qualitative limitation. The paper's contribution thus shifts the research landscape: scaling model size has been demonstrated as a viable path to scientific capability, but unlocking that capability at inference time—through search, revision, or agentic planning—requires solving the verifier robustness problem that Intern-S1 identified and Intern-S1-Pro inherits unchanged.

The role of data curation in scientific AI becomes elevated. The paper's caption pipeline—producing 270B tokens of dense, descriptive scientific image captions from PDFs—represents a specific methodological contribution, but its broader implication is that domain-specific data engineering is not a pre-training afterthought but a first-class research contribution for scientific models. The pipeline's scale (270B tokens is roughly the size of several major LLM pre-training corpora) and its design philosophy (generating ~1000-word captions that explicitly describe every visual element rather than the ~100-word associative captions typical of web data, as illustrated in Figure 6) set a new standard for what "high-quality training data" means for scientific VLMs. Future scientific models will be expected to document their data curation with the same rigor as their architectural innovations; the days of training on whatever web-scraped image-text pairs are available and hoping the model absorbs scientific knowledge through osmosis are, if not over, at least challenged by the existence proof that a dedicated pipeline yields 40-point advantages on scientific reasoning benchmarks.

Agent capabilities become a standard evaluation dimension for scientific models. Intern-S1-Pro's inclusion of GAIA (77.4), Tau2-Bench (80.9), and ScreenSpot V2 (93.6) alongside traditional scientific and general benchmarks signals a shift in what constitutes a complete evaluation of a scientific foundation model. The paper argues, through its evaluation choices, that scientific AI is not merely about answering questions—it is about interacting with tools, navigating interfaces, and executing multi-step workflows. This is a substantive claim: a model that scores 74.8 on SmolInstruct but cannot use a web browser to look up reaction conditions is not a useful scientific assistant, regardless of its benchmark scores. The paper does not evaluate scientific agent tasks specifically (GAIA and Tau2-Bench test general-purpose agent capabilities), but the inclusion of these benchmarks in the main results table alongside scientific benchmarks establishes an expectation that future scientific models will report agent performance as a core metric, not an optional add-on.

Follow-Up Research This Work Enables

Characterize the generalist-superiority threshold through controlled scaling experiments. The paper establishes that at 1T-A22B scale, Intern-S1-Pro outperforms a biology specialist on most biological sequence tasks. What is unknown—and what a follow-up study could determine—is the shape of the scaling curve between specialist and generalist performance. A strong experiment would train Intern-S1-Pro variants at multiple scales (e.g., 200B, 400B, 600B, 800B, 1T parameters) using identical data and training recipes, then evaluate each against the same Biology-Instruction specialist on the same 21-task benchmark (Table 4). The outcome would be a curve showing at what parameter count the generalist crosses the specialist's performance on each task, revealing whether the transition is continuous (gradual improvement with scale) or discontinuous (a sharp threshold). The Protein-Fluorescence task, where the specialist scores 2.57 and Intern-S1-Pro scores 78.14, is a candidate for the most dramatic threshold; understanding whether this task becomes learnable at 500B, 750B, or only at 1T parameters would provide the first empirical characterization of when general reasoning transfers to specialized scientific domains. If the threshold varies substantially across tasks—some requiring 200B, others requiring beyond 1T—that would refine the "Specializable Generalist" concept into a more nuanced picture of which scientific capabilities are emergent at which scales.

Replicate the generalist-over-specialist finding in chemistry and materials science. The Biology-Instruction case study (Section 5.5, Table 4) is a single data point. A direct replication in chemistry would train a specialist chemistry model on SmolInstruct data (the same 14 tasks and three million samples used in the benchmark) and compare its performance against Intern-S1-Pro, controlling for data quality by training the specialist on the same augmented data (with reasoning chains and diversified prompts from Section 3.2) that Intern-S1-Pro received. The prediction is that Intern-S1-Pro would show a similar advantage to the biology case—but the null result (specialist matches or outperforms the generalist) would be equally informative, suggesting that the generalist advantage is domain-specific and may depend on the degree to which general reasoning transfers to the domain's problem structures. Chemistry tasks like forward synthesis and property prediction have different reasoning patterns than biological sequence tasks; if the generalist advantage is smaller or absent in chemistry, the "Specializable Generalist" claim would need to be scoped to domains where general reasoning provides strong inductive bias. A materials science replication using MatBench (13 tasks) would provide a third test, particularly valuable because materials properties (band gaps, formation energies) are governed by physical laws that may be less amenable to transfer from general reasoning than biological sequences, where pattern recognition from diverse training data may be more directly applicable.

Develop and evaluate scientific agent workflows using Intern-S1-Pro as the reasoning backbone. The paper evaluates agent capabilities on general benchmarks (GAIA, Tau2-Bench, ScreenSpot V2) but not on scientific workflows. A natural follow-up would design a benchmark for scientific agent tasks: given a research question (e.g., "find the melting point of this compound and propose a recrystallization solvent"), the model must search the literature (PubMed, PubChem), interpret retrieved figures and tables, reason about chemical compatibility, and output a protocol. Intern-S1-Pro's native time series module, scientific image captioning, and biology/chemistry knowledge make it a strong candidate for the reasoning backbone of such a system, but the integration challenge is non-trivial. A concrete experiment would implement an agent scaffold around Intern-S1-Pro with tool access (web search, PubChem API, molecular visualization) and evaluate on expert-curated scientific workflow tasks, measuring both task success rate and the quality of intermediate reasoning steps (e.g., by domain expert evaluation). The paper's 77.4 on GAIA demonstrates general agent competence; the open question is whether this transfers to scientific workflows where the tools and reasoning patterns are domain-specific. A negative result—Intern-S1-Pro performs well on general web tasks but poorly on scientific workflow execution—would indicate that domain-specific agent training (not just domain-specific knowledge) is required, a finding that would redirect research toward scientific tool-use fine-tuning rather than assuming general agent capabilities transfer automatically.

Ablate the caption pipeline's contribution to isolate the value of dense scientific image descriptions. The paper reports a 270B-token scientific caption dataset but does not ablate its contribution to final performance (see Section 6). A controlled follow-up would train two Intern-S1-Pro variants: one with the full caption pipeline data (270B tokens of dense, descriptive captions), and one where the same 270B tokens are replaced with web-scraped captions of equal volume (e.g., from LAION or COYO). Evaluating both on scientific VQA benchmarks (SFE, MicroVQA, XLRS-Bench, MSEarth-MCQ) would reveal the marginal value of caption quality and density independent of caption volume. The prediction, based on the Figure 6 comparison showing the gap between typical ~100-word natural captions and the pipeline's ~1000-word dense captions, is a substantial gap in scientific image understanding—perhaps 10–20 points on visual scientific benchmarks. Quantifying this gap would serve two purposes: (a) justifying the substantial compute investment in the caption pipeline (running InternVL3.5-241B at scale), and (b) providing guidance to practitioners about whether building a similar pipeline should be a priority. If the gap is small (dense captions provide only marginal benefit over web-scraped captions when the base model is large enough), the finding would redirect effort away from caption engineering and toward architectural scaling.

Investigate verifier robustness at trillion-parameter scale using Intern-S1-Pro as the base proposal distribution. Intern-S1 demonstrated that process reward model (PRM) search and iterative revisions can improve test-time performance by up to 4× in compute efficiency, but also that verifier over-optimization is the binding constraint—beam search degrades on easy problems at high budgets (Figure 3 right in the reference paper). Intern-S1-Pro's 1T-parameter scale intensifies both sides of this tradeoff: the base model produces higher-quality candidate solutions (raising the ceiling of what test-time compute can achieve), but the diversity and sophistication of its outputs make verifier training more challenging (because the verifier must discriminate more subtle errors). A systematic study would train a PRM on Intern-S1-Pro's output distribution using the Monte Carlo rollout procedure from Intern-S1, then replicate the compute-optimal scaling analysis—sweeping best-of-N, beam search, and lookahead search under varying budgets, stratifying by difficulty—on Intern-S1-Pro's performance on SciReasoner or SFE. The key outcome would be whether the over-optimization ceiling shifts: does Intern-S1-Pro's stronger base model push the point where verifier over-optimization kicks in to higher budgets, or does the increased output sophistication make the verifier more vulnerable to exploitation? The answer determines whether the next frontier for scientific AI is improving base models (scaling further) or improving verifiers (solving the over-optimization problem). If verifier over-optimization remains the bottleneck even at 1T parameters, it would establish that verifier robustness—not model scale—is the critical path for unlocking test-time compute at frontier scale, redirecting research investment accordingly.

Stress-test the Grouped Router and STE at expert counts beyond Intern-S1-Pro's configuration. The Grouped Router (Section 2.1) guarantees absolute load balancing by design when the number of groups equals the expert parallelism degree and each group selects Top-1. Intern-S1-Pro uses K = 8 experts per token with EP8, mapping cleanly to 8 groups each selecting 1 expert. But what happens when these constraints are relaxed—for example, with K = 16 and EP8 (each group would need to select Top-2, reintroducing within-group imbalance), or with hundreds of experts per group (where the Top-1 selection might degrade routing quality because the single best expert in a large group may not be as good as the Top-2 in a smaller group)? A scaling study varying K and the number of groups at fixed total expert count would characterize the routing quality versus load balance tradeoff in the Grouped Router design space. The paper's validation on a 30B proxy model (Section 2) establishes feasibility; extending this to larger scales with systematic variation of routing parameters would provide the engineering guidance needed for the next generation of even larger MoE models. Similarly, the STE's benefit (Section 2.2) is asserted but not measured; training with and without STE on a model with hundreds of experts and measuring router embedding convergence speed and final routing entropy would quantify whether STE's dense gradient updates provide practical convergence acceleration or are unnecessary at scale.

Practical Applications and Downstream Use Cases

Scientific literature understanding and cross-domain synthesis at scale. Intern-S1-Pro's combination of dense scientific image captioning (270B tokens of training data from the pipeline in Section 3.1), dedicated time series processing (Table 3: 99.5 F1 on EAU01, 88.3 on BIU03), and strong agent capabilities (77.4 GAIA, 80.9 Tau2-Bench) makes it immediately deployable as a literature analysis engine for research institutions and pharmaceutical R&D. The specific use case is processing the scientific literature at scale: given a corpus of PDFs across chemistry, biology, and materials science, Intern-S1-Pro can extract and interpret figures (not just text), reason about time series data embedded in papers (e.g., comparing kinetic curves across publications), and answer cross-domain queries that require synthesizing information from figures in one paper with text from another. The 55.5 on SciReasoner (versus 14.7 for Gemini-3-Pro) and 74.8 on SmolInstruct suggest that the model can reason about scientific content with substantially greater depth than alternatives currently available to practitioners. For a pharmaceutical company evaluating drug candidates, this translates to an automated system that can read thousands of papers, extract quantitative data from figures (binding affinities, toxicity curves, pharmacokinetic time series), and flag compounds meeting specified criteria—a workflow that currently requires teams of human scientists reading papers individually.

On-device or edge deployment of specialized scientific assistants via distillation from Intern-S1-Pro. While Intern-S1-Pro itself is a 1T-parameter model requiring substantial compute, its demonstrated ability to serve as a Specializable Generalist—a single model that performs at expert level across multiple scientific domains—makes it an ideal teacher model for distillation into smaller, domain-specific student models. The specific pipeline: use Intern-S1-Pro to generate high-quality reasoning traces, answers, and figure interpretations on scientific training data (leveraging the caption pipeline's output and the Rollout mechanism from Section 3.2), then fine-tune a much smaller model (e.g., 7B–70B parameters) on these synthetic examples. The Biology-Instruction comparison (Table 4) demonstrates that Intern-S1-Pro extracts substantially more value from biological training data than a smaller specialized model—the distillation hypothesis is that this value can be transferred to a smaller model through synthetic data generation, potentially achieving most of Intern-S1-Pro's scientific capability at a fraction of the inference cost. This matters for scenarios where a scientist needs a specialized assistant on their laptop or a lab instrument needs embedded intelligence: running a 1T-parameter model is infeasible, but running a 7B model distilled from Intern-S1-Pro's outputs on a specific scientific domain (e.g., protein property prediction) may be practical. The paper's strong performance on benchmarks where small models fail entirely (Protein-Fluorescence: 78.14 vs. 2.57 for the specialist) suggests that Intern-S1-Pro has learned reasoning strategies that a smaller model cannot discover from raw data but might learn from Intern-S1-Pro's chain-of-thought examples.

Multimodal scientific data integration for experimental labs. Intern-S1-Pro's native support for images (native ViT, Section 2.3), time series (adaptive subsampling module, Section 2.5), and text (the language model backbone) makes it uniquely suited for environments where multiple data modalities converge. A concrete deployment scenario is a materials characterization lab: an experiment produces X-ray diffraction patterns (images), thermogravimetric analysis curves (time series), and synthesis notes (text). Intern-S1-Pro can ingest all three modalities simultaneously—interpreting the diffraction pattern to identify phases, analyzing the TGA curve to quantify thermal stability, and reasoning across modalities to determine whether the synthesis produced the intended material. The 72.8 on MatBench and 99.5 F1 on the EAU01 time series task (Table 3) suggest the model has genuine capability in both domains; the integration question is whether these capabilities compose when multiple modalities are presented together. For a lab scientist, this replaces the current workflow of using separate specialized software for each data type with a single unified interface, potentially catching cross-modal inconsistencies (e.g., the diffraction pattern suggests phase A but the TGA decomposition temperature is inconsistent with phase A's known properties) that isolated analysis would miss.

When to Prefer This Method

The paper's architecture and training methodology position Intern-S1-Pro against two distinct alternatives: (a) training separate specialized models for each scientific domain (the pre-Intern-S1-Pro status quo), and (b) using general-purpose proprietary VLMs (GPT-5.2, Gemini-3-Pro) for scientific tasks. The decision rules that emerge from the paper's results are:

  • Prefer the Intern-S1-Pro approach (trillion-parameter generalist with dedicated scientific data and modality modules) when:

    • The problem spans multiple scientific disciplines and requires cross-domain reasoning. Intern-S1-Pro's 14.2-point lead on Mol-Instructions and 40.5-point lead on Biology-Instruction over proprietary models (Table 2) indicate that general-purpose VLMs lack the domain depth, while separate specialized models cannot integrate across biology and chemistry.
    • The application requires processing scientific data in native modalities (time series signals, high-resolution microscopy, remote sensing imagery) rather than as text or downsampled images. The 20–70 point advantages on SciTS time series tasks (Table 3) over both text-only and VL LLMs demonstrate that the dedicated time series module and native-resolution ViT provide capabilities that alternative approaches fundamentally lack.
    • There is a large corpus of domain-specific PDFs available for caption extraction. The 270B-token caption pipeline (Section 3.1) is a major investment, but for organizations with access to proprietary scientific document collections (pharmaceutical companies, national labs, research universities), the return on building a similar pipeline may be substantial given the 40-point SciReasoner advantage that the dense caption data likely contributes to.
    • The deployment involves distillation to smaller models, where Intern-S1-Pro's "Specializable Generalist" property makes it a versatile teacher for multiple domain-specific students. The Biology-Instruction comparison (Table 4) suggests that Intern-S1-Pro extracts more value from specialized data than a smaller specialist can, and this extracted knowledge can potentially be transferred through synthetic data generation.

  • Prefer specialized models over the Intern-S1-Pro approach when:

    • The problem is confined to a single, narrow scientific domain where the generalist advantage has not been demonstrated. The three Biology-Instruction tasks where the specialist outperforms Intern-S1-Pro—RNA-NoncodingRNAFamily (63.09 vs. 34.50, a 28.6-point specialist advantage), Multi_sequence-rna_protein_interaction (74.26 vs. 58.51), and promoter-enhancer interaction (4.77 vs. -1.30)—indicate that specialization still benefits certain problem types. If the target task resembles these (highly specific biological sequence classification with limited transfer from general reasoning), a purpose-built specialist may outperform even a trillion-parameter generalist.
    • The training budget is constrained and cannot support a trillion-parameter model. The paper's ~20% training efficiency reduction at 4× scale (Section 1) suggests that Intern-S1-Pro training is feasible with the described infrastructure but remains a substantial investment. Without characterized scaling curves (see Section 6), there is no evidence about what fraction of Intern-S1-Pro's scientific capability could be achieved at, say, 100B or 500B parameters with the same data—practitioners with limited budgets face uncertainty about the return on scaling investment.
    • The primary requirement is frontier mathematical reasoning or code generation. Intern-S1-Pro trails GPT-5.2 by 6.9 points on AIME-2025 (93.1 vs. 100.0), by 9.0 points on IMO-Answer-Bench (77.3 vs. 86.3), and by 13.4 points on LCB V6 (74.3 vs. 87.7). For applications where math or code are the dominant modalities, proprietary general-purpose models or specialized math/code models remain preferable based on current evidence.

  • Prefer general-purpose proprietary VLMs (GPT-5.2, Gemini-3-Pro) over Intern-S1-Pro when:

    • The task is general multimodal reasoning without deep scientific domain requirements. Intern-S1-Pro trails Gemini-3-Pro by 8.2 points on MMMU-Pro (72.8 vs. 81.0) and by 2.7 points on MMLU-Pro (86.6 vs. 89.3), and trails GPT-5.2 by 4.2 points on IFBench (71.2 vs. 75.4). For applications that do not leverage Intern-S1-Pro's scientific depth, proprietary models maintain an edge on general multimodal understanding.
    • Latency or throughput requirements preclude running a 1T-parameter MoE model. The paper does not report inference latency or throughput, but a 1T-A22B model with 22B activated parameters per token will be substantially more expensive per query than smaller or dense models, regardless of any efficiency optimizations. For high-throughput or latency-sensitive applications without scientific requirements, smaller proprietary or open-source models may be more practical.
    • The deployment environment cannot accommodate the infrastructure requirements of a trillion-parameter model (distributed inference across multiple GPUs). Intern-S1-Pro is explicitly positioned as an open-source release, but the hardware requirements for inference at this scale are non-trivial and may exclude single-GPU or consumer-hardware deployments.