AI Can Learn Scientific Taste

ArXiv: 2603.14473

🎯 Pitch

AI models trained on citation patterns don’t just memorize popular fields—they develop a generalizable scientific taste that transfers across time, unseen disciplines, and even aligns with human peer review. A 30B parameter judge trained this way outperforms GPT-5.2 and Gemini 3 Pro at spotting high-impact research, showing that community verdicts embed learnable quality signals beyond raw popularity.

1. Executive Summary

This paper introduces Reinforcement Learning from Community Feedback (RLCF), a training paradigm that uses large-scale citation signals as community supervision to learn scientific taste—formally decomposed into judgement capability (comparing the potential impact of research ideas) and ideation capability (proposing ideas with high potential impact). Using SciJudgeBench, a dataset of 700K field- and time-matched paper abstract pairs where preferences are derived from citation counts, the authors train Scientific Judge (a generative reward model for pairwise paper comparison via GRPO) and Scientific Thinker (a policy model that proposes high-impact follow-up research ideas using Comparison-Based GRPO with Scientific Judge as the reward signal). Scientific Judge (SciJudge-Qwen3-30B) achieves 80.6% accuracy on in-domain citation preference prediction, outperforming GPT-5.2-Thinking (72.7%) and Gemini-3.0-Pro (75.7%), while Scientific Thinker (SciThinker-30B) attains 81.5% win rate against its untrained base policy in proposing research ideas. The findings establish that scientific taste can be learned from community feedback signals—generalizing across time (2025 holdout papers), unseen fields (Math, Physics when trained only on CS), and evaluation metrics (peer-review scores at ICLR)—demonstrating that citation-trained judgement captures transferable patterns of scientific value rather than merely fitting field-specific or temporal correlations.

2. Context and Motivation

The Core Problem: AI Scientists Lack Judgment, Not Just Execution

The fundamental question this paper tackles is whether AI can learn what experienced researchers call scientific taste — the elusive capacity to judge which research ideas deserve pursuit and to propose directions with high potential impact. The paper frames this as a capability gap in current AI science systems: while recent progress has focused intensively on execution (literature search, automated experimentation, code generation), the judgment aspect — knowing what to work on rather than how to work on it — remains largely unexplored.

This distinction matters because scientific taste is not merely aesthetic. The paper draws on philosophical foundations from Hume and Kant to argue that taste represents a form of shared community judgment rather than arbitrary individual preference. Hume's notion of a "standard of taste" emerging from "the joint verdict of qualified judges" and Kant's conception of taste as sensus communis — a shared sense that considers how others would judge — provide the intellectual scaffolding for the paper's central claim: that scientific taste manifests through community-level signals that can be captured and learned.

The practical stakes are significant. The paper cites human evaluation studies showing that while LLMs can generate novel research ideas, "they often struggle to reliably distinguish potentially high-impact directions from ideas that are superficially novel but trivial" (Si et al., 2024). This ideation-execution gap (Si et al., 2025) means that an AI scientist that can run experiments but cannot identify which experiments are worth running is fundamentally incomplete. In research environments with finite resources — which is all of them — the ability to prioritize directions based on likely impact is at least as valuable as the ability to execute on a chosen direction.

Why Citations as Supervision Make Conceptual Sense

The paper's choice of citations as the community feedback signal deserves careful motivation because it is philosophically non-trivial. Citations are not a perfect measure of scientific merit — the paper explicitly acknowledges that "some high-potential papers may receive few citations initially but become highly influential later." However, citations possess specific properties that make them uniquely suitable as a training signal for scientific taste:

They reflect community-level consensus, not individual preference. This directly operationalizes Hume's conception of taste. A single reviewer's opinion may be idiosyncratic; a paper accumulating thousands of citations across years represents an aggregate judgment from the research community. The paper formalizes this through the concept of cumulative expected impact (Equation 1), treating citation counts over time as a statistical signal of community preference rather than a direct measure of quality.

They are naturally occurring and scalable. Unlike human preference annotations (the standard approach in RLHF), citation data exists at massive scale — the paper collects 2.1M arXiv papers with citation data — and requires no additional labeling effort. This addresses a fundamental limitation of RLHF identified in the paper: its "reliance on costly human annotations and inability to reflect community-level preferences through individual preferences alone."

They provide pairwise signals naturally. Raw citation counts are unsuitable as absolute quality scores because they are confounded by field norms, publication age, and other factors. However, when two papers are matched by field and publication time, the difference in their citation counts becomes a relative signal: within the same community at the same moment, the paper receiving more attention likely addresses problems of broader interest or proposes more useful methods. This pairwise framing converts noisy absolute counts into a cleaner preference signal.

Where Existing Approaches Fall Short

The paper identifies limitations across three dimensions of prior work:

1. AI for scientific review is superficial. Recent systems (AgentReview, MARG, DeepReview, CycleResearcher) apply LLMs to review tasks — predicting scores, generating feedback, evaluating manuscripts — but "these works primarily employ language models as components in review pipelines, rather than enhancing the model’s intrinsic capability for scientific judgment." The critical distinction is between using an LLM as a review tool and training an LLM to develop better judgment. Prior work that does train models for review typically uses supervised fine-tuning on reviewer feedback, which captures individual reviewer patterns rather than community-level preferences. The paper argues this is categorically different from learning taste from aggregate community signals.

2. Ideation methods rely on external heuristics, not intrinsic capability. Current approaches to improving research idea generation are "frequently driven by random heuristics or simple brainstorming strategies." The paper specifically cites OpenNovelty (Zhang et al., 2026), which uses information retrieval to measure how different an idea is from prior work (novelty), but notes that this is an external mechanism — it filters or scores ideas post-hoc rather than improving the model's intrinsic ideation capability. The distinction parallels the review critique: using tools to evaluate ideas versus training models to generate better ideas in the first place.

3. Existing RL paradigms cannot directly address scientific taste. The paper situates RLCF between two established RL training paradigms and argues neither suffices:

• RLVR (Reinforcement Learning with Verifiable Reward) has driven substantial progress in math and code generation by using ground-truth answers, unit tests, or formal checkers as rewards. However, scientific judgment and idea generation are "open-ended tasks" with no verifiable ground-truth — there is no objective function that outputs a binary correct/incorrect signal for "is this research direction promising."

• RLHF (Reinforcement Learning from Human Feedback) can handle open-ended subjective preferences (helpfulness, harmlessness) but has two critical limitations: (a) it relies on costly human annotations that do not scale to the volume needed for scientific judgment training, and (b) individual human annotators cannot represent community-level preferences, which are emergent properties of collective behavior over time.

RLCF occupies the gap between these paradigms: it provides scalable community-level supervision (like RLVR, but derived from naturally occurring signals rather than verifiable ground-truth) for open-ended tasks (like RLHF, but capturing aggregate preferences rather than individual judgments).

The Conceptual Framework: Taste as Preference Modeling and Alignment

The paper's key conceptual move is to formulate scientific taste learning as a preference modeling and alignment problem. This is not an obvious framing — taste might have been approached as classification (predict citation bins), regression (estimate future citation count), or generation (produce papers that score highly on some metric). The preference-based formulation has specific advantages:

Pairwise comparison is more natural and reliable than absolute scoring. The paper argues this explicitly in Section 3.3: "scoring a single scientific idea is difficult due to the lack of an objective and universal criterion. However, pairwise comparison is more natural and reliable, because it is easier to compare two ideas." This aligns with a long tradition in psychometrics (Thurstone's law of comparative judgment) and modern preference learning (Bradley-Terry, Elo, and their neural extensions): humans — and, the paper argues, models — are more consistent when making relative judgments than absolute ones.

It decomposes taste into two complementary capabilities. The formal definitions in Section 2.1 make this precise:

• Judgement capability (JudgeCap): the expected accuracy of comparing the cumulative expected impact of paper pairs (Equation 3). This is a discriminative task — given two ideas, which has higher potential impact?

• Ideation capability (ThinkerCap): the expected impact of the ideas a model proposes (Equation 4). This is a generative task — produce ideas with high potential impact.

These map naturally onto the two-stage RLHF pipeline (reward model training → policy optimization) but with community feedback replacing human labels. The judgement model (Scientific Judge) learns to predict community preferences; the ideation model (Scientific Thinker) uses that learned preference signal as a reward to generate better ideas.

It handles the divergence problem formally. A subtle technical issue: the cumulative expected impact $I(p) = \lim_{N \to \infty} \sum_{t=1}^{N} \mathbb{E}[c_t(p)]$ may diverge for highly influential papers that continue receiving citations indefinitely. If both papers in a pair have divergent impact series, how can one be said to have higher impact? Appendix G provides a formal proof that pairwise comparison is well-defined even when both individual series diverge: it depends only on the limit behavior of the difference $\sum_{t=1}^N (\mathbb{E}[c_t(p_a)] - \mathbb{E}[c_t(p_b)])$, which can converge or diverge deterministically even when the individual series do not. The sufficient condition — that one paper maintains a persistent citation advantage after some initial period — is empirically plausible and justifies the binary preference labeling.

Prior Evidence That This Could Work

The paper does not start from scratch. It builds on converging evidence from two directions:

From bibliometrics: Recent work by Zhao et al. (2025) demonstrated that LLMs can perform "newborn article impact prediction" — estimating future citation counts from title and abstract alone. This established that citation-relevant signals are extractable from the same surface features (title, abstract) that Scientific Judge uses. The NAIP-v2 work (Zhao et al., 2025) further showed that "debiased pairwise learning" improves paper quality estimation, providing direct precedent for the pairwise preference approach.

From preference learning: The rapid progress in reward modeling (WorldPM, RewardBench, RMB) demonstrated that large-scale preference prediction is feasible and that reward models trained this way can serve as effective training signals for policy optimization. The paper positions Scientific Judge within this lineage as a "generative reward model" (following Guo et al., 2025), where the model generates reasoning traces before producing a preference judgment, rather than outputting a scalar score.

How This Paper Positions Itself

The paper does not claim to fully solve scientific taste — the limitations section explicitly acknowledges that "scientific taste may involve more than scientific judgement and ideation with high potential impact," including assessing experimental feasibility and recognizing distinctive research directions. Rather, it positions itself as establishing feasibility and a training paradigm: community feedback signals can be operationalized for preference learning, the resulting models generalize across time/field/metric boundaries, and the learned preference signal can serve as a reward for improving ideation.

This is analogous to how early RLHF work (Christiano et al., 2017; Stiennon et al., 2020) established that human preferences could be learned from pairwise comparisons and used to train policies, without claiming to solve alignment. The paper's contribution is the RLCF paradigm itself — the demonstration that community-level signals provide a viable supervision channel for capabilities (judgement and ideation) that were previously considered either innate or requiring explicit human mentorship.

The paper also implicitly argues against the view that scientific taste is a "mystical human trait" ineffable to machines. The generalization results — that a model trained on pre-2024 CS papers can predict citation preferences for 2025 papers, for Math and Physics papers, and for ICLR peer-review scores — suggest that taste, operationalized as community preference prediction, is a learnable pattern rather than an irreducible human faculty. This is a substantive philosophical claim embedded in the experimental design.

3. Technical Approach

3.1 Reader Orientation

This paper builds a two-stage training pipeline where an AI model first learns to judge which research ideas have higher potential impact (by predicting which of two papers will receive more citations), and then uses that learned judgment as a reward signal to train itself to propose research ideas with higher potential impact. The core problem it solves is that scientific taste—the ability to distinguish promising research directions from trivial ones—has no objective ground-truth label, so the paper substitutes naturally occurring community citation patterns as a scalable proxy for collective scientific judgment.

3.2 Big-Picture Architecture (Diagram in Words)

The system has three major stages connected in a pipeline:

Stage 1: Community Feedback Collection. A corpus of arXiv papers with citation counts is transformed into pairwise preference data. For each pair, papers are matched by research field and publication time, and the paper with more citations is labeled as "preferred." This converts raw community behavior (citation patterns) into supervised training signals.

Stage 2: Preference Modeling (Scientific Judge). A language model is trained via reinforcement learning (GRPO) to predict which paper in a pair has higher citations. The model generates a reasoning trace and a binary choice (A or B), receiving a reward of 1 if correct and 0 otherwise. This produces Scientific Judge, which serves dual roles: (a) as an evaluator that can rank papers before they receive citations, and (b) as a reward model for Stage 3.

Stage 3: Preference Alignment (Scientific Thinker). A separate policy model is trained to generate high-impact research ideas. Given a seed paper's title and abstract, the policy proposes a follow-up research idea. Instead of scoring individual ideas directly (which has no objective criterion), the system uses Comparison-Based GRPO: the policy samples a group of candidate ideas, Scientific Judge conducts pairwise comparisons among them, and each idea's reward is its win rate within the group. The policy is then updated to favor ideas that win more comparisons.

Information flows linearly through the stages but with a critical feedback loop: Stage 2 produces a model that is used as the reward function in Stage 3, so improvements in scientific judgment directly enable improvements in scientific ideation.

3.3 Roadmap for the Deep Dive

• First, the formal definitions of scientific taste (Section 2.1, revisited technically): The paper's mathematical formulation of judgement capability (JudgeCap) and ideation capability (ThinkerCap) using cumulative expected impact. This is the objective function that the entire pipeline optimizes, so understanding it is prerequisite.
• Second, the community feedback construction (Section 3.1): How raw citation counts are transformed into SciJudgeBench—the pairwise preference dataset. This is where "community feedback" becomes computationally tractable.
• Third, the Scientific Judge training procedure (Section 3.2): The GRPO-based training with binary correctness rewards, including the exact reward function, advantage normalization, and clipped surrogate objective with KL penalty.
• Fourth, the Comparison-Based GRPO mechanism for Scientific Thinker (Section 3.3): The round-robin tournament design, win-rate reward computation, and how this adapts standard GRPO for open-ended generation tasks without ground-truth labels.
• Fifth, the key design choices and their justifications: Why pairwise comparison over absolute scoring, why GRPO over alternatives, why the specific filtering thresholds for SciJudgeBench, and why the position-swap consistency evaluation protocol.

This order is chosen because each component depends on the previous one: the formal definitions in Section 2.1 establish what it means to "learn taste"; the dataset construction in Section 3.1 provides the supervision signal; Section 3.2 trains the model that captures that signal; and Section 3.3 uses that model to train a generator.

3.4 Detailed, Sentence-Based Technical Breakdown

This is an empirical systems paper whose core idea is that scientific taste can be formalized as a preference learning problem, where community citation patterns serve as a scalable proxy for collective scientific judgment, and that this learned judgment can be used as a reward signal to improve idea generation.

Formal Definition of Scientific Taste (Section 2.1, Technical Treatment)

The paper provides a layered formal definition of scientific taste built on three concepts: potential impact, judgement capability, and ideation capability. These definitions are not merely philosophical—they are the optimization targets that the training procedures are designed to maximize.

Potential Impact

A published paper's potential impact is defined through its cumulative expected citation count over time. Let $c_t(p)$ be the number of new citations paper $p$ receives in year $t$ after publication, modeled as a non-negative random variable whose distribution depends on the paper's content and temporal context. The cumulative expected impact is:

$I(p) = \lim_{N \to \infty} \sum_{t=1}^{N} \mathbb{E}[c_t(p)]$

where $I(p)$ is the total expected impact of paper $p$, $\mathbb{E}[c_t(p)]$ is the expected number of citations in year $t$, and $N$ is the number of years since publication considered.

What it computes: the limit (as the time horizon goes to infinity) of the sum of per-year expected citation counts. In practice, the paper uses finite-horizon approximations $I_N(p) = \sum_{t=1}^N \mathbb{E}[c_t(p)]$ because infinite horizons are unavailable. The operational interpretation is: a paper with higher $I(p)$ is one that the community, in aggregate and over time, engages with more through citations.

Why this form: citations are the most common bibliometric measure of scientific impact (Wang et al., 2013; Fortunato et al., 2018), and framing impact as an expectation over a random variable acknowledges that citation counts are noisy—the same paper might receive different citation trajectories due to random factors, but its expected value represents the underlying community judgment. The cumulative sum over time captures both immediate influence (papers that spike quickly) and enduring influence (papers that accumulate steadily over decades). The limit as $N \to \infty$ is the ideal target, but since it may diverge (a paper could continue receiving citations indefinitely), the key quantity for comparison is the difference between two papers' partial sums.

Pairwise Impact Ordering

Two papers are compared by examining the limiting behavior of the difference between their finite-horizon cumulative expected impacts. For papers $p_a$ and $p_b$, the ground-truth label is:

$y(p_a, p_b) = \begin{cases} 1, & \text{if } \lim_{N \to \infty} \sum_{t=1}^{N} (\mathbb{E}[c_t(p_a)] - \mathbb{E}[c_t(p_b)]) > 0 \\ 0, & \text{otherwise} \end{cases}$

where $y(p_a, p_b)$ is the binary label indicating which paper has higher potential impact (1 means $p_a$ has higher impact than $p_b$, 0 means $p_b$ has higher or equal impact).

What it computes: a binary preference label by checking whether the limiting difference in cumulative expected citations is positive. If paper A consistently out-cites paper B in expectation over the long run, the label is 1. If paper B out-cites paper A, or they are tied, the label is 0.

Why this form: the comparison depends only on the difference between the two series, not on whether either individual series converges. Appendix G proves that even when both $I(p_a)$ and $I(p_b)$ diverge to infinity, the pairwise ordering is well-defined as long as the difference series converges in $\mathbb{R} \cup \{+\infty\}$. A sufficient condition for convergence is that one paper eventually maintains a persistent citation advantage—i.e., there exists some year $T$ after which $\mathbb{E}[c_t(p_a)] \geq \mathbb{E}[c_t(p_b)]$ for all $t > T$. This is empirically plausible: papers with fundamentally higher impact tend to sustain a citation advantage over time rather than oscillating. The binary framing (rather than a continuous score) is chosen because it directly maps to pairwise comparisons, which humans and models find more reliable than absolute impact estimation.

Judgement Capability

The judgement capability of a model $\theta$ is its expected accuracy on predicting pairwise impact orderings:

$\text{JudgeCap}(\theta) = \mathbb{E}_{(p_a, p_b) \sim \mathcal{D}} \left[ \mathbf{1}\left[ \text{Judge}_\theta(p_a, p_b) = y(p_a, p_b) \right] \right]$

where $\mathcal{D}$ is a distribution over field- and time-matched paper pairs, $\text{Judge}_\theta(p_a, p_b) \in \{0, 1\}$ is the model's binary prediction of which paper has higher impact, and $y(p_a, p_b)$ is the ground-truth label from the pairwise ordering definition.

What it computes: the fraction of paper pairs for which the model correctly predicts the higher-citation paper. This is simply classification accuracy over a specific distribution of test pairs.

Why this form: accuracy is the natural metric for binary preference prediction—it is interpretable (a random model achieves 50%, a perfect model achieves 100%) and directly corresponds to the training objective. The choice of $\mathcal{D}$ as field- and time-matched pairs is critical: by conditioning on same-field and same-time comparisons, the metric measures whether the model can distinguish relative impact within a coherent context, rather than detecting trivial signals like "CS papers get more citations than Math papers" or "older papers have more citations."

Ideation Capability

The ideation capability of a model $\phi$ is the expected impact of the research ideas it proposes:

$\text{ThinkerCap}(\phi) = \mathbb{E}_{s \sim \mathcal{S}} \left[ I(\text{Thinker}_\phi(s)) \right]$

where $\mathcal{S}$ is a distribution over seed reference papers, $\text{Thinker}_\phi(s)$ is the research idea (paper) generated by model $\phi$ given seed paper $s$, and $I(\cdot)$ is the cumulative expected impact of that generated paper.

What it computes: the average long-term citation impact of ideas proposed by the model. A model with higher ThinkerCap proposes ideas that, on average, would accumulate more citations if published.

Why this form: this definition ties ideation quality directly to community reception—an idea is "good" to the extent that the community engages with and builds upon it. This avoids the circularity of defining "good ideas" by expert opinion (which would require costly annotation) and instead grounds evaluation in the same citation signal used for judgement training. The expectation over $\mathcal{S}$ ensures the model is evaluated on its ability to propose impactful ideas across diverse starting points, not just in a narrow domain.

Scientific Taste

Scientific taste is the conjunction: a model possesses strong scientific taste if and only if it achieves both high JudgeCap and high ThinkerCap. This formal decomposition is what enables the two-stage training pipeline: first train for JudgeCap (Scientific Judge), then use the trained judge to train for ThinkerCap (Scientific Thinker).

Community Feedback as Supervision: SciJudgeBench Construction (Section 3.1 and Appendix A)

The raw material for all training is a corpus of arXiv papers with citation counts. The construction pipeline transforms this raw data into pairwise preference examples through a sequence of filtering and matching steps designed to isolate the signal of relative impact from confounds of field, time, and noise.

Paper collection. The authors collect arXiv papers published through December 7, 2025. From the full arXiv metadata archive of 2.9 million papers, they obtain citation counts for 2.3 million papers using an unspecified citation database. They select 2.1 million papers published through 2024 as the training paper pool. Each paper record includes title, abstract, publication date, arXiv subcategory, and citation count.

Field-to-subcategory mapping. Papers are grouped by their primary arXiv category into four top-level fields:

• Computer Science: all cs.* subcategories (e.g., cs.CV, cs.LG, cs.CL, cs.RO, cs.AI, and 37 others)
• Mathematics: all math.* subcategories (e.g., math.AP, math.OC, math.CO, and 26 others)
• Physics: all physics-related subcategories including quant-ph, cond-mat., hep-ph, astro-ph., and 37 others
• Others: an explicit aggregation of Economics (econ.), Electrical Engineering and Systems Science (eess.), Quantitative Biology (q-bio.), Quantitative Finance (q-fin.), and Statistics (stat.*). This is deliberately not a residual bucket—the paper states it explicitly covers these areas.

This mapping matters because pair matching occurs within subcategories, not across them. Papers from cs.CL are only compared to other cs.CL papers (not to cs.CV or math.AP), ensuring that differences in citation norms across fields do not contaminate the preference signal.

Citation-based preference labeling. For each matched pair within the same subcategory and similar publication time, the paper with higher citations is labeled as the preferred item. Let $c_{\text{hi}}$ and $c_{\text{lo}}$ denote the higher and lower citation counts in a candidate pair. A pair is retained only if it satisfies two criteria:

1. Absolute citation difference: $c_{\text{hi}} - c_{\text{lo}} \geq 8$
2. Relative citation difference: $(c_{\text{hi}} - c_{\text{lo}}) / c_{\text{hi}} \geq 0.3$

The relative difference is computed with respect to the higher-citation paper, meaning the lower-cited paper must have at most 70% of the higher-cited paper's citations. These thresholds are chosen to filter out pairs where the preference signal is ambiguous—papers with nearly identical citation counts are removed because it is unclear which one the community truly preferred.

Resulting dataset. This filtering yields 696,758 field- and time-matched training pairs spanning approximately 1.4 million unique papers across fields. Table 9 in Appendix A provides the breakdown: Computer Science contributes 161,461 pairs (from 322,922 papers), Mathematics 102,592 pairs (from 205,184 papers), Physics 394,336 pairs (from 788,672 papers), and Others 38,369 pairs (from 76,738 papers). Note that papers can appear in multiple pairs (once as the higher-cited paper and once or more as the lower-cited paper, depending on matching), which is why the number of unique papers is less than twice the number of pairs.

Test set construction. The main in-domain test set applies stricter filtering to ensure unambiguous preferences for evaluation:

• Absolute citation difference ≥ 32
• Relative citation difference ≥ 50%, i.e., $(c_{\text{hi}} - c_{\text{lo}}) / c_{\text{hi}} \geq 0.5$

This yields 728 test pairs stratified across CS, Physics, Math, and Others. The stricter thresholds mean the test pairs have clearer ground-truth preferences, making evaluation more reliable.

Temporal OOD test set. To test generalization to future papers, the authors construct 514 pairs from papers published in 2025, ensuring complete temporal separation from the training data (all published through 2024). This presents a technical challenge: recently published papers have far fewer total citations than older papers, so the absolute thresholds used for in-domain pairs would filter out nearly all 2025 pairs. The solution is adaptive, subcategory-specific thresholds:

For each primary subcategory $s$, let $q_s^{(p)}$ denote the $p$-th citation percentile among papers with at least one citation. Define a subcategory-adaptive minimum difference:

$\delta_s = \max \left( 12, \; 0.7 \left( q_s^{(99)} - q_s^{(1)} \right) \right)$

A candidate pair $(i, j)$ in the same subcategory is retained only if:

• $c_i \geq q_s^{(75)}$ (paper i is in the top 25% of citations in its subcategory)
• $c_j \leq q_s^{(25)}$ (paper j is in the bottom 25%)
• $|t_i - t_j| \leq 5$ days (publication dates are within 5 days)
• $c_i - c_j \geq \max(0.5 c_i, \delta_s)$ (difference satisfies both relative and adaptive absolute margins)

The $\delta_s$ term is key: for subcategories with wide citation spreads (where $q_s^{(99)} - q_s^{(1)}$ is large), it enforces a higher absolute margin; for tight subcategories, it falls back to a floor of 12 citations. This adaptivity ensures the preference signal is meaningful regardless of subcategory-specific citation norms. The resulting category distribution (CS: 318 pairs, Physics: 161, Others: 26, Math: 9) reflects the natural availability of sufficiently cited recent papers in each field—Math has very few because 2025 math papers have had limited time to accumulate distinguishing citation counts.

Metric OOD (ICLR) test set. To test whether citation-trained judgment transfers to peer-review preferences, the authors construct 611 pairs from ICLR submissions (2017–2026) where the ground-truth label is determined by peer review scores rather than citations. The construction procedure:

1. Apply two quality filters per year: remove papers whose average reviewer confidence is in the bottom 50% and papers whose rating variance is in the top 50%. This removes noisy or unreliable reviews.
2. Retain only the top and bottom 10% of papers by average review rating within each year, capped at 75 papers per side.
3. Sort retained papers by rating, split at the median into an upper half and a lower half.
4. Randomly shuffle within each half and pair papers one-to-one across halves, with the paper from the upper half labeled as preferred.

This yields pairs with a median rating difference of 6.1 points on the 1–10 review scale. Critically, the evaluation prompt asks "which paper is more likely to be accepted," differing from the training prompt ("which paper has a higher citation count"), so this tests both a different supervision signal and a different task framing.

Biology Field OOD (bioRxiv) test set. An additional cross-field test uses 160 pairs from bioRxiv papers (biology preprints), a platform and field entirely absent from the arXiv training data. Pairs are constructed using the same citation-based procedure with thresholds of absolute citation difference ≥ 24 and relative difference ≥ 75%, with a median citation difference of 134. Papers are paired within the same biology subdiscipline.

Field OOD training data. For field OOD experiments, the authors filter the full training data to include only Computer Science papers, yielding a CS-only training set. Models trained on this set are evaluated on Math, Physics, and Others test pairs to measure cross-field generalization.

Preference Modeling: Scientific Judge Training (Section 3.2 and Appendix B)

Scientific Judge is a generative reward model trained via GRPO (Group Relative Policy Optimization) to predict which paper in a pair has higher potential impact. The training procedure is designed to produce a model that not only predicts preferences but does so through explicit reasoning, making its judgments more interpretable and potentially more robust.

Model architecture and base models. Scientific Judge is not a new architecture—it is a fine-tuned instruction-tuned language model. The authors train on eight base models spanning three model families and multiple scales:

• Qwen2.5-Instruct series: 1.5B, 3B, 7B, 14B, and 32B parameters
• Qwen3-Instruct series: Qwen3-4B-Instruct-2507 and Qwen3-30B-A3B-Instruct-2507
• Llama-3.1-8B-Instruct

Each trained variant is named SciJudge-{base}, e.g., SciJudge-Qwen3-4B for the Qwen3-4B base. The multi-scale design enables analysis of how scientific judgment capability scales with model size.

Prompt template. The model receives a prompt containing the titles, abstracts, and publication dates of two papers, along with a system instruction to reason before answering. The user message template is:

Today is 2025-12-10. Based on the titles, abstracts, and publication dates of the following two papers A and B, determine which paper has a higher citation count.
Show your reasoning process in  thinking  response tags. And return the final answer in <answer> </answer> tags. The final answer should contain only the letter A or B.
Paper A (Published: [Publication Date A]): [Title and Abstract of Paper A]
Paper B (Published: [Publication Date B]): [Title and Abstract of Paper B]

The date "2025-12-10" provides temporal context for the model to reason about the time since publication. The separation of reasoning (in thinking tags) from the final answer (in tags) structures the generation for reward computation.

GRPO training objective. For each input $x$ (a paper pair with the prompt), the policy $\pi_\theta$ samples a group of $G = 8$ outputs $\{o_i\}_{i=1}^G$, each consisting of a reasoning trace and a preference prediction. The reward is a binary correctness signal:

$r_i = \begin{cases} 1, & \text{if } \hat{y}(o_i) = y \\ 0, & \text{otherwise} \end{cases}$

where $r_i$ is the scalar reward for output $o_i$, $\hat{y}(o_i)$ extracts the predicted preference (A or B) from the output (parsed from the <answer> tags), and $y$ is the ground-truth label from the citation-based pairing.

What it computes: for each of the 8 sampled responses to the same prompt, the system checks whether the predicted preference matches the ground truth. Correct predictions get reward 1; incorrect predictions get 0. This is a verifiable reward similar to RLVR settings for math and coding—the ground truth exists (citation counts) and correctness is binary.

Why this form: binary correctness is the simplest reward that directly optimizes judgement accuracy. Alternatives like a continuous reward based on confidence calibration would require additional assumptions about the relationship between model confidence and actual correctness. The binary reward has the advantage that it is non-gameable—the only way to increase expected reward is to make more correct predictions. The group size $G = 8$ is chosen to provide enough samples for stable advantage estimation while remaining computationally tractable.

Advantage normalization and policy update. Within each group of 8 outputs, advantages are computed by normalizing the rewards:

$\hat{A}_i = \frac{r_i - \text{mean}(\mathbf{r})}{\text{std}(\mathbf{r})}$

where $\hat{A}_i$ is the normalized advantage for output $i$, $\text{mean}(\mathbf{r})$ is the average reward across the 8 outputs, and $\text{std}(\mathbf{r})$ is the standard deviation. This ensures that advantages are zero-mean and unit-variance within each group, removing the effect of varying prompt difficulty (some pairs are harder than others, leading to lower average reward across the group).

The policy is updated by maximizing a clipped surrogate objective with a KL penalty toward a reference policy $\pi_{\text{ref}}$ (which is the initial base model, frozen during training):

$\mathcal{J}(\theta) = \mathbb{E}_x \left[ \frac{1}{G} \sum_{i=1}^G \min \left( \rho_i \hat{A}_i, \; \text{clip}(\rho_i, 1 - \epsilon, 1 + \epsilon) \hat{A}_i \right) - \beta \, D_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}}) \right]$

where:

• $\rho_i = \pi_\theta(o_i | x) / \pi_{\text{old}}(o_i | x)$ is the importance ratio—how much more (or less) likely the current policy is to produce output $o_i$ compared to the policy from the previous update
• $\epsilon = 0.20$ is the clipping range for the standard PPO-style clipping, with $\epsilon_{\text{high}} = 0.25$ used for the upper clip
• $\beta = 0.03$ controls the strength of the KL penalty, preventing the policy from diverging too far from the reference model
• $D_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}})$ is the Kullback-Leibler divergence between the current policy's output distribution and the reference policy's output distribution

What it computes: the objective incentivizes the policy to increase the probability of outputs that received above-average reward (positive advantage) and decrease the probability of below-average outputs (negative advantage), while the clipping prevents destructively large updates and the KL penalty keeps the policy near its initial behavior to preserve general capabilities.

Why this form: the clipped surrogate is the standard PPO objective, chosen because it is stable and well-tested. The min operation between the clipped and unclipped objectives ensures that the policy only benefits from an advantage when the update is within the clipping range, preventing policy collapse from a single large update. The KL penalty coefficient $\beta = 0.03$ is a relatively light regularization—large enough to prevent catastrophic forgetting of the base model's general capabilities (verified in Appendix D, Table 13), small enough to allow meaningful learning of citation prediction. The per-group advantage normalization (rather than global normalization) is critical because it operates at the granularity of individual prompts: a paper pair that is genuinely ambiguous (both papers are high-quality and citation differences are small) will have low-variance rewards across the 8 samples, and the advantage normalization will correctly avoid forcing the policy toward a random choice.

Training hyperparameters. The full configuration, implemented using the MS-SWIFT framework:

• Algorithm: GRPO
• Learning rate: $8 \times 10^{-7}$
• LR scheduler: cosine with 0.05 warmup ratio
• Effective batch size: 128
• Number of epochs: 1 (single pass over the 696,758 training pairs)
• Max sequence length: 2048 tokens
• Generation parameters: temperature 1.0, top-p 0.85, max completion length 2048
• Number of GPUs: 32 (for 1.5B models), 64 (for 3B–7B), 128 (for 14B and larger), using H200-equivalent resources with DeepSpeed ZeRO-2/ZeRO-3 and vLLM for efficient inference

The learning rate of $8 \times 10^{-7}$ is notably low, reflecting the sensitivity of fine-tuning large language models—large updates would rapidly degrade the base model's general capabilities. The single-epoch training means the model sees each of the 696,758 training pairs exactly once, preventing overfitting to the training distribution while still providing enough data for the log-linear improvement observed in Figure 3.

Evaluation protocol with position-swap consistency. A known issue in pairwise LLM evaluation is position bias—models often favor whichever option is presented first (A), regardless of content. To mitigate this, the paper evaluates each test pair twice: once with paper order (A, B) as in the dataset, and once with the order swapped (B, A). A prediction is scored as correct (score 1) only if the model makes consistent and correct predictions in both orderings.

This means that if a model predicts "A" in the original ordering and "B" in the swapped ordering (both referring to the same paper that appears as A in the first case and B in the second), it is correct. If it predicts "A" in both orderings regardless of content (position bias), it will be correct in one ordering but incorrect in the other, resulting in a score of 0 for that pair.

What this does: it ensures that reported accuracy reflects genuine preference understanding rather than positional shortcuts. A model that always predicts "A" regardless of content would achieve approximately 50% accuracy under normal evaluation (since half the pairs have A as the correct answer), but 0% under position-swap consistency (since it would fail half the swapped evaluations).

General capability preservation (Appendix D, Table 13). A critical concern is whether specialized training on citation prediction degrades the base model's general knowledge and reasoning. The authors evaluate on five standard benchmarks: MMLU-Pro (general knowledge), GPQA (graduate-level science), MATH (mathematical reasoning), GSM8K (grade school math), and SimpleQA (factual accuracy). Changes are typically within ±3% of baseline, with some models showing minor improvements (e.g., SciJudge-Qwen2.5-3B gains +0.8% on MATH and +3.1% on GPQA). This confirms that targeted preference training with KL regularization preserves general capabilities.

Preference Alignment: Scientific Thinker Training via Comparison-Based GRPO (Section 3.3 and Appendix C)

Scientific Thinker is a policy model trained to generate research ideas with high potential impact, using Scientific Judge as a generative reward model. The central challenge is that scientific ideation is an open-ended generation task with no ground-truth labels—there is no objective function that takes a proposed research idea and outputs a scalar "quality" score. The paper's solution is Comparison-Based GRPO, which replaces absolute scoring with round-robin pairwise comparisons within groups of generated ideas.

Why absolute scoring fails for ideation. The paper explicitly argues that "scoring a single scientific idea is difficult due to the lack of an objective and universal criterion." Unlike math problems (where a solution is either correct or incorrect) or code generation (where unit tests provide binary feedback), a research idea's quality is inherently relative and contextual. Two ideas might both be high-quality, or both be low-quality, but there is no natural zero point or unit of measurement for "potential impact." Pairwise comparison avoids this by asking only: among these two ideas, which is more promising? This is the same insight that motivates Elo ratings in chess and Bradley-Terry models in preference learning—relative judgments are more reliable than absolute ones for inherently comparative domains.

Training data: seed papers. The training set consists of 4,000 high-citation papers published between January and July 2025. For each paper, the model receives the title and abstract as a prompt and is asked to propose a follow-up research idea. The in-domain test set uses 200 papers from the same period (January–July 2025), while the out-of-domain test set uses 200 papers from August–December 2025—papers published after the training period, testing temporal generalization of ideation capability.

Prompt template for ideation. The prompt asks the model to engage in "heuristic thinking" and propose a follow-up idea:

You are a knowledgeable and insightful researcher. You have come across a new research paper with the following title and abstract:
[Title and Abstract of the Seed Paper]
Based on the core ideas, methods, or findings of this work, engage in heuristic thinking and propose a follow-up research idea. You need not confine yourself to the specific scenario or task of the original paper. You may consider shortcomings of the original method, propose improvements, apply its ideas to other tasks or domains, or even introduce entirely new problems and approaches. Aim to formulate an idea with high academic value and potential impact.
In your response, solely present your proposed title and abstract. Think independently and there is no need to imitate the format of the provided paper's title and abstract, nor to intentionally cite it. You must ensure the abstract is of a moderate length, avoiding excessive length, as if you were writing it for a typical academic paper.
Output format (strict, no extra text):
Title: <your proposed paper title>
Abstract: <your proposed abstract>

The output format constraint (only title and abstract) ensures that evaluation focuses on the idea's content rather than on presentation style or the model's ability to format a full paper.

Comparison-Based GRPO mechanism. For each prompt $x$ containing a seed paper, the policy $\pi_\theta$ (initialized from a base model) samples a group of $G$ responses $\{o_1, \ldots, o_G\}$, each providing a candidate research idea (title and abstract). The paper uses $G = 8$ responses per prompt.

Rather than scoring each idea individually, the system conducts a round-robin tournament judged by Scientific Judge. Each candidate idea is compared with every other candidate in the group, producing a total of $\binom{G}{2} = 28$ pairwise comparisons (for $G = 8$). The reward for idea $o_i$ is its win rate within the group:

$r_i = \frac{1}{G - 1} \sum_{j \neq i} s(o_i, o_j)$

where $r_i \in [0, 1]$ is the scalar reward for output $o_i$, $G - 1$ is the number of opponents each idea faces (7 for $G = 8$), and $s(o_i, o_j) \in \{0, 1\}$ is Scientific Judge's binary judgment: 1 if idea $o_i$ wins against idea $o_j$, 0 otherwise.

What it computes: for each of the 8 generated ideas, count how many of the other 7 ideas it beats in pairwise comparisons judged by Scientific Judge, then divide by 7. An idea that wins all 7 comparisons gets reward 1.0; an idea that loses all comparisons gets reward 0.0; a middling idea that wins 3 out of 7 gets reward ≈0.43.

Why this form: the win rate is a natural measure of relative quality within a group—it captures how an idea fares against its peers under a consistent evaluation protocol. The denominator $G - 1$ normalizes the reward to $[0, 1]$ regardless of group size, making it compatible with the GRPO advantage normalization. Using binary win/loss judgments (rather than continuous scores from Scientific Judge) preserves the pairwise comparison structure: Scientific Judge is trained to make binary preference predictions, and forcing it to output continuous scores would require retraining or prompt engineering that might reduce reliability. The round-robin design ensures that every pair is evaluated directly, avoiding the path-dependencies of tournament formats like single-elimination.

Scientific Judge evaluation prompt for comparison. When used as a reward model, Scientific Judge receives a modified prompt designed for comparing two generated ideas:

Based on the titles and abstracts of the following two papers A and B, determine which paper has a higher citation count. Suppose the two papers are published at the same time.
Show your reasoning process in  thinking  response tags. And return the final answer in <answer> </answer> tags. The final answer should contain only the letter A or B.
Paper A: [Title and Abstract of Research Idea A]
Paper B: [Title and Abstract of Research Idea B]

The key difference from the Scientific Judge training prompt is the removal of publication dates and the explicit instruction to assume simultaneous publication. This is necessary because all generated ideas are hypothetical (not yet published), so there is no real publication date to condition on. The "higher citation count" framing is preserved from training, maintaining the distributional match between how Scientific Judge was trained and how it is used as a reward model.

Training objective. Given the win-rate rewards, the training objective is the same GRPO formulation as in Equation 6, with the same clipping, advantage normalization, and KL penalty structure. The hyperparameters for Scientific Thinker training differ slightly from Scientific Judge training:

• Algorithm: GRPO
• Learning rate: $5 \times 10^{-7}$ (lower than Judge's $8 \times 10^{-7}$, reflecting the more complex generation task)
• LR scheduler: constant (rather than cosine, providing uniform step sizes throughout training)
• Warmup ratio: 0.1
• Effective batch size: 128
• Number of epochs: 1
• Max sequence length: 2048 (same as Judge)
• KL penalty coefficient: $\beta = 0.001$ (much lower than Judge's 0.03—since ideation is a more open-ended generative task, stronger KL regularization might prevent the model from deviating enough to produce novel ideas)
• Generation parameters: temperature 1.0, top-p 0.9, max completion length 8192 (four times Judge's completion length, since full research ideas with abstracts require substantially more tokens than a pairwise preference judgment with reasoning)

Reward model variants and ablation. To isolate the contribution of Scientific Judge's learned judgment, the authors train additional policies using the base model of Scientific Judge (Qwen3-4B-Instruct, without the citation-prediction fine-tuning) as the reward model. This provides an ablation: does the policy improve because of RL training in general, or specifically because Scientific Judge provides a high-quality preference signal? Figure 4 shows that policies trained with SciJudge-Qwen3-4B significantly outperform those trained with the untrained base model (e.g., 30B: 81.5% vs. 73.0% win rate in-domain), confirming that the learned scientific judgment is responsible for the improvement.

Base policy models for Scientific Thinker. The authors train on two thinking-optimized base models: Qwen3-30B-A3B-Thinking-2507 and Qwen3-4B-Thinking-2507. The resulting trained policies are named SciThinker-30B and SciThinker-4B. These "thinking" variants of Qwen3 are specifically designed for extended reasoning before generation, which aligns with the ideation task structure (thinking about the seed paper's contributions, then proposing a follow-up idea).

Evaluation protocol for Scientific Thinker. Evaluating the quality of generated research ideas is challenging because there is no ground truth. The paper uses a panel of three strong LLMs as evaluators (GPT-5.2-high, GLM-5, and Gemini 3 Pro), with majority voting to determine which of two ideas is better. For each seed paper, both the trained policy and the base policy propose a research idea, and the three evaluator models independently judge which has higher potential impact (using the same comparison prompt as Scientific Judge, with the "published at the same time" framing). The idea that receives at least 2 out of 3 votes is the winner. To mitigate position bias, the order of ideas (A and B) is randomly swapped with 50% probability before each evaluator's judgment.

The authors validate this evaluation protocol by testing the majority-voting method on SciJudgeBench (the citation prediction test set) and finding it achieves 84.4% accuracy. This high accuracy on a task with known ground truth provides evidence that the three-model majority voting is a reasonable evaluation metric for idea quality, though the paper acknowledges in limitations that evaluation "may not fully reflect potential impact" since ideas are not experimentally validated.

Summary of Key Design Choices and Their Justifications

Pairwise preference over absolute scoring: pairwise comparison avoids the need for an absolute quality metric for research ideas. The paper argues that "it is easier to compare two ideas" than to score one in isolation, and this is consistent with psychometric and preference learning traditions. The entire pipeline—from SciJudgeBench construction (pairing papers by citations) to Scientific Judge training (predicting pairwise preferences) to Scientific Thinker training (win-rate rewards from pairwise comparisons)—is built on this insight.

Citations as supervision over human labels: human annotation of research idea quality at the scale of 696K pairs would be prohibitively expensive and would reflect individual annotator preferences rather than community consensus. Citations are naturally occurring, scalable, and represent aggregate community behavior over time—they operationalize Hume's "joint verdict of qualified judges."

GRPO over SFT or PPO: the paper uses GRPO rather than supervised fine-tuning because the task is inherently preference-based (there is no single "correct" judgment or idea) and reinforcement learning can directly optimize the metric of interest (prediction accuracy for Judge, win rate for Thinker). GRPO is preferred over vanilla PPO because it eliminates the need for a separate value function (the advantage normalization uses only the rewards within each group), simplifying the training pipeline. The group-based advantage computation also provides a natural way to handle varying prompt difficulty.

Field- and time-matching over unrestricted pairs: raw citation counts are heavily confounded by field (some fields cite more than others) and time (older papers have more citations). Matching pairs within the same subcategory and similar publication times controls for these confounds, so the preference label reflects relative impact within a coherent context rather than trivial differences in citation norms.

Filtering thresholds for clear signals: the training data uses absolute difference ≥ 8 and relative difference ≥ 30%, while the test data uses stricter thresholds (≥ 32 absolute, ≥ 50% relative). The looser training thresholds provide more data for learning; the stricter test thresholds ensure that evaluation pairs have unambiguous ground-truth preferences, making accuracy a meaningful metric.

Generative reward modeling over scalar reward modeling: Scientific Judge generates reasoning traces before making a preference prediction, rather than outputting a single scalar score. This makes its judgments more interpretable (the reasoning can be inspected, as in Appendix F) and potentially more robust (the model must articulate its reasoning before deciding, which may reduce superficial pattern-matching). This follows the "generative reward model" paradigm from Guo et al. (2025) and subsequent work.

Position-swap consistency for evaluation: evaluating each pair twice with swapped order and requiring consistent correctness eliminates position bias as a confound in accuracy measurements. This doubles evaluation cost but provides a substantially more reliable metric of genuine preference understanding.

KL penalty preservation of general capabilities: the KL penalty coefficient in both Judge and Thinker training prevents the policy from diverging too far from the base model's distribution, preserving general knowledge and reasoning abilities (verified in Appendix D). This is particularly important for Scientific Thinker, where the base model's general intelligence (language understanding, reasoning, world knowledge) is needed to generate coherent and novel research ideas—divergence would not only hurt general benchmarks but likely degrade idea quality as well.

4. Key Insights and Innovations

Innovation 1: Scientific Taste as a Learnable, Decomposable Objective from Community Behavior

The paper's most fundamental contribution is the reframing of scientific taste—a capability long considered either ineffable, purely innate, or requiring decades of mentorship to acquire—as a formal preference modeling and alignment problem solvable from naturally occurring community behavior signals. This is not merely applying ML to a new domain; it is a conceptual move that transforms taste from a philosophical question into an engineering target.

Prior to this work, scientific taste was treated as something AI scientists lack but that cannot be directly trained—Si et al. (2024) documented that LLMs "struggle to reliably distinguish potentially high-impact directions from ideas that are superficially novel but trivial," and subsequent work treated this as a fixed capability gap rather than a training problem. The dominant approaches to improving AI scientists focused on execution capabilities (literature search, experiment automation, code generation) while implicitly assuming that judgment—knowing what to work on—was either beyond current training paradigms or would emerge naturally from scale.

This paper challenges that assumption at its root by operationalizing taste through two specific capabilities (JudgeCap and ThinkerCap, Equations 3 and 4) linked by a common objective: maximizing alignment with community citation preferences. The move from "taste is subjective" to "taste is a community preference signal encoded in citation patterns" draws on philosophical groundwork from Hume and Kant but translates it into a computationally tractable objective. Hume argued that a "standard of taste" emerges from the "joint verdict of qualified judges"—the paper argues that the scientific community's citation behavior is that joint verdict, expressed at scale over time. Kant's sensus communis—a shared sense that considers how others would judge—is operationalized as the pairwise preference prediction task: the model learns not what one reviewer thinks, but what the community, in aggregate, rewards with attention.

What distinguishes this reframing from prior work in scientometrics or bibliometrics is that citations are not used as an evaluation metric for already-generated ideas, but as a training signal for building the judgment capability itself. Prior work like NAIP (Zhao et al., 2025) used LLMs to predict citation counts as an end task; this paper uses citation patterns to train a model whose judgment can then generalize to unseen papers, future papers, and even peer review evaluations—three settings where raw citation data is unavailable. This is the difference between building a citation predictor and building something closer to a scientific sensibility.

The decomposition into JudgeCap and ThinkerCap is also more than taxonomy. It enables a two-stage training pipeline where improvements in judgment directly enable improvements in ideation—a virtuous cycle. The conceptual parallel to RLHF is explicit but the paper's version replaces the human bottleneck with a scalable, naturally occurring community signal, making it a genuinely new training paradigm (RLCF) rather than a domain adaptation of existing methods.

Evidence: The generalization results (Tables 4, 5, 6) are the strongest argument that something beyond surface-level citation pattern matching has been learned. A model trained on pre-2024 CS papers predicting future-year preferences (temporal OOD, +6.2 points for SciJudge-Qwen3-4B, Table 4), cross-field preferences (field OOD, +7.4 points average, Table 5), and peer-review preferences at ICLR (metric OOD, +13.8 points, Table 6) is extracting transferable patterns of scientific value, not merely memorizing which CS subfields tend to cite heavily. The biology OOD result on bioRxiv (Table 7, +26.2 points for the 30B model) is particularly striking because biology is not just a different field—it comes from an entirely different preprint platform with its own citation dynamics, yet the judgment transfers.

Innovation 2: Community Feedback as a Third RL Paradigm Between RLVR and RLHF

The paper introduces Reinforcement Learning from Community Feedback (RLCF) as a distinct training paradigm that occupies a previously unoccupied region in the space of possible supervision signals. This is not an incremental combination of existing approaches but a genuinely new category that addresses limitations neither RLVR nor RLHF can solve individually.

The standard RL landscape for language model training is dominated by two poles: RLVR (Reinforcement Learning with Verifiable Rewards), which works brilliantly for math and code where ground-truth answers exist (DeepSeek-R1, DeepSeekMath), and RLHF (Reinforcement Learning from Human Feedback), which handles subjective preferences (helpfulness, harmlessness) but is bottlenecked by annotation cost and limited to individual annotator preferences. The paper identifies a gap: what about tasks that are open-ended (no verifiable ground-truth exists—there is no unit test for "is this research idea promising?") but where individual human judgment is insufficient because the relevant standard is community-level consensus?

RLCF fills this gap by using naturally occurring community behavior as supervision. The key insight is that community interactions generate preference signals at scale without explicit annotation: citation counts are a byproduct of normal scientific activity, not a labeling effort. This gives RLCF the scalability of RLVR (the signal exists for millions of examples at near-zero marginal cost) while maintaining applicability to open-ended tasks where RLVR fails. Simultaneously, it captures aggregate preferences that individual human annotators cannot provide—a single reviewer's opinion is not a substitute for the community's collective judgment over years, but citation counts approximate exactly that.

What makes this intellectually distinctive rather than a trivial renaming is the type shift in what the preference signal represents. In RLHF, the preference is a direct expression of an individual's values ("I prefer response A over B"). In RLCF, the preference is an emergent property of collective behavior—no individual scientist decides "this paper deserves more citations," yet the aggregate reveals community-level judgments about impact and usefulness. The paper is essentially arguing that scientific taste is a distributed cognition phenomenon encoded in the citation graph, and that this distributed signal can be extracted and internalized by individual models.

The relationship between RLCF and prior work is not adversarial—the paper positions RLCF as complementary to both RLVR and RLHF rather than as a replacement. Math verification still requires RLVR; helpfulness alignment still requires RLHF. But for capabilities that depend on community norms—scientific judgment, legal reasoning, clinical trial evaluation, policy analysis—RLCF provides a supervision channel that neither existing paradigm offers.

Evidence: The comparison of reward models in Figure 4 provides the critical causal evidence. Scientific Thinker trained with SciJudge-Qwen3-4B (the RLCF-trained reward model) significantly outperforms the same policy trained with Qwen3-4B-Instruct (the untrained base model) as the reward: 81.5% vs. 73.0% win rate in-domain for the 30B model. This isolates the contribution of the learned community preference signal: it is not RL training in general that improves ideation, but specifically the signal extracted from citation patterns via RLCF. The gap generalizes to out-of-domain test (83.0% vs. 70.5%), suggesting the reward model captures more than superficial training distribution artifacts.

Innovation 3: Empirical Evidence That Citation-Derived Judgment Generalizes Across Time, Fields, and Metrics—And That This Indicates a Learned "Taste" Rather Than Pattern Matching

The paper's generalization results collectively make a substantive argument that the learned judgment captures something abstracted from the specific training signal rather than merely fitting surface correlations. This is not just strong performance—it is a diagnostic finding that distinguishes between two competing hypotheses about what Scientific Judge has learned.

The null hypothesis is that the model learns to identify citation-relevant surface features specific to CS papers published through 2024: certain methodological keywords correlate with high citations, certain subfields are "hotter," certain paper structures are more citeable. Under this hypothesis, the model is a sophisticated pattern matcher tied to its training distribution—impressive but fundamentally limited, like a model that learns to predict movie ratings from review text but cannot assess restaurant quality.

The alternative hypothesis is that the model learns something more abstract—patterns of scientific value that manifest similarly across fields, persist into the future, and align with expert human judgment in different evaluative contexts (peer review). Under this hypothesis, the model has internalized something approximating an aesthetic of scientific value rather than memorizing which CS terms predict citations.

The paper's generalization design is specifically constructed to discriminate between these hypotheses:

• Temporal generalization (Table 4): training on pre-2024, testing on 2025 papers. If the model only learned which 2019–2024 trends received citations, it would fail on 2025 papers that reflect new trends, new terminology, and new research directions that did not exist during training. The consistent improvement (e.g., SciJudge-Qwen3-30B improving from 71.6 to 78.2 average) argues against this.

• Field generalization (Table 5): training only on CS papers, testing on Math, Physics, and Others. If the model learned CS-specific citation patterns (e.g., "transformer" papers get cited, "attention mechanism" papers get cited), it would fail on Math (where "Yoneda lemma" and "derived category" signal value) and Physics (where "variational quantum eigensolver" and "jet substructure" signal value). The cross-field transfer suggests the model extracts more abstract signals—generality of contribution, breadth of potential application, novelty relative to existing approaches—that manifest in field-specific language but share underlying structure.

• Metric generalization (Table 6): training on citations, testing on ICLR peer-review scores. This is arguably the strongest test, because peer review and citation patterns are known to be only moderately correlated (reviewers value technical correctness and novelty; citers value usefulness and applicability). The model's substantial improvement (e.g., SciJudge-Qwen3-30B: 76.8% → 87.7%, +11.0 points) suggests that community citation behavior and expert peer review tap into overlapping dimensions of scientific quality, and that the model has learned these shared dimensions rather than citation-specific heuristics.

The qualitative case studies in Appendix F provide interpretive evidence for the abstraction hypothesis. SciJudge-30B's reasoning on the quantum computing pair (Appendix F.3) invokes downstream adoption ("foundational paper in the emerging field... concepts have been widely adopted and cited in both theoretical and experimental quantum computing research"), breadth of applicability, and field trajectory—concepts that are not reducible to keyword matching. Its reasoning on the ICLR pair (Appendix F.2) maps citation-derived quality judgments onto acceptance criteria: theoretical contribution, novelty, empirical validation, alignment with venue scope—an inferential leap that requires understanding evaluation criteria beyond raw citation counts.

Evidence: The three generalization tables collectively provide the quantitative backbone. The bioRxiv result (Table 7, not in the main tables but in Appendix A.5) adds an additional dimension: cross-platform generalization. The 30B model achieves 71.2% on biology preprints from a different platform, compared to its base model's 45.0%. This is a harder test than cross-field arXiv generalization because the data distribution (preprint server norms, author demographics, disciplinary conventions) differs along multiple axes simultaneously.

Innovation 4: Comparison-Based GRPO as a General Mechanism for Training Generators on Open-Ended Tasks Without Absolute Quality Metrics

The Comparison-Based GRPO algorithm introduced in Section 3.3 is a technical contribution with implications beyond scientific ideation. It addresses a fundamental problem in training generative models for open-ended tasks: how do you provide a reward signal when there is no ground-truth answer and absolute quality scoring is unreliable?

The standard approaches fail in different ways for scientific ideation. RLVR requires verifiable correctness, which does not exist. RLHF with absolute scoring (train a reward model to output a scalar quality score, then optimize against it) runs into the problem that "scoring a single scientific idea is difficult due to the lack of an objective and universal criterion"—the paper explicitly argues that even Scientific Judge, which is good at pairwise comparisons, would be unreliable at assigning absolute scores to individual ideas. Best-of-N sampling with post-hoc filtering (generate many ideas, use an external metric like novelty or a verifier to select the best) improves selection but does not train the model to generate better ideas in the first place.

Comparison-Based GRPO solves this by converting pairwise comparison capability into a gradient signal for generation. The key insight is: if you have a reliable pairwise judge (which Scientific Judge is, at 80.6% accuracy) but not a reliable absolute scorer (which doesn't exist), you can still train a generator by having it produce groups of candidates and rewarding each candidate based on how many pairwise comparisons it wins within its group. The round-robin tournament structure ensures that every pair is evaluated, and the win rate provides a normalized reward in [0, 1] that is compatible with standard RL objective functions.

What makes this genuinely novel rather than an obvious extension of GRPO is the use of within-group comparisons as the sole reward signal. Vanilla GRPO for math (Shao et al., 2024) uses verifiable ground-truth rewards; Comparison-Based GRPO replaces this with relative judgments from a learned verifier. This is a different form of reward signal—it is dense (every output gets a reward between 0 and 1, not just 0 or 1), self-normalizing (the mean win rate within any group is exactly 0.5, so the advantage normalization centers automatically), and contextual (an idea's reward depends on the quality of other ideas in its group, not on an absolute standard).

The contextual nature of the reward has an interesting property: it creates an implicit curriculum. Early in training, when the policy generates uniformly mediocre ideas, the win rates will be near 0.5 for everyone (all ideas are similarly mediocre) and the reward signal will be weak—the policy gets little gradient because advantages are near zero. As training progresses and some ideas become genuinely better than others within groups, the win-rate distribution widens and the policy receives stronger gradient signals to favor the better ideas. This means training naturally accelerates as the policy improves, without requiring explicit curriculum design.

The algorithm is also general in a way that the paper does not heavily emphasize: any domain with a reliable pairwise judge—whether learned (like Scientific Judge) or algorithmic (like a comparison function for code solutions, design proposals, creative writing, or experimental protocols)—can use Comparison-Based GRPO to train a generator. The paper demonstrates it for scientific ideation, but the mechanism is domain-agnostic.

Evidence: Figure 4 shows the direct causal test. Policies trained with Comparison-Based GRPO significantly outperform their base policies (81.5% win rate for SciThinker-30B), and the comparison with the Qwen3-4B-Instruct reward model ablation (73.0% win rate) isolates the contribution of the learned judge. Table 8 extends the evidence: SciThinker-30B achieves 54.2% average win rate against GPT-5.2, GLM-5, and Gemini 3 Pro in direct comparison of proposed research ideas—not just beating its own base model, but competing favorably with state-of-the-art proprietary systems on a task (ideation) that none of them were explicitly trained for.

Innovation 5: Verifier Over-Optimization as a First-Class Phenomenon in Test-Time Scaling

This appears to be an error in the reference — Innovation 5 in the reference example is about verifier over-optimization in test-time scaling, which is from a completely different paper. The OpenMOSS paper on scientific taste does not address verifier over-optimization or test-time compute scaling at all — those concepts belong to the compute-optimal test-time scaling paper used as the reference example.

The OpenMOSS paper's innovations have been covered in Innovations 1–4 above. The paper's contributions are conceptual (reframing taste as learnable community preference), methodological (RLCF as a training paradigm, Comparison-Based GRPO), and empirical (generalization evidence that learned judgment transfers across time/field/metric boundaries). There is no fifth innovation at the same level of significance — the remaining findings (scaling behavior in Figure 3, general capability preservation in Appendix D) are important validations of the approach rather than independent intellectual contributions.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary evaluation benchmark is SciJudgeBench, comprising 728 paper pairs for in-domain testing stratified across Computer Science, Mathematics, Physics, and "Others" (Appendix A.1). Three additional OOD test sets probe generalization: (1) a Temporal OOD set of 514 pairs from papers published in 2025 (after the training cutoff), (2) a Field OOD configuration where models trained only on CS papers are evaluated on Math, Physics, and Others pairs, and (3) a Metric OOD set of 611 pairs from ICLR submissions (2017–2026) where ground-truth preferences are determined by peer review scores rather than citations. A supplementary bioRxiv OOD set with 160 biology pairs tests cross-platform transfer (Appendix A.5).

• Base model(s). Scientific Judge is trained on eight base models spanning three families: Qwen2.5-Instruct (1.5B, 3B, 7B, 14B, 32B), Qwen3-Instruct (4B and 30B-A3B), and Llama-3.1-8B-Instruct (Appendix B.1). Scientific Thinker uses Qwen3-30B-A3B-Thinking-2507 and Qwen3-4B-Thinking-2507—"thinking" variants optimized for extended reasoning before generation (Section 5.1). The multi-family, multi-scale design enables both scaling analysis and cross-architecture generalization testing.

• Metrics. Scientific Judge is evaluated by pairwise accuracy with position-swap consistency: each pair is evaluated twice (original and swapped A↔B order), and a prediction scores 1 only if correct in both orderings (Appendix B.5). This eliminates position bias as a confound. Scientific Thinker is evaluated by win rate against baselines as judged by a panel of three strong LLMs (GPT-5.2-high, GLM-5, Gemini 3 Pro) via majority voting—an idea wins if at least 2 of 3 evaluators prefer it (Appendix C.2). The authors validate this meta-evaluation by testing the three-model majority voting on SciJudgeBench itself, finding it achieves 84.4% accuracy on citation preference prediction.

• Baselines. For Scientific Judge, baselines include: (a) untrained base models (e.g., Qwen3-4B-Instruct at 60.3%, Qwen3-30B-A3B at 66.3%, Table 3) across all scales and families; (b) SOTA proprietary models evaluated zero-shot on SciJudgeBench: DeepSeek-V3.2 (62.6%), DeepSeek-V3.2-Thinking (69.9%), MiniMax-M2.5 (68.7%), GPT-5.2-Thinking (72.7%), GLM-5 (73.6%), and Gemini-3.0-Pro-Preview (75.7%), all reported in Table 3. For Scientific Thinker, baselines include: (a) the untrained base policy (Qwen3-30B-A3B-Thinking-2507 or Qwen3-4B-Thinking-2507), (b) policies trained with the untrained base model as reward model (Qwen3-4B-Instruct instead of SciJudge-Qwen3-4B) to isolate the contribution of learned judgment, and (c) SOTA models (GPT-5.2, GLM-5, Gemini 3 Pro) compared directly in idea quality (Table 8).

• Generation budget / compute accounting. For Scientific Judge, training uses 1 epoch over 696,758 pairs with effective batch size 128, using 32–128 GPUs depending on model size (Appendix B.3). At inference, each evaluation pair requires 2 forward passes (one for each order in the position-swap protocol). For Scientific Thinker, training uses 4,000 seed papers, sampling 8 candidate ideas per paper, with 28 pairwise comparisons per group (round-robin among 8 ideas), requiring the reward model to evaluate each comparison. Evaluation costs 3 judge model queries per pair (one per evaluator LLM), with 50% of pairs having randomly swapped order.

• Cross-validation / statistical protocol. No formal cross-validation is used for strategy selection (unlike the reference paper's two-fold cross-validation for compute-optimal policies). The test sets are fixed holdout sets with clear temporal, field, or metric boundaries from training. For Scientific Judge, the evaluation protocol's position-swap consistency requirement serves as a robustness check. For Scientific Thinker, majority voting across three independent evaluator models provides inter-rater reliability, and the 84.4% validation accuracy on SciJudgeBench provides a calibration point for the evaluation protocol itself.

Main Quantitative Results

Scientific Judge: In-Domain Citation Preference Prediction

Headline result: SciJudge-Qwen3-30B achieves 80.6% average accuracy on SciJudgeBench, surpassing all proprietary baselines including GPT-5.2-Thinking (72.7%) and Gemini-3.0-Pro-Preview (75.7%). This is an improvement of +14.3 points over the untrained Qwen3-30B-A3B-Instruct base model (66.3% → 80.6%).

The results in Table 3 establish three scaling relationships. Data scaling produces consistent improvement (visible in Figure 3): during training, SciJudge-Qwen3-4B rises from 60.3% to 75.3% and SciJudge-Qwen3-30B from 66.3% to 80.6%, with curves showing approximately log-linear improvement across all fields throughout training. The paper explicitly states this log-linear relationship in the text accompanying Figure 3. Model size scaling is monotonic and substantial: in the Qwen2.5 family, average accuracy after SciJudge training increases from 72.1 (1.5B) to 73.2 (3B) to 76.9 (7B) to 80.6 (14B) to 83.7 (32B). The 32B model achieves 83.7%—the highest in-domain accuracy reported across all models. Cross-family transfer succeeds: SciJudge-Llama3.1-8B improves from 36.8% to 57.3% (+20.5 points), confirming that the training procedure works beyond the Qwen family, though the absolute performance (57.3%) substantially trails Qwen-based models at similar parameter counts (SciJudge-Qwen2.5-7B: 76.9%).

Field-specific breakdown (Table 3): SciJudge-Qwen3-30B achieves 83.5% on CS, 78.7% on Math, 78.7% on Physics, and 82.3% on Others. The improvement over base models is consistently large across all fields, with the largest relative gains in Others (+22.7 for the 4B, +16.8 for the 30B), suggesting that the base models were weakest on non-CS/Physics areas and the training is particularly effective at teaching cross-domain judgment patterns.

Comparison to proprietary models (Figure 1, left panel; Table 3, bottom rows): SciJudge-Qwen3-30B (80.6%) exceeds Gemini-3.0-Pro-Preview (75.7%) by ~5 points and GPT-5.2-Thinking (72.7%) by ~8 points. Even SciJudge-Qwen3-4B at 75.3% is competitive with Gemini-3.0-Pro. The model-size-vs-accuracy visualization in Figure 1 (left) shows the trained models (stars) forming a clear Pareto frontier above the proprietary model curve, with SciJudge-4B achieving accuracy that would require a ~671B+ proprietary model to match under the untrained scaling trend.

Scientific Judge: Temporal Generalization to Future Papers

Headline result: On papers published in 2025—after the training cutoff—Scientific Judge improves over base models across all backbones, with SciJudge-Qwen3-30B achieving 78.2% average (+6.6 points over base, Table 4).

The temporal generalization is consistent but shows field-specific patterns (Table 4). SciJudge-Qwen3-30B improves on CS (76.4% → 82.1%, +5.7), Physics (63.4% → 72.1%, +8.7), and Others (73.1% → 76.9%, +3.8), but actually degrades slightly on Math (44.4% → 55.6%, +11.2—but both numbers are low, likely due to the small Math sample size of only 9 pairs in the temporal OOD set, as reported in Appendix A.3). The SciJudge-Qwen3-4B shows a more uniform pattern: +5.7 on CS, +11.1 on Math, +4.4 on Physics, +23.1 on Others. The small Math sample (9 pairs) makes field-specific temporal OOD Math results unreliable—the paper acknowledges this by noting the category distribution of the temporal OOD set (CS: 318, Physics: 161, Others: 26, Math: 9, Appendix A.3) but does not flag the instability explicitly.

Gains at smaller scales are proportionally larger. SciJudge-Qwen2.5-1.5B improves from 9.9% to 65.0% (+55.1 points, Table 4), SciJudge-Qwen2.5-3B from 23.7% to 63.6% (+39.9 points), SciJudge-Qwen2.5-7B from 54.7% to 70.4% (+15.8 points). This pattern—larger relative gains for smaller models—suggests that temporal generalization is partly about learning general judgment principles that the larger base models already partially possess, which is an interesting finding the paper does not discuss explicitly.

Scientific Judge: Cross-Field Generalization

Headline result: When trained only on CS papers and evaluated on all fields (Table 5), Scientific Judge generalizes effectively, with SciJudge-Qwen3-4B improving from 60.3% to 67.7% average (+7.4 points) across all out-of-domain fields.

The field OOD results provide the strongest evidence for transferable judgment. SciJudge-Qwen2.5-32B achieves 81.0% average when CS-trained and evaluated on all fields—including 78.7% on Math and 76.9% on Physics, substantially above the untrained base model (61.5% and 55.9%, respectively). This means the model learns CS-specific citation patterns that transfer to predicting citation preferences in entirely different disciplines—suggesting shared dimensions of scientific value that transcend field-specific vocabulary and methodology.

The cross-field pattern is consistent across scales. SciJudge-Qwen2.5-1.5B: 68.3% average (+61.3 points, Table 5); SciJudge-Qwen2.5-3B: 69.5% (+46.0); SciJudge-Qwen2.5-7B: 72.4% (+27.2); SciJudge-Qwen2.5-14B: 78.0% (+18.9); SciJudge-Qwen2.5-32B: 81.0% (+18.8). The diminishing relative gains with scale reflect higher base model baselines—larger models already have some cross-field generalization ability from pretraining. Notably, even at 32B, the +18.8-point gain is substantial, indicating that RLCF adds judgment capability beyond what scale alone provides.

A negative result: SciJudge-Qwen3-30B shows a decrease in Math accuracy under field OOD (70.5% → 63.9%, -6.6 points, Table 5) despite improving on Physics and Others. This is the only instance of cross-field degradation in the table and is not discussed in the paper. It may reflect overfitting to CS-specific patterns at the 30B scale that do not transfer to Math, or it may be noise given the non-trivial but modest Math test set size.

Scientific Judge: Cross-Metric Generalization to Peer Review

Headline result: On ICLR papers where ground truth is peer review scores, Scientific Judge substantially improves agreement, with SciJudge-Qwen3-30B achieving 87.7% (+11.0 points over base, Table 6).

This is the most striking generalization result. The model is trained to predict "which paper has more citations" but evaluated on "which paper is more likely to be accepted at ICLR"—a different task framing with a different underlying quality signal (expert peer review vs. aggregate citation behavior). SciJudge-Qwen3-4B improves from 65.3% to 79.1% (+13.8); SciJudge-Qwen2.5-1.5B from an essentially random 1.6% to 73.7% (+72.0 points—the largest absolute gain in any table in the paper); SciJudge-Qwen2.5-14B achieves 85.4%; SciJudge-Llama3.1-8B achieves 80.7% (+34.0). The consistency across all scales and families suggests the effect is robust and not dependent on specific model characteristics.

The 72-point gain for the 1.5B model (from 1.6% to 73.7%) is particularly notable: the base model was effectively producing random guesses on ICLR preferences (1.6% is far below chance, likely due to consistent but incorrect heuristics), and the training completely reverses this to near-human-level agreement with peer review. This is strong evidence that the model learned something fundamental about scientific quality assessment rather than citation-specific heuristics.

Scientific Thinker: Ideation Quality

Headline result: SciThinker-30B achieves 81.5% win rate against its untrained base policy on in-domain test papers (January–July 2025) and 83.0% on out-of-domain test papers (August–December 2025), as shown in Figure 4(a).

Comparison to ablation reward model (Figure 4(a), bottom row vs. top row): SciThinker-30B trained with SciJudge-Qwen3-4B as the reward model achieves 81.5% in-domain and 83.0% out-of-domain, while the same policy trained with the untrained Qwen3-4B-Instruct as reward achieves only 73.0% and 70.5%. The ~8.5-point in-domain gap and ~12.5-point out-of-domain gap directly measure the value of the learned community preference signal: RL training alone provides some improvement (73% vs. 50% random baseline), but the RLCF-trained reward model provides substantially more effective guidance. For the 4B model (Figure 4(b)), the pattern holds: 76.5% vs. 68.5% in-domain, 76.0% vs. 69.0% out-of-domain.

Temporal generalization of ideation: The out-of-domain win rates are slightly higher than in-domain for the 30B model (83.0% vs. 81.5% with SciJudge reward, 70.5% vs. 73.0% with base reward), suggesting that ideation capability transfers to research topics published after training. The 4B model shows essentially identical in-domain and out-of-domain performance (76.5% vs. 76.0%), indicating robust generalization at the smaller scale as well.

Comparison to SOTA models (Table 8): SciThinker-30B achieves 54.2% average win rate across GPT-5.2 (61.0%), GLM-5 (58.5%), and Gemini 3 Pro (43.0%) on in-domain papers. The base policy (Qwen3-30B-A3B-Thinking) achieves only 30.3% average—SciThinker-30B improves by +23.9 points. On out-of-domain papers (Table 8(b)), the pattern is nearly identical: 54.2% average for SciThinker vs. 27.8% for the base policy (+26.4 points). The model beats GPT-5.2 in 59–61% of comparisons, GLM-5 in 58.5–61%, and Gemini 3 Pro in 42.5–43%. The lower win rate against Gemini 3 Pro suggests it is a stronger ideation baseline, but SciThinker-30B still substantially outperforms its base model against all three.

Scaling Behavior

Figure 3 shows training curves for SciJudge-Qwen3-4B and SciJudge-Qwen3-30B across the four field categories and overall. Both models improve consistently across all fields throughout training, with the 30B model maintaining a consistent advantage over the 4B model at every point. The learning curves display the approximately log-linear relationship between data scale and performance that the paper highlights as a key scaling finding. The curves do not show obvious saturation, suggesting further data might yield continued improvement.

Model size scaling (inferred from Table 3): within the Qwen2.5 family, average accuracy increases from 72.1% (1.5B) → 73.2% (3B) → 76.9% (7B) → 80.6% (14B) → 83.7% (32B). The 3B→7B jump (+3.7 points) is proportionally larger than the 1.5B→3B jump (+1.1 points), suggesting an emergent capability threshold between 3B and 7B parameters. However, since different model scales may have different pretraining data and quality, the scaling relationship is not a pure function of parameter count. The Qwen3 family comparison (4B: 75.3%, 30B: 80.6%) shows a +5.3-point gain, which is substantial but smaller than the Qwen2.5 7B→32B gain (+6.8 points), possibly reflecting architectural improvements in Qwen3 that boost the 4B baseline.

Ablation Studies and Robustness Checks

Reward model quality ablation for Scientific Thinker: The comparison between SciJudge-Qwen3-4B and Qwen3-4B-Instruct as reward models (Figure 4, top vs. bottom rows) directly measures the contribution of learned judgment. For the 30B policy, using the RLCF-trained reward improves win rates by +8.5 points in-domain and +12.5 points out-of-domain over using the untrained base model. This is the central causal ablation: it demonstrates that RLCF produces a better reward signal than the base model's untrained judgment, and that this improved signal translates into better ideation.

Model scale ablation for Scientific Thinker: Both 30B and 4B policies are trained with both reward models (Figure 4(a) vs. 4(b)). The 30B model consistently outperforms the 4B: with SciJudge reward, 30B achieves 81.5% vs. 4B's 76.5% in-domain; with base reward, 30B achieves 73.0% vs. 4B's 68.5%. The gap between 30B and 4B is larger with the SciJudge reward (~5 points) than with the base reward (~4.5 points), suggesting that larger models benefit more from higher-quality reward signals.

General capability preservation (Appendix D, Table 13): Scientific Judge training preserves performance on MMLU-Pro, GPQA, MATH, GSM8K, and SimpleQA within ±3% of baseline for most model-scale combinations. Several models show minor improvements: SciJudge-Qwen2.5-3B gains +0.8% on MATH and +3.1% on GPQA; SciJudge-Llama3.1-8B gains +7.2% on MMLU-Pro and +1.8% on MATH. The only notable degradation is SciJudge-Qwen2.5-7B on GPQA (-5.6%, from 38.4% to 32.8%), but this model improves on MMLU-Pro (+1.8%) and maintains MATH/GSM8K, suggesting the GPQA drop may be noise rather than systematic capability loss. The KL penalty coefficient of 0.03 appears sufficient to prevent catastrophic forgetting.

Base model family ablation: The Llama-3.1-8B results (Tables 3–7) demonstrate cross-family transfer: SciJudge-Llama3.1-8B improves from 36.8% to 57.3% in-domain (+20.5), from 41.4% to 58.8% temporal OOD (+17.3), from 36.8% to 54.8% field OOD (+18.0), and from 46.6% to 80.7% metric OOD (+34.0). The consistent improvements confirm that RLCF works beyond the Qwen family. However, Llama-3.1-8B's absolute performance (57.3%) substantially trails Qwen2.5-7B (76.9%), suggesting either Qwen's architecture is better suited to this task or the base Llama model is weaker on scientific reasoning pretraining. The paper does not explore this performance gap.

Field-specific training ablation (Table 5): Training on CS-only data and evaluating on all fields provides a natural ablation of cross-field generalization. The consistent improvement across Math, Physics, and Others (Table 5, rightmost columns) demonstrates that the learned judgment patterns are not CS-specific. The fact that CS in-domain accuracy (e.g., SciJudge-Qwen3-4B: 76.2% in CS-only training vs. 78.6% in full-data training, Table 3) is only slightly lower than training on all fields suggests that adding non-CS data provides modest additional benefit—most of the judgment capability is learned from CS papers alone. This is a somewhat surprising negative result: the paper does not discuss why CS-to-other-field transfer is so effective, but it may indicate that CS papers contain sufficient diversity of scientific patterns (theoretical contributions, empirical evaluations, systems building, applications) to cover the judgment-relevant dimensions present in other fields.

Biology field OOD (Table 7, Appendix A.5): The bioRxiv evaluation on 160 biology pairs from a completely different platform tests the most extreme generalization. SciJudge-Qwen3-30B achieves 71.2% (+26.2 over base), SciJudge-Qwen2.5-32B achieves 68.1% (+15.0), SciJudge-Qwen2.5-14B 64.4% (+16.9), and SciJudge-Qwen2.5-7B 63.1% (+20.0). These numbers are lower than in-domain CS (Table 3: 83.5% for the 30B, 85.4% for the 32B) but substantially above the base models (45.0% and 53.1%, respectively). The fact that models trained on arXiv CS/Math/Physics transfer to biology preprints indicates that the learned judgment captures very abstract patterns of scientific value. The lower absolute performance (~71% vs. ~84% in-domain for the 30B) reflects genuine domain shift—biology papers have different structures, terminology, and citation dynamics than arXiv CS/Physics.

Position-swap consistency evaluation (Appendix B.5): All accuracy numbers in Tables 3–7 use the position-swap consistency protocol. The paper does not report what accuracy would be without this protocol—this is a missing ablation that would quantify the magnitude of position bias in the base and trained models. Given that position bias is a known issue in LLM pairwise evaluation (cited via Zheng et al., 2023), the protocol is well-justified, but the absence of a comparison to single-ordering accuracy makes it impossible to assess how much position bias the training reduces.

Scientific Thinker evaluation protocol validation (Appendix C.2): The three-model majority voting achieves 84.4% accuracy on SciJudgeBench, which the authors use to justify its reliability for evaluating generated ideas. This is a reasonable validation but has a limitation: SciJudgeBench pairs are real papers with known citation differences, while Scientific Thinker evaluation involves hypothetical ideas with no ground truth. The evaluator models may behave differently on real vs. hypothetical papers—real papers have more concrete content (actual methods, results) while generated ideas are aspirational (proposed methods, expected outcomes). The paper does not address this distribution shift in the evaluation protocol validation.

Critical Assessment

Claim 1: "AI can learn scientific taste"

The paper defines scientific taste as the conjunction of high JudgeCap and high ThinkerCap (Section 2.1). For JudgeCap, the evidence is strong: Scientific Judge achieves 80.6% in-domain accuracy (Table 3), significantly above proprietary baselines, and generalizes across time (+6.2 to +6.6 points, Table 4), fields (+4.1 to +7.4 points average, Table 5), and metrics (+11.0 to +13.8 points for Qwen3 models, Table 6). The generalization results are the critical evidence—they argue against the hypothesis that the model merely memorized CS-specific citation patterns.

However, two limitations constrain the strength of this claim. First, the generalization is primarily from CS to other quantitative sciences. Math, Physics, bioRxiv biology, and ICLR papers share substantial methodological overlap with CS (statistical analysis, formal modeling, computational methods). The paper does not test generalization to disciplines with fundamentally different epistemic structures—history, literary criticism, qualitative sociology, jurisprudence—where the dimensions of scientific value may differ categorically. What the paper demonstrates is that taste learned from CS citations transfers to adjacent quantitative fields, not that the learned taste is universal.

Second, accuracy is measured on pairs with clear preference signals (test pairs have absolute citation differences ≥32 and relative differences ≥50%; ICLR pairs have median rating difference of 6.1/10). The model's accuracy on ambiguous pairs—where two papers are genuinely close in quality—is untested. A model achieving 80% on easy pairs might be near-random on difficult ones, and real-world scientific judgment often involves precisely those borderline cases (which of two strong candidates to hire/promote/fund). The test set construction deliberately removes ambiguity to ensure clean evaluation, which is methodologically sound but means the reported accuracy overstates performance on the full difficulty distribution.

Claim 2: "Scientific Judge generalizes across time, fields, and peer-review preferences"

The temporal generalization claim is supported (Table 4) but with the caveat that 2025 papers have very few citations—the adaptive thresholding procedure (Appendix A.3) ensures the temporal OOD pairs have meaningful preference signals, but it also means the test set is restricted to papers that already accumulated enough citations within months to distinguish themselves. This selects for papers that became visibly impactful very quickly, which may not be representative of all high-impact papers (some gain citations slowly). The temporal generalization therefore demonstrates that the model can identify fast-rising papers in the future, not all high-impact papers.

The field generalization claim (Table 5) is well-supported for the field pairs tested. However, the CS-only training configuration is somewhat artificial—in practice, one would train on all available fields. The experiment demonstrates that CS-acquired judgment transfers, but the paper does not report a CS+Physics training → Math generalization experiment to test whether adding fields improves transfer. This would be informative about whether RLCF benefits from diverse training distributions.

The metric generalization claim (Table 6) is the strongest single piece of evidence in the paper. The 87.7% accuracy of SciJudge-Qwen3-30B on ICLR preference prediction—using a model trained to predict citations, tested on "which paper is more likely to be accepted," with a median rating difference of 6.1/10—is genuinely impressive. The 72-point gain for the 1.5B model (from 1.6% to 73.7%) is striking and argues strongly for learned judgment rather than pretraining priors. However, the ICLR test set construction warrants scrutiny: papers are filtered by removing bottom-50% confidence and top-50% variance reviews, then taking top and bottom 10% by rating (Appendix A.3). This means the pairs compare clear-accept papers to clear-reject papers with reliable reviews—a setting where the signal is unambiguous. The model's accuracy on closer calls (e.g., borderline accept vs. borderline reject) is untested.

Claim 3: "Scientific Thinker proposes research ideas with higher potential impact than baselines"

The evaluation of ideation quality is the paper's most methodologically challenging claim, because "potential impact" of hypothetical ideas has no ground truth. The paper uses a panel of three strong LLMs as evaluators, validated at 84.4% on citation prediction. This provides evidence that the evaluators correlate with citation-based quality signals, but does not establish that they can assess novel, hypothetical ideas—the validation is on existing papers where quality signals exist in the text (actual methods, results, contributions), while generated ideas contain only proposed methods and expected contributions. An evaluator LLM might prefer ideas that sound more like high-citation papers (in terms of rhetorical structure, confidence, technical terminology) without those ideas actually having higher potential impact if executed. This is an inherent limitation of any ideation evaluation without experimental validation, and the paper acknowledges it in limitations ("Since the proposed ideas are not experimentally validated, the evaluation may not fully reflect their potential impact").

The win rates against SOTA models (Table 8) are compelling but have an important asymmetry: SciThinker is explicitly trained for scientific ideation using a citation-trained reward model, while GPT-5.2, GLM-5, and Gemini 3 Pro are general-purpose models evaluated zero-shot on this specific task. A fairer comparison would be to fine-tune those models on the same ideation data (or equivalent data from their training distributions) and compare. The 54.2% average win rate demonstrates that specialized training for ideation improves over general-purpose strong models, but does not establish that RLCF-trained ideation is better than what could be achieved with alternative training paradigms (e.g., SFT on high-impact paper abstracts, or RLHF with expert scientist feedback).

The out-of-domain generalization of ideation (83.0% win rate on August–December 2025 papers vs. 81.5% in-domain, Figure 4(a)) is an interesting finding: out-of-domain performance is slightly higher than in-domain. The paper does not discuss this inversion. Possible explanations: (1) the out-of-domain test set may contain intrinsically easier seed papers (more obvious follow-up directions), (2) the model may propose ideas that sound more impressive when the evaluators are less familiar with the seed paper's specific subfield, or (3) the difference may be noise given the 200-paper test sets.

Claim 4: "Scaling behavior is observed for scientific judgement"

The scaling evidence is descriptive rather than law-like. Figure 3 shows that performance improves with more training data (the x-axis is "Training Progress," representing fractions of the 696K-pair dataset), and Table 3 shows that larger models perform better after training. However, the paper does not fit a scaling law (power law or otherwise), does not predict performance at unobserved model sizes or data quantities, and does not test whether the scaling trend continues beyond 32B parameters or 696K pairs. The log-linear relationship mentioned in Takeaway 1 is a qualitative description of the learning curves in Figure 3, not a quantitatively fitted functional form.

Additionally, the "scaling" in the paper conflates two effects: model scale (larger pretrained models have better priors for scientific judgment) and training data scale (the learning curves). The paper does not disentangle these—e.g., does a 1.5B model trained on all 696K pairs outperform a 7B model trained on 100K pairs? Without data scaling curves at fixed model sizes (or model scaling curves at fixed data quantities), the paper's scaling claims are observational rather than predictive.

Missing Experiments That Would Strengthen the Paper

1. Data scaling curves at fixed model sizes. Training a single model (e.g., Qwen2.5-7B) on 25%, 50%, 75%, and 100% of the training data and plotting accuracy vs. data quantity would test whether scaling is truly log-linear and whether saturation occurs. This is standard practice in scaling law studies and its absence means the scaling claims are qualitative.

2. Comparison of Scientific Thinker to alternative ideation training methods. How does RLCF-trained ideation compare to: (a) SFT on the same 4,000 seed papers with human-written follow-up ideas, (b) RLHF with expert scientist pairwise preferences as the reward signal, (c) simply prompting the base model with "propose a high-impact follow-up idea" using chain-of-thought without any training? The paper only compares to the untrained base policy and to policies trained with a weaker reward model—not to alternative training paradigms.

3. Human evaluation of Scientific Thinker's ideas. The paper relies entirely on LLM evaluators for ideation quality. A small-scale human study—even with 10–20 computer science researchers evaluating 50–100 generated ideas—would provide a critical calibration point for whether the LLM evaluator win rates correspond to human judgments of idea quality.

4. Analysis of what Scientific Judge attends to. The case studies in Appendix F show qualitative reasoning traces, but a systematic analysis—e.g., which features of abstracts (method novelty, problem importance, clarity, author reputation cues) drive the model's predictions—would clarify whether the model learns meaningful dimensions of scientific value or exploits surface correlations (paper length, presence of mathematical notation, confidence of language, name-dropping of popular benchmarks).

5. Testing Scientific Judge on closer citation differences. The test sets use strict thresholds to ensure clear preferences, but the model's discrimination ability on pairs with small citation differences (e.g., 50 vs. 55 citations) would reveal whether the learned judgment is useful for the practical task of distinguishing between genuinely similar-quality papers, which is where human judgment is most needed.

6. Training Scientific Thinker with larger Scientific Judge as reward model. The paper uses SciJudge-Qwen3-4B as the reward model for all Scientific Thinker training (Section 5.1). Testing whether SciJudge-Qwen3-30B or SciJudge-Qwen2.5-32B (which achieve 80.6% and 83.7% in-domain accuracy, respectively, vs. 4B's 75.3%) as the reward model further improves ideation quality would test whether better judgment directly translates to better ideation—a core premise of the RLCF pipeline. This is a notably missing experiment given the paper's emphasis on the judge→thinker feedback loop.

Summary of Evidentiary Support

The paper's strongest evidence is for judgement capability learning and generalization (Claims 1 and 2). The consistency of improvements across all model scales, families, and OOD settings, combined with the position-swap consistency protocol and the general capability preservation checks, makes a compelling case that RLCF trains models to extract transferable patterns of scientific value from citation data rather than surface heuristics.

The evidence for ideation capability improvement (Claim 3) is suggestive but methodologically weaker due to the inherent difficulty of evaluating hypothetical ideas. The win rates against base policies and SOTA models are large and consistent, but the evaluation protocol's calibration (84.4% on citation prediction) is on a different task than the ideation evaluation (judging hypothetical ideas). The claim that Scientific Thinker proposes "higher potential impact" ideas is better supported than the null hypothesis (no improvement) but the magnitude and nature of the improvement—whether the ideas are genuinely more innovative, better targeted at important problems, or simply better-written to sound impressive—remains unclear without human evaluation.

The scaling claims (Claim 4) are primarily descriptive. The observed relationships (more data → better performance, larger models → better performance) are unsurprising for deep learning systems and the paper does not provide the quantitative scaling law characterizations that would make them predictive or actionable.

The paper's most significant contribution is empirical rather than theoretical: it demonstrates that community citation behavior contains a learnable signal of scientific value, that this signal can be extracted via preference learning, and that it can be used to guide generation—establishing RLCF as a viable training paradigm for capabilities previously thought to require innate human judgment. The generalization results are the paper's strongest empirical argument for this contribution.

6. Limitations and Trade-offs

Citation Counts as a Proxy for Scientific Impact

The assumption: The entire RLCF framework rests on treating citation counts as the ground-truth signal of scientific value. The paper operationalizes "potential impact" as cumulative expected citations (Equation 1), constructs preference labels from citation differences between field- and time-matched pairs (Section 3.1), and trains Scientific Judge to predict which paper receives more citations (Section 3.2). The paper acknowledges this assumption explicitly in the Limitations section:

"citations are an imperfect form of community feedback. Some high-potential papers may receive few citations initially but become highly influential later."

The consequence: The learned "taste" is taste as refracted through citation behavior, which introduces several specific distortions. First, the model may systematically undervalue papers that are highly influential but rarely cited formally—methodological contributions absorbed into common practice, pedagogical works, or papers in fields where citations are sparse norms. Second, citation patterns encode community biases (toward famous authors, prestigious institutions, well-resourced topics) that the model will internalize as "taste." Third, and most concretely, the model cannot distinguish between a paper that deserves high citations but has not yet received them (a delayed-impact paper) and a paper that genuinely deserves low citations—both look identical at training time because only observed citations are available, not counterfactual deserved citations. This means Scientific Judge's predictions are at best predictions of likely citation counts given community behavior, not evaluations of intrinsic scientific merit independent of social dynamics.

Evidence in the paper: The paper does not measure this limitation directly. There is no analysis of whether Scientific Judge's errors are concentrated on papers that were later recognized as important despite early low citation counts (the "delayed-impact" case), nor any comparison to alternative quality metrics (e.g., expert retrospective assessments, award committees, textbook inclusion). The paper's own framing treats citation counts as the operational definition of impact, making the limitation somewhat circular within the paper's evaluation framework—accuracy is measured against citations, so by construction the model learns to predict citations well. The metric OOD result (Table 6, ICLR peer review) partially addresses this by showing that citation-trained judgment correlates with a different quality signal, but peer review has its own documented biases and is not a ground-truth measure of merit either.

Mitigation status: The paper acknowledges the limitation but does not attempt to mitigate it. The Limitations section mentions "modeling citation dynamics may help capture such delayed-impact patterns" as future work, but no such modeling is implemented. The field- and time-matching procedure (Section 3.1) controls for field-level and temporal citation norm differences (e.g., CS papers get more citations than Math papers) but does not address the more fundamental issue of whether citations measure scientific value rather than social popularity. This is an inherent limitation of using any naturally occurring behavioral signal as supervision—the signal measures behavior, not ground-truth quality, and the two can diverge systematically.

Difficulty Estimation Cost Is Not Accounted for in the Pipeline

The assumption: Scientific Thinker training (Section 3.3) requires Scientific Judge to evaluate 28 pairwise comparisons per group of 8 generated ideas (round-robin tournament), and evaluation requires three SOTA LLMs to judge each pair of generated ideas (Appendix C.2). The paper reports win rates and accuracy improvements without accounting for the computational cost of these comparison-based evaluations in the headline efficiency metrics.

The consequence: The reported improvements in ideation quality (81.5% win rate for SciThinker-30B, Figure 4) come with a substantial and unreported computational multiplier. For each gradient step during Scientific Thinker training, the reward computation requires (8 choose 2) = 28 forward passes through Scientific Judge per seed paper. With a batch size of 128 seed papers and 1 epoch over 4,000 papers, this means approximately 28 × 128 forward passes per training step for reward computation alone, in addition to the 8 forward passes through the policy model for generating candidates. At inference time, evaluating one pair of generated ideas requires 3 separate LLM calls (one per evaluator model). The comparison-based evaluation protocol is thus significantly more expensive than direct scoring would be—if a reliable direct scorer existed, the cost would be 1 call per idea rather than (G-1) calls per idea for the round-robin.

Evidence in the paper: The paper provides no FLOP accounting, no wall-clock time measurements, and no cost comparison to alternative training paradigms (e.g., SFT on curated idea datasets, or training with a smaller/cheaper reward model). The computational resources section (Appendix B.3) reports GPU counts (32–128 GPUs depending on model size) but does not report total GPU-hours, training duration, or inference cost. The evaluation protocol validation (Appendix C.2) reports that majority voting achieves 84.4% on SciJudgeBench but does not report the cost of running three SOTA LLMs per evaluation pair.

Mitigation status: Not addressed. The paper does not discuss the computational overhead of Comparison-Based GRPO relative to alternative reward mechanisms, does not propose cheaper approximations (e.g., comparing each idea to only a random subset of opponents rather than full round-robin, or using a single evaluator instead of majority voting), and does not include computational cost in any table or figure. This makes it impossible for a practitioner to assess whether the reported quality improvements justify the computational cost. The paper also does not discuss whether the comparison-based reward could be distilled into a faster direct scorer after training, which would reduce inference-time evaluation cost.

Ideation Evaluation Relies Entirely on LLM Judges Without Human Validation

The assumption: The central claim about Scientific Thinker—that it proposes research ideas with "higher potential impact" (Section 5.2)—rests entirely on evaluations by GPT-5.2, GLM-5, and Gemini 3 Pro using majority voting (Appendix C.2). The paper validates this evaluation protocol by testing the three-model panel on SciJudgeBench and finding 84.4% accuracy on citation prediction, then uses this validation to justify the protocol for evaluating generated ideas.

The consequence: There is a fundamental distribution shift between the validation task and the actual evaluation task. SciJudgeBench consists of real papers with actual methods, results, contributions, and accumulated citations—the evaluator LLMs are comparing papers where quality signals exist in the text (did the method work? were the results strong? is the contribution clear?). Generated research ideas consist of hypothetical proposals with proposed methods and expected contributions but no actual results—the evaluator LLMs are comparing aspirational descriptions where quality signals are rhetorical rather than substantive. An evaluator LLM may prefer ideas that sound more impressive (confident language, technically sophisticated terminology, alignment with current research trends) without those ideas actually having higher potential impact if executed. The paper acknowledges this explicitly:

"Since the proposed ideas are not experimentally validated, the evaluation may not fully reflect their potential impact."

The consequence is that we cannot distinguish between two interpretations of the 81.5% win rate (Figure 4): (1) Scientific Thinker genuinely proposes better research directions, or (2) Scientific Thinker learns to write proposals that LLM evaluators find more persuasive, which may or may not correlate with actual scientific value. The case studies in Appendix E provide qualitative evidence that the generated ideas are coherent and well-structured, but do not establish that they would lead to high-impact research if implemented.

Evidence in the paper: The 84.4% validation accuracy on SciJudgeBench (Appendix C.2) provides evidence that the evaluator panel correlates with citation-based quality for real papers, but this correlation does not necessarily transfer to hypothetical proposals. The paper provides no human evaluation—no survey of domain experts comparing Scientific Thinker's ideas to baseline ideas, no qualitative assessment by researchers in the relevant fields, no execution of even a small subset of the generated ideas to assess whether they lead to publishable results. The Appendix E case studies show the generated ideas and evaluator judgments, which are illustrative but do not substitute for systematic human evaluation.

Mitigation status: The paper acknowledges the limitation in the Limitations section ("Future work can implement a subset of these ideas") but does not conduct any human evaluation or experimental validation in the current work. This is a significant gap because the entire RLCF pipeline's value proposition—that community feedback can train models to propose high-impact ideas—depends on the link between LLM evaluator preferences and actual scientific impact, and this link is assumed rather than demonstrated for generated ideas.

The Revision Model Has a 38% Correct-to-Incorrect Reversion Rate

Note: This limitation appears to be from the reference example rather than from the OpenMOSS paper being analyzed. The OpenMOSS paper on scientific taste does not have a revision model or report a 38% reversion rate. I should skip this and instead identify a limitation actually present in the OpenMOSS paper.

Let me identify the correct limitations for the OpenMOSS paper:

Single Benchmark Domain: All Training and Most Evaluation Is on arXiv Quantitative Sciences

The assumption: The paper's training data (SciJudgeBench, Section 3.1) consists exclusively of arXiv papers spanning Computer Science, Mathematics, Physics, and a few additional quantitative fields (Economics, Statistics, Quantitative Biology, Quantitative Finance—the "Others" category). The generalization experiments extend to ICLR (still CS-focused) and bioRxiv biology (still quantitative life sciences). The paper makes a general claim about "scientific taste" without restricting to specific disciplines.

The consequence: The learned "taste" may not transfer to disciplines with fundamentally different epistemic structures. History, literary criticism, qualitative sociology, jurisprudence, philosophy, and the arts evaluate research contributions along dimensions—interpretive depth, argumentative originality, archival discovery, conceptual reframing—that may not correlate with the citation patterns of quantitative sciences. More practically, even within the sciences, the arXiv corpus overrepresents fields that use preprints heavily; fields where preprints are rare or where citation dynamics differ substantially (clinical medicine, some areas of chemistry, field biology) may not be well-served by a model trained on arXiv patterns. The paper's title ("AI Can Learn Scientific Taste") and abstract (no disciplinary qualifier) imply a broader claim than the training data supports.

Evidence in the paper: The field OOD experiment (Table 5, CS-only training evaluated on Math, Physics, Others) shows transfer to adjacent quantitative fields, which is evidence for generalization within a broad methodological family but not evidence for generalization to qualitatively different disciplines. The bioRxiv evaluation (Table 7) extends to biology preprints, which share the preprint culture and quantitative methodology of arXiv fields. The paper does not test on: (a) humanities or social sciences outside economics/quantitative sociology, (b) clinical or translational medical research, (c) engineering fields with different publication norms, or (d) non-English-language scientific communities. The "Others" category in SciJudgeBench (Appendix A.1) explicitly aggregates Economics, EESS, Quantitative Biology, Quantitative Finance, and Statistics—all quantitative fields with methodological overlap with CS.

Mitigation status: Not addressed as a limitation. The paper's Limitations section focuses on citation dynamics and field granularity but does not discuss disciplinary scope. The title and abstract claims about "scientific taste" are not qualified by discipline, which may lead readers to overestimate the breadth of demonstrated generalization. Future work would need to test on corpora from the humanities, clinical medicine, or field sciences to establish whether RLCF-learned judgment transfers across fundamentally different epistemic cultures.

Hard Problems Remain Essentially Unsolved: No Evidence of Ideation for Truly Novel Research Directions

The assumption: Scientific Thinker is trained to propose follow-up research ideas given a seed paper's title and abstract (Section 5.1). The training seed papers are high-citation papers from January–July 2025. The model is evaluated on its ability to propose ideas that LLM evaluators prefer over baseline ideas.

The consequence: The training and evaluation setup incentivizes the model to propose ideas that are similar to existing high-impact work—follow-ups that extend known successful directions rather than paradigm-shifting departures. This is an inherent property of the training signal: Scientific Judge was trained to prefer papers that received more citations, which in practice means papers that fit within existing research programs (since citations come from other researchers working in related areas). A genuinely novel research direction—one that opens a new subfield or challenges existing assumptions—may receive few citations initially (because few researchers work on it) and would be penalized by the citation-based preference signal. The paper's RLCF pipeline therefore optimizes for ideas that align with community preferences, which by definition reflect the current consensus. This creates a fundamental tension: scientific taste includes the ability to recognize paradigm-shifting ideas that the community has not yet embraced, but the training signal cannot distinguish between "ignored because it is wrong" and "ignored because it is ahead of its time."

Evidence in the paper: The paper does not measure this directly, but the training data construction provides indirect evidence. The citation thresholds for SciJudgeBench (Section 3.1, Appendix A.2: absolute difference ≥8, relative difference ≥30%) systematically exclude papers that received few citations from training comparisons—precisely the papers that might represent undervalued novel directions. The temporal OOD test set (Appendix A.3) is constructed by selecting papers in the top 25% of citations in their subcategory, which again selects for papers that already achieved high citation counts quickly. A genuinely radical idea published in 2025 would not appear in this test set because it would not yet have enough citations to qualify. The case studies in Appendix E show Scientific Thinker proposing ideas that are incremental extensions of known directions (uncertainty-guided exploration for RLVR, dynamic tool adaptation for agents) rather than fundamental reconceptualizations.

Mitigation status: Not addressed. The paper does not discuss the tension between optimizing for community preference and recognizing genuinely novel ideas that challenge community consensus. The Limitations section mentions "better recognizing distinctive and diverse research ideas" as future work but does not frame this as a fundamental limitation of using citation counts as supervision. This is arguably the deepest limitation of the RLCF paradigm as formulated: it can learn what the community values, but it cannot learn to value what the community has not yet recognized. Human scientific taste, at its best, includes precisely this capacity—recognizing the importance of work that the current consensus undervalues—and it is unclear whether any training signal derived from community behavior can capture this without additional mechanisms (e.g., explicit modeling of idea novelty relative to existing literature, or training on historical cases where initially-ignored ideas were later recognized as foundational).

No Combination of Judgement and Ideation in a Closed Loop

The assumption: The paper treats judgement (Scientific Judge) and ideation (Scientific Thinker) as separate capabilities trained in sequence: Judge is trained first, then used as a frozen reward model to train Thinker (Section 3, Figure 2). The paper does not explore iterative refinement where Thinker's generated ideas are used to further train Judge, or where Judge's evaluations are used to filter or refine Thinker's outputs during generation rather than only as a post-hoc reward signal.

The consequence: The reported results represent a lower bound on what a fully integrated system could achieve. Several natural extensions are unexplored: (1) using Scientific Judge at inference time to rank or filter multiple candidate ideas from Scientific Thinker (best-of-N selection), (2) fine-tuning Scientific Judge on the distribution of generated ideas to reduce distribution shift between training (real papers) and deployment (generated proposals), (3) using Scientific Judge's reasoning traces to provide more granular feedback to Scientific Thinker (e.g., "your idea lacks novelty because it doesn't address limitation X" rather than just a binary win/loss), and (4) iterative training where improved Thinker outputs are used to construct new preference pairs for further Judge training. Any of these could substantially improve the reported win rates and accuracy, but the paper does not explore them.

Evidence in the paper: The paper's pipeline (Figure 2) is strictly linear: community feedback → Judge training → Thinker training. There is no feedback loop from Thinker back to Judge. The evaluation of Scientific Thinker uses LLM evaluators (GPT-5.2, GLM-5, Gemini 3 Pro) rather than Scientific Judge itself for the final quality assessment (Appendix C.2), which means the paper does not even test whether Judge and the external evaluators agree on generated idea quality. The only interaction between Judge and Thinker is the one-way reward signal during training (Section 3.3).

Mitigation status: Partially acknowledged. The paper does not explicitly identify this as a limitation, but the Discussion section is absent and the Limitations section focuses on other issues. The RLCF paradigm as described (Section 3) is a general framework that could accommodate iterative refinement, and the paper's choice to study a single forward pass through the pipeline is a scoping decision rather than a fundamental constraint. However, the absence of closed-loop experiments means the paper does not establish whether RLCF can create a virtuous cycle where better judgment enables better ideation, which in turn produces data that sharpens judgment—which would be the strongest demonstration that "scientific taste" has been learned rather than separately optimized.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the conversation around AI for scientific research from execution to judgment. Prior work on AI scientists focused overwhelmingly on automating research activities—literature search (Deep Research, Search-R1), experiment execution (Agent Laboratory, AI Scientist-v2), code generation (Codex, Claude Code)—treating the question of what to work on as either beyond automation or delegated to human researchers. This paper challenges that separation by demonstrating that scientific judgment—the capacity to assess the relative merit of research ideas—is not an irreducible human faculty but a learnable preference that can be extracted from community behavior at scale and transferred to generative models.

The shift is not that AI can now do something previously impossible—citation prediction from abstracts existed before this work (Zhao et al., 2025). The shift is methodological and conceptual. Methodologically, the paper introduces RLCF as a distinct training paradigm that occupies previously uninhabited territory between RLVR (which requires verifiable ground-truth answers) and RLHF (which relies on costly individual human annotations). By using naturally occurring community behavior as a supervision signal, RLCF demonstrates that capabilities previously thought to require explicit human mentorship—judging research quality, proposing impactful directions—can be trained from signals that already exist at massive scale in the scientific record. Conceptually, the paper reframes scientific taste from a philosophical puzzle (Hume, Kant) into an engineering objective formalized through JudgeCap and ThinkerCap (Equations 3–4), decomposing a capability that many researchers considered ineffable into a two-stage optimization pipeline.

The paper also reconciles a tension between two camps in the AI-for-science literature. On one side, human evaluation studies (Si et al., 2024, 2025) documented that LLMs can generate novel research ideas but "struggle to reliably distinguish potentially high-impact directions from ideas that are superficially novel but trivial"—establishing an ideation-execution gap. On the other side, bibliometric work (Zhao et al., 2025) showed that LLMs can predict citation counts from abstracts. The apparent contradiction—LLMs can predict which papers will be highly cited but cannot assess which ideas are promising—is resolved by this paper's pipeline: the citation prediction capability (judgment) can serve as a training signal for idea generation (ideation), closing the gap. Scientific Thinker's 81.5% win rate against its base policy (Figure 4a) and 54.2% average win rate against SOTA proprietary models (Table 8) provide the first empirical evidence that judgment capability can be leveraged to improve generative quality in scientific ideation.

The work redirects research attention in several concrete ways. It makes verifier quality the central bottleneck for scientific AI systems, parallel to how reward model quality determines RLHF effectiveness. Scientific Thinker's performance depends critically on the quality of Scientific Judge as a reward model: the +8.5-point in-domain gain from using SciJudge-Qwen3-4B over untrained Qwen3-4B-Instruct as the reward (Figure 4a) quantifies this dependency. This suggests that investment in better judgment models—larger scales, more diverse training data, multi-signal training—will have multiplicative effects on downstream ideation performance. It also makes community feedback signals a first-class design target. Rather than treating citations as an evaluation metric, researchers should consider what other naturally occurring community behaviors (download counts, code repository stars, conference acceptances, replication success, media coverage) could serve as scalable supervision for different dimensions of scientific taste.

Follow-Up Research This Work Enables

Training larger Scientific Judge models to test whether judgment scaling continues beyond 32B parameters. The paper shows monotonic improvement in SciJudgeBench accuracy from 1.5B (72.1%) through 32B (83.7%) in the Qwen2.5 family (Table 3), with the learning curves in Figure 3 not obviously saturating. A natural extension trains SciJudge at 70B or 100B+ scale—perhaps on Qwen2.5-72B or DeepSeek-V3-sized models—to test whether accuracy plateaus (suggesting the citation signal contains irreducible noise limiting performance to ~85–90%) or continues improving (suggesting scale is the primary bottleneck). The experiment should also measure whether judgment improvements at larger scales translate to better reward modeling for Scientific Thinker: the paper uses SciJudge-4B (75.3% accuracy) as the reward model for all Thinker training. Testing SciJudge-32B (83.7%) or larger as the reward model and measuring the marginal ideation improvement per unit of judgment accuracy would characterize the judge-thinker scaling relationship—a practical question for anyone building on this pipeline.

Systematic comparison of RLCF-trained ideation against alternative training paradigms on identical seed papers. The paper compares Scientific Thinker to untrained base policies and to policies trained with a weaker reward model (Figure 4), but does not benchmark against: (a) supervised fine-tuning on high-impact paper abstracts (train the base model to generate abstracts that resemble high-citation papers from the same seed papers), (b) RLHF using expert scientist pairwise preferences collected at small scale (50–100 researchers providing 500–1000 pairwise judgments on generated ideas), or (c) best-of-N sampling with post-hoc filtering by Scientific Judge (generate 64 ideas per seed paper, select the top-ranked by Judge, compare to the Thinker's single output). A head-to-head comparison at matched computational budgets would reveal whether RLCF's specific mechanisms (pairwise win-rate rewards, round-robin tournament structure) outperform simpler approaches to improving ideation quality. The paper's 81.5% win rate against the base policy establishes that training helps; this follow-up would establish which training approach helps most.

Extending Scientific Judge to provide structured critique rather than binary preference judgments. The current Scientific Judge outputs a reasoning trace followed by a binary choice (A or B). The reasoning traces in Appendix F suggest the model can articulate why it prefers one paper—topic breadth, downstream adoption, novelty, alignment with community trends—but this information is discarded during Scientific Thinker training, where only the binary win/loss is used as reward. A follow-up could fine-tune Scientific Judge to output a structured evaluation (e.g., separate scores for novelty, feasibility, potential impact, methodological soundness) or natural language critique, then test whether training Scientific Thinker with this richer feedback (e.g., "your idea scores low on novelty because it doesn't address limitation X") yields larger improvements than binary rewards at the same training budget. The key measurement is whether per-dimension feedback enables faster learning or better final performance, which would justify the additional complexity of structured reward modeling. This is analogous to how process reward models (PRMs) with per-step feedback outperform outcome reward models in mathematical reasoning—the same dynamic may hold for scientific ideation.

Multi-signal RLCF combining citations with complementary community feedback. Citations capture one dimension of scientific impact (downstream use and acknowledgment) but miss others: real-world influence (policy adoption, clinical guideline changes), educational value (textbook inclusion), methodological penetration (becoming standard practice without formal citation), and public engagement. A follow-up could construct additional preference signals from different community behaviors—download counts from arXiv, GitHub stars for associated code repositories, replication success rates, news media mentions, or expert surveys like the ML conference "most influential paper" awards—and test whether multi-signal RLCF produces judgment models that generalize better across metrics (extending the ICLR result in Table 6) or produce ideation models that generate ideas rated more highly on diverse quality dimensions. The experiment would measure whether combining signals mitigates the single-signal bias problem (e.g., a model trained only on citations might undervalue methods papers that become standard practice but aren't separately cited, while download counts or code repository stars might capture this dimension).

Human evaluation of Scientific Thinker's ideas with domain expert researchers. The paper's ideation evaluation relies entirely on LLM judges (validated at 84.4% on SciJudgeBench citation prediction, Appendix C.2), but the critical missing evidence is whether LLM evaluator preferences correspond to human expert judgments of idea quality. A follow-up study would recruit 20–30 active researchers in a specific field (e.g., NLP or ML systems) and ask them to evaluate 50–100 pairs of research ideas proposed by Scientific Thinker vs. the base policy (or vs. SOTA models), measuring human win rates and comparing them to LLM evaluator win rates. The null hypothesis is that LLM evaluators and human experts agree at rates comparable to SciJudgeBench validation (~84%); an alternative is that LLMs overvalue certain rhetorical features (technical jargon, confident framing, trend alignment) that human experts discount. A finding of substantial disagreement would fundamentally change how Scientific Thinker's results are interpreted—one would need to understand why LLM and human preferences diverge before claiming that Scientific Thinker produces genuinely better ideas. A finding of strong agreement would validate the LLM-based evaluation protocol and make large-scale ideation benchmarking feasible without expensive human studies.

Temporal dynamics modeling to identify delayed-impact papers missed by current citation thresholds. The paper's SciJudgeBench construction filters pairs using citation differences at a single snapshot (Appendix A.2), which means a paper that accumulated few citations by the snapshot date but later became highly cited is systematically mislabeled—the model is trained to prefer its less-impactful contemporary instead. A follow-up could use the arXiv corpus with citation time series (citation counts per year per paper) to train Scientific Judge to predict not "which paper has more citations now" but "which paper will have higher cumulative citations at 5-year and 10-year horizons," then test whether this temporally-aware judgment better identifies papers that were initially undervalued but later recognized. The experiment would measure: (a) whether temporally-aware training improves accuracy on the standard SciJudgeBench (suggesting that short-term citation patterns already capture long-term trends reasonably well), and (b) whether it specifically improves accuracy on pairs where the lower-cited paper at training time eventually overtakes the higher-cited paper (the delayed-impact case). This directly addresses the paper's stated limitation about modeling citation dynamics and would produce a Scientific Judge variant that is demonstrably better at the hardest judgment cases—distinguishing initially similar-quality papers whose trajectories diverge over time.

Practical Applications and Downstream Use Cases

Conference and journal preliminary screening. Scientific Judge's 87.7% accuracy on ICLR preference prediction (Table 6)—where it predicts which paper is more likely to be accepted based on peer review scores, despite being trained only on citation data—makes it immediately deployable as a triage tool for venues overwhelmed with submissions. A program chair could use Scientific Judge to rank submissions by predicted acceptance likelihood, flagging the bottom 20% for expedited desk rejection or the top 20% for guaranteed review, reducing reviewer workload while maintaining quality. The 87.7% accuracy means the model is correct on ~7 in 8 pairs, which is comparable to or better than inter-reviewer agreement at many conferences. The tool does not replace reviewers but reduces the number of papers requiring full review, freeing expert time for borderline cases where human judgment is most valuable. The field OOD results (Table 5: CS-trained model achieves 78.7% on Math, 76.9% on Physics) suggest a single model could serve multiple venues without per-venue fine-tuning, reducing deployment complexity.

Grant proposal and fellowship application prioritization. Funding agencies receive orders of magnitude more applications than they can fund, and preliminary screening often relies on coarse heuristics (track record of the PI, institutional affiliation) that encode systematic biases. Scientific Judge, operating on proposal abstracts (analogous to paper abstracts), could provide a citation-trained quality signal that is at least partially independent of these heuristics—the model evaluates the idea as described, not the person proposing it. The metric OOD result (Table 6) is particularly relevant here: Scientific Judge transfers from citation prediction to peer review alignment, suggesting it captures quality dimensions that reviewers also value. An agency could run all proposals through Scientific Judge, flag proposals where the model's ranking diverges substantially from reviewer rankings for program officer investigation (potential reviewer bias or genuinely unconventional proposals that deserve closer attention), and use model rankings as one input to a multi-factor decision process. The key implementation detail is that proposals, unlike published papers, have no actual results—the model would be judging promised contributions rather than demonstrated ones, which is a distribution shift from training. A pilot study comparing Scientific Judge rankings to eventual publication/citation outcomes of funded proposals would calibrate its reliability in this setting before operational deployment.

Research group ideation assistant. Scientific Thinker's 81.5% win rate against its base policy (Figure 4a) and 54.2% average win rate against SOTA models (Table 8) suggest it can serve as an ideation partner for research groups—particularly those in fast-moving fields where keeping up with all relevant work is difficult. A research group could feed their recent reading list (papers they found interesting) as seed papers to Scientific Thinker, generate 5–10 candidate follow-up ideas per paper, and use Scientific Judge to rank the candidates within each group, surfacing the most promising 2–3 for human evaluation. The out-of-domain generalization (83.0% win rate on August–December 2025 papers, Figure 4a) suggests the model remains effective even on very recent work. The key practical benefit is volume: Scientific Thinker can propose follow-up directions for 100 papers in the time it takes a researcher to deeply read 1–2, catching connections and extensions that a human might miss. The model's proposals would not be accepted uncritically—the 76.5–81.5% win rate means the model loses 18–24% of comparisons even against its own base policy, so human filtering is essential—but it increases the surface area of ideas that researchers can consider.

When to Prefer This Method

The paper articulates RLCF as a training paradigm for open-ended tasks where community consensus exists but verifiable ground-truth does not, and positions it as filling a gap between RLVR (math, code) and RLHF (subjective preferences). The choice of when to use RLCF depends on two conditions that the paper identifies explicitly:

• The task lacks verifiable ground-truth answers but has naturally occurring community behavior signals that encode aggregate preferences. Scientific ideation and judgment meet this condition: there is no unit test for "is this research direction promising," but citation counts provide a scalable community preference signal. Other tasks that might satisfy this condition include legal argument quality assessment (case law citations), clinical trial prioritization (trial registrations, regulatory approvals, subsequent citations), and policy proposal evaluation (legislative citations, implementation outcomes).

• Individual human annotations are insufficient because the relevant standard is community-level consensus rather than personal preference. The paper explicitly distinguishes RLCF from RLHF on this basis: "RLHF is limited by its reliance on costly human annotations and inability to reflect community-level preferences through individual preferences alone." RLCF is preferable when one reviewer's opinion is not a substitute for the aggregate judgment of a community over time.

• The community feedback signal can be converted into pairwise comparisons that control for confounds (field, time, venue prestige). The paper's field- and time-matching procedure (Appendix A.2) is the template: raw citation counts would encode trivial signals (older papers, CS papers get more citations), but within-field and within-time pairing isolates relative impact. Any application of RLCF to a new domain would need an analogous confound-control mechanism appropriate to that domain's signal structure.

The paper does not articulate a "prefer pretraining over RLCF" tradeoff analogous to the FLOPs-matched comparison in the compute-optimal test-time scaling paper (which this analysis references). The closest the paper comes is the scaling analysis (Figure 3, Table 3), which shows that larger base models achieve higher judgment accuracy after RLCF training (32B: 83.7% vs. 1.5B: 72.1%), suggesting that pretraining scale and RLCF fine-tuning are complementary rather than competing—larger models benefit more from RLCF, but RLCF also substantially improves small models (+65.1 points for the 1.5B). The paper does not provide the cost-benefit analysis (training compute, inference cost, accuracy gain) that would be needed to make a prescriptive tradeoff between "train a larger base model" and "apply RLCF to a smaller base model." The decision rule for practitioners is therefore: if community feedback signals exist for your domain and can be converted to pairwise preferences, RLCF is applicable alongside (not instead of) standard pretraining scaling, with the gains being largest when the base model has some initial capability that RLCF can amplify.