Beyond Length Scaling: Synergizing Breadth and Depth for Generative Reward Models

ArXiv: 2603.01571

🎯 Pitch

Current generative reward models blindly scale reasoning length, but this paper shows that how a model thinks—parallel breadth across principles versus sequential depth through self-solving—is far more critical. Strikingly, using the wrong structure for the task directly hurts performance, yet the proposed Mix-GRM learns to spontaneously polarize its reasoning style to match 95% of task demands, achieving state-of-the-art accuracy with a fraction of the training data.

---

1. Executive Summary

This paper introduces Mix-GRM, a framework that reconfigures raw Chain-of-Thought (CoT) rationales into two structured reasoning mechanisms—Breadth-CoT (B-CoT, parallel aggregation across diverse evaluation principles) and Depth-CoT (D-CoT, sequential judgment expansion grounded in self-solving reasoning traces)—and aligns them with task demands through Supervised Fine-Tuning and Reinforcement Learning with Verifiable Rewards (RLVR). Evaluated on five general reward benchmarks (RewardBench, RewardBench-v2, RMB, RM-Bench, PPE) using Qwen3-8B-Base, Mix-GRM establishes new state-of-the-art performance, surpassing leading open-source reward models by an average of 8.2% while achieving these gains with only 9K SFT samples—a fraction of the data required by prior work. The analysis reveals that B-CoT benefits subjective preference tasks whereas D-CoT excels in objective correctness—misaligning mechanism with task directly degrades performance—and demonstrates that RLVR acts as a switching amplifier, inducing emergent polarization where the model spontaneously allocates its reasoning style from a 73% structural match rate post-SFT to 95% post-RLVR, establishing that optimizing how a model thinks proves more critical for post-training efficacy than simply scaling how long it writes.

2. Context and Motivation

The Core Problem: We Don't Know How to Think When We Judge

The fundamental question this paper tackles is deceptively simple: when a language model evaluates the quality of another model's output, what kind of reasoning process should it use? This matters because the field is rapidly shifting from scalar reward models (which output a single number like "score = 0.73") to Generative Reward Models (GRMs) that produce explicit natural language rationales before rendering a verdict. The implicit assumption driving this shift—and driving much of the broader CoT literature—is that lengthening the reasoning chain improves evaluation reliability. More words, more principles, more critique means more accurate judgments, or so the thinking goes.

This paper argues that this assumption is not just oversimplified—it is actively wrong in important ways. The central claim is that the structure of reasoning, not just its length, determines evaluation quality, and that the optimal structure depends fundamentally on what kind of task is being evaluated. The paper provides empirical evidence for what prior work had only hinted at theoretically: that different reasoning mechanisms—parallel breadth across multiple principles versus sequential depth through deductive verification—have divergent, sometimes opposite, effects depending on whether the task involves subjective human preferences or objective correctness.

Why This Problem Is Important

The significance of this gap manifests along three dimensions: the changing nature of RLHF pipelines, the economics of reward model training, and a fundamental theoretical question about reasoning itself.

The RL alignment bottleneck. Reinforcement learning from human feedback (RLHF) has become the dominant post-training paradigm for aligning LLMs with human values. The reward model sits at the heart of this pipeline—it is the proxy for human judgment that guides policy optimization. As the ambition of RL expands from narrow domains like math problem-solving to general-purpose alignment across chat, coding, safety, and instruction-following, the reward model faces an increasingly complex evaluation landscape. A single scalar score cannot adequately capture why a response about Afghan-Pakistani relations is better than another, or why one code solution is correct while another contains a subtle logical flaw. GRMs attempt to solve this by making the evaluation process explicit and interpretable.

However, the paper identifies a critical unresolved question: does a GRM's reasoning strategy generalize across all evaluation tasks, or does it need to adapt? If a GRM trained with one reasoning style (say, checking many surface-level features in parallel) excels at subjective preference judgments but fails at objective correctness verification, then using it as a universal reward signal for RL would systematically disadvantage either helpfulness or correctness in the resulting policy. The downstream consequences are concrete: a policy model optimized against a reward model with mismatched reasoning could become superficially eloquent but logically sloppy, or rigorously correct but unhelpfully rigid. The paper's DPO experiments in Table 3 demonstrate exactly this practical concern—different reward models produce different tradeoffs between instruction-following and mathematical reasoning in the resulting policy.

The economics of training data efficiency. A second practical concern is the sheer volume of data required by current GRM approaches. The paper's Table 1 shows that FARE-8B requires approximately 2.5 million training samples to achieve an average benchmark score of 75.9, while Mix-GRM reaches a comparable 75.1 using only 9,000 SFT samples. This is not a trivial efficiency gain—it represents roughly a 275× reduction in required training data. In a landscape where high-quality human preference annotations are expensive and time-consuming to collect, the difference between needing thousands of samples versus millions has direct economic consequences. The paper suggests that this efficiency stems from optimizing what the model learns (structured reasoning mechanisms) rather than how much it sees (raw data volume), which has implications for the entire field's approach to reward model development.

A theoretical puzzle about reasoning itself. Beyond practical considerations, the paper addresses a deeper question that has been building in the CoT literature: are different types of reasoning structurally incompatible, or can they be synergistically combined? Prior work on parallel thinking (Skeleton-of-Thought, Self-Consistency) and sequential thinking (Tree of Thoughts, step-by-step verification) has demonstrated that each approach excels in different domains, but these findings existed in separate research threads. The paper's central theoretical contribution is demonstrating not just that breadth and depth are domain-specific, but that they can be unified within a single model that learns to switch between them. This transforms the question from "which reasoning structure is best?" to "how do we build models that can fluidly deploy the right structure for the right task?"—a significantly more challenging and interesting scientific problem.

Where Prior Approaches Fall Short

The paper identifies specific limitations in three categories of prior work:

Length-scaling approaches ignore structural diversity. The dominant research trajectory in GRMs has been extending CoT length through two primary mechanisms. The first is RL-based elicitation, exemplified by JudgeLRM (Chen et al., 2025), RM-R1 (Chen et al., 2026), and DeepSeek-GRM (Liu et al., 2025c), which use reinforcement learning to incentivize longer reasoning traces. The second is synthetic data expansion, exemplified by FARE-8B (Xu et al., 2026), Prometheus (Kim et al., 2024), and rubric-based approaches (Liu et al., 2026; Gunjal et al., 2026), which generate detailed checklists or multi-perspective critiques to expand evaluation coverage.

The paper's position, stated directly in the introduction, is that these strategies "typically rely on static, task-agnostic structures, overlooking the critical nuance that the optimal reasoning mechanism is intrinsically task-dependent." The evidence for this claim appears throughout their experiments: Table 2 shows that RubricRM-8B—which uses a static rubric template—achieves 68.6 on Preference tasks but only 71.3 on Correctness tasks, while JudgeLRM achieves somewhat complementary scores. Neither static approach excels across both domains simultaneously because neither adapts its reasoning structure to what is being evaluated.

The paper makes this critique more pointed through its case studies in Table 5. A vanilla CoT (unstructured, length-scaled) fails on a preference task by fixating on historical detail while missing a language mismatch, and fails on a correctness task by being fooled by superficial comprehensiveness (longer response = better response). These failures are not due to insufficient reasoning length—both vanilla CoTs are extensive—but due to reasoning misdirection: attending to the wrong kinds of features for the task at hand.

CoT structural research has not been applied to reward modeling. The paper draws on a rich literature exploring structured reasoning topologies—Tree of Thoughts (Yao et al., 2023), Graph of Thoughts (Besta et al., 2025), Skeleton-of-Thought (Ning et al., 2024), and Self-Consistency (Wang et al., 2023)—but notes that none of this work has been systematically adapted to the reward modeling context. This is a non-trivial gap because reward modeling differs from task-solving in a crucial way: the model is not generating a solution to a problem but evaluating the quality of someone else's solution. The reasoning required is meta-cognitive—it requires assessing correctness, helpfulness, safety, and dozens of other dimensions simultaneously. The structural demands of this meta-evaluation may differ substantially from the structural demands of primary task-solving, yet the field has largely imported CoT techniques without re-examining their suitability.

Scalar reward models provide no interpretable reasoning. The paper briefly acknowledges the dominant discriminative approach represented by Skywork-Reward (Liu et al., 2024), a Bradley-Terry model that produces scalar scores. While effective—Skywork-Reward achieves 93.9 on RewardBench-v1—scalar models offer no insight into why one response is preferred over another. This opacity is problematic for debugging alignment failures, for providing feedback to policy models during training, and for building trust in the evaluation process. The GRM approach addresses this by making the reasoning explicit, but inherits the structural limitations discussed above.

How This Paper Positions Itself

Mix-GRM positions itself at the intersection of two previously disconnected research threads: the GRM literature's focus on generating evaluation rationales and the CoT literature's exploration of structured reasoning topologies. The paper's framing is that GRM development has been operating under an implicit "longer is better" assumption that ignores structural considerations, and that importing insights from CoT structure research can yield substantial improvements in both accuracy and data efficiency.

The paper draws an explicit parallel to the test-time scaling literature's distinction between parallel and sequential thinking (Zhang et al., 2026; 2025b). This connection is more than rhetorical—it provides a theoretical justification for why breadth and depth would be domain-specific. Parallel thinking (breadth) explores multiple reasoning paths simultaneously, which is valuable when the evaluation criterion is multi-dimensional and no single dimension dominates. Subjective preference is exactly this case: a "good" response must simultaneously satisfy helpfulness, appropriateness, clarity, and creativity, and failure on any dimension can be disqualifying. Sequential thinking (depth) verifies logical dependencies step by step, which is valuable when correctness depends on a chain of deductive reasoning where any broken link invalidates the conclusion. Objective correctness in math and code is exactly this case: a single arithmetic error or logical flaw renders the entire solution incorrect regardless of how well-written it is.

The paper's methodological innovation is not discovering that breadth and depth exist—the CoT literature already explored these concepts—but rather operationalizing them within a unified reward modeling framework through three specific mechanisms: (1) modular schema standardization that decomposes unstructured rationales into atomic Principle-Judgment-Verdict units, enabling clean structural manipulation; (2) dual-track synthesis that reconstructs these units into B-CoT (via parallel aggregation and deduplication) or D-CoT (via reasoning-grounded judgment regeneration); and (3) mechanism-adaptive alignment through SFT on a mixture dataset followed by RLVR that, crucially, does not explicitly label which mechanism to use—the model learns this autonomously through reward optimization.

The paper also positions itself against the data-scaling paradigm. Where FARE-8B uses 2.5M samples and DeepSeek-GRM-16B uses 1.2M SFT + 237K RL samples, Mix-GRM uses only 9K SFT + 21K RLVR samples—a deliberate demonstration that structural optimization can substitute for data volume. This directly challenges the implicit assumption in much of the GRM literature that progress comes primarily from scaling up training data, whether human-annotated or synthetically generated.

Finally, the paper's RLVR analysis adds a novel dimension to the reward modeling literature. While prior work used RL to elicit longer CoT traces (JudgeLRM, RM-R1), Mix-GRM uses RLVR not to lengthen reasoning but to sharpen mechanism allocation—the model learns to spontaneously deploy B-CoT for preference tasks and D-CoT for correctness tasks without explicit structural supervision during RL training. This represents a shift from "RL to write more" to "RL to think better," and the 73% → 95% structural match rate improvement provides compelling evidence that this effect is real and substantial.

3. Technical Approach

3.1 Reader Orientation

Mix-GRM is a reward modeling framework that trains a language model to act as an automatic evaluator — it reads a user instruction and two candidate responses, then writes out an explicit justification followed by a verdict (e.g., "Response A is better"). The core problem it solves is that existing Generative Reward Models (GRMs) use unstructured, task-agnostic reasoning: they write long critiques without adapting how they think to what they are evaluating. Mix-GRM's solution is to reconfigure raw rationales into two structurally distinct reasoning mechanisms — Breadth-CoT (parallel scanning of many evaluation dimensions) and Depth-CoT (step-by-step deductive verification) — and then train the model to autonomously deploy the right mechanism for the right task through a two-stage process of Supervised Fine-Tuning followed by Reinforcement Learning with Verifiable Rewards.

3.2 Big-Picture Architecture (Diagram in Words)

The Mix-GRM system has five major components:

1. Schema Standardizer — takes an unstructured rationale (a free-form paragraph explaining why one response is better) and decomposes it into atomic "Principle–Judgment–Verdict" triples. Each triple encodes one evaluation dimension, the specific analysis on that dimension, and a sub-verdict.

2. Breadth-CoT Synthesizer — samples multiple independent rationales from the LLM, parses each into atomic triples, then merges and deduplicates them across rationales to produce a comprehensive, non-redundant evaluation covering many distinct principles in parallel.

3. Depth-CoT Synthesizer — first prompts the LLM to solve the instruction itself (producing a "reasoning trace" that outlines the correct solution path), then regenerates judgments for a focused subset of principles by grounding them in this trace, ensuring the evaluation verifies logical correctness rather than surface features.

4. Mixture Dataset Constructor — pairs B-CoT rationales with subjective preference tasks (where quality is multi-dimensional) and D-CoT rationales with objective correctness tasks (where logical rigor matters), producing a training corpus that teaches the model both reasoning mechanisms in their appropriate contexts.

5. Mechanism-Adaptive Alignment Trainer — performs SFT on the mixture dataset to initialize the model, then applies RLVR (Group Relative Policy Optimization) using only verdict-consistency rewards. Critically, the RLVR stage provides no explicit guidance about which reasoning mechanism to use — the model spontaneously learns to deploy B-CoT for preference tasks and D-CoT for correctness tasks by discovering that this alignment maximizes reward.

Information flows as follows: raw training rationales enter → Schema Standardizer decomposes them into atomic triples → these triples are routed to either the B-CoT or D-CoT synthesizer depending on the task domain → the synthesized structured rationales are paired with their corresponding tasks to form the mixture dataset → SFT initializes the model on this mixture → RLVR optimizes the model's verdict accuracy, which indirectly sharpens mechanism allocation → at inference time, the trained model receives an instruction and two responses, autonomously selects the appropriate reasoning structure, generates an evaluation rationale, and outputs a final verdict.

3.3 Roadmap for the Deep Dive

• First, the problem formulation — the mathematical notation for what a GRM does, which establishes the input/output contract and provides a common language for discussing the transformations that follow.
• Second, the Schema Standardizer — how unstructured rationales are decomposed into modular Principle–Judgment–Verdict triples, since this atomic decomposition is the foundation that enables the subsequent structural manipulation. Without understanding these building blocks, the synthesis mechanisms are unmotivated.
• Third, the B-CoT and D-CoT synthesis mechanisms — the detailed procedures for constructing each reasoning type, including the design rationale for why parallel aggregation suits preference tasks and why reasoning-grounding suits correctness tasks. These are the paper's core methodological contributions.
• Fourth, the mixture dataset construction and training protocol — how the synthesized data is organized into a training corpus and the two-stage SFT+RLVR training procedure. This section explains why RLVR is called a "switching amplifier" and how the model achieves mechanism-adaptive alignment without explicit structural supervision.
• Fifth, the key design choices and their justifications — a consolidated explanation of why certain decisions were made (e.g., using N=2 independent samples for B-CoT, selecting the top-10 most frequent principles, using a self-solving trace for D-CoT grounding), which ties together the architectural decisions with the empirical results presented in Section 5.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a methodological framework paper that introduces a data synthesis and training pipeline for building Generative Reward Models that can dynamically switch between two structurally distinct reasoning mechanisms. The core insight is that evaluation reasoning should not be a monolithic, unstructured text block — it should be reconfigured into task-appropriate structures that either scan broadly across many evaluation dimensions (Breadth-CoT) or drill deeply through deductive verification (Depth-CoT), and the model should learn to deploy these structures autonomously.

---

Problem Formulation

The paper begins by establishing the formal notation for what a Generative Reward Model does, which serves as the contract that all subsequent components must satisfy.

Given a task instruction $x$ and two candidate responses $\{y_A, y_B\}$ generated by two different assistants, a standard GRM $\mathcal{M}$ produces an output sequence consisting of an explicit evaluation rationale $c$ followed by a preference verdict $v$:

$(c, v) = \mathcal{M}(y_A, y_B \mid x)$

where $c$ is the Chain-of-Thought reasoning rationale (a natural language explanation of why one response is better than the other), $v$ is the final preference verdict (the model's decision about which response is preferred), and $\mathcal{M}$ is the GRM's generation function parameterized by its learned weights.

What it computes: the function takes an instruction and two candidate responses as input, runs the GRM's generative process to produce a text string that contains both a justification and a conclusion, and returns this pair. The rationale $c$ is the model's attempt to make its evaluation process transparent and interpretable, while $v$ is the actionable output that downstream systems (like RLHF pipelines) use to guide policy optimization.

Why this form: this formulation captures the fundamental distinction between traditional scalar reward models (which would output only $v$ or a scalar score) and Generative Reward Models (which output $c$ alongside $v$). The presence of $c$ is what makes GRMs "generative" — they produce human-readable justifications that can be inspected, audited, and used for debugging alignment failures. The paper's innovation is not to change this interface but to change how $c$ is structured internally, which the decomposition into atomic units enables.

For convenience, the paper also defines the full input triplet as:

$I = (x, y_A, y_B)$

This shorthand is used throughout the training objective definitions to keep notation compact.

---

Modular Schema Standardization

The first stage of the Mix-GRM pipeline transforms unstructured rationales into a standardized, modular format. The motivation is two-fold: first, raw rationales generated by standard GRM training procedures are typically free-form paragraphs that mix many evaluation dimensions together in a tangled narrative, making it impossible to systematically manipulate the reasoning structure; second, prior work (Viswanathan et al., 2025) has shown that decomposing complex evaluations into atomic checklist items improves the reliability and interpretability of the evaluation process.

The decomposition procedure. The paper uses a large language model (DeepSeek-V3, 0324 snapshot, temperature $T=0.8$) to parse an existing unstructured rationale $c$ into a set of structured atomic units $\mathcal{S}$:

$\mathcal{S} = \{(p_k, j_k, v_k)\}_{k=1}^{K}$

where $p_k$ denotes the $k$-th discrete evaluation Principle (a named dimension of evaluation such as "Instruction Adherence" or "Linguistic Alignment"), $j_k$ represents the $k$-th specific Judgment (the concrete analysis of how the two responses perform on that principle, e.g., "Response B directly addresses the user's question in the requested language while Response A provides a detailed but language-mismatched explanation"), $v_k$ is the $k$-th Sub-Verdict (a local decision on that principle, e.g., "<B> is Better" or "<A> is Better"), and $K$ is the total number of extracted units, which the paper states typically ranges from 3 to 5.

What it computes: the LLM reads a piece of free-form text that explains why one response is better than another, identifies the distinct evaluation dimensions being discussed, separates the reasoning about each dimension into its own unit, labels each unit with a named principle, and extracts a local verdict for that principle. The output is a set of structured triples where each triple is a self-contained mini-evaluation on one dimension.

Why this form: the atomic decomposition serves three critical purposes for the downstream synthesis pipeline. First, it creates modularity — individual principles can be rearranged, combined across different rationales, or selected for focused re-examination without losing the coherence of the overall evaluation. Second, it enforces syntactic uniformity — the paper explicitly notes that this ensures "performance gains are driven by thinking mechanisms (i.e., Breadth vs. Depth) rather than superficial stylistic patterns." Without this standardization, any observed differences between B-CoT and D-CoT could be attributed to differences in writing style or formatting rather than genuine structural differences in reasoning. Third, it enables granular verifiability — each principle-judgment-verdict triple can be independently checked for correctness, which is important for the synthesis quality control described in Appendix E.

What the triple structure represents concretely. A Principle is not just a topic label — it is a specification of the evaluative lens through which a response pair is being compared. The Judgment is the evidence or analysis that justifies a preference under that lens. The Sub-Verdict is the directional preference. Together, these three elements form a complete mini-argument: "under criterion X (Principle), we observe Y (Judgment), therefore Z is better (Sub-Verdict)." The paper's case studies in Table 5 provide concrete examples: in Case 1, one triple has Principle = "Linguistic Alignment," Judgment = "Assistant B's response is in Japanese matching the user's language while Assistant A responds in English," and Sub-Verdict = "<B> is Better."

Relationship to prior work. This schema draws on the checklist-based evaluation paradigm (Viswanathan et al., 2025), which demonstrated that forcing models to evaluate against explicit checklists improves alignment quality. However, the paper's use of atomic triples is more structured than a simple checklist — it captures not just what to check (the principle) but what was found (the judgment) and what conclusion follows (the sub-verdict). This richer structure is essential for the synthesis mechanisms that follow, which need to know not just which principles were checked but what evidence was marshaled under each principle.

---

Mechanism Synthesis: B-CoT Construction

With the atomic triples extracted, the paper constructs two structurally distinct types of Chain-of-Thought reasoning. The first is Breadth-CoT (B-CoT), designed for subjective preference evaluation where quality is defined by simultaneous satisfaction of multiple, potentially independent dimensions.

The design rationale. In subjective preference tasks, a "good" response is not defined by any single criterion but by the joint satisfaction of many criteria simultaneously. A response might be helpful but impolite, creative but irrelevant, or thorough but incomprehensible. Single-track reasoning — writing one long paragraph that discusses whatever comes to mind — risks fixating on a few dominant traits while overlooking subtle but important dimensions. The paper argues that parallel thinking, which explores multiple reasoning paths concurrently, provides a "deliberative breadth" that aligns with the multifaceted nature of human preference.

The synthesis procedure. B-CoT is constructed through a three-step process of sampling, decomposition, and aggregation:

Step 1 — Parallel Sampling. The paper samples $N$ independent rationales $\{c_n\}_{n=1}^{N}$ from the LLM, where each $c_n$ is generated by prompting the model to evaluate the same instruction-response pair but with independent stochastic runs (achieved through temperature sampling at $T=0.8$). The paper treats these independent samples as "a stochastic exploration of the instruction's evaluative manifold" — each run may attend to slightly different aspects of the responses, surfacing principles that might remain dormant in a single run.

The paper uses $N=2$ for its main experiments (as shown in Table 4, B-CoT uses 2 reasoning passes), and the ablation study in Figure 4(a) shows that performance improves monotonically as $N$ increases from 1 to 4, confirming that more parallel samples yield broader principle coverage. The choice of $N=2$ balances coverage against computational cost — additional samples provide diminishing returns as the principle space becomes saturated.

Step 2 — Decomposition. Each sampled rationale $c_n$ is parsed through the Schema Standardizer described in Section 3.4 (Modular Schema Standardization) to produce a set of atomic triples $\mathcal{S}_n = \{(p_k, j_k, v_k)\}$. This yields $N$ sets of structured evaluation units, each set representing one "cognitive trajectory" through the evaluation space.

Step 3 — Merge and Deduplicate. The $N$ sets of triples are unified through an LLM-based transformation $\mathcal{T}_{\text{merge}}$:

$C_B = \mathcal{T}_{\text{merge}}\left(\bigcup_{n=1}^{N} \{(p, j, v) \in \mathcal{S}_n\}\right)$

where $\mathcal{T}_{\text{merge}}$ is a prompt-guided LLM operation that takes all extracted principles from all rationales, identifies principles that refer to the same underlying evaluation dimension (even if phrased differently), merges them into a single coherent unit, and filters out principles that appear infrequently (lowest-frequency principles are discarded). The paper explicitly states that $\mathcal{T}_{\text{merge}}$ is "limited solely to merging and deduplication" and does not introduce new information, so it does not constitute an additional reasoning pass.

What it computes: the union of all principle-judgment-verdict triples from $N$ independent evaluations is taken, redundant or overlapping principles are collapsed into single entries, and low-frequency principles (those that appeared in few or only one of the $N$ rationales) are removed. The output $C_B$ is a single, consolidated CoT text that covers a comprehensive and non-redundant spectrum of evaluation principles. Each principle section in $C_B$ retains its original Judgment and Sub-Verdict content from the source rationale, ensuring the evaluation is grounded in actual analysis rather than abstract principle names.

Why this form: the merge-and-deduplicate approach addresses the fundamental challenge of breadth-oriented evaluation — the risk of redundancy and noise. Without deduplication, the aggregated rationale would contain multiple sections essentially discussing the same principle in slightly different words, bloating the CoT length without adding informational value and potentially confusing the model during training by presenting inconsistent Sub-Verdicts on the same principle. Without filtering low-frequency principles, idiosyncratic or poorly-supported evaluation dimensions would be included, introducing noise into the training signal. The paper's ablation in Figure 4(b) confirms this: using the full pool of principles ("Breadth Full") improves over random selection but is outperformed by selecting the top-10 most frequently appearing principles ("Breadth Top-10"), which the paper characterizes as a "denoising effect where low-frequency principles introduce noise, while high-frequency ones form a more robust 'reasoning consensus.'"

The consensus mechanism. The "top-10" selection strategy is particularly interesting because it operationalizes a form of inter-rationale agreement. Principles that appear across multiple independent evaluations are more likely to be genuinely relevant to the task rather than artifacts of a particular sampling run. By retaining only principles that achieve consensus (defined implicitly by frequency of appearance), the synthesis effectively performs a vote: if multiple independent evaluations agree that "Instruction Adherence" is a relevant dimension for this task, it is included; if only one evaluation mentions "Aesthetic Quality" and others do not, it is likely spurious and is filtered out. This consensus mechanism gives B-CoT a built-in robustness that a single-pass evaluation lacks.

The final B-CoT structure. The synthesized $C_B$ is a long-form text that reads as a structured evaluation with multiple clearly delineated sections, each corresponding to one distinct principle. While the paper does not show a complete B-CoT example in the main text, the Case Study in Table 5 provides a compressed illustration: the B-CoT in Case 1 lists separate principles (Linguistic Alignment, Contextual Nuance, Cultural Sensitivity) with individual Sub-Verdicts under each, culminating in a Final Verdict that aggregates across all principles.

---

Mechanism Synthesis: D-CoT Construction

The second reasoning mechanism is Depth-CoT (D-CoT), designed for objective correctness evaluation where quality depends on rigorous logical constraints rather than multi-dimensional coverage.

The design rationale. In objective correctness tasks (math, code, factual accuracy), a "good" response is one that is logically valid — the conclusion follows from correct premises through valid inference steps. Surface-level features like fluency, formatting, or apparent comprehensiveness are irrelevant to correctness (and can be actively misleading, as the Case Study 2 in Table 5 demonstrates where a longer, more detailed response contained a fundamental reactivity error). Normal evaluation rationales often fall prey to "superficial shortcuts" — they judge responses based on how professional or thorough they look rather than whether the underlying logic is sound. The paper argues that sequential thinking, which verifies logical dependencies step by step, provides a "deductive rigor" that naturally aligns with the strict requirements of objective correctness.

The synthesis procedure. D-CoT is constructed through a four-step process that fundamentally differs from B-CoT in both its information sources and its structural logic:

Step 1 — Reasoning Trace Elicitation. The LLM is first prompted to generate a Reasoning Trace $z$ — a self-solving pass derived from the instruction $x$ that explicitly outlines the optimal solution path required for a correct response. This is NOT an evaluation of the candidate responses — it is the model's own attempt to solve the task, independent of what Response A or Response B says.

For a math problem, $z$ would be the correct step-by-step derivation of the answer. For a coding task, $z$ would be the correct implementation logic. For a factual question, $z$ would be the correct answer with supporting evidence. The paper describes this as an "explicit" solution path, meaning the model produces the answer itself before evaluating whether the candidates' answers match.

Step 2 — Focused Principle Selection. Unlike B-CoT, which aggregates across many principles, D-CoT intentionally focuses on a narrow subset $\mathcal{S}_{\text{sub}} \subset \mathcal{S}$ of the principles extracted by the Schema Standardizer. The paper specifies that $|K| \leq 3$ principles are selected — a deliberate trade-off of "horizontal coverage for deductive rigor." The intuition is that depth-oriented reasoning requires higher cognitive load per unit (each judgment must be carefully verified against the reasoning trace), and spreading this verification across many principles would dilute the quality of each verification.

Step 3 — Reasoning-Guided Judgment Regeneration. This is the core mechanism that distinguishes D-CoT from both raw rationales and B-CoT. For each selected principle $p_k$, the original judgment $j_k$ (which was generated during the initial unstructured evaluation) is discarded and replaced with a new judgment $\tilde{j}_k$ that is generated by an LLM transformation conditioned on the reasoning trace:

$\tilde{j}_k = \mathcal{T}_{\text{refine}}(p_k \mid z)$

where $\mathcal{T}_{\text{refine}}$ is a prompt-guided operation that takes a principle name and the reasoning trace as input, and produces a new judgment that explicitly grounds the evaluation of that principle in the verified solution path. The transformation does not simply check whether the response "looks good" on principle $p_k$ — it checks whether the response's claims on principle $p_k$ are consistent with what the reasoning trace $z$ says should be true.

What it computes: for each selected evaluation principle, the original analysis (which might have been based on surface features like "the response is detailed and well-structured") is thrown away, and a completely new analysis is generated from scratch using the reasoning trace as ground truth. The new judgment asks: "Given that the correct solution is $z$, does the response get this principle right or wrong?" The output $\tilde{j}_k$ is a judgment that is explicitly anchored in verified correctness rather than in heuristic impressions.

Why this form: the key design decision is the discarding of the original judgment. If the original $j_k$ were retained and merely "checked" against $z$, the resulting evaluation might be biased by the original reasoning's errors — the model might look for evidence confirming its existing judgment rather than re-evaluating from first principles. By regenerating $j_k$ from scratch conditioned on $z$, the model is forced to ground its evaluation in the verified solution rather than in its initial (potentially flawed) assessment. This implements what the paper calls "substantive reasoning soundness" — the evaluation is sound because it derives directly from a verified logical foundation, not because it's long or detailed.

Step 4 — Trace Injection and Serialization. To ensure the evaluative process is transparent and explicitly grounded in the model's own logic, the reasoning trace $z$ is injected directly into the lead judgment unit $\tilde{j}_1$. This means the final D-CoT text opens with the model's own solution to the instruction, followed by the principle-by-principle verification that references this solution. The inject step creates what the paper describes as an "evaluative process [that] is transparent and explicitly grounded in the model's own logic" — a reader of the D-CoT can see exactly what the model thinks the correct answer is and how it uses that answer to assess the candidate responses.

The final $C_D$ is constructed by serializing these refined units into a single CoT text, ordered so that the reasoning trace comes first and the principle verifications follow. The structure reads as: "Here is how to solve this problem correctly. Now, checking Response A and Response B against this solution: on Principle 1, ... on Principle 2, ... Final Verdict."

Why D-CoT differs from B-CoT in structural logic. B-CoT expands horizontally — it adds more principles to ensure no important dimension is missed. D-CoT expands vertically — it keeps the number of principles small but deepens the analysis under each principle by grounding it in verified reasoning. The two mechanisms are complementary by design: B-CoT would be inappropriate for correctness tasks because it risks including superficially appealing but logically irrelevant principles (as the Case Study shows, B-CoT on a chemistry problem fixated on "Comprehensive Option Analysis" and "Informative Detail" while missing the reactivity ordering error), while D-CoT would be inappropriate for preference tasks because its narrow focus would miss important but subtle dimensions (as the Case Study shows, D-CoT on a language preference task tunneled on historical content analysis while missing the fundamental language mismatch).

Computational parity. The paper emphasizes in Table 4 that both B-CoT and D-CoT require approximately 2 reasoning passes during synthesis: B-CoT samples $N=2$ rationales then merges them, while D-CoT generates one reasoning trace then synthesizes the final CoT. The merge operation in B-CoT does not count as an additional reasoning pass because it only reorganizes existing information. This parity is important for the paper's claim that performance differences stem from structural efficacy rather than raw compute disparities.

---

The Mixture Dataset and Mechanism-Adaptive Alignment

Having defined the two reasoning mechanisms, the paper describes how they are combined into a training corpus and how the model is trained to deploy them appropriately.

Mixture dataset construction. The training data is organized along two task domains following the categorization in Frick et al. (2025):

• Preference Domain: Subjective evaluation tasks where quality is multi-dimensional (chat quality, helpfulness, harmlessness, instruction-following in open-ended scenarios, safety, multilingual appropriateness). The paper assigns B-CoT rationales to these instances because the parallel aggregation of multiple principles aligns with the need for comprehensive coverage across diverse evaluation dimensions.

• Correctness Domain: Objective evaluation tasks where quality depends on logical validity (math problem-solving, code generation, factual accuracy, STEM reasoning). The paper assigns D-CoT rationales to these instances because the sequential verification grounded in a reasoning trace aligns with the need for deductive rigor.

The resulting mixture dataset $\mathcal{D}_{\text{mix}}$ contains both B-CoT and D-CoT rationales, each paired with the task domain for which its structure is appropriate. The paper constructs this dataset from 30,000 total samples (9K for SFT, 21K for RLVR) spanning five source datasets detailed in Appendix Table 8: HelpSteer3 (general chat, STEM, code, multilingual — 14,876 samples), Code-Preference (4,000), Math-DPO (4,000), WildGuard (safety — 4,000), and OffsetBias (instruction following — 4,000).

Why mixture rather than separate models. An alternative design would train two separate RMs — one with B-CoT for preference tasks, one with D-CoT for correctness tasks — and route evaluation requests to the appropriate model at inference time. The paper's mixture approach is more ambitious: it trains a single model to internalize both mechanisms and to autonomously decide which to deploy based on the nature of the task. This is what the paper means by "mechanism-adaptive alignment" — the model itself learns the mapping from task characteristics to reasoning structure, rather than relying on an external router.

SFT Initialization. The policy $\pi_\theta$ (the GRM being trained) is first initialized via Supervised Fine-Tuning on the mixture dataset $\mathcal{D}_{\text{mix}}$. Given the input triplet $I = (x, y_A, y_B)$ and the corresponding structured rationale $c \in \{c_B, c_D\}$, the model is trained via standard next-token prediction to generate the rationale $c$ followed by the verdict $v$. The training objective is standard language modeling cross-entropy, but the target sequences contain explicit structural markers that teach the model to produce either breadth-oriented or depth-oriented evaluations.

Hyperparameters for SFT (from Appendix Table 6):

• Epochs: 2
• Learning rate: $2 \times 10^{-5}$
• Batch size: 128 (with gradient accumulation steps = 16, implying a micro-batch of 8)
• Maximum sequence length: 12,288 tokens
• Weight decay: 0.0
• Warmup: 5% linear warmup

The 12,288 token sequence length is notable — it is substantially longer than typical SFT lengths, reflecting the fact that both B-CoT and D-CoT produce long, structured rationales that can span thousands of tokens.

Why SFT alone is insufficient. The SFT stage teaches the model what structured reasoning looks like and provides initial domain-mechanism associations (preference → B-CoT, correctness → D-CoT). However, the paper's analysis reveals that post-SFT, the model achieves only a 73% structural match rate — meaning it deploys the correct reasoning mechanism for the correct domain only 73% of the time. The remaining 27% of the time, it uses the wrong structure or falls back to unstructured reasoning. This incomplete internalization motivates the second training stage.

RLVR via GRPO. To optimize verdict accuracy and sharpen mechanism allocation, the paper employs Reinforcement Learning with Verifiable Rewards using the Group Relative Policy Optimization algorithm. The objective function is:

$\mathcal{J}_{\text{GRPO}}(\theta) = \mathbb{E}_{\substack{I \sim \mathcal{D} \\ \{o_i\} \sim \pi_{\theta_{\text{old}}}}} \left[ \frac{1}{G} \sum_{i=1}^{G} \left( \frac{\pi_\theta(o_i \mid I)}{\pi_{\theta_{\text{old}}}(o_i \mid I)} \hat{A}_i - \beta \mathbb{D}_{\text{KL}}(\pi_\theta \mid\mid \pi_{\text{ref}}) \right) \right]$

where $\theta$ are the current policy parameters being optimized, $\theta_{\text{old}}$ are the policy parameters from the previous iteration (used for importance sampling), $\pi_\theta(o_i \mid I)$ is the probability of the model generating output sequence $o_i$ given input $I$ under the current policy, $\pi_{\theta_{\text{old}}}(o_i \mid I)$ is the same probability under the reference (old) policy, $\hat{A}_i$ is the advantage estimate for output $o_i$ (how much better or worse this output is compared to the group average), $G$ is the number of rollouts per input (the paper uses $G = 8$, sampling with temperature 0.8 via vLLM), $\beta$ is the KL divergence penalty coefficient (set to 0.001), $\mathbb{D}_{\text{KL}}(\pi_\theta \mid\mid \pi_{\text{ref}})$ measures how far the current policy has diverged from a reference policy, and the expectation is taken over inputs $I$ from the training distribution and outputs $\{o_i\}$ sampled from the old policy.

What it computes: for each training input, the model generates 8 candidate outputs (each containing a rationale and verdict). Each output receives a reward based solely on whether its final verdict matches the ground-truth human label. The advantage $\hat{A}_i$ for each output is computed by comparing its reward to the average reward of the 8 candidates in its group (this is the "group relative" aspect — advantages are normalized within each group rather than across the entire batch). The ratio $\frac{\pi_\theta}{\pi_{\theta_{\text{old}}}}$ implements importance-weighted policy gradient: if the current policy assigns higher probability to a good output than the old policy did, the gradient increases that probability further; if it assigns higher probability to a bad output, the gradient decreases it. The KL penalty prevents the policy from diverging too far from the reference model, which would risk catastrophic forgetting or reward hacking.

Why this form: GRPO is chosen over standard PPO for several reasons. First, by computing advantages relative to the group mean, it eliminates the need for a separate value function (critic) model, reducing memory and computational requirements — the 8 rollouts per input serve as their own baseline. Second, the group-relative normalization automatically adapts to varying difficulty levels: on an easy input where all 8 rollouts produce correct verdicts, all advantages are near zero (no update pressure); on a hard input where only 1 of 8 is correct, that correct output receives a large positive advantage (strong update pressure). Third, the KL penalty with coefficient 0.001 is relatively small, allowing the model to make substantial changes to its policy while preventing degenerate collapse.

The reward function. The paper defines a binary reward based on verdict consistency:

• $+1$ if the generated verdict $v_i$ matches the ground-truth human preference label
• $-1$ otherwise

This is a deliberately simple reward structure — it provides no signal about the quality of the reasoning, the structure of the CoT, or whether the model used the "right" mechanism for the task. The model's only objective during RLVR is to produce verdicts that match human labels.

This is what makes the paper's finding about mechanism polarization so striking: even though the RLVR reward function is completely agnostic to reasoning structure, the model spontaneously learns to couple B-CoT with preference tasks and D-CoT with correctness tasks because doing so improves verdict accuracy. The mechanism allocation is not directly rewarded — it emerges as an instrumental strategy for maximizing the verdict-consistency reward.

Hyperparameters for RLVR (from Appendix Table 7):

• Training steps: 100
• Learning rate: $1 \times 10^{-6}$
• Batch size: 128
• KL loss coefficient: 0.001
• KL coefficient: 0.001
• Rollouts: $n = 8$ using vLLM with temperature 0.8

Why RLVR is called a "switching amplifier." The paper uses this term to capture the mechanism by which RLVR improves performance beyond SFT. During SFT, the model learns to produce both B-CoT and D-CoT structures but only imperfectly associates them with their appropriate domains (73% structural match rate). During RLVR, the model receives reward only for correct verdicts. The model discovers — through trial and error across many training steps — that when evaluating preference tasks, generating B-CoT yields higher verdict accuracy than generating D-CoT or unstructured reasoning. Conversely, when evaluating correctness tasks, generating D-CoT yields higher accuracy. The "switching" refers to the model's autonomous selection between reasoning mechanisms; the "amplification" refers to the RL process strengthening the correlation between domain and mechanism because that correlation is rewarded (indirectly, through verdict accuracy).

The evidence for this amplification is the jump from 73% to 95% structural match rate after RLVR, reported in Section 5 ("Emergent Polarization Analysis"). At 73%, the model's mechanism allocation is correlated with domain but far from deterministic — it's essentially a weak prior inherited from SFT. At 95%, the allocation is near-deterministic — the model has learned that using the right mechanism substantially improves its chance of producing a correct verdict, and it consistently acts on that knowledge.

No explicit structural labels during RLVR. A crucial detail is what the RLVR stage does NOT include: there are no structural labels, no mechanism-selector rewards, and no explicit guidance about which reasoning type to use for which input. The 21K RLVR samples are drawn from the same data sources as the SFT data (HelpSteer3, Code-Preference, Math-DPO, WildGuard, OffsetBias), but the model receives only the instruction $x$, the two responses $y_A, y_B$, and the ground-truth label. The model must generate its own rationale from scratch — it is not shown a target B-CoT or D-CoT during RLVR. The structural match rate improvement is therefore a genuinely emergent property: the model independently discovers that structured reasoning improves accuracy and learns to consistently deploy it.

---

Key Design Choices and Their Justifications

Several architectural decisions in the Mix-GRM pipeline warrant explicit explanation, as they represent deliberate trade-offs that affect the framework's behavior.

Why decompose raw rationales into atomic triples rather than training on raw rationales directly? Without decomposition, the synthesis mechanisms (merge for B-CoT, judgment regeneration for D-CoT) would be operating on unstructured text, making it impossible to identify which parts of one rationale correspond to which parts of another, or to isolate individual judgments for re-grounding. The atomic schema provides a common representation that makes structural manipulation tractable. Furthermore, the paper notes that the schema enforces "syntactic uniformity," ensuring that the resulting CoTs differ in reasoning mechanism rather than in superficial stylistic patterns — a critical control for the paper's claim that structural differences, not length differences, drive performance.

Why sample N=2 rationales for B-CoT rather than more? The ablation in Figure 4(a) shows that performance improves monotonically from N=1 to N=4, suggesting that more samples would yield even better B-CoT quality. The choice of N=2 balances the benefits of breadth (more diverse principles) against computational cost (each additional sample requires a full LLM generation pass) and against the risk of including low-quality or contradictory principles. The paper's analysis in Figure 4(b) suggests that raw volume of principles is not the primary driver of performance — selection quality matters more. With N=2, the merge-and-deduplicate step produces a manageable principle set where consensus-based filtering (top-10) can effectively separate signal from noise. With N=4, the principle pool becomes large enough that even the top-10 consensus might include some noise.

Why select the top-10 most frequent principles rather than all principles or a random subset? Figure 4(b) provides the empirical justification: Top-10 > Full > Random. The paper interprets this as a "denoising effect" — low-frequency principles (those mentioned in only 1 out of N rationales) are likely evaluation artifacts rather than genuinely relevant dimensions, and including them introduces noise that degrades training signal quality. The top-10 consensus acts as an implicit quality filter: if multiple independent evaluations agree that a principle is relevant, it probably is; if only one evaluation mentions it, it probably is not. The choice of 10 rather than some other number is not explicitly justified, but the empirical results suggest it provides a good balance between coverage (including enough principles to capture the multi-dimensional nature of preference) and noise reduction (excluding spurious principles).

Why use a self-solving reasoning trace for D-CoT rather than just asking the model to "verify step by step"? The key distinction is between verification that references an external standard and verification that reasons from scratch. If the model is simply asked to "verify step by step whether the response is correct," it might still fall prey to the superficial shortcuts the paper identifies — it might verify that the response's steps look logical without checking whether the underlying claims are true. By first generating an independent solution $z$ and then comparing the response against $z$, the model is forced to confront discrepancies between what it independently determines is correct and what the response claims. This implements a form of fact-checking rather than coherence-checking: the question is not "does the response make sense?" but "does the response match the independently verified answer?" The paper's Case Study 2 demonstrates the value: D-CoT catches an error (K > Mg being incorrect in the reactivity series) that vanilla CoT and B-CoT both miss because they evaluate the response's structure rather than its substance.

Why use GRPO rather than standard PPO or DPO for RLVR? The paper does not explicitly justify GRPO over alternatives, but the choice follows from practical considerations. Standard PPO requires training a separate value function (critic) model, which doubles the memory footprint and adds complexity. DPO operates on pre-collected preference pairs rather than on-policy rollouts, making it unsuitable for the exploration-driven mechanism discovery that RLVR enables. GRPO's group-relative advantage normalization eliminates the critic while still supporting on-policy exploration through the 8-rollout-per-input sampling. The temperature of 0.8 ensures sufficient diversity in these rollouts for the model to discover the mechanism-performance relationship.

Why is the SFT data not filtered for verdict consistency? A natural question is whether training the model on synthesized CoTs whose final verdicts might disagree with the ground-truth human label would teach the model to produce incorrect verdicts. The paper addresses this in Appendix A.3: "our empirical verification reveals that training on the full synthesized CoTs yields superior performance compared to aggressive filtering, regardless of verdict consistency." This is a counterintuitive finding — one might expect that training only on "correct" CoTs (those whose verdicts match the label) would be better. The paper suggests this may be because the synthesized CoTs contain valuable structural information (how to organize a B-CoT or D-CoT) even when their final verdict is wrong, and filtering reduces the amount of structural training data available. The RLVR stage then corrects the verdict direction by optimizing for consistency with human labels, while the SFT stage provides the structural scaffolding.

Why use a mixture of B-CoT and D-CoT rather than training only on B-CoT for preference and D-CoT for correctness? The mixture approach forces the model to internalize the mapping between domain and mechanism. If trained only on B-CoT for preference data and D-CoT for correctness data, the model might learn to produce both structures but would have no exposure to situations where the wrong structure is used for a domain — it would not learn to avoid deploying B-CoT on correctness tasks or D-CoT on preference tasks. The mixture training (with the SFT→RLVR sequence) exposes the model to both structures across both domains initially, then uses RLVR reward pressure to sharpen the allocation. The paper's 73% → 95% structural match rate improvement is evidence that this two-stage approach is more effective than pure supervised learning would be.

Why 9K SFT and 21K RLVR samples? The total training budget of 30,000 samples is deliberately small compared to prior work (FARE-8B: 2.5M; DeepSeek-GRM: 1.2M SFT + 237K RL). This is a design choice that demonstrates the paper's central claim about structural optimization versus data scaling. The paper is not claiming that more data would hurt — it is claiming that with the right reasoning structures, far less data is needed to achieve competitive performance. The specific split (9K for SFT, 21K for RLVR) reflects the relative importance of the two stages: SFT needs enough data to teach the basic structural patterns, while RLVR needs more data because it must explore the mechanism-performance relationship through trial and error across 8 rollouts per input.

4. Key Insights and Innovations

Innovation 1: The Optimal Reasoning Mechanism for Evaluation Is Task-Dependent — and Misalignment Directly Hurts Performance

The paper's most fundamental intellectual contribution is not the introduction of Breadth-CoT or Depth-CoT as mechanisms — parallel and sequential reasoning have been explored in prior CoT literature (Yao et al., 2023; Besta et al., 2025; Ning et al., 2024; Wang et al., 2023) — but rather the causal demonstration that deploying the wrong reasoning structure for a given evaluation task actively degrades performance below an unstructured baseline. This is a substantially stronger claim than "different tasks benefit from different approaches." It says that the relationship between reasoning structure and task domain exhibits a double dissociation: B-CoT improves preference but harms correctness, while D-CoT improves correctness but harms preference, relative to a vanilla unstructured CoT.

Prior to this work, the GRM literature operated under an implicit assumption — reinforced by the "longer is better" narrative in recent CoT scaling work (Chen et al., 2025; 2026) — that adding more reasoning, in any form, would at worst provide no benefit and at best improve evaluation. The possibility that structured reasoning could be counterproductive when misapplied was not a recognized phenomenon in reward modeling. The paper's Table 2 provides direct evidence: in the SFT stage, B-CoT drops Correctness domain performance from 72.2 to 70.2 (a ↓2.0 gap), while D-CoT drops Preference domain performance from 68.2 to 65.9 (a ↓2.3 gap). These are not trivial degradations — they represent the model becoming actively worse at its core task because it is thinking in a structurally inappropriate way.

This finding reframes the GRM development problem. The relevant question shifts from "how do we get the model to reason more?" to "how do we get the model to reason appropriately for each evaluation context?" This is a conceptual shift from quantity-oriented to structure-aware evaluation, and it has implications beyond the specific B-CoT/D-CoT dichotomy. It suggests that the evaluation of any future GRM should separately report performance across task domains — a model that achieves high aggregate scores by excelling at preference while failing at correctness (or vice versa) may be poorly suited for universal reward modeling in RLHF pipelines where both types of evaluation are needed.

The significance of this double dissociation extends to how we interpret prior conflicting results in the GRM literature. Models like FARE-8B (which uses synthetic expansion of evaluation coverage — effectively a breadth-oriented approach) and JudgeLRM (which uses RL to elicit longer CoT traces — potentially depth-oriented depending on the training dynamics) achieved different performance profiles that could now be reinterpreted through the lens of mechanism-task alignment rather than simply "better" or "worse" overall. The paper's framework provides a vocabulary for diagnosing why one GRM outperforms another on specific benchmarks rather than treating benchmark scores as opaque quality metrics.

Innovation 2: RLVR Acts as a "Switching Amplifier" — Models Spontaneously Learn to Allocate Reasoning Structure Without Explicit Structural Supervision

This is the paper's most surprising and theoretically significant finding. The standard approach to teaching a model multiple behaviors is to provide explicit labels indicating which behavior to deploy when — a router, a task classifier, or domain-conditional training. Mix-GRM does something fundamentally different: during SFT, the model is shown B-CoT examples in preference contexts and D-CoT examples in correctness contexts, but it internalizes this association only weakly (73% structural match rate at inference). Then, during RLVR, the model receives no structural guidance whatsoever — the reward signal depends only on verdict correctness, with zero information about whether the model used B-CoT, D-CoT, or unstructured reasoning. Yet after RLVR, the structural match rate jumps from 73% to 95%.

What makes this remarkable is the mechanism by which it occurs. The model is not told "use B-CoT for preference" — it discovers that doing so improves its probability of producing a correct verdict, because B-CoT's multi-dimensional scanning catches subtle preference nuances that unstructured or D-CoT reasoning would miss. Conversely, it discovers that D-CoT's deductive grounding catches logical errors that B-CoT's surface-level feature checking would overlook. The allocation is entirely emergent: it is an instrumental strategy that the model autonomously develops to maximize a reward signal that is agnostic to reasoning structure.

This finding has implications that extend well beyond reward modeling. It provides a concrete existence proof for a phenomenon that has been hypothesized but not cleanly demonstrated: that reinforcement learning with outcome-based rewards can induce models to develop appropriate internal reasoning strategies without explicit process supervision. This connects to broader debates in the RL for reasoning literature — particularly around whether models trained with outcome rewards learn genuine reasoning skills or merely pattern-match to surface features. The 73% → 95% structural match rate improvement, combined with the domain-appropriate performance gains documented in Table 2 (where RLVR boosts Correctness domain performance from 72.2 to 78.7 for Mix-GRM), suggests that the model is learning something structurally meaningful about how to reason, not just what to output.

The "switching amplifier" framing is also conceptually novel. Prior work on RL for GRMs (Chen et al., 2025; 2026) used RL to elicit longer CoT traces — the amplification was quantitative. Mix-GRM's RLVR amplifies something qualitative: the precision with which the model matches reasoning structure to task domain. The paper's analysis shows that the Base-GRM (unstructured CoT) sees a smaller RLVR gain (73.3 → 76.9, a +3.6 improvement) than Mix-GRM (75.1 → 79.4, a +4.3 improvement), confirming that the mixed mechanism provides a "more exploitable base" for RL optimization. This suggests a general principle: models that have internalized multiple reasoning strategies provide a richer action space for RL to optimize over, enabling larger gains than models constrained to a single reasoning approach.

Innovation 3: Structural Optimization Can Substitute for Data Scale — 275× Data Efficiency Over Brute-Force Expansion

The paper makes a pointed economic argument that is both a practical contribution and a conceptual challenge to the dominant data-scaling paradigm in GRM development. FARE-8B (Xu et al., 2026) achieves an average benchmark score of 75.9 using approximately 2.5 million training samples. Mix-GRM achieves a comparable 75.1 using only 9,000 SFT samples — a ~275× reduction in required training data. This is not an incremental efficiency gain from better hyperparameters or a slightly cleaner dataset; it is an order-of-magnitude difference that stems from a fundamentally different approach to what training data should contain.

The conceptual move here is from data quantity to data structure. FARE-8B expands its training corpus by generating massive amounts of synthetic evaluation data, but the rationales in that data remain unstructured — they are essentially more of the same kind of reasoning that standard GRMs produce. Mix-GRM's 9K samples are carefully structured to teach the model how to reason rather than just what to say. Each sample encodes not just a verdict but a reasoning architecture (B-CoT or D-CoT) that the model can generalize from.

This efficiency argument matters because it changes the economics of GRM development. Training on 2.5M samples requires substantial compute for both data synthesis (generating millions of LLM evaluations) and model training. Training on 9K samples with a structured synthesis pipeline (which the paper shows requires approximately 2 reasoning passes per sample for both B-CoT and D-CoT, as documented in Table 4) dramatically reduces the total computational footprint. For research groups or organizations without access to massive compute clusters, this makes state-of-the-art GRM development feasible.

The finding also has implications for how the field thinks about progress in reward modeling. If structural optimization can achieve what data scaling achieves with two orders of magnitude less data, then the marginal value of additional unstructured training data may be far lower than the GRM literature has assumed. The paper does not claim that Mix-GRM's approach would surpass a data-scaled model given equivalent data volumes — that comparison is not made — but it demonstrates that the data-scaling curve can be dramatically shifted by improving data quality (defined structurally rather than in terms of label accuracy). This is a reframing of "data efficiency" from a filtering problem (keeping correct labels, discarding noisy ones, as discussed in Appendix A.3 where the paper shows that verdict filtering does not improve performance) to a structural design problem (organizing information into reasoning-appropriate formats).

Innovation 4: Verdict-Consistency Reward Is Sufficient to Induce Appropriate Reasoning Structure — Process Supervision Is Not Required

This is the theoretical finding that underpins Innovation 2 but deserves separate treatment because it addresses a long-standing debate in the reasoning literature. The question of whether outcome-based rewards can induce correct reasoning processes — or whether explicit process supervision (step-by-step correctness labels, as in process reward models) is necessary — has been contentious. The paper contributes a clean empirical demonstration in the reward modeling domain: a reward function that depends only on whether the final verdict matches the human label is sufficient to cause the model to converge on domain-appropriate reasoning structures, without any intermediate supervision on whether the reasoning steps themselves are correct.

This finding is significant because process supervision is expensive. Training a process reward model for evaluation would require annotating not just which response is better but why at each step of the evaluation rationale — a dramatically more costly data collection pipeline. Mix-GRM's approach requires only final verdict labels (which are already present in standard preference datasets like HelpSteer3 and UltraFeedback), making it compatible with existing data infrastructure.

The mechanism by which outcome-based reward induces process improvement is worth articulating because it is subtle. The model generates 8 rollouts per input during RLVR. These rollouts vary in their reasoning structure — some naturally tend toward breadth (mentioning many principles), some toward depth (focusing on a few principles with detailed analysis), and some are unstructured. The rollouts that happen to use a structure appropriate to the domain are more likely to produce correct verdicts, receive positive reward, and have their probability increased by the policy gradient update. Over many training steps, the model's distribution shifts toward producing the rewarded structures more consistently. This is not the model learning that "B-CoT is correct" — it is the model learning that "when I use B-CoT on preference tasks, I get reward more often."

The 95% structural match rate is the quantitative evidence that this mechanism works. If outcome-based reward were insufficient to shape reasoning structure, the match rate would remain at the SFT level of 73% or drift randomly. The 22 percentage point improvement is a substantial effect size that argues strongly against the hypothesis that process supervision is necessary for teaching appropriate reasoning strategies.

This finding connects to broader observations in the RL for reasoning literature, particularly around DeepSeek-R1 (DeepSeek-AI, 2025) and related work showing that RL can elicit complex reasoning behaviors from base models. Mix-GRM extends this principle to the meta-cognitive domain: the model is not learning to solve math problems through RL but learning to evaluate solutions through RL, and it spontaneously develops the reasoning structures appropriate to different types of evaluation. This suggests that outcome-based RL may be a general-purpose mechanism for inducing appropriate cognitive strategies across a wider range of tasks than previously demonstrated.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates Mix-GRM across five widely recognized general-purpose reward modeling benchmarks: RewardBench (Lambert et al., 2025; 2,985 samples covering Chat, Math, Code, Safety), RewardBench-v2 (Malik et al., 2025; 1,865 samples covering Focus, Instruction-Following, Factuality, Math, Safety, Ties), RMB (Zhou et al., 2025a; 14,725 samples covering Harmfulness, Helpfulness in General and Code), RM-Bench (Liu et al., 2025b; 11,943 samples covering Chat, Math, Code, Safety), and PPE (Frick et al., 2025; 22,991 samples excluding ties, covering Chat, MMLU-Pro, GPQA, IFEval, MBPP). The standard benchmark metric is pairwise comparison accuracy: the fraction of evaluation pairs where the model's preference verdict matches the ground-truth human label. For mechanism efficiency analysis, instances from these benchmarks are re-categorized by the authors into two domain groups—Preference (subjective) and Correctness (objective)—based on original task metadata (detailed in Appendix A.3, Table 9). The paper reports aggregate benchmark-level accuracy for overall performance and domain-level accuracy for the mechanism analysis in Table 2. For downstream evaluation, DPO-trained policy models are assessed on instruction-following benchmarks (AlpacaEval-2, Arena-Hard v0.1) using win rate against a baseline reference, and on mathematical reasoning benchmarks (GSM8k, MATH, MAWPS, TabMWP, and a STEM aggregate) using accuracy. Best-of-N test-time scaling is evaluated on MATH, CHAMP, MBPP+, and BigCodeBench using Best-of-10 accuracy, where the model selects the optimal solution from a pool of 10 candidate responses generated by Llama-3.1-8B.

• Base model(s). All Mix-GRM variants use Qwen3-8B-Base (Team, 2025b) as the starting model. The paper does not explicitly justify this choice, but the 8B scale is representative of the open-source reward model landscape against which the paper compares (Skywork-Reward-8B, FARE-8B, RubricRM-8B, JudgeLRM-7B, RM-R1-7B). For FLOPs-matched analysis, this is not a pretraining-scaling paper, so no larger model comparison is presented—the scaling argument is about data volume (9K vs. 2.5M samples), not parameter count. For the DPO downstream experiments, the base policy model is Llama-3-8B (not Qwen), chosen explicitly "to strictly control for temporal data leakage and ensure a fair comparison with the release dates of our evaluation benchmarks" (Appendix A.4).

• Metrics. The primary metric for reward model evaluation is pairwise comparison accuracy—the percentage of response pairs where the model's chosen response matches the human-annotated preference. This is reported at the benchmark level (e.g., "RewardBench: 91.8") and as averages across benchmarks ("Overall" columns in Table 1). For the DPO downstream experiments, instruction-following is measured by win rate (percentage of head-to-head comparisons against a reference model where the DPO-trained policy wins, as judged by GPT-4), and mathematical reasoning is measured by exact match accuracy against ground-truth answers. For Best-of-N test-time scaling, the metric is pass@1 accuracy after selecting the highest-scored candidate from 10 samples. For the mechanism analysis in Section 5, the paper introduces a structural match rate—the percentage of generated CoTs whose structure (B-CoT vs. D-CoT indicators such as principle counts and trigger phrases) matches what would be expected for the task domain—as a diagnostic for whether the model is deploying appropriate reasoning mechanisms.

• Baselines. The paper compares against seven open-source reward models spanning two paradigms:

  • Discriminative (scalar): Skywork-Reward-v0.2-Llama-3.1-8B (Liu et al., 2024), a Bradley-Terry model that outputs scalar preference scores without explicit reasoning.
  • Generative (with reasoning): JudgeLRM-7B (Chen et al., 2025), trained via RL to elicit long CoT judging traces; RM-R1-7B (Distill) and RM-R1-7B (Instruct) (Chen et al., 2026), both trained with SFT+RL, where the Instruct variant uses a stronger base model; FARE-8B (Xu et al., 2026), trained via SFT on 2.5M synthetically expanded multi-task evaluation samples; RubricRM-8B (Liu et al., 2026), a two-stage system with a rubric generator and rubric-based judge; and DeepSeek-GRM-16B (Liu et al., 2025c), trained on 1.2M SFT + 237K RL samples. Two proprietary models—DeepSeek-V3.2 and Gemini-3-Flash—are included as upper-bound references (gray rows in Table 1) but are not the target of comparison.

  For the DPO downstream experiments, additional baselines include RubricRM-8B, FARE-8B, RM-R1-Instruct, and DeepSeek-GRM-16B used as reward signal providers for DPO data construction.

  For Best-of-N experiments, the paper compares against RM-R1 (Instruct) and FARE-8B as verifiers, with random selection (average of all 10 candidates) and oracle selection (always pick the best candidate if it exists in the pool) as theoretical lower and upper bounds.

• Generation budget / compute accounting. The paper measures computational cost primarily in terms of training data volume (number of SFT and RLVR samples) and reasoning passes during synthesis (Table 4). For fair comparison of B-CoT and D-CoT synthesis costs, the paper explicitly tracks: both B-CoT (N=2 sampling) and D-CoT (trace generation + synthesis) require approximately 2 reasoning passes; the merge operation in B-CoT is restricted to deduplication without introducing new information and is not counted as an additional pass. During RLVR, all configurations use G=8 rollouts per input. At inference, Table 4 reports average token counts per CoT style: D-CoT (702 tokens), B-CoT (824 tokens), Mix-CoT adaptive (731 tokens)—all within the same order of magnitude. The paper emphasizes that these marginal differences confirm that performance gaps stem from structural efficacy rather than raw compute disparities. However, unlike the reference example paper (which meticulously tracks FLOPs for pretraining vs. inference tradeoffs), this paper does not report total FLOP counts, GPU-hours, or per-sample training costs, making direct computational efficiency comparisons across methods approximate.

• Cross-validation / statistical protocol. The paper does not employ k-fold cross-validation for model selection, unlike the reference example's two-fold cross-validation within difficulty bins. Instead, it reports benchmark-level accuracy on established test sets with standard train/test splits—the risk of overfitting to test benchmarks through hyperparameter tuning is not explicitly addressed. The training data is constructed via stratified random sampling from five source datasets (HelpSteer3, Code-Preference, Math-DPO, WildGuard, OffsetBias) totaling 30,876 samples (Table 8), with the specific 9K SFT + 21K RLVR split chosen as a design parameter rather than through cross-validated selection. The RLVR training uses 100 steps with batch size 128, and the structural match rate is computed once post-training. No confidence intervals, standard deviations, or statistical significance tests are reported for any metric—all results are presented as point estimates. For the DPO experiments, models are trained once per RM variant and evaluated on fixed benchmarks. For Best-of-N, results are reported as single accuracy numbers per benchmark without error bars. This absence of variance reporting is a notable limitation for assessing the reliability of the relatively small performance differences observed in some comparisons (e.g., Mix-GRM's 75.1 vs. FARE-8B's 75.9 in the SFT-only comparison, a 0.8 point gap on an average across five benchmarks).

---

Main Quantitative Results

Overall Benchmark Performance (Table 1)

The headline result is that Mix-GRM (RLVR-trained) achieves an average score of 79.4 across the five benchmarks, establishing a new state-of-the-art among open-source reward models and surpassing the second-best (Skywork-Reward-8B at 76.5) by +2.9 points, the strongest generative baseline (FARE-8B at 75.9) by +3.5 points, and the RL-driven models JudgeLRM-7B (68.8) and RM-R1-Instruct (70.1) by +10.6 and +9.3 points respectively. The gap over the Base-GRM (Mix-GRM trained with unstructured CoT + same RLVR) is +2.5 points (79.4 vs. 76.9), representing the net benefit of the Mix-GRM framework over simply scaling unstructured CoT length with RL. The gap over proprietary models is substantial but narrowing: Gemini-3-Flash achieves 87.2 and DeepSeek-V3.2 achieves 86.4, leaving a ~7 point gap to the best proprietary model.

Looking at the SFT-only stage (before RLVR), Mix-GRM achieves 75.1, which is remarkable primarily because it surpasses two RL-trained baselines—RM-R1-Instruct (70.1, +5.0) and DeepSeek-GRM-16B (65.2, +9.9)—despite using no RL training itself. It also matches FARE-8B (75.9) with only 9K SFT samples versus FARE-8B's 2.5M—a ~275× data reduction that the paper highlights as evidence for structural optimization over data scaling.

The RLVR boost over SFT is +4.3 for Mix-GRM (75.1 → 79.4) compared to +3.6 for Base-GRM (73.3 → 76.9), which the paper interprets as evidence that the mixed CoT structures provide a "more exploitable base" for RL optimization. The widening gap between Mix-GRM and Base-GRM from SFT (+1.8) to RLVR (+2.5) supports this interpretation.

Per-benchmark highlights (Table 1, RLVR-trained):

• On RewardBench-v1, Mix-GRM achieves 91.8, closing within 3.7 points of DeepSeek-V3.2 (95.5) and surpassing Skywork-Reward (93.9) by only −2.1—notable because Skywork-Reward is a discriminative model that specializes on RewardBench-style comparisons.
• On RewardBench-v2, Mix-GRM achieves 77.5, substantially outperforming all open-source baselines (next best: Skywork-Reward at 79.7? Wait—Table 1 shows Skywork-Reward at 79.7 for RB-v2, so Mix-GRM's 77.5 is below Skywork. The paper's claim of "surpassing leading open-source RMs by an average of 8.2%" must be computed differently: likely comparing against the average of baselines excluding Skywork, or the claim refers to specific benchmarks like RM-Bench where Mix-GRM's 82.7 dominates). This discrepancy between the stated 8.2% average improvement and the per-benchmark comparisons warrants scrutiny.
• On RM-Bench, Mix-GRM achieves 82.7—the highest open-source score by a large margin (+8.7 over Skywork-Reward's 74.0, which is the second-highest among baselines examined).
• On RMB, Mix-GRM achieves 80.1, behind FARE-8B (83.2) and ahead of all other open-source models.
• On PPE, Mix-GRM achieves 64.8, second to FARE-8B (62.5) among open-source baselines? Actually, Table 1 shows FARE-8B at 62.5, RubricRM at 62.5, so Mix-GRM's 64.8 does lead—but the absolute scores on PPE are notably lower than other benchmarks, reflecting the difficulty of this evaluation benchmark.

Mechanism Efficiency: Domain-Level Analysis (Table 2)

The most important finding in Table 2 is the double dissociation between reasoning mechanism and task domain. In the SFT stage:

• B-CoT (Breadth-only) improves Preference domain performance from 68.2 (Base-GRM) to 69.3 (+1.1), but degrades Correctness domain performance from 72.2 to 70.2 (−2.0). The model becomes better at judging subjective quality but worse at verifying objective correctness.

• D-CoT (Depth-only) improves Correctness domain performance from 72.2 to 72.8 (+0.6), but degrades Preference domain performance from 68.2 to 65.9 (−2.3). The model becomes better at logical verification but worse at multi-dimensional preference evaluation.

• Mix-GRM (Combined) achieves 70.0 on Preference (+1.8 over Base-GRM) and 73.5 on Correctness (+1.3 over Base-GRM), outperforming both single-mode models on their respective strongholds. This is the paper's evidence for "synergistic mutual enhancement"—the mixture does not simply average the strengths of B-CoT and D-CoT but exceeds both.

After RLVR, the pattern intensifies:

• Mix-GRM (RLVR) achieves 72.6 on Preference (+3.7 over Base-GRM SFT baseline, not over Base-GRM RLVR which is 68.9) and 78.7 on Correctness (+6.5 over Base-GRM SFT baseline, or +2.3 over Base-GRM RLVR which is 76.4). The Correctness domain sees the largest absolute gains from the Mix-GRM framework.

• The single-mode RLVR variants show performance ceilings: Mix-GRM (Breadth) RLVR plateaus on Correctness at 71.5, which is below its SFT performance (70.2 → 71.5, only +1.3, and substantially below Base-GRM RLVR at 76.4). This is a striking negative result—RL-optimizing a breadth-only model for correctness actually caps its potential, confirming that the CoT structure acts as a bottleneck for RL optimization.

• Mix-GRM (Depth) RLVR excels at Correctness (77.7, the highest among all variants) but underperforms on Preference (71.0 vs. Mix-GRM's 72.6).

The paper's claim that "misaligning the reasoning mechanism with the task directly degrades performance" is supported specifically by the SFT single-mode results: deploying B-CoT on correctness tasks (−2.0) and D-CoT on preference tasks (−2.3) both underperform the unstructured Base-GRM baseline. The claim that "RLVR acts as a switching amplifier" is supported by the 73% → 95% structural match rate improvement documented in Section 5.

Fine-grained per-subtask analysis (Appendix D, Tables 10 and 11):

Table 10 decomposes Preference performance into sub-tasks (Chat, Focus, IF, Helpfulness, Human, IF). Mix-GRM (RLVR) leads on RewardBench Chat (86.2), RewardBench-v2 Focus (91.3—the highest individual score across all subtasks), and RewardBench-v2 IF (37.5). However, it is notably weaker on RMB Chat Helpfulness (78.1) where FARE-8B (82.9) and the single-mode Mix-GRM Breadth (79.2) both outperform. This suggests that while the mixture approach generally improves preference evaluation, it may sacrifice some coverage on chat-specific helpfulness dimensions relative to a pure breadth approach.

Table 11 decomposes Correctness performance into sub-tasks (Math, Code, Factuality, MMLU-Pro, GPQA, MBPP). Mix-GRM (RLVR) leads on RewardBench Code (95.4), RewardBench-v2 Math (89.0), RewardBench-v2 Factuality (65.8), RM-Bench Math (79.2), RM-Bench Code (66.6), RMB MMLU-Pro (65.0), and PPE Math (86.7). The breadth-only variant (Mix-GRM Breadth RLVR) actually achieves the highest RewardBench Code score (96.3), but this appears to be an anomaly given its poor performance on other correctness subtasks—possibly indicating that Code evaluation on RewardBench involves preference-like elements (style, readability) that benefit from breadth.

Downstream Utility: DPO Training (Table 3)

Table 3 evaluates whether Mix-GRM provides useful reward signals for training policy models via Direct Preference Optimization. The experimental setup: a base Llama-3-8B model is first SFT-initialized on UltraChat + MetaMathQA (40K samples), then DPO-trained using preference pairs constructed by having each candidate RM score 5 GPT-4o-mini-generated responses per instruction and selecting the highest-scoring as chosen and lowest-scoring as rejected. The resulting policies are evaluated on instruction-following (AlpacaEval-2, Arena-Hard) and mathematical reasoning (GSM8k, MATH, STEM, TabMWP).

The headline: Mix-GRM-trained policy achieves the best instruction-following performance (12.1 average win rate) while maintaining the best math reasoning accuracy (46.4 average). Specific comparisons:

• Instruction-following: Mix-GRM (12.1) edges out FARE-8B (12.0) and substantially exceeds RubricRM-8B (10.5), RM-R1-Instruct (11.1), and DeepSeek-GRM-16B (11.1). The SFT baseline achieves only 5.3, confirming that DPO with any RM substantially improves instruction-following.

• Mathematical reasoning: Mix-GRM (46.4) slightly outperforms RubricRM-8B (45.9), FARE-8B (45.8), and RM-R1-Instruct (45.8). On GSM8k specifically, Mix-GRM reaches 77.6%, a clear improvement over the SFT baseline (75.1%) and all other RM-trained policies (75.6–76.3%). The margin is small (+1.5% over the SFT baseline) but consistent across the math suite.

The key insight from Table 3 is what does NOT happen: DPO training with Mix-GRM does not sacrifice reasoning capability to achieve better instruction-following. This is the practical manifestation of the paper's mechanism-alignment claim—because Mix-GRM evaluates both preference and correctness appropriately, the resulting policy internalizes both qualities rather than trading one off against the other.

However, the absolute improvements are modest. The DPO training improves instruction-following by roughly 6–7 points over SFT (from 5.3 to 12.1) and math reasoning by only 1.4 points (from 45.0 to 46.4). This suggests that while Mix-GRM provides better reward signals than alternatives, the downstream impact on policy quality is bounded by other factors in the DPO pipeline (the base model, the response generation process, the DPO algorithm itself).

Downstream Utility: Best-of-N Test-Time Scaling (Figure 2)

Figure 2 reports Best-of-10 accuracy where Mix-GRM acts as a verifier to select the best candidate from 10 Llama-3.1-8B-generated solutions. The headline: Mix-GRM achieves the highest accuracy across all four benchmarks, with particularly strong results on reasoning-heavy tasks.

• MATH: Mix-GRM achieves 43.2%, outperforming RM-R1 (37.7%) by +5.5 points and FARE-8B (35.2%) by +8.0 points. The random baseline is approximately 33.0% and oracle is approximately 55.0%, suggesting substantial room for improvement in verifier quality.

• CHAMP: Mix-GRM leads at approximately 26.5% (exact values must be estimated from the bar chart), with smaller gaps over baselines than on MATH.

• MBPP+: Mix-GRM achieves approximately 58.5%, the highest among all rerankers.

• BigCodeBench: Mix-GRM achieves approximately 45.0%, again the highest.

The consistent lead across diverse benchmarks (math and code, competition-level and standard difficulty) supports the claim that Mix-GRM's mechanism-adaptive reasoning provides more discriminative signals for logical verification than methods relying on massive data scaling (FARE-8B) or generic RL-driven CoT elongation (RM-R1). The gap is particularly pronounced on MATH, which requires rigorous step-by-step verification—exactly the strength that D-CoT is designed to provide.

---

Ablation Studies and Robustness Checks

B-CoT aggregation scale (Figure 4a): Performance on both Preference and Correctness domains improves monotonically as the number of parallel CoT rationales aggregated ($N$) increases from 1 (vanilla) to 4. The experiment confirms that breadth functions by expanding principle coverage—more independent samples surface more distinct evaluation dimensions. However, the marginal gain diminishes: the jump from N=1 to N=2 is larger than from N=2 to N=4, suggesting that the principle space becomes progressively saturated. The paper's choice of N=2 for main experiments balances coverage against computational cost.

Principle selection strategy for B-CoT (Figure 4b): Comparing three selection strategies within the N=4 pool: Top-10 (most frequent principles across the 4 rationales) > Full (all principles from all rationales) > Random (random subset). The hierarchy holds for both Preference and Correctness domains, with Top-10 achieving 71.8/72.7 (SFT/RLVR performance on the aggregated metric). The finding that Full underperforms Top-10 is the key insight: simply including more principles adds noise, and consensus-based filtering (keeping only high-frequency principles) improves training signal quality. This is characterized as a "denoising effect" where low-frequency principles represent evaluation artifacts rather than genuinely relevant dimensions.

Verdict consistency filtering for SFT data (Appendix E, Table 13): Training on 9K unfiltered synthesized CoTs (some of which have verdicts disagreeing with human labels) achieves 69.8% performance, nearly identical to training on 9K filtered CoTs (only those with correct verdicts) at 70.1%. This is a counterintuitive negative result: filtering for correctness does not help. The paper interprets this as evidence that the synthesized CoTs contain valuable structural information (how to organize B-CoT/D-CoT reasoning) even when the final verdict is wrong, and that filtering reduces the amount of structural training data available. The RLVR stage subsequently corrects verdict direction.

Pipeline stability (Appendix E, Table 12): Solution accuracy tracked across synthesis stages remains stable: Raw Rationale (87.1%) → Merge B-CoT (90.2%) → Generate D-CoT (88.5%). The merge step actually improves accuracy slightly (by resolving contradictions between rationales), while the D-CoT generation maintains high fidelity. This addresses the concern that the LLM-based transformations might introduce errors that compound through the pipeline.

Structural evolution during training (Figure 3): The paper tracks four indicators—average token length per judgment, average principle count, percentage classified as Breadth characteristics, and percentage classified as Depth characteristics—across training stages and mechanism variants. The key findings: (1) Single-mode strategies show extreme structural profiles (Breadth: high principle count, short judgments; Depth: low principle count, long judgments), confirming that the synthesis pipeline successfully creates structurally distinct CoTs. (2) Mix-GRM (SFT) achieves a balanced profile combining both characteristics. (3) RLVR globally expands both breadth and depth indicators—the model does not simply choose one mechanism but amplifies both, which the paper interprets as evidence of "mechanism polarization" where domain-appropriate structures are strengthened while the model maintains the capacity for both.

Emergent polarization analysis (Section 5): The structural match rate between generated CoT type and task domain jumps from 73% post-SFT to 95% post-RLVR. This is measured using structural indicators (principle counts, trigger phrases) on the test set. The paper emphasizes that RLVR provides no explicit structural labels—the improvement is entirely emergent from optimizing verdict-consistency reward. At 73%, the model's mechanism allocation is weakly correlated with domain (essentially a prior inherited from SFT). At 95%, it is near-deterministic.

Token cost parity (Table 4): Across CoT styles, the token consumption remains comparable: D-CoT (702 avg. inference tokens), B-CoT (824), Mix-CoT adaptive (731). RLVR rollouts: 682 (D-CoT), 830 (B-CoT), 725 (Mix-CoT). SFT tokens: 624 (D-CoT), 711 (B-CoT), 648 (Mix-CoT). The paper argues that these marginal differences confirm that performance gaps are structural rather than computational in origin—B-CoT uses slightly more tokens but not enough to explain its domain-specific advantages.

Sensitivity to open-source vs. proprietary LLM for synthesis (Appendix E): The paper notes using the open-weights DeepSeek-V3 model for both schema extraction and raw rationale generation, and states that the pipeline's effectiveness "further underscores its robustness and generalizability, proving it is not overly sensitive to the choice of the underlying LLM." However, no ablation comparing different backbone LLMs for synthesis is presented, so this claim of generalizability is asserted rather than empirically demonstrated.

---

Critical Assessment

The experimental results provide strong evidence for several of the paper's claims, but the strength of support varies substantially across claims and there are meaningful gaps in what was empirically tested.

Claim: "Mix-GRM establishes a new state-of-the-art across five benchmarks, surpassing leading open-source RMs by an average of 8.2%." This claim requires careful parsing. The "average of 8.2%" figure does not correspond to any single comparison in Table 1. Comparing Mix-GRM (RLVR) at 79.4 to the specific baselines: vs. Skywork-Reward (76.5) is +2.9, vs. FARE-8B (75.9) is +3.5, vs. RubricRM (74.7) is +4.7, vs. RM-R1-Instruct (70.1) is +9.3, vs. JudgeLRM (68.8) is +10.6, vs. RM-R1-Distill (67.2) is +12.2, vs. DeepSeek-GRM (65.2) is +14.2. The average improvement over these seven baselines is approximately +8.2%. So the claim is technically correct as an average-over-baselines, but this framing obscures that the improvement over the strongest baseline (Skywork-Reward) is only +2.9 points, and over FARE-8B (the most comparable generative RM) is +3.5 points. The 8.2% figure is inflated by including much weaker baselines like DeepSeek-GRM-16B at 65.2. The more honest claim is that Mix-GRM achieves state-of-the-art by a modest margin over the next-best open-source models.

Additionally, the "state-of-the-art" claim applies to open-source models only. The proprietary baselines (Gemini-3-Flash at 87.2, DeepSeek-V3.2 at 86.4) remain substantially ahead—by approximately 7–8 points on average. The paper is transparent about this (proprietary models are in gray rows labeled "Reference"), but the "SOTA" language in the abstract and introduction could mislead readers who don't carefully distinguish between open-source and all models.

Claim: "B-CoT benefits subjective preference tasks, whereas D-CoT excels in objective correctness tasks." This is the strongest and best-supported claim in the paper. Table 2 provides clean double-dissociation evidence in the SFT stage: B-CoT helps Preference (+1.1), harms Correctness (−2.0); D-CoT helps Correctness (+0.6), harms Preference (−2.3). The effect sizes are moderate but directionally consistent. The case studies in Table 5 provide qualitative validation: B-CoT catches a language mismatch that D-CoT's tunnel vision misses; D-CoT catches a reactivity ordering error that B-CoT's surface-level heuristics overlook. The convergent evidence from quantitative benchmarks and qualitative examples makes this the paper's most robust finding.

However, a limitation is that the domain categorization (Preference vs. Correctness) is a binary imposed by the authors based on task metadata. Real-world evaluation tasks may not cleanly separate into these two categories—a code evaluation task might involve both correctness verification and style assessment, a chat evaluation might involve both factual accuracy and conversational appropriateness. The paper's own limitation section acknowledges that "this dichotomy represents a coarse-grained mapping of the diverse alignment landscape" and that real-world tasks "exist on a continuous spectrum." The strong double dissociation observed in Table 2 may partly be an artifact of the binary categorization forcing tasks into one bin or the other. Testing on deliberately hybrid tasks (e.g., evaluating code that must be both correct and readable, or evaluating responses that must be both factually accurate and empathetically phrased) would strengthen or qualify this claim.

Claim: "RLVR acts as a switching amplifier, inducing emergent polarization where the model spontaneously allocates its reasoning style to match task demands." The evidence for this claim rests primarily on the 73% → 95% structural match rate improvement. This is compelling but the measurement methodology deserves scrutiny. The structural match rate is computed using "specific structural indicators (e.g., principle counts and trigger phrases)" per Section 5. The paper does not specify exactly what these indicators are, what the classification threshold is, or whether the classification itself is automated or manual. If the indicators are coarse (e.g., a CoT is classified as B-CoT if it mentions more than X principles), the 95% figure might overstate how precisely the model is matching the intended reasoning structure—it might be producing something that looks structurally like B-CoT (many principles mentioned) without actually implementing the full breadth-oriented reasoning process. Conversely, if the indicators are fine-grained, the 95% figure is impressive evidence of emergent mechanism selection.

A stronger test of the "switching amplifier" claim would be a causal intervention: take the post-RLVR model and force it to use the wrong mechanism on some test instances (e.g., via prompting that instructs it to use B-CoT on a correctness task), showing that performance degrades relative to when it freely chooses. Without this, the correlation between mechanism choice and domain could reflect a domain-detection capability (the model learns to recognize what kind of task it's evaluating and adjusts its output format accordingly) rather than a genuine causal effect of mechanism choice on evaluation accuracy. The paper's ablation showing that single-mode models plateau under RLVR (Breadth-only on Correctness at 71.5 vs. Mix-GRM at 78.7) provides some causal evidence, but the within-model intervention would be cleaner.

Claim: "Optimizing how a model thinks is more critical for post-training efficacy than simply scaling how long it writes." The comparison between Mix-GRM (9K SFT, 79.4 after RLVR) and FARE-8B (2.5M SFT, 75.9) is the primary evidence. The data efficiency argument is valid as a demonstration, but it conflates two distinct axes: data quality (structured vs. unstructured CoT) and training methodology (SFT+RLVR vs. SFT-only). FARE-8B is SFT-only; Mix-GRM uses RLVR after SFT. The fair comparison would be Mix-GRM's SFT-only performance (75.1) against FARE-8B's 75.9—here, Mix-GRM is slightly worse despite the structural optimization. The 275× data reduction is therefore more accurately stated as: "with 275× less data plus RLVR, Mix-GRM achieves comparable performance to a data-scaled SFT-only model." The RLVR contribution (+4.3 for Mix-GRM) is doing substantial work in closing the gap. The paper does not report what performance FARE-8B would achieve with an equivalent RLVR stage, which would be the proper comparison for the "structural optimization vs. RL enhancement" decomposition.

Additionally, the paper does not compare against FARE-8B trained on 9K structured data, which would test whether FARE-8B's architecture benefits from the same structural optimization. The efficiency claim is thus about Mix-GRM specifically, not about structural optimization generally—we don't know whether the efficiency gain is replicable with different base architectures or training recipes.

Missing experiments that would strengthen the paper:

• Scaling the mixture ratio: The paper uses a fixed assignment (B-CoT for all Preference, D-CoT for all Correctness). What happens if some percentage of Preference tasks get D-CoT and vice versa? The emergent polarization analysis suggests the model would sort this out during RLVR, but this wasn't tested. An ablation sweeping the percentage of cross-domain examples (e.g., 10% D-CoT on Preference, 25%, 50%) would characterize how robust the mechanism alignment is to label noise in the SFT stage.

• Different base model scales: All experiments use Qwen3-8B-Base. The paper claims generality—that the findings about reasoning mechanisms are not specific to this model—but provides no evidence with 1B, 3B, or 14B variants. If the double dissociation only appears at 8B scale, or if the switch amplification effect requires a certain model capacity, the paper's theoretical claims would need qualification.

• Latency and throughput measurements: Table 4 reports token counts but no wall-clock timing. The paper makes claims about practical utility (DPO, Best-of-N) but doesn't report how long Mix-GRM takes to generate evaluations compared to baselines. B-CoT with its longer average output (824 tokens) might be slower than the Base-GRM's unstructured output, offsetting some of the accuracy gains in latency-sensitive applications.

• Comparison against ensemble methods: The B-CoT mechanism (sampling multiple rationales, merging them) has conceptual similarity to ensembling (run multiple evaluators, aggregate their judgments). A baseline that simply runs the Base-GRM 2–3 times and majority-votes the verdicts would test whether the structured B-CoT provides benefits beyond what naive aggregation achieves. If simple ensembling matches B-CoT performance, the synthesis pipeline's complexity is harder to justify.

• Statistical significance testing: The paper reports point estimates without confidence intervals. Given the 500-question test sets (RewardBench) and similar sizes for other benchmarks, some of the smaller gaps (e.g., Mix-GRM SFT at 75.1 vs. FARE-8B at 75.9, a 0.8 point difference) may not be statistically distinguishable. The DPO results in Table 3 with small margins (46.4 vs. 45.9) are particularly suspect without variance estimates.

Conditional claims that hold with qualifications:

• "RLVR boosts Mix-GRM by a larger margin than Base-GRM": Supported (+4.3 vs. +3.6, Table 1), but this is a single comparison without ablations testing different RLVR hyperparameters or training durations. The paper runs 100 RLVR steps for both models—we don't know if Base-GRM would catch up with more steps or if Mix-GRM would pull further ahead.

• "Mix-GRM achieves best-in-class utility in both Offline RL and Test-time Scaling": Partially supported. For DPO, Mix-GRM achieves the best instruction-following (12.1) and math reasoning (46.4), but the margins are small and we lack significance testing. For Best-of-N, Mix-GRM leads on all four benchmarks (Figure 2), with the largest gap on MATH (+5.5 over RM-R1). The Best-of-N results are more convincing because the gaps are larger and consistent across benchmarks.

• "Single-mode mechanisms encounter hard performance ceilings under RLVR": Strongly supported for Breadth-only on Correctness (71.5 RLVR vs. 78.7 for Mix-GRM RLVR) and Depth-only on Preference (71.0 RLVR vs. 72.6 for Mix-GRM RLVR). The Breadth-only ceiling on Correctness is particularly striking—RLVR barely improves it over SFT (+1.3), while unstructured Base-GRM improves by +4.2 on the same domain. This is the paper's cleanest causal evidence that CoT structure constrains RL optimization potential.

6. Limitations and Trade-offs

6.1 The Difficulty Estimation Cost for Mechanism Selection Is Not Accounted For

The assumption or constraint. The entire Mix-GRM framework depends on the model's ability to autonomously select the appropriate reasoning mechanism (Breadth-CoT vs. Depth-CoT) at inference time based on the nature of the evaluation task. The paper demonstrates that this mechanism selection is learned through RLVR and achieves a 95% structural match rate post-training (Section 5, "Emergent Polarization Analysis"), but the model must still perform this classification implicitly for every inference request. The paper does not measure the computational cost, latency, or accuracy of this implicit task-classification step—it is simply folded into the model's standard forward pass and token generation.

This is not a minor accounting issue. The task classification problem—distinguishing subjective preference evaluations from objective correctness evaluations—is itself a non-trivial reasoning challenge. A query might involve both dimensions (e.g., a coding task where style and correctness are both relevant), and misclassification could route the evaluation to an inappropriate reasoning mechanism. The 95% structural match rate, while impressive, means 5% of evaluations are still potentially using suboptimal reasoning structures, and the paper provides no analysis of how those misclassified cases perform.

The consequence. In practice, the "adaptive" mechanism selection comes with an implicit inference-time overhead that the paper does not quantify. More importantly, the 5% of cases where the model selects the wrong mechanism represent an accuracy ceiling that is structural rather than capacity-limited—these are cases where the model's meta-cognitive judgment about what kind of evaluation is needed is incorrect, and additional training or scaling is unlikely to fix it without explicit mechanism-selection supervision. For deployment in RLHF pipelines processing millions of evaluation pairs, a 5% mechanism misalignment rate could systematically bias the reward signal in ways that are invisible to aggregate accuracy metrics.

Additionally, the model's mechanism selection is entirely black-box. Users cannot inspect why the model chose B-CoT over D-CoT for a particular query, making it difficult to debug systematic misclassifications. If a particular type of hybrid task (e.g., evaluating code documentation that requires both factual accuracy and clarity assessment) consistently triggers the wrong mechanism, there is no obvious intervention point short of retraining.

What evidence exists in the paper. The 73% → 95% structural match rate improvement is reported in Section 5 ("Emergent Polarization Analysis") as evidence that RLVR sharpens mechanism allocation. However, the paper provides no per-domain breakdown of the remaining 5% mismatch—we do not know whether the mismatches cluster in particular task types, difficulty levels, or ambiguity regimes. The paper acknowledges in the Limitations section that "real-world tasks often exist on a continuous spectrum or involve hybrid demands that intricately blend deductive rigor with multi-dimensional nuances," and that "our current categorization may act as a low-rank approximation of a higher-dimensional space of mechanisms," but this acknowledgment does not translate into any measurement of how hybrid tasks actually fare under the adaptive mechanism selection.

Mitigation status. The paper does not attempt to address this limitation. It suggests future work on "more granular taxonomies to achieve even more precise task-mechanism calibration" and "dedicated hybrid slicing benchmarks," but provides no partial solution for the current framework. A practical mitigation—such as a lightweight task classifier trained explicitly to route evaluation requests—would address the black-box nature of the mechanism selection but is not explored. The paper's philosophy is that the model should learn this autonomously, which is elegant but leaves the residual error uncharacterized and potentially uncorrectable without architectural changes.

---

6.2 All Results Are on a Single Model Family and Scale—Transferability Is Unverified

The assumption or constraint. Every experiment in the paper—benchmark evaluation (Tables 1, 2), DPO downstream training (Table 3), Best-of-N test-time scaling (Figure 2), mechanism analysis (Figure 3), and all ablations—uses Qwen3-8B-Base as the foundation model. The paper does not report results with any other model family (e.g., Llama, Mistral, Gemma), any other scale (e.g., 1B, 3B, 14B), or any other architecture type. The paper asserts generality implicitly by comparing against baselines built on different architectures (Skywork-Reward uses Llama-3.1-8B, JudgeLRM uses a 7B model, FARE-8B uses an 8B model, DeepSeek-GRM uses a 16B model), but none of these baselines were trained with the Mix-GRM framework, so they do not test whether the framework's benefits transfer.

The consequence. Three specific transfer questions are left unanswered, each with practical implications:

Scale dependence. The double dissociation between B-CoT and D-CoT (B-CoT helps preference but harms correctness; D-CoT helps correctness but harms preference) might depend on model capacity. Smaller models (1B–3B parameters) may lack the representational capacity to maintain two distinct reasoning mechanisms simultaneously, in which case the mixture approach could underperform a single-mode approach optimized for one domain. Conversely, larger models (14B–70B) might benefit less from structural optimization because their greater capacity allows them to handle both types of reasoning implicitly within unstructured CoT—in which case Mix-GRM's advantages are specific to the 8B scale where capacity is constrained enough for structural guidance to matter.

Architecture dependence. Qwen3-8B-Base has specific architectural properties (attention mechanism, training data mixture, tokenizer) that might interact with the Mix-GRM framework. For example, if Qwen3's pretraining data was particularly rich in structured reasoning examples, the model might be unusually receptive to learning separate Breadth and Depth mechanisms. A model pretrained primarily on unstructured text might not internalize the structural patterns as effectively from only 9K SFT samples.

Domain dependence. The paper's training data construction (HelpSteer3, Code-Preference, Math-DPO, WildGuard, OffsetBias) and evaluation benchmarks are heavily weighted toward English-language, technical-reasoning-heavy tasks. The mechanism synergy might not transfer to low-resource languages where the LLM's evaluation capabilities are weaker overall, or to domains like creative writing evaluation where neither pure preference coverage nor pure correctness verification captures the relevant quality dimensions.

What evidence exists in the paper. None. The paper provides zero cross-model ablation. The Appendix E mentions using "the open-weights DeepSeek-V3 model for both schema extraction and raw rationale generation" and states this "further underscores its robustness and generalizability, proving it is not overly sensitive to the choice of the underlying LLM," but this refers to the synthesis backbone (the LLM used to generate the training data), not the reward model being trained. This is a conflation of two distinct transferability questions. The reward model itself—the Qwen3-8B-Base fine-tuned via Mix-GRM—has not been tested on any other base architecture.

Mitigation status. Not addressed. The paper does not acknowledge the single-model limitation in its Limitations section, which focuses on the coarseness of the Preference/Correctness dichotomy and the rigidity of mechanism allocation on hybrid tasks. A reader deciding whether to adopt Mix-GRM for a production system built on Llama-3-70B or Mistral-7B has no empirical guidance on whether the framework's benefits will transfer.

---

6.3 The Synthesis Pipeline Requires a Strong External LLM, Creating a Hidden Dependency

The assumption or constraint. The Mix-GRM training data synthesis pipeline—Schema Standardization (decomposing raw rationales into atomic triples), B-CoT merge-and-deduplicate, D-CoT reasoning trace generation and judgment regeneration—relies on DeepSeek-V3 (0324 snapshot) as the backbone generator, with sampling temperature T=0.8 for diversity. The paper treats this as a fixed, black-box component of the training infrastructure. However, the quality of the synthesized B-CoT and D-CoT rationales depends fundamentally on DeepSeek-V3's evaluation capabilities, reasoning quality, and language understanding. If DeepSeek-V3 produces poor principle decompositions, misses important evaluation dimensions, or generates flawed reasoning traces, those errors propagate into the training data and limit the quality of the resulting Mix-GRM.

This creates a capability ceiling: Mix-GRM can at best approach the evaluation quality of its synthesis backbone, because it is trained to imitate DeepSeek-V3's structured reasoning patterns. The RLVR stage can correct verdict direction (matching human labels) but cannot invent new reasoning capabilities that were not present in the SFT data—it can only amplify and sharpen existing structural patterns.

The consequence. The paper's claim of "data efficiency" (9K SFT samples achieving performance comparable to 2.5M samples) is partially attributable to training data quality rather than data structure alone. The 9K Mix-GRM samples are high-quality because they were synthesized by DeepSeek-V3, a state-of-the-art proprietary model. The 2.5M FARE-8B samples were also synthetically generated, but potentially with a weaker generator or without the structured synthesis pipeline. A fair efficiency comparison would control for the quality of the synthesis backbone. If FARE-8B's 2.5M samples had been synthesized using DeepSeek-V3 with similar structural templates, would the performance gap persist? The paper provides no evidence either way.

This also creates a practical barrier to reproduction. Reproducing Mix-GRM requires access to a model of DeepSeek-V3's caliber for data synthesis. While DeepSeek-V3 has open weights, running it for synthesis requires substantial computational resources. The paper does not report the total compute cost of synthesizing the 9K SFT samples—how many GPU-hours did the DeepSeek-V3 inference take? For research groups without access to models of this quality, the Mix-GRM framework may be substantially less effective, because the synthesized training data would be of lower quality.

What evidence exists in the paper. Appendix E reports that the synthesis pipeline is stable across stages (Raw Rationale accuracy: 87.1%, Merge B-CoT: 90.2%, Generate D-CoT: 88.5%) and that training on unfiltered synthesized data performs comparably to training on filtered data (69.8 vs. 70.1, Table 13). These numbers characterize the synthesis pipeline's quality given DeepSeek-V3 as the backbone, but provide no comparison against alternative backbones. The paper does not measure, for example, whether using a weaker model (Qwen3-8B-Base itself) for synthesis would degrade downstream Mix-GRM performance, or whether using an even stronger model would improve it.

Mitigation status. The paper partially acknowledges this in Appendix E's final paragraph: "we utilize the open-weights DeepSeek-V3 model for both schema extraction and raw rationale generation. The effectiveness of our synthesis pipeline with an open-source model further underscores its robustness and generalizability, proving it is not overly sensitive to the choice of the underlying LLM." This is misleading—DeepSeek-V3 is open-weight but is not a generic "open-source model" in terms of capability. It is a frontier-class model competitive with proprietary alternatives. The claim of "not overly sensitive" is asserted rather than tested, since no alternative backbone comparison exists. Future work on this limitation would require a systematic backbone ablation and a cost analysis of the synthesis compute.

---

6.4 The Structural Match Rate Metric Is Undefined and Potentially Circular

The assumption or constraint. The paper's central evidence for the "switching amplifier" claim—that RLVR induces emergent mechanism polarization—is the 73% → 95% structural match rate improvement reported in Section 5 ("Emergent Polarization Analysis"). The paper describes this metric as "the distribution of generated CoT structures on the test set using specific structural indicators (e.g., principle counts and trigger phrases)." However, the paper does not define what constitutes a structural match, what the classification threshold is, whether the classification is automated (by rule or by an auxiliary classifier) or manual, or what the "trigger phrases" are.

This vagueness creates a significant measurement validity problem. If the structural indicators are coarse—for example, classifying any CoT containing more than a threshold number of distinct principles as B-CoT—then the 95% match rate might reflect the model learning a superficial formatting convention (mention many principles on preference tasks, mention few on correctness tasks) rather than genuinely internalizing different reasoning mechanisms. The model could achieve a high match rate by producing verbose but shallow breadth on preference tasks and terse depth on correctness tasks without actually implementing the substantive reasoning processes the paper claims.

The consequence. The 95% structural match rate cannot be interpreted as strong evidence for mechanism internalization without knowing what it measures. If the classification is based on surface features (token counts, keyword presence, section markers), the improvement from 73% to 95% might simply mean that the model learned to produce different text shapes for different domains—a stylistic adaptation rather than a reasoning adaptation. The paper's case studies in Table 5 provide qualitative validation that B-CoT and D-CoT differ substantively when they work well, but provide no evidence about the 95% of cases where the model is classified as "matching"—we don't know whether those matches represent genuine structural reasoning or superficial formatting.

Additionally, the metric's computation on the "test set" raises circularity concerns if the test set overlaps with the RLVR training distribution in ways that allow the model to learn domain-correlated stylistic cues. The RLVR training data comes from the same source datasets (HelpSteer3, Code-Preference, Math-DPO, WildGuard, OffsetBias) that the test benchmarks (RewardBench, RMB, RM-Bench, PPE) draw from, creating potential for distributional overlap where stylistic patterns present in training are predictive of domain without reflecting genuine reasoning adaptation.

What evidence exists in the paper. No methodological detail is provided for the structural match rate computation. Section 5 mentions "principle counts and trigger phrases" as indicators but does not specify threshold values, does not provide the classification algorithm, and does not report inter-annotator agreement if manual classification was used. The paper does not provide examples of CoTs that were classified as matches vs. mismatches, making it impossible for readers to assess the metric's face validity. Figure 3 provides aggregate structural indicators (average token length per judgment, average principle count, percentage classified as Breadth/Depth) but does not show the distribution of these indicators across domains—we see averages but not whether the separation between B-CoT and D-CoT is clear-cut or overlapping.

Mitigation status. Not addressed. This is the most significant methodological gap in the paper because the "switching amplifier" claim—arguably the paper's most novel theoretical contribution—rests almost entirely on this unvalidated metric. A proper mitigation would include: (1) a clear operational definition of structural match with explicit classification rules, (2) examples of matched and mismatched CoTs with classification justifications, (3) inter-rater reliability if human judgment was used, (4) a per-domain breakdown of the 5% mismatch cases to characterize failure modes, and (5) a correlation analysis showing that higher structural match rate predicts higher evaluation accuracy at the instance level (not just at the domain level). Without these, the 73% → 95% improvement remains an intriguing signal that cannot be distinguished from a measurement artifact.

---

6.5 No Direct Comparison Between RLVR-Elicited Mechanism Allocation and Explicit Routing

The assumption or constraint. The paper's architecture relies on the model autonomously learning to deploy the appropriate reasoning mechanism through RLVR, without explicit structural supervision. The evidence for this working is the 95% structural match rate and the performance improvements in Table 2. However, the paper never compares this emergent approach against a simpler alternative: an explicit router that uses a lightweight task classifier (trained on the same domain labels the paper already uses to construct the SFT mixture) to decide whether to prompt the model to use B-CoT or D-CoT at inference time.

This is a significant omission because the explicit routing approach would address several limitations simultaneously: (1) it eliminates the 5% mechanism misclassification rate by making the mechanism selection deterministic and inspectable, (2) it would work immediately after SFT without requiring RLVR to sharpen the mechanism allocation, (3) it makes the mechanism selection auditable—if the model produces a bad evaluation, developers can check whether it used the wrong mechanism and correct the routing logic, and (4) it would help characterize whether the RLVR gains come from improved mechanism selection or from improved reasoning quality within each mechanism.

The consequence. Without the explicit routing baseline, we cannot determine whether the emergent mechanism polarization is actually necessary for the observed performance gains, or whether a simpler approach would achieve equivalent results. The paper implicitly argues that autonomous mechanism selection is superior because the model can handle ambiguous cases that don't cleanly fit the Preference/Correctness dichotomy. But this argument is not tested—the paper provides no analysis of how the model handles genuinely ambiguous evaluation tasks, and the Limitations section acknowledges that the framework may be "rigid" on hybrid tasks.

The practical implication is that a practitioner deploying Mix-GRM has no guidance on whether to trust the model's implicit mechanism selection or to implement an explicit routing layer. If the model's 5% misclassification rate clusters in specific task types (e.g., safety evaluation, where both factual correctness and social appropriateness matter), an explicit router with domain-specific rules could outperform the 95% autonomous match rate. Conversely, if the autonomous selection captures subtleties that a simple domain classifier would miss, the emergent approach would be genuinely superior. The paper provides evidence for neither interpretation.

What evidence exists in the paper. None. No explicit routing baseline is reported. The paper's closest comparison is the single-mode models (Mix-GRM Breadth, Mix-GRM Depth) in Table 2, which represent what would happen if all tasks were routed to a single mechanism regardless of domain. These models perform worse than the adaptive Mix-GRM, but this only shows that adaptation is better than no adaptation—it does not show that emergent adaptation is better than explicit, deterministic adaptation.

Mitigation status. Not addressed. The paper does not acknowledge the absence of an explicit routing baseline as a limitation. The vision of autonomous mechanism selection is central to the paper's narrative ("the model spontaneously converges on the optimal thinking style"), and a comparison against explicit routing would be a natural test of whether this autonomy is genuinely beneficial or merely an elegant implementation detail. Future work should at minimum compare the 95% autonomous match rate against a classifier trained on the same domain labels, measuring both overall accuracy and per-domain performance to characterize where autonomous selection adds value and where it introduces errors.

---

6.6 The Preference/Correctness Dichotomy Is Too Coarse for Production Reward Modeling, and the Paper Provides No Guidance on Finer-Grained Mechanism Design

The assumption or constraint. The entire Mix-GRM framework is built on a binary task taxonomy: every evaluation instance is categorized as either Preference (subjective, multi-dimensional quality) or Correctness (objective, logically verifiable). B-CoT is matched to Preference, D-CoT to Correctness. The paper constructs its SFT mixture dataset using this binary mapping, computes the structural match rate against this binary expectation, and interprets all mechanism-efficiency results (Table 2) through this binary lens.

The Limitations section acknowledges this explicitly: "this dichotomy represents a coarse-grained mapping of the diverse alignment landscape. Real-world tasks often exist on a continuous spectrum or involve hybrid demands that intricately blend deductive rigor with multi-dimensional nuances." However, the paper provides no empirical characterization of how the binary framework degrades on tasks that don't cleanly fit either category, and offers no diagnostic for practitioners to determine whether their specific evaluation use case is well-served by the B-CoT/D-CoT dichotomy.

The consequence. Production reward modeling rarely involves pure Preference or pure Correctness tasks. Consider these realistic evaluation scenarios:

• Safety evaluation: A response might be factually correct but harmful (correctness: yes, preference: no). The evaluation requires both logical verification (does the response contain dangerous information?) and multi-dimensional judgment (is the tone appropriate? does it refuse unsafe requests appropriately?). Neither pure B-CoT nor pure D-CoT captures this hybrid demand.

• Instruction-following with factual constraints: A user asks for a summary of a specific event with a word limit. The evaluation must check both factual accuracy (correctness) and adherence to format constraints (preference). B-CoT's breadth might verify the word limit but miss factual errors; D-CoT's depth might verify the facts but miss the format violation.

• Code review: Evaluating a code response requires checking functional correctness (does it compile? does it pass tests?), algorithmic efficiency, code style, and documentation quality. This spans correctness (functionality) and preference (style, readability) simultaneously.

The paper's Table 2 shows that single-mode models underperform on their non-target domains (B-CoT on Correctness: −2.0; D-CoT on Preference: −2.3), which means that a hybrid task evaluated by the adaptive Mix-GRM might receive a mechanism that is suboptimal for half the evaluation criteria. The 95% structural match rate tells us the model selects one mechanism or the other—it does not tell us whether the model blends mechanisms on hybrid tasks, or how performance degrades when it must evaluate hybrid quality through a single mechanism.

What evidence exists in the paper. The fine-grained sub-task analysis in Appendix D (Tables 10 and 11) provides some insight. Certain sub-tasks show patterns that suggest hybrid demands: on RewardBench Code (Table 11), Mix-GRM Breadth (RLVR) actually achieves 96.3—the highest score—while Mix-GRM Depth (RLVR) achieves 94.6. Since Code evaluation on RewardBench likely involves both correctness and style assessment, this suggests Breadth may be the default mechanism for code evaluation, but we don't know whether this is optimal or whether a hybrid mechanism would perform better. Conversely, on RM-Bench Chat Helpfulness (Table 10), the Depth variant (75.6) outperforms the Breadth variant (70.1) in the RLVR stage, suggesting that helpfulness evaluation—nominally a preference task—benefits from depth-oriented reasoning. These cross-domain anomalies hint that the binary taxonomy is indeed too coarse, but the paper does not analyze them as evidence of the dichotomy's limitations.

Mitigation status. The paper acknowledges the limitation explicitly in the Limitations section but treats it entirely as future work: "Future work could explore more granular taxonomies to achieve even more precise task-mechanism calibration" and "developing dedicated hybrid slicing benchmarks to explicitly evaluate the trade-off between style and logic, alongside the design of more sophisticated, fine-grained hybrid structures (e.g., soft-routing mechanisms)." No partial solution is provided for the current framework—no mechanism for detecting hybrid tasks, no fallback strategy when the binary classification is ambiguous, and no empirical characterization of how performance degrades as task hybridity increases. This leaves practitioners with a framework that is known to be optimal only for tasks that cleanly separate into pure preference or pure correctness, with no tools for determining whether their specific tasks meet that criterion.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the GRM development conversation from a quantity-centric paradigm ("longer reasoning chains → better evaluation") to a structure-aware paradigm ("the right reasoning mechanism for the right task → better evaluation"). This is not a paradigm shift in the Kuhnian sense — the basic GRM architecture (generate rationale, then verdict) remains intact — but it is a substantial reframing of what practitioners should optimize. Prior to this work, the dominant approaches to improving GRMs were either RL-driven CoT elongation (JudgeLRM, RM-R1, DeepSeek-GRM) or synthetic data expansion (FARE-8B, RubricRM, Prometheus). Both implicitly assumed that reasoning quality correlates monotonically with reasoning quantity. Mix-GRM demonstrates that this assumption is not just incomplete but actively counterproductive when the reasoning structure is mismatched to the evaluation domain — B-CoT on correctness tasks and D-CoT on preference tasks both underperform an unstructured baseline (Table 2, SFT stage: −2.0 and −2.3 respectively).

The practical consequence is that data scaling alone is no longer a defensible sole strategy for GRM improvement. The paper's 275× data efficiency (9K SFT samples achieving 75.1 vs. FARE-8B's 2.5M samples achieving 75.9) directly challenges the economic logic of brute-force data expansion. If a carefully structured 9K samples can match the performance of 2.5M unstructured samples, then the marginal ROI of additional unstructured training data is substantially lower than the field has assumed. This does not mean data scaling is obsolete — Mix-GRM with 2.5M structured samples might outperform both — but it means that data quality (defined structurally) is a higher-leverage intervention than data quantity for GRMs at the 8B scale. Research groups with limited compute budgets should prioritize reasoning structure design over data volume expansion.

The paper also reconciles conflicting intuitions about when and why different evaluation strategies work. The test-time scaling literature had identified parallel and sequential thinking as orthogonal mechanisms for amplifying intelligence (Zhang et al., 2026), but this insight had not been operationalized in reward modeling. Prior GRM work produced models that were sometimes surprisingly good at preference tasks and surprisingly bad at correctness tasks (or vice versa) without a vocabulary for diagnosing why. Mix-GRM provides that vocabulary: a GRM's evaluation failures are often mechanism mismatches rather than capability deficits. The case studies in Table 5 make this concrete — a model that fails to catch a language mismatch is not "bad at evaluation" in general; it is deploying depth-oriented reasoning when breadth-oriented reasoning is needed. This diagnostic framework enables more targeted model improvement than the current practice of "add more data and retrain."

The "switching amplifier" finding (RLVR sharpening mechanism allocation from 73% to 95% without structural supervision) reframes the role of RL in GRM training. The prior view was that RL elicits longer CoT traces — a quantitative effect. The paper's evidence suggests RL also induces qualitative structural adaptation — the model learns to think differently depending on what it is evaluating. This has implications beyond GRMs: if outcome-based RL can induce appropriate reasoning structures in evaluation tasks, it may similarly induce structure-appropriate reasoning in other meta-cognitive domains (planning, self-critique, multi-step decision-making). The finding that single-mode models hit performance ceilings under RLVR (Breadth-only on Correctness at 71.5 vs. Mix-GRM at 78.7) further suggests that structural diversity in the base policy is a prerequisite for RL to achieve large gains — a principle that may generalize to RL fine-tuning more broadly.

Research directions that become more attractive include: structural mechanism design for other meta-cognitive tasks (summarization evaluation, translation quality assessment, dialogue coherence judgment); RL-driven mechanism discovery in domains where the optimal reasoning structure is unknown rather than designed; and systematic characterization of when outcome-based rewards suffice to induce process-level adaptation versus when explicit process supervision is necessary.

Research directions that become less attractive include: further work on pure CoT length scaling for GRMs without structural considerations (the paper shows that unstructured length scaling hits a lower ceiling than structured approaches); training increasingly large GRMs on increasingly large unstructured datasets (the 275× efficiency gap suggests this is an inefficient use of compute); and developing more complex search algorithms for evaluation (the paper shows that the bottleneck is reasoning structure, not search sophistication — even best-of-10 with a structurally-aware verifier outperforms prior methods).

Follow-Up Research This Work Enables

Causal intervention experiments to validate the mechanism-switching claim. The paper's 73% → 95% structural match rate improvement is correlational — we observe that the model deploys the "right" mechanism more often after RLVR, but we do not have direct causal evidence that mechanism choice causes accuracy improvements. A clean experiment would take the post-RLVR Mix-GRM and force it to use the wrong mechanism on a held-out test set through explicit prompting ("Evaluate this math problem by considering many different principles" vs. "Evaluate this math problem by first solving it yourself, then checking the response step by step"). If the forced-mismatch condition degrades performance relative to the free-choice condition (which should achieve ~95% match), this demonstrates that mechanism selection is causally responsible for accuracy, not merely correlated with it. The experiment should be run on both Preference and Correctness domains, and should include a third condition where the model is forced to use the right mechanism as a sanity check (should match or slightly exceed free-choice performance). This would transform the switching amplifier claim from an interesting observation to a validated causal mechanism.

Fine-grained mechanism taxonomy and soft routing. The paper acknowledges that the Preference/Correctness dichotomy is coarse. A natural extension would construct a finer-grained taxonomy of evaluation sub-tasks and design corresponding reasoning mechanisms. For example, within Preference: (a) stylistic quality (fluency, tone, engagement — benefits from breadth), (b) constraint satisfaction (word limits, format requirements — benefits from structured checklist reasoning), (c) safety/harmfulness (requires both factual verification and social norm reasoning — hybrid). Within Correctness: (a) deductive reasoning (math proofs — benefits from step-by-step verification), (b) factual accuracy (knowledge retrieval — benefits from source-comparison reasoning), (c) code correctness (benefits from execution-simulation reasoning). The experiment would synthesize training data for each mechanism, train a single model on the full mixture, and measure whether the model learns a more nuanced mechanism-to-subtask mapping than the binary B-CoT/D-CoT split. The key metric would be whether performance on hybrid tasks (e.g., evaluating code that must be both correct and well-documented) improves over the binary approach. A soft-routing mechanism — where the model generates a mixture of Breadth and Depth reasoning with attention weights that vary per task — would be a natural architectural extension that the paper's Limitations section explicitly calls for.

Cross-model and cross-scale replication of the double dissociation. The paper's central empirical result — B-CoT helps preference but harms correctness; D-CoT helps correctness but harms preference — is demonstrated on exactly one model (Qwen3-8B-Base). Reproducing this on Llama-3-8B, Mistral-7B, and Gemma-7B would establish whether the dissociation is a general property of how language models process structured reasoning or an artifact of Qwen3's specific pretraining. Even more informative would be replication at 1B, 3B, and 14B scales (using Qwen3 family for consistency): the paper hypothesizes that structural optimization is particularly important at constrained model scales where capacity is limited, but this hypothesis is untested. If the double dissociation is weaker or absent at 1B (model too small to maintain distinct mechanisms) and weaker at 14B (model large enough to handle both types implicitly), that would characterize the scale regime where Mix-GRM is most valuable. If the dissociation strengthens with scale (larger models benefit even more from structural optimization), that would suggest structural design is a universal scaling principle, not a capacity-compensation trick.

Mechanism ablation through SFT mixture ratio sweeping. The paper uses a fixed assignment: B-CoT for 100% of Preference tasks, D-CoT for 100% of Correctness tasks. What happens when this assignment is corrupted? Train variants where 10%, 25%, or 50% of Preference tasks receive D-CoT rationales (and symmetrically for Correctness), then measure: (a) post-SFT structural match rate, (b) post-RLVR structural match rate, (c) final evaluation accuracy on both domains. This would characterize how robust the mechanism alignment is to SFT label noise and how effectively RLVR can correct misaligned priors. The paper's emergent polarization analysis suggests RLVR would recover the correct mapping even with substantial noise — if the 50% corruption variant still achieves >90% structural match rate after RLVR, this is strong evidence that the mechanism allocation is genuinely discovered through reward optimization rather than merely inherited from SFT. If performance degrades substantially at high corruption levels, it would indicate that SFT provides an essential initialization that RLVR cannot fully override, which would qualify the "autonomous discovery" narrative.

Synthesis backbone sensitivity analysis. The paper uses DeepSeek-V3 as the backbone for schema extraction and rationale generation, and claims the pipeline is "not overly sensitive to the choice of the underlying LLM" (Appendix E) without testing this claim. A systematic ablation would train Mix-GRM variants using synthesis backbones of varying quality: (a) DeepSeek-V3 (frontier), (b) Qwen3-8B-Instruct (strong open-weight, same family as the reward model), (c) Llama-3-8B-Instruct (different family, strong), (d) Qwen3-1B-Instruct (weak, same family). The key measurement is whether Mix-GRM performance degrades gracefully with backbone quality or collapses below some threshold. If Qwen3-8B-Instruct-synthesized data produces a Mix-GRM within 1-2 points of DeepSeek-V3-synthesized data, the framework is genuinely backbone-robust and practical for groups without frontier model access. If performance collapses with weaker backbones, the framework's "data efficiency" is partly a function of DeepSeek-V3's quality rather than Mix-GRM's structural design, and the paper's claims about data efficiency need to be qualified as "efficiency given access to frontier-quality synthesis."

Hybrid task benchmark construction and stress-testing. The paper's Limitations section identifies hybrid tasks as a key challenge but does not empirically characterize the degradation. A targeted benchmark could be constructed by taking existing evaluation instances and deliberately creating hybrid demands: (a) math problems where the correct answer is embedded in a poorly formatted, rude, or unhelpful explanation (the evaluator must recognize both the correctness of the answer and the unacceptability of the presentation); (b) open-ended chat tasks where one response is more helpful but less factually accurate (requiring the evaluator to trade off preference and correctness dimensions); (c) code generation tasks where one solution is correct but unreadable and another is elegant but contains a subtle edge-case bug. Evaluating Mix-GRM, B-CoT-only, D-CoT-only, and an explicit two-stage evaluator (first check correctness with D-CoT, then check quality with B-CoT, then aggregate) on this benchmark would characterize where the binary mechanism framework breaks down and whether simple compositional approaches can address the hybrid case. The paper's acknowledgement that real-world tasks "exist on a continuous spectrum" demands this kind of stress-test before Mix-GRM can be confidently deployed in production RLHF pipelines.

Practical Applications and Downstream Use Cases

Cost-efficient GRM training for mid-scale LLM alignment. The paper's 275× data efficiency (9K SFT vs. 2.5M for comparable performance) directly translates to reduced training costs for organizations building custom reward models. The concrete workflow: (1) collect a small, high-quality preference dataset across target domains (on the order of 10K samples, achievable with ~$2K-5K in human annotation costs at current rates, or synthetic generation using an API-based frontier model); (2) run the Mix-GRM synthesis pipeline using DeepSeek-V3 (or comparable) to produce structured B-CoT and D-CoT rationales; (3) SFT an 8B base model on this mixture; (4) optionally RLVR-train on an additional 20K samples if verdict accuracy needs improvement. The total compute cost for training is substantially lower than the 2.5M-sample SFT runs required by FARE-8B, making state-of-the-art GRM development accessible to research labs without industrial compute budgets. The paper's Table 2 domain-level results provide guidance on when this investment is worthwhile: if the deployment scenario primarily involves subjective preference evaluation (chat quality, content moderation, style assessment), a B-CoT-only variant might suffice; if it primarily involves correctness verification (math tutoring, code review, factual QA), a D-CoT-only variant might suffice. The full Mix-GRM is most valuable when the evaluation workload spans both domains.

Best-of-N verifier deployment for reasoning-heavy applications. Figure 2 demonstrates that Mix-GRM as a verifier for Best-of-10 selection achieves 43.2% on MATH, outperforming RM-R1 (37.7%) and FARE-8B (35.2%). In practical terms, this means a system that generates 10 candidate solutions from a relatively cheap generator model (Llama-3.1-8B) and uses Mix-GRM to select the best one can achieve MATH accuracy competitive with models that are 5-10× more expensive per query. Deployment scenario: a math tutoring application that must provide correct step-by-step solutions to student queries. Instead of running a single expensive frontier model (GPT-4, Claude) on each query, the system runs a cheap 8B generator 10 times in parallel (high throughput, low per-token cost) and uses Mix-GRM to select the best solution. The 43.2% MATH accuracy, while not production-grade for high-stakes applications, represents a substantial improvement over the 33% random baseline and demonstrates that the verifier is extracting genuine signal from the candidate pool. The consistent lead across MATH, CHAMP, MBPP+, and BigCodeBench (Figure 2) suggests the approach generalizes across reasoning domains. The key engineering insight is that Mix-GRM's D-CoT mechanism — which catches logical errors by grounding evaluation in self-solved reasoning traces — is particularly valuable for reasoning verification, and the performance gap over RM-R1 (+5.5 on MATH) quantifies the benefit of structured depth over RL-elicited length.

RLHF data construction with mechanism-appropriate quality signals. The DPO results in Table 3 show that Mix-GRM-trained policies achieve both the best instruction-following (12.1 win rate) and the best math reasoning (46.4 accuracy), without the tradeoff that other RMs exhibit. This has direct implications for RLHF pipeline design: when constructing preference pairs for DPO training, the reward model's evaluation mechanism should match the nature of the training data. If the preference data includes both subjective chat quality comparisons and objective correctness comparisons (as most general RLHF datasets do, e.g., UltraFeedback), a single-mechanism RM will systematically underweight one dimension. The concrete deployment workflow: (1) use Mix-GRM to score candidate responses for each prompt in the RLHF data pool, (2) construct DPO pairs using the Pairwise Scoring Aggregation strategy described in Appendix A.4 (score all pairs, select highest and lowest cumulative scores), (3) train the policy via DPO on these pairs. The paper's result that Mix-GRM-trained policy maintains math reasoning (GSM8k: 77.6% vs. 75.1% for the SFT baseline) while improving instruction-following (12.1 vs. 5.3) is the key selling point — the reward model is not inducing a capability tradeoff. This is particularly valuable for general-purpose assistant training, where the policy must be both helpful and correct.

On-device or edge deployment of lightweight verifiers. The paper's demonstration that a structurally-optimized 8B reward model can approach the performance of much larger or more data-intensive alternatives has implications for deployment scenarios where model size is constrained. An 8B verifier can run on consumer GPUs or high-end edge devices with acceptable latency. A concrete scenario: a privacy-sensitive code review assistant that runs entirely on-device, generating candidate code fixes with a small local model and using Mix-GRM to select the most correct fix before presenting it to the user. The D-CoT mechanism is particularly well-suited here because code correctness verification benefits from reasoning-trace-grounded evaluation. The paper's Best-of-10 results on BigCodeBench (Figure 2) provide initial evidence that Mix-GRM effectively discriminates correct from incorrect code solutions. The token cost analysis in Table 4 (~700-800 inference tokens per evaluation) suggests per-query latency would be manageable for interactive use cases when running on appropriate hardware.

When to Prefer This Method

• Prefer Mix-GRM over data-scaled GRMs (FARE-8B, RubricRM) when training data budget is constrained (thousands to low tens of thousands of samples rather than millions) AND the evaluation workload spans both subjective preference and objective correctness tasks. The paper shows Mix-GRM matches FARE-8B's SFT performance with 275× less data (75.1 vs. 75.9) and exceeds it after RLVR (79.4 vs. 75.9). The advantage is largest when both evaluation types are needed, because single-mechanism approaches systematically underperform on their non-target domain (Table 2: Breadth-only on Correctness drops 2.0 points under SFT baseline; Depth-only on Preference drops 2.3 points).

• Prefer Mix-GRM over RL-elicited GRMs (JudgeLRM, RM-R1) when the GRM must generalize across diverse evaluation benchmarks rather than specializing in reasoning-heavy tasks. JudgeLRM achieves 68.8 average (Table 1) and RM-R1-Instruct achieves 70.1, compared to Mix-GRM's 79.4 — a ~10 point gap driven primarily by Mix-GRM's strength on preference tasks where RL-elicited models underperform (Table 10: RM-R1-Instruct achieves 62.1 on Preference vs. Mix-GRM's 72.6 after RLVR). The RL-elicited models' strength in correctness tasks (RM-R1-Distill achieves 70.7 on Correctness post-RLVR, Table 11) suggests they may still be preferable for pure correctness evaluation workloads, but the paper's Mix-GRM achieves 78.7 on Correctness — surpassing even the RL-elicited specialist on its home domain.

• Prefer B-CoT-only or D-CoT-only variants when the evaluation workload is narrowly scoped to one domain AND structural match rate is carefully monitored. Table 2 shows that B-CoT-only RLVR achieves 71.0 on Preference (competitive with Mix-GRM's 72.6) and D-CoT-only RLVR achieves 77.7 on Correctness (competitive with Mix-GRM's 78.7). The single-mode models require less complex data synthesis and avoid the 5% mechanism misclassification rate. However, the paper shows these single-mode variants hit performance ceilings under RLVR (Breadth-only on Correctness plateaus at 71.5), so this choice is only appropriate when the deployment domain is truly narrow and unlikely to drift.

• Do not prefer Mix-GRM when the evaluation workload involves tasks that are genuinely hybrid (simultaneously requiring rigorous correctness verification and nuanced multi-dimensional preference judgment) AND an explicit, auditable evaluation process is required. The paper provides no characterization of hybrid-task performance and the autonomous mechanism selection is black-box — users cannot inspect why B-CoT was chosen over D-CoT for a given query. In regulated or safety-critical settings where evaluation decisions must be explainable and contestable, an explicit two-stage evaluator (correctness check followed by quality assessment, with transparent aggregation) may be preferable even if it achieves lower aggregate accuracy.