Fin-PRM: A Domain-Specialized Process Reward Model for Financial Reasoning in Large Language Models

ArXiv: 2508.15202

🎯 Pitch

General-purpose process reward models fail catastrophically in finance because they cannot distinguish plausible-sounding reasoning from factually correct financial logic. Fin-PRM solves this by injecting domain knowledge into reward training, enabling 12.9% gains in downstream task accuracy simply by selecting better reasoning traces.

---

1. Executive Summary

This paper introduces Fin-PRM, a domain-specialized process reward model designed to evaluate intermediate reasoning steps in financial tasks, trained on a newly constructed 3,000-sample dataset derived from the CFLUE benchmark using Deepseek-R1 as the teacher model. Fin-PRM integrates step-level and trajectory-level reward supervision through a dual-level training paradigm — combining Monte Carlo rollouts (importance scores), LLM-as-a-judge assessments (qualitative scores), and knowledge-verification checks (accuracy and coverage scores) — to provide fine-grained, factually-grounded evaluation of reasoning traces. When applied to three downstream tasks using Qwen2.5-7B-Instruct as the policy model, Fin-PRM yields gains of 12.9% in supervised fine-tuning via offline data selection, 5.2% in reinforcement learning via GRPO, and 5.1% in test-time Best-of-N selection on the CFLUE benchmark, establishing that domain-specialized, knowledge-aware process supervision substantially outperforms general-purpose PRMs only when the reward model is trained with verifiable signals grounded in expert-derived financial knowledge.

2. Context and Motivation

The Core Problem: General-Purpose PRMs Fail in High-Stakes, Knowledge-Intensive Domains

The fundamental problem this paper addresses is that process reward models trained on general or STEM domains systematically underperform when applied to domain-specific reasoning tasks like finance, where correctness depends not just on logical coherence but on factual accuracy, regulatory compliance, and domain-specific procedural knowledge. This is not merely a matter of incremental performance loss — the paper argues that general-purpose PRMs can be actively misleading in finance because they lack the expert knowledge needed to distinguish between a reasoning step that sounds plausible from one that is factually correct.

The authors frame this around three specific failure modes of existing PRMs when applied to finance (Section 1):

• Factual hallucination goes undetected: A general PRM might assign a high score to a reasoning step that is logically coherent but makes factually incorrect claims about financial concepts (e.g., mischaracterizing how price-to-book ratios work or misclassifying on-balance-sheet vs. off-balance-sheet items). Since general PRMs evaluate form (logical flow) rather than content (factual grounding), they cannot catch these errors.
• Domain-specific reasoning conventions are invisible: Financial reasoning follows particular structured patterns — distinguishing between asset, liability, and intermediary businesses; correctly applying regulatory frameworks; interpreting financial ratios with their standard meanings. A PRM trained on math proofs or science explanations has no exposure to these conventions and cannot assess whether a step follows valid financial logic.
• Reward hacking is amplified: In reinforcement learning settings, the policy model learns to exploit the PRM's weaknesses — generating reasoning that scores highly under the PRM but is actually incorrect. When the PRM lacks domain knowledge, the space of exploitable reasoning patterns is larger, because the PRM cannot distinguish between genuine financial reasoning and plausible-sounding nonsense.

This gap is significant because finance is a domain where errors have direct, quantifiable costs. A model that confidently produces incorrect financial analysis — misclassifying an off-balance-sheet exposure, miscalculating a regulatory capital requirement, or recommending an investment based on flawed reasoning about valuation metrics — can cause real financial harm. The authors position this as a domain where "truth is non-negotiable" (Section 7), making the unreliability of general-purpose PRMs an urgent practical problem.

What Prior Work Provides — and Where It Falls Short

The paper builds on several established lines of research, each of which provides useful foundations but fails to solve the domain-specialization problem:

General-domain PRMs are powerful but untethered from domain knowledge. State-of-the-art PRMs like Math-Shepherd (Wang et al., 2024), Skywork-PRM (He et al., 2024), and Qwen2.5-Math-PRM (Zhang et al., 2025) use human-annotated supervision or synthetic Monte Carlo rollouts to train step-level verifiers for mathematical reasoning. More recent work like ReasonFlux-PRM (Zou et al., 2025) combines step-level and trajectory-level signals, and Open-PRM (Zhang et al., 2024) uses outcome reward models (ORMs) to reverse-engineer process-level labels. These models achieve strong results on math and science benchmarks, but their training data consists entirely of general reasoning problems — algebraic derivations, physics calculations, code generation — with no exposure to financial concepts, terminology, or reasoning patterns. The paper's experiments make this failure concrete: Qwen2.5-Math-PRM-7B, a strong general-domain PRM, performs worse than majority voting at higher sample counts on financial tasks (Figure 2, Section 5.2), strongly suggesting that its scoring signal becomes anti-correlated with correctness when applied out-of-domain. This is a striking finding: it means that using a math-specialized PRM for finance can actively harm performance compared to simple voting, a clear indication of domain mismatch.

LLM-as-a-judge provides scalable evaluation but is opaque and ungrounded. Recent work has established that powerful LLMs can serve as effective evaluators of model outputs — the "LLM-as-a-judge" paradigm (Gu et al., 2025; Zheng et al., 2023). This approach is attractive because it eliminates the need for human annotation: you prompt a strong model to assess reasoning quality and use its outputs as training labels. However, the authors identify a critical limitation: LLM-as-a-judge evaluations are "often opaque and uninterpretable" (Section 2.2). The judge model makes assessments based on its own internal knowledge and biases, which may be incomplete or incorrect for specialized domains, and there is no mechanism to verify whether its judgments align with external facts. In finance, where correctness depends on verifiable facts (regulatory definitions, accounting standards, established financial ratios), this opacity is particularly dangerous — the judge might confidently assert that a reasoning step is correct while being factually wrong, and there is no built-in check against this.

Data synthesis pipelines exist for reasoning but aren't adapted for knowledge-intensive domains. The OpenThoughts framework (Guha et al., 2025) established systematic methodologies for reasoning data synthesis — prompting strong teacher models (like Deepseek-R1) to generate structured Chain-of-Thought traces, then using these traces to train student models. This provides a proven recipe for generating reasoning data at scale. However, OpenThoughts was validated on general reasoning tasks; in finance, there is an additional requirement that the synthesized reasoning be factually grounded in domain knowledge. A teacher model like Deepseek-R1 can produce plausible-sounding financial reasoning, but its outputs may contain factual errors or hallucinated financial concepts that a naive data synthesis pipeline would propagate into the training data without detection. The Central challenge, as the authors frame it in Section 1, is creating training data that features "not only verified outcomes but also granular, step-by-step demonstrations of expert financial reasoning."

Existing reward labeling methods lack verifiability. Prior approaches to generating reward labels for PRM training typically rely on outcome-based checking (did the final answer match the ground truth?) or Monte Carlo rollout success rates. These methods assess whether a reasoning path led to a correct answer but don't evaluate whether each step is independently correct. A step might be factually wrong but happen to lead to the right answer (or vice versa), creating noisy training signals. The authors argue that in finance, where factual precision is paramount, reward labels must be grounded in verifiable external knowledge — something no prior PRM training pipeline provides.

How This Paper Positions Itself

The paper positions itself at the intersection of three converging trends — PRM development, automated data synthesis, and LLM-as-a-judge evaluation — and argues that none of them individually solves the domain-specialization problem, but that combining them in a knowledge-aware framework does. The key intellectual move is to treat verifiability as a first-class requirement in reward modeling. Rather than relying solely on the teacher model's internal reasoning or on Monte Carlo success rates, Fin-PRM constructs reward signals that are anchored in an externally extracted, human-verified knowledge base derived from expert financial analyses.

This positioning is reflected in several distinctive design choices that differentiate Fin-PRM from prior work:

• Against ReasonFlux-PRM and similar trajectory-aware PRMs: While ReasonFlux-PRM also uses dual-level (step + trajectory) supervision, its reward signals are not grounded in external knowledge bases — they derive from template-guided trace-level scoring that evaluates reasoning form but not factual content. Fin-PRM adds the accuracy score ($r_{\text{acc}}$) and knowledge coverage score ($r_{\text{cover}}$), which explicitly check generated reasoning against a trusted knowledge base $\mathcal{K}$, creating a factual "anchor" that prior trajectory-aware PRMs lack.

• Against OpenThoughts: The data synthesis pipeline follows OpenThoughts' methodology for generating reasoning traces from a teacher model, but the authors add a crucial post-processing step: the generated traces are not just used as-is for SFT training. Instead, the paper treats the teacher model's final answer $a$ as silver truth only — explicitly acknowledging that the teacher model can be wrong ("We treat $a$ as a silver truth because the answer teacher model gives maybe wrong," Section 3.2) — and builds the reward model's training labels from multi-faceted verification against ground truth $y$ and knowledge base $\mathcal{K}$, not from the teacher's output alone.

• Against standard PRM training with only Monte Carlo rollouts: The importance score ($r_{\text{importance}}$) uses standard MC rollout methodology (generate $N=8$ completions from each step, check correctness rate), but the paper does not rely on this as the sole reward signal. It is one of three step-level components, balanced by qualitative assessment ($r_{\text{qual}}$) and, critically, accuracy verification ($r_{\text{acc}}$), which provides the factual grounding that pure MC-based PRMs lack.

• Against LLM-as-a-judge alone: While the qualitative score ($r_{\text{qual}}$) uses LLM-as-a-judge for assessing reasoning quality (semantic coherence, logic soundness, target progression), this is just one signal among several. The accuracy and coverage components serve as a cross-check, preventing the LLM judge's potential hallucinations or biases from dominating the reward signal. The authors note (Appendix B.1) that they compared Qwen3-235b-a22b with GPT-4.1 for this task and observed "almost the same score," suggesting the qualitative assessment is robust but still insufficient alone — hence the need for the knowledge-based verification components.

The paper also positions itself as a methodological template for domain-specialized PRMs beyond finance. The framework of (1) extracting a knowledge base from expert analyses, (2) generating reasoning traces from a teacher model, (3) constructing multi-faceted reward signals that combine importance estimation, qualitative assessment, and knowledge verification, and (4) training a dual-level reward model is explicitly presented as a "blueprint" for other high-stakes domains like law and medicine (Section 8). This positions the contribution not as finance-specific engineering but as a generalizable architecture for building PRMs that are both logically coherent and factually grounded in any domain where expert knowledge exists in structured form.

Why This Matters Beyond Finance

The paper implicitly argues for a broader shift in how the field thinks about reward model training. The dominant paradigm — train PRMs on large corpora of general reasoning problems using outcome-based or rollout-based labels — implicitly assumes that reasoning quality is domain-invariant: a good reasoning step in math looks like a good reasoning step in finance. The experimental evidence in this paper challenges that assumption directly. The finding that Qwen2.5-Math-PRM-7B underperforms majority voting on financial BoN selection (Figure 2) is not just a domain-transfer failure — it suggests that reasoning evaluation is partially domain-dependent, and that without domain-specific knowledge, a PRM's scoring signal can become actively harmful. This has implications for any field where reasoning depends on specialized factual knowledge: medicine (diagnostic reasoning requires medical knowledge), law (legal reasoning requires knowledge of statutes and precedents), engineering (design reasoning requires domain-specific physical constraints). The paper suggests that the future of reliable AI evaluation may require "a portfolio of specialized, reliable models" (Section 8) rather than a single general-purpose PRM, shifting the conversation from scaling up general evaluators to building communities of domain-expert evaluators.

3. Technical Approach

3.1 Reader Orientation

This paper presents a system for training a process reward model that evaluates the quality of step-by-step financial reasoning produced by large language models. The problem it solves is that general-purpose reward models, trained on math or science problems, systematically fail at assessing financial reasoning because they cannot distinguish between reasoning that sounds plausible and reasoning that is factually grounded in financial knowledge. The solution takes the shape of a dual-level, knowledge-verifiable reward architecture: the PRM is trained to score both individual reasoning steps (are they logically sound, factually correct, and useful?) and entire reasoning trajectories (do they reach the right answer and cover the necessary financial concepts?), with reward labels that are explicitly anchored in an external knowledge base extracted from expert financial analyses.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components connected in a pipeline:

1. Data Source (CFLUE benchmark) — provides financial questions along with ground-truth answers and human-written expert analyses. These expert analyses serve as the raw material for building a verified knowledge base.
2. Reasoning Trace Synthesizer (Deepseek-R1) — takes each CFLUE question and generates a multi-step Chain-of-Thought reasoning trace followed by a final answer. This produces candidate (question, reasoning trace, answer) triplets.
3. Knowledge Extraction Module (Qwen3-235b-a22b) — reads the expert analyses from CFLUE and extracts structured financial terms with their definitions into a knowledge base $\mathcal{K}$. This knowledge base becomes the factual reference against which the PRM's reward labels are verified.
4. Reward Signal Construction Pipeline — computes five distinct reward signals for each reasoning trace: three step-level scores (importance, qualitative quality, accuracy) and two trajectory-level scores (outcome correctness, knowledge coverage). These signals are then aggregated into binary labels for training.
5. Fin-PRM Model (fine-tuned Qwen2.5-7B-Instruct) — trained on the labeled data to predict both step-level and trajectory-level correctness scores using a joint binary cross-entropy objective. Once trained, it serves as a drop-in reward model for downstream applications.

Information flows as follows: CFLUE questions → Deepseek-R1 synthesizes reasoning traces → Qwen3 extracts financial knowledge from expert analyses → the reward pipeline computes multi-faceted scores for each step and trajectory using the question, trace, ground truth, and knowledge base → binary labels are constructed via aggregation and thresholding → Fin-PRM is fine-tuned to predict these labels → trained Fin-PRM is deployed for SFT data selection, Best-of-N inference, or RL reward shaping.

3.3 Roadmap for the Deep Dive

• First, the data synthesis pipeline — how reasoning traces are generated from CFLUE using Deepseek-R1, and how the financial knowledge base is extracted from expert analyses. This produces the raw material on which everything else depends.
• Second, the three step-level reward components ($r_{\text{importance}}$, $r_{\text{qual}}$, $r_{\text{acc}}$) — what each one measures, how it is computed, and why all three are needed rather than any single one.
• Third, the two trajectory-level reward components ($r_{\text{out}}$, $r_{\text{cover}}$) — how they capture global reasoning quality and knowledge usage, and why they complement step-level signals.
• Fourth, the reward label construction process — how the five raw signals are aggregated into binary step-level and trajectory-level labels using dynamic weighting and thresholding.
• Fifth, the Fin-PRM training objective — the joint step-level and trajectory-level binary cross-entropy loss, the hyperparameters, and what the model physically predicts at inference time.
• Sixth, the deployment recipes — how the trained Fin-PRM is used in three downstream applications (SFT data selection, Best-of-N inference, GRPO reinforcement learning), including the scoring formulas and hyperparameter choices.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems-building paper whose core idea is that domain-specialized process reward models must be trained with reward signals that are verifiable against external expert knowledge, not just derived from the teacher model's own reasoning or from outcome-based checking. The paper constructs an end-to-end pipeline — data synthesis, knowledge extraction, multi-faceted reward labeling, dual-level training — and validates it through three downstream applications.

---

Data Synthesis: Generating Reasoning Traces and Extracting Knowledge

The training of Fin-PRM starts with two parallel data generation processes that produce the two foundational artifacts: reasoning traces (the inputs to be evaluated) and a knowledge base (the factual reference for verification).

Synthesizing reasoning traces from CFLUE. The authors select CFLUE (Zhu et al., 2024) as the base data source because it is a Chinese financial benchmark where every question comes with a detailed, human-written expert analysis. This expert analysis is critical — it provides a ground-truth account of what correct financial reasoning for that question should look like, including the specific concepts that should be invoked and how they should be applied. The paper uses Deepseek-R1 (DeepSeek-AI et al., 2025) as the teacher model to generate structured reasoning traces. For each question $x$ in CFLUE, Deepseek-R1 is prompted with a detailed instruction template (shown in Listing 1 in Appendix A.2) that requires the model to produce output in two delimited sections:

• A Thought section (bounded by <|begin_of_thought|> and <|end_of_thought|> tags) containing the step-by-step reasoning process, with individual steps separated by \n\n (double newlines). The prompt instructs the model to engage in "a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration."
• A Solution section (bounded by <|begin_of_solution|> and <|end_of_solution|> tags) containing the final consolidated answer derived from the reasoning.

The model's output is parsed into a reasoning trace $s = (s_1, s_2, \dots, s_T)$, where each $s_t$ is an individual reasoning step (extracted by splitting on the double-newline separators), and a final answer $a$. Each resulting triplet $(x, s, a)$ becomes a candidate for which reward signals will later be generated. The paper explicitly notes that $a$ is treated as silver truth — not golden truth — because the teacher model can produce incorrect answers. This is an important design decision: the system does not assume the teacher is always correct, which is why downstream reward signals are verified against the external ground truth $y$ and knowledge base $\mathcal{K}$ rather than against the teacher's output.

Extracting the financial knowledge base. In parallel with trace generation, the authors construct a structured knowledge base $\mathcal{K}$ from the expert analyses provided by CFLUE. The extraction process uses Qwen3-235b-a22b (a powerful general-purpose LLM) prompted as a "financial knowledge extraction expert" using the template shown in Listing 2 (Appendix A.2). The prompt presents the expert analysis text from CFLUE and asks the model to identify all key financial terms and concepts, providing a concise definition for each based on the text. The output is a JSON list of objects, each containing a Term field (the financial concept) and an Explanation field (its definition).

For example, from an analysis discussing company valuation, the model might extract a term like "Price-to-Book Ratio" with an explanation like "A financial ratio used to compare a company's current market price to its book value." The resulting knowledge base $\mathcal{K}$ is global (spanning all questions in the training set), but for each specific question $x$, the system identifies the subset $\mathcal{K}_x \subseteq \mathcal{K}$ containing only the terms relevant to that question. This subset serves as the factual reference for the accuracy and coverage reward signals described later.

Why this approach over alternatives. The paper could have used the teacher model's own internal knowledge as the factual reference — essentially trusting Deepseek-R1 to know the correct financial concepts. Instead, the knowledge is extracted from human-written expert analyses that come with the CFLUE benchmark. This means the knowledge base is grounded in verified, curated domain expertise, not in whatever a large language model happens to have memorized (which could include errors, outdated information, or hallucinations). The authors frame this as a deliberate choice to make the reward signals "factually grounded" and to "mitigate issues like LLM hallucination and reward hacking" (Section 4.2).

Dataset scale and composition. The final dataset $\mathcal{D}$ for training Fin-PRM consists of 3,000 samples, where each sample is a tuple:

$(x, s, a, \mathcal{K}_x, y, y_{\text{analysis}})$

where $x$ is the original CFLUE question, $s$ is the reasoning trace from Deepseek-R1, $a$ is the silver-truth answer from Deepseek-R1, $\mathcal{K}_x$ is the question-specific knowledge subset, $y$ is the ground-truth correct answer from CFLUE, and $y_{\text{analysis}}$ is the expert analysis text from CFLUE. This structure means every training example provides the PRM with access to the question, the model's reasoning, the teacher's answer, the relevant domain knowledge, and the actual correct answer — giving the PRM all the information needed to learn what separates good financial reasoning from bad.

---

Step-Level Reward Modeling: Three Complementary Signals

The core innovation of Fin-PRM is that each reasoning step $s_t$ is evaluated using three independent reward signals, each capturing a different dimension of reasoning quality. These are then aggregated into a single score (Section 4.4). The authors argue that any single signal is insufficient — importance scores can be noisy when rollouts are stochastic, qualitative scores can reflect an LLM judge's biases, and accuracy scores are only as good as the knowledge base — but together they provide a robust, multi-faceted assessment.

Importance Score ($r_{\text{importance}}$): Monte Carlo Rollout-Based Utility

The importance score measures how useful a step is by estimating the probability that continuing from this step will lead to a correct final answer. This is the standard Monte Carlo rollout methodology used in prior PRM work (Wang et al., 2024; Lightman et al., 2023), adapted here with a specific model choice.

Computation procedure. For a given step $s_t$ in a reasoning trace, the system prompts a separate generator model — Qwen2.5-7B-Math — to complete the reasoning from that point onward. Specifically, for each step, the generator produces $N = 8$ independent completions (rollouts) starting from step $s_t$, each continuing until a final answer is reached. The importance score is then:

$r_{\text{importance}} = \frac{1}{N} \sum_{i=1}^{N} \mathbf{I}\left(\xi\left(R_{\mu,i}(s_t \mid x, s_{<t}, y)\right)\right)$

where $R_{\mu,i}(s_t \mid x, s_{<t}, y)$ is the $i$-th generated completion starting from step $s_t$ (conditioned on the question $x$, the preceding reasoning history $s_{<t}$, and the ground truth $y$), $\xi$ is the answer-checking process that determines whether the final answer of the rollout is correct, and $\mathbf{I}(\cdot)$ is the indicator function returning 1 if the rollout's final answer is correct and 0 otherwise.

What it computes. The importance score is a soft label in $[0, 1]$ representing the empirical success rate of completions from this step. If 6 out of 8 rollouts reach the correct answer, $r_{\text{importance}} = 0.75$. If none do, it is 0. The paper also defines a hard-label variant: if the soft label is not 0, the hard label is set to 1 (meaning the step is considered useful if even a single rollout succeeds). This binarization is used later in label construction.

Why this form. Monte Carlo rollouts provide a principled, model-free estimate of a step's expected utility. The key insight is that a good reasoning step should make it easier to reach the correct answer — not harder — and the proportion of successful completions is a direct measure of this. The choice of $N = 8$ balances statistical reliability (more rollouts reduce variance) against computational cost (each rollout requires a full generation). The use of Qwen2.5-7B-Math as the rollout generator (rather than Deepseek-R1 or the base Qwen model) is a specific design choice: a math-specialized model is chosen because the rollouts need to follow logical reasoning chains, and a smaller model (7B) is used to keep the cost manageable across thousands of steps.

Limitation acknowledged. The importance score alone is insufficient because it measures utility but not correctness. A step could have a high importance score because it happens to lead to correct completions by chance, even if the step itself contains factual errors. This is why the other two components are necessary.

Qualitative Score ($r_{\text{qual}}$): LLM-as-a-Judge Assessment of Reasoning Quality

The qualitative score captures the abstract, form-based quality of a reasoning step — its logical coherence, semantic soundness, and contribution toward solving the problem. This uses the LLM-as-a-judge paradigm with a specific judge model and structured evaluation rubric.

Judge model and prompt. The judge is Qwen3-235b-a22b, a large general-purpose LLM. The prompt template (shown in full in Listing 3, Appendix C.1) presents the judge with the original question $x$, the reasoning history $s_{<t}$ (all steps before the current one), and the current step $s_t$. The judge is instructed to evaluate the current step on three criteria, each scored as a float from 0.0 to 1.0:

• Logical Soundness: How coherent and logically valid is the reasoning within this specific step? (Does it follow from the previous steps? Is there a logical gap?)
• Step Correctness: Is the information presented in the step factually or procedurally correct, independent of the overall strategy? (This is about the step's internal validity, not whether it's on the right path.)
• Target Progression: How effectively does this step move the overall reasoning process closer to a correct final answer? (Is it productive, or is it a tangent?)

The judge returns a JSON object with three fields (logical_soundness, step_correctness, target_progression), each a float. The qualitative score $r_{\text{qual}}$ is then:

$r_{\text{qual}} = R_{\theta}(s_t \mid x, s_{<t}, a)$

where $R_{\theta}$ represents the judge model's scoring function, and the output is a scalar between 0 and 1 (the paper does not specify the exact aggregation of the three sub-scores, but the context suggests they are averaged or the model produces a single holistic score — the prompt structure implies three separate values that are combined).

Why this form. LLM-as-a-judge provides scalable, automated assessment of reasoning quality without requiring human annotation. The three-criteria rubric decomposes "good reasoning" into distinct aspects, making the evaluation more structured and interpretable than a single holistic score. The choice of Qwen3-235b-a22b as the judge is motivated by its strong general reasoning capabilities, and the authors note they also tested GPT-4.1 and observed "almost the same score" (Appendix B.1), suggesting the qualitative assessment is robust to the choice of strong judge model.

Critical nuance: the judge does not use ground truth. The qualitative score is computed using the teacher model's answer $a$ as context, not the ground-truth answer $y$. This is an explicit design choice: the authors state that "against prior works like OpenThoughts and ReasonFlux-PRM treat $a$ as a golden truth, we consider that reasoning data constructed for SFT is not suitable for PRM training" (Section 4.2). In other words, the qualitative assessment evaluates whether the step is good relative to the teacher's answer, but does not assume the teacher's answer is correct. This prevents the qualitative score from being contaminated by errors in the teacher model — the judge might assess a step as logically sound and progressive toward an incorrect answer $a$, which would still be useful information for the PRM to learn, but the qualitative score alone would be misleading as a reward signal if $a$ is wrong. This is precisely why the accuracy score (which uses ground truth $y$) is needed as a cross-check.

Accuracy Score ($r_{\text{acc}}$): Knowledge-Grounded Factual Verification

The accuracy score is the paper's primary mechanism for grounding reward signals in verifiable external facts. It provides a robust, quantitative check on whether a reasoning step is factually and procedurally correct, specifically designed to counteract LLM hallucination and reward hacking. It is composed of two sub-scores:

Procedural Correctness ($r_{\text{step\_correctness}}$). This sub-score answers the question: "Is this step a valid move toward the known correct answer?" It uses an LLM verifier prompted with the question $x$, the reasoning history $s_{<t}$, the current step $s_t$, and the ground-truth answer $y$ (from CFLUE). The verifier makes a binary assertion: 1 if the step is a logically sound and productive move toward $y$, 0 otherwise. The prompt (Listing 4, Appendix C.2) explicitly asks: "Is this step a valid, logical progression towards the correct final answer? Respond with a JSON object containing a binary value."

The key difference between $r_{\text{step\_correctness}}$ and $r_{\text{qual}}$ is what they use as their reference standard. $r_{\text{qual}}$ evaluates the step relative to the teacher model's answer $a$ (which may be wrong). $r_{\text{step\_correctness}}$ evaluates the step relative to the ground-truth correct answer $y$ (which is authoritative). A step could score high on $r_{\text{qual}}$ (it's logically coherent and advances toward the teacher's answer) but low on $r_{\text{step\_correctness}}$ (the teacher's answer is wrong, so the step is actually moving in the wrong direction). This dual-reference design is what makes the reward signal robust: the qualitative score captures reasoning form, while the procedural correctness score captures reasoning direction relative to the truth.

Factual Accuracy ($r_{\text{knowledge\_acc}}$). This sub-score answers the question: "Are all factual claims in this step supported by the knowledge base?" It systematically validates every identifiable financial term and claim within $s_t$ against the question-specific knowledge base $\mathcal{K}_x$. The verifier (again an LLM, prompted as a "fact-checking agent" using Listing 5, Appendix C.2) produces a binary output: 1 if all claims and terms in the step are supported by the knowledge base, 0 otherwise.

This is the paper's anti-hallucination mechanism. Even if a step is logically sound and oriented toward the correct answer, it might contain fabricated financial facts — inventing a ratio that doesn't exist, mischaracterizing a regulatory requirement, or hallucinating a definition. The factual accuracy check catches these errors by requiring that every claim be traceable to the expert-derived knowledge base. If the step says "the Price-to-Book ratio measures liquidity" but $\mathcal{K}_x$ defines Price-to-Book ratio as measuring market value relative to book value, the factual accuracy check returns 0.

Aggregation into the accuracy score. The two sub-scores are combined in a weighted sum:

$r_{\text{acc}} = 0.5 \left(r_{\text{step\_correctness}}(s_t, y) + \omega_k \cdot r_{\text{knowledge\_acc}}(s_t, \mathcal{K}_x)\right)$

where $\omega_k$ is a hyperparameter controlling the relative importance of factual grounding versus procedural correctness. The authors set $\omega_k = 1.0$, giving equal weight to both sub-scores.

What it computes. The accuracy score is a value $\in [0, 1]$ (since both sub-scores are binary 0 or 1, and $\omega_k = 1.0$, the result is 0, 0.5, or 1.0 depending on how many sub-checks pass). A score of 1.0 means the step is both on the right track toward the correct answer and factually supported by domain knowledge. A score of 0 means it fails on both dimensions.

Why this form. The two-component design separates procedural correctness (is this the right thing to do?) from factual correctness (are the facts right?). These can diverge: a step might use the correct procedure but reference a hallucinated financial term, or it might contain all true facts but apply them in a way that doesn't advance toward the answer. The weighted-sum formulation with $\omega_k = 1.0$ treats them as equally important, reflecting the paper's position that in finance, factual precision is non-negotiable. The binary nature of the sub-scores (rather than continuous scores) makes the reward signal crisper and less ambiguous — a claim is either supported by the knowledge base or it isn't, a step is either on the right path or it isn't — which is appropriate for a domain where correctness is often binary (a regulatory definition is either right or wrong, a calculation is either accurate or it isn't).

---

Trajectory-Level Reward Modeling: Global Quality and Knowledge Coverage

While step-level rewards evaluate local correctness, trajectory-level rewards evaluate the reasoning process as a whole. The paper introduces two trajectory-level signals to address a specific vulnerability: a trajectory where every individual step is correct but the overall reasoning still reaches the wrong answer (e.g., because the steps are correct in isolation but don't connect properly). The trajectory-level signals provide global oversight.

Outcome Correctness Score ($r_{\text{out}}$)

This is the simplest reward signal: does the final answer match the ground truth?

$r_{\text{out}} \in \{0, 1\}$

For the CFLUE benchmark, questions typically require selecting a final option (e.g., 'A', 'B', 'ACD'), so the score is computed by extracting the model's chosen option and comparing it directly to the ground-truth correct option. This yields a strict binary signal: 1 if the exact option matches, 0 otherwise.

Why this form. Outcome correctness is the most direct measure of whether the reasoning succeeded, but it is a blunt instrument. A trajectory could receive $r_{\text{out}} = 1$ despite containing some incorrect steps (if the errors happen to cancel out or don't affect the final answer), or $r_{\text{out}} = 0$ despite mostly correct steps (if a single critical error derails the conclusion). This is why outcome correctness alone is insufficient as a reward signal — the PRM needs to learn where in the trajectory errors occurred, not just whether they occurred. But as one component of a trajectory-level assessment, it provides a crucial anchoring signal.

Knowledge Coverage Score ($r_{\text{cover}}$)

The knowledge coverage score measures how thoroughly the reasoning process utilizes the relevant domain knowledge. It is defined as:

$r_{\text{cover}} = \frac{|\phi_{\text{ext}}(s \oplus a) \cap \mathcal{K}_x|}{|\mathcal{K}_x|}$

where $\mathcal{K}_x \subseteq \mathcal{K}$ is the subset of the knowledge base containing all terms deemed relevant to the question $x$, $\phi_{\text{ext}}(\cdot)$ is an extraction function (implemented via an LLM) that identifies all financial concepts mentioned in the generated text, $s \oplus a$ denotes the concatenation of the full reasoning trace and final answer, and $|\cdot|$ denotes set cardinality (number of elements).

What it computes. The function $\phi_{\text{ext}}$ scans the model's complete output (reasoning trace $s$ plus final answer $a$) and extracts all financial terms it can identify. These are compared against the set of required terms $\mathcal{K}_x$ (the concepts that the expert analysis says should be invoked). The coverage ratio is the fraction of required terms that appear in the model's output. If $\mathcal{K}_x$ contains 5 financial concepts and the model mentions 4 of them, $r_{\text{cover}} = 0.8$. If it mentions all 5, $r_{\text{cover}} = 1.0$.

The extraction function $\phi_{\text{ext}}$ is itself implemented via an LLM prompted as a verification agent (Listing 6, Appendix C.2). The prompt provides the list of required knowledge points (terms and definitions from $\mathcal{K}_x$) and the model's full generated response, and asks the model to count how many of the required knowledge points were actually covered. The output is a JSON object with a coverage_number (integer count) and coverage_index (list of which knowledge points were covered).

Why this form. Knowledge coverage provides a measure of reasoning comprehensiveness that is independent of correctness. A reasoning trace might reach the right answer for the wrong reasons (using only partial knowledge or skipping important concepts), and $r_{\text{out}}$ alone would give it a perfect score. The coverage score penalizes such traces by requiring that the reasoning demonstrates awareness of all relevant financial concepts. This is particularly important in finance, where thoroughness is often a requirement — a financial analysis that reaches the right conclusion but omits mention of key risk factors or regulatory considerations is incomplete, even if the bottom-line answer matches.

The set-based formulation (using intersection cardinality) provides a simple, interpretable measure: coverage is just "what fraction of the required concepts did the model use." The LLM-based extraction $\phi_{\text{ext}}$ is necessary because concepts can be expressed in varied language — the model doesn't need to use the exact term from the knowledge base, it just needs to refer to the concept. The LLM verifier handles this semantic matching.

---

Reward Data Construction: From Raw Signals to Binary Training Labels

Once the five raw reward signals have been computed for each step and trajectory, they must be converted into binary labels suitable for training a classifier. The paper uses distinct aggregation procedures for step-level and trajectory-level labels, each designed to produce a single, unambiguous prediction target.

Step-Level Label Construction: Dynamic Weighted Aggregation

For each step $s_t$, three raw scores are available: $r_t^{\text{imp}}$ (importance), $r_t^{\text{qual}}$ (qualitative), and $r_t^{\text{acc}}$ (accuracy). These are aggregated into a single continuous score $r_t^{\text{step}}$ using a dynamic weighting scheme based on the softmax function:

$r_t^{\text{step}} = \sum_{k \in \{\text{imp}, \text{qual}, \text{acc}\}} \text{softmax}\left(r_t^{\text{imp}}, r_t^{\text{qual}}, r_t^{\text{acc}}\right)_k \cdot r_t^k$

where $r_t^k$ is the raw score for component $k$ at step $t$, and $\text{softmax}(\cdot)_k$ is the $k$-th element of the softmax probability distribution over the three raw scores. The softmax function is defined as:

$\text{softmax}(v_1, v_2, v_3)_k = \frac{e^{v_k}}{e^{v_1} + e^{v_2} + e^{v_3}}$

What it computes. The three raw scores are treated as logits for a softmax distribution, which converts them into a set of weights that sum to 1. These weights are then used to take a weighted average of the raw scores. If one score is significantly higher than the others, it receives a disproportionately larger weight (due to the exponential in softmax), allowing its signal to dominate the aggregation. For example, if accuracy is 0.9, importance is 0.3, and quality is 0.4, the softmax weights will heavily favor accuracy, and the aggregated score will be close to 0.9. If all three scores are similar, the weights will be close to $1/3$ each, and the aggregation approximates a simple average.

The continuous score $r_t^{\text{step}}$ is then binarized using a threshold of 0.5:

$L_t^{\text{step}} = \begin{cases} 1 & \text{if } r_t^{\text{step}} \geq 0.5 \\ 0 & \text{otherwise} \end{cases}$

Why this form. The softmax-based dynamic weighting is the paper's alternative to a fixed linear combination (e.g., $0.33 \times r_{\text{imp}} + 0.33 \times r_{\text{qual}} + 0.33 \times r_{\text{acc}}$). Fixed weights assume that all three signals are equally informative across all steps, which is unlikely to be true — a step that scores very high on one dimension but mediocre on others might still be a good step, but a fixed-weight average would dilute its high score. The dynamic weighting functions as an attention mechanism: it identifies which signal is providing the strongest evidence and amplifies it, while still incorporating information from the other signals. This is more robust than fixed weights because it adapts to the specific characteristics of each step.

The threshold of 0.5 is a natural choice for binarization given that the component scores are all in $[0, 1]$ or binary. It means a step is labeled as "good" (1) if the aggregated score is at least halfway between "definitely bad" (0) and "definitely good" (1). The paper does not explore sensitivity to this threshold choice.

Trajectory-Level Label Construction: Outcome-Coverage Combination

For each trajectory, the outcome correctness score $r_{\text{out}}$ and knowledge coverage score $r_{\text{cover}}$ are combined into a single score:

$R_{\text{traj}}(s, a) = r_{\text{out}}(a) + \eta \cdot r_{\text{cover}}(s, a)$

where $\eta = 1.5$ is a weighting hyperparameter.

What it computes. The trajectory score is a simple linear combination: the outcome correctness (0 or 1) plus 1.5 times the knowledge coverage ratio (0 to 1). This means a trajectory that gets the correct answer ($r_{\text{out}} = 1$) with full knowledge coverage ($r_{\text{cover}} = 1$) receives a score of $1 + 1.5 \times 1 = 2.5$. A trajectory that gets the wrong answer with no knowledge coverage receives 0. A trajectory that gets the right answer but with only partial coverage (say $r_{\text{cover}} = 0.6$) receives $1 + 1.5 \times 0.6 = 1.9$.

This continuous score is binarized using a threshold of 1.25:

$L_{\text{traj}} = \begin{cases} 1 & \text{if } R_{\text{traj}} \geq 1.25 \\ 0 & \text{otherwise} \end{cases}$

Why this form and these specific parameter values. The weighting $\eta = 1.5$ means that knowledge coverage can contribute up to 1.5 points to the total score, compared to 1 point from outcome correctness. This makes knowledge coverage more influential than outcome correctness in the continuous score — a trajectory with full coverage but wrong answer scores 1.5, which is above the 1.25 threshold (so it would be labeled as 1), while a trajectory with correct answer but zero coverage scores 1.0, which is below the threshold (labeled as 0). This encoding reflects the paper's philosophy that in finance, reasoning that demonstrates comprehensive domain knowledge is valuable even if the final answer happens to be wrong (perhaps due to a minor arithmetic error), while reasoning that reaches the right answer without engaging with the relevant concepts is suspicious (it might be guessing or using shallow heuristics). The threshold of 1.25 is set to the mean of the two weights ($(1 + 1.5)/2 = 1.25$), which the authors state "gives the ability to change the hard label to each reward signal" — meaning the threshold is balanced such that neither outcome nor coverage can unilaterally determine the label; both must contribute for the label to be positive.

---

Fin-PRM Training: Dual-Level Joint Objective

Fin-PRM is implemented by fine-tuning the Qwen2.5-7B-Instruct model on the labeled dataset. The training uses a joint binary cross-entropy objective that combines step-level and trajectory-level supervision.

Model architecture and input format. The base model is Qwen2.5-7B-Instruct, a 7-billion-parameter instruction-tuned language model. Fin-PRM is trained to predict correctness scores given the context of a question, reasoning history, and (for trajectory-level prediction) the full reasoning trace. The model's scoring function $R_{\phi}$ takes different forms depending on the evaluation granularity:

• Step-level evaluation: $R_{\phi}(s_t \mid x, s_{<t}, a)$ — the model receives the question $x$, all reasoning steps before the current one ($s_{<t}$), and the silver-truth answer $a$, and outputs a scalar logit for step $s_t$ representing its predicted probability of correctness.
• Trajectory-level evaluation: $R_{\phi}(s, a \mid x)$ — the model receives the question $x$ and the full reasoning trace $s$ with answer $a$, and outputs a scalar logit for the entire trajectory.

The paper does not specify the exact input format (tokenization, special tokens, or prompting template), but the principle is that the same model backbone produces both step-level and trajectory-level predictions by varying the context it receives.

The joint loss function. The total training loss is:

$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{step}} + \lambda \cdot \mathcal{L}_{\text{traj}}$

where $\lambda$ is a hyperparameter balancing the two loss terms. The paper does not specify the exact value of $\lambda$ in the main text or appendix, but the ablation study in Figure 4 (and associated discussion of $\zeta$ in Section 6) provides some insight into the relative importance of step vs. trajectory signals, suggesting that approximately equal weighting is effective.

The step-level loss. For a reasoning trace with $T$ steps:

$\mathcal{L}_{\text{step}} = \frac{1}{T} \sum_{t=1}^{T} \mathcal{L}_{\text{BCE}}\left(R_{\phi}(s_t \mid x, s_{<t}, a), L_{\text{step}}(s_t)\right)$

What it computes. The step-level loss is the average binary cross-entropy over all $T$ steps in the trace. For each step, the model's predicted logit $R_{\phi}(s_t \mid \dots)$ is compared against the ground-truth binary label $L_{\text{step}}(s_t)$ (0 or 1) using the BCE loss. The predictions are independent across steps — the model sees the full history before each step, but the loss for each step is computed separately and averaged.

The trajectory-level loss:

$\mathcal{L}_{\text{traj}} = \mathcal{L}_{\text{BCE}}\left(R_{\phi}(s, a \mid x), L_{\text{traj}}(s, a)\right)$

What it computes. Unlike the step-level loss which averages over multiple predictions, the trajectory loss is a single BCE term comparing the model's prediction for the entire trajectory against the binary trajectory label $L_{\text{traj}}$.

The binary cross-entropy loss. Both $\mathcal{L}_{\text{step}}$ and $\mathcal{L}_{\text{traj}}$ use the standard binary cross-entropy formulation:

$\mathcal{L}_{\text{BCE}}(R_{\phi}, L) = -\left[L \log \sigma(R_{\phi}) + (1 - L) \log(1 - \sigma(R_{\phi}))\right]$

where $L \in \{0, 1\}$ is the ground-truth binary label, $R_{\phi}$ is the model's raw logit output, and $\sigma(\cdot)$ is the sigmoid function that converts the logit to a probability $\in (0, 1)$. The sigmoid function is:

$\sigma(R_{\phi}) = \frac{1}{1 + e^{-R_{\phi}}}$

What it computes. BCE loss penalizes the model when its predicted probability $\sigma(R_{\phi})$ diverges from the ground-truth label $L$. When $L = 1$, the loss is $-\log(\sigma(R_{\phi}))$, which is large when $\sigma(R_{\phi})$ is close to 0 (the model is incorrectly confident that the step/trajectory is bad) and small when $\sigma(R_{\phi})$ is close to 1 (correctly confident). When $L = 0$, the loss is $-\log(1 - \sigma(R_{\phi}))$, which is large when $\sigma(R_{\phi})$ is close to 1 (incorrectly confident that it's good) and small when $\sigma(R_{\phi})$ is close to 0.

Why this form. BCE is the maximum-likelihood objective for binary classification with a Bernoulli target, making it the appropriate loss function for learning to predict binary labels. The sigmoid function maps unbounded logits to valid probabilities, ensuring the model's outputs can be interpreted as confidence scores. The joint objective with both step-level and trajectory-level losses ensures that the model learns to make both local (per-step) and global (whole-trajectory) assessments, which the ablation study confirms is superior to either alone.

Training hyperparameters (Table 3, Appendix D.1). The paper provides specific training configurations:

• Base model: Qwen2.5-7B-Instruct
• Training data: 3,000 samples
• Batch size: 128
• Learning rate: $1 \times 10^{-5}$
• Dropout: 0.0 (no dropout)
• Optimizer: AdamW with betas $(0.9, 0.95)$
• Hardware: NVIDIA A100 GPUs (count not specified)

The learning rate of $1 \times 10^{-5}$ is relatively conservative for fine-tuning a 7B model, appropriate for a small dataset (3,000 samples) where aggressive optimization could lead to overfitting. The absence of dropout (0.0) suggests the authors rely on the limited dataset size and early stopping (presumably based on validation loss, though not explicitly stated) rather than explicit regularization to prevent overfitting.

What Fin-PRM predicts at inference time. After training, Fin-PRM can evaluate any reasoning step or trajectory by producing a scalar score. The paper does not specify whether the model outputs raw logits or post-sigmoid probabilities, but downstream applications use the scores as continuous values (not binarized), suggesting the sigmoid is applied at inference time to produce a probability $\in (0, 1)$. For step $s_t$, the model produces:

$\hat{R}_{\text{step}}(s_t \mid x, s_{<t}, a) = \sigma(R_{\phi}(s_t \mid x, s_{<t}, a))$

For a full trajectory:

$\hat{R}_{\text{traj}}(s, a \mid x) = \sigma(R_{\phi}(s, a \mid x))$

These predicted scores form the basis for all three downstream applications.

---

Deployment Recipe 1: Offline Data Selection for SFT

The first application of Fin-PRM is to filter a large pool of candidate reasoning traces, selecting only the highest-quality examples for supervised fine-tuning. This is an offline application — the PRM is used once to curate a training dataset, and the curated dataset is then used to train a student model in a standard SFT pipeline.

Candidate generation. The authors use Qwen3-8B (a different model than the one used for PRM training) to generate multiple distinct reasoning trajectories for each question. This creates a pool of candidate (question, trace, answer) triplets with varying quality — some will be correct and well-reasoned, others will contain errors or poor reasoning.

Scoring function. Each candidate trajectory is scored using a combined step-trajectory formula:

$\hat{R} = \frac{1}{T} \sum_{t=1}^{T} \hat{R}_{\text{step}}(s_t \mid x, s_{<t}, a) + \zeta \cdot \hat{R}_{\text{traj}}(s, a \mid x)$

where $\hat{R}_{\text{step}}$ and $\hat{R}_{\text{traj}}$ are the predicted scores from Fin-PRM (post-sigmoid probabilities), $T$ is the number of steps in the trace, and $\zeta = 1.0$ is a hyperparameter balancing the contribution of the averaged step-level score versus the trajectory-level score.

What it computes. The first term is the mean step-level score across all steps in the trace — this captures the average local quality of individual reasoning steps. The second term is the trajectory-level score (weighted by $\zeta$), capturing the global quality of the entire reasoning process. The sum is a scalar representing the overall quality of the trace. With $\zeta = 1.0$, step-level and trajectory-level assessments contribute equally to the final score.

Selection process. For each question, all generated trajectories are scored using $\hat{R}$, and only the top-scoring trajectory (or trajectories) is retained. The paper reports using 1,000 selected samples to fine-tune the student model (Qwen2.5-7B-Instruct), though it does not specify exactly how selection was performed across questions (e.g., whether exactly one trace per question was selected, or whether multiple top traces from some questions were included to reach 1,000 total).

Why this form. The combined scoring formula mirrors the training objective's dual-level design: both local (step) and global (trajectory) quality matter for selecting good reasoning examples. The hyperparameter $\zeta$ controls the relative importance, and the ablation study (Figure 4, Section 6) validates that $\zeta = 1.0$ is optimal — giving equal weight to both granularities. Selection using this score is superior to random selection because it filters out noisy, incorrect, or poorly-reasoned traces that would degrade the student model's training.

---

Deployment Recipe 2: Best-of-N Test-Time Selection

The second application uses Fin-PRM at inference time to select the best answer from multiple candidate responses. This is the standard Best-of-N (BoN) setting: a policy model generates $N$ complete responses, Fin-PRM scores each one, and the highest-scoring response is chosen as the final output.

Candidate generation. The policy model (Qwen2.5-7B-Instruct in the experiments) generates $N$ independent candidate responses for each question, where $N \in \{4, 8, 16\}$ in the reported experiments. Each candidate consists of a full reasoning trace and final answer.

Scoring and selection. Each of the $N$ candidates is scored using the same $\hat{R}$ formula from Equation 13 (the combined step-trajectory score with $\zeta = 1.0$). The candidate with the highest $\hat{R}$ value is selected as the model's final output.

Baselines for comparison. The paper compares Fin-PRM against:

• Majority voting: Select the most common final answer among the $N$ candidates (no learned verifier involved).
• Qwen2.5-Math-PRM-7B: A general-purpose math-specialized PRM of the same size as Fin-PRM, used as a direct ablation to test whether domain specialization matters.
• Qwen2.5-Math-PRM-72B: A larger (72B) version of the math PRM, included as an upper-bound reference for general-domain PRM performance.

What the results showed. Figure 2 reveals that Fin-PRM consistently outperforms both majority voting and the math-specialized PRMs on the CFLUE financial benchmark. The math PRMs actually perform worse than majority voting at higher values of $N$, which the paper interprets as evidence that their scoring signal becomes anti-correlated with correctness when applied out-of-domain. On the out-of-domain Math500 benchmark, Fin-PRM maintains competitive performance (close to the math-specialized PRMs), demonstrating that domain specialization does not catastrophically destroy general reasoning assessment capability.

---

Deployment Recipe 3: Reward Signal for GRPO Reinforcement Learning

The third and most dynamic application uses Fin-PRM as a reward function during online policy optimization with Group Relative Policy Optimization (GRPO; Shao et al., 2024). In this setting, the policy model is actively being updated, and Fin-PRM provides the reward signal that guides the updates.

GRPO background. GRPO is a variant of policy gradient RL that optimizes a policy by comparing the rewards of multiple responses generated from the same prompt. For a group of $N$ responses to a question, the raw rewards are normalized (subtract mean, divide by standard deviation) to compute advantages, and the policy is updated to increase the probability of above-average responses while decreasing the probability of below-average ones. By default, GRPO uses only an outcome-level reward (whether the final answer is correct).

Fin-PRM-augmented reward. The paper augments the standard GRPO reward with Fin-PRM's process-level assessment:

$r_{\text{rl}} = (1 - \delta) \cdot r_{\text{out}} + \delta \cdot \hat{R}$

where $r_{\text{out}} \in \{0, 1\}$ is the binary outcome correctness (does the final answer match ground truth?), $\hat{R}$ is Fin-PRM's combined step-trajectory score (from Equation 13), and $\delta$ is a hyperparameter controlling how much weight is given to the process-level reward versus the outcome reward. The paper does not specify the exact value of $\delta$ used in the GRPO experiments, but Table 4 (Appendix D.2) lists the GRPO hyperparameters, suggesting $\delta$ is tuned.

What it computes. The composite reward $r_{\text{rl}}$ is a convex combination of outcome and process signals. When $\delta = 0$, the RL agent receives only outcome feedback (the standard GRPO setup) — it learns whether its final answer was correct but gets no signal about the quality of its reasoning. When $\delta = 1$, the agent receives only process feedback from Fin-PRM — it learns about reasoning quality regardless of whether the final answer is correct. Intermediate values blend the two signals. The paper's experimental results (Figure 3) show that incorporating Fin-PRM's process-level rewards (with some $\delta > 0$) substantially improves the final policy's performance on both CFLUE (70.5% vs. 67.2% with rule-based reward alone) and FinQA (62.8% vs. 59.5%).

Advantage computation for GRPO. For a group of $N$ responses, the advantages are computed by normalizing the composite rewards:

$A_{\text{rl}} = \frac{r_{\text{rl}} - \text{mean}(\{r_{\text{rl}}\}_{j=1}^{N})}{\text{std}(\{r_{\text{rl}}\}_{j=1}^{N})}$

where $\text{mean}(\cdot)$ and $\text{std}(\cdot)$ are computed over the $N$ responses in the group. This normalization ensures that advantages are centered at zero with unit variance, which stabilizes policy gradient updates by preventing the scale of rewards from affecting the learning rate.

The GRPO objective with Fin-PRM. The full GRPO policy gradient objective (Equation 16 in the paper) incorporates the Fin-PRM-derived advantages into a clipped surrogate objective:

$\mathcal{J}_{\text{GRPO}}(\theta) = \mathbb{E}_{x, \{s_i\} \sim \pi_{\theta_{\text{old}}}} \left[\frac{1}{N} \sum_{i=1}^{N} \frac{1}{T_i} \sum_{t=1}^{T_i} \left(\min\left\{ \frac{\pi_\theta(s_{i,t}|x, s_{i,<t})}{\pi_{\theta_{\text{old}}}(s_{i,t}|x, s_{i,<t})} A_{\text{rl},i}, \text{clip}\left(\frac{\pi_\theta(s_{i,t}|x, s_{i,<t})}{\pi_{\theta_{\text{old}}}(s_{i,t}|x, s_{i,<t})}, 1-\epsilon, 1+\epsilon\right) A_{\text{rl},i} \right\} - \beta_{\text{KL}} \mathcal{D}_{\text{KL}}(\pi_\theta \| \pi_{\text{ref}}) \right) \right]$

where $\pi_\theta$ is the current policy being optimized, $\pi_{\theta_{\text{old}}}$ is the policy that generated the responses (frozen during the update), $A_{\text{rl},i}$ is the normalized advantage for response $i$, $\epsilon$ is the clipping hyperparameter (typically 0.2) that prevents overly large policy updates, $\beta_{\text{KL}}$ controls the strength of the KL-divergence penalty against a reference policy $\pi_{\text{ref}}$ (preventing the policy from drifting too far from its initial behavior), and $\mathcal{D}_{\text{KL}}$ is the KL divergence. The clipping operator $\text{clip}(r, 1-\epsilon, 1+\epsilon)$ constrains the importance sampling ratio $r = \frac{\pi_\theta}{\pi_{\theta_{\text{old}}}}$ to the interval $[1-\epsilon, 1+\epsilon]$. The $\min$ operation selects the more conservative of the unclipped or clipped objective, preventing the policy from changing too dramatically in any single update.

What it computes. This is the standard PPO/GRPO clipped objective adapted to use Fin-PRM's advantages. For each token position $t$ in each response $i$, the ratio of the new policy's probability to the old policy's probability is computed. This ratio tells us how much more (or less) likely the new policy makes that token. The ratio is multiplied by the advantage $A_{\text{rl},i}$ (which is the same for all tokens in the same response — GRPO uses response-level advantages, not token-level). The clipping and min operations ensure that the update is conservative: if the advantage is positive (we want to make this response more likely), the ratio is capped at $1+\epsilon$; if the advantage is negative (we want to make this response less likely), the ratio is floored at $1-\epsilon$. The KL penalty prevents the new policy from diverging too far from the reference policy, which is typically the initial SFT model.

Why this form. The PPO-style clipped objective is the standard approach for stable RL fine-tuning of language models. The key innovation here is not in the optimization algorithm but in the reward signal: Fin-PRM provides process-level advantages that give the policy more nuanced feedback than outcome-only rewards. When the policy generates a response that reaches the wrong answer but has high-quality reasoning (high $\hat{R}$), the composite reward $r_{\text{rl}}$ will be higher than the outcome-only reward would suggest, providing a partial positive signal that encourages the policy to retain its good reasoning patterns even when the answer is wrong. This should lead to more sample-efficient learning because the policy receives feedback on how it reasoned, not just what it concluded.

GRPO training hyperparameters (Table 4, Appendix D.2).

• Policy model: Qwen2.5-7B-Instruct
• Number of responses per prompt ($N$): 8
• Clipping parameter ($\epsilon$): 0.2
• KL penalty coefficient ($\beta_{\text{KL}}$): 0.04
• Learning rate: $1 \times 10^{-6}$ (policy) and $5 \times 10^{-6}$ (critic, if applicable)
• Training epochs: 3

The small learning rate ($10^{-6}$ for the policy, an order of magnitude smaller than the PRM training learning rate) is typical for RL fine-tuning, where large updates can quickly degrade the policy's language capabilities. The KL penalty of 0.04 provides moderate regularization against policy drift.

---

Summary of Design Choices and Their Justifications

• Three-component step-level reward over single-component: Prevents any single weakness (noisy MC rollouts, LLM judge bias, imperfect knowledge base) from dominating the training signal. The dynamic softmax weighting ensures the strongest signal for each step is amplified.
• Knowledge base extracted from human expert analyses rather than from the teacher model: Ensures factual ground truth is independent of the models being evaluated, preventing circular verification where the teacher model's errors are used to validate itself.
• Silver-truth treatment of teacher answers ($a$ treated as potentially wrong): Prevents the PRM from learning to trust the teacher model's outputs unconditionally, and forces verification against the actual ground truth $y$ and knowledge base $\mathcal{K}$.
• Binary thresholding for label construction rather than using continuous scores directly: Creates clean, unambiguous training targets suitable for classification-based training, and simplifies the downstream use of the PRM (scores are interpretable as "good" or "bad").
• Joint step-trajectory training loss rather than separate models: A single model learns to evaluate both granularities, which is more parameter-efficient and allows the step-level and trajectory-level representations to share knowledge.
• Qwen2.5-7B-Instruct as the base model rather than a larger model: Balances capability with practical deployability — a 7B model can be used for BoN selection on thousands of candidates without prohibitive computational cost.
• $\zeta = 1.0$ for combined scoring in deployment based on the ablation study (Figure 4): Equal weighting of step-level and trajectory-level signals is empirically optimal, confirming the dual-level design.
• GRPO composite reward with $\delta$ blending outcome and process signals rather than replacing outcome rewards entirely: Maintains the anchoring to ground-truth correctness while adding process-level guidance, preventing the RL agent from optimizing for PRM-pleasing reasoning that doesn't lead to correct answers.

4. Key Insights and Innovations

Innovation 1: Verifiability as a First-Class Design Constraint for Reward Models

The paper's most fundamental conceptual move is elevating verifiability — the property that a reward signal can be checked against an external, authoritative source — from an afterthought to a central architectural principle. Prior PRM work treats reward construction as a label-generation problem: use Monte Carlo rollouts (Math-Shepherd, Lightman et al.), LLM-as-a-judge assessments (Open-PRM, ReasonFlux-PRM), or human annotation to produce training targets, then train a model to imitate those targets. The implicit assumption is that a well-trained reward model will generalize to evaluate reasoning quality in any domain, because good reasoning has universal structural properties (logical coherence, step-by-step progression, valid inference).

Fin-PRM challenges this assumption not by arguing that domain knowledge matters — that's an obvious claim — but by identifying a specific, testable failure mode: general-purpose PRMs can become anti-correlated with correctness when applied outside their training domain. The evidence in Figure 2 makes this concrete: Qwen2.5-Math-PRM-7B, a state-of-the-art math PRM, performs worse than majority voting at higher sample counts on financial Best-of-N selection. This is not merely suboptimal performance — it means the PRM's scoring signal is actively harmful, steering selection toward wrong answers more often than random voting would. The authors diagnose this as a consequence of domain mismatch: the math PRM evaluates reasoning form (does this look like a valid derivation?) but has no capacity to evaluate reasoning content (are these financial claims true?), so it systematically rewards plausible-sounding but factually incorrect financial reasoning.

The innovation is not just adding a knowledge base to a PRM — it's the insight that the knowledge base must be external and independently verified, not derived from the teacher model used for data synthesis. This is a subtle but critical distinction. Prior work like ReasonFlux-PRM and Open-PRM constructs reward signals from the teacher model's own outputs (using template-guided scoring or ORM-reverse-engineered labels). In those frameworks, if the teacher model hallucinates a financial concept, the PRM learns to reward that hallucination. Fin-PRM breaks this circularity by extracting its knowledge base $\mathcal{K}$ from human-written expert analyses that come with the CFLUE benchmark — a source independent of any LLM's generation. The accuracy score $r_{\text{acc}}$ and coverage score $r_{\text{cover}}$ are grounded in this external reference, meaning the PRM's training signal is anchored to facts that exist independently of the models being evaluated.

This reframes reward modeling from a purely statistical problem (learn to predict what evaluators would say) to a partially deductive one (verify claims against a trusted knowledge repository). It is a fundamental shift because it changes what kind of errors a PRM can make: a statistics-only PRM can be confidently wrong about factual claims it has never seen; a verification-augmented PRM can still be wrong if its knowledge base is incomplete, but its errors are bounded by the coverage of the knowledge base rather than by the distribution of its training data. For high-stakes domains where the cost of a factual error is high (finance, medicine, law), this bounded-error property is qualitatively more trustworthy than the unbounded errors of a purely statistical evaluator.

This contribution is fundamental rather than incremental because it establishes a new design axis for reward models — verifiability — that is orthogonal to the standard axes of model architecture, training data scale, or reward granularity. It is not a refinement of existing PRM training methods; it is an assertion about what information a reward signal should be allowed to depend on, and where that information should come from.

---

Innovation 2: Multi-Faceted Reward Decomposition as a Defense Against Signal Fragility

The decision to decompose the step-level reward into three independent signals — importance ($r_{\text{importance}}$), quality ($r_{\text{qual}}$), and accuracy ($r_{\text{acc}}$) — might appear to be a straightforward engineering choice (more signals = more information). But the paper surfaces a deeper rationale: no single reward signal type is robust across all failure modes of reasoning evaluation, and the redundancy created by multiple signals is itself a defense mechanism.

Each signal type has a known fragility. Monte Carlo importance scores (Equation 4) are sensitive to the stochasticity of rollouts and the capability of the rollout generator — if Qwen2.5-7B-Math happens to produce unrepresentative completions, the importance score can be noisy. LLM-as-a-judge qualitative assessments (Equation 5) are subject to the judge model's biases, knowledge gaps, and hallucinations — Qwen3-235b-a22b might confidently assert that a step is logically sound while missing that it relies on a false premise. Knowledge-grounded accuracy scores (Equation 6) are only as complete as the knowledge base — if $\mathcal{K}_x$ fails to capture a relevant concept, the factual accuracy check cannot detect errors related to that concept.

Prior work typically relies on one or two of these signal types: Math-Shepherd uses only MC rollouts, Open-PRM uses only ORM-derived labels, ReasonFlux-PRM uses MC rollouts plus template-guided trajectory scores. The implicit assumption is that the chosen signal type is sufficiently robust for the target domain. Fin-PRM's three-component design encodes a different assumption: in a domain where errors have high cost, robustness to signal failure is worth the additional annotation cost.

The dynamic softmax weighting in Equation 8 operationalizes this philosophy. Rather than averaging the three scores with fixed weights (which would mean a failed signal drags down the aggregate even when the other two are confident), the softmax mechanism functions as an attention mechanism that amplifies the most confident signal. If the accuracy check is highly confident (score close to 1.0) while the importance and quality scores are moderate, the softmax weights will heavily favor accuracy, producing an aggregate score close to 1.0. Conversely, if none of the signals is confident, the weights will be more balanced, reflecting genuine ambiguity. This is a form of implicit ensembling where the signals compete to be the primary determinant of the label, rather than cooperating in a fixed-weight average where a single weak signal can dilute a strong one.

The significance of this design extends beyond the specific application to finance. It establishes a modular architecture for reward signal construction where new signal types can be added or removed depending on domain requirements: a domain with a comprehensive knowledge base (finance, medicine) can weight accuracy heavily; a domain with less structured knowledge but clear outcome signals (mathematics, code) can rely more on importance scores; a domain requiring nuanced qualitative judgment (essay evaluation, creative writing) can lean on LLM-as-a-judge. The framework doesn't prescribe which signals to use — it prescribes how to combine them robustly, which is a more general contribution.

This innovation is incremental in mechanism but fundamental in framing. The individual signal types (MC rollouts, LLM-as-a-judge, knowledge verification) all exist in prior work. What's novel is treating them as interchangeable, fail-able modules whose combination should be designed to gracefully handle individual module failures. The ablation study (Figure 4) provides indirect evidence for this claim: performance peaks when step-level and trajectory-level signals are given equal weight ($\zeta = 1.0$), suggesting that neither granularity alone provides sufficient information, and that the model benefits from the redundancy.

---

Innovation 3: Trajectory-Level Knowledge Coverage as a Measure of Reasoning Comprehensiveness

The knowledge coverage score $r_{\text{cover}}$ (Equation 7) introduces a dimension of evaluation that has no analogue in prior PRM work: not just whether the reasoning reaches the right answer, but whether it engages with all the relevant domain knowledge. This is a conceptual departure from the standard PRM evaluation framework, which focuses exclusively on correctness (is the step valid? does the trajectory lead to the right answer?).

The motivation is domain-specific but the concept is generalizable. In finance, a reasoning trace might reach the correct answer while omitting critical considerations — calculating a valuation without mentioning relevant risk factors, classifying a financial instrument without referencing the applicable regulatory framework, or making an investment recommendation without considering alternative scenarios. Such reasoning is correct but incomplete, and in high-stakes domains, incompleteness can be as dangerous as incorrectness because it creates overconfidence in conclusions that haven't properly considered the full picture.

$r_{\text{cover}}$ operationalizes comprehensiveness as coverage of a knowledge set: what fraction of the concepts that an expert would invoke did the model actually invoke? The set-based formulation (intersection cardinality over total required concepts) is deliberately simple — it doesn't assess how well the concepts were used, only whether they appeared. This simplicity is a strength for a training signal: it provides a clear, unambiguous target (mention all required concepts) that is computationally cheap to evaluate (via LLM-based extraction and set comparison) and interpretable (a coverage of 0.8 means "you mentioned 4 out of 5 required concepts").

The trajectory-level label construction (Equation 9, with $\eta = 1.5$ and threshold 1.25) encodes an interesting value judgment: knowledge coverage matters more than outcome correctness in determining the binary label. A trajectory with the wrong answer ($r_{\text{out}} = 0$) but full coverage ($r_{\text{cover}} = 1$) scores $0 + 1.5 \times 1 = 1.5$, which is above the 1.25 threshold and thus labeled as positive. A trajectory with the correct answer ($r_{\text{out}} = 1$) but no coverage ($r_{\text{cover}} = 0$) scores $1 + 1.5 \times 0 = 1.0$, which is below the threshold and labeled as negative. This weighting reflects a philosophical stance: a model that demonstrates comprehensive domain understanding but makes a calculation error is closer to expert reasoning than a model that guesses the right answer using superficial heuristics.

This is fundamental rather than incremental because it expands the definition of what a PRM should evaluate. Prior PRMs are essentially correctness verifiers — they assess whether reasoning is valid and leads to the right conclusion. Fin-PRM adds a completeness evaluator — it assesses whether reasoning is thorough and well-supported by domain knowledge. These are orthogonal dimensions: a reasoning trace can be correct but incomplete, complete but incorrect, both, or neither. The dual-level training objective encodes both dimensions, and the trajectory-level label construction explicitly penalizes incompleteness even when the answer is right. This reframes the PRM's role from "did you reason correctly?" to "did you reason like an expert would?" — a higher standard that is appropriate for high-stakes domains.

---

Innovation 4: Empirical Proof That Domain Mismatch Can Make PRMs Actively Harmful (Not Just Ineffective)

The finding in Figure 2 that Qwen2.5-Math-PRM-7B underperforms majority voting on financial Best-of-N selection is, in the authors' own framing, evidence that a domain-mismatched PRM is worse than no PRM at all. This is a stronger claim than "general PRMs don't transfer well" — it means the PRM's scoring signal is systematically anti-correlated with correctness, actively steering selection toward wrong answers.

The mechanism for this anti-correlation is plausible though not proven in the paper: a math-specialized PRM has learned to recognize patterns that are correlated with correct reasoning in mathematics — symbolic manipulation, step-by-step deduction, explicit intermediate calculations, formal logical structure. When applied to financial reasoning, these same patterns may be negatively correlated with correctness, because they indicate the model is applying mathematical reasoning templates to problems that require financial judgment rather than formal derivation. A financial reasoning step that says "the Price-to-Book ratio is 1.5, therefore the stock is undervalued" has the surface form of a mathematical inference but is a financial non-sequitur (P/B must be interpreted relative to industry averages, growth prospects, and asset composition). A math PRM would see the numeric comparison and reward the step; a finance-knowledgeable PRM would recognize the missing context and penalize it.

This is a negative result with significant diagnostic value. It contradicts the reasonable default assumption that a PRM from a related technical domain (mathematics) would provide at least weak positive signal on another technical domain (finance). The fact that the signal is actively negative implies that PRM evaluation is not just imperfectly transferred but qualitatively domain-dependent — the features that constitute "good reasoning" differ between domains in ways that are not captured by surface-level structural similarity.

The practical implication is a caution against the natural impulse to deploy general-purpose PRMs in specialized domains as an "better than nothing" solution. If the finding generalizes — and the paper provides suggestive evidence that it might, by showing Fin-PRM itself performs competitively on Math500 (Table 2) even though the math PRM fails on finance — then the asymmetry is important: a domain-specialized PRM can generalize to general reasoning, but a general PRM cannot safely be specialized by deployment alone. This has implications for how organizations should invest in evaluation infrastructure: rather than building one PRM to rule them all, the evidence suggests a portfolio of domain-specialized evaluators may be necessary for reliable performance.

This innovation is incremental as a finding but fundamental in its implications. The specific result (Math-PRM < majority voting) is a single data point from one benchmark with one model family. But if it replicates across domains and model families, it constitutes a paradigm-level challenge to the "general evaluation" approach that currently dominates PRM research. The paper doesn't overclaim this — it presents the result as motivation for domain specialization rather than as a universal law — but the implication is clearly stated: the future may require "a portfolio of specialized, reliable models" (Section 8) rather than a single general-purpose evaluator.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary evaluation uses the CFLUE benchmark (Zhu et al., 2024), a Chinese financial language understanding dataset. The paper evaluates on a 1,000-sample subset of the CFLUE test set for Best-of-N and SFT data selection experiments (Section 5.1, 5.2). For reinforcement learning experiments (Section 5.3), results are reported on both CFLUE (full test set, exact size unspecified but likely the same 1,000-sample subset) and FinQA, an English financial QA benchmark. Out-of-domain generalization is tested on the Math500 benchmark, consisting of 500 competition-level mathematics problems (Section 5.2).

• Base model(s). All core experiments use Qwen2.5-7B-Instruct as the policy model (the model being evaluated or fine-tuned). This 7-billion-parameter instruction-tuned model serves three roles: (i) the base model fine-tuned into Fin-PRM itself, (ii) the student model in SFT data selection experiments, and (iii) the policy model in GRPO reinforcement learning and Best-of-N generation. For candidate generation in the SFT data selection pipeline, Qwen3-8B is used to produce multiple reasoning trajectories per question. For the initial reasoning trace synthesis (constructing the training data for Fin-PRM), Deepseek-R1 serves as the teacher model. The knowledge extraction module uses Qwen3-235b-a22b. The choice of Qwen2.5-7B-Instruct as the primary experimental model reflects a practical scale that enables extensive experimentation (generating thousands of candidate responses, running RL training loops, performing BoN selection at multiple values of N) while remaining representative of production-capable model sizes.

• Metrics. Accuracy is the sole reported metric across all experiments, measured as the fraction of questions for which the model's selected or generated final answer exactly matches the ground-truth correct answer. For CFLUE, which uses multiple-choice format, this means exact option matching (e.g., selecting 'A', 'B', or 'ACD' must match precisely). For Math500, standard answer matching is used. For FinQA, the paper reports accuracy as the primary metric. No other metrics (F1, partial credit, calibration) are reported.

• Baselines. The paper evaluates against several distinct baselines depending on the experimental setting:

  • For Best-of-N selection (Section 5.2): Majority voting (select the most common final answer among N candidates) serves as the no-PRM baseline. Qwen2.5-Math-PRM-7B (Zhang et al., 2025) is the primary PRM baseline — a math-specialized process reward model of identical parameter count to Fin-PRM, enabling direct assessment of domain specialization effects. Qwen2.5-Math-PRM-72B is included as a larger-scale reference point for general-domain PRM capability.
  • For SFT data selection (Section 5.1): Random data selection (fine-tune on 1,000 randomly sampled trajectories from the candidate pool) serves as the negative control. The base model without fine-tuning (45.3% accuracy on CFLUE) provides the zero-shot baseline. Additional PRM-based selection methods using the math PRMs are compared, though specific numbers are only partially reported in Table 1.
  • For GRPO reinforcement learning (Section 5.3): Rule-based outcome reward using only $r_{\text{out}}$ (binary correctness of final answer) serves as the standard GRPO baseline. Qwen2.5-Math-PRM-7B used as a reward source provides the general-domain PRM baseline for RL.

• Generation budget / compute accounting. For Best-of-N experiments, the "budget" is simply N — the number of complete candidate responses generated by the policy model per question. Experiments sweep N = 4, 8, and 16 (Figure 2), with majority voting and PRM-based selection compared at each budget level. For SFT data selection, the candidate pool consists of multiple trajectories per question generated by Qwen3-8B (exact number per question not specified), from which the top 1,000 are selected for fine-tuning. For GRPO experiments, each training step generates 8 responses per prompt (the group size for advantage normalization), and training runs for 3 epochs (Appendix D.2, Table 4). Compute cost of PRM inference itself is not accounted for in any budget comparisons — the experiments measure downstream accuracy gains from PRM usage without measuring the FLOPs or latency cost of running the PRM to score candidates or compute rewards.

• Cross-validation / statistical protocol. The paper does not describe any cross-validation protocol, statistical significance testing, or confidence intervals for the reported results. The Best-of-N experiments are conducted on a fixed 1,000-sample subset of CFLUE (Section 5.2). The SFT data selection experiments use the same base model and training hyperparameters across conditions, with 1,000 selected samples for fine-tuning. The GRPO experiments report "mean accuracy over multiple runs" (Figure 3 caption), but the number of runs, variance across runs, and any statistical testing are not specified. Ablation studies (Section 6) sweep hyperparameter values on the same 1,000-sample test set, with no mention of held-out validation or cross-validation for hyperparameter selection, meaning the reported optimal $\zeta = 1.0$ may be partially tuned to the test set.

---

Main Quantitative Results

Supervised Fine-Tuning with Offline Data Selection (Section 5.1)

The headline result in Table 1 establishes the core claim: a student model fine-tuned on Fin-PRM-selected data substantially outperforms both random selection and general-purpose PRM selection. On the CFLUE benchmark, fine-tuning Qwen2.5-7B-Instruct on 1,000 trajectories selected by Fin-PRM achieves 58.2% accuracy, compared to 43.8% for random selection and 54.8% for selection using Qwen2.5-Math-PRM-7B (Table 1). The base model without fine-tuning achieves 45.3%, meaning random selection actually degrades performance by 1.5 percentage points, while Fin-PRM selection improves it by 12.9 percentage points.

Several details merit attention. First, the baseline model accuracy (45.3%) is substantially lower than what might be expected, suggesting CFLUE is a challenging benchmark where the base model struggles — this provides headroom for improvement from data selection. Second, the random selection result (43.8%) being worse than the base model indicates that the synthetic data pool contains a substantial fraction of low-quality or incorrect reasoning traces that actively harm the student model when included in training. This validates the paper's motivation: in financial domains, teacher model outputs (from Qwen3-8B in this case) are noisy enough that naive inclusion degrades performance. Third, the math-specialized PRM (Qwen2.5-Math-PRM-7B) provides a substantial improvement over random selection (54.8% vs. 43.8%), demonstrating that even domain-mismatched process supervision is better than no supervision for data filtering. However, the 3.4-point gap between Fin-PRM (58.2%) and the math PRM (54.8%) represents the marginal value of domain specialization specifically.

The paper does not provide a full breakdown of Table 1 in the main text — the table itself is only partially visible in the paper excerpts — but the comparisons that are reported suggest a clear ranking: Fin-PRM > Math-PRM-7B > base model > random selection. Notably, the larger Math-PRM-72B is not evaluated in the SFT data selection setting (it appears only in BoN experiments), leaving open the question of whether scale could compensate for domain mismatch in offline data filtering.

Best-of-N Test-Time Scaling (Section 5.2)

Figure 2 presents Best-of-N accuracy curves on CFLUE as N increases from 4 to 8 to 16. The central finding is that Fin-PRM consistently outperforms all baselines on financial reasoning, with the performance gap widening as N increases. At N = 16, Fin-PRM achieves approximately 5.1% higher accuracy than majority voting (the paper reports "more than 5.1% at N=16" in Section 5.2). The absolute accuracy values are not explicitly tabulated, but can be read approximately from Figure 2: Fin-PRM at N=16 reaches roughly 53–54%, while majority voting reaches roughly 48%.

The comparison with domain-mismatched PRMs reveals a striking pattern. Qwen2.5-Math-PRM-7B initially tracks close to Fin-PRM at N=4, but its performance plateaus or declines as N increases — at N=16, it falls below majority voting. The paper explicitly states this: "Qwen2.5-Math-PRM-7B shows performance below majority-voting when the number of N becomes larger" (Figure 2 caption). This is the empirical basis for the paper's claim that general-purpose PRMs can be actively harmful in domain-specific settings: the math PRM's scoring signal becomes anti-correlated with correctness as more candidates are generated, steering selection toward wrong answers more often than random voting would. The larger Qwen2.5-Math-PRM-72B also underperforms Fin-PRM on CFLUE, though the paper does not specify whether it falls below majority voting or merely underperforms Fin-PRM.

The out-of-domain generalization test on Math500 (Table 2) provides a critical complement. Fin-PRM achieves comparable or slightly lower performance than the math-specialized PRMs on mathematics problems, with the 7B versions performing similarly (both around the majority voting baseline). The larger Math-PRM-72B achieves the highest performance on Math500 (consistent with scale advantages for in-domain evaluation). The key result is not that Fin-PRM beats math PRMs on math — it doesn't — but that Fin-PRM does not catastrophically fail on out-of-domain evaluation. It maintains "respectable baseline capability" and "remains competitive with majority voting" (Section 5.2). This establishes an important asymmetry: domain specialization helps dramatically in-domain without destroying general evaluation capability, whereas general-domain PRMs can be harmful in specialized domains.

The Math500 results also validate that majority voting is a strong baseline for reasoning tasks when the generator model has non-trivial accuracy. The fact that both Fin-PRM and the math PRMs only marginally improve over majority voting on Math500 suggests that for well-studied domains where the policy model already produces reasonably calibrated outputs, PRM-based selection provides limited additional value — the gains from process supervision are most pronounced when the generator's outputs are noisy or when domain knowledge is needed to distinguish superficially similar but factually different reasoning paths.

Reinforcement Learning with GRPO (Section 5.3)

Figure 3 presents the downstream reasoning performance after GRPO policy optimization using different reward sources. The policy model (Qwen2.5-7B-Instruct) is trained with the GRPO algorithm using one of three reward signals: rule-based outcome reward ($r_{\text{out}}$ only), Qwen2.5-Math-PRM-7B as process reward, or Fin-PRM as process reward augmented with outcome blending (Equation 14, with $\delta$ controlling the mixture).

On CFLUE, Fin-PRM-guided RL achieves 70.5% accuracy, compared to 67.2% with rule-based outcome reward and an intermediate value for the math PRM baseline (exact number not reported in text, but visible in Figure 3 as approximately 65–66%, notably below the rule-based baseline). This represents a 3.3 percentage point gain from incorporating Fin-PRM's process-level rewards. On FinQA, a separate English financial QA benchmark, Fin-PRM-guided RL achieves 62.8%, compared to 59.5% with rule-based reward, again a 3.3-point gain. The math PRM baseline on FinQA performs roughly at 60% (again approximately at or below the rule-based baseline, based on Figure 3).

Several patterns in these results are noteworthy. First, the math PRM actually underperforms the simple rule-based outcome reward on CFLUE when used as an RL reward source. This extends the Best-of-N finding (math PRM worse than majority voting) to the RL setting: not only does the math PRM fail to help at inference time, it provides actively harmful training signals when used to guide policy optimization. The policy learns to produce reasoning that scores well under the math PRM but is actually incorrect by financial standards. This is a more damaging failure mode than the BoN case because the harm is baked into the model's parameters rather than just affecting test-time selection.

Second, the absolute performance levels are high — 70.5% on CFLUE represents strong performance on what the paper describes as a challenging benchmark (base model accuracy is only 45.3%). The GRPO training with Fin-PRM rewards improves over the base model by more than 25 percentage points, though this improvement is not solely attributable to Fin-PRM (outcome-based GRPO alone provides most of the gains, with Fin-PRM contributing the final 3.3 points). This suggests that while process-level rewards provide meaningful additional signal, the bulk of improvement in RL-based financial reasoning comes from the outcome-based RL optimization itself.

Third, the consistent 3.3-point gain across both benchmarks (CFLUE and FinQA) suggests the Fin-PRM reward signal transfers across languages and specific financial task formulations — CFLUE is Chinese, FinQA is English, and yet the marginal value of process supervision is nearly identical. This provides indirect evidence that the knowledge base and verification mechanisms capture general financial reasoning quality rather than benchmark-specific patterns.

Table 4 (Appendix D.2) provides the GRPO hyperparameters: 8 responses per prompt for group advantage computation, PPO clipping $\epsilon = 0.2$, KL penalty coefficient $\beta_{\text{KL}} = 0.04$, policy learning rate $1 \times 10^{-6}$, and 3 training epochs. The small learning rate and moderate KL penalty are standard for stable RL fine-tuning. The paper does not specify the value of $\delta$ (the mixture weight between outcome and process rewards in Equation 14) used in the reported experiments, which limits reproducibility and leaves unclear how sensitive the results are to this hyperparameter choice.

---

Ablation Studies and Robustness Checks

Ranking score weight $\zeta$ (Section 6, Figure 4): The hyperparameter $\zeta$ in Equation 13 controls the balance between averaged step-level rewards and trajectory-level rewards when computing the final ranking score for BoN selection. The ablation sweeps $\zeta$ across [0.0, 2.0] at N = 2, 4, 8, and 16 on the 1,000-sample CFLUE test set. The results show a clear inverted-U pattern: accuracy peaks at $\zeta = 1.0$ for N = 8 and N = 16, with performance declining on both sides. At $\zeta = 0$ (step-level rewards only), performance is "consistently suboptimal" (Section 6.3). As $\zeta$ increases beyond 1.0, performance degrades, indicating that over-relying on trajectory-level signals while ignoring step-level details leads to poorer selection — trajectories that look good globally but contain local flaws are selected over trajectories with stronger individual steps. For smaller N (2 and 4), the peak is less pronounced, likely because with fewer candidates, the discriminative power of the combined score matters less. The paper interprets the $\zeta = 1.0$ optimum as validation of the dual-granularity design: "a balanced integration of both local and global signals is essential for accurately identifying superior reasoning processes" (Section 6.3). However, this ablation was conducted on the test set, raising the concern that $\zeta = 1.0$ may be partially tuned to evaluation data rather than selected via cross-validation.

Additional parameter ablations (Section 6.3, Appendix): The paper states that "more ablation study about each parameters we used in paper can be found in appendix" (Section 6.3), but the provided appendix content (Appendix A through E in the paper) does not include additional parameter ablation results beyond Figure 4. The appendix covers model implementations (A.1), dataset details (A.2), prompt templates for reward signal construction (B.1, B.2), and training hyperparameters for Fin-PRM, SFT, and GRPO (C.1, C.2), but no ablation tables or figures beyond those in the main text are present. This is a notable gap: key hyperparameters like the softmax temperature (implicitly 1.0 in the standard softmax formulation), the binarization thresholds (0.5 for step labels, 1.25 for trajectory labels), the trajectory weight $\eta = 1.5$, the knowledge accuracy weight $\omega_k = 1.0$, the number of Monte Carlo rollouts N = 8, and the GRPO mixture weight $\delta$ are all set to fixed values without reported sensitivity analysis. The absence of these ablations means the paper cannot demonstrate which design choices are load-bearing and which are incidental.

Out-of-domain generalization (Section 5.2, Table 2): While not framed as an ablation, the Math500 experiment serves as an implicit robustness check on domain specialization. Fin-PRM, trained exclusively on Chinese financial reasoning data, achieves competitive performance on English mathematical reasoning problems. Table 2 shows that Fin-PRM and Qwen2.5-Math-PRM-7B perform similarly on Math500 across N = 4, 8, and 16, with both tracking close to majority voting. The 72B math PRM achieves the best performance, consistent with scale advantages. This demonstrates that Fin-PRM's training does not catastrophically overfit to financial reasoning patterns at the expense of general reasoning evaluation — a concern that would naturally arise from training on only 3,000 domain-specific samples. The paper frames this as evidence that Fin-PRM "retains a foundational ability to assess logical structure" (Section 5.2) even when financial knowledge is not applicable.

Case study with per-step reward values (Appendix E): The appendix presents a complete worked example — a CFLUE question about commercial bank business classification — with the full reasoning trace, each step's aggregated reward score, and the final trajectory label. The six reasoning steps receive step-level scores of 0.644, 0.769, 0.753, 0.686, 0.839, and 0.837, and the trajectory receives a label of 1 (Correct). This example is illustrative rather than evaluative — it shows what the reward signals look like on a correct trajectory but provides no comparison with incorrect trajectories or analysis of how the individual reward components (importance, quality, accuracy) contributed to each aggregated score. The scores are relatively high and consistent (0.64–0.84 range), which is expected for a correct trajectory, but the case study does not demonstrate the model's behavior on borderline cases where different reward components might disagree.

---

Critical Assessment

The experiments reported in this paper provide moderate support for the central claim that domain-specialized, knowledge-aware process supervision improves financial reasoning across multiple applications. However, several aspects of the experimental design limit the strength of the conclusions, and some important claims are supported only indirectly or not at all.

Claim: Fin-PRM improves SFT by 12.9% over the base model and substantially over general-purpose PRMs. The evidence in Table 1 supports this claim, but with important caveats. The base model accuracy of 45.3% provides a clear reference point, and the 58.2% achieved with Fin-PRM-selected data represents a substantial improvement. However, Table 1 is not fully shown in the paper, making it impossible to verify exactly which baselines were compared and at what performance levels. The random selection baseline (43.8%) being worse than the base model raises a methodological concern: if synthetic data quality is so poor that random inclusion degrades performance, then simply filtering out obviously wrong answers (using the ground-truth answer $y$, which is available during offline data construction) might account for a substantial fraction of the improvement. The paper does not include a baseline that selects trajectories based solely on outcome correctness ($r_{\text{out}}$), which would isolate how much of Fin-PRM's advantage comes from process-level evaluation versus simple answer filtering. This is a significant missing ablation: if outcome-only filtering achieves, say, 55%, then the marginal value of process supervision for SFT data selection is only 3 percentage points, not 12.9.

Claim: Fin-PRM enables Best-of-N selection that outperforms majority voting by 5.1% at N=16. The evidence in Figure 2 supports this claim for the specific model (Qwen2.5-7B-Instruct) and benchmark (CFLUE). However, the absolute performance scale matters. At N=16, Fin-PRM achieves roughly 53–54% accuracy — a 5-point improvement over majority voting's ~48%. This is a meaningful gain, but the absolute accuracy remains modest, and the improvement over N=8 (where Fin-PRM achieves roughly 50–51%) is only about 3 percentage points for a 2× increase in compute. The paper does not test beyond N=16, leaving open whether the Fin-PRM advantage continues to scale, plateaus, or eventually declines (as the math PRM does). The comparison to the math PRM's negative scaling is compelling evidence for domain specialization, but the absolute value of the improvement — 5 points at 2× the compute of N=8 — should be weighed against the computational cost of running Fin-PRM to score all candidates.

Claim: General-purpose PRMs are actively harmful in domain-specific settings. This is the paper's most striking finding, and the evidence in Figure 2 is clear: Qwen2.5-Math-PRM-7B falls below majority voting at larger N. However, this result is demonstrated for one general-purpose PRM on one benchmark. The paper does not test other general PRMs (e.g., Skywork-PRM, ReasonFlux-PRM, Open-PRM) to establish whether this is a general phenomenon or specific to Qwen2.5-Math-PRM's training. The larger Qwen2.5-Math-PRM-72B is tested but its performance relative to majority voting at high N is not explicitly stated. More importantly, the mechanism for the harmful effect — the math PRM rewarding plausible-sounding but incorrect financial reasoning — is hypothesized but not empirically validated. The paper does not analyze which types of errors the math PRM systematically misses or what patterns in financial reasoning cause the anti-correlation. Without this diagnostic analysis, the claim that domain mismatch causes active harm is supported observationally but not mechanistically.

Claim: Fin-PRM improves GRPO reinforcement learning by 3.3 points on both CFLUE and FinQA. Figure 3 supports this, with Fin-PRM-guided RL achieving 70.5% on CFLUE vs. 67.2% for outcome-only RL. The consistency across benchmarks (3.3 points on both) is striking and suggests a genuine effect rather than noise. However, the GRPO experiments have several reporting gaps. The value of $\delta$ (the mixture weight between outcome and process rewards) is not specified, making reproduction impossible and leaving unclear whether the 3.3-point gain is robust to this hyperparameter choice or only achievable at a narrow sweet spot. The "multiple runs" over which accuracy is averaged are not quantified — if this is 2–3 runs with unknown variance, the 3.3-point difference may not be statistically significant. The math PRM baseline for GRPO shows performance below the rule-based outcome reward on CFLUE (Figure 3), a negative result that is consistent with the BoN finding but not discussed in sufficient detail — understanding why process rewards from a math PRM harm RL training would strengthen the paper's argument for domain specialization.

Missing experiments that would strengthen the paper:

• Outcome-only filtering baseline for SFT. Selecting trajectories based solely on whether the final answer matches ground truth would isolate the value of process-level evaluation for data curation.
• Ablation of individual reward components. Removing $r_{\text{importance}}$, $r_{\text{qual}}$, or $r_{\text{acc}}$ one at a time from the training objective would reveal which components are essential and which are redundant. The dynamic softmax weighting design implies all three matter, but this is untested.
• Sensitivity to dataset size. The 3,000-sample training set is modest; performance as a function of training data quantity would inform whether Fin-PRM's gains are data-limited or architecture-limited.
• Scaling with model size. All experiments use 7B-scale models. Testing whether Fin-PRM's advantages persist or diminish with larger policy models (e.g., Qwen2.5-72B) would clarify whether domain-specialized PRMs are most valuable for smaller, less capable generators or scale to any model size.
• Confidence intervals or significance tests. None of the quantitative results include error bars, confidence intervals, or significance tests. With a 1,000-sample test set, differences of 3–5 percentage points may or may not be statistically reliable.
• Beyond Qwen model family. All experiments use Qwen-family models (Qwen2.5-7B-Instruct, Qwen3-8B, Qwen3-235b-a22b, Deepseek-R1). Testing with a different base model architecture (e.g., LLaMA, Mistral) would demonstrate that the approach generalizes beyond a single model family's reasoning characteristics.

Conditional nature of the claims. The paper's findings are conditional on several factors that are not varied experimentally:

• Domain characteristics. CFLUE is a Chinese-language financial benchmark with multiple-choice format and expert-provided analyses. The paper's approach relies on having high-quality expert analyses from which to extract a knowledge base. In financial domains without such curated expert content (or in domains where expert knowledge is less structured), the knowledge extraction and verification pipeline may not function.
• Teacher model capability. The reasoning traces are generated by Deepseek-R1, a state-of-the-art reasoning model. If a less capable teacher model were used, the synthetic traces would be lower quality, potentially degrading Fin-PRM's training. The paper does not test sensitivity to teacher model quality.
• Knowledge base coverage. The accuracy and coverage rewards can only verify claims against terms present in the extracted knowledge base. If $\mathcal{K}_x$ is incomplete — missing relevant financial concepts or containing outdated information — the verification checks will have false negatives (marking correct steps as unsupported) or false positives (failing to catch unsupported claims). The paper does not evaluate knowledge base completeness or its impact on reward quality.
• Language and benchmark format. CFLUE is Chinese and multiple-choice. The FinQA benchmark used in GRPO is English. The implications for other financial reasoning formats (numerical answer, open-ended analysis, regulatory compliance checking) are not tested.

Overall, the experimental evidence demonstrates that Fin-PRM provides consistent, moderate improvements over general-purpose baselines across three distinct applications (SFT data selection, BoN, and RL). The results are directionally consistent with the paper's thesis that domain specialization and knowledge grounding matter. However, the experiments leave unquantified the marginal contribution of Fin-PRM's specific architectural innovations (multi-faceted rewards, dual-level training, dynamic softmax weighting) versus simpler domain-adaptation approaches (e.g., fine-tuning a general PRM on financial outcome labels). The absence of component ablations, sensitivity analyses, and statistical rigor means the paper demonstrates that the approach works for this specific setup without fully establishing why each design choice matters or how broadly the findings generalize.

6. Limitations and Trade-offs

Knowledge Base Is Static and Extraction Is Brittle

The assumption or constraint. Fin-PRM's accuracy score ($r_{\text{acc}}$) and coverage score ($r_{\text{cover}}$) depend entirely on the knowledge base $\mathcal{K}$ extracted from CFLUE's expert analyses using Qwen3-235b-a22b. The paper explicitly acknowledges this limitation in Section 7: "our knowledge base $\mathcal{K}$ is static, which poses a risk in a dynamic field like finance where regulations and market conditions evolve." The extraction process itself relies on an LLM (Qwen3-235b-a22b) to identify and define financial terms from human-written analyses (Appendix B.2, Listing 2), meaning the knowledge base is only as complete and accurate as both the source expert analyses and the extraction model's capability.

The consequence. A static knowledge base creates two distinct failure modes. First, knowledge decay: financial regulations, accounting standards, and market conventions change over time. Terms extracted today may become outdated — a regulatory definition from the CFLUE dataset might not reflect current requirements, causing the accuracy verifier to mark factually correct steps (by current standards) as unsupported, or worse, to validate steps that rely on obsolete information. Second, coverage gaps: if the CFLUE expert analyses omit certain financial concepts relevant to a question, or if the extraction model fails to identify a concept that is implicitly present, $\mathcal{K}_x$ will be incomplete. When $\mathcal{K}_x$ is incomplete, the factual accuracy check ($r_{\text{knowledge\_acc}}$ in Equation 6) cannot detect errors related to missing concepts — a step could make false claims about an un-extracted term and still receive a perfect accuracy score. Similarly, the coverage score ($r_{\text{cover}}$ in Equation 7) would penalize the model for not mentioning concepts it couldn't possibly know were required. The paper provides no mechanism to detect or quantify these knowledge gaps, meaning the PRM's confidence is uncalibrated with respect to its own ignorance — it may assign high scores to reasoning that is incorrect relative to concepts outside its knowledge base.

What evidence exists in the paper. No experiment quantifies knowledge base completeness, extraction accuracy, or the impact of knowledge gaps on reward quality. The case study in Appendix E demonstrates the pipeline working on a single correct example but provides no analysis of failures caused by missing or incorrect knowledge base entries. The paper's out-of-domain test on Math500 (Table 2) shows Fin-PRM maintaining competitive performance on general math, which indirectly suggests the model is not over-relying on financial knowledge for all evaluation, but this does not address the specific failure mode of knowledge gaps within financial reasoning. The paper does not evaluate Fin-PRM on financial questions that require knowledge outside the CFLUE expert analyses, meaning the generalization to novel financial concepts is completely untested.

Mitigation status. The paper acknowledges this limitation explicitly in Section 7 and proposes a future direction: "Integrating dynamic knowledge sources, such as real-time financial news feeds or regulatory update APIs, would be a critical enhancement to prevent knowledge decay and ensure long-term reliability." This is presented as future work with no implementation or evaluation. No attempt is made to characterize the current knowledge base's coverage, to measure extraction accuracy against human-verified financial knowledge, or to test sensitivity to incomplete knowledge (e.g., by artificially removing concepts from $\mathcal{K}_x$ and measuring the impact on reward accuracy).

Difficulty Estimation Cost Is Unaccounted For and Prohibitive

The assumption or constraint. The paper reports accuracy improvements from Fin-PRM's three applications — 12.9% in SFT data selection, 5.1% in Best-of-N selection, 5.2% in reinforcement learning — without accounting for the computational cost of Fin-PRM inference itself in any of the comparisons. For SFT data selection, Fin-PRM must score every candidate trajectory in the pool (potentially multiple trajectories per question across the entire training set) before selecting the top 1,000. For Best-of-N selection, every one of the N candidate responses must be scored by Fin-PRM before the best can be selected — meaning the PRM inference cost scales linearly with N. For GRPO reinforcement learning, Fin-PRM must score 8 responses per prompt at every training step across 3 epochs. None of these costs appear in the computation budgets, latency measurements, or efficiency analyses.

The consequence. The reported gains are gross improvements, not net improvements accounting for total compute. In the Best-of-N setting, comparing Fin-PRM at N=16 to majority voting at N=16 is not a FLOPs-matched comparison: majority voting requires only answer extraction and counting, while Fin-PRM requires running a full 7B-parameter model on every candidate response (potentially generating per-step and per-trajectory scores). If Fin-PRM inference is even 25% as expensive as the policy model's generation, the effective budget comparison shifts — a system using Fin-PRM at N=16 might actually be spending the compute equivalent of N=20 or more, making the accuracy gains partially attributable to higher effective compute rather than better selection quality. For reinforcement learning, the additional cost of running Fin-PRM as a reward model at every GRPO step could make training substantially more expensive than outcome-only RL, potentially eroding or negating the 3.3-point accuracy gain when compared under a fixed training compute budget. The paper's silence on PRM inference cost means practitioners cannot assess whether the reported accuracy improvements justify the additional computational expenditure.

What evidence exists in the paper. The paper provides no measurements of Fin-PRM inference latency, FLOPs, or cost relative to policy model generation. The training hyperparameters in Table 3 (batch size 128, learning rate $1 \times 10^{-5}$, A100 GPUs) describe Fin-PRM training cost but not inference cost. The GRPO hyperparameters in Table 4 (group size 8, 3 epochs) allow a rough inference of PRM inference budget during RL training, but no comparison is made to outcome-only RL training time or compute. The BoN experiments (Figure 2) compare accuracy against N but do not include a compute-normalized comparison (e.g., accuracy per FLOP) that would account for PRM inference overhead.

Mitigation status. The paper does not acknowledge this as a limitation. There is no discussion of Fin-PRM inference cost, no efficiency comparison between methods under equal compute budgets, and no suggestion for reducing deployment overhead (e.g., distillation, quantization, or amortized scoring). The omission is particularly notable given that the Best-of-N results compare against majority voting — a method with near-zero additional inference cost — without any acknowledgment that the comparison is not FLOPs-matched.

Framework Validated on a Single Model Family and Narrow Task Format

The assumption or constraint. All experiments use Qwen-family models: Qwen2.5-7B-Instruct as Fin-PRM's base, the SFT student, and the GRPO/BoN policy model; Qwen3-8B for candidate generation in SFT data selection; Qwen3-235b-a22b for knowledge extraction and LLM-as-a-judge scoring; Qwen2.5-7B-Math for Monte Carlo rollouts; and Deepseek-R1 as the teacher model for initial reasoning trace synthesis. The primary evaluation benchmark (CFLUE) is Chinese-language, multiple-choice format, with expert-written analyses provided as part of the dataset. The secondary RL benchmark (FinQA) is English and also QA-format. The paper does not specify whether CFLUE questions span different financial subdomains (accounting, regulation, investment analysis, risk management) or are concentrated in particular areas.

The consequence. The paper cannot distinguish between what is specific to Qwen models and what generalizes. Several aspects of the pipeline could be model-dependent: (a) Deepseek-R1's reasoning trace quality shapes the entire training dataset — if Deepseek-R1 produces particularly structured or financial-domain-savvy traces, Fin-PRM may be learning to evaluate Deepseek-R1-style reasoning rather than financial reasoning in general; (b) Qwen3-235b-a22b's capability as the LLM judge for qualitative scoring ($r_{\text{qual}}$ in Equation 5) and knowledge extraction determines reward signal quality — a weaker judge model might produce noisier labels, and a stronger one might mask the value of the other reward components; (c) Qwen2.5-7B-Instruct's base capability affects the headroom available for improvement — a model family with different financial reasoning baseline might show larger or smaller gains from Fin-PRM. The multiple-choice format of CFLUE is also consequential: distinguishing correct from incorrect reasoning is harder in open-ended financial analysis tasks where correctness is nuanced rather than binary. The paper's approach to accuracy checking ($r_{\text{step\_correctness}}$ uses the ground-truth answer $y$, and $r_{\text{out}}$ checks exact option matching) depends on having a single verifiable correct answer — this does not extend to tasks where multiple valid answers exist or where correctness is a matter of degree.

What evidence exists in the paper. The Math500 out-of-domain test (Table 2) provides the only evidence of cross-domain generalization, but it tests mathematics (a different reasoning domain) rather than a different model family or open-ended financial task format. The FinQA benchmark used in GRPO experiments (Section 5.3) is English and separate from CFLUE, but it is also QA-format and the paper does not analyze whether CFLUE and FinQA cover different financial subdomains. No experiments test Fin-PRM with non-Qwen policy models, different teacher models for trace generation, or open-ended financial reasoning tasks.

Mitigation status. The paper does not discuss model-family dependence or task format limitations. It does not claim generalization beyond the tested setup, but it also does not acknowledge that the findings are conditionally valid only for Qwen-family models on multiple-choice Chinese financial questions. The "blueprint for developing trustworthy AI evaluators in other high-stakes fields like law and medicine" (Section 8) implicitly assumes the approach transfers across domains, model families, and task formats, but no evidence supports this assumption.

Binarization Thresholds Are Arbitrary and Their Sensitivity Is Unexplored

The assumption or constraint. The conversion from continuous reward signals to binary training labels requires threshold choices at two critical points: step-level labels are binarized at 0.5 (after softmax aggregation in Equation 8), and trajectory-level labels are binarized at 1.25 (after combining outcome and coverage scores in Equation 9). These thresholds are set to fixed values with brief justifications: the 0.5 step threshold is described as a natural midpoint ("a step is labeled as 'good' if the aggregated score is at least halfway between 'definitely bad' and 'definitely good'"), and the 1.25 trajectory threshold is set to the mean of the two component weights ($(1 + 1.5)/2 = 1.25$), which the paper says "gives the ability to change the hard label to each reward signal." The trajectory weight $\eta = 1.5$ is similarly fixed without optimization.

The consequence. Binary classification training with cross-entropy loss means Fin-PRM learns to predict these binary labels. If the thresholds are poorly chosen — too high, labeling genuinely acceptable steps as negative — the PRM learns an overly strict evaluation criterion that rejects valid reasoning. If too low, the PRM becomes permissive and fails to catch errors. More subtly, the relationship between the 0.5 step threshold and the 1.25 trajectory threshold determines the relative difficulty of the two classification tasks. Since both contribute to the joint loss (Equation 10, with an unspecified $\lambda$ balancing them), imbalanced difficulty could cause the model to prioritize one granularity over the other regardless of which would be more useful for downstream applications. The trajectory threshold of 1.25 encodes a specific value judgment: as noted in Section 4 (Innovation 3), it means a trajectory with wrong answer but full coverage ($r_{\text{out}} = 0$, $r_{\text{cover}} = 1$, score = 1.5) is labeled positive, while a trajectory with correct answer but no coverage ($r_{\text{out}} = 1$, $r_{\text{cover}} = 0$, score = 1.0) is labeled negative. Different threshold choices would encode different value judgments — a threshold of 1.0 would treat these cases symmetrically, while a threshold of 0.75 would prioritize outcome correctness. The paper provides no evidence that 1.25 is the "right" tradeoff for downstream performance.

What evidence exists in the paper. No ablation study examines sensitivity to the binarization thresholds. The ablation in Figure 4 only sweeps the deployment-time ranking weight $\zeta$, not any of the training-time threshold parameters. Table 3 (Fin-PRM training hyperparameters) does not list the binarization thresholds, suggesting they are treated as fixed design choices rather than tunable hyperparameters. The appendix states that "more ablation study about each parameters we used in paper can be found in appendix" (Section 6.3), but no such ablation results appear in the provided appendix content.

Mitigation status. The paper provides only qualitative justifications for the threshold choices without empirical validation. The sensitivity of downstream performance (SFT data selection quality, BoN accuracy, GRPO policy improvement) to these thresholds is completely unknown. A practitioner attempting to replicate Fin-PRM for a different domain would have no guidance on how to set analogous thresholds — whether the 0.5 step threshold and mean-of-weights trajectory threshold are general principles or artifacts that happen to work for CFLUE.

Hard Problems Show Near-Zero Improvement — the Approach Amplifies Existing Capability but Cannot Create It

The assumption or constraint. Fin-PRM evaluates and selects among existing reasoning traces — it does not generate new reasoning or correct errors in generated traces. This means the approach is fundamentally bounded by the quality of the best available reasoning trace for a given question. If the policy model (or the candidate pool for SFT selection) contains no correct or partially-correct reasoning for a problem, Fin-PRM cannot improve performance beyond what is already present.

The consequence. On questions that genuinely require financial reasoning capability beyond what the base model possesses, Fin-PRM provides zero benefit. This is visible in the paper's results, though not broken out by difficulty. The base model achieves 45.3% on CFLUE, meaning it fails on 54.7% of questions. After SFT with Fin-PRM-selected data, accuracy reaches 58.2% — a 12.9-point improvement — but 41.8% of questions remain unsolved. For at least some fraction of these remaining failures, the limitation is likely not that Fin-PRM selected poor training data, but that no trajectory in the candidate pool contained correct reasoning — the policy model simply cannot solve these problems, and process supervision cannot create capability that isn't present in any candidate. This is the same fundamental limitation observed in the compute-optimal test-time scaling literature (Snell et al., 2024), where the hardest difficulty bin shows flat scaling curves regardless of compute budget. In GRPO (Section 5.3), Fin-PRM achieves 70.5% — a 25-point improvement over the base model's 45.3%, but still leaving ~30% of problems unsolved. Some of these may be genuinely outside the capability range that RL with process rewards can reach. The paper does not distinguish between failures caused by poor candidate quality (which better process supervision could address) and failures caused by fundamental capability gaps (which no amount of evaluation can fix). This distinction matters for practitioners: if the remaining errors are predominantly on problems where the model has no path to correctness, further investment in Fin-PRM will yield diminishing returns, and the correct response is to improve the base model's financial reasoning capability rather than the evaluation infrastructure.

What evidence exists in the paper. The paper does not provide a difficulty-stratified analysis of where Fin-PRM's gains occur and where they don't. There is no breakdown of accuracy by question difficulty, by financial subdomain, or by reasoning trace length/complexity. The base model's 45.3% accuracy implies there is a substantial fraction of problems where the model's pass@1 is very low (potentially near zero for some subset), but the paper does not analyze whether Fin-PRM provides any benefit on these hardest problems or whether its gains are concentrated on problems where the base model already has non-trivial capability. The GRPO results showing improvement from 67.2% to 70.5% with Fin-PRM suggest that even after RL training, there are problems where the policy produces no correct answers in a group of 8, meaning Fin-PRM has no correct option to select.

Mitigation status. The paper does not acknowledge this capability bound or discuss it as a limitation. There is no analysis of which types of financial reasoning problems Fin-PRM helps with and which it doesn't, no measurement of the relationship between base model pass@1 and Fin-PRM's marginal improvement, and no discussion of the implications for practitioners deciding whether to invest in better PRMs versus better base models for financial reasoning.

No Quantified Comparison Against Simpler Domain-Adaptation Baselines

The assumption or constraint. Fin-PRM's architecture combines three step-level reward signals (importance, quality, accuracy) with two trajectory-level signals (outcome, coverage), uses dynamic softmax weighting for step-level aggregation, trains with a joint step-trajectory loss, and deploys with a combined scoring formula (Equation 13). The paper compares Fin-PRM against general-purpose PRMs (Qwen2.5-Math-PRM-7B and -72B), majority voting, random data selection, and rule-based outcome rewards. However, it does not compare against the most obvious baseline for domain specialization: fine-tuning an existing general-purpose PRM on financial outcome labels — without knowledge base extraction, without multi-component reward decomposition, and without trajectory-level coverage scores. Such a baseline would isolate how much of Fin-PRM's gain comes from its specific architectural innovations versus the simple act of exposing a PRM to financial data.

The consequence. The 12.9% SFT improvement, 5.1% BoN improvement, and 5.2% RL improvement attributed to Fin-PRM may substantially overstate the marginal value of the paper's architectural contributions. If simply fine-tuning Qwen2.5-Math-PRM-7B on CFLUE outcome labels achieves, say, 55% in SFT data selection (vs. Fin-PRM's 58.2%) and 51% in BoN at N=16 (vs. Fin-PRM's ~53%), then the paper's specific innovations account for only ~3 and ~2 percentage points respectively — useful but far less transformative than the headline numbers suggest. The dynamic softmax weighting, dual-level training, and knowledge verification components all add complexity to the training pipeline (requiring MC rollouts, LLM-as-a-judge scoring, knowledge base extraction, and coverage computation). Without an ablation showing that each component contributes independently to performance, a practitioner has no basis to decide which parts of the Fin-PRM recipe are load-bearing versus which can be simplified for their domain.

What evidence exists in the paper. The paper provides no domain-adapted baseline beyond the out-of-box math PRMs. There is no experiment where a general PRM is fine-tuned on any financial data, with or without the knowledge verification components. The ablation study (Figure 4) only varies the deployment-time hyperparameter $\zeta$, not any training-time architectural choices. Component-level ablations — removing $r_{\text{qual}}$, removing $r_{\text{acc}}$, removing trajectory-level training, using fixed weights instead of dynamic softmax, using outcome-only labels for training — are entirely absent. The paper claims that each component is important based on conceptual arguments (Section 4) rather than empirical evidence.

Mitigation status. The paper does not acknowledge the absence of these baselines as a limitation. The related work section (Section 2.1) discusses general PRMs and notes their domain limitations, but the experimental design jumps directly from "general PRMs fail" to "our full multi-component domain-specialized PRM succeeds" without establishing whether intermediate approaches would also succeed. The paper's framing in Section 7 — "The key to Fin-PRM's performance is its dual-level, knowledge-aware architecture. By integrating verifiable reward components ($r_{\text{acc}}$ and $r_{\text{cover}}$) grounded in an expert-derived knowledge base, Fin-PRM moves beyond assessing mere logical plausibility to penalizing factual hallucinations" — attributes the gains to specific architectural features that were never empirically isolated. A skeptical reader could equally attribute the gains to the simpler explanation that Fin-PRM was trained on in-domain financial data while the baseline PRMs were not, and the knowledge verification components are unnecessary overhead.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a new design axis for process reward models — verifiability against external domain knowledge — that is orthogonal to the standard PRM research axes of model architecture, training data scale, or reward granularity. Before Fin-PRM, PRM development operated under an implicit assumption that reasoning evaluation is largely domain-invariant: a well-trained PRM learns universal structural properties of good reasoning (logical coherence, step-by-step progression, valid inference), and while domain transfer might require some fine-tuning, the core evaluation capability should generalize. The experimental finding in Figure 2 — that Qwen2.5-Math-PRM-7B performs worse than majority voting on financial reasoning at higher sample counts — provides a concrete counterexample to this assumption. It is not that the math PRM is merely suboptimal in finance; its scoring signal becomes actively anti-correlated with correctness, steering selection toward wrong answers more often than random voting would. This is a diagnostic result with significant implications: it establishes that PRM evaluation quality can be qualitatively domain-dependent, not just imperfectly transferred.

The paper's central methodological contribution is the framework for constructing reward signals that are anchored in an independently verified knowledge base rather than derived solely from the teacher model's outputs or from statistical rollout success. This reframes reward modeling from a purely statistical problem (learn to predict what evaluators would say) to a partially deductive one (verify claims against a trusted external reference). In a statistical-only PRM, errors are unbounded — the model can be confidently wrong about factual claims it has never seen, because its training objective is to imitate labels, not to verify facts. In a verification-augmented PRM like Fin-PRM, errors are bounded by the coverage of the knowledge base: the model can still be wrong if $\mathcal{K}$ is incomplete, but it cannot be confidently wrong about claims that $\mathcal{K}$ covers, because those claims are explicitly checked during training label construction. For high-stakes domains where factual errors carry direct costs (finance, medicine, law, engineering), this bounded-error property represents a qualitatively different trust model — the PRM's judgments are falsifiable against a specific, inspectable knowledge source, rather than being the opaque output of a statistical model trained to mimic evaluators.

This work also reconciles a tension in the PRM literature between the apparent power of general-purpose evaluators (math PRMs achieve strong results on math benchmarks, and LLM-as-a-judge approaches work across many domains) and the documented fragility of these evaluators under distribution shift (reward hacking, over-optimization, and the paper's own finding of anti-correlation with correctness in finance). The tension is not that general evaluators are useless — it's that their reliability depends on the evaluator's training distribution matching the deployment distribution in ways that are not captured by surface-level similarity. A math PRM and a financial reasoning problem both involve "step-by-step logical deduction," but the features that correlate with correctness in math (symbolic manipulation, explicit intermediate calculations, formal proof structure) are not the same features that correlate with correctness in finance (proper application of domain concepts, regulatory awareness, appropriate consideration of context). The Fin-PRM framework resolves this tension not by arguing against general evaluators but by proposing that domain specialization requires more than fine-tuning on domain data — it requires restructuring the reward signal to incorporate external verification, because the evaluation-relevant features differ qualitatively across domains.

The implication for the field is a shift away from the implicit goal of building "one PRM to rule them all" toward a portfolio approach where specialized evaluators serve different domains, unified by a common architecture (multi-faceted reward decomposition, external knowledge grounding, dual-level training) but distinguished by their domain-specific knowledge bases. The paper explicitly proposes this in Section 8: "this framework serves as a blueprint for developing trustworthy AI evaluators in other high-stakes fields like law and medicine, promoting a shift towards a portfolio of specialized, reliable models." This is not a call for bespoke PRMs for every niche — the Math500 results (Table 2) show that Fin-PRM retains competitive general reasoning evaluation capability even though it was trained on Chinese financial data. Rather, it suggests that the default deployment strategy for PRMs should be domain-specialized, with general evaluation capability retained as a byproduct of good training rather than as the primary design goal. The asymmetry is important: domain-specialized PRMs can generalize outward (Fin-PRM on Math500), but general PRMs cannot safely be specialized by deployment alone (Math-PRM on CFLUE). This inverts the natural engineering impulse to build general tools first and specialize later.

The paper also provides a new diagnostic for PRM quality that goes beyond benchmark accuracy: the comparison against majority voting at increasing sample sizes. The finding that a domain-mismatched PRM falls below majority voting at high N (Figure 2) provides a simple, computationally cheap test for whether a PRM is appropriate for a given domain — if it underperforms majority voting, its scoring signal is harmful and should not be used. This diagnostic does not require ground-truth labels for the target domain (majority voting only needs multiple candidate answers), making it practical for domain transfer evaluation. If this diagnostic generalizes, it could become a standard part of the PRM evaluation toolkit, analogous to how the RLHF community checks for reward over-optimization by comparing policy performance under the learned reward against true performance.

Follow-Up Research This Work Enables

Isolate the marginal value of each reward component through systematic ablation. The paper's central architectural claim — that the three-component step-level reward decomposition (importance, quality, accuracy) and dual-level training are essential — is untested. A high-priority follow-up would train and evaluate at least six Fin-PRM variants: (1) importance-only, (2) quality-only, (3) accuracy-only, (4) importance + quality (removing knowledge verification), (5) all step-level components but no trajectory-level loss, and (6) the full model. Each variant would be evaluated on the same three downstream tasks (SFT data selection, BoN, GRPO) on CFLUE. The key question is whether accuracy ($r_{\text{acc}}$) provides unique value beyond what importance and quality can capture, and whether trajectory-level training improves performance beyond step-level training alone. If accuracy-only performs nearly as well as the full model, the knowledge extraction pipeline is load-bearing and the other components are auxiliary; if importance + quality matches the full model, the knowledge verification components add complexity without marginal value. This ablation would also reveal whether the dynamic softmax weighting (Equation 8) is necessary — comparing it against a fixed-weight average (e.g., $1/3$ each) would show whether the attention mechanism provides robustness or is an unnecessary complication.

Test whether domain-specialized PRMs scale with policy model capability. The paper's experiments use Qwen2.5-7B-Instruct as the policy model, which achieves only 45.3% base accuracy on CFLUE. A critical open question is whether Fin-PRM's advantages persist, shrink, or grow when the policy model is more capable. A strong follow-up would replicate the BoN and GRPO experiments with Qwen2.5-72B-Instruct (or a comparable 70B-scale model) as the policy model. If the base model's CFLUE accuracy is, say, 65%, does Fin-PRM still provide a 5-point BoN improvement over majority voting? Or does the gap narrow because a more capable policy model produces reasoning that is already better-structured, reducing the value of process supervision? Conversely, if the policy model is strong but its errors are concentrated on the hardest problems where financial knowledge matters most, Fin-PRM's knowledge verification might become more important, not less. This experiment would establish the ceiling and floor for domain-specialized PRMs across the capability spectrum and inform whether they are most valuable as training wheels for weak models or as quality assurance for strong ones.

Evaluate Fin-PRM on open-ended financial reasoning tasks without a single correct answer. The CFLUE benchmark uses multiple-choice format, meaning every question has exactly one correct answer, and the accuracy score ($r_{\text{step\_correctness}}$ and $r_{\text{out}}$) can rely on exact match against the ground truth. Many real-world financial reasoning tasks — writing an investment memorandum, assessing regulatory compliance, analyzing a company's risk exposure — have nuanced, multi-dimensional correctness where an answer can be partially right, well-reasoned but reaching a wrong conclusion, or correct but incomplete. A strong follow-up would adapt Fin-PRM to an open-ended financial reasoning benchmark (e.g., FinBen, or a constructed dataset of financial analyst reports graded by expert humans). The key adaptation would be replacing binary outcome correctness with a graded evaluation rubric, and testing whether the knowledge coverage score ($r_{\text{cover}}$) becomes more important relative to outcome correctness when correctness is continuous rather than binary. This would reveal whether the Fin-PRM framework is specific to QA-format tasks or genuinely generalizes to the broader landscape of financial reasoning.

Measure and mitigate the cost of knowledge base construction and maintenance. The paper's knowledge base $\mathcal{K}$ is extracted from CFLUE's expert analyses using Qwen3-235b-a22b, with no measurement of extraction accuracy, coverage, or the human effort required to verify and correct the extracted knowledge. A replication study that carefully measures these costs — how many extracted terms are correct, how many relevant terms are missed, how much expert time is needed to clean the knowledge base — would quantify the practical barrier to deploying Fin-PRM-style PRMs in new domains. Additionally, testing whether a smaller extraction model (e.g., Qwen2.5-7B) produces knowledge bases of acceptable quality would inform whether the extraction step requires a large, expensive LLM or can be done cheaply. A follow-up that measures downstream Fin-PRM performance as a function of knowledge base quality (by artificially degrading $\mathcal{K}$ through random deletion, insertion of false terms, or substitution of outdated definitions) would establish the sensitivity curve and inform how much investment in knowledge base quality is justified for a given performance target.

Build a Fin-PRM for a different high-stakes domain and test cross-domain generalization. The paper claims the framework is a "blueprint" for law and medicine (Section 8), but provides no evidence beyond finance. A direct follow-up would replicate the entire Fin-PRM pipeline for a legal reasoning benchmark (e.g., LexGLUE, or a dataset of bar exam questions with expert analyses) or a medical reasoning benchmark (e.g., MedQA, with textbook-derived knowledge bases). The key measurements would be: (1) does the domain-specialized PRM outperform general PRMs on the target domain? (2) does it maintain competitive performance on out-of-domain benchmarks (the equivalent of Math500 for each domain)? and (3) is the performance gain proportional to the knowledge-intensity of the domain? If law and medicine show similar 5–13% improvements over general PRMs, the blueprint claim is validated. If one domain shows large gains and another shows none, the framework's applicability boundaries would be established, revealing which domain characteristics (structured knowledge, verifiable facts, expert-written reference materials) are preconditions for success.

Investigate whether Fin-PRM can be used to improve its own knowledge base through iterative refinement. The paper's knowledge base is static — extracted once from CFLUE expert analyses and never updated. A more ambitious follow-up would use Fin-PRM itself to identify gaps in $\mathcal{K}$: when the PRM flags a reasoning step as incorrect due to factual inaccuracy ($r_{\text{knowledge\_acc}}$), but the step is actually correct (based on outcome verification or human judgment), that discrepancy signals a missing or incorrect knowledge base entry. These signals could be aggregated across many evaluations to produce a prioritized list of knowledge base updates, creating a feedback loop where the PRM's evaluations become progressively more accurate as the knowledge base improves. This would address the paper's acknowledged limitation that $\mathcal{K}$ is static and incomplete, and would demonstrate a path toward self-improving evaluation systems for dynamic domains.

Practical Applications and Downstream Use Cases

Financial SFT data curation at scale. Organizations building financial LLMs — banks training internal models on proprietary financial data, fintech companies developing customer-facing financial assistants, regulatory technology firms building compliance-checking systems — routinely generate large pools of synthetic reasoning data from teacher models. The paper's SFT data selection result (Table 1) demonstrates that filtering this data with a domain-specialized PRM improves downstream model accuracy by 12.9 percentage points over random selection (58.2% vs. 43.8%) and by 3.4 points over selection with a general math PRM (54.8%). For a financial institution generating millions of synthetic training examples, this translates to substantially better model performance at fixed training cost, or equivalent performance with 4× fewer training examples (since random selection's 43.8% is below the base model's 45.3%, meaning much of the synthetic data is actively harmful). The specific recipe — score all candidates with Fin-PRM using Equation 13 with $\zeta = 1.0$, select the top-scoring trajectories, fine-tune — is directly implementable given access to a Fin-PRM model and a pool of candidate reasoning traces.

Test-time verification for high-stakes financial outputs. In deployment scenarios where a single incorrect answer carries significant cost — automated financial reporting, regulatory filing preparation, investment recommendation generation — Best-of-N selection with Fin-PRM provides a practical safety layer. Figure 2 shows that at N=16, Fin-PRM achieves approximately 53–54% accuracy on CFLUE, compared to ~48% for majority voting, a 5-point improvement that represents roughly 10% fewer errors. The deployment pattern is straightforward: for each user query, generate N=8–16 candidate responses from the policy model, score each with Fin-PRM, return only the highest-scoring response (or flag for human review if all scores fall below a threshold). The additional latency from PRM inference (running a 7B model on each candidate) may be acceptable for asynchronous or batch use cases where correctness is prioritized over speed. The paper's finding that Fin-PRM maintains competitive general evaluation on Math500 (Table 2) also suggests that the same deployed PRM can serve as a general-purpose output filter without requiring separate evaluators for different query types.

Process-aware reinforcement learning for financial model alignment. The GRPO results (Figure 3) demonstrate that incorporating Fin-PRM's process-level rewards improves RL-trained policy accuracy by 3.3 points on both CFLUE (to 70.5%) and FinQA (to 62.8%) compared to outcome-only RL. For organizations already using RLHF or GRPO to align their financial models, integrating a domain-specialized PRM as an auxiliary reward source is a drop-in improvement — the GRPO objective in Equation 16 accepts any reward function, and the composite reward in Equation 14 requires only a hyperparameter $\delta$ to balance outcome and process signals. The practical benefit is not just the 3.3-point accuracy gain but the nature of the improvement: process-level rewards provide feedback on how the model reasons, not just what it concludes. This should produce models whose reasoning is more robust, more factually grounded, and less prone to reward hacking because the reward signal penalizes reasoning that scores well superficially but is not supported by domain knowledge. The paper does not measure these qualitative improvements directly, but they are the logical consequence of training with rewards that explicitly check factual accuracy and knowledge coverage.

When to Prefer This Method

The paper positions Fin-PRM as preferable to general-purpose PRMs specifically when the target domain has high factual precision requirements and when structured expert knowledge exists that can be extracted into a verifiable knowledge base. This is not a claim that Fin-PRM should replace general PRMs universally — the Math500 results show Fin-PRM is merely competitive with, not superior to, math-specialized PRMs on general reasoning. The choice is domain-contingent:

• Prefer a domain-specialized, knowledge-verified PRM like Fin-PRM when: (1) the domain has high factual stakes where incorrect reasoning carries direct costs (finance, medicine, law, engineering safety); (2) expert-written reference materials exist from which a knowledge base can be extracted; (3) the policy model's outputs are noisy enough that general PRMs risk anti-correlation with correctness (as demonstrated by Math-PRM falling below majority voting on CFLUE); and (4) the deployment setting allows the additional inference cost of running a 7B PRM on candidate outputs (acceptable for offline data curation, batch processing, or asynchronous applications).

• A general-purpose PRM may be sufficient when: (1) the domain's reasoning patterns closely match the PRM's training distribution (e.g., using a math PRM for mathematical reasoning); (2) the cost of knowledge base construction and maintenance outweighs the expected accuracy gain; (3) the deployment requires minimal latency and the PRM inference overhead is prohibitive; or (4) the base model's pass@1 is already high enough that process supervision provides diminishing returns (the Math500 results in Table 2 show that even specialized PRMs provide only marginal gains over majority voting when the policy model is already strong).

The paper does not explicitly articulate this tradeoff matrix, but the logic follows directly from the experimental results: the value of domain specialization is proportional to the domain's distance from the general PRM's training distribution and the cost of factual errors in that domain.