PhyCritic: Multimodal Critic Models for Physical AI

🎯 Pitch

Existing multimodal judge models fail on physical AI tasks because they lack internal physics-aware reasoning—PhyCritic fixes this by forcing the model to first predict the answer itself before judging others, surging 12–17 points ahead of open-source baselines. The critic’s own “self-prediction” acts as an anchor, making judgments on robotics and driving scenarios dramatically more stable and physically correct.

1. Executive Summary

This paper introduces PhyCritic, a multimodal critic model optimized for physical AI evaluation tasks—perception, causal reasoning, and planning—through a two-stage RLVR (Reinforcement Learning with Verifiable Rewards) pipeline that combines a physical skill warmup stage with a novel self-referential critic finetuning mechanism (where the critic first generates its own prediction for the question, then uses that self-prediction as an internal reference when judging candidate responses). Evaluated on PhyCritic-Bench, a newly constructed benchmark spanning robotics and autonomous driving scenarios, PhyCritic-7B achieves 68.0% pairwise judgment accuracy—outperforming all open-source 7B/8B baselines by margins of 12.0 to 16.9 points—while simultaneously improving physical reasoning as a policy model on CosmosReason1-Bench (63.9%), establishing that a critic model's judgment becomes significantly more stable and physically correct when grounded in the model's own physics-aware reasoning, though the gains manifest primarily when ground-truth answers are available to supply the self-prediction reward signal.

2. Context and Motivation

The Core Problem: Multimodal Evaluation Systems Cannot Judge Physical AI Outputs

The fundamental gap this paper addresses is deceptively simple: existing multimodal judge and critic models are trained for general visual domains (captioning, image QA, STEM reasoning) and fail when asked to evaluate outputs that involve physical perception, causal reasoning, or action planning. When one model generates a response about object affordances in a robotics video, or about the causal sequence of events in a manipulation task, or about the next action a driver should take—current judges cannot reliably determine whether that response is grounded in physical reality, consistent with visual evidence, and causally valid. They may confuse linguistically plausible but physically impossible reasoning with correct reasoning, or fail to detect subtle violations of spatial and temporal constraints.

This gap is not a niche concern. It sits at the intersection of three rapidly converging trends:

Physical AI is becoming a dominant application domain. As the paper's introduction notes, physical AI—which "involves visual perception, physical commonsense, spatial reasoning, and action-centric decision making"—is no longer confined to research labs. The paper references concrete systems spanning robotics (RoboVQA, BridgeData V2, HoloAssist, AgiBot World), autonomous driving (LingoQA, DriveLM, Reason2Drive), and embodied manipulation (RT-2, OpenVLA). These systems require evaluation at scale: training data quality depends on reliable reward signals, deployed systems need monitoring, and model developers need to compare architectures. An unreliable judge propagates errors through the entire development pipeline.

Critic models are becoming essential infrastructure for VLMs. As the paper makes clear in Section 1, the multimodal community has increasingly adopted a paradigm where models evaluate other models' outputs—providing pairwise preferences, numerical scores, and explanatory justifications. This is critical for:

Preference alignment: RLHF and RLAIF require reliable reward signals to align models with human expectations. If those signals misjudge physical reasoning, the aligned model inherits those blind spots.
Open-ended evaluation: Many physical AI outputs (action plans, causal explanations, affordance reasoning) cannot be evaluated by simple string matching against a ground-truth answer. A judge is necessary.
Automated benchmarking: As benchmarks grow, manual evaluation becomes intractable. Automated judges that correlate with human judgment are essential.

If the judges are unreliable for physical domains, the entire alignment and evaluation pipeline degrades—models become fine-tuned to optimize metrics that don't reflect physical correctness, and the community loses the ability to measure progress.

RLVR and GRPO are producing strong policy models but haven't been transferred to critics for physical domains. The paper explicitly notes in Section 1 that "recent advances in reinforcement-finetuned multimodal policy models show that RLVR-style verifiable rewards... and physically grounded reasoning datasets... can significantly improve multimodal reasoning and temporal consistency." Models like Cosmos-Reason1 demonstrate that physically-oriented RL training can improve a model's own reasoning. But these insights have not been applied to critic training—the task of evaluating other models' reasoning. The capabilities that make a model a good policy solver (physical perception, causal reasoning) don't automatically make it a good judge (consistent evaluation, grounded justification, resistance to superficial cues).

Where Prior Approaches Fall Short

The paper identifies three specific limitations of existing critic and judge models:

1. They lack physics awareness. General-domain judges (e.g., Prometheus-Vision, LLaVA-Critic, UnifiedReward-Think) are trained on data from captioning, VQA, and STEM domains. When presented with a physical AI problem—say, a video of a robot arm with a question about what it should do next—these judges may produce evaluations that are linguistically fluent but physically incoherent. The paper's Figure 1 provides a concrete example: the question describes an oven-opening scenario where proper causal sequence matters. A naive judge might not recognize that "open the oven before pulling out the tray" is the correct causal ordering, while PhyCritic's self-referential process catches this. The judge needs to understand the physical world, not just match response patterns.

2. Training data is confined to broad multimodal evaluation domains. The paper's related work section (Section 2) surveys the landscape: datasets like VL-RewardBench and MultimodalRewardBench focus on general visual understanding, hallucination detection, and image reasoning. None systematically cover physically grounded scenarios involving manipulation, affordance reasoning, 3D spatial understanding, or embodied interaction. A model trained on "is this caption accurate?" or "does this reasoning contain a hallucination?" has never encountered the judgment challenges unique to physical AI: distinguishing whether a visual explanation adheres to actual 3D configurations, assessing whether an action plan respects temporal and dynamical constraints, or evaluating whether reasoning about physical causality is valid.

3. They do not ground their decisions in their own physical understanding. This is the paper's most distinctive critique. In standard critic training, the judge is presented with a question and candidate responses, and it produces an evaluation. It is not required to first solve the problem itself. The paper argues—and provides empirical evidence for—that this leads to "inconsistent or superficial verdicts." A judge may pick the response that sounds more authoritative, uses more technical terminology, or has a more fluent structure, without recognizing that it contains a subtle physical error. Conversely, it may penalize a correct but terse response because it seems less detailed. The core intuition, illustrated in Figure 1, is that "a strong physical critic should behave like an expert human judge: before evaluating other models' responses, it should first solve the problem itself."

Attempts at Physical AI Critics: Limited and Narrow

The paper acknowledges two prior efforts that have begun to bridge the gap but are insufficient:

DriveCritic (Song et al., 2025) applies LMM-based critics to evaluate trajectory pairs in autonomous driving. This is the closest prior work—it explicitly targets a physical domain with safety-critical evaluation needs. However, the paper notes its scope "remains [limited] and does not apply to diverse, grounded multimodal text responses." DriveCritic focuses on trajectory comparison—a relatively structured judgment task—not on evaluating open-ended physical reasoning, affordance explanation, or multi-step planning in text form. PhyCritic "broadens the critic capabilities... toward general physical AI domains."

WorldModelBench (Li et al., 2025a) evaluates video-generation models on instruction following and physics adherence. While it "underscores the need for physics-aware multimodal evaluation," it evaluates generated videos—not model responses to physical reasoning questions. The physics awareness it assesses is in the output medium (does the video obey gravity?), not in the reasoning process of a language model.

Neither of these systems provides a general-purpose physics-aware judge that can evaluate natural language responses about perception, causation, and action across diverse embodied scenarios. The paper positions PhyCritic to fill exactly this gap.

The Missing Mechanism: Self-Referential Grounding

The paper's most important conceptual contribution—beyond the training pipeline—is the identification of self-referential grounding as the mechanism that makes judgments physically correct and stable. The intuition, drawn from the literature on human critical thinking (the paper cites Lai, 2011), is that an expert evaluator doesn't just read responses and rate them—they first form their own understanding of the problem, then use that understanding as a benchmark against which to assess others' answers.

Without this, a judge is reduced to surface-level comparison between two responses. If both responses agree on a physically incorrect claim, the judge may still declare a winner based on fluency or detail. If one response is correct but awkwardly phrased, and the other is wrong but eloquently argued, the judge may pick the wrong one. The self-referential mechanism breaks this degeneracy: the judge first solves the problem, then asks "which response aligns with my understanding of what the correct answer should be?"

The paper's innovation is to operationalize this in a two-stage RLVR training framework where:

Stage 1 builds the model's capacity to solve physical problems accurately (via standard GRPO with accuracy rewards).
Stage 2 teaches the model to generate its own prediction, then reference it when judging (via a combined reward that incentivizes both correct self-prediction and correct judgment).

This is not a simple prompt engineering trick—it requires that during training, the critic be explicitly rewarded for getting the self-prediction right (the $r_{\mathrm{sp}}$ term) in addition to getting the judgment right ( $r_{\mathrm{crit}}$ ). The ablation in Table 5 (Section 5.2) confirms this: removing the self-referential process drops PhyCritic-Bench performance from 68.0 to 64.4 ( $-3.6$ ), and removing the self-prediction reward drops it to 65.8 ( $-2.2$ ). The mechanism genuinely drives the gains.

How This Differs from Prior Judge Training Paradigms

The paper distinguishes itself from two existing paradigms for training multimodal judge models (Section 2):

BT-style reward modeling (Sun et al., 2023; Zang et al., 2025; Zhang et al., 2025d) replaces the language model head with a scalar prediction head that outputs a preference score. This approach produces a reward score but no explanation. The judge is a black box. In physical domains where errors are subtle and auditability matters, this opacity is problematic—you cannot tell why the judge preferred one response to another, making it hard to debug or improve.

Generative judges (Lee et al., 2024; Xiong et al., 2025b; Zhang et al., 2025c; Wang et al., 2025a) produce textual reasoning and verdicts autoregressively. This is PhyCritic's paradigm, and it is more suitable for physical AI because the reasoning trace can be inspected. However, these methods train the judge directly on preference pairs without requiring self-prediction. The judge learns to compare outputs, not to ground judgments in its own understanding. Later work incorporated RL (Zhang et al., 2025c; Wang et al., 2025a) to improve accuracy, but the training signal remains purely the judgment correctness—not the internal consistency between self-prediction and judgment.

PhyCritic adds the self-referential dimension: during RL fine-tuning, the reward $r_{\mathrm{total}} = \alpha_{\mathrm{sp}} r_{\mathrm{sp}} + \alpha_{\mathrm{crit}} r_{\mathrm{crit}} + \alpha_{\mathrm{form}} r_{\mathrm{form}}$ includes a term ( $r_{\mathrm{sp}}$ ) that explicitly rewards the model for getting its own answer right. This creates a joint optimization: the model must become a better problem solver in order to become a better judge, because the judgment reward alone can be gamed by superficial pattern matching. The paper's chi-square analysis (Section 5.3) provides evidence that this coupling works: the correlation between self-prediction correctness and judgment correctness strengthens significantly after Stage 2 training ( $\chi^2$ increases from 51.07 to 161.76), indicating that the model is learning to use its own correct understanding as the basis for its judgments.

Why the Problem Requires a New Approach

At a higher level, the paper is arguing that physical AI evaluation is fundamentally harder than general-domain multimodal evaluation, in ways that require a different training methodology:

Correctness is physically grounded, not linguistically grounded. In a captioning evaluation task, a judge can often assess quality by examining linguistic fluency, visual-textual alignment, and completeness. In physical AI, a response can be perfectly eloquent, well-structured, and visually grounded—yet physically wrong because it misunderstands causality, ignores dynamical constraints, or proposes an impossible action sequence. The judge needs to reason physically, not just match patterns. The self-referential mechanism ensures the judge actually does that reasoning.

Ground truth is verifiable but requires physical understanding to use. Many physical AI tasks have verifiable correct answers (the paper uses accuracy-based ground truth for preference labels), but arriving at those answers requires the model to interpret video, reason about spatial relationships, and infer causal sequences. The Stage 1 warmup explicitly targets this capacity before the judge is asked to evaluate others.

Training data for physical critics is scarce. General judge training leverages large-scale preference datasets from general visual domains. Physical AI datasets with paired response preferences are rare. The paper constructs its own training dataset (Section 3.4) by collecting responses from seven models on 800 Cosmos-Reason1 questions and using GPT-4o to label correct/incorrect answers, yielding only 3,258 training samples. The two-stage pipeline's data efficiency (4,058 total training samples, 380 RL steps) is critical—a data-hungry approach wouldn't work.

Judgment consistency requires explicit structure. The ablation on critic prompts (Table 7) shows that removing detailed evaluation criteria (truthfulness, visual groundedness, logical validity, efficiency) from the prompt drops PhyCritic-Bench from 68.0 to 63.9. The paper notes this "contrasts with prior work such as Wang et al. (2025a), which uses a simple pairwise critic prompt for general-domain critic training." Physical AI judging requires more explicit guidance to shape the judge's reasoning, and the self-referential structure provides scaffolding for that guidance.

Summary of the Gap

What the community has: General-domain multimodal judges trained on captioning/VQA/STEM data (Prometheus-Vision, LLaVA-Critic, UnifiedReward-Think), plus domain-specific evaluation systems for narrow applications (DriveCritic for driving trajectories, WorldModelBench for video generation physics adherence).
What the community lacks: A general-purpose, physics-aware critic that can evaluate open-ended physical reasoning, perception, and planning responses across diverse embodied domains, with judgments that are grounded in the model's own physical understanding rather than superficial response comparison.
What the paper contributes to fill this gap: PhyCritic with its two-stage RLVR pipeline (physical skill warmup → self-referential critic finetuning), a training dataset of 3,258 physical-domain preference pairs, and PhyCritic-Bench (225 evaluation samples spanning robotics and autonomous driving) for systematic evaluation of physical AI critics.

3. Technical Approach

This is primarily a methods paper whose core idea is that a multimodal critic model for physical AI should generate its own answer to a question before evaluating candidate responses, grounding its judgment in that self-generated understanding through a two-stage RLVR (Reinforcement Learning from Verifiable Rewards) training pipeline.

3.1 Reader orientation

PhyCritic is a system that evaluates pairs of multimodal model responses to physical reasoning questions—questions about what a robot should do next, why an object behaves a certain way, or what action a driver should take. The core problem it solves is that existing judge models compare responses at a surface level without first understanding the problem themselves, leading to judgments that can be physically incorrect even when they sound reasonable. The solution takes the form of a two-stage training process: first teach the model to answer physical questions correctly (Stage 1: physical skill warmup), then teach it to generate its own correct answer as a reference before judging which of two candidate responses is better (Stage 2: self-referential critic finetuning).

3.2 Big-picture architecture (diagram in words)

The system has five major components arranged in a training pipeline:

Base Vision-Language Model (Qwen2.5-VL-7B-Instruct) — the starting point: a pretrained multimodal model that can process visual inputs (images, video frames) and text queries, and generate text responses. It has general multimodal capabilities but no specialized physical reasoning or critic training.
Physical QA Training Data (Stage 1 input) — a small set of (question, visual input, ground-truth answer) tuples drawn from the Cosmos-Reason1 RL dataset. These are problems where the correct answer is known and can be verified programmatically.
Critic Training Data (Stage 2 input) — tuples of the form $(Q, L_A, L_B, A_Q, P)$ where $Q$ is a multimodal prompt (question + visual input), $L_A$ and $L_B$ are two candidate responses generated by different models, $A_Q$ is the ground-truth answer to $Q$ , and $P \in \{A, B\}$ is a binary preference label indicating which response is higher quality. This data is constructed by the authors from embodied robotics datasets.
GRPO Optimizer — the reinforcement learning algorithm used in both stages. It takes the model's generated outputs, computes rewards (accuracy for Stage 1; combined self-prediction + critic + format rewards for Stage 2), derives advantages by comparing multiple sampled trajectories as a group, and updates the model parameters.
Self-Referential Critic Prompt Template (Table 1) — a structured prompt used during Stage 2 that instructs the model to: (a) think through the problem, (b) produce its own answer, then (c) evaluate both candidate responses with explicit reference to its own answer, all within a specific formatting structure enforced by the format reward.

Information flows as follows: the base model enters Stage 1 → it generates answers to physical QA problems → accuracy rewards are computed by string-matching against ground truth → GRPO updates the model to improve physical reasoning → the Stage-1 model enters Stage 2 → it receives critic tuples with the self-referential prompt → it generates long-form outputs containing both a self-prediction and a preference judgment → three reward components are computed ( $r_{\mathrm{sp}}$ for self-prediction accuracy, $r_{\mathrm{crit}}$ for judgment accuracy, $r_{\mathrm{form}}$ for format compliance) → a weighted sum reward drives GRPO updates → the final PhyCritic model emerges.

3.3 Roadmap for the deep dive

First, the task formulation (Section 3.1): I will define the exact data structure the critic operates on, establishing the notation used throughout the pipeline. This is the "interface" specification—what the model sees as input and what it must produce.
Second, Stage 1: Physical skill warmup (Section 3.2): I will walk through how the base model is first trained to answer physical questions correctly using standard GRPO with accuracy rewards, including the exact training data, reward definition, and number of steps. This stage builds the foundation that Stage 2 depends on.
Third, Stage 2: Self-referential critic finetuning (Sections 3.2–3.3): This is the paper's core technical contribution. I will explain: (a) the prompt template and its required output structure, (b) the three reward components and their weighting, (c) the GRPO optimization algorithm, and (d) the design rationale for why self-referential grounding improves critic reliability.
Fourth, the critic training dataset construction (Section 3.4): I will explain where the videos, questions, and candidate responses come from, how preference labels are assigned, and the final dataset size and composition. Understanding data provenance is essential because the critic's capabilities are bounded by the quality and diversity of its training pairs.
Fifth, the evaluation benchmark PhyCritic-Bench (Section 4): I will detail the construction of the held-out evaluation set, including its composition (robotics and autonomous driving subsets, 225 samples), the response generation and labeling procedure, and the evaluation metric. This is the yardstick against which the paper's claims are measured.
Sixth, hyperparameter summary and implementation details (Section 5): I will aggregate the key numbers—learning rates, batch sizes, reward weights, training steps, model sizes—into a reference table for reproducibility.

3.4 Detailed, sentence-based technical breakdown

Task Formulation: What the Critic Sees and Must Produce

The critic training data are organized as tuples $(Q, L_A, L_B, A_Q, P)$ , where:

$Q$ denotes a multimodal prompt consisting of a user question together with visual inputs (frames from a video);
$L_A$ and $L_B$ are two candidate responses to be evaluated—these are the outputs generated by other models (or the same model with different sampling) that the critic must judge;
$A_Q$ denotes the ground-truth answer to $Q$ —a verifiable correct answer that can be used to compute accuracy rewards and preference labels;
$P \in \{A, B\}$ is a binary preference label indicating which response demonstrates higher quality—for this paper, "higher quality" means "the response whose final answer matches $A_Q$ ," since only one response in each pair is correct.

At inference time, the critic receives $(Q, L_A, L_B)$ and must produce a judgment indicating which response is better, along with a reasoning trace explaining why. During Stage 2 training, the critic is additionally required (via the prompt template and reward structure) to first generate its own answer $\hat{A}_{\mathrm{pred}}$ to $Q$ before evaluating the pair—this self-prediction serves as the internal reference against which candidate responses are assessed.

The paper formulates the critic's task as pairwise preference prediction: given $(Q, L_A, L_B)$ , output $\hat{P}_{\mathrm{pred}} \in \{A, B\}$ such that $\hat{P}_{\mathrm{pred}}$ matches the ground-truth $P$ . Accuracy is measured as:

$\text{Acc} = \mathbb{I}(\text{VLM}(q, l_a, l_b) = p)$

where $\mathbb{I}$ is the indicator function (returns 1 if the condition is true, 0 otherwise), and $\text{VLM}(q, l_a, l_b)$ is the model's predicted preference when shown the multimodal prompt $q$ and the two candidate responses $l_a$ and $l_b$ .

What it computes: the fraction of test instances where the critic's preferred response matches the ground-truth preference label. Each test instance gets a score of 1 if the critic selects the correct response (the one whose answer matches $A_Q$ ), and 0 otherwise. The metric is then averaged over all test instances.

Why this form: Pairwise accuracy is the standard metric in the judge-model literature (used by JudgeBench, VL-RewardBench, and Multimodal RewardBench). It is simple, interpretable, and directly measures whether the critic can distinguish higher-quality from lower-quality responses. The binary formulation avoids the calibration issues of scalar reward prediction and aligns with the training objective (the critic reward $r_{\mathrm{crit}}$ uses the same indicator function).

Stage 1: Physical Skill Warmup with RLVR

Before the model can serve as a reliable critic, it must possess solid foundational perception and reasoning capabilities within the physical domain. Stage 1 addresses this by training the base model (Qwen2.5-VL-7B-Instruct) on a verifiable QA dataset using standard GRPO with an accuracy-only reward.

Training data. The paper uses the Cosmos-Reason1 RL dataset (Azzolini et al., 2025), which provides 800 high-quality question–answer pairs requiring physical perception, planning, and reasoning. Each sample is a tuple $(Q, A_Q)$ —a multimodal prompt with its ground-truth answer—without any candidate responses or preference labels. This is the same dataset used by the Cosmos-Reason1-7B baseline model, but crucially PhyCritic uses only the RL portion (not the millions of supervised fine-tuning distillation traces that Cosmos-Reason1 uses), making the approach more data-efficient.

Reward definition. The reward is a simple binary indicator of answer correctness:

$r = \mathbb{I}(\hat{A}_{\mathrm{pred}}(Q) = A_Q)$

where $\hat{A}_{\mathrm{pred}}(Q)$ is the answer the model generates for prompt $Q$ , and $A_Q$ is the ground-truth answer.

What it computes: a binary reward—1 if the model's generated answer matches the ground-truth answer exactly, 0 otherwise. The comparison is a string match after parsing the model's output to extract the final answer (presumably enclosed in \boxed{} or a designated format following DeepSeek-R1 conventions).

Why this form: For physical QA tasks with verifiable answers (e.g., multiple-choice questions about what action to take next, yes/no questions about whether an object is present), accuracy is the natural reward signal. It is cheap to compute (no human labeling needed), unambiguous (the answer is either correct or not), and directly incentivizes the behavior we want: the model producing physically correct answers. The indicator function provides a sparse but clean signal—models must learn to produce exactly the right answer, not an approximately correct one.

Training configuration. The model is trained using vanilla GRPO on the Cosmos-Reason1-RL dataset for 80 steps, with a batch size of 128, a learning rate of $1 \times 10^{-6}$ , and a KL coefficient of 0.01. The veRL framework is used for RL finetuning. The base model is Qwen2.5-VL-7B-Instruct (Bai et al., 2025), a 7-billion-parameter vision-language model.

Design rationale. Why warm up the model with QA before introducing the critic task? The paper identifies two reasons. First, existing VLMs are "primarily trained on general image–text pairs with limited exposure to physically grounded data"—the base model may not reliably understand physical scenarios, making it a poor judge regardless of how well the critic mechanism works. Second, the self-referential mechanism in Stage 2 requires the model to generate a correct self-prediction; if the model cannot solve the problems itself, the self-prediction reward $r_{\mathrm{sp}}$ will be near zero for most samples, providing no useful training signal. Stage 1 ensures the model enters Stage 2 with non-trivial physical reasoning capability.

The ablation in Table 4 confirms this design choice: Stage 1 alone improves CosmosReason1-Bench accuracy from 54.3 (base Qwen2.5-VL-7B) to 61.8 (+7.5 points), but only marginally improves PhyCritic-Bench judgment accuracy from 51.6 to 53.6 (+2.0 points). Physical reasoning ability is necessary but not sufficient for good criticism—it requires Stage 2's explicit critic training to translate reasoning into judgment.

Stage 2: Self-Referential Critic Finetuning

The Prompt Template (Table 1)

The critic prompt is the structural backbone of Stage 2 and the mechanism by which self-referential behavior is enforced. The paper presents it in Table 1 with a full template (the exact text is shown in the paper's figure). The key structural elements are:

System instruction: The model is told it is a rigorous evaluator that must assess candidate responses according to specific criteria (truthfulness, visual groundedness, logical validity, and efficiency/clarity for reasoning assessments).
Self-prediction requirement: The prompt explicitly instructs the model to first generate its own reasoning and prediction for the given question, enclosed in <pred_think> and <pred> tags.
Critic requirement: After the self-prediction, the model must evaluate both candidate responses with explicit reference to its self-prediction, providing a detailed comparison and final verdict.
Format prompt: Following DeepSeek-R1 conventions (Guo et al., 2025), the prompt ends with a format specification that standardizes the model's reasoning behavior and output structure. The format requires specific tags including thinking (for the critic reasoning) and \boxed{} (for the final verdict).

The complete prompt is presented to the model during training for every critic sample. At inference time, the same prompt is used (the self-prediction is generated dynamically per query, not stored or cached).

Why this structure? The separation of <pred_think> and the critic thinking into distinct sections forces the model to complete its own problem-solving before evaluating others. This is enforced not just by the prompt instruction but by the reward design—the format reward explicitly requires the <pred_think> and <pred> tags to be present (contributing 1.0 to the format reward if all tags exist, 0.5 if only the critic thinking and boxed answer exist, 0 otherwise). The structure prevents the model from "cheating" by doing the critic evaluation first and retroactively constructing a self-prediction that matches—the output order is enforced by the format reward.

Reward Design

The total reward $r_{\mathrm{total}}$ during Stage 2 is a weighted sum of three components:

$r_{\mathrm{total}} = r_{\mathrm{acc}} + r_{\mathrm{form}} \cdot \alpha_{\mathrm{form}}$

where $r_{\mathrm{acc}}$ is the accuracy reward (itself a weighted sum of two sub-components), $r_{\mathrm{form}}$ is the format reward, and $\alpha_{\mathrm{form}} = 0.1$ is a weighting coefficient for the format reward.

What it computes: a scalar reward value for each generated trajectory (a complete model output containing both self-prediction and critic judgment). The reward is computed by extracting the self-prediction answer, the critic preference verdict, and the format tags from the model's raw text output, then checking each against the ground truth or format specification.

Why this form: The weighted sum allows the training to balance three competing objectives: self-prediction accuracy (the model should be good at solving problems), judgment accuracy (the model should be good at evaluating others), and format compliance (the outputs should be parseable and well-structured). Without all three components, the model might optimize for one at the expense of others—for example, producing correct judgments with no interpretable reasoning trace, or producing well-formatted but physically incorrect outputs.

Accuracy Reward ( $r_{\mathrm{acc}}$ ). The accuracy reward itself has two components:

$r_{\mathrm{acc}} = \alpha_{\mathrm{sp}} r_{\mathrm{sp}} + \alpha_{\mathrm{crit}} r_{\mathrm{crit}}$

where $\alpha_{\mathrm{sp}} = 0.2$ and $\alpha_{\mathrm{crit}} = 0.7$ (from the paper's final configuration; the ablation in Table 6 shows $\alpha_{\mathrm{sp}}$ is swept across $\{0.1, 0.2, 0.3, 0.4\}$ with $\alpha_{\mathrm{crit}} = 0.9 - \alpha_{\mathrm{sp}}$ ).

What it computes: a weighted accuracy score between 0 and 0.9 (since the weights sum to 0.9 in the final configuration, with the remaining 0.1 going to format). The reward is 0.9 if both self-prediction and critic judgment are correct; 0.7 if only the judgment is correct; 0.2 if only the self-prediction is correct; and 0 if neither is correct.

Why these weights: The choice of $\alpha_{\mathrm{crit}} = 0.7$ and $\alpha_{\mathrm{sp}} = 0.2$ reflects the relative importance of the two tasks: the primary objective is to produce an accurate critic (hence the heavier weight on $r_{\mathrm{crit}}$ ), but the self-prediction component must be non-zero to incentivize the model to actually solve the problem before judging (the ablation in Table 5 shows that removing $\alpha_{\mathrm{sp}}$ entirely drops performance by 2.2 points on PhyCritic-Bench). The weights were determined empirically: the ablation in Table 6 evaluates $\alpha_{\mathrm{sp}} \in \{0.1, 0.2, 0.3, 0.4\}$ and finds $\alpha_{\mathrm{sp}} = 0.2$ provides the best balance across judgment and reasoning benchmarks.

Self-Prediction Reward ( $r_{\mathrm{sp}}$ ):

$r_{\mathrm{sp}} = \mathbb{I}(\hat{A}_{\mathrm{pred}} = A_Q)$

where $\hat{A}_{\mathrm{pred}}$ is the model's self-generated answer extracted from the <pred> section of its output, and $A_Q$ is the ground-truth answer.

What it computes: a binary reward—1 if the model's self-prediction matches the ground-truth answer, 0 otherwise. This is the same accuracy-based reward used in Stage 1, but now it operates within the larger critic output.

Why this form: The indicator function provides a clean, verifiable signal that the model has correctly understood the problem before judging. It is the mechanism that makes the self-referential process faithful: the model cannot simply claim to have a certain understanding—it must actually produce the correct answer to receive this reward component. Without $r_{\mathrm{sp}}$ , the model could write a plausible-sounding but incorrect self-prediction and still receive full credit if its judgment happens to be right, potentially reinforcing a pattern where the model's expressed reasoning is decoupled from its actual judgment process.

Critic Reward ( $r_{\mathrm{crit}}$ ):

$r_{\mathrm{crit}} = \mathbb{I}(\hat{P}_{\mathrm{crit}}(Q, L_A, L_B) = P)$

where $\hat{P}_{\mathrm{crit}}(Q, L_A, L_B)$ is the model's predicted preference between responses A and B, and $P$ is the ground-truth preference label.

What it computes: a binary reward—1 if the critic correctly identifies which response is higher quality (i.e., which response's answer matches $A_Q$ ), 0 otherwise. The comparison is a string match between the model's final verdict and the ground-truth label.

Why this form: This is the primary objective of the critic—distinguishing correct from incorrect responses. The indicator function is appropriate because preference in this dataset is binary and deterministic (one response is correct, the other is not). Unlike typical RLHF preference data where both responses might be reasonable but one is slightly preferred, the physical QA setting provides an unambiguous correctness signal: a response is "better" if and only if it arrives at the correct final answer.

Format Reward ( $r_{\mathrm{form}}$ ):

$r_{\mathrm{form}} = \begin{cases} 1.0 & \text{if } \texttt{<pred\_think>}, \texttt{<pred>}, \texttt{ thinking} \text{ and } \texttt{\textbackslash boxed\{\}} \text{ exist,} \\ 0.5 & \text{if only } \texttt{ thinking} \text{ and } \texttt{\textbackslash boxed\{\}} \text{ exist,} \\ 0 & \text{otherwise.} \end{cases}$

What it computes: a three-level reward based on whether the model's output contains the required formatting tags. A score of 1.0 requires all four elements: <pred_think> section, <pred> section, thinking section, and a \boxed{} final answer. A score of 0.5 is given if only the critic-related tags (thinking and \boxed{}) are present but the self-prediction tags are missing. A score of 0 is given for any other format.

Why this discrete structure: The format reward serves two purposes. First, it makes the output parseable—the training pipeline needs to extract $\hat{A}_{\mathrm{pred}}$ and $\hat{P}_{\mathrm{crit}}$ from specific tagged sections to compute $r_{\mathrm{sp}}$ and $r_{\mathrm{crit}}$ . Without format enforcement, these extractions would fail, breaking the reward computation. Second, the three-tier structure creates a gradient toward the full self-referential format: the model gets partial credit (0.5) for producing a well-structured critic output even if it omits the self-prediction, but full credit (1.0) requires the complete self-referential structure. This shapes the model's behavior toward the desired output pattern without being so punitive that early training steps collapse.

Optimization Algorithm: Group Relative Policy Optimization (GRPO)

The paper employs GRPO (Shao et al., 2024), a policy-gradient method that was originally introduced for mathematical reasoning and later adopted by DeepSeek-R1 (Guo et al., 2025). GRPO modifies the standard PPO (Proximal Policy Optimization) objective by eliminating the learned value network and instead computing advantages by comparing multiple sampled trajectories as a group.

The GRPO objective is:

$\mathcal{L}_{\text{GRPO}} = \mathbb{E}_{o \sim \pi_{\theta}}\left[\min(\rho_o A_o, \text{clip}(\rho_o, 1 - \epsilon, 1 + \epsilon) A_o)\right] - \beta \mathcal{D}_{\mathrm{KL}}(\pi_{\theta} \| \pi_{\mathrm{ref}})$

where:

$\pi_{\theta}$ is the current policy (the model being trained),
$\pi_{\mathrm{ref}}$ is the reference policy (the model before the current RL update, used for KL regularization),
$o$ is a sampled trajectory (a complete model output for one training instance),
$\rho_o = \frac{\pi_{\theta}(o)}{\pi_{\mathrm{ref}}(o)}$ is the probability ratio—the ratio of the current policy's likelihood of generating trajectory $o$ to the reference policy's likelihood,
$A_o = \frac{r_o - \bar{r}}{\text{std}(r)}$ is the group-relative advantage, computed by normalizing each outcome reward $r_o$ (the total reward $r_{\mathrm{total}}$ for trajectory $o$ ) by subtracting the mean reward $\bar{r}$ across all trajectories in the same group and dividing by the standard deviation $\text{std}(r)$ of the rewards in the group,
$\epsilon$ is the clipping parameter (controls how far the probability ratio can deviate from 1 before the gradient is clipped; typical value is 0.2),
$\beta = 0.01$ is the KL penalty coefficient (from the paper's implementation details),
$\mathcal{D}_{\mathrm{KL}}$ is the KL divergence between the current and reference policies.

What it computes: a scalar loss that is minimized via stochastic gradient descent. The loss has two terms. The first term (the expectation over $\min(\rho_o A_o, \text{clip}(\dots))$ ) is a clipped surrogate objective: when the advantage $A_o$ is positive, the objective encourages increasing $\rho_o$ (making the trajectory more likely), but clips the gradient if $\rho_o$ exceeds $1 + \epsilon$ to prevent overly aggressive updates. When $A_o$ is negative, the objective encourages decreasing $\rho_o$ , but clips if $\rho_o$ falls below $1 - \epsilon$ . The second term ( $-\beta \mathcal{D}_{\mathrm{KL}}$ ) penalizes the policy for deviating too far from the reference policy, preventing catastrophic forgetting of the base model's general capabilities.

Why this form—and why GRPO over PPO?

Group-relative advantage vs. learned value function: In standard PPO, the advantage $A_o$ is computed as $r_o - V(s)$ , where $V(s)$ is a separately learned value network that estimates expected future reward from state $s$ . GRPO replaces $V(s)$ with the group mean $\bar{r}$ and divides by $\text{std}(r)$ . This eliminates the need to train a separate value network (which would double the model size and training complexity), and the group-based normalization provides a natural baseline: a trajectory is "good" if it performs better than other trajectories sampled for the same input under the same policy. This is particularly appropriate for the critic setting because the rewards are bounded between 0 and 1, making normalization straightforward.
Clipped surrogate objective: The clipping prevents the policy from changing too much in a single update. Without clipping, a trajectory with a very high advantage could cause an extremely large gradient update, potentially destabilizing training. The $\min$ operation ensures that the gradient is zero (or clipped) when the probability ratio moves too far from 1 in the favorable direction, preventing the policy from overfitting to a small number of high-reward trajectories.
KL regularization to reference policy: The $\beta \mathcal{D}_{\mathrm{KL}}$ term is critical in the critic setting because the model is being trained on a relatively small dataset (3,258 samples) for a task that requires maintaining general multimodal reasoning capabilities. Without KL regularization, the policy might collapse to a degenerate strategy that maximizes reward on the training distribution but loses the base model's visual understanding and language fluency—a form of reward hacking at the capability level.

Training configuration for Stage 2. Both GRPO stages use the veRL framework, a batch size of 128, a learning rate of $1 \times 10^{-6}$ , and a KL coefficient $\beta = 0.01$ . Stage 2 runs for 300 steps on the critic training data (the 3,258-sample dataset described below). This is a remarkably small amount of training compared to typical RL fine-tuning runs, reflecting the data efficiency of the self-referential approach.

Critic Training Dataset Construction (Section 3.4)

The critic training dataset is constructed from scratch because no existing dataset provides the combination of physical-domain videos, high-quality reasoning questions, and paired model responses with verifiable preference labels that critic training requires.

Video sources. Videos are selected from four major robotics and embodied datasets:

RoboVQA (Sermanet et al., 2024): long-horizon reasoning videos for robotics, with manipulation tasks in simulated and real environments;
BridgeData V2 (Walke et al., 2023): a large-scale robot learning dataset with diverse manipulation behaviors (grasping, stacking, folding, assembling) from third-person camera views;
HoloAssist (Wang et al., 2023): an egocentric human interaction dataset capturing real-world task execution from a first-person perspective;
AgiBot World (Bu et al., 2025): a large-scale manipulation platform with scalable embodied data collection in diverse physical contexts from kitchens to workspaces.

The selection spans egocentric and third-person views, a wide range of manipulation behaviors, and diverse physical contexts—ensuring the critic encounters varied visual perspectives, action types, and environmental conditions.

Question source. Questions are drawn from the Cosmos-Reason1 RL dataset (Azzolini et al., 2025), which provides 800 high-quality question–answer pairs "requiring rich physical perception, planning, and reasoning capacities." These questions probe perception (e.g., "Is the red block on the table?"), causal reasoning (e.g., "Why did the robot drop the object?"), affordance understanding (e.g., "Can the gripper grasp the cup from this angle?"), and action planning (e.g., "What should the robot do next to complete the task?").

Response generation. Candidate responses are collected from seven multimodal models spanning different families:

Proprietary models: GPT-4o (Hurst et al., 2024) and Gemini-2.5-Flash (Comanici et al., 2025);
Open-source general VLMs: Qwen2.5-VL-72B (Bai et al., 2025) and InternVL3-38B (Zhu et al., 2025);
RL-finetuned systems with enhanced reasoning: Cosmos-Reason1-7B (Azzolini et al., 2025), Video-R1 (Feng et al., 2025), and MiMo-VL-7B (Team et al., 2025b).

All models are prompted in a Chain-of-Thought (CoT) manner to produce explicit reasoning traces alongside their final answers. This diversity of response sources ensures the critic encounters varied reasoning styles, accuracy levels, and error patterns during training.

Preference label construction. The ground-truth critic preference uses an accuracy-based labeling procedure: GPT-4o verifies each candidate response against the ground-truth answer $A_Q$ and assigns a binary score (1 = preferred, 0 = rejected). The key assumption is that a response is "preferred" if and only if its final answer matches the ground truth—this reduces the complex task of "which response is better" to the verifiable task of "which response is correct." Response pairs are then constructed, each consisting of one chosen (correct) and one rejected (incorrect) response. After balancing response lengths and distribution (to prevent the critic from learning spurious length-based heuristics), the final dataset contains 3,258 training samples.

Design rationale. The dataset construction makes several deliberate choices. Using seven diverse response sources prevents the critic from overfitting to the output patterns of any single model. Using accuracy as the sole preference criterion eliminates the ambiguity of human preference annotation (where two responses might both be partially correct, making the "better" judgment subjective). The length balancing prevents the critic from learning that longer responses are better—a common failure mode in judge models trained on naturally-imbalanced data where correct models tend to produce longer answers.

The relatively small size (3,258 samples) reflects both the cost of construction (requires running seven models on 800 questions) and the paper's emphasis on data efficiency. Combined with the 800 QA samples from Stage 1, the total training data across both stages is 4,058 samples—several orders of magnitude less than the millions of samples used by general-domain judge models like LLaVA-Critic or UnifiedReward-Think.

PhyCritic-Bench: Evaluation Benchmark (Section 4)

To evaluate critic performance in physical domains, the paper introduces PhyCritic-Bench, a held-out benchmark distinct from the training data.

Composition. PhyCritic-Bench comprises 225 evaluation samples covering two physical-AI scenarios:

Robotics tasks: Videos from RoboVQA, BridgeData V2, HoloAssist, AgiBot World, and RoboFail (Liu et al., 2023), with questions adapted from CosmosReason1-Bench. RoboFail is specifically included to test the critic's ability to identify failure cases—videos where the robot fails at the task, requiring the critic to recognize that correct responses should identify the failure rather than describe a successful execution.
Autonomous driving: Videos and action- or prediction-related questions from LingoQA (Marcu et al., 2024). This domain tests the critic's ability to evaluate responses in safety-critical, temporally-extended scenarios with different perception requirements from manipulation tasks.

The distribution of prompt sources and model responses is visualized in Figure 3. Response models span the same seven-model pool used for training data generation.

Response pair construction. Following the JudgeBench pipeline (Tan et al., 2024), each prompt is assigned to one model from a pool, which generates $N = 8$ CoT responses via temperature sampling (with temperature $\tau = 0.6$ , creating diversity while maintaining basic coherence). GPT-4o then verifies each response against the ground-truth answer, identifying correct and incorrect predictions. The final pair for each evaluation instance comprises one correct and one incorrect response, with correctness serving as the ground-truth preference label. This ensures that every evaluation instance has a clear, objective ground truth—the critic should prefer the correct response over the incorrect one.

Evaluation metric. Each VLM acts as a pairwise judge: given the multimodal query $q$ and the two responses $l_a$ and $l_b$ , the model must output which response it prefers. Performance is measured by preference consistency with the ground truth:

$\text{Acc} = \mathbb{I}(\text{VLM}(q, l_a, l_b) = p)$

Why this benchmark is necessary. Existing multimodal reward benchmarks (VL-RewardBench, Multimodal RewardBench) cover general visual domains (captioning, STEM reasoning, image QA) but do not "cover physical AI domains that involve judgments related to physical perception, actions and planning." PhyCritic-Bench fills this gap with a targeted evaluation that directly measures whether a critic can distinguish physically correct from physically incorrect reasoning in embodied and driving contexts. The inclusion of RoboFail as a specific sub-suite is noteworthy: it tests whether the critic can recognize failure—a capability that requires understanding not just what the robot is doing but also what the correct outcome should be, making it a strong test of physical understanding rather than pattern matching.

Implementation Details Summary (Section 5)

Component	Configuration
Base model	Qwen2.5-VL-7B-Instruct
RL framework	veRL
Stage 1 steps	80
Stage 2 steps	300
Batch size (both stages)	128
Learning rate (both stages)	$1 \times 10^{-6}$
KL coefficient $\beta$	0.01
$\alpha_{\mathrm{sp}}$ (self-prediction weight)	0.2
$\alpha_{\mathrm{crit}}$ (critic weight)	0.7
$\alpha_{\mathrm{form}}$ (format weight)	0.1
Training samples (total)	4,058 (800 Stage 1 + 3,258 Stage 2)
Evaluation samples (PhyCritic-Bench)	225

Additional Analysis Components (Section 5.3 and Appendix A)

Best-of-N test-time scaling. The paper evaluates PhyCritic as a test-time judge for best-of-N sampling (Appendix A, Figure 4). Qwen2.5-VL-7B-Instruct generates $N$ candidate reasoning trajectories (with temperature $\tau = 0.6$ ) per question on CosmosReason1-Bench. PhyCritic-7B is applied through a pairwise knockout procedure: starting from the first two trajectories, PhyCritic judges the pair and advances the preferred response to compete with the next candidate. This continues for $N - 1$ rounds until a final winner is selected. This procedure tests whether PhyCritic can reliably identify the highest-quality trajectory among multiple candidates at test time, independent of its training objective.

Chi-square test for self-prediction-judgment coupling (Section 5.3). To quantify the relationship between self-prediction accuracy and judgment quality, the paper conducts chi-square tests of independence. For the Stage 1 model (physical RL warmup only), $\chi^2 = 51.07$ with $p = 8.93 \times 10^{-13}$ —a statistically significant association, but modest in strength. For the final PhyCritic model after Stage 2, $\chi^2 = 161.76$ with $p = 4.66 \times 10^{-37}$ —a much stronger association, indicating that self-referential critic finetuning substantially tightens the coupling between the model's own problem-solving ability and its judgment accuracy. This is the empirical evidence that the self-referential mechanism actually works as intended: the model learns to base its judgments on its own understanding, not on superficial response features.

Downstream policy training (Appendix A, Table 8). To validate PhyCritic as a reward signal for improving other models, the paper applies it within a DPO (Direct Preference Optimization) self-improvement framework. Starting from Qwen2.5-VL-7B-Instruct, 8 responses are generated per prompt on Cosmos-RL data. PhyCritic-7B scores all ordered response pairs and selects best–worst responses as preference pairs for DPO training (learning rate $1 \times 10^{-6}$ , 5 epochs). The resulting policy improves CosmosReason1-Bench accuracy from 54.3 to 60.0, substantially outperforming an answer-verifier baseline that simply pairs correct with incorrect responses (57.5). This demonstrates that PhyCritic provides useful reward signals beyond simple correctness matching, likely by selecting preference pairs where the "better" response has higher-quality reasoning even when both are correct, or where the "worse" response has subtle physical errors that a simple answer match would miss.

4. Key Insights and Innovations

Innovation 1: Self-Referential Grounding as a Mechanism for Critic Reliability

The paper's most conceptually distinctive contribution is the principle that a judge model should solve the problem before evaluating others' solutions, and that this self-referential process can be operationalized as a trainable behavior through reward design. This is not simply a prompting trick—it is a claim about what makes critic judgments trustworthy, backed by a training methodology that enforces the coupling between a model's own understanding and its evaluations.

What the field did before. Prior multimodal critic training—whether via BT-style reward modeling (Sun et al., 2023; Zang et al., 2025) or generative judge fine-tuning (Lee et al., 2024; Xiong et al., 2025b; Wang et al., 2025a)—treated the critic's task as a direct mapping from (question, response A, response B) to a preference. The critic learned to compare outputs, but there was no mechanism ensuring its internal understanding of the problem aligned with its judgment. A critic could learn to prefer well-structured but physically incorrect responses, or to penalize correct but terse ones, because the training signal only rewarded judgment accuracy—not the consistency between the critic's own problem-solving and its evaluation.

Why this is a conceptual advance, not just a methodological tweak. The paper identifies a failure mode specific to physical AI evaluation that general-domain critics don't face: in captioning or VQA, surface-level features (fluency, visual-textual alignment, keyword matching) are often sufficient correlates of quality. In physical reasoning, a response can be eloquent, well-structured, and visually grounded yet physically wrong—because it misunderstands causality, ignores dynamical constraints, or proposes an impossible action. A critic that judges by pattern matching will fail silently. The self-referential mechanism breaks this degeneracy by forcing the critic to form its own physically-grounded understanding first, then use that understanding as a benchmark. This reframes the critic's task from "which response looks better?" to "which response agrees with what I know to be correct?"

Evidence for the mechanism actually working. The chi-square analysis in Section 5.3 provides the cleanest evidence. After Stage 1 (physical RL warmup, no self-referential training), the association between self-prediction correctness and judgment correctness is statistically significant but modest ( $\chi^2 = 51.07$ , $p = 8.93 \times 10^{-13}$ ). After Stage 2 (self-referential critic finetuning), the association strengthens dramatically ( $\chi^2 = 161.76$ , $p = 4.66 \times 10^{-37}$ ). This is not a foregone conclusion—the critic could have learned to produce correct judgments by relying on superficial cues while its self-predictions remained unrelated to judgment quality. The strengthening association demonstrates that the model is genuinely learning to use its own correct understanding as the basis for its evaluations, not just getting better at both tasks independently.

The ablations in Table 5 reinforce this: removing the explicit self-referential process drops PhyCritic-Bench accuracy by 3.6 points (68.0 → 64.4), and removing only the self-prediction reward $r_{\mathrm{sp}}$ while keeping the self-referential prompt drops it by 2.2 points (68.0 → 65.8). The prompt alone provides some structure, but the reward signal is necessary to make the coupling robust.

Why this matters beyond the numbers. The self-referential mechanism addresses a fundamental trust problem in automated evaluation. When a critic produces a judgment without explaining its own understanding, there is no way to audit whether the judgment is grounded in physical reasoning or surface-level heuristics. The self-referential structure makes the critic's reasoning inspectable: one can check whether the self-prediction is correct and whether the judgment follows from it. For safety-critical physical AI applications (autonomous driving, medical robotics), this auditability is not optional—it is essential for determining whether to trust the critic's output. The paper's qualitative examples (Appendix B, Table 10) illustrate this concretely: the base Qwen2.5-VL model produces a superficially structured evaluation that misses the key physical insight (the truck's stationary status and lane position), while PhyCritic first identifies "the oven is closed" or "the truck is not obstructing the right lane" in its self-prediction, then uses that insight to ground its judgment. The difference is not just accuracy—it's that PhyCritic's judgment comes with a verifiable chain of physical reasoning.

This is a fundamental conceptual contribution rather than an incremental improvement. It introduces a new design principle for critic models—self-referential grounding—that is independent of the specific architecture, base model, or domain, and it provides both a training methodology and empirical validation for that principle.

Innovation 2: Physical AI Evaluation as a Distinct Challenge Requiring Domain-Specific Critic Training

The paper makes a strong case—empirically and conceptually—that evaluating physical AI outputs is qualitatively different from evaluating general visual-language outputs, and that this difference demands purpose-built critics rather than adaptation of general-domain judges.

What the field assumed before. The dominant approach in multimodal judge training has been to build general-purpose critics trained on broad mixtures of captioning, VQA, and STEM reasoning data (Lee et al., 2024; Xiong et al., 2025b; Wang et al., 2025c, 2025a; Zhang et al., 2025c). The implicit assumption is that a critic trained to evaluate image captions or STEM problem solutions can generalize to evaluating physical reasoning about robot actions, object affordances, or driving decisions. PhyCritic challenges this assumption directly.

The empirical case for domain specificity. The comparison with UnifiedReward-Think (Appendix A, Table 9) is revealing. UnifiedReward-Think is a general-domain critic trained via RL on broad understanding and generation preference data, also initialized from Qwen2.5-VL-7B-Instruct. It performs strongly on general reward benchmarks (73.8 on VL-RewardBench vs. PhyCritic's 57.3) but shows only marginal gains on physical-domain judgment (52.4 on PhyCritic-Bench vs. the base model's 51.6). Meanwhile, PhyCritic achieves 68.0 on PhyCritic-Bench and 57.3 on VL-RewardBench. The asymmetry is striking: general-domain critic training does not transfer to physical evaluation, but physical critic training does transfer—at least partially—to general evaluation. This suggests that physical AI judging requires capabilities (causal reasoning, spatial understanding, temporal consistency assessment) that are a superset of what general judging requires.

What makes physical AI evaluation fundamentally harder. The paper identifies—implicitly through its prompt design ablation (Table 7)—that physical AI judging requires explicit, structured evaluation criteria. Removing the detailed criteria (truthfulness, visual groundedness, logical validity, efficiency) from the critic prompt drops PhyCritic-Bench from 68.0 to 63.9. The paper explicitly notes this "contrasts with prior work such as Wang et al. (2025a), which uses a simple pairwise critic prompt for general-domain critic training." A simple "which response is better?" prompt suffices for general domains where quality correlates with surface features, but for physical AI, the critic needs to be instructed to check specific dimensions of correctness—and needs the capability to actually perform those checks.

This is a diagnostic reframing: the paper is effectively arguing that the community's approach of building ever-larger generalist judges is insufficient for physical domains, and that the path forward involves domain-aware critic architectures (of which self-referential grounding is one instance) with explicit evaluation criteria tailored to the domain's correctness requirements. This is not an incremental finding—it challenges the dominant scaling assumption in the judge-model literature and suggests a research direction toward domain-specialized critics rather than one-size-fits-all evaluators.

Innovation 3: RLVR as a Data-Efficient Alternative to Supervised Fine-Tuning for Critic Training

The paper demonstrates that reinforcement learning with verifiable rewards can produce a strong critic model using orders of magnitude less data than supervised fine-tuning approaches, challenging the assumption that critic training requires large-scale preference datasets.

The data efficiency contrast. PhyCritic is trained on a total of 4,058 samples (800 Stage 1 QA pairs + 3,258 Stage 2 critic pairs) across 380 RL steps. In comparison, Cosmos-Reason1-7B—the closest physical-reasoning baseline—uses "millions of supervised finetuning traces distilled from DeepSeek-R1" followed by RL finetuning. General-domain critics like LLaVA-Critic (Xiong et al., 2025b) and Prometheus-Vision (Lee et al., 2024) are trained on datasets of tens to hundreds of thousands of preference pairs. Despite this data disparity, PhyCritic matches or exceeds Cosmos-Reason1-7B on physical reasoning (63.9 vs. 63.0 on CosmosReason1-Bench, Table 3) and dramatically exceeds it on critic judgment (68.0 vs. 51.1 on PhyCritic-Bench, Table 2).

Why RLVR achieves this efficiency. The paper doesn't theorize extensively about this, but the mechanism is implied by the reward structure. Supervised fine-tuning on preference pairs teaches the model to imitate correct judgments but provides no signal about why those judgments are correct. The model sees (question, response A, response B, label: A is better) and learns a statistical mapping. RLVR, by contrast, provides a dense, multi-component reward that decomposes the critic task into sub-skills: get the answer right ( $r_{\mathrm{sp}}$ ), make the right preference call ( $r_{\mathrm{crit}}$ ), and follow the reasoning structure ( $r_{\mathrm{form}}$ ). The GRPO advantage normalization further ensures that the model learns from relative performance within a batch, making efficient use of each training sample. The result is that a small number of high-quality, verifiable training instances—with carefully designed rewards—can substitute for massive supervised datasets.

This is a methodological contribution with practical implications. For any physical AI domain where ground-truth answers are verifiable (robotics task success, driving decision correctness, physical state classification), the PhyCritic training recipe provides a template for building domain-specific critics without the prohibitive cost of collecting large-scale human preference annotations or generating millions of synthetic labeled pairs. The paper's downstream DPO experiment (Table 8) further validates this: a critic trained on 4,058 samples can serve as an effective reward model for improving a policy model, outperforming a simple answer-verifier baseline. This closes the loop from critic training to policy improvement, demonstrating that data-efficient critic models are not just evaluation tools but can drive the alignment pipeline itself.

Innovation 4: The Two-Stage Pipeline as a Necessary Condition for Physical Critic Performance

The paper's ablation on training strategy (Table 4) reveals a non-obvious finding: physical reasoning skill and critic judgment capability are complementary but non-interchangeable, and attempting to learn both simultaneously is suboptimal. This has implications for how multi-capability RL systems should be designed.

The empirical pattern in Table 4. Starting from the base Qwen2.5-VL-7B (PhyCritic-Bench: 51.6, CosmosReason1-Bench: 54.3):

Physical RL only (Stage 1 alone): Physical reasoning jumps to 61.8 (+7.5), but critic judgment barely moves to 53.6 (+2.0). The model becomes a better problem-solver but not a substantially better judge.
Critic RL only (skip Stage 1, go directly to Stage 2 self-referential training): Critic judgment improves to 62.2 (+10.6), but physical reasoning drops to 57.1 (−4.7 relative to Stage 1 alone). The model learns to judge but at the cost of some physical reasoning capability.
Mixed RL (single-stage training with combined physical QA + critic rewards): Intermediate performance: 66.7 on PhyCritic-Bench and 60.2 on CosmosReason1-Bench. Better than either alone, but worse than the two-stage pipeline.
Two-stage (Stage 1 → Stage 2): Best on all metrics: 68.0 on PhyCritic-Bench and 63.9 on CosmosReason1-Bench.

What this reveals about capability acquisition. The two-stage pipeline is not merely "more training." It represents a curriculum learning insight: the model must first acquire physical reasoning as a stable foundation before it can productively learn to use that reasoning as a reference for judging others. Attempting to learn both simultaneously (the mixed RL variant) creates interference—the model is simultaneously optimizing for generating correct answers and evaluating others' answers, and the two objectives can conflict (e.g., the model might learn to produce a correct self-prediction but a lazy judgment, or a sophisticated judgment but a sloppy self-prediction, and the weighted reward doesn't distinguish these failure modes). The sequential structure prevents this by ensuring that the physical reasoning capability is consolidated before the self-referential mechanism is introduced.

This is a conceptual contribution to understanding multi-objective RL training. It suggests that when downstream tasks depend on upstream capabilities (here, critic judgment depends on physical understanding), sequential training with capability-specific rewards is more effective than joint optimization with a combined reward. This principle is likely to generalize beyond the specific critic setting—to any system where one capability (e.g., perception) is a prerequisite for another (e.g., reasoning about perceived objects). The paper doesn't explicitly frame this as a general principle, but the ablation data makes the case empirically, and it opens a research question about optimal curriculum design for multi-capability RL systems.

Innovation 5: The Critic-as-Policy Dual-Use Finding

Perhaps the paper's most surprising empirical result is that training a model to be a better critic simultaneously improves its capabilities as a policy model for physical reasoning tasks. This goes beyond the stated goal of building a judge—it reveals a bidirectional relationship between evaluation and generation that has implications for self-improving AI systems.

The evidence for dual-use capability. On CosmosReason1-Bench (Table 3), PhyCritic-7B achieves 63.9, which:

Exceeds its base model Qwen2.5-VL-7B (54.3, +9.6 points)
Exceeds the physical-reasoning specialist Cosmos-Reason1-7B (63.0, +0.9 points), despite Cosmos-Reason1 being trained on "millions of in-domain distilled reasoning traces"
Is competitive with proprietary models (GPT-4o: 56.3, Gemini-2.5-Flash: 57.9)

On CV-Bench 3D reasoning, PhyCritic achieves the best open-source 7B/8B score (83.9). On EgoPlanBench2, it ranks second overall (42.3) with top-two performance across all sub-domains.

What makes this surprising. The critic training objective (Stage 2) is designed to optimize judgment accuracy—the model learns to evaluate others' responses, not to generate better responses itself. The self-prediction reward ( $r_{\mathrm{sp}}$ ) incentivizes the model to produce a correct answer, but only as an intermediate step toward the critic judgment. Yet the reinforcement learning process appears to transfer back to the model's standalone problem-solving capability. The paper's explanation (Section 5.1) is that "critic training further pushes the model's problem-solving capacities," but the mechanism is not fully theorized.

A plausible interpretation: the self-referential process during Stage 2 exposes the model to diverse reasoning traces (from the seven candidate models in the training data) paired with correctness feedback. Even though the model is trained to judge these traces rather than generate them, the exposure to varied reasoning patterns—some correct, some incorrect—with explicit feedback about which are better likely serves as an implicit form of reasoning training. The model learns not just to recognize good reasoning but to internalize what makes reasoning physically correct, and this internalization benefits its own generation.

Why this matters for the broader AI landscape. This finding suggests a virtuous cycle: better critics enable better policy training (via DPO or RLHF), and the process of training critics also improves the critic model's own policy capabilities. This dual-use property could make critic training a more efficient route to general capability improvement than direct policy training—especially in domains like physical AI where high-quality supervised reasoning traces are scarce but verifiable ground-truth answers are available. The paper's DPO experiment (Table 8), where a policy model improved from 54.3 to 60.0 on CosmosReason1-Bench using PhyCritic-guided preference pairs, provides initial evidence that this virtuous cycle is practical.

This is a finding with implications beyond the paper's stated scope. It challenges the separation between "evaluator" and "solver" that underlies much of the RLHF and judge-model literature, and suggests that in domains requiring deep reasoning, the boundary between evaluation capability and generation capability is porous—improving one improves the other. If this finding generalizes, it has significant implications for how self-improving AI systems should be architected: training critic modules may be a more data-efficient path to capability improvement than direct behavioral cloning or policy optimization.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper introduces PhyCritic-Bench, a 225-sample pairwise preference benchmark covering physical AI scenarios, specifically robotics tasks (videos from RoboVQA, BridgeData V2, HoloAssist, AgiBot World, and RoboFail, with questions from CosmosReason1-Bench) and autonomous driving (videos and questions from LingoQA). This is held out from training data. The paper also evaluates on two established general-domain reward benchmarks: VL-RewardBench (Li et al., 2025b) and Multimodal RewardBench (Yasunaga et al., 2025), plus three physical reasoning benchmarks used to assess policy-model capabilities: CosmosReason1-Bench (Azzolini et al., 2025), CV-Bench (Tong et al., 2024), and EgoPlanBench2 (Qiu et al., 2024). The physical reasoning benchmarks are evaluated in a standard QA mode (not pairwise judgment)—the model answers questions directly and its final answer is compared to ground truth.
Base model. All PhyCritic variants are initialized from Qwen2.5-VL-7B-Instruct (Bai et al., 2025), a 7-billion-parameter vision-language model. The paper chooses this base model because it represents a standard open-source VLM scale (7B–8B parameters), making comparisons against other open-source models at similar scale fair, and because it has "limited exposure to physically grounded data" (Section 3.2), providing a meaningful test of whether the two-stage pipeline can inject physical AI capabilities from scratch.
Metrics. For critic evaluation on PhyCritic-Bench, VL-RewardBench, and Multimodal RewardBench, the metric is pairwise preference consistency with the ground-truth label: Acc = 1 if the critic's preferred response matches the ground-truth preference, 0 otherwise, averaged over all test instances (Section 4). For physical reasoning benchmarks (CosmosReason1-Bench, CV-Bench, EgoPlanBench2), the metric is accuracy—whether the model's generated answer matches the ground-truth correct answer, using whatever grading procedure is standard for each benchmark (Section 5, with references to the original benchmark papers for CV-Bench and EgoPlanBench2). For CosmosReason1-Bench, accuracy is reported overall and broken down by sub-task (AgiBot, HoloAssist, RoboVQA, Bridge, RoboFail).
Baselines. The paper compares against two groups (Section 5):
- General-purpose VLMs at 7B/8B scale: Qwen2.5-VL-7B (Bai et al., 2025, the base model itself, serving as a zero-shot critic and policy baseline) and Eagle-2.5-8B (Chen et al., 2025).
- Physical reasoning–oriented VLMs built on Qwen2.5-VL-7B: Cosmos-Reason1-7B (Azzolini et al., 2025), which is trained on millions of SFT reasoning traces distilled from DeepSeek-R1 followed by RL finetuning on object interaction and temporal consistency tasks; and RoboBrain2.0-7B (Team et al., 2025a), a perception-to-action model trained via multi-stage SFT and RL emphasizing task decomposition and subgoal generation.
- Proprietary frontier models are reported as upper-bound references: GPT-4o (Hurst et al., 2024) and Gemini-2.5-Pro/Flash (Comanici et al., 2025).
- One additional comparison appears in Appendix A (Table 9): UnifiedReward-Think-Qwen-7B (Wang et al., 2025c), a general-domain critic trained via RL on broad understanding and generation preference data, initialized from the same Qwen2.5-VL-7B-Instruct.
Generation budget / compute accounting. Compute is measured in RL training steps, not FLOPs. Stage 1 uses 80 GRPO steps on 800 QA samples (batch size 128); Stage 2 uses 300 GRPO steps on 3,258 critic samples (batch size 128). Total training is 380 RL steps on 4,058 total samples. The paper emphasizes this as a small compute budget compared to baselines like Cosmos-Reason1-7B (millions of SFT traces + RL). For the best-of-N test-time scaling experiment (Appendix A, Figure 4), compute is measured by $N$ , the number of candidate responses sampled, with PhyCritic applied through $N - 1$ pairwise comparisons in a knockout procedure. For the DPO experiment (Appendix A, Table 8), the policy model is trained for 5 epochs on PhyCritic-selected preference pairs.
Cross-validation / statistical protocol. The paper does not report cross-validation for hyperparameter selection or strategy optimization in the main results. The only statistical test reported is the chi-square test of independence between self-prediction correctness and judgment correctness (Section 5.3), with $\chi^2$ and $p$ -values reported for the Stage 1 model and the final PhyCritic model. All main results tables (Tables 2, 3) report single-run accuracy numbers without confidence intervals or standard deviations.

Main Quantitative Results

The experimental results fall into four categories: critic performance on physical and general benchmarks, policy-model performance on physical reasoning tasks, ablation studies on the training pipeline, and analysis of PhyCritic as a test-time and training-time reward signal.

Critic Performance: Physical AI vs. General Domains

The headline result for PhyCritic as a judge appears in Table 2: 68.0% pairwise accuracy on PhyCritic-Bench, the highest among all open-source 7B/8B models tested, with margins of +12.0 over Eagle-2.5-8B (56.0), +16.4 over the base Qwen2.5-VL-7B (51.6), +13.3 over RoboBrain2.0-7B (54.7), and +16.9 over Cosmos-R1-7B (51.1). The base model's 51.6% accuracy is barely above chance (50%), confirming that zero-shot physical AI judging is essentially random for an untuned general-purpose VLM.

The per-sub-suite breakdown reveals domain-specific patterns. PhyCritic achieves best or tied-best open-source performance on AgiBot (78.8), HoloAssist (65.5, tied with RoboBrain2.0), and RoboVQA (86.7), while ranking second on Bridge-V2 (65.6). Notably, on RoboFail (57.4) and LingoQA (60.0), PhyCritic's performance is tied with the base Qwen2.5-VL (57.4) and with the best open-source model (60.0) respectively—the gains on these subsets are modest or zero, suggesting the self-referential mechanism may not help as much when the visual evidence for failure or correct driving action is ambiguous. The proprietary models Gemini-2.5-Pro (78.2 overall) and Gemini-2.5-Flash (67.1 overall) still dominate, establishing a performance ceiling that open-source critics are far from reaching.

Generalization to general-domain benchmarks is tested on VL-RewardBench and Multimodal RewardBench (Table 2). PhyCritic-7B achieves 57.3 overall on VL-RewardBench (+4.1 over the base Qwen2.5-VL-7B's 53.2) and 65.9 on Multimodal RewardBench (+1.9 over the base model's 64.0), with gains observed across General, Hallucination, and Reasoning sub-categories. The comparison with UnifiedReward-Think (Appendix A, Table 9) is instructive: UnifiedReward-Think achieves 73.8 on VL-RewardBench vs. PhyCritic's 57.3—a 16.5-point gap favoring the general-domain critic—but scores only 52.4 on PhyCritic-Bench vs. PhyCritic's 68.0. This asymmetry confirms the paper's claim that physical and general critic capabilities are not interchangeable: general-domain critic training transfers poorly to physical evaluation, while physical critic training transfers partially to general evaluation.

Policy Performance: PhyCritic as a Physical Reasoner

Beyond judging, PhyCritic is evaluated as a policy model on three physical reasoning benchmarks (Table 3). On CosmosReason1-Bench, PhyCritic-7B achieves 63.9% overall, the best among open-source 7B/8B models:

Exceeds the base Qwen2.5-VL-7B (54.3, +9.6 points)
Exceeds Cosmos-R1-7B (63.0, +0.9 points) despite the latter's training on millions of SFT distillation traces
Exceeds Eagle-2.5-8B (54.3, +9.6) and RoboBrain2.0-7B (50.4, +13.5)

Sub-task performance shows strong results on HoloAssist (68.0), RoboVQA (95.5), and Bridge (63.0), but notably weak on AgiBot (53.0) and RoboFail (37.0). The RoboFail score (37.0) is particularly low—below even the base model's score on the critic version of the dataset (57.4 in Table 2)—suggesting that the policy-model evaluation of failure recognition is substantially harder than the critic evaluation of failure recognition, perhaps because generating a correct failure description is more demanding than recognizing that one candidate response correctly identifies the failure.

On CV-Bench, PhyCritic achieves 79.7 overall, second among open-source 7B/8B models behind RoboBrain2.0-7B (85.8), with the best 3D score of 83.9 among open-source models. The 3D improvement over the base model (82.7 → 83.9) is modest, but the direction is consistent with the claim that critic training improves spatial grounding. On EgoPlanBench2, PhyCritic scores 42.3 overall, second behind Eagle-2.5-8B (43.0), with top-two performance across all sub-domains (Daily: 45.6, Hobbies: 38.3, Recreation: 47.5, Work: 39.4). The consistent improvement over the base model (32.6, +9.7 points) indicates that the physical reasoning gains transfer to egocentric planning tasks not seen during training.

Best-of-N Scaling with PhyCritic as Judge

In Appendix A (Figure 4), PhyCritic is applied as a test-time judge for best-of-N ensemble selection. Qwen2.5-VL-7B-Instruct generates N candidate reasoning trajectories, and PhyCritic selects the best via pairwise knockout. The results show:

At N = 32, PhyCritic-guided selection achieves 60.8 on CosmosReason1-Bench, a +6.5 improvement over the base model's single-sample accuracy of 54.3
Using the base Qwen2.5-VL as the judge yields negligible improvements at any N
Using Cosmos-Reason1-7B as the judge yields smaller gains than PhyCritic at all N values

The key pattern is that PhyCritic's advantage as a selector grows with N—at N = 8, the gain over the best baseline judge is roughly 2 points; at N = 32, it is roughly 5 points. This scaling behavior supports the claim that PhyCritic's physical-domain judgment capability is robust at discriminating among diverse reasoning traces, not just distinguishing obviously correct from obviously incorrect responses.

PhyCritic as a Reward Model for Policy Training

In Appendix A (Table 8), PhyCritic is used as a reward signal for DPO training on Qwen2.5-VL-7B-Instruct. PhyCritic scores all response pairs and selects best–worst pairs as DPO preferences. The resulting policy achieves 60.0 on CosmosReason1-Bench, compared to 57.5 for an answer-verifier baseline (which constructs preference pairs by simply pairing correct with incorrect responses) and 54.3 for the base model. The 2.5-point gap over the answer-verifier baseline suggests that PhyCritic is selecting preference pairs where the "better" response has higher-quality reasoning even when both are correct, or where the "worse" response has subtle physical errors that a simple answer match would miss.

Ablation Studies and Robustness Checks

Training Strategy Ablation (Table 4)

The paper compares five training strategies to isolate the contribution of each pipeline component, all evaluated on PhyCritic-Bench, CosmosReason1-Bench, and VL-RewardBench:

Base Qwen2.5-VL-7B (no RL): PhyCritic-Bench 51.6, CosmosReason1-Bench 54.3, VL-Reward 53.2. Serves as the zero-shot baseline.
Physical RL only (Stage 1, 80 steps): PhyCritic-Bench 53.6 (+2.0), CosmosReason1-Bench 61.8 (+7.5), VL-Reward 52.0 (−1.2). Physical reasoning improves dramatically but critic judgment barely moves; VL-Reward slightly drops, suggesting some forgetting of general evaluation capability.
Physical RL extended (Stage 1 only, 80+300 steps): PhyCritic-Bench 52.7 (+1.1), CosmosReason1-Bench 63.1 (+8.8), VL-Reward 53.0 (−0.2). Extending Stage 1 training further improves physical reasoning but degrades critic performance relative to the 80-step variant—a finding consistent with the paper's claim that physical QA training alone does not teach critic skills and may cause overfitting that interferes with judgment.
Critic RL only (Stage 2 only, 300 steps, skip Stage 1): PhyCritic-Bench 62.2 (+10.6), CosmosReason1-Bench 57.1 (+2.8), VL-Reward 54.0 (+0.8). Direct critic training substantially improves judgment but physical reasoning gains are modest compared to Stage 1 training, indicating that critic RL alone does not adequately build the foundational physical understanding.
Mixed RL (single-stage, 80+300 steps, combined physical QA + critic rewards): PhyCritic-Bench 66.7 (+15.1), CosmosReason1-Bench 60.2 (+5.9), VL-Reward 55.5 (+2.3). Better than any single-stage variant but worse than the two-stage pipeline on all three metrics.
Two-stage RL (Stage 1 → Stage 2, 80 + 300 steps): PhyCritic-Bench 68.0 (+16.4), CosmosReason1-Bench 63.9 (+9.6), VL-Reward 57.3 (+4.1). Best on all metrics.

The non-obvious finding is that the two-stage sequential structure is strictly better than joint training (mixed RL)—the 1.3-point gap on PhyCritic-Bench (68.0 vs. 66.7) and the 3.7-point gap on CosmosReason1-Bench (63.9 vs. 60.2) demonstrate that the order of capability acquisition matters. The physical skill warmup must precede critic finetuning; attempting to learn both simultaneously creates negative interference between the two objectives.

Self-Referential Mechanism Ablations (Table 5)

Two ablations test the importance of self-referential grounding:

No self-referential process ("no self-refer."): removing the explicit self-prediction requirement from the critic prompt and training reward reduces PhyCritic-Bench from 68.0 to 64.4 (−3.6), CosmosReason1-Bench from 63.9 to 62.6 (−1.3), and VL-Reward from 57.3 to 56.6 (−0.7). The largest drop is on physical critic judgment, consistent with the mechanism being most impactful in domains requiring grounded physical reasoning.
No self-prediction reward ("no r_sp"): keeping the self-referential prompt structure but removing the $r_{\mathrm{sp}}$ reward term (setting $\alpha_{\mathrm{sp}} = 0$ ) reduces PhyCritic-Bench from 68.0 to 65.8 (−2.2), CosmosReason1-Bench from 63.9 to 63.5 (−0.4), and VL-Reward from 57.3 to 56.5 (−0.8). The smaller drops compared to "no self-refer." suggest that the prompt structure alone provides meaningful scaffolding, but the reward signal is necessary for full effectiveness—the model benefits from being told to self-reference, but it needs the accuracy incentive to make the self-referencing faithful rather than perfunctory.

The gap between "no self-refer." (−3.6) and "no r_sp" (−2.2) is 1.4 points on PhyCritic-Bench, indicating that the prompt structure alone accounts for roughly 60% of the self-referential benefit, with the reward signal contributing the remaining 40%.

Self-Prediction Reward Weight Ablation (Table 6)

The paper sweeps $\alpha_{\mathrm{sp}}$ across {0.1, 0.2, 0.3, 0.4} with $\alpha_{\mathrm{crit}} = 0.9 - \alpha_{\mathrm{sp}}$ and $\alpha_{\mathrm{form}} = 0.1$ fixed. PhyCritic-Bench performance is:

$\alpha_{\mathrm{sp}} = 0.1$ : 65.8 overall, 66.1 macro
$\alpha_{\mathrm{sp}} = 0.2$ : 68.0 overall, 69.0 macro (best)
$\alpha_{\mathrm{sp}} = 0.3$ : 66.2 overall, 66.4 macro
$\alpha_{\mathrm{sp}} = 0.4$ : 65.3 overall, 65.2 macro

The results show an inverted-U relationship: too little self-prediction weight (0.1) under-incentivizes the model to build its own understanding, while too much (0.4) over-emphasizes self-prediction at the expense of judgment accuracy (since $\alpha_{\mathrm{crit}}$ shrinks to 0.5). VL-RewardBench interestingly peaks at $\alpha_{\mathrm{sp}} = 0.1$ (58.6), suggesting that for general-domain evaluation, self-prediction grounding is less important—or even slightly detrimental—compared to focusing more weight on the critic judgment signal. This domain-dependence of the optimal $\alpha_{\mathrm{sp}}$ is not discussed in the paper but is visible in the table.

Critic Prompt Criteria Ablation (Table 7)

Removing the detailed evaluation criteria (truthfulness, visual groundedness, logical validity, efficiency) from the critic prompt reduces PhyCritic-Bench from 68.0 to 63.9 (−4.1), CosmosReason1-Bench from 63.9 to 62.0 (−1.9), and VL-Reward from 57.3 to 55.1 (−2.2). This 4.1-point drop on the physical critic benchmark is notable—the largest single ablation effect on critic performance. The paper explicitly contrasts this with prior work (Wang et al., 2025a), which uses a simple pairwise prompt for general-domain critic training, arguing that "building reliable critic capabilities for physical-AI tasks is considerably more challenging and requires explicit, structured critic guidance." The finding supports the paper's central thesis that physical AI evaluation is qualitatively harder and requires more scaffolding than general-domain evaluation.

General-Domain Critic Comparison (Table 9)

PhyCritic is compared with UnifiedReward-Think-Qwen-7B, a general-domain critic trained via RL on broad understanding and generation preference data, initialized from the same Qwen2.5-VL-7B-Instruct. UnifiedReward-Think achieves 73.8 on VL-RewardBench and 66.1 on Multimodal RewardBench—substantially higher than PhyCritic's 57.3 and 65.9 respectively—but only 52.4 on PhyCritic-Bench (vs. PhyCritic's 68.0) and 51.8 on CosmosReason1-Bench (vs. PhyCritic's 63.9). The asymmetry confirms that general-domain critic training does not transfer to physical evaluation, while physical critic training provides moderate benefits to general evaluation. The paper does not explore whether this transfer is due to improved visual perception, better reasoning, or better output formatting.

Downstream DPO Training (Table 8)

PhyCritic-guided DPO on Qwen2.5-VL-7B-Instruct produces a policy achieving 60.0 on CosmosReason1-Bench, outperforming an answer-verifier DPO baseline (57.5) by +2.5 points. Sub-task breakdown shows the largest gains on HoloAssist (66.0 vs. 60.0, +6.0) and RoboVQA (90.0 vs. 86.4, +3.6), with minimal improvement on AgiBot (41.0 vs. 39.0, +2.0) and RoboFail (61.0 vs. 60.0, +1.0). The pattern suggests PhyCritic's reward signal is most valuable in tasks requiring nuanced physical reasoning (HoloAssist involves egocentric interaction understanding) and less valuable in tasks where answer correctness alone is a strong enough signal (bridge data tasks with clearer success/failure criteria).

Critical Assessment

The experiments demonstrate that PhyCritic improves physical-domain critic accuracy over all open-source 7B/8B baselines and that the self-referential mechanism contributes meaningfully to this improvement. However, the paper's claims must be evaluated against several limitations in the experimental design that constrain the strength of the conclusions.

Claim 1: Self-referential grounding improves critic reliability and physical correctness

The ablation in Table 5 supports this: removing the self-referential process drops PhyCritic-Bench by 3.6 points, and removing the self-prediction reward drops it by 2.2 points. The chi-square analysis (Section 5.3) provides additional circumstantial evidence—the correlation between self-prediction correctness and judgment correctness strengthens after Stage 2 training, consistent with the model learning to ground judgments in its own understanding.

However, the experimental design does not rule out an alternative explanation: the self-prediction requirement may function primarily as a regularizer that shapes the model's output structure rather than genuinely causing it to use its own reasoning as a reference. The model receives a format reward for producing a <pred_think> section and an accuracy reward for getting the answer right. A model that produces a correct self-prediction and a correct judgment receives full reward regardless of whether the judgment causally depends on the self-prediction or simply co-occurs with it. The strengthened $\chi^2$ correlation could result from the model independently improving at both tasks through a shared representation, not from the judgment process actually referencing the self-prediction. The paper's qualitative examples (Table 10) show PhyCritic explicitly referencing its self-prediction in judgment reasoning, but this is cherry-picked evidence—no systematic analysis of whether the judgment reasoning actually follows from the self-prediction is provided.

A stronger test would be counterfactual evaluation: present the model with its own self-prediction artificially modified to be incorrect and measure whether the judgment changes accordingly. If the judgment flips when the artificial self-prediction contradicts the original, that would demonstrate genuine causal dependency. Without such a test, the mechanism is plausibly a training regularizer rather than a runtime reasoning strategy.

Claim 2: PhyCritic achieves SOTA open-source 7B/8B performance on physical critic evaluation

Supported by Table 2: 68.0 vs. the next-best open-source model (Eagle-2.5-8B at 56.0, a 12-point gap). However, the comparison is against models that were never trained for critic tasks—Eagle-2.5-8B is a general VLM, and Cosmos-R1-7B and RoboBrain2.0-7B are trained as policy models for physical reasoning, not as critics. The paper provides no comparison against an open-source physical critic because no such model exists, which is the gap the paper fills, but it means the "SOTA" claim is against a weak baseline field. The fair comparison would be a version of Cosmos-R1-7B or RoboBrain2.0-7B fine-tuned on the same critic data with a standard pairwise training objective (no self-referential mechanism)—this would disentangle the effect of the training data and the effect of the self-referential mechanism. This ablation is not present.

The gap to proprietary models (Gemini-2.5-Pro at 78.2, GPT-4o at 64.7) is substantial. PhyCritic exceeds GPT-4o (68.0 vs. 64.7) but trails Gemini-2.5-Pro by 10.2 points. The paper frames this as strong performance, but the 10.2-point gap suggests that model scale and proprietary training data provide benefits that the self-referential mechanism does not compensate for.

Claim 3: Physical critic training transfers to general-domain judging

Partially supported. PhyCritic improves over the base model on VL-RewardBench (53.2 → 57.3, +4.1) and Multimodal RewardBench (64.0 → 65.9, +1.9). However, the absolute performance on VL-RewardBench (57.3) remains far below the general-domain specialist UnifiedReward-Think (73.8), and the gain over the base model is modest (+4.1). The paper claims this demonstrates "effective transfer," but an equally valid interpretation is that physical critic training provides only minor spillover benefits to general evaluation—the 4.1-point gain leaves PhyCritic far from competitive as a general-domain judge.

Furthermore, the VL-RewardBench and Multimodal RewardBench results may partially reflect improved visual perception from the physical training data rather than genuine critic capability transfer. The base model's VL-RewardBench score (53.2) is only slightly above chance, suggesting the base model has room for improvement purely from better visual understanding, independent of critic-specific training. The paper does not isolate whether the VL-RewardBench gains come from improved perception or improved judgment—an ablation with a model trained only on physical QA (Stage 1) evaluated on VL-RewardBench would address this, but the Stage 1 result on VL-Reward is 52.0 (Table 4), which is slightly worse than the base model (53.2), suggesting that physical QA training alone actually degrades general evaluation capability. The Stage 2 critic training then brings it up to 57.3, but the gain relative to the Stage 1 trough is 5.3 points, not 4.1—a nuance the paper's framing obscures.

Claim 4: PhyCritic improves physical reasoning as a policy model

Supported by Table 3: 63.9 on CosmosReason1-Bench vs. base model 54.3 (+9.6). The comparison with Cosmos-R1-7B (63.0) is particularly notable since Cosmos-R1-7B was trained on millions of SFT reasoning traces while PhyCritic used only 4,058 RL samples. However, two qualifications apply.

First, the Cosmos-R1-7B comparison is not entirely fair: Cosmos-R1-7B may have been optimized for a different distribution or may have been trained with different hyperparameters. The paper reports Cosmos-R1-7B's scores from the original benchmark paper for some tasks (marked with † in Table 3), while PhyCritic's scores are from the authors' own evaluation. Different evaluation protocols could account for some of the gap.

Second, the policy-model gains are concentrated on CosmosReason1-Bench (63.9, +9.6) and EgoPlanBench2 (42.3, +9.7), but more modest on CV-Bench (79.7 vs. base 78.9, +0.8). Since CosmosReason1-Bench questions are drawn from the same source as the training data (Cosmos-Reason1), the large gain there could reflect training-test distribution overlap rather than genuine generalization of physical reasoning. The CV-Bench and EgoPlanBench2 results are more informative as generalization tests: the +9.7 on EgoPlanBench2 is impressive and suggests genuine transfer to a different physical planning domain, while the +0.8 on CV-Bench suggests limited improvement on spatial reasoning tasks.

Claim 5: The two-stage pipeline is necessary for optimal performance

Strongly supported by Table 4: the two-stage pipeline (68.0) outperforms mixed RL (66.7), physical RL only (53.6), and critic RL only (62.2) on PhyCritic-Bench. The sequential structure is clearly beneficial. However, the paper does not explore whether the order could be reversed (critic training first, then physical QA refinement) or whether a longer single-stage training with carefully tuned reward weights could match the two-stage performance. Mixed RL uses the same total steps (80+300) but the weighting between physical QA and critic objectives during the single stage is not reported—different weightings might close the gap.

The "physical RL extended" row (80+300 steps of physical QA only) shows that more Stage 1 training degrades critic performance (52.7 vs. 53.6 for 80 steps alone), which is an interesting negative result but is not explored further. The mechanism for this degradation—whether it is overfitting to the QA format, forgetting of general multimodal capabilities, or something else—is not investigated.

Claim 6: PhyCritic is data-efficient, requiring only 4,058 training samples

The paper emphasizes this, but the claim needs context. The 4,058 samples are highly curated: 800 QA pairs from a specific RL dataset pre-filtered for physical reasoning quality, and 3,258 critic pairs with candidate responses from seven diverse models, length-balanced, and verified by GPT-4o. The sample count may reflect the curation cost rather than a fundamental data efficiency property of the method. A fair test of "data efficiency" would require showing that performance degrades gracefully with fewer samples—no learning curve or data scaling experiment is reported. The claim that PhyCritic is "data-efficient" is better stated as: PhyCritic achieves strong performance using a small, carefully curated dataset, but we do not know how performance would change with more or less data.

Missing experiments that would strengthen the paper

Several experiments would significantly strengthen the paper's claims:

A standard pairwise critic baseline: train Qwen2.5-VL-7B on the same 3,258 critic pairs using supervised fine-tuning or basic RL without self-referential structure. This is the most natural baseline for isolating the self-referential mechanism's contribution—the current "critic RL only" variant in Table 4 still includes the self-referential prompt and rewards, so it only tests whether Stage 1 warmup is necessary, not whether the self-referential structure itself is necessary relative to a simpler critic training approach.
Data scaling curve: how does PhyCritic-Bench accuracy change as the number of training samples increases (e.g., 10%, 25%, 50%, 100% of the 3,258 samples)? Is the method genuinely data-efficient (high accuracy at low sample counts) or just operating in a regime where more data would help?
Multiple random seeds: all results are reported as single-run numbers without error bars. Given the 225-sample test set, a difference of 1-2 points between configurations could easily be noise. Confidence intervals would clarify which differences are statistically reliable.
Human evaluation correlation: PhyCritic-Bench uses GPT-4o-labeled preferences as ground truth. If GPT-4o's labeling is imperfect (and on CosmosReason1-Bench, GPT-4o itself scores only 56.3), the ground truth contains systematic errors. Human evaluation of a subset of PhyCritic-Bench instances would validate whether the GPT-4o labels align with human judgment and whether PhyCritic's improvements over baselines represent genuine alignment with human preferences or alignment with GPT-4o's particular biases.
Model scale analysis: all experiments use the 7B base model. The paper does not report whether the self-referential mechanism provides benefits at smaller scales (does it help a 2B model?) or whether larger models (72B) close the gap with proprietary models. This limits the generality of the findings.
Failure mode analysis: the paper provides qualitative successes (Tables 10-12) but no systematic categorization of failure modes. On which types of questions does PhyCritic still fail? Are failures due to incorrect self-predictions (the model solves the problem wrong, then judges based on wrong understanding) or incorrect judgments despite correct self-predictions (the model understands correctly but misapplies that understanding when evaluating)? These failure modes have very different implications for the mechanism's reliability.

Summary of confidence in claims

Self-referential grounding improves critic performance: Supported with qualifications. The ablation evidence is clear but does not establish causality. The mechanism is best characterized as a proven-effective training technique whose runtime behavior is not fully validated.
PhyCritic is the best open-source 7B/8B physical critic: Supported, but against a weak baseline field. The claim reflects the absence of competing physical critics as much as PhyCritic's absolute quality.
Physical critic training transfers to general evaluation: Weakly supported. The transfer is real but small (+4.1 on VL-RewardBench), and PhyCritic remains far from competitive with general-domain specialist critics.
PhyCritic improves physical reasoning as a policy: Supported, with the caveat that gains are largest on the benchmark closest to the training distribution and smallest on the most distinct benchmark (CV-Bench).
Data efficiency: Supported only in the weak sense that the method works with a small dataset. Without data scaling experiments, the claim of efficiency is untested.

6. Limitations and Trade-offs

6.1 The Self-Referential Training Mechanism Requires Ground-Truth Answers, Limiting Applicability to Open-Ended Domains

The assumption or constraint. The self-referential critic finetuning mechanism depends on a verifiable ground-truth answer $A_Q$ for every training instance. This answer is used both to construct the preference labels ( $P$ is determined by which response matches $A_Q$ ) and to supply the self-prediction reward ( $r_{\mathrm{sp}} = \mathbb{I}(\hat{A}_{\mathrm{pred}} = A_Q)$ ). The paper explicitly acknowledges this in Section 5.3:

"While self-referential critic fine-tuning has proven more effective than conventional pairwise critic training, it additionally requires ground-truth answers for the multimodal prompts. Although actions and planning can often be verified through physical-world feedback, this requirement limits its applicability to fully open-ended scenarios."

The consequence. This constraint fundamentally bounds the scope of problems to which PhyCritic's training recipe can be applied. Any domain where correctness is ambiguous, multi-dimensional, or subjective—open-ended physical reasoning (e.g., "explain why this robot arm's motion is efficient"), creative task planning, safety justification, or aesthetic evaluation—cannot provide the binary accuracy signal that drives both the preference labeling and the self-prediction reward. In these settings, one of two things happens: (1) the method cannot be applied at all, or (2) a proxy for correctness must be used (e.g., a heuristic or a noisy automated labeler), which may introduce systematic biases into the critic's training. The paper's downstream DPO experiment (Table 8) shows PhyCritic providing useful reward signals beyond simple answer matching, but even this experiment operates in the CosmosReason1 domain where ground-truth answers exist—the paper never demonstrates the method working without verifiable correctness.

Furthermore, the self-prediction reward creates a fundamental asymmetry: the critic learns to prefer responses that agree with its own (hopefully correct) understanding, but what happens when the critic's self-prediction is wrong? In the training data, responses are binary-labeled as correct or incorrect based on ground truth, so the critic never encounters a scenario where its wrong self-prediction aligns with a wrong candidate response—this pattern would receive zero reward (both $r_{\mathrm{sp}} = 0$ and $r_{\mathrm{crit}} = 0$ if the judgment is also wrong, or $r_{\mathrm{crit}} = 1$ if by chance the judgment flips the label). The training distribution therefore underrepresents the regime where the critic confidently holds a wrong belief and must evaluate responses that may or may not share that wrong belief. At deployment, if the critic misinterprets a video, its judgments may be consistently biased toward responses that share its misconception.

What evidence exists in the paper. The paper does not evaluate PhyCritic on any open-ended benchmark without verifiable answers. All evaluation datasets—PhyCritic-Bench, CosmosReason1-Bench, CV-Bench, EgoPlanBench2, VL-RewardBench, Multimodal RewardBench—have deterministic ground-truth answers. The paper does not report what fraction of PhyCritic's incorrect judgments co-occur with incorrect self-predictions (a failure mode where the critic's own misunderstanding leads it to prefer a wrong response over a correct one), nor does it compare this to the fraction where the self-prediction is correct but the judgment is wrong (a failure mode where the mechanism breaks down despite correct understanding). The chi-square association (Section 5.3) shows correlation but does not decompose error types.

Mitigation status. The paper acknowledges this as a limitation in Section 5.3 and gestures toward future work: "Future work may explore self-verification or meta-judging strategies to replace the explicit accuracy reward used for self-prediction." However, no experiments explore these alternatives. The limitation is inherent to the self-referential training design—removing the self-prediction reward eliminates the mechanism that makes the approach distinctive (the ablation in Table 5 shows this drops performance by 2-3 points). Until a replacement for ground-truth-dependent self-prediction rewards is demonstrated, the method is confined to domains with verifiable correctness signals.

6.2 The Difficulty Estimation / Pre-Solving Overhead Is Not Accounted For in the Critic's Inference Cost

The assumption or constraint. At inference time, PhyCritic must generate a complete self-prediction before it can produce its critic judgment. This is not an optional feature—the model's output structure is enforced by the training prompt and format reward, requiring <pred_think>, <pred>, and the critic thinking sections in sequence. A standard critic model (e.g., UnifiedReward-Think, Prometheus-Vision) only needs to generate the judgment reasoning, roughly halving the output length compared to PhyCritic's self-prediction + judgment structure.

The consequence. The self-referential mechanism imposes a latency and compute overhead at inference time that is not reflected in the paper's accuracy metrics. Every judgment requires the model to autoregressively generate tokens for solving the problem before it begins evaluating responses—tokens that a conventional critic never generates. For a practitioner deciding whether to deploy PhyCritic versus a simpler judge, the relevant comparison is not just accuracy but accuracy per unit of inference cost or wall-clock time. If PhyCritic takes 2× longer per judgment than a baseline judge, the 16.4-point accuracy gain over the base model (Table 2) must be weighed against the doubled latency. In latency-sensitive applications (real-time robot monitoring, interactive driving assistance), this overhead may be prohibitive regardless of accuracy improvements. The best-of-N test-time scaling experiment (Figure 4) amplifies this concern: PhyCritic performs $N-1$ pairwise comparisons for $N$ candidates, each comparison generating a full self-prediction. At $N = 32$ , this means 31 full self-predictions are generated to select one answer—a cost that dwarfs the base model's single-sample generation.

More subtly, the self-prediction may be redundant computation in many cases. If the critic is evaluating responses where one is obviously correct and the other obviously wrong (e.g., one response describes a physically impossible action), generating a full reasoning trace to solve the problem is unnecessary—a simpler judge could identify the error through surface-level checking. The self-referential mechanism treats all problems uniformly, generating a self-prediction regardless of how obvious the judgment is.

What evidence exists in the paper. The paper provides no measurements of latency, token counts, or inference cost. All evaluations report accuracy only. The format reward structure (Section 3.3) and the prompt template (Table 1) make it clear that the self-prediction is mandatory, but the paper never quantifies the overhead relative to a non-self-referential critic baseline. The comparison with UnifiedReward-Think (Table 9) shows that a general-domain critic achieves higher accuracy on VL-RewardBench (73.8 vs. 57.3) without self-referential structure, but the paper does not report whether UnifiedReward-Think's inferences are faster or shorter—they likely are, since no self-prediction is generated.

Mitigation status. Not addressed. The paper does not discuss inference cost optimization, potential caching strategies for the self-prediction (e.g., if the same question is evaluated multiple times with different response pairs), or any mechanism to skip the self-prediction when confidence is high. The prompt template enforces the self-prediction as a fixed output requirement with no provision for adaptive computation.

6.3 Results Are Demonstrated on a Single Model Family (Qwen2.5-VL-7B) and a Narrow Set of Physical Benchmarks

The assumption or constraint. All PhyCritic variants are initialized from Qwen2.5-VL-7B-Instruct, and all training and evaluation data is drawn from a specific set of embodied robotics and driving datasets with questions from Cosmos-Reason1. The paper states in Section 5 that it chose Qwen2.5-VL-7B because it is "representative of a standard open-source VLM scale (7B–8B parameters)," but no experiments vary the base model architecture, pretraining data, or scale.

The consequence. The paper's claims—that self-referential critic finetuning improves judgment accuracy, that physical critic training transfers to general evaluation, and that the two-stage pipeline is data-efficient—are all conditioned on this specific base model and data distribution. We do not know whether:

The self-referential mechanism provides benefits for models with different pretraining distributions. A model pretrained heavily on robotics data (e.g., RT-2, OpenVLA) might have sufficiently strong physical reasoning out of the box that the Stage 1 warmup is unnecessary, or might benefit less from self-referential grounding because its base judgments are already physically informed.
The mechanism scales to larger models. A 72B model might have strong enough zero-shot physical reasoning that self-referential grounding adds little, or might benefit more because its larger capacity enables more sophisticated self-prediction traces.
The training recipe transfers to models with different architectural inductive biases (e.g., models with different visual encoders, different context lengths, or different multimodal fusion mechanisms).
The specific hyperparameter choices ( $\alpha_{\mathrm{sp}} = 0.2$ , 80+300 steps, batch size 128) generalize across model scales and families.

Furthermore, the evaluation benchmarks represent a narrow slice of physical AI. CosmosReason1-Bench, CV-Bench, and EgoPlanBench2 all test perception, spatial reasoning, and planning, but they do not cover important physical AI capabilities like:

Force and dynamics reasoning: inferring masses, friction, or collision outcomes from visual observation.
Temporal prediction at scale: predicting long-horizon outcomes (minutes of robot operation, not single next steps).
Multi-agent physical interaction: reasoning about how multiple robots or humans interact physically.
Open-vocabulary physical reasoning: answering physical questions not constrained to a predefined action space or answer set.

The strong performance on HoloAssist (68.0 policy accuracy, Table 3) and the weak performance on AgiBot (53.0) demonstrate that the method's effectiveness varies significantly across domains within the same benchmark family. Without broader evaluation, we cannot tell whether PhyCritic's capabilities are genuinely "physical AI critic" capabilities or "Cosmos-Reason1-domain critic" capabilities.

What evidence exists in the paper. The paper reports all results on a single base model. The closest thing to a model-family ablation is Table 3's comparison with different physical-reasoning models (Cosmos-R1-7B, RoboBrain2.0-7B) on the same base architecture—but these are different training recipes, not different base models. The evaluation on CV-Bench and EgoPlanBench2 (Table 3) provides some evidence of generalization beyond the Cosmos-Reason1 domain, but both benchmarks remain in the perception-planning-reasoning space—they test breadth within a category, not breadth across categories. The paper does not discuss or measure performance on dynamics reasoning, long-horizon prediction, or multi-agent scenarios.

Mitigation status. Not addressed. The paper does not discuss model-family sensitivity as a limitation, nor does it suggest replication on other architectures as future work. The choice to build on Qwen2.5-VL is reasonable for an initial study, but the absence of model-family experiments limits the generality of the contributions as presented.

6.4 The Ground-Truth Preference Labels Are Generated by GPT-4o, Creating a Risk of Systematic Label Bias

The assumption or constraint. The preference labels for both the training data (3,258 critic pairs, Section 3.4) and the evaluation benchmark (225 PhyCritic-Bench samples, Section 4) are determined by GPT-4o verifying each candidate response against the ground-truth answer. The paper states: "GPT-4o verifies each response against the ground-truth answer and assigns a binary score (1 = preferred, 0 = rejected)." For training pairs, one correct and one incorrect response are paired, with correctness derived from GPT-4o's verification. For PhyCritic-Bench, the JudgeBench protocol generates N = 8 responses per prompt, GPT-4o identifies correct and incorrect predictions, and the final pair comprises one correct and one incorrect response.

The consequence. GPT-4o's verification is treated as ground truth, but GPT-4o is not a perfect verifier. On CosmosReason1-Bench, GPT-4o itself achieves only 56.3% accuracy (Table 3). While answer verification is a simpler task than answering (it requires matching a final answer against ground truth, not generating the answer), there are still failure modes: a candidate response might have the right final answer but flawed reasoning (GPT-4o would label it "correct"), or might have the wrong final answer format that GPT-4o misparses as correct. The paper's accuracy-based labeling methodology assumes that a response is preferred if and only if its final answer matches ground truth—this conflates "answer accuracy" with "response quality." A response with hallucinated visual details that accidentally arrives at the correct multiple-choice answer would be labeled "preferred" over a response with sound physical reasoning that arrives at the wrong answer due to a subtle visual misinterpretation.

More importantly, if GPT-4o has systematic biases in answer verification (e.g., preferring longer answers, being more lenient with certain answer formats, or systematically misreading certain physical scenarios), those biases become ground truth for both training and evaluation. The critic learns to reproduce GPT-4o's verification patterns, and the benchmark measures how well models reproduce GPT-4o's verification patterns—not necessarily how well they align with human judgment of physical correctness. The paper's single human-correlation datapoint is implicit in Table 2: Gemini-2.5-Pro achieves 78.2 on PhyCritic-Bench, while GPT-4o (the labeler) achieves 64.7. This 13.5-point gap suggests that GPT-4o's own verification standards are not perfectly aligned with Gemini-2.5-Pro's judgment, which in turn may be more or less aligned with human judgment—we cannot tell from the data.

What evidence exists in the paper. The paper provides no human evaluation of PhyCritic-Bench labels. No subset of the benchmark was verified by human annotators. No inter-annotator agreement statistics are reported (even between GPT-4o and another model). The paper does not report what fraction of GPT-4o's own answers on CosmosReason1-Bench are correct (the 56.3 in Table 3 is the average across all sub-tasks but the distribution across difficulty levels is not shown), so we cannot estimate how often GPT-4o makes verification errors on the specific question set used for labeling.

The construction protocol (Section 4) includes a potential safeguard: "GPT-4o verifies each response against the ground-truth answer." This suggests GPT-4o is given the ground-truth answer and asked whether the response matches it—a verification task, not a generation task. Verification is typically easier than generation, so GPT-4o's verification accuracy is likely higher than its 56.3% generation accuracy. However, the paper provides no measurement of GPT-4o's verification accuracy on this data, leaving the error rate unknown.

Mitigation status. Not addressed. The paper does not acknowledge label bias as a potential limitation or suggest human verification as a validation step. The reliance on GPT-4o for both training label generation and benchmark construction is practical but creates a potential circularity: the critic is trained to reproduce a specific model's concept of answer correctness, and evaluated on how well it reproduces that same model's concept of answer correctness. The high baseline performance of proprietary models (Gemini-2.5-Pro at 78.2) may partly reflect that these models share GPT-4o's verification tendencies rather than genuinely superior physical reasoning evaluation.

6.5 The Critic Training Data and Evaluation Benchmark Overlap in Question Source, Potentially Inflating Measured Performance

The assumption or constraint. Both the critic training data (Section 3.4) and the evaluation benchmark PhyCritic-Bench (Section 4) draw questions from the Cosmos-Reason1 dataset. The training data uses "800 high-quality question–answer pairs" from Cosmos-Reason1 RL, and the benchmark uses "questions adapted from CosmosReason1-Bench" for the robotics tasks. While the paper constructs training and evaluation sets from different splits (the training data is separate from the 225 evaluation samples), the questions originate from the same distribution—they share the same question-writing style, the same types of physical scenarios, the same answer formats, and the same difficulty profile.

The consequence. This creates a train-test distribution overlap that likely inflates PhyCritic's measured performance relative to its true generalization capability. The model is trained to answer and judge questions from the Cosmos-Reason1 distribution, then evaluated on questions from the same distribution. A model that memorizes domain-specific patterns (the typical structure of a Cosmos-Reason1 question, the typical visual features of the embodied datasets, the distribution of correct answer formats) will score well on PhyCritic-Bench even if it has not genuinely learned generalizable physical reasoning evaluation.

The per-sub-suite breakdown in Table 2 provides suggestive evidence of this overlap effect. PhyCritic achieves its highest scores on sub-suites whose videos and questions most closely match the training distribution: RoboVQA (86.7), AgiBot (78.8), and HoloAssist (65.5)—all datasets that contributed videos to the training data. On LingoQA (60.0), an autonomous driving dataset that was not part of the training video sources, PhyCritic's score ties with the best open-source model and is only marginally above the base model (which scored 48.0 in Table 2). Similarly, on RoboFail (57.4), a failure-recognition subset that tests a different type of physical understanding, performance is tied with the base model. The pattern suggests that PhyCritic's large gains are concentrated on the domains closest to its training data, with minimal transfer to out-of-distribution physical evaluation tasks.

What evidence exists in the paper. The comparison across sub-suites in Table 2 provides the primary evidence, but the paper does not explicitly analyze it in terms of train-test overlap. The paper does report results on the two general-domain benchmarks (VL-RewardBench and Multimodal RewardBench, Table 2) as evidence of generalization, but these benchmarks test general visual evaluation—not physical AI evaluation in a different domain. There is no held-out physical AI benchmark from a completely different source (e.g., a physics simulation QA dataset, a different robotics benchmark with distinct visual style and question format) against which PhyCritic's generalization could be tested.

The CV-Bench and EgoPlanBench2 results (Table 3), evaluated in policy mode, provide the closest thing to a cross-domain physical reasoning test. PhyCritic's +9.7 gain on EgoPlanBench2 over the base model and +0.8 gain on CV-Bench suggest some transfer, but these are accuracy-on-own-answers metrics, not critic judgment metrics—they test whether PhyCritic can solve physical problems, not whether it can evaluate others' solutions to physical problems from different domains. The paper never evaluates PhyCritic's critic capability on a physical AI benchmark outside the Cosmos-Reason1 sphere.

Mitigation status. The paper does not acknowledge this overlap as a limitation. It does not include an out-of-distribution physical critic evaluation set, nor does it analyze the per-sub-suite results in terms of training data proximity. The construction of PhyCritic-Bench with questions from the same distribution as the training data is a natural and defensible choice given the scarcity of physical AI evaluation resources, but it leaves open the question of whether PhyCritic's critic capabilities generalize to physical AI domains with different visual characteristics, question formats, or reasoning requirements.

6.6 The Critic's Policy-Model Gains Are Concentrated on Training-Adjacent Benchmarks, While Spatial Reasoning Gains Are Near Zero

The assumption or constraint. The paper presents PhyCritic's policy-model performance as a key finding—that "critic training further pushes the model's problem-solving capacities" (Section 5.1) and that a critic model "is secretly a strong policy model" (citing Wang et al., 2025a in related work). This positions critic training as a dual-use method that improves both evaluation and generation capabilities.

The consequence. The policy-model gains are highly uneven across benchmarks. On CosmosReason1-Bench, PhyCritic achieves 63.9 vs. the base model's 54.3—a +9.6 point gain (Table 3). On EgoPlanBench2, the gain is +9.7 (42.3 vs. 32.6). But on CV-Bench, the gain is only +0.8 (79.7 vs. 78.9). The CV-Bench result is particularly informative because it tests a different capability profile—2D and 3D spatial understanding, cross-view consistency—that is central to physical AI but is not the primary focus of the Cosmos-Reason1 training data. PhyCritic's near-zero gain on CV-Bench (and its second-place ranking behind RoboBrain2.0-7B's 85.8) suggests that the critic training process improves reasoning about actions and plans more than it improves spatial perception and geometric understanding.

This uneven transfer pattern means that PhyCritic as a policy model is not a general-purpose physical AI improvement—it is a targeted improvement on the reasoning dimensions that are well-represented in its training data. A practitioner hoping to use PhyCritic-style training to improve a model's 3D spatial reasoning or cross-view consistency would likely be disappointed. The paper's framing of "improved physical reasoning" is accurate but incomplete—it should specify which physical reasoning dimensions improve and which do not.

What evidence exists in the paper. Table 3 provides the direct evidence. The CV-Bench sub-scores are instructive: 2D improves from 75.1 (base) to 75.5 (+0.4), and 3D improves from 82.7 to 83.9 (+1.2). Both gains are within the range of run-to-run variance for a 500-sample benchmark, and neither approaches the gains seen on CosmosReason1-Bench (+9.6) or EgoPlanBench2 (+9.7). The paper does not discuss this discrepancy or acknowledge that the policy gains are domain-dependent. On the AgiBot subset of CosmosReason1-Bench, PhyCritic scores 53.0—only somewhat better than the base model's 40.4 but worse than Cosmos-R1-7B's 49.4—further demonstrating that gains are not uniform even within the CosmosReason1 domain.

The paper does not provide any analysis of why CV-Bench gains are small. Possibilities include: (1) the Stage 1 physical QA training does not sufficiently exercise spatial reasoning, (2) the critic training data emphasizes action planning over geometric reasoning, (3) the self-referential mechanism is less helpful when the reasoning is primarily perceptual rather than inferential, or (4) the base model's CV-Bench performance is already near its architecture's ceiling for spatial tasks. Without investigating which explanation is correct, the paper cannot provide guidance on when critic training will and will not improve policy-model capabilities.

Mitigation status. Not addressed. The paper treats the policy-model results as uniformly positive evidence that critic training improves physical reasoning, without disaggregating which reasoning capabilities benefit and which do not. The CV-Bench result is reported in Table 3 without commentary on the small magnitude of improvement.

7. Implications and Future Directions

How This Work Changes the Landscape

PhyCritic introduces a new design principle for multimodal critic models: that a judge should first solve the problem itself, then use that solution as an internal reference when evaluating others' outputs. This is not a prompt engineering trick—it is a training methodology enforced through reward design, with the self-prediction reward ( $r_{\mathrm{sp}}$ ) creating a coupling between the model's own problem-solving accuracy and its judgment accuracy. The paper's chi-square evidence (Section 5.3) shows this coupling is not just asserted but actually strengthened by training: the association between self-prediction correctness and judgment correctness increases from $\chi^2 = 51.07$ after Stage 1 to $\chi^2 = 161.76$ after Stage 2, indicating that the model genuinely learns to ground its evaluations in its own understanding rather than producing judgments that merely co-occur with correct self-predictions.

This matters because it addresses a trust problem that is particularly acute in physical AI. When a critic judges robot action plans or driving decisions, the stakes are safety-critical. A critic that produces correct-sounding but ungrounded evaluations—picking the more eloquent response rather than the physically correct one—is not just inaccurate but dangerous. The self-referential mechanism provides an audit trail: one can inspect the critic's self-prediction to verify that it understood the problem correctly before trusting its judgment. This is a conceptual contribution to critic reliability that operates orthogonally to scale—a larger model without self-referential grounding may still produce ungrounded evaluations, while a smaller model with self-referential grounding may produce more trustworthy ones.

Beyond the specific mechanism, the paper reframes physical AI evaluation as a distinct capability from general-domain multimodal judging, requiring purpose-built critics rather than adaptation of generalist evaluators. The empirical evidence for this reframing is stark. The comparison with UnifiedReward-Think (Table 9) shows a 16.5-point asymmetry: UnifiedReward-Think dominates on general-domain judging (VL-RewardBench 73.8 vs. PhyCritic's 57.3) but barely moves the needle on physical judging (PhyCritic-Bench 52.4 vs. the base model's 51.6). General-domain critic training does not transfer to physical evaluation. Yet PhyCritic's physical training does transfer partially to general evaluation (VL-RewardBench +4.1 over base model), suggesting that the capabilities required for physical judging—causal reasoning, spatial understanding, temporal consistency assessment—subsume some of what general judging requires. This asymmetry challenges the dominant assumption in the judge-model literature that one large generalist critic can evaluate everything. It redirects attention toward domain-specialized critics as a research program, especially for domains where surface-level pattern matching is insufficient for reliable evaluation.

The paper also provides one of the first systematic demonstrations that RLVR can produce a critic model with dramatically less data than supervised fine-tuning approaches require. PhyCritic uses 4,058 total training samples (800 physical QA pairs for Stage 1 + 3,258 critic pairs for Stage 2) across 380 RL steps, compared to the "millions of supervised finetuning traces" used by the Cosmos-Reason1-7B baseline, which PhyCritic matches or exceeds on both physical reasoning (63.9 vs. 63.0 on CosmosReason1-Bench) and dramatically exceeds on critic judgment (68.0 vs. 51.1 on PhyCritic-Bench). While this data efficiency claim needs tempering—the 4,058 samples are highly curated, and no data scaling curve is provided—the finding that a carefully designed multi-component reward with GRPO can substitute for massive SFT datasets has practical implications for any domain where preference data is scarce but verifiable ground truth exists. It makes critic training feasible for niche physical AI domains that will never accumulate the scale of data that general-domain critics enjoy.

The critic-as-policy dual-use finding is perhaps the paper's most surprising result with the broadest implications. Training a model to evaluate others' physical reasoning simultaneously improves its own physical reasoning—by 9.6 points on CosmosReason1-Bench over the base model (Table 3)—even though the training objective never directly incentivizes better answer generation beyond the relatively small self-prediction reward weight ( $\alpha_{\mathrm{sp}} = 0.2$ ). This challenges the conceptual separation between evaluation and generation capabilities that underlies the standard RLHF pipeline (where a frozen reward model scores a separate policy model). It suggests that in domains requiring deep reasoning, the process of learning to evaluate—including exposure to diverse correct and incorrect reasoning traces during training—feeds back into the evaluator's own generative capabilities. If this finding generalizes, it implies that critic training may be a more data-efficient route to capability improvement than direct policy training, because the critic learns from both positive and negative examples of reasoning while a policy model trained via behavioral cloning sees only positive examples. The paper's downstream DPO experiment (Table 8) closes the loop: a policy model trained with PhyCritic-guided preference pairs improves from 54.3 to 60.0 on CosmosReason1-Bench, outperforming an answer-verifier baseline (57.5). The critic, having learned to distinguish high-quality from low-quality physical reasoning, provides a better training signal than simple correctness checking.

The paper also resolves a latent tension in the multimodal judge literature. Prior work (e.g., Wang et al., 2025a) found that simple pairwise critic prompts sufficed for general-domain critic training. PhyCritic's prompt ablation (Table 7) shows that removing detailed evaluation criteria drops physical judging accuracy by 4.1 points—the largest single ablation effect on critic performance—while the paper explicitly notes this "contrasts with prior work such as Wang et al. (2025a)." This reconciles the apparent contradiction: physical AI judging requires explicit scaffolding (evaluation criteria) that general-domain judging does not, explaining why generalist critics fail on physical tasks and why a more structured approach is necessary.

Two research directions become less attractive in light of this work. First, the paper makes the case that simply scaling generalist judge models to larger sizes and broader training distributions is unlikely to close the gap on physical evaluation—the UnifiedReward-Think comparison shows that general-domain RL critic training, even with strong performance on general benchmarks, does not transfer to physical domains. The path forward involves domain-aware critic architectures, not just larger generalists. Second, the paper's ablation on mixed RL training (Table 4) shows that attempting to learn physical reasoning and critic judgment simultaneously is strictly worse than sequential training (66.7 vs. 68.0 on PhyCritic-Bench, and 60.2 vs. 63.9 on CosmosReason1-Bench). This challenges the assumption that multi-objective RL with weighted rewards can substitute for curriculum learning when downstream capabilities depend on upstream ones—a finding with implications for any multi-capability training pipeline.

Follow-Up Research This Work Enables

Counterfactual evaluation of the self-referential mechanism to establish causality. The paper shows correlation between self-prediction accuracy and judgment accuracy (the chi-square test in Section 5.3), but correlation does not demonstrate that judgments causally depend on self-predictions. A direct test: take a trained PhyCritic model, present it with a question and two responses, but intervene by replacing its generated self-prediction with an artificially constructed incorrect prediction before it produces the judgment. If the judgment flips to align with the incorrect self-prediction, this demonstrates genuine causal dependency. If the judgment stays correct despite the wrong self-prediction (or if it becomes incoherent), the mechanism is more of a training regularizer than a runtime reasoning strategy. A strong follow-up would classify failure modes by whether the self-prediction was correct but the judgment was wrong (the model understood the problem but failed to apply that understanding), vs. the self-prediction was wrong and the judgment followed it (the model confidently held a wrong belief and judged accordingly). These two failure modes have radically different implications for the safety and auditability of the self-referential approach.

Data scaling laws for physical critic training. The paper claims data efficiency (4,058 total samples) but provides no learning curve to support this. A systematic study would train PhyCritic variants on 10%, 25%, 50%, 75%, and 100% of the training data (all with the same two-stage pipeline and hyperparameters), measuring both PhyCritic-Bench accuracy and CosmosReason1-Bench policy accuracy. This would answer two critical questions: (1) Is the method genuinely data-efficient—does it achieve high performance at very low sample counts relative to a supervised fine-tuning baseline? (2) Is performance still improving at 3,258 samples, or has it saturated? If the curve is flat beyond 25% of the data, the data efficiency claim would be much stronger. If it is still rising steeply, the 3,258-sample result may significantly understate what the method could achieve with more data, and the claim of data efficiency would be weaker. A strong version of this experiment would compare against a supervised fine-tuning baseline trained on the same data at each data fraction, to isolate the contribution of RLVR from the contribution of data scale.

Cross-model-family replication of the two-stage pipeline. All PhyCritic results are on Qwen2.5-VL-7B-Instruct. A replication on a different base model architecture—for example, InternVL3-7B or LLaVA-OneVision-7B—would test whether the self-referential mechanism and two-stage curriculum are general or Qwen2.5-VL-specific. The optimal $\alpha_{\mathrm{sp}}$ value (0.2) and the number of Stage 1 steps (80) were tuned for Qwen2.5-VL; different architectures may require different tradeoffs between physical reasoning warmup and critic training. If the mechanism transfers cleanly with the same hyperparameters, the design principle is robust. If it requires architecture-specific tuning or fails entirely, the contribution is narrower. Additionally, a scale analysis—repeating the two-stage pipeline on 2B, 7B, and 72B variants of the same model family—would test whether self-referential grounding provides diminishing, constant, or increasing returns with model scale. The paper's finding that PhyCritic-7B (68.0) still trails Gemini-2.5-Pro (78.2) by 10.2 points on PhyCritic-Bench suggests that scale may provide complementary benefits, but we cannot tell without data on how a 72B PhyCritic would perform.

Human correlation study for PhyCritic-Bench ground-truth labels. The entire training and evaluation pipeline rests on GPT-4o-verified correctness labels. A study that collects human judgments on a representative subset (e.g., 100 instances) of PhyCritic-Bench, measuring inter-annotator agreement and human–GPT-4o agreement, would calibrate the paper's claims. If GPT-4o's verification labels have high agreement with human physical reasoning experts (say, Cohen's $\kappa > 0.8$ ), the benchmark is trustworthy. If agreement is moderate ( $\kappa \approx 0.6$ ), the "ground truth" contains systematic noise that may explain some portion of the gap between open-source and proprietary models (Gemini-2.5-Pro's 78.2 may partly reflect alignment with GPT-4o's labeling tendencies). If agreement is low (below 0.5), the benchmark measures something other than human-aligned physical judgment. The study would also reveal whether any PhyCritic-Bench sub-suites (RoboFail, LingoQA) have particularly low human–GPT-4o agreement, potentially explaining why PhyCritic's gains on those sub-suites are minimal.

Generalization to out-of-distribution physical AI evaluation. PhyCritic-Bench draws questions from the same Cosmos-Reason1 distribution as the training data, and the per-sub-suite results in Table 2 show the largest gains on domains closest to training (RoboVQA 86.7, AgiBot 78.8) and minimal gains on out-of-distribution domains (LingoQA 60.0, tied with baseline). A strong stress-test would construct a physical critic evaluation set from a completely independent source—for example, using questions from a physics simulation benchmark (PHYRE, CRAFT), a different robotics dataset with distinct visual style (something from the Open-X-Embodiment collection not overlapping with the training sources), or a physical commonsense QA dataset like PIQA adapted to video format. If PhyCritic's critic accuracy on this out-of-distribution set is significantly above the base model (say, 10+ points), the method genuinely learns transferable physical evaluation capabilities. If accuracy degrades to near-baseline levels, the method learns domain-specific patterns that do not generalize, and the "physical AI critic" framing overstates the contribution relative to "Cosmos-Reason1-domain critic."

Combining self-referential critics with online policy improvement loops. The paper demonstrates PhyCritic as an offline reward model for DPO (Table 8), but the critic's ability to provide judgment at test time opens a more ambitious possibility: online self-improvement where a policy model generates candidate actions or reasoning traces, PhyCritic evaluates them in real time (using the self-referential process), and the preferred traces are immediately used for policy updates. This would close the loop between evaluation and generation continuously rather than in batch. The key research question is whether the critic's self-prediction accuracy remains calibrated under distribution shift—as the policy model improves and generates responses from a progressively different distribution than the critic's training data, does the critic's judgment remain accurate, or does it suffer from the same out-of-distribution degradation that afflicts standard reward models? A concrete experiment: run 5–10 iterations of online DPO with PhyCritic as the reward signal, measuring both policy accuracy (on CosmosReason1-Bench) and critic accuracy (on a held-out set of policy outputs from each iteration) to track whether improvement plateaus due to critic degradation. This would test the practical limits of the virtuous cycle suggested by the dual-use finding.

Extension to open-ended physical evaluation via learned verifiers. The self-prediction reward requires ground-truth answers, limiting applicability to domains with verifiable correctness (Section 5.3). A concrete extension: replace the binary ground-truth verification in $r_{\mathrm{sp}}$ with a learned verifier trained on a small set of human-annotated physical reasoning quality scores (e.g., "is this explanation of the robot's failure physically plausible and internally consistent?"). The verifier would provide a continuous self-prediction reward that doesn't require exact answer matching, enabling self-referential critic training on open-ended physical reasoning tasks. The research question is whether a learned verifier provides a training signal strong enough to drive the coupling that the binary reward achieves—the paper's chi-square evidence shows that the coupling strength depends on the accuracy of self-predictions, and a noisy verifier might weaken the coupling rather than strengthen it. A strong follow-up would compare PhyCritic trained with ground-truth binary rewards vs. trained with a learned verifier on a task where both can be evaluated (e.g., a task where the ground truth is available but the verifier sees only a subset), measuring whether the verifier-based training achieves comparable critic accuracy and self-prediction–judgment coupling.

Practical Applications and Downstream Use Cases

Automated evaluation for physical AI model development. The most direct application is using PhyCritic-style critics as automated evaluators in the development loop for physical AI models—replacing or augmenting human evaluation of model outputs on robotics, embodied AI, and autonomous driving tasks. The paper shows PhyCritic achieves 68.0% pairwise accuracy on physical judgment vs. the base model's near-chance 51.6% (Table 2), and when used for best-of-N selection, it improves the base model's accuracy from 54.3 to 60.8 at N=32 (Figure 4). For a robotics team developing a new VLA model, this means they can automatically compare model variants on hundreds of physical reasoning questions, identifying which checkpoint produces better action plans or explanations, without the cost and latency of human evaluation. The critic's performance on RoboVQA (86.7%) and AgiBot (78.8%) specifically suggests it is reliable enough for deployment on these common embodied evaluation domains. The practical value is not that PhyCritic replaces human evaluation entirely—it still trails Gemini-2.5-Pro by 10.2 points—but that it provides a zero-cost, instant evaluation signal for the many comparisons that happen during model development (hyperparameter sweeps, data ablation studies, architecture comparisons) where human evaluation at scale is impractical.

Data filtering and curation for physical AI training sets. PhyCritic can be deployed as a quality filter for large-scale physical AI training data. When generating synthetic training data—for example, using a VLM to produce reasoning traces or action plans for millions of robotics videos—a large fraction of generated outputs will contain physical errors, hallucinations, or inconsistent reasoning. Training on unfiltered data propagates these errors into the policy model. PhyCritic can score candidate outputs pairwise against a known-good reference or against each other, filtering out low-quality generations before they enter the training set. The paper's DPO experiment (Table 8) provides proof of concept: using PhyCritic to select preference pairs for DPO training produced a 2.5-point improvement over a simple answer-verifier baseline, demonstrating that the critic can distinguish response quality beyond binary correctness. At scale, this filtering could substantially improve the efficiency of physical AI training by ensuring that models learn from high-quality reasoning traces rather than all synthetically generated outputs.

Safety monitoring for deployed physical AI systems. In safety-critical physical AI deployments—autonomous vehicles, surgical robots, industrial manipulation systems—a critic model can serve as a runtime monitor, evaluating the system's own outputs (planned actions, predicted outcomes, explanations of decisions) against physical correctness criteria. If the critic judges a planned action as physically unsound or inconsistent with visual evidence, it can trigger a human override or a fallback to a safer policy. PhyCritic's performance on the LingoQA autonomous driving subset (60.0%, Table 2) is modest—tied with the best open-source model but far from ceiling—so this application is aspirational rather than ready for deployment. However, the architecture (generate self-prediction, then evaluate) is well-suited to monitoring because it produces an auditable reasoning trace: a safety engineer can inspect the critic's self-prediction to understand what physical understanding drove the safety judgment. The path from current performance to deployment-grade reliability likely requires domain-specific fine-tuning (training a LingoQA-specialized critic, for example) and calibration against human safety judgments, but the framework is in place.

When to Prefer This Method

The paper does not articulate an explicit tradeoff against named alternatives in a way that supports a clear decision rule. It benchmarks against general-purpose VLMs and physical-reasoning policy models, but these are not alternative critic training methods—they are untrained or differently-trained baselines that PhyCritic outperforms. No comparison is provided against a standard pairwise critic trained on the same data with supervised fine-tuning or basic RL (without self-referential structure), so we cannot say when the self-referential approach should be preferred over simpler critic training methods.

The comparison with UnifiedReward-Think (Table 9) provides one tradeoff signal: if the evaluation domain is general visual understanding (captioning, VQA, STEM), a general-domain RL critic significantly outperforms a physical-domain critic (73.8 vs. 57.3 on VL-RewardBench). If the domain is physical AI (robot action evaluation, driving decisions, embodied reasoning), a physical-domain critic with self-referential grounding dramatically outperforms a general-domain critic (68.0 vs. 52.4 on PhyCritic-Bench). This suggests a domain-matching principle: use a critic trained on data from the target evaluation distribution. But this is not a distinctive property of the method—it's a general property of machine learning that would apply to any critic training approach. The paper does not provide the ablation (self-referential vs. standard pairwise critic, both trained on the same physical data) that would enable a stronger "prefer self-referential when..." recommendation.