ArXiv: 2605.09269

🎯 Pitch

A single MLLM trained to first plan verification checklists targeting disagreements, then check them, outperforms standard evaluators by huge margins—+22.6 points on a multimodal reward benchmark—because it is forced to visually verify its own criteria rather than lazily relying on text patterns.

1. Executive Summary

This paper introduces DeltaRubric, a framework that reformulates multimodal preference evaluation as a plan-and-execute process within a single shared MLLM, decomposing judgment into a Disagreement Planner (which generates a neutral, instance-specific verification checklist isolating factual divergences between candidate responses) and a Checklist Verifier (which executes each self-generated check against the image to produce a grounded final verdict). Training Qwen3-VL 4B and 8B Instruct models via multi-role reinforcement learning—where the Planner is rewarded for exposing the Verifier's blind spots and the Verifier is rewarded for accurate checklist-guided execution—DeltaRubric improves base model overall accuracy on VL-RewardBench by +22.6 (4B) and +18.8 (8B) points, consistently outperforming standard no-rubric baselines (+4.3 and +8.1 points respectively), while on the text-only RewardBench it simultaneously prevents catastrophic forgetting and improves reasoning performance by +6.7 points over the base model. The results establish that decomposing multimodal evaluation into structured, disagreement-driven verification steps yields reliable and generalizable reward modeling, with the Planner synthesizing targeted rubrics that actively enforce visual grounding rather than passively relying on language priors.

2. Context and Motivation

The Core Problem: Multimodal Reward Models Fall Prey to "Lazy Judging"

The fundamental question this paper addresses is deceptively simple: how do we build a reward model that actually looks at the image before deciding which response is better? This matters because the dominant approach to aligning Multimodal Large Language Models (MLLMs) with human preferences—Reinforcement Learning from Human Feedback (RLHF)—depends entirely on the reliability of the reward signal [2, 27]. If the reward model grades responses based on superficial textual patterns rather than grounded visual evidence, the entire alignment pipeline is compromised: the policy model learns to produce responses that sound correct rather than responses that are visually correct.

The gap the paper identifies is specific and well-motivated. Current multimodal reward models predominantly operate as single-step evaluators: they take an image, a question, and two candidate responses, and directly output a preference verdict or a scalar score. In text-only settings, rubric-based evaluation has emerged as an effective countermeasure against the "lazy judging" phenomenon documented by Zheng et al. [50]—the tendency of LLM judges to exploit language priors, length biases, and formatting heuristics rather than engaging in genuine evaluative reasoning [12, 15, 25, 30, 41]. However, as the authors explicitly state in Section 1:

"Although rubric-based evaluation has proven effective at mitigating these issues in text-only tasks, it remains largely underexplored in the multimodal domain. The primary bottleneck is the complexity of visual reasoning: the critical differences between two multimodal responses often depend on highly specific, instance-level visual details, such as object counts, spatial relationships, or localized hallucinations."

The problem is not merely that existing multimodal reward models are inaccurate—it's that their failure mode is systematic and pernicious. Because fine-grained visual grounding is computationally demanding, models learn to bypass it. A reward model that relies on textual priors will penalize responses that use uncommon phrasing even when factually correct, reward verbose but visually inaccurate descriptions over concise accurate ones, and fail to catch subtle hallucinations that contradict specific image details. This is not a hypothetical concern: prior work has documented that MLLM judges exhibit length correlations [33] and fail to verify image-grounded claims [14], and the paper provides concrete qualitative evidence in Figure 3 where the no-rubric baseline incorrectly selects a response that hallucinates "cars driving on the road" simply because the hallucinated response uses more confident, detailed language.

Why This Problem Is Important: Both Practical and Theoretical Stakes

The practical importance is immediate and concrete. The post-training pipeline for MLLMs increasingly relies on automated reward signals to scale beyond the bottleneck of human annotation. As the authors note, for tasks with ground-truth verifiers (mathematics, coding), alignment can be achieved with rule-based checks [5, 13, 19, 22–24, 35, 51]. But for the vast space of open-ended multimodal tasks—visual question answering, detailed image description, spatial reasoning, visual instruction following—there is no automatic correctness oracle. These are precisely the domains where learned reward models are indispensable, and also where lazy judging does the most damage. If the reward model's errors are systematic (favoring plausible-sounding over visually-grounded responses), the policy model trained against that reward will learn to produce increasingly fluent and confident hallucinations—the opposite of what alignment is meant to achieve.

There is also a theoretical significance that the paper's framing surfaces. The authors argue in Section 1 that:

"multimodal evaluation should not be treated as a passive scoring task, but rather an active reasoning process."

This is more than a rhetorical claim—it represents a shift in how evaluation itself is conceptualized. A passive scorer receives inputs and maps them to an output; an active reasoner must investigate the input, identifying what needs to be checked and then checking it. The distinction matters because passive scoring collapses the entire reasoning burden into a single forward pass, creating a situation where the path of least resistance—exploiting textual correlations—is also the computationally cheapest path. By reframing evaluation as active investigation, the paper argues for architectures that structurally enforce visual engagement, making it harder for the model to take shortcuts. This is analogous to how chain-of-thought prompting [39] forces LLMs to externalize intermediate reasoning steps, but applied to the verification side of the alignment pipeline.

Where Prior Approaches Fall Short

The paper identifies limitations in existing work along several dimensions, grounded in specific citations rather than straw-man arguments.

The LLM-as-a-Judge paradigm adapts poorly to multimodality. In text-only settings, the LLM-as-a-judge framework [17, 29, 50] has proven effective: models generate chain-of-thought rationales alongside preference verdicts, producing more calibrated and explainable judgments. When extended to the multimodal domain [40, 45], the same architecture—process image and text, generate rationale and verdict in one pass—is preserved. However, the paper (Section 2) notes a critical difference: text-only evaluation requires checking claims against the model's internal knowledge, while multimodal evaluation requires checking claims against external visual evidence that must be actively perceived. A single-pass judge can skip the perception step entirely and fall back on language priors without any architectural mechanism to prevent it.

Direct scalar reward models miss the reasoning. Recent work has produced multimodal reward models that predict scalar preferences either directly [49] or with CoT reasoning [37, 38, 47]. The paper positions these as incremental rather than fundamental advances over single-step evaluation: while CoT reasoning improves interpretability, the model still generates the entire evaluation trajectory in one unbroken sequence. There is no structural decomposition that isolates what needs to be verified from the act of verification itself. This means the model can produce a plausible-sounding rationale that never actually engages with the image—a failure mode the paper calls out in its qualitative examples, where the no-rubric baseline generates detailed analysis that contradicts the image (Figure 3, and Figures 4–6 in Appendix A.5).

Rubric-based methods have not been properly adapted to the visual domain. The most relevant prior line of work is rubric-based reward modeling in text-only settings. Approaches like those in [12, 15, 25, 30, 41] decompose complex judgments into criteria (e.g., "check for factual accuracy," "evaluate coherence") and evaluate each criterion explicitly. The paper acknowledges these as strong baselines but identifies a fundamental mismatch when extending them to multimodality. Text-only rubrics can be generic because the space of errors is relatively well-characterized: hallucinations, logical inconsistencies, missing information. Visual errors are instance-specific: whether Response A is better than Response B often depends on whether there are actually cars in a specific street scene, or whether a tissue box is green or white in a particular photograph. A generic rubric ("check for hallucinations") does not tell the model what to look for in the image. As the authors put it in Section 1:

"multimodal evaluation demands an active mechanism capable of dynamically synthesizing instance-specific rubrics that isolate the critical factual and spatial discrepancies between responses."

The critical word here is dynamically synthesizing. Existing rubric-based approaches use static, human-designed rubrics (as in the paper's static-rubric baseline in Table 5) or generate rubrics using a separate model with no joint optimization. Neither approach produces rubrics that are specifically targeted to the visual disagreements in the current example, and neither ensures that the rubric generator and the rubric executor develop complementary capabilities through co-training.

Architectural separation creates brittle pipelines. The paper specifically cites [41] as an approach that uses separate models for rubric generation and preference evaluation. This creates what the authors describe as "cascading errors" (Section 2): if the rubric generator produces a checklist that misses a critical visual discrepancy—or produces one that is irrelevant or misleading—the evaluator has no mechanism to recover. The evaluator is an obedient executor of whatever rubric it receives, with no ability to recognize that the checklist itself is flawed. The paper's key architectural claim is that joint optimization of planning and verification within a single model avoids this brittleness: the Planner learns to produce checklists that actually help the Verifier, because both are updated based on the same downstream objective.

How DeltaRubric Positions Itself Relative to Existing Work

The paper's self-positioning (articulated across Sections 1–3) can be understood through three conceptual moves:

First, it reframes evaluation as investigation rather than classification. Rather than treating multimodal evaluation as a mapping from inputs to a preference label—the paradigm shared by essentially all prior work—the paper models it as a two-phase process: planning (identifying what to check) and execution (performing the checks). This framing has a concrete architectural consequence: the model generates an explicit intermediate artifact (the checklist) that serves as both a plan for subsequent computation and a training signal for the planning capability itself. The checklist is not merely an output; it is a latent variable that structures the model's subsequent reasoning.

Second, it argues for dynamic, instance-specific rubrics over static, generic ones. The static-rubric baseline (Table 5) demonstrates that providing generic evaluation criteria does help—it raises the 8B model's overall accuracy from 72.0 to 75.3. But the gap from 75.3 to DeltaRubric's 80.1 is where the instance-specific contribution lives. The authors' claim is that generic rubrics tell the model what kinds of things to check for, but instance-specific rubrics tell the model exactly which visual facts to verify in this specific case. This is particularly important for the Reasoning subcategory on VL-RewardBench, where the 8B model shows a +13.0 point gain over the static baseline (Table 5). Complex spatial reasoning requires checking specific relationships that no generic rubric could anticipate.

Third, it introduces multi-role RL as the mechanism for joint optimization. The paper does not simply propose a two-step architecture; it proposes a specific training methodology that aligns the incentives of the two steps. Previous approaches that used separate models for rubric generation and evaluation [41] treat the rubric as a fixed input to the evaluator; there is no feedback loop where the Planner learns from the Verifier's failures. DeltaRubric's multi-role RL formulation in Section 3.3—with independent advantage estimation for Planner and Verifier—ensures that the Planner receives positive reinforcement only when its checklists actually improve verdict accuracy over the no-checklist baseline (Equation 1: Rplan(ci)=I(zi=z∗)−I(z0=z∗)R_{\text{plan}}(c_i) = \mathbb{I}(z_i = z^*) - \mathbb{I}(z_0 = z^*)), while the Verifier receives a bonus for correctly executing checklist-guided judgments that outperform unguided ones (Equation 2). This creates a symbiotic relationship: the Planner is incentivized to hunt for the specific factual divergences that the Verifier would otherwise miss, and the Verifier is incentivized to faithfully execute those checks rather than ignore them.

The paper's position is also clarified by what it explicitly does NOT claim. It does not argue that rubrics are a new idea—the acknowledgement of [12, 15, 25, 30, 41] is thorough and honest. It does not claim to have invented process-based evaluation, which has roots in the PRM literature [Lightman et al., 2023; Wang et al., 2023] and recent rubric-reward scaling work. And it does not claim to solve all multimodal evaluation problems—the hard questions in the Reasoning subcategory on VL-RewardBench remain challenging, and the approach is validated on a single model family (Qwen3-VL) with a bounded set of benchmarks. What it claims is a specific architectural and training innovation—dynamically synthesized, disagreement-focused rubrics optimized jointly with the evaluator that executes them—that addresses a specific documented failure mode (lazy judging in multimodal evaluation) with a measurable improvement over both no-rubric and static-rubric baselines across two independent benchmarks.

The Gap This Paper Fills, Precisely Stated

To summarize with precision: prior work demonstrated that (a) single-step multimodal reward models exhibit lazy judging [50, 14, 33], that (b) rubric-based evaluation mitigates this in text-only settings [12, 15, 25], and that (c) process-based or decomposed evaluation improves reliability in general [30, 41]. What was missing—and what DeltaRubric provides—is a practical method for synthesizing and executing instance-specific visual verification rubrics within a single jointly optimized model, where the rubric generation is explicitly trained to expose the evaluator's blind spots and the evaluation is explicitly trained to ground its verdicts in the generated checklist. This gap is not merely incremental: it addresses the central challenge that has prevented rubric-based methods from being effectively deployed in the multimodal domain, namely that generic rubrics cannot anticipate the specific visual facts that matter for each individual comparison.

3. Technical Approach

3.1 Reader Orientation

DeltaRubric is a system that trains a single Multimodal Large Language Model (MLLM) to act as its own evaluation architect — first planning what visual facts need checking, then executing those checks against the image — rather than jumping straight to a verdict. The core idea is that multimodal evaluation is not a passive classification problem but an active investigation problem, and the solution is a framework that decomposes judgment into a plan-then-execute process, where the planning step generates an instance-specific checklist isolating the exact visual disagreements between two candidate responses, and the execution step grounds each checklist item in visual evidence before reaching a final decision. Both capabilities are trained jointly through multi-role reinforcement learning, with separate reward signals for planning quality (did the checklist help the verifier?) and verification accuracy (did the check produce the correct verdict?), ensuring the Planner learns to expose the Verifier's blind spots rather than generate generic criteria.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components, all implemented within a single shared MLLM policy $\pi_\theta$:

  1. Disagreement Planner — Given the full input tuple (image $I$, question $q$, response $y_A$, response $y_B$), produces a neutral, evidence-seeking verification checklist $c$ consisting of 2–4 concrete, verifiable constraints that isolate the critical factual divergences between $y_A$ and $y_B$. The Planner does NOT express a preference; it merely identifies what needs to be checked. This is a generative rollout $c \sim \pi_\theta(\cdot \mid x)$ where $x = (I, q, y_A, y_B)$.

  2. Checklist Verifier — Takes the original input $x$ plus the self-generated checklist $c$ and produces a step-by-step reasoning trajectory $r$ followed by a final verdict $z \in \{A, B\}$. The Verifier explicitly evaluates each checklist item against the image $I$, aggregating evidence across items, and is instructed to ignore any checklist items that are hallucinated, irrelevant, or contradicted by the image. This is a conditional generative rollout $(r, z) \sim \pi_\theta(\cdot \mid x, c)$.

  3. Multi-Role RL Training Loop — During training, $\pi_\theta$ performs both roles sequentially. For each training example: (a) sample $N=5$ candidate checklists from the Planner, (b) score each checklist by querying the Verifier with a lightweight "cheap probe" prompt to see whether the checklist improves verdict accuracy over a no-checklist baseline, (c) select the best checklist via greedy decoding, (d) sample $M=5$ full verification trajectories conditioned on that best checklist, (e) compute independent advantages for Planner and Verifier based on their respective task objectives, and (f) update the shared policy with a joint loss that combines both sets of experiences.

  4. Cheap Probe Verifier — A lightweight variant of the Verifier used during Planner scoring that produces only a verdict $z$ without extended reasoning. This reduces the computational cost of evaluating $N=5$ candidate checklists per training example, since generating full reasoning trajectories for each would be prohibitively expensive. The cheap probe uses a minimal prompt: "Answer ONLY with [[A]] or [[B]]."

Information flows as follows: a training example $(I, q, y_A, y_B)$ with ground-truth preference $z^*$ enters → the Planner generates $N$ candidate checklists → each checklist is scored by the cheap-probe Verifier against the ground truth → the Planner reward is computed from whether the checklist corrected a no-checklist baseline error → the greedy-best checklist $c^*$ is selected → the full Verifier generates $M$ trajectories conditioned on $c^*$ → the Verifier reward is computed from final accuracy plus a conditional guidance bonus → advantages are normalized independently within each group → the joint loss combines both sets of clipped surrogate objectives → the shared policy $\pi_\theta$ is updated.

3.3 Roadmap for the Deep Dive

  • First, the problem formulation (Section 3.1), which defines the mathematical framing that justifies why a latent intermediate variable (the checklist) is necessary and how it differs from standard RLHF.

  • Second, the Planner-Verifier architecture (Section 3.2), covering what each component generates, how the checklist is structured, and the prompt templates that enforce neutrality for the Planner and checklist-grounded execution for the Verifier.

  • Third, the multi-role RL optimization (Section 3.3), which is the core technical contribution: how Planner rewards are computed via cheap-probe scoring against a no-rubric baseline (Equation 1), how Verifier rewards incorporate both accuracy and a conditional guidance bonus (Equation 2), how advantages are computed independently per task group, and how the joint loss (Equation 3) aggregates these separate signals into a single policy update.

  • Fourth, implementation details and hyperparameters, covering the RL algorithm (GRPO), model architecture (Qwen3-VL 4B and 8B Instruct), training dataset (30K RLAIF-V samples), sampling budgets ($N=5$, $M=5$), bonus coefficient ($\lambda=0.4$), and optimizer settings.

  • Fifth, design rationale, explaining why each architectural and optimization choice was made relative to alternatives (joint model vs. separate models, instance-specific checklists vs. static rubrics, relative Planner reward vs. absolute reward, decoupled advantage estimation vs. shared advantage).

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an architectural and training methodology paper whose core idea is that multimodal reward modeling should be decomposed into explicit planning and grounded execution steps, with both optimized jointly through decoupled multi-role reinforcement learning so that the planning step learns to generate checklists that specifically correct the Verifier's weaknesses.

Problem Formulation: Why a Latent Checklist Variable?

The paper begins by defining the standard multimodal preference evaluation task. Each training sample is a tuple:

x=(I,q,yA,yB)x = (I, q, y_A, y_B)

where $I$ is the image, $q$ is the question, and $y_A$ and $y_B$ are two candidate responses, with $z^* \in \{A, B\}$ representing the ground-truth preferred response. The standard RLHF approach models either $\pi_\theta(z \mid x)$ (direct verdict prediction) or $\pi_\theta(r, z \mid x)$ (reasoning-then-verdict prediction). The paper argues both formulations share a fundamental flaw: they compress the entire evaluation burden into a single generative trajectory, meaning the model can produce a plausible-sounding rationale $r$ without ever actually engaging with the visual evidence in $I$. This is the formal characterization of lazy judging: the model learns to sample $r$ from a distribution that is conditionally independent of $I$ given sufficient textual cues from $y_A$ and $y_B$.

The reformulation introduces an explicit intermediate variable — the checklist $c$ — that mediates between input and output. Rather than modeling $\pi_\theta(z \mid x)$ directly, the paper decomposes the evaluation into:

  1. Planning: $c \sim \pi_\theta(\cdot \mid x)$ — generate a verification structure from the input.
  2. Verification: $(r, z) \sim \pi_\theta(\cdot \mid x, c)$ — generate reasoning and verdict conditioned on both the input and the plan.

Why this decomposition matters. The checklist $c$ is not just an additional output token — it is a structural bottleneck that forces the model to externalize its hypotheses about what needs to be verified before it can produce a verdict. This has three consequences: (1) it makes the verification step explicitly conditional on the plan, creating a dependency that is hard for the model to ignore; (2) it provides an intermediate signal that can be evaluated and optimized independently (via the Planner reward in Equation 1), enabling targeted training of the planning capability; and (3) it creates a natural interface for interpretability, since the checklist is human-readable and explicitly identifies what visual facts the model considered decisive.

The paper's framing draws a parallel to latent-variable models in structured prediction: just as a parser might first predict a parse tree and then use that tree to guide semantic interpretation, DeltaRubric first predicts a verification structure and then uses that structure to guide evidence collection and judgment. The checklist plays the role of the latent structure — it is not observed in the training data but is inferred during both training and inference to improve downstream accuracy.

The Disagreement Planner: Generating Instance-Specific Checklists

The Planner takes the full input $x = (I, q, y_A, y_B)$ and generates a checklist $c$ consisting of a short sequence of numbered, verifiable constraints. Each constraint must describe exactly one concrete fact, relation, or attribute to verify from the image. The Planner is prompted (see Appendix A.6 for the full template) with explicit rules:

  • Each check must describe exactly one concrete fact, relation, or constraint to verify from the image.
  • The checklist must focus strictly on decisive disagreements or contradictory claims in the responses, not generic evaluation advice.
  • Each check must remain neutral and evidence-seeking — no mention of Response A or Response B by name, no indication of which response is better, no preamble or explanation.
  • The output must be only the numbered checks, with 2–4 items total.

What the Planner sees and doesn't see. Crucially, the Planner is given the image $I$ during checklist generation (the ablation in Table 7 tests removing visual context, finding performance drops particularly in Reasoning). This means the Planner can visually inspect the scene to identify which disagreements between $y_A$ and $y_B$ are actually verifiable from the image, rather than merely extracting textual contradictions. For example, if Response A says "the tissue box is green" and Response B says "the tissue box is white," the Planner can look at the image and generate a checklist item like "Does the tissue box have a green exterior or cover?" rather than a generic "Check the color of the tissue box." The former guides the Verifier to the specific visual attribute in dispute; the latter forces the Verifier to independently determine what to look for and how to assess it.

Post-generation filtering. The paper notes (Section 3.2) that "a post-generation filtering step is applied to further enforce this impartiality." The exact filtering mechanism is not detailed, but the intent is clear: if the Planner accidentally leaks preference information (e.g., "Check whether Response A's claim about the green tissue box is correct," which already implies which response made the claim), the filtering step catches and removes or revises such biased checklists. This is important because the Planner's output serves as input to the Verifier, and any bias in the checklist could cascade into biased verification.

Concrete example from Figure 3. For the question "Describe this image in detail," where Response A claims "cars driving on the road" and Response B does not mention cars, the Planner generates:

  1. Is the woman holding a checkered umbrella, not just any umbrella?
  2. Is there no visible car traffic on the street, only pedestrians and a bus?

These checklist items are narrowly targeted to the specific disagreements: item 1 isolates the umbrella pattern (a detail both responses mention differently), and item 2 isolates the car hallucination (a decisive factual error in Response A). Neither item mentions A or B by name; both are framed as neutral visual verification tasks. This is the Planner's core capability: translating the abstract notion of "these responses disagree" into a concrete, executable sequence of visual checks.

The Checklist Verifier: Executing the Plan Against Visual Evidence

The Verifier takes the original input $x$ plus the generated checklist $c$ and produces a two-part output: a step-by-step reasoning trajectory $r$ followed by a final verdict $z \in \{A, B\}$. The Verifier operates under explicit instructions (full prompt in Appendix A.6):

  • Treat the checklist as a shortlist of checks to execute, not as evidence. This is a critical instruction: the Verifier must independently verify each checklist item against the image $I$, not simply assume the checklist items are true statements about the scene.
  • Execute the checklist item by item. For each item, the Verifier must state the visual evidence it observes and indicate which response (if either) the evidence favors. This produces an intermediate alignment between checklist items and response preferences (e.g., "Item 1: The umbrella has a clear checkered pattern. Response A says 'an umbrella,' doesn't specify checkered. Response B says 'an umbrella,' doesn't specify checkered. Favors: Neither.")
  • Ignore irrelevant or contradicted items. If a checklist item asks about something not visible in the image, or if the image contradicts the premise of the item, the Verifier is instructed to skip it. This provides robustness against Planner errors — if the Planner generates a hallucinated or irrelevant check, the Verifier can recognize and disregard it rather than being forced to produce a nonsensical evaluation.

Interaction between checklist specificity and evaluation difficulty. The Verifier's task is made easier by good checklists and harder by poor ones. A checklist item like "Is there no visible car traffic on the street, only pedestrians and a bus?" tells the Verifier exactly what to look for (absence of cars, presence of pedestrians and bus) and implicitly identifies which response made which claim (Response A claimed cars, Response B did not). The Verifier's job reduces to: look for cars, report what you see, and map that finding to a response preference. By contrast, a generic checklist item like "Check for hallucinations" leaves the Verifier to independently determine what constitutes a hallucination, which claims to verify, and how to map verification results to a preference — essentially the same burden as the no-rubric baseline. This is why the Planner's quality directly impacts Verifier accuracy, and why training the Planner to generate decision-useful checklists (as verified by the Planner probe accuracy in Figure 2c) is essential.

The role of reasoning $r$. The Verifier's reasoning trajectory $r$ serves both functional and training purposes. Functionally, it provides an auditable record of how each checklist item was resolved, making the Verifier's judgments interpretable and debuggable. For training, it provides a surface over which the Verifier reward (Equation 2) can be defined — the reward is based on the final verdict $z$, which is the endpoint of $r$, meaning the RL optimization shapes the entire reasoning trajectory, not just the final token. This is consistent with the generative reward modeling paradigm [30, 41] where the model's evaluative reasoning process itself is the object of optimization, rather than a scalar reward head appended to a frozen representation.

Final verdict aggregation. After executing all checklist items, the Verifier aggregates the item-level evidence into a single justification (one sentence summarizing the checklist results) and a final verdict in the format [[A]] or [[B]]. The paper does not prescribe a specific aggregation function (e.g., majority vote across items, weighted sum, etc.) — the Verifier learns through RL to produce whatever aggregation strategy maximizes its reward. The checklist items provide structured evidence; the Verifier's learned policy determines how to weight and combine that evidence into a final decision.

Joint Optimization via Multi-Role RL: The Core Training Algorithm

This section describes the training procedure that jointly optimizes both capabilities of the shared model $\pi_\theta$. The key insight is that the Planner and Verifier have different, potentially conflicting, learning signals, and naively averaging their gradients would produce noisy updates that fail to optimize either capability effectively. The solution is decoupled advantage estimation: compute separate advantages for each role, then combine them into a single joint loss.

Planner Learning: Rewarding Checklists That Fix Blind Spots

For a given training example $x$ with ground-truth preference $z^*$, the Planner generates $N = 5$ candidate checklists:

C={c1,c2,…,cN}∼πθ(⋅∣x)C = \{c_1, c_2, \dots, c_N\} \sim \pi_\theta(\cdot \mid x)

To evaluate each checklist $c_i$ without incurring the cost of full verification trajectories (which would require $N \times M$ full rollouts), the paper uses a cheap probe: the Verifier is queried with a minimal prompt to produce only a verdict $z_i$ without extended reasoning:

zi∼πθ(z∣x,ci)z_i \sim \pi_\theta(z \mid x, c_i)

The cheap probe prompt (Appendix A.6) is: "You are a fair judge. Decide which response better answers the question below based on the image. [...] Use the verification checklist only as a shortlist of checks. Answer ONLY with [[A]] or [[B]]." This reduces the per-checklist evaluation cost to a single short generation, making it practical to score $N=5$ candidates per example.

Simultaneously, a baseline verdict is obtained without any checklist:

z0∼πθ(z∣x)z_0 \sim \pi_\theta(z \mid x)

This baseline represents the model's performance under the standard no-rubric paradigm and serves as the reference point for measuring checklist utility.

The Planner reward for checklist $c_i$ is defined as the relative improvement over the no-rubric baseline:

Rplan(ci)=I(zi=z∗)−I(z0=z∗)R_{\text{plan}}(c_i) = \mathbb{I}(z_i = z^*) - \mathbb{I}(z_0 = z^*)

where $\mathbb{I}(\cdot)$ is the indicator function (1 if the condition is true, 0 otherwise). $R_{\text{plan}}(c_i)$ takes values in $\{-1, 0, +1\}$, where $z_i$ is the cheap-probe verdict using checklist $c_i$, and $z_0$ is the baseline verdict.

What this reward computes operationally. For each candidate checklist $c_i$, the system runs the cheap-probe Verifier with $c_i$ to get verdict $z_i$, runs the cheap-probe Verifier without any checklist to get $z_0$, and compares both to the ground truth $z^*$. If $z_i$ is correct and $z_0$ is wrong, the checklist receives $+1$ (it flipped an error to a correct answer). If $z_i$ is wrong and $z_0$ is correct, the checklist receives $-1$ (it misled the Verifier into an error). If both are correct or both are wrong, the reward is $0$ (the checklist made no difference). The result is a single scalar per checklist.

Why this form — the relative reward design. The paper explicitly ablates an alternative formulation where the Planner is rewarded for absolute accuracy: $R_{\text{plan}}(c_i) = \mathbb{I}(z_i = z^*)$. This alternative (Table 4, "Ours (Abs.)") degrades performance substantially — Overall accuracy drops from 80.1 to 77.6 and Reasoning drops from 72.6 to 69.1. The reason (explained in Section 4.3) is that absolute reward gives positive reinforcement even for checklists generated on easy examples where the baseline Verifier would have been correct anyway. The Planner receives credit for "helping" on cases that needed no help, which dilutes the learning signal with noise. The relative reward focuses learning on the marginal cases where the checklist actually changes the outcome: the Planner is only rewarded for checklists that correct Verifier errors, and penalized for checklists that introduce errors, creating a clean signal that directly incentivizes the Planner to hunt for the Verifier's specific blind spots.

Depending on initialization, the Planner advantage for candidate $c_i$ is computed by normalizing within the group of $N$ checklists:

Aplan(i)=Rplan(ci)−μCσCA^{(i)}_{\text{plan}} = \frac{R_{\text{plan}}(c_i) - \mu_C}{\sigma_C}

where $\mu_C$ is the mean of $\{R_{\text{plan}}(c_1), \ldots, R_{\text{plan}}(c_N)\}$ and $\sigma_C$ is the standard deviation of the same set.

What this normalization does. GRPO (Group Relative Policy Optimization [31]) computes advantages relative to a group baseline rather than a learned value function. By normalizing rewards within the group of $N$ checklists generated for the same prompt, the advantage $A^{(i)}_{\text{plan}}$ measures how much better or worse checklist $c_i$ is compared to the average checklist produced by the current policy for this prompt. Positive advantages go to above-average checklists; negative advantages go to below-average ones. This removes the need to estimate a global baseline (a value function) and instead uses the group mean as a local reference point, which is more stable and scales better with model size.

How this shapes Planner behavior. Over many training iterations, the Planner learns to produce checklists that consistently earn positive advantages — meaning checklists that, when fed to the cheap-probe Verifier, produce correct verdicts on examples where the baseline Verifier would have been wrong. Since the baseline Verifier uses the same model $\pi_\theta$ (just without a checklist), the Planner is essentially learning to generate instructions that steer the model away from its own systematic errors. This is the "blind spot hunting" dynamic: the Planner learns where the Verifier tends to fail and generates checklists that specifically preempt those failures.

Verifier Learning: Rewarding Accurate Checklist-Guided Execution

After scoring the Planner candidates and selecting the best one, the system runs a greedy forward pass through the Planner to obtain the deterministic best checklist:

c∗=arg⁡max⁡πθ(⋅∣x)c^* = \arg\max \pi_\theta(\cdot \mid x)

This greedy checklist $c^*$ is then provided to the Verifier, which samples $M = 5$ full reasoning trajectories:

R={(r1,z1),…,(rM,zM)}∼πθ(⋅∣x,c∗)R = \{(r_1, z_1), \dots, (r_M, z_M)\} \sim \pi_\theta(\cdot \mid x, c^*)

Each trajectory includes both the step-by-step reasoning $r_j$ (the analysis of each checklist item against the image) and the final verdict $z_j$. The Verifier reward for trajectory $(r_j, z_j)$ is:

Rverify(rj,zj)=I(zj=z∗)+λmax⁡(0,I(zj=z∗)−I(z0=z∗))R_{\text{verify}}(r_j, z_j) = \mathbb{I}(z_j = z^*) + \lambda \max(0, \mathbb{I}(z_j = z^*) - \mathbb{I}(z_0 = z^*))

where $z_j$ is the verdict from trajectory $j$, $z^*$ is the ground-truth preference, $z_0$ is the baseline no-checklist verdict, and $\lambda = 0.4$ is the guidance bonus coefficient.

What this reward computes operationally. The first term $\mathbb{I}(z_j = z^*)$ is a standard accuracy reward: 1 if the verdict is correct, 0 otherwise. The second term is a conditional bonus: it adds $\lambda \times 1$ if the checklist-guided verdict $z_j$ is correct AND the baseline no-checklist verdict $z_0$ was incorrect — that is, cases where the checklist actually improved upon the unguided baseline. The $\max(0, \cdot)$ wrapper ensures the bonus is never negative: if $z_j$ is wrong while $z_0$ was right (checklist misled the Verifier), the bonus term is zero, but the accuracy term is already zero, so the penalty is implicit in the loss of the accuracy reward. If both are correct or both are wrong, the bonus is zero.

The total reward ranges from 0 (wrong verdict) to $1 + \lambda$ (correct verdict that improved upon baseline), making $1 + 0.4 = 1.4$ the maximum possible reward. The bonus creates an asymmetric incentive: correct verdicts that beat the baseline are rewarded more than correct verdicts on already-easy examples.

Why this form — the guidance bonus design. The sensitivity analysis in Table 8 (Appendix A.2) tests values $\lambda \in \{0.0, 0.2, 0.4, 0.6\}$. Without the bonus ($\lambda = 0.0$), the Verifier receives the same reward for correct verdicts regardless of whether the checklist was actually helpful — it gets full credit even when the baseline would have been correct anyway. This leads to weaker checklist adherence because the Verifier has no incentive to actually use the checklist; it can produce a correct verdict through its own (possibly checklist-ignoring) reasoning and still receive maximum reward. The bonus $\lambda = 0.4$ specifically incentivizes the Verifier to produce correct verdicts in the subpopulation where the baseline fails, which is exactly where the checklist adds value. However, $\lambda = 0.6$ degrades reasoning performance (Reasoning drops from 72.6 to 66.3), suggesting that an overly strong bonus causes the Verifier to over-optimize for checklist adherence at the expense of independent reasoning — it becomes too obedient to the checklist, even when checklist items are subtly flawed or incomplete, rather than using the checklist as guidance while applying its own judgment.

The Verifier advantage for trajectory $j$ is normalized within the group of $M$ trajectories:

Averify(j)=Rverify(rj,zj)−μRσRA^{(j)}_{\text{verify}} = \frac{R_{\text{verify}}(r_j, z_j) - \mu_R}{\sigma_R}

where $\mu_R$ is the mean of $\{R_{\text{verify}}(r_1, z_1), \ldots, R_{\text{verify}}(r_M, z_M)\}$ and $\sigma_R$ is the standard deviation. As with the Planner advantage, this group-relative normalization removes dependence on a learned value function and measures how each trajectory compares to the average trajectory for the same prompt under the current policy.

How this shapes Verifier behavior. Over training, the Verifier learns to produce trajectories that maximize the combination of accuracy and baseline-beating bonus. Since the bonus is only available when the checklist actually helps, the Verifier is implicitly incentivized to (a) attend to the checklist, (b) faithfully execute each item against the image, and (c) produce verdicts that align with the evidence rather than reverting to the textual shortcuts that cause the baseline to fail. The group-relative normalization also creates implicit competition: for a given prompt, there is pressure to produce trajectories that are better than the average trajectory for that prompt, which drives continuous improvement as the average itself rises.

Joint Multi-Role Loss: Combining Separate Experiences into One Update

The final policy update combines the Planner and Verifier experiences into a single optimization step. Let $\mathcal{L}(\theta \mid \text{data}, A)$ denote the standard RL clipped surrogate objective (GRPO's loss function). The shared model $\theta$ is updated by minimizing the joint loss:

Ltotal(θ)=Ex∼D[1N∑i=1NL(θ∣ci,Aplan(i))+1M∑j=1ML(θ∣rj,zj,Averify(j))]\mathcal{L}_{\text{total}}(\theta) = \mathbb{E}_{x \sim \mathcal{D}} \left[ \frac{1}{N} \sum_{i=1}^{N} \mathcal{L}\left(\theta \mid c_i, A^{(i)}_{\text{plan}}\right) + \frac{1}{M} \sum_{j=1}^{M} \mathcal{L}\left(\theta \mid r_j, z_j, A^{(j)}_{\text{verify}}\right) \right]

where $\mathcal{D}$ is the training data distribution, $N = 5$ is the number of Planner rollouts, $M = 5$ is the number of Verifier rollouts, $\mathcal{L}(\theta \mid c_i, A^{(i)}_{\text{plan}})$ is the GRPO loss for the Planner rollout $c_i$ using its group-normalized advantage $A^{(i)}_{\text{plan}}$, and $\mathcal{L}(\theta \mid r_j, z_j, A^{(j)}_{\text{verify}})$ is the GRPO loss for the Verifier rollout $(r_j, z_j)$ using its group-normalized advantage $A^{(j)}_{\text{verify}}$.

What this loss computes operationally. For each training example $x$, the system generates $N$ Planner rollouts (the checklists) and $M$ Verifier rollouts (the reasoning-and-verdict trajectories). Each rollout has an associated advantage computed independently within its task group. The GRPO loss for each rollout is the standard clipped surrogate objective — it encourages the policy to increase the probability of tokens that led to positive advantages and decrease the probability of tokens that led to negative advantages, subject to a clipping constraint that prevents too-large updates. The joint loss averages these per-rollout losses across all $N + M$ rollouts, treating Planner and Verifier contributions symmetrically in the sum. The policy update moves $\theta$ in a direction that simultaneously improves checklist generation quality (Planner) and checklist-grounded verification quality (Verifier).

Why this form — decoupled advantage estimation within a shared policy. The critical design choice is that advantages are computed separately for each task group (Planner group of size $N$, Verifier group of size $M$) and only normalized within their respective groups. This prevents cross-task variance from corrupting the RL signals. If advantages were pooled across both tasks, a particularly high Planner advantage could dominate the gradient and cause the policy to over-optimize for checklist generation at the expense of verification quality, or vice versa. By computing independent group statistics ($\mu_C, \sigma_C$ for Planner, $\mu_R, \sigma_R$ for Verifier), the learning signal for each task is calibrated to the difficulty and variance of that specific task, ensuring balanced improvement.

The paper explicitly argues (Section 3.3): "By computing advantages separately for each task group, we ensure that the Planner gradients are strictly driven by checklist quality, and Verifier gradients are strictly driven by execution quality, preventing cross-task variance from corrupting the RL signals." This is the formal justification for decoupling: the Planner should not be penalized because the Verifier happened to produce noisy rollouts for a different prompt, and the Verifier should not be penalized because the Planner happened to produce a particularly good checklist on a different example. Each task's gradient reflects only the quality of its own outputs relative to other outputs from the same task on the same prompt.

Interaction with GRPO's group-relative formulation. GRPO [31] was originally designed for single-task RL where a group of $G$ rollouts is sampled for the same prompt, and advantages are computed relative to the group mean. DeltaRubric extends this to the multi-task setting by partitioning the rollouts into two task-specific groups. Within each group, the GRPO mechanism operates as usual: the group mean serves as a local baseline, and the policy is updated to favor above-average rollouts. The innovation is the partitioning, not the baseline mechanism itself. This works because the Planner and Verifier generate structurally different outputs (checklists vs. reasoning trajectories) that should not be compared directly — a mediocre checklist might have a higher absolute reward than an excellent verification trajectory simply because the tasks have different difficulty distributions, creating misleading relative rankings if pooled.

Implementation Details and Hyperparameters

Base models. All experiments fine-tune Qwen3-VL-4B-Instruct and Qwen3-VL-8B-Instruct [1] using direct RL (no supervised fine-tuning warmup). The choice of Qwen3-VL is practical — the models are open-weight and represent state-of-the-art in their size class — but the paper's framework is model-agnostic by design since both Planner and Verifier use the same architecture with different prompting.

Training data. The training set consists of 30,000 instances randomly sampled from the RLAIF-V dataset [45]. Each instance provides an image-query pair, two candidate responses, and a human preference label indicating which response is better. The paper explicitly states that the data is "strictly decontaminated" to ensure zero overlap with evaluation sets (VL-RewardBench and Multimodal RewardBench), which is essential for the out-of-domain generalization claims.

RL algorithm. Training uses GRPO (Group Relative Policy Optimization) [31] as the base RL algorithm, with the decoupled multi-role extension described above. GRPO was chosen for its simplicity (no learned value function) and stability at scale. The paper also validates DeltaRubric with DAPO [43] (Table 9 in Appendix A.3), demonstrating that the framework is algorithm-agnostic — the decoupled advantage estimation approach integrates with any group-based RL algorithm.

Sampling budgets. For each training example, the Planner samples $N = 5$ candidate checklists, and the Verifier samples $M = 5$ full reasoning trajectories. These values represent a tradeoff: larger $N$ provides more exploration for the Planner but increases computational cost (each candidate requires a cheap-probe Verifier forward pass); larger $M$ provides more stable Verifier gradients but similarly increases cost. The choice of 5 for both reflects a practical balance validated by the observed training dynamics (Figure 2).

Training duration and optimizer. Models are trained for 120 steps using the AdamW optimizer [26] with learning rate $1 \times 10^{-6}$, weight decay 0.01, global batch size 128, rollout batch size 256, and generation temperature 1.0. The learning rate is relatively low (typical for RL fine-tuning of pretrained models to avoid catastrophic forgetting), and the temperature of 1.0 encourages diverse rollouts for exploration. The implementation uses the EasyR1 framework [52].

Guidance bonus coefficient. The Verifier reward (Equation 2) uses $\lambda = 0.4$, selected via the sensitivity analysis in Appendix A.2 (Table 8). Values tested were $\{0.0, 0.2, 0.4, 0.6\}$. At $\lambda = 0.0$, the bonus is disabled, and performance is suboptimal (Overall 78.2 vs. 80.1 at $\lambda = 0.4$), indicating that incentivizing checklist adherence is important. At $\lambda = 0.6$, Reasoning drops sharply (66.3 vs. 72.6 at $\lambda = 0.4$), indicating over-optimization — the Verifier becomes too obedient to the checklist and loses the ability to recognize when checklist items are flawed or incomplete. The value 0.4 represents the sweet spot where the bonus is strong enough to enforce checklist usage but not so strong that it overrides independent reasoning.

Cheap probe configuration. During Planner scoring, the cheap-probe Verifier is prompted with a minimal instruction (Appendix A.6): "You are a fair judge. Decide which response better answers the question below based on the image. [...] Use the verification checklist only as a shortlist of checks. Answer ONLY with [[A]] or [[B]]." This prompt omits the request for reasoning, producing only the verdict token, which substantially reduces generation cost compared to full verification trajectories.

Post-generation filtering. The paper mentions (Section 3.2) that "a post-generation filtering step is applied to further enforce this impartiality" for the Planner's checklists. The filtering likely uses simple string-matching heuristics to detect and discard checklists that mention Response A or Response B by name, express a preference, or otherwise violate the neutrality constraint specified in the Planner prompt. The filtering is not extensively detailed but is conceptually straightforward: any checklist that leaks bias is removed before it reaches the Verifier, preventing the Verifier from being primed toward a particular answer by the checklist itself.

Design Rationale: Why Each Choice Over Alternatives

Joint model vs. separate Planner and Verifier models. The paper uses a single shared model $\pi_\theta$ for both roles rather than training separate models. This has three advantages: (1) parameter efficiency — a single model serves both functions, halving memory and storage requirements; (2) representational alignment — the Planner and Verifier develop compatible internal representations since they share weights, meaning the Planner learns to generate checklists in a "language" the Verifier naturally understands; (3) simplified deployment — at inference time, the same model is called twice (first as Planner, then as Verifier) without requiring model switching or loading. The cost is potential capacity competition — the two tasks might compete for model capacity — but the empirical results suggest that for models of this scale (4B–8B parameters), the shared representation is beneficial rather than constraining.

Instance-specific checklists vs. static, dataset-level rubrics. The static-rubric baseline in Table 5 conditions the Verifier on a fixed set of generic criteria: "Directly answers the question using the information relevant to the image," "Makes factual claims that are consistent with the image," "Correctly identifies important visual information," "Uses sound reasoning," "Gives a clear and complete answer." This baseline improves over no-rubric (75.3 vs. 72.0 Overall for 8B), confirming that any structural prior helps. But the gap to DeltaRubric (80.1 Overall) demonstrates the value of instance-specificity. A generic rubric tells the model what categories of error to look for; an instance-specific checklist tells the model exactly which claims to verify against which visual evidence. The Reasoning subcategory shows the largest gain (+13.0 points for 8B over static rubric; Table 5) because complex reasoning tasks involve specific spatial, logical, or numerical relationships that no generic rubric could enumerate — the checklist must be synthesized on-the-fly based on the actual content of the two responses.

Relative Planner reward (Equation 1) vs. absolute reward. The ablation in Table 4 directly compares $R_{\text{plan}} = \mathbb{I}(z_i = z^*) - \mathbb{I}(z_0 = z^*)$ (relative) against $R_{\text{plan}} = \mathbb{I}(z_i = z^*)$ (absolute). The relative formulation outperforms by +2.5 points Overall and +3.5 points Reasoning. The absolute reward fails because it gives the Planner credit for checklists that are irrelevant — if the baseline Verifier would have been correct anyway, any checklist that doesn't break the Verifier receives $+1$, regardless of whether the checklist contained useful content. The Planner can achieve high reward by generating vacuous checklists (e.g., "Check if the image contains the objects mentioned") that never interfere with the Verifier's existing capability. The relative reward forces the Planner to add value beyond what the Verifier can do alone, which requires generating checklists that specifically address cases where the Verifier's default reasoning fails.

Decoupled advantage estimation vs. shared baseline. Computing advantages separately for Planner and Verifier groups is essential because the two tasks have different difficulty distributions and variance characteristics. A Planner advantage depends on whether a checklist flips a verdict (a relatively rare event — most checklists will have zero reward since they neither help nor hurt). A Verifier advantage depends on whether a trajectory produces the correct verdict (a more common event). If advantages were normalized across a pooled group of both Planner and Verifier rollouts, the Verifier's more frequent non-zero rewards would dominate the normalization, making Planner advantages appear systematically small and suppressing Planner learning. Decoupling ensures each task's learning signal is calibrated to its own reward distribution.

GRPO with group normalization vs. PPO with learned value function. The paper uses GRPO's group-relative baseline rather than PPO's learned value function for two reasons: (1) simplicity — no separate value network needs to be trained, reducing memory and implementation complexity; (2) stability — the group mean is an unbiased baseline that adapts immediately to policy changes, whereas a learned value function lags behind and can introduce bias during rapid policy improvement. The tradeoff is that GRPO requires generating multiple rollouts per prompt, which increases per-step computation, but at $N=5, M=5$ this overhead is manageable.

Greedy best-checklist selection vs. sampling. During Verifier training, the system selects $c^* = \arg\max \pi_\theta(\cdot \mid x)$ (greedy decoding) rather than sampling from the Planner's distribution. This is a deliberate choice: the Verifier should be trained on the best checklist the Planner can produce, since at inference time the system will use greedy or near-greedy decoding for the Planner (there's no benefit to sampling a worse checklist). Training on sampled (potentially suboptimal) checklists would teach the Verifier to be robust to checklist quality variation, which is a useful capability, but would also mean the Verifier receives a weaker training signal on average because many sampled checklists would be mediocre. The greedy selection ensures the Verifier trains primarily on high-quality checklists, maximizing the signal-to-noise ratio in Verifier gradients. The Planner still explores diverse checklists during its own training (via the $N=5$ samples), so it learns to produce high-quality greedy outputs.

Two-step inference cost vs. single-step baselines. At inference time, DeltaRubric requires two sequential forward passes: first the Planner generates a checklist, then the Verifier generates reasoning and verdict conditioned on that checklist. This doubles the latency compared to a single-step judge, which is a genuine tradeoff. The paper does not explicitly discuss inference cost, but the architecture implies that the checklist generation step is relatively cheap — the Planner outputs 2–4 short sentences (the checklist items), while the Verifier outputs a longer reasoning trajectory. In practice, the Planner's generation cost is a small fraction of the total, and the accuracy gains (+4.3 to +8.1 points on VL-RewardBench over no-rubric, +4.5 points on Multimodal RewardBench) likely justify the modest latency increase for most applications where evaluation quality matters more than speed.

4. Key Insights and Innovations

Innovation 1: Reframing Multimodal Evaluation as Active Investigation Rather Than Passive Scoring

The paper's most foundational conceptual move is not any specific architectural choice, but rather the reframing of what multimodal evaluation is. Prior work — across LLM-as-a-judge [17, 29, 50], direct scalar reward models [49], and even CoT-augmented generative reward models [37, 38, 47] — implicitly treats evaluation as a classification or regression problem: inputs go in, a verdict (or score) comes out, with optional reasoning produced as a byproduct. The model processes the image, question, and responses in a single forward pass and produces its best guess. This paradigm treats the model as a passive assessor that receives evidence and renders judgment.

DeltaRubric fundamentally reorients this framing by arguing (Section 1) that:

"multimodal evaluation should not be treated as a passive scoring task, but rather an active reasoning process."

This is not merely rhetorical repositioning — it has concrete architectural and training implications. An active reasoner must decide what to investigate before investigating it. This introduces an explicit planning phase that does not exist in the passive paradigm. In the passive view, a model that produces a plausible-sounding rationale without consulting the image has succeeded in producing a well-formed output but failed at its actual task. In the active investigation view, such behavior is structurally impossible because the evaluation pipeline is decomposed: the model must externalize its investigative agenda (the checklist) before it can execute that agenda (the verification step). If the checklist is vacuous, the subsequent verification has nothing to ground itself in; if the checklist is specific and targeted, the verification step is forced to engage with the image to resolve each item.

Why this is distinctive rather than incremental. The field has recognized lazy judging as a problem [50] and has developed various mitigations: better prompting, chain-of-thought, process reward models, and rubric-based decomposition. But all existing mitigations operate within the passive scoring paradigm — they change how the model arrives at a verdict, not the fundamental structure of the evaluation process. Chain-of-thought adds reasoning tokens but does not structurally prevent the model from hallucinating that reasoning. Rubrics tell the model what to check but don't force the model to actually perform the checks. DeltaRubric's reframing makes visual investigation a necessary intermediate computation rather than an optional reasoning step, creating an architectural enforcement of grounded evaluation that goes beyond prompt engineering.

This insight connects to broader trends in AI where decomposing complex tasks into explicit sub-goals improves reliability — think of tool-use agents that must produce an API call before receiving results, or retrieval-augmented models that must query a database before generating. The common thread is that forcing the model to externalize its information needs as explicit intermediate outputs creates a dependency that is harder to bypass than a purely end-to-end mapping.

Innovation 2: Disagreement-Focused Rubrics as a Dynamic, Instance-Specific Alternative to Static Evaluation Criteria

The paper identifies a specific failure mode in prior rubric-based evaluation approaches that extends beyond the obvious "generic rubrics are less helpful than specific ones." The deeper insight is that the type of specificity matters: rubrics should not be fine-grained versions of general quality criteria (e.g., "check attribute binding" vs. "check if the tissue box color matches"), but rather should be synthesized from the disagreement structure between the candidate responses themselves.

To understand why this is a conceptual contribution rather than an implementation detail, consider what static rubrics do. A static rubric — whether human-designed [15, 25] or model-generated from the question alone [16] — enumerates dimensions of quality that are relevant to the task in general. For image description, the rubric might include "checks for object hallucinations," "verifies spatial relationships," "assesses completeness." This is useful because it reminds the model to attend to these dimensions, but it suffers from a fundamental limitation: it does not tell the model which specific objects, relationships, or attributes are contested in this particular comparison. The model must still independently discover that Response A claims cars are present while Response B does not, and then independently decide to check for cars in the image. The entire cognitive burden of identifying the locus of disagreement falls on the Verifier, which is exactly where lazy judging shortcuts can occur.

DeltaRubric's innovation is to make the disagreement itself the object of analysis. The Planner does not ask "what qualities matter for image description?" but rather "where do these two responses make conflicting claims about the visual world?" The resulting checklist items — "Is there no visible car traffic on the street, only pedestrians and a bus?" (Figure 3), "Does the tissue box have a green exterior or cover?" (Figure 6) — are not generic evaluation criteria. They are forensic hypotheses about specific visual facts that will discriminate between the two responses. Each item is a binary question whose answer, once determined from the image, directly implicates which response is more accurate.

Comparison to prior rubric work. The closest prior approach is AutoRubric-R1V [16], which generates rubrics for multimodal reasoning but uses a "disjointed architecture prone to cascading errors" (as the paper describes in Section 2) where rubric generation and evaluation are separate stages without joint optimization. DeltaRubric's contribution is twofold: (1) the rubrics are disagreement-focused rather than criteria-focused, making them inherently more decision-relevant for preference evaluation, and (2) the Planner and Verifier are jointly optimized so the Planner learns to generate checklists that the Verifier actually finds useful, rather than checklists that merely look plausible.

The evidence for this innovation's importance is in the static-rubric ablation (Table 5). The static rubric baseline (which provides generic evaluation criteria) improves over no-rubric (8B: 75.3 vs. 72.0 Overall), confirming that any structural prior helps. But DeltaRubric's dynamic, disagreement-focused checklists push Overall accuracy substantially further (80.1), with the largest gap appearing in the Reasoning subcategory (72.6 vs. 59.6, a +13.0 point gain). This gap is telling: reasoning tasks involve complex logical or spatial relationships that generic criteria cannot anticipate. Only by reading the specific claims made by each response and identifying their points of conflict can the Planner generate items like "Is the giraffe taking the food from the palm, not from a branch or tree trunk?" (Figure 5), which directly resolves the factual dispute in a way that a generic "check for hallucinated objects" rubric never could.

Innovation 3: Multi-Role RL with Decoupled Advantage Estimation as a Mechanism for Symbiotic Capability Development

The paper's training methodology is not merely an application of GRPO to a two-step pipeline — it represents a principled solution to the alignment problem between a planner and an executor that share parameters. The challenge is subtle: if the Planner and Verifier share a single model and are trained with a single unified objective, how do you ensure that each capability improves for the right reasons?

The naive approach would be to concatenate the Planner and Verifier trajectories into a single long sequence and compute a single advantage over the entire sequence. This fails because it conflates two distinct sources of variance: (1) whether the checklist was good (Planner quality), and (2) whether the verification was executed correctly given that checklist (Verifier quality). A correct verdict could result from a great checklist + sloppy verification, or from a mediocre checklist + excellent verification. A single advantage cannot disentangle these contributions, so the policy receives ambiguous gradients — should it increase probability of the checklist tokens, the verification tokens, or both?

The paper's solution, decoupled advantage estimation (Section 3.3), is conceptually elegant: compute separate advantages for Planner and Verifier rollouts within their respective task groups, then sum the losses. This means:

  • The Planner advantage depends only on whether the checklist improved verdict accuracy over the no-checklist baseline (Equation 1). A checklist that produces a correct verdict when the baseline was wrong receives +1 advantage regardless of whether the Verifier's reasoning trajectory for that checklist was eloquent or clumsy. The Planner is rewarded purely for checklist utility.

  • The Verifier advantage depends only on whether the verification trajectory produced the correct verdict, with a bonus for beating the baseline (Equation 2). The Verifier is rewarded for accuracy and checklist-groundedness, but its advantage is computed relative to other Verifier trajectories for the same checklist — meaning the Verifier learns to execute a given checklist well, not to compensate for poor checklists.

Why this is distinctive rather than standard multi-task RL. Standard multi-task RL (e.g., training a model on multiple tasks by summing losses) does not solve the credit assignment problem. If the Planner and Verifier losses are simply added without decoupled advantage estimation, the gradient for the Planner's checklist tokens would also depend on the Verifier's execution quality — a good checklist might receive a low advantage because the Verifier happened to produce a noisy trajectory for unrelated reasons. Decoupled advantage estimation isolates each task's signal to its own output tokens, creating a symbiotic but independent relationship: the Planner gets credit for helping the Verifier, and the Verifier gets credit for executing well, but neither gets penalized for the other's failures.

The evidence for this innovation's importance is distributed across multiple results. The Planner optimization ablation (Table 3) shows that freezing the Planner (no RL training) hurts Reasoning by -6.3 points compared to the fully trained system, confirming that the Planner's learning signal is essential. The Planner reward formulation ablation (Table 4) shows that using absolute rather than relative reward — which would conflate Planner and Verifier contributions by rewarding the Planner even on easy examples — degrades performance. And the sensitivity analysis (Table 8) shows that the Verifier's guidance bonus λ must be carefully tuned: too low (0.0) and the Verifier ignores the checklist, too high (0.6) and the Verifier over-optimizes for checklist adherence at the expense of reasoning. These results collectively demonstrate that the decoupled, calibrated reward structure is not incidental to performance — it is load-bearing.

This innovation has broader implications beyond this specific paper. It suggests a general template for training models that must both plan and execute: identify an intermediate artifact whose quality can be independently evaluated (here, the checklist's ability to improve verdict accuracy), define separate task-specific rewards that isolate each capability's contribution, and compute advantage estimates within homogeneous groups to prevent cross-task variance. This template could apply to any setting where a model must generate its own instructions, plans, or sub-goals before executing them — from hierarchical RL to tool-use agents to self-improving reasoning systems.

Innovation 4: Empirical Demonstration That Multimodal Fine-Tuning Can Actively Improve Text-Only Reasoning

This is a negative-result-turned-positive-finding that carries significant implications for how the field thinks about catastrophic forgetting in multimodal training. The default expectation — grounded in substantial prior evidence — is that fine-tuning an MLLM on multimodal data degrades its text-only capabilities. The visual modality introduces new parameters (the vision encoder, cross-attention, etc.) and new training objectives that can overwrite or interfere with the representations learned during language pretraining.

The paper's finding on the text-only RewardBench (Table 6) contradicts this expectation in a specific and instructive way. The base Qwen3-VL-8B-Instruct achieves 81.4 Overall accuracy on RewardBench — already strong text-only performance. The no-rubric multimodal fine-tuning baseline (which applies standard preference optimization without rubrics) roughly preserves this (81.9 Overall), which is itself a positive result: multimodal training did not cause catastrophic forgetting. But DeltaRubric goes further: it improves text-only performance to 84.6 Overall, with the largest gain in the Reasoning subset (+6.7 points, from 79.0 to 85.7).

What makes this a conceptual innovation rather than just a nice empirical result is the hypothesized mechanism. The paper argues (Section 4.3) that:

"disagreement-driven checklist generation encourages the model to decompose evaluations into structured, verifiable reasoning steps, benefiting reasoning even without visual input."

In other words, the Planner-Verifier decomposition teaches the model a domain-general reasoning skill — the ability to identify points of disagreement, formulate testable hypotheses about those disagreements, and then execute those tests — that transfers to text-only settings. The Planner learns to read two text passages, locate where they make conflicting claims, and generate neutral verification questions. The Verifier learns to answer those questions by consulting available evidence (which in the text-only case is the model's own knowledge or the provided context). These skills are modality-agnostic: whether the evidence comes from pixels or from internal knowledge, the process of decompose-then-verify is the same.

Why this matters beyond the result itself. The finding challenges the implicit assumption in multimodal training that language and vision capabilities are in zero-sum competition. If training on multimodal evaluation tasks with structural decomposition actually improves text-only reasoning, it suggests that the bottleneck in text-only evaluators is not knowledge or language understanding, but rather the habit of structured verification. Text-only models can reason well when prompted to do so (as the base model's 81.4 Overall accuracy shows), but they may not spontaneously apply the kind of systematic, disagreement-driven decomposition that DeltaRubric enforces. The multimodal training teaches this habit, and the habit transfers.

This has practical implications for model development: it suggests that multimodal evaluation training could serve as a reasoning curriculum that strengthens capabilities applicable to text-only tasks, potentially reducing or eliminating the tradeoff between multimodal and unimodal performance that has historically complicated MLLM post-training. It also raises a deeper question about the nature of reasoning: to what extent is "good reasoning" about learning to externalize and test hypotheses, independent of the modality of the evidence? If DeltaRubric's text-only improvement is replicable across other models and benchmarks, it would suggest that structured evaluation is a transferable meta-skill that can be taught in one domain and applied in another.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper evaluates on two multimodal reward modeling benchmarks and one text-only benchmark. The primary multimodal benchmark is VL-RewardBench[21], described as an "out-of-domain set designed to probe robustness to common failure modes such as visual hallucinations and spatial reasoning errors." It reports accuracy across General, Hallucination, and Reasoning subcategories, plus Overall and Macro Average (mean of subcategory accuracies). The secondary multimodal benchmark is Multimodal RewardBench[42], which "evaluates general vision-language preference alignment" across eight subcategories: General, Knowledge, Reasoning, Safety, VQA, Correctness, Preference, Math, and Coding, reporting Overall and subcategory accuracy. The text-only benchmark is RewardBench[20], which evaluates language-only reward models across Chat, Hard, Safety, and Reasoning subsets. The training data is 30K randomly sampled instances from the RLAIF-V dataset [45], with strict decontamination to ensure zero overlap with evaluation sets. All evaluation follows the protocols established by the respective benchmark creators.

Base model(s). All experiments use Qwen3-VL-4B-Instruct and Qwen3-VL-8B-Instruct[1] as the starting checkpoints. The choice is motivated by practical considerations: these are open-weight models representing the state-of-the-art in their size class at the time of experimentation. The paper also evaluates several external open-source models for comparison: VITA-1.5-7B [10], SliME-7B/8B [46], Molmo-7B [8], MM-RLHF-Reward-7B [48], InternVL2-8B [4], InternVL3-8B [54], LLaVA-Critic-8B [40], Llama-3.2-11B and Llama-3.2-90B [11], and NVLM-D-72B [6]. These serve as reference points but are not directly comparable since they use different base architectures, training data, and training procedures.

Metrics. The primary metric across all benchmarks is accuracy — the fraction of test instances where the model's predicted preference verdict matches the ground-truth preference. For VL-RewardBench, accuracy is computed via greedy decoding and reported per subcategory (General, Hallucination, Reasoning), as Overall accuracy (across the full dataset), and as Macro Average (mean of all subcategory accuracies). For Multimodal RewardBench, Overall and subcategory accuracies are reported following the evaluation protocol of [42]. For the text-only RewardBench, Overall and subset accuracies (Chat, Hard, Safety, Reasoning) are reported. All evaluations use the grading procedures defined by the respective benchmark papers.

Baselines. The paper defines two controlled baselines, both using the same Qwen3-VL base models:

  • Zero-shot base model: The off-the-shelf Qwen3-VL Instruct model prompted to act as a judge without any RL fine-tuning. This represents the model's capability before any training.
  • No-rubric: The same base model fine-tuned via standard RL (GRPO) to generate a CoT rationale followed by a verdict, without any checklist or rubric structure. This represents the standard generative reward modeling paradigm where the model produces $(r, z) \sim \pi_\theta(\cdot \mid x)$ — the model generates reasoning tokens and a verdict in one unified trajectory, but without the Planner-Verifier decomposition.

Additionally, the paper includes a static-rubric baseline in the ablation studies (Table 5): the RL-fine-tuned Verifier is conditioned on a fixed, dataset-level set of generic evaluation criteria (see Appendix for the exact rubric: "Directly answers the question using the information relevant to the image," "Makes factual claims that are consistent with the image and avoids unsupported details," "Correctly identifies important visual information when it matters for the question," "Uses sound reasoning and logical inference where needed," "Gives a clear and complete answer") rather than a dynamically generated checklist. This isolates the value of instance-specific rubric synthesis from the general benefit of having any evaluation structure.

For broader context, the paper also reports evaluated numbers from several external open-source models (listed above under Base models). These are not directly controlled baselines but provide reference points for the absolute performance levels of DeltaRubric-trained models.

Generation budget / compute accounting. The paper accounts for training compute through the sampling budget: for each training example, the Planner generates N = 5 candidate checklists, and the Verifier generates M = 5 full reasoning trajectories. Each checklist is scored via a "cheap probe" Verifier forward pass (verdict only, no reasoning), and the best checklist is selected greedily for full Verifier training. Training runs for 120 steps with a global batch size of 128. Inference compute is not explicitly budgeted or compared across methods — the paper focuses on accuracy improvements rather than compute-matched comparisons. This is a notable omission: DeltaRubric requires two sequential forward passes at inference time (Planner then Verifier) versus one for the no-rubric baseline, and the paper does not quantify whether the accuracy gains justify the additional inference cost in latency-sensitive applications.

Cross-validation / statistical protocol. The paper does not employ cross-validation or statistical significance testing for its main results. The training data is strictly decontaminated from evaluation sets, which prevents train-test leakage, but there is no mechanism to assess the variance of the reported results (no confidence intervals, no multiple training runs with different random seeds). The composition of the evaluation sets is not detailed — VL-RewardBench and Multimodal RewardBench are external benchmarks with presumably fixed test splits, but the paper does not report their sizes. This makes it difficult to assess whether observed differences (e.g., +4.3 points for 4B on VL-RewardBench) are statistically reliable or within the noise floor of these benchmarks.

Main Quantitative Results

VL-RewardBench Results: DeltaRubric Substantially Outperforms Both Zero-Shot and No-Rubric Baselines

The headline result on VL-RewardBench (Table 1) is that DeltaRubric improves the base Qwen3-VL-4B-Instruct model's Overall accuracy from 54.9 (zero-shot) to 77.5 (+22.6 points), and the 8B model from 61.3 to 80.1 (+18.8 points). These are large absolute improvements that move the models from below to well above the best previously reported open-source results (Llama-3.2-90B at 56.2 Overall, MM-RLHF-Reward-7B at 50.2).

More informative is the comparison against the no-rubric baseline, which isolates the contribution of DeltaRubric's Planner-Verifier decomposition from the general benefits of RL fine-tuning on preference data. For the 4B model, the no-rubric baseline achieves 73.2 Overall; DeltaRubric pushes this to 77.5, a gain of +4.3 points. For the 8B model, the no-rubric baseline achieves 72.0 Overall; DeltaRubric reaches 80.1, a gain of +8.1 points. These gains are not uniform across subcategories:

  • General: 4B shows +3.4 over no-rubric (55.3 vs. 51.9); 8B shows +3.9 (59.7 vs. 55.8).
  • Hallucination: Both models show small gains (+0.6 for 4B, +2.2 for 8B), but this subcategory is already high under the no-rubric baseline (87.1 for 4B, 86.1 for 8B), suggesting a ceiling effect.
  • Reasoning: This is where DeltaRubric shows its largest relative gains. The 4B model improves from 50.8 to 65.9 (+15.1 points over no-rubric), and the 8B model improves from 48.3 to 72.6 (+24.3 points over no-rubric). This disproportionate improvement in reasoning — the subcategory most demanding of structured visual verification — is the strongest evidence that the Planner's disagreement-focused checklists provide value beyond generic RL training.

The pattern across subcategories reveals something important about where DeltaRubric helps. The Hallucination subcategory likely involves fairly explicit visual contradictions (an object either is or isn't present), which even a no-rubric model can often catch with sufficient RL training. The Reasoning subcategory involves more complex inferences — spatial relationships, logical deductions, counting — where identifying what to check is nontrivial. The Planner's checklist decomposes these complex reasoning tasks into a sequence of simple visual verification steps, which dramatically improves the Verifier's accuracy.

Looking at external models in Table 1 provides context for DeltaRubric's absolute performance. The best previously reported open-source model on VL-RewardBench is Llama-3.2-90B at 56.2 Overall and 53.9 Macro Avg — a model with approximately 11x more parameters than DeltaRubric's 8B variant. DeltaRubric-8B achieves 80.1 Overall and 73.5 Macro Avg, substantially exceeding this much larger model. However, this comparison is confounded by differences in base model architecture, pretraining data, and training procedures — it demonstrates DeltaRubric's strong absolute performance but does not isolate the DeltaRubric framework's contribution.

The Macro Average metric (mean of subcategory accuracies) penalizes methods that have imbalanced performance across subcategories. DeltaRubric-8B achieves 73.5 Macro Avg versus 63.4 for no-rubric, a +10.1 point improvement — larger than the +8.1 Overall improvement — indicating that DeltaRubric not only improves average accuracy but also makes performance more balanced across subcategories. The no-rubric baseline has a large gap between Hallucination (86.1) and Reasoning (48.3), while DeltaRubric narrows this gap to Hallucination 88.3 vs. Reasoning 72.6.

Multimodal RewardBench Results: Consistent Gains Across a Broader Set of Subcategories

On Multimodal RewardBench (Table 2), the 8B model shows more modest but consistent gains: Overall accuracy improves from 67.7 (zero-shot base) to 73.2 with DeltaRubric, compared to 68.7 for the no-rubric baseline — a +4.5 point DeltaRubric advantage over no-rubric. The 4B model shows similar patterns: 69.1 Overall for DeltaRubric vs. 66.4 for no-rubric (+2.7 points).

The subcategory breakdown reveals which visual tasks benefit most from checklist-guided verification:

  • VQA: The 8B model improves from 71.4 (zero-shot) to 84.9 with DeltaRubric, compared to 77.0 for no-rubric (+7.9 points over no-rubric). This is the largest subcategory gain and makes intuitive sense: VQA tasks require checking specific visual claims against the image, which is exactly what the disagreement-focused checklist enforces.
  • Safety: The 8B model reaches 93.3 with DeltaRubric vs. 91.5 for no-rubric (+1.8 points). The ceiling is high here, limiting room for improvement.
  • Reasoning: The 8B model reaches 69.5 vs. 56.7 for no-rubric (+12.8 points), again showing that structured decomposition disproportionately helps complex inference.
  • Correctness: The 4B model shows an interesting pattern — DeltaRubric at 52.0 is slightly below no-rubric at 55.0 (-3.0 points). This is the only subcategory where DeltaRubric underperforms the no-rubric baseline, and the paper does not discuss it. It may indicate that on correctness-oriented tasks (which might involve verifying factual knowledge rather than visual details), the checklist structure occasionally adds unnecessary complexity or misdirects the Verifier.
  • Math and Coding: Both show large gains for both model sizes (4B: 80.8 vs. 71.3 no-rubric; 8B: 84.9 vs. 77.0 no-rubric). These tasks may benefit from the structured decomposition even when the "visual" component is minimal, consistent with the finding that checklist-based reasoning improves reasoning quality in general.

The comparison against external models in Table 2 shows that DeltaRubric-8B at 73.2 Overall trails InternVL3-8B at 63.6 and Llama-3.2-90B at 61.2, but direct comparison is confounded by different base models, training data, and evaluation prompts. The within-model comparison (no-rubric vs. DeltaRubric) is the cleanest measure of DeltaRubric's contribution.

Training Dynamics: Planner and Verifier Improve Together

Figure 2 shows the training trajectories for the 8B model, revealing how the Planner and Verifier capabilities co-evolve:

  • Figure 2a (Verifier training accuracy): Both DeltaRubric and the no-rubric baseline improve over the 120 training steps, but DeltaRubric consistently achieves higher mean accuracy. The gap widens in the later stages of training, suggesting that as the Planner learns to generate better checklists (see Figure 2c), the Verifier benefits increasingly from the improved guidance.
  • Figure 2b (Verifier validation accuracy, evaluated every 5 steps): The validation curve shows a similar pattern, confirming that the training accuracy improvements generalize to held-out data. The no-rubric baseline appears to plateau earlier, while DeltaRubric continues to improve, indicating that the Planner's ongoing learning provides a sustained advantage.
  • Figure 2c (Planner probe accuracy): This metric measures "the fraction of sampled checklists that successfully guide a lightweight verdict probe to the correct ground-truth winner." It serves as a proxy for checklist quality. The steady upward trajectory from approximately 0.55 to approximately 0.65 over 120 steps confirms that the Planner is learning to generate checklists that are increasingly decision-useful. The improvement in Verifier accuracy (Figures 2a and 2b) correlates with Planner improvement, consistent with the hypothesis that better checklists cause better verification.

The paper does not report whether the Planner and Verifier improvements are causally linked or merely correlated — for example, does freezing the Planner at step 60 cause Verifier accuracy to plateau? The frozen-Planner ablation (Table 3) addresses this for the final model but not for the training dynamics.

Text-Only RewardBench: Multimodal Training Improves Language-Only Reasoning

Table 6 shows results on the text-only RewardBench [20], which serves as both a control for modality effects and a check for catastrophic forgetting. The base Qwen3-VL-8B-Instruct starts at 81.4 Overall, which is strong text-only performance despite being a vision-language model. The no-rubric multimodal fine-tuning baseline achieves 81.9 — essentially unchanged, confirming that multimodal RL training on preference data does not degrade text-only capabilities. DeltaRubric improves Overall to 84.6, a gain of +3.2 points over the base model and +2.7 over no-rubric.

The subcategory breakdown is revealing:

  • Chat: Minimal changes (90.8 → 91.3), near ceiling.
  • Hard: Small improvement (70.8 → 71.9).
  • Safety: Modest improvement (86.8 → 89.1).
  • Reasoning: The largest gain: 79.0 → 85.7 (+6.7 points over the base model, +5.2 over no-rubric).

This pattern mirrors the multimodal results: DeltaRubric's largest gains are in Reasoning, the subcategory that most benefits from structured decomposition. The paper's interpretation is that the Planner-Verifier training teaches a domain-general skill — decomposing evaluations into structured, verifiable steps — that transfers to text-only settings. The fact that text-only Reasoning improves more than multimodal Hallucination detection (which relies on visual perception specifically) supports the transfer hypothesis.

This finding is notable because it contradicts the common concern that multimodal fine-tuning causes catastrophic forgetting of language capabilities. Not only is there no forgetting, but there is active improvement in reasoning — a capability that is arguably more foundational than the specific visual grounding skills the model was directly trained on.

Qualitative Evidence: Checklists Catch Hallucinations That No-Rubric Baselines Miss

Figure 3 (and the additional examples in Appendix A.5, Figures 4–6) provides qualitative evidence for the mechanism by which DeltaRubric improves over the no-rubric baseline. In the Figure 3 example (question: "Describe this image in detail"), Response A hallucinates "cars driving on the road," while Response B correctly describes only pedestrians and a bus. The no-rubric baseline produces analysis that incorrectly claims "Response A avoids adding unsupported details" and selects Response A. DeltaRubric's Planner generates a targeted checklist: "Is there no visible car traffic on the street, only pedestrians and a bus?" The Verifier executes this check, finds that "No cars are visible; only pedestrians and a bus," and correctly selects Response B.

The key qualitative insight is not just that DeltaRubric gets the answer right — it's how it gets it right. The no-rubric baseline's analysis is plausible-sounding but factually wrong; it claims Response A "correctly describes the scene without fabricating elements like cars," which directly contradicts the image. This is a textbook example of lazy judging: the model produces fluent, confident analysis that never actually verifies the contested claims against the image. DeltaRubric's checklist prevents this failure mode by forcing explicit visual verification of the specific claim in dispute. The Verifier cannot produce a plausible-sounding analysis that ignores the "cars" issue because the checklist item directly asks about car presence.

The additional examples in Appendix A.5 reinforce this pattern:

  • Figure 4: The no-rubric baseline misidentifies shoe color (claiming shoes are "white with black laces" when they are black with white soles) and selects the wrong response. DeltaRubric's checklist isolates the shoe color as the decisive disagreement and correctly identifies Response A as more accurate.
  • Figure 5: The no-rubric baseline exhibits logical inconsistency — it correctly notes that no tree branch is visible but still selects the response claiming the giraffe is eating from a branch. DeltaRubric's checklist systematically verifies the food delivery mechanism (palm vs. branch) and correctly selects Response B.
  • Figure 6: The no-rubric baseline struggles with attribute binding, confusing the white tissues with the color of the tissue box itself. DeltaRubric's checklist explicitly isolates the "green exterior" of the box, preventing this confusion.

These examples collectively demonstrate that DeltaRubric's improvement is not merely a quantitative accuracy gain but a qualitative change in evaluation behavior — from surface-level textual analysis to grounded visual verification. The checklists function as an external memory that prevents the Verifier from losing track of what needs to be checked, which is particularly valuable when the evidence contradicts the fluent narrative that a lazy judge might otherwise construct.

Ablation Studies and Robustness Checks

Planner optimization (Table 3): Removing RL training from the Planner (using a frozen, zero-shot Planner with greedy decoding while the Verifier is trained independently) degrades VL-RewardBench Overall accuracy from 80.1 to 78.8 (-1.3 points) and Reasoning from 72.6 to 66.3 (-6.3 points) for the 8B model. The frozen Planner still outperforms the base model (78.8 vs. 61.3 Overall), confirming that even static structural guidance helps, but the gap to fully trained DeltaRubric — particularly in Reasoning — demonstrates that the Planner must learn to generate decision-useful checklists through RL. The frozen Planner generates checklists that provide a useful structural prior but lack the instance-specific targeting that the trained Planner develops. This ablation isolates the contribution of Planner RL: it is responsible for a substantial fraction of the Reasoning improvement but only a modest fraction of the Overall improvement, consistent with the idea that Planner training matters most for complex tasks where identifying the right verification targets is nontrivial.

Planner reward formulation (Table 4): Using an absolute reward ($R_{\text{plan}}(c_i) = \mathbb{I}(z_i = z^*)$) instead of the relative reward ($R_{\text{plan}}(c_i) = \mathbb{I}(z_i = z^*) - \mathbb{I}(z_0 = z^*)$) degrades Overall accuracy from 80.1 to 77.6 (-2.5 points) and Reasoning from 72.6 to 69.1 (-3.5 points). The absolute reward formulation gives the Planner positive reinforcement for generating checklists on easy examples where the baseline Verifier would have been correct anyway, leading to noisy gradient updates that do not specifically incentivize blind-spot targeting. This is a concrete validation of the paper's design rationale: the Planner should be rewarded only for marginal improvements, not for baseline performance. The degradation is moderate rather than catastrophic, suggesting that even an imperfectly incentivized Planner can learn somewhat useful checklists (the structural prior still helps), but the relative reward is clearly superior.

Static rubric baseline (Table 5): Conditioning the RL-fine-tuned Verifier on a fixed set of generic evaluation criteria (the static rubric) rather than dynamically generated checklists improves over the no-rubric baseline (8B: 75.3 vs. 72.0 Overall), confirming that any structural prior provides benefit. However, the gap to DeltaRubric is substantial: 8B Overall accuracy drops from 80.1 to 75.3 (-4.8 points), and Reasoning drops from 72.6 to 59.6 (-13.0 points, the largest single ablation gap in the paper). This is the clearest evidence that instance-specific, disagreement-focused checklists provide value beyond generic evaluation criteria. The static rubric tells the Verifier what categories of error to look for; DeltaRubric's checklist tells the Verifier exactly which claims to verify against the image. For complex reasoning tasks, the former is insufficient — the model must still independently identify which specific visual facts are disputed — while the latter provides the precise verification targets needed.

The static rubric result also contextualizes the external model comparisons. The static-rubric 8B model at 75.3 Overall already exceeds all external models in Table 1 except Llama-3.2-90B (56.2). This suggests that even a simple structural prior combined with RL fine-tuning on preference data yields strong performance, and that DeltaRubric's additional gains (+4.8 Overall, +13.0 Reasoning) represent the specific contribution of dynamic, disagreement-focused rubric synthesis.

Text-only planning (Table 7): Removing visual context from the Planner (providing only the question and two responses, not the image) yields an 8B Overall accuracy of 79.1 vs. 80.1 for full DeltaRubric (-1.0 points), with Reasoning dropping from 72.6 to 68.0 (-4.6 points). The text-only Planner still substantially outperforms the base model (79.1 vs. 61.3), demonstrating that much of the checklist's value comes from identifying textual disagreements between responses. However, the drop in Reasoning confirms that seeing the image helps the Planner generate more targeted, context-aware checklist items — for complex visual reasoning, knowing that Response A claims "the tissue box is green" and Response B claims "it is white" is useful, but actually seeing the tissue box helps the Planner formulate checklist items that precisely specify what visual attribute to verify (e.g., "Does the tissue box have a green exterior or cover?" rather than "What color is the tissue box?"). The text-only Planner may produce checklists that are less precisely targeted to the available visual evidence, making the Verifier's job harder.

Guidance bonus coefficient λ sensitivity (Table 8): The Verifier reward includes a bonus term $\lambda \max(0, \mathbb{I}(z_j = z^*) - \mathbb{I}(z_0 = z^*))$ scaled by λ. Testing values {0.0, 0.2, 0.4, 0.6} on the 8B model reveals a non-monotonic relationship: λ = 0.4 achieves the best Overall accuracy (80.1) and Macro Average (73.5). Disabling the bonus (λ = 0.0) reduces Overall to 78.2 (-1.9 points) and Hallucination from 88.3 to 87.9, suggesting the bonus provides a small but consistent incentive for checklist adherence. Setting λ = 0.6 causes a sharp drop in Reasoning (72.6 → 66.3, -6.3 points) while slightly improving General (59.7 → 58.6, -1.1) and maintaining Hallucination (88.3 → 87.3). The Reasoning degradation at high λ is the paper's clearest evidence of over-optimization: when the guidance bonus is too strong, the Verifier becomes overly obedient to the checklist, losing the ability to recognize when checklist items are flawed, incomplete, or need to be overridden based on broader contextual reasoning. At λ = 0.4, the bonus is strong enough to ensure the Verifier attempts to use the checklist but weak enough that it can still exercise independent judgment.

The optimal λ value (0.4) is moderate relative to the base accuracy reward (which is 1.0 when correct), meaning the guidance bonus contributes at most 0.4/(1.0 + 0.4) ≈ 28.6% of the maximum possible reward. This ratio appears to be in the right regime: enough to matter (λ = 0.0 performs worse) but not so much that it dominates (λ = 0.6 degrades Reasoning).

Generalization to alternative RL algorithm (Table 9, Appendix A.3): Using DAPO [43] instead of GRPO as the underlying RL algorithm, DeltaRubric maintains its advantage over baselines: the 8B model achieves 77.9 Overall vs. 74.7 for static-rubric and 74.2 for no-rubric. The gain of +3.7 points over no-rubric is smaller than with GRPO (+8.1 points, Table 1), and the absolute performance (77.9) is lower than with GRPO (80.1), but the qualitative pattern — DeltaRubric > static-rubric > no-rubric — is preserved. This demonstrates that the DeltaRubric framework's benefits are not tied to a specific RL algorithm, but the magnitude of improvement does depend on algorithm choice. The paper does not investigate why DAPO underperforms GRPO in this setting; it may be due to hyperparameter sensitivity, interaction between DAPO's clipping mechanism and the multi-role advantage structure, or simply because GRPO's group-relative baseline is particularly well-suited to the decoupled group structure of DeltaRubric.

Training data scale: The paper does not ablate the amount of training data (30K RLAIF-V instances). This is a notable omission, as it leaves open the question of whether DeltaRubric's gains would persist with substantially less data (important for domains where preference annotations are scarce) or whether they would continue to scale with more data. The RLAIF-V dataset is relatively small (30K) compared to typical RLHF datasets, so the results already demonstrate data efficiency to some degree, but a formal ablation would strengthen this claim.

Number of Planner and Verifier samples (N, M): The paper uses N = 5 candidate checklists and M = 5 verification trajectories per example, but does not ablate these values. Larger N might improve Planner learning by providing more exploration, while larger M might provide more stable Verifier gradients. The choice of 5 appears to be a practical default rather than an empirically optimized value. An ablation showing whether performance saturates at N=5 or continues to improve with larger N would clarify whether the Planner benefits from more extensive exploration.

Checklist length: The Planner is prompted to generate 2–4 checklist items. The paper does not ablate this range — for example, testing whether allowing longer checklists (5+ items) improves or degrades performance (longer checklists might provide more thorough coverage but could also overwhelm the Verifier or introduce more opportunities for Planner errors). The qualitative examples (Figures 3–6) show checklists of exactly the prompted length (2–4 items), suggesting the model generally adheres to the instruction, but the optimal length is unexplored.

Post-generation filtering: The paper mentions that "a post-generation filtering step is applied to further enforce this impartiality" for Planner checklists. The ablation studies do not test the impact of removing this filtering, so its contribution to overall performance is unknown. If the filtering is important (e.g., it removes biased checklists that would prime the Verifier toward a particular answer), then DeltaRubric's performance is partly dependent on this heuristic post-processing step, which would complicate deployment in settings where the appropriate filtering criteria are unclear.

Critical Assessment

The experiments reported in this paper collectively support the central claim that decomposing multimodal evaluation into a disagreement-focused planning step and a checklist-grounded verification step, optimized jointly via multi-role RL, yields more accurate and reliable reward modeling than single-step evaluation. But whether they support the magnitude of the claim — and its generalizability — requires examining what was and was not tested.

The primary empirical claim — improvement over no-rubric baselines — is well-supported within the tested setting. Both VL-RewardBench and Multimodal RewardBench show consistent gains for DeltaRubric over the no-rubric baseline for both model sizes (Tables 1 and 2). The gains are not uniform — they are largest in Reasoning-heavy subcategories and smallest in Hallucination, which makes sense given the hypothesized mechanism — but the overall pattern is robust. The static-rubric ablation (Table 5) demonstrates that instance-specific, disagreement-focused checklists contribute value beyond generic evaluation criteria, and the Planner reward formulation ablation (Table 4) validates the specific design choice of relative over absolute reward. The qualitative examples (Figures 3–6) provide mechanistic evidence that the improvement comes from genuine visual grounding rather than some other confound. These results, taken together, establish that DeltaRubric works, in the sense of improving accuracy on established multimodal reward modeling benchmarks, for the specific base models and training configuration tested.

However, several aspects of the claim require qualification when scrutinized against what the experiments actually demonstrate.

First, the paper compares against a specific no-rubric baseline (GRPO-trained CoT judge) but not against other plausible strong baselines. The no-rubric baseline is a single-step evaluator that generates reasoning and verdict in one trajectory. But the literature contains other approaches that the paper does not compare against: (1) a two-step baseline where the model generates CoT reasoning first, then produces a verdict in a separate forward pass (decoupling reasoning from judgment without explicit checklist structure); (2) a baseline that uses the same two-step Planner-Verifier architecture but with a different intermediate representation — for example, having the Planner generate a summary of disagreements in natural language rather than a structured checklist, which would test whether the checklist format specifically matters or whether any intermediate text artifact provides similar benefits; (3) a majority-voting baseline where multiple sampled CoT judgments are aggregated, which could improve accuracy at the cost of additional compute without any architectural changes. The absence of these baselines makes it difficult to isolate the specific contribution of the checklist structure from the general benefits of any intermediate planning step.

Second, the inference-time compute cost is never accounted for. DeltaRubric requires two sequential forward passes at inference: first the Planner generates a checklist, then the Verifier generates reasoning and verdict conditioned on that checklist. The no-rubric baseline requires one forward pass. The paper reports accuracy gains but never quantifies the inference latency or FLOPs cost of the additional Planner forward pass. For latency-sensitive applications, a +4.3 point accuracy gain might not justify a 2x increase in inference time, especially if a simpler method (e.g., sampling multiple no-rubric judgments and majority voting) could achieve similar accuracy with comparable compute. The paper's framing as a "more reliable" evaluator is valid, but the practical deployment tradeoff between accuracy and latency is unexplored. A fair comparison would include: DeltaRubric with budget $2 \times B$ vs. no-rubric with majority voting over $2 \times B$ independent samples, or DeltaRubric vs. no-rubric with a longer CoT (matching total output tokens rather than number of forward passes).

Third, the single model family and training dataset limit generalizability claims. All experiments use Qwen3-VL 4B and 8B Instruct models, fine-tuned on 30K RLAIF-V instances. The paper does not test on other model families (e.g., LLaVA, InternVL, or proprietary models), other model scales (e.g., 1B, 70B), or other training datasets. The strong text-only RewardBench results (Table 6) provide some evidence of cross-domain transfer, but they use the same base model trained on the same data — they demonstrate transfer from multimodal to text-only evaluation within the same model, not generalizability across model architectures. Whether DeltaRubric would improve a LLaVA-based evaluator or a proprietary API model is unknown. The paper's claim that "decomposing evaluation into structured, verifiable steps leads to more reliable and generalizable multimodal reward modeling" is supported for Qwen3-VL but the "generalizable" part — meaning generalizes across models — is not tested.

Fourth, the benchmarks may not fully capture the lazy judging phenomenon the paper claims to address. VL-RewardBench is "designed to probe robustness to common failure modes such as visual hallucinations and spatial reasoning errors" [21], and the Hallucination subcategory specifically tests whether models can detect factually incorrect visual claims. But detecting a hallucination in a benchmark is not the same as overcoming lazy judging in deployment. A model might learn to detect benchmark-specific hallucination patterns (e.g., looking for object presence/absence contradictions) without genuinely developing the active visual investigation capability the paper claims. The paper's qualitative examples are compelling but are selected post-hoc (they illustrate successes, not random samples). A systematic analysis of failure modes — cases where DeltaRubric still fails, and whether those failures resemble lazy judging or some other error type — would provide stronger evidence that the mechanism works as hypothesized, but such analysis is absent.

Fifth, the experimental results reveal an asymmetry that the paper does not fully address. The gains on VL-RewardBench are large (+4.3 to +8.1 points over no-rubric for Overall accuracy), while the gains on Multimodal RewardBench are more modest (+2.7 to +4.5 points). The paper does not analyze why the improvement differs across benchmarks. One hypothesis: VL-RewardBench places greater emphasis on visual hallucination detection and spatial reasoning, where disagreement-focused checklists provide the most value, while Multimodal RewardBench includes subcategories (Knowledge, Math, Coding) where visual grounding is less critical and text-based reasoning dominates. If this hypothesis is correct, it would mean DeltaRubric's benefits are proportional to the visual grounding demands of the task — a condition on the claim that the paper states qualitatively but does not quantify.

Sixth, the Planner training relies on a cheap probe whose relationship to full verification is not analyzed. The Planner reward (Equation 1) uses a cheap-probe Verifier that produces only verdicts without reasoning. But the Planner's ultimate value is measured by whether its checklists improve the full Verifier's accuracy (as in the actual evaluation pipeline). If the cheap probe and full Verifier have systematically different failure modes — for example, the cheap probe might be more susceptible to checklist bias because it doesn't generate reasoning, making checklists appear more helpful than they actually are — then the Planner could be optimizing for a proxy that doesn't align with downstream performance. The paper does not compare cheap-probe accuracy against full-Verifier accuracy or analyze cases where they disagree, leaving open the possibility that the Planner is learning to generate checklists that help the cheap probe without helping (or even hurting) the full Verifier. The strong final performance suggests this is not a severe problem, but the mechanism is underexplored.

Seventh, the paper does not report the inference-time behavior of the Planner — specifically, whether the Planner ever generates biased, irrelevant, or hallucinated checklists, and how frequently. The post-generation filtering step is mentioned but not detailed, and its effectiveness is not measured. If the Planner generates flawed checklists on, say, 5% of examples, and the Verifier's instruction to "ignore any checks that are hallucinated or contradicted by the image" successfully mitigates this, then the system is robust. But if the Verifier sometimes trusts flawed checklists (as the λ = 0.6 results suggest can happen), then the Planner's error rate matters. No such analysis is provided.

What experiments would have strengthened the paper?

  1. A compute-matched comparison: Compare DeltaRubric (2 forward passes) against no-rubric with 2x sampling and majority voting, or no-rubric with 2x longer CoT, to determine whether the architectural innovation provides benefits beyond simply spending more compute at inference.

  2. A Planner-only ablation: Test whether the Planner's checklists, when provided to a frozen Verifier (no RL training), improve that Verifier's accuracy. This would isolate the checklist's intrinsic value from the Verifier's adaptation to checklist-guided reasoning. If checklists help even without Verifier fine-tuning, the Planner's contribution is robust; if they only help when the Verifier is co-trained, the benefit is more fragile.

  3. A Verifier-only ablation: Train the Verifier on human-written checklists (or oracle checklists derived from ground-truth annotations) to see the performance ceiling for checklist-guided verification with perfect planning. This would bound the Planner's contribution and indicate how much room remains for Planner improvement.

  4. Systematic error analysis: Categorize DeltaRubric's remaining errors — are they Planner errors (checklist missed critical disagreement), Verifier errors (checklist item was correctly formulated but incorrectly executed), or fundamental failures (even with perfect checklist and execution, the ground-truth preference is ambiguous)? This would identify the bottleneck for future improvement.

  5. Scaling the training data: Test performance at 5K, 10K, 30K, and 60K training instances to determine whether DeltaRubric's advantage over no-rubric increases, decreases, or remains constant with data scale. This would inform whether the approach is particularly valuable in low-data regimes (where structural priors matter more) or high-data regimes (where the Planner has enough data to learn sophisticated checklist generation).

  6. Confidence calibration: Report whether DeltaRubric's judgments are better calibrated than the no-rubric baseline — that is, whether the model's confidence in its verdicts (measurable via token probabilities or multiple samples) better predicts actual correctness. A more reliable evaluator should not only be more accurate but should also know when it's likely to be wrong.

In summary, the experiments convincingly demonstrate that DeltaRubric improves multimodal reward model accuracy over a no-rubric GRPO baseline for Qwen3-VL models on two established benchmarks, with the largest gains in reasoning-heavy subcategories. The ablation studies isolate the contributions of Planner RL training, relative reward formulation, and instance-specific checklist generation. However, the paper overclaims in implying that these results demonstrate a general principle ("decomposing evaluation into structured, verifiable steps leads to more reliable and generalizable multimodal reward modeling") when the evidence is limited to one model family, one training dataset, and two benchmarks, without compute-matched comparisons, systematic error analysis, or investigation of the Planner's failure modes. The mechanism — that checklists force visual grounding — is supported qualitatively and through the pattern of subcategory gains, but the quantitative contribution of each component (checklist structure vs. two-step decomposition vs. multi-role RL) is not fully isolated. The paper's contributions are real but narrower than its framing suggests: it introduces and validates a specific training recipe (disagreement-focused Planner + checklist-grounded Verifier + decoupled multi-role GRPO) that improves Qwen3-VL evaluators on multimodal reward modeling benchmarks, with evidence that the improvement comes partly from better visual grounding.

6. Limitations and Trade-offs

6.1 Inference-Time Cost Doubles Without Accounting in Headline Comparisons

The constraint. DeltaRubric requires two sequential forward passes at inference: first the Planner generates the checklist, then the Verifier generates reasoning and a verdict conditioned on that checklist. The no-rubric baseline requires a single forward pass. The paper's headline accuracy comparisons (Tables 1, 2, 5) treat these as equal-cost alternatives — they compare DeltaRubric's two-pass inference against the no-rubric baseline's one-pass inference without any compute normalization. The paper never quantifies inference latency, throughput, or FLOPs, and the evaluations use greedy decoding for both methods, meaning DeltaRubric generates strictly more tokens (the checklist plus the verification reasoning) than the baseline (verification reasoning only).

The consequence. A practitioner choosing between DeltaRubric and a no-rubric baseline cannot determine from the paper's results whether the accuracy gains justify the additional inference cost. The +4.3 point gain for the 4B model on VL-RewardBench (77.5 vs. 73.2 Overall, Table 1) and the +8.1 point gain for the 8B model (80.1 vs. 72.0 Overall, Table 1) might be achievable with similar or better efficiency by simply running the no-rubric baseline twice with majority voting, or by allowing the no-rubric baseline to generate a longer chain-of-thought (matching total output tokens rather than forward passes). For latency-sensitive applications (interactive evaluation, real-time reward scoring during RLHF training), a 2× increase in per-sample latency could be prohibitive regardless of accuracy. There is no evidence in the paper about whether DeltaRubric's accuracy-at-fixed-total-compute is superior to simpler methods that spend the same compute budget differently.

What evidence exists in the paper. None. The paper does not include a compute-matched comparison, does not report inference times or token counts for Planner vs. Verifier generations, and does not compare DeltaRubric against a no-rubric baseline with matched total generation budget (e.g., no-rubric with 2× sampling or 2× longer reasoning). The training cost accounting (N=5 Planner rollouts, M=5 Verifier rollouts per example, Section 3.3) suggests the authors are aware of compute budgets, but this awareness is not extended to the inference-time evaluation.

Mitigation status. Not addressed. The paper does not discuss inference cost as a limitation, does not propose any mechanism for reducing Planner cost (e.g., using a smaller model for checklist generation, caching checklists for similar queries, or adaptive triggering where the Planner is only invoked when initial confidence is low), and does not flag this as an area for future work in Section A.4. The open question is whether the two-pass architecture is fundamentally necessary for the observed gains, or whether a single-pass model with extended reasoning could internalize the plan-then-execute dynamic without the explicit intermediate artifact and its associated latency cost.

6.2 Difficulty Estimation: Checklists May Not Help — or May Actively Hurt — on Easy Examples, Yet Are Always Generated

The structural assumption. DeltaRubric generates a checklist for every evaluation instance, regardless of whether the Verifier needs it. The Planner reward (Equation 1) is designed to assign zero reward to checklists on examples where the baseline was already correct (R_plan = 0 when z_0 is correct), so the Planner is not positively reinforced for generating useless checklists. However, the Planner is never penalized for generating a checklist on an easy example (as long as it doesn't mislead the Verifier into an error), and there is no mechanism to skip checklist generation when it's unnecessary.

The consequence. On a distribution of evaluation queries that skews toward easy examples (where the baseline Verifier already performs well), the Planner still incurs full inference cost while providing zero or near-zero marginal benefit. More concerning is the possibility that checklists could degrade performance on some fraction of easy examples: a poorly formulated checklist item — one that is ambiguous, misleading, or focuses the Verifier's attention on a non-decisive detail — could flip a correct baseline verdict to incorrect. The Planner reward in Equation 1 includes a -1 penalty when z_i ≠ z* while z_0 = z*, but the Planner sees this penalty only during training; at inference time, a single checklist is sampled (greedy decoding, Section 3.3), and if that checklist happens to be misleading, the error propagates. The paper does not quantify how often the Planner generates checklists that reduce accuracy relative to the no-checklist baseline.

The implicit assumption is that the Planner's training reliably converges to generating neutral, helpful checklists on all examples, and that the downward-trending validation loss in Figure 2b implies decreasing rates of harmful checklists. But this is an extrapolation, not a demonstrated property: the validation accuracy metric in Figure 2b measures average correctness across the Verifier's outputs given the best checklist, which would mask cases where a fraction of checklists are harmful if the majority are helpful. A Verifier that correctly ignores bad checklists (as instructed by the prompt) would also mask Planner errors, but that same instruction — "ignore any checks that are hallucinated or contradicted by the image" (Appendix A.6) — places the burden of checklist validation on the Verifier, whose ability to do so is not independently assessed.

What evidence exists in the paper. The paper provides suggestive but incomplete evidence. The Planner probe accuracy in Figure 2c shows the fraction of sampled checklists that guide the cheap-probe Verifier to the correct answer; this metric rises from ~0.55 to ~0.65 over training, meaning at the end of training, roughly 35% of sampled checklists still fail to guide the cheap-probe Verifier to the correct answer. This is the best available estimate of Planner error rate, but it has two limitations: (1) the cheap-probe Verifier is not the same as the full Verifier, so "failure to guide the cheap probe" may overestimate the actual harm to the full pipeline (the full Verifier could compensate via its independent reasoning, as instructed); (2) the probe accuracy measures whether the checklist helps reach the correct answer, not whether it prevents a correct answer — a checklist that is merely unhelpful (adds no information) would have the same probe accuracy outcome as a checklist that is actively misleading (causes an error), as long as both produce the wrong verdict. The paper does not decompose Planner failures into "benign" (zero reward) and "harmful" (negative reward) categories, making it impossible to assess the risk of checklist-induced errors.

The frozen-Planner ablation (Table 3) provides indirect evidence: the frozen Planner achieves 78.8 Overall vs. the trained Planner's 80.1, a difference of -1.3 points. Since the frozen Planner's checklists are presumably on average less helpful, the small gap could indicate that Verifier robustness (ability to ignore bad checklist items) limits the damage from Planner errors. But this is speculative; the paper does not directly measure Verifier robustness to Planner errors.

Mitigation status. Partially acknowledged. The Verifier prompt includes the instruction "If a checklist item is irrelevant, too vague, or contradicted by the image/question, ignore that item" (Appendix A.6), which is a robustness mechanism. The post-generation filtering step (Section 3.2) attempts to remove biased checklists before they reach the Verifier. However, neither mechanism is ablated or measured — there is no experiment showing how often the Verifier correctly ignores bad checklist items, how often it incorrectly ignores good items, or how effective the post-filtering is at catching erroneous checklists. The paper's future work discussion (Appendix A.4) mentions "employing dynamic routing to trigger checklist generation only for highly ambiguous cases" as a potential direction, which implicitly acknowledges the cost of always-on checklist generation but does not address the risk of checklist-induced errors on easy examples.

6.3 Single Model Family and Single Training Dataset: Generalizability Is Unverified

The constraint. All experiments use Qwen3-VL 4B and 8B Instruct [1] as base models, fine-tuned on 30K randomly sampled instances from the RLAIF-V dataset [45]. The paper does not test on other model families (e.g., LLaVA, InternVL, proprietary API models), other model scales (1B, 70B), or other training datasets. While the text-only RewardBench results (Table 6) demonstrate cross-task transfer (multimodal training improves text-only reasoning), they use the same base model and training data — they show transfer within the same model, not generalizability of the DeltaRubric framework across different model architectures or training distributions.

The consequence. Practitioners using non-Qwen3-VL base models cannot assume DeltaRubric will produce similar gains. Different model families have different visual encoding architectures (Qwen3-VL uses a specific vision encoder and cross-attention design), different pretraining data mixtures (affecting prior visual grounding capabilities), and different instruction-tuning procedures (affecting how well the model follows the Planner and Verifier prompt formats). A model with weaker visual grounding capabilities might benefit more from checklist-guided verification (the Planner compensates for a larger gap) or might benefit less (the Planner itself generates lower-quality checklists because it can't accurately perceive the image, and the Verifier can't reliably execute the checks).

Similarly, the RLAIF-V dataset has specific characteristics — its preference labels, image domains, question types, and response distributions — that may influence what the Planner learns to attend to. A different training dataset with different error patterns (e.g., more subtle hallucinations, different types of visual reasoning failures) might produce a Planner that is less effective at identifying disagreements, or a Verifier that is less robust to checklist quality variation. The paper employs "strict decontamination" to ensure zero overlap with evaluation sets, which is good practice for assessing out-of-distribution generalization within the benchmark, but does not address generalization across model architectures or training data distributions.

What evidence exists in the paper. The consistency of results across two model sizes (4B and 8B, both Qwen3-VL) provides some evidence of within-family robustness — the approach works for both sizes with qualitatively similar patterns (Tables 1, 2). The text-only RewardBench results (Table 6) provide evidence of cross-domain transfer (to text-only evaluation), which is a stronger test of generalization. However, these results still use the same model family. No experiments with LLaVA-Critic, InternVL, Molmo, or any other vision-language architecture are reported, despite these models appearing as baselines in Tables 1 and 2 — the baselines are evaluated zero-shot, not fine-tuned with DeltaRubric.

The Multimodal RewardBench results (Table 2) show substantial variation in subcategory gains: large improvements in VQA (+7.9 points for 8B over no-rubric), Reasoning (+12.8 points), Math (+7.9), and Coding (+7.9), but smaller gains in General (+1.9) and a slight degradation in Correctness for the 4B model (-3.0 points). This variation suggests that the framework's effectiveness depends on task characteristics, and that the specific pattern observed on these two benchmarks may not transfer to tasks with different visual reasoning demands.

Mitigation status. Not addressed. The paper does not discuss model-family or training-data generalizability as a limitation, does not test on additional architectures, and does not explore whether the gains are sensitive to the specific training data used. The conclusions (Section 5) state that "decomposing evaluation into structured, verifiable steps leads to more reliable and generalizable multimodal reward modeling," with "generalizable" supported only by cross-benchmark (within-model) and cross-task (multimodal-to-text) evidence, not cross-architecture or cross-training-data evidence.

6.4 The Planner-Verifier Decomposition Is Never Compared Against Alternative Two-Stage or Multi-Sample Architectures

The missing baselines. The paper compares DeltaRubric against two primary baselines: (1) a zero-shot base model (no RL fine-tuning), and (2) a "no-rubric" baseline where the model generates CoT reasoning and a verdict in a single trajectory, fine-tuned with the same RL algorithm (GRPO). Both baselines represent the standard paradigm of monolithic, single-pass evaluation. What the paper does not compare against is any alternative two-stage architecture that performs planning-and-execution but with a different intermediate representation — for instance, a baseline where the model first generates a natural-language summary of disagreements (without the structured checklist format) and then generates a verdict conditioned on that summary, or a baseline where the model generates multiple candidate verdicts and then revises its judgment.

The consequence. The paper cannot distinguish between three hypotheses for why DeltaRubric outperforms the no-rubric baseline: (1) the checklist format specifically (structured, itemized, neutral verification items) is responsible for the gain; (2) any intermediate planning step (generating any text artifact that analyzes disagreements before rendering judgment) would produce similar gains, regardless of format; or (3) the two-pass inference with more total computation (more tokens generated, more forward passes) accounts for some or all of the gain, independent of the checklist structure. If hypothesis (2) or (3) is correct, then simpler architectures (e.g., "first describe the disagreements, then judge") could achieve similar accuracy improvements with less implementation complexity (no need for post-generation filtering, no need for neutral-language prompting, no need for decoupled multi-role RL). If hypothesis (3) is correct, then the gains are a compute effect rather than an architectural innovation, and the same compute could be spent on longer CoT reasoning or multiple samples with majority voting.

The static-rubric ablation (Table 5) partially addresses hypothesis (1) by showing that generic evaluation criteria (step 1 of a plan) are less effective than instance-specific checklists (step 2 of an adaptive plan). But the static rubric is one specific type of intermediate structure; other intermediate representations — particularly those that, like DeltaRubric's checklists, are generated dynamically per instance rather than fixed across the dataset — are not tested. A baseline that generates free-form disagreement analysis (not structured as checklist items) would directly test whether the structure of the checklist (numbered items, neutral framing, constraint-level granularity) matters beyond the general value of generating any intermediate analysis.

What evidence exists in the paper. The ablation studies (Tables 3–5, 7–9) all vary components within the DeltaRubric framework (Planner training, reward formulation, visual context, guidance bonus λ, RL algorithm) but none test alternative frameworks that achieve planning-and-execution through different mechanisms. The paper does not discuss why it chose only single-pass baselines or acknowledge the absence of multi-stage alternatives. The qualitative examples (Figures 3–6) show that DeltaRubric's checklists succeed where the no-rubric baseline fails, but they don't show whether a simple "analyze disagreements first, then judge" prompt would have caught the same errors.

Mitigation status. Not addressed. The paper does not acknowledge this as a limitation, does not propose such baselines as future work, and implicitly treats the no-rubric baseline as the only relevant comparison point for single-step vs. multi-step evaluation. A practitioner wondering whether to adopt the full DeltaRubric framework (with checklist generation, post-filtering, decoupled multi-role RL) vs. a simpler multi-step prompting strategy has no evidence in this paper to guide that decision.

6.5 The Hardest Evaluation Tasks Remain Unsolved: Checklist Guidance Does Not Help When Visual Reasoning Is Fundamentally Beyond the Model

The capability bound. Across all experiments, the paper's results show a consistent pattern: the largest absolute gains from DeltaRubric occur in subcategories where the base model already has non-trivial performance, while the hardest tasks show more modest (or even negative) improvement. On VL-RewardBench (Table 1), the 8B base model's weakest subcategory is Reasoning at 43.2 (zero-shot); DeltaRubric improves this to 72.6, a gain of +29.4 points over the base and +24.3 over no-rubric — this is the paper's largest gain and demonstrates that the approach works well when the model has some reasoning capability that structured verification can amplify. However, on Multimodal RewardBench (Table 2), the 8B base model's Correctness subcategory starts at 64.6 and DeltaRubric actually reduces it slightly to 64.0 (though no-rubric itself drops to 64.0), while the 4B model shows a -3.0 point drop (52.0 for DeltaRubric vs. 55.0 for no-rubric). This suggests that on tasks where visual correctness verification is fundamentally challenging, the checklist structure may add complexity without commensurate benefit, or may occasionally misdirect the Verifier.

More broadly, the paper demonstrates that DeltaRubric improves evaluation accuracy on its benchmarks by 4–8 points over a no-rubric baseline. But this leaves substantial headroom: even DeltaRubric-8B achieves only 80.1 on VL-RewardBench (meaning it is wrong on ~20% of examples) and only 73.2 on Multimodal RewardBench (wrong on ~27% of examples). These are error rates that remain too high for high-stakes reward modeling where systematic biases in the reward signal could distort policy learning. The paper does not characterize the remaining errors — whether they are Planner failures (checklist didn't identify the critical disagreement), Verifier failures (checklist was good but execution was flawed), or fundamental failures (even perfect planning and execution would not resolve the ambiguity). Without this characterization, a practitioner cannot assess whether further investment in DeltaRubric (better Planner training, larger models, more data) is likely to close the remaining gap, or whether the approach has hit a ceiling.

What evidence exists in the paper. The subcategory breakdowns in Tables 1 and 2 provide evidence of differential effectiveness across task types. The Correctness subcategory degradation for the 4B model (-3.0 on Multimodal RewardBench) is the most direct evidence that DeltaRubric can underperform the no-rubric baseline on certain task types, though the paper does not discuss this result. The static-rubric ablation (Table 5) shows that for the 8B model, the static rubric achieves 75.3 Overall vs. DeltaRubric's 80.1 — a 4.8-point gap that represents the value of instance-specific checklists over generic criteria. But this also means that 75.3 — and the remaining ~20 points to perfect accuracy — is attributable to factors that neither generic rubrics nor instance-specific checklists fully address (base model capability, verifier quality, inherent ambiguity). The paper does not analyze what fraction of remaining errors falls into each category.

The text-only RewardBench results (Table 6) provide additional evidence of a capability bound: even on a text-only task where visual grounding is not required, DeltaRubric-8B achieves only 84.6 Overall, leaving a ~15% error rate. The base model's text-only capability (81.4) is strong, and DeltaRubric's improvement is meaningful but bounded. This suggests that the Planner-Verifier decomposition improves the model's process of evaluation (it reasons more systematically) but does not fundamentally expand its knowledge or reasoning depth — it makes the model more reliable at applying its existing capabilities, but does not grant new capabilities.

Mitigation status. Not explicitly addressed. The paper's future work (Appendix A.4) mentions "extending the DeltaRubric framework to temporal modalities, such as video evaluation" as an exciting direction, but does not discuss the capability ceiling observed in the current results or strategies for improving performance on the hardest subtasks. The paper does not provide an error taxonomy, confidence calibration analysis, or investigation of whether remaining errors cluster in particular failure modes that could be systematically addressed.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper reshapes the conversation around multimodal reward modeling by introducing a simple but consequential idea: evaluation is an active investigation process, not a passive classification task. Before DeltaRubric, the field's approach to multimodal reward models was converging on a unified paradigm: train a monolithic model to process image-question-response tuples and output preference verdicts, optionally with chain-of-thought reasoning. The debate centered on what kind of reasoning (scalar vs. generative, CoT vs. direct), not on whether reasoning should be structurally decomposed. DeltaRubric challenges that consensus by demonstrating that decomposing evaluation into explicit planning (what to check) and execution (checking it) — with the plan expressed as an instance-specific, disagreement-focused checklist — yields accuracy gains that cannot be recovered by simply training the same model harder or longer with the monolithic approach.

The paper establishes this not as a minor tweak but as a qualitatively different evaluation paradigm. The critical evidence comes from the static-rubric ablation (Table 5): providing a generic evaluation rubric helps (8B Overall: 72.0 with no-rubric → 75.3 with static rubric), but instance-specific, dynamically synthesized checklists push performance substantially further (80.1). The +13.0-point gap in the Reasoning subcategory between static and dynamic rubrics is particularly telling — it shows that the value of DeltaRubric is not just "having some structure," but having precisely the right structure for this specific comparison. A static rubric tells the model what kinds of errors to look for; a disagreement-driven checklist tells the model exactly which claims to verify against which visual evidence. This is the difference between giving someone a general checklist for inspecting a house ("check the plumbing, check the electrical, check the roof") versus giving them the seller's disclosure form with the specific discrepancies highlighted for this particular property. The latter doesn't just remind you what to look for; it directs your attention to where the problems actually are.

The finding partially resolves a tension in the prior literature that the paper itself describes (Section 2). On one side, rubric-based evaluation has proven effective at mitigating lazy judging in text-only settings [12, 15, 25, 30, 41]. On the other side, extending these approaches to multimodal evaluation has been bottlenecked by the difficulty of specifying rubrics that are simultaneously comprehensive enough to catch diverse visual errors and specific enough to guide fine-grained verification. DeltaRubric's resolution is to abandon the goal of comprehensiveness in favor of disagreement-specificity: rather than trying to enumerate all possible dimensions of quality (which is impossible for open-ended visual tasks), generate a checklist that only covers the points where these two specific responses actually conflict. This shifts the Planner's job from "predict what evaluation criteria matter for this task type" to "find where these responses say different things about the image," which is a more tractable problem with a cleaner training signal.

The paper also provides the first systematic evidence — albeit limited to Qwen3-VL models — that multimodal evaluation training with structural decomposition can actively improve text-only reasoning rather than causing catastrophic forgetting (Table 6: +6.7 points on text-only RewardBench Reasoning). This finding, if replicated, has significant implications for how the field thinks about the relationship between modalities in post-training. The default assumption has been that multimodal fine-tuning introduces interference — visual representations compete with linguistic ones for model capacity, and the result is degraded text-only performance unless explicit mitigation (replay, elastic weight consolidation, etc.) is applied. DeltaRubric's result suggests that when the multimodal training task is evaluative reasoning — decomposing claims, identifying disagreements, verifying evidence — the skill transfers bidirectionally. Learning to hunt for visual discrepancies between responses makes you better at hunting for logical discrepancies between text passages. This opens the possibility of using multimodal evaluation as a reasoning curriculum that strengthens general-purpose critical thinking, not just visual grounding. If validated, this means post-training pipelines could intermix text and multimodal data without a tradeoff, and might even prefer multimodal data for its richer disagreement structure.

However, it would be a mistake to frame this as a paradigm shift in reward modeling. DeltaRubric does not replace or obsolete existing approaches — it augments them with a specific structural decomposition. The pipeline still requires a base model capable of visual perception and language understanding, and the Planner-Verifier architecture is a training-and-inference protocol, not a new model class. The gains, while significant (+4.3 to +8.1 points over no-rubric on VL-RewardBench), are measured improvements over a strong baseline, not the kind of 2× leap that would signal a fundamentally new capability. The framework is more accurately described as a methodological reframing with strong empirical backing: it changes how practitioners should think about building evaluation systems (as active investigation pipelines rather than monolithic classifiers) and provides a concrete recipe for implementing that vision, but the underlying components — MLLMs, RL fine-tuning, structured prompting — are drawn from the existing toolkit.

The paper redirects research attention in several ways, both constructive and cautionary. On the constructive side, it makes checklist generation as a learnable skill a first-class research problem. Prior work on rubrics treated them as static artifacts (human-designed or once-generated) fed to an evaluator; DeltaRubric shows that optimizing the rubric generator jointly with the evaluator yields better results than either a frozen generator (Table 3: -1.3 Overall, -6.3 Reasoning) or an absolute-reward-trained generator (Table 4: -2.5 Overall, -3.5 Reasoning). This opens a line of inquiry into how to train rubric generators — the reward structure, the exploration budget, the interaction between rubric specificity and verifier robustness — that essentially didn't exist before. On the cautionary side, the sensitivity analysis on the guidance bonus λ (Table 8) serves as a warning: pushing the Verifier too hard toward checklist obedience (λ=0.6) degrades reasoning performance. Checklist adherence and independent judgment are in tension, and the optimal balance is a tunable parameter, not a resolved design principle. This means future work in this space cannot simply assume "more checklist influence is better" — it must calibrate the tradeoff, and the calibration may be task-dependent.

Follow-Up Research This Work Enables

Directly combining DeltaRubric with process reward models (PRMs) for step-level verification feedback. The Planner generates 2–4 checklist items, each specifying a concrete visual fact to verify. The Verifier evaluates each item against the image and produces item-level evidence. But the current training only rewards the Verifier for final verdict accuracy (Equation 2) — there is no per-item supervision signal. A natural extension is to train the Verifier with step-level process rewards: for each checklist item, reward the Verifier for correctly determining which response (if either) is favored by the visual evidence, independent of the final verdict. This would require annotated data where each checklist item's ground-truth resolution is known (e.g., from human annotators or from the image metadata), which is more expensive than preference labels but could provide a much denser training signal. A strong experiment here would train a DeltaRubric Verifier with PRM-style per-item rewards on a subset of annotated data, then evaluate whether it achieves better generalization to unseen checklist structures or to out-of-domain images, compared to the outcome-reward-only training in the current paper. The hypothesis is that per-item rewards would make the Verifier more robust to Planner errors (since it learns to evaluate each check independently rather than optimizing for the final verdict given the checklist as a whole), and would make the Planner's job easier (since the Verifier's per-item behavior becomes more predictable). The paper's current finding that λ=0.6 degrades Reasoning (Table 8) suggests that over-optimizing for checklist adherence at the aggregate level is harmful, but per-item rewards might allow finer-grained adherence without the same over-optimization risk, since the Verifier would be rewarded for accurate item execution rather than obedient item execution.

Adaptive or selective checklist generation — only planning when the baseline is uncertain. The current system generates a checklist for every evaluation instance, doubling inference cost regardless of whether the checklist adds value. On easy examples where the Verifier would be correct without a checklist, the Planner's output is wasted computation. On hard examples where even a perfect checklist wouldn't help (the Verifier lacks the visual reasoning capability to execute the checks), the Planner's output is wasted effort. A natural extension is to train a gating mechanism — either a separate lightweight classifier or a self-assessment capability within the Verifier — that decides, before invoking the Planner, whether checklist generation is likely to change the verdict. The gating signal could be the Verifier's confidence in its no-checklist judgment (e.g., token probability of the verdict, or entropy across multiple cheap-probe samples). A concrete experiment: take the trained DeltaRubric model, run the cheap-probe Verifier without a checklist on each evaluation instance, and bin instances by the Verifier's confidence. Then measure DeltaRubric's accuracy gain over the no-rubric baseline within each confidence bin. The hypothesis is that gains concentrate in medium-confidence bins — where the baseline is uncertain enough to be wrong but capable enough to benefit from guidance — while low-confidence bins (hard examples) show no gain and high-confidence bins (easy examples) show negligible gain. If this pattern holds, a gating system that invokes the Planner only for medium-confidence instances could recover most of DeltaRubric's accuracy gain at a fraction of the inference cost. This would directly address the inference-cost limitation (Section 6.1) and would be a practical contribution for deployment.

Cross-model transfer of Planner-generated checklists — does a checklist from a strong Planner help a weak Verifier from a different model family? The paper demonstrates that jointly training Planner and Verifier within a single model yields synergistic improvements. An open question is whether the Planner's checklists are model-specific (tailored to the particular blind spots of its co-trained Verifier) or universally useful (providing structural guidance that helps any competent MLLM evaluator). A cross-model transfer experiment would test this: take the Planner from a fully trained DeltaRubric-8B model, generate checklists on an evaluation set, and feed those checklists to a different Verifier — say, a Qwen3-VL-4B Verifier trained only with the no-rubric objective, or a zero-shot LLaVA-Critic-8B, or an InternVL3-8B prompted as a judge. Measure whether the externally-generated checklists improve these Verifiers' accuracy relative to their own no-checklist baselines. If checklists transfer well, it means the Planner has learned to generate generally useful verification structures, not just structures that specifically compensate for its own Verifier's weaknesses. This would make DeltaRubric's Planner a deployable standalone component: a "disagreement analyzer" that can be paired with any evaluator to improve its accuracy. If checklists don't transfer well, it suggests the Planner-Verifier co-adaptation is specific to the shared representations developed during joint training, and the framework is less modular than it appears. Either result is informative: successful transfer opens a path to decoupled deployment (expensive Planner, cheap Verifier), while failed transfer clarifies that the gains come from tight integration.

Stress-testing Planner robustness to adversarial response pairs designed to induce biased or misleading checklists. The paper demonstrates that DeltaRubric catches specific hallucinations (Figures 3–6) and that the Planner learns to generate neutral checklists. But the Planner's neutrality is enforced only through a prompt ("Do NOT mention Response A or Response B by name") and a post-generation filtering step whose effectiveness is unmeasured. A stress-test would construct adversarial response pairs where one response is subtly biased in its framing — e.g., Response A uses more confident, authoritative language about a hallucinated detail, while Response B uses hedging language about a true detail — and test whether the Planner generates checklists that implicitly favor Response A (e.g., framing checks that assume A's claims as the default to verify against). A more sophisticated stress-test would construct pairs where one response contains a "poisoned" claim designed to hijack the Planner's attention — e.g., Response A makes a factually correct but boring claim, while Response B makes a similar but slightly incorrect claim along with a vivid but irrelevant description that draws the Planner to generate a checklist focused on the vivid detail rather than the factual error. If the Planner is robust, it should consistently generate checklists that isolate the adjudicatively decisive disagreements regardless of response framing. If it's fragile, the failure modes would reveal the Planner's reliance on surface-level textual cues (e.g., checklist items targeting claims that use assertive language rather than claims that are factually wrong). This stress-test would clarify whether DeltaRubric genuinely mitigates lazy judging or merely shifts it: instead of the Verifier exploiting textual priors, perhaps the Planner exploits textual priors to decide which disagreements to encode as checklist items. The λ=0.6 result (Table 8) already suggests fragility to over-compliance; adversarial response pairs would probe the Planner side of the equation.

Scaling checklist generation to video and temporal reasoning tasks. The paper's future work discussion (Appendix A.4) explicitly mentions extending to temporal modalities. The natural first step is video QA preference evaluation: given a video, a question, and two candidate responses, the Planner must generate checklist items that isolate temporal disagreements — e.g., "Does the person pick up the object before or after entering the room?" or "Is the car visible in the first 5 seconds of the clip?" This is a harder planning problem because the dimension of disagreement is temporal (sequence of events, object persistence, action ordering) rather than purely spatial, and the Planner must specify not just what to check but when in the video to check it. The Verifier's job also becomes harder: executing a checklist item against a video requires temporal localization (finding the relevant clip segment) in addition to visual recognition. A concrete experiment would fine-tune the DeltaRubric framework on a video QA preference dataset (e.g., Video-MME or a video-specific subset of RLAIF-V if available) and measure accuracy gains over a no-rubric video evaluation baseline. The key question is whether the Planner-Verifier decomposition scales to the increased complexity of temporal evidence gathering, or whether the Planner's checklist items become too coarse to guide effective verification (because temporal disagreements are harder to specify in 2–4 short sentences). A negative result (no gain over baseline) would indicate that the checklist format needs temporal grounding annotations beyond natural language instructions; a positive result would substantially broaden DeltaRubric's applicability.

Practical Applications and Downstream Use Cases

Automated quality filtering in multimodal preference data construction. The primary bottleneck in scaling RLHF for MLLMs is the cost and difficulty of obtaining reliable preference labels for image-question-response triples. Human annotation is expensive and slow; automated annotation using MLLM judges is fast but unreliable due to lazy judging. DeltaRubric improves the accuracy of automated judges by +4.3 to +8.1 points on VL-RewardBench (Table 1), and particularly in the Hallucination subcategory where automated judges must catch factually incorrect visual claims (88.3 for 8B DeltaRubric vs. 86.1 for no-rubric, Table 1). In a data construction pipeline, a DeltaRubric-trained judge could be used to filter or re-rank candidate response pairs before human review, reducing the annotation budget: pairs where the judge has high confidence (across multiple samples) could be accepted automatically, while ambiguous pairs are routed to humans. This is a near-term application because the training recipe is defined (30K RLAIF-V instances, GRPO with decoupled advantages) and the model weights are presumably releasable, enabling practitioners to fine-tune their own MLLM judges with DeltaRubric before deploying them in preference data pipelines.

Online reward signal during RLHF training of MLLMs. The standard RLHF training loop for MLLMs alternates between sampling responses from the policy model, scoring them with a reward model, and updating the policy based on the reward signal. If the reward model suffers from lazy judging — systematically preferring responses with confident, fluent language over visually accurate ones — the policy model learns to produce visually-plausible hallucinations. Replacing a monolithic reward model with a DeltaRubric-trained variant provides a more reliable reward signal without changing the training loop or the policy model architecture, since DeltaRubric is a drop-in replacement for the reward scoring step (it takes the same inputs — image, question, response pair — and produces a preference verdict). The +8.1-point gain on VL-RewardBench (Table 1, 8B model) suggests that for every 100 preference comparisons, DeltaRubric would correctly resolve approximately 8 additional disputes compared to the no-rubric baseline. Over the course of RLHF training with thousands of comparisons, this could meaningfully shift the policy's learned distribution away from fluent hallucinations and toward visually grounded responses. The inference latency cost (2× forward passes) is less problematic in offline RLHF training than in interactive deployment, since training is throughput-bound rather than latency-bound, and the Planner and Verifier forward passes can potentially be batched.

Evaluation-time guardrails for high-stakes multimodal QA systems. In applications where MLLMs are deployed to answer visual questions with factual accountability — medical image interpretation, insurance claim processing from photos, accessibility tools describing visual scenes to blind users — the consequences of a hallucinated or incorrect answer can be severe. DeltaRubric can serve as an evaluation-time verification layer: when the primary model generates a response, a separate DeltaRubric-trained judge compares that response against (a) a reference response from a different model or a different decoding run, or (b) a deliberately generated "adversarial" response designed to probe potential hallucinations, and the judge's verdict can flag potentially unsafe responses for human review. The large gains in Safety (93.3 for 8B DeltaRubric vs. 91.5 for no-rubric on Multimodal RewardBench, Table 2) and VQA (84.9 vs. 77.0, Table 2) subcategories are directly relevant: these are task types where factual visual accuracy is paramount. The checklist's interpretability is an additional asset in high-stakes settings — a human reviewer receiving a flagged response can see exactly which visual claims the judge considered dubious, enabling faster and more targeted human-in-the-loop verification.