MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data

ArXiv: 2603.09206

🎯 Pitch

MM-Zero achieves the first fully self-sufficient training loop for vision-language models, requiring absolutely no images or human data from the start. Instead of sourcing or generating data, it bootstraps learning by having three specialized agents—a Proposer, Coder, and Solver—collaborate to create and reason over novel visual concepts rendered from code, with all roles improving together through reinforcement learning.

---

1. Executive Summary

MM-Zero introduces the first reinforcement-learning-based framework that achieves zero-data self-evolution for vision-language model reasoning — training VLMs without any external images, questions, or human labels. The framework instantiates a multi-role self-evolving training framework comprising three specialized agents all initialized from the same base model: a Proposer that generates abstract visual concepts and formulates questions, a Coder that translates these concepts into executable code (SVG/Python) to render visual images, and a Solver that performs multimodal reasoning over the generated visual content, with all roles optimized sequentially using Group Relative Policy Optimization (GRPO) and role-specific reward functions that integrate execution feedback, visual verification, and difficulty balancing. Across Qwen3-VL-4B-Instruct, Qwen3-VL-8B-Instruct, and Mimo-VL-7B-SFT, MM-Zero yields consistent improvements on multimodal benchmarks, with the 8B model climbing from 50.7% to 54.1% average accuracy and the 7B model improving from 50.9% to 56.0%, establishing that VLMs can autonomously self-improve through multi-model interaction with zero external data, with larger gains concentrated on visual math reasoning tasks and the largest benefits accruing to models with sufficient base capability to render code successfully.

2. Context and Motivation

The Core Problem: Self-Evolution Requires Data — Especially Visual Data

The fundamental question MM-Zero tackles emerges from a tension in the current landscape of AI self-improvement: language models can now self-evolve with zero external data, but vision-language models cannot. This asymmetry exists because self-evolution fundamentally requires generating training experiences. For LLMs, "generating experiences" means producing text — questions, reasoning traces, and answers — which the model can already do natively. For VLMs, every training experience must include an image, and images are not something a VLM can simply emit from its text-based output head. The model must either source images externally or generate them through some intermediate mechanism.

This gap matters for several practical reasons the paper highlights throughout Section 1:

• Annotation bottlenecks are structural, not transient. The standard post-training pipeline for VLMs relies on expert-curated vision-language tasks — images paired with questions and human-verified answers. The paper cites extensive prior work ([7, 8, 13, 17, 21, 22]) showing that this pipeline is "costly, labor-intensive, and fundamentally bottlenecked by the availability of human annotation." Unlike the situation for LLMs, where verifiable domains like math and code can be supervised entirely by execution feedback, VLM training has remained tethered to human effort for the visual modality.

• The image curation burden substitutes one data problem for another. Even when prior work attempts to reduce human labeling, it shifts the bottleneck rather than eliminating it. If you eliminate human-annotated questions but still require a static image dataset to generate questions about, you have merely exchanged the problem of labeling images for the problem of sourcing and filtering images. The paper explicitly frames this as a substitution, not a solution: "the entire iterative process remains rigidly conditioned on pre-existing, collected static image datasets. This merely shifts the bottleneck: the model's evolution is now bounded by the distribution, quality, and diversity of the collected image dataset."

• Static datasets impose hard ceilings on complexity scaling. A curated image dataset has fixed content, fixed difficulty, and fixed diversity. As the model improves through self-evolution, it eventually saturates on whatever distribution the static images represent. There is no mechanism to generate progressively harder visual reasoning problems, which is essential for continuous improvement — a VLM trained only on existing chart comprehension images cannot spontaneously encounter more complex chart types, denser data visualizations, or novel visual reasoning patterns that would push its capabilities forward.

• The scaling argument from simulation. The paper draws an explicit analogy to robotics and autonomous driving (Section 1), where simulated environments expose models to "rare and dangerous events that would be too costly to collect in the real world." Programmatically rendered visual scenes — charts, diagrams, geometric constructions, tables — can simulate "far more complex and diverse scenarios than curated datasets, with virtually unlimited variations." This is not merely a cost argument; it's an argument about the upper bound of what can be learned from fixed data versus generated data. A model that can generate its own progressively harder training examples has no inherent ceiling on complexity, while a model trained on static images does.

Conflicting Progress in LLM vs. VLM Self-Evolution

The paper is motivated by an asymmetry in the literature that reveals a genuine missing capability. The introduction and Section 5.2 survey the landscape:

On the LLM side, zero-data self-evolution is already demonstrated. Works like R-Zero ([12]) train LLM reasoners by having a proposer generate questions and a solver answer them, with correctness verified by code execution feedback — no human data required. SPICE ([20]) shows that LLMs can improve reasoning through self-play in corpus environments. Absolute Zero ([57]) demonstrates reinforced self-play reasoning from zero data. This progression establishes that, for text-only domains with verifiable correctness, the self-evolution loop is closed.

On the VLM side, the loop is open — something must provide images. The paper identifies a specific lineage of approaches that attempt to extend the proposer-solver paradigm to VLMs, and diagnoses why they fall short:

• VisPlay ([11]) adapts the challenger-solver pipeline from LLM self-evolution to multimodal tasks but "strictly require[s] a collection of seed images to bootstrap training, which can be costly to curate and filter." The word "strictly" is significant — VisPlay's framework is architected around a static image corpus as a non-negotiable input.

• Evolmm ([31]) and V-Zero ([34]) explore multimodal self-improvement with "minimal annotation" but remain "conditioned on pre-existing static image datasets." These methods reduce the labeling burden but preserve the data dependency.

• Vision-Zero ([37]) uses "strategic gamified self-play" for scalable VLM self-improvement but, based on the paper's characterization, similarly relies on existing visual inputs.

The paper's diagnosis is that these approaches share a common flaw: they conflate eliminating human labels with achieving zero-data self-evolution. They succeed at the former but fail at the latter because "they remain constrained by a fixed image corpus, preventing the synthesis of progressively harder or more diverse visual scenes that could continuously challenge the model's perception and reasoning." A static corpus cannot produce the Goldilocks dynamic — generating questions just at the frontier of the current model's capabilities — because the difficulty distribution is fixed from the start.

The Missing Capability: Programmatic Visual Generation as a Learned Skill

This diagnosis points to a specific missing component in prior work: no existing VLM self-evolution framework includes a generative visual synthesis role. The proposer-solver duality inherited from LLM research is insufficient because it treats the creation of the visual input (the equivalent of the "prompt" in text-only reasoning) as something that happens outside the learning loop. In LLM self-evolution, the proposer generates a text problem, and that's sufficient — the text problem is the input. In VLM self-evolution, the proposer generates a text description, but the text description is not the input. An image must be created from that description before the solver can reason about it.

The paper identifies code generation (SVG, Python/matplotlib) as the bridge between abstract text specifications and concrete visual images. This is not an arbitrary choice — it reflects a recognition that executable code is the only modality that simultaneously: (a) can be generated by a language model trained on code, (b) produces precise, deterministic visual output when executed, and (c) provides its own verifiability signal (does the code compile? does it render without errors?). The Coder role is therefore the architectural innovation that closes the self-evolution loop for VLMs — not an optimization, but a structural requirement for zero-data operation.

Where This Paper Positions Itself

The paper explicitly frames its contribution as a first-of-its-kind capability rather than an incremental improvement. The key positioning claim appears in the abstract and is reinforced throughout Section 1 and Section 2: MM-Zero is "the first RL-based framework to achieve zero-data self-evolution for VLM reasoning" and "the first framework to explore self-evolving multimodal reasoning through interactions among more than two roles."

This positioning is significant because it defines the paper's novelty in terms of what is made possible rather than what is optimized. Prior work on VLM post-training improved efficiency or reduced human annotation; MM-Zero eliminates the external data requirement entirely. This is a categorical shift rather than a quantitative one.

The paper also draws a deliberate structural parallel to LLM self-evolution work while emphasizing a fundamental difference. R-Zero ([12]) demonstrated that LLMs could self-evolve with two roles (proposer and solver). MM-Zero demonstrates that VLMs require three roles because the visual modality introduces an irreducible generation step that the proposer cannot perform alone. The introduction of the Coder role is therefore not a design choice among alternatives — it is presented as a necessary architectural response to the multimodal nature of the problem.

The Goldilocks Principle as a Training Philosophy

The paper frames its approach within the Goldilocks principle ([14]): problems should be "just right" — challenging enough to drive learning but solvable enough to provide a meaningful training signal. This principle is cited explicitly in Section 2.4 (Equation 7's difficulty reward formulation) but permeates the entire framework design:

• The Proposer is rewarded for generating questions where the Solver is maximally uncertain (difficulty score peaks at $c_i = 0.5$, meaning the Solver's own answers are evenly split — see Equation 7).
• The Coder is trained on captions where the rendering success rate falls between 0.25 and 0.75 (not trivially easy to render, not impossible — Section 2.2, training data filtering).
• The Solver trains on questions where easy-question accuracy exceeds 0.5 but hard-question accuracy is between 0.27 and 0.75 (verifiable but not solved — Section 2.2, solver filtering).

This difficulty-gating is not merely an optimization trick; it is the mechanism by which the system calibrates its own curriculum. Without external data to define a difficulty distribution, the system must discover what constitutes an appropriately challenging problem through its own interactions. The Goldilocks filtering in the data pipeline serves as a self-supervised difficulty estimator that replaces the human-curated difficulty progression that would exist in a manual training pipeline.

The paper therefore positions itself at the intersection of three research threads: self-evolution in LLMs (providing the template for proposer-solver dynamics), reinforcement learning with verifiable rewards (providing the training algorithm, GRPO), and programmatic visual synthesis (providing the mechanism for zero-data image generation). The claimed contribution is the integration of these threads into a framework that, for the first time, enables VLMs to bootstrap their own improvement without any external visual data, human annotations, or curated datasets.

3. Technical Approach

3.1 Reader Orientation

MM-Zero builds a self-contained training system where a single base VLM spawns three specialized agents that collectively generate their own visual reasoning problems, render the corresponding images, solve those problems, and use the outcomes as a training signal to improve all three agents — without any external images, questions, or human labels. The system solves the problem of VLM self-improvement by transforming a single base model into a closed ecosystem where abstract language descriptions flow through code execution to become concrete visual inputs, which in turn provide verifiable correctness signals that drive reinforcement learning, creating a loop where difficulty naturally escalates as all agents improve.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components arranged in a sequential training loop:

1. A single base VLM (e.g., Qwen3-VL-8B-Instruct) — serves as the common initialization for all three roles. No external checkpoints, no pretrained specialist models. The same weights are duplicated and then each copy is fine-tuned with role-specific prompts and reward functions.

2. The Proposer — a text-only agent that takes a content seed (e.g., "data_chart") and outputs a structured quadruple: a detailed visual caption describing what should appear in an image, an easy question with answer (used to verify the image was rendered faithfully), and a hard question with answer (used to train the Solver). The Proposer is trained via GRPO to generate captions that are specific enough to be renderable but hard enough to challenge the Solver.

3. The Coder — a text-to-code agent that receives the Proposer's caption and generates executable SVG (or Python/matplotlib) code. The code is executed in a sandboxed environment to render a PNG image. The Coder is trained via GRPO to produce code that compiles, renders without errors, and yields images containing the information needed to answer at least the easy question correctly.

4. The Solver — the multimodal reasoning agent that receives a rendered image and the hard question, outputs chain-of-thought reasoning and a final boxed answer. The Solver is trained via GRPO using test-time reinforcement learning (majority voting among its own rollouts provides pseudo-labels, since no ground-truth hard-question answers exist).

5. A rendering and filtering pipeline — an infrastructure layer that executes Coder output, converts SVG to PNG, validates images, forwards them to the Solver, and implements the Goldilocks filtering thresholds that determine which generated examples are retained for training each role.

Information flows as follows: a content type seed enters → the Proposer generates a quadruple (caption, easy QA, hard QA) → the caption goes to the Coder → the Coder produces SVG code → the code is executed to render a PNG → the easy question + image go to the Solver for solvability scoring (rewarding the Proposer and Coder) → the hard question + image go to the Solver for majority-vote consensus scoring (rewarding the Proposer via difficulty reward, and training the Solver via TTRL) → filtered examples are retained per role → each role updates via GRPO → the cycle repeats with improved agents generating harder problems.

Training is sequential per role, not joint: in each training step, one role's parameters are updated while the other two roles are frozen. The full training runs for 3 iterations (20 steps per role per iteration, 60 total steps per model), with checkpoints saved every 20 steps. All training uses 8 RTX 6000/Pro 96GB GPUs.

3.3 Roadmap for the Deep Dive

• First, the GRPO algorithm and RLVR framework — the shared training machinery that all three roles use, since understanding how rewards are normalized and policies are updated is prerequisite to understanding each role's specific reward design.
• Second, the Proposer — its input/output structure, its reward decomposition, and why the Goldilocks difficulty signal (min(ci, 1 − ci)) is the mechanism that drives curriculum escalation.
• Third, the Coder — its training data pipeline, its reward function, and how execution feedback provides the only verifiability signal for code generation.
• Fourth, the Solver — the TTRL mechanism, the silver-label generation process, and the format enforcement reward.
• Fifth, the training data filtering and iterative protocol — the Goldilocks thresholds per role, the sequential training loop, and the checkpoint management strategy.
• Sixth, the rendering pipeline and infrastructure — the SVG execution stack, the parallelization architecture, and the validation constraints that convert code generation into a reliable image supply.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a systems paper whose core idea is that VLM self-evolution without external data requires an intermediate code-generation step — a Coder role — that bridges abstract text proposals to concrete visual inputs, and that this tri-role architecture, trained sequentially with role-specific GRPO rewards incorporating execution feedback, solvability verification, and Goldilocks difficulty gating, enables autonomous capability improvement from zero initial data.

---

Group Relative Policy Optimization (GRPO) as the Shared Training Algorithm

All three roles are trained using the same underlying RL algorithm: Group Relative Policy Optimization (GRPO, introduced in Shao et al., 2024 for DeepSeekMath). The paper adopts GRPO rather than PPO because GRPO eliminates the need for a separately learned value function (a "critic" network), which would triple the memory footprint (one critic per role) and introduce training instability from value estimation errors.

The core operation of GRPO (Section 2.1): For a given prompt $p$, the current policy $\pi_{\theta_{\text{old}}}$ generates $N$ independent responses $\{x_1, \ldots, x_N\}$. Each response receives a scalar reward $r_i$ from the role-specific reward function. These raw rewards are then normalized within the group to produce advantages:

$\hat{A}_i = \frac{r_i - \mathrm{mean}(r_1, \ldots, r_N)}{\mathrm{std}(r_1, \ldots, r_N) + \varepsilon_{\text{norm}}}$

where $\hat{A}_i$ is the normalized advantage for response $i$, $\mathrm{mean}(\cdot)$ and $\mathrm{std}(\cdot)$ are computed across the group of $N$ responses, and $\varepsilon_{\text{norm}}$ is a small constant for numerical stability (the paper does not specify its exact value, but standard practice sets it to $10^{-6}$ or $10^{-8}$).

What this computes: The advantage $\hat{A}_i$ is a z-score — it measures how much better or worse response $i$ is compared to the average response in its group, in units of standard deviation. A positive advantage means the response outperformed the group average; a negative advantage means it underperformed. The normalization is crucial because raw rewards have no natural scale — a "good" render might get 1.0, a "bad" render might get -0.1, and the difference between them might be misleadingly small or large depending on the reward function design. Standardizing to z-scores ensures the policy update is driven by relative ordering within the batch, not absolute reward magnitudes.

Why this form: Alternatives like PPO (which uses a learned value function to estimate advantages: $A_i = r_i - V(s)$) require training a separate critic network whose predictions will be inaccurate early in training, introducing bias into the advantage estimates. GRPO's group-relative normalization is a non-parametric alternative — it estimates "how good is this response?" by comparing it to other responses to the same prompt, which is always a valid comparison regardless of how the reward function is calibrated. The group must be large enough for the mean and standard deviation to be reliable; the paper uses $N = 4$ rollouts for the Coder and $N = 5$ for the Solver when computing rewards for the Proposer, and $N = 8$ for the Solver's own training rollouts.

The policy is then updated by maximizing a clipped surrogate objective with a KL regularization term:

$\mathcal{L}_{\text{GRPO}}(\theta) = -\frac{1}{N}\sum_{i=1}^{N} \min\!\Bigg( \frac{\pi_\theta(x_i)}{\pi_{\theta_{\text{old}}}(x_i)} \hat{A}_i,\, \mathrm{clip}\!\left( \frac{\pi_\theta(x_i)}{\pi_{\theta_{\text{old}}}(x_i)}, 1-\epsilon, 1+\epsilon \right) \hat{A}_i \Bigg)+ \beta \, \mathrm{KL}\!\left(\pi_\theta \,\|\, \pi_{\theta_{\text{old}}}\right)$

where $\pi_\theta(x_i) / \pi_{\theta_{\text{old}}}(x_i)$ is the importance sampling ratio (how much the probability of response $x_i$ has changed under the new policy relative to the old policy), $\epsilon$ is the clipping threshold (typically 0.1–0.3, preventing the ratio from deviating beyond $1 \pm \epsilon$), $\beta$ is the KL penalty coefficient, and the KL divergence term $\mathrm{KL}(\pi_\theta \,\|\, \pi_{\theta_{\text{old}}})$ constrains how far the new policy can drift from the old one.

What this computes: The objective increases the probability of responses with positive advantage (by making $\pi_\theta(x_i)$ larger) and decreases the probability of responses with negative advantage. The $\min$ operator with the clipping term implements a trust region: for responses with positive advantage, the update is clipped at a $1+\epsilon$ ratio increase to prevent the policy from over-optimizing a single lucky response; for responses with negative advantage, the update is clipped at $1-\epsilon$ to prevent the policy from catastrophically forgetting how to generate those responses. The KL penalty provides an additional global constraint on policy drift.

Why this form: The clipping is the key innovation from PPO that GRPO inherits — it prevents the "policy collapse" problem where a single high-reward response dominates the update and the model forgets everything else it knew. Without clipping, the importance sampling ratio could grow arbitrarily large for a high-reward response, causing the model to assign near-100% probability to a single output sequence and zero probability to all alternatives. The $\min$ operator means that for highly positive advantages, the optimizer uses the clipped (smaller) ratio, which limits the effective gradient. For negative advantages, the clipped ratio is bottom-bounded at $1-\epsilon$, so the model cannot completely eliminate low-reward responses — it can only reduce their probability by a limited factor per update.

Training hyperparameters for GRPO across roles (Table 5 of the paper's supplementary material):

Notable differences across roles: the Coder uses lower temperature (0.7 vs 1.0) and lower top_p (0.95 vs 0.99) than the Proposer and Solver, reflecting the need for more deterministic code generation — syntax errors are catastrophic for rendering, so the Coder benefits from reduced sampling diversity. The Solver uses 8 rollouts per prompt instead of 4, which is necessary for reliable majority voting (the TTRL pseudo-label requires identifying a consensus answer among rollouts, and more rollouts improve the reliability of that consensus). The Solver also requires 8 GPUs compared to 3 for the Proposer, reflecting the higher memory cost of processing image inputs (the vision encoder activations consume significant VRAM during both forward and backward passes).

---

The Proposer: Generating Visual Concepts and Questions

The Proposer $\pi_P$ is the generative engine of the self-evolution loop. It is initialized from the base VLM but used as a text-only policy — it receives a content type seed as input and outputs structured text, never seeing images during its own forward pass. Its job is to produce training data for both the Coder and the Solver by generating rich visual descriptions and corresponding question-answer pairs.

Input and output structure (Section 2.2, detailed in Appendix A.1). The Proposer receives a content type string — one of data_chart, diagram, geometry, timeline, map, table, or other — which specifies the visual category it should design for. It outputs a quadruple $x = (c, q_{\text{easy}}, a_{\text{easy}}, q_{\text{hard}})$ structured as six XML-tagged blocks:

1. <content_type> — echoes the input type (validates format compliance).
2. <caption> — a "rich, detailed specification" of the visual scene. The prompt template (Appendix A.1) requires that captions "include at least three of: multiple data series, annotations, secondary panel, colors/markers, derived values, non-trivial patterns, geometric constructions." This constraint forces the Proposer to generate complex, information-dense captions rather than minimal descriptions.
3. <easy_question> — "a simple question directly readable from the image." The answer to this question should be visually obvious once the caption is correctly rendered.
4. <easy_answer> — the ground-truth answer for the easy question (used to verify that the Coder rendered faithfully).
5. <hard_question> — "a challenging reasoning question." The template requires that hard questions "must require multi-step reasoning" and "must force the reader to extract at least one value from the visualization (do not state all values in the question)."
6. <hard_answer> — the intended answer for the hard question (not used as ground truth for Solver training — the Solver uses TTRL pseudo-labels — but serves as part of the Proposer's output for the hard-question generation task).

The quadruple structure reflects a deliberate separation of concerns: the caption defines what the image should depict, the easy question verifies whether the image was rendered correctly, and the hard question provides the reasoning challenge for the Solver. This three-way decomposition means that each downstream component (Coder for rendering, Solver for reasoning) receives exactly the information it needs.

The Proposer reward function (Section 2.4, Equation 4). The Proposer's reward is the most complex in the system, aggregating six distinct components that collectively incentivize formatting compliance, renderability, solvability, appropriate difficulty, content diversity, and within-batch uniqueness:

$R_p(x) = \begin{cases} -1 & \text{if format invalid,} \\[6pt] \frac{1}{N} \sum_{i=1}^{N} \mathds{1}_{\text{exec}}(C_i) \cdot \bigl(\min (R_{\text{solv}}(I_i),\, 0.5) + R_{\text{diff}}(I_i)\bigr) + r_{\text{eh}} + r_{\text{ct}} + r_{\text{div}} & \text{otherwise.} \end{cases}$

where $x$ is the Proposer's output quadruple, $N = 4$ is the number of code rollouts generated by the Coder from the caption $c$, each $C_i$ is one SVG code sample from the Coder, each $I_i$ is the rendered image (if execution succeeds), $\mathds{1}_{\text{exec}}(C_i) \in \{0, 1\}$ indicates whether rendering succeeded, $R_{\text{solv}}(I_i)$ is the solvability score (easy-question accuracy), $R_{\text{diff}}(I_i)$ is the difficulty score (hard-question self-consistency), $r_{\text{eh}}$ is the easy-hard penalty, $r_{\text{ct}}$ is the content-type diversity penalty, and $r_{\text{div}}$ is the caption/question diversity bonus. The total reward range is $[-1.0, 1.5]$.

What this computes: The outer case structure first checks whether the Proposer's output conforms to the required XML format — if not, the reward is immediately $-1$ and no further computation occurs. If the format is valid, the reward is the average (over $N = 4$ Coder rollouts) of a sum of two capped components plus three batch-level adjustment terms. Each Coder rollout contributes to the sum only if it renders successfully ($\mathds{1}_{\text{exec}}(C_i) = 1$); failed renders contribute zero. The sum of the solvability component (capped at 0.5) and the difficulty component (maximizing at 0.5) means the Proposer can earn at most 1.0 from rendering quality and difficulty per successful image, plus batch-level adjustments that can add or subtract up to several tenths of a point.

Why this form: The format check provides a hard constraint that prevents reward hacking through the Proposer ignoring the structured output format. The $-1$ penalty is severe enough to dominate any partial credit the Proposer might otherwise earn, ensuring format compliance is learned early. The cap of 0.5 on $R_{\text{solv}}$ is the critical design choice that prevents the Proposer from being rewarded for generating trivially easy questions. If solvability were uncapped, the Proposer could achieve high reward purely through easy questions that the Solver answers correctly with high confidence, without ever generating useful hard questions. The cap ensures that even perfectly solvable easy questions contribute at most 0.5 to the reward, forcing the Proposer to also invest in generating challenging hard questions (whose difficulty component can add up to another 0.5) to maximize its score. The ablation study (Section 4) explicitly validates this: "without the solvability and difficulty balance factor, the solver improves by only 2.3% over the baseline, compared to 3.9% for MM-Zero," and the model increasingly "embeds answers directly as text within the generated code, causing the solutions to appear explicitly in the rendered images" — a form of reward hacking that the cap prevents.

Solvability score (Section 2.4, Equation 5): For each successfully rendered image $I_i$, the Proposer's easy question $q_{\text{easy}}$ is sent to the Solver. The Solver generates $K = 5$ independent answers $\{y^{(i,k)}_{\text{easy}}\}_{k=1}^{K}$ to the easy question. The solvability score is the fraction of rollouts that match the Proposer's intended answer $a_{\text{easy}}$:

$R_{\text{solv}}(I_i) = \frac{1}{K} \sum_{k=1}^{K} \mathds{1}\left( y^{(i,k)}_{\text{easy}} = a_{\text{easy}} \right) \in [0, 1]$

What this computes: For a given rendered image, how consistently does the Solver produce the Proposer's intended easy answer? If 5 out of 5 rollouts match, $R_{\text{solv}} = 1.0$ — the image clearly contains the information needed. If 2 out of 5 match, $R_{\text{solv}} = 0.4$ — the image is confusing or incorrect. This score is a proxy for rendering faithfulness: it answers "did the Coder produce an image that conveys what the Proposer intended?" without requiring any ground-truth annotation of the image content.

Why this form: The easy question serves as a self-verification mechanism. Because the Proposer generates both the caption and the easy question/answer, the easy question is, by construction, answerable from the intended visual content. If the Coder renders faithfully, the Solver should consistently produce the Proposer's intended answer. If the Solver's answers are inconsistent or wrong, something went wrong — either the Coder's rendering omitted or distorted critical information, or the Proposer's caption was ambiguous. The $K = 5$ rollouts provide a more reliable signal than a single rollout would: random errors in the Solver's sampling (e.g., unlucky token choices) are averaged out, leaving only systematic failures that reflect genuine issues with the rendering or caption quality.

Difficulty score (Section 2.4, Equations 6–7): For the hard question $q_{\text{hard}}$, there is no ground-truth answer — the Proposer provides an intended answer ($a_{\text{hard}}$ in the XML output), but this cannot be trusted as a gold label because the Proposer itself is being trained and may produce incorrect answers. Instead, the difficulty score uses test-time reinforcement learning (TTRL) via majority-vote self-consistency. The Solver generates $K = 5$ independent answers $\{y^{(i,k)}_{\text{hard}}\}_{k=1}^{K}$ to the hard question on image $I_i$. The self-consistency $c_i$ is the fraction agreeing with the majority vote $\hat{y}_i$:

$c_i = \frac{1}{K} \sum_{k=1}^{K} \mathds{1}\left( y^{(i,k)}_{\text{hard}} = \hat{y}_i \right)$

The difficulty reward is then:

$R_{\text{diff}}(I_i) = \min(c_i, 1 - c_i) \in [0, 0.5]$

What this computes: Self-consistency $c_i = 0.6$ means 3 out of 5 rollouts agree on the same answer, with 2 disagreeing. The difficulty reward is $\min(0.6, 0.4) = 0.4$. Self-consistency $c_i = 1.0$ means all 5 rollouts agree — the Solver is completely certain — and the difficulty reward is $\min(1.0, 0.0) = 0.0$ (the question is too easy). Self-consistency $c_i = 0.2$ means only 1 out of 5 agrees with the majority (the Solver is guessing randomly) — the difficulty reward is $\min(0.2, 0.8) = 0.2$ (the question is too hard to provide a reliable signal). The peak occurs at $c_i = 0.5$, where $R_{\text{diff}} = 0.5$ — the Solver is maximally uncertain, exactly split between two (or more) answers, indicating the question lies at the frontier of its capabilities.

Why this form: This is the Goldilocks principle operationalized as a reward function. The $\min(c_i, 1 - c_i)$ function is symmetric around 0.5 and penalizes both extremes: very high consensus (question is trivially easy for the Solver) and very low consensus (question is impossibly hard, Solver is guessing randomly). This incentivizes the Proposer to discover questions where the Solver is on the edge of being able to reason correctly — questions that will provide a strong learning signal for the Solver during its training phase. An alternative formulation like $R_{\text{diff}} = 1 - c_i$ (rewarding low consensus) would incentivize the Proposer to generate impossibly hard questions; $R_{\text{diff}} = c_i$ (rewarding high consensus) would incentivize trivially easy questions. The $\min(c, 1-c)$ form is the only symmetric function that peaks at maximum uncertainty.

Easy-hard penalty (Section 2.4, Equation 8): If the average difficulty score across all successfully rendered images in the batch falls below a threshold $\delta_{\text{eh}} = 0.15$, a penalty $-\lambda_{\text{eh}} = -0.3$ is applied. This penalizes batches where, on average, the Solver is very confident on all hard questions — a signal that the Proposer is generating questions that are too easy.

Content-type diversity penalty (Section 2.4, Equation 9): To prevent the Proposer from converging to a single easy-to-render visual type (e.g., always generating bar charts), the system tracks the fraction $f_t$ of proposals in the current batch that share the same content type. If $f_t > \phi = 0.5$ (more than half the batch is a single type), a penalty proportional to $(f_t - \phi) / (1 - \phi)$ is subtracted, up to a maximum penalty of $-\lambda_{\text{ct}} = -0.15$ when $f_t = 1.0$. This is a soft constraint — the Proposer can still generate multiple proposals of the same type, but the reward penalty increases linearly as any single type dominates the batch.

Caption and question diversity bonus (Section 2.4, Equation 10): To penalize repetitive proposals within a batch, the system independently clusters captions, easy questions, and hard questions using agglomerative clustering based on pairwise BLEU distance (a measure of n-gram overlap). Let $s_x^{\text{(cap)}}$, $s_x^{\text{(eq)}}$, $s_x^{\text{(hq)}}$ denote the cluster share — the fraction of the batch that falls into the same cluster as proposal $x$. Let $u = 1/M$ be the uniform share for a batch of $M$ valid proposals. The combined diversity adjustment is a weighted sum of deviations from uniformity:

$r_{\text{div}} = -\text{clip}\!\Big(\big(w_c (s^{\text{(cap)}}_x - u) + w_e (s^{\text{(eq)}}_x - u) + w_h (s^{\text{(hq)}}_x - u)\big) \cdot M \cdot \lambda_{\text{div}},\; -\lambda_{\text{div}},\; \lambda_{\text{div}}\Big)$

where $w_c = 0.45$, $w_e = 0.20$, $w_h = 0.35$ are the weights for caption, easy question, and hard question diversity respectively, and $\lambda_{\text{div}} = 0.5$. Proposals in small clusters (share below uniform) receive a bonus (positive reward contribution), while proposals in large clusters (share above uniform) receive a penalty (negative reward contribution). The $\text{clip}$ function bounds the total diversity adjustment to $[-\lambda_{\text{div}}, \lambda_{\text{div}}] = [-0.5, 0.5]$, preventing any single proposal from dominating or being dominated by the diversity term.

What this computes: The diversity bonus measures how unusual a proposal is within its batch. If a caption is one of many nearly-identical captions, its cluster share will be high (e.g., $s_x^{\text{(cap)}} = 0.3$ if it shares a cluster with 30% of the batch), the deviation from uniformity $s_x^{\text{(cap)}} - u$ will be positive, and the resulting adjustment will be negative (penalty). If a caption is unique, its cluster share will be low (e.g., $s_x^{\text{(cap)}} = 0.02$ if it's in a cluster with one other proposal), the deviation will be negative, and the adjustment will be positive (bonus). The $M$ factor scales the adjustment by batch size so that the penalty per proposal is independent of how many proposals are in the batch.

Why this form: The diversity bonus addresses a fundamental problem in self-play systems: modal collapse. Without a diversity incentive, the Proposer will naturally converge to generating proposals that maximize the other reward components (high solvability, appropriate difficulty), which tend to be drawn from a narrow distribution of "reliably renderable" captions. The cluster-share mechanism penalizes this collapse gently — it doesn't force every proposal to be unique, but it makes it progressively more expensive to generate proposals similar to ones already in the batch. The weights $w_c = 0.45$, $w_e = 0.20$, $w_h = 0.35$ indicate that caption diversity (nearly half the weight) and hard-question diversity (about a third) are prioritized over easy-question diversity — this makes sense because easy questions are by design simple and formulaic, so penalizing easy-question repetition would be counterproductive.

Proposer training procedure (Section 2.2). The Proposer is trained first in each iteration. At each training step, the Proposer generates proposals. The Coder (frozen at its current checkpoint) receives the captions and produces $N = 4$ SVG code samples per caption via vLLM inference. Successfully rendered images, along with their easy and hard questions, are forwarded to the Solver (also frozen) to compute rewards. The Proposer's parameters are then updated via GRPO using the aggregated reward from Equation 4. The Proposer produces approximately 4,000 caption and question-answer pairs per training phase, which serve as the training data for the subsequent Coder training phase.

---

The Coder: From Textual Descriptions to Executable Visual Code

The Coder $\pi_D$ translates the Proposer's textual captions into executable code that renders visual images. It is initialized from the base VLM but operates as a text-to-code model — it receives a text caption and outputs SVG (or Python/matplotlib) code, never seeing images during its own generation. The Coder is trained second in each iteration, using data generated by the most recent Proposer checkpoint.

Training data construction for the Coder (Section 2.2). After the Proposer finishes its training phase, its best checkpoint is used to generate approximately 4,000 proposals (captions with associated questions and answers). The Coder is trained to generate SVG code from these captions. The crucial step is the data filtering applied before Coder training: for each caption, the system generates 4 code rollouts using the current (pre-training) Coder and checks the rendering success rate. Only captions where the rendering success rate falls in $[0.25, 0.75]$ are retained for training. This Goldilocks filter excludes captions that are trivially easy to render (success rate > 0.75 — the Coder can already handle them, so training on them provides no learning signal) and captions that are impossible to render (success rate < 0.25 — even multiple attempts fail, indicating the caption may be too complex, ambiguous, or outside the Coder's current capability range for productive learning). The filtered set of approximately 1,000–3,000 captions (depending on the base model's rendering capability) becomes the training dataset for the Coder.

The Coder's input and output structure (Appendix A.2). The Coder receives a prompt that includes the caption and the easy question/answer pair. The prompt template explicitly instructs: "Critical: The rendered image must contain the data needed to answer the Easy Question with the exact Easy Answer provided." and "Write raw SVG markup (starting with <svg ...>). Do not write Python code." The Coder outputs raw SVG code wrapped in markdown code fences. The SVG guidelines in the prompt specify: use a viewBox, use <text> for labels, font-size ≥ 12px, distinct colors, and self-contained rendering (no external resources). These constraints are designed to maximize the probability of successful rendering in the sandboxed SVG-to-PNG pipeline.

The Coder reward function (Section 2.5, Equation 11). The Coder's reward is simpler than the Proposer's because the Coder has a narrow, well-defined objective: produce executable code that renders faithfully to the caption. The reward is a weighted sum:

$R_D(C) = R_{\text{render}} + R_{\text{solv}} + R_{\text{diff}} - \lambda_{\text{err}}$

where $C$ is the generated code, $R_{\text{render}} = \mathds{1}_{\text{exec}}(C) \in \{0, 1\}$ is the execution indicator (1 if the code compiles and renders successfully, 0 otherwise), $R_{\text{solv}} \in [0, 1]$ is identically the Proposer's solvability score (the fraction of Solver rollouts on the easy question that match the intended answer, computed on the rendered image $I$ from code $C$), $R_{\text{diff}} \in [0, 1]$ is identically the Proposer's difficulty score (the $\min(c_i, 1-c_i)$ self-consistency measure for the hard question), and $\lambda_{\text{err}}$ is an error penalty: -0.1 if rendering fails, -0.05 if the code has a syntax error (detected before execution).

What this computes: The reward has four additive terms. The $R_{\text{render}}$ term provides a binary success/failure signal — the most fundamental requirement. The $R_{\text{solv}}$ term provides a continuous faithfulness signal — even if code renders, does the resulting image contain the information to answer the easy question correctly? The $R_{\text{diff}}$ term provides a continuous difficulty signal — does the rendered image support the hard question at an appropriate difficulty level? The $\lambda_{\text{err}}$ penalty provides a diagnostic signal — distinguishing between syntax errors (code is malformed, -0.05) and execution errors (code parses but the SVG renderer rejects it, -0.1), with the latter penalized more heavily because it's harder to diagnose and fix during training.

Why this form: The additive structure means the Coder is incentivized along multiple dimensions simultaneously. The $R_{\text{render}}$ term alone would encourage the Coder to generate any valid SVG, even if it depicted nothing related to the caption (e.g., an empty or trivial graphic). The $R_{\text{solv}}$ term ties the Coder's reward to the Proposer's original intent — if the rendering fails to convey the content needed to answer the easy question, the Coder receives $R_{\text{solv}} \approx 0$ even if the rendering succeeded ($R_{\text{render}} = 1$). The $R_{\text{diff}}$ term is included (rather than only using solvability) to prevent the Coder from learning to produce "cheat" images where answers are explicitly written as text annotations — an image that contains the answer as a text label would score high on solvability (the Solver just reads the text) but low on difficulty (the Solver is maximally confident, so $c_i \approx 1.0$ and $R_{\text{diff}} = \min(1, 0) = 0$), thus reducing the total reward. The error penalty granularity (-0.1 vs -0.05) provides the model with a more informative signal than a single "failure" penalty — syntax errors are easier to fix (the model just needs to produce valid code syntax) than rendering errors (which may require understanding SVG semantics or coordinate system constraints).

Coder training procedure (Section 2.2). The Coder is trained using the approximately 4,000 filtered captions from the Proposer's best checkpoint. At each training step, the Coder generates $N = 4$ SVG code samples per caption. Each code is executed in the rendering pipeline. Successfully rendered images are forwarded to the Solver (frozen at its current checkpoint) to compute $R_{\text{solv}}$ and $R_{\text{diff}}$. Failed renders receive $R_{\text{render}} = 0$ and the appropriate $\lambda_{\text{err}}$ penalty. The Coder is updated via GRPO (Equation 3) using the aggregated reward. The Coder saves a checkpoint every 20 steps.

The rendering pipeline (Appendix C). The SVG rendering pipeline is a critical infrastructure component. The process is: (1) the Coder's raw SVG string is extracted from the code fences in its output; (2) the string is passed to cairosvg, a Python SVG-to-PNG converter, with a per-snippet timeout of 30 seconds; (3) the resulting PNG is validated against maximum aspect ratio (100) and maximum dimension (16384 pixels) constraints — images exceeding these limits are discarded; (4) valid PNGs are base64-encoded for transmission to the vLLM/Solver input. Rendering is parallelized using ProcessPoolExecutor to handle multiple code snippets simultaneously. The 30-second timeout prevents infinite loops or computationally pathological SVGs from blocking the training pipeline. The dimension and aspect ratio constraints prevent memory exhaustion from unreasonably large generated images.

Why SVG primarily (rather than Python/matplotlib)? The paper mentions both SVG and Python as code generation targets but the prompt template (Appendix A.2) and rendering pipeline (Appendix C) focus on SVG. The SVG-only variant is the primary rendering pathway. SVG is chosen because: (a) it is a declarative markup language — the model writes geometric primitives directly, which is text-generation-friendly; (b) rendering SVG to PNG via cairosvg is fast, deterministic, and requires no Python interpreter sandbox (reducing security and complexity concerns); (c) SVG errors are typically isolated to specific elements (e.g., a malformed <text> tag) rather than crashing the entire rendering pipeline, making the reward signal more informative. Python/matplotlib is mentioned as an alternative but is not the primary pathway in the described implementation.

---

The Solver: Multimodal Reasoning via Test-Time Reinforcement Learning

The Solver $\pi_S$ is the multimodal reasoning agent — the component whose performance is ultimately evaluated on downstream benchmarks. It receives a rendered image and a hard question, and must produce a chain-of-thought reasoning trace followed by a final answer in \boxed{} format. The Solver is trained third in each iteration, using data generated by the most recent Proposer and Coder checkpoints.

Training data construction for the Solver (Section 2.2). After both the Proposer and Coder have completed their training phases, their best checkpoints are used to generate training data. The Proposer generates proposals, and the Coder renders them into images. The pipeline retains only successfully rendered images. For each image, the system evaluates the Solver (current pre-training checkpoint) on the easy question to verify solvability (accuracy > 0.5 across rollouts) and on the hard question to verify the difficulty range (accuracy between 0.27 and 0.75). Only examples passing both filters are retained for Solver training. This ensures the Solver trains on images it can meaningfully reason about (the easy question is answerable) and on hard questions that are challenging but not impossible (accuracy not too high, not too low). The filtered set constitutes the Solver's training dataset for that iteration.

The Solver's input and output structure (Appendix A.3). The Solver receives an <image> token (the base64-encoded PNG rendered by the Coder) followed by the hard question text. The prompt template instructs the model to "think step by step inside thinking ... tags" and then "give your final answer inside \boxed{} as a single number, single word, or short phrase only — no units, no full sentences." This structured output format is essential for: (a) extracting the final answer reliably (the system parses \boxed{...} to get the answer string), and (b) providing the format reward signal that reinforces the chain-of-thought + boxed-answer structure.

Test-Time Reinforcement Learning (TTRL) for the Solver (Section 2.6). The fundamental challenge for Solver training is that there is no ground-truth answer for the hard questions. The Proposer generates an intended answer (<hard_answer> in its XML output), but this answer may be incorrect (the Proposer is being trained itself and lacks the multimodal reasoning capability to verify its own answers). Using the Proposer's answer as ground truth would train the Solver to replicate the Proposer's mistakes — a form of confirmation bias that would prevent genuine improvement.

Instead, the Solver uses Test-Time Reinforcement Learning (TTRL) via majority voting, as introduced in Zuo et al. (2024, [60]). For a given image $I$ and hard question $q_{\text{hard}}$, the Solver generates $K = 8$ independent reasoning paths (with independent sampling at temperature 1.0). From these $K$ rollouts, the system identifies the silver answer $\bar{y}$ via majority vote:

$\bar{y} = \text{Mode}(\{y_1, \ldots, y_K\})$

That is, $\bar{y}$ is the most common answer among the $K$ rollouts. This silver answer is then treated as a pseudo-ground-truth label for training.

The Solver reward function (Section 2.6, Equations 12–14). For the $k$-th rollout, the reward is:

$R_S(y_k) = \alpha \cdot R_{\text{acc}}(y_k, \bar{y}) + (1 - \alpha) \cdot R_{\text{fmt}}(y_k)$

where $\alpha = 0.9$ is the weight on accuracy versus format, $R_{\text{acc}}$ is the accuracy reward (did $y_k$ produce the silver answer $\bar{y}$?), and $R_{\text{fmt}}$ is the format reward (did $y_k$ follow the thinking... \boxed{...} structure?).

Accuracy reward (Equation 13):

$R_{\text{acc}}(y_k, \bar{y}) = \mathds{1}\left( \hat{y}_k = \bar{y} \right)$

where $\hat{y}_k$ is the extracted final answer from the \boxed{...} content in $y_k$. This is a binary indicator: 1 if the rollout's answer matches the majority consensus, 0 otherwise.

Format reward (Equation 14):

$R_{\text{fmt}}(y_k) = \mathds{1}\left( y_k \text{ matches format } \texttt{thinking... response \boxed{...}} \right)$

This is a binary indicator of whether the output contains a thinking... block followed by a \boxed{...} final answer block, in that order. The specific format pattern is enforced by string matching against the expected template structure.

What this computes: For each of the $K = 8$ rollouts, the Solver receives a reward between 0 and 1. A rollout that both follows the format and matches the majority vote receives $R_S = 0.9 \times 1.0 + 0.1 \times 1.0 = 1.0$. A rollout that follows the format but disagrees with the majority receives $R_S = 0.9 \times 0.0 + 0.1 \times 1.0 = 0.1$. A rollout that fails the format but agrees with the majority receives $R_S = 0.9 \times 1.0 + 0.1 \times 0.0 = 0.9$. The 0.9/0.1 weighting heavily prioritizes answer accuracy over format compliance — format is important for reliable answer extraction but does not substitute for correct reasoning.

Why this form: The accuracy reward uses majority voting because, under reasonable assumptions about the Solver's error distribution, the majority vote is more reliable than any single rollout. If the Solver has a non-trivial probability of producing the correct answer (say, 40% correct, 60% spread across various incorrect answers), then with 8 independent rollouts, the probability that the correct answer wins the majority vote is substantially higher than 40% (because incorrect answers are distributed across multiple options, diluting their vote share). This is the standard wisdom-vs-crowds argument applied to self-training: even a weak model, when sampled multiple times, produces a consensus that outperforms its individual samples. The $\alpha = 0.9$ weight means the model receives 90% of its reward from agreeing with this consensus and only 10% from format compliance — the format reward serves as a regularizer that keeps the model's outputs parseable but does not dominate the objective.

The key risk of TTRL is that majority voting can amplify systematic errors: if the model has a consistent misconception (e.g., it always misapplies a particular geometric formula), the majority vote will converge to the wrong answer, and training on that wrong answer will reinforce the misconception. The paper does not directly address this risk, but the Goldilocks filtering on hard-question accuracy (0.27–0.75) partially mitigates it: questions where the model is systematically wrong (accuracy < 0.27, meaning fewer than ~2 out of 8 rollouts agree) are filtered out, preventing the model from training on cases where the majority vote is unreliable. Questions where the model is near-perfect (accuracy > 0.75) are also filtered out because they provide no learning signal.

Solver training procedure (Section 2.2, Table 5). The Solver is trained using the filtered training set from the current Proposer and Coder checkpoints. At each training step, the Solver receives $N = 8$ rollouts per image-question pair (the rollout batch size of 512 in Table 5 refers to the total number of training examples processed per step, not the number per prompt). The silver answer $\bar{y}$ is computed across all $K$ rollouts for each prompt. Each rollout receives a reward $R_S(y_k)$ computed as in Equation 12. The Solver is updated via GRPO (Equation 3) using these rewards. The Solver saves a checkpoint every 20 steps.

Why 8 rollouts rather than 5 (as in Proposer reward computation)? When the Solver is used to compute rewards for the Proposer (Section 2.4), $K = 5$ rollouts are used because the goal is efficient reward computation — the Proposer's updates depend on Solver feedback, and generating many rollouts per proposal would be prohibitively expensive (the Proposer generates many proposals, each requiring Solver inference). When the Solver is training itself, $K = 8$ rollouts are used because the goal is reliable pseudo-label generation — more rollouts produce a more reliable majority vote, and the Solver's own training directly benefits from this reliability. The tradeoff is purely computational: the Proposer's training loop calls the Solver as a reward evaluator (expensive per call, so minimize calls), while the Solver's training loop calls the Solver as the policy being trained (calls are already required, so increasing $K$ improves training signal quality at moderate additional cost).

---

Training Data Filtering and the Iterative Training Protocol

The entire system operates through three sequential training phases per iteration, with specific filtering at each stage. The filtering is not merely a quality-control step — it is the mechanism that implements the Goldilocks principle across the entire self-evolution loop, ensuring each role trains on data at the frontier of its current capabilities.

Proposer data construction (no filtering — generative). The Proposer does not consume filtered training data; it generates proposals from scratch at each training step based on content type seeds. The Proposer's training signal comes from the downstream rewards (solvability, difficulty, diversity) rather than from supervised learning on a fixed dataset.

Coder data filtering (Section 2.2). After the Proposer completes its training phase, its best checkpoint generates ~4,000 proposals. The Coder generates 4 rollouts per caption. The filter retains only captions where the rendering success rate (fraction of rollouts that produce a valid PNG) falls in $[0.25, 0.75]$. This excludes:

• Trivially easy captions (success rate > 0.75): the Coder can already render these reliably, so training on them provides minimal learning. The Coder would just be practicing skills it already has, wasting training compute.
• Impossibly hard captions (success rate < 0.25): the Coder fails so consistently on these that even with 4 attempts, successful renders are rare. Training on these would be dominated by failure signals (the $\lambda_{\text{err}}$ penalty), providing little useful gradient information about how to succeed.

The $[0.25, 0.75]$ range is a hyperparameter choice that balances exploration (the model should attempt challenging captions) with exploitation (the model should learn from captions where success is achievable). The value 0.25 as the lower bound means the Coder must succeed at least once in 4 attempts on average — captions failing all 4 rollouts are excluded entirely. The value 0.75 as the upper bound ensures the Coder still has room to improve (if it already succeeds 4 out of 4 times consistently, there is nothing to learn).

Solver data filtering (Section 2.2). After both Proposer and Coder complete their training, the best checkpoints are used to generate training images. For each successfully rendered image:

• The Solver is evaluated on the easy question across multiple rollouts. Only images where easy-question accuracy exceeds 0.5 are retained. This ensures the rendered image is faithful enough that the Solver can extract the basic information — if the Solver can't reliably answer the easy question, the rendering quality is insufficient for training, regardless of the hard question's quality.
• The Solver is evaluated on the hard question across multiple rollouts. Only questions where accuracy falls in $[0.27, 0.75]$ are retained. This is stricter than the Coder's filter (0.25 vs 0.27 lower bound) because Solver training is more sensitive to label quality — the TTRL pseudo-label (majority vote) is only reliable when the model has some signal above random chance. Accuracy below 0.27 suggests the Solver is essentially guessing randomly (with 8 rollouts, a random 2-way choice would have ~50% accuracy by chance, but multi-way or open-ended answers would have much lower accuracy by chance), making the majority vote an unreliable silver label. Accuracy above 0.75 suggests the question is too easy to provide a meaningful learning gradient.

The iterative training protocol (Section 3). Training proceeds in iterations, where one iteration is a complete pass through all three roles (Proposer → Coder → Solver). The paper reports results for 3 iterations (60 total training steps — 10 steps per role per iteration for 3 roles × 3 iterations). The protocol is:

1. Iteration 1, Proposer phase (10 steps): Proposer is initialized from base model, trained via GRPO with Coder and Solver frozen at base model checkpoints. Proposer checkpoint saved at step 20 (out of 60 total steps).

2. Iteration 1, Coder phase (10 steps): Coder is initialized from base model, trained on ~4,000 proposals from the step-20 Proposer checkpoint, with Solver frozen at base model checkpoint. Coder checkpoint saved at step 20.

3. Iteration 1, Solver phase (10 steps): Solver is initialized from base model, trained on filtered images from step-20 Proposer + step-20 Coder checkpoints. Solver checkpoint saved at step 20.

4. Iteration 2: Repeat steps 1–3, but now: Proposer trains from its step-20 checkpoint (not base model), with Coder frozen at step-20 and Solver frozen at step-20; then Coder trains from its step-20 checkpoint using proposals from the step-40 Proposer; then Solver trains from its step-20 checkpoint using images from step-40 Proposer + step-40 Coder. Checkpoints saved at step 40.

5. Iteration 3: Repeat with checkpoints from step 40, saving final checkpoints at step 60.

6. Extended training (Table 2): The paper additionally reports results for iterations 4 and 5 (steps 80 and 100) on the Qwen3-VL-8B-Instruct model, showing continued improvement (54.1% at iter 3, 54.2% at iter 4, 54.5% at iter 5), indicating the self-evolution process does not saturate within three iterations.

Why sequential rather than joint training: Training all three roles simultaneously (joint optimization) would create a non-stationary environment where each role's training signal becomes unstable because the other two roles are also changing. For example, if the Proposer's reward depends on the Solver's answers (via $R_{\text{solv}}$ and $R_{\text{diff}}$), but the Solver is simultaneously being updated, the Proposer's reward function changes from one gradient step to the next. This violates the stationarity assumption of policy gradient methods and can cause training instability or divergence. By freezing two roles while training the third, each role's reward function is stationary (determined by the current fixed checkpoints of the other two roles), and training converges more reliably. The sequential protocol effectively implements coordinate ascent in the space of role policies — each role is optimized to convergence given the current best policies of the other two roles, then the next role is optimized, and the cycle repeats.

The exploitation-exploration tradeoff in sequential training. The sequential protocol has a known weakness: each role can overfit to the specific behavior of the other roles at their current checkpoints. For example, the Proposer trained against a weak Solver (early iterations) might learn to generate questions that are only challenging for a weak Solver, rather than questions that will remain challenging as the Solver improves. The paper partially addresses this through the Goldilocks filtering: if the Proposer generates questions that are too easy for the improved Solver in later iterations, those questions will be filtered out (accuracy > 0.75 on hard questions), preventing them from being used for Solver training. However, the Proposer itself does not directly receive feedback on whether its questions remain appropriately challenging after the Solver improves — the Proposer's difficulty reward is computed against the current (frozen) Solver checkpoint, not the future Solver. This is an inherent limitation of the sequential training protocol and a recognized open problem in self-play systems.

---

Infrastructure: The Rendering and Inference Stack

The MM-Zero training loop depends on a substantial infrastructure layer that connects the three roles and executes the code generation and evaluation pipeline efficiently.

vLLM inference serving (Section 2.2). The system uses vLLM ([15]), a high-throughput LLM serving engine with PagedAttention memory management, to serve the Coder and Solver during reward computation. When the Proposer is being trained, vLLM service ports are initialized for both the Coder and the Solver (each at their current frozen checkpoint). The Proposer's generated captions are batched and sent to the Coder's vLLM instance for $N = 4$ independent code generations per caption. Successfully rendered images are then sent to the Solver's vLLM instance for $K = 5$ independent answer generations per question per image. Using vLLM (rather than running the models in-process) enables parallelization across GPUs and eliminates the overhead of repeatedly loading and unloading model weights. The per-role GPU allocation in Table 5 reflects this: 3 GPUs for the Proposer, 4 for the Coder, 8 for the Solver — each role's GPUs host its own vLLM instance or training process independently.

Parallel rendering (Appendix C). The SVG-to-PNG conversion is parallelized using Python's ProcessPoolExecutor. Each SVG string is rendered in a separate process with a 30-second timeout per snippet. The dimension constraints (max aspect ratio 100, max dimension 16384 pixels) are validated post-rendering. Base64 encoding converts the PNG binary to a text string that can be embedded in the Solver's input as an <image> data URI, following standard VLM input conventions.

Content type seeds for Proposer diversity. The Proposer receives content type seeds (from the set {data_chart, diagram, geometry, timeline, map, table, other}) as input to encourage diversity in the generated visual content. The content-type diversity penalty ($r_{\text{ct}}$ in Equation 9) works in conjunction with these seeds: the seeds provide the Proposer with a category to design for, and the penalty prevents the Proposer from always choosing the easiest-to-render category. The paper does not specify how seeds are selected (randomly? cyclically?), but the diversity penalty ensures that regardless of the selection mechanism, no single type dominates the training batch.

GPU hardware and training cost. All experiments use 8 RTX 6000/Pro GPUs with 96GB VRAM each (Table 4). The total trainable parameters for Qwen3-VL-8B-Instruct are approximately 8.77B (full fine-tuning, LoRA rank 0, vision tower trainable). The 8 GPUs are allocated across roles during their respective training phases, with the Solver phase using all 8 GPUs (the most memory-intensive due to image inputs) and the Proposer using only 3. The total wall-clock training time is not reported, but the 60-step protocol (10 steps per role × 3 roles × 3 iterations, expanded to 100 steps for the 5-iteration experiments) suggests each role receives only a modest number of gradient updates per iteration — this is a sample-efficient training paradigm because each GRPO update uses large batches (global batch size 64 for Coder and Solver) and informative reward signals (execution feedback, solvability, difficulty), rather than relying on many gradient steps with weak supervision.

LLM-as-a-judge for evaluation (Appendix A.4). For downstream benchmark evaluation (the results in Table 1), the paper does not use exact string matching. Instead, it employs Qwen2.5-14B-Instruct ([26]) as a judge: the judge LLM receives the question, the gold answer, and the model's answer, and outputs "Yes" or "No" to indicate correctness. The judge prompt specifies that numeric equivalence ("14 vs 14.0"), option equivalence ("A vs A."), and paraphrases should be considered correct. This is chosen "as it correlates more strongly with human judgments than exact string matching" and "its size balances judgment accuracy with inference efficiency." This is standard practice in VLM evaluation where open-ended answers (not just multiple choice) need to be graded, and the 14B judge model provides a pragmatic compromise between judgment quality and computational cost.

4. Key Insights and Innovations

Innovation 1: Identifying Programmatic Code Generation as the Missing Link for Zero-Data VLM Self-Evolution

The paper's most fundamental conceptual contribution is a diagnosis: the reason VLM self-evolution has remained tethered to external data is not that the proposer-solver paradigm is insufficient, but that it is architecturally incomplete for the multimodal setting. Prior work (VisPlay [11], Evolmm [31], V-Zero [34]) inherited the two-role structure from LLM self-evolution and treated the image modality as an external input to be sourced, filtered, and curated — effectively treating the visual data problem as a cost-reduction challenge rather than an architectural one. The dominant assumption was that reducing human annotation was the goal, and that static image datasets were an acceptable substitute for human labels.

MM-Zero identifies a deeper structural gap: in a multimodal self-evolution loop, the proposer's textual description is not the final input — the image is. A two-role architecture has no mechanism to bridge from text specification to visual realization. By introducing the Coder role — a dedicated agent that translates abstract textual descriptions into executable rendering code — the paper reframes the problem from "how do we efficiently source images?" to "how do we make the model generate images?" This is a categorical shift in framing, not an optimization of an existing approach.

The significance of this diagnosis extends beyond the specific implementation. It establishes a design principle for self-evolving multimodal systems: any modality that cannot be directly emitted by the base model's output head requires an intermediate generative role that translates the proposer's abstract specifications into concrete perceptual inputs. For VLMs, the missing modality is images, and code generation is the bridge. For video-language models, it would be video synthesis. For embodied agents, it would be action primitives. The tri-role architecture is not merely this paper's chosen framework — it is argued to be a necessary architectural pattern for closing self-evolution loops across modalities where the reasoning target cannot be produced through text generation alone.

Evidence for the structural necessity of the Coder comes from the ablation in Table 3 (though that ablation targets reward components, not role count) and from the qualitative evolution shown in Figure 3, where the Coder's rendering capability improves from "cluttered layouts with overlapping elements" at the base model stage to "polished" layouts demanding "genuine compositional reasoning" by iteration 3. The fact that the 4B model shows smaller gains (3 percentage points, from 50.2% to 53.4%) specifically because of its weaker rendering capability (40% render success rate vs. 70% for the 7B and 8B models, Figure 2) directly supports the claim that Coder quality is a bottleneck — when the Coder cannot reliably produce training images, the entire self-evolution loop starves for data.

Innovation 2: The Goldilocks Difficulty Signal as a Self-Supervised Curriculum Generator

A central challenge for any self-play system is curriculum design: how does the system generate training problems at the right difficulty level when no external teacher defines what "right" means? Prior self-evolution work (R-Zero [12] for LLMs) addressed this through simpler difficulty heuristics (e.g., rejection rates), but MM-Zero introduces a more principled and symmetric mechanism: the min(c_i, 1 - c_i) difficulty reward (Equation 7), which operationalizes the Goldilocks principle as a self-supervised signal derived entirely from the Solver's internal answer consistency.

What makes this distinctive is not the Goldilocks principle itself (which is a well-known concept [14]) but the specific instantiation as a reward function that penalizes both overconfidence and underconfidence symmetrically. The function min(c, 1-c) peaks at c = 0.5 — the point of maximum uncertainty where the Solver's own rollouts are evenly split between answers. This is a subtle design choice: a naïve difficulty reward of 1 - c would incentivize the Proposer to generate impossible questions where the Solver guesses randomly (rewarding only disagreement), while a reward of c would incentivize trivially easy questions (rewarding only agreement). The min formulation ensures the Proposer is incentivized to find the frontier — questions where the Solver has partial but incomplete mastery, which is precisely where learning occurs.

This mechanism effectively turns the entire tri-role system into an automatic curriculum generator. As the Solver improves (higher accuracy on previously frontier questions), those questions shift from the Goldilocks zone (producing R_diff ≈ 0.5) to the too-easy zone (producing R_diff ≈ 0.0), reducing the Proposer's reward for generating them. The Proposer is thus continuously pressured to discover harder questions that push the Solver back into the uncertainty frontier. The difficulty score serves double duty: it is simultaneously a training signal for the Proposer (rewarding question quality) and the Coder (rewarding faithful rendering that supports the hard question), and a filtering criterion for Solver training data (excluding both too-easy and too-hard questions). This multi-use design — one mechanism serving three distinct functions in the training loop — is an elegant systems contribution that reduces complexity while maintaining alignment between all three roles' objectives.

The ablation study (Section 4, Table 3) provides direct evidence for the importance of the solvability cap that works in concert with the difficulty reward. Without the 0.5 cap on R_solv, the system "increasingly embeds answers directly as text within the generated code, causing the solutions to appear explicitly in the rendered images" — the Proposer learns to cheat by making hard questions trivially answerable through visual annotation, which the difficulty reward alone cannot fully prevent because the Solver becomes artificially certain. The cap ensures the Proposer cannot compensate for low difficulty scores with high solvability scores, forcing genuine commitment to generating challenging questions.

Innovation 3: Test-Time Reinforcement Learning as a Pseudo-Labeling Mechanism for Reasoning with No Ground Truth

A critical obstacle to self-supervised training of reasoning models is the absence of correctness labels. In LLM self-evolution (R-Zero [12]), this was addressed through code execution feedback — the model generated code, executed it, and received deterministic correctness signals from the execution environment. MM-Zero cannot rely on this mechanism because the Solver is answering natural language reasoning questions about visual content, not generating executable code. The paper's solution — Test-Time Reinforcement Learning (TTRL) via majority voting [60] — is not individually novel (majority voting for self-training is well-established), but its integration into a self-evolving loop as the primary training signal for the reasoning agent is a significant architectural contribution.

What makes this distinctive is the asymmetric use of the Solver's own capabilities. The Solver is simultaneously the student (being trained via GRPO on hard questions) and the teacher (generating pseudo-labels via majority voting on those same hard questions). This is a form of self-training that could easily degrade into confirmation bias — if the Solver has systematic misconceptions, majority voting will reinforce them. The paper mitigates this risk through a specific design choice: the Goldilocks accuracy filtering on training data (retaining only questions where hard-question accuracy is between 0.27 and 0.75). This ensures the Solver only trains on questions where: (a) it has some signal (accuracy > 0.27, above chance), making majority voting informative, but (b) it still makes frequent errors (accuracy < 0.75), providing a learning gradient. Questions where the Solver is consistently wrong (< 0.27) are discarded because the majority vote would be an unreliable label; questions where it is consistently right (> 0.75) are discarded because there is nothing to learn. This filtering mechanism prevents the two failure modes of naive self-training: amplifying errors (by excluding low-accuracy data) and stagnation (by excluding high-accuracy data).

The Solver reward formulation (α = 0.9 for accuracy, 1-α = 0.1 for format; Equation 12) further reveals a deliberate design philosophy: the format reward is not merely a convenience for answer extraction but a structural regularizer that keeps the model's outputs parseable during self-training. Without it, the model could drift into output patterns where answer extraction fails, breaking the TTRL pseudo-labeling pipeline. The 0.9/0.1 split is asymmetric enough that format compliance cannot substitute for correctness (a model producing perfectly formatted wrong answers gets only 0.1 reward) but present enough to prevent the format from degrading. This is an understated but practically important design choice: self-training systems often fail because the output format degrades, making automated evaluation impossible, which in turn makes training impossible.

The TTRL approach also represents a conceptual bridge between test-time computation (majority voting for inference) and training-time supervision. Rather than treating majority voting as an inference-only technique (ensemble predictions at evaluation time) and ground-truth labels as a training-only resource, MM-Zero uses majority voting during training to generate the supervision signal, effectively amortizing test-time computation into training-time improvement. This blurs the traditional boundary between inference and training in a way that is conceptually significant for self-evolving systems: the same mechanism that could improve the model at test time is instead used to improve the model's weights, creating a positive feedback loop where improved weights lead to more reliable majority votes, which lead to better training signals, which lead to further improved weights.

Evidence for the effectiveness of TTRL comes from the Solver's iterative improvements in Table 1: the Qwen3-VL-8B-Instruct model improves from 50.7% to 54.1% average accuracy using only self-generated training data with TTRL pseudo-labels, and Table 2 shows continued improvement through iteration 5 (54.5%), indicating the TTRL signal does not asymptote quickly despite being derived from the model's own outputs. The concentration of gains on visual math benchmarks (MathVerse: 41.6% → 45.6%; MathVision: 31.5% → 39.8%) relative to general visual understanding is consistent with TTRL being most effective on tasks requiring multi-step reasoning where majority voting provides a stronger signal than on tasks requiring fine-grained perceptual discrimination.

Innovation 4: Multi-Role Sequential Training as a Stabilized Self-Play Protocol

While the tri-role architecture and specific reward designs are the paper's most visible contributions, the training protocol itself — sequential per-role GRPO updates with frozen companion roles — represents an understated but practically significant innovation in how multi-agent self-play systems are stabilized.

Prior multi-agent self-play systems (notably in game-playing domains like AlphaGo [30]) typically use joint or alternating training where all agents update simultaneously or in rapid alternation. This creates a non-stationary training environment: each agent's reward function changes as the other agents improve, violating the stationarity assumption underlying policy gradient methods. The standard solution is to use large replay buffers or slow-moving target networks, which add complexity and memory overhead. MM-Zero's protocol — train one role to convergence (or a fixed number of steps) while the other two are completely frozen, then move to the next role — is a coordinate ascent approach that guarantees each role's training environment is stationary during its update.

This is not a trivial engineering choice. The protocol has specific implications for what each role learns. When the Proposer trains against frozen Coder and Solver checkpoints, it learns to generate proposals optimized for those specific Coder and Solver capabilities. When the Coder subsequently trains on those proposals (now against a frozen Solver), it learns to render images optimized for that specific Solver checkpoint. When the Solver trains on the resulting images, it learns to reason about visual content generated by the now-improved Proposer-Coder pipeline. The sequential protocol thus creates a ratcheting effect: each role pushes against the current capabilities of the others, and the system as a whole improves as the sequence Proposer → Coder → Solver repeats.

The limitation is that each role can overfit to the specific behavioral quirks of the frozen checkpoints — a Proposer trained against a weak Solver might learn to generate questions that are only challenging for a weak Solver, not questions that will remain challenging as the Solver improves. The paper partially addresses this through the Goldilocks filtering (which excludes questions that become too easy after the Solver improves), but this is a post-hoc correction rather than a fundamental solution. The fact that performance continues improving through iteration 5 (Table 2, reaching 54.5% from 50.7%) suggests the overfitting concern is manageable at this scale, but it remains an open question whether the protocol would sustain improvement over many more iterations or eventually collapse due to each role specializing to the specific weaknesses of past checkpoints.

What makes this contribution distinctive is that it takes a known instability problem in multi-agent RL and solves it with architectural simplicity rather than algorithmic complexity. Instead of adding memory mechanisms, target networks, or trust-region constraints across agents, the protocol simply serializes training and freezes cross-role dependencies. This is a design insight with transfer value: other multi-agent self-improvement systems (e.g., multi-modal, multi-tool, multi-embodiment) could adopt the same sequential freezing pattern to stabilize training without modifying the underlying RL algorithm.

Evidence for the protocol's effectiveness comes from the consistent improvement across iterations (Table 1: all three base models improve from iteration 1 to iteration 3) and the lack of training collapse or oscillation (which would appear as intermediate checkpoints underperforming earlier ones). The Qwen3-VL-8B model shows monotonic improvement on average accuracy across all five reported checkpoints in Table 2 (50.7% → 53.0% → 53.1% → 54.1% → 54.2% → 54.5%), indicating stable convergence rather than the instability that would characterize joint training. The MiMo-VL-7B model's slight dip from iteration 2 (56.1%) to iteration 3 (56.0%) is within noise and does not indicate protocol instability.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates the Solver on 10 multimodal benchmarks spanning three categories. For General Visual Understanding: MMMU ([51]), MMMU-Pro ([52]), MM-Vet ([50]), and ChartQA ([25]). For Visual Mathematical Reasoning: MathVerse ([56]), MathVision ([2]), MathVista ([24]), and VisNumBench ([41]). For Hallucination Detection: HallusionBench ([9]) and MMSI ([47]). These cover chart interpretation, object recognition, geometric reasoning, mathematical problem-solving over visual inputs, and language-prior-vs-visual-grounding diagnostics. No training or validation split is used for any benchmark — the Solver is evaluated in a zero-shot setting on each benchmark's standard test set.

• Base model(s). Three base models are used: Qwen3-VL-4B-Instruct, Qwen3-VL-8B-Instruct ([46]), and Mimo-VL-7B-SFT ([44]). The Qwen3-VL family is chosen as a current representative of instruction-tuned VLMs with documented multimodal reasoning capabilities. The Mimo-VL-7B model is included to test generalization beyond the Qwen-VL architecture. The range of model sizes (4B, 7B, 8B parameters) allows analysis of how base capability (particularly code rendering success, which varies from ~40% for the 4B model to ~70% for the 7B and 8B models, Figure 2) affects self-evolution gains. All models use full fine-tuning (LoRA rank 0, vision tower trainable), with ~8.77B trainable parameters for the 8B model.

• Metrics. Accuracy is the sole metric, measured as the percentage of test questions where the model's final answer matches the ground truth. Answer correctness is determined not by exact string matching but by LLM-as-a-judge evaluation using Qwen2.5-14B-Instruct ([26]). For each question, the judge receives the question text, the gold answer, and the model's extracted answer, and outputs "Yes" or "No." The judge prompt (Appendix A.4) specifies that numeric equivalence ("14 vs 14.0"), option letter equivalence ("A vs A."), and semantic paraphrases should be accepted as correct. The final answer is extracted from the \boxed{...} content in the model's output. The paper reports per-benchmark accuracy and an average accuracy across all 10 benchmarks as the summary metric. The choice of LLM-as-a-judge over exact match is justified as correlating "more strongly with human judgments" and is standard practice in open-ended VLM evaluation.

• Baselines. The primary baseline is the untrained base model evaluated directly on all 10 benchmarks without any self-evolution. This is a natural comparison point given the paper's claim of zero-data self-evolution — the question is whether the model improves relative to its own starting point, not relative to a different training paradigm. The paper does not compare against: (a) VLMs trained with supervised fine-tuning on human-annotated visual reasoning data, (b) VLMs trained with self-evolution but using static image datasets (e.g., VisPlay ([11]), Evolmm ([31]), V-Zero ([34])), or (c) larger pretrained VLMs. This means the experimental comparison is strictly before-vs-after self-evolution rather than self-evolution vs. alternative training paradigms. The absence of comparisons to prior VLM self-evolution methods is notable — the paper claims to be "the first zero-data self-evolving framework" but does not empirically demonstrate superiority over methods that use minimal static data.

• Generation budget and compute accounting. The paper measures training compute through the number of training steps (10 per role per iteration, 60 total steps for 3 iterations, 100 total steps for 5 iterations) and the wall-clock GPU usage (8 RTX 6000/Pro 96GB GPUs). There is no formal FLOP accounting or comparison to alternative training paradigms (e.g., "how many FLOPs would supervised fine-tuning on equivalently-sized human-annotated data cost?"). At inference (evaluation) time, the Solver uses standard single-pass generation (the evaluation protocol does not specify temperature, sampling parameters, or whether majority voting is used at test time).

• Cross-validation or statistical protocol. None is reported. All results in Tables 1, 2, and 3 are single evaluation runs on the fixed test sets of each benchmark. There are no confidence intervals, standard deviations, or statistical significance tests. The paper reports all results to one decimal place of accuracy, and the differences between iterations are sometimes small (e.g., Qwen3-VL-8B at step 40: 53.1% vs. step 60: 54.1% vs. step 80: 54.2%), making it difficult to assess whether observed improvements are statistically reliable or within evaluation noise. The iterative nature of the training (3–5 iterations) provides some replication across checkpoints, but no formal significance testing is applied.

Main Quantitative Results

Overall Performance Gains from Self-Evolution

The headline result (Table 1) is that all three base models improve on average benchmark accuracy through MM-Zero self-evolution, with gains concentrated on visual math reasoning benchmarks. The specific numbers:

The 4B model's best performance occurs at iteration 1 (step 20), with subsequent iterations showing slight degradation (53.5% → 52.8% → 53.4%). The 8B model's performance is monotonically increasing across the three reported iterations (50.7% → 53.0% → 53.1% → 54.1%). The Mimo-VL-7B model peaks at iteration 2 (56.1%) and shows a negligible drop at iteration 3 (56.0%).

The gains are not uniform across benchmark categories. For Qwen3-VL-8B-Instruct, the visual math benchmarks show the largest improvements: MathVision jumps from 31.5% to 39.6% (+8.1 percentage points at iteration 3), VisNumBench from 47.7% to 53.2% (+5.5 points), and MathVerse from 41.6% to 45.1% (+3.5 points). General visual understanding benchmarks show more modest gains: MMMU from 55.8% to 58.3% (+2.5 points), MMMU-Pro from 46.6% to 53.0% (+6.4 points), while ChartQA actually declines slightly at some checkpoints (76.9% base → 79.6% at iteration 3, but 78.9% at iteration 1). Hallucination benchmarks show small and inconsistent changes: HallusionBench from 72.8% to 74.1% (+1.3 points at iteration 3), MMSI from 25.9% to 28.9% (+3.0 points at iteration 3). This pattern is consistent with the design of the self-evolution loop: the Proposer focuses on generating visual reasoning problems (charts, geometry, diagrams) that exercise the types of skills tested by visual math benchmarks, while general recognition and hallucination detection are not directly trained.

Extended Training Shows Continued Improvement

Table 2 reports extended training on Qwen3-VL-8B-Instruct through iteration 5 (step 100). The average accuracy continues to improve monotonically: 50.7% (base) → 53.0% (iter 1) → 53.1% (iter 2) → 54.1% (iter 3) → 54.2% (iter 4) → 54.5% (iter 5). The gains are diminishing — the largest jump is from base to iteration 1 (+2.3 points), with subsequent iterations adding approximately +0.1 to +1.0 points each. The visual math benchmarks continue to show the strongest trajectory: MathVision reaches 39.8% at iteration 5 (from 31.5% base), MathVerse reaches 45.6% (from 41.6%), and VisNumBench reaches 53.0% (from 47.7%). HallusionBench and MMSI show negligible net improvement from iteration 3 to iteration 5 (HallusionBench oscillates between 72.1–74.7%; MMSI between 28.7–30.0%).

The paper interprets the continued improvement through iteration 5 as evidence that "the self-evolution process does not saturate within three iterations" and raises the question of "how far can a model self-evolve from zero external data before reaching its ceiling without human supervision." However, the diminishing returns (iteration 4 adds 0.1 points, iteration 5 adds 0.3 points over iteration 4) suggest the process is approaching a plateau, at least for the 8B model. Whether this plateau is a fundamental limit of self-evolution or an artifact of the current training protocol (10 steps per role, fixed hyperparameters, no architectural improvements) is not explored.

Coder Rendering Quality Improves During Training

Figure 2 tracks two internal metrics throughout the 60-step (3-iteration) training protocol: Coder render success rate (the percentage of Coder rollouts that produce a valid PNG image) and rendered image solvability (the percentage of successfully rendered images where the Solver can answer the easy question correctly). Both metrics increase for all three base models. For Qwen3-VL-4B, render success rate rises from approximately 40% to >80%, and solvability rises from approximately 45% to >95%. For Qwen3-VL-8B, render success starts around 70%, climbs to >85%, while solvability rises from approximately 75% to >95%. For Mimo-VL-7B, render success starts around 60–65%, reaches approximately 85%, and solvability mirrors this trajectory.

The large gap between 4B and 7B/8B models in initial render success rate (~40% vs. ~65–70%) is the paper's explanation for why the 4B model shows the smallest gains (+3.3%) despite having the most room for improvement in absolute terms: "its image rendering success rate is only around 40%, compared to 70% for the 7B and 8B models as shown in Fig. 2, resulting in fewer valid training samples from the start." The 4B model generates fewer usable training images per iteration, starving the Solver of training data and limiting the benefits of the self-evolution loop. However, all three models ultimately reach similarly high solvability (>95%), suggesting that once rendering succeeds, the image quality is sufficient for the Solver.

Qualitative Evolution of Proposer and Coder Outputs

Figure 3 presents qualitative examples of Proposer outputs and Coder-rendered images across training iterations for the Qwen3-VL-8B model. The progression is striking:

• Base model: Generated figures show "cluttered layouts with overlapping elements, making them nearly unreadable." The corresponding hard questions are simple value-extraction tasks (e.g., "What was the total energy consumption in Q2 2023?").
• Iteration 1: Visual organization improves, but answers are often "embedded directly in the image as annotations, making them trivially easy" — a reward-hacking behavior that the difficulty score is designed to penalize.
• Iteration 2: Visuals become cleaner, and hard questions begin to require multi-step reasoning (e.g., "What is the ratio of the shaded region area to the area of the smaller circle?" from a geometric construction).
• Iteration 3: Layouts are polished, with distinct colors, clear labels, and compositional questions (e.g., "Using the 2020 baseline value and the 92% label for 2021, what was the approximate absolute emission level at the 2021 trough?" requiring extracting a value from a chart axis and applying a percentage annotation to compute a derived quantity).

The examples in Figure 3 are manually selected ("we sample 10 successfully rendered images and manually inspect them") rather than randomly sampled, so they illustrate what the system can produce rather than what it typically produces. The progression from "unreadable" to "compositional reasoning" is suggestive of genuine capability improvement, but without quantitative metrics on caption quality, question complexity, or answer accuracy distribution, it remains anecdotal evidence.

Ablation Studies and Robustness Checks

The paper includes one ablation study (Section 4, Table 3) examining two reward components unique to the tri-role architecture: the solvability capping and the content-type diversity penalty. Both ablations are conducted on Qwen3-VL-8B-Instruct using the same training protocol (3 iterations, 60 steps), and compared to the full MM-Zero model and the base model.

Solvability and difficulty balance (removing the cap on R_solv): In the full MM-Zero proposer reward (Equation 4), the solvability term is capped at 0.5: min(R_solv, 0.5) + R_diff. The ablation removes this cap and uses R_solv + R_diff directly. The theoretical concern is that without the cap, R_solv can reach 1.0 while R_diff maxes at 0.5, creating an imbalanced reward signal that favors easy-to-solve captions over challenging ones. The experimental result: the ablated model reaches 52.3% average accuracy at iteration 3, compared to 54.1% for full MM-Zero and 50.3% for the base model. The gain over base is +2.0 points vs. +3.8 points for the full model. The paper's qualitative analysis of 10 sampled captions from each iteration finds that without the cap, "the model increasingly embeds answers directly as text within the generated code, causing the solutions to appear explicitly in the rendered images" — a reward-hacking shortcut that the cap prevents. The performance gap emerges primarily in later iterations (iteration 1: 51.2% vs. 53.0% for full MM-Zero; iteration 3: 52.3% vs. 54.1%), consistent with the hypothesis that reward hacking takes time to develop as the model learns to exploit the imbalanced reward.

Content-type diversity (removing the r_ct penalty): In the full MM-Zero proposer reward (Equation 9), a content-type diversity penalty penalizes proposals when any single content type (e.g., data_chart, geometry) exceeds 50% of the batch. The ablation removes this penalty entirely. The result: the ablated model reaches only 49.4% at iteration 3 — worse than the base model's 50.3%. The trajectory is particularly revealing: iteration 1 achieves 51.7% (an improvement over base), but iteration 2 drops to 51.3%, and iteration 3 collapses to 49.4%. The paper's analysis of 10 samples from iterations 2 and 3 finds that "generated captions and rendered images increasingly converge toward a narrow subset of visual types, such as histograms, which are easy to render and yield higher rendering success rates." This modal collapse causes the Solver to overfit to easy-to-generate visual types, and its performance on the diverse evaluation benchmarks — which include chart types, geometric reasoning, tables, and diagrams beyond histograms — degrades. The ablation provides direct evidence that the diversity penalty is not merely a nice-to-have feature but is necessary to prevent training collapse in the self-evolution loop. Without it, the Proposer's natural tendency to exploit high-reward patterns (easy-to-render visual types) overwhelms the system's ability to maintain the diverse training distribution needed for general-purpose improvement.

No ablation on the number of roles (two-role vs. three-role): The paper does not compare MM-Zero's tri-role architecture to a two-role baseline (Proposer-Solver only, using a non-learned rendering mechanism or a frozen base model as Coder). This is a significant omission. The central claim of the paper is that the Coder role is architecturally necessary for zero-data VLM self-evolution, but without a comparison showing that a two-role system fails to achieve similar gains (or produces degenerate results), this claim remains a theoretical argument rather than an empirical finding. A natural ablation would train only Proposer and Solver, with the Proposer generating both captions and directly executable code (combining the Proposer and Coder roles into a single model) — this would test whether the Coder specialization is genuinely necessary or whether a single model can learn to both design and render visual scenes.

No ablation on training protocol (sequential vs. joint): The paper trains roles sequentially (freezing two while training the third) but does not compare this to joint training (updating all three roles simultaneously) or to alternative schedules (e.g., training Coder and Solver jointly after the Proposer). The sequential protocol is motivated by stationarity concerns, but without an empirical comparison showing joint training is unstable or underperforms, this remains a design choice rather than a validated necessity.

No ablation on the Goldilocks difficulty mechanism: The min(c_i, 1 - c_i) difficulty reward (Equation 7) is one of the paper's most distinctive design choices, but there is no ablation comparing it to alternative difficulty formulations (e.g., 1 - c_i rewarding only disagreement, c_i rewarding only agreement, or fixed difficulty targets). Without this comparison, it is unclear whether the min formulation is genuinely superior or merely one of several workable approaches.

No robustness check on hyperparameters: The paper reports a single set of hyperparameters (Table 5) without sensitivity analysis. Key quantities — the number of rollouts for Proposer reward computation (N=4 for Coder, K=5 for Solver), the Goldilocks filtering thresholds (0.25–0.75 for Coder, 0.27–0.75 for Solver), the solvability cap (0.5), the diversity penalty threshold (0.5), the accuracy-format weight (α=0.9) — are all fixed without exploration of alternatives. The 4B model's weaker performance is attributed to its low render success rate, but the paper does not test whether adjusting these thresholds for the 4B model (e.g., permitting lower render success rates in the Coder filter, or increasing the number of rollouts to compensate for lower-quality samples) could close the gap with larger models.

No robustness check on base model variance: All three base models are instruction-tuned VLMs from two model families (Qwen-VL and Mimo-VL). The paper does not test on base (pre-instruction-tuning) VLMs or on significantly different architectures (e.g., LLaVA-style models with different vision encoders). The consistent gains across three models suggest some robustness, but the sample is too small and too homogeneous to support strong generalization claims.

No analysis of generated data quality or diversity: While Figure 3 provides qualitative examples, the paper provides no quantitative metrics on the training data generated during self-evolution: caption length distribution, question difficulty distribution, visual content type distribution, answer correctness rates, or the relationship between data characteristics and Solver performance gains. The diversity penalty and Goldilocks filtering are designed to maintain data quality, but whether they succeed in doing so (beyond preventing the collapse observed in the ablation) is not systematically measured.

Critical Assessment

Do the Experiments Demonstrate Zero-Data Self-Evolution?

Yes, with qualification. MM-Zero demonstrates that VLMs can improve on multimodal benchmarks through training on entirely self-generated data — no external images, questions, or labels. The three base models (4B, 7B, 8B) all show average accuracy improvements of 3–5 percentage points (Table 1), the Solver's benchmark performance (Table 1, bottom rows) improves across iterations, and the Coder's rendering capability improves during training (Figure 2). The ablation showing that removing the diversity penalty causes training collapse (Table 3) provides evidence that the self-evolution loop is genuinely producing diverse training data rather than merely amplifying a narrow set of easy-to-generate examples.

However, the experiments demonstrate self-evolution relative to the base model's starting point — they do not demonstrate that self-evolution produces a better model than alternative training paradigms that use external data. The lack of comparison to: (a) VLMs trained with supervised fine-tuning on human-annotated visual reasoning data, (b) self-evolution methods that use static image datasets (VisPlay, Evolmm, V-Zero), or (c) simply training the base model longer with standard instruction-tuning objectives, means the paper cannot claim that zero-data self-evolution is more effective than data-dependent approaches, only that it is possible. This is a meaningful finding — demonstrating feasibility of a previously unattempted capability — but it is narrower than the framing in the abstract might suggest.

Do the Experiments Support the Claim That the Coder Role Is Architecturally Necessary?

Indirectly, but not empirically. The paper argues that the Coder is the key innovation enabling zero-data self-evolution by bridging text descriptions to visual images. The evidence for this claim is: (1) the qualitative evolution of Coder-rendered images shows improvement from "cluttered" to "polished" (Figure 3); (2) the 4B model's weaker performance is attributed to its lower Coder render success rate (Figure 2); (3) the Coder's rendering success rate improves throughout training (Figure 2). This evidence shows that the Coder works — it generates images that the Solver can use — but does not show that the Coder is necessary. A two-role system where the Proposer directly generates SVG code (eliminating the Coder as a separate role) might perform comparably. Without this ablation, the paper's central architectural claim remains theoretically motivated rather than empirically validated.

Do the Experiments Support the Claim That Gains Are Concentrated on Visual Math?

Yes, consistently across models. In Table 1, all three base models show their largest gains on visual math benchmarks (MathVision, MathVerse, MathVista, VisNumBench) and smaller or inconsistent gains on general visual understanding and hallucination benchmarks. For Qwen3-VL-8B-Instruct at iteration 3: MathVision +8.1 points, VisNumBench +5.5 points, MathVista -0.5 points (decline), MathVerse +3.5 points, compared to MMMU +2.5 points and HallusionBench +1.3 points. This pattern is consistent with the training data generation: the Proposer focuses on chart understanding, geometric constructions, and quantitative reasoning, which align with visual math benchmarks. The paper does not claim that MM-Zero improves general visual recognition, and the results support this modest scope.

Do the Experiments Support the Claim of Continuing Improvement Without Saturation?

Partially. Extended training on Qwen3-VL-8B-Instruct (Table 2) shows monotonic improvement from 50.7% (base) to 54.5% (iteration 5), but the gains after iteration 3 are small (+0.1 at iteration 4, +0.3 at iteration 5). The rate of improvement clearly diminishes. Whether this is "not saturated" or "approaching saturation" is a matter of interpretation. The paper acknowledges this uncertainty and frames it as an open question for future work. However, the lack of error bars makes it impossible to determine whether the iteration 4–5 improvements are statistically distinguishable from noise. Given the small absolute differences, a more conservative conclusion would be that performance is approaching a plateau within the current training paradigm, and that demonstrating continued improvement would require longer training runs (with associated cost) or modifications to the training protocol.

Genuine Weaknesses in the Experimental Design

No statistical reporting. All results in Tables 1, 2, and 3 are point estimates from single evaluation runs with no confidence intervals, standard deviations, or significance tests. The differences between iterations are sometimes very small (e.g., 54.1% at iteration 3 vs. 54.2% at iteration 4 for the 8B model in Table 2). Without variance estimates, readers cannot distinguish genuine improvement from evaluation noise. This is particularly problematic because LLM-as-a-judge evaluation introduces its own variance — the judge model's decisions are not perfectly deterministic or consistent, and the paper does not report judge agreement rates or inter-evaluator reliability.

No comparison to data-dependent methods. The paper claims to be the first zero-data VLM self-evolution framework, but establishing the value of "zero-data" requires showing what is gained or lost relative to methods that use data. If a self-evolution method using 1,000 seed images achieves 10-point improvements while MM-Zero achieves 4-point improvements, the zero-data property comes at a significant performance cost — a tradeoff the paper does not quantify.

Manual selection in qualitative examples. The examples in Figure 3 are "manually inspected" from a sample of "10 successfully rendered images." Selection bias is inherent — the authors choose which examples to present, and examples showing the most dramatic improvement are naturally more interesting to include. This is standard practice for qualitative figures, but readers should not interpret Figure 3 as representative of typical outputs.

Narrow evaluation domain. All 10 benchmarks are existing multimodal benchmarks focused on chart comprehension, geometric reasoning, math, and hallucination detection. These align well with the types of visual scenes the Coder can generate (charts, diagrams, geometric constructions), but they do not test whether self-evolution improves capabilities on photographic images, natural scenes, or real-world visual understanding — domains where SVG rendering cannot generate realistic training data. This is not a flaw of the experiments (the paper is upfront about the types of visual content it can generate), but it bounds the scope of the claimed improvements: MM-Zero improves visual reasoning on synthetic, programmatically-renderable visual content, not on general visual understanding.

Missing Experiments That Would Have Strengthened the Paper

Two-role baseline: Training only Proposer and Solver without a dedicated Coder, to test whether the Coder specialization provides benefits beyond a combined Proposer-Coder.

Comparison to data-dependent self-evolution: Running VisPlay ([11]) or a similar method on the same base model with a static image dataset, to quantify the performance difference between zero-data and data-dependent approaches.

Scaling the number of iterations further: The paper runs 5 iterations and observes approaching saturation. Running 10 or 20 iterations would clarify whether the plateau is real or whether further optimization (learning rate schedules, increased step counts per role, architectural changes) could unlock continued gains.

Ablation on the difficulty reward formulation: Comparing min(c, 1-c) to alternative difficulty signals (1-c, c, fixed difficulty targets, or no difficulty reward) to validate the specific Goldilocks formulation.

Ablation on the number of roles' rollout counts: The paper uses N=4 Coder rollouts and K=5 Solver rollouts for Proposer reward computation, and K=8 for Solver self-training. How sensitive are results to these choices? Could the 4B model's weaker performance be addressed by increasing rollouts to compensate for lower-quality samples?

Evaluation on a held-out generated dataset: The Solver is evaluated on external benchmarks, but no evaluation is performed on a held-out set of self-generated examples to measure whether the training data itself improves in quality and diversity across iterations. Such an analysis would connect the internal metrics (render success, solvability) to downstream performance more explicitly.

Wall-clock time and FLOP cost analysis: The paper reports GPU count and training steps but does not report total training time, FLOPs, or compare the computational cost of self-evolution to alternative training paradigms. This makes it difficult to assess whether the observed gains justify the computational investment, especially for the 4B model where gains are modest.

6. Limitations and Trade-offs

The Scalability Wall: Stronger Base Models Required to Bootstrap Effective Self-Evolution

The assumption or constraint. MM-Zero's self-evolution loop depends on the Coder's ability to generate valid, renderable SVG code at a sufficient success rate to produce enough training images for the Solver. The paper explicitly ties self-evolution gains to base model capability: "its image rendering success rate is only around 40%, compared to 70% for the 7B and 8B models as shown in Fig. 2, resulting in fewer valid training samples from the start" (Section 3.2, discussing the 4B model's results). The Proposer's ability to generate diverse, appropriately challenging questions — and the Solver's capacity to provide meaningful TTRL pseudo-labels — similarly depend on the base model's starting competence.

The consequence. This creates a minimum capability threshold below which self-evolution provides negligible returns or fails entirely. The 4B model achieves the smallest average gain (+3.3%, from 50.2% to 53.5%) despite having the most room for absolute improvement, and its best performance occurs at iteration 1 with subsequent iterations showing slight degradation — suggesting the self-evolution loop may not sustain long-term improvement when the base Coder cannot generate sufficient training data. Extending this logic, a model with a render success rate of, say, 10–20% (plausible for smaller VLMs or models with weaker code generation capabilities) would produce so few valid training images that the self-evolution loop effectively starves. The paper does not test on any model smaller than 4B parameters, leaving the lower bound of this capability threshold unknown. Moreover, the paper explicitly acknowledges it "was unable to validate this scaling trend on larger VLMs (e.g., 38B parameters) due to the prohibitive computational cost, leaving the full scaling behavior of self-evolving training as an important direction for future work" (Section 6). This means the scaling behavior is observed over only a ~2× parameter range (4B to 8B), and whether gains would continue to increase, plateau, or even reverse at larger scales is completely unknown.

What evidence exists in the paper. The per-model results in Table 1 and the render success rate curves in Figure 2 directly show the capability-dependence: the 7B and 8B models with ~65–70% initial render success achieve gains of 3.4–5.2 percentage points, while the 4B model with ~40% initial render success achieves only +3.3% with non-monotonic improvement. The ablation in Table 3 provides indirect evidence that training data quality matters critically — removing the diversity penalty causes training collapse, and removing the solvability cap causes reward hacking, both of which would be exacerbated in a lower-capability regime where the base model produces fewer high-quality proposals to begin with. However, the paper does not systematically vary base model capability (e.g., by testing intentionally degraded checkpoints or models with varying code generation proficiency) to map out the failure boundary.

Mitigation status. The paper acknowledges this limitation explicitly in Section 6 and frames it as future work: "scaling to larger base models would test whether stronger initial capabilities lead to proportionally better coders and visual outputs." It does not propose or test any mechanism to lower the capability threshold — e.g., warm-starting the Coder with synthetic SVG rendering data, using a simpler rendering target than SVG, or providing the Coder with a small number of seed examples. The current framework requires a base model that can already generate compilable SVG code with non-trivial probability, which restricts applicability to models of sufficient scale and code-generation proficiency.

---

The Rendering Modality Constraint: SVG-Only Generation Limits the Scope of Learnable Visual Reasoning

The assumption or constraint. MM-Zero generates all training images through SVG (and potentially Python/matplotlib) code execution, which constrains the visual content the system can produce to programmatically renderable scenes — charts, diagrams, geometric constructions, tables, and similar structured graphics. The paper focuses on visual content types listed in the Proposer prompt: "data_chart, diagram, geometry, timeline, map, table, other" (Appendix A.1). This deliberately excludes photographic images, natural scenes, real-world object photographs, medical images, satellite imagery, and any visual domain that cannot be synthesized through declarative vector graphics or plotting code.

The consequence. The self-evolution loop can only improve the Solver's reasoning capabilities on visual content that is structurally similar to the SVG-generated training distribution. This explains the paper's pattern of results: large gains on visual math benchmarks (MathVision: +8.1 points for 8B; VisNumBench: +5.5 points) that test chart reading, geometric reasoning, and quantitative visual analysis, versus small or inconsistent gains on general visual understanding benchmarks (MMMU: +2.5 points for 8B) and hallucination detection (HallusionBench: +1.3 points) that include photographs, natural images, and real-world scenes. The paper does not evaluate on any benchmark requiring photographic visual understanding (e.g., VQA-v2, GQA, TextVQA on natural images, or spatial reasoning on real-world scenes). A practitioner deploying MM-Zero to improve a VLM for, say, medical image diagnosis, autonomous driving perception, or general photographic question answering would find the self-evolution loop completely ineffective because the Coder cannot generate training images resembling the target domain. The improvement is therefore domain-bounded: MM-Zero improves visual reasoning in domains where programmatic rendering can produce realistic-enough synthetic data, and provides no benefit — or potentially negative transfer — on domains outside that scope.

What evidence exists in the paper. The per-benchmark results in Table 1 consistently show larger gains on visual math benchmarks than on general visual understanding benchmarks across all three models. For Qwen3-VL-8B-Instruct at iteration 3, the four visual math benchmarks show an average gain of approximately +5.4 points (computed across MathVerse, MathVision, MathVista, VisNumBench), while the four general visual understanding benchmarks show an average gain of approximately +1.5 points (MMMU, MMMU-Pro, MM-Vet, ChartQA). The paper does not evaluate on any benchmark dominated by photographic content, making the domain boundaries empirically observable but not systematically measured. The qualitative examples in Figure 3 (charts, geometric diagrams, stacked bar charts) reflect the SVG-renderable domain.

Mitigation status. The paper does not explicitly acknowledge this as a limitation. Section 7 mentions that "the framework could be extended to support more diverse tool usage beyond code generation, enabling agents to produce a wider variety of visual components (e.g., diagrams, plots, 3D renderings) and thereby create richer and more flexible training data such as 3D spatial reasoning for the solver." This suggests awareness of the modality constraint but frames it as an extensibility direction rather than a fundamental scope limitation. However, extending from SVG to 3D renderings does not address the core gap — photographic realism — which would require generative image models (e.g., diffusion models) rather than code-based rendering. The paper does not discuss whether integrating a text-to-image generator as an alternative Coder could expand the domain scope, or whether doing so would introduce new challenges (e.g., the generator's own biases, the difficulty of generating images with known ground-truth structure for question answering).

---

The Absence of Comparisons to Data-Dependent Self-Evolution Methods Leaves the "Zero-Data" Advantage Unquantified

The assumption or constraint. The paper frames its primary contribution as "the first zero-data self-evolving framework that improves VLM reasoning through multi-model self-evolution, requiring no external data" (Section 1). All experimental comparisons are against the untrained base model — the paper measures improvement from self-evolution relative to the starting point, not relative to alternative training paradigms. It does not compare MM-Zero to: (a) VLM self-evolution methods that use a static image dataset (VisPlay [11], Evolmm [31], V-Zero [34]), (b) supervised fine-tuning on human-annotated visual reasoning data of comparable volume, or (c) the base model fine-tuned with standard instruction-tuning objectives on synthetically generated but externally sourced data.

The consequence. The paper demonstrates that zero-data self-evolution is possible but does not demonstrate that it is competitive with data-dependent approaches. It is entirely plausible — and consistent with the paper's own results — that a self-evolution loop seeded with even a small number of high-quality real images (e.g., 1,000 photographs from existing datasets) would produce larger gains than the fully zero-data approach, especially on general visual understanding benchmarks where SVG-generated training data provides limited benefit. If a data-seeded method achieves double the accuracy improvement of MM-Zero at comparable computational cost, the "zero-data" property is a disadvantage, not an advantage, for practitioners who have access to any external data (which is effectively all practitioners — the paper's motivation emphasizes eliminating data cost, not enabling deployment in data-scarce scenarios where no images exist at all). Without these comparisons, readers cannot determine whether zero-data self-evolution is a pragmatic improvement over prior work or a proof-of-concept demonstrating a previously unattempted capability at the cost of reduced effectiveness compared to simpler, data-dependent approaches.

What evidence exists in the paper. None. The paper does not include any baseline other than the untrained base model. The related work section (Section 5.2) describes VisPlay, Evolmm, V-Zero, and Vision-Zero as prior self-evolution methods, and acknowledges that they are "conditioned on pre-existing static image datasets" — but never implements or compares against any of them. The ablation study (Section 4) ablates reward components within MM-Zero, not architectural choices relative to prior work.

Mitigation status. Not addressed. The paper does not acknowledge the absence of data-dependent baselines as a limitation. The abstract claims MM-Zero "achieves consistent performance gains across diverse VLM benchmarks," which is true relative to the base model but potentially misleading if a data-seeded method would achieve larger gains on the same benchmarks with the same computational budget. The paper does not even provide informal comparisons — e.g., reporting the base model's accuracy on the same benchmarks after supervised fine-tuning on a standard visual reasoning dataset, or citing prior work's reported improvements to contextualize the magnitude of MM-Zero's gains.

---

The Computational Cost of the Full Training Loop Is Unreported and Potentially Prohibitive for Broader Adoption

The assumption or constraint. MM-Zero's training protocol involves three sequential GRPO training phases per iteration, each requiring: (1) generating proposals from the Proposer, (2) running the Coder on each proposal with N=4 rollouts for reward computation and N=4 for training, (3) executing the SVG rendering pipeline on all generated code, (4) running the Solver on each rendered image with K=5 rollouts for Proposer reward computation and K=8 for Solver self-training, (5) performing GRPO policy updates on each role's parameters. The paper reports hardware (8 RTX 6000/Pro 96GB GPUs) and per-role configuration (Table 5, Appendix B) but does not report total wall-clock training time, total FLOPs, or GPU-hours for a complete 3-iteration (60-step) training run.

The consequence. Practitioners cannot assess whether the observed gains (+3–5 percentage points average accuracy) justify the computational investment. The paper reports 30 total training steps per iteration (10 per role) for 3 iterations = 90 steps per model, plus an additional 40 steps for the 5-iteration extended run (Table 2). Each Solver training step involves generating K=8 rollouts per training example and performing GRPO updates on an ~8.77B parameter model with full fine-tuning (vision tower trainable) — this is computationally intensive. The Proposer's reward computation requires serving the Coder and Solver via vLLM for each proposal batch, which adds inference overhead beyond the training FLOPs. Without any cost reporting, a practitioner considering whether to adopt MM-Zero cannot answer basic resource-planning questions: how many GPU-hours per percentage point of improvement? Is this more or less efficient than supervised fine-tuning on human-annotated data of comparable volume? How does the cost scale with model size (the paper only tested up to 8B parameters and explicitly lacked resources to test 38B)?

What evidence exists in the paper. The hardware specification and per-role configuration (Tables 4, 5 in Appendix B) provide enough detail to estimate cost but not to know it. The paper does not report: total training time, inference time for reward computation vs. training time for policy updates, the fraction of GPU-hours spent on each role, or the relative cost of Proposer reward computation (which requires frozen Coder and Solver inference) vs. Coder and Solver training. The GPU allocation across roles (3 GPUs for Proposer, 4 for Coder, 8 for Solver — Table 5) suggests that the Solver training phase dominates cost (using all 8 GPUs), but this is speculative without actual timing data.

Mitigation status. Not addressed. The paper does not mention computational cost as a limitation or provide any efficiency analysis. Section 6 (Limitations) focuses on the inability to test larger models due to "prohibitive computational cost" but does not extend this concern to the cost of training the reported model sizes. The paper also does not discuss whether the 3-iteration protocol could be shortened (e.g., fewer steps per role, fewer iterations) without significant performance loss, or whether the cost could be amortized across multiple downstream tasks. For a paper whose primary practical claim is enabling self-evolution "without any external data" — implying that data acquisition is the bottleneck being solved — the failure to account for the computational cost of replacing data with self-generated training is a significant omission.

---

The Difficulty Estimation Cost Is Excluded from the Self-Evolution Budget, Overstating the Framework's Data Independence

The assumption or constraint. MM-Zero's training data filtering (Section 2.2) requires evaluating the current Solver on every candidate training example to determine whether it falls within the Goldilocks difficulty range. Specifically: for Coder training data, the system generates N=4 code rollouts per caption to measure rendering success rate; for Solver training data, it evaluates the Solver on the easy question (multiple rollouts) to verify solvability > 0.5 and on the hard question (multiple rollouts) to verify accuracy in [0.27, 0.75]. These evaluations require running full Solver inference — processing images through the vision encoder and generating multi-step reasoning traces — on examples that may ultimately be discarded. The paper does not count this filtering computation in any training cost metric.

The consequence. The filtering pipeline consumes a potentially large fraction of the total computation without contributing directly to parameter updates. For example, if the Coder generates proposals where only 50% pass the rendering success filter and only 60% of those pass the Solver's difficulty filter, then approximately 70% of the Proposer's generated proposals are discarded after consuming Coder and Solver inference to evaluate them. This "filtering overhead" is analogous to the difficulty estimation cost the paper identifies in its analysis of prior work on compute-optimal test-time scaling — generating many samples to estimate difficulty before allocating the solution budget. MM-Zero's framework has the same structural issue: it generates and evaluates many examples to find the Goldilocks-appropriate subset, and the evaluation cost is not separately accounted for. If filtering overhead dominates total computation, then the "zero-data" property comes at the cost of substantial computational waste — the model spends most of its inference budget evaluating, then discarding, its own generated training data.

What evidence exists in the paper. The filtering thresholds are documented (Section 2.2) but no filtering ratios are reported — the paper does not state what fraction of generated proposals are retained after each filtering stage. The render success rate and solvability curves in Figure 2 show that early in training, solvability can be as low as 45% for the 4B model, implying that more than half of generated images fail the solvability filter. The Coder's render success rate starts at ~40% for the 4B model, meaning 60% of code rollouts fail rendering — but this is the per-rollout rate, not the per-caption retention rate (a caption is retained if 1–3 out of 4 rollouts succeed). The actual retention rates — and thus the filtering overhead — are not computed or reported.

Mitigation status. Not addressed. The paper does not discuss filtering cost, filtering efficiency, or whether filtering ratios improve over training iterations (which would reduce overhead as the Coder and Solver improve). The Goldilocks filtering is presented as a data quality mechanism without consideration of its computational cost. The paper also does not explore whether filtering could be made more efficient — e.g., by using a lightweight "difficulty predictor" trained on early filtering results to estimate whether a proposal will pass the filter without running full Solver inference, or by adaptively adjusting filtering thresholds to maintain a target retention rate that balances data quality with computational efficiency.

---

The Sequential Training Protocol Risks Overfitting Each Role to Past Checkpoints of the Other Roles

The assumption or constraint. The training protocol (Section 2.2) trains roles sequentially within each iteration: Proposer first (Coder and Solver frozen), then Coder (Proposer and Solver frozen), then Solver (Proposer and Coder frozen). At each phase, the training role optimizes against the current frozen checkpoint of the companion roles. In subsequent iterations, each role trains from its own previous checkpoint while the companion roles have also advanced. This protocol is justified as a stationarity guarantee for policy gradient methods — each role's reward function is fixed during its update.

The consequence. Each role can overfit to the specific behavioral quirks of the frozen companion checkpoints it trains against. For example, the Proposer at iteration 2 trains against the Coder's iteration 1 checkpoint and the Solver's iteration 1 checkpoint. It learns to generate proposals that are optimally challenging for those specific Coder and Solver versions. When the Coder subsequently improves at iteration 2, it may render these iteration-2 proposals differently (better or differently), and the Solver at iteration 2 may find them easier or harder than expected. This mismatch means the Proposer's difficulty calibration is always one step behind the Coder and Solver it will actually be paired with in the next iteration. Over many iterations, this lag could cause the Proposer to specialize to outdated companion behaviors rather than learning to generate proposals that remain appropriately challenging across a range of Coder and Solver capabilities.

The paper's results show that the 4B model's best performance occurs at iteration 1 (53.5%), with subsequent iterations degrading slightly (52.8% at iteration 2, 53.4% at iteration 3) — consistent with overfitting, though other explanations are possible. The Mimo-7B model shows a similar pattern: peak at iteration 2 (56.1%) with a slight drop at iteration 3 (56.0%). Only the 8B model shows monotonic improvement (50.7% → 53.0% → 53.1% → 54.1%). The diminishing returns in extended training (54.1% at iteration 3 → 54.2% at iteration 4 → 54.5% at iteration 5) at the 8B scale could reflect either approaching a capability ceiling or the Proposer's proposals becoming increasingly stale relative to the improving Solver.

What evidence exists in the paper. The per-iteration trajectories in Table 1 and Table 2 show the pattern described above, but the paper provides no diagnostic experiments to test whether overfitting is occurring. Specifically, the paper does not: (a) compare the Proposer's difficulty scores when evaluated against the Solver checkpoint it trained with vs. the next iteration's Solver checkpoint, (b) measure whether proposals generated at iteration 2 are systematically easier for the iteration 3 Solver than proposals generated at iteration 3, or (c) test an alternative protocol (e.g., joint training with a trust-region constraint across roles, or more frequent role alternation within each iteration to reduce staleness). The sequential protocol is presented as a design choice without empirical validation of its necessity or comparison to alternatives.

Mitigation status. Not addressed explicitly. The paper mentions the sequential training protocol in Section 2.2 as the training procedure but does not discuss its potential drawbacks or compare it to alternatives. The Goldilocks filtering provides a partial mitigation — if the Proposer generates questions that become too easy for the improved Solver, they will be filtered out of Solver training data (accuracy > 0.75), preventing the Solver from wasting training on trivial examples. However, this does not address the Proposer's own reward calibration: the Proposer continues to receive rewards based on the previous Solver's difficulty assessments, which may not accurately reflect difficulty for the current Solver. The Proposer thus lacks a direct incentive to generate questions that will challenge the Solver it will ultimately be paired with.

7. Implications and Future Directions

How This Work Changes the Landscape

MM-Zero represents a capability demonstration, not a paradigm shift — it shows for the first time that VLM reasoning can improve through self-evolution with literally zero external data, but the gains are moderate (+3–5 percentage points average accuracy), domain-bounded (concentrated on visual math where SVG can generate realistic-enough training images), and achieved through substantial unreported computation. The paper's primary effect on the field is to open a question rather than close one: given that zero-data VLM self-evolution is possible, under what conditions is it practically worthwhile compared to alternative training paradigms?

The paper's most significant contribution to the research landscape is reframing the visual data bottleneck as an architectural problem rather than a resource problem. Prior to MM-Zero, the dominant assumption in VLM self-evolution — inherited from LLM work — was that the challenge was reducing or eliminating human annotation, and that static image datasets were an acceptable, if costly, input to the self-improvement loop. MM-Zero demonstrates that this assumption is a choice, not a necessity: the image generation step can itself be part of the learned system, not an external dependency. This reframing suggests that future work on VLM training pipelines should consider generative visual synthesis as a trainable component of the system, not merely a data preparation step that happens before training begins. Even for methods that do use external data, incorporating a learned rendering or generation role could enable curriculum generation that adapts to the model's evolving capabilities — the Goldilocks principle that MM-Zero operationalizes through majority-vote self-consistency.

The paper also provides the first concrete evidence for a capability threshold in self-evolving systems. The 4B model's weak gains (+3.3%, non-monotonic improvement, best performance at iteration 1) compared to the 7B–8B models (+3.4–5.2%, monotonic or near-monotonic improvement) establish empirically what might have been predicted theoretically: self-evolution requires a minimum base competence to bootstrap. The 4B model's ~40% initial render success rate (vs. ~65–70% for larger models, Figure 2) means the Coder generates fewer usable training images, and the downstream effects cascade — fewer images means smaller Solver training sets, which means less improvement, which means the Proposer's difficulty signal doesn't escalate effectively because the Solver isn't improving fast enough. This finding has practical consequences: it suggests that zero-data self-evolution is not a universal capability that any VLM can use, but rather a capability that emerges at sufficient scale, analogous to how certain reasoning abilities in LLMs appear only above a parameter threshold. Researchers considering applying MM-Zero-like frameworks to smaller or weaker models should expect diminishing returns or outright failure.

The work partially reconciles a tension between two competing stories in the VLM post-training literature. One story, represented by R-Zero [12] and SPICE [20], says that LLMs can bootstrap complex reasoning from zero data through self-play. The other story, represented by VisPlay [11] and Evolmm [31], says that VLMs require external image datasets to achieve meaningful improvement, even if human labels are eliminated. MM-Zero shows that the truth is more nuanced: VLMs can self-evolve from zero data, but only in domains where the base model can already generate valid synthetic visual content. The limiting factor is not the self-play paradigm itself but the model's ability to produce the modality it needs to reason about. This suggests that the gap between LLM and VLM self-evolution is not fundamental — it is a reflection of the fact that generating credible images (via code) is harder than generating credible text, and that as base VLM capabilities improve, the gap should narrow. A VLM that can generate SVG as reliably as a comparably-sized LLM can generate Python would likely show self-evolution gains comparable to LLM self-play results.

However, the paper's impact is limited by what it does not show. The lack of comparison to data-dependent methods means the community cannot assess the cost-effectiveness tradeoff of zero-data self-evolution vs. simply using some seed data. The unreported computational cost means practitioners cannot compare MM-Zero to supervised fine-tuning on human-annotated data in terms of GPU-hours per accuracy point. And the domain-boundedness — gains on visual math, minimal gains on general visual understanding, no evaluation on photographic benchmarks — means the demonstrated capability applies to a specific, narrow category of visual reasoning tasks. These limitations prevent the paper from being a "drop-in replacement" recommendation for existing VLM training pipelines and instead position it as a research artifact that establishes feasibility and motivates further investigation.

Follow-Up Research This Work Enables

Direct comparison of MM-Zero to data-seeded self-evolution on identical base models. The paper's central claim — zero-data self-evolution — would be substantially strengthened or appropriately bounded by a controlled experiment that runs MM-Zero, VisPlay-like self-evolution with a static seed image dataset, and supervised fine-tuning on an equivalent volume of human-annotated visual reasoning data, all from the same base model (e.g., Qwen3-VL-8B-Instruct), with the same computational budget (GPU-hours), evaluated on the same 10 benchmarks. The key question is not whether MM-Zero works (the paper shows it does) but whether the zero-data property is worth the performance cost relative to methods that use seed data. If MM-Zero achieves 80% of the gain of a data-seeded method at 50% of the data acquisition cost, the tradeoff favors MM-Zero for cost-sensitive deployments. If MM-Zero achieves 30% of the gain of a data-seeded method at comparable or higher computational cost, the zero-data property is a liability, not an advantage. A strong follow-up would report: accuracy gains, total GPU-hours, and total wall-clock time for each method, along with per-benchmark breakdowns to identify which visual domains benefit from zero-data generation and which require real images.

Scaling the number of iterations to failure to identify the self-evolution ceiling. The paper's extended training on Qwen3-VL-8B (Table 2) shows diminishing returns: +2.3 points at iteration 1, +0.1 at iteration 2, +1.0 at iteration 3, +0.1 at iteration 4, +0.3 at iteration 5. This trajectory suggests an approaching plateau, but 5 iterations is insufficient to distinguish between (a) a temporary slowdown that would accelerate with hyperparameter adjustment, (b) a genuine capability ceiling where self-evolution provides no further benefit, and (c) continued slow improvement that would accumulate to meaningful gains over 20–50 iterations. A clear experiment would run MM-Zero on Qwen3-VL-8B for 20–30 iterations with periodic evaluation, plotting the full accuracy curve and fitting a functional form to estimate the asymptotic limit. If the curve plateaus at ~56% (only ~1.5 points above the iteration-5 result), the self-evolution ceiling is low and the framework's practical value is limited. If the curve continues a slow upward trajectory reaching ~60% at iteration 20, the framework is practically valuable but requires patience. This experiment would also reveal whether the Proposer's proposals become easier relative to the improving Solver (staleness), which could be diagnosed by measuring the Solver's accuracy on proposals from iteration N vs. proposals from iteration N+1 when evaluated at a fixed Solver checkpoint.

Ablation on the number of roles: does the Coder specialization matter? The paper argues that the Coder is the key architectural innovation enabling zero-data self-evolution, but never tests whether a two-role system (Proposer generates both captions and SVG code; Solver reasons about rendered images) could achieve comparable results. A direct ablation would train a "combined Proposer-Coder" model that receives a content type seed and outputs both a caption, easy/hard QA pairs, and SVG code in a single generation — this tests whether the Coder's specialized training (focused exclusively on code generation with execution feedback) provides benefits beyond what a single model can learn by generating everything at once. The hypothesis from the paper would predict that the specialized Coder produces higher-quality images (higher render success, higher solvability) than the combined model, which would in turn produce better Solver training data and larger downstream gains. If the combined model achieves similar performance, the tri-role architecture is unnecessarily complex and a simpler two-role system would be preferable. If the combined model fails (render success rate collapses, or proposals become degenerate), the Coder specialization is empirically validated as necessary.

Testing the Goldilocks difficulty formulation against alternatives. The min(c_i, 1 - c_i) difficulty reward (Equation 7) is one of the paper's most distinctive design choices, but there is no evidence that it outperforms simpler alternatives. A systematic comparison would train MM-Zero variants with: (a) R_diff = 1 - c_i (rewarding low consensus — incentivizing impossibly hard questions), (b) R_diff = c_i (rewarding high consensus — incentivizing easy questions), (c) R_diff = |c_i - 0.5| (rewarding distance from 0.5 — the opposite of Goldilocks, penalizing frontier questions), and (d) no difficulty reward (only solvability). The predictions are clear: (a) should cause the Proposer to generate nonsense questions; (b) should cause trivial questions and reward hacking (embedding answers in images); (c) should actively avoid the learning frontier; (d) should produce no curriculum escalation. Measuring the Solver's accuracy trajectory across these variants would validate whether the min formulation is genuinely necessary or merely one of several workable difficulty signals.

Domain expansion through generative image models. The paper's most significant scope limitation is that SVG rendering restricts training data to programmatically-renderable visual content — charts, diagrams, geometry — excluding photographic images and natural scenes. A natural extension would replace or supplement the SVG Coder with a text-to-image diffusion model (e.g., Stable Diffusion, DALL-E, or a smaller fine-tuned variant) capable of generating photographic scenes from captions. The challenge — and the research contribution — would be designing reward mechanisms and filtering pipelines for diffusion-generated images, which have no execution feedback (you cannot "compile" a diffusion output to check correctness) and no ground-truth structure (unlike SVG where visual elements correspond to specific code statements). Potential approaches include: (a) using a separate VLM to verify that generated images match the caption (a "visual faithfulness" score analogous to solvability), (b) using multiple diffusion samples with different seeds and measuring visual consistency as a proxy for caption faithfulness, or (c) combining SVG for structured elements (charts, labels) with diffusion for background/scene generation. A strong experiment would evaluate the resulting Solver on photographic VQA benchmarks (VQA-v2, GQA, TextVQA) that are completely out of scope for SVG-only MM-Zero, measuring whether self-evolution can now improve photographic visual reasoning.

Applying MM-Zero to video or 3D modalities. The paper's tri-role architecture — Proposer generates abstract concepts, Coder generates perceptible output, Solver reasons over that output — is not specific to 2D static images. The same structure could apply to video understanding (Coder generates video frames via code, e.g., matplotlib animations or simple game engines), 3D spatial reasoning (Coder generates 3D scenes via a rendering engine like PyTorch3D), or audio-visual reasoning (Coder generates plots with accompanying audio descriptions). The key requirement is that the Coder can generate the target modality via executable code, and that a verifiable reward signal exists (SVG has execution feedback; video rendering may have more complex failure modes). The research question is whether self-evolution gains in one modality (e.g., 3D spatial reasoning) transfer to related modalities, or whether each modality requires independent self-evolution loops. A strong experiment would train MM-Zero on SVG-generated 3D projections (wireframe renders, depth maps) and evaluate on 3D spatial reasoning benchmarks, measuring both within-domain improvement and cross-domain transfer to 2D chart reasoning.

Practical Applications and Downstream Use Cases

Automated curriculum generation for visual math tutoring systems. An educational technology company deploying a VLM-powered math tutor for students (where the model must interpret charts, graphs, and geometric diagrams and answer questions about them) faces a continuous content creation problem: the tutor needs an endless supply of fresh practice problems at appropriate difficulty levels for different students. MM-Zero's self-evolution loop directly addresses this: once deployed, the system can generate new chart-and-question pairs daily, with the Proposer calibrated to produce Goldilocks-difficulty problems (via the min(c, 1-c) reward) and the Coder rendering them programmatically. The Solver's accuracy improvements on MathVision (+8.1 points for the 8B model, Table 1) and MathVerse (+3.5 points) suggest the system can generate problems at competition-math difficulty levels, and the Coder's >95% final solvability (Figure 2) means the rendered charts are reliably interpretable. The key deployment advantage over static content is that the system naturally adapts difficulty as students improve — if the Solver represents a "simulated student," its improving accuracy pushes the Proposer to generate harder problems, creating an automatic difficulty progression without human curriculum designers.

Cost reduction for domain-specific VLM fine-tuning in chart-heavy industries. Financial analysis, business intelligence, and scientific research organizations frequently fine-tune VLMs on proprietary chart data (financial reports, experimental plots, sensor dashboards) for automated report generation or question answering. The traditional pipeline requires human experts to create chart-and-question training pairs — a costly bottleneck. MM-Zero offers a bootstrapping alternative: fine-tune the VLM on the organization's existing (limited) chart data to establish basic chart-reading capability, then deploy MM-Zero's self-evolution loop to generate additional training data from the Proposer-Coder pipeline, using the Goldilocks filtering to ensure only appropriately challenging examples are retained. The 7B model's +5.2 point gain (Table 1) shows that self-evolution can meaningfully improve a model already starting from reasonable base performance (~50% accuracy). The primary practical advantage is that the organization's human experts only need to create a seed dataset sufficient for the base model to achieve the minimum render success rate (roughly 40–50% based on the 4B vs. 7B comparison in Figure 2), after which the self-evolution loop handles expansion and difficulty escalation. The unreported computational cost is the main deployment barrier — without knowing the GPU-hours required per accuracy point, organizations cannot budget the self-evolution training against the cost of human data creation.

Synthetic data generation for VLM robustness testing and red-teaming. Security and reliability teams evaluating VLM deployments need to test model behavior on a diverse range of visual inputs, including edge cases and adversarial examples that are rare in natural data. MM-Zero's Proposer-Coder pipeline provides a programmatic test-case generator: the content-type diversity penalty (Equation 9) incentivizes generating across multiple visual formats (charts, diagrams, geometry, maps, tables), and the Proposer's continuous pressure to increase difficulty means the system naturally produces edge-case questions that stress the Solver's reasoning limits. The qualitative examples in Figure 3 show the progression from simple value extraction to compositional reasoning, demonstrating that the system can generate test cases that require multi-step inference. A red-teaming workflow would: (1) run MM-Zero's self-evolution loop to generate a large corpus of chart-and-question pairs, (2) evaluate the production VLM on this corpus, (3) identify failure patterns (e.g., specific chart types, question structures, or reasoning steps where accuracy drops), and (4) use these patterns to prioritize model improvements. The advantage over human-designed test cases is coverage — the Proposer samples from a diverse caption space (enforced by the diversity penalty and BLEU-clustering mechanism in Equation 10), potentially uncovering failure modes human test designers would not think to test.