MedXIAOHE: A Comprehensive Recipe for Building Medical MLLMs

🎯 Pitch

MedXIAOHE achieves a ~2.5× relative leap over the next best model on hard differential-diagnosis questions and beats GPT-5.2 Thinking on visual medical benchmarks—yet it still stumbles on mundane tasks like radiology report narratives, revealing a deep frontier in clinical reliability that raw benchmark scores hide.

1. Executive Summary

This report introduces MedXIAOHE, a medical vision-language foundation model that achieves state-of-the-art performance across 30+ public and in-house medical benchmarks, surpassing closed-source systems including GPT-5.2 Thinking, Gemini 3.0 Pro, and Gemini 2.5 Pro on multiple capabilities. The model builds on the Seed-VL architecture through an entity-aware continual pretraining framework that organizes heterogeneous medical corpora around a hierarchical Medical Entity Tree (a 1.4-million-entity taxonomy spanning 640B tokens across public web data, licensed books, and clinical images), explicitly targeting long-tail coverage gaps such as rare diseases and atypical presentations. For medical reasoning, MedXIAOHE incorporates diverse patterns via reinforcement learning and tool-augmented agentic training (enabling multi-step diagnostic reasoning with verifiable decision traces through general search, medical-specific drug and clinical record lookup, and image manipulation tools like zoom and rotate), while reliability is improved through user-preference rubrics, evidence-grounded reasoning, and low-hallucination long-form report generation with improved instruction following. The model outperforms GPT-5.2 Thinking on the MMMU medical series (87.53 vs. 79.73 on MMMU_val-Med and 73.88 vs. 71.85 on MMMU_Pro-Med) and achieves a relative improvement of ~2.5× over the next best model on Medbullets_op5 (94.16 vs. 88.31), establishing that a single medical VLM can match or exceed frontier general-domain models across visual diagnosis, medical imaging, diagnostic reasoning, text-based clinical QA, and report generation when domain-specific continual pretraining, multi-expert reasoning synthesis, and tool-augmented agentic training are combined — while also demonstrating that even this approach leaves headroom on narrative report generation (IU-Xray: 65.66 vs. Gemini 3.0 Pro's 73.51) and that hard diagnostic reasoning problems remain challenging (HLE-Med: 25.77, DiagnosisArena: 42.08).

2. Context and Motivation

The Core Problem: Medical VLMs Are Brittle, Not Clinically Ready

The fundamental problem this paper addresses is the gap between the impressive benchmark performance of existing medical vision-language models (VLMs) and their real-world clinical deployability. While multiple medical VLM systems have demonstrated strong results on curated datasets — Gemini in Medicine (Saab et al., 2024), Med-Gemini, LLaVA-Med, HuatuoGPT-Vision, and Med-Flamingo, among others — the paper argues that these results mask deeper reliability issues that make current models unsuitable for actual clinical workflows. The gap manifests along several dimensions that the paper systematically identifies:

Long-tail coverage failures. Medical knowledge follows a Zipfian distribution: common conditions (hypertension, diabetes, pneumonia) appear frequently in training data, while rare diseases, atypical presentations, and complex comorbidities occupy a sparse long tail. Current medical VLMs, the paper argues, inherit this distributional bias and tend to produce "confident but incorrect answers" when queried about rare conditions or unusual symptom combinations (Section 1, paragraph 2). This is particularly dangerous in medicine because rare diseases collectively affect a large number of patients — approximately 300 million people worldwide — and diagnostic errors in these cases carry disproportionate clinical consequences. The paper explicitly calls out that "many high-value scenarios sit in the medical long tail (e.g., rare diseases, atypical presentations, complex comorbidities), where coverage gaps and spurious shortcuts can surface as confident but incorrect answers."

Heterogeneous input modalities with uncontrolled variation. Clinical images are not the clean, standardized inputs that typical vision benchmarks provide. The paper highlights that "imaging protocols vary across institutions, visual inspection images are captured under uncontrolled conditions, and reports can contain subtle details whose omission or distortion changes clinical meaning" (Section 1). Photographs of skin lesions, for example, may be taken with different cameras, lighting conditions, angles, and backgrounds. Medical documents — lab reports, prescriptions, radiology findings — may suffer from perspective distortion, blur, shadows, glare, partial occlusion, and cluttered backgrounds when captured by patients or clinicians in routine practice. A model that performs well on de-identified, curated datasets may fail catastrophically when faced with the messy, uncontrolled inputs of real clinical workflows.

Hallucination in long-form generation. Medical report generation requires describing findings without introducing fabricated details — a capability the paper terms "faithfulness." The authors note that even when models generate seemingly fluent and coherent reports, they may introduce "subtle hallucinations or unsupported claims" (Section 1). In a clinical context, an incorrectly described lesion location, a hallucinated measurement, or an over-interpreted finding can lead to inappropriate treatment decisions. The paper frames this as a reliability problem that goes beyond simple accuracy metrics: a model can achieve high scores on standard benchmarks (which often evaluate against reference reports) while still producing clinically dangerous outputs because standard metrics like BLEU, ROUGE, or even RadGraph F1 do not adequately penalize unsupported assertions.

Fragmented and non-reproducible evaluation. The paper identifies a critical methodological problem in the medical VLM literature: "medical multimodal progress is often reported across a patchwork of benchmarks with inconsistent prompting, scoring, and contamination control, which makes results hard to reproduce and hard to compare" (Section 1). Different research groups select different subsets of benchmarks, use different prompt templates, apply different answer parsing strategies, and handle data contamination differently — making cross-paper comparisons essentially meaningless. Furthermore, "capabilities that strongly influence product readiness, such as medical report OCR robustness and lesion-level description faithfulness, are absent or under-measured from many public leaderboards." This creates a situation where model developers can optimize for leaderboard metrics that have weak correlation with clinical usefulness, while deployment-critical failure modes remain invisible.

Lack of interactive, tool-using reasoning. The paper argues that "beyond static question answering, real clinical workflows are interactive: high-quality assistance often requires gathering missing information, consulting references, and verifying intermediate hypotheses" (Section 1). Most medical VLMs are evaluated on single-turn question answering — show an image, ask a question, get an answer — but real clinical reasoning involves multi-turn information gathering, cross-referencing of evidence sources, and iterative hypothesis refinement. A model that cannot ask clarifying questions, look up drug interactions, or verify its reasoning against external evidence is fundamentally limited in clinical utility regardless of its single-turn accuracy.

Why This Problem Matters

The paper argues this problem is important along several dimensions:

Clinical impact and patient safety. The motivating context is that healthcare decision-making is "inherently multimodal, involving patient symptoms, medical histories, physical examinations, radiology and pathology images, and structured reports" (Section 1). If a medical VLM could reliably integrate these diverse signals, it could support clinicians in diagnosis, reduce errors, and improve access to specialist-level expertise in underserved areas. But the reliability failures documented above — hallucinations, long-tail gaps, brittleness to input variation — mean that deploying current models in clinical settings risks causing harm. The paper's framing is that the problem is not whether medical VLMs can achieve high benchmark scores (many can), but whether they can be trusted in practice.

Scalability of healthcare expertise. The paper situates medical VLMs within the broader challenge of healthcare access. Specialist expertise — radiology interpretation, dermatological diagnosis, complex differential diagnosis — is concentrated in well-resourced institutions and scarce in rural or low-resource settings. A genuinely reliable medical VLM could democratize access to this expertise. But the paper argues that current systems are not reliable enough to fulfill this promise, particularly because their failure modes are concentrated in the rare and complex cases where specialist input is most needed.

Methodological maturity of the field. By highlighting the fragmentation of evaluation practices, the paper argues that the medical VLM community faces a methodological bottleneck. Without standardized, reproducible, and clinically meaningful evaluation, it is difficult to determine whether progress on benchmarks translates to progress on clinical outcomes. The paper positions its Unified Med-VLM Benchmark as addressing this gap, arguing that establishing rigorous evaluation infrastructure is as important as developing better models.

Where Prior Approaches Fall Short

The paper identifies specific limitations in prior work, organized around what existing models do well versus what they cannot do:

Existing medical VLMs achieve strong benchmark scores but lack clinical robustness. The paper cites recent large-scale medical VLM efforts including Gemini in Medicine (Saab et al., 2024), Med-Gemini (Sellergren et al., 2025), Med-Flamingo (Moor et al., 2023), LLaVA-Med (Li et al., 2023), MedVLM-R1 (Pan et al., 2025), HuatuoGPT-Vision (Chen et al., 2024a), and LingShu (Xu et al., 2025b). These systems have demonstrated strong performance on standard medical VQA and report generation benchmarks, establishing that VLMs can acquire substantial medical knowledge. However, the paper argues that these systems were not designed or evaluated for the specific reliability challenges that matter in deployment: long-tail robustness, input variation tolerance, hallucination control in long-form outputs, and interactive reasoning with external tools. The benchmarks they were optimized for may not measure these capabilities.

General-domain VLMs are competitive on medical tasks but lack depth. The paper compares against GPT-5.2 Thinking, Gemini 3.0 Pro, and Gemini 2.5 Pro (Section 2, Table 1), showing that these general-domain systems perform remarkably well on medical benchmarks without medical-specific pretraining. However, the paper argues that general-domain VLMs suffer from specific gaps: they may lack coverage of specialized medical knowledge (rare diseases, drug interactions, institution-specific protocols), they are not trained on the specific modalities and image qualities encountered in clinical practice, and their instruction-following behavior may not align with medical safety requirements (e.g., handling prohibitive rules, maintaining appropriate uncertainty communication). The paper's positioning is that domain-specific training is necessary not because general models perform poorly — they don't — but because clinical deployment requires a level of reliability, coverage, and behavioral alignment that general pretraining alone cannot guarantee.

Knowledge coverage gaps in pretraining data are poorly characterized. The paper argues that prior medical VLM efforts have not systematically measured or addressed the knowledge coverage of their pretraining corpora. The Medical Entity Tree (Section 3.2) and the associated coverage metrics represent an attempt to make this coverage problem explicit and measurable. The paper shows (Table 2) that existing knowledge resources like CMeKG have a backward coverage score of only 0.79 relative to the paper's taxonomy, meaning they miss roughly 21% of the medical concepts in the paper's reference set — many of which are likely long-tail entities. This suggests that previous medical VLMs, trained on data aligned with these existing knowledge resources, inherit their coverage gaps.

Self-correction and iterative reasoning are underexplored. The paper notes that "simply prompting existing LLMs to correct their own mistakes tends to be largely ineffective for obtaining performance improvements on reasoning problems" (referencing the broader self-correction literature, though this specific claim is from the Scaffold provided context rather than this paper directly). In the medical domain, this means that naively asking a VLM to "check its work" or "think step by step" is unlikely to produce reliable improvements — the model needs structured training on clinical reasoning patterns, including how to gather additional evidence, how to verify hypotheses against external sources, and how to recognize when its initial assessment may be wrong. The paper's mid-training and post-training stages (Sections 4 and 5) are explicitly designed to address this gap through multi-expert reasoning synthesis, structured Chain-of-Thought training, and reinforcement learning with process supervision.

Tool use for medical reasoning is nascent. The paper cites DeepResearch-style systems (Tongyi DeepResearch, DeepDive; Team et al., 2025; Lu et al., 2025) that combine search with reasoning for general-domain fact-seeking. However, it argues that these systems are not well-adapted to medical reasoning specifically: "General DeepResearch models are good at fact-seeking tasks but lack medical reasoning expertise" (Section 4.1.2). Medical scenarios require multi-hop decision-making in consistent clinical contexts, appropriate tool selection (when to search the literature vs. when to look up a drug interaction vs. when to examine an image region more closely), and tool chaining (searching for drug information, then searching for clinical cases involving that drug, then integrating both sets of evidence). Prior work has not developed training methodologies specifically for these medical tool-use patterns.

Reasoning training can degrade perceptual capabilities. The paper identifies an "industry-wide bottleneck" that it calls the "perception-reasoning conflict": aggressive training on long reasoning chains can paradoxically degrade visual pattern recognition because "complex, text-heavy reasoning chains tend to detach from visual evidence, causing the model to operate purely in the textual space" (Section 4.1.1, Personalized Visual CoT subsection). This means that simply training medical VLMs to produce longer, more detailed reasoning — the natural approach to improving diagnostic accuracy — can backfire by making the model less attentive to subtle visual findings that are critical for correct diagnosis. The paper claims this is pervasive across the industry but has not been systematically addressed in prior work.

How This Paper Positions Itself

The paper frames its contributions not as a single novel technique but as a comprehensive recipe — a systematic integration of data curation, knowledge organization, multi-stage training, and unified evaluation that collectively addresses the gaps identified above. This is reflected in the subtitle "A Comprehensive Recipe for Building Medical MLLMs."

Entity-aware continual pretraining as a coverage solution. The Medical Entity Tree (MET) and associated data curation pipeline represent the paper's answer to the long-tail coverage problem. Rather than simply collecting more data (which would disproportionately add common conditions), the MET provides an explicit taxonomy that enables: (1) measuring knowledge coverage quantitatively, (2) identifying sparse regions for targeted data acquisition, and (3) balancing entity representation during training to prevent common conditions from dominating the optimization signal. This is positioned as a principled alternative to the implicit coverage that results from uncontrolled data collection — a shift from "hope that the data covers everything" to "verify and ensure that the data covers everything."

Structured post-training as a reasoning solution. The mid-training and post-training stages (Sections 4 and 5) are positioned as a systematic approach to building medical reasoning capabilities that existing models lack. The key innovation relative to prior work is the diversity of reasoning structures and the explicit handling of the perception-reasoning tradeoff: the paper synthesizes data covering basic logical inference, complex multi-hop deduction, tool-augmented agentic reasoning, anatomically grounded visual reasoning, and evidence-verified report generation — all within a unified training framework that explicitly balances these heterogeneous objectives through curriculum learning, progressive distribution warm-up, and iterative RFT-enhanced reinforcement learning. This is a departure from prior approaches that either focused on a single reasoning paradigm (e.g., chain-of-thought only, or tool use only) or trained on reasoning data without accounting for perceptual degradation.

Tool-augmented agentic training as an interactive reasoning solution. The integration of medical DeepResearch (search, scholar search, drug lookup, clinical record lookup) and medical image manipulation tools (zoom, rotate) is positioned as addressing the gap between static single-turn QA and the interactive, evidence-verifying workflows of real clinical practice. The paper argues that prior medical VLMs have largely been evaluated as passive question-answerers, while MedXIAOHE is explicitly trained to actively gather information, verify hypotheses, and produce reasoning traces that include tool calls and their results. The CureBench competition results (#1 place in both Internal Reasoning and Agentic Reasoning tracks at NeurIPS 2025) are cited as external validation of this capability.

Unified evaluation as a methodological contribution. The Unified Med-VLM Benchmark (Section 6) is positioned as addressing the evaluation fragmentation problem. By standardizing prompting, answer parsing, scoring, and contamination control across 30+ public benchmarks, and supplementing with targeted in-house benchmarks for capabilities not measured by public leaderboards (OCR robustness, lesion-level caption faithfulness), the paper provides both a measurement framework for its own model and a contribution to the broader community. The paper explicitly contrasts this with the status quo where "evaluation practice in the medical VLM literature remains fragmented" and "reported numbers are often not directly comparable due to protocol-level mismatches rather than model differences."

Architecture is not the differentiator. Notably, the paper explicitly states that MedXIAOHE "extends the Seed foundation through continual training rather than architecture re-design" (Section 2, Difference in Design), and that "for clinical-facing systems, reliability and coverage often depend more on (1) domain-specific data quality, (2) knowledge coverage in the long tail, and (3) post-training alignment for reasoning and preference, rather than on introducing complex architectural novelty." This positions the paper's contributions as systematic engineering and data-centric AI rather than architectural innovation — a deliberate choice that implies the field's progress on medical VLMs is bottlenecked not by model architecture but by data quality, knowledge coverage, and training methodology.

3. Technical Approach

This is primarily a systems and methodology paper whose core idea is that building a clinically reliable medical VLM requires systematic integration of three components—(1) entity-aware continual pretraining for broad and balanced medical knowledge coverage, (2) structured reasoning synthesis with explicit perception-reasoning balancing during mid-training, and (3) multi-layered reinforcement learning with tool-augmented agentic capabilities during post-training—each of which addresses specific failure modes that make existing medical VLMs unsuitable for real-world deployment, and which are validated through a unified evaluation framework that standardizes measurement across 30+ benchmarks.

3.1 Reader Orientation

MedXIAOHE is a medical vision-language model that takes images (X-rays, CT scans, pathology slides, clinical photos, medical documents) and text (patient histories, lab results, clinical questions) as input, and produces text outputs ranging from short diagnostic answers to detailed radiology reports with reasoning traces. The system solves the problem that existing medical VLMs fail in deployment because they lack broad knowledge coverage (especially for rare conditions), hallucinate details in long-form outputs, degrade perceptual accuracy when trained to reason extensively, and cannot interactively gather information or verify claims. The solution's "shape" is a three-stage training pipeline—continual pretraining, mid-training, post-training—that progressively builds knowledge breadth, reasoning depth, and interactive reliability, all scaffolded by an explicit medical knowledge taxonomy that ensures balanced coverage rather than passive data accumulation.

3.2 Big-Picture Architecture (Diagram in Words)

The MedXIAOHE system has seven major components organized in a pipeline, with some components operating in parallel during data construction:

Medical Entity Tree (MET) — a five-tier hierarchical taxonomy of 1.4 million medical entities constructed through LLM-driven extraction, clustering, and ReAct-agent conflict resolution from a 640B-token pretraining corpus. It serves as the coverage scaffold: it quantifies which medical concepts are represented in training data, identifies sparse regions for targeted acquisition, and enables balanced entity training to prevent common conditions from dominating optimization.
Continual Pretraining Corpus — a curated dataset of approximately 640 billion tokens assembled from four sources (310B public web, 280B licensed books/papers, 28B clinical lesion images, 22B open-source datasets), processed through a three-stage cleaning pipeline (hash-based deduplication, rule-based filtering/normalization, model-based quality classification) and organized into knowledge-grounded instruction data through the MET-guided synthesis pipeline.
Seed-VL Architecture — the pretrained Seed vision-language foundation model consisting of a Seed-ViT vision encoder, an MLP adapter projecting visual features into the LLM embedding space, and an autoregressive LLM decoder. This architecture processes interleaved image and text tokens and is kept architecturally unchanged throughout training.
Mid-Training Data Engine — a multi-dimensional synthesis pipeline that generates structured reasoning data across four paradigms: KG-guided multi-hop QA, multi-expert reject sampling with dual quality gates (outcome verification and thinking verification), reverse structured Chain-of-Thought with six-dimensional quality checks, and agentic training data including medical DeepResearch and think-with-image tool trajectories. This engine produces the diverse, high-quality supervision needed for the model to learn clinical reasoning patterns.
Continual Pretraining + Mid-Training Optimizer — a unified training process that applies data ordering (UMAP embedding → HDBSCAN clustering → compactness-based curriculum) to organize the heterogeneous corpus into semantically coherent groups with smooth transitions, enabling joint optimization of medical knowledge and multimodal reasoning without gradient interference from conflicting objectives. During mid-training, the vision encoder is unfrozen for joint optimization with the LLM.
Post-Training Pipeline — combines supervised fine-tuning on highest-quality human-annotated and synthetic data (including explicit instruction-following training with graph-structured reasoning) with RFT-enhanced iterative curriculum reinforcement learning. The RL stage uses a Multi-Layered Hybrid Reward System that routes samples to rule-based graders or rubric-based evaluators, fuses signals from outcome correctness, process supervision on reasoning traces, and soft shaping multipliers, and applies curricular phases (foundation → specialization → alignment) with dynamic hint-based scaffolding and entropy-aware exploration regulation.
Unified Med-VLM Benchmark — a standardized evaluation harness wrapping 30+ public benchmarks under consistent prompting, answer parsing, scoring, and contamination control, supplemented by in-house benchmarks targeting deployment-critical capabilities (real-world clinical image VQA, deformed medical document OCR, lesion-level caption faithfulness with dual-physician verification and reward-penalty scoring).

Information flows as follows: raw data from four sources enters the cleaning pipeline → cleaned text and image data pass through the MET-guided synthesis engine to produce knowledge-grounded training instances → these instances are organized into a compactness-ordered curriculum → the Seed-VL model undergoes single-stage continual pretraining on this curriculum with unfrozen vision encoder → mid-training data from the synthesis engine (structured CoT, multi-expert reasoning, agentic trajectories) is progressively mixed in → SFT on highest-quality human-annotated and synthetic data → RFT-enhanced iterative RL with multi-layered rewards → final MedXIAOHE model evaluated through the unified benchmark suite.

3.3 Roadmap for the Deep Dive

First, the Medical Entity Tree (Section 3.2) — the foundational knowledge organization structure that enables all subsequent data curation decisions. Understanding the MET is prerequisite to understanding why the pretraining data is constructed as it is.
Second, the continual pretraining data construction (Section 3.3) — how text knowledge, image knowledge, captions, OCR data, grounding data, and human-in-the-loop annotations are generated, filtered, and assembled, all guided by the MET's coverage requirements.
Third, the continual pretraining training recipe (Section 3.4) — the data ordering and curriculum strategy that enables joint optimization over heterogeneous data without gradient interference. This explains how the diverse data sources are combined, not just what they are.
Fourth, the mid-training data construction (Section 4.1) — the synthesis engine for internal reasoning (KG-guided QA, multi-expert reject sampling, structured CoT, personalized visual CoT) and agentic reasoning (medical DeepResearch, think-with-image). This is where the model acquires clinical reasoning patterns.
Fifth, the mid-training training recipe (Section 4.2) — the curriculum learning, ViT joint optimization, progressive distribution warm-up, and perception-reasoning balancing strategies that enable the model to learn reasoning without degrading visual capabilities.
Sixth, the post-training pipeline (Section 5) — SFT data construction and instruction following training, followed by the Multi-Layered Hybrid Reward System for RL and the RFT-enhanced iterative curriculum strategy. This is where the model's capabilities are refined and aligned for deployment.

3.4 Detailed, Sentence-Based Technical Breakdown

This paper is a comprehensive systems contribution that integrates data engineering, knowledge organization, multi-stage training, and unified evaluation into a single recipe. The technical approach spans data curation (how to build a balanced, high-coverage medical corpus), knowledge structuring (how to organize medical concepts into a measurable taxonomy), training methodology (how to optimize heterogeneous objectives without destructive interference), and reward design (how to supervise clinical reasoning with process-level feedback). I will walk through each component in the order it appears in the pipeline.

3.4.1 Medical Entity Tree (MET) Construction and Usage

The Medical Entity Tree is the central knowledge-organizing structure that distinguishes MedXIAOHE's data strategy from simple data accumulation. Rather than collecting massive amounts of medical text and hoping it covers the necessary concepts, the MET provides an explicit, measurable taxonomy that enables three capabilities: (1) quantifying knowledge coverage to identify gaps, (2) guiding targeted data acquisition to fill those gaps, and (3) balancing entity representation during training to prevent common conditions from overwhelming rare ones. The construction proceeds through three stages, progressively refining a flat list of extracted entities into a structured five-tier taxonomy of 1.4 million entities.

Stage 1: High-Efficiency Entity Extraction

The first stage extracts medical entities from the massive 640B-token pretraining corpus using LLMs with a batch chunking strategy. Rather than processing each sentence individually — which would be prohibitively slow — the system groups multiple sentences into consolidated input units and identifies entities within these batches. The authors report this yields a "processing speedup of approximately 30× while maintaining high extraction recall." The prompt design enforces strict JSON formatting and domain filtering, with explicit handling of null cases by outputting "None" when no medical entities are present. This structured output format significantly facilitates downstream parsing, since the extraction results are machine-readable and filterable.

The purpose of this stage is to convert a massive, unstructured text corpus into a manageable list of candidate medical entities — nouns and noun phrases that refer to diseases, symptoms, drugs, anatomical structures, procedures, and other medical concepts. This is a coarse extraction that prioritizes recall over precision: the goal is to capture as many potential entities as possible, with refinement happening in later stages.

Stage 2: Joint Typing and Hierarchical Clustering

The second stage evolves the flat entity list into a structured taxonomy. The key innovation here is that the prompt engineering strategy is advanced to simultaneously perform entity extraction AND type abstraction — that is, the LLM not only identifies entities but also assigns them to abstract categories. The prompt specifically addresses challenges in medical texts such as noisy characters (from OCR artifacts or formatting inconsistencies) and inappropriate entity granularity (splitting overly long phrases into constituent entities).

After extraction, a rigorous cleansing protocol is applied: entities appearing fewer than 10 times across the corpus are removed, and types containing fewer than 5 entities are filtered out. These thresholds balance noise reduction against coverage preservation — rare but legitimate entities that appear at least 10 times survive, while artifacts and extraction errors that appear only a handful of times are removed.

The clustering step applies K-Means clustering on entity embeddings, using the Silhouette Coefficient to dynamically optimize the cluster count for each type. The Silhouette Coefficient measures how similar an entity is to its own cluster compared to other clusters, with values ranging from -1 to 1. By using this metric to determine the optimal number of clusters, the method avoids the need to manually specify how many subcategories each entity type should have. The clustering is applied bottom-up: entity embeddings within each type are clustered into secondary types, then these secondary types are clustered into higher-level types, recursively constructing a multi-level hierarchy through frequency-weighted aggregation of embeddings. This means that more frequent entities contribute more strongly to the representation of their parent categories, which is appropriate because frequent entities are better estimated and more representative of the category.

Stage 3: Controlled Expansion via Tree Attachment

Upon completing Stages 1 and 2, the system has a concise, high-quality core taxonomy. Stage 3 scales this repository by integrating entities extracted from the broader corpus that were not captured in the core extraction. Three mechanisms are used:

Incremental Tree Attachment. Instead of rebuilding the tree from scratch (which would be computationally expensive and would discard the validated core structure), new candidate entities are grafted onto the existing tree. The LLM acts as a taxonomic expert, given the core taxonomy as context and instructed to identify the correct insertion path for each candidate entity without altering existing nodes. This preserves the integrity of the validated core while allowing the taxonomy to grow.

Deferred Insertion. As the taxonomy expands, fitting the entire tree structure into the LLM's context window becomes infeasible. The solution is to freeze the high-quality core tree as a context anchor: for each new entity, the LLM predicts the insertion path relative to this frozen core. These insertion operations are buffered and executed in batch updates, preventing context overflow while maintaining global consistency. The tradeoff is that relationships between newly inserted entities (which might suggest new intermediate categories) are not immediately captured — but this is acceptable because the core tree already provides the high-level structure, and the primary goal of expansion is coverage, not structural refinement.

Conflict Resolution via ReAct Agent. The deferred insertion strategy introduces structural ambiguity: a single entity may develop high affinity with multiple parent nodes. The paper gives the example of "Klüver-Bucy syndrome appearing under both 'Neurological Disorders' and 'Digestive Symptoms'" — both parent categories are legitimate because the syndrome has both neurological and behavioral manifestations that include eating disturbances. Solely relying on the LLM's internal parametric knowledge to resolve such conflicts would risk hallucinations (the LLM might confidently assign the entity to one category based on spurious pattern matching rather than clinical reasoning). To address this, the system implements a ReAct Agent framework (Yao et al., 2022) that requires the LLM to perform Retrieval-Augmented Generation (RAG) — searching external evidence in real time to arbitrate the correct classification. The paper identifies three benefits: (1) hallucination mitigation through grounding in objective external evidence rather than potentially outdated or hallucinated internal weights, (2) explainability and auditability through structured logs containing <Reasoning> and <SearchEvidence> fields that create a transparent audit trail, and (3) dynamic knowledge adaptation since the agent connects to the live web and can correctly classify newly discovered rare diseases or recently approved drugs beyond the pretraining data cutoff.

The final output is a rigorous five-tier taxonomy comprising 1.4 million entities.

Scalable Mapping of Corpus to MET

Once the taxonomy is built, the challenge shifts to efficiently mapping the tens of millions of entries in the pretraining corpus to their corresponding MET nodes. The system uses the Aho-Corasick (AC) Automaton algorithm, which constructs a Trie tree with Fail Pointers from the finalized entity dictionary. This achieves O(N) time complexity for corpus scanning, where N is the corpus size — meaning the processing time grows linearly with corpus size and is independent of dictionary size. This is critical because the alternative (parallel LLM scanning) would scale with both corpus size and entity count, making it infeasible for a 1.4M-entity taxonomy. The paper reports approximately 20 hours for the full scan, a 3× speedup over parallel LLM scanning.

Quantitative Evaluation of Knowledge Coverage

To validate that the MET actually provides comprehensive coverage, the paper introduces a semantic coverage metric called AMCS (presumably "Average Maximum Cosine Similarity"):

$\text{AMCS}(A, Ref) = \frac{1}{|A|} \sum_{i=1}^{|A|} \max_{j \in [1, |Ref|]} \text{CosineSimilarity}(\mathbf{a}_i, \mathbf{r}_j)$

where $\mathbf{a}_i$ and $\mathbf{r}_j$ are embedding vectors of entities in set $A$ (target set) and set $Ref$ (reference set) respectively, generated by a pretrained encoder such as Sentence-BERT. $|A|$ is the number of entities in the target set, and $|Ref|$ is the number of entities in the reference set.

What it computes: For each entity in the target set A, find the most similar entity in the reference set Ref (as measured by cosine similarity of their Sentence-BERT embeddings), then average these maximum similarities across all entities in A. This produces a score between 0 and 1 (since cosine similarity ranges from -1 to 1, but in practice embeddings of related medical concepts will be positively correlated).

What it enables: Two directional interpretations. Forward Coverage AMCS(A, Ref) measures what fraction of concepts in A are semantically present in Ref. A high value (>0.95) means the reference taxonomy covers nearly all concepts in the benchmark. Backward Coverage AMCS(Ref, A) measures what fraction of concepts in Ref are present in A. A lower value means the reference taxonomy contains substantial additional concepts beyond the benchmark, indicating superior comprehensiveness.

Why this form: Cosine similarity is appropriate because it measures directional similarity in embedding space independently of vector magnitude — two medical concepts that are semantically related will have similar embedding directions even if one is more specific than the other. The max operation ensures that for each target entity, the best-matching reference entity is found (avoiding penalizing the reference set for containing multiple related concepts where only one match is needed). The asymmetry of the metric (AMCS(A, Ref) ≠ AMCS(Ref, A)) is deliberate and useful: it separately measures inclusion of A in Ref and inclusion of Ref in A, providing a more nuanced picture than a symmetric similarity measure like average pairwise cosine similarity.

Results interpretation (Table 2): The MET achieves Forward Coverage scores of 0.96, 0.95, and 0.97 against Clinical Knowledge Data, Common Crawl Medical Corpus, and CMeKG respectively, confirming that it encompasses the vast majority of medical concepts in existing benchmarks. The Backward Coverage scores are lower (0.68, 0.89, 0.79), indicating that the MET contains substantial long-tail content not captured by these baselines. The paper interprets this as empirical validation that extracting entities directly from authoritative literature (medical textbooks, papers) yields a more comprehensive ontology than existing collections like CMeKG.

3.4.2 Continual Pretraining Data Construction

With the MET providing coverage guidance, the paper constructs a balanced pretraining dataset across five categories: text knowledge, image knowledge, captions, OCR, and grounding, plus a human-in-the-loop pipeline for other data types. Each category has a distinct synthesis or curation methodology designed to address specific failure modes.

Text Knowledge

The text knowledge construction addresses the core problem that raw web corpora are "often compromised by noise and factual hallucinations." The paper's solution is a knowledge-graph-grounded synthesis strategy that produces two categories of QA data from a large-scale medical KG.

The KG itself is constructed by collecting raw documents from medical textbooks and academic papers, segmenting and filtering them, then extracting precise medical entities and relationships. These entities and relationships form nodes and edges in the KG.

Atomic QA: Independent QA pairs are generated from individual KG nodes (entities) and edges (relationships). For example, a node describing "metformin" might generate "What is the mechanism of action of metformin?" and a relationship connecting "metformin" to "lactic acidosis" might generate "What rare but serious adverse effect is associated with metformin?" These atomic pairs inject discrete, factual knowledge points into the training data.

Aggregated QA: Subgraphs are sampled via random walks from the KG, and the interconnected entities and relationships are rephrased into coherent passages. An LLM then generates QA pairs that require synthesizing multiple pieces of information from these passages. For example, a subgraph connecting "type 2 diabetes," "metformin," "renal impairment," and "lactic acidosis" might generate a question like "Why is metformin contraindicated in patients with severe renal impairment?" that requires integrating knowledge about the drug's mechanism, its clearance pathway, and the risk of accumulation leading to lactic acidosis.

Both categories undergo filtering for non-self-contained expressions (questions that cannot be answered without external context that isn't provided) and high-quality sample selection based on Perplexity. Perplexity-based filtering removes samples where the model assigns low probability to the correct answer sequence, indicating that the sample is either too noisy, too difficult relative to the provided context, or contains factual errors that make the answer unpredictable from the question.

Image Knowledge

The image knowledge pipeline addresses the gap between the simple object-caption pairs used in general-domain VLM training and the nuanced anatomical and pathological understanding required for medical imaging. The core idea is to construct a "dense narrative" for every image — a detailed, structured description that goes far beyond sparse classification labels.

Dual-source data acquisition: The system uses two complementary data sources. Public datasets (MIMIC-CXR, CheXpert, ROCO) provide breadth — large volumes of samples covering common pathologies and standard anatomical views. However, these datasets typically have noisy or sparse annotations (e.g., binary labels like "Pneumonia: Positive"). In-house clinical data from partner medical institutions provide depth — high-resolution imaging with expert-level annotations from board-certified radiologists, including "precise lesion localization, measurements of lesion progression, subtle textural changes in tissue, and differential diagnosis notes." This dual-source strategy ensures the model sees both the diversity needed for generalization and the precision needed for clinical accuracy.

Caption construction pipeline: The transformation from sparse labels to dense narratives proceeds through three stages:

Stage 1 (Structured Attribute Extraction): All available metadata is aggregated into a structured JSON object. For public data, this includes DICOM headers and tabular labels. For in-house data, it includes structured reports with findings, impressions, and ROI coordinates. The JSON contains key-value pairs for anatomical location, finding type, severity, laterality, and technical quality. This provides a machine-readable intermediate representation.
Stage 2 (LLM-Driven Semantic Expansion): The structured attributes are fed into a general-domain LLM with a specific system prompt: "Act as a senior radiologist. Convert the following structured findings into a detailed diagnostic report description. Ensure medical terminology is accurate, describe the visual appearance of the findings." The LLM expands the sparse attributes into fluent, coherent medical text. The outputs are then reviewed by human annotators to ensure accuracy. This stage converts structured data into natural language that the VLM can learn from.
Stage 3 (Multi-Granularity Instruction Formatting): Rather than pairing each image with a single caption, the system generates diverse instruction-response pairs to cover different clinical intents: descriptive instructions ("Describe the findings in this chest X-ray"), diagnostic Q&A ("Is there evidence of pleural effusion?"), localization tasks ("Identify the region responsible for the patient's shortness of breath"), and reasoning prompts ("Based on the visual evidence, what is the most likely diagnosis and what follow-up is recommended?"). This diversity ensures the model learns to handle different types of clinical queries rather than overfitting to a single captioning style.

Caption

The caption pipeline specifically addresses hallucination control in medical image captions by introducing a rubric-based Generative Reward Model (GenRM). The synthesis process begins with an initial draft caption derived from the source image's existing alt text in large-scale raw data. Key medical entities (pharmaceutical names, disease names) are extracted from this draft and used to query an internal knowledge base. The retrieved domain knowledge, combined with the initial draft, is fed into an LLM which generates specific revision suggestions (e.g., "the description of lesion margins is imprecise — specify whether they are well-circumscribed or infiltrative"). Based on these expert-level recommendations, the initial draft undergoes modification to produce a refined caption.

The critical innovation is the GenRM-based filtration. Full-scale manual inspection of synthetic data is prohibitively expensive, so a subset is sampled for detailed human annotation, categorizing errors related to visual omissions, incorrect identifications, and lack of rigorous phrasing. These annotations are converted into structured rubrics — weighted scoring criteria for automated evaluation. The GenRM is trained via supervised fine-tuning followed by reinforcement learning to score captions based on weighted objective criteria including anatomical localization, morphological accuracy, and the strictness of evidence-based assertions. This enables automatic filtering of captions that hallucinate findings not actually present in the image or make ungrounded diagnostic claims.

OCR

The OCR data pipeline addresses the practical challenge of extracting information from medical documents (lab reports, prescriptions, radiology findings) that may suffer from perspective distortion, blur, shadows, and other real-world artifacts. The pipeline has three components:

Data source and collection: Medical images with text are selected from the medical database and processed through a standardized preprocessing pipeline including image deduplication, low-resolution filtering, aspect-ratio filtering, and corrupted image removal. To selectively recall medical-report-type images from the corpus, a two-stage weakly supervised strategy is used. First, a VLM generates pseudo-labels for 10K sampled images, with "medical report" as one target category. Second, these pseudo-labeled samples are used to train a ViT-Base classifier, which is applied to the full database to retrieve approximately 9 million medical report images.

Data distillation and filtering: For the recalled images, the Seed1.5-VL model distills OCR-style textual descriptions, yielding paired image-text supervision. Rule-based filtering removes approximately 0.01% of anomalous, noisy samples. A subset undergoes human review to ensure quality. This two-stage process (automated distillation followed by targeted human verification) balances scale and accuracy.

Data augmentation: A lightweight augmentation strategy randomly rotates 10% of images to improve robustness to orientation variations commonly observed in real-world medical documents (e.g., photos of reports taken at an angle).

Grounding

The grounding pipeline addresses the need for precise object localization — the ability to associate textual descriptions with specific image regions. The paper calls this capability "indispensable for verifiable medical diagnostics." The pipeline begins with an internal model captioning images, followed by an MLLM-based visual verifier that performs "rigorous, image-grounded entity extraction." The verifier cross-references textual mentions with visual evidence, extracting only entities that are verifiably present in the image while filtering out abstract concepts. This establishes a robust foundation for object-level recognition by ensuring that grounding annotations are tied to actual visual content rather than merely reflecting textual co-occurrence patterns.

Human-in-the-Loop Data Generation

For other data types, the paper develops a human-in-the-loop pipeline that combines physician annotation with automated synthesis and multi-stage filtering:

Label-based annotation framework: Licensed physicians participate in image annotation workflows including image filtering (discarding low-quality or non-medical images), label assignment (following standardized annotation guidelines covering anatomical regions, lesion types, lesion attributes, disease categories, and treatment recommendations), and result verification (review by a senior medical expert panel). The paper reports "entity-level precision and recall exceeding 95% after expert auditing."

VQA synthesis from annotated labels: The curated labels and images are used to prompt LLMs to automatically synthesize VQA instructions. LLMs are preferred over VLMs for this synthesis stage because of their stronger instruction-following capabilities. Rewriting techniques are incorporated to increase question complexity.

General VLM-based data filtering: A two-dimensional filtering process using a general-purpose VLM evaluates each synthesized VQA sample along two axes: image relevance (relevant/uncertain/irrelevant) and answer plausibility (plausible/uncertain/implausible). This removes "extreme or abnormal cases, such as answers that are irrelevant, hallucinated, template-based, or not grounded in the image content."

Expert VLM rollout-based sampling: A medical expert VLM trained on the filtered data performs pass@k rollouts — generating k independent answers for each question and comparing them against ground-truth annotations. Samples where k rollouts are fully or partially correct are retained. Samples with all incorrect outputs undergo additional sampling or physician manual review. This specifically targets "subtle medical inaccuracies that may pass general-purpose filtering."

3.4.3 Continual Pretraining Training Recipe

The training recipe addresses a fundamental optimization challenge: how to train on heterogeneous data (medical images, reports, captions, OCR text, general-domain QA) without gradient interference between conflicting objectives. The paper's central hypothesis is that "the conventional random shuffling paradigm obscures informative structure in multimodal medical data, including domain, modality, and task difficulty, thereby interleaving unrelated objectives and increasing gradient variance and conflicts during optimization."

Rather than separating training into multiple phases (which would introduce phase-transition forgetting and complicate hyperparameter tuning), the system uses a single-stage recipe that relies on data ordering — organizing the corpus into a structured curriculum where semantically similar samples appear together, enabling locally consistent gradient updates.

Data Ordering via Clustering

The procedure operates as follows:

Warm-up model training: A lightweight warm-up model is trained on a small random subset of the data (10%). This model serves as a feature extractor — it produces embeddings that capture semantic similarity between samples without requiring full training.
Embedding extraction: For every training sample, a fixed-length embedding is computed by extracting token representations from an intermediate decoder layer (chosen for efficiency rather than representational quality — earlier layers are faster to compute and still capture semantic content) followed by token-wise average pooling. This produces a single vector per sample.
Dimensionality reduction and clustering: The embeddings are reduced using UMAP (Uniform Manifold Approximation and Projection) for visualization and computational efficiency, then clustered using HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise). HDBSCAN is chosen because it does not require specifying the number of clusters in advance and can identify clusters of varying density — appropriate for medical data where some concept clusters (e.g., chest X-rays with pneumonia) are denser than others (e.g., rare disease descriptions).
Compactness-based curriculum: Clusters are ranked using a compactness score, defined as the mean pairwise Euclidean distance between embeddings of randomly sampled instances within each cluster. More compact clusters (lower scores) are treated as "easier" because they exhibit higher intra-cluster similarity and are expected to incur less gradient conflict — the model updates from samples in these clusters will be more consistent with each other. Clusters are ordered in ascending compactness score, creating an easy-to-hard curriculum.

Smooth Transitions and Replay

To avoid sharp optimization shifts at cluster boundaries (which could cause temporary performance degradation as the optimizer adapts to a new data distribution), the system introduces smooth transitions. Between consecutive clusters, there are mixed regions that jointly sample from both clusters and randomly shuffle the samples. Each mixed region additionally includes a small replay buffer drawn from earlier clusters, which reduces forgetting of previously learned concepts while maintaining forward progress through the curriculum.

This single-stage, ordered training schedule produces a unified pretraining process that preserves the benefits of joint optimization (all parameters are updated together, avoiding the need to freeze and unfreeze components) while systematically reducing gradient conflict through structured exposure. The paper claims this improves "stability and data efficiency without requiring explicit stage-wise parameter freezing or capability-specific phases."

3.4.4 Mid-Training Data Construction

Mid-training is positioned as a critical stage that bridges foundational pretraining (which provides broad medical knowledge) and post-training (which refines behavior for deployment). The objective is to systematically build the model's advanced reasoning abilities, addressing three goals: (1) developing atomic combinational skills (integrating individual capabilities like visual grounding and tool use into coherent workflows), (2) substantially improving medical reasoning (moving from simple information retrieval to rigorous multi-step clinical inference), and (3) building high-quality supervision for reinforcement learning initialization (supplying reliable policy priors for later alignment).

The core challenge is that large medical corpora "mainly provide final diagnoses and rarely include the explicit step-by-step reasoning needed to learn clinical logic." The paper's solution is a multi-dimensional data synthesis engine that systematically reconstructs the clinical reasoning space across four paradigms.

KG-Guided QA Synthesis

This paradigm leverages the medical KG to synthesize multi-hop QA pairs that require chaining multiple facts together. The process identifies "weak entities" — KG entities where the model's pass@k performance is low — indicating knowledge gaps. For these weak entities, supplementary information is retrieved via the retrieval system, extracting new entities and relationships to update the KG. Starting from weak entities, complex paths exceeding 5 hops are sampled via random walks. The terminal node is designated as the answer, while some intermediate attributes along the path are masked or obfuscated to increase difficulty. This generates complex multi-hop QA pairs and reasoning trajectories that specifically target the model's knowledge gaps.

For more complex problems requiring verifiable reasoning, a KG-based verifiable reasoning synthesis pipeline is used. For an existing QA pair, entities are extracted from both the question and answer and mapped to KG nodes through multi-stage matching. Potential multi-hop paths between question entities and answer entities are searched and re-ranked to retain only relevant and correct paths. Using these filtered reliable paths as conditions, the model generates verifiable reasoning trajectories — the key being that each step in the reasoning chain can be traced back to a specific KG edge, making the reasoning process auditable.

A critical design choice: the synthesis processes preserve "the native reasoning patterns of our base model. This ensures the training focus remains on the reasoning content itself, avoiding performance degradation potentially caused by pattern distribution shifts." In other words, the reasoning data uses the same style and format as the base model's natural outputs, so the model doesn't need to learn a new reasoning format simultaneously with learning to reason — it can focus entirely on the content.

Multi-Expert Reject Sampling

Complementing the structured KG-guided synthesis with broader cognitive diversity, this pipeline synthesizes reasoning trajectories by orchestrating expert models with varying cognitive configurations and employing best-of-N rejection sampling. The paper describes this as shifting from "single-source generation to an ensemble-driven distillation process" (Section 4.1.1). Multiple expert models — each with different strengths, training distributions, or reasoning styles — generate candidate reasoning trajectories for each input. Best-of-N rejection sampling selects the trajectories that produce correct answers, yielding a diverse set of valid reasoning patterns.

The critical quality control mechanism is a dual-quality gate:

Outcome-Verify: A comprehensive verification mechanism that checks whether the final answer is correct across multiple question types (judgment, selection, fill-in-the-blank, Q&A). This is a necessary but not sufficient condition — a correct answer can be reached through flawed reasoning.
Thinking-Verify: A mechanism that concurrently performs logical verification of the reasoning chain to ensure its soundness. This checks whether each step follows from previous steps and whether the chain as a whole supports the conclusion, independent of whether the conclusion happens to be correct.

This dual-gate system ensures that only trajectories with both correct outcomes AND valid reasoning processes are retained for training. The paper claims this enables "building fundamental medical reasoning capabilities from scratch" — meaning the model doesn't need to already have good reasoning to benefit from this data, because the expert models provide the initial reasoning patterns.

Structured Chain-of-Thought (CoT)

The paper identifies that standard forward rejection sampling often fails in complex medical tasks due to "open-ended divergence" — when generating long reasoning chains, the model may drift away from clinically valid reasoning even if it eventually produces the right answer. The solution is a Reverse Structured CoT Synthesis Framework that anchors reasoning on specific constraints.

For image-based tasks, the approach introduces a "low-hallucination caption" as an anchoring step to prevent perceptual errors from propagating through the reasoning chain. The model first generates a conservative, evidence-grounded description of what it sees in the image, then uses this caption (rather than directly reasoning from raw pixels) as the basis for clinical reasoning. For pure text inputs, the synthesis proceeds directly to the reasoning phase.

To simulate authentic clinical cognition, a "medical expert role-play strategy" is employed that strictly operates without hindsight — the reasoning path is constructed without premature reliance on the ground-truth answer. This ensures the model learns to reason forward from evidence to conclusion, rather than working backward from the answer (which would not generalize to cases where the answer is unknown).

The reasoning structure is enforced through a four-stage constraint:

Understanding: Clarifying what the question is asking and what information is provided.
Visual Observation / Knowledge Recall: For imaging data, performing systematic visual observation. For textual queries, recalling relevant prior medical knowledge.
Reasoning: Applying clinical logic to connect observations/knowledge to diagnostic hypotheses.
Conclusion: Deriving the final answer from the reasoning chain.

An automatic quality check validates the generated rationale across six dimensions: hindsight-free logic (whether the reasoning would be valid without knowing the answer), logical soundness (whether each inference step is valid), instruction following (whether the response addresses the specific question asked), answer consistency (whether the conclusion matches the reasoning), authentic thinking (whether the reasoning resembles human clinical cognition rather than pattern-matched justifications), and global consistency (whether all parts of the response are internally coherent). The paper emphasizes this is not "merely a linguistic expansion but a verifiable reproduction of expert clinical derivation, effectively eliminating reasoning drift."

Personalized Visual CoT

This paradigm addresses the "perception-reasoning conflict" — the empirically observed phenomenon that aggressive training on long reasoning chains can degrade visual pattern recognition. The paper's initial experiments confirmed that "complex, text-heavy reasoning chains tend to detach from visual evidence, causing the model to operate purely in the textual space and leading to significant regressions in fine-grained perceptual tasks."

The core insight is that "brevity preserves fidelity — where concise reasoning prevents the dilution of visual signals and sustains visual attention." Short chain-of-thought is inherently superior for perception-intensive tasks because it keeps the model's attention anchored to the visual input rather than allowing it to drift into extended textual reasoning that may become disconnected from the image.

The solution is a Personalized CoT Closed-Loop Pipeline that separates logical content from cognitive style:

Generate a low-hallucination caption that isolates core visual details (similar to the Structured CoT pipeline).
Construct a "first-principles logical core" that is strictly bound to visual observations — the reasoning cannot reference any finding not explicitly grounded in the caption.
Employ a multi-agent architecture (image analysts, reasoning experts, style transfer specialists) to adapt this core logic into task-specific formats. This means the same logical structure can be expressed in different styles appropriate for different clinical contexts (e.g., a terse radiology finding vs. a detailed differential diagnosis discussion).

A dual-layer quality gate validates both the structural logic (ensuring the reasoning is sound) and the post-adaptation style consistency (ensuring the reasoning is expressed appropriately for the task type). This pipeline is claimed to "produce high-fidelity training data that enhances reasoning depth without compromising the model's fundamental sensitivity to visual stimuli, successfully breaking the stagnation observed in standard one-size-fits-all reasoning paradigms."

Agentic Reasoning: Medical DeepResearch

This paradigm addresses the gap between static QA and the interactive, evidence-gathering workflows of real clinical practice. DeepResearch systems (Tongyi DeepResearch, DeepDive) couple search with reasoning: the model decomposes a problem step by step, verifies critical details against reliable external evidence, and then produces an answer. The paper specializes this for medicine by equipping the model with medical-specific tools and training data.

Tool set: The system integrates standard DeepResearch tools (General Search for web retrieval, Scholar Search for academic literature, Visit for opening specific URLs and extracting content) plus medical-specific tools — Search Drug for drug information (dosage, contraindications, interactions) and Search Clinical for clinical case records. These tools enable the model to look up information that may not be in its parametric knowledge, reducing hallucinations and knowledge cutoff issues.

Training data construction: The key challenge is that "general DeepResearch models are good at fact-seeking tasks but lack medical reasoning expertise." Medical scenarios require multi-hop decision-making in consistent clinical contexts, including appropriate tool selection and chaining. The paper creates multi-hop medical questions and solution paths by building entity chains through random walks on the internal medical KG. Two specific challenges are addressed:

Coverage: Using common diseases and drugs within each specialty as starting nodes and frequently restarting walks avoids biased learning concentrated in small subgraphs. Without frequent restarts, random walks would tend to revisit high-degree nodes, oversampling common entities.
Relational plausibility: Using an LLM-based sampling method to pick logically and clinically valid next-hop nodes from candidate neighbors. A purely random walk might chain unrelated entities together, producing nonsensical question chains that don't reflect real clinical reasoning.

The generated multi-hop questions undergo three-tier filtering to ensure difficulty and quality. The result is a dataset of challenging questions that genuinely require tool use — questions that cannot be answered from parametric knowledge alone and require the model to search for and integrate external evidence.

Two filtering stages are used during RL training specifically (Section 4.1.2):

Fine-grained perception filtering: A multi-agent uncertainty consensus framework identifies cases requiring localized, region-specific perception rather than global image context. Cases where multiple expert agents agree on the answer (low uncertainty) are filtered out as too easy; only cases with substantial disagreement (indicating genuine diagnostic challenge) are retained.
Task-with-instrument effectiveness filtering: An iterative process assesses example difficulty with a base model (without tools) and checks if tool-augmented inference improves correctness. Only tool-effective cases (where tools actually help) are retained. This ensures the model learns appropriate tool use rather than being trained on examples where tools are unnecessary or unhelpful. The filtering is interleaved with training to adapt to the model's evolving competence — as the model improves, previously difficult cases become easy and are filtered out, while new challenging cases are introduced.

Agentic Reasoning: Think with Medical Image

This paradigm addresses visual reasoning by integrating two complementary capabilities: anatomically grounded reasoning and image manipulation tools.

Anatomically grounded reasoning: Moving beyond generic object detection, the system establishes "anatomical landmarks as the fundamental anchors for vision-language alignment." The reasoning follows a structured Analyze-Reason-Conclude workflow. In the analysis phase, the model systematically traverses anatomical landmarks following radiological protocol, interleaving textual findings with precise bounding box coordinates. This explicitly binds reasoning steps to visual evidence — every claim about a finding is associated with a specific image region. In the reasoning phase, these grounded observations are synthesized to deduce pathologies, ensuring every diagnostic conclusion can be traced to a specific visual region.

Image manipulation tools: To address cases where visual limitations prevent fine-grained detection — for example, when a lesion is too small to characterize at the default image resolution — the system integrates "Think with Image" tools: Zoom In for magnifying regions of interest (enabling inspection of subtle lesions, tissue texture, and lesion margins) and Rotate for maintaining spatial orientation during interpretation of complex images. These tools are particularly valuable in medical imaging where diagnostic cues may be visible only at specific magnifications or orientations.

Training data construction and filtering: The training dataset is built from radiology imaging data (X-rays, CT scans, MRI) and public de-identified radiology datasets with metadata and expert annotations. A two-stage filtering pipeline selects cases for RL training:

Fine-grained perception filtering: A multi-agent uncertainty consensus framework selects cases requiring localized perception, filtering out examples solvable from global image context.
Task-with-instrument effectiveness filtering: An iterative process checks whether tool-augmented inference improves over base model performance, retaining only tool-effective cases. This is interleaved with training to adapt to the model's evolving competence.

The resulting RL dataset focuses on diagnostically challenging, tool-enhanced cases, guiding the model to learn appropriate tool use — when to zoom, when to rotate, and how to integrate the additional visual information from these operations into the diagnostic reasoning process.

3.4.5 Mid-Training Training Recipe

The mid-training recipe addresses the challenge of synthesizing the heterogeneous cognitive architectures (foundational logic, adaptive execution, perceptual grounding) into a unified clinical system without destructive interference. The paper implements four strategic pillars:

Curriculum Learning: A progressive training strategy builds medical reasoning capabilities in sequence: foundational logic initialization → complex long-horizon deduction → perceptual-logic alignment → adaptive hybrid fusion. This hierarchical approach mimics human cognitive development — building simple skills before complex ones, then integrating them. Each phase addresses specific limitations: missing reasoning baselines in early training, logical discontinuities in complex tasks, and the perception-reasoning trade-off in later stages.

ViT Joint Training Optimization: The visual backbone (ViT) is unfrozen to enable joint optimization with the language model. This is motivated by the observation that "medical VQA relies heavily on distinguishing minute visual discrepancies that frozen encoders might overlook." Unlike general-domain VLMs where frozen vision encoders often suffice (because the visual concepts are coarser and better captured by pretrained features), medical imaging requires sensitivity to density changes and subtle anomalies that may not be well-represented in general-domain pretraining. Joint optimization allows the extracted visual features to adapt specifically to medical image characteristics.

Progressive Distribution Warm-up: To mitigate catastrophic forgetting and capability imbalance from the distributional shift between pretraining and mid-training, the proportion of target reasoning data is gradually annealed throughout the training steps. Rather than abruptly switching from pretraining data to reasoning data, the system incrementally increases the density of complex medical reasoning samples while maintaining a replay buffer of general-domain data. This "smoothes the distribution fluctuations, allowing the model to securely anchor new capabilities without eroding its original competencies."

Balancing Perception and Reasoning: A dual-track alignment strategy reconciles the verbose logic of complex derivations with fine-grained perception. This strategy features anatomy-centered grounding (tying reasoning steps to specific visual regions) and tool-augmented trajectories derived from the medical KG. The tool augmentation transforms the model into "a reasoner that maintains perceptual fidelity while verifying critical details with external evidence" — the external verification serves as a check on reasoning that might otherwise drift away from perceptual reality.

3.4.6 Post-Training Pipeline

Post-training aims to establish foundational clinical capabilities (focusing on accuracy and regulatory compliance) through SFT, then improve reasoning in complex scenarios through RL. The pipeline is illustrated in Figure 7.

Supervised Fine-Tuning (SFT)

SFT uses the highest-quality data in the training pipeline, distinguishing it from mid-training through broader task coverage and stronger instruction-following capabilities.

Human annotation data: Human preference data is triggered by a multi-expert consistency mechanism. For the same image-text prompt, multiple mutually independent VLMs produce candidate responses. An expert-consistency metric aggregates and scores these candidates. If model disagreement is substantial, or if key medical entities and conclusions cannot be reliably determined from consistency signals, the sample is escalated to a human annotation workflow. Medically trained annotators consult the multi-model outputs and their consensus evidence to fact-check candidate answers and provide preference rankings. This ensures that human annotation effort is focused on cases where automated methods are unreliable, maximizing the value of expensive clinical expertise.

Synthetic preference data: The synthetic data follows the same closed-loop synthesis pipeline as mid-training (Figure 5) with additional enhancements for diversity and robustness. Prompt rewriting generates expression variants without altering medical facts or ground-truth answers. A data-balancing module performs coverage-based sampling over a "site/disease/lesion × difficulty" grid, applying targeted oversampling to long-tail and high-risk regions. This ensures balanced coverage across clinical domains while providing stronger supervision on error-prone critical points (e.g., drug interactions, rare disease presentations, complex comorbidities).

Instruction Following

The paper distinguishes instruction following as a critical capability for clinical deployment: "in clinical settings, this requirement is stricter because instructions are often long-context, multi-turn, and may include implicit constraints or conflicts." The SFT stage explicitly targets robust medical instruction following through two complementary approaches:

Instruction data analysis and synthesis: Diverse medical tasks are collected from open-source datasets and online resources, then expanded in depth and breadth using LLMs. These tasks are standardized into an atomic instruction set — a library of elementary instruction types. Conflict-free combinations of atomic instructions are sampled to construct both single-turn and multi-turn instruction-following data, ensuring coverage of departments, task types, and constraint patterns.

To model real-world medical requests that often contain hidden reasoning steps (implicit requirements that are not explicitly stated but must be inferred), the system builds a graph-structured representation linking knowledge nodes (medical concepts), relations (how concepts connect), and operations (what actions to take). This graph is converted into natural-language instructions that preserve implicit reasoning requirements, and used as a reference to generate structured rationales for training.

Instruction understanding and reasoning: The paper notes that "existing models are prone to non-compliance under strict medical constraints, which may lead to safety risks (e.g., violating prohibitive rules)." Large-scale SFT with the synthesized instruction data strengthens basic compliance. However, forward-only synthesis (generating instructions and then generating responses) can introduce subtle instruction-response misalignment. The solution is a reverse construction strategy: start from high-quality clinician-like responses, infer the most compatible instructions, and form high-quality instruction-response pairs. This ensures that the responses genuinely address the instructions rather than being loosely associated text.

To further strengthen reasoning under constraints, structured "thinking" supervision is introduced in SFT through two types of rationales: (1) prompt-guided structured rationales with explicit constraint checking — the model is trained to enumerate and verify each constraint before producing the final answer, and (2) rationales expanded from the explicit reasoning graph to encourage fine-grained logical decomposition.

Multi-Layered Hybrid Reward System

Following SFT, the model "possesses extensive domain knowledge but lacks the proficiency to fully mobilize it for complex clinical reasoning." The RL stage bridges this gap through a sophisticated reward system (Figure 8) with several components:

Data Router: Rather than using a monolithic reward model, the system dynamically directs input samples to specific evaluation pipelines based on metadata. The router interprets prompt intent to select the appropriate grader — routing alignment tasks to the Rule-based Reward and the Rubrics Reward model. This ensures reward signals are contextually tailored to specific training objectives rather than applying a one-size-fits-all evaluation.

Rule-based Reward: A deterministic evaluator handles tasks with well-defined ground truth that can be verified through exact string matching or regular expression patterns (e.g., multiple-choice questions, numerical answers, structured extraction tasks). Using set-theoretic principles, model predictions are compared against reference answers, with proportional credit for partial overlaps and maximum scores for exact matches. All calculations are normalized to a unified scale to ensure compatibility within the broader optimization framework.

Rubrics Reward: For tasks requiring nuanced evaluation beyond exact matching, the Rubric Reward mechanism evaluates candidate responses against multidimensional assessment criteria. Reference responses are generated through best-of-N sampling from expert models fine-tuned on domain-specific datasets curated and labeled by human specialists, establishing reliable scoring anchors.

The rubric construction uses a dual-pathway approach:

Dynamic pathway: An Expert Analyst model analyzes the system prompt, dialogue history, and user query to construct instance-specific rubrics capturing contextual requirements (logical coherence, empathetic expression, intent alignment). These dynamic rubrics adapt to the specific interaction context.
Static pathway: Domain-specific gold standard rubrics are established through multi-model consensus filtering followed by human expert refinement. For medical dialogue systems, these encompass critical assessment dimensions such as Medical Knowledge and Patient Experience.

For tasks with explicit reference answers, rubrics incorporate semantic alignment criteria where LLMs/VLMs assess meaning equivalence beyond surface-level lexical matching. This is important for medical tasks where the same clinical meaning can be expressed in various ways.

Process-Supervised Reasoning Verification: To enhance multi-step reasoning capabilities, dense supervision is introduced on the model's internal deliberation process encoded in thinking blocks. A dedicated evaluation module assesses this intermediate chain of thought across three dimensions: framework completeness (whether all relevant constraints are identified), logical robustness (validity of inference steps), and exploratory depth (thoroughness of solution space examination). This process supervision "compels the model to explicitly analyze task constraints and formulate execution strategies before generating responses, substantially improving performance on complex reasoning tasks."

Reference-Augmented Rubric Evaluation: To mitigate instability inherent in pointwise scoring for nuanced medical criteria, the evaluation is reformulated as a pairwise semantic matching task against gold-standard references. This anchoring significantly reduces variance because clinical reasoning is validated against concrete ground truth rather than abstract definitions.

Soft Shaping: Auxiliary quality metrics (response length, formatting, Pharmacist Evaluation) are optimized using continuous multipliers that scale the base evaluation scores. These multiplicative adjustments compound penalties for deviations while preserving value for high-quality outputs, "creating smooth optimization gradients that drive balanced performance without compromising semantic accuracy."

Reward Signal Fusion: The unified optimization objective nonlinearly fuses criterion-based medical scoring, fundamental evaluations, and reasoning process rewards with strict safety constraints. A binary gating signal nullifies the entire reward if critical safety rules are violated — ensuring that safety is a hard constraint, not a soft preference. The core reward aggregates these components with dynamically computed weights to balance task characteristics, and the dominant rubrics-based score is further modulated by soft shaping multipliers. The formula is not explicitly stated in the paper, but is described as ensuring "the final signal rigorously promotes clinical accuracy, logical coherence, and safety alignment."

RFT-Enhanced Curriculum Reinforcement Learning

The final training stage addresses a fundamental challenge: "reconciling heterogeneous multi-modal data — spanning basic instruction following to long-horizon clinical reasoning over high-resolution imaging — under layered, expert-driven, safety-constrained rewards." Preliminary experiments revealed that simultaneous training (Fusion Paradigm) induces "gradient conflicts and capability oscillations" while sequential training (Staged Paradigm) causes "entropy collapse, where overconfidence on simple tasks eliminates plasticity for complex reasoning." The solution is an Iterative Curriculum strategy grounded in Pareto multi-task learning theory.

Iterative Curriculum Strategy: Each iteration comprises a four-phase progression:

RFT Phase (Rejection Sampling Fine-Tuning): The model's latent competence is converted into systematic supervision by sampling multiple responses from the current policy for each training instance, identifying cases where at least one output meets the multi-layered reward criteria, and distilling these superior responses back into the training corpus. This "converts sporadic successes into reliable behavioral cloning signals."
Foundation Phase: Reward stability is established through short-context instructions and simple multimodal samples, anchoring gradients while maintaining core instruction-following capabilities. This phase uses RFT-enhanced data to provide cleaner behavioral cloning signals for stable gradient anchoring.
Specialization Phase: High-difficulty clinical reasoning is targeted by substantially upweighting complex samples — long-horizon cases and high-resolution imaging — to push performance boundaries on capabilities that failed to emerge under staged training due to premature entropy collapse. The improved supervision from the RFT phase enables "effective reward bootstrapping on complex clinical cases: the model can reliably generate intermediate-quality responses that receive positive reinforcement, enabling productive exploration."
Alignment Phase: General domain and safety-constrained samples are reintroduced as both regularization and plasticity preservation, preventing catastrophic forgetting while maintaining safe distributional boundaries. The reduced distribution shift between RFT-supervised and RL-optimized policies ensures regularization operates on a coherent capability manifold.

Dynamic Hint-based Curriculum: This addresses the cold start problem in early iterations where sparse rewards from complex clinical conditions hinder bootstrapping of valid reasoning chains. Hierarchical hints — ranging from visual perception cues to intermediate logical steps — are injected into the input space as "gradient primers that transform sparse reward signals into dense learning opportunities." A linear decay schedule is applied to hint density across iterations, forcing a gradual transition from external assistance to internalized parameters, ensuring zero-shot capability by intermediate stages.

Entropy-aware Adaptive Regulation: This addresses exploration stagnation in later iterations where accumulated confidence suppresses discovery of alternative solution paths. A real-time monitoring system tracks policy entropy as a proxy for reasoning diversity. When low-entropy periods are detected — indicating overconfidence in familiar patterns — the system dynamically introduces an entropy bonus into the loss function. This prevents premature convergence to local optima, "reinvigorating exploration of alternative reasoning paths and ensuring continuous optimization even as performance approaches the ceiling."

3.4.7 Summary of Design Choices

This subsection synthesizes the key design decisions across the pipeline and their justifications:

Entity-aware continual pretraining over passive data accumulation: The MET makes coverage gaps explicit and measurable, enabling targeted data acquisition and balanced entity training rather than hoping that massive scale alone provides adequate coverage. This is motivated by the observation that medical knowledge follows a Zipfian distribution where rare conditions are systematically underrepresented in uncontrolled data collection.
Data ordering over random shuffling: Organizing the pretraining corpus into semantic clusters with compactness-based curriculum reduces gradient interference between conflicting objectives. Random shuffling would interleave unrelated modalities and difficulty levels, increasing optimization noise.
Multi-expert reject sampling with dual quality gates over single-model synthesis: Ensemble-driven distillation with outcome and thinking verification produces diverse, valid reasoning patterns. Single-model synthesis risks systematic errors (if the model has consistent blind spots) and lacks the diversity needed to teach robust reasoning.
Reverse structured CoT over forward generation: Forward generation of long reasoning chains often diverges from clinically valid logic, particularly in open-ended tasks. The reverse approach anchors reasoning on constraints (low-hallucination captions, four-stage structure) and validates against six quality dimensions.
Personalized visual CoT over uniform reasoning training: Separate treatment of perception-intensive tasks (short CoT, visual anchoring) and reasoning-intensive tasks prevents the perception-reasoning conflict where verbose reasoning degrades visual pattern recognition.
Tool-augmented agentic training over static QA: Integrating medical DeepResearch and think-with-image tools enables interactive evidence-gathering and hypothesis verification that mirrors real clinical workflows, rather than evaluating the model as a passive answerer.
Unfrozen ViT in mid-training over frozen encoder: Medical imaging requires sensitivity to subtle visual discrepancies that frozen general-domain encoders may overlook. Joint optimization adapts visual features to medical image characteristics.
RFT-enhanced iterative curriculum RL over single-stage RL or sequential training: Single-stage RL induces gradient conflicts; sequential training causes entropy collapse. The iterative curriculum with RFT capability distillation, dynamic hints for cold starts, and entropy-aware regulation for exploration stagnation addresses both failure modes.
Multi-Layered Hybrid Reward System over monolithic reward models: Different medical tasks require different evaluation criteria (exact match for MCQs, rubric-based for diagnostic reasoning, process supervision for multi-step reasoning). A modular router with task-specific graders provides contextually appropriate reward signals.

4. Key Insights and Innovations

Innovation 1: Knowledge Coverage as a Measurable, Optimizable Quantity Rather Than an Emergent Property of Scale

The paper's most fundamental conceptual move is treating medical knowledge coverage not as something that passively emerges from training on massive datasets, but as a first-class, measurable, and optimizable objective in model development. This is a genuine reframing rather than an incremental improvement. Prior to this work, the dominant assumption in medical VLM development — and in large-scale pretraining more broadly — was that collecting more data from diverse sources would naturally improve coverage, and that coverage gaps could be addressed simply by scaling up. The Medical Entity Tree (MET) and its associated AMCS metric (Table 2) make coverage explicit, quantitative, and auditable, enabling a shift from "hope the data covers everything" to "verify and ensure the data covers everything."

The significance of this reframing extends beyond the immediate gain of identifying sparse regions for targeted data acquisition. It introduces a coverage epistemology into medical AI: the idea that we can and should measure which medical concepts a model has been exposed to, quantify gaps, and systematically close them. The AMCS metric (Section 3.2.3, Equation 1) is asymmetric by design — forward coverage measures inclusion of benchmarks in the taxonomy (validating that the taxonomy captures known concepts), while backward coverage measures how much the taxonomy exceeds benchmarks (quantifying long-tail richness). This two-directional measurement is conceptually sophisticated: it simultaneously answers "does our knowledge structure cover what we already know matters?" and "does our knowledge structure contain things beyond what existing benchmarks test?" — both questions that prior work could not answer.

The empirical validation is concrete: Table 2 shows backward coverage scores of 0.68, 0.89, and 0.79 for Clinical Knowledge Data, Common Crawl Medical Corpus, and CMeKG respectively. These numbers mean that the paper's entity tree contains 11–32% more medical concepts than these baselines — concepts that are predominantly long-tail entities (rare diseases, atypical presentations, emerging treatments) that would be systematically underrepresented in models trained without explicit coverage optimization. This is significant not because 1.4 million entities is a large number (scale claims are common), but because the paper provides a verifiable methodology for measuring what that scale actually means in terms of concept coverage, and demonstrates that existing resources — including the widely-used CMeKG — have substantial gaps.

The reframing has downstream implications for the entire training pipeline: the MET guides pretraining data construction (Section 3.3), the KG-guided QA synthesis (Section 4.1.1), and the "weak entity" targeting mechanism. Without the MET as a coverage scaffold, these pipeline components would be operating blind — generating data without knowing whether they are filling coverage gaps or redundantly reinforcing already-covered concepts. In this sense, the MET is not just a data structure but the architectural backbone of the entire knowledge acquisition strategy — a role that no prior medical VLM work has assigned to explicit knowledge organization.

Innovation 2: The Perception-Reasoning Conflict as a Diagnostic Construct with a Systematic Resolution

The paper identifies what it calls a "pervasive industry-wide bottleneck known as the perception-reasoning conflict" (Section 4.1.1, Personalized Visual CoT): the empirically observed phenomenon that aggressive training on long, text-heavy reasoning chains causes models to "detach from visual evidence, causing the model to operate purely in the textual space and leading to significant regressions in fine-grained perceptual tasks." While the general phenomenon of capability interference in multi-task learning is well-documented, the paper's contribution is diagnosing the specific mechanism (verbose reasoning dilutes visual attention rather than general gradient conflict) and developing a targeted solution (Personalized Visual CoT that separates logical content from cognitive style and enforces visual anchoring through low-hallucination captions and first-principles logical cores strictly bound to visual observations).

This is a fundamental insight because it challenges the implicit assumption in the reasoning literature that "more reasoning is always better." The standard approach to improving diagnostic accuracy in medical VLMs would naturally be to train them to produce longer, more detailed, more step-by-step reasoning — mirroring the success of chain-of-thought prompting in text-only LLMs. The paper's finding that this can actively harm performance on perception-intensive tasks is a negative result with significant practical implications: it means that reasoning training must be modality-aware, with different reasoning strategies for different input types, rather than applying a uniform reasoning format across all tasks.

The novelty lies not in the observation that models can degrade (catastrophic forgetting is well-studied) but in the mechanism identification and the solution architecture. The paper's Personalized Visual CoT pipeline separates logical content (what the reasoning says) from cognitive style (how it is expressed), uses a multi-agent architecture to adapt core logic into task-specific formats, and enforces a dual-layer quality gate that validates both logical structure and post-adaptation style consistency. This is a fundamentally different approach from prior work that either (a) treated reasoning as a uniform capability to be trained uniformly, or (b) separated perception and reasoning into different model components or training phases. The paper instead keeps them in a single model but trains them with explicit awareness of their interaction, using visual anchoring (low-hallucination captions, bounding-box-grounded observations) to prevent reasoning from drifting away from perceptual evidence.

The practical significance is that this insight enables the model to achieve strong performance on both reasoning-intensive tasks (MedXpertQA-MM: 71.00, DiagnosisArena: 42.08) and perception-intensive tasks (Inhouse VQA: 76.77, SLAKE: 82.62) — a simultaneous achievement that the paper implies would not be possible with standard uniform reasoning training. This is not just a performance claim but a capability composition claim: the paper argues that prior models face an inherent tradeoff between perceptual accuracy and reasoning depth, and that the Personalized Visual CoT approach resolves this tradeoff.

Innovation 3: Difficulty-Aware, Modality-Specific Tool Augmentation as a Distinct Capability

The paper's approach to tool-augmented medical reasoning is conceptually distinctive in a way that distinguishes it from both general-domain DeepResearch systems and prior medical tool-use efforts. Standard DeepResearch (Team et al., 2025; Lu et al., 2025) couples search with reasoning for fact-seeking tasks — the model decomposes a question, searches for relevant information, and synthesizes an answer. The paper identifies that this paradigm is insufficient for medicine because "medical scenarios require multi-hop decision-making in consistent clinical contexts, including appropriate tool selection and chaining" (Section 4.1.2) — that is, knowing which tool to use when and how to sequence tool calls in a clinically coherent way.

The innovation is threefold. First, the paper introduces medical-specific tools (Search Drug, Search Clinical, Zoom In, Rotate) that go beyond generic web search — reflecting the insight that different clinical reasoning subtasks require access to different types of external evidence (drug databases, clinical case records, high-resolution image regions). Second, it develops a training data construction methodology specifically for medical tool use, including entity-chain random walks for multi-hop question generation and two-stage filtering (fine-grained perception filtering via multi-agent uncertainty consensus, task-with-instrument effectiveness filtering) that selects cases where tools genuinely improve diagnostic accuracy. Third, and most distinctively, it integrates tool use as part of the reasoning curriculum rather than as a separate capability bolted on after the fact — the RFT-enhanced iterative curriculum RL (Section 5.2.3) trains tool use alongside diagnostic reasoning in the same optimization process.

This is significant beyond the performance gains on tool-intensive benchmarks (MedBrowseComp: 29.00 vs. GPT-5.2 Thinking's 19.00; CureBench #1 rankings) because it demonstrates that tool use is not just about having access to tools — it is about knowing when and how to use them within a clinical reasoning framework. The two-stage filtering for tool-augmented training data is critical here: by filtering for cases where tools actually help (task-with-instrument effectiveness filtering) and interleaving this filtering with training, the system avoids training the model to use tools on cases where parametric knowledge suffices, which would produce unnecessary tool calls and slow inference. This is a subtle but important practical consideration that prior work on tool-augmented LLMs has not systematically addressed.

The conceptual significance is that this approach treats tool use as a clinical skill to be learned rather than an infrastructure capability to be provided. The model learns not just how to call tools (the mechanics) but when tool use is diagnostically appropriate (the judgment) — much as a clinician learns when to order a lab test versus rely on physical examination findings. This framing aligns the technical capability with clinical workflow in a way that prior tool-use systems, which tend to treat tools as universally available information sources, do not.

Innovation 4: Process-Verified Multi-Expert Reasoning Synthesis with Dual Quality Gates

The paper's approach to synthesizing medical reasoning training data introduces a methodological innovation in how to generate reliable, diverse reasoning trajectories from models that may themselves be imperfect reasoners. The standard approach in the reasoning literature is to generate chain-of-thought data through forward sampling (prompt the model with a question and collect the reasoning it produces) or through single-model best-of-N rejection sampling (generate multiple reasoning chains from one model, keep the ones that produce correct answers). These approaches have two well-known failure modes: (1) a correct answer can be reached through flawed reasoning (outcome-correct but reasoning-incorrect), and (2) a single model's reasoning patterns are limited to that model's cognitive biases and blind spots.

The paper's Multi-Expert Reject Sampling with dual quality gates (Section 4.1.1) addresses both simultaneously. The Outcome-Verify gate checks whether the final answer is correct — necessary but insufficient. The Thinking-Verify gate concurrently checks whether the reasoning chain itself is logically sound — evaluating whether each inference step follows from previous steps and whether the chain supports the conclusion. The "multi-expert" aspect means that reasoning trajectories are drawn from multiple models with different training distributions and cognitive configurations, producing a diverse set of valid reasoning patterns rather than a single model's characteristic style.

This is conceptually distinctive because it treats reasoning validity and reasoning diversity as independent objectives in data synthesis, and uses ensemble-based generation with separate verification for each objective. Prior work has addressed reasoning validity (through self-consistency or outcome-based filtering) and reasoning diversity (through ensemble methods) separately, but the dual-gate architecture explicitly verifies both simultaneously, producing training data that is both correct in outcome and diverse in approach — each trajectory being an independently valid path to the correct answer.

The significance extends to the quality of the RL initialization. The paper argues that mid-training "builds a high-quality supervision signal that offers a strong initialization for reinforcement learning, supplying reliable policy priors for later alignment" (Section 4, introduction). The dual-gate synthesis ensures that these policy priors are reliable — the model starts RL with reasoning patterns that are both outcome-verified and thinking-verified, reducing the exploration burden during RL and preventing the model from learning to produce answers that happen to be correct but are reached through bad reasoning (which would be rewarded by an outcome-only reward signal). This is a practical insight with theoretical implications: the quality of RL initialization matters not just for sample efficiency but for the validity of the learned reasoning process itself.

5. Experimental Analysis

Evaluation Methodology

Datasets. The primary evaluation framework is the Unified Med-VLM Benchmark (Section 6), which consolidates 30+ public benchmarks spanning six capability categories (Visual Diagnosis & Image Recognition, Medical Imaging, Diagnosis, Medical Text, Medical Report, Instruction Following). These are supplemented by three in-house benchmarks (Inhouse VQA, Inhouse OCR, Inhouse Caption) designed to measure capabilities that are "under-measured in the public landscape." A complete listing with benchmark descriptions appears in Tables 3–8 in Appendix 10. The model is additionally evaluated on CureBench (a competition benchmark for drug decision-making and treatment planning hosted at NeurIPS 2025), where results are reported in Section 9.1.
Base model. MedXIAOHE is built on the Seed vision-language foundation model (Seed-VL; Guo et al., 2025a), which consists of a Seed-ViT vision encoder, an MLP adapter, and an autoregressive LLM decoder. The paper does not disclose the exact parameter count, training FLOPs, or LLM backbone identifier for either the base Seed model or the resulting MedXIAOHE model. This is a notable omission in a paper that otherwise provides extensive methodological detail. The choice of Seed-VL is justified by its "unified multimodal decoder" architecture that processes interleaved image and text tokens (Section 2, Figure 2), with the explicit claim that architecture innovation is not the differentiator — domain-specific data quality, knowledge coverage, and post-training alignment matter more.
Metrics. The paper reports Pass@1 scores for all public and in-house benchmarks (Section 2, Table 1). For closed-form tasks (MCQ and short QA), deterministic answer parsing is applied (option extraction, numeric normalization, whitespace/punctuation normalization). For open-form tasks (dialogue, report, caption generation), task-appropriate metrics are used alongside "rubric-style structured judging to distinguish clinical correctness, completeness, and unsupported claims" (Section 6). The in-house caption benchmark uses a custom scoring scheme where "correctly covered key points receive positive credit, while incorrect statements are penalized" (Section 6.2). The AMCS metric (Equation 1) measures semantic coverage of the Medical Entity Tree against reference sets. No confidence intervals, standard deviations, or statistical significance tests are reported for any result in the main paper or appendices.
Baselines. The primary baselines are GPT-5.2 Thinking, Gemini 3.0 Pro, and Gemini 2.5 Pro (Table 1), all evaluated with greedy decoding in "thinking mode" where applicable. These represent frontier closed-source general-domain multimodal systems. No open-source medical VLM baselines are reported in the main results table (Table 1), though the paper cites numerous prior medical VLM systems in its related work discussion (Saab et al., 2024; Sellergren et al., 2025; Moor et al., 2023; Li et al., 2023; Pan et al., 2025; Chen et al., 2024a; Xu et al., 2025b). The absence of direct comparisons against these open-source medical VLMs in Table 1 is a limitation — it prevents a controlled comparison isolating the effect of domain-specific continual pretraining versus general-domain pretraining at scale. Gemini 3.0 Pro is excluded from in-house benchmarks "because of changes in its privacy protocols."
Generation budget / compute accounting. The paper does not report total training FLOPs, inference FLOPs, or parameter counts, making it impossible to assess the computational cost of MedXIAOHE relative to baselines or to perform any FLOPs-matched comparison of the kind exemplified by the pretraining-inference tradeoff analysis in the example paper. All evaluations use greedy decoding (Table 1 caption: "All benchmarks are evaluated with greedy decoding"), which is a single-generation setting. For tool-augmented inference, the generation budget would include tool calls and their results, but these costs are not quantified.
Cross-validation / statistical protocol. No cross-validation, bootstrapping, or statistical testing is reported for benchmark evaluations. The difficulty estimation and strategy selection procedures described in the mid-training and post-training sections use internal validation sets (e.g., the fine-grained perception filtering uses a "multi-agent uncertainty consensus framework" and task-with-instrument effectiveness filtering is interleaved with training), but the specific validation splits or protocols are not disclosed. The CureBench results (Section 9.1) are presumably on the competition's official test set, though this is not explicitly stated.

Main Quantitative Results

The paper presents results organized around the six capability categories of the Unified Med-VLM Benchmark, supplemented by in-house evaluations and competition results. I walk through each category, noting comparative performance and specific claims.

Overall Performance (Table 1, Figure 1)

MedXIAOHE is reported to achieve state-of-the-art performance across the benchmark suite. The left panel of Figure 1 shows an overall average score across 30+ benchmarks, with MedXIAOHE positioned ahead of GPT-5.2 Thinking and Gemini models, though no numerical aggregate score is reported in the text. The paper describes MedXIAOHE as "competitive across a broad range of medical benchmarks" (Section 7), which is a more measured claim than "outperforms on every benchmark." Indeed, the model does not uniformly win: it loses to Gemini 3.0 Pro on IU-Xray (65.66 vs. 73.51), to GPT-5.2 Thinking on MedXpertQA-MM and MedXpertQA-Text (71.00 vs. 73.25 and 63.14 vs. 55.27 respectively — but note the text benchmark scores show MedXIAOHE winning relative to GPT-5.2 Thinking's 55.27 vs. MedXIAOHE's 63.14, which contradicts the narrative of GPT-5.2 Thinking dominance on this benchmark), and to GPT-5.2 Thinking on GMAI-MMBench (55.21 vs. 60.42) and VQA_RAD (66.96 vs. 68.96). The pattern is one of broad competitiveness with specific strengths rather than uniform superiority.

Visual Diagnosis & Image Recognition (Table 1)

MedXIAOHE achieves 87.53 on MMMU_val-Med (vs. 79.73 for GPT-5.2 Thinking, 83.33 for Gemini 3.0 Pro, 79.60 for Gemini 2.5 Pro) and 73.88 on MMMU_Pro-Med (vs. 71.85 for GPT-5.2 Thinking, 73.43 for Gemini 3.0 Pro, 65.91 for Gemini 2.5 Pro). The MMMU_Pro-Med result is essentially tied with Gemini 3.0 Pro (73.88 vs. 73.43), though the paper claims outperformance without qualification. On in-house benchmarks in this category, MedXIAOHE achieves 76.77 on Inhouse VQA, 46.89 on Inhouse Caption, and 81.92 on Inhouse OCR. Direct comparisons are only against Gemini 2.5 Pro for these in-house benchmarks (VQA: 76.77 vs. 69.64; Caption: 46.89 vs. 45.92; OCR: 81.92 vs. 80.82), since GPT-5.2 Thinking and Gemini 3.0 Pro are marked as not evaluated on in-house data (the GPT-5.2 Thinking column shows "—" for in-house benchmarks, and the Gemini 3.0 Pro column shows "—" for all in-house measures). This means the claim of outperforming "leading closed-source multimodal systems on multiple capabilities" (Abstract) is based on a subset of benchmarks where comparison was possible, not the full suite.

Medical Imaging (Table 1)

Performance on medical imaging benchmarks is mixed. MedXIAOHE achieves strong results on SLAKE (82.62 vs. GPT-5.2 Thinking's 72.83 — a ~10-point absolute gap), PATH_VQA (59.15 vs. GPT-5.2 Thinking's 47.88 — an ~11-point gap), and OmniMedVQA (83.40 vs. GPT-5.2 Thinking's 73.95 — a ~9-point gap). These are substantial margins that suggest effective cross-modal reasoning on clinical imagery. However, it underperforms on GMAI-MMBench (55.21 vs. GPT-5.2 Thinking's 60.42 and Gemini 3.0 Pro's 66.81), VQA_RAD (66.96 vs. 68.96 and 74.50), and PMC_VQA (63.90 vs. 68.65 and 70.15). The pattern is notable: MedXIAOHE wins on datasets focused on specific modalities with knowledge-enhanced or large-scale coverage (SLAKE, PATH_VQA, OmniMedVQA) but loses on broad-coverage benchmarks (GMAI-MMBench) and specialized radiology QA (VQA_RAD). This may reflect a tradeoff between breadth and depth in the pretraining data — the entity-aware continual pretraining may strengthen specific knowledge domains at the expense of the broad-coverage generalization that general-domain models acquire through massive-scale pretraining.

Diagnosis (Table 1)

Diagnosis benchmarks reveal specific strengths in rare disease reasoning (RareBench: 46.79 vs. 42.19 for GPT-5.2 Thinking) and medical deep search (MedBrowseComp: 29.00 vs. 19.00 for GPT-5.2 Thinking — a ~10-point absolute gap), consistent with the paper's emphasis on long-tail coverage and tool-augmented reasoning. On MedXpertQA-MM (multimodal expert-level reasoning), MedXIAOHE's 71.00 trails GPT-5.2 Thinking's 73.25 and Gemini 3.0 Pro's 77.20. On MedXpertQA-Text, MedXIAOHE's 63.14 outperforms GPT-5.2 Thinking's 55.27 but trails Gemini 3.0 Pro's 69.47. On DiagnosisArena (case-based diagnostic reasoning), MedXIAOHE's 42.08 is competitive with GPT-5.2 Thinking's 45.57 and substantially ahead of both Gemini models (26.45 and 20.87). On HLE-Med (expert-challenging problems), MedXIAOHE's 25.77 is essentially tied with GPT-5.2 Thinking's 24.70, while Gemini 3.0 Pro leads at 36.91.

The paper claims these results "indicate robust medical decision-making capability in complex and rare disease diagnosis" (Section 2). This is partially supported: the strong RareBench and MedBrowseComp results are consistent with the MET and agentic training contributions, but the mixed performance on MedXpertQA and the relatively low absolute scores on DiagnosisArena (42.08) and HLE-Med (25.77) suggest that complex diagnostic reasoning remains challenging — MedXIAOHE gets roughly half of DiagnosisArena questions wrong and three-quarters of HLE-Med questions wrong.

Medical Text (Table 1)

MedXIAOHE demonstrates strong text-only clinical reasoning, achieving the highest score on most benchmarks in this category: PubMedQA (86.00 vs. 80.80 for GPT-5.2 Thinking), MedQA_MCMLE (96.21 vs. 94.48), MedQA_USMLE (97.88 vs. 96.39), Medbullets_op4 (95.78 vs. 91.88), Medbullets_op5 (94.16 vs. 88.31 — a ~6-point gap), CMExam (96.12 vs. 93.13), MedMCQA (86.54 vs. 86.01), HealthBench-hard (46.10 vs. 42.04), and HealthBench-normal (65.20 vs. 63.34). The single exception is SuperGPQA-Med, where MedXIAOHE's 71.94 trails Gemini 3.0 Pro's 72.45 but leads GPT-5.2 Thinking's 65.37. The consistent strength across USMLE-style, Chinese licensing, and multi-turn dialogue benchmarks suggests the knowledge-grounded pretraining and medical text synthesis pipelines (Section 3.3.1) are effective at building robust text-based clinical knowledge.

The HealthBench results deserve particular attention: HealthBench-hard (46.10) and HealthBench-normal (65.20) are multi-turn medical dialogue benchmarks evaluating correctness, safety, and communication quality. MedXIAOHE's lead here — particularly the ~28-point gap over Gemini 3.0 Pro on HealthBench-hard (46.10 vs. 17.70) — supports the paper's claim of improved instruction following and alignment with health-oriented preferences (Section 5.1.2). This suggests the SFT-stage instruction-following training and the post-training alignment are producing meaningful behavioral improvements beyond simple accuracy.

Medical Report (Table 1)

MedXIAOHE achieves 50.86 on MIMIC-CXR (vs. GPT-5.2 Thinking's 46.44), 49.43 on CheXpert Plus (vs. 42.15 for Gemini 3.0 Pro), and 65.66 on IU-Xray (vs. Gemini 3.0 Pro's 73.51). The IU-Xray result is the most notable underperformance: a ~8-point gap behind the leader. The paper acknowledges this specifically: "Results on IU-Xray are comparatively lower than the best baseline, suggesting remaining headroom for narrative in certain report-generation regimes" (Section 2). IU-Xray evaluates descriptive faithfulness and content coverage in radiology report/caption generation (Table 7). The gap may reflect the paper's emphasis on low-hallucination generation — the GenRM filtering (Section 3.3.3) and evidence-grounded captioning may produce conservative reports that score lower on reference-based metrics like BLEU/ROUGE (which reward n-gram overlap with reference text) while being clinically more faithful. The paper's in-house caption benchmark (with its "reward-and-penalty design" that penalizes incorrect statements) may capture this tradeoff, but the IU-Xray evaluation metric is not specified in sufficient detail to verify this hypothesis.

Instruction Following (Table 1)

MedXIAOHE achieves 61.90 on MultiChallenge (vs. 66.80 for GPT-5.2 Thinking), 78.70 on MulDimIF (vs. 78.60 for GPT-5.2 Thinking — essentially tied), and 63.75 on MedMTbench (vs. 51.30 for GPT-5.2 Thinking — a ~12-point gap). The strong MedMTbench result is noteworthy because MedMTbench evaluates "long-context medical instruction following with implicit demands" (Table 8), directly testing the capabilities developed through the instruction-following training described in Section 5.1.2. The competitive-but-not-leading MultiChallenge result suggests that complex multi-turn medical dialogue with constraint tracking remains challenging, consistent with the broader finding that interactive clinical reasoning is still a work in progress.

Agentic Reasoning: CureBench (Section 9.1)

The paper reports achieving "#1 place in both the Internal Reasoning and Agentic Reasoning tracks of CureBench @ NeurIPS 2025." No scores are provided beyond the ranking claim — no absolute accuracy, no comparison against runner-up systems, and no description of the evaluation protocol or number of competing systems. This result is presented as qualitative validation of the agentic training approach but cannot be evaluated quantitatively from the information provided.

Putative Relation to Medical Image Interpretation

The grounding and think-with-image results are not evaluated through a separate benchmark in the main results table. The qualitative examples in Section 9 (Figures 9–14) demonstrate the model's ability to use tools (zoom, rotate, search) and produce grounded reasoning with bounding boxes, but no quantitative accuracy metric for these specific capabilities is reported. The Inhouse VQA and Medical Imaging benchmarks likely partially capture these capabilities, but the paper does not isolate grounding accuracy or tool-use effectiveness as distinct metrics.

Putative Impact of Approach on Report Generation Quality

The evidence-grounded captioning and report pipeline (Section 3.3.3, caption construction; Section 5, post-training RL) is evaluated through the public report benchmarks (MIMIC-CXR, CheXpert Plus, IU-Xray) and the in-house caption benchmark. As noted above, the public benchmark results are competitive for MIMIC-CXR and CheXpert Plus but substantially behind on IU-Xray. The in-house caption benchmark uses a custom "reward-and-penalty" scoring scheme that rewards correct key points and penalizes incorrect statements, but no comparison against baselines on this metric is reported — we only see the in-house caption score for MedXIAOHE (46.89) and Gemini 2.5 Pro (45.92) in Table 1, without clarity on what these numbers represent (percentage of maximum possible score? raw score?) or how they were computed. The claim of "low-hallucination long-form report generation" (Abstract) therefore rests on qualitative examples (Figure 13) and the paper's methodological description rather than on direct quantitative hallucination measurement.

Ablation Studies and Robustness Checks

The paper does not report systematic ablation studies isolating individual components of the training pipeline. This is a significant departure from the experimental norms established by papers like the example (which provided detailed ablation of search algorithms, aggregation strategies, revision vs. parallel sampling, PRM vs. ORM, oracle vs. predicted difficulty, and more). The following are the closest approximations to ablations that can be extracted from the paper's descriptions:

Coverage of Medical Entity Tree vs. existing knowledge resources (Table 2): The AMCS analysis compares forward and backward coverage of the MET against Clinical Knowledge Data, Common Crawl Medical Corpus, and CMeKG. Forward coverage scores exceed 0.95 for all comparisons, confirming the MET encompasses concepts from these baselines. Backward coverage scores (0.68, 0.89, 0.79) are lower, indicating the MET contains additional concepts not present in the baselines. This validates the MET's comprehensiveness but does not directly demonstrate that increased coverage leads to improved model performance — it shows the taxonomy is comprehensive, not that comprehensive taxonomy-guided training produces better models than taxonomy-agnostic training at equivalent data scale.

Data ordering curriculum vs. random shuffling (Section 3.4): The paper describes the data ordering paradigm (UMAP + HDBSCAN clustering + compactness-based curriculum) as motivated by the hypothesis that random shuffling "interleaves unrelated objectives and increases gradient variance and conflicts." However, no experimental comparison of ordered vs. randomly shuffled training is reported — no loss curves, no downstream benchmark comparisons, no ablation showing that the ordering procedure improves performance over a random baseline. This is a notable gap for a paper whose central methodology claim is that data ordering matters.

Personalized Visual CoT vs. standard reasoning training (Section 4.1.1): The paper describes the "perception-reasoning conflict" and the Personalized Visual CoT solution in detail, stating that "initial experiments confirmed a pervasive industry-wide bottleneck." However, no quantitative ablation is reported — no comparison of model performance with and without the personalized visual CoT pipeline, no comparison of short vs. long CoT on perception-intensive tasks, no demonstration that the perception-reasoning tradeoff actually manifests in MedXIAOHE or that the proposed solution mitigates it.

Multi-expert reject sampling with dual quality gates (Section 4.1.1): The dual-gate mechanism (Outcome-Verify + Thinking-Verify) is described as producing reasoning data that enables "building fundamental medical reasoning capabilities from scratch." No ablation is reported comparing training with and without the Thinking-Verify gate — that is, we cannot determine whether the process-level verification adds value beyond outcome-level verification. Similarly, the multi-expert ensemble approach is not compared against single-model reject sampling.

RFT-enhanced iterative curriculum RL vs. alternatives (Section 5.2.3): The paper describes preliminary experiments that revealed "simultaneous training (Fusion Paradigm) induces gradient conflicts and capability oscillations, while sequential training (Staged Paradigm) causes entropy collapse," motivating the proposed iterative curriculum strategy. These are the closest the paper comes to reporting ablation results, but no quantitative data from these preliminary experiments is presented — no learning curves, no policy entropy measurements, no performance comparisons between Fusion, Staged, and Iterative paradigms on downstream benchmarks. The claims about gradient conflict and entropy collapse are therefore asserted rather than demonstrated.

ReST $^{EM}$ -style training degradation (negative result): The paper does not report a ReST $^{EM}$ experiment analogous to the example paper's Appendix K. There is no reported attempt to further optimize the model using on-policy RL that degraded performance.

Tool-augmented vs. tool-free inference (Section 4.1.2): The task-with-instrument effectiveness filtering interleaves tool-effectiveness assessment with training, which implicitly compares tool-augmented and tool-free performance on each example. However, no aggregate comparison of the final model's performance with and without tool access is reported. The MedBrowseComp benchmark (29.00 vs. GPT-5.2 Thinking's 19.00) is the closest proxy, since MedBrowseComp specifically evaluates medical deep-search/browsing agents, but this compares across models rather than within MedXIAOHE.

In summary, the paper provides no systematic ablation studies isolating the contributions of individual training components, data strategies, or architectural choices. Every methodological component described in Sections 3–5 is evaluated only through the final model's aggregate performance on the full benchmark suite (Table 1). This makes it impossible to attribute performance gains to specific innovations, to identify which components are essential versus incidental, or to guide practitioners on which parts of the "comprehensive recipe" to prioritize.

Critical Assessment

The paper makes several major claims, which I examine against the experimental evidence:

Claim 1: MedXIAOHE achieves state-of-the-art performance across diverse medical benchmarks and surpasses leading closed-source multimodal systems on multiple capabilities (Abstract, Section 2).

The evidence for this claim is Table 1, which shows MedXIAOHE achieving the highest score on 22 of the 30 reported benchmark entries (not counting in-house benchmarks where comparison is incomplete). However, several qualifications apply:

The claim of "surpassing leading closed-source multimodal systems on multiple capabilities" is true in a literal sense — MedXIAOHE wins on more benchmarks than it loses — but the pattern of results is heterogeneous. MedXIAOHE underperforms on GMAI-MMBench (55.21 vs. 66.81), VQA_RAD (66.96 vs. 74.50), PMC_VQA (63.90 vs. 70.15), MedXpertQA-MM (71.00 vs. 77.20), MedXpertQA-Text (63.14 vs. 69.47 — though note MedXIAOHE beats GPT-5.2 Thinking here), IU-Xray (65.66 vs. 73.51), and MultiChallenge (61.90 vs. 66.80). These are not marginal gaps — some are 8–11 absolute points. A fair characterization would be "competitive with frontier models across most benchmarks, with specific areas of strength and specific areas of weakness" rather than the unqualified "surpasses" claim.
The baselines are all closed-source general-domain models. The absence of direct comparisons against open-source medical VLMs (LLaVA-Med, Med-Flamingo, HuatuoGPT-Vision, MedVLM-R1) means we cannot assess how much of MedXIAOHE's performance comes from the base Seed-VL model's capabilities versus the domain-specific continual pretraining and post-training. A fairer comparison would include at minimum the base Seed-VL model (without medical training) and one or more open-source medical VLMs trained at comparable or larger scale. Without these baselines, the claim that the paper's specific methodology produces gains over simpler medical adaptation approaches is unverified.
The in-house benchmarks are evaluated against only Gemini 2.5 Pro, not GPT-5.2 Thinking or Gemini 3.0 Pro (these are marked "—" in Table 1). This means the strongest baseline systems are not measured on the benchmarks that the paper specifically designed to stress deployment-critical capabilities. The claim of superiority on in-house measures is therefore based on comparison against a single baseline that is not the strongest available.

Claim 2: Entity-aware continual pretraining broadens knowledge coverage and reduces long-tail gaps (Abstract, Section 3).

The evidence for this claim comes from the AMCS coverage analysis (Table 2) showing that the MET contains 11–32% more medical concepts than existing resources. This establishes that the taxonomy is comprehensive, but it does not establish that this comprehensiveness translates to improved model performance on long-tail tasks. The paper reports one long-tail-specific benchmark result: RareBench (46.79 vs. 42.19 for GPT-5.2 Thinking). This is positive but represents a single data point — a ~4.6-point improvement on one benchmark. The paper does not report:

Performance on RareBench stratified by entity frequency in the pretraining data (which would directly test whether improved coverage of rare entities improves performance on questions about those entities).
Comparison against a version of MedXIAOHE trained without the MET-guided data curation (which would isolate the effect of the MET).
Any measure of the relationship between AMCS coverage scores and downstream task performance.
Performance on other long-tail distributions (rare disease subsets of MMMU, MedXpertQA, or DiagnosisArena).

The claim that entity-aware pretraining reduces long-tail gaps is therefore plausible but not demonstrated experimentally. The AMCS analysis provides evidence of coverage, but coverage is a necessary condition for performance, not a sufficient one — a concept being present in the taxonomy does not guarantee the model can reason about it correctly.

Claim 3: Tool-augmented agentic training enables multi-step diagnostic reasoning with verifiable decision traces (Abstract, Section 4.1.2).

The evidence for this claim includes: MedBrowseComp (29.00 vs. 19.00 for GPT-5.2 Thinking), the CureBench #1 ranking, and qualitative examples (Figures 9–14). The MedBrowseComp result is the strongest quantitative evidence — a 10-point gap on a benchmark specifically designed to evaluate medical deep-search agents. However:

MedBrowseComp is a single benchmark with an unspecified number of test instances. The robustness of this result cannot be assessed.
The CureBench results are reported as rankings without scores, making quantitative comparison impossible.
The qualitative examples demonstrate the model's ability to use tools and produce grounded reasoning, but they are cherry-picked — they show what the model can do, not how often it succeeds.
No comparison is provided between MedXIAOHE with and without tool access on the same benchmarks (which would isolate the tool contribution). The tool-augmented training is integrated into the full pipeline, so the model with tools is essentially the only version evaluated.
The "verifiability" of the decision traces is asserted (the traces include tool calls, search evidence, and grounded bounding boxes) but not empirically validated — no human evaluation of trace correctness or usefulness is reported.

Claim 4: The model integrates user-preference rubrics, evidence-grounded reasoning, and low-hallucination long-form report generation with improved instruction following (Abstract, Section 5).

The instruction-following claim is supported by strong results on MedMTbench (63.75 vs. 51.30 for GPT-5.2 Thinking) and HealthBench-hard (46.10 vs. 17.70 for Gemini 3.0 Pro). These are substantial margins on benchmarks specifically designed to test instruction adherence and safe multi-turn dialogue.

The low-hallucination report generation claim is more difficult to assess. The public report benchmark results are positive for MIMIC-CXR (50.86 vs. 46.44) and CheXpert Plus (49.43 vs. 42.15) but negative for IU-Xray (65.66 vs. 73.51). The paper does not report any direct hallucination measurement — no metric counting fabricated findings, no comparison of generated reports against ground-truth annotations for unsupported claims, and no human evaluation of report faithfulness. The GenRM filtering pipeline (Section 3.3.3) is described as filtering "hallucinations or ungrounded diagnostic claims," but its effectiveness is not evaluated. The in-house caption benchmark's "reward-and-penalty" scoring is designed to incentivize faithful captions, but the absolute score (46.89) is not contextualized against a maximum possible score, a human performance baseline, or an ablation of the GenRM filtering.

Missing experiments that would substantially strengthen the paper:

Base model ablation: Evaluate the Seed-VL base model (without any medical pretraining) on the Unified Med-VLM Benchmark. This would establish the baseline from which all domain-specific improvements are measured, making it possible to attribute gains to the paper's methodology rather than the base model's capabilities.
Component ablations: Systematically ablate major pipeline components: MET-guided data curation vs. random data sampling, data ordering vs. random shuffling, multi-expert reject sampling vs. single-model synthesis, Personalized Visual CoT vs. uniform CoT, RFT-enhanced iterative curriculum RL vs. single-stage RL, and tool-augmented training vs. tool-free training. Without these ablations, the "comprehensive recipe" claim is unfalsifiable — we cannot determine which ingredients are essential.
Scaling analysis: Report how performance varies with pretraining data scale, model size, and compute budget. The paper claims that "scaling training data and compute — together with stronger evaluation — remains an immediate path to improving performance" (Section 7), but provides no scaling data to support or characterize this claim.
Hallucination measurement: Report a direct hallucination metric on report-generation tasks — for example, the rate of clinically significant fabricated findings as judged by medical experts, or an automated metric counting unsupported assertions against reference annotations. Without this, the "low-hallucination" claim is asserted rather than demonstrated.
Statistical significance: Report confidence intervals or standard deviations for benchmark scores, especially given that some margins (MMMU_Pro-Med: 73.88 vs. 73.43; MulDimIF: 78.70 vs. 78.60; SuperGPQA-Med: 71.94 vs. 72.45) are sufficiently small that they could fall within sampling noise. Without variance estimates, we cannot determine whether these differences are reliable.
Latency and cost analysis: For tool-augmented inference, report the average number of tool calls per query, the additional inference time introduced by tool use, and any computational cost analysis. The paper claims tool use improves diagnostic reasoning, but the practical cost of this improvement is not quantified.
Comparison against open-source medical VLMs: Evaluate LLaVA-Med, Med-Flamingo, HuatuoGPT-Vision, and MedVLM-R1 on the Unified Med-VLM Benchmark under the same protocol. This would establish whether MedXIAOHE's gains come from methodology, base model quality, or training data scale.

Overall assessment: The paper provides a single, comprehensive evaluation (Table 1) that establishes MedXIAOHE as broadly competitive with frontier closed-source models on a standardized benchmark suite. This is a useful contribution to the medical VLM literature, particularly given the evaluation fragmentation the paper identifies. The specific strengths on RareBench, MedBrowseComp, and HealthBench are consistent with the paper's methodological emphasis on long-tail coverage, tool use, and instruction alignment. However, the paper's central claims about the causal role of specific methodological innovations — the Medical Entity Tree, data ordering, multi-expert reasoning synthesis, Personalized Visual CoT, and the RFT-enhanced RL curriculum — are asserted rather than experimentally demonstrated. Without component ablations, scaling analyses, or direct comparisons against simpler baselines, the "comprehensive recipe" remains a description of what was done rather than evidence of what works. The paper would be strengthened substantially by trading some of its methodological breadth for experimental depth — fewer pipeline components described in detail, more controlled experiments demonstrating that specific components produce measurable improvements over simpler alternatives.

6. Limitations and Trade-offs

Difficulty Estimation Cost Is Unaccounted for in Headline Efficiency Claims

The assumption or constraint. The Medical Entity Tree (Section 3.2) and the subsequent data ordering curriculum (Section 3.4) require computationally expensive preprocessing: LLM-driven entity extraction from a 640B-token corpus (with a reported 30× speedup through batch chunking, implying the original cost was substantial), K-Means clustering on entity embeddings with Silhouette Coefficient optimization, ReAct-agent conflict resolution requiring live web search for ambiguous entity classifications, AC-Automaton scanning taking "approximately 20 hours," and UMAP+HDBSCAN clustering on embeddings from the entire pretraining corpus. The paper never accounts for this preprocessing cost in any efficiency or resource analysis. The pretraining data construction pipelines — caption generation with GenRM filtering, OCR distillation, image knowledge synthesis with human annotation review — similarly involve substantial computational overhead that is not quantified. The paper explicitly states none of these costs are factored into the training or inference budget.

The consequence. A practitioner attempting to replicate or extend MedXIAOHE faces unknown — but likely substantial — preprocessing costs before training even begins. The MET construction pipeline alone requires LLM inference over a 640B-token corpus (for entity extraction), embedding computation over all training samples (for the data-ordering curriculum), and automated web search for thousands or millions of entity disambiguation queries (for the ReAct agent). These costs may rival or exceed the actual training cost, particularly for the MET-guided data balancing that is central to the paper's coverage claims. The "comprehensive recipe" framing does not inform practitioners how to allocate their total compute budget between preprocessing infrastructure and model training — a critical consideration when deciding whether to adopt this approach versus a simpler, less curated alternative. Furthermore, the difficulty and cost of reproducing these preprocessing steps may limit independent verification of the paper's claims, since only organizations with comparable infrastructure could replicate the full pipeline.

What evidence exists in the paper. No evidence. The paper reports only one cost-related number — the 20-hour AC-Automaton scan time and the 30× speedup for batch-chunked entity extraction — but neither is contextualized (What hardware? What LLM? What corpus size for the 30× claim? What was the baseline cost?). No FLOPs analysis, dollar-cost estimate, or preprocessing-to-training compute ratio is provided anywhere in the paper or appendices.

Mitigation status. Not addressed. The paper does not acknowledge preprocessing cost as a limitation, does not suggest how costs could be reduced, and does not compare the approach to simpler alternatives (e.g., random data sampling without entity balancing, random shuffling without curriculum ordering) that would trade coverage or training stability for dramatically lower preprocessing overhead. The lack of ablation studies (Section 5) means we cannot assess whether these preprocessing investments produce proportional performance gains.

No Systematic Ablation Studies — Causal Attribution Is Ungrounded

The assumption or constraint. The paper presents MedXIAOHE as a "comprehensive recipe" integrating entity-aware continual pretraining, structured reasoning synthesis with dual quality gates, Personalized Visual CoT, tool-augmented agentic training, data-ordering curricula, and RFT-enhanced iterative curriculum RL. However, the paper reports zero systematic ablation studies isolating the contribution of any single component. The evaluation consists entirely of end-to-end benchmark comparisons against external models (Table 1). While the paper describes preliminary experiments that motivated design choices — e.g., the Fusion vs. Staged paradigm comparison that revealed "gradient conflicts and entropy collapse" (Section 5.2.3) — no quantitative data from these experiments is reported. The AMCS coverage analysis (Table 2) validates the MET's comprehensiveness but does not demonstrate that MET-guided training produces better models than taxonomy-agnostic training at equivalent data scale.

The consequence. The paper's central claims about specific methodological innovations are unfalsifiable from the reported evidence. We cannot determine whether the strong RareBench performance (46.79 vs. 42.19) is due to the MET-guided long-tail coverage, the KG-guided QA synthesis targeting weak entities, the multi-expert reasoning data, the RL curriculum, or simply the scale of medical pretraining data. We cannot determine whether the Personalized Visual CoT pipeline actually resolves the perception-reasoning conflict (the paper reports no comparison of model performance on perceptual tasks with and without this pipeline). We cannot determine whether the RFT-enhanced iterative curriculum produces better results than single-stage RL, staged training, or SFT alone. Practitioners reading this paper to identify which components to adopt for their own medical VLMs receive no evidence-based guidance — every component is described and claimed to be important, but none is validated independently. This is particularly problematic for components like data ordering (Section 3.4) and the dual-gate reasoning verification (Section 4.1.1), which add substantial implementation complexity and computational overhead without demonstrated benefit over simpler alternatives.

What evidence exists in the paper. None for component-level ablation. The only quantitative variation in the paper is the AMCS analysis comparing MET coverage against external knowledge resources (Table 2), which is a coverage measurement, not a performance ablation. The paper does not report performance of: (a) Seed-VL base model without medical training on the Unified Med-VLM Benchmark, (b) MedXIAOHE trained without the MET-guided data balancing, (c) MedXIAOHE trained with random data shuffling instead of the ordered curriculum, (d) MedXIAOHE trained with only outcome-based reject sampling (no thinking-verify gate), (e) MedXIAOHE trained without the Personalized Visual CoT pipeline, (f) MedXIAOHE trained with standard single-stage RL instead of the RFT-enhanced iterative curriculum, or (g) MedXIAOHE without tool-augmented training, evaluated on the same benchmarks with the same tools available at inference.

Mitigation status. Not addressed. The paper presents the pipeline as an integrated system and does not frame the absence of ablations as a limitation. The CureBench results (Section 9.1) provide some external validation of the agentic reasoning approach, but without internal ablation, this validation attributes performance to the full system rather than to specific innovations. The paper does not suggest future work on component isolation.

Hard Diagnostic Reasoning Remains Largely Unsolved Despite Long-Tail Coverage Claims

The assumption or constraint. The paper's central motivation is that existing medical VLMs fail on long-tail and complex cases — rare diseases, atypical presentations, complex comorbidities — and that entity-aware continual pretraining with explicit coverage optimization can close this gap. The MET is explicitly designed to "mitigate long-tail distribution issues" (Section 3.2). However, the paper's own results show that on the hardest diagnostic reasoning benchmarks, performance remains poor in absolute terms, and the model does not consistently outperform baselines.

The consequence. Test-time compute, tool augmentation, and balanced pretraining amplify existing capabilities but cannot create them from nothing. On DiagnosisArena (case-based diagnostic reasoning requiring end-to-end evidence synthesis), MedXIAOHE achieves 42.08 — meaning it gets more than half of these diagnostic cases wrong. On HLE-Med (expert-challenging closed-form problems probing near-ceiling reasoning), MedXIAOHE achieves 25.77 — failing on roughly three-quarters of questions. On HealthBench-hard (difficult multi-turn medical dialogues), MedXIAOHE achieves 46.10. These numbers represent genuine difficulty: the model is substantially below clinical reliability thresholds on tasks that most closely resemble real diagnostic workflows. Even on MedXpertQA-MM (expert-level multimodal medical questions), MedXIAOHE's 71.00 trails Gemini 3.0 Pro's 77.20 by ~6 points, suggesting that the coverage-balanced pretraining does not compensate for the reasoning depth of a larger general-domain model on sufficiently challenging multimodal diagnostic tasks. For practitioners considering deployment, these results indicate that MedXIAOHE — despite its comprehensive training recipe — would not be reliable enough for autonomous clinical decision-making on complex cases. The model performs well on "exam-style" benchmarks (MedQA_USMLE: 97.88; Medbullets_op5: 94.16) where questions have clear correct answers and follow standardized formats, but degrades on open-ended diagnostic reasoning where evidence synthesis, differential diagnosis, and uncertainty management are required.

What evidence exists in the paper. The DiagnosisArena (42.08), HLE-Med (25.77), HealthBench-hard (46.10), and MedXpertQA multimodal/text scores (71.00/63.14) in Table 1 quantify the failure regime. Figure 12 provides a qualitative example of successful complex diagnosis (hairy cell leukemia identification), but this is cherry-picked and does not characterize failure frequency or failure modes. The paper acknowledges in Section 7 that "hard diagnostic reasoning problems remain challenging" (as quoted in the Executive Summary), but does not characterize why they remain challenging — whether failures stem from knowledge gaps, reasoning errors, perceptual misses, or tool-use failures. No error analysis or confusion matrix is provided for any benchmark.

Mitigation status. The paper partially acknowledges this limitation in Section 7: "we identify limitations shared by current medical VLMs, including mitigating hallucinations in long-form generation, improving reliability under distribution shift, and strengthening multi-step medical reasoning. Addressing these challenges is a core part of our ongoing work." However, this framing presents the limitations as "shared by current medical VLMs" rather than as residual weaknesses of the specific methodology, and does not attribute them to specific failure modes in the MedXIAOHE pipeline. The paper does not quantify how much headroom exists between MedXIAOHE's current performance and clinical viability, or characterize which types of hard cases the model handles successfully vs. unsuccessfully. The suggestion to pursue "deeper evidence-grounding, more reliable tool-use, and broader coverage" does not specify which of these is the binding constraint for the observed diagnostic reasoning failures.

Base Model Details and Resource Requirements Are Undisclosed, Blocking Reproducibility and Fair Comparison

The assumption or constraint. The paper provides no information about the underlying model architecture beyond the high-level description of "Seed-ViT vision encoder + MLP adapter + autoregressive LLM decoder" (Section 2). The parameter count, training FLOPs, LLM backbone identifier, tokenizer details, number of layers, hidden dimensions, attention head configuration, and context window length are not disclosed for either the base Seed-VL model or the resulting MedXIAOHE model. No training hardware, wall-clock training time, or approximate compute budget is reported. The pretraining data is described in aggregate token counts (640B total) but individual source sizes are not precisely specified, and the data mixing ratios are not disclosed. The mid-training and post-training data volumes are not reported at all. The GenRM used for caption filtering, the LLM used for entity extraction and QA synthesis, and the multi-expert models used for reject sampling are identified only as "a general-domain LLM," "LLMs," or "expert models" without specification.

The consequence. The paper's results are not reproducible from the report alone. A practitioner who wants to build a medical VLM using the described methodology cannot determine whether MedXIAOHE's performance is attributable to model scale (if the underlying Seed-VL is a large model), training data volume (640B tokens is substantial), specific training hyperparameters (none reported), or the described methodological innovations. The comparison against GPT-5.2 Thinking, Gemini 3.0 Pro, and Gemini 2.5 Pro is confounded by unknown differences in model scale, architecture, and pretraining data — we cannot determine whether MedXIAOHE outperforms these baselines because of superior methodology or because the underlying Seed-VL model is comparable or larger in scale, or because 640B tokens of domain-specific pretraining provides an advantage independent of the MET-based curation strategy. The absence of comparisons against open-source medical VLMs (LLaVA-Med, Med-Flamingo, HuatuoGPT-Vision) — models for which architecture, parameter count, and training data are publicly known — means we cannot assess whether MedXIAOHE's gains come from methodology, base model quality, or training scale. This is particularly concerning given that the paper explicitly states architecture innovation is not the differentiator (Section 2), implying that the claimed contributions are the data curation and training methodology — contributions that cannot be independently validated without knowing what model they were applied to.

What evidence exists in the paper. None that addresses scale, architecture details, or resource requirements. The only model-specific information is the name "Seed-VL" and a citation to Guo et al. (2025a). The only training data volume reported is the 640B token pretraining corpus aggregate. The only compute-related number is the 20-hour AC-Automaton scan time for entity mapping. Training hyperparameters (learning rate, batch size, optimizer, sequence length, number of training steps, warmup schedule, gradient accumulation) are not reported anywhere in the paper or appendices for any training stage (continual pretraining, mid-training, SFT, RL). This stands in contrast to the example paper, which provided detailed hyperparameters (learning rate, batch size, dropout, optimizer, betas, early stopping criteria) for PRM training, revision model fine-tuning, and all search configurations.

Mitigation status. Not addressed. The paper does not acknowledge the lack of model and resource specifications as a limitation, does not indicate whether these details will be released in a future version or technical addendum, and does not provide even order-of-magnitude estimates (e.g., "a model with approximately X parameters, trained on Y GPUs for Z days"). The "comprehensive recipe" framing is undermined by the absence of the most basic recipe components: quantities and settings.

Report-Generation Faithfulness and Hallucination Are Claimed but Not Measured

The assumption or constraint. One of MedXIAOHE's headline capabilities is "low-hallucination long-form report generation with improved adherence to medical instructions" (Abstract). The paper describes an extensive pipeline for hallucination control: the caption construction pipeline (Section 3.3.3) with rubric-based GenRM filtering, the in-house caption benchmark with "reward-and-penalty design" that "encourages clinically faithful, high-utility captions," and the evidence-grounded report generation training. However, the paper reports no direct hallucination measurement for generated reports — no metric counting fabricated findings, no comparison of generated reports against ground-truth annotations for unsupported claims, and no human evaluation of report faithfulness. The public report benchmark evaluations (MIMIC-CXR: 50.86, CheXpert Plus: 49.43, IU-Xray: 65.66) use unspecified metrics that may rely heavily on n-gram overlap with reference reports, which can reward verbose but partially hallucinated text if it contains overlapping terminology.

The consequence. We cannot determine whether MedXIAOHE actually produces fewer hallucinations than baseline models, or whether its report generation simply optimizes for different surface-level characteristics. The underperformance on IU-Xray (65.66 vs. Gemini 3.0 Pro's 73.51) is acknowledged in Section 2 as "suggesting remaining headroom for narrative in certain report-generation regimes," but this framing assumes the metric captures desirable properties. If MedXIAOHE's reports are more conservative — making fewer unsupported claims in exchange for lower n-gram overlap with reference reports — then the IU-Xray gap might reflect a desirable safety-accuracy tradeoff. Alternatively, if MedXIAOHE hallucinates at similar rates to baselines but scores lower because its reports are less stylistically similar to radiologist-written references, then the GenRM filtering and evidence-grounding pipeline are not improving faithfulness. Without direct hallucination measurement, these competing interpretations cannot be disambiguated. For clinical deployment, the hallucination rate of a report-generation system is perhaps the single most important safety metric — a model that gets 95% of findings correct but fabricates 5% could cause more harm than one that gets 80% correct but makes no false claims. The paper's failure to measure this directly is a substantial gap between its claims and its evidence.

What evidence exists in the paper. The in-house caption benchmark uses a custom scoring scheme that "penalizes incorrect statements" (Section 6.2), which superficially addresses faithfulness. However, the paper does not report: how penalty and reward weights are calibrated, whether incorrect statements are weighted equally regardless of clinical severity, inter-annotator agreement on incorrectness judgments, the maximum possible score, or human baseline performance. The raw score of 46.89 (with Gemini 2.5 Pro at 45.92) is uninterpretable without this context — a 1-point margin could represent meaningful reduction in hallucinations or could be noise. Figure 13 provides a qualitative example of structured report generation with step-by-step reasoning, but this is a single cherry-picked case that demonstrates successful generation, not typical behavior or hallucination frequency.

Mitigation status. Not acknowledged. The paper presents the GenRM filtering, evidence-grounding pipeline, and in-house caption benchmark as solutions to the hallucination problem without acknowledging that hallucination rates are never directly measured. The Section 7 future work mentions "mitigating hallucinations in long-form generation" as a continuing challenge, but frames this as an area for improvement rather than an unvalidated claim in the current work. A direct hallucination measurement — for example, having radiologists annotate a sample of generated reports for fabricated vs. supported findings, and comparing rates across MedXIAOHE and baselines — would substantially strengthen the paper's central safety claim and is feasible within the paper's evaluation framework.

The Benchmark-Defined Success Narrative Is Vulnerable to the Evaluation Fragmentation the Paper Itself Critiques

The assumption or constraint. A core motivation of the paper is that "evaluation practice in the medical VLM literature remains fragmented" and that "reported numbers are often not directly comparable due to protocol-level mismatches rather than model differences" (Section 6). The Unified Med-VLM Benchmark is presented as a solution: a standardized harness with consistent prompting, answer parsing, scoring, and contamination control across 30+ benchmarks. However, the paper's own evaluation framework introduces new sources of incomparability and assumes — without validation — that the benchmark suite covers all deployment-critical capabilities.

The consequence. The Unified Med-VLM Benchmark, while a methodological improvement over ad-hoc evaluation, does not escape the fundamental problem it diagnoses: the mapping from benchmark performance to clinical utility is unknown. The paper does not provide evidence that higher scores on the Unified Med-VLM Benchmark correlate with better clinical outcomes, reduced diagnostic errors, or improved patient safety. Several deployment-critical capabilities that the paper identifies as important — robustness to input variation (uncontrolled imaging conditions, document deformations), reliability under distribution shift, appropriate uncertainty communication — are tested only through in-house benchmarks (Inhouse VQA, Inhouse OCR, Inhouse Caption) for which no public baseline comparisons are available beyond Gemini 2.5 Pro. The strongest baseline systems (GPT-5.2 Thinking, Gemini 3.0 Pro) are not evaluated on these in-house benchmarks at all (marked "—" in Table 1). This creates a situation where MedXIAOHE can claim superiority on exactly the benchmarks that are hardest to independently verify, while the benchmarks where independent comparison is possible show mixed results. Furthermore, the in-house benchmarks themselves are described in only general terms (Section 6.2) — the Inhouse VQA "contains more than 100k questions and fully aligns with real-world clinical applications," but the question distribution, difficulty composition, answer format, and clinical domain coverage are not specified. The in-house OCR benchmark includes "perspective distortion, blur, shadows, glare, partial occlusion, and cluttered backgrounds," but the severity distribution of these artifacts and the accuracy metric are not detailed. This makes it impossible for external researchers to assess whether the in-house benchmarks are fairly constructed or whether they inadvertently favor MedXIAOHE's specific training distribution. The dedicated in-house benchmarks were motivated by the observation that "capabilities that strongly influence product readiness... are absent or under-measured from many public leaderboards" (Section 1), but by making these benchmarks private, the paper recreates the very evaluation opacity it critiques — claims of deployment readiness on unverifiable metrics.

What evidence exists in the paper. Table 1 shows MedXIAOHE with higher scores than Gemini 2.5 Pro on Inhouse VQA (76.77 vs. 69.64), Inhouse Caption (46.89 vs. 45.92), and Inhouse OCR (81.92 vs. 80.82). These margins are small (1–7 points) and are evaluated against only one baseline system that is not the strongest available. The paper does not report the maximum possible scores or human performance on these benchmarks. Section 6.2 describes the in-house benchmarks at a high level but does not provide the detail necessary for independent replication or critical assessment. No correlation analysis between in-house and public benchmark performance is reported, which would help validate that the in-house metrics measure something distinct from the public ones.

Mitigation status. Not acknowledged. The paper presents the Unified Med-VLM Benchmark as a contribution to standardized evaluation without acknowledging that the in-house components undermine this standardization. The paper does not indicate whether the in-house benchmarks will be released publicly (a standard practice for benchmark contributions), describe plans for third-party auditing of benchmark quality, or discuss the tension between using private benchmarks to demonstrate deployment readiness and the paper's stated goal of "standardized, interpretable, and actionable" evaluation.

7. Implications and Future Directions

How This Work Changes the Landscape

This work does not introduce a paradigm shift — it does not propose a new architecture, a new learning algorithm, or a new theoretical framework. Its contribution is better understood as a methodological reframing with significant practical consequences for how medical VLM development is organized. The paper changes the landscape in two specific ways that, together, raise the bar for what counts as evidence in medical VLM research.

First, it makes knowledge coverage an explicit, measurable engineering objective rather than an implicit hope. Prior to this work, medical VLMs were built by collecting large datasets from diverse sources and assuming coverage would be adequate — an assumption that was rarely tested and virtually never quantified. The Medical Entity Tree (Section 3.2) and its associated AMCS metric (Table 2) introduce a verifiable coverage epistemology: you can now measure which clinical concepts your training data covers, identify gaps quantitatively, and track whether your data curation is closing those gaps. This matters because it converts a previously invisible failure mode (long-tail knowledge gaps that surface as confident but incorrect answers on rare conditions) into a visible, manageable engineering problem. A developer building a medical VLM after this paper can — and arguably should — be expected to report coverage statistics, just as a developer of a general-domain LLM is now expected to report training FLOPs and data scale after Chinchilla scaling laws became standard.

However, it is important to be precise about what this reframing does NOT do. The paper demonstrates that the MET is comprehensive (Table 2: forward coverage exceeding 0.95 against multiple baselines), which is a measurement of an engineering artifact — the taxonomy — not a measurement of model capability. The paper does NOT demonstrate that MET-guided training produces better models than taxonomy-agnostic training at equivalent data scale. Without that ablation, the reframing is methodologically compelling but empirically unvalidated: it tells the field what to measure (coverage) but not how much measurement precision matters for downstream performance. This is analogous to early work on data quality metrics in NLP that introduced better ways to characterize training data but took years of follow-up work to establish the relationship between data quality metrics and model performance. The paper's contribution is thus best understood as a diagnostic tool for the medical VLM development process that has strong face validity but awaits causal validation.

Second, it identifies the perception-reasoning conflict as a specific, named failure mode with a concrete resolution strategy. The observation that aggressive reasoning training can degrade perceptual capabilities is not new — multi-task learning researchers have documented capability interference for years. But the paper's contribution is naming the specific mechanism (verbose reasoning diluting visual attention, causing the model to "operate purely in the textual space") and providing a specific architectural response (Personalized Visual CoT with separated logical content and cognitive style, enforced visual anchoring through low-hallucination captions, and dual-layer quality gates). This changes the landscape by giving medical VLM developers a concrete thing to watch for and a concrete thing to try: when your model's diagnostic accuracy improves but its lesion detection worsens, you are likely seeing perception-reasoning conflict, and the remedy is not to scale up or add more data but to restructure your reasoning training to keep it anchored to visual evidence.

The practical impact of this identification is that it redirects research attention from "how do we make models reason better?" to "how do we make models reason better without sacrificing perception?" — a more constrained, more productive question. Prior to this work, a medical VLM developer seeing perceptual degradation after reasoning training might have attributed it to generic catastrophic forgetting and responded with replay buffers or regularization. The paper's analysis suggests this would be treating the symptom rather than the cause: the issue is not forgetting old visual patterns but learning new reasoning patterns that overwrite the model's tendency to attend to visual evidence. The Personalized Visual CoT approach — separation of logical content from style, multi-agent architecture for style adaptation, dual-layer quality gates validating both logic and style consistency — provides a concrete template for how to address this specific failure mode.

Third, the paper implicitly raises the standard for medical VLM evaluation by demonstrating a unified benchmark harness and then showing that it reveals heterogeneity that single-benchmark reporting obscures. Table 1 shows MedXIAOHE winning on some benchmarks (SLAKE: 82.62 vs. 72.83; Medbullets_op5: 94.16 vs. 88.31; MedBrowseComp: 29.00 vs. 19.00) while losing on others (GMAI-MMBench: 55.21 vs. 66.81; IU-Xray: 65.66 vs. 73.51; MedXpertQA-MM: 71.00 vs. 77.20). The co-existence of strong wins and strong losses on the same model, evaluated under the same protocol, demonstrates that medical VLM capability is not a scalar — you cannot summarize a medical VLM with an average score across benchmarks, because different models have different capability profiles that are obscured by aggregation. This is a methodological contribution to the field's evaluation practices: the paper shows, by example, what a properly multidimensional evaluation looks like, and demonstrates that the heterogeneity of results is itself informative (it reveals where the model's strengths lie — rare disease knowledge, tool-augmented search — versus where its weaknesses remain — broad-coverage radiology QA, narrative report generation).

The paper also partially reconciles a contradiction in the literature regarding how well general-domain VLMs perform on medical tasks. Prior to this work, it was known that general-domain models (GPT-4V, Gemini) could achieve surprisingly strong results on medical benchmarks without domain-specific training, raising questions about whether domain-specific pretraining was necessary. Table 1 provides a nuanced answer: general-domain models are competitive on broad-coverage benchmarks (GMAI-MMBench, VQA_RAD, IU-Xray) and on text-based clinical reasoning (MedQA, Medbullets), but domain-specific training produces meaningful gains on benchmarks requiring specialized medical knowledge (RareBench: +4.6 points over GPT-5.2 Thinking; MedBrowseComp: +10 points; SLAKE: +10 points; PATH_VQA: +11 points). The implication is that domain-specific training matters — but specifically for the long tail and for modality-specific reasoning, not for exam-style QA where general reasoning and broad knowledge suffice. This is a more actionable answer than the binary "domain training helps" or "domain training doesn't help" positions that characterized earlier debates.

Follow-Up Research This Work Enables

Coverage-to-performance causal analysis: does MET-guided training produce better models than taxonomy-agnostic training? The paper provides a coverage measurement framework (AMCS metric, Table 2) but does not establish that improved coverage causes improved downstream performance. A strong follow-up would train two versions of MedXIAOHE — one with the MET-guided data balancing and entity-targeted synthesis described in Sections 3.2–3.4, and one with the same total data volume but randomly sampled (no entity balancing, no "weak entity" targeting for the KG-guided QA synthesis) — and compare performance on benchmarks stratified by entity frequency. The key question: does the MET-guided model outperform the random-sampling model on questions involving rare entities (below-median frequency in the pretraining corpus), and by how much? A null result (no difference) would suggest that the MET's value is in diagnosis and measurement, not in training optimization, and that simpler data collection strategies with larger scale may achieve equivalent coverage. A positive result (meaningful gains on rare-entity questions) would validate the central methodological claim of the paper and set a precedent for coverage-aware training in medical AI. The RareBench results (46.79 vs. 42.19 for GPT-5.2 Thinking) provide suggestive evidence but cannot distinguish MET-guided training from other pipeline components or from the sheer volume of medical data.

Perception-reasoning conflict: quantitative measurement and intervention comparison. The paper identifies the perception-reasoning conflict through "initial experiments" (Section 4.1.1) but does not report the data. A strong follow-up would design a controlled measurement: take a base medical VLM, fine-tune it with increasing amounts of long CoT reasoning data (varying chain length and verbosity), and measure both diagnostic accuracy (on reasoning-intensive tasks) and perceptual accuracy (on lesion detection, abnormality classification) at each training stage. The hypothesis is that perceptual accuracy will show an inverted-U relationship with reasoning training intensity — improving initially (as reasoning helps the model attend to relevant features) then declining (as verbose reasoning dilutes visual attention). The follow-up would then compare three interventions at the point of perceptual decline: (a) Personalized Visual CoT (short reasoning chains with visual anchoring, as the paper describes), (b) standard catastrophic forgetting mitigation (replay buffers, elastic weight consolidation), and (c) architectural separation (frozen vision encoder). The key outcome is whether Personalized Visual CoT produces better perceptual retention than generic forgetting mitigation, which would validate that the mechanism is attention dilution rather than weight interference, and whether architectural separation (keeping the vision encoder frozen) is sufficient, which would provide a simpler alternative to the paper's multi-agent CoT pipeline.

Tool-use necessity: when do tools actually help, and for which model scales? The paper's task-with-instrument effectiveness filtering (Section 4.1.2) selects training cases where tools improve performance, but the filtering is interleaved with training and the final tool-use policy is not characterized. A strong follow-up would evaluate MedXIAOHE on the same diagnostic benchmarks with and without tool access at inference time, measuring both accuracy change and average tool calls per query. The hypothesis is that tools provide the largest gains on questions requiring external knowledge retrieval (drug interactions, rare disease facts, recent clinical guidelines) and minimal gains on questions answerable from parametric knowledge. If this hypothesis holds, the tool-use policy could be optimized: route queries to tool-free inference when the question type suggests parametric sufficiency (e.g., common-disease MCQ), and to tool-augmented inference when external evidence is likely needed (e.g., rare disease diagnosis, drug interaction checking). This would address the practical concern that tool-augmented inference is slower and more expensive than parametric inference, and should be deployed selectively. A crucial negative result would be if tools do NOT provide measurable benefit over parametric inference on most benchmarks — this would suggest the tool-augmented training in Section 4.1.2 primarily serves as a regularization mechanism (teaching the model to be more careful and evidence-grounded) rather than an inference-time capability, and the MedBrowseComp gains (29.00 vs. 19.00) might reflect improved general reasoning rather than effective tool use specifically.

Unified Med-VLM Benchmark as a community standard: replication across model families. The paper introduces a standardized evaluation harness but evaluates only one model (MedXIAOHE) against three closed-source baselines. A strong follow-up would apply the Unified Med-VLM Benchmark protocol to a broader set of models — specifically, open-source medical VLMs (LLaVA-Med, Med-Flamingo, HuatuoGPT-Vision, MedVLM-R1) and the base Seed-VL without medical training — to establish a public leaderboard with standardized prompting, answer parsing, and decontamination. This would serve three purposes: (1) it would validate that the benchmark protocol produces consistent, reproducible results across independent evaluations (since these models are publicly available and can be re-evaluated by anyone), (2) it would establish whether MedXIAOHE's gains over the baselines in Table 1 are attributable to domain-specific methodology or base model quality (if the base Seed-VL scores similarly to MedXIAOHE, the medical training pipeline adds little; if it scores substantially lower, the domain-specific training is doing heavy lifting), and (3) it would make the benchmark suite useful to the broader community — a contribution that would persist even if MedXIAOHE itself is superseded. The in-house benchmarks (Inhouse VQA, Inhouse OCR, Inhouse Caption) pose a problem for community adoption; the follow-up would need to either release these benchmarks publicly (with appropriate privacy protections) or develop public proxies that capture the same deployment-critical capabilities (e.g., using publicly available deformed-document datasets for OCR robustness, existing clinical image datasets with detailed annotation for lesion-level faithfulness).

Computational cost of the comprehensive recipe: a Pareto analysis. The paper provides no resource quantification, making it impossible for practitioners to determine which components of the "comprehensive recipe" provide returns proportional to their cost. A strong follow-up would perform a systematic resource analysis: (a) measure the preprocessing compute for MET construction (LLM entity extraction, K-Means clustering with Silhouette Coefficient optimization, ReAct-agent conflict resolution), the data ordering curriculum (UMAP+HDBSCAN on the full corpus), and each synthesis pipeline (text QA, image captions, OCR distillation, grounding, CoT generation), (b) measure the training compute for continual pretraining, mid-training, and post-training, (c) measure inference compute (with and without tools), and (d) conduct a series of "subtractive ablations" — removing one major pipeline component at a time and measuring both the compute savings and the performance impact. The output would be a Pareto frontier showing, for a given total compute budget, which combination of pipeline components maximizes benchmark performance. This would transform the paper's "comprehensive recipe" (which implies all components are necessary) into an actionable resource-allocation guide (which identifies the most cost-effective components and the points of diminishing returns). The paper's claim that "scaling training data and compute — together with stronger evaluation — remains an immediate path to improving performance" (Section 7) is untestable without baseline resource measurements.

Hallucination measurement in report generation: a direct, clinically meaningful metric. The paper claims "low-hallucination long-form report generation" (Abstract) but reports no direct hallucination measurement. A strong follow-up would design a targeted hallucination evaluation: sample a set of generated reports (from MedXIAOHE and baselines) paired with their input images, have board-certified radiologists annotate each sentence as "supported by image evidence," "unsupported but clinically plausible," or "unsupported and clinically incorrect," and compute hallucination rates per model. The key comparison would be MedXIAOHE vs. Gemini 3.0 Pro on IU-Xray — where Gemini leads on the existing metric (73.51 vs. 65.66) — to determine whether MedXIAOHE's lower score reflects more conservative (less hallucinated) reporting or genuinely worse quality. If MedXIAOHE hallucinates less, the IU-Xray gap would be reinterpreted as a desirable safety-accuracy tradeoff, and the benchmark community would need to develop metrics that reward faithfulness alongside fluency. If MedXIAOHE hallucinates at similar rates to Gemini, the GenRM filtering and evidence-grounding pipeline (Section 3.3.3) would be shown to be ineffective for their stated purpose. The paper's in-house caption benchmark with its "reward-and-penalty design" (Section 6.2) partially addresses this, but the lack of public access and the absence of hallucination-rate reporting for this benchmark prevent independent verification.

Practical Applications and Downstream Use Cases

Medical education and exam preparation. The paper's strongest and most consistent results are on text-based clinical QA benchmarks modeled after licensing examinations: MedQA_USMLE (97.88), MedQA_MCMLE (96.21), Medbullets_op5 (94.16), CMExam (96.12). These scores represent near-ceiling performance on standardized medical knowledge assessments, with gaps to perfect scores ranging from 2.1 to 5.8 points. For medical students and residents preparing for licensing exams, a system that achieves 94–98% accuracy on practice questions — with the added capability to explain its reasoning through the structured CoT trained during mid-training — could serve as a high-quality study aid that not only provides correct answers but models the clinical reasoning process. The multi-turn dialogue capability demonstrated on HealthBench-normal (65.20, leading all baselines) suggests the model can engage in sustained educational interactions rather than single-shot Q&A. The key practical advantage over general-domain models: MedXIAOHE's entity-aware pretraining provides broader coverage of rare conditions and specialized knowledge (RareBench: 46.79) that may appear on advanced exams but be underrepresented in general-domain training. The limitation: HLE-Med (25.77) and DiagnosisArena (42.08) show that the model still struggles with the hardest diagnostic reasoning, so it would be most appropriate for foundational and intermediate education rather than expert-level challenge problems.

Clinical decision support for rare disease screening. RareBench (46.79 vs. 42.19 for GPT-5.2 Thinking) and MedBrowseComp (29.00 vs. 19.00 for GPT-5.2 Thinking) indicate specific strengths in rare disease reasoning and evidence retrieval. The absolute scores are modest — the model gets roughly half of rare disease questions wrong and 71% of deep-search questions wrong — but the relative improvement over the best general-domain system (4.6 and 10.0 absolute points, respectively) suggests the entity-aware pretraining and tool-augmented training provide advantages specifically for these long-tail tasks. In a clinical decision support context, a system that can retrieve and synthesize evidence about rare conditions — even imperfectly — could accelerate the diagnostic process for patients with atypical presentations. The practical deployment model would be a "second reader" or differential diagnosis generator: a clinician inputs a complex case with unusual features, the model retrieves relevant literature and case reports (via Search Drug, Search Clinical, Scholar Search), and proposes candidate diagnoses with supporting evidence and reasoning traces. The model's tool-use capability (zoom for image inspection, search for evidence lookup) makes it more suited to this interactive workflow than a static QA system. The limitation: the 71% error rate on MedBrowseComp means the model would miss or misrank relevant evidence in the majority of cases, so it would need to be positioned as "suggestions to consider" rather than "diagnoses to rely on," with human clinicians making final decisions.

Automated radiology report drafting with human-in-the-loop review. The report generation results on MIMIC-CXR (50.86) and CheXpert Plus (49.43) indicate the model can produce clinically relevant report drafts. In a high-volume radiology practice, an automated system that generates preliminary findings from chest X-rays could reduce radiologist workload by providing a structured first draft that the radiologist reviews and edits, rather than requiring de novo report composition. The evidence-grounded reasoning pipeline (structured CoT with visual observation → reasoning → conclusion, exemplified in Figure 13) produces reports with explicit reasoning traces that a reviewing radiologist can verify against the image — a significant advantage over black-box generation. The practical workflow: the model ingests a chest X-ray, produces a structured findings section with anatomical region-by-region analysis and an impression section summarizing key abnormalities, and the radiologist reviews both the report and the reasoning trace, accepting, modifying, or rejecting findings as appropriate. The key requirement: the model must "fail safe" — that is, it must not fabricate findings that a busy radiologist might overlook during review. The paper's GenRM filtering (Section 3.3.3) and in-house caption benchmark's penalty for incorrect statements are designed to encourage this conservatism, but as noted in Section 6, hallucination rates are not directly measured, so deployment would require institution-specific validation of false positive rates. The IU-Xray result (65.66 vs. 73.51 for Gemini 3.0 Pro) suggests the model may indeed be more conservative — producing fewer findings overall and thus achieving lower n-gram overlap with reference reports — which would be a desirable property for this workflow if confirmed by direct faithfulness measurement.