Towards Autonomous Mechanistic Reasoning in Virtual Cells

ArXiv: 2604.11661

🎯 Pitch

LLMs struggle to produce reliably accurate biological explanations—but here, structuring reasoning as verifiable mechanistic graphs and pairing it with domain-grounded retrieval slashes hallucination and lifts gene expression prediction F1 by over 15 points. The key is replacing unbounded generation with a verifier-filtered multi-agent loop that checks each causal claim against real binding and expression data, making virtual cell explanations finally falsifiable and useful downstream.

1. Executive Summary

This paper introduces VCR-Agent, a multi-agent framework that generates and validates structured mechanistic explanations for virtual cells by decomposing the reasoning process into two specialized modules—a report generator that retrieves and synthesizes factual biological knowledge from external databases, and an explanation constructor that transforms this grounded report into a formal Directed Acyclic Graph of mechanistic actions connected by causal dependencies. Using this framework on the Tahoe-100M atlas with Claude 4 as the backbone LLM, the authors release VC-Traces, a dataset of verified mechanistic explanation traces filtered through specialized verifier-based pipelines (drug-target interaction verifiers using Boltz-2 for binding plausibility, differential expression verifiers querying ground-truth data for gene regulation claims). On the downstream TahoeQA gene expression prediction task, models trained with these structured explanations as supervision signals achieve up to 15.3 percentage point improvement in differential expression F1-score over standard supervised fine-tuning (e.g., 0.435 average F1 for SFT-Prompt vs. 0.292 for vanilla SFT), establishing that grounding high-dimensional transcriptomic data in explicit biological reasoning improves predictive accuracy on novel compounds only when explanations are both factually grounded through retrieval and structurally constrained to enable systematic verification and falsification.

2. Context and Motivation

The Core Problem: Bridging the Gap Between LLM Reasoning and Scientific Discovery

The fundamental challenge this paper addresses is a critical mismatch: large language models (LLMs) have demonstrated remarkable reasoning capabilities in structured domains like mathematics and programming, but these same capabilities fail to transfer to open-ended scientific domains such as biology. The paper opens by stating this directly:

"However, directly transferring the reasoning training paradigms from mathematics or coding to scientific discovery is not straightforward."

This gap is not merely an inconvenience—it represents a fundamental obstacle to using LLMs as genuine scientific reasoning engines rather than sophisticated pattern matchers. The authors frame this around the emerging concept of virtual cells: computational models that simulate cellular behavior, predict responses to perturbations (like drug treatments or genetic knockouts), and ideally produce mechanistically grounded explanations for their predictions. As they note in Section 1, a central goal of these models is to move "beyond correlation-based prediction toward actionable insight," but generating explanations that are "both biologically plausible and reliable remains a critical bottleneck."

Why does this matter? The practical stakes are substantial. Virtual cells promise to accelerate drug discovery by predicting how compounds will affect cellular systems before running expensive wet-lab experiments. But prediction without explanation is of limited value to biologists—a model that says "drug X will kill cancer cell Y" without explaining how (which pathways, which molecular targets, which downstream effects) offers no mechanistic insight that could guide iterative drug design or help researchers understand why a prediction might fail in a different cellular context. The authors position their work within the broader vision articulated by Noutahi et al. (2025), Bunne et al. (2024), and Adduri et al. (2025), where virtual cells must "predict, explain, discover"—and explanation is the missing piece of this triad.

Two Dimensions of the Problem: Data Scarcity and Verification Impossibility

The paper identifies two intertwined reasons why LLM reasoning fails to transfer to biology. Understanding both is essential because they require fundamentally different solutions.

Data Scarcity: We Cannot Annotate Our Way Out of This

The first problem is a data-centric bottleneck. The most successful LLM reasoning systems—those that power mathematical problem-solving (e.g., DeepSeekMath, o1) and code generation—are trained on vast corpora of high-quality reasoning traces. These traces come from two sources:

1. High-quality human annotations (Cobbe et al., 2021 for MATH; Gao et al., 2024 for Omni-MATH; Hendrycks et al., 2021 for the MATH dataset itself), where domain experts write out step-by-step solutions that serve as supervision signals for training reasoning models.
2. Large-scale synthetic generation by LLMs (Wang et al., 2023's Self-Instruct approach; Moshkov et al., 2025 for mathematical reasoning; Guha et al., 2025 for the OpenThoughts dataset), where powerful models generate reasoning traces that are then filtered or used as training data.

Neither approach works well for biology. Human annotation requires specialized expertise—you need trained biologists, pharmacologists, or bioinformaticians to write mechanistic explanations of how a drug affects a specific cell type. This is prohibitively expensive and non-scalable: the paper notes that such annotation is "particularly in domains that require specialized expertise, such as biology." Unlike math problems, where a single expert can verify thousands of solutions, biological reasoning spans molecular interactions, pathway dynamics, transcriptional regulation, and phenotypic outcomes—each requiring different domain knowledge.

Synthetic generation is equally problematic. LLMs without sufficient biological grounding "frequently exhibit factual inconsistency or domain hallucination," as stated in Section 6. An LLM prompted to "explain how Binimetinib affects C32 melanoma cells" might produce a superficially plausible but factually incorrect reasoning chain—inventing non-existent protein interactions, misattributing gene regulation directions, or fabricating binding affinities. In mathematics, you can check the answer against a ground truth; in biology, there is no single ground truth to check against, and the generated reasoning may be plausible-sounding yet biologically wrong in ways that only domain experts can detect.

This data scarcity creates a chicken-and-egg problem: you need high-quality reasoning traces to train models that generate good biological explanations, but generating those traces requires either expensive human experts or reliable verifiers that don't yet exist.

Verification Impossibility: Biology Has No Compiler

The second problem is even more fundamental. In mathematics and programming, reasoning traces can be automatically verified for correctness:

• Code can be executed against test cases—if the program passes the tests, the reasoning trace that produced it is at least partially validated.
• Mathematical proofs can be checked for logical consistency, and final answers can be compared to known solutions.
• Symbolic manipulation systems can verify algebraic derivations step by step.

Biological reasoning admits no such direct verification. As the paper explains:

"Biological reasoning, in contrast, rarely admits such direct verification because it relies on disjointed knowledge from scientific literature rather than deterministic rules."

Consider a reasoning trace that claims "Binimetinib binds to MAP2K1 with 12 nM affinity, inhibiting MEK kinase activity, which reduces ERK phosphorylation, leading to decreased MYC transcription and ultimately G1/S cell cycle arrest." Each step in this chain involves biological claims that cannot be automatically verified in the way code can be executed:

• The binding claim depends on experimental measurements that may vary across assays and conditions.
• The pathway modulation claim involves complex signaling dynamics that are context-dependent.
• The transcriptional regulation claim may hold in some cell types but not others.
• The phenotypic outcome may be partially true (cytostatic effect) while missing important nuances (compensatory pathway activation, resistance mechanisms).

This "inherent ambiguity," as the paper calls it, creates a critical bottleneck: you cannot simply generate thousands of reasoning traces, filter them with an automatic verifier (as is done in mathematical reasoning with outcome reward models or process reward models), and train on the survivors. Without reliable verification, LLM-generated explanations remain untrustworthy, and the entire supervised fine-tuning pipeline for reasoning fails.

Prior Approaches and Their Specific Failures

The paper situates itself against three categories of prior work, each of which addresses part of the problem but leaves critical gaps.

Unstructured Free-Form Reasoning Without Grounding

The most direct prior approach is simply prompting LLMs to generate natural-language explanations of biological mechanisms. This is what the paper calls "unstructured, free-form natural language rationales" (Section 6). Recent work in this vein includes:

• SynthPert (Phillips et al., 2025): SFT on GPT-4o reasoning traces for cellular perturbation prediction.
• Inference-time compression of gene-centric knowledge graphs (Wu et al., 2025).
• RL frameworks with soft verifiers (Istrate et al., 2025), which train biological reasoning LLMs using biological world models as soft verifiers.

The paper identifies three specific limitations of this body of work:

1. Lack of reliable fact discovery. These models "rely on internal parametric weights without sufficient integration of external knowledge, leading to biologically implausible hallucinations" (Section 6). An LLM's parametric knowledge about biology—learned during pretraining from the internet and scientific literature—is incomplete, sometimes outdated, and mixed with non-scientific content. When asked to reason about a specific perturbation in a specific cell type, the model may confidently assert facts that are plausible in general but wrong in this specific context. For example, it might claim a drug binds to a particular target because that target is mentioned in the drug's Wikipedia article, without knowing that the binding affinity is too weak to be biologically relevant at therapeutic concentrations.

2. Structural ambiguity. Because free-form natural language imposes no constraints on the form of the explanation, "unstructured rationales prevent the formalization of mechanistic dependencies, rendering them unsuitable for systematic verification" (Section 6). Two explanations of the same phenomenon might use different terminology, different levels of granularity, or different causal structures, making it impossible to automatically compare them or check their internal consistency. A biologist reading "the drug inhibits the pathway, which stops cell growth" cannot programmatically verify whether this claim is consistent with known pathway-regulatory relationships because the language is too vague.

3. Cross-modal insufficiency. Prior work "focuses almost exclusively on gene-centric perturbations, neglecting the complexity of diverse modalities, such as drug-induced responses" (Section 6). Many biological reasoning systems are designed for genetic perturbations (knockdowns, knockouts, overexpression) where the causal agent is a gene and the reasoning traces can be constrained to gene regulatory networks. But drug perturbations introduce entirely different modalities: chemical compounds with specific binding affinities, pharmacokinetic properties, off-target effects, and metabolism. A reasoning framework designed for "knockdown of gene X reduces expression of gene Y" cannot easily handle "compound Z with IC50 of 12 nM for target A and 4.6 nM for target B, with 97% plasma protein binding and 3.5-hour half-life."

The paper's perspective is that these limitations are not independent—they compound each other. Lack of factual grounding makes unstructured explanations unreliable; structural ambiguity prevents you from detecting when they're unreliable; and modality restrictions mean even if you solved the first two problems for genetic perturbations, you'd still be unable to handle the chemical perturbations that dominate drug discovery.

Knowledge-Grounded Approaches Without Structured Reasoning

A separate line of work focuses on augmenting LLMs with external knowledge bases for biological tasks. This addresses the factual grounding problem but leaves the reasoning problem unsolved. The paper's report generator (Section 3.1) builds on this tradition by integrating:

• StarkPrimeKG (Wu et al., 2024b): A biomedical knowledge graph providing relational information about genes, proteins, drugs, diseases, and their interactions.
• Harmonizome (Diamant et al., 2024): A gene-centric database aggregating information from diverse multi-omics resources.
• PubMed: The biomedical literature database for finding relevant research papers.
• Wikipedia: General background knowledge about drugs, genes, and diseases.

Prior systems might query one or more of these sources and present the retrieved information to the LLM as context, then ask the LLM to generate an explanation. The limitation, from the paper's perspective, is that this still produces unstructured natural language that cannot be systematically verified. The retrieved knowledge grounds the explanation in facts, but the explanation itself remains ambiguous—you cannot automatically check whether the reasoning chain connecting those facts is logically valid or whether important steps are missing.

Structured Biological Representations Without LLM Flexibility

A third body of work represents biological knowledge in structured formats—pathway databases like Reactome (Croft et al., 2010), gene ontology annotations (Ashburner et al., 2000), protein-protein interaction networks, and formal models of signaling cascades. These representations are precise, verifiable, and machine-readable. However, they are typically:

• Static: Representing what is known about biology in general, not what happens when a specific perturbation is applied to a specific cell type under specific conditions.
• Manually curated: Requiring enormous human effort to construct and maintain, which limits their coverage and scalability.
• Inflexible: Cannot easily accommodate the combinatorial explosion of possible perturbations × cell types × conditions that a virtual cell must reason about.

The paper's insight is that neither the fully unstructured (free-form text) nor the fully structured (hand-curated databases) approaches work alone. What's needed is a framework that generates structured reasoning on demand, combining the flexibility of LLM generation with the verifiability of structured representations.

How This Paper Positions Itself

The authors frame their contribution as a paradigm shift in how we think about biological reasoning with LLMs:

"We propose a paradigm shift toward structured mechanistic reasoning: treating biological reasoning as the autonomous construction of structured mechanistic graphs."

This is not merely an engineering contribution—it's a conceptual reframing. Rather than treating the LLM as an oracle that produces natural-language explanations (which we then hope are correct), the paper treats the LLM as a mechanistic architect that assembles explanations from a constrained vocabulary of biologically grounded actions. Each action (binds_to, modulates_pathway_activity, regulates_expression, etc.) has a fixed argument schema, and actions are connected by directed edges representing mechanistic dependencies, forming a Directed Acyclic Graph (DAG).

This framing addresses all three limitations of prior work simultaneously:

• Factual grounding: The report generator enforces this by retrieving relevant knowledge from external databases before any reasoning begins. The LLM does not generate explanations from its parametric knowledge alone—it synthesizes retrieved facts into a structured form.

• Structural verification: Because each action has a fixed schema with specific arguments (e.g., binds_to requires an actor, a target, and optionally an affinity), specialized verifiers can automatically check individual claims against reference databases. A drug-target interaction (DTI) verifier can check whether Binimetinib actually binds to MAP2K1 by running Boltz-2 (Passaro et al., 2025) to predict binding affinity. A differential expression (DE) verifier can check whether ERK1/2 inhibition actually downregulates MYC by querying the Tahoe-100M perturbation atlas.

• Cross-modal coverage: The action space (Section 2.2, Figure 3) spans molecular interactions, signaling modulation, protein regulation, transcriptional and translational regulation, epigenetic modifications, metabolic conversions, functional perturbations, phenotypic outcomes, and more. This covers both drug-induced and genetic perturbations within a unified framework.

Crucially, the paper does not claim to solve causal discovery or produce formal interventional causal graphs. It explicitly distinguishes its approach:

"This representation encodes mechanistic plausibility and downstream biological consequences, thereby enhancing the interpretability of the reasoning model's logic while remaining distinct from formal, interventional causal discovery."

This is an important humility: the generated DAGs represent mechanistic reasoning—plausible chains of biological events consistent with known facts—not proven causal relationships. The distinction matters because in biology, true causal discovery requires interventional experiments that are beyond the scope of any purely computational system.

The Specific Gap This Paper Fills

To state it precisely: prior to this work, there was no framework that could:

1. Generate biological explanations for arbitrary perturbation-context pairs,
2. In a structured, machine-verifiable format,
3. Grounded in retrieved factual knowledge rather than parametric memory,
4. With automated verification that filters out factually incorrect claims,
5. And demonstrate that training on these verified explanations improves downstream predictive performance.

Individual pieces existed—knowledge retrieval systems, structured biological databases, LLM-based explanation generation—but no prior work integrated them into a coherent pipeline that could produce verified, structured reasoning at scale and then use those reasoning traces as supervision signals for improved prediction. The Tahoe-100M atlas provided the raw perturbation data for verification and downstream training, but lacked the mechanistic reasoning layer connecting perturbations to outcomes. VCR-Agent fills this gap by providing the missing layer: a systematic way to construct, verify, and utilize mechanistic explanations that bridge the space between molecular perturbation and cellular phenotype.

The paper's ambitions are practical as well as conceptual. By releasing VC-Traces—a dataset of verified mechanistic explanations derived from 18,950 perturbation-context pairs—the authors aim to provide the research community with a resource that can serve a role analogous to what mathematical reasoning datasets (MATH, GSM8K) and code reasoning datasets (APPS, HumanEval) have done for those domains: a training signal that teaches models not just to predict outcomes, but to reason about how those outcomes arise. The downstream TahoeQA experiments demonstrate that this supervision signal indeed transfers: models trained to predict gene expression changes while conditioning on structured explanations substantially outperform models trained to predict directly, suggesting that the reasoning traces encode generalizable biological knowledge rather than mere pattern memorization.

3. Technical Approach

3.1 Reader Orientation

VCR-Agent is a multi-agent software pipeline that takes a biological perturbation (e.g., a drug compound) and a cellular context (e.g., a specific cancer cell line) as input, and produces a structured, verifiable mechanistic explanation of how that perturbation affects the cell, represented as a Directed Acyclic Graph (DAG) of discrete biological actions. The core problem it solves is that LLMs can generate plausible-sounding biological explanations but cannot ground them in factual knowledge or verify their correctness automatically—VCR-Agent addresses this by decoupling knowledge retrieval from structured reasoning generation and then filtering generated explanations through specialized verifiers that check individual claims against external databases, transforming the LLM from an unreliable narrator into a constrained mechanistic architect whose outputs can be systematically validated and falsified.

3.2 Big-Picture Architecture (Diagram in Words)

The system operates as a two-stage pipeline with a post-hoc verification step, visualized in Figure 1. Information flows through these major components:

1. Input: A perturbation-context pair $x = (p, c)$, where $p$ specifies the perturbation (chemical compound with SMILES structure, known targets, mechanism of action type) and $c$ specifies the cellular context (cell type, disease model, genetic background).

2. Report Generator (Stage 1): This module performs three sequential operations—named entity recognition to extract biomedical entities from the input, multi-source knowledge retrieval from four external databases (StarkPrimeKG, Harmonizome, PubMed, Wikipedia), and report synthesis where Claude 4 generates a comprehensive natural-language mechanistic report grounded in the retrieved facts.

3. Explanation Constructor (Stage 2): This module takes the knowledge-grounded report as input and uses Claude 4 with a carefully designed prompt to transform the free-text report into a structured explanation, consisting of nodes (mechanistic actions from a fixed action space of 20 primitives) and edges (directed dependencies forming a DAG).

4. Verifier-Based Filtering (Post-hoc): The generated DAG passes through specialized verifiers—a Drug-Target Interaction (DTI) verifier that checks binding claims using Boltz-2 binding affinity predictions, a Differential Expression (DE) verifier that checks gene regulation claims against ground-truth Tahoe-100M data, and optionally a subcellular Localization (LOC) verifier and Phenotype (PHENO) verifier—which score individual actions and filter out factually inconsistent claims.

The output is a filtered, verified structured explanation that can be used either as a standalone mechanistic hypothesis or as a supervision signal for training downstream prediction models (as demonstrated in the TahoeQA experiments).

3.3 Roadmap for the Deep Dive

• First, the problem formulation (Section 2.1)—what exactly is being generated, the DAG representation, and the formal definition of the reasoning task—because this defines the output space that constrains everything downstream.
• Second, the action space (Section 2.2)—the 20 action primitives and their argument schemas—because these are the vocabulary from which all explanations are built and the targets that verifiers check against.
• Third, the report generator (Section 3.1)—the entity extraction, multi-source retrieval, and report synthesis pipeline—because factual grounding is the prerequisite for reliable reasoning.
• Fourth, the explanation constructor (Section 3.2)—how the grounded report is transformed into the structured DAG format—because this is where the LLM's flexibility is channeled into verifiable structures.
• Fifth, the verifier-based filtering pipeline (Section 4)—the four specialized verifiers, their scoring mechanisms, and the filtering logic—because this is what separates factually correct claims from hallucinations.
• Sixth, the training configurations for downstream use (Section 5.2)—the two supervised fine-tuning strategies and their hyperparameters—because these show how the verified traces serve as supervision signals.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and dataset paper whose core idea is that structured, verifiable mechanistic explanations can be generated at scale by combining (a) multi-source biological knowledge retrieval to ground the LLM in facts, (b) a constrained action space that forces explanations into a machine-verifiable format, and (c) specialized verifiers that automatically filter out factually incorrect claims before the explanations are used as training signals.

Problem Formulation: What Does "Structured Reasoning" Mean?

The paper formalizes the reasoning task in Section 2.1 with mathematical precision, which is worth examining carefully because every downstream design choice follows from this definition.

Given an input $x = (p, c)$, where $p$ denotes the perturbation and $c$ denotes the cellular context, the goal is to generate a reasoning graph $G$ that captures a chain of mechanistic actions triggered by the perturbation in that context. The graph is defined as:

$G = (V, E)$

where $V = \{n_1, n_2, \ldots, n_k\}$ is the set of nodes, each representing a mechanistic action, and $E$ is the set of directed edges.

What it represents: Each node $n_i$ belongs to the predefined action space $\mathcal{A}$ (the 20 primitives defined in Section 2.2) and carries specific arguments (e.g., for a binds_to action, the node includes actor, target, affinity, etc.). Each directed edge $(n_i, n_j) \in E$ represents a mechanistic dependency—the outcome of action $n_i$ enables or influences action $n_j$. For example, a ligand-receptor binding event might precede a downstream signaling modulation.

What it computes: The reasoning model $f_\theta$ is a function:

$f_\theta : x \rightarrow G$

This means the model takes a perturbation-context pair and produces both the set of mechanistic actions (nodes) and their dependencies (edges) that together form a DAG explaining how the perturbation leads to observable cellular outcomes. Critically, this DAG is not a formal causal graph derived from interventional experiments—the paper explicitly states it "encodes mechanistic plausibility and downstream biological consequences, thereby enhancing the interpretability of the reasoning model's logic while remaining distinct from formal, interventional causal discovery."

Why this form: The DAG representation is chosen over free-form text for three specific reasons. First, it constrains the reasoning space: rather than allowing the model to produce arbitrary natural language (which might be vague, internally inconsistent, or unverifiable), the DAG structure forces the explanation into a finite vocabulary of actions with explicit dependencies. Second, it enables automatic verification: because each node is a specific action type with a known argument schema, specialized verifiers can check individual claims against databases—a DTI verifier can look up whether the drug actually binds to the claimed target, and a DE verifier can check whether the claimed gene regulation direction matches experimental data. Third, it preserves biological topology: cellular signaling naturally propagates through cascades of mechanistic events (binding → activation → phosphorylation → translocation → transcription → phenotype), and the DAG structure mirrors this inherent directionality. The alternative—generating unstructured text—would lose all three properties: no constraint on what can be claimed, no systematic way to verify individual claims, and no enforced representation of causal direction.

The paper's example in Figure 2a illustrates this concretely: for the perturbation Binimetinib (a MEK inhibitor) applied to C32 melanoma cells, the generated DAG in Figure 2b traces a path from drug binding (n1: binds_to MAP2K1, n2: binds_to MAP2K2) through activity modulation (n3, n4: inhibits MEK1/2), pathway disruption (n5: downregulates MAPK signaling), protein modification (n6: reduces ERK phosphorylation, n7: ERK mislocalization), transcriptional changes (n8-n13: regulates expression of MYC, CCND1, CDK4/6, metabolic genes, apoptotic genes), complex formation (n11: Rb-E2F1 complex formation), and finally phenotypic outcomes (n19: induces G1/S arrest and apoptosis). The <dag> tag encodes the edge list (n1, n3), (n2, n4), (n3, n5), (n4, n5), (n5, n6), (n6, n7), (n7, n8), (n7, n9), (n9, n10), (n10, n11), (n11, n12), (n7, n13), (n5, n14), (n7, n15), (n7, n16), (n17, n18), (n12, n19), (n13, n19), (n18, n19), defining exactly which actions depend on which prior actions. This is not a suggestion or a narrative—it is a formal graph that can be traversed, verified node by node, and used as structured input to downstream models.

The Action Space: A Constrained Vocabulary for Biological Reasoning

Section 2.2 and Figure 3 define the action space $\mathcal{A}$—the finite set of permissible reasoning actions from which all DAG nodes must be drawn. This is the single most important design choice in the paper because it determines what can be expressed, what can be verified, and what cannot be claimed.

The paper defines 20 action primitives organized into 8 categories (the paper text mentions 7 categories in Section 2.2 but Appendix A expands to 8: system initialization, metabolic, regulation, functional, interaction, phenotype, proteostasis, and descriptive). Each primitive has a specific argument schema—a fixed signature that specifies required and optional arguments. Here is the complete breakdown from Appendix A, Table Appx.1:

System Initialization (1 primitive):

• set_context(cell_type, genotype, disease, prior_perturbation, extras): Defines the biological background before the perturbation is applied, including the cell model, genotype, disease state, and any prior treatments. This is always the first node in any DAG because it establishes the cellular environment in which all subsequent actions occur.

Metabolic (1 primitive):

• converts_substrate(id, enzyme, substrate, product, {via, confidence}): Enzymatic conversion of one chemical entity into another, covering metabolic reactions or proteolytic processing.

Regulation (7 primitives, the largest category):

• Activity modulation:
  • modulates_molecule_activity(id, target, direction, {via, confidence}): Increases or decreases the catalytic or signaling activity of a single protein, enzyme, transporter, transcription factor, or RNA.
  • modulates_pathway_activity(id, pathway, direction, {via, confidence}): Increases or decreases the activity of a named pathway or biological process (e.g., MAPK, autophagy, EMT).
• Protein regulation:
  • modulates_complex(id, members, complex, direction, {stoichiometry, via, confidence}): Promotes assembly or causes disassembly of a multi-subunit complex, optionally including stoichiometry changes.
  • post_translational_modification(id, protein, mod_type, site, direction, {via, confidence}): Adds or removes a specific post-translational modification (phosphorylation, ubiquitination, acetylation, etc.) or proteolytic cleavage on a protein site.
• Transcriptional regulation:
  • regulates_expression(id, regulator, gene_list, direction, {mechanism, via, confidence}): Alters steady-state mRNA level or isoform ratio of one or more genes. The mechanism argument can tag cases like alternative splicing.
• Translational regulation:
  • regulates_translation(id, regulator, rna_id, direction, {mechanism, via, confidence}): Post-transcriptional control at the ribosome level (e.g., eIF inhibition, uORF usage, IRES activation).
• Epigenetic regulation:
  • chromatin_modification(id, mark, locus, direction, {via, confidence}): Adds or removes histone or DNA marks at a defined genomic locus, affecting chromatin accessibility.

Functional (5 primitives):

• Functional perturbation:
  • gain_of_function(id, variant_id, protein, {via, confidence}): Genetic variant or edit that increases the normal activity of the specified protein.
  • loss_of_function(id, variant_id, protein, {via, confidence}): Genetic variant, knock-down, or knock-out that reduces or abolishes activity of the specified protein.
• Functional association:
  • similar_to(id, entity_a, entity_b, evidence_type, {confidence, via}): Non-causal functional similarity (transcriptomic, phenotypic, structural) between two entities.
  • correlates_with(id, entity_a, entity_b, evidence_type, {confidence}): Statistical association without established causality (GWAS hit, literature co-mention, co-expression).
  • participates_in(id, entity, ontology_id, {evidence_type, confidence}): Links an entity to a GO term, pathway, or compartment (background annotation).

Interaction (2 primitives):

• binds_to(id, actor, target, {affinity, unit, residues_actor, residues_target, via, confidence}): Direct physical binding of two biomolecules such as drug-target, protein-protein, or ligand-receptor pairs.
• cell_cell_interaction(id, sender, receiver, ligand, receptor, outcome, {via, confidence}): Ligand-receptor signaling from one cell type to another plus the immediate downstream outcome.

Phenotype (2 primitives):

• induces_phenotype(id, source, phenotype, {via, confidence, from_state, to_state}): Creates or worsens a measurable phenotype or cell-state transition. Optional from_state and to_state capture events like EMT or senescence.
• alleviates_phenotype(id, actor, phenotype, {via, confidence, from_state, to_state}): Reverts or mitigates an abnormal phenotype back toward normal (rescue).

Proteostasis (2 primitives):

• localizes_to(id, entity, from_loc, to_loc, {mechanism, via, confidence}): Moves a molecule between compartments. Use to="extracellular" for secretion or from="extracellular" for uptake.
• degrades_or_stabilizes(id, regulator, target, direction, {via, confidence}): Changes protein abundance by altering half-life (ubiquitin-proteasome degradation, PROTAC, chaperone rescue).

Why this particular action space? The design reflects several deliberate choices. First, the primitives span the full causal chain from molecular interaction to phenotypic outcome—this ensures that a complete DAG can trace a perturbation all the way from initial target binding through signaling cascades to observable cellular effects. Second, each primitive has a fixed argument schema with specific, named parameters rather than free-text descriptions—this is what enables automatic verification because a verifier knows exactly which fields to check (e.g., a DTI verifier looks at the actor and target fields of binds_to, not at an unstructured sentence). Third, the primitives are organized hierarchically (categories and sub-categories) which provides the LLM with a structured vocabulary that mirrors how biologists think about cellular mechanisms—activity modulation is separate from transcriptional regulation, which is separate from phenotypic outcomes. Fourth, many arguments are mapped to biological ontologies (compounds, proteins, affinity scores, etc.), which means verifiers can resolve entities to standardized identifiers and cross-reference them against databases.

The paper notes that among these 20 primitives, the ones with implemented verifiers are highlighted in purple in Figure 3. In the current implementation, only a subset have verifiers (DTI for binds_to, DE for regulates_expression, LOC for localizes_to, PHENO for induces_phenotype and alleviates_phenotype), but the framework is designed to be extensible: as reliable computational tools become available for additional action types, new verifiers can be plugged in without modifying the overall architecture.

A concrete example from Figure 2a illustrates how an action is parameterized:

binds_to(id="n1", actor="Binimetinib", target="MAP2K1", affinity="12 nM", via="allosteric site binding")

Here, id="n1" is the node identifier for DAG construction, actor and target specify the interacting entities (drawn from the input perturbation and retrieved knowledge), affinity provides a quantitative binding measurement (12 nM, the IC50 value retrieved from knowledge bases), and via explains the mechanism of binding. This structured format means a DTI verifier can programmatically extract (actor="Binimetinib", target="MAP2K1"), feed these to Boltz-2, and receive a binding probability score that either confirms or refutes the claim—something impossible with unstructured text like "Binimetinib potently inhibits MEK1."

Report Generator: Grounding the LLM in Retrieved Facts

Section 3.1 details the report generator, which is the first stage of the two-stage pipeline and is responsible for resolving what the paper identifies as the primary failure mode of LLM biological reasoning: factual hallucination due to reliance on parametric knowledge.

The report generator operates in a strict three-step sequence:

Step 1: Biomedical Named Entity Recognition (NER). The system first extracts relevant biomedical entities from the input perturbation and cellular context using HunFlair2 (Sänger et al., 2024), a biomedical NER tool. The extracted entity types include chemical compounds, genes, diseases, and other biomedical concepts. This NER step serves a specific purpose: it converts the complex natural-language description of a perturbation into a set of searchable entity identifiers, enabling "entity-based search instead of relying on complex natural language queries" (Section 3.1). For the Binimetinib-C32 example from Figure 2a, the NER would extract entities like "Binimetinib" (Compound), "MAP2K1" (Gene), "MAP2K2" (Gene), and "C32" (Cell type), as shown in Figure 1's "Extracted entities" box.

Why NER rather than direct natural-language retrieval? The paper's rationale is that biomedical knowledge bases are indexed by standardized entity identifiers, not by free-text descriptions. If you query "How does a MEK inhibitor affect BRAF-mutant melanoma cells?" against StarkPrimeKG, you'll get poor results because the knowledge graph stores relations between specific entities (e.g., Binimetinib -[targets]-> MAP2K1), not between conceptual categories. The NER step bridges this gap by grounding the vague input description in specific entity names that can be matched against database indices.

Step 2: Multi-Source Knowledge Retrieval. The extracted entities are then used to query four external knowledge bases, each serving a complementary role:

• StarkPrimeKG (Wu et al., 2024b): A biomedical knowledge graph. For each extracted entity and its synonyms, the system searches for a matching node in the knowledge graph. If no exact match is found (e.g., due to name variations), the system identifies the most similar node based on cosine similarity of PubMedBERT embeddings (Gu et al., 2021). Once the node is located, the system aggregates 1-hop neighbor nodes into a textual context, providing relational information. For example, querying "MAP2K1" retrieves protein-protein interactions (e.g., MAP2K1 interacts with EGR1, MYC, GSK3B), drug targets (e.g., Selumetinib, Cobimetinib, Trametinib also target MAP2K1), disease associations (e.g., Noonan syndrome, melanoma), cellular component annotations (e.g., nucleus, cytoplasm, Golgi apparatus), molecular functions (e.g., MAP kinase kinase activity, ATP binding), and pathway memberships (e.g., MAPK3/ERK1 activation). The 1-hop neighborhood is crucial because it captures not just the entity itself but its biological context—what it interacts with, what diseases it's associated with, where in the cell it localizes.

• Harmonizome (Diamant et al., 2024): A gene-centric database aggregating information from diverse multi-omics resources. The system queries gene entities to enrich gene-specific information, targeting two categories: (1) the genes encoding protein targets of compound perturbations (e.g., for Binimetinib, the targets MAP2K1 and MAP2K2), and (2) the genes targeted by genetic perturbations. For each queried gene, Harmonizome returns gene names, symbols, synonyms, protein descriptions, functional summaries, and—importantly for context-specific reasoning—cell-type-specific gene associations. In the C32 example from Appendix B, Harmonizome returns a list of genes related to C32 melanoma cells (GANAB, CNOT11, HNRNPR, MAP2K1, CASP3, HABP2, etc.), each with functional annotations. This cell-type-specific information is what enables the system to reason about context-dependent effects—Binimetinib might affect different genes in C32 cells versus HepG2 cells because the baseline gene expression programs differ.

• PubMed: The biomedical literature database. The system queries using the extracted entities and retrieves relevant papers, "prioritizing papers whose abstracts demonstrate the highest similarity to the input entities" (Section 3.1). This captures literature-grounded evidence that may not be in curated databases—recent findings, context-specific experimental results, and nuanced mechanistic details. For the Binimetinib example, Appendix B shows retrieved papers including "Binimetinib, a novel MEK1/2 inhibitor, exerts anti-leukemic effects under inactive status of PI3Kinase/Akt pathway" and "Phase Ib Study of Combination Therapy with MEK Inhibitor Binimetinib and Phosphatidylinositol 3-Kinase Inhibitor Buparlisib in Patients with Advanced Solid Tumors with RAS/RAF Alterations." These papers provide evidence about compensatory pathway activation (PI3K/AKT upregulation upon MEK inhibition) that might not be captured in static knowledge graphs.

• Wikipedia: The system queries each entity to retrieve the best-matching Wikipedia documents. Wikipedia provides broad background knowledge about drugs, genes, and diseases—information like mechanism of action descriptions, clinical trial results, and functional summaries that complement the more structured database entries. For Binimetinib, Wikipedia returns the drug's mechanism of action ("uncompetitive with ATP, binds to and inhibits the activity of MEK1/2 kinase"), its development history, and its approved indications.

Why four sources rather than one comprehensive knowledge base? The paper's ablation study (Appendix D.1, Table Appx.3) demonstrates that no single source approaches the performance of the full combination. Using PubMed alone achieves a DTI score of 0.432 and DE score of 0.435; StarkPrimeKG alone achieves 0.382 and 0.449; Harmonizome alone achieves 0.399 and 0.457; Wikipedia alone achieves 0.469 and 0.485. The full combination achieves DTI score of 0.766 and DE score of 0.487. Each database contributes complementary information: StarkPrimeKG provides relational context (what interacts with what), Harmonizome provides gene-level detail (functional annotations and cell-type associations), PubMed provides literature-grounded evidence (experimental findings and mechanistic studies), and Wikipedia provides broad background (drug mechanisms, disease context). The combination ensures that the report generator has access to structured relationships, functional annotations, experimental evidence, and general knowledge simultaneously.

Step 3: Report Generation. All retrieved information is aggregated and fed to Claude 4 (Anthropic, 2025) with a detailed prompt (provided in Appendix C) that instructs the model to generate a comprehensive mechanistic report. The prompt specifies six requirements:

1. Describe the perturbation in detail (type, primary targets, binding affinities, mechanism of action).
2. Map the full causal chain step by step, explicitly marking each step as causal (direct mechanistic or experimental evidence) or correlative (statistical or inferred association).
3. Prioritize measurable end nodes (transcriptomic changes, phenomic signatures) over abstract higher-order phenotypes.
4. Include associative evidence and ontological evidence when available.
5. Summarize the final phenotypic outcome.
6. Provide quantitative details (IC50, Kd, phosphorylation sites, gene modulation directions, morphological metrics) whenever available.

The resulting report (example in Appendix B.2) is a detailed natural-language document structured into sections: Perturbation Description, Cellular Context, Step-by-Step Mechanistic Analysis (with each step labeled as CAUSAL or CORRELATIVE), Measurable Biological Endpoints (transcriptomic and phenomic signatures), and Final Phenotypic Outcome. For Binimetinib in C32 cells, the report spans approximately 3,000 words covering pharmacological properties, the C32 mutational landscape (BRAF V600E, CDKN2A loss, PTEN loss, TERT activation), and seven mechanistic steps from target engagement through MAPK pathway disruption, transcriptional reprogramming, cell cycle arrest, metabolic reprogramming, compensatory PI3K/AKT activation, apoptotic priming, and resistance mechanisms.

Why this two-stage design? The paper's ablation (Appendix D.3, Table Appx.5) directly tests the alternative: a one-step pipeline that generates structured explanations directly from the perturbation input and retrieved information without an intermediate report. The two-step pipeline nearly doubles both DTI score (0.766 vs. 0.329) and DE score (0.487 vs. 0.267) compared to one-step generation, while also achieving higher verifiability (0.601 vs. 0.534). The paper's interpretation is that "decoupling knowledge synthesis from structured reasoning generation is critical"—the intermediate report serves as a knowledge bottleneck that forces the system to first aggregate and synthesize all relevant facts before attempting to structure them. Without this step, the LLM must simultaneously retrieve knowledge from its context and format it into the constrained action space, which leads to lower-quality outputs because the two tasks compete for the model's attention and the temptation to hallucinate missing facts is stronger when the model is also struggling with format constraints.

Explanation Constructor: From Free-Text Report to Structured DAG

Section 3.2 describes the explanation constructor, which is the second stage of the pipeline. It takes the knowledge-grounded report as input and transforms it into the structured reasoning format defined in Section 2.1. The model used is again Claude 4 (Anthropic, 2025), and the prompt is provided in Appendix C.

What the prompt enforces. The explanation constructor prompt is substantially more detailed than the report generator prompt, specifying four required outputs:

1. Private Reasoning (wrapped in <think> tags): Step-by-step biological reasoning where the model "proceeds as if discovering the answer for the first time," simulating binding events, cascades, interactions, and regulatory outcomes while marking each connection as causal or correlative. This is the model's internal reasoning trace, analogous to chain-of-thought, and is not part of the structured output but guides the model's construction.

2. Mechanism-of-Action Summary (wrapped in <answer> tags): A one-paragraph summary that clearly describes the mechanism of action, distinguishes mechanistic from correlative findings, and concludes with the predicted phenotypic outcome. This serves as a human-readable abstract of the structured explanation.

3. Structured Explanation (wrapped in <explain> tags): The core output—a sequence of action primitives, one per line, each following the specific argument schema from Section 2.2. Key constraints enforced by the prompt:

   • Must begin with set_context(...) to define the biological background.
   • Each line must include a unique id (e.g., id="n3") for DAG referencing.
   • The final step must be a measurable biological output: induces_phenotype(...), alleviates_phenotype(...), regulates_expression(...), or regulates_translation(...).
   • Must not end on upstream mechanistic nodes—"always ground the explanation in a testable, measurable biological output."
   • The via argument should reference upstream mechanisms for causal steps or explain the evidence basis for correlative ones.

4. Causal DAG (wrapped in <dag> tags): An edge list referencing the id values from the <explain> block, with each edge specifying a relation type of either "causal" (direct mechanistic consequence) or "correlative" (statistical or inferred association). For example: edge("n1", "n2", relation="causal").

Why these four outputs? The design reflects a deliberate separation of concerns. The private reasoning ( thinking) gives the model space for open-ended deliberation without format constraints—it can consider alternative mechanistic paths, weigh evidence, and identify gaps before committing to a structured output. The summary (<answer>) provides a human-readable checkpoint that can be quickly evaluated for overall plausibility. The structured explanation (<explain>) is the machine-verifiable artifact that downstream verifiers and training pipelines consume. The DAG (<dag>) captures the dependency structure explicitly, enabling graph-based analyses (e.g., identifying which nodes are upstream of a phenotype, checking for cycles, computing path lengths).

What the example reveals about the framework's capabilities. The full example in Figure 2a (reproduced in Appendix B.2 context) contains 19 nodes spanning the entire causal chain: set_context (n0, implicitly the first action), binds_to for two targets (n1, n2), modulates_molecule_activity (n3, n4), modulates_pathway_activity (n5), post_translational_modification (n6), localizes_to (n7), regulates_expression for multiple gene sets (n8, n9, n12, n13, n15, n16), post_translational_modification of RB1 (n10), modulates_complex formation (n11), modulates_pathway_activity for compensatory PI3K/AKT signaling (n14), modulates_pathway_activity for mitochondrial apoptosis (n17), post_translational_modification of caspase-3 (n18), and finally induces_phenotype (n19). The edge list in the <dag> tag defines exactly which actions depend on which: for instance, (n7, n15) and (n7, n16) indicate that ERK mislocalization to the cytoplasm is the mechanistic cause of both pro-apoptotic gene upregulation and anti-apoptotic gene downregulation, while (n12, n19), (n13, n19), and (n18, n19) indicate that the phenotype emerges from three concurrent pathways: E2F1-mediated cell cycle arrest, metabolic gene downregulation, and caspase-dependent apoptosis.

This level of detail—specifying not just what happens but what depends on what—is what distinguishes the structured approach from a free-text narrative. A biologist reading the explanation can trace exactly which molecular events are claimed to cause which downstream effects, and a verifier can programmatically check individual claims (e.g., "does Binimetinib actually bind MAP2K1?" by querying the DTI verifier with (actor="Binimetinib", target="MAP2K1")).

Backbone model choice. The paper's ablation (Appendix D.2, Table Appx.4) compares Claude against GPT-4.1 and Gemini-2.5-Flash as the backbone LLM for the explanation constructor. Claude achieves perfect trace validity (1.000, meaning all generated traces are syntactically correct and structurally valid), while GPT-4.1 achieves 0.970 and Gemini-2.5-Flash achieves only 0.060. On DTI score, Claude substantially outperforms both alternatives (0.766 vs. 0.399 for GPT-4.1 and 0.338 for Gemini-2.5-Flash). GPT-4.1 achieves a higher DE score (0.527 vs. 0.487), but "at the cost of lower verifiability" (0.299 vs. 0.601), meaning a large fraction of its outputs cannot be reliably evaluated because the generated arguments don't map to valid biomedical entities. The authors conclude that Claude "balances format adherence with biological accuracy," making it the appropriate backbone choice.

The full pipeline is not a single LLM call. It's worth emphasizing that VCR-Agent makes multiple LLM calls in sequence: HunFlair2 for NER (non-LLM), embedding-based similarity search for knowledge graph matching (non-LLM), then Claude 4 for report generation (first LLM call), then Claude 4 again for explanation construction (second LLM call). The separation is not arbitrary—it reflects the paper's core thesis that knowledge synthesis and structured reasoning are distinct cognitive tasks that benefit from being performed separately, with the intermediate report serving as a verifiable checkpoint where factual errors can be caught before they propagate into the structured output.

Verifier-Based Filtering: From Generated Claims to Verified Facts

Section 4 introduces the verifier-based filtering pipeline, which is the quality control mechanism that transforms raw generated explanations into the verified VC-Traces dataset. This is the component that addresses the "verification impossibility" problem identified in the introduction—biology has no compiler, but specialized verifiers can check specific types of claims against reference data.

The filtering operates at two levels (Section 4 introduction): per-action verification and trace-level filtering.

Per-action verification: Each individual action within a generated explanation trace is evaluated by a specialized verifier corresponding to its action type. The paper implements four verifiers, with two serving as the primary filters:

1. DTI Verifier (for binds_to actions): This verifier predicts the physical plausibility of a binding between a given actor (drug) and target (protein). It leverages Boltz-2 (Passaro et al., 2025), a state-of-the-art binding affinity prediction model, to model the protein-ligand interaction and characterize the binding interface. The output is a continuous binding probability score. For example, for the claim binds_to(actor="Binimetinib", target="MAP2K1"), the DTI verifier takes the drug and protein as input, runs Boltz-2 to predict whether binding is physically plausible, and returns a score—in the Figure 5 example, a score of 7.35, which exceeds the threshold of 6 and is therefore kept.

2. DE Verifier (for regulates_expression actions): This verifier validates whether a perturbation actually up- or down-regulates the claimed target gene in the claimed direction. It queries ground-truth differential expression datasets—specifically the Tahoe-100M atlas (Zhang et al., 2025)—to check whether the predicted target gene is significantly regulated in the claimed direction. For example, for the claim regulates_expression(regulator="ERK1/2", gene_list=["MYC","CCND1","FOS","JUN"], direction="down"), the DE verifier queries Tahoe-100M to see whether Binimetinib treatment actually downregulates these genes in C32 cells. In the Figure 5 example, the DE verifier confirms that MYC and CCND1 are indeed downregulated, but removes JUN from the gene list because the experimental data does not support that claim.

3. LOC Verifier (for localizes_to actions): Described in Appendix E, this verifier cross-references claimed subcellular localizations against curated annotations from UniProt (Ahmad et al., 2025) and the Human Protein Atlas (Thul & Lindskog, 2018). For each localizes_to claim, it checks whether the specified entity is annotated to the claimed from_loc and to_loc compartments in these databases.

4. PHENO Verifier (for induces_phenotype and alleviates_phenotype actions): Also described in Appendix E, this verifier queries the Cellular Phenotype Database (Kirsanova et al., 2015) to check whether the claimed phenotype is consistent with documented phenotypic associations for the perturbation or its downstream targets.

Why these particular verifiers? The choice is pragmatic: these cover the most frequently occurring action types. The paper notes (Section 4.2) that DTI (binds_to) and DE (regulates_expression) actions "jointly appear in 91.5% of all explanation traces in VC-Traces," meaning these two verifiers alone provide broad filtering coverage. The LOC and PHENO verifiers are described but less emphasized, likely because they cover a smaller fraction of actions. The framework is explicitly designed to be extensible (Section 5 impact statement): "as reliable computational tools become available for additional action types, new verifiers can be incorporated into the pipeline without modifying the overall architecture."

Trace-level filtering with thresholds: Once individual actions receive verification scores, the system applies two filtering rules:

1. DTI validity constraint: Any trace containing a binds_to action with a DTI confidence score below a pre-defined threshold $\tau$ is discarded entirely. The paper does not explicitly state the value of $\tau$, but the example in Figure 5 shows a score of 7.35 being kept against a threshold of 6. The rationale is that if the fundamental drug-target binding claim is unsupported, the entire downstream reasoning chain is built on a false premise and should be rejected wholesale.

2. DE consistency pruning: The DE verifier eliminates gene arguments that correspond to incorrectly identified or directionally mismatched gene expression changes. Unlike DTI filtering, which discards entire traces, DE filtering operates at the argument level—individual genes within a gene_list are removed if they don't match experimental data. The paper emphasizes (Section 4.2): "our filtering is designed to prevent false positives, i.e., it only removes claims that directly contradict established biological evidence, leaving others unchanged." This is a conservative strategy: the verifiers catch clear errors but don't penalize claims for which evidence is simply absent, avoiding over-filtering that might discard correct but sparsely annotated facts.

Quantitative impact of filtering: The paper reports (Section 5.1) that during the construction of VC-Traces, the verification pipeline "successfully excluded 28.2% of faulty DTI claims and refined 87.3% of DE actions to eliminate hallucinations." These numbers are substantial—more than a quarter of drug-target binding claims were factually wrong, and nearly 90% of gene regulation claims contained at least one hallucinated gene. This validates the paper's central concern: LLM-generated biological explanations contain high rates of factual errors, and without verification, using them as training data would propagate these errors downstream.

Verifier design avoids test-time label leakage. A crucial technical detail for the downstream TahoeQA experiments (Section 5.2): the paper explicitly verifies that the DE verifier does not introduce data leakage between training and test sets. The overlap between test genes and those appearing in regulates_expression actions during training is "minimal (0.2% for DE, 0.1% for DOC [Direction of Change])," meaning the verifier is not accidentally revealing test-set labels to the model during training. This is a subtle but important point—if the DE verifier's filtering decisions were correlated with test-set labels, the downstream improvements might be attributable to label leakage rather than genuine reasoning capability.

LLM-judge evaluation cannot fully capture verifier benefits. The ablation study in Appendix D.5 (Table Appx.7) evaluates filtered vs. unfiltered traces using LLM-as-judge metrics (scientific accuracy, logical consistency, mechanistic clarity). The improvements are modest: scientific accuracy increases from 0.641 to 0.645, logical consistency from 0.720 to 0.724, mechanistic clarity unchanged at 0.725. The paper attributes this to the LLM-judge's "lack of the specialized regulatory knowledge such as those from Boltz-2 and Tahoe-100M, which are used by our verifiers." In other words, the LLM-judge cannot detect the specific factual errors that the specialized verifiers catch—it rates a trace with a hallucinated binding claim similarly to a trace with a verified binding claim because it doesn't have access to binding affinity predictions or differential expression databases. This finding highlights both the value of the specialized verifiers (they catch errors invisible to general-purpose evaluators) and the limitation of current evaluation methods (LLM-judges are necessary but not sufficient for scientific fact-checking).

Human evaluation validates verifier alignment. Appendix D.6 reports a human expert evaluation on a randomized subset of 10 explanation traces, with domain experts in molecular biology and pharmacology scoring traces on the same three criteria. The Pearson correlation between human expert ratings and LLM-judge scores averages $r = 0.72$ across criteria (scientific accuracy: 0.72, logical consistency: 0.69, mechanistic clarity: 0.76), confirming that the LLM-judge serves as a "reliable proxy for domain expert assessment." Additionally, traces judged as plausible by experts cluster in the upper-right region of the score space while implausible traces receive lower scores from both humans and LLM-judges (visualized in Figure Appx.1), providing confidence that the evaluation framework captures biologically meaningful quality distinctions.

Downstream Training: Using Verified Traces as Supervision Signals

Section 5.2 describes how the verified structured explanations are used for the TahoeQA downstream task, which is designed to predict transcriptional responses to chemical compounds. This section is essential for understanding the paper's central claim that "grounding high-dimensional transcriptomic data in biological reasoning improves performance in sparse-data and out-of-distribution scenarios."

Task formulation. The TahoeQA task is inspired by the PerturbQA benchmark (Wu et al., 2024a) and is a two-fold binary classification:

1. Differential Expression (DE) task: Given a perturbation, cellular context, and target gene, predict whether the perturbation causes the target gene to be differentially expressed (yes/no). This is inherently imbalanced—for each perturbation-context pair, there are 50 positive examples (top 25 up-regulated, top 25 down-regulated) and 100 negative examples (random non-regulated genes).

2. Direction of Change (DOC) task: Given that a gene is differentially expressed, predict whether its expression increases or decreases (up/down).

Both tasks share the same input structure: a perturbation (chemical compound with SMILES, name, known targets, mechanism of action) and a cellular context (cell type, disease model), with the target gene as the prediction target.

Dataset construction. The labeled dataset is derived from Tahoe-100M using the following procedure (Section 5.2):

1. For each perturbation-context pair, perform differential expression analysis by fitting a negative binomial-based general linear model to pseudo-bulked counts and running Wald's test (Wald, 1943) to determine if the log-fold change differs significantly from 0, as implemented in DESeq2 (Love et al., 2014).

2. Define differentially expressed genes following the Benjamini-Hochberg procedure (Benjamini & Hochberg, 1995) with adjusted $p < 0.05$.

3. Select the top 25 up-regulated, top 25 down-regulated (based on magnitude of log2 fold change), and 100 random non-regulated genes for each pair.

4. Select five cell types following the few-shot test split of Tahoe-X1 (Gandhi et al., 2025): C32, HOP62, HepG2/C3A, Hs 766T, PANC-1.

5. Split by perturbation to ensure no overlap between training and test perturbations, with approximately 1,000 test examples randomly selected for evaluation. This perturbation-split design is critical: it means the test set contains compounds the model has never seen during training, testing genuine generalization rather than memorization of compound-specific patterns.

Two supervised fine-tuning strategies. The paper fine-tunes Qwen3-4B-Instruct-2507 (Yang et al., 2025) in two configurations, both using SFT only (no reinforcement learning, which the paper notes "often suffers from sparse reward signals" in pure classification settings and leaves for future work):

SFT-Prompt (context-augmented prediction): The model is trained to predict the answer label (DE or DOC) given three inputs: the perturbation description, the cellular context, and the verified structured explanation as additional context. Concretely, during training, the model sees something like:

[Perturbation: Binimetinib, MEK1/2 inhibitor, ...]
[Context: C32 melanoma, BRAF V600E, ...]
[Structured Explanation: set_context(...) binds_to(...) modulates_pathway_activity(...) regulates_expression(...) ...]
Question: Is MYC differentially expressed in response to this perturbation?
Answer: Yes, downregulated

This is the simpler approach—the model doesn't need to generate reasoning, only to condition on it. It tests whether having access to mechanistic reasoning improves prediction.

SFT-Generate (generative reasoning): The model is trained to generate the structured explanation followed by the answer, given only the perturbation and cellular context. During training, the model learns to produce:

[Perturbation: Binimetinib, ...]
[Context: C32 melanoma, ...]
<explain>
set_context(...)
binds_to(id="n1", ...)
...
</explain>
<dag>...</dag>
Answer: MYC is differentially expressed (downregulated)

This is the more ambitious approach—the model must autonomously construct the reasoning chain before predicting. It tests whether the process of generating mechanistic reasoning improves the model's underlying understanding of perturbation biology.

Training hyperparameters (Appendix C.3):

• Learning rate: $2 \times 10^{-4}$
• Linear scheduler with warmup ratio of 0.05
• Weight decay: 0.01
• Gradient accumulation steps: 4
• LoRA with rank $r = 64$
• Random seed: 11

Inference hyperparameters:

• Temperature: 0.2
• Nucleus sampling (top-p): 0.8
• Top-k: 20

These generation parameters are relatively conservative (low temperature, moderate top-p and top-k), which is appropriate for tasks where factual precision matters more than creative diversity.

Why LoRA? The paper does not explicitly justify this choice, but LoRA (Low-Rank Adaptation) is standard for parameter-efficient fine-tuning when the base model is large and full fine-tuning would be computationally expensive. The rank of 64 is relatively high (typical LoRA ranks are 8-32), suggesting the authors wanted sufficient capacity for the model to learn the structured explanation format and the biological reasoning patterns.

Why Qwen3-4B? The paper does not justify this specific model choice, but 4 billion parameters is relatively small by modern standards, which is deliberate—the experiments aim to show that structured reasoning provides a strong supervision signal even for modestly-sized models, making the approach accessible without requiring massive compute. The downstream performance improvements are therefore attributable to the quality of the training data (verified structured explanations) rather than raw model scale.

Metrics and imbalance handling. Both tasks use F1-score as the primary metric "due to the label imbalance problem (i.e., 50 positive and 100 negative labels per perturbation-context pair) in the DE task" (Section 5.2). F1-score—the harmonic mean of precision and recall—is appropriate here because accuracy would be misleading: a model that always predicts "not differentially expressed" would achieve 67% accuracy (100/150 correct) but an F1 of 0, correctly reflecting its uselessness for the practical task of identifying which genes actually change.

No test-time label leakage from DE verifier. This is a critical validation step. The paper confirms that "the overlap between test genes and those appearing in regulates_expression actions is minimal (0.2% for DE, 0.1% for DOC)." To understand why this matters: if the DE verifier had filtered regulates_expression claims during VC-Traces construction based on Tahoe-100M data, and if the test-set genes appeared in those training claims, then a model trained on VC-Traces might learn the test labels directly from the training data rather than from generalizable biological reasoning. The 0.2% overlap is negligible, meaning the downstream improvements cannot be attributed to this form of leakage.

Why exclude reinforcement learning? The paper explicitly notes: "we focus on SFT and exclude reinforcement learning (RL) from this study, as reasoning in pure classification settings often suffers from sparse reward signals (Wang et al., 2024; He et al., 2025; Sprague et al., 2025) and leave RL-based optimization for future work." This is a candid acknowledgment of a practical limitation: RL for reasoning requires the model to generate long chains of actions (the structured explanation) before receiving a binary reward (correct/incorrect prediction), creating a sparse reward problem where credit assignment is extremely difficult. In mathematical reasoning, intermediate steps can sometimes be verified (e.g., checking whether an intermediate equation is correct), providing denser rewards, but in biological reasoning, intermediate steps like "ERK1/2 localizes to cytoplasm" are hard to verify independently, making RL optimization particularly challenging. The SFT-only approach sidesteps this problem by providing dense supervision at every token of the structured explanation.

Complete Pipeline Integration: What Actually Happens End-to-End

To make the full system concrete, here is the end-to-end flow for a single perturbation-context pair:

1. Input: $x = (p = \text{"Binimetinib" with SMILES CN1C=NC2=...}, c = \text{"C32 melanoma cell line"})$

2. NER (HunFlair2): Extracts entities → {"Binimetinib": Compound, "MAP2K1": Gene, "MAP2K2": Gene, "C32": Cell type}

3. Retrieval:

   • StarkPrimeKG: Query "Binimetinib" → drug node with targets MAP2K1, MAP2K2; query "MAP2K1" → protein node with interactions (MYC, GSK3B, ...), pathways (MAPK signaling), diseases (melanoma)
   • Harmonizome: Query "MAP2K1", "MAP2K2" → gene summaries, C32-associated genes
   • PubMed: Query "Binimetinib MAP2K1 C32" → top relevant papers (binimetinib in melanoma, combination therapies)
   • Wikipedia: Query "Binimetinib", "MAP2K1", "MAP2K2" → background articles

4. Report Generation (Claude 4): Synthesizes all retrieved information into a 3,000-word mechanistic report covering pharmacological properties, C32 mutational landscape, step-by-step causal chain from target engagement to phenotype, measurable endpoints, and final outcome.

5. Explanation Construction (Claude 4): Transforms report into structured format → 19 nodes (1 set_context, 2 binds_to, 2 modulates_molecule_activity, 1 modulates_pathway_activity, 2 post_translational_modification, 1 localizes_to, 8 regulates_expression, 1 modulates_complex, 1 induces_phenotype) with 18 edges forming a DAG.

6. Verification:

   • DTI verifier checks each binds_to claim → Binimetinib-MAP2K1: score 7.35 (keep), Binimetinib-MAP2K2: score passes (keep)
   • DE verifier checks each regulates_expression claim → MYC: downregulated ✓, CCND1: downregulated ✓, FOS: downregulated ✓, JUN: not significantly downregulated ✗ (removed from gene_list)
   • If any binds_to claim had failed, the entire trace would be discarded

7. Output: A filtered, verified structured explanation trace suitable for inclusion in VC-Traces or use as training data for downstream models.

The entire pipeline requires multiple API calls (HunFlair2, embedding similarity search, two Claude 4 calls, verifier computations) and accesses four external databases and one binding affinity predictor. The computational cost is not explicitly quantified in the paper, but the two-stage design with intermediate report generation means each perturbation-context pair requires two full LLM generations, which is a non-trivial expense at scale (18,950 pairs for the full VC-Traces dataset).

4. Key Insights and Innovations

Innovation 1: Structured Mechanistic Reasoning as a Verifiability-Enabling Formalism

The paper's most fundamental intellectual move is redefining what it means for an LLM to "reason" about biology. Before this work, the dominant paradigm—inherited from the mathematical and code reasoning literature—treated reasoning as free-form natural language chain-of-thought. Models like those from Istrate et al. (2025), Phillips et al. (2025), and Wu et al. (2025) all produce unstructured textual rationales: "the drug inhibits MEK, which reduces ERK phosphorylation, leading to cell cycle arrest." These narratives are flexible, expressive, and completely unverifiable by machine.

The conceptual shift VCR-Agent makes is treating biological reasoning not as text generation but as graph construction from a constrained vocabulary. By formalizing reasoning as a Directed Acyclic Graph where each node is one of 20 predefined action primitives with fixed argument schemas, and each edge represents a mechanistic dependency, the paper transforms reasoning from an interpretation problem into a verification problem. This is not merely an engineering convenience—it fundamentally changes what can be done with the output. A free-text explanation can be read by a human; a structured DAG can be programmatically checked against databases, scored by specialized verifiers, filtered for factual errors, and used as structured input to downstream models.

Prior work recognized the verification problem but addressed it indirectly. Istrate et al. (2025) used "soft verifiers"—learned models that score the plausibility of entire reasoning traces—but these provide holistic scores without identifying which specific claims are false. The mathematical reasoning literature (Cobbe et al., 2021; Lightman et al., 2023) uses outcome and process reward models, but these work because math has deterministic correctness; biology doesn't. VCR-Agent's insight is that by decomposing reasoning into discrete, typed actions—each with a known argument schema mapped to biological ontologies—you can build specialized per-action verifiers that check individual claims against ground-truth data, something impossible with unstructured text.

The significance of this shift extends beyond the paper's immediate results. It establishes a design principle for scientific reasoning systems: the output format should be chosen to maximize verifiability, not expressiveness. Free-form text maximizes expressiveness (you can say anything) but minimizes verifiability (you can check almost nothing automatically). The structured action space constrains expressiveness (you can only say things expressible as binds_to(actor, target, affinity, ...) or regulates_expression(regulator, gene_list, direction, ...)) but maximizes verifiability (each action type has a corresponding specialized verifier). The paper demonstrates that this tradeoff is worthwhile: the constrained format retains sufficient expressiveness to capture complex mechanistic chains (the Binimetinib example spans 19 nodes from binding to apoptosis), while enabling automated filtering that catches 28.2% of faulty DTI claims and refines 87.3% of DE actions.

This is a fundamental reframing, not an incremental improvement. The field previously asked: "How can we make LLMs generate better biological explanations?" VCR-Agent asks instead: "What form must an explanation take for us to know whether it's correct?" The answer—structured actions with specialized verifiers—is generalizable beyond biology to any scientific domain where verification can be decomposed along ontological lines.

Innovation 2: Knowledge Retrieval as a Separate, Verifiable Pre-Reasoning Stage

The paper's two-stage pipeline (report generator → explanation constructor) might appear to be a straightforward engineering choice, but it embodies a diagnostic insight about LLM failure modes that challenges how the field typically approaches knowledge-grounded generation.

The standard approach in retrieval-augmented generation (RAG) and tool-use systems is to interleave retrieval with generation: the model retrieves some facts, generates some text, retrieves more facts, generates more text, and so on. This is how systems like Wu et al. (2025) operate—the LLM queries a knowledge graph during reasoning, incorporating retrieved facts on-the-fly. The implicit assumption is that the LLM can effectively integrate retrieval and generation in a single cognitive pass.

VCR-Agent's ablation study (Appendix D.3, Table Appx.5) provides direct empirical evidence against this assumption in the biological domain. The one-step baseline—which generates structured explanations directly from retrieved information without an intermediate report—achieves DTI score of 0.329 and DE score of 0.267. The two-step pipeline nearly doubles these to 0.766 and 0.487. The magnitude of this gap (roughly 2× improvement in factual accuracy) is striking and suggests that knowledge synthesis and structured reasoning are competing for the same cognitive resources in the LLM. When forced to do both simultaneously, the model does both poorly; when separated, each can be optimized independently.

The conceptual innovation here is recognizing the intermediate report as a knowledge bottleneck—a verifiable checkpoint where factual errors can be detected before they propagate into structured reasoning. The report is human-readable, grounded in specific database queries, and can be inspected for hallucinations or missing information. If the report contains errors (e.g., incorrectly stating that Binimetinib targets BRAF rather than MEK), these errors will be systematically reflected in the structured DAG. But the separation means that improving the report generator (better retrieval, better summarization) directly improves downstream reasoning quality without requiring changes to the explanation constructor. This modularity is a design principle that the paper implicitly demonstrates rather than explicitly argues: by decoupling the stages, each component's failures become diagnosable and fixable independently.

Prior work in scientific reasoning with LLMs largely took one of two approaches: either rely entirely on parametric knowledge (Phillips et al., 2025; Istrate et al., 2025) or use interleaved retrieval during reasoning (Wu et al., 2025). The parametric approach suffers from hallucination because LLM knowledge about biology is incomplete and sometimes wrong. The interleaved approach suffers from cognitive overload because the model must simultaneously decide what to retrieve, integrate retrieved facts, and maintain coherent reasoning. The VCR-Agent two-stage design solves both problems: the report generator handles retrieval and synthesis without the distraction of format constraints; the explanation constructor handles formatting without the distraction of fact-checking.

This is an incremental but important contribution—incremental because two-stage pipelines exist in other contexts, but important because it provides controlled evidence (via the ablation) that the separation is causally responsible for the performance improvement, and because it identifies a specific cognitive bottleneck (simultaneous retrieval + structured generation) that likely applies to other scientific domains where the output format is constrained.

Innovation 3: Verifier Over-Optimization Avoided Through Conservative Filtering

The paper's verifier-based filtering pipeline (Section 4) might appear to be a straightforward quality control mechanism, but it embodies a subtle and practically crucial design philosophy: verifiers should be used to eliminate false claims, not to optimize for high scores. This distinction matters because it prevents a failure mode well-documented in the LLM alignment literature but largely unexamined in scientific reasoning: reward hacking through verifier over-optimization.

In the mathematical reasoning literature, the reference paper (Snell et al., 2024, analyzed in the example) demonstrates that process reward models (PRMs) can be over-optimized—beam search guided by PRM scores eventually produces solutions that score highly under the PRM but are actually incorrect, because the PRM has blind spots that aggressive optimization exploits. The same dynamic could occur in biological reasoning: if the DTI verifier's binding probability score were used to select the best binds_to claims or to rank entire traces for filtering, the LLM might learn to generate claims that score highly on the verifier without being biologically true—for instance, claiming implausible bindings that happen to have favorable physicochemical properties according to Boltz-2.

VCR-Agent's approach avoids this by design. The verifiers operate in a conservative, false-positive-prevention mode: they remove claims that directly contradict established evidence but leave unverified claims intact. The DTI verifier discards entire traces only when the binding score falls below a threshold; it does not rank traces by score or optimize for the highest-scoring claims. The DE verifier prunes individual gene arguments that are directionally wrong but does not penalize genes for which evidence is simply absent. As the paper states (Section 4.2): "our filtering is designed to prevent false positives, i.e., it only removes claims that directly contradict established biological evidence, leaving others unchanged."

This conservative philosophy has two important consequences. First, it means the verifiers cannot be gamed through optimization—there is no gradient signal pushing the LLM toward "high-scoring but wrong" claims because the verifiers are used only for rejection, not for ranking or selection. Second, it means the verified dataset is conservative but trustworthy: claims that survive filtering are those for which no contradictory evidence exists, not necessarily those for which positive evidence exists. This is appropriate for a scientific reasoning system where false claims (hallucinations) are more dangerous than missing claims (incomplete reasoning).

This design choice is a diagnostic contribution rather than a novel algorithm. It identifies a failure mode that the field should anticipate—verifier over-optimization in scientific domains where verifiers are imperfect—and demonstrates a simple structural solution: use verifiers as binary filters, not as optimization objectives. The contrast with the mathematical reasoning literature, where verifier-guided search can improve performance up to a point before degrading, highlights that the appropriate use of verifiers depends on domain characteristics. In biology, where verifiers are sparse (only 4 of 20 action types covered) and imperfect (Boltz-2 predicts binding plausibility, not certainty), conservative filtering is the only defensible strategy.

The evidence for this insight is partially explicit (the filtering design described in Section 4.2) and partially implicit (the paper does not attempt to use verifier scores for ranking or optimization, and the results in Appendix D.5 show that filtering produces only modest LLM-judge improvements, suggesting the verifiers are catching errors that would otherwise propagate undetected). The conceptual takeaway is that the purpose of verification in scientific reasoning is error elimination, not reward maximization—a principle that should guide future work on verifier design for scientific domains.

Innovation 4: Mechanistic Reasoning as an Inductive Bias for Generalization

The downstream TahoeQA results (Section 5.2, Figure 6) demonstrate that models trained with structured mechanistic explanations substantially outperform models trained only on input-output pairs, even though both see the same perturbations and labels. This is more than a performance improvement—it reveals something about what kind of supervision signal teaches models to generalize.

The standard SFT baseline (Qwen3-4B fine-tuned to predict DE directly from perturbation + context) achieves an average F1 of only 0.292 on the DE task, barely above the random baseline of 0.259. This is despite having access to the same perturbation descriptions and cellular contexts as the structured reasoning models. The SFT-Prompt model—which receives the verified structured explanation as additional context—more than doubles this to 0.435. The SFT-Generate model—which learns to produce the reasoning chain before predicting—achieves 0.388.

What explains this dramatic gap? The paper's interpretation (Section 5.2) is that "grounding high-dimensional transcriptomic data in biological reasoning improves performance in sparse-data and out-of-distribution scenarios." But the deeper insight is about the nature of the supervision signal. Direct label prediction teaches the model a mapping from perturbation descriptions to gene expression changes—a pattern that may work for seen compounds but fails on novel ones because the mapping is underdetermined (many mechanisms can produce the same expression change). Structured reasoning teaches the model the causal structure connecting perturbations to outcomes: Binimetinib inhibits MEK → reduces ERK phosphorylation → downregulates MYC, CCND1, etc. This causal knowledge generalizes because if a new compound also inhibits MEK, the same downstream effects should follow, even if the compound's name and structure are different.

This is an instance of a broader principle in machine learning: inductive biases that match the causal structure of the domain improve generalization. The STATE model (Adduri et al., 2025), which learns from raw transcriptomic data without explicit mechanistic structure, achieves only 0.257 average F1 on DE—comparable to random—despite being trained on the full Tahoe-100M corpus. The K-nearest-neighbor baseline, which uses chemical fingerprint similarity, achieves 0.406, showing that chemical structure alone provides some generalization signal. But the structured reasoning approach outperforms both, suggesting that mechanistic knowledge (how the drug works) provides generalization benefits beyond what chemical similarity (what the drug looks like) or expression pattern matching (what the drug does to expression on average) can provide.

The significance of this finding extends beyond the specific TahoeQA task. It suggests that for scientific machine learning problems where data is sparse and generalization to novel interventions is required, explicit mechanistic reasoning traces may be more valuable as training signals than raw experimental data alone. This is a fundamental insight about the relationship between scientific knowledge representation and model generalization, not merely an incremental performance improvement. The paper provides controlled evidence for this claim: same base model, same perturbation data, same labels—the only difference is whether structured reasoning is included in the training signal, and that difference accounts for up to 15.3 percentage points in F1.

A critical nuance: the SFT-Prompt model (which receives pre-computed explanations) outperforms SFT-Generate (which must produce explanations). This suggests that the benefit comes from having access to correct mechanistic knowledge, not from the process of generating it. SFT-Generate may be limited by the difficulty of learning to produce correct explanations—a harder task than conditioning on them—which is consistent with the paper's decision to exclude RL due to sparse reward problems. The practical implication is that for downstream applications, conditioning on verified explanations (perhaps generated by a separate, more powerful system) may be more effective than training a single model to both reason and predict.

5. Experimental Analysis

Evaluation Methodology

Dataset. The experiments use the Tahoe-100M atlas (Zhang et al., 2025), a giga-scale single-cell perturbation atlas, from which the authors extract 18,950 unique compound perturbation-context pairs. These pairs form the basis for generating the VC-Traces dataset through the VCR-Agent framework. For the explanation quality experiments (Section 5.1), the evaluation focuses on a representative subset of five cell lines—C32, HOP62, HepG2/C3A, Hs 766T, and PANC-1—chosen to align with the test split of Tahoe-X1 (Gandhi et al., 2025). For the downstream TahoeQA task (Section 5.2), labeled data is constructed by performing differential expression analysis using DESeq2 (Love et al., 2014) on Tahoe-100M: for each perturbation-context pair, the top 25 up-regulated genes, top 25 down-regulated genes (by log2 fold change magnitude), and 100 random non-regulated genes are selected, creating a classification dataset with 50 positive and 100 negative examples per pair. The data is split by perturbation (not by gene or cell line) to ensure no overlap between training and test perturbations, with approximately 1,000 test examples randomly selected for evaluation. This perturbation-based split means the test set contains compounds the model has never seen, testing generalization rather than memorization.

Base model(s). For the VCR-Agent pipeline itself (report generation and explanation construction), the backbone LLM is Claude 4 (Anthropic, 2025), chosen based on an ablation study (Appendix D.2, Table Appx.4) showing it achieves perfect trace validity (1.000) and the highest DTI verifier score (0.766) compared to GPT-4.1 and Gemini-2.5-Flash. For the downstream TahoeQA fine-tuning experiments, the base model is Qwen3-4B-Instruct-2507 (Yang et al., 2025), a 4-billion-parameter model chosen to demonstrate that structured reasoning benefits transfer even to modestly-sized models, isolating the effect of training data quality from model scale. For explanation quality baselines, three open-source models are compared: Qwen3-30B-A3B, DeepSeek-R1-0528-Qwen3-8B (DeepSeek-AI et al., 2025), and Llama3.3-70B-Instruct (Grattafiori et al., 2024), alongside Claude-Sonnet-4 as a closed-source baseline using the same model family as VCR-Agent.

Metrics. The paper evaluates along two dimensions with distinct metric suites. For explanation quality (Section 5.1), four metrics are reported: (1) Validity—the proportion of generated traces that are both syntactically correct (containing proper <explain> and <dag> tags) and structurally valid (all action primitives adhere to the defined action space); (2) Verifiability—the proportion of generated arguments that can be successfully mapped to valid biomedical entities for verification; (3) DTI score—the average binding probability score from the drug-target interaction verifier across all binds_to actions in a trace; and (4) DE score—the proportion of traces where at least one differential expression step included in the trace is correct according to the DE verifier. For the TahoeQA downstream task (Section 5.2), the primary metric is F1-score (harmonic mean of precision and recall), chosen due to label imbalance (50 positive, 100 negative examples per perturbation-context pair) where accuracy would be misleading. Results are reported both as average F1 across the five individual cell-line test sets and as union F1 on a combined test set. For the Direction of Change subtask, the same F1 metric is used. Standard deviations are reported across cell lines for explanation quality metrics and per-cell-line values are provided for TahoeQA in Appendix C, Table Appx.2.

Baselines. For explanation quality (Section 5.1), the paper compares VCR-Agent against both open-source and closed-source LLMs: Claude-Sonnet-4 (Anthropic, 2025, the same base model family as VCR-Agent), DeepSeek-R1-8B (DeepSeek-AI et al., 2025), Qwen3-30B (Yang et al., 2025), and Llama3.3-70B (Grattafiori et al., 2024). For the TahoeQA task (Section 5.2), baselines span three categories: (1) simple statistical models—a random baseline, a mean baseline that predicts labels based on the average gene expression response for a given compound, and a k-nearest-neighbor baseline that classifies by aggregating labels of the k most similar compounds using extended-connectivity fingerprints (ECFP; Rogers & Hahn, 2010); (2) a transcriptomic foundation model—the STATE Transition (ST) model (Adduri et al., 2025), which learns state-transition functions over gene expression from the full Tahoe-100M corpus and is evaluated under the few-shot setting where test perturbations are held out; and (3) LLM baselines—zero-shot prompting with Qwen3-4B and vanilla supervised fine-tuning (SFT) on Qwen3-4B using the same perturbation-context pairs but without structured explanations, predicting answer labels directly.

Generation budget / compute accounting. The paper does not use a unified compute budget metric across experiments since different components have fundamentally different cost structures. For the VCR-Agent pipeline, compute cost is implicit in the number of LLM calls (one for report generation, one for explanation construction) plus retrieval operations (four database queries, embedding similarity search, NER). For downstream SFT training, all models use the same Qwen3-4B-Instruct-2507 backbone with identical hyperparameters (learning rate 2×10⁻⁴, LoRA rank 64, gradient accumulation steps 4, weight decay 0.01), ensuring fair comparison. Training is conducted on H100 GPUs. The inference hyperparameters for generating structured explanations are fixed across experiments: temperature 0.2, nucleus sampling (top-p) 0.8, top-k 20. The paper does not report wall-clock time or total FLOPs for any experiment.

Cross-validation / statistical protocol. The paper does not employ cross-validation in the traditional sense. For TahoeQA, the perturbation-based train-test split serves as the evaluation protocol: test perturbations are entirely held out from training, ensuring that performance reflects generalization to novel compounds. For explanation quality, standard deviations are computed across the five cell lines, providing a measure of consistency across biological contexts. The human evaluation (Appendix D.6) uses a stratified random sample of 10 traces across the five cell lines, scored independently by domain experts in molecular biology and pharmacology, with agreement measured via Pearson correlation against LLM-judge scores.

Main Quantitative Results

Explanation Quality: VCR-Agent vs. Baseline LLMs

The headline result from Table 1 is that VCR-Agent achieves perfect validity (1.000 ± 0.000) and the highest DTI verifier score (0.725 ± 0.017), substantially outperforming all baseline LLMs on the metrics that matter most for downstream usability—structural correctness and factual accuracy of drug-target binding claims.

Looking at the numbers in detail from Table 1:

• Validity: VCR-Agent achieves 1.000, matching Claude-Sonnet-4 (also 1.000), while open-source models fall short: DeepSeek-R1-8B at 0.003 (essentially zero—it cannot produce syntactically correct structured explanations), Qwen3-30B at 0.860, and Llama3.3-70B at 0.841. This means VCR-Agent and Claude-Sonnet-4 never produce malformed outputs, while open-source models produce invalid traces 14–16% of the time and DeepSeek fails almost entirely.

• Verifiability: VCR-Agent achieves 0.945 ± 0.011, second only to DeepSeek-R1-8B's 1.000. However, DeepSeek's perfect verifiability is misleading because it only produces valid traces 0.3% of the time—when it does produce valid output, those outputs happen to be verifiable, but 99.7% of attempts fail completely. Claude-Sonnet-4 scores 0.867, Qwen3-30B scores 0.965, and Llama3.3-70B scores 0.887. VCR-Agent's high verifiability means its generated arguments reliably map to real biomedical entities that can be checked by verifiers.

• DTI verifier score: VCR-Agent's 0.725 substantially exceeds Claude-Sonnet-4 (0.657), Qwen3-30B (0.528), Llama3.3-70B (0.322), and DeepSeek-R1-8B (0.000—it never produces valid binds_to claims). This is the most direct measure of factual accuracy for drug-target interactions, and VCR-Agent's advantage of roughly 10 percentage points over the next-best model (Claude-Sonnet-4) suggests the knowledge retrieval pipeline meaningfully improves binding claim accuracy beyond what the base LLM can produce from parametric knowledge alone.

• DE verifier score: VCR-Agent achieves 0.528 ± 0.060, slightly ahead of Claude-Sonnet-4 (0.504 ± 0.064). Qwen3-30B scores 0.272, Llama3.3-70B scores 0.090, and DeepSeek-R1-8B scores 0.000. The DE scores are lower than DTI scores across all models, suggesting that predicting correct gene regulation claims is inherently harder than predicting drug-target binding—unsurprising given that gene regulation involves context-dependent transcriptional programs rather than binary binding events.

A critical detail: these metrics evaluate raw generative performance before any filtering. The paper notes that during VC-Traces construction, the verification pipeline further improves quality by "excluding 28.2% of faulty DTI claims and refining 87.3% of DE actions to eliminate hallucinations." This means the numbers in Table 1 understate the final dataset quality—after filtering, the surviving traces have substantially higher factual accuracy than the raw generation metrics suggest.

The comparison between VCR-Agent and Claude-Sonnet-4 is particularly informative because both use the same base model family. VCR-Agent's advantage (DTI: 0.725 vs. 0.657; DE: 0.528 vs. 0.504) is attributable entirely to the two-stage pipeline with knowledge retrieval and structured prompting, not to a more capable base LLM. This provides a controlled demonstration that the framework design, not model scale, drives the improvement.

Figure 5 provides a concrete illustration of the filtering process: an initial regulates_expression claim for gene_list=["MYC","CCND1","FOS","JUN"] is refined by the DE verifier, which confirms MYC, CCND1, and FOS as correctly downregulated but removes JUN because the experimental data does not support that claim. This example demonstrates that even high-quality traces (the example in Figure 2a achieves high verifier scores) contain hallucinated individual genes that the verifiers catch.

TahoeQA Downstream Performance: Structured Reasoning as Supervision Signal

The headline result from Figure 6 and Table Appx.2 is that models trained with structured mechanistic explanations substantially outperform all baselines on the differential expression prediction task, with SFT-Prompt achieving average F1 of 0.435 compared to 0.292 for vanilla SFT (a 15.3 percentage point absolute improvement, roughly 49% relative improvement) and 0.406 for the best non-LLM baseline (K-nearest-neighbor).

Looking at the differential expression results in detail from Figure 6 and Table Appx.2:

Statistical baselines:

• Random: 0.259 average F1 (0.286 union)
• Mean: 0.393 average (0.401 union)
• K-nearest-neighbor: 0.406 average (0.403 union)

The K-nearest-neighbor baseline using ECFP fingerprints is notably strong, outperforming both STATE (0.257 average) and vanilla SFT (0.292 average). This confirms prior findings (Kernfeld et al., 2025; Wenkel et al., 2025) that simple chemical-similarity-based approaches are competitive baselines for perturbation prediction tasks.

Transcriptomic foundation model:

• STATE (ST): 0.257 average (0.251 union)

STATE's poor performance (barely above random) is surprising given it was trained on the full Tahoe-100M corpus—but the paper notes it was "evaluated under our few-shot setting, where test perturbations are held out," meaning the model must generalize to compounds it never saw during training. This suggests STATE's learned representations may not transfer well to truly novel compounds, consistent with the paper's argument that raw numerical representations alone are insufficient for out-of-distribution generalization.

LLM baselines:

• Zero-shot (Qwen3-4B): 0.367 average (0.338 union)
• Vanilla SFT (Qwen3-4B): 0.292 average (0.291 union)

The zero-shot LLM outperforming SFT is counterintuitive but explicable: SFT on only 1,000 training examples with direct label prediction may overfit to surface patterns that don't generalize, while zero-shot prompting leverages the model's pretrained biological knowledge (however incomplete). The SFT result of 0.292—barely above random (0.259)—underscores how difficult this task is when the model lacks mechanistic guidance.

Structured reasoning models:

• SFT-Prompt (Ours): 0.435 average (0.452 union)
• SFT-Generate (Ours): 0.388 average (0.441 union)

SFT-Prompt achieves the best overall average (0.435), and notably, its union F1 of 0.452 is the highest reported for the DE task. The per-cell-line breakdown (Table Appx.2) shows consistent improvement: SFT-Prompt outperforms vanilla SFT on every cell line (C32: 0.470 vs. 0.344; HOP62: 0.470 vs. 0.353; HepG2/C3A: 0.362 vs. 0.163; Hs 766T: 0.470 vs. 0.363; PANC-1: 0.405 vs. 0.236). The HepG2/C3A improvement is particularly dramatic—0.362 vs. 0.163, more than doubling—suggesting structured reasoning is especially valuable when the baseline model performs poorly, perhaps because the mechanistic reasoning compensates for cell-type-specific knowledge gaps.

SFT-Generate (0.388 average) outperforms all non-structured baselines but trails SFT-Prompt by 4.7 percentage points on average. This gap is consistent with the paper's interpretation: learning to generate correct explanations is harder than conditioning on pre-computed correct explanations. SFT-Generate must learn both the reasoning format and the biological content simultaneously, while SFT-Prompt only needs to learn how to extract predictive signals from already-correct explanations. The union F1 of SFT-Generate (0.441) is notably competitive with SFT-Prompt (0.452), suggesting that on the combined test set—which may have different cell-type composition—the generative model's performance is closer to the prompt-conditioned model.

Direction of Change results (Figure 6, right panel; Table Appx.2):

The DOC task shows a different pattern, with the strongest baselines already performing well:

• Random: 0.530 average (0.513 union)
• Mean: 0.817 average (0.810 union)
• K-nearest-neighbor: 0.771 average (0.757 union)
• STATE: 0.721 average (0.695 union)
• Vanilla SFT: 0.823 average (0.820 union)
• SFT-Prompt (Ours): 0.832 average (0.820 union)
• SFT-Generate (Ours): 0.715 average (0.727 union)

The DOC task is fundamentally easier than DE prediction (once you know a gene is differentially expressed, predicting the direction is a simpler problem with stronger statistical signal), and the improvements from structured reasoning are smaller. SFT-Prompt achieves 0.832, a modest improvement over vanilla SFT (0.823), and union F1 is equal at 0.820. The mean baseline (0.817) is surprisingly strong, suggesting that for many compounds, the average direction of gene regulation is a reliable predictor. SFT-Generate (0.715) underperforms both SFT-Prompt and vanilla SFT on DOC, possibly because the model struggles to correctly generate the direction of regulation in its reasoning chains, and errors in generated reasoning propagate to prediction errors. The zero-shot LLM performs catastrophically on DOC (0.066 average), indicating that without fine-tuning, Qwen3-4B cannot reliably predict the direction of gene expression changes from perturbation descriptions alone.

Key insight from the gap between SFT-Prompt and SFT-Generate: The fact that SFT-Prompt (conditioning on explanations) substantially outperforms SFT-Generate (generating explanations) demonstrates that the benefit of structured reasoning comes from having access to correct mechanistic knowledge, not from the act of generating it. This has practical implications: for downstream applications, it may be more effective to generate explanations with a separate, more powerful system (like VCR-Agent with Claude 4) and condition downstream models on those explanations, rather than training a single model to both reason and predict. The paper acknowledges this implicitly by leaving RL-based optimization (which could potentially close the generation gap) for future work, citing sparse reward problems in classification settings.

Ablation Studies and Robustness Checks

Retrieval source ablation (Appendix D.1, Table Appx.3): Each knowledge source contributes complementary information, and no single source approaches the performance of the full combination. Using PubMed alone achieves DTI score of 0.432 ± 0.059 and DE score of 0.435 ± 0.050; StarkPrimeKG alone achieves 0.382 ± 0.031 and 0.449 ± 0.054; Harmonizome alone achieves 0.399 ± 0.013 and 0.457 ± 0.051; Wikipedia alone achieves 0.469 ± 0.028 and 0.485 ± 0.063. The full combination achieves 0.766 ± 0.008 and 0.487 ± 0.054. The DTI score nearly doubles with the full combination (0.766 vs. 0.382–0.469 range), while the DE score shows more modest gains (0.487 vs. 0.435–0.485), suggesting that drug-target binding knowledge is distributed across databases in complementary ways while gene regulation information has more overlap. Verifiability is substantially higher with the full combination (0.601) compared to any single source (0.292–0.300), because having multiple sources increases the likelihood that entity names are recognized and mappable to ontologies.

Backbone LLM ablation (Appendix D.2, Table Appx.4): Claude substantially outperforms GPT-4.1 and Gemini-2.5-Flash on the metrics that matter for downstream usability. Claude achieves validity 1.000, verifiability 0.601, DTI 0.766, DE 0.487. GPT-4.1 achieves validity 0.970, verifiability 0.299, DTI 0.399, DE 0.527. Gemini-2.5-Flash achieves validity 0.060, verifiability 0.963, DTI 0.338, DE 0.207. GPT-4.1's higher DE score (0.527 vs. 0.487) comes at the cost of drastically lower verifiability (0.299 vs. 0.601), meaning a large fraction of its outputs cannot be reliably evaluated—the higher DE score may reflect selection bias where only the most plausible traces survive structural validation. Gemini-2.5-Flash's near-zero validity (0.060) makes it effectively unusable for structured explanation generation despite its high verifiability on the rare valid outputs.

One-step vs. two-step pipeline ablation (Appendix D.3, Table Appx.5): The two-step pipeline (report generation then explanation construction) nearly doubles both DTI score (0.766 vs. 0.329) and DE score (0.487 vs. 0.267) compared to one-step generation that produces structured explanations directly from retrieved information without an intermediate report. Verifiability also improves (0.601 vs. 0.534). This is the paper's most important ablation because it directly tests the core architectural hypothesis—that decoupling knowledge synthesis from structured reasoning is necessary—and the magnitude of the gap (roughly 2× for DTI, 1.8× for DE) provides strong evidence for the two-stage design.

Same retrieved report ablation (Appendix D.4, Table Appx.6): When all baseline models receive the identical retrieved report generated by VCR-Agent's report generator as input, the performance gap narrows but VCR-Agent maintains advantages. Qwen3 achieves the highest DE score (0.556 vs. VCR-Agent's 0.487) but with substantially lower verifiability (0.340 vs. 1.000), meaning many of its outputs cannot be reliably used. DeepSeek and Llama3.3 achieve poor DTI scores (0.185 and 0.079) even with access to the same retrieved report, indicating that model capability—not just access to knowledge—determines whether structured explanations can be correctly generated. This ablation disentangles the contribution of retrieval quality from explanation construction quality: VCR-Agent's advantage is not solely attributable to better retrieval; the structured prompting and two-stage design contribute independently.

Verifier-based filtering ablation (Appendix D.5, Table Appx.7): Verifier-based filtering yields modest but consistent improvements in LLM-judge scores: scientific accuracy increases from 0.641 (no verifiers) to 0.645 (full pipeline), logical consistency from 0.720 to 0.724, and mechanistic clarity remains at 0.725. The paper acknowledges that these gains are small because the LLM-judge "lacks the specialized regulatory knowledge such as those from Boltz-2 and Tahoe-100M, which are used by our verifiers"—the LLM-judge cannot detect the specific factual errors that the specialized verifiers catch, so filtered and unfiltered traces receive similar scores despite the verifiers having removed concrete hallucinations. This finding is methodologically important: it demonstrates that general-purpose LLM evaluators are insufficient for assessing factual accuracy in specialized scientific domains, and that domain-specific verifiers provide a complementary quality signal.

Human evaluation (Appendix D.6, Table Appx.8, Figure Appx.1): On a randomized subset of 10 traces scored by domain experts, Pearson correlation between human expert ratings and LLM-judge scores averages r = 0.72 across criteria (scientific accuracy: 0.72, logical consistency: 0.69, mechanistic clarity: 0.76). Average LLM-judge scores on the full dataset are 0.65 (scientific accuracy), 0.72 (logical consistency), 0.73 (mechanistic clarity). Traces judged as plausible by human experts consistently cluster in the upper-right region of the LLM-judge score space, while implausible traces receive lower scores from both humans and the LLM-judge. This validation is important because it establishes that the LLM-judge metrics—while imperfect for detecting specialized factual errors—correlate meaningfully with domain expert assessment, providing confidence that the evaluation framework captures biologically meaningful quality distinctions.

Critical Assessment

Claim 1: VCR-Agent produces explanations with higher factual accuracy than baseline LLMs.

What the experiments demonstrate: Table 1 shows VCR-Agent achieving DTI score of 0.725 vs. the next-best closed-source model (Claude-Sonnet-4) at 0.657—a 10.3% relative improvement. DE scores show a narrower gap (0.528 vs. 0.504). These improvements are measured on raw generated outputs before any filtering, meaning they reflect the pipeline's ability to produce more factually accurate claims from the outset, not just to filter out errors post-hoc.

What is not demonstrated: The comparison is against models generating explanations directly from perturbation-context pairs without retrieval augmentation. The ablation in Appendix D.4 shows that when baselines receive the same retrieved report, Qwen3 actually achieves a higher DE score (0.556 vs. 0.487), albeit with lower verifiability. This makes the claim more nuanced: VCR-Agent's advantage on DTI is robust even when baselines get retrieval, but its advantage on DE may be partially attributable to better retrieval rather than better explanation construction. Additionally, all evaluations are on a subset of five cell lines; whether the advantage holds across the full diversity of Tahoe-100M cell types is untested.

Genuine weaknesses: The metrics have a selection bias problem. Validity and verifiability act as implicit filters—traces that fail these format checks cannot be scored by verifiers, and their exclusion from DTI/DE averages may inflate apparent performance for models with low validity. For example, DeepSeek-R1-8B achieves 0.000 DTI and DE scores because it almost never produces valid traces; if its invalid traces happened to contain correct biological claims (in malformed syntax), those claims would be invisible to the metrics. The paper doesn't report what fraction of baseline outputs are excluded due to validity/verifiability failures, making it difficult to assess whether performance differences reflect genuine accuracy improvements or differential filtering rates.

Claim 2: Training with verified structured explanations improves downstream gene expression prediction.

What the experiments demonstrate: Figure 6 and Table Appx.2 show SFT-Prompt achieving 0.435 average F1 on DE vs. 0.292 for vanilla SFT—a substantial 15.3 percentage point absolute improvement. The improvement is consistent across all five cell lines (Table Appx.2), with particularly large gains on HepG2/C3A (0.362 vs. 0.163). SFT-Generate (0.388) also substantially outperforms vanilla SFT, confirming that the benefit generalizes across training configurations.

What is not demonstrated: The claim is about "improving predictive accuracy on novel compounds," but the test set is only approximately 1,000 examples across five cell lines, and the paper does not report how many unique perturbations this represents. If the test perturbations are few in number or structurally similar to training perturbations (despite being "held out" by perturbation identity), the generalization claim is weaker than it appears. The ECFP-based K-nearest-neighbor baseline—which relies purely on chemical structure similarity—achieves 0.406, which is competitive with SFT-Generate (0.388) and not far below SFT-Prompt (0.435). This suggests that a substantial portion of the predictable signal in the DE task comes from chemical structure similarity alone, and the incremental benefit of mechanistic reasoning—while real—is in the range of 3–4 F1 points for SFT-Generate, not 15 points.

Missing ablation: The paper does not ablate whether the structured explanation format itself matters, or whether an equivalent amount of unstructured biological text (e.g., the raw report from the report generator) would provide similar benefits. If conditioning on a 3,000-word natural-language report about Binimetinib's mechanism of action improves prediction similarly to conditioning on the structured DAG, then the structured format is not the active ingredient—it's the factual content gained through retrieval. This ablation would disentangle the contribution of the VCR-Agent's structured formalism from the contribution of its retrieval pipeline.

Claim 3: The verifier-based filtering pipeline eliminates hallucinations and improves explanation quality.

What the experiments demonstrate: The paper reports that filtering excludes 28.2% of DTI claims and refines 87.3% of DE actions (Section 5.1). The ablation in Table Appx.7 shows modest improvements in LLM-judge scores with filtering. The human evaluation (Appendix D.6) shows that filtered traces receive high expert ratings and that LLM-judge scores correlate with expert judgment.

What is not demonstrated: The 28.2% and 87.3% figures are aggregate statistics without per-cell-line breakdown, and the paper doesn't report what fraction of complete traces are entirely discarded (as opposed to having individual arguments pruned). More importantly, the verifiers themselves have unknown false positive/negative rates—the DTI verifier using Boltz-2 predicts binding probability, not experimental binding confirmation, and the DE verifier queries Tahoe-100M data which has its own noise and biases. A binds_to claim that fails the DTI verifier threshold might actually be correct (e.g., if Boltz-2 has poor performance on that drug-target class), and a gene removed by the DE verifier might be contextually regulated in ways not captured by the specific Tahoe-100M experiment. The paper's conservative filtering philosophy ("only removes claims that directly contradict established biological evidence") is reasonable but untested: there is no experiment measuring whether correct claims are accidentally filtered out.

Missing experiment: A human expert audit of filtered vs. unfiltered traces—where biologists judge specific verifier decisions (was this claim correctly removed? was this claim incorrectly kept?)—would provide ground-truth validation of verifier accuracy. Without this, the verifiers' reliability is assumed rather than demonstrated.

Claim 4: Grounding predictions in biological reasoning improves generalization to novel compounds.

What the experiments demonstrate: SFT-Prompt (0.435) substantially outperforms STATE (0.257) and K-nearest-neighbor (0.406) on the DE task with held-out perturbations. The perturbation-based train-test split ensures the test compounds are novel, and the improvement over baselines that use only chemical structure or expression patterns suggests mechanistic knowledge provides generalization benefits beyond surface similarity.

What is not demonstrated: The paper only tests five cell lines, all cancer cell lines. Whether the approach generalizes to non-cancer contexts, primary cells, or in-vivo settings is entirely untested. The claim is about "sparse-data and out-of-distribution scenarios," but the test perturbations may still be within the distribution of drug-like small molecules—the paper doesn't characterize how chemically dissimilar the test compounds are from training compounds. If test compounds are close analogs of training compounds (e.g., both sets contain kinase inhibitors with similar scaffolds), then the "generalization" being measured is relatively narrow.

Additional missing baselines and experiments:

• No direct comparison to fine-tuning on unstructured reports. The paper doesn't test whether SFT on the raw natural-language reports (without structured formatting) would achieve similar downstream improvements. This is the most important missing ablation for establishing that the structured formalism itself—rather than the factual knowledge retrieved and synthesized—drives the downstream gains.

• No scale analysis for the downstream model. All downstream experiments use Qwen3-4B. The paper doesn't show whether larger models benefit more or less from structured reasoning supervision—a scaling analysis would reveal whether structured reasoning provides diminishing returns as model capacity increases (suggesting larger models already encode mechanistic knowledge parametrically) or increasing returns (suggesting structured reasoning helps organize knowledge that larger models possess but cannot effectively deploy).

• Single backbone for downstream experiments. The paper only fine-tunes Qwen3-4B for TahoeQA. Results might differ with other model families (Llama, Mistral, DeepSeek) that have different pretraining mixtures and different amounts of biological knowledge encoded parametrically.

• No temporal or version split. The retrieved knowledge (from StarkPrimeKG, PubMed, Wikipedia) may contain information about the specific perturbations being tested, and the paper doesn't verify that the knowledge bases used for retrieval were frozen at a time point before the test perturbations were characterized in the literature. If the retrieval includes papers or database entries that describe the transcriptional effects of test compounds, the downstream model could benefit from test-set leakage through the retrieval pipeline.

• Five cell lines is a small sample for cell-type generalization claims. The Tahoe-100M atlas contains many more cell types. Demonstrating consistent improvements across a broader diversity of cell types (primary vs. immortalized, different tissue origins, different disease states) would strengthen the generalization claim considerably.

Summary assessment: The experiments provide credible evidence that structured mechanistic reasoning, when grounded in retrieved knowledge and filtered through specialized verifiers, can serve as an effective supervision signal for downstream biological prediction tasks—improving performance on held-out compounds beyond what chemical similarity or expression pattern matching alone can achieve. The evidence is strongest for the differential expression task (where improvements are large and consistent across cell lines) and weaker for the direction of change task (where improvements are modest and the SFT-Generate variant actually underperforms). The paper's broader claim—that the structured formalism and verifier pipeline represent a general solution for scientific reasoning—is supported in principle but demonstrated only in a specific biological context (drug perturbations in cancer cell lines) with a specific downstream task (binary gene expression classification) and a specific model family (Qwen3-4B). The missing ablations (unstructured report baseline, scaling analysis, verifier accuracy audit) leave open important questions about which components of the framework are most responsible for the observed improvements and whether the approach generalizes beyond the tested setting.

6. Limitations and Trade-offs

The Cost of Difficulty Estimation (Factual Grounding) Is Unaccounted For

The assumption or constraint. The VCR-Agent framework requires a substantial retrieval and report generation step before any structured reasoning can occur. For every perturbation–context pair, the system queries four external knowledge bases (StarkPrimeKG, Harmonizome, PubMed, Wikipedia), runs biomedical NER with HunFlair2, performs embedding-based similarity search for knowledge graph matching, and makes a full Claude 4 inference call to synthesize the retrieved information into a comprehensive mechanistic report—before the explanation constructor is ever invoked. The paper does not quantify this cost in any experiment. When evaluating downstream performance in Section 5.2, the SFT-Prompt and SFT-Generate models receive pre-computed verified explanations that were generated offline by this expensive pipeline. The paper never reports the total compute budget or wall-clock time required to construct VC-Traces from the 18,950 perturbation–context pairs, nor does it amortize this cost into the reported downstream efficiency gains.

The consequence. This creates a significant ambiguity about practical deployment. The headline result—that training with structured explanations improves TahoeQA differential expression F1 by 15.3 percentage points—is measured after the expensive explanation generation has already been done by a separate, more powerful system (Claude 4 with multi-source retrieval). A practitioner who wants to use this approach on new perturbations not in VC-Traces faces the full cost of the VCR-Agent pipeline for each new compound: NER, four database queries, two Claude 4 API calls (report generation + explanation construction), and verifier filtering. If the per-perturbation cost is high (e.g., minutes of wall-clock time and non-trivial API expenses for the LLM calls), the approach may be impractical for large-scale screening where thousands of compounds need explanations. More subtly, the retrieval pipeline itself depends on external knowledge bases that must be maintained and queried—the cost and latency of these queries varies by database and is not characterized. For latency-sensitive applications (e.g., interactive drug discovery tools where a biologist wants an explanation immediately), the two-stage pipeline with Claude 4 calls may be too slow regardless of accuracy improvements.

What evidence exists in the paper. The paper provides no experiment that quantifies the retrieval, report generation, or explanation construction cost. Section 3.1 describes the retrieval pipeline in detail, and Appendix C.4 notes that H100 GPUs were used for TahoeQA training, but the cost of generating VC-Traces itself is never reported—not in GPU-hours, not in API calls, not in wall-clock time, not in dollars. The paper does not amortize the explanation generation cost into any downstream performance metric. The ablation in Appendix D.3 compares one-step vs. two-step generation in terms of quality (DTI score, DE score) but not in terms of cost. The retrieval source ablation in Appendix D.1 shows that all four databases are needed for best performance, which implicitly means the full cost of querying four databases must be paid for optimal results.

Mitigation status. The paper does not address this limitation or acknowledge it as a deployment concern. The authors do not discuss strategies for reducing per-perturbation cost—for instance, caching retrieved knowledge for structurally similar compounds, using smaller models for report generation, or amortizing retrieval across batches of related perturbations. There is no suggestion that the cost could be reduced or that a cost–accuracy tradeoff analysis would be valuable. This is a notable gap given the paper's framing of VC-Traces as a resource for "scalable" reasoning generation.

Verifier Coverage Is Fragmentary and Verifier Accuracy Is Unvalidated

The assumption or constraint. The verifier-based filtering pipeline—which is the paper's central mechanism for ensuring factual reliability—covers only a subset of the 20 action primitives and operates with unvalidated accuracy. The paper implements four verifiers: DTI (for binds_to), DE (regulates_expression), LOC (localizes_to), and PHENO (induces_phenotype / alleviates_phenotype). Section 4.2 notes that DTI and DE actions "jointly appear in 91.5% of all explanation traces in VC-Traces," which is intended to demonstrate broad coverage. However, this means ~8.5% of actions are never verified, and even among the "covered" actions, the verifiers themselves are imperfect. The DTI verifier uses Boltz-2 (Passaro et al., 2025) to predict binding probability—a computational prediction, not an experimental measurement. The DE verifier queries Tahoe-100M differential expression data, which has its own statistical noise, batch effects, and limitations (the atlas does not cover all possible cell types, treatment conditions, or time points). The LOC verifier cross-references UniProt and Human Protein Atlas annotations, which are incomplete and sometimes contradictory. The PHENO verifier queries the Cellular Phenotype Database, which has limited coverage.

The consequence. The verifier pipeline introduces two types of errors that are never measured: false negatives (correct claims filtered out because the verifier lacks evidence or makes a mistake) and false positives (incorrect claims that pass filtering because the verifier lacks coverage or sensitivity). A binds_to claim might be correct (experimentally validated binding) but receive a low Boltz-2 score because the model performs poorly on that drug–target class—the entire trace would then be discarded, losing valid mechanistic reasoning. Conversely, a regulates_expression claim for a gene not present in the relevant Tahoe-100M experiment would not be filtered (the paper's conservative design leaves unverified claims intact), meaning hallucinated gene regulation claims that happen not to contradict available evidence survive into VC-Traces. More fundamentally, 8 of the 20 action primitives—including modulates_pathway_activity, post_translational_modification, modulates_complex, and chromatin_modification—have no verifier at all. Claims about pathway modulation or protein phosphorylation are never checked, meaning substantial portions of the mechanistic reasoning chain are unverified. A trace could contain a hallucinated pathway modulation claim that cascades into downstream effects, and if the binds_to and regulates_expression nodes happen to pass their verifiers, the trace would be accepted despite containing a critical factual error at an unverified node.

What evidence exists in the paper. The paper acknowledges the verifier coverage limitation indirectly in the Impact Statement: "our current implementation includes only the verifiers most critical to our downstream tasks—Drug-Target Interactions and Differential Expression—this represents an extensible foundation rather than an exhaustive solution." The ablation in Appendix D.5 (Table Appx.7) shows that verifier-based filtering produces only modest improvements in LLM-judge scores (scientific accuracy from 0.641 to 0.645), which the paper attributes to the LLM-judge's inability to detect the errors verifiers catch—but this also means the LLM-judge cannot validate whether the verifiers are making correct decisions. The paper reports that filtering excluded 28.2% of DTI claims and refined 87.3% of DE actions (Section 5.1), but it does not report what fraction of those exclusions were correct (i.e., the filtered claims were genuinely false) versus incorrect (filtered claims were actually true). The human evaluation in Appendix D.6 covers only 10 traces and uses expert assessment of overall plausibility, not a claim-by-claim audit of verifier decisions.

Mitigation status. The paper acknowledges the extensibility limitation and frames the verification suite as a foundation that can be expanded. However, there is no acknowledgment that the existing verifiers may themselves produce errors, no experiment measuring verifier false positive/negative rates, and no discussion of what level of verifier accuracy would be needed for the filtering pipeline to be trustworthy. The conservative filtering philosophy ("only removes claims that directly contradict established biological evidence") is reasonable but remains an untested assumption about verifier behavior.

Generalization Is Demonstrated on a Narrow Slice of Biology (5 Cancer Cell Lines, Drug Perturbations Only)

The assumption or constraint. All experiments—both explanation quality assessment (Section 5.1) and downstream TahoeQA evaluation (Section 5.2)—are conducted on a subset of five cell lines (C32, HOP62, HepG2/C3A, Hs 766T, PANC-1) selected from the Tahoe-100M atlas to align with the Tahoe-X1 test split. All five are cancer cell lines (melanoma, lung adenocarcinoma, hepatocellular carcinoma, pancreatic carcinoma), and all experiments use chemical compound perturbations. The paper's action space includes primitives for genetic perturbations (gain_of_function, loss_of_function), cell–cell interactions, metabolic conversions, epigenetic modifications, and more—but none of these are tested with verifiers or in downstream tasks. The TahoeQA task itself is a binary classification of whether individual genes are differentially expressed, which captures only one dimension of cellular response (transcriptional changes) and ignores proteomic, metabolomic, or phenotypic outcomes that the action space is designed to represent.

The consequence. The paper's claims about "improving performance in sparse-data and out-of-distribution scenarios" and providing a framework for "autonomous mechanistic reasoning in virtual cells" are supported only for a specific, narrow regime: predicting transcriptional responses to small-molecule drugs in cancer cell lines. It is entirely unknown whether the structured reasoning approach transfers to genetic perturbations (CRISPR knockouts, overexpression), to non-cancer cell types (primary cells, stem cells, differentiated tissues), to other perturbation modalities (cytokines, antibodies, environmental stressors), or to other prediction tasks (phenotypic outcomes, pathway activity, cell state transitions). The K-nearest-neighbor baseline achieving 0.406 F1 on DE—competitive with SFT-Generate at 0.388—suggests that a substantial portion of predictable signal in the tested regime comes from chemical structure similarity rather than mechanistic reasoning. A practitioner working with genetic perturbations or non-cancer contexts has no evidence that the framework would be effective. More subtly, cancer cell lines have extensively studied biology with rich knowledge base coverage (many papers, well-characterized mutations, known drug responses), which may make the retrieval step unusually effective—in poorly characterized cell types or organisms where knowledge bases are sparse, the report generator may produce thin or inaccurate reports, undermining the entire pipeline.

What evidence exists in the paper. All quantitative results are restricted to the five cell lines. Figure 6 shows per-cell-line F1 scores (Table Appx.2), all within the cancer domain. The paper does not include any experiment with genetic perturbations, despite the action space including gain_of_function and loss_of_function. The paper does not include any non-cancer cell type. The paper does not include any prediction task beyond binary gene expression classification. The limitation is not explicitly acknowledged as a generalization concern—the paper frames VC-Traces and the TahoeQA results as evidence that "grounding high-dimensional transcriptomic data in biological reasoning improves performance" without qualification about domain specificity.

Mitigation status. Not addressed. The paper does not discuss domain generalization limitations, does not suggest that results might differ for genetic perturbations or non-cancer contexts, and does not propose experiments to test transfer. The extensible design of the action space and verifier suite implies that the framework could be applied to other domains, but no evidence is provided that it would work.

The Structured Formalism May Be Unnecessarily Constraining for Many Biological Phenomena

The assumption or constraint. The paper formalizes all biological reasoning as a Directed Acyclic Graph of 20 predefined action primitives with fixed argument schemas. This representation enforces several structural commitments: (a) all mechanistic relationships must be expressible as discrete, typed actions with specific arguments; (b) all dependencies must be directional and acyclic (no feedback loops allowed); (c) the explanation must terminate in a "measurable biological output" (induces_phenotype, regulates_expression, etc.). The paper's example in Figure 2a demonstrates a clean, linear causal chain from drug binding through pathway modulation to phenotype. However, real biological systems are characterized by feedback loops (e.g., ERK activating phosphatases that dephosphorylate MEK, creating negative feedback), compensatory mechanisms that operate in parallel (e.g., PI3K/AKT activation upon MAPK inhibition, which the example includes as a separate branch but cannot represent as a feedback interaction with the MAPK branch), combinatorial regulation (multiple transcription factors cooperatively regulating a gene), and stochastic or context-dependent effects that don't fit neatly into deterministic action sequences.

The consequence. The DAG formalism forces biological complexity into a representation that may systematically distort it. Feedback loops—ubiquitous in signaling networks—must be either omitted or approximated as feedforward effects, potentially producing explanations that are mechanistically incomplete or misleading. The requirement that every explanation terminate in a measurable output means the model must choose a single phenotypic endpoint, when in reality drug perturbations produce multiple simultaneous phenotypes with complex temporal dynamics. The action space, while covering a broad range of biological processes, may not capture important phenomena: post-transcriptional regulation beyond translation (mRNA stability, microRNA effects), protein allostery beyond simple binding events, cellular mechanical effects, metabolic flux redistribution that cannot be reduced to single enzyme–substrate conversions, or immune–cancer cell interactions beyond the cell_cell_interaction primitive. A biologist reading a VCR-Agent explanation might accept it as complete when in fact it omits known feedback mechanisms that are critical for understanding drug response (e.g., the explanation does not include the ERK→MEK negative feedback that partially counteracts MEK inhibition, which is why MEK inhibitors show limited efficacy as monotherapies).

What evidence exists in the paper. The paper gestures at this limitation through its fig. 1 example, which includes compensatory PI3K/AKT activation (node n14: modulates_pathway_activity(pathway="PI3K/AKT signaling", direction="up")) as a separate branch parallel to the MAPK branch, but does not represent this as feedback from the MAPK branch—it is simply an additional effect rather than a systemic response to MAPK inhibition. The paper does not report how often generated DAGs contain cycles that must be broken or simplified during construction, how often the model attempts to represent feedback and fails, or what fraction of known biological interactions in standard pathway databases (Reactome, KEGG) would be inexpressible in the action space. The paper acknowledges in Section 2.1 that the formalism "remains distinct from formal, interventional causal discovery," but does not discuss the specific biological phenomena that the DAG representation is structurally unable to capture.

Mitigation status. Minimally addressed. The paper's Impact Statement notes that "the generated reasoning traces are intended for mechanistic plausibility rather than formal causal discovery or direct clinical implementation," which acknowledges a limitation of scope but not a limitation of the formalism itself. There is no discussion of feedback loops, combinatorial effects, or temporal dynamics as representational challenges. The extensible verifier design is presented as a path to covering more action types, but no path is suggested for representing phenomena that don't fit the discrete-action DAG paradigm.

The Downstream Evaluation Has a Weak SFT Baseline and Missing Ablations That Undermine Causal Attribution

The assumption or constraint. The central empirical claim of the paper is that structured mechanistic explanations improve downstream prediction beyond what standard supervised learning can achieve. To test this, the paper compares SFT-Prompt and SFT-Generate (both trained with structured explanations) against vanilla SFT (trained on direct label prediction). However, this comparison does not isolate the effect of structured reasoning from the effect of having additional factual information from the retrieval pipeline. The vanilla SFT baseline receives only the perturbation description and cellular context—it does not receive the retrieved knowledge, the natural-language report, or any factual enrichment. A more informative baseline would be: SFT on the raw natural-language report (without structured formatting) to test whether the structured formalism specifically matters, or SFT on the retrieved knowledge without report synthesis to test whether knowledge grounding alone provides the benefit. Neither ablation appears in the paper.

The consequence. The 15.3 percentage point improvement of SFT-Prompt over vanilla SFT cannot be confidently attributed to the structured explanation format. It could be largely or entirely attributable to the factual enrichment provided by the retrieval pipeline—the model learns more about Binimetinib's mechanism from a 3,000-word report than from a one-sentence description, regardless of whether that report is formatted as a DAG or as prose. If conditioning on the unstructured report achieves similar downstream performance to conditioning on the structured DAG, then the paper's core contribution—the structured reasoning formalism—would not be the active ingredient driving the downstream gains; the retrieval pipeline would be. This is a critical ambiguity because the paper's framing is explicitly about the value of structured, verifiable reasoning, not about the value of knowledge retrieval. A practitioner wanting to improve perturbation prediction could achieve similar gains by simply appending retrieved biological context to the input, without implementing the structured action space, DAG construction, or verifier pipeline.

What evidence exists in the paper. The paper does not include any ablation comparing structured explanations to unstructured reports. The ablation in Appendix D.4 tests baseline LLMs given the same retrieved report as VCR-Agent, but this measures explanation generation quality (DTI score, DE score), not downstream prediction performance. There is no experiment that trains a Qwen3-4B model on the raw natural-language reports and evaluates it on TahoeQA. The ablation in Appendix D.3 compares one-step vs. two-step generation quality, not downstream utility. The closest the paper comes to addressing this is the SFT-Generate model, which learns to produce structured explanations—but this tests whether generating the structured format helps, not whether the format itself is necessary, and SFT-Generate underperforms SFT-Prompt, further complicating causal attribution.

Mitigation status. Not addressed. The paper does not acknowledge this as a missing control, does not discuss why the unstructured report baseline was excluded, and does not qualify its claims about structured reasoning in light of this gap. The paper's framing consistently attributes downstream improvements to structured reasoning, without considering the alternative explanation that factual enrichment from retrieval is the primary driver.

The Revision Model (SFT-Generate) Has Substantially Worse Performance Than Conditioning, and No Path Is Shown to Close This Gap

The assumption or constraint. The SFT-Generate configuration—where a single model (Qwen3-4B) is trained to produce the structured explanation and then predict the answer—aims to demonstrate that a model can learn to both reason mechanistically and predict outcomes. This is the more ambitious use case: rather than relying on a separate, more powerful system (Claude 4 with retrieval) to produce explanations, a single downstream model would autonomously generate its own reasoning before predicting. However, SFT-Generate achieves average DE F1 of 0.388, substantially below SFT-Prompt at 0.435—a gap of 4.7 percentage points. On the Direction of Change task, SFT-Generate (0.715) actually underperforms vanilla SFT (0.823), dropping 10.8 points—a catastrophic degradation. The paper attributes this to the difficulty of the generation task and the decision to exclude reinforcement learning due to "sparse reward signals" in classification settings.

The consequence. The large and consistent gap between SFT-Prompt and SFT-Generate means that the framework's most autonomous configuration—where a single model reasons and predicts without external explanation generation—is substantially less effective than the configuration that depends on pre-computed explanations from a more powerful system. This undermines the vision of "autonomous mechanistic reasoning" suggested by the paper's title. In practice, a deployment would need to either (a) run the expensive VCR-Agent pipeline (Claude 4 + retrieval) for every new perturbation to generate explanations for SFT-Prompt conditioning, or (b) accept the substantially lower performance of SFT-Generate. On DOC, SFT-Generate is actually worse than not using structured reasoning at all. The paper's decision to exclude RL provides a diagnosis (sparse rewards make it hard to learn explanation generation) but no solution. Future work on RL-based optimization is suggested, but the paper provides no evidence that RL can close the 4.7-point DE gap or recover the 10.8-point DOC loss—the biological reasoning domain may be fundamentally harder for RL than mathematics or coding because intermediate verification signals are sparse and noisy.

What evidence exists in the paper. Figure 6 and Table Appx.2 directly show the SFT-Prompt vs. SFT-Generate gap across both tasks and all cell lines. On DE, SFT-Generate trails SFT-Prompt on every cell line: C32 (0.412 vs. 0.470), HOP62 (0.446 vs. 0.470), HepG2/C3A (0.328 vs. 0.362), Hs 766T (0.424 vs. 0.470), PANC-1 (0.329 vs. 0.405). On DOC, the gap is even larger: SFT-Generate scores range from 0.688–0.773 across cell lines compared to 0.818–0.855 for SFT-Prompt. The paper acknowledges the RL limitation explicitly (Section 5.2): "we focus on SFT and exclude reinforcement learning (RL) from this study, as reasoning in pure classification settings often suffers from sparse reward signals... and leave RL-based optimization for future work." The appendix K ablation mentioned in the paper's text does not exist in the provided content (the appendix ends at E), so any further evidence about revision model optimization is absent from the paper as presented.

Mitigation status. Partially addressed through acknowledged future work. The paper is transparent about excluding RL and cites prior work (Wang et al., 2024; He et al., 2025; Sprague et al., 2025) on the sparse reward problem. However, the paper does not discuss alternative approaches to closing the generation gap—such as distillation from a teacher model that produces explanations, iterative refinement where SFT-Generate outputs are verified and used for further training, or hybrid approaches where a small model generates draft explanations that are verified and corrected before prediction. The 10.8-point DOC drop for SFT-Generate vs. vanilla SFT is particularly concerning and receives no specific discussion—it suggests that forcing the model to generate explanations before predicting direction of change actively harms performance on this subtask, a finding that deserves more attention than it receives.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a diagnostic reframing rather than a paradigm shift: the core insight is not that LLMs should reason about biology (many papers attempt this), but that the form of the reasoning output determines whether it can be verified, and verifiability—not expressiveness—is the binding constraint for scientific reasoning systems. This reframing has several specific consequences for how the field should approach LLM-based scientific discovery.

It reorients the conversation from "better reasoning" to "verifiable reasoning." Prior work on biological LLM reasoning (Istrate et al., 2025; Phillips et al., 2025; Wu et al., 2025) implicitly treated reasoning quality as a generation problem: if we can get the LLM to produce more detailed, more coherent explanations, they will be more useful. VCR-Agent demonstrates that even high-quality sounding explanations from strong models contain substantial factual errors—28.2% of drug-target binding claims are faulty, 87.3% of gene regulation claims contain at least one hallucinated gene—and that the limiting factor is not the LLM's ability to construct plausible narratives, but the absence of mechanisms to check those narratives against ground truth. This implies that future work on scientific LLM reasoning should prioritize verifier development over prompt engineering or model scaling. A system with perfect verifiers and mediocre generation would outperform a system with state-of-the-art generation and no verification, because the former can filter its outputs to retain only correct claims while the latter cannot distinguish fact from hallucination.

It resolves a latent tension in the literature about the role of retrieval in scientific reasoning. Prior work followed two incompatible strategies: some systems relied entirely on parametric LLM knowledge (Phillips et al., 2025), producing fluent but factually unreliable explanations; others interleaved retrieval with generation (Wu et al., 2025), which the one-step ablation in Appendix D.3 shows produces substantially worse structured outputs than separating the stages. VCR-Agent's two-stage architecture—retrieve and synthesize first, then structure—provides a design principle for resolving this tension: knowledge acquisition and reasoning formatting are competing cognitive tasks for LLMs, and forcing them to be done simultaneously degrades both. The nearly 2× improvement in DTI score (0.766 vs. 0.329) from the two-stage over the one-stage pipeline is not merely an engineering optimization; it is empirical evidence that retrieval-augmented generation needs a dedicated synthesis phase before structured output generation, at least in domains where the output format is constrained. This principle likely generalizes to other scientific reasoning tasks where the output must conform to a formal schema (chemical reaction prediction with structured product representations, clinical reasoning with structured diagnostic criteria, materials science with structured property predictions).

It identifies verifier over-optimization as a lurking failure mode and demonstrates a structural solution. The mathematical reasoning literature has documented that verifier-guided search can degrade performance through over-optimization (the reference paper from Snell et al., 2024, discussed extensively in the example analysis). VCR-Agent sidesteps this entirely by using verifiers only for error elimination, not for optimization. The DTI verifier discards traces below a threshold; it never ranks traces by verifier score or guides generation toward higher scores. This conservative design philosophy—verifiers as binary filters, not as reward functions—is a diagnostic contribution that future work on scientific verifiers should adopt unless and until verifiers are demonstrated to be robust against optimization pressure. The modest LLM-judge improvements from filtering (Appendix D.5, Table Appx.7: scientific accuracy from 0.641 to 0.645) actually support this philosophy: the verifiers catch errors that the LLM-judge cannot detect, meaning they provide a quality signal orthogonal to surface-level plausibility metrics. If verifiers were used for optimization, the LLM might learn to exploit this orthogonality—producing outputs that score well on verifiers while degrading along dimensions the verifiers don't measure.

It makes certain research directions more attractive, and others less so. The paper's results strongly motivate investment in structured output formats for scientific domains coupled with specialized per-claim verifiers, because this combination is what enables automatic quality control. Conversely, the results argue against continuing to invest in unstructured free-text explanation generation without verification—the baseline LLMs in Table 1 produce traces with DTI scores as low as 0.000 (DeepSeek-R1-8B) and DE scores as low as 0.090 (Llama3.3-70B), and even the strong Claude-Sonnet-4 baseline (0.657 DTI, 0.504 DE) would propagate errors at unacceptably high rates if used unfiltered for downstream training. The paper also argues against interleaved retrieval-generation for constrained-output tasks, given the one-step ablation results. More broadly, the paper makes scaling up retrieval coverage and verifier accuracy a higher-priority research direction than scaling model size—the downstream improvements come from having access to correct mechanistic knowledge (SFT-Prompt at 0.435 F1), not from a larger model generating better reasoning from scratch (all baseline LLMs substantially underperform VCR-Agent on DTI and DE scores despite some being much larger, e.g., Llama3.3-70B).

However, the paper also reveals sharp boundaries on what the approach can achieve that should temper expectations. The SFT-Generate model (0.388 average DE F1) substantially underperforms SFT-Prompt (0.435), and on Direction of Change, SFT-Generate (0.715) actually underperforms vanilla SFT (0.823) by 10.8 points. This means the vision of a single model that autonomously reasons and predicts—implied by the paper's title, "Towards Autonomous Mechanistic Reasoning"—is not realized by the current approach, and the 10.8-point DOC degradation suggests that forcing a model to generate explanations before predicting can actively harm performance on some tasks. The field should therefore distinguish between two distinct goals: (a) using structured reasoning as a supervision signal for downstream prediction (SFT-Prompt paradigm), which the paper demonstrates works well, and (b) building models that autonomously produce verified reasoning (SFT-Generate paradigm), which the paper shows is substantially harder and not solved. Conflating these goals risks overclaiming the paper's contributions toward autonomy.

Follow-Up Research This Work Enables

Cheap difficulty estimation for biological perturbations. The most immediate practical bottleneck is the cost of the retrieval and report generation pipeline. The paper never quantifies this cost, but generating VC-Traces from 18,950 perturbation–context pairs required two Claude 4 calls per pair plus four database queries—likely thousands of GPU-hours or API dollars. A natural follow-up would train a lightweight classifier to predict, directly from the perturbation structure and cell type name, whether a perturbation–context pair will yield high-quality structured explanations. The classifier could be trained on VC-Traces itself: for each of the 18,950 pairs, the DTI score, DE score, validity, and verifiability are already computed (the metrics in Table 1). A model that takes a SMILES string and cell type as input and predicts the expected DTI score—trained on 80% of VC-Traces and evaluated on the remaining 20%—would tell practitioners whether the expensive VCR-Agent pipeline is worth running for a given compound before investing the compute. If the classifier achieves, say, Pearson r > 0.7 with actual DTI scores, it could serve as a difficulty estimator analogous to the prompt difficulty estimation in the mathematical reasoning literature, routing easy perturbations (well-characterized drugs with clear targets) through the full pipeline and flagging hard perturbations (novel compounds with unknown targets) for alternative strategies. The ablation in Appendix D.1 showing that different retrieval sources contribute complementary information (no single source achieves DTI > 0.47 vs. 0.77 for all four) suggests that the difficulty signal would need to capture multi-source knowledge coverage, not just single-database presence.

Verifier accuracy audit through expert claim-by-claim evaluation. The paper's verifiers are assumed accurate but never validated against human expert ground truth. A critical follow-up would be a structured expert audit: sample 200–500 individual verifier decisions (e.g., 100 DTI decisions where Boltz-2 scored below threshold and the trace was discarded, 100 where it scored above and the trace was kept; similarly for DE decisions where individual genes were removed or retained), present each decision to 2–3 domain experts in molecular pharmacology or transcriptomics, and ask them to judge whether the verifier's decision was correct. This would produce a confusion matrix for each verifier with measurable false positive and false negative rates. The paper reports that 28.2% of DTI claims were excluded—if, say, 15% of those exclusions were incorrect (the drug actually does bind the target, but Boltz-2 gave a low score), that would mean ~4% of all DTI claims were correctly excluded but ~4% were incorrectly discarded—a nontrivial error rate that would need to be weighed against the benefit of removing the 24% that were genuinely faulty. Similarly, if the DE verifier removes genes that are genuinely regulated but in conditions slightly different from the Tahoe-100M experiment queried, the "refined" gene lists in VC-Traces may be systematically biased toward well-studied genes in well-studied conditions. This audit would transform the verifiers from assumed-reliable components to characterized-reliable components with documented error profiles, enabling informed decisions about filtering thresholds and downstream use cases where verifier errors are acceptable versus unacceptable.

Unstructured report baseline for downstream prediction. The paper's central claim—that structured explanations improve downstream prediction—remains confounded with the effect of factual enrichment from retrieval. A direct follow-up experiment would train Qwen3-4B on the raw natural-language reports produced by VCR-Agent's report generator (Appendix B.2 provides an example: a ~3,000-word mechanistic report for Binimetinib in C32 cells) and evaluate on TahoeQA, comparing against the SFT-Prompt model that receives the structured DAG. If the unstructured report achieves comparable F1 (say, 0.42–0.44 on DE vs. 0.435 for SFT-Prompt), then the paper's contribution would be primarily the retrieval and synthesis pipeline, with the structured format serving as a convenience for verifiability rather than a necessity for prediction. If the unstructured report substantially underperforms (say, 0.35–0.38), it would validate the paper's claim that the structured formalism specifically encodes mechanistic dependencies in a way that improves generalization—perhaps because the DAG's explicit edge list forces the model to attend to causal relationships rather than treating the report as an undifferentiated bag of facts. A third condition—training on the report after verifier-based filtering but still in unstructured form—would further disentangle whether the benefit comes from factual accuracy (filtering out wrong claims helps regardless of format) or from structural representation (the DAG format is specifically beneficial). This is a single experiment that would substantially clarify which components of VCR-Agent are necessary versus nice-to-have for the downstream performance gains.

Scaling analysis across model sizes and perturbation modalities. The paper demonstrates structured reasoning benefits on Qwen3-4B with drug perturbations in 5 cancer cell lines. A scaling analysis would test whether larger models (Qwen3-14B, Qwen3-30B, or equivalent from other families) benefit more or less from structured reasoning supervision, trained on the same VC-Traces data with the same SFT-Prompt and SFT-Generate configurations. The hypothesis space is interesting: on one hand, larger models may encode more mechanistic knowledge parametrically, reducing the marginal benefit of explicit structured supervision; on the other hand, larger models may be more capable of leveraging structured explanations because they can better integrate the mechanistic constraints with their existing knowledge. The SFT-Prompt vs. SFT-Generate gap as a function of model size would also be revealing—if the gap narrows with scale, it would suggest larger models are better at learning to generate explanations, making the autonomous reasoning vision more plausible at larger scales. A complementary scaling dimension is perturbation modality: replicate the TahoeQA experiment with genetic perturbations (CRISPR knockouts from Tahoe-100M, using loss_of_function and gain_of_function primitives in the structured explanations) to test whether the framework transfers beyond drug perturbations. The paper's action space explicitly supports these primitives, but the verifiers currently don't cover them—a follow-up would need to develop DTI-equivalent verifiers for genetic perturbations (e.g., checking whether the claimed gene is genuinely knocked out in the relevant experiment) before structured explanations for genetic perturbations could be meaningfully filtered and used as supervision signals.

Combined verifier-guided search with structured reasoning generation. The paper uses verifiers only for post-hoc filtering, explicitly avoiding optimization. But the structure of the DAG—where nodes are discrete typed actions and edges are explicit dependencies—naturally supports constrained search: at each step of DAG construction, the model could generate multiple candidate next actions, score them with the appropriate verifier (DTI for binds_to, DE for regulates_expression), and select the highest-scoring candidates to expand, analogous to beam search against a process reward model. The key difference from the mathematical reasoning setting is that different action types require different verifiers, so the search would need to route each candidate to the correct specialized verifier rather than using a single unified scoring function. A follow-up experiment would implement this as a verifier-guided tree search where, at each expansion step, the LLM proposes k candidate next actions, each is scored by its corresponding verifier (if available) or a default plausibility score (for unverified action types), and the top b actions are kept for further expansion, with the final DAG selected by some aggregation of per-action verifier scores. The ablation in Appendix D.5 showing that filtering improves LLM-judge scores modestly suggests that this search approach might find higher-quality DAGs by exploring multiple reasoning paths—but the over-optimization risk documented in the mathematical reasoning literature warns that verifier-guided search could also find DAGs that score well on individual verifiers while being globally incoherent. A first experiment would test whether beam search with verifier guidance improves DTI and DE scores over the current single-pass generation, on a held-out set of perturbations, measuring both per-action verifier scores and global LLM-judge metrics to detect over-optimization.

Cross-domain stress test: bacterial perturbations, plant biology, or rare diseases. The paper's experiments are entirely in human cancer cell lines, which benefit from the richest knowledge base coverage in biology—decades of research, well-characterized pathways, extensive drug-target databases. A stress test in a knowledge-sparse domain would reveal whether the structured reasoning framework is robust to retrieval quality degradation, or whether it fundamentally depends on dense knowledge base coverage. For example: applying VCR-Agent to predict transcriptional responses to antibiotics in bacterial species with sparse literature (e.g., Acinetobacter baumannii), or to predict gene expression changes under drought stress in a non-model plant species, or to predict drug responses in a rare genetic disease where StarkPrimeKG has only a handful of relations for the relevant genes. In these domains, the report generator would produce thinner reports with more gaps and lower-confidence claims. The question is whether the two-stage pipeline degrades gracefully (structured explanations become sparser but remaining claims are still accurate, and downstream prediction still benefits from whatever mechanistic knowledge is available) or catastrophically (the LLM hallucinates to fill gaps, verifiers cannot catch the hallucinations because reference data is also sparse, and downstream performance collapses). This experiment would establish the boundary conditions for the framework's applicability: does it require the rich knowledge ecosystem of human cancer biology, or does it generalize to the many biological domains where knowledge is fragmentary? A strong result would be maintenance of the SFT-Prompt vs. vanilla SFT gap even in sparse-knowledge domains (suggesting the framework extracts signal from whatever knowledge exists), while a null result would suggest the framework is best reserved for well-studied systems and alternative approaches are needed for knowledge-sparse biology.

Practical Applications and Downstream Use Cases

Automated mechanistic annotation of perturbation atlases. The Tahoe-100M atlas contains millions of perturbation experiments, but each is characterized primarily by numerical gene expression changes without mechanistic interpretation. VCR-Agent could be run on the entire atlas—or on focused subsets of high-interest perturbations—to generate structured mechanistic explanations for each perturbation–cell type pair, effectively annotating the atlas with why each perturbation produces its observed expression changes, not just what those changes are. The VC-Traces dataset already demonstrates this at scale: 18,950 perturbation–context pairs annotated with verified structured explanations. For a pharmaceutical company using Tahoe-100M to screen drug candidates, these annotations would enable queries like "find all compounds that modulate MAPK signaling through a mechanism similar to Binimetinib" or "find all perturbations that induce G1/S arrest through CDK4/6 downregulation"—queries that are impossible with raw expression data alone. The practical benefit is connecting the atlas's rich expression data to actionable mechanistic hypotheses, enabling researchers to move from "this compound changes expression of these genes" to "this compound likely works through this pathway because of this binding interaction" without manually reviewing literature for each hit.

Training data generation for virtual cell models that must explain predictions. The field of virtual cell modeling (Noutahi et al., 2025; Bunne et al., 2024) envisions models that not only predict cellular responses but explain their predictions mechanistically. The VC-Traces dataset provides exactly the supervision signal needed to train such models: perturbation–context pairs paired with verified mechanistic explanations. A virtual cell model could be fine-tuned on VC-Traces to generate structured explanations for novel perturbations, following the SFT-Generate paradigm. While the paper shows SFT-Generate underperforms SFT-Prompt (0.388 vs. 0.435 on DE), a dedicated virtual cell model—potentially with a larger architecture, domain-specific pretraining on biological sequences, and iterative training with verification feedback—could close this gap. The practical benefit is enabling virtual cell systems that don't just output "drug X will kill cancer cell Y" but output "drug X binds target A with affinity Kd, inhibiting pathway B, which reduces transcription of genes C, D, E, leading to phenotype F"—with each claim programmatically verified against databases, giving biologists specific, testable hypotheses rather than black-box predictions.

Prioritization and triage in drug discovery screening pipelines. In early-stage drug discovery, high-throughput screening produces thousands of hit compounds, and medicinal chemists must prioritize which to pursue. Current prioritization relies heavily on chemical structure considerations (drug-likeness, synthetic accessibility) and primary assay results (potency, selectivity). Structured mechanistic explanations add an orthogonal prioritization dimension: mechanistic plausibility and specificity. A compound predicted to work through a well-characterized, disease-relevant mechanism with high verifier scores across multiple nodes in its DAG is a stronger candidate than a compound with similar potency but a fragmentary or low-confidence mechanistic explanation, because the former provides a clearer path to optimization (you know which target engagement to measure, which downstream biomarkers to track, which resistance mechanisms to anticipate). The VCR-Agent pipeline could be run on primary screening hits, and compounds could be ranked by a composite score combining assay potency with explanation quality metrics (DTI score, DE score, fraction of nodes verified). The paper's numbers are directly applicable: if 28.2% of DTI claims are faulty and 87.3% of DE actions contain hallucinations, an unfiltered mechanistic narrative for a screening hit has a high probability of being partially wrong, potentially sending a medicinal chemistry program down a blind alley. Filtered, verified explanations substantially reduce this risk by flagging which mechanistic claims are actually supported by evidence.

Interactive hypothesis generation for bench biologists. A deployed version of VCR-Agent—with the retrieval and explanation generation wrapped behind a user interface—could serve as an interactive tool for biologists designing experiments. A researcher planning to test a new drug in a specific cell line could input the drug's structure and the cell line, receive a structured mechanistic explanation with verifier scores for each claim, and use this to identify (a) which mechanistic steps are well-supported and can be assumed in experimental design, (b) which steps are plausible but unverified (no verifier exists for that action type, or the verifier score is borderline), representing hypotheses to test, and (c) which downstream readouts (genes, phenotypes) are predicted to change, guiding assay selection. For example, for Binimetinib in C32 cells (Figure 2a), the DTI-verified binding to MAP2K1 and MAP2K2 gives confidence in the target engagement hypothesis; the DE-verified downregulation of MYC and CCND1 suggests qPCR assays to monitor transcriptional response; the unverified post_translational_modification of RB1 (node n10) and modulates_complex formation of Rb-E2F1 (node n11) represent mechanistic hypotheses that could be tested with phospho-specific antibodies or co-immunoprecipitation. The practical benefit is reducing the time from "I have a drug and a cell line" to "I know which experiments to run to validate the hypothesized mechanism," with the verifier scores providing a transparent confidence estimate that helps the biologist decide which claims to trust and which to test.