JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence

🎯 Pitch

A unified 7B model surpasses GPT-4o on multimodal coding benchmarks by synthesizing 800K cross-domain examples, proving that bridging code and vision does not require proprietary scale—only a data engine that rewards visual fidelity, not just executability.

1. Executive Summary

This paper introduces JanusCoder, a suite of foundational models that establish a unified visual–programmatic interface for multimodal code intelligence, spanning tasks from chart-to-code generation to interactive scientific demonstration authoring. The core enabler is a versatile data synthesis toolkit that leverages reciprocal synergies across data modalities to automatically curate JanusCode-800K—the largest multimodal code corpus to date—which trains JanusCoder (text-centric) and JanusCoderV (vision-centric) models at 7B to 14B scales using a pipeline of guided evolution (evolving seed instruction-code pairs under high-level conceptual keywords), re-contextualization (deepening existing instructions from verified code), reverse instruction (transforming raw code into instruction-code pairs), and bidirectional translation (translating between semantically analogous domains like Manim and Mathematica). On benchmarks spanning PandasPlotBench, ChartMimic, DesignBench, WebCode2M, InteractScience, and the newly proposed DTVBench, the JanusCoder series approaches or exceeds GPT-4o's performance, with JanusCoder-7B setting new state-of-the-art structural correctness on WebCode2M (TreeBLEU scores surpassing all proprietary counterparts) and JanusCoderV-7B substantially outperforming specialized chart-to-code models on ChartMimic, establishing that cross-domain data synergies enable strong multimodal code intelligence even at modest model scales—but only when data quality is enforced through VLM-based reward modeling rather than executability checks alone.

2. Context and Motivation

The Core Problem: Bridging Code and Vision in a General Way

The fundamental problem this paper addresses is that the field of code intelligence has remained largely fragmented across modalities and domains. While large language models have made remarkable progress on text-centric code generation—producing Python scripts, HTML pages, or algorithmic solutions from natural language instructions—the ability to work across the boundary between code and its visual outputs has developed in isolated silos. There is no generalist model that can flexibly handle the full spectrum of visual–programmatic tasks: generating a Matplotlib chart from a description, replicating a webpage from a screenshot, editing a visualization's code based on a user's instruction, or building an interactive scientific demonstration from a reference image.

This gap matters because vision is not an optional add-on for code intelligence; it is the primary way humans interact with many kinds of programs. Data visualizations, web interfaces, scientific simulations, and explanatory animations are designed to be seen and interpreted, not just executed correctly. A model that cannot connect code's symbolic logic to its visual manifestation cannot participate in the full lifecycle of these artifacts—from authoring to debugging to iterative refinement. The paper's opening figure (Figure 1) illustrates this breadth: the same underlying capability to map between code and vision is needed for chart-to-code, WebUI generation, animation creation, and scientific demonstration authoring. Yet prior work has treated each as a separate problem requiring separate models and separate datasets.

The authors frame this as establishing a "visual–programmatic interface"—a modeling paradigm where code generation, visual understanding, and cross-modal reasoning are unified rather than compartmentalized. This is not merely a convenience argument; it is a hypothesis about what enables generalization. If the same model learns to generate Python plotting code from text descriptions and HTML from screenshots and Mathematica visualizations from Wolfram demonstrations, the cross-domain exposure should create transferable representations that benefit each individual task—what the paper calls "cross-domain synergies" (Section 3.3).

Why This Matters: Real-World and Research Significance

The practical significance of this problem is immediate and growing. Several converging trends make multimodal code intelligence critical:

The rise of AI-assisted data visualization. Scientific workflows, business analytics, and journalism increasingly demand automated chart generation from data specifications or reference images. Benchmarks like ChartMimic (Yang et al., 2025b) and PandasPlotBench (Galimzyanov et al., 2025) formalize this need, but existing models either handle text-to-code or image-to-code, rarely both, and almost never with the same architecture.

The demand for automated front-end engineering. Web development involves tight coupling between visual design and code implementation. As demonstrated by WebCode2M (Gui et al., 2025) and DesignBench (Xiao et al., 2025), the ability to generate HTML/CSS from screenshots, or edit existing pages based on visual instructions, has direct economic value. Current approaches require separate models for generation versus editing, and for text-driven versus vision-driven inputs.

Emerging needs for interactive scientific communication. The paper's proposed DTVBench highlights a frontier application: translating mathematical theorems and scientific concepts into dynamic, code-driven animations (Manim) or interactive visualizations (Wolfram Mathematica). This is what creators like 3Blue1Brown do manually—building explanatory visualizations of concepts from "Attention Is All You Need" to "Gibbs free energy." Automating this requires a model that understands both the abstract concept and the concrete code syntax, plus the visual conventions that make animations pedagogically effective.

The gap between execution correctness and visual quality. A crucial insight motivating this work is that compilable code ≠ good visual output. A Matplotlib script can run without errors yet produce a chart with misaligned labels, wrong colors, or misleading scales. Traditional code intelligence metrics (pass@k, execution rate) are blind to this. The paper argues for evaluation regimes that assess visual fidelity, instruction alignment, and perceptual quality—dimensions that require models to understand code through its visual outputs, not just its syntax.

From a research perspective, the paper addresses a deeper question: Can a single model architecture, trained on a sufficiently diverse and well-curated corpus, serve as a foundation for all visual–programmatic tasks? This is the multimodal-code analog of the foundational model paradigm that has reshaped NLP and computer vision. If successful, it would mean that progress on, say, chart-to-code translation could directly improve WebUI generation, without task-specific architectures or training pipelines—a more efficient and principled path forward than the current landscape of siloed solutions.

Prior Approaches and Their Limitations

The paper identifies several categories of prior work, each of which falls short of the unified vision–programmatic interface:

1. Text-centric code generation with visual outputs. Early work focused on LLMs generating plotting code (Python libraries like Matplotlib, Seaborn) from natural language (Yang et al., 2024; Zhang et al., 2024b; Sun et al., 2025b), or creating web artifacts from text descriptions (Zhang et al., 2025; Cheng et al., 2024). These approaches demonstrate that LLMs can author executable visual content, but they are restricted to text-driven inputs. They cannot incorporate visual references—a screenshot of a chart to replicate, an image of a webpage to edit—which limits their utility in real workflows where users often start from visual examples rather than written descriptions.

2. Vision-grounded code generation and understanding. A complementary line of work emphasizes multimodal inputs, where models interpret visual information to produce or reason about code. Representative efforts include chart understanding and chart-to-code generation (Zhao et al., 2025b; Xia et al., 2025; Wu et al., 2025), theorem visualization (Ku et al., 2025), multimodal algorithmic problem solving (Li et al., 2024), and structured vector graphics generation (Yang et al., 2025c; Nishina & Matsui, 2024). While these demonstrate progress, they share a critical limitation: each targets an isolated domain. A model trained for chart-to-code knows nothing about WebUI generation; a model for SVG editing cannot produce Matplotlib plotting code. The paper contrasts this with its own approach explicitly:

"we move beyond these constraints by unifying diverse domains and modalities across charts, web UIs, animations, symbolic computation, and more, taking a leap forward in advancing multimodal code intelligence." (Section 2)

3. Specialized models for isolated tasks. The paper emphasizes that even for well-explored scenarios, "existing works often build specialized models for isolated targets (e.g., one for chart-to-code, another for WebUI-to-code)" (Section 1). This leads to models that "can neither generalize across scenarios nor scale effectively." The specialization is not just inconvenient—it precludes the cross-domain knowledge transfer that the paper argues is essential for data-scarce domains like scientific demonstration generation or animation authoring.

4. The data scarcity bottleneck. Perhaps most fundamentally, the paper argues that progress has been impeded by "the scarcity of high-quality, diverse multimodal code data" (Section 1). The heterogeneity of content requirements is daunting: different programming languages (Python, HTML/CSS, Wolfram Language, SVG, R, MATLAB), different styles of natural language instructions (vague user requests, precise technical specifications), and a vast array of visual outputs (static charts, interactive widgets, extended animations, rendered webpages). Existing corpora are either narrowly domain-specific (e.g., chart-only datasets), lack visual validation, or suffer from insufficient instruction diversity. The paper notes that "creating a comprehensive corpus that covers this spectrum is a formidable task. It requires not only large-scale data collection and processing but also well-matched validation environments (e.g., computation/rendering engines), and rigorous quality control over the diverse visual contents" (Section 1).

5. The executability trap. A recurring failure mode in prior work is treating executability as a sufficient signal of data quality. The paper highlights that "while a program may pass compiler or rendering checks, its actual visual output can drastically diverge from user instructions or requirements" (Section 3.4). Prior data pipelines that filter by executability alone admit samples where code runs but produces visually misaligned, incomplete, or irrelevant outputs. The paper's introduction of VLM-based reward modeling directly addresses this gap—evaluating rendered outputs holistically across task relevance, completion, code quality, and visual clarity rather than binary pass/fail.

How This Paper Positions Itself

The paper positions itself as bridging the perception–symbolic gap through three coordinated contributions:

First, a data-first philosophy. Rather than proposing a novel model architecture, the paper argues that the primary bottleneck is data—its quantity, diversity, quality, and cross-domain coverage. The synthesis toolkit is therefore the central methodological contribution, designed to be "versatile" and "scalable" in ways that prior data curation efforts were not. By releasing the toolkit alongside the corpus, the paper aims to "significantly reduce the engineering efforts required for data curation in future research" (Section 1).

Second, a unification agenda. The paper explicitly contrasts with the specialization paradigm that dominates prior work. The goal is not "JanusCoder for charts" plus "JanusCoder for WebUI"—it is a single interface that handles text-centric and vision-centric tasks, generation and editing, static and dynamic outputs. The model naming reflects this: JanusCoder (text-centric) and JanusCoderV (vision-centric) are variants of the same underlying approach, distinguished by which data they train on, not by architectural divergence.

Third, a quality-over-executability standard. The paper's emphasis on VLM-based reward modeling is a deliberate departure from prior work that treats execution success as the gold standard. By introducing multi-dimensional quality assessment (task completion, solution coherence, visual clarity, task relevance) scored via frontier VLMs, the paper establishes a higher bar for what constitutes acceptable training data. This is not merely a filtering step—it fundamentally shapes the corpus composition, removing executably correct but visually poor samples that would degrade model learning.

The positioning can be understood through what the paper is not: it is not a new vision-language architecture, not a novel decoding or search strategy, not a fine-tuning recipe for a specific benchmark. It is a data-centric foundation for a new subfield—multimodal code intelligence as a general capability rather than a collection of specialized tasks. The claim is that if the data is sufficiently broad, diverse, and carefully curated, existing model architectures (Qwen3, Qwen2.5-VL, InternVL3.5) can be trained to handle the full spectrum of visual–programmatic tasks, achieving results that approach or surpass both specialized models and commercial systems like GPT-4o.

This framing echoes the foundational model philosophy that has transformed NLP—the idea that a single model, trained on a sufficiently large and diverse corpus, can serve as a baseline for many downstream tasks. The paper extends this philosophy across the code–vision boundary, arguing that the same data diversity that benefits general code intelligence (Sun et al., 2023; 2024a) applies with equal force to multimodal settings. The release of JanusCode-800K—the largest corpus of its kind—is intended to enable the community to build on this foundation, much as ImageNet enabled computer vision or The Stack enabled code LLMs.

3. Technical Approach

3.1 Reader Orientation

The system being built is a data synthesis toolkit and training pipeline that produces a unified model capable of generating executable code from natural language instructions, visual inputs (screenshots, reference images), or combinations of both, where the generated code produces visual outputs—charts, webpages, animations, interactive demonstrations—that are both functionally correct and visually faithful. The core problem is the scarcity of high-quality, diverse multimodal code data, and the solution takes the shape of a multi-strategy data synthesis engine that systematically generates instruction-code pairs and instruction-code-vision triplets across heterogeneous domains, filters them through execution validation and VLM-based quality assessment, and then trains standard vision-language models on the resulting corpus to serve as a general-purpose visual–programmatic interface.

3.2 Big-Picture Architecture (Diagram in Words)

The system has three major stages, each containing multiple components:

Stage 1: Data Sourcing and Collection — raw assets (code repositories, public datasets, web corpora) are aggregated and categorized into two formats: paired data (instruction-code pairs, optionally with visual outputs) and code-only data (unlabeled code snippets). Complex long-form code files (e.g., Manim animation scripts) are decomposed into semantically coherent units using Abstract Syntax Tree (AST) parsing.

Stage 2: Data Synthesis and Curation — four synthesis strategies operate on the sourced data to produce new training samples: Guided Evolution (evolving seed pairs under high-level conceptual guidance), Re-contextualization (deepening existing instructions from verified code), Reverse Instruction (transforming raw code snippets into instruction-code pairs), and Bidirectional Translation (mapping between semantically analogous domains like Manim and Mathematica). Every synthesized code sample is validated through formal execution in a sandbox environment; failures are rerouted to the synthesis engine for refinement.

Stage 3: Quality Control — a reward modeling pipeline employing VLMs (for vision-centric data) or LLMs (for text-centric data) scores each sample across four dimensions (task relevance, task completion, code quality, visual clarity) on a 1–5 scale. Only samples exceeding a predefined score threshold are retained.

The output of this pipeline is JanusCode-800K, a corpus containing approximately 407K text-centric samples and 393K vision-centric samples. This corpus trains two model variants: JanusCoder (trained on text-centric data only, using Qwen3 backbones) and JanusCoderV (trained on the full corpus, using Qwen2.5-VL and InternVL3.5 backbones).

3.3 Roadmap for the Deep Dive

First, the data collection and AST-based preprocessing pipeline, because it establishes the raw material types (paired vs. code-only) and the decomposition strategy that makes complex source files usable for training.
Second, the four synthesis strategies in detail—Guided Evolution, Re-contextualization, Reverse Instruction, and Bidirectional Translation—since these are the core mechanisms that transform raw assets into diverse, high-quality training samples across heterogeneous domains.
Third, the execution validation sandbox, which acts as a first-pass quality filter by ensuring every synthesized code sample actually runs and produces a visual output, and how samples that fail this check are rerouted for refinement.
Fourth, the cross-domain synergy principle, because it explains the deliberate design choice to transfer knowledge between semantically related domains (e.g., R code reinforcing Mathematica tasks) and across modalities (e.g., visual outputs of Python tasks constructing chart-to-code data), which is essential for data-scarce specialized domains.
Fifth, the reward modeling pipeline for quality control, covering how VLMs and LLMs evaluate samples across multiple dimensions, how thresholds are set, and why executability alone is an insufficient quality signal.
Sixth, the training configuration and model variants, including backbone choices, hyperparameters, and the distinction between JanusCoder and JanusCoderV.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a data-centric systems paper whose core idea is that a unified visual–programmatic interface for code intelligence can be achieved not through novel architecture design, but through a principled data synthesis and curation pipeline that produces training data spanning diverse domains, modalities, and tasks, with rigorous quality control that goes beyond executability to assess visual fidelity and instruction alignment.

3.4.1 Data Collection and Categorization

What happens first. The pipeline begins by aggregating raw data from a deliberately heterogeneous set of sources. The paper lists: "large-scale public datasets (e.g., StackV2; Lozhkov et al., 2024), extensive web corpora (e.g., WebCode2M; Gui et al., 2025), specialized knowledge bases like the Wolfram Demonstrations Project, and competitive programming problems (Xu et al., 2025; Sun et al., 2025a)." The detailed breakdown in Appendix B (Table 6) specifies quantities: 200,000 Python visualization samples from VisCoder (Ni et al., 2025), 77,000 chart-to-code samples also from VisCoder, 210,000 samples from VisCodex (Jiang et al., 2025), 70,000 algorithm samples from CodeEvo (Sun et al., 2025a), 129,000 algorithm samples from VisCodex, 68,778 Manim animation samples from a 3Blue1Brown video dataset, 414 Manim samples from the Kaggle Manim Dataset, 20,000 SVG samples from MMSVG-Icon and MMSVG-Illustration (Yang et al., 2025c), 500,000 scientific PL samples from TheStackV2 (Lozhkov et al., 2024), 200,000 WebUI generation samples and 69,501 WebUI editing samples from WebCode2M (Gui et al., 2025), plus additional assets from the WebDev Arena and Wolfram Demonstrations Project.

All sourced data is classified into two primary formats. The first is Paired Data, denoted $\mathcal{D}_{\text{paired}}$ , which contains instruction-code pairs $(I, C)$ . When a visual output is available, it is included as an optional third component, forming a triplet $(I, C, V)$ . The second is Code-Only Data, denoted $\mathcal{D}_{\text{code}}$ , which consists solely of code snippets $C$ with no associated instruction or visual output.

The AST-based decomposition for long-form code. A significant practical challenge arises within $\mathcal{D}_{\text{code}}$ : "the long-form, complex code files, such as a single Manim script that generates a 5-minute-long mathematical animation." A single 5-minute animation script may contain dozens of semantically distinct conceptual steps (creating shapes, animating transformations, adding text labels, transitioning between scenes), but training a model on the entire monolithic file would not teach it to map specific instructions to specific visual effects. To address this, the paper employs "a sophisticated decomposition strategy utilizing Abstract Syntax Trees (AST)."

The procedure, detailed in Appendix A.1, works as follows:

AST-based static analysis: Each .py script is parsed into an Abstract Syntax Tree without executing it, ensuring reproducibility and avoiding environment-specific dependencies. An AST is a tree representation of the syntactic structure of source code, where each node represents a construct occurring in the source (e.g., a function definition, a method call, a class declaration). Parsing without execution means the pipeline can process code that might depend on unavailable libraries or specific runtime environments, since it only examines structure, not behavior.
Scene identification: Within the AST, the system detects classes that inherit from canonical Manim base classes such as Scene and ThreeDScene. Inheritance is determined by examining the class definition node's base classes in the AST. For each scene class found, the system locates its construct() method, which encodes the primary animation logic—this is the standard entry point in Manim where all animation instructions are sequenced.
Feature extraction: The system traverses the body of the construct() method to extract semantically meaningful features. These include instantiated objects (e.g., Circle, Text—the visual elements that appear in the animation), invoked Animations (e.g., Create, Write—the actions that manipulate those elements), and embedded textual content (any text that appears on screen). The system also records import statements and captures concise code excerpts while filtering out project-specific dependencies such as manim_imports_ext, which would not be available in a general training environment.
Data structuring: The extracted elements are consolidated into structured JSONL entries. Each entry contains the file identifier, the scene class name, the extracted features (objects, animations, text), and a prompt template. This representation "preserves the semantic intent of the animation in a format suitable for our data toolkit"—meaning each self-contained scene becomes a unit that can later be paired with a synthesized instruction describing what it does.

Design choices in collection. The authors deliberately cast a wide net across heterogeneous sources rather than collecting narrowly from a single domain. This is motivated by the cross-domain synergy hypothesis (Section 3.3)—that exposure to diverse programming languages and visual output types creates transferable representations. The inclusion of scientific PLs (R, MATLAB), symbolic computation (Wolfram Mathematica), animation (Manim), and vector graphics (SVG) alongside the more common Python visualization and WebUI data is what distinguishes JanusCode-800K from prior corpora that focus on one or two domains.

3.4.2 Synthesis Strategy 1: Guided Evolution

What it is. Guided Evolution adapts the interaction-driven synthesis approach from the authors' prior work (CodeEvo; Sun et al., 2025a) to the multimodal code domain. The goal is to increase data complexity and diversity beyond what exists in the seed corpus by evolving existing instruction-code pairs under high-level conceptual guidance.

How it works. Starting with a seed triplet $(I, C) \in \mathcal{D}_{\text{paired}}$ , the evolution is guided by a high-level concept $K$ , which is represented as keywords (e.g., a specific chart type) or a web meta-task (e.g., "add a widget"). The meta-task abstraction is defined in Appendix A.2 as "an abstract, canonicalized edit operation on a web page that captures the essential type of user intent while remaining agnostic to the specific context, location, or wording." For example, "Change the color of a button" is a meta-task; it can be instantiated into concrete instructions like "Change the submit button to green" or "Make the navigation buttons blue" depending on the specific page context.

The new instruction is generated via: $I' = f_{\text{evolve}}(I, C, K)$ where $f_{\text{evolve}}$ is the instruction evolution function (implemented via gpt-oss-120b), $I$ is the original instruction, $C$ is the original code, and $K$ is the high-level concept that guides the direction of evolution.

What it computes: the function takes an existing instruction-code pair and a conceptual guidance signal, and produces a new instruction that is grounded in the original task but modified according to the concept. For instance, given a seed pair that describes "a bar chart showing monthly sales," and a concept keyword "scatter plot," the function might generate "a scatter plot showing the relationship between advertising spend and monthly sales."

Why this form: previous heuristic-based evolution methods (the paper cites WizardLM; Xu et al., 2024a) apply generic transformations like "make it more complex" without domain-specific guidance. For visual coding tasks, such blind evolution would frequently produce instructions that are incoherent or impossible to render (e.g., asking for a chart type that doesn't match the data structure). The concept-guided approach ensures that evolved instructions remain grounded in executable visual specifications—the concept $K$ constrains the evolution to meaningful variations within the target domain.

After generating $I'$ , the model produces code $C'$ for the new instruction. This code is then validated in an execution environment $\mathcal{E}$ , producing a verification result $V' = \text{Exec}(C', \mathcal{E})$ . The feedback from this validation—whether the code executed successfully and what visual output it produced—drives the next synthesis iteration. Samples that fail execution are rerouted to the synthesis engine for retry and refinement, creating an iterative loop that converges toward executable, instruction-aligned outputs.

Why iterative feedback matters. The retry mechanism is essential because the initial code generation may fail for many reasons: missing imports, incorrect API usage, structural errors in the generated visualization. By feeding execution errors back into the generation process, the system can produce corrected versions rather than discarding the sample. This is analogous to the "self-refinement" paradigm in code generation, but integrated directly into the data synthesis pipeline rather than applied at inference time.

3.4.3 Synthesis Strategy 2: Re-Contextualization

What it is. Re-contextualization enhances the semantic quality of existing paired data without synthesizing new code. The insight is that many existing instruction-code pairs have imprecise or underspecified instructions relative to what the code actually implements. Strengthening the instruction to better reflect the code's actual behavior produces higher-fidelity training pairs.

How it works. For a given pair $(I, C) \in \mathcal{D}_{\text{paired}}$ , the function $f_{\text{recontext}}$ performs a deep analysis of the code $C$ to uncover implicit logic, edge cases, or contextual details not specified in the original instruction $I$ . It then generates a more descriptive and precise instruction: $I' = f_{\text{recontext}}(I, C)$ where $f_{\text{recontext}}$ is the re-contextualization function (also implemented via gpt-oss-120b), which takes the original instruction and the verified code as input.

What it computes: the function reads the code to understand its actual behavior—what chart type it produces, what colors it uses, what edge cases it handles, what data transformations it applies—and rewrites the instruction to explicitly mention these details. For example, if the original instruction said "plot the data" but the code creates a normalized stacked area chart with a custom color palette and annotated outliers, the re-contextualized instruction would describe all of these specifics.

Why this form rather than synthesizing new code. The primary advantage is efficiency: "it creates a higher-fidelity pair $(I', C)$ by improving the quality of the instruction without the computational overhead of synthesizing and validating entirely new code." The code $C$ is already verified as executable and correct—it has passed through the execution sandbox. Rather than risking the introduction of errors through code regeneration, the strategy improves the weaker half of the pair (the instruction) while preserving the stronger half (the code). This ensures that the model is trained on a "semantically richer dataset where language and code are more tightly aligned."

The paper does not specify the exact prompt or few-shot examples used for re-contextualization, but the principle is clear: leverage the information asymmetry between a terse user instruction and the detailed implementation it produced to create training data that teaches the model the mapping from detailed specifications to precise implementations.

3.4.4 Synthesis Strategy 3: Reverse Instruction

What it is. Reverse Instruction transforms raw code snippets into aligned instruction-code pairs. This strategy addresses the fundamental asymmetry in data availability: there is vastly more open-source code than there are high-quality instruction-code pairs. The paper explicitly states that this approach is "inspired by prior practices that exploit large-scale open-source code to synthesize realistic tasks," citing Magicoder (Wei et al., 2024).

How it works. Given a reference file $C_{\text{ref}} \in \mathcal{D}_{\text{code}}$ , a snippet of $K$ lines $C_{\text{sample}}$ is sampled and passed to a function $f_{\text{reverse}}$ to produce a plausible natural language instruction: $I'' = f_{\text{reverse}}(C_{\text{sample}})$ where $f_{\text{reverse}}$ is implemented via gpt-oss-120b.

What it computes: the function reads the sampled code snippet—which could be an R data analysis script, a MATLAB signal processing routine, a Mathematica symbolic computation, or any other code from the scientific PL corpus—and generates an instruction that could plausibly have motivated writing that code. For instance, if $C_{\text{sample}}$ contains code that performs a Fourier transform and plots the frequency spectrum, $f_{\text{reverse}}$ might generate "Analyze the frequency components of a signal using a Fourier transform and visualize the power spectrum."

After generating the instruction, a model then produces $C'$ conditioned on $I''$ , optionally leveraging $C_{\text{ref}}$ as broader context. This means the generated code $C'$ may differ from the original $C_{\text{sample}}$ —the model is not simply memorizing the snippet but generating a fresh implementation that satisfies the synthesized instruction. The paper emphasizes that "this pipeline enables the systematic repurposing of theorems and data analysis code from scientific PLs like R and Matlab into instruction-following samples $(I', C')$ , effectively populating our dataset with a rich variety of domain-specific tasks."

Design choice: optional reference context. The paper notes that $C_{\text{ref}}$ (the broader file from which $C_{\text{sample}}$ was extracted) can optionally be provided as context during code generation. This is important for cases where the sampled snippet relies on helper functions, data loading routines, or configuration defined elsewhere in the same file. Without this context, the generated code might be incomplete or non-executable. The optional nature of this context balances completeness (using the full reference) against realism (in real usage, users don't provide the full implementation context).

Why this strategy matters for data coverage. The paper's data sources include 500,000 samples from TheStackV2 for scientific PLs alone. Directly training on these as code-only data would teach syntax but not instruction-following. Reverse Instruction converts these raw code assets into the instruction-code format needed for supervised fine-tuning, dramatically expanding coverage into programming languages and domains where human-written instruction-code pairs are scarce.

3.4.5 Synthesis Strategy 4: Bidirectional Translation

What it is. Bidirectional Translation fosters the learning of abstract, syntax-independent representations by translating conceptual intent between semantically analogous domains. The paper gives the specific example of translating between Manim (Python animation library) and Mathematica (Wolfram symbolic computation), two domains that can express similar mathematical concepts through different programming paradigms.

How it works in the forward direction. Given a sample $(I_A, C_A)$ from a source domain $A$ , a new instruction for the target domain $B$ is first generated: $I_B = f_{\text{translate}}(I_A)$ where $f_{\text{translate}}$ extracts the conceptual intent from the source domain instruction and re-expresses it in the target domain's idiom. For instance, an instruction to "animate a particle moving along a parametric curve" in Manim might be translated to "create an interactive Manipulate that traces a particle along a parametric curve" in Mathematica.

Subsequently, the model generates the target code $C_B$ that uses the source code $C_A$ as a structural template: $C_B = f_{\text{translate}}(I_B, C_A)$ where the function now takes both the translated instruction and the source code as inputs. The source code provides a structural template—the sequence of operations, the parameterization of the curve, the timing or interactivity logic—that the model adapts to the target domain's syntax and API.

What it computes: The bidirectional translation effectively creates a parallel corpus of equivalent programs in different languages, similar to how machine translation creates parallel corpora of equivalent sentences in different languages. A Manim animation of a damped harmonic oscillator and its Mathematica equivalent express the same physical phenomenon through different code; by training on both, the model learns that the concept "damped harmonic oscillator" maps to a particular code structure in each language, abstracting away from syntax-specific details.

Why bidirectional. The process is fully bidirectional—Mathematica samples can be translated to Manim as well. This pragmatically addresses "the challenge of generating complex code from scratch": it is far easier for a model to translate an existing structurally correct implementation from one syntax to another than to generate a complex program from a natural language description alone. The structural template provides a scaffold that constrains the generation problem, making it more tractable.

Why this matters for data-scarce domains. The paper notes that some domains (e.g., Mathematica, Manim) have relatively small seed corpora compared to Python or WebUI (Table 6 shows only 5K animation samples and 11K Mathematica samples post-synthesis). Bidirectional translation effectively multiplies the value of these specialized datasets by creating cross-domain equivalents. A single Manim animation can generate a Mathematica sample and vice versa, doubling the effective data coverage in both domains.

3.4.6 Execution Validation Sandbox

What it is and where it fits. After any of the four synthesis strategies produces new code $C'$ , that code must pass through a formal execution validation step before proceeding to quality control. The paper describes this as: "We leverage a sandbox $\mathcal{E}$ that provides the necessary backends (e.g., Python interpreters, web renderers)."

How it works. Every newly generated code sample $C'$ must pass through a formal execution function: $V' = \text{Exec}(C', \mathcal{E})$ where $\mathcal{E}$ is the execution environment appropriate to the programming language (Python interpreter for Matplotlib/Manim, Playwright browser automation for WebUI/HTML, the Wolfram Engine for Mathematica, appropriate runtimes for R and MATLAB). The output $V'$ is either a successful visual output (a rendered chart image, a webpage screenshot, an animation frame) or a failure signal (compilation error, runtime exception, timeout).

The feedback loop. The critical design choice is what happens when validation fails: "Samples that fail this validation are rerouted to the synthesis engine for retry and refinement." This means the synthesis pipeline is iterative—code that doesn't execute is not simply discarded; instead, the execution error is used as feedback to guide a second attempt at code generation. This is a form of "self-correcting" data generation that increases yield from the synthesis process. The paper does not specify the maximum number of retry attempts or the exact mechanism by which execution errors are fed back into the generation prompt, but the principle is that validation is not just a filter but an active component of the synthesis loop.

Why execution alone is insufficient. The paper emphasizes that "executability alone is an insufficient proxy for the quality of the generated visual content." A program may pass compiler or rendering checks—it runs without errors and produces some visual output—yet "its actual visual output can drastically diverge from user instructions or requirements." For instance, a Matplotlib script might run successfully but produce a chart with wrong axis labels, misaligned legends, or colors that don't match the specification. This is why execution validation is positioned as a necessary but not sufficient condition, preceding the more sophisticated reward modeling stage that assesses visual quality.

Multiple backends for heterogeneous domains. The sandbox must support fundamentally different execution models: Python scripts that run once and produce static images; HTML/CSS/JavaScript that renders in a headless browser and may involve user interactions; Wolfram Language that runs in the proprietary Wolfram Engine; Manim scripts that produce video files. The paper mentions Playwright specifically for web rendering, which is a browser automation library that can load an HTML page, execute its JavaScript, and capture screenshots of the rendered result. This diversity of backends is necessary precisely because the corpus spans such heterogeneous visual output types.

3.4.7 Cross-Domain Synergies

What it is. The cross-domain synergy principle is a design philosophy rather than a separate processing step: "Rather than treating data sources in isolation, we deliberately exploit synergies across heterogeneous domains and modalities. The central idea is that knowledge can be transferred between semantically related domains (e.g., R code reinforcing Mathematica tasks) and across different modalities (e.g., the visual output of a Python data visualization task can be used to construct chart-to-code data)."

How it manifests in practice. The paper provides concrete examples of cross-domain transfer in the synthesis strategies:

Scientific computing to animation: "The wealth of scientific computing logic in R and Matlab corpora is generalized to synthesize new data for Manim and Mathematica using our Reverse Instruction and Bidirectional Translation strategies." R and MATLAB code for numerical analysis, signal processing, or statistical modeling expresses computational concepts that are equally relevant in animation (e.g., parametric curves, Fourier synthesis, particle systems). By translating these concepts into Manim syntax, the pipeline creates animation training data from sources that have nothing to do with animation originally.
WebDev to scientific demonstration: "Foundational data from WebDev, including HTML and SVG code, provides a robust basis for generating complex, interactive scientific demonstrations." The structural knowledge of how to build interactive web interfaces (DOM manipulation, event handling, CSS layouts) transfers to building interactive scientific widgets, even though the domain knowledge (physics, chemistry, biology) is different. The paper reports that this synergy is "crucial for broadening task diversity and strengthening model generalization."
Text-centric to vision-centric: "The visual output of a Python data visualization task can be used to construct chart-to-code data." A Python visualization sample $(I, C)$ that passes execution validation produces a visual output $V$ (the rendered chart). This $V$ can then be paired with the code $C$ to create a chart-to-code training sample $(I, C, V)$ where the model learns to generate code from a visual reference. This is a modality transformation that requires no additional code synthesis—it simply captures the visual output of already-verified code and repurposes it for a different task format.

Why this matters for data-scarce domains. The paper explicitly claims that "this approach is highly effective for mitigating data scarcity in specialized areas, such as scientific demonstration." Domains like interactive scientific demonstrations or Manim animations have orders of magnitude less raw data available than Python visualization or web development. Direct data collection in these domains would be prohibitively expensive, both in terms of finding qualified human authors and in terms of the domain expertise required to write correct code. Cross-domain transfer provides a mechanism to bootstrap training data in these sparse domains by leveraging knowledge from more abundant, semantically related domains.

The validation in Section 6.1. The paper tests this claim through ablation studies (Table 5), which we have not yet analyzed in detail but which the technical approach section should reference as the empirical grounding for why cross-domain synergy is designed into the pipeline rather than being a post-hoc rationalization. The removal of specific data categories consistently degrades performance on tasks outside those categories, confirming that the transfer is real and not merely aspirational.

3.4.8 Reward Modeling for Quality Control

What it is and why it's needed. The reward modeling pipeline is the final quality filter that distinguishes JanusCode-800K from prior corpora that rely solely on executability checks. The paper frames this sharply: "While our synthesis pipeline generates substantial executable text-centric and vision-centric code, executability alone is an insufficient proxy for the quality of the generated visual content." The reward model addresses this by systematically assessing and filtering out misaligned or low-quality data at scale.

The reward model architecture. The reward model employs a VLM as its core engine to assess the quality of data, though different backbones are used for different data types. For vision-centric data (Python visualizations, rendered webpages), the paper uses Qwen2.5-VL-72B-Instruct (Bai et al., 2025). For text-centric data lacking visual outputs (e.g., Mathematica code, algorithmic implementations), the paper uses Qwen3-235B-A22B (Yang et al., 2025a), an LLM rather than a VLM.

The two-stage evaluation process. The reward process, denoted by the function $\mathcal{R}$ , takes three inputs: the natural language instruction $I$ , the generated code $C$ , and the resulting visual output $V$ . These elements are organized within a structured prompt that guides the VLM through two stages:

Stage 1: Task Understanding. The VLM summarizes its interpretation of the instruction, analyzing what the user actually requested. This stage forces the model to articulate its understanding before scoring, making the subsequent evaluation more interpretable and reducing the risk of the model scoring based on surface features rather than semantic alignment.

Stage 2: Multi-dimensional Rating and Scoring. The VLM rates the sample across four key metrics, each assigned an integer score on a scale of $[1, 5]$ :

Task Completion (denoted $s_{\text{complete}} \in \{1, 2, 3, 4, 5\}$ ): Measures the extent to which the final visual output and code successfully fulfill all aspects of the instructed task. This includes accuracy (does the image correctly represent the data and adhere to specified chart types, labels, titles?) and completeness (are all parts of the instruction addressed?).
Solution Coherence and Code Quality (denoted $s_{\text{code}} \in \{1, 2, 3, 4, 5\}$ ): Assesses the logic, efficiency, and readability of the generated code. For vision-centric data, this checks whether the code follows a logical sequence, uses appropriate libraries correctly, and is well-structured. For text-centric data, this evaluates programming best practices.
Visual Clarity (denoted $s_{\text{visual}} \in \{1, 2, 3, 4, 5\}$ ): Assesses the aesthetic and communicative quality of the final visual output. This covers readability (are fonts, colors, and labels clear?), aesthetics (is the visualization well-designed and visually appealing?), and layout (is the output balanced and free of clutter?).
Task Relevance (denoted $s_{\text{relevance}} \in \{1, 2, 3, 4, 5\}$ ): Measures the practical, real-world value of the assigned task. This is a meta-evaluation: does the instruction represent a realistic and useful scenario? Does the task serve as a meaningful benchmark?

Computing the final score. The final reward score $S$ is calculated as: $S = \mathcal{R}(I, C, V) = \frac{s_{\text{complete}} + s_{\text{code}} + s_{\text{visual}} + s_{\text{relevance}}}{4}$

where $s_{\text{complete}}$ , $s_{\text{code}}$ , $s_{\text{visual}}$ , and $s_{\text{relevance}}$ are the four dimensional scores.

What it computes: the arithmetic mean of four expert judgments, each on a 1–5 integer scale. The result is a scalar score in $[1, 5]$ that summarizes the overall quality of the $(I, C, V)$ triplet across execution-independent quality dimensions. Only data samples whose score $S$ exceeds a predefined threshold are retained in the final corpus.

Why this form rather than binary pass/fail. A binary executability filter would admit many low-quality samples (correctly executing but visually misaligned code) while rejecting some high-quality samples (code that fails execution due to environment mismatches rather than logic errors). The multi-dimensional 1–5 scale provides a more nuanced signal: a sample might score high on task completion (the right chart type was generated) but low on visual clarity (the labels overlap), and the aggregate score reflects this tradeoff. The use of four independent dimensions also prevents the model from collapsing quality assessment to a single axis (e.g., "does it look pretty?") and forces consideration of code quality and task realism.

The threshold choice. The paper states that for text-centric data evaluated by LLMs, "only samples achieving a score of 5 are retained to ensure high-fidelity instruction following in the final dataset" (Appendix C.1). For vision-centric data, the threshold varies by task type. For visual editing tasks specifically, Appendix C.2 describes "a binary filtering strategy" where "samples are marked as valid only if they achieve a minimum threshold (≥3) across key dimensions, including Task Completion and Visual Clarity." This asymmetric thresholding reflects the different difficulty distributions: text-centric data evaluated by an LLM can achieve perfect instruction alignment more readily than vision-centric data where visual subtleties introduce inevitable imperfection.

Cross-model adjudication validation. To verify that the reward scores are not biased by the specific judge model used, the paper conducted a cross-model adjudication study (Appendix G.3). They randomly sampled 30K unfiltered data samples and evaluated them using three distinct frontier models: Qwen2.5-VL-72B-Instruct, Gemini-2.5-Pro, and GPT-5. The results (Table 14) show a Krippendorff's $\alpha$ above 0.70 across both Python and WebUI artifact domains, indicating "strong inter-annotator agreement among different model families." This is important because it demonstrates that the reward scores are "robust, objective, and generalizable indicators of data quality" rather than artifacts of a particular VLM's idiosyncratic preferences.

The ablation on reward modeling (Section 6.1, Table 5). When reward modeling is removed ("w/o Rewarding"), the paper observes "a clear performance drop" across multiple benchmarks. For JANUSCODER on PandasPlotBench, the Visual score drops from 63 to 60 and Task score from 80 to 77. For JANUSCODERV on ChartMimic, the overall score drops from 68.74 to 58.26. This validates "the critical role of our reward modeling for multimodal parts, demonstrating that successful execution alone is insufficient to guarantee high-quality data."

Design choice: VLM-based judging vs. human evaluation. The authors supplement the automated VLM judging with human evaluation on a subset of 100 generated samples (Appendix G.4). Three annotators evaluated outputs on a 1–5 scale for visual fidelity and instruction alignment. The JanusCoder series consistently outperforms baselines in this human evaluation, correlating strongly with the automated VLM-based scores. This cross-validation addresses the concern that VLM judges might have systematic biases that diverge from human preferences—the correlation between human and VLM judgments provides confidence that the automated quality filtering is aligned with human notions of visual quality.

3.4.9 Training Configuration and Model Variants

Backbone models. The paper uses different backbones for the two model variants:

JanusCoder (text-centric): Uses Qwen3-{8B, 14B} (Yang et al., 2025a) as backbones. In ablation studies, Qwen3-4B and Qwen2.5-Coder-7B-Instruct (Hui et al., 2024) are additionally tested.
JanusCoderV (vision-centric): Uses Qwen2.5-VL-7B-Instruct (Bai et al., 2025) and InternVL3.5-8B (Wang et al., 2025b) as backbones. In ablation studies, InternVL3.5-4B is additionally tested.

Training framework and precision. All training experiments are conducted using the LLaMA-Factory framework (Zheng et al., 2024) with bfloat16 precision. The paper uses several acceleration techniques: FlashAttention-2 (Dao, 2024) for efficient attention computation, Liger-Kernel (Hsu et al., 2025) for optimized Triton kernel implementations, and the DeepSpeed framework (Rasley et al., 2020) for distributed training.

Hardware configurations by model scale. The hardware scales with model size:

For 4B, 7B, and 8B models: training on 8 × NVIDIA H800 GPUs with ZeRO-2 sharding and a per-device batch size of 2.
For 14B models: training on 16 × NVIDIA H800 GPUs with ZeRO-3 sharding and a per-device batch size of 1.

With a gradient accumulation step of 8, the total batch size is fixed at 128 across all configurations. This means: for 8-GPU setups, $8 \times 2 \times 8 = 128$ ; for 16-GPU setups, $16 \times 1 \times 8 = 128$ .

Training hyperparameters. The paper states: "Following prior work (Ni et al., 2025) and our own observations, we adopt a learning rate of $1 \times 10^{-5}$ and train for three epochs across all settings." No learning rate schedule, weight decay, or warmup parameters are specified.

Training data assignment. JanusCoder is trained exclusively on the text-centric portion of JanusCode-800K, while JanusCoderV utilizes the entire corpus (both text-centric and vision-centric). The text-centric portion constitutes approximately 50.9% of the corpus (407K samples), and the vision-centric portion approximately 49.1% (393K samples). The balanced split is by design, as shown in Figure 3, ensuring neither modality dominates training for the vision-centric model variant.

Data synthesis model. All natural language instructions and code in the synthesis pipeline are generated using gpt-oss-120b (OpenAI, 2025). This is a 120-billion-parameter model, substantially larger than the 7B–14B models being trained on the synthesized data, following the common pattern of using stronger models to generate training data for weaker models.

4. Key Insights and Innovations

Innovation 1: Reframing Multimodal Code Intelligence as a Unification Problem Rather Than a Specialization Problem

The dominant paradigm in multimodal code intelligence prior to this paper was task-specific specialization: build one model for chart-to-code, another for WebUI generation, a third for SVG editing, and so on. Each task required its own dataset, its own training pipeline, and often its own architectural modifications. The paper's survey of related work (Section 2) makes this pattern explicit—works on chart understanding (Masry et al., 2022; Zhang et al., 2024a), chart-to-code (Zhao et al., 2025b; Xia et al., 2025), theorem visualization (Ku et al., 2025), and structured vector graphics (Yang et al., 2025c) occupy separate literatures with separate benchmarks and separate model releases.

JanusCoder's central intellectual move is to reject this specialization paradigm in favor of unification. The claim is not merely that a single model can handle multiple tasks—a convenience argument—but that cross-domain exposure creates transferable representations that benefit each individual task. This is a hypothesis about the nature of the visual–programmatic mapping: that the cognitive skill of translating between visual intent and executable code is fundamentally domain-general, and that learning it across charts, web pages, animations, and scientific demonstrations reinforces the underlying abstraction rather than creating interference.

What distinguishes this from typical multi-task learning is the deliberate exploitation of cross-domain synergies (Section 3.3). The paper doesn't just throw diverse data into a single training run and hope for positive transfer. It designs synthesis strategies—Bidirectional Translation between Manim and Mathematica, Reverse Instruction repurposing R code for animation—that actively construct cross-domain connections. The transfer is engineered, not merely hoped for. This is a fundamentally different framing from prior work, where each domain's data pipeline was independently optimized.

The evidence for this claim is the ablation studies in Table 5: removing algorithm data from JanusCode-800K degrades JanusCoder's performance on ArtifactsBench VLM Judge from 25.14 to 17.71 (a 30% relative drop), and removing text-centric data from JanusCoderV drops InteractScience functional score from 17.73 to 12.93 (a 27% relative drop). These are not small effects, and they demonstrate transfer across modalities (text-centric data helping vision-centric tasks) in ways that specialization-based approaches would never capture. The unified model also outperforms recently released specialized chart-to-code models on ChartMimic (Appendix G), which is the strongest empirical validation: a generalist beating specialists on their own target domain implies that the cross-domain exposure is providing genuine benefit, not merely not hurting.

This reframing is fundamental rather than incremental because it changes the research question from "how do we build a better chart-to-code model?" to "what is the minimal data recipe that produces general visual–programmatic competence?"—a question that presumes the existence of a unified interface and investigates its boundary conditions. The paper's release of both the data toolkit and the 800K corpus is designed to operationalize this reframing for the community.

Innovation 2: Executability Is Not Quality—Establishing VLM-Based Reward Modeling as the New Standard for Visual Code Data

The default assumption in code data curation—implicit in virtually all prior work on code generation datasets—is that if code executes correctly, it is good training data. This is the logic behind execution-based filtering in datasets like APPS, MBPP, and HumanEval, and behind the "execution rate" metric that code generation papers routinely report. The assumption is reasonable for text-centric code: if a function passes its unit tests, it satisfies the specification.

JanusCoder demonstrates that this assumption catastrophically breaks down for visual code outputs. The paper's core diagnostic observation (Section 3.4) is that "while a program may pass compiler or rendering checks, its actual visual output can drastically diverge from user instructions or requirements." A Matplotlib script that runs without errors can produce a chart with wrong colors, missing titles, misaligned legends, or confusing scale choices. An HTML page that renders without JavaScript errors can have a layout that is semantically wrong relative to the instruction. The failure mode is not execution failure but visual misalignment, and execution-based filtering is blind to it.

The innovation is not the use of VLM judges per se—prior work has used LLM-as-judge for text evaluation—but the systematic integration of multi-dimensional VLM assessment as a mandatory quality gate in a large-scale data synthesis pipeline. The four-dimensional rubric (task completion, solution coherence, visual clarity, task relevance) is carefully designed to capture aspects of quality that execution cannot measure, including aesthetic judgment ("is the visualization well-designed and visually appealing?") and meta-task assessment ("does the instruction represent a realistic scenario?"). The cross-model adjudication study (Appendix G.3, Table 14) showing Krippendorff's α > 0.70 across three different frontier VLMs validates that these judgments are not idiosyncratic to a single judge model—they capture something objective about visual quality.

The ablation in Table 5 ("w/o Rewarding") provides the empirical grounding: removing the reward modeling step while keeping training set size constant produces clear performance drops—on JanusCoderV's ChartMimic score, from 68.74 to 58.26, a 15% relative decline. This magnitude demonstrates that execution-filtered but unrewarded data is not merely slightly worse; it is substantially degraded, containing enough visually misaligned samples to measurably harm downstream model performance.

This insight is fundamental because it changes what "data quality" means for an entire subfield. If the community adopts VLM-based reward modeling as standard practice for visual code data, it would represent a shift analogous to the adoption of RLHF for instruction following—a recognition that the target behavior (producing visually faithful outputs) requires a quality signal that goes beyond binary correctness. The paper provides the first large-scale demonstration that this signal is both computable and consequential.

Innovation 3: Cross-Domain Data Transfer as a Systematic Remedy for Data Scarcity in Specialized Visual Programming Domains

The paper identifies a structural asymmetry in data availability: domains like Python data visualization and web development have abundant training corpora (hundreds of thousands of samples from VisCoder, WebCode2M, and TheStackV2), while specialized domains like Manim animation, Wolfram Mathematica, and scientific demonstrations have orders of magnitude less. The conventional approach to this asymmetry would be to accept the limitation—train specialized models on whatever data exists in each domain and live with the performance ceiling—or to invest in expensive human annotation to close the gap.

JanusCoder's approach is qualitatively different: deliberately engineer cross-domain transfer pathways that convert abundant-domain data into scarce-domain training samples. The mechanism is Bidirectional Translation (Section 3.2), which translates conceptual intent between semantically analogous domains. A Manim animation of a Fourier synthesis and its Mathematica equivalent express the same mathematical concept through different programming paradigms; by training the model to translate between them, the scarce-domain data is effectively multiplied—each existing sample generates a cross-domain counterpart.

What makes this intellectually distinctive is not the translation mechanism itself (which is a standard application of LLM-based code generation) but the recognition that visual programming domains with different syntaxes can be semantically equivalent at the level of abstract operations. Manim's Create(circle) and Mathematica's Graphics[{Circle[]}] are syntactically unrelated but semantically identical—both instantiate a circular visual element. The paper's synthesis design exploits this semantic equivalence systematically, treating it not as a curiosity but as a data multiplication strategy.

The practical significance is demonstrated through negative results rather than positive ones: the ablation in Table 5 shows that when text-centric data is removed from JanusCoderV's training, the InteractScience functional score drops from 17.73 to 12.93—the largest relative decline in any ablation condition. InteractScience evaluates scientific demonstration code generation, precisely the kind of data-scarce domain where cross-domain transfer should matter most. The magnitude of the drop (27% relative) suggests that cross-domain data is not merely supplementing in-domain data but providing capabilities the model cannot acquire from scarce in-domain data alone.

This is a fundamental insight because it provides a generalizable strategy for any visual programming domain with limited training data: identify semantically analogous abundant domains and engineer translation pathways, rather than accepting data scarcity as a ceiling. The paper's release of the synthesis toolkit makes this strategy operational for future researchers, reducing the engineering effort required to apply the same principle to new domain pairs (e.g., translating between Plotly and ggplot2, or between D3.js and Observable).

5. Experimental Analysis

Evaluation Methodology

Datasets. The paper evaluates on seven benchmarks spanning both unimodal (text-to-code) and multimodal (vision-to-code) settings, plus two general coding benchmarks:

PandasPlotBench (Galimzyanov et al., 2025): Evaluates Python-based visualization code generation from natural language descriptions based on DataFrames. Metrics include incorrect code percentage, mean visual score (0–100), mean task score (0–100), and the fraction of outputs rated "Good" (≥75 on visual and task dimensions).
ArtifactsBench (Zhang et al., 2025): Evaluates generation of interactive visual artifacts (games, SVGs, web applications, simulations, management systems) from text descriptions. Uses VLM-as-judge scoring via GPT-4o.
ChartMimic (Yang et al., 2025b): Evaluates chart-to-code generation, with two sub-tasks—Direct Mimic (reproducing a chart from a reference image + caption) and Customized Mimic (reproducing with modifications)—using low-level metrics (text, layout, type, color matching) and high-level metrics (overall quality).
DesignBench (Xiao et al., 2025): Evaluates WebUI generation (code from webpage screenshots) and editing (code modification per user instructions given screenshots and source code), using CLIP similarity, MLLM Score (GPT-4o-as-judge on a 0–10 scale), and Code Match Score (CMS, Jaccard similarity of modified code lines).
WebCode2M (Gui et al., 2025): Evaluates webpage-to-code generation from screenshots, measuring Visual similarity (image-level alignment) and TreeBLEU (structural correctness at the DOM tree level, formally the fraction of all 1-height subtrees in a candidate tree matched in a reference tree: TreeBLEU = |S(t) ∩ S(ŝ)| / |S(ŝ)|).
InteractScience (Chen et al., 2025b): Evaluates scientific demonstration code generation via two components: a Programmatic Functional Test (PFT) measuring functional pass rate (Overall %, Average %, Perfect %), and a Visually-Grounded Qualitative Test (VQT) measuring Action score (whether intended interaction sequences execute correctly), CLIP similarity, and VLM-Judge score.
DTVBench (proposed in this paper): Evaluates generation of dynamic theorem visualizations using two engines—Manim (Python animations) and Wolfram Mathematica (interactive visualizations)—with 102 manually curated tasks. Scoring: Executability (binary), Code Similarity (1–5, judged by GPT-4o against reference), Instruction Alignment (1–5, judged by GPT-4o), and Faithfulness (1–5, optional subjective human score). Overall score = s_exec × (s_sim + s_align + s_faith), ensuring only executable code receives further evaluation.
BigCodeBench (Zhuo et al., 2025a) and LiveCodeBench (Jain et al., 2025): General coding benchmarks for complex instruction following and algorithmic capability.

Base models. The paper constructs two model variants from different backbones. JanusCoder (text-centric) uses Qwen3-8B and Qwen3-14B (Yang et al., 2025a). JanusCoderV (vision-centric) uses Qwen2.5-VL-7B-Instruct (Bai et al., 2025) and InternVL3.5-8B (Wang et al., 2025b). For ablation experiments on backbone robustness, the paper additionally tests Qwen3-4B, Qwen2.5-Coder-7B-Instruct (Hui et al., 2024), and InternVL3.5-4B. The backbones were chosen because Qwen3 and Qwen2.5-VL represent strong open-weight models at the time of development, and InternVL3.5 represents a state-of-the-art vision-language architecture, allowing the paper to demonstrate that the data pipeline's benefits generalize across model families.

Metrics. The metrics vary by benchmark and are detailed in Appendix F. For PandasPlotBench: Incorrect code % (fraction of samples producing execution errors or unparseable outputs), Visual score and Task score (GPT-4o judge ratings on 0–100 scales), and "Good (≥75)" rates (fraction scoring ≥75 on each dimension). For ChartMimic: Execution Rate (% of samples producing runnable code), Low-Level scores (Text, Layout, Type, Color matching between generated and reference charts), High-Level average, and Overall score. For DesignBench: CLIP similarity (semantic alignment between generated and reference screenshots), MLLM Score (GPT-4o judge on 0–10 scale validated against human evaluation per the original paper), and CMS (Jaccard similarity of modified lines between generated and ground-truth code). For WebCode2M: Visual (image-level resemblance) and TreeBLEU (DOM tree structural correctness). For InteractScience: PFT Overall % (fraction of all test cases passed), Average % (mean accuracy across samples), Perfect % (percentage of cases where all tests for one sample pass), plus VQT Action %, CLIP, and VLM-Judge scores. For DTVBench: Executability (binary), Code Similarity (1–5), Instruction Alignment (1–5), and Faithfulness (1–5, optionally human-judged), with overall score computed as described above.

Baselines. The paper compares against a range of open-weight and proprietary models. For unimodal tasks: LLaMA3-8B-Instruct (Dubey et al., 2024), Qwen3-8B and Qwen3-14B base models (pre-training, Yang et al., 2025a), Qwen2.5-Coder-7B-Instruct and Qwen2.5-Coder-14B-Instruct (Hui et al., 2024), and GPT-4o (Hurst et al., 2024) as the proprietary ceiling. For multimodal tasks: Qwen2.5-VL-7B-Instruct (Bai et al., 2025), InternVL3-8B and InternVL3.5-8B (Wang et al., 2025b), MiniCPM-V-2-6 (Yao et al., 2024), Llama-3.2-11B-Vision-Instruct (Meta, 2024), and GPT-4o. For DTVBench specifically, additional baselines include Qwen2.5-Coder-32B-Instruct. ChartMimic baselines also include GeminiProVision, Claude-3-opus, IDEFICS2-8B, DeepSeek-VL-7B, LLaVA-Next-Yi-34B, LLaVA-Next-Mistral-7B, Qwen2-VL-2B, Cogvlm2-llama3-chat-19B, InternVL2 variants (2B, 4B, 8B, 26B), Phi-3-Vision-128K, and MiniCPM-Llama3-V-2.5, with results taken directly from the ChartMimic paper. The paper also compares against specialized models: VisCoder (Ni et al., 2025) for Python visualization and recently released chart-to-code MLLMs (Xia et al., 2025; Zhao et al., 2025b) for chart-to-code.

Generation budget / compute accounting. The paper does not standardize around a unified "generation budget" concept as seen in inference-time scaling work, since the primary contribution is a training data pipeline rather than an inference-time strategy. All models in the main comparisons generate single outputs (greedy or single-sample), so there is no budget sweep. For PandasPlotBench and ArtifactsBench, each model produces one code output per test instance, which is then executed to produce a visual output for evaluation. For ChartMimic and WebCode2M, the same single-generation protocol applies. For BigCodeBench and LiveCodeBench, pass@1 is reported. For DTVBench, a single generation per task is evaluated.

Cross-validation / statistical protocol. For the proposed DTVBench, the paper uses human evaluation with three college-level annotators rating outputs on a 1–5 scale for visual fidelity and instruction alignment (Appendix G.4). For the cross-model adjudication study validating the reward model (Appendix G.3), the paper randomly samples 30K unfiltered data samples and evaluates them using three distinct frontier models (Qwen2.5-VL-72B-Instruct, Gemini-2.5-Pro, and GPT-5), computing Krippendorff's α for inter-annotator agreement. No other cross-validation or statistical significance testing is reported across the main benchmark evaluations—results are point estimates on standard test splits.

Main Quantitative Results

Unimodal Tasks (Text-to-Code Generation)

PandasPlotBench (Table 7, Table 3). JANUSCODER-14B achieves the best results among open-weight models, with an incorrect code rate of 9.7%—matching GPT-4o exactly and substantially outperforming Qwen3-14B-Base (11.4%), Qwen2.5-Coder-14B-Instruct (16.0%), and LLaMA3-8B-Instruct (26.9%). On the Visual score, JANUSCODER-14B reaches 67 (vs. GPT-4o's 72), and on Task score it reaches 86 (vs. GPT-4o's 85), demonstrating near-proprietary performance on task completion while trailing modestly on visual quality. The "Good (≥75)" rates paint a consistent picture: JANUSCODER-14B achieves 0.57 on Visual (GPT-4o: 0.63) and 0.87 on Task (GPT-4o: 0.85), exceeding GPT-4o on task completion rate. JANUSCODER-8B shows more modest but still competitive results: 14.9% incorrect code rate, Visual score of 63, Task score of 80, with Good rates of 0.59 (Visual) and 0.80 (Task). Notably, JANUSCODER-8B outperforms the Qwen3-8B base model (20.0% incorrect, Visual 63, Task 74) across the board, and approaches Qwen2.5-Coder-7B-Instruct (21.1% incorrect, Visual 63, Task 76). The vision-centric JANUSCODERV variants also perform competitively on this unimodal benchmark: JANUSCODERV-7B achieves 18.9% incorrect, Visual 63, Task 80, substantially outperforming its backbone Qwen2.5-VL-7B-Instruct (18.3% incorrect, Visual 63, Task 74). JANUSCODERV-8B underperforms relative to its backbone InternVL3.5-8B, which the paper attributes to the InternVL3.5 base model's weaker inherent Python coding capability (the backbone itself scores 36.0% incorrect, Visual 52, Task 63).

ArtifactsBench (Table 3, Table 13). JANUSCODER-14B achieves the highest average VLM Judge score of 41.10, surpassing Qwen3-14B (39.79), Qwen3-8B (36.52), and GPT-4o (37.97). This is a particularly strong result given that GPT-4o is the proprietary baseline. JANUSCODER-8B achieves 39.60, substantially exceeding its Qwen3-8B backbone (36.52). The sub-domain breakdown in Table 13 reveals where the gains concentrate: JANUSCODER-14B achieves the best results on WEB (44.47 vs. GPT-4o's 39.27), SI—Simulation (41.49 vs. GPT-4o's 35.73), and MS—Management System (45.04 vs. GPT-4o's 35.83), while underperforming on SVG Generation (24.72 vs. GPT-4o's 39.54). The authors attribute the WEB/SI/MS improvements to "our data pipeline that combines challenging webdev data for complex interactive components with theorem-related resources and cross-language code to enrich structural diversity and enhance generalization." The SVG weakness is noted but not explained—it may reflect a tradeoff where the unified model's exposure to diverse domains dilutes performance on a specific format, or where the SVG synthesis strategies in the pipeline are less effective than those for other domains. JANUSCODER-8B shows a similar pattern: strong on WEB (40.07 vs. Qwen3-8B's 38.08), SI (41.92 vs. 36.15), and MS (44.75 vs. 35.92), but weak on SVG (30.47 vs. 36.37).

DTVBench (Table 3). The DTVBench results are reported separately for Manim and Wolfram sub-tasks. For Manim, JANUSCODER-8B achieves 9.70 overall score, substantially outperforming Qwen3-8B (6.20), Qwen2.5-Coder-7B-Instruct (8.56), and Qwen3-14B (6.63), while approaching GPT-4o (10.60). JANUSCODER-14B scores 8.41, which is below JANUSCODER-8B—an unusual inverse scaling pattern that the paper does not discuss, potentially reflecting differences in how the 8B and 14B models handle the specific Manim API conventions or the relatively small training data volume in the animation domain. For Wolfram, JANUSCODER-8B achieves 6.07, outperforming Qwen3-8B (5.18) and approaching GPT-4o (4.92), while JANUSCODER-14B achieves 5.97. The overall pattern suggests that JANUSCODER models achieve competitive or superior performance to GPT-4o on dynamic content generation, particularly in the Manim domain where the paper's 3Blue1Brown-derived training data and AST-based decomposition pipeline provide the strongest domain coverage. However, the absolute scores are low across all models (maximum 10.60 out of a theoretical maximum of 15 for GPT-4o on Manim), indicating that dynamic theorem visualization remains a challenging frontier task where even the strongest models produce outputs with significant imperfections.

Multimodal Tasks (Vision-to-Code Generation)

ChartMimic (Table 4, Table 8, Table 9). JANUSCODERV-7B achieves 64.72 on Direct Mimic Low-Level metrics and 72.77 on High-Level, with an Overall score of 69.2 (Table 8). JANUSCODERV-8B achieves comparable results: 65.8 Low-Level, 73.2 High-Level, 69.5 Overall. Both variants substantially outperform GPT-4o (57.2 Low, 64.6 High, 60.9 Overall) and all open-weight baselines, including strong specialized chart models. The closest open-weight competitor is InternVL3.5-8B at 46.0 Low, 53.4 High, 49.7 Overall—a gap of approximately 20 points in Overall score. On Customized Mimic (Table 9), the pattern is consistent: JANUSCODERV-7B achieves 64.7 Low, 72.8 High, 68.7 Overall vs. GPT-4o's 59.4 Low, 67.4 High, 63.4 Overall. The Execution Rate is notably high for the JanusCoder models (80.6% for Direct, 80.3–80.7% for Customized), substantially exceeding GPT-4o (73.0% Direct, 73.2% Customized) and all open-weight baselines. This high execution rate suggests that JanusCode-800K's execution validation during synthesis trains the model to produce syntactically correct chart code reliably. The paper explicitly notes: "As a unified model, it also outperforms recently released specialized chart-to-code MLLMs (Xia et al., 2025; Zhao et al., 2025b), highlighting the effectiveness of leveraging cross-task data synergy."

DesignBench (Table 4, Table 10). On generation tasks, JANUSCODERV-7B achieves 73.31 overall score (combining CLIP, MLLM, and CMS metrics), making it the strongest open-weight model and competitive with proprietary systems (GPT-4o: 76.83, Claude-3.7-sonnet: 81.32). On editing tasks, JANUSCODERV-7B achieves an MLLM Score of 8.79 and CMS of 27.49, placing it just behind GPT-4o (MLLM 9.23, CMS 33.94) but ahead of InternVL3.5-8B (MLLM 8.63, CMS 28.65). JANUSCODERV-8B shows slightly lower generation performance (68.86 overall) but competitive editing (MLLM 8.63, CMS 25.60). A notable finding is the CLIP similarity across models: JANUSCODERV-7B achieves 73.31 on generation, comparable to InternVL3.5-8B (71.73) and Qwen2.5-VL-7B-Instruct (72.73), suggesting that visual similarity improvements from JanusCode-800K training are modest, while the structural correctness (CMS) and instruction-following (MLLM Score) gains are more substantial.

WebCode2M (Table 4, Table 11). JANUSCODERV-7B achieves a Visual score of 75.78 (averaged across Short, Mid, and Long webpage lengths from Table 11) and a TreeBLEU score of 26.21, which the paper characterizes as "significantly higher TreeBLEU scores, surpassing all proprietary counterparts and setting the state-of-the-art in structural preservation of generated code." This is a striking finding: GPT-4o achieves Visual 82.67 and TreeBLEU 13.00—far superior visual fidelity but far inferior structural correctness. The TreeBLEU advantage holds across all webpage length categories: for Short, JANUSCODERV-7B scores 0.25 vs. GPT-4o's 0.15; for Mid, 0.28 vs. 0.13; for Long, 0.26 vs. 0.11. This suggests that JanusCode-800K's training data, which includes WebCode2M-derived samples with execution validation and VLM quality filtering, teaches the model to produce webpages that are structurally sound at the DOM level even when visual appearance is less precisely matched. The paper frames this as a "favorable trade-off: JanusCoder prioritizes structural faithfulness without severely sacrificing appearance quality." JANUSCODERV-8B achieves lower scores: Visual 66.34, TreeBLEU 18.28, still substantially above GPT-4o on TreeBLEU but with a larger visual fidelity gap.

InteractScience (Table 4, Table 12). JANUSCODERV-7B achieves a Programmatic Functional Test (PFT) Overall score of 17.73%, Average of 16.91%, and Perfect of 4.20%, substantially outperforming all open-weight baselines: Qwen2.5-VL-7B-Instruct (8.40%, 7.05%, 0.70%), InternVL3-8B (8.93%, 8.13%, 1.40%), InternVL3.5-8B (11.47%, 10.92%, 2.10%), MiniCPM-V-2-6 (0.13%, 0.08%, 0.00%), and Llama-3.2-11B-Vision-Instruct (6.67%, 5.63%, 0.70%). The gap between JanusCoder and InternVL3.5-8B—roughly 6 percentage points on Overall and 5 points on Average—is substantial given the difficulty of this task. However, both JanusCoder variants remain significantly behind proprietary models: GPT-4o achieves 31.07% Overall, 28.59% Average, 10.49% Perfect, and Gemini-2.5-Pro achieves 41.87% Overall, 38.56% Average, 13.99% Perfect. On the Visually-Grounded Qualitative Test (VQT), JANUSCODERV-8B achieves a VLM-Judge score of 33.32, exceeding GPT-4o (46.01) but approaching within ~13 points. The CLIP similarity scores show JANUSCODERV-8B at 61.52 (GPT-4o: 71.18), and Action scores at 81.86% (GPT-4o: 88.47%), indicating that the functional interaction sequences are correctly executed at rates approaching GPT-4o, but the overall visual quality and comprehensiveness still lag. JANUSCODERV-7B achieves slightly lower VLM-Judge (27.67) but higher Action (83.22%) and comparable CLIP (60.56), suggesting a different strength profile across the two backbone architectures.

Cross-Model Adjudication Validation

Reward model inter-annotator agreement (Table 14). The cross-model adjudication study evaluates whether the VLM-based reward scores are robust to the choice of judge model. On 30K randomly sampled unfiltered data samples spanning Python and WebUI artifacts, evaluated by Qwen2.5-VL-72B-Instruct, Gemini-2.5-Pro, and GPT-5, the paper reports Krippendorff's α of 0.76 for Python data and 0.71 for WebUI artifacts, both above the 0.70 threshold conventionally considered strong agreement. The mean scores across judges are consistent: for Python, Qwen2.5-VL gives mean 4.59 (variance 0.41), Gemini-2.5-Pro gives 4.36 (0.58), GPT-5 gives 4.41 (0.53). For WebUI artifacts, the means are 4.28 (0.47), 4.05 (0.61), and 4.16 (0.52), respectively. The Qwen2.5-VL judge (the one used in the main pipeline) gives slightly higher and lower-variance scores than the other two models, but the differences are modest and the strong α values confirm that the reward signal is not an artifact of a single judge model's idiosyncratic preferences.

General Coding Capabilities

BigCodeBench and LiveCodeBench (Figure 5, Figure 6 in Appendix). The paper evaluates JANUSCODER on general coding benchmarks to assess whether the visual–programmatic training degrades standard code generation capability. Results are presented as scatter plots in Figure 5 and Figure 6, showing various models on pairs of axes: PandasPlotBench Task score vs. BigCodeBench-Hard pass@1, and ArtifactsBench VLM Judge vs. LiveCodeBench pass@1. JANUSCODER-14B achieves competitive general coding performance while dominating visualization tasks: it outperforms Qwen3-8B, LLaMA3-8B-Instruct, and Qwen2.5-Coder-7B-Instruct on both dimensions simultaneously. JANUSCODER-8B also shows strong balanced performance, exceeding VisCoder (a specialized visualization model) on PandasPlotBench while maintaining higher BigCodeBench scores. The paper specifically highlights that JANUSCODER "surpasses even specialist approaches" and "outperforms specialist models like VisCoder in their own target visualization domain," demonstrating that unified training does not require sacrificing domain-specific excellence.

Ablation Studies and Robustness Checks

Data synergies (Table 5). The paper tests the cross-domain synergy hypothesis by selectively removing data categories from JanusCode-800K and measuring performance impact:

w/o Algorithm (removing the 100K algorithm data samples): On JANUSCODER, PandasPlotBench Visual score drops from 63 to 62 (minimal), Task score drops from 80 to 83 (actually improves slightly), LiveCodeBench-V6 drops from 40.99 to 40.31 (small), but ArtifactsBench VLM Judge drops substantially from 25.14 to 17.71—a 29.6% relative decline. On JANUSCODERV, ChartMimic improves from 68.74 to 70.16, InteractScience improves from 17.73 to 18.13 (both slight), but WebCode2M drops from 75.78 to 72.18. These results are nuanced: algorithm data appears modestly beneficial for WebUI structural correctness and essential for artifact generation quality, but removing it may even slightly improve chart-related tasks, possibly because the algorithmic reasoning training data introduces a different linguistic style that creates mild interference for chart-to-code specifically.
w/o SVG (removing the 20K SVG data samples): On JANUSCODER, ArtifactsBench VLM Judge drops from 25.14 to 22.86 (a 9.1% relative decline), while PandasPlotBench and LiveCodeBench show minimal changes. This is expected given that ArtifactsBench includes an explicit SVG sub-domain, but the magnitude suggests SVG training data provides structural knowledge that transfers beyond SVG-specific tasks—the drop is larger than what would be expected from removing only 20K samples out of 800K if the benefit were purely SVG-specific.
w/o Chart2Code (removing the 70K chart-to-code data samples): On JANUSCODERV, ChartMimic drops from 68.74 to 56.50—a 17.8% relative decline, the largest absolute drop in any ablation condition. WebCode2M drops from 75.78 to 71.92. InteractScience drops from 17.73 to 16.27. This confirms that the chart-to-code data is the primary driver of ChartMimic performance, as expected, but the spillover to WebCode2M suggests that the visual-to-code mapping learned from chart images transfers to webpage screenshots as well, consistent with the cross-domain synergy hypothesis.
w/o Text-centric (removing all text-centric data from JANUSCODERV's training): ChartMimic drops from 68.74 to 60.73 (11.7% relative), WebCode2M drops from 75.78 to 71.82 (5.2%), and InteractScience drops from 17.73 to 12.93—a 27.1% relative decline, the largest relative drop in the multimodal ablation set. This is the strongest evidence for cross-modal synergy: removing text-only code training data (which has no visual components) substantially degrades vision-to-code tasks, especially the most challenging scientific demonstration generation task. The paper interprets this as confirming that "text-centric data contributing to multimodal coding scenarios" provides transferable coding capabilities.

Reward modeling (Table 5). The paper compares training with the full pipeline (execution validation + VLM reward filtering) against training on execution-validated but unrewarded data, keeping training set size constant:

w/o Rewarding on JANUSCODER: PandasPlotBench Visual drops from 63 to 60, Task drops from 80 to 77, LiveCodeBench-V6 drops from 40.99 to 38.58, and ArtifactsBench VLM Judge drops from 25.14 to 24.57. The drops are consistent (2–5% relative) but moderate, suggesting that for text-centric data, executability is a more reasonable quality proxy than for vision-centric data, though reward modeling still provides a measurable benefit.
w/o Rewarding on JANUSCODERV: ChartMimic drops from 68.74 to 58.26—a 15.2% relative decline, the largest effect of any single ablation on a multimodal benchmark. InteractScience drops from 17.73 to 17.20 (small), and WebCode2M drops from 75.78 to 73.78 (small). The large effect specifically on ChartMimic is noteworthy: chart-to-code is precisely the type of task where executability alone is a poor quality proxy, since a chart script can run correctly while producing visually misaligned output (wrong colors, mislabeled axes, inappropriate chart type). The reward model's visual quality assessment is most essential for these samples, and removing it introduces enough visually misaligned training data to substantially degrade downstream chart-to-code performance.

Effect of backbones (Figure 4, Figure 7 in Appendix). The paper tests JanusCode-800K training on multiple backbone architectures and scales to assess whether the data pipeline's benefits are backbone-specific:

For unimodal models: Training on JanusCode-800K improves PandasPlotBench Task score for Qwen3-14B (from ~78 to ~86), Qwen3-8B (from ~74 to ~80), and Qwen2.5-Coder-7B (from ~76 to ~84). On the ArtifactsBench Lite subset, similar consistent improvements are observed across all three backbones. The gains are largest for Qwen2.5-Coder-7B, which starts from the strongest base performance—suggesting that JanusCode-800K provides complementary data that benefits even code-specialized models.
For multimodal models: Training on JanusCode-800K improves ChartMimic Direct score for Qwen2.5-VL-7B (from ~41 to ~69), InternVL3.5-8B (from ~50 to ~68), and InternVL3.5-4B (from ~35 to ~55). On InteractScience Functional score, Qwen2.5-VL-7B improves from ~8 to ~18, InternVL3.5-8B from ~12 to ~18, and InternVL3.5-4B from ~5 to ~13. The consistent pattern across all three backbones—including different model families (Qwen vs. InternVL) and scales (4B, 7B, 8B)—confirms that the data pipeline's effectiveness is not an artifact of a particular architecture's inductive biases. The paper states this "confirms the soundness of our data design and can empower diverse backbones to become more generalist models for multimodal code intelligence."

Human evaluation on subjective tasks (Table 15). The paper conducts human evaluation on 100 generated samples spanning Manim Code2Video and InteractScience WebUI tasks, with three college-level annotators rating outputs on 1–5 scales for visual fidelity and instruction alignment. For Manim, JANUSCODER-14B achieves 3.14, outperforming Qwen3-14B (2.64), Qwen2.5-Coder-32B (2.82), and GPT-4o (2.78). For InteractScience WebUI, JANUSCODERV-7B achieves 3.19, substantially outperforming InternVL3.5-8B (1.57) and GPT-4o (2.78). These human evaluations correlate with the automated VLM-based metrics used in the main benchmarks, providing cross-validation that the automated metrics capture human-relevant quality signals. The absolute scores remain moderate (3.14–3.19 out of 5), confirming that dynamic visualization and scientific demonstration generation remain challenging.

Critical Assessment

Claim 1: "JanusCode-800K, powered by a versatile data synthesis toolkit, enables training unified models that outperform specialized models and approach or exceed GPT-4o."

This claim is substantially supported but with specific boundary conditions. The evidence is strongest on ChartMimic, where JANUSCODERV-7B surpasses GPT-4o by a wide margin (69.2 vs. 60.9 Overall on Direct Mimic) and also outperforms specialized chart-to-code models specifically trained for this task. On PandasPlotBench, JANUSCODER-14B matches GPT-4o on the incorrect code rate (9.7% each) and Task score (86 vs. 85), though it trails on Visual score (67 vs. 72). On ArtifactsBench, JANUSCODER-14B exceeds GPT-4o (41.10 vs. 37.97) but only after averaging across sub-domains—the SVG sub-domain is a notable weakness (24.72 vs. GPT-4o's 39.54). On WebCode2M, JANUSCODERV-7B sets a new state-of-the-art on TreeBLEU (26.21 vs. GPT-4o's 13.00) while trailing on Visual score (75.78 vs. 82.67). The claim of "approaching or exceeding GPT-4o" is therefore domain-dependent: JanusCoder wins on structural code correctness and task completion, while GPT-4o retains an advantage on raw visual fidelity.

A critical nuance: the "specialized models" comparison in the ChartMimic domain is mentioned but the specific numerical comparisons against Xia et al. (2025) and Zhao et al. (2025b) are described qualitatively ("outperforms recently released specialized chart-to-code MLLMs") with the detailed numbers deferred to Appendix G—but Appendix G as provided in the paper excerpt does not contain these specific numbers. The claim is credible given the large margin over GPT-4o (which itself is presumably stronger than specialized chart models), but the exact evidence is not fully presented in the main body.

Claim 2: "Cross-domain synergies enable transfer between semantically related domains and across modalities."

This claim is supported with strong ablation evidence but the mechanisms remain underspecified. The ablation studies in Table 5 provide clear empirical evidence for transfer: removing text-centric data degrades InteractScience by 27.1%, removing algorithm data degrades ArtifactsBench by 29.6%, and removing chart-to-code data degrades WebCode2M by 5.2%. These are cross-domain effects that would not occur if each domain were processed independently. However, the paper does not provide a mechanistic analysis of why these transfers occur—is it shared syntax (Python appearing in both text-centric and vision-centric tasks), shared visual reasoning (understanding spatial layouts transferring from charts to webpages), or something else? The bidirectional translation strategy between Manim and Mathematica is described as a mechanism, but no ablation specifically tests whether removing bidirectional translation (while keeping the same quantity of in-domain data) degrades performance, which would provide direct evidence for that specific synergy mechanism.

Claim 3: "VLM-based reward modeling is essential—executability alone is insufficient."

This claim is strongly supported for vision-centric tasks, moderately supported for text-centric tasks. The ablation in Table 5 shows that removing reward modeling from JANUSCODERV drops ChartMimic from 68.74 to 58.26—a 15.2% relative decline—which is large and clearly consequential. For JANUSCODER (text-centric), the drops are more modest (e.g., ArtifactsBench from 25.14 to 24.57, or 2.3% relative), suggesting that executability is a more reasonable quality proxy for text-only code data. This makes intuitive sense: a text-centric programming problem's correctness is largely determined by its logic, whereas a visualization's quality depends on visual attributes that execution doesn't check. The paper's cross-model adjudication study (Krippendorff's α > 0.70) strengthens the claim by showing that VLM judgments are not idiosyncratic to the chosen judge model.

A limitation: the paper reports that for text-centric data "only samples achieving a score of 5 are retained" (Appendix C.1), while for vision-centric data the threshold varies (≥3 for key dimensions in editing tasks). This asymmetric thresholding is not explored as an ablation—would a strict score-of-5 threshold for vision-centric data yield even better results, or would it filter out too many samples? The threshold choice is presented as a design decision rather than an empirically optimized parameter.

Claim 4: "The JanusCoder series serves as a strong open-source foundational model for multimodal code intelligence."

This claim is supported by the breadth of evaluation but limited by the absence of certain baselines and scales. The paper evaluates on seven benchmarks spanning unimodal and multimodal settings, covering chart generation, webpage generation, animation authoring, and scientific demonstration creation—an unusually broad evaluation suite for a single model. The performance is consistently strong relative to open-weight alternatives and competitive with GPT-4o.

However, several limitations qualify this claim:

Missing model scales. The paper evaluates 7B–14B models, but "foundational model" status typically implies a family spanning larger scales (e.g., 70B, 405B). Training on JanusCode-800K with larger backbones (Qwen3-70B, Qwen2.5-VL-72B) would test whether the data pipeline's benefits scale with model size—would the gap to GPT-4o close further on visual fidelity tasks like WebCode2M and InteractScience where larger models might better absorb the diverse training data? This experiment is absent.
No comparison to simply scaling up in-domain data. A key unanswered question: does JanusCode-800K's cross-domain diversity provide benefits beyond training on an equivalently sized corpus of in-domain-only data? For example, would a ChartMimic-specialized model trained on 800K chart-only samples outperform JANUSCODERV? The ablation studies remove data categories but don't replace them with additional in-domain samples to isolate the diversity vs. quantity effect.
The SVG weakness is unexplained. JANUSCODER-14B's ArtifactsBench SVG sub-score of 24.72 (vs. GPT-4o's 39.54 and even Qwen3-14B's 39.50) is a significant regression that the paper does not analyze. If the unified model sacrifices performance on a specific output format in exchange for breadth, this is a genuine tradeoff that should be characterized—is it due to SVG data quality in the synthesis pipeline, interference from non-SVG training, or insufficient SVG representation in the corpus (only 20K of 800K samples, or 2.5%)?
No few-shot or chain-of-thought baselines. The paper evaluates all models under single-generation protocols, but GPT-4o with few-shot prompting or chain-of-thought reasoning might close or reverse some of the gaps. For complex tasks like InteractScience and DTVBench, GPT-4o's reasoning capabilities might benefit disproportionately from few-shot examples or explicit planning prompts, and the paper does not test whether JanusCoder's advantage persists under these more favorable inference conditions for the baseline.
Test set contamination risk. JanusCode-800K is synthesized from sources that include WebCode2M and VisCoder data (Table 6: 200K WebUI generation samples and 69,501 editing samples from WebCode2M; 200K Python visualization samples from VisCoder). If the evaluation benchmarks (WebCode2M, PandasPlotBench) share any test instances with the source datasets, the reported results could reflect memorization rather than generalization. The paper does not discuss decontamination procedures.

Claim 5: "DTVBench provides a meaningful new evaluation for dynamic theorem visualization."

This is a useful contribution but with validation gaps. The benchmark's 102 manually curated tasks spanning two complementary engines (Manim, Wolfram Mathematica) address a genuine gap—no prior benchmark specifically evaluates dynamic mathematical visualization generation. The multi-dimensional scoring (executability × similarity + alignment + faithfulness) is well-motivated for the domain. However: (1) the human evaluation is small-scale (100 samples, 3 annotators) and the annotators are college-level students, not domain experts in mathematical visualization or animation design; (2) the Code Similarity and Instruction Alignment dimensions are judged by GPT-4o, an automated metric that may not correlate well with human judgments of mathematical correctness and pedagogical effectiveness; (3) the optional "Faithfulness" subjective score is described as evaluating "plausibility and visual correctness," which conflates mathematical accuracy with perceptual quality; (4) the absolute scores across all models are very low (maximum ~10.6 out of 15 on Manim), which could indicate either that the task is genuinely hard or that the evaluation rubric is miscalibrated—distinguishing between these requires more extensive human calibration studies than the paper provides.

Overall assessment of the experimental design: strengths and gaps.

Strengths: The breadth of evaluation is genuinely impressive for a single paper introducing a new training corpus and model family. Seven diverse benchmarks, multiple model scales, both automated and human evaluation, and cross-model adjudication for the reward pipeline constitute a thorough evaluation program. The ablation studies systematically probe the cross-domain synergy hypothesis and the importance of reward modeling, providing empirical grounding for the paper's methodological claims.

Gaps: (1) No scaling study—how does performance change if JanusCode-800K is 200K, 400K, 800K, 1.6M samples? (2) No controlled experiment isolating cross-domain diversity benefits—training on 800K in-domain vs. 800K cross-domain vs. 800K mixed would clarify whether diversity or sheer quantity drives the gains. (3) No qualitative error analysis—when does JanusCoder fail? What types of visualizations are systematically challenging? The case studies in Appendix H show cherry-picked successes but no failure analyses. (4) The training cost is not reported in terms of GPU-hours or total FLOPs, making it difficult for practitioners to assess the resource requirements for reproducing the results. (5) No inference-time efficiency analysis—the models are 7B–14B parameters, but the paper doesn't discuss inference latency or throughput, which matters for interactive applications like WebUI generation.

6. Limitations and Trade-offs

Limitation 1: The Reward Modeling Pipeline's Cost and Dependence on Frontier VLMs

The assumption or constraint. The quality control stage of the data synthesis pipeline relies on frontier vision-language models—Qwen2.5-VL-72B-Instruct for vision-centric data and Qwen3-235B-A22B for text-centric data—to score every training sample across four dimensions on a 1–5 scale. These are 72B and 235B parameter models, respectively, which are substantially larger and more computationally expensive than the 7B–14B models being trained on the filtered data. The paper acknowledges this architectural choice explicitly in Section 3.4: "Our reward model employs a VLM as its core engine to assess the quality of data."

The consequence. The pipeline creates a circular dependency on proprietary or very large open-weight models to generate training data for smaller models. To reproduce the JanusCode-800K corpus, a practitioner needs access to models (Qwen2.5-VL-72B, Qwen3-235B, plus gpt-oss-120b for data synthesis) whose combined inference cost likely exceeds the training cost of the 7B–14B target models. This means the headline result—"small open-weight models approaching GPT-4o"—is achieved by standing on the shoulders of models that are themselves comparable to or larger than GPT-4o. If the reward models were replaced with weaker, cheaper alternatives (e.g., 7B VLM judges), the data quality would presumably degrade, but the paper does not measure how much. The reward pipeline also introduces a latency bottleneck: scoring 800K samples with a 72B VLM is computationally intensive in a way that is not accounted for in any efficiency analysis.

What evidence exists in the paper. The paper partially addresses the robustness concern through the cross-model adjudication study (Table 14, Appendix G.3), showing Krippendorff's α > 0.70 across Qwen2.5-VL-72B, Gemini-2.5-Pro, and GPT-5, confirming that the scores are not idiosyncratic to a single judge. However, this only demonstrates that different large frontier models agree with each other—it does not test whether a smaller, cheaper judge model would produce comparably useful quality filtering. The ablation in Table 5 shows that removing reward modeling entirely causes substantial drops (ChartMimic from 68.74 to 58.26, a 15.2% relative decline), confirming the pipeline is sensitive to reward quality, but it does not establish the minimum viable judge model scale.

Mitigation status. Not addressed. The paper treats the choice of reward model backbone as a fixed design decision rather than a hyperparameter to be studied. There is no experiment comparing reward filtering using, say, a 7B Qwen2.5-VL vs. the 72B version, which would help practitioners understand the quality–cost tradeoff. The paper's release of the final corpus (rather than requiring users to re-run the pipeline) partially mitigates the reproduction burden, but any researcher wanting to extend the pipeline to new domains or data sources would face the same cost barrier.

Limitation 2: The Difficulty Estimation and Synthesis Cost Is Unaccounted for in Any Efficiency Analysis

The assumption or constraint. The entire data synthesis pipeline—including the four synthesis strategies (Guided Evolution, Re-contextualization, Reverse Instruction, Bidirectional Translation), the AST-based decomposition of complex code files, the execution validation sandbox, and the VLM-based reward scoring—is treated as a one-time cost external to model training and evaluation. The paper reports no metrics on the computational cost of constructing JanusCode-800K: no GPU-hours for data synthesis, no wall-clock time for the synthesis-validate-reward loop, no FLOPs accounting for the gpt-oss-120b API calls that generate all natural language instructions and code. Section 3.5 simply states the corpus size (800K samples) without discussing the resources required to produce it.

The consequence. The reported efficiency gains over baselines—"approaching or exceeding the performance of leading commercial models" with 7B–14B parameters—are computed without amortizing the data creation cost. If synthesizing JanusCode-800K required, for instance, 10× more compute than training the final model, then the total cost of producing JanusCoder could exceed that of training a much larger model on existing open-source data. A practitioner evaluating whether to adopt the JanusCoder pipeline vs. simply using GPT-4o directly would need to know the total cost (synthesis + training), not just the training cost. More importantly, the iterative refinement loop in the synthesis pipeline (Section 3.4.6: "Samples that fail this validation are rerouted to the synthesis engine for retry and refinement") implies that some unknown fraction of samples required multiple generation attempts before passing execution validation, multiplying the synthesis cost for those samples. The paper does not report the average number of retry attempts per accepted sample.

What evidence exists in the paper. None. The paper contains no cost accounting for the data pipeline. The training configuration is specified in detail (Section 3.4.9: 8–16 H800 GPUs, ZeRO-2/3, gradient accumulation, etc.), but the synthesis cost is completely uncharacterized. This is a significant asymmetry: the paper is transparent about training hyperparameters while being opaque about the larger computational investment that precedes training.

Mitigation status. Not addressed and not acknowledged as a limitation. The paper frames its contribution as releasing both the toolkit and the corpus, which means future researchers can use the pre-built corpus without incurring synthesis costs—but only for the domains already covered. Any extension to new programming languages, new visualization libraries, or new task formats would require re-running the full pipeline at unknown cost. The paper does not discuss strategies for reducing synthesis cost (e.g., using smaller models for initial generation, or amortizing the cost across multiple downstream models).

Limitation 3: Single Benchmark Domain and No Evidence of Generalization Beyond Technical Visual Programming

The assumption or constraint. All evaluation benchmarks in the paper—PandasPlotBench, ChartMimic, DesignBench, WebCode2M, InteractScience, ArtifactsBench, and DTVBench—fall within the same broad category: generating executable code that produces visual outputs from technical specifications or reference images. This is a coherent domain, but it excludes large classes of code intelligence tasks that practitioners might care about: general algorithmic problem solving (only lightly tested via BigCodeBench and LiveCodeBench in Figure 5), code translation between programming languages, code documentation and explanation, bug detection and repair, API usage recommendation, or natural-language-to-SQL. The paper explicitly scopes itself to "multimodal code intelligence" (Section 1), but the risk is that training on JanusCode-800K may specialize the model toward visual–programmatic tasks at the expense of general coding competence.

The consequence. A practitioner considering JanusCoder as a "foundational model for code intelligence" (the paper's stated ambition) cannot know whether it remains competitive on non-visual coding tasks. The general coding results in Figure 5 and Figure 6 are encouraging—JanusCoder-14B achieves competitive BigCodeBench and LiveCodeBench scores while dominating visualization benchmarks—but these are two general benchmarks tested at a single pass@1 setting. There is no systematic evaluation across a broad code intelligence benchmark suite (e.g., HumanEval, MBPP, DS-1000, CodeXGLUE tasks) that would characterize the breadth of capability. The risk of catastrophic forgetting of general coding competence due to domain-specialized fine-tuning is not assessed: the paper does not compare JanusCoder against its pre-fine-tuning backbones on a diverse set of standard code benchmarks.

Moreover, all benchmarks evaluate English-language instructions only. The paper does not test whether the visual–programmatic mapping learned from English training data transfers to instructions in other languages—a significant gap for a model positioned as "foundational."

What evidence exists in the paper. The paper provides only two general coding data points (BigCodeBench-Hard pass@1 and LiveCodeBench-V6 pass@1 in Figure 5 and Figure 6), and these are presented as scatter plots alongside visualization benchmarks to demonstrate "balanced capabilities" rather than as a thorough general coding evaluation. No pass@k (k > 1) results are reported. No non-English evaluation is discussed. The backbone model comparison in Figure 4 shows that JanusCode-800K training consistently improves PandasPlotBench and InteractScience scores, but does not test whether general coding performance is preserved or degraded.

Mitigation status. Partially acknowledged. The paper explicitly evaluates on BigCodeBench and LiveCodeBench (Section 5.2) to "highlight its capability in following complex instructions and algorithmic capability," recognizing that general coding competence is a relevant metric. However, this gesture toward generality is limited to two benchmarks and does not constitute a systematic analysis of the breadth–depth tradeoff. The paper does not discuss whether the unified visual–programmatic training causes regression on any coding sub-skills.

Limitation 4: SVG Performance Regression and Unexplained Domain-Specific Weaknesses

The assumption or constraint. The paper presents JanusCoder as a unified model that achieves strong performance across a broad spectrum of visual–programmatic tasks. The implicit assumption is that training on diverse domains produces uniformly improved or at least non-degraded performance across all covered domains.

The consequence. This assumption is violated for at least one domain: SVG generation. Table 13 shows that JANUSCODER-14B achieves an ArtifactsBench SVG sub-score of 24.72, compared to 39.50 for Qwen3-14B (the pre-fine-tuning backbone) and 39.54 for GPT-4o. This is a 37.4% relative regression from the base model—JanusCode-800K training made the model substantially worse at SVG generation. Similarly, JANUSCODER-8B scores 30.47 vs. Qwen3-8B's 36.37, a 16.2% relative decline. The paper acknowledges the SVG weakness in passing (Section 5.3: JANUSCODER-14B is noted as "underperforming on SVG Generation") but offers no explanation, no ablation, and no analysis of why this domain-specific regression occurs. This matters because it demonstrates that the unified training approach involves genuine tradeoffs: gains in some domains (WEB, SI, MS) come at the cost of losses in others (SVG). A practitioner whose primary use case is SVG generation would be actively harmed by using JanusCoder over the base Qwen3 model.

The broader concern is whether similar regressions exist in other sub-domains that are not independently measured. The benchmarks in the paper aggregate performance at the task level (chart generation, webpage generation, etc.), but within each task there may be specific output types or difficulty levels where JanusCoder underperforms its backbone. The paper provides no per-category breakdown for ChartMimic (e.g., bar charts vs. line charts vs. scatter plots) or for WebCode2M (e.g., simple landing pages vs. complex dashboards) that would reveal whether the aggregate improvements mask sub-domain regressions.

What evidence exists in the paper. Table 13 provides the raw numbers. No ablation isolates which component of JanusCode-800K (which synthesis strategy, which data source, which training data category) causes the SVG degradation. The SVG data in JanusCode-800K comes from MMSVG-Icon and MMSVG-Illustration (Table 6: 20K samples), which constitutes only 2.5% of the total corpus. It is plausible that the model's limited exposure to SVG during training, combined with interference from structurally dissimilar domains (Python visualizations, HTML/CSS), dilutes the SVG-specific knowledge from pre-training. But this hypothesis is not tested—the paper does not, for instance, train an ablation model with increased SVG sampling weight to see if the regression can be mitigated.

Mitigation status. Not addressed. The SVG result is mentioned in the main text as an observation without follow-up. The paper does not discuss rebalancing the corpus to address domain-specific weaknesses, curriculum learning strategies to prevent interference, or post-hoc fine-tuning on the affected domain. For a paper that claims the model "can serve as a strong open-source foundational model for future research and applications," the unexplained regression on a covered domain is a significant gap in the characterization of model reliability.

Limitation 5: The Training Data Synthesis Model (gpt-oss-120b) Creates an Undisclosed Dependency and Potential for Systematic Bias

The assumption or constraint. All natural language instructions and code in the JanusCode-800K synthesis pipeline are generated using gpt-oss-120b (OpenAI, 2025), a 120-billion-parameter model mentioned once in Section 5.1 and once in the Appendix. The paper's four synthesis strategies (Guided Evolution, Re-contextualization, Reverse Instruction, Bidirectional Translation) all depend on this single model to produce the training data on which JanusCoder is fine-tuned. The assumption is that gpt-oss-120b's outputs are sufficiently diverse and unbiased that models trained on them will develop generalizable visual–programmatic competence rather than learning to imitate gpt-oss-120b's specific coding style, error patterns, and visual aesthetic preferences.

The consequence. JanusCoder's capabilities may be bounded by gpt-oss-120b's capabilities and biases in ways that are invisible in the evaluation. If gpt-oss-120b systematically prefers certain chart color palettes, certain HTML layout patterns, or certain Mathematica idioms, JanusCoder will learn these as "correct" and may fail when users request styles outside gpt-oss-120b's typical output distribution. More subtly, if gpt-oss-120b makes systematic errors in certain coding domains (e.g., specific Manim API versions, edge cases in Mathematica symbolic computation), these errors will be propagated into the training data and possibly amplified through the synthesis strategies that iterate on model outputs. The reverse instruction strategy is particularly vulnerable to this: if f_reverse misinterprets a code snippet's intent, the resulting instruction-code pair will be semantically misaligned, teaching the target model an incorrect mapping.

The paper's cross-model adjudication study (Table 14) validates that the reward scoring is robust across judge models, but this does not validate the data generation itself. The generated instructions and code could be systematically biased or limited in ways that all three adjudication models agree are high-quality because they share similar training distributions and biases. There is no human evaluation of the synthesized training data's diversity, correctness, or representativeness.

What evidence exists in the paper. Very little. The paper does not characterize gpt-oss-120b's performance on the synthesis tasks—no pass rate, no error analysis, no comparison to alternative synthesis models (e.g., Claude, Gemini, open-weight models). The paper does not report the fraction of synthesized samples that pass execution validation on the first attempt vs. require retries, which would provide a signal about gpt-oss-120b's code generation quality. The paper does not measure the diversity of the synthesized corpus (e.g., vocabulary diversity, code pattern entropy, visual style coverage) relative to human-written code corpora. The human evaluation in Appendix G.4 (Table 15) evaluates the final JanusCoder outputs, not the intermediate training data, so it cannot detect training data quality issues that the model overcomes through scale.

Mitigation status. Not addressed. The choice of a single proprietary model as the data generation engine is treated as an implementation detail rather than a methodological decision with implications for reproducibility and bias. Researchers attempting to replicate the work without access to gpt-oss-120b (which is not specified as publicly available in the paper) would need to substitute a different model, producing a potentially different corpus with unknown effects on downstream model quality. The paper does not discuss whether the synthesis strategies would work with open-weight alternatives.

Limitation 6: Test Set Contamination Risk from Source Datasets Used in Corpus Construction

The assumption or constraint. JanusCode-800K is constructed by aggregating and synthesizing data from multiple publicly available sources, including WebCode2M (Gui et al., 2025) for WebUI data and VisCoder (Ni et al., 2025) for Python visualization data. Table 6 shows that 200,000 WebUI generation samples and 69,501 WebUI editing samples come from WebCode2M, while 200,000 Python visualization samples come from VisCoder. The assumption is that these source datasets' training splits do not overlap with the test splits of the evaluation benchmarks (particularly WebCode2M and PandasPlotBench, respectively), and that the synthesis strategies (Guided Evolution, Re-contextualization, Reverse Instruction) produce sufficiently transformed samples that even if source-test overlap exists, the resulting JanusCode-800K samples are not memorizable copies of test instances.

The consequence. If test set contamination exists—for example, if WebCode2M's public release includes test split URLs or screenshots that were inadvertently included in the source data ingested by the JanusCode-800K pipeline—then the reported results on WebCode2M, PandasPlotBench, and potentially other benchmarks would overstate JanusCoder's generalization ability. The extremely strong TreeBLEU results on WebCode2M (JanusCoderV-7B achieving 26.21 vs. GPT-4o's 13.00, Table 11) are particularly striking and could be partially explained by memorization if WebCode2M test instances were present in the training data. Similarly, the PandasPlotBench results (JanusCoder-14B matching GPT-4o at 9.7% incorrect code rate) benefit from training on VisCoder data, which shares the same domain and likely the same underlying DataFrame structures and plotting conventions.

This is not an accusation of contamination—the paper may have carefully decontaminated—but rather the observation that the paper does not discuss decontamination at all, making the risk unquantifiable for readers. This is a standard concern in the LLM training literature (e.g., the BigCode project's detailed decontamination reports for StarCoder), and its absence here is notable given that multiple evaluation benchmarks share data sources with the training corpus.

What evidence exists in the paper. None. The paper does not mention decontamination procedures, does not report overlap analysis between JanusCode-800K and the evaluation benchmarks, and does not discuss whether the synthesis strategies are sufficient to prevent memorization of source data. The VisCoder and WebCode2M samples in JanusCode-800K are listed as "sampled" in Table 6, but the sampling procedure is not described—did the authors verify that sampled instances do not include test split data? Were the synthesis strategies applied to all sourced data or only a subset? The paper does not say.

Mitigation status. Not addressed and not acknowledged as a limitation. This is the most straightforward risk to address (e.g., by reporting n-gram overlap or embedding similarity between training and test data, or by describing the data split verification process), and its complete absence from the paper is a significant gap in the experimental methodology. The paper's strong performance on benchmarks that share data sources with the training corpus should be interpreted with appropriate caution until decontamination is independently verified.

7. Implications and Future Directions

How This Work Changes the Landscape

JanusCoder represents a reframing rather than a paradigm shift—it does not introduce a novel architecture, training objective, or inference algorithm, but rather demonstrates that the primary bottleneck in multimodal code intelligence is data, not modeling. The paper's core methodological move—building a versatile data synthesis toolkit that produces 800K high-quality, cross-domain samples—produces models that approach or exceed GPT-4o on visual–programmatic tasks using standard open-weight backbones (Qwen3, Qwen2.5-VL, InternVL3.5). This shifts the conversation from "how do we build a better chart-to-code architecture?" to "what is the data recipe that produces general visual–programmatic competence?"—a reframing that redirects research effort from model design toward data engineering.

The reframing has several concrete consequences for how the field operates:

It destabilizes the specialization paradigm. Prior to this work, the dominant approach to visual–programmatic tasks was task-specific: one dataset and one model for chart-to-code (ChartCoder; Zhao et al., 2025b), another for WebUI generation (DesignCoder; Chen et al., 2025d), another for Python visualization (VisCoder; Ni et al., 2025). JanusCoder demonstrates that a single unified model outperforms specialized models on their own target tasks (Section 5.4: JanusCoderV surpasses "recently released specialized chart-to-code MLLMs" on ChartMimic, and JanusCoder outperforms VisCoder on PandasPlotBench in Figure 5). If a generalist beats specialists, the economic argument for building specialists collapses—why maintain separate training pipelines, separate evaluation suites, and separate deployments when a single model handles all tasks at superior or competitive quality? This does not mean specialization is dead (there will always be tasks where in-domain overfitting provides an edge), but it raises the bar: a specialist must now demonstrate a substantial margin over a well-trained generalist to justify its existence, rather than assuming the generalist approach is inherently limited.

It resolves the cross-domain transfer debate from speculation to measurement. The question of whether training data from one visual programming domain transfers to another has been largely speculative in prior work. Some researchers assumed positive transfer (the "rising tide lifts all boats" hypothesis), while others warned about negative interference (the "jack of all trades, master of none" concern). JanusCoder's ablation studies (Table 5) provide the first large-scale empirical resolution: cross-domain transfer is real, substantial, and domain-asymmetric. Removing text-centric data degrades InteractScience functional score by 27.1% relative (from 17.73 to 12.93)—this is strong evidence for positive cross-modal transfer from text-only code to vision-grounded scientific demonstrations. Removing algorithm data degrades ArtifactsBench by 29.6% relative (from 25.14 to 17.71)—evidence that algorithmic reasoning data transfers to visual artifact generation. But removing chart-to-code data also degrades WebCode2M (from 75.78 to 71.92, a 5.2% relative decline), suggesting that chart-to-code training provides transferable visual-to-code mapping skills that benefit webpage generation. These numbers convert a philosophical debate into an empirical science: for any proposed pair of domains, one can now measure the transfer coefficient by training with and without cross-domain data, characterizing the synergy landscape quantitatively.

Critically, the paper also provides evidence for negative transfer: JanusCoder-14B's ArtifactsBench SVG sub-score drops from 39.50 (Qwen3-14B base) to 24.72 after JanusCode-800K training—a 37.4% relative regression (Table 13). This demonstrates that the "unified training always helps" assumption is false; some domains suffer interference when trained alongside others, and a foundational model for multimodal code intelligence must contend with these tradeoffs rather than ignoring them. The implication is not that unification should be abandoned, but that domain weighting during training becomes a critical hyperparameter—future work will need to characterize the interference surface across visual programming domains, identifying which combinations are synergistic and which are antagonistic.

It establishes VLM-based quality assessment as a necessary component of visual code data pipelines. The paper's most impactful methodological contribution may be the demonstration that executability is not a sufficient quality signal for visual code training data, and that multi-dimensional VLM assessment (task completion, code quality, visual clarity, task relevance) provides a quality signal that measurably improves downstream model performance. The ablation in Table 5 quantifies this: removing reward modeling drops ChartMimic from 68.74 to 58.26—a 15.2% relative decline that dwarfs most other ablations. This result will likely shift community standards: future visual code datasets that filter by executability alone will be viewed as incomplete, much as text-code datasets that filter by compilation alone are now viewed as insufficient relative to those that include unit test execution. The cross-model adjudication study (Table 14, Krippendorff's α > 0.70 across three frontier VLMs) provides the necessary validation that VLM judgments are objective enough to serve as a quality gate, addressing the natural concern that VLM-based scoring might encode arbitrary aesthetic preferences.

It makes certain research directions more attractive and others less so. Research on novel architectures for specific visual–programmatic tasks (e.g., specialized visual encoders for chart understanding, layout-aware decoders for WebUI generation) becomes harder to justify: if standard VLMs fine-tuned on diverse data match or exceed specialized architectures, the marginal benefit of architecture innovation is small unless it addresses a capability that data diversity cannot provide (e.g., handling novel chart types not represented in training, or generating code in programming languages with extremely limited training data). Conversely, research on data synthesis methodologies becomes far more attractive—the paper demonstrates that careful design of synthesis strategies (guided evolution, re-contextualization, reverse instruction, bidirectional translation) can produce training data that teaches general visual–programmatic competence, and the space of possible synthesis strategies is largely unexplored. Similarly, research on cross-domain interference and optimal data mixing becomes critical: if training on domain A helps domain B but hurts domain C, the obvious next step is to learn the mixing weights that maximize desired capabilities while minimizing regressions, following the data mixing literature that has emerged in LLM pre-training.

Follow-Up Research This Work Enables

Characterizing the cross-domain transfer matrix for visual programming. JanusCoder's ablation studies (Table 5) provide point estimates of cross-domain transfer for specific domain pairs (text-centric → InteractScience, algorithm → ArtifactsBench, chart-to-code → WebCode2M), but the full transfer matrix—showing how every domain in JanusCode-800K affects every other domain at each scale—is unknown. A direct follow-up would train a panel of models, each omitting one domain from the corpus, and evaluate all benchmarks to construct an N×N transfer matrix. This would answer questions the current ablations cannot: does Manim animation data help or hurt WebUI generation? Does SVG data transfer to chart-to-code (shared vector graphics knowledge) or interfere with it (different coordinate systems and APIs)? The matrix would also reveal clusters of synergistic domains—perhaps all Python-based visualizations (Matplotlib, Manim, scientific computing) form a positive cluster, while declarative formats (SVG, HTML/CSS) form another, with limited transfer between clusters. This is the quantitative foundation needed to move from "cross-domain synergy is good" to "here is the optimal domain mixture for a given target task profile."

Scaling the data pipeline: does JanusCode-800K quality scale with corpus size? The paper trains on 800K samples but provides no scaling curve—does performance saturate at 200K, 400K, or continue improving to 1.6M and beyond? A direct follow-up would train JanusCoder variants on randomly sampled subsets of JanusCode-800K at sizes of 100K, 200K, 400K, 800K, and (if the synthesis pipeline can produce it) 1.6M samples, measuring performance on all benchmarks. This scaling curve would reveal (a) whether the current 800K corpus is operating in the diminishing returns regime, in which case additional data synthesis effort should focus on new domains rather than more samples from existing domains, and (b) whether different benchmarks have different saturation points—ChartMimic might saturate at 200K while InteractScience continues improving to 1.6M, which would guide data allocation for domain-specific applications. The experiment would also test whether the data quality ceiling imposed by gpt-oss-120b (the synthesis model) can be overcome by sheer quantity, or whether more data from the same generator eventually hits a quality plateau determined by the generator's own capability limits.

Testing whether smaller, cheaper reward models can provide equivalent quality filtering. The paper uses Qwen2.5-VL-72B-Instruct and Qwen3-235B-A22B as reward judges—models far larger than the 7B–14B targets being trained. A critical practical question is whether this gap is necessary. A direct follow-up would replicate the reward filtering step using judge models at multiple scales (e.g., Qwen2.5-VL at 7B, 32B, 72B) and train JanusCoder variants on the resulting filtered corpora, measuring downstream performance. If a 7B judge produces a corpus that yields performance within, say, 2% of the 72B judge on the main benchmarks, the computational barrier to extending the pipeline to new domains drops by orders of magnitude. If the gap is large, it establishes a "judge scaling law" that quantifies how much compute must be invested in quality assessment to achieve a given downstream performance, which would be a novel contribution to the data curation literature. The cross-model adjudication study (Table 14) already shows that three large frontier models agree strongly (α > 0.70), but this does not address the scale axis—do small and large models of the same family agree? The experiment would also test whether distillation (training a small judge model on the large judge's scores) can recover most of the benefit at lower cost.

Stress-testing JanusCoder on adversarial visual-programmatic inputs. All benchmarks in the paper use relatively clean inputs: well-formatted reference charts (ChartMimic), professionally designed webpage screenshots (WebCode2M, DesignBench), clear textual instructions. Real-world usage involves inputs that are noisy, ambiguous, or adversarially challenging: low-resolution screenshots, charts with overlapping or missing labels, instructions with contradictory requirements, webpages with unusual layout conventions. A stress-test suite would systematically degrade inputs along multiple axes—resolution, occlusion, linguistic ambiguity, visual clutter—and measure performance degradation curves. This would characterize JanusCoder's robustness in deployment-like conditions and identify which components of the training pipeline (reward modeling, cross-domain diversity, reverse instruction) contribute most to robustness. If JanusCoder proves brittle to specific degradations (e.g., low-resolution charts cause catastrophic accuracy drops), that would motivate augmenting the synthesis pipeline with data that explicitly introduces such degradations during training, following the adversarial training paradigm.

Extending the synthesis toolkit to non-visual code intelligence domains. The paper's synthesis strategies (Guided Evolution, Re-contextualization, Reverse Instruction, Bidirectional Translation) are described in the context of visual programming, but nothing about them is inherently visual. Reverse Instruction transforming raw code into instruction-code pairs could be applied to general algorithmic code (e.g., converting competitive programming solutions into instruction-following training data for algorithmic reasoning). Bidirectional Translation between semantically analogous domains could be applied to programming language translation (Python ↔ JavaScript, C++ ↔ Rust) or even to translating between natural languages for the same code task. A follow-up would adapt the toolkit to produce training data for non-visual code intelligence tasks and measure whether the same synthesis strategies yield quality improvements comparable to those observed for visual tasks. This would test the generality of the "data diversity through synthesis" thesis and potentially yield a JanusCode-800K-equivalent corpus for general code intelligence, which would be a major resource for the field.

Ablating individual quality dimensions in the reward model. The reward model scores across four dimensions (task completion, code quality, visual clarity, task relevance), but the paper treats them as a composite score. Do all four dimensions contribute to downstream performance, or are some dimensions redundant or even harmful? A follow-up would train JanusCoder variants on corpora filtered by each dimension individually, by all pairs, and by all four dimensions, measuring downstream performance. It's possible that "visual clarity" filtering is the primary driver of improvement on ChartMimic and WebCode2M, while "task completion" is what matters for InteractScience and PandasPlotBench. If so, the reward model cost could be reduced by scoring only the dimensions relevant to the target deployment domain. Conversely, some dimensions might introduce noise—"task relevance" assesses the realism of the instruction itself, which could filter out unusual but valid tasks that would improve model robustness. Characterizing this would turn the reward model from a fixed quality gate into a tunable filter that can be configured per deployment scenario.

Practical Applications and Downstream Use Cases

Automated front-end prototyping from design mockups. A design team produces webpage mockups as screenshots (from Figma, Sketch, or hand-drawn wireframes) and needs to generate working HTML/CSS prototypes for usability testing or stakeholder review. JanusCoderV-7B achieves a Visual score of 75.78 and TreeBLEU of 26.21 on WebCode2M (Table 4, Table 11), producing structurally correct DOM trees (TreeBLEU surpassing GPT-4o by 2×) while maintaining competitive visual fidelity. The high TreeBLEU is particularly valuable in this setting—a prototype that looks roughly right but has a well-structured DOM is more useful for downstream engineering than one that looks perfect but is structurally incoherent, since the former can be refined by developers while the latter must be rebuilt from scratch. The model's moderate visual fidelity gap (75.78 vs. GPT-4o's 82.67) is acceptable for prototyping, where pixel-perfect accuracy is less important than structural soundness and speed. A team using JanusCoderV-7B (open-weight, self-hosted, no API cost per query) rather than GPT-4o (proprietary, per-token pricing) for a pipeline that generates hundreds of prototypes per day could see substantial cost savings while achieving structurally superior outputs.

Data visualization generation in scientific workflows. A research lab routinely generates figures for papers—scatter plots, heatmaps, time series, distribution plots—from DataFrames containing experimental results. JanusCoder-14B achieves a 9.7% incorrect code rate on PandasPlotBench (Table 7, matching GPT-4o) with a Task score of 86 (exceeding GPT-4o's 85), meaning it produces correct, instruction-aligned plotting code at a rate comparable to the strongest proprietary model. The key practical advantage is batch processing: generating 50 figures for a paper from a spreadsheet of experimental conditions can be done with a self-hosted 14B model at negligible inference cost, avoiding per-query API fees for each figure. The model also handles editing (Table 3: Python Visualization Editing, 51.8K training samples in the corpus), so iterative refinement—"make the y-axis log scale," "change the color palette to viridis," "add error bars"—can be done in a conversation loop without regenerating figures from scratch. The moderate Visual score (67 vs. GPT-4o's 72) means a human-in-the-loop may need to make minor aesthetic adjustments, but the task-level correctness (86 vs. 85) means the core data representation is reliable.

Interactive educational content authoring. An educator wants to create interactive visualizations for a physics course—a damped harmonic oscillator with sliders for mass, spring constant, and damping coefficient; a Gibbs free energy phase diagram for a binary mixture; a cellular automaton simulation—without writing HTML/JavaScript from scratch. On InteractScience, JanusCoderV-7B achieves a PFT Overall score of 17.73% and VLM-Judge of 27.67 (Table 12), substantially outperforming all open-weight alternatives (next best: InternVL3.5-8B at 11.47% and 24.17) but still well behind GPT-4o (31.07% and 46.01) and Gemini-2.5-Pro (41.87% and 55.26). The absolute numbers indicate that fully automated generation of complex interactive demonstrations is not yet production-ready for any model—success rates of 18–42% mean a human must be prepared to debug failures. However, in an interactive co-creation setting where the educator provides a reference image plus natural language instructions, and iteratively refines the output through code editing (which JanusCoder also supports, per DesignBench editing results: MLLM Score 8.79 vs. GPT-4o's 9.23 in Table 10), the model can serve as a productivity multiplier. The human expert provides domain knowledge and error correction; the model provides initial code generation and implements requested modifications. As a self-hosted open-weight model, JanusCoderV can be deployed without data privacy concerns (educational content data stays on the institution's servers), which is relevant in academic settings where proprietary API usage may be restricted.

Self-improving data generation pipelines for code models. The paper's synthesis toolkit uses gpt-oss-120b (a 120B model) to generate training data for 7B–14B models. This follows a general pattern—strong model generates data, weaker model trains on it—but the cost of repeated API calls to the strong model is a barrier to scale. A self-improvement loop would work as follows: (1) JanusCoder-14B generates code for a large batch of visual–programmatic tasks, (2) execution validation and VLM reward modeling filter the outputs to retain only high-quality samples, (3) these samples are added to the training corpus for a subsequent training round. If the quality of JanusCoder-generated data approaches that of gpt-oss-120b-generated data, the loop becomes self-sustaining—each generation of the model produces the training data for the next generation, analogous to the self-improvement loops explored in STaR and ReST EM for reasoning tasks but applied to visual–programmatic code. The paper's reward modeling pipeline provides the quality gate necessary for this loop—without it, self-generated data would degrade through accumulation of errors—and the cross-model adjudication study (Table 14) validates that the quality gate is objective. A successful self-improvement loop could produce a family of increasingly capable JanusCoder models at decreasing reliance on external proprietary models, which has both practical (cost reduction) and scientific (understanding the limits of self-supervised improvement) implications.