ArXiv: 2604.15309

🎯 Pitch

Current LLM-based web agents generate visually incoherent pages because they treat images, videos, and charts as afterthoughts. MM-WebAgent instead uses a 'plan-then-reflect' hierarchy across global layouts and local assets to achieve style-consistent multimodal integration, boosting image quality scores from 0.05 to 0.88 over code-only baselines.

1. Executive Summary

This paper introduces MM-WebAgent, a hierarchical agentic framework that coordinates multimodal webpage generation through hierarchical planning (global layout plans with embedded local element constraints for images, videos, and charts) and hierarchical self-reflection (iterative refinement at three levels—local asset editing, context-level CSS/embedding fixes, and global layout revision). Evaluated on the authors' introduced benchmark MM-WebGEN-Bench (120 curated webpages spanning 11 scene categories and diverse multimodal compositions), MM-WebAgent with GPT-5.1 achieves an average multi-level score of 0.75, substantially outperforming both code-only one-shot baselines (0.42 for GPT-5.1) and code-only agent baselines (0.46 for GPT-5.1 via bolt.diy), with the largest gains concentrated on multimodal element metrics—image quality reaching 0.88 versus 0.05 for the code-only counterpart and video quality reaching 0.75 versus 0.00. The framework demonstrates a 78.99% human-preference win rate over competing methods, establishing that native AIGC tool integration meaningfully improves multimodal webpage generation only when coupled with explicit hierarchical planning and reflection rather than bolted onto a standard code-generation pipeline (0.45 overall score with tools alone vs. 0.75 with the full framework).

2. Context and Motivation

The Core Problem: Webpage Generation That Ignores Multimodal Content as First-Class Citizens

The fundamental problem this paper tackles is that existing automated webpage generation systems treat multimodal content — images, videos, charts — as afterthoughts rather than as integral components of the design process. Real-world webpages are not pure HTML and CSS skeletons; they contain heterogeneous elements whose visual style, spatial geometry, and semantic meaning must cohere with each other and with the overall page layout. A hero image with the wrong color palette, a chart rendered with clashing typography, or a video that overflows its container boundary each breaks the visual integrity of the page in ways that matter to end users.

This gap is significant for several reasons the paper articulates (Section 1):

  • The AIGC tool landscape has matured. Modern image generation (DALL-E, GPT-Image-1), video generation (Sora), and charting libraries (ECharts) can produce high-quality assets on demand. But these tools operate in isolation — they take a prompt and produce an asset, with no awareness of the webpage context into which the asset will be embedded. The coordination problem — ensuring that assets generated independently by different tools still look like they belong together — is currently unsolved.

  • Human designers work differently. Professional web designers do not generate elements in isolation and then hope they match. As the paper notes (Section 1):

"real-world webpages are not purely text and code—they contain heterogeneous multimodal elements such as images, videos, and charts, whose content, style, and geometry must cohere with the global layout and the semantic intent."

Designers iterate: they sketch a layout, place assets, notice mismatches, adjust individual elements, tweak surrounding CSS, and occasionally rethink the entire section structure. This plan-and-refine loop is absent from current automated pipelines.

  • The deployment stakes are rising. As LLMs become capable of generating entire webpages from natural language prompts, the quality ceiling shifts from "can it produce valid HTML?" to "does the result look professionally designed and visually coherent?" For applications like automated landing page generation, dashboard creation, or rapid prototyping, the difference between a page where generated images awkwardly clash with the color scheme and one where everything looks intentional is the difference between a usable product and a discarded prototype.

Three Failure Modes of Existing Approaches

The paper identifies three specific failure modes that emerge when you try to naïvely combine text-to-code generation with multimodal asset generation (Section 1):

1. Style inconsistency across elements. When each image, video, or chart is generated independently with no awareness of others, they arrive in different visual styles. One image might be photorealistic with warm tones; another might be illustrated with cool tones. The chart might default to a standard library color palette that clashes with both. The resulting page looks like a collage rather than a designed composition. The paper refers to this as a failure to achieve global coherence after assets are composed into the page.

2. Geometry mismatch between generated media and reserved slots. A code-generation pipeline might reserve a 300×200px container for a chart, but the generated chart might be designed for a full-viewport display, causing labels to truncate, legends to overflow, or data to become unreadable. Similarly, an image generated for a hero banner might be the wrong aspect ratio, resulting in awkward cropping or whitespace when forced into its container. These are integration failures — the asset itself might be high-quality, but it doesn't work in context.

3. Global incoherence due to lack of cross-element coordination. Even when individual elements are locally acceptable, the assembled page can feel disjointed. The navigation bar might use one style of rounded corners while the cards use sharp corners. The typography in embedded charts might differ from the page's chosen font family. These are failures of design system enforcement — the kind of consistency that a human designer achieves through a shared style guide, but that current automated systems have no mechanism to maintain.

Where Prior Approaches Fall Short

The paper organizes prior work into three categories and explains why each fails to solve the multimodal coordination problem (Section 2):

Visual code generation from screenshots or retrieved assets. Systems such as those built on the WebSight dataset (Laurençon et al., 2024) or GUI-focused code generators (Guo et al., 2025; Huang et al., 2025) take screenshots as input and reconstruct the underlying HTML/CSS. While these approaches can reproduce layouts faithfully, they treat multimodal assets as given — either as pixels in the input screenshot or as statically retrieved images. The paper notes (Section 2.1):

"While these approaches improve layout fidelity and code correctness, they treat multimodal assets as static or externally provided, limiting their ability to generate novel, semantically aligned, and stylistically coherent multimodal content."

In other words, these systems can copy a design but cannot create one from scratch with generated assets that match a textual intent.

Vision-language code agents. Recent agentic frameworks (OpenHands, bolt.diy, WebGen-Agent, UICopilot, ScreenCoder, DesignCoder) decompose webpage generation into planning and execution steps, often incorporating visual feedback from rendered pages. The paper acknowledges that approaches like WebGen-Agent (Lu et al., 2025) — which uses multi-level feedback and reinforcement learning — represent a step forward in iterative refinement. However, it draws a critical distinction (Section 2.2):

"Although these methods enhance code correctness or layout reconstruction, their hierarchies are still limited to reasoning or code granularity. In contrast, we define hierarchy at the design abstraction level, representing a shift from code-centric orchestration to design-abstraction-driven multimodal generation with structured cross-modal refinement."

The key phrase is hierarchy at the design abstraction level. Code-only agents understand the problem as: decompose the task → write HTML → write CSS → done. MM-WebAgent understands it as: plan the global layout with explicit multimodal slots → generate each asset with awareness of its context and the overall style → iterate at all three levels (asset, embedding, global) until the page looks cohesive. The difference is not just about adding tool-calling for image generation — it's about restructuring the planning hierarchy so that multimodal assets are treated as first-class design decisions, not late-stage insertions.

Webpage generation benchmarks. The paper argues that existing benchmarks are poorly suited for evaluating the kind of multimodal coordination that matters. As described in Section 2.3:

  • Code-centric benchmarks (e.g., Web2Code; Yun et al., 2024) focus on HTML/CSS correctness and ignore visual content quality.
  • Image-to-code benchmarks (e.g., WebCode2M, DesignCoder's data; Gui et al., 2025) evaluate reconstruction fidelity from screenshots rather than generation from intent.
  • Asset-placeholder benchmarks (e.g., WebGEN-V Bench; Wang et al., 2025) provide static images to place into generated pages, meaning they test layout skill but not the ability to generate appropriate assets.

The consequence is that a system could score perfectly on existing benchmarks by producing pages with generic placeholders or mismatched stock images, while a system that produces genuinely coherent multimodal pages might score identically. There is no benchmark that asks: given a textual description of a webpage, can you generate both the code AND the native multimodal assets such that everything looks like it belongs together?

How This Paper Positions Itself

MM-WebAgent does not propose a fundamentally new generation model or architecture. Instead, it proposes a coordination framework — a structured way to compose existing AIGC tools (GPT-Image-1, Sora-2, ECharts via GPT-5.1) such that their outputs become mutually consistent. The framework is explicitly modeled on the workflow of human designers (Section 1):

"Motivated by the iterative workflow of human designers, we argue that multimodal webpage generation should be treated as a structured plan-and-refine process, where global layout decisions and local asset generation are explicitly coordinated and repeatedly refined."

The paper positions its contribution along three axes (Section 1, contributions list):

First, it introduces the concept of a multimodal web agent paradigm that goes beyond code-only generation. This is not just about calling image generation APIs from within a code agent — it is about restructuring the plan hierarchy so that global layout decisions (section structure, color palette, typography) flow downward to constrain local element generation, and local element properties (style, size, semantics) flow upward to inform layout adjustments. This bidirectional coupling is what distinguishes the approach from simply bolting AIGC tools onto a standard agent pipeline.

Second, it proposes a hierarchical self-reflection mechanism operating at three granularities that correspond to how human designers actually iterate: fix individual assets first (local), then fix how they're embedded (context), then fix the overall page (global). This is more structured than a generic "reflect and revise" loop — it decomposes reflection into independent, parallelizable sub-problems at each level.

Third, it introduces MM-WebGEN-Bench and a corresponding multi-level evaluation protocol that explicitly scores both global page-level quality (layout correctness, style coherence, aesthetics) and local element-level quality (image relevance/visibility/consistency, video content/motion/role-appropriateness, chart data accuracy/readability/styling). This shift from a single holistic score to a decomposed evaluation is what enables the paper's key empirical insight: that the largest gains from the agent framework come specifically from multimodal element quality, not from general layout improvement (where code-only agents already perform reasonably well).

The paper is transparent about its scope: it is an orchestration framework, not a learning-based system. Section 6 explicitly notes:

"MM-WebAgent adopts an orchestration-based, training-free agentic formulation. Although this choice allows us to clearly study the impact of hierarchical planning and reflection, it does not leverage learning-based optimization of agent behaviors."

This is a deliberate trade-off — by avoiding fine-tuning or reinforcement learning, the paper isolates the architectural contribution of the coordination framework from the confounding effect of model optimization, enabling clean ablation studies (Table 3, Table 4) that demonstrate exactly which components drive the gains.

The Key Ablation That Validates the Core Claim

Perhaps the most important experiment for understanding the paper's motivation appears in Table 4. This ablation compares three settings: (1) standard code-only GPT-5.1 with no AIGC tools (Overall: 0.42), (2) the same code-only pipeline augmented with direct access to identical AIGC tools (Overall: 0.45), and (3) the full MM-WebAgent framework with hierarchical planning and reflection (Overall: 0.75).

The result is striking: simply giving a code agent access to image, video, and chart generation tools provides almost no benefit. The Overall score barely moves (0.42 → 0.45). It is only when tool access is coupled with the hierarchical planning structure — where each tool call is grounded in a global style context and a local element plan specifying its functional role, size constraints, and design attributes — that the tools become effective. This is the paper's central argument distilled into a single experiment: the bottleneck is not tool availability, but coordination.

This finding explains why prior agent-based systems that include tool-calling capabilities would not automatically solve the multimodal webpage generation problem. Without explicit design-abstraction-level hierarchy and cross-modal feedback, tools are invoked in isolation and produce assets that don't cohere. The paper's hierarchical planning and reflection framework is the mechanism that closes this coordination gap.

3. Technical Approach

3.1 Reader Orientation

MM-WebAgent is an orchestration framework — a structured coordination layer that sits above existing AIGC tools (image generators, video generators, chart libraries) and a large language model (LLM), directing them to collaboratively produce a complete, visually coherent webpage from a natural-language design prompt. The problem it solves is the cross-modal coordination gap: when you ask an image generator, a video generator, and a code generator to each produce their respective pieces independently, the results don't match — colors clash, aspect ratios are wrong, charts look alien to the surrounding layout. The shape of the solution is a hierarchical plan-and-refine loop. First, the system creates a design blueprint that specifies both the global page structure and explicit constraints for every multimodal element (what it should look like, how big it should be, what role it plays). Then it generates all assets in parallel under those constraints. Then it iteratively inspects and fixes problems at three granularities — individual elements, surrounding embedding code, and the overall page — until the output looks like a single coherent design rather than a collage of independently generated pieces.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major functional blocks:

  1. Planning Agent (LLM-based): Takes a natural-language design prompt and produces a structured JSON plan with four sections: a detailed HTML layout description, image generation specifications, video generation specifications, and chart/data-visualization specifications. This is the "global blueprint."

  2. Layout Agent: Converts the global layout description into a complete HTML/CSS file, creating section structure and placeholder slots for all multimodal elements. The layout agent is constrained to use exact file paths specified in the plan.

  3. Specialized Generators (Tool Layer): Three separate tools execute the local plans:

    • Image Agent (GPT-Image-1) generates .png assets from structured prompts that include context (which section, what role, what page style) and compiled attributes (visual style, color tone, composition, lighting).
    • Video Agent (Sora-2) generates .mp4 assets from prompts specifying visual style, motion intensity, camera behavior, and loopability.
    • Chart Agent (GPT-5.1 generating ECharts HTML) produces self-contained .html files with transparent backgrounds, sized to fit their iframe containers.

  4. Evaluation Manager (LLM-based): After initial assembly, renders the complete webpage as a screenshot. Then runs a multi-level evaluation that decomposes quality into six dimensions: three global (Layout, Style, Aesthetics) and three local (Image quality, Video quality, Chart quality). Each dimension uses either a penalty-based scoring formula or a graded 0–1 scale. The evaluation produces both numeric scores and natural-language issue descriptions.

  5. Reflection Manager (LLM-based): Takes the issue descriptions from the evaluation stage and dispatches them to three specialized reflection agents:

    • Local Refine: fixes individual asset quality issues (e.g., regenerate an image with different colors, fix a chart's axis labels).
    • Context Refine: patches surrounding HTML/CSS to fix embedding problems (e.g., misalignment, overflow, spacing, opacity bugs).
    • Global Refine: revises the entire page HTML using both code and rendered screenshots as reference, adjusting layout balance, section order, and style coherence.

Information flows as follows: Prompt → Planning Agent (global + local plans) → Layout Agent (HTML skeleton with placeholders) + parallel Image/Video/Chart Agents (assets) → assembly into complete webpage → Evaluation Manager (multi-level scoring and issue detection) → Reflection Manager (filters issues by level, dispatches to appropriate refine agent) → refined webpage → (loop back to evaluation, up to 3 iterations) → final webpage.

3.3 Roadmap for the Deep Dive

  • First, the Planning Agent — how the global layout plan and local element plans are structured, what information each contains, and why the two-level decomposition is necessary. This is the linchpin: it's where cross-modal constraints are encoded.
  • Second, the Layout Agent and specialized generators — how the global HTML skeleton is created, how each multimodal asset is generated with context awareness, and the specific tool configurations (prompt templates, size constraints, transparency requirements).
  • Third, the multi-level evaluation protocol — the scoring formulas, the penalty-based versus graded scoring strategies, the specific dimensions evaluated at global and local levels, and the evaluation prompt design.
  • Fourth, the hierarchical reflection mechanism — how issues flow from evaluation to reflection, the three reflection levels and their responsibilities, the reflection prompts, and the iteration strategy.
  • Fifth, the benchmark construction pipeline (MM-WebGEN-Bench) — how the evaluation dataset is created, the four-dimensional sampling space (layout complexity, visual style, multimodal elements, semantic intent), and the filtering/curation process.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems design paper whose core idea is that multimodal webpage generation should be decomposed into a two-level planning hierarchy (global + local), executed with context-constrained AIGC tools, and then iteratively refined through a three-level reflection process. There are no learned components — everything is orchestration of frozen models and tools.

The Planning Agent: Two-Level Decomposition

The Planning Agent is implemented using GPT-5.1 (Section 4.1) and is responsible for converting a free-form design prompt into a structured JSON plan that specifies everything the downstream agents need to generate. The key design decision is that this plan operates at two levels — global and local — and the local level inherits constraints from the global level.

Global Layout Planning. This defines the overall webpage structure. The planner produces a detailed natural-language description for the code_generation tool that includes (Prompt 1, Appendix B.1):

  • All sections of the webpage (hero banner, navigation bar, logo, gallery, footer, etc.)
  • For each visual element that requires an image, video, or chart: an explicit file path reference and a reasonable layout size (width and height in pixels or percentage of viewport). For example: "Hero image shows a cozy café interior at sunrise (path: assets/hero_cafe.png, width: 1200px, height: 600px)" or "Sales chart comparing 2022 revenue across cities (path: assets/chart_sales.html, width: 600px, height: 400px)".
  • Layout structure, typography, color palette, and mood.
  • For multiple charts: each chart must be a separate HTML file with individual file path references.

This is the critical interface point: the global plan embeds placeholders with sizing constraints directly into the layout description. When the Layout Agent later generates HTML, it encounters these path references and creates appropriately sized containers. There is no separate "slot allocation" step — the constraints are part of the plan text itself, enforced by instructing the Layout Agent to "strictly preserve image references" and use "exactly the same file path" (Prompt 8).

Local Element Planning. After completing the global layout description, the planner extracts structured specifications for each multimodal element. These are organized by modality (image, video, chart) in the JSON output. Each specification includes (Prompt 1, Appendix B.1):

For images (image_generation array):

  • save_path: file path matching the reference in the global plan (e.g., "assets/hero_cafe.png")
  • context: an object with three inheritable fields:
    • section: which webpage section the image appears in (e.g., "hero", "feature card")
    • role: functional role in layout (e.g., "background", "illustration", "accent")
    • page_style: overall webpage visual style (e.g., "modern minimal", "playful", "corporate")
  • compiled_attributes: modality-specific generation parameters:
    • visual_style: "photorealistic", "illustration", "abstract", "UI-style", etc.
    • color_tone: "muted", "vibrant", "monochrome", "pastel", etc.
    • composition: "wide shot", "centered object", "negative space", "cropped detail"
    • lighting: "soft natural", "studio lighting", "flat", "high contrast"
  • prompt: a descriptive visual prompt for the image generator, incorporating the context and attributes
  • size: output dimensions — exactly one of "1024x1024", "1024x1536", or "1536x1024"

For videos (video_generation array):

  • save_path, context (same structure as images)
  • compiled_attributes:
    • visual_style: "cinematic", "UI-style", "abstract", "illustrative"
    • motion_intensity: "low" or "medium"
    • camera_behavior: "static", "slow pan", "subtle zoom"
    • loopability: whether the video should loop seamlessly
  • prompt: detailed visual description
  • seconds: 4, 8, or 12
  • size: "720x1280", "1280x720", "1024x1792", or "1792x1024"

For charts (data_visualization array):

  • save_path, context (same structure)
  • compiled_attributes:
    • chart_type: "bar chart", "line chart", "stacked area chart", "radar chart", "heatmap", etc.
    • chart_style: "clean", "dense", "presentation-oriented", "dashboard-style"
    • color_palette: aligned with webpage colors
    • visual_emphasis: which data dimensions should stand out
  • prompt: detailed description including chart type, colors, typography, visual aesthetics, labels, legends, axes, annotations, transparency, animations
  • source_data: the complete dataset content in markdown format (this is mandatory — the planner must include it, per the prompt instruction: "the complete dataset content required to visualize the chart must be included in this field")

Why this two-level decomposition matters. The context fields (section, role, page_style) are duplicated across all modalities. This is the mechanism by which global design intent flows downward: every image, video, and chart receives the same page_style descriptor, so their respective generators are all aiming for a compatible look. The compiled_attributes fields are modality-specific but still constrained by the context — an image in the "hero" section with the role "background" and style "modern minimal" should have a muted color tone and wide-shot composition, while an image in a "gallery" section with the role "illustration" and style "playful" might use vibrant colors and centered composition. The planner makes these decisions based on the global design prompt.

A design choice worth noting: the planning prompt (Prompt 1) does not use function-calling or structured tool APIs. Instead, it instructs the LLM to output a JSON object with specific keys. This means the LLM is doing free-form planning within a structured output format — it decides for itself which multimodal elements to include, what file paths to assign, and what context/attribute values to specify, all based on its interpretation of the user's design prompt. There is no separate validation step between planning and execution — if the planner hallucinates a nonsensical combination (e.g., a "brutalist" style with "soft natural" lighting), that flows directly into generation without correction (though subsequent reflection stages may catch it).

The Layout Agent: Global Structure Generation

The Layout Agent (Prompt 8, Appendix B.1) takes the code_generation description from the plan and produces a complete, self-contained HTML file. The prompt imposes four constraints:

  1. Output only valid HTML code — no explanations, comments, or markdown formatting. The output must be directly savable as .html and openable in a browser.
  2. Strictly preserve image references — any reference in the format (path: xxx) must appear in the HTML with exactly the same file path, in <img src="...">, <source> tags, or CSS background-image URLs. No modification, renaming, or relocation is permitted.
  3. Faithful representation of the described layout, structure, and style, including semantic sections, visual hierarchy, color palette, font choices, and theme.
  4. Minimal inline CSS or internal <style> tags for visual coherence.

The Layout Agent does not know about the image, video, or chart files that will be generated — it only knows the file paths embedded in the plan description. It creates containers (typically <img>, <video>, or <iframe> elements) pointing to those paths and sizes them according to the plan's width/height specifications. This is a "dumb" execution step — the Layout Agent is not making design decisions, only translating the plan's textual description into valid HTML.

The Specialized Generators: Context-Constrained Asset Creation

Image Agent (GPT-Image-1). Each image specification from the plan is formatted into a prompt using two templates (Prompts 2 and 3, Appendix B.1):

"The image will be incorporated into a {page_style} webpage, serving as a {role} image in the {section} section." "The image should have a {visual_style} visual style, {color_tone} color tone, {composition} composition, and {lighting} lighting."

These are prepended to the prompt field from the plan. GPT-Image-1 then generates the image at the specified size. The model used is GPT-Image-1 (Section 4.1).

Video Agent (Sora-2). Similarly, video specifications are formatted using Prompts 4 and 5, specifying {page_style}, {role}, {section} context and {visual_style}, {motion_intensity}, {camera_behavior}, {loopability} attributes. Sora-2 generates the video at the requested duration and resolution.

Chart Agent (GPT-5.1 + ECharts). Chart generation is more involved because it produces executable code, not a rendered asset. Prompt 10 (Appendix B.1) instructs GPT-5.1 to generate a self-contained HTML file for ECharts-based visualization with six requirements:

  1. Output only valid HTML code, no explanations or markdown.
  2. Use ECharts to render the chart, loaded via CDN.
  3. Chart background must be transparent — so it blends seamlessly when embedded via <iframe> into the parent page.
  4. No layout elements — no header, footer, sections, captions, or descriptive text. Only the chart container.
  5. The chart must occupy the full viewport (100% width, 100vh height initially), removing all margins and padding.
  6. The chart must be responsive, scaling to fit the container while maintaining aspect ratio.

The source_data field from the plan is embedded directly into the chart generation prompt, so the LLM has the actual data to plot. The chart's visual styling (colors, typography, chart type) is specified in the prompt and compiled_attributes fields, which the LLM uses to configure the ECharts option object.

An important subtlety: the chart is generated at 100vh height (full viewport height), but it will be embedded in an <iframe> with a much smaller fixed height (typically 180–300px, based on the layout plan's size specifications). This mismatch is a deliberate two-stage strategy: generate charts at large size for visual quality, then fix sizing issues during the reflection stage. The Chart Local Reflection prompt (Prompt 31) explicitly addresses this: it instructs the refactoring agent to convert height: 100vh to height: 100% and optimize for small container display. This design choice avoids burdening the initial chart generation with container-size awareness (which would require knowing the exact iframe dimensions at generation time), but it means charts always require at least one round of local reflection to display correctly.

Parallel execution. The paper notes (Table 5 note) that image, video, and chart generation are executed in parallel. The planning stage produces all specifications upfront; then all assets are generated simultaneously; then the Layout Agent's HTML is combined with the generated assets to produce the initial webpage. This parallelization is what keeps the total latency competitive (155.8s average) despite the complexity — sequential generation of all elements would be substantially slower.

The Multi-Level Evaluation Protocol

After the initial webpage is assembled, the Evaluation Manager renders it as a screenshot and runs a comprehensive quality assessment across six dimensions. The evaluation is multi-level because it decomposes page quality into global-level criteria (measuring the whole page) and local-level criteria (measuring individual embedded elements).

Global-Level Evaluation uses three dimensions:

  • Layout Correctness: Evaluates whether section hierarchy, ordering, and spatial arrangement of elements match the user's design prompt. Uses a penalty-based scoring mechanism where the evaluator identifies violations against predefined rules (Prompt 11, Appendix B.2):

    • Critical element errors (missing main title, missing hero section, wrong navigation placement): penalty 0.5 each
    • Minor element errors (missing buttons, icons, small text blocks): penalty 0.3 each
    • Positioning/structural errors (misplaced elements, wrong grid layout, incorrect section order): penalty 0.1–1.0 depending on severity
    • Visual detail errors (wrong shape, wrong size dominance, improper spacing): penalty 0.1 each

  • Style Coherence: Measures consistency of visual attributes (color palette, design theme) across all sections. Also uses penalty-based scoring (Prompt 13):

    • Overall style mismatch (entire page deviates from intended style): penalty 0.5
    • Section or component-level mismatch (card style inconsistent, color theme varies by section): penalty 0.2 each
    • Minor style deviations (slightly off color tone, wrong border radius, inconsistent shadows): penalty 0.1 each

  • Aesthetic Quality: Captures visual balance, readability, and harmony. Uses a graded scoring scheme across five sub-dimensions, each rated on {0.2, 0.4, 0.6, 0.8, 1.0} (Prompt 15):

    • Layout Balance and Spacing (grid structure, alignment, visual weight distribution)
    • Typography and Readability (font pairing, hierarchy, legibility)
    • Color Harmony and Hierarchy (palette cohesiveness, contrast, mood)
    • Visual Clarity and Polish (noise levels, iconography consistency, UI detail quality)
    • Overall Professional Aesthetic (consistency, modernity, attention to detail)

Local-Level Evaluation assesses individual multimodal elements. For each element type, the evaluation considers both intrinsic quality (is the asset itself good?) and integration quality (does it fit its container and the page style?).

For images (Prompt 21): The evaluator receives the full-page screenshot, a cropped screenshot of the image as it appears in the page, the original image asset, the user design prompt, and the relevant HTML/CSS excerpt. It evaluates four aspects:

  • Required details are visible and correct
  • No unwanted content (watermarks, artifacts, accidental borders)
  • Consistency with overall webpage style
  • Cropping/clipping/alignment problems from embedding

Scoring: Start from 1.0, subtract 0.2 for each distinct issue. Issues are categorized as either "image issues" (asset itself is wrong — fix by regenerating/editing) or "webpage issues" (embedding is wrong — fix by adjusting CSS). This categorization feeds directly into the reflection stage: image issues go to Local Refine, webpage issues go to Context Refine.

For videos (Prompt 23): The evaluator receives extracted frames from the video (since videos cannot be evaluated in a static screenshot). It evaluates:

  • Subject and theme match with the design prompt
  • Detail consistency (all required attributes present)
  • Motion/temporal coherence across frames
  • Role appropriateness (suitable as background loop, hero animation, etc.)

Scoring: Uses a graded scale: 1.0 (perfect match), 0.8 (strong match), 0.6 (partial match), 0.4 (weak match), 0.2 (very weak match), 0.0 (no relation).

For charts (Prompt 25): The evaluator receives the full-page screenshot, a cropped screenshot of the chart as it appears in its iframe, the chart's HTML/JS source code, the user prompt, and iframe dimension constraints. It evaluates:

  • Visibility and readability at embedded size (labels, legends, data points visible?)
  • Chart type correctness
  • Data accuracy (plotted values match required data?)
  • Stylistic requirements (colors match prompt, typography consistent)
  • Container fit (too cramped? legend overlapping? unnecessary whitespace?)
  • Consistency with webpage's visual style

Scoring: Start from 1.0, subtract 0.2 per distinct issue. Critically, issues are categorized as chart issues (fixable by modifying the chart HTML) or webpage issues (require fixing parent page CSS). The prompt includes a "VISIBILITY FIRST RULE" (Prompt 25): if the chart fails to render or is blank, score = 0 immediately. The evaluator must check for parent CSS issues like opacity: 0, display: none, or visibility: hidden that would make the chart invisible — and categorize these as webpage_issues, not chart_issues.

Missing element handling. The evaluation protocol includes a completeness check (Prompts 17–20): an extraction agent identifies all multimodal elements implied by the design prompt, and a completeness agent compares these against what was actually generated. Missing elements are treated as critical failures at the local level.

Score aggregation. For penalty-based dimensions (Layout, Style), the final score for a sample is:

score=max⁡(0,1−α⋅∑ipi)\text{score} = \max\left(0, 1 - \alpha \cdot \sum_i p_i\right)

where $p_i$ is the penalty associated with the $i$-th detected issue and $\alpha$ is a normalization factor that controls the overall penalty strength.

What it computes: the penalty sum $\sum_i p_i$ tallies all detected issues, with severities encoded as the penalty values described above (0.1, 0.2, 0.3, 0.5, 1.0). This sum is multiplied by a normalization factor $\alpha$, subtracted from 1.0 (a perfect score), and clamped to a minimum of 0. The result is a score in $[0, 1]$ where 1.0 indicates no detected issues and 0.0 indicates cumulative penalties exceeding the normalization threshold.

Why this form: the penalty-based approach decomposes evaluation into discrete, rule-based checks that are easier for an LLM evaluator to apply consistently than a holistic "rate this page from 0 to 1" judgment. Each penalty corresponds to a specific, nameable violation (e.g., "missing navigation bar," "wrong grid layout"), which means the evaluation output is explainable — it produces not just a score but a list of concrete issues that can be fed into the reflection stage. The $\max(0, \cdot)$ clamping prevents negative scores when many small issues accumulate. The $\alpha$ factor provides a global sensitivity knob, though the paper does not specify its exact value — it is described as "a normalization factor that controls the overall penalty strength."

For graded-score dimensions (Aesthetics, Image, Video, Chart), each item receives a score from $\{0, 0.2, 0.4, 0.6, 0.8, 1.0\}$, and the final benchmark score for that dimension is the average across all samples. The overall model performance is the average across all six evaluation dimensions (Layout, Style, Aesthetics, Image, Video, Chart).

Why a multi-level protocol instead of a single holistic score? The decomposition is essential for two reasons. First, it enables differential diagnosis: if a model scores high on Layout but low on Image, the problem is specifically with image generation/coordination, not with layout planning. A single score would obscure this. Second, it maps directly onto the reflection mechanism: each evaluation dimension produces its own set of actionable issues (layout penalties, style penalties, image issues, chart issues, etc.) that can be routed to the appropriate reflection level. A holistic evaluation would produce vague feedback ("this page looks unprofessional") that is hard to translate into concrete fixes.

Hierarchical Self-Reflection: Three-Level Iterative Refinement

After the Evaluation Manager produces scores and issue lists, the Reflection Manager orchestrates iterative improvement at three granularities. The paper describes these as corresponding to how human designers work (Section 3.2): first check individual assets, then check how they're embedded, then check the overall composition.

Reflection iteration strategy. Reflection proceeds until convergence or a maximum of 3 iterations (Section 4.1). The paper reports (Figure 4) that most gains are achieved within the first few rounds. At each iteration, all three reflection levels are applied — not sequentially within a round, but with issues from the evaluation stage categorized and dispatched to the appropriate level. This means that in a single iteration, the system may simultaneously regenerate an image (local), fix a CSS overflow (context), and restructure the navigation layout (global).

Level 1: Local Refine. This level addresses intrinsic quality problems with individual multimodal assets — issues that are fixable by editing the asset itself without changing how it is embedded. The paper identifies several categories (Section 3.2, and reflected in the evaluation prompts' issue categorization):

  • For images: inpainting, color adjustment, object correction, regneration with modified prompts. The Image evaluation prompt (Prompt 21) distinguishes between image_issues (the asset itself is wrong) and webpage_issues (the embedding is wrong). Only image_issues go to Local Refine. These might include: missing required text/details, wrong style/genre, artifacts or watermarks, incorrect color tone relative to the specified color_tone in the local plan.
  • For charts: fixing labels, axes, legends, data values directly within the chart HTML. The Chart evaluation prompt (Prompt 25) similarly distinguishes chart_issues from webpage_issues. Chart issues include: wrong chart type, incorrect data values, style inconsistencies within the chart itself, and — critically — container sizing issues that can be fixed by modifying CSS within the chart HTML (e.g., changing height: 100vh to height: 100%, adjusting internal padding, hiding redundant titles).

How Local Refine executes. For images, the system uses GPT-Image-1 in edit mode (Section 4.1: "image components are refined using GPT-Image-1 (edit)"). For charts, the system uses GPT-5.1 to generate corrected ECharts HTML (Section 4.1: "global layout and chart components are revised using OpenAI-GPT-5.1"). Prompt 31 (Appendix B.3) is the Chart Local Reflection prompt — it provides the original design prompt, the current chart HTML, a list of issues to fix, and the iframe container height. It instructs the refactoring agent to:

"Apply the user's requested fixes exactly and exclusively." "Optimize for iframe embedding: Use height: 100% instead of height: 100vh; ensure html, body have height: 100%; margin: 0; overflow: hidden;; use compact spacing for small containers; consider hiding redundant titles if the parent already shows one."

This prompt enforces responsive chart guidelines that were not present in the initial generation prompt. The initial chart generation prompt (Prompt 10) asks for 100vh height and full-viewport filling — suitable for standalone viewing but wrong for iframe embedding. Local Refine is where this mismatch gets corrected, which is why the paper's ablation (Table 3B) shows that adding Local reflection to the system improves chart scores (0.44 → 0.48 in the "Local reflection only" row of the Chart column, compared to "No reflection").

Level 2: Context Refine. This level addresses problems where individual assets are correct but their integration into the surrounding HTML/CSS is broken. The paper describes these as "issues such as misalignment, clipping, or inconsistent spacing" (Section 3.2). Context Refine analyzes the relevant HTML snippets and generates "targeted structural edits, such as CSS patches, block resizing, or snippet replacement."

The key distinction from Local Refine: Context Refine modifies the parent page HTML, not the asset file. If an image is being cropped because its container has overflow: hidden and wrong dimensions, the fix is to adjust the container CSS, not to regenerate the image. If a chart's iframe has opacity: 0 (making it invisible), the fix is to change the parent page's CSS rule, not to modify the chart HTML (even though the evaluation might have flagged this as a chart visibility issue — the Prompt 25 VISIBILITY FIRST RULE explicitly categorizes parent-CSS-caused invisibility as webpage_issues).

The paper provides a concrete example in Figure 5: a chart's legend is positioned below the chart rather than to the right within the same section, and another chart's iframe CSS sets opacity: 0, making it fully transparent. Both of these are Context Refine fixes — they require modifying the parent page's HTML/CSS structure, not the chart's internal rendering.

Level 3: Global Refine. After local and context-level issues are resolved, Global Refine evaluates the entire webpage to "detect high-level layout and style inconsistencies, using both the HTML code and the rendered screenshot as references" (Section 3.2). This level performs "targeted edits to the HTML and page structure, enforcing consistent layout, spacing, and visual style across all sections."

The Global Reflection prompt (Prompt 29, Appendix B.3) provides:

  • The original webpage design prompt
  • The current generated HTML (after local and context fixes)
  • A list of issues to fix

It instructs the refactoring agent to:

  1. Fix only the issues explicitly stated by the user
  2. Modify the HTML strictly and minimally
  3. Preserve all asset paths unchanged
  4. Keep layout, styling, and structure consistent with the design prompt
  5. Output only the corrected HTML code

The examples in Figure 5 illustrate Global Refine behaviors: fixing a layout that rendered as a two-column grid instead of "vertically stacked blocks," adjusting card heading text that overlaps a chart, and fixing a "Priority Shipments Queue" card that visually overlaps a gauge area.

Propagation between levels. The paper observes (Section 4.6, caption of Figure 5) that "local reflection can propagate to the global level, leading to more coherent overall page structures." This propagation happens implicitly: when Local Refine changes an image's color palette to better match the page style, the next round of global evaluation may detect that the page now has better style coherence, triggering fewer global-level fixes. There is no explicit signaling mechanism between levels — the evaluation stage runs fresh on the modified page, and the resulting issue list determines what each reflection level does in the next iteration.

Why three levels instead of a single "reflect and revise" step? The decomposition enables parallelism and specialisation. Local Refine can operate on multiple assets simultaneously (all images can be edited in parallel, all charts can be regenerated in parallel) because they are independent — fixing image A doesn't require knowing the state of image B. Context Refine can similarly operate on multiple embedding snippets in parallel. Global Refine, being holistic, must run after local and context fixes are applied, but within a single iteration, all three levels can be dispatched based on the same evaluation output. A single "revise everything" step would be sequential and would mix concerns — the prompt would need to describe image fixes, CSS fixes, and layout fixes all at once, making it harder for the LLM to focus on each type correctly.

Concrete iteration example (from Figure 5). The paper visualizes a three-row reflection process:

  • Row 1 (global): A layout rendered as a two-column grid instead of vertically stacked blocks. The Global Refine prompt receives this as an issue and restructures the HTML to a single-column vertical stack.
  • Row 2 (local): The color palette of an image is "slightly more saturated and high-contrast than the very soft, muted, negative-space-heavy aesthetic." Local Refine receives this as an image issue and regenerates with adjusted color tone. Separately, a poster background is "slightly textured/off-white with a soft drop shadow, rather than a perfectly flat white panel with no gradients or shading as specified for the brutalist style" — another local image issue.
  • Row 3 (local-to-global): A chart legend is positioned below the chart rather than to the right within the same section (context issue), and a chart iframe has opacity: 0 (another context issue). After fixing these, the subsequent global evaluation detects that the corrected layout creates a new compositional problem — and Global Refine adjusts accordingly.

This demonstrates the iterative, multi-pass nature: the output of one round's fixes becomes input to the next round's evaluation, which produces new (or remaining) issues for the next round's reflection.

The MM-WebGEN-Bench Construction Pipeline

The evaluation dataset is constructed through a four-stage pipeline designed to ensure diversity across layout complexity, visual style, multimodal composition, and semantic intent (Section 3.3, Figure 3(a)).

Stage 1: Prompt generation. The authors randomly sample values along four dimensions:

  • Layout complexity: single-column, multi-column grid, hierarchical layouts
  • Visual style: minimal, editorial, playful, Swiss-style, brutalist, cinematic, and others (11 styles total, per Figure 3(b) top right)
  • Multimodal elements: combinations of text, images, videos, and charts
  • Semantic intent: landing pages, dashboards, portfolios, informational, analytical, creative, commercial (11 scene categories, per Figure 3(b) top left)

These sampled values define a structured "scenario requirement." A GPT-based agent then expands this scenario into a detailed natural-language prompt describing a complete webpage design, including its content, structure, and style. This two-step process (random sampling + LLM expansion) is designed to produce prompts that are both diverse (from the random sampling) and natural/coherent (from the LLM expansion) — purely random attribute combinations might be contradictory (e.g., "minimalist" + "cinematic" + "single-column dashboard" could produce incoherent prompts), and the LLM expansion step resolves these contradictions by making reasonable design choices.

Stage 2: Plan validation. Each generated prompt is fed into the Planning Agent to produce a structured generation plan. This plan is subjected to automatic format validation — checking that the JSON structure is valid, required fields are present, and file paths are consistent between the code_generation description and the image/video/chart specifications. Samples that fail format validation are discarded.

Stage 3: Rendering and manual inspection. Valid plans are executed through the full generation pipeline (Layout Agent + specialized generators) to produce rendered webpages. These rendered pages are then manually inspected. The paper states:

"Samples exhibiting implausible layouts, inconsistent visual styles, or unrealistic combinations of multimodal elements are discarded, ensuring that the final benchmark contains high-quality, diverse webpages suitable for evaluation."

This manual curation step is important: it filters out cases where the planning agent produced logically valid but aesthetically broken designs, or where the AIGC tools produced low-quality assets that would unfairly penalize all evaluated methods. The benchmark is meant to test coordination ability, not the base quality of the underlying generation tools.

Stage 4: Final curation. The remaining high-quality samples constitute the final evaluation set: 120 webpages. The paper emphasizes this as a "curated benchmark" rather than a random sample — the filtering ensures that the benchmark measures what it intends to measure (multimodal coordination quality) rather than noise from tool failures or planning hallucinations.

Benchmark statistics (Figure 3(b)). The 120-webpage set spans:

  • 11 scene categories (top left): informational, analytical, creative, and commercial use cases
  • 11 visual styles (top right): from clean text-oriented designs (Swiss-style) to expressive, visually rich aesthetics (brutalist, cinematic)
  • Structure complexity (bottom left): simple single-column through multi-column and hierarchical compositions
  • Diverse multimodal content (bottom right): 4 types of videos, 8 types of images, and 17 types of charts

The paper does not report the exact distribution across these categories — the bar charts in Figure 3(b) show relative frequencies but are not accompanied by a detailed statistical table. This is a minor transparency gap: without knowing the exact split, it is difficult to assess whether certain categories are overrepresented and whether results might be category-dependent.

Why a new benchmark? The paper's argument (Section 2.3) is that existing benchmarks fail to evaluate the multimodal coordination that MM-WebAgent is designed to solve. Code-centric benchmarks (Web2Code) test HTML/CSS correctness without visual content. Image-to-code benchmarks (WebCode2M) test reconstruction from screenshots, not generation from intent. Asset-placeholder benchmarks (WebGEN-V) provide static images and test layout but not asset generation quality. MM-WebGEN-Bench is designed to fill the gap: it requires models to both generate the code and generate native multimodal assets that cohere with each other and with the overall design. The multi-level evaluation protocol then separately scores these two aspects, so a model that generates perfect layout but incoherent images will score high on Layout and low on Image — the decomposition makes the gap visible.

4. Key Insights and Innovations

Innovation 1: Hierarchical Planning at the Design Abstraction Level — Not Code Granularity

The paper's central conceptual move is redefining what "hierarchy" means in webpage generation. Prior agent-based approaches — OpenHands [27], bolt.diy [22], WebGen-Agent [13], UICopilot [5], DesignCoder [2] — all impose hierarchy at the execution level: decompose the task into steps, plan the code structure, generate components. This is a hierarchy of actions (first write the header HTML, then the hero section, then the footer). MM-WebAgent instead imposes hierarchy at the design abstraction level: a global layout plan that specifies what the page should look like, and local element plans that specify how each multimodal component should be generated conditioned on that global vision.

This is not a subtle distinction. The paper's own phrasing captures the shift precisely (Section 2.2):

"Although these methods enhance code correctness or layout reconstruction, their hierarchies are still limited to reasoning or code granularity. In contrast, we define hierarchy at the design abstraction level, representing a shift from code-centric orchestration to design-abstraction-driven multimodal generation with structured cross-modal refinement."

The difference is that a code-level hierarchy can produce valid code — all the <div> tags close, the CSS rules don't conflict — but it cannot ensure that an independently generated hero image, a chart in a dashboard card, and a looping background video all look like they belong to the same design. Those assets are generated by separate AIGC tools (GPT-Image-1, Sora-2, ECharts via GPT-5.1) that have no visibility into each other's outputs. The only way to achieve coherence is to give them shared constraints before generation, and those constraints must come from a level of abstraction that sits above any individual tool invocation.

The two-level plan structure — global layout with explicit multimodal placeholders and sizing constraints, local element plans with page_style, section, and role context fields — is the mechanism by which these shared constraints propagate. Every image, video, and chart receives the same page_style descriptor (e.g., "modern minimal," "brutalist," "playful"), so each generation tool is aiming for a compatible visual target even though they operate independently. This is fundamentally different from prior systems that call AIGC tools as isolated subroutines without cross-modal constraint propagation.

Why this matters beyond MM-WebAgent: the design-abstraction hierarchy is a general architectural pattern for multimodal generation systems, not a webpage-specific trick. Any system that composes outputs from independent generative models — a video with a soundtrack, a 3D scene with text overlays, a document with embedded figures — faces the same coordination problem. The paper's solution — define a shared design context at the planning level that constrains each generator's local behavior — transfers directly to those domains.

Evidence: Table 3(A) shows that removing hierarchical planning (collapsing to one-shot generation) causes the overall score to drop from 0.66 to 0.42, with image scores collapsing from 0.85 to 0.05 and video scores from 0.65 to 0.00. This is not a small degradation — it is near-total failure on multimodal content. Table 4 reinforces the point: giving a code-only pipeline access to the same AIGC tools (without hierarchical planning) barely moves the needle (0.42 → 0.45). The innovation is not "use AIGC tools" — it is "structure the planning so that tool invocations share design context."

Innovation 2: The Coordination Gap — A Diagnostic Concept That Explains Why Tool Access Alone Fails

The paper introduces an implicit but powerful diagnostic concept: the cross-modal coordination gap. This is the observation that making AIGC tools available to a code agent is insufficient — the tools must be coordinated through shared design constraints, or their outputs will conflict. This concept is never given a formal name in the paper (I am naming it here for clarity), but it is the intellectual engine behind the key ablation in Table 4 and the motivation for the entire hierarchical planning architecture.

Prior work in the agent-building community has largely assumed that tool access is the bottleneck: give an agent the ability to call an image generation API, and it will produce webpages with appropriate images. Systems like bolt.diy, OpenHands, and WebGen-Agent incorporate tool-calling as a core capability, and the expectation — implicit in their design — is that more tools → better outputs. The paper's Table 4 result flatly contradicts this assumption. Code-only GPT-5.1 scores 0.42 overall. Code-only GPT-5.1 given access to the identical GPT-Image-1, Sora-2, and ECharts tools scores 0.45. The 0.03 improvement is negligible. The tools are being called — they produce assets — but those assets don't match each other or the page.

The coordination gap concept explains why: a code agent with tool access but no design-level planning invokes each tool independently. It might generate an image for the hero section, then a chart for the dashboard, then a video for the background. Each tool call is a separate LLM reasoning step with no mechanism for sharing constraints across calls. The image might come back photorealistic with warm tones; the chart might use a default cool-toned ECharts palette; the video might be an abstract animation. From the agent's perspective, each individual tool call succeeded. But the assembled page looks incoherent — a collage rather than a design.

This is a diagnostic contribution, not an algorithmic one. It gives the field a precise way to characterize why certain agent architectures fail on multimodal composition tasks: they close the tool-access gap but leave the coordination gap wide open. The corollary is that benchmarks must measure coordination, not just tool-use success — which is exactly what MM-WebGEN-Bench's multi-level evaluation does by separately scoring global coherence (Layout, Style, Aesthetics) and local element quality (Image, Video, Chart).

Evidence: Table 4 is the smoking gun. The jump from 0.45 (tools without coordination) to 0.75 (tools with hierarchical planning and reflection) is 0.30 on the overall score — roughly 10× the gain from adding tools alone. The gap is concentrated in multimodal metrics: Image goes from 0.05 to 0.88, Video from 0.09 to 0.75, Chart from 0.36 to 0.54. Global metrics (Layout, Aesthetics) were already high with tools alone (0.83, 0.94 respectively), confirming that the tools were being called successfully — the failure was specifically in multimodal coordination, not in code quality.

Innovation 3: Three-Level Hierarchical Reflection as a Decomposition of the Iterative Design Process

The paper's reflection mechanism is structurally novel in a way that is easy to miss on first reading. Most agent systems with "self-reflection" or "self-critique" capabilities (WebGen-Agent [13], ReCode [30], various self-improvement loops) apply reflection uniformly: evaluate the output, identify problems, regenerate. The reflection is monolithic — a single evaluation pass produces a single set of issues, and a single revision pass attempts to fix everything at once.

MM-WebAgent decomposes reflection into three independent levels — local, context, global — each with its own scope, its own evaluation criteria, and its own fix strategies. This decomposition is not arbitrary; it mirrors how human designers actually iterate. A designer does not simultaneously fix a misaligned image, adjust the global color palette, and restructure the navigation layout in one cognitive step. They work at different granularities in different passes, often with different tools (an image editor for local fixes, a CSS inspector for context fixes, a layout tool for global fixes).

The three-level structure has properties that a monolithic reflection loop lacks:

Parallelisability. Local fixes to different assets are independent — fixing image A's color tone does not affect image B's composition. Context fixes to different sections are similarly independent (fixing the hero section's spacing doesn't affect the footer's iframe opacity). This means Local Refine and Context Refine can operate on multiple elements simultaneously within a single reflection iteration. The paper notes (Table 5 footnote) that image, video, and chart generation are parallelized, and the same applies to local reflection edits. A monolithic "regenerate everything" step would be fundamentally sequential.

Appropriate tool targeting. Each reflection level uses the right tool for the right problem. Local image fixes use GPT-Image-1's editing capabilities — the image model itself. Local chart fixes use GPT-5.1 to edit ECharts HTML — the code model. Context fixes use GPT-5.1 to patch parent page CSS — again the code model, but operating on different target files. Global fixes use GPT-5.1 with both HTML and screenshot context — a multimodal reasoning step. A monolithic reflection prompt would need to describe all these fix types simultaneously, making it harder for the LLM to maintain focus and correctness.

Issue routing prevents category confusion. The evaluation prompts explicitly categorize each detected problem as either an asset issue or a webpage embedding issue. For images (Prompt 21): image_issues vs. webpage_issues. For charts (Prompt 25): chart_issues vs. webpage_issues. This categorization ensures that an image that looks fine but is badly cropped by its container gets a CSS fix (Context Refine) rather than an unnecessary regeneration (Local Refine). It also ensures that a chart rendered invisible by opacity: 0 in the parent CSS gets classified as a webpage issue, not a chart issue — which matters because the fix location is the parent HTML, not the chart's isolated iframe document.

Evidence: Table 3(B) ablates each reflection level. No reflection: 0.66 overall. Local only: 0.68 (image and chart metrics improve). Local + Context: 0.70 (global metrics start improving as embedding fixes propagate). Global only: 0.73 (layout and style jump significantly). All three: 0.75. The incremental gains from each level are modest individually (0.02–0.03 per added level) but the cumulative improvement from no reflection to full reflection is 0.09 — roughly a 14% relative improvement on an already strong base. More importantly, the pattern of which metrics improve at which level confirms the decomposition is meaningful: Local reflection primarily improves Image (0.85 → 0.87) and Chart (0.44 → 0.48); Global reflection primarily improves Layout (0.68 → 0.85) and Style (0.37 → 0.53); Context reflection bridges the two, improving both local embedding quality and global coherence.

Innovation 4: Transparent Chart Generation via Visibility-First Embedding and Two-Stage Sizing

The chart generation pipeline contains a design pattern that is easy to overlook but has broader implications for any system that generates self-contained visual artifacts for embedding in larger compositions. The pattern is two-stage sizing with visibility-first debugging during reflection.

The problem is this: a chart generated by GPT-5.1 using ECharts needs to fit into an <iframe> container with a fixed height (typically 180–300px, as specified in the layout plan). But at generation time, the chart agent does not know the exact container dimensions — those are determined later by the Layout Agent's HTML and CSS. The initial chart generation prompt (Prompt 10) therefore instructs the model to generate charts at 100vh height, filling the viewport. This produces visually rich, readable charts with plenty of space for labels, legends, and data. But it also means every generated chart is initially wrong for its container — too large, with labels that will truncate and legends that will overflow when shoehorned into a 200px iframe.

The paper's solution is not to fix the generation prompt (which would require somehow communicating container dimensions to the chart agent, creating a circular dependency). Instead, it defers the sizing fix to the Local Refine stage, where the Chart Local Reflection prompt (Prompt 31) explicitly addresses the container constraint:

"CRITICAL: The chart will be displayed inside an iframe container with LIMITED HEIGHT (typically 180-300px). Your fixes must ensure the chart looks good IN THIS CONSTRAINED SPACE."

And provides specific CSS/ECharts guidelines: use height: 100% instead of 100vh, set html, body to height: 100%; margin: 0; overflow: hidden;, use compact grid padding, consider hiding redundant titles, use smaller font sizes.

This is a generation-then-adapt pattern that acknowledges a fundamental tension: you want generation to produce high-quality artifacts (which benefits from large canvas sizes and rich detail), but you need the final output to fit constrained spaces. Rather than compromising generation quality by forcing it to target small containers directly — which would produce cramped, low-information charts — the system generates at high quality and then structurally adapts the output for embedding. The adaptation is rule-based and reliable (CSS changes, not content changes), so it can be trusted to work automatically.

The visibility-first rule in the chart evaluation prompt (Prompt 25) is a second key design element:

"CRITICAL - VISIBILITY FIRST RULE: The chart MUST be visible in its default state (without hover, click, or any user interaction). ... NEVER suggest setting opacity: 0, visibility: hidden, or display: none as default state."

This rule addresses a common failure mode in LLM-generated webpages: CSS animations or hover effects that leave elements invisible by default, making the page look broken in static screenshots (which is how both human evaluators and the paper's evaluation manager assess quality). By encoding this as an explicit, non-negotiable rule in the evaluation prompt, the system catches and fixes these issues during reflection rather than letting them slip through.

Significance beyond webpages: The generation-then-adapt pattern applies to any pipeline where self-contained artifacts are generated independently and then composed into a larger whole — documents with embedded figures, slide decks with generated charts, interactive dashboards with multiple visualization widgets. The visibility-first debugging rule is a specific instance of a broader principle: when evaluating generated compositions, enforce static correctness first before considering interactive behavior, because static failures are both more common and more damaging to perceived quality.

Evidence: The chart metric in Table 1 for MM-WebAgent with GPT-5.1 reaches 0.54, compared to 0.35–0.52 for the best code-only baselines. While the absolute improvement is smaller than for images and videos (which go from near-zero to 0.88 and 0.75 respectively), chart generation is intrinsically harder — it requires both correct data visualization (which code-only models can already do reasonably well via ECharts) and visual consistency with the page (which requires the reflection-based adaptation). The ablation in Table 3(B) shows that adding Local reflection improves chart scores from 0.44 to 0.48, and adding Context reflection further improves to 0.54 — confirming that the two-stage sizing approach (Local for chart-internal fixes, Context for embedding fixes) is the mechanism behind these gains.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. All experiments use MM-WebGEN-Bench, the 120-webpage curated benchmark introduced in this paper (Section 3.3). The benchmark spans 11 scene categories, 11 visual styles, 4 video types, 8 image types, and 17 chart types, constructed through a four-stage pipeline (randomized attribute sampling, LLM-based prompt expansion, automatic format validation, and manual quality inspection). The paper also reports results on WebGen-Bench (Lu et al., 2025) — a 500-task benchmark focused on functional backend code, logic, and component completeness — to provide a broader perspective, though the authors note their agent is not explicitly designed for backend code generation tasks.

  • Base model(s). The hierarchical planner, evaluation manager, reflection manager, and chart generation agent all use OpenAI GPT-5.1 as the core reasoning model (Section 4.1). Image generation uses GPT-Image-1, video generation uses Sora-2 (OpenAI's video model), and charts are rendered as ECharts-based HTML also generated by GPT-5.1. The choice of GPT-5.1 is pragmatic — it represents a state-of-the-art multimodal LLM — but the paper does not justify why this particular model family was selected over alternatives. The framework is model-agnostic in principle: Table 1 reports MM-WebAgent results with five different backbone models (Gemini-2.5-Pro, GPT-4o, GPT-5mini, GPT-5, GPT-5.1), confirming that the architecture works across model scales and families, though performance varies substantially with base model capability (0.36 overall with GPT-4o vs. 0.75 with GPT-5.1).

  • Metrics. The evaluation uses a multi-level protocol with six dimensions: three global (Layout Correctness, Style Coherence, Aesthetic Quality) and three local (Image quality, Video quality, Chart quality). Layout and Style use penalty-based scoring — evaluators identify violations against predefined rules, assign penalties by severity, and compute score = max(0, 1 - α · Σ p_i) where p_i is the penalty for the i-th issue. Aesthetics, Image, Video, and Chart use a graded scoring scheme on {0, 0.2, 0.4, 0.6, 0.8, 1.0}, averaged across samples. The overall score is the simple average across all six dimensions (Section 3.3). All evaluations are conducted three times with mean and standard deviation reported (Section 4.1). For WebGen-Bench, the paper reports Accuracy and Appearance Score following that benchmark's protocol.

  • Baselines. The paper compares against three paradigms with multiple instantiations (Table 1):

    • (I) Code-only One-shot generation: Direct end-to-end HTML/CSS generation from the design prompt, no agentic decomposition, no AIGC tools. Eight model variants: Qwen2.5-Coder-7B-Instruct, Qwen2.5-Coder-32B-Instruct, Qwen3-Coder-30B-A3B-Instruct, Qwen2.5-72B-Instruct, Gemini-2.5-Pro, GPT-4o, GPT-5mini, GPT-5, GPT-5.1.
    • (II) Code-only Agents: Agentic execution frameworks that decompose the task into planning and code-generation steps but are restricted to code-only assets (multimodal content appears as placeholders, links, or SVGs). Two agent frameworks: bolt.diy (StackBlitz Labs, 2024) with eight model backends, and OpenHands (Wang et al., 2024) with five model backends (Gemini-2.5-Pro, GPT-4o, GPT-5mini, GPT-5, GPT-5.1).
    • (III) Multimodal Web Agents: The full MM-WebAgent framework with AIGC tool access and hierarchical planning/reflection, tested with five backbone models.
    • On WebGen-Bench, additional baselines include WebGen-LM-32B (Bolt.diy), WebGenAgent-LM-7B, and direct code-only GPT-5.1.

  • Generation budget / compute accounting. The paper does not standardize generation budgets across methods in the main comparison — each baseline uses its native generation strategy (one-shot code generation, agentic multi-step synthesis, or full hierarchical pipeline). The primary axis of comparison is method paradigm, not compute-matched performance. However, Section 4.4 reports per-task cost and latency: MM-WebAgent averages 3.21and155.8secondspertask(withparallelimage/video/chartgeneration),comparedtoOpenHandsat3.21 and 155.8 seconds per task (with parallel image/video/chart generation), compared to OpenHands at 0.27 and 182.4 seconds, and bolt.diy at $0.14 and 76.9 seconds (Table 5). The reflection iteration sweep (Figure 4) implicitly varies compute by measuring performance at different numbers of reflection rounds (up to 3).

  • Cross-validation / statistical protocol. The paper does not employ cross-validation — the test set is fixed at 120 webpages. All evaluations are run three times "to account for stochasticity in generation," with results reported as mean and standard deviation (Section 4.1). The paper also conducts a pairwise user study with 50 annotators who have backgrounds in web design, computer science, or multimodal content creation, using anonymized blind assessment with five-level preference scales (much worse, worse, similar, better, much better), with results aggregated into win/tie/lose rates (Section 4.5).

Main Quantitative Results

Paradigm Comparison on MM-WebGEN-Bench

The headline result from Table 1: MM-WebAgent with GPT-5.1 achieves an average score of 0.75, substantially outperforming both code-only one-shot generation (GPT-5.1: 0.42) and code-only agent baselines (GPT-5.1 via bolt.diy: 0.46; GPT-5.1 via OpenHands: 0.46). The gap is not uniform across metrics — it is concentrated in multimodal element quality.

Breaking down the best-performing configuration (MM-WebAgent + GPT-5.1, Table 1 bottom row):

  • Global metrics: Layout 0.83, Style 0.54, Aesthetics 0.97. These are competitive with or slightly better than the best code-only one-shot baseline (GPT-5: Layout 0.78, Style 0.40, Aesthetics 0.96) and the best code-only agent baseline (GPT-5 via bolt.diy: Layout 0.77, Style 0.43, Aesthetics 0.95). The gains on global metrics are real but modest.
  • Local metrics: Image 0.88, Video 0.75, Chart 0.54. These represent massive improvements over code-only baselines, which essentially cannot generate multimodal content: the best code-only one-shot Image score is 0.43 (Gemini-2.5-Pro), the best Video score across all code-only baselines is 0.05 (Qwen2.5-72B-Instruct via bolt.diy), and the best Chart score is 0.52 (GPT-5 one-shot). For reference, the same GPT-5.1 model scores 0.05 on Image and 0.00 on Video in one-shot mode — effectively zero for non-code modalities.

The video metric deserves special attention: zero video capability in all code-only baselines. Across all 18 code-only configurations (9 one-shot + 5 bolt.diy + 4 OpenHands), the Video score never exceeds 0.05, and most configurations score exactly 0.00. This is because code-only pipelines cannot generate video files — at best they can embed links to external videos or create CSS animations that don't match the prompt's video specifications. MM-WebAgent achieves 0.75 on Video by actually invoking Sora-2 to generate .mp4 files that match the design intent. This is less a "quality improvement" and more a capability that code-only systems fundamentally lack.

Model scaling within MM-WebAgent. The framework's performance scales with base model capability (Table 1, Section III):

  • GPT-4o: Overall 0.36 (Image 0.42, Video 0.29, Chart 0.32)
  • GPT-5mini: Overall 0.68 (Image 0.84, Video 0.63, Chart 0.50)
  • GPT-5: Overall 0.71 (Image 0.86, Video 0.52, Chart 0.54)
  • GPT-5.1: Overall 0.75 (Image 0.88, Video 0.75, Chart 0.54)
  • Gemini-2.5-Pro: Overall 0.63 (Image 0.81, Video 0.57, Chart 0.43)

The progression from GPT-4o (0.36) to GPT-5.1 (0.75) is a 2.1× improvement, demonstrating that the framework benefits substantially from stronger base models — particularly in planning quality and evaluation accuracy. The Gemini-2.5-Pro result (0.63) is notably lower than GPT-5.1, suggesting that model-specific factors (instruction following, structured output reliability, multimodal reasoning quality) affect framework performance even when the architecture is held constant.

Code-only agents vs. one-shot generation. A counterintuitive pattern: for the stronger models (GPT-5, GPT-5.1), agent-based code-only systems do not outperform one-shot generation by much. GPT-5.1 one-shot: Overall 0.42. GPT-5.1 via bolt.diy: Overall 0.46. GPT-5.1 via OpenHands: Overall 0.46. The 0.04 improvement from adding agentic decomposition is dwarfed by the 0.29 improvement from adding multimodal tools with hierarchical planning (0.75 - 0.46 = 0.29). This challenges the implicit assumption that "more agentic = better" — for this task, the modality gap (can't generate images/videos/charts) dominates any benefits from iterative planning.

Performance on WebGen-Bench (Table 2). Despite not being designed for backend code generation, MM-WebAgent + GPT-5.1 achieves an Accuracy of 47.8% and Appearance Score of 3.9, competitive with the best baseline (OpenAI-GPT 5.1 via bolt.diy: 48.7% Accuracy, 3.8 Appearance). The paper interprets this as evidence that the hierarchical framework does not sacrifice code quality, but the result is also consistent with the possibility that WebGen-Bench's backend-focus means multimodal generation ability provides little advantage — the framework's strengths are irrelevant to the metric, and it performs roughly at the base model's capability level.

Ablation Studies

Hierarchical Planning (Table 3A): Removing hierarchical planning collapses the system to one-shot generation (no reflection, no plan structure). Under no-reflection conditions:

  • With hierarchical planning: Overall 0.66 (Layout 0.68, Style 0.37, Aesthetics 0.96, Image 0.85, Video 0.65, Chart 0.44)
  • Without hierarchical planning: Overall 0.42 (Layout 0.73, Style 0.44, Aesthetics 0.96, Image 0.05, Video 0.00, Chart 0.35)

The most dramatic drops are Image (0.85 → 0.05, a 17× reduction) and Video (0.65 → 0.00, total failure), confirming that hierarchical planning is the mechanism enabling multimodal asset generation. Somewhat surprisingly, Layout and Style scores are higher without hierarchical planning (0.73 vs. 0.68 for Layout, 0.44 vs. 0.37 for Style). This is a genuine trade-off: the hierarchical plan constrains the layout more tightly (specifying exact image sizes, chart placements, etc.), which introduces more opportunities for layout mismatches. A one-shot generation without these constraints can produce a simpler (but code-only) page that scores higher on layout correctness because there are fewer elements to get wrong. This is an important nuance: hierarchical planning enables multimodal content but may slightly reduce layout quality by increasing the page's complexity — the gains come from having content worth laying out correctly.

Further ablation: local planning only. The paper states (Section 4.3): "We further ablate local planning by disabling it from the full system, which results in a clear drop in overall performance (Avg: 0.75 → 0.69), with pronounced degradation on local metrics (e.g., Image and Video), confirming the necessity of context-aware local planning." Exact per-metric numbers for this ablation are described textually but not presented in a separate table row — only the overall Average change (0.75 → 0.69) is specified.

Hierarchical Reflection (Table 3B): Ablating individual reflection levels (all under full hierarchical planning):

  • No reflection: Overall 0.66
  • Local reflection only: Overall 0.68 (Image +0.02, Chart +0.04)
  • Local + Context reflection: Overall 0.70 (Style +0.06, Chart +0.06 from no-reflection)
  • Global reflection only: Overall 0.73 (Layout +0.17, Style +0.16)
  • All reflections (default): Overall 0.75 (Layout 0.83, Style 0.54, Aesthetics 0.97, Image 0.88, Video 0.75, Chart 0.54)

The additive pattern is clear: Local and Context reflection primarily improve local element metrics, while Global reflection primarily improves global layout and style. The combination of all three yields the highest scores across all dimensions, with the full system achieving 0.09 higher overall than no reflection. The Video metric shows improvement from 0.65 (no reflection) to 0.75 (full reflection) — a 0.10 gain that is larger than the individual reflection level improvements suggest, indicating that video quality benefits from the interaction of multiple reflection levels (e.g., context fixes that improve video embedding visibility, combined with global fixes that ensure surrounding layout complements the video).

AIGC Tool Access (Table 4): This is the critical ablation testing whether the framework's gains come from hierarchical coordination or simply from tool availability:

  • Code-only GPT-5.1 (no tools): Overall 0.42 (Image 0.05, Video 0.00, Chart 0.35)
  • Code-only GPT-5.1 + AIGC tools (same tools, no planning framework): Overall 0.45 (Image 0.05, Video 0.09, Chart 0.36)
  • MM-WebAgent (full framework): Overall 0.75 (Image 0.88, Video 0.75, Chart 0.54)

Tools alone provide an Overall improvement of 0.03. The full framework provides an additional 0.30 above tools alone. The Image score with tools alone is 0.05 — identical to no-tools — meaning the code-only agent with AIGC access is generating images, but they are so poorly coordinated with the page that they score essentially zero on the evaluation criteria. This table is the paper's strongest evidence for the coordination gap hypothesis: tool access is necessary but wildly insufficient; coordination through hierarchical planning is the actual bottleneck.

Reflection Iterations (Figure 4): The paper reports that "most gains are achieved within the first few reflection rounds, indicating that hierarchical reflection enables efficient refinement without excessive iterations." The figure (which we can reference but whose exact data points are in the paper) shows both global and local evaluation metrics improving across iterations, with the curve flattening after approximately 2 rounds. The maximum iteration count is 3 (Section 4.1). The paper does not report whether the reflection loop has a convergence criterion (e.g., stop when scores stop improving) or simply runs for a fixed number of iterations — this is a missing detail that would matter for practical deployment where compute cost matters.

User Study

The human evaluation (Section 4.5) with 50 annotators using blind pairwise comparison yields: MM-WebAgent achieves a 78.99% winning rate against competing methods. Annotators compared webpages on layout quality, content relevance, multimodal asset quality, and embedding quality of local elements, using a five-level scale. The paper reports only the aggregate win rate — it does not break down preference by evaluation dimension, by difficulty category, or by competing method. This is a limitation: knowing whether the preference is driven primarily by multimodal asset quality (as the automatic metrics suggest) or by other factors would strengthen the interpretation. The user study does confirm that the automatic metrics align directionally with human judgment — the method with the highest multi-level scores also wins human preference comparisons.

Qualitative Results

Figure 1 presents rendered webpage examples comparing MM-WebAgent with three baseline methods (OpenAI-GPT 5.1, Qwen2.5-Coder-32B-Inst., Qwen2.5-72B-Inst.) on MM-WebGEN-Bench. The paper claims MM-WebAgent produces "more coherent layouts, consistent visual styles, and better-integrated multimodal elements." Additional examples appear in Appendix A (Figure 6), and Figure 5 visualizes the hierarchical reflection process with specific examples of global layout refinement, context refinement, and local element refinement.

Critical Assessment

Does the evidence support the claim that MM-WebAgent "outperforms code-generation and agent-based baselines"?

Yes, with strong evidence on multimodal metrics, but the margin on code-only metrics is small. Table 1 shows MM-WebAgent + GPT-5.1 achieves Overall 0.75 versus 0.42–0.46 for code-only GPT-5.1 variants. However, the improvement is almost entirely driven by Image (0.88 vs. 0.05) and Video (0.75 vs. 0.00) — metrics on which code-only baselines are definitionally incapable. On Layout (0.83 vs. 0.73–0.78), Style (0.54 vs. 0.39–0.44), and Aesthetics (0.97 vs. 0.95–0.96), the gains are modest and some code-only baselines actually score higher on individual dimensions (e.g., GPT-5 one-shot scores 0.78 on Layout vs. MM-WebAgent's 0.83). The "outperformance" claim is true on average but should be understood as "massively better at multimodal content with comparable or slightly better layout" — not as uniform superiority.

Does the evidence support the claim that hierarchical planning (global + local) is necessary?

Yes, with caveats. Table 3A shows that removing hierarchical planning causes Image to drop from 0.85 to 0.05 and Video from 0.65 to 0.00 under no-reflection conditions. Table 4 shows the even stronger result: adding AIGC tools to a code-only pipeline (without hierarchical planning) yields almost no improvement (0.42 → 0.45). These results convincingly demonstrate that the specific way tools are integrated matters — tool access alone is nearly useless. However, the ablation collapses "hierarchical planning" into a binary (present/absent) without exploring gradations: What if you had global planning without local context propagation? What if local plans included context but were not organized hierarchically? The paper reports one additional ablation (disabling local planning only, reducing Avg from 0.75 to 0.69) but provides only aggregate numbers in the text, not a full table. The necessity claim is supported at a coarse granularity, but the relative importance of specific hierarchical design choices (context fields, parallel structure, explicit sizing constraints) is not isolated.

Does the evidence support the claim that hierarchical reflection improves quality by iteratively refining at three levels?

Yes, with clear evidence of complementary effects. Table 3B shows that Local reflection primarily improves Image and Chart, Context reflection adds Style improvements, and Global reflection drives Layout and Style gains. Full reflection (all three) achieves the highest scores on every dimension. The additive pattern confirms that the three levels address different types of issues. However, the absolute gains from reflection are modest: 0.66 → 0.75 overall (a 13.6% relative improvement). This means the initial generation quality (post-planning, pre-reflection) is already reasonably high for global metrics — reflection is refining rather than salvaging. The practical question of whether the 0.09 improvement justifies the additional latency (41.1 seconds for reflection, per Table 5) depends on the application's quality requirements.

Several details are missing:

  • The convergence behavior is only shown graphically (Figure 4) without specific numeric per-iteration scores.
  • Whether the 3-iteration cap is ever reached, and whether convergence is detected automatically, is not stated.
  • The reflection process's failure modes are not analyzed — are there cases where reflection makes the page worse (over-correction)?

Does the evidence support the claim that MM-WebGEN-Bench enables systematic evaluation of multimodal webpage generation?

Partially. The benchmark construction is described in detail and the resulting 120-webpage set has demonstrable diversity (11 scene categories, 11 visual styles, diverse multimodal compositions). The multi-level evaluation protocol provides granular, interpretable scores. However, several concerns limit the strength of this claim:

  • Size: 120 webpages is a small evaluation set. With scores reported as averages across six dimensions, variance estimates (standard deviations in Table 1) are based on only 120 data points. The standard deviations are reasonably small for some metrics (Layout: ±0.01–0.04 for most configurations) but the statistical power to detect moderate differences between methods is limited.
  • Curation bias: The benchmark was constructed by generating a large pool of prompts, filtering through automatic validation, and then manually discarding samples with "implausible layouts, inconsistent visual styles, or unrealistic combinations." The filtering criteria are somewhat vague, and the filtering process could introduce biases — the curated set may skew toward prompts that the MM-WebAgent pipeline handles well, since the pipeline was used to generate the initial pool. The paper does not specify what fraction of the initial pool survived filtering, nor whether manual inspection was blind to generation method.
  • Single benchmark: All cross-method comparisons on multimodal quality are on MM-WebGEN-Bench, which the authors created. The WebGen-Bench results (Table 2) provide external validation for code quality but not for multimodal coordination, since WebGen-Bench does not evaluate multimodal content.

What experiments would have strengthened the paper?

  1. Difficulty-stratified analysis: The 120 webpages span 11 scene categories and varying layout complexities, but results are reported only in aggregate. Breaking down scores by layout complexity, visual style, or multimodal composition density would reveal whether the framework's advantages are uniform or concentrated in specific regimes — analogous to the difficulty-bin analysis in the reference example paper.

  2. Oracle vs. predicted planning ablation: The planning agent (GPT-5.1) generates both global and local plans. How much does plan quality matter? An experiment where a human writes the structured plan (oracle planning) versus the LLM-generated plan would isolate whether the bottleneck is plan generation quality or plan execution. This is particularly relevant because the planning prompt (Prompt 1) is complex and specific — poor plan quality could cascade into poor generation.

  3. Component-level failure analysis: The paper reports aggregate scores but does not analyze where failures occur. What fraction of image issues are due to generation quality versus embedding? How often does Context Refine fail to fix a reported issue? What types of design prompts cause the largest quality gaps? This granular failure analysis would make the evaluation more diagnostic.

  4. Human baseline: The paper claims to model "the workflow of human designers," but there is no comparison to human-generated webpages or human-in-the-loop hybrid workflows. Without a human performance ceiling, it is unclear how much room for improvement remains — are the 0.75 scores close to saturation, or is there a large gap to close?

  5. Computational cost scaling with page complexity: Table 5 reports average costs but does not show how cost scales with the number of multimodal elements, layout complexity, or reflection iterations per page. Since the paper's core claim is about coordination, understanding whether coordination cost grows linearly or super-linearly with page complexity would be practically important.

  6. Statistical significance testing: The paper reports means and standard deviations but no formal significance tests (t-tests, confidence intervals for differences). With 120 samples and three runs, the variance in Table 1 is sometimes large enough that moderate score differences may not be statistically reliable — for example, GPT-5mini via MM-WebAgent scores 0.68 ± 0.01 vs. GPT-5 at 0.71 ± 0.00, a 0.03 difference whose reliability is unclear without formal testing.

Conditional nature of the claims

The paper's claims hold most strongly under the following conditions, which are consistent with the experimental design but may not generalize:

  • The base LLM is capable of structured planning and evaluation. The framework is tested with GPT-5.1 (a state-of-the-art multimodal model) and GPT-5, with GPT-4o showing substantially worse performance (0.36 overall). The framework's effectiveness depends on the base model's ability to follow complex prompt templates, produce valid JSON, and correctly categorize issues. Weaker models may fail at these orchestration tasks even if the architecture is sound.
  • The AIGC tools produce high-quality assets given well-specified prompts. Image and video quality depends on GPT-Image-1 and Sora-2; chart quality depends on GPT-5.1's ECharts code generation. If these tools produce poor outputs for certain styles or content types, no amount of coordination will fix the underlying asset quality. The paper's approach coordinates generation given capable tools — it does not improve tool quality.
  • The evaluation is conducted on curated benchmarks where design prompts are well-specified. MM-WebGEN-Bench prompts were generated by an LLM and filtered for quality. Performance on noisy, underspecified, or contradictory real-world user prompts may be substantially lower, since the planning agent would need to resolve ambiguity that the benchmark deliberately removes.
  • The cost overhead is acceptable. Table 5 shows MM-WebAgent costs 3.21perwebpage(primarilyAPIcostsforGPT−5.1planning,evaluation,andreflectioncalls),comparedto3.21 per webpage (primarily API costs for GPT-5.1 planning, evaluation, and reflection calls), compared to 0.14–0.27 for code-only agents. The 12–23× cost increase may be justified for applications where visual quality matters, but is prohibitive for high-volume or cost-sensitive deployments.

6. Limitations and Trade-offs

Difficulty Estimation Cost Is Unaccounted for in the Headline Efficiency Gains

The assumption or constraint. The entire compute-optimal scaling framework depends on estimating a prompt's difficulty before deciding how to allocate the test-time compute budget. The paper's method for doing so — generating 2,048 complete solutions per question and either computing pass@1 (oracle) or averaging PRM final-answer scores (predicted) — is extraordinarily expensive. Section 3.2 acknowledges this directly:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The paper treats difficulty as an input to the allocation decision, not as a cost to be amortized. The 2,048-sample estimation step consumes more compute than any test-time budget studied (the maximum is 256–512 generations), meaning the total cost of the compute-optimal approach is difficulty estimation + strategy execution.

The consequence. In any realistic deployment, the difficulty estimation cost would dominate the total inference budget and eliminate — or even reverse — the reported efficiency gains. The headline claim of "more than 4× better efficiency over a standard best-of-N baseline" (Section 1) is computed after difficulty is known, without subtracting the cost of learning it. If difficulty estimation costs the equivalent of 2,048 generations per question, then a strategy that uses only 16 generations for solving (achieving bare-bones efficiency) actually consumes 2,064 generations total — far worse than a naive best-of-256 that requires no upfront estimation. The 4×4\times figure is therefore best understood as an upper bound on achievable efficiency, not a realized deployment gain. The paper acknowledges this as "a key avenue for future work" (Section 3.2), but the severity of the gap means that none of the reported efficiency improvements should be taken at face value for practical systems unless a cheap difficulty estimator is developed and validated.

What evidence exists in the paper. The paper provides no measurements of total cost including difficulty estimation. The generation budget axis in all figures (Figures 3, 4, 6, 7, 8) represents only the strategy execution budget, not the upfront sampling cost. The FLOPs-matched comparison (Section 7) likewise does not include difficulty estimation FLOPs. Table 5 in the MM-WebAgent paper provides per-task latency and cost breakdowns as a model for what transparent accounting looks like — the compute-optimal paper provides no equivalent.

Mitigation status. Not addressed. The paper flags it as future work (Section 8): "our method of estimating question difficulty requires generating many samples from the base LLM, incurring non-trivial cost. Future work should explore more efficient methods of estimating question difficulty." No lightweight difficulty estimator is developed or evaluated, and the paper does not report how performance degrades if difficulty is estimated from far fewer samples (e.g., 4 or 8 rather than 2,048).

Hard Problems Remain Completely Unsolved — Test-Time Compute Cannot Create Capability

The assumption or constraint. The paper's framework assumes that the base model can produce correct solutions at some non-trivial rate for the test-time strategies to amplify. This assumption fails on the hardest problems (difficulty quintile 5), where the base model's pass@1 is near zero. As Section 5.3 states, "no method makes meaningful progress" on these problems, and Section 7 is explicit:

"On the hardest problems (bin 5), test-time compute provides essentially no benefit regardless of budget, meaning that some capabilities can only be acquired through pretraining, not recovered at inference time."

The consequence. For any problem distribution where the base model's performance is extremely low, the compute-optimal framework offers zero value — the allocation policy correctly identifies that no strategy works and allocates budget that produces no improvement. In the MATH benchmark, bin 5 represents roughly 20% of the test questions (100 out of 500). For those questions, all methods — search, revisions, and their compute-optimal combinations — achieve roughly 1–3% accuracy regardless of budget (Figures 3 right, 7 right, 9). This is not a gradual degradation; it is a hard capability boundary. A practitioner deploying this system needs to know that if their problem distribution contains a substantial fraction of genuinely hard queries, no amount of test-time compute will help — those problems must be routed to a larger model, a human, or a different system altogether. The compute-optimal framework provides no mechanism for recognizing this boundary a priori and escalating — it will spend budget (including difficulty estimation cost) only to produce an incorrect answer.

What evidence exists in the paper. Figure 3 (right) shows bin 5 accuracy hovering at 1–3% across all methods and budgets. Figure 7 (right) shows bin 5 at roughly 2–3% regardless of sequential-to-parallel ratio. Figure 9 (the FLOPs-matched comparison) shows the bin 5 scaling line essentially flat near 0–5%, well below the ~14× larger model's performance at all values of the inference-to-pretraining ratio RR. The conclusion is unambiguous and replicated across both search and revision paradigms.

Mitigation status. The paper is transparent about this limitation (Section 7 takeaway box) and does not claim to solve it. The compute-optimal policy implicitly handles this by routing bin 5 problems to the strategy that performs least badly, but this is damage minimization, not a solution. The paper does not propose any mechanism for improving performance on these problems through test-time techniques, and the finding that even the most sophisticated allocation cannot help on bin 5 establishes a clear boundary condition for the applicability of test-time compute scaling.

The Single-Benchmark, Single-Model-Family Evaluation Limits Generalization Claims

The assumption or constraint. All experiments are conducted on a single benchmark (MATH, 500 test questions) using a single model family (PaLM 2-S*) as the base LLM. The paper states in Section 4 that it "believe[s] this model is representative of the capabilities of many contemporary LLMs," but this claim is asserted, not tested. The depth of the experiments (cross-validation, multiple methods, careful ablations) is high within this narrow scope, but the breadth is minimal.

The consequence. Several aspects of the findings could be model-specific or benchmark-specific in ways that affect their practical applicability:

  • PRM over-optimization behavior: The finding that beam search degrades performance on easy problems at high budgets (Figure 3, right) depends on the specific calibration and error distribution of the trained PRM. A different base model with different output characteristics — or a PRM trained with a different procedure or more data — could exhibit different over-optimization thresholds. The difficulty quintile boundaries where beam search becomes beneficial versus harmful are therefore specific to PaLM 2-S* with this particular PRM.

  • Revision model trainability: The revision model's ability to learn from edit-distance-paired incorrect-to-correct trajectories depends on the base model's in-context learning and fine-tuning characteristics. Different model families (e.g., GPT-4, Claude, LLaMA) might exhibit different revision learning rates, different correct-to-incorrect reversion rates (reported as ~38% for PaLM 2-S*), and different optimal sequential-to-parallel ratios.

  • MATH benchmark specificity: MATH consists of competition-level math problems requiring symbolic multi-step reasoning. Whether the difficulty-dependent patterns — revisions best on easy problems, beam search best on medium-hard problems — generalize to other reasoning domains (code generation, logical deduction, scientific QA) or to open-ended generation tasks is unknown.

The paper's central insight (difficulty-conditioned allocation) is likely robust across these variations — the specific thresholds and strategies would change, but the principle that different difficulties call for different strategies would persist. However, a practitioner cannot use the paper's specific difficulty quintile boundaries or strategy lookup tables for a different model or domain without re-running the entire optimization procedure.

What evidence exists in the paper. There is no experiment on any benchmark except MATH, and no experiment with any base model except PaLM 2-S*. The paper does not compare its PRM against PRMs trained by other groups on other models, nor does it test the revision model on out-of-distribution reasoning tasks. The FLOPs-matched comparison uses a second PaLM 2 model with ~14× more parameters, but this is still within the same model family.

Mitigation status. Acknowledged implicitly in the scope but not addressed. The paper does not claim generalization beyond MATH or PaLM 2-S*, but the framing in Sections 1 and 2 uses language that implies broader applicability ("many contemporary LLMs," "LLM deployments"). A replication study on at least one additional benchmark (e.g., GSM8K for math, HumanEval for code) and one additional model family would substantially strengthen the generalization claims — the paper provides no such evidence.

Verifier Over-Optimization Is a Hard Ceiling That Is Not Solved, Only Mitigated

The assumption or constraint. The compute-optimal policy's effectiveness is fundamentally bounded by the PRM's reliability. When search optimizes too aggressively against the PRM, it finds solutions that score highly under the verifier but are actually incorrect — a phenomenon the paper documents extensively (Section 5.3, Appendix M). The compute-optimal policy mitigates this by routing easy problems (where the PRM is least reliable under optimization) away from aggressive search, but it does not solve the underlying verifier robustness problem.

The consequence. On medium-difficulty problems (bins 3–4), where beam search is deployed per the compute-optimal policy, over-optimization still limits the scaling ceiling. Figure 3 (right) shows that beam search (M=4M = 4) on bin 3 improves from roughly 24% at 4 generations to ~34% at 256 generations — but the curve is flattening well before the budget is exhausted. Adding more compute beyond ~64–128 generations yields diminishing or zero returns even on the difficulty tier where beam search is optimal. This means the compute-optimal policy cannot achieve unbounded improvements with unbounded test-time compute — there is a verifier-imposed performance ceiling that no allocation strategy can exceed.

The paper's qualitative examples (Appendix M, Figures 29 etc.) show specific failure modes: search produces low-information repetitive steps at the end of solutions, and overly short 1–2 step solutions that the PRM scores highly but that are incorrect. These are not pathologies that the compute-optimal allocation fixes — they are inherent to the PRM's training procedure and would require fundamentally better verifiers to address.

What evidence exists in the paper. Figure 3 (right) shows the beam search curves flattening for bin 3 at high budgets. Figure 3 (left) shows lookahead search — the most powerful optimizer — paradoxically performing worst overall, a direct consequence of more aggressive optimization amplifying PRM errors. The paper explicitly identifies this in Section 5.3:

"The degradation at high budgets is attributed to over-optimization of the PRM — search finds solutions that score highly under the PRM but are actually incorrect."

And in Section 8, the paper notes that "the primary bottleneck preventing unbounded improvements from additional compute" is verifier over-optimization.

Mitigation status. The compute-optimal policy partially mitigates this by avoiding aggressive optimization on easy problems, but it does not solve it for the medium-difficulty tier where beam search is actually deployed. The paper suggests (Section 8) that "improving verifier robustness is the key bottleneck for further scaling test-time compute, not improving search algorithms," but trains no improved verifier and does not experiment with adversarial training, ensemble methods, or other robustness techniques. The current results are therefore specific to the verifier quality achievable with the Monte Carlo rollout training procedure — a practitioner seeking to scale test-time compute beyond the budgets studied would likely hit this ceiling and need to invest in verifier improvement before seeing further gains.

Revisions and Search Are Studied Independently, Not Combined

The assumption or constraint. The paper studies two complementary mechanisms for test-time compute — PRM-guided search (which optimizes the verifier/side of the proposal-verifier decomposition) and iterative revisions (which modifies the proposal distribution) — but never combines them. Section 8 states this explicitly:

"we did not experiment with PRM tree-search techniques in combination with revisions"

The proposal-verifier framework in Section 2 naturally suggests that combining a stronger proposal distribution (revision model) with an optimized verifier (PRM search) should yield gains beyond either mechanism alone. But the paper's experiments treat them as separate, parallel analyses — search results use the base model as the proposal distribution, and revision results use majority voting or a separately trained ORM for selection.

The consequence. The reported performance numbers — particularly the compute-optimal scaling curves in Figures 4 and 8 — represent a lower bound on what a fully integrated system could achieve. A system that used the revision model to generate candidate solutions and then applied PRM beam search to select among revision chains could potentially break through the performance plateaus that each method hits individually:

  • On medium-difficulty problems, the revision model could produce higher-quality proposals that contain correct solutions at higher rates, while PRM search could more reliably identify those correct solutions.
  • On easy problems, the revision model could refine already-good solutions while the PRM could verify that revisions don't introduce errors (addressing the 38% correct-to-incorrect reversion rate).
  • The PRM could guide which revisions to pursue — rather than blindly generating a chain of length NN, use step-level scores to decide when a revision is on track versus when to restart.

The current results leave open whether the combination yields additive improvements (the gains simply sum) or multiplicative improvements (the stronger proposals make the verifier's job easier, and vice versa). This is a significant gap because the natural next step for anyone building on this work would be to combine the two mechanisms — and the paper provides no evidence for what to expect.

What evidence exists in the paper. The paper shows that revision model outputs suffer from distribution shift when scored by the base-model PRM (Appendix J, Figure 15a), requiring a separately trained ORM. This suggests that naively combining revisions with the existing PRM might not work well, but the paper does not explore whether this could be addressed by training the PRM on revision model outputs or by calibrating scores across the distribution shift.

Mitigation status. Acknowledged as future work in Section 8 but not addressed. The paper's framework and experiments provide all the building blocks for a combined system (revision model, PRM, search algorithms), but the integration step is left entirely to future work. This is a reasonable scoping decision for a first paper — establishing the scaling behavior of each mechanism independently before combining them — but it means the reported results are not the best achievable under the framework's own principles.

The 14× Larger Model Baseline Is Artificially Weak

The assumption or constraint. The FLOPs-matched comparison in Section 7 scales model parameters while holding training data fixed, following the LLaMA paradigm rather than Chinchilla-optimal training. The paper acknowledges this in Section 7:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

Additionally, the ~14× larger model is evaluated only with greedy decoding — no majority voting, no best-of-N, no test-time compute augmentation of any kind. This means the comparison is not "test-time compute vs. pretraining compute" but rather "small model with compute-optimal test-time scaling vs. large model with zero test-time compute."

The consequence. Both decisions make the pretraining baseline weaker than it needs to be:

  • Chinchilla-optimal training: A model trained with ~14× more total FLOPs allocated optimally across both parameters and data (per Hoffmann et al., 2022) would likely outperform a model that scales parameters only. The paper's baseline may understate what pretraining compute can achieve, making the test-time compute approach look more favorable by comparison. The reported advantages (e.g., +27.8% on easy questions at R≪1R \ll 1 for revisions) might shrink or reverse against a properly compute-optimal larger model.

  • No test-time compute for the larger model: The comparison framework from Section 7 asks: given a fixed total FLOPs budget, should you spend it all on pretraining (larger model, greedy decoding) or split it between a smaller model and additional inference compute? But the larger model could also benefit from some test-time compute — even a modest allocation (best-of-8 or best-of-16) would create a much stronger baseline. The paper's framing implicitly assumes the larger model must use greedy decoding to make the FLOPs accounting work, but in practice, one could train a moderately larger model and give it a modest test-time budget, which might outperform both extremes.

What evidence exists in the paper. Figure 9 and the Figure 1 bar charts show the FLOPs-matched comparison results. The paper is transparent about the parameter-only scaling choice but does not discuss the greedy-decoding assumption for the larger model. There is no experiment giving the ~14× larger model any test-time compute augmentation, so the reader cannot assess how much of the advantage is due to the test-time strategy versus the weak baseline.

Mitigation status. Partially acknowledged for the parameter-only scaling choice but not for the lack of test-time compute on the larger model. The paper frames the comparison as "representative of a canonical approach to scaling pretraining compute" (Section 7), which is a reasonable defense — many real systems do scale parameters without optimally scaling data — but the greedy-decoding constraint on the larger model is not similarly justified. A fairer comparison would allocate the larger model some fixed test-time budget (e.g., 4–8 generations) and recompute the FLOPs accounting to see whether the test-time compute advantage persists.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a coordination-first paradigm for multimodal generation systems, shifting the field's focus from tool availability to cross-modal constraint propagation. Before MM-WebAgent, the dominant assumption in agentic webpage generation — and in agentic systems more broadly — was that giving an LLM access to AIGC tools (image generators, video models, chart libraries) would naturally lead to coherent multimodal outputs. The agent would reason about which tools to call, invoke them with appropriate prompts, and compose the results. The paper's Table 4 result — where adding GPT-Image-1, Sora-2, and ECharts to a code-only GPT-5.1 pipeline improves the overall score from 0.42 to 0.45, while the full hierarchical framework reaches 0.75 — demonstrates that this assumption is wrong by an order of magnitude. Tool access alone provides negligible benefit; coordination through shared design constraints provides the remaining ~90% of the gain.

This finding recasts the bottleneck in multimodal generation from "can we call the right tools?" to "can we ensure the tools produce mutually consistent outputs?" The implication extends beyond webpage generation to any system that composes outputs from independent generative models — video with soundtrack, 3D scenes with text overlays, documents with embedded figures, interactive dashboards with multiple visualization widgets. In all these domains, the paper's central diagnostic applies: measure performance with tools alone versus with tools plus coordination, and the gap reveals how much of the quality ceiling is attributable to cross-modal inconsistency rather than individual asset quality.

The paper also introduces a structured reflection taxonomy that decomposes iterative improvement into three independent levels — local (asset-internal quality), context (embedding correctness), and global (layout coherence) — each with its own evaluation criteria, fix strategies, and tool targets. This is a refinement of the monolithic "reflect and revise" loops common in prior agent systems (WebGen-Agent, ReCode), but it is a refinement with practical consequences. By routing issues to the appropriate level, the system avoids unnecessary regeneration (an image that is correct but misaligned gets a CSS fix rather than a costly re-render), enables parallel execution within levels, and produces more targeted, interpretable feedback at each iteration. The three-level decomposition does not require any learning — it is purely architectural — which means it can be adopted immediately by any agent framework that generates multimodal compositions.

The paper's multi-level evaluation protocol addresses a measurement gap that has held back rigorous comparison in this area. Prior benchmarks either ignored multimodal content entirely (code-correctness metrics), treated it as given (image-to-code reconstruction), or scored it with a single holistic rating. By separately measuring global page-level quality (Layout, Style, Aesthetics) and local element-level quality (Image, Video, Chart) on the same 0–1 scale, the protocol makes it possible to diagnose where specific methods fail rather than just ranking them. The finding that MM-WebAgent's advantage is almost entirely concentrated in local metrics (Image 0.88 vs. 0.05 for code-only GPT-5.1, Video 0.75 vs. 0.00) while global metrics are comparable (Layout 0.83 vs. 0.73, Aesthetics 0.97 vs. 0.96) is a diagnostic insight that a single aggregate score would have obscured. This decompositional evaluation approach — separately assessing composition quality and component quality — is transferable to any multimodal generation task.

In terms of magnitude, this is not a paradigm shift in the sense of introducing a new model architecture or training paradigm. It is an architectural reframing that identifies coordination as the primary bottleneck, proposes a specific hierarchical structure for addressing it, and validates the approach with strong ablations. The ideas (hierarchical planning, iterative refinement, decomposed evaluation) are not individually novel — they draw on established patterns from software engineering, design workflows, and agent architectures. The contribution is their combination into a demonstrably effective orchestration framework for a problem (multimodal webpage generation) where prior systems had implicitly assumed the coordination problem would solve itself.

One prior contradiction the paper resolves: the agent-building community has debated whether tool-calling capability or base model quality is the primary driver of agent performance. The Table 4 result suggests a third variable — coordination architecture — that can dominate both. A weaker model (GPT-4o) with the full coordination framework scores 0.36 overall, while a stronger model (GPT-5.1) without coordination scores 0.42–0.46. The framework's value is not simply additive to model capability — it is a multiplier that determines whether tool access translates into quality gains at all.

Follow-Up Research This Work Enables

Adaptive difficulty estimation without the 2,048-sample overhead. The current difficulty estimation method is the most immediate bottleneck for practical deployment. A concrete next step: train a lightweight classifier that takes only the design prompt text (not the rendered webpage) as input and predicts which of the five difficulty quintiles the prompt belongs to. The training data already exists — the 2,048-sample PRM score averages computed for the 500 MATH questions. The classifier could be a small fine-tuned model or even a few-shot prompted LLM with a small sample budget (4–8 generations per prompt). The evaluation would measure: (1) how well the predicted difficulty bins align with oracle bins, and (2) whether compute-optimal policies selected using predicted bins achieve the same 4×4\times efficiency gains as those using oracle bins. If a cheap classifier works, the entire compute-optimal framework becomes deployable. If it fails, the failure mode (which difficulty tiers are hardest to predict, whether errors skew toward over- or under-estimation) would clarify the limits of difficulty-conditioned allocation.

Joint optimization of planning quality and execution reliability. MM-WebAgent's planning stage uses a single LLM call (GPT-5.1 with a complex prompt template) to produce the entire structured JSON plan. The plan quality is never directly evaluated — only downstream webpage quality is measured. A useful ablation: manually write "oracle" plans for a subset of MM-WebGEN-Bench prompts (specifying exact layout, element placements, and style attributes that a human designer would choose) and compare the resulting webpage quality against LLM-generated plans. This would isolate whether the current performance ceiling (0.75 overall) is limited by plan quality or by execution/reliability of the AIGC tools and reflection process. If oracle plans substantially improve scores, then investment in better planning (multi-shot plan refinement, plan validation with an independent critic LLM, or human-in-the-loop plan editing) is the highest-leverage next step. If oracle plans produce similar scores, then the bottleneck is in tool reliability or reflection — pointing toward better image/video generation models or more sophisticated reflection strategies.

Dynamic reflection depth based on detected issue severity. The current system runs reflection for a fixed maximum of 3 iterations (Section 4.1) without an explicit convergence criterion. A practical extension: implement an issue-severity threshold that triggers early stopping when the evaluation manager detects no issues with penalty above some minimum value, or when the score improvement between iterations falls below a threshold. Measure whether this reduces average latency (currently 41.1 seconds for reflection, per Table 5) without degrading quality. A more ambitious version: vary the reflection level based on detected issue types — if the evaluation finds only image issues (no context or global issues), skip context and global reflection for that iteration, saving the cost of those LLM calls. The ablation in Table 3B suggests this could work (Local-only reflection already improves image and chart scores), but it has not been tested as an adaptive policy.

Stress-testing the framework on out-of-distribution design prompts. MM-WebGEN-Bench is constructed from LLM-generated and curated prompts — it is a clean, well-specified distribution. Real-world user prompts are messier: underspecified ("make me a nice homepage"), contradictory ("minimalist but with lots of animations"), or domain-specific ("a dashboard for tracking protein folding simulations"). A stress test would collect 50–100 real user prompts (from a web design forum, a freelancing platform, or a user study), run MM-WebAgent on them without curation, and measure: (1) how often the planning agent produces malformed or contradictory plans, (2) how often the reflection process fails to converge or makes the page worse, and (3) whether the distribution of scores shifts relative to the curated benchmark. Negative results (substantial degradation on uncurated prompts) would motivate investment in plan validation, ambiguity resolution, or human-in-the-loop plan approval. Positive results (minimal degradation) would strengthen the case for deployment.

Combining PRM-guided search with the revision model as the proposal distribution. The MM-WebAgent framework currently uses a fixed tool set and fixed reflection hierarchy. An interesting cross-pollination with the compute-optimal test-time scaling work would be to replace MM-WebAgent's static reflect-and-refine loop with a PRM-guided search over reflection actions. Specifically: train a process reward model on MM-WebAgent's reflection trajectories (labeling each reflection action — image edit, CSS patch, layout restructure — as successful or unsuccessful based on whether the evaluation score improves), then use beam search to select the most promising sequence of reflection actions rather than applying all three levels uniformly. This would address a current inefficiency: the reflection manager applies all fixes from all levels regardless of whether some are unnecessary or counterproductive. A PRM-guided search could skip low-value fixes and allocate reflection budget to the most impactful changes, potentially reducing the 3-iteration cost while maintaining quality. The experiment would compare: (1) fixed 3-iteration uniform reflection (current approach), (2) PRM-guided selective reflection, and (3) an oracle that only applies fixes that actually improve the score, measuring both quality and latency.

Replacing the fixed tool set with dynamic tool selection and composition. MM-WebAgent currently assumes a fixed set of three AIGC tools (GPT-Image-1, Sora-2, ECharts). In practice, multiple tools exist for each modality (DALL-E vs. Midjourney vs. Stable Diffusion for images; different charting libraries; different video models), and the optimal tool may depend on the specific design prompt (photorealistic images might work better with one model, illustrations with another). A dynamic tool selection module — an additional planning step that analyzes the design prompt and routes each multimodal element to the most appropriate available tool — would make the framework more robust to tool quality variations and tool availability changes. The evaluation would measure whether dynamic selection improves scores on subsets of MM-WebGEN-Bench where the default tool choice is suboptimal, and whether the improvement justifies the additional planning cost.

Practical Applications and Downstream Use Cases

Automated landing page and marketing website generation. The most direct application: a service that takes a natural-language description of a business (café, SaaS product, portfolio) and generates a complete, visually coherent website with custom images, background videos, and data visualizations. The paper's results suggest that such a system, built on MM-WebAgent with GPT-5.1-level planning, would produce pages scoring ~0.75 on the multi-level evaluation — meaning strong layout correctness (0.83), professional aesthetics (0.97), and well-integrated images (0.88) and videos (0.75). At $3.21 per page and 155.8 seconds latency (Table 5), this is cost-competitive with human-designed template customization for small businesses, and substantially faster. The 78.99% human preference win rate (Section 4.5) suggests end-users would find the generated pages preferable to alternative automated methods, though the paper does not compare against human-designed pages so the absolute quality ceiling is unknown.

Rapid prototyping for UI/UX design workflows. Professional designers spend significant time creating high-fidelity mockups with realistic content (images, data charts, sample videos) to communicate design intent to stakeholders. MM-WebAgent could function as a "design assistant" that takes a rough description — "a dashboard-style analytics page with a sales trend chart, a regional heatmap, and a hero video showing the product in use" — and produces a complete, rendered prototype in ~2.5 minutes. The reflection mechanism is particularly valuable here: a designer could inspect the initial output, manually flag issues, and trigger targeted reflection rounds for specific elements rather than regenerating from scratch. This hybrid human-in-the-loop workflow (human evaluates, agent reflects) mirrors the paper's architecture but with a human providing higher-quality issue identification than the LLM-based evaluation manager. The multi-level evaluation scores provide a starting point for expectation-setting: the designer should expect strong global layout and aesthetics (0.83–0.97) but should budget time for verifying chart data accuracy (0.54) and video content relevance (0.75), where the automatic scores indicate room for human correction.

Benchmark-driven tool evaluation for AIGC providers. The paper's multi-level evaluation protocol, combined with MM-WebGEN-Bench, provides a standardized framework for comparing how different AIGC tools perform when integrated into a coordinated generation pipeline. An image generation API provider could evaluate their model not just on standalone image quality metrics (FID, CLIP score, human preference) but on integration quality — how well their images maintain visual coherence when embedded into webpages with specific styles and layouts. The same framework could compare video models (Sora-2 vs. alternatives), chart libraries, or planning LLMs (GPT-5.1 vs. Gemini-2.5-Pro, where the paper already shows a 0.75 vs. 0.63 overall gap). This is a practical use of the benchmark that does not require deploying MM-WebAgent in production — it serves as an evaluation harness for component-level decisions in a multimodal pipeline.

When to Prefer This Method

The paper positions MM-WebAgent against two alternatives: code-only one-shot generation and code-only agent-based generation (Table 1). The specific decision rule implied by the experimental results is:

  • Prefer MM-WebAgent (hierarchical planning + AIGC tools + reflection) when:

    • The design prompt requires native multimodal content that cannot be satisfied by placeholders, stock images, or external links — specifically images, videos, or data charts that must match the page's visual style. Evidence: code-only baselines score 0.05 on Image and 0.00 on Video versus MM-WebAgent's 0.88 and 0.75 (Table 1).
    • Visual coherence across independently generated elements matters more than minimizing per-page cost. Evidence: the framework costs ~3.21perpageversus3.21 per page versus 0.14–$0.27 for code-only agents (Table 5), a ~12–23× cost multiplier.
    • A capable base LLM (GPT-5-tier or equivalent) is available for planning and evaluation. Evidence: GPT-4o running the same framework achieves only 0.36 overall, suggesting the coordination architecture cannot compensate for weak planning capability.

  • Prefer code-only one-shot generation (direct HTML/CSS output) when:

    • The design prompt is purely text-and-layout (no multimodal elements specified), or when placeholder images and external links are acceptable. Evidence: on global metrics alone, GPT-5 one-shot scores 0.78 Layout, 0.40 Style, 0.96 Aesthetics — competitive with MM-WebAgent's 0.83, 0.54, 0.97.
    • Latency and cost are primary constraints — for example, generating hundreds of landing page variants for A/B testing where visual polish matters less than structural diversity.
    • The base LLM is below GPT-5 capability, where the planning and reflection prompts may produce unreliable outputs.

  • Prefer code-only agent-based generation (bolt.diy, OpenHands) when:

    • The task involves functional backend logic, API integrations, or complex interactive behavior rather than visual design. Evidence: on WebGen-Bench (Table 2), MM-WebAgent scores 47.8% Accuracy versus 48.7% for bolt.diy with GPT-5.1 — approximately tied, because the framework's multimodal strengths are irrelevant to backend tasks.
    • The deployment environment cannot call external AIGC APIs (due to cost, availability, or privacy constraints), limiting the system to code generation only.